EBookClubs

Read Books & Download eBooks Full Online

EBookClubs

Read Books & Download eBooks Full Online

Book Impacts of Multidimensionality and Content Misclassification on Ability Estimation in Computerized Adaptive Sequential Testing CAST

Download or read book Impacts of Multidimensionality and Content Misclassification on Ability Estimation in Computerized Adaptive Sequential Testing CAST written by and published by . This book was released on 2006 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Computer Adaptive Sequential Testing (CAST) is a test delivery model that combines features of the traditional conventional paper-and-pencil testing and item-based computerized adaptive testing (CAT). The basic structure of CAST is a panel composed of multiple testlets adaptively administered to examinees at different stages. Current applications of CAST reply on the item response theory (IRT) and assume a unidimensional IRT model for scoring. This study evaluated the robustness of CAST when tests were constructed, administered, and scored by a unidimensional IRT model but item responses were multidimensional. Various conditions of multidimensionality were simulated in item pools, as well as different levels of content misclassification through manipulation of the correspondence between content area and dimension of items. An automated test assembly (ATA) process constructed CAST panels from the item pools, each representing a unique combination of multidimensionality and content misclassification. Administration of the panels was simulated and multidimensional response data were scored by the unidimensional IRT model. The ability scores, routing decisions, and pass-fail decisions were evaluated against "true" ability scores and decisions to assess the impacts of multidimensionality and content misclassification. Results showed that, when multidimensionality was mild as measured by the angle distance between item clusters, unidimensional ability estimates and routing decisions were not sensitive to the level of content misclassification in item pools. Only when multidimensionality was severe, panels without content misclassification yielded more accurate ability estimates and routing decisions. However, type I errors of pass-fail decisions (defined as passing unqualified examinees) were the smallest for panels without content misclassification, regardless of the severity of multidimensionality. The conclusion was that, although content classification, affecting dimensional structure during test assembly, might not be a serious concern for unidimensional ability estimation and routing decisions for CAST with multidimensional data, it played a significant role in the accuracy of pass-fail decisions.

Book Robustness of Ability Estimation to Multidimensionality in CAST with Implications to Test Assembly

Download or read book Robustness of Ability Estimation to Multidimensionality in CAST with Implications to Test Assembly written by Yanwei Zhang and published by . This book was released on 2006 with total page 52 pages. Available in PDF, EPUB and Kindle. Book excerpt: Computer Adaptive Sequential Testing (CAST) is a test delivery model that combines features of the traditional conventional paper-and-pencil testing and item-based computerized adaptive testing (CAT). The basic structure of CAST is a panel composed of multiple testlets adaptively administered to examinees at different stages. Current applications of CAST reply on the item response theory (IRT) and assume a unidimensional IRT model for scoring. This study evaluated the robustness of CAST when tests were constructed, administered, and scored by a unidimensional IRT model but item responses were multidimensional. Various conditions of multidimensionality were simulated in item pools, as well as different levels of content misclassification through manipulation of the correspondence between content area and dimension of items. An automated test assembly (ATA) process constructed CAST panels from the item pools, each representing a unique combination of multidimensionality and content misclassification. Administration of the panels was simulated and multidimensional response data were scored by the unidimensional IRT model. The ability scores, routing decisions, and pass-fail decisions were evaluated against "true" ability scores and decisions to assess the impacts of multidimensionality and content misclassification. Results showed that, when multidimensionality was mild as measured by the angle distance between item clusters, unidimensional ability estimates and routing decisions were not sensitive to the level of content misclassification in item pools. (Contains 8 tables and 14 figures.).

Book The Impact of Missing Data on Parameter Estimation in Computerized Adaptive Testing

Download or read book The Impact of Missing Data on Parameter Estimation in Computerized Adaptive Testing written by Xiaowen Liu and published by . This book was released on 2021 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Computer adaptive testing (CAT) targets items to student ability level and creates a missing data pattern that may not be ignorable, depending on the use of the testing data. The current study investigates the impact of missingness on parameter recovery in CAT by examining parameter estimation and the sampling distributions of parameter estimates in several testing conditions. Overall, recalibration of item parameters after an operational CAT was reasonable despite the missing data challenges. Under some conditions, a subset of item discrimination parameters were estimated to be negative, resulting in a discontinuity in the ability continuum. A cluster analysis revealed the extent to which post-CAT data segments students and items. The ignorability of missingness in process data in CAT was also explored. Results for the estimation of response time and accuracy models, t separately and jointly, showed the importance of including the observed responses relevant to the missing data mechanism.

Book Computerized Adaptive Testing Theory and Practice

Download or read book Computerized Adaptive Testing Theory and Practice written by Wim J. van der Linden and published by Springer Science & Business Media. This book was released on 2005-12-30 with total page 327 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book offers a comprehensive introduction to the latest developments in the theory and practice of CAT. It can be used both as a basic reference and a valuable resource on test theory. It covers such topics as item selection and ability estimation, item pool development and maintenance, item calibration and model fit, and testlet-based adaptive testing, as well as the operational aspects of existing large-scale CAT programs.

Book Dissertation Abstracts International

Download or read book Dissertation Abstracts International written by and published by . This book was released on 2006 with total page 614 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Book Effect of Person Cluster on Accuracy of Ability Estimation of Computerized Adaptive Testing in K 12 Education Assessment

Download or read book Effect of Person Cluster on Accuracy of Ability Estimation of Computerized Adaptive Testing in K 12 Education Assessment written by Shudong Wang and published by . This book was released on 2011 with total page 31 pages. Available in PDF, EPUB and Kindle. Book excerpt: The ability estimation procedure is one of the most important components in a computerized adaptive testing (CAT) system. Currently, all CATs that provide K-12 student scores are based on the item response theory (IRT) model(s); while such application directly violates the assumption of independent sample of a person in IRT models because ability estimation is mostly based on cluster (or correlated) educational data in which students usually are clustered in certain groups or settings (classrooms or schools). The consequences of such violations are commonly ignored. The purpose of this study is to investigate the effect of ignoring hierarchical data structures of students sample on the accuracy of ability estimation by using a regular Rasch model. Results show that ICCs have not only statistically significant effect on the accuracy of a person's ability estimation, but also large effect sizes. (Contains 13 tables and 7 figures.).

Book Statistical Aspects Of Computerized Adaptive Testing

Download or read book Statistical Aspects Of Computerized Adaptive Testing written by Haskell Sie and published by . This book was released on 2014 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: In the past several decades, Computerized Adaptive Testing (CAT) has received much attention in educational and psychological research due to the efficiency in achieving the goal of assessment, whether it is to estimate the latent trait of test takers with high precision or to accurately classify them into one of several latent classes. In the latter case, the adaptive nature of CAT is used in educational testing to make inferences about the location of examinees' latent ability relative to one or more pre-specified cut-off points along the ability continuum. When there is only one cut-off point and two proficiency groups, this type of CAT is commonly referred to as Adaptive Mastery Testing (AMT). A well-known approach in AMT is to combine the Sequential Probability Ratio Test (SPRT) stopping rule with item selection to maximize Fisher information at the mastery threshold. In the first part of this dissertation, a new approach is proposed in which a time limit is defined for the test and examinees' response times are considered in both item selection and test termination. Item selection is performed by maximizing Fisher information per time unit, rather than Fisher information itself. The test is terminated once the SPRT makes a classification decision, the time limit is exceeded, or there is no remaining item that has a high enough probability of being answered before the time limit. In a simulation study, the new procedure showed a substantial reduction in average testing time while slightly improving classification accuracy compared to the original method. In addition, the new procedure reduced the percentage of examinees who exceeded the time limit. Another well-known stopping rule in AMT is to terminate the assessment once the examinee's two-sided ability confidence interval lies entirely above or below the cut score. The second part of this dissertation proposes new procedures that seek to improve such a variable-length stopping rule by coupling it with curtailment and stochastic curtailment. Under the new procedures, test termination can occur earlier if the probability is high enough that the current classification decision remains the same should the test continue. Computation of this probability utilizes normality of an asymptotically equivalent version of the maximum likelihood estimate (MLE) of ability. In two simulation studies, the new procedures showed a substantial reduction in average test length (ATL) while maintaining similar classification accuracy to the original stopping rule based on the ability confidence interval. In the last part of this dissertation, generalization to multidimensional CAT (MCAT) is examined. Research has shown that MCAT improves the precision of both subscores and overall scores compared to its unidimensional counterpart. Several studies have investigated the performance of MCAT in recovering examinees' multiple abilities depending on the item selection methods. None of these studies, however, considered an item pool containing a mixture of multiple-choice (MC) and constructed-response (CR) items. With many assessments currently containing such a mixture of item types that measure more than one trait, there is an obvious need to understand how different item selection methods choose different types of items depending on their dimensional loadings (simple-structure versus complex-structured) and location of maximum information. In a simulation study, performance of five MCAT item selection methods were compared using an item pool consisting of a mixture between MC and CR items for mixed-format assessments. Ability recovery as well as item preferences of each method (simple- versus complex-structured items and location of maximum information) were examined.

Book Evaluating the Impacts of Item Exposure Procedures on Ability Estimates in CAT When Items are Disclosed

Download or read book Evaluating the Impacts of Item Exposure Procedures on Ability Estimates in CAT When Items are Disclosed written by Wei He and published by . This book was released on 2008 with total page 30 pages. Available in PDF, EPUB and Kindle. Book excerpt: Test security has been a concern for computerized adaptive tests (CAT) due to the nature of continuous testing. This concern becomes unprecedentedly severe with increasingly easy access to the World-Wide-Web where some examinees post on the internet their recollections of items they are administered, leaving future examinees with opportunities to get their scores inflated by having pre-knowledge of the items that appear in their tests. This study investigated, given some items in the item pool are disclosed, which item exposure control procedure results in minimum negative effect on ability estimate. The disclosed items in this study are defined as items released to the public or intentionally memorized by the test-takers and shared with future examinees and items to which examinees can respond correctly. The results from both simulation and statistical hypothesis testing are in consistent with Schnipke & Scrams (1999) in terms that the presence of disclosed items can inflate the final ability estimates in the context of CATs by an unacceptably high degree, especially for the low-ability examinees. Among all item exposure procedures under study, the a-stratified test with b-blocking, i.e., a design that stratifies the item pool but takes care of the relationship between item difficulty and discrimination parameters, performs the best. (Contains 18 tables, 3 figures, and 4 footnotes.).

Book Handbook of Test Development

Download or read book Handbook of Test Development written by Suzanne Lane and published by Routledge. This book was released on 2015-10-08 with total page 676 pages. Available in PDF, EPUB and Kindle. Book excerpt: The second edition of the Handbook of Test Development provides graduate students and professionals with an up-to-date, research-oriented guide to the latest developments in the field. Including thirty-two chapters by well-known scholars and practitioners, it is divided into five sections, covering the foundations of test development, content definition, item development, test design and form assembly, and the processes of test administration, documentation, and evaluation. Keenly aware of developments in the field since the publication of the first edition, including changes in technology, the evolution of psychometric theory, and the increased demands for effective tests via educational policy, the editors of this edition include new chapters on assessing noncognitive skills, measuring growth and learning progressions, automated item generation and test assembly, and computerized scoring of constructed responses. The volume also includes expanded coverage of performance testing, validity, fairness, and numerous other topics. Edited by Suzanne Lane, Mark R. Raymond, and Thomas M. Haladyna, The Handbook of Test Development, 2nd edition, is based on the revised Standards for Educational and Psychological Testing, and is appropriate for graduate courses and seminars that deal with test development and usage, professional testing services and credentialing agencies, state and local boards of education, and academic libraries serving these groups.

Book Standards for Educational and Psychological Testing

Download or read book Standards for Educational and Psychological Testing written by American Educational Research Association and published by . This book was released on 2014 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: "Prepared by the Joint Committee on Standards for Educatioanl and Psychological Testing of the American Educational Research Association, American Psychological Association and National Council on Measurement in Education"--T.p. verso.

Book Automated Machine Learning

Download or read book Automated Machine Learning written by Frank Hutter and published by Springer. This book was released on 2019-05-17 with total page 223 pages. Available in PDF, EPUB and Kindle. Book excerpt: This open access book presents the first comprehensive overview of general methods in Automated Machine Learning (AutoML), collects descriptions of existing systems based on these methods, and discusses the first series of international challenges of AutoML systems. The recent success of commercial ML applications and the rapid growth of the field has created a high demand for off-the-shelf ML methods that can be used easily and without expert knowledge. However, many of the recent machine learning successes crucially rely on human experts, who manually select appropriate ML architectures (deep learning architectures or more traditional ML workflows) and their hyperparameters. To overcome this problem, the field of AutoML targets a progressive automation of machine learning, based on principles from optimization and machine learning itself. This book serves as a point of entry into this quickly-developing field for researchers and advanced students alike, as well as providing a reference for practitioners aiming to use AutoML in their work.

Book Design and Analysis of DNA Microarray Investigations

Download or read book Design and Analysis of DNA Microarray Investigations written by Richard M. Simon and published by Springer Science & Business Media. This book was released on 2006-05-09 with total page 205 pages. Available in PDF, EPUB and Kindle. Book excerpt: The analysis of gene expression profile data from DNA micorarray studies are discussed in this book. It provides a review of available methods and presents it in a manner that is intelligible to biologists. It offers an understanding of the design and analysis of experiments utilizing microarrays to benefit scientists. It includes an Appendix tutorial on the use of BRB-ArrayTools and step by step analyses of several major datasets using this software which is available from the National Cancer Institute.