Download or read book Effects of Scale Transformation and Test Termination Rule on the Precision of Ability Estimates in CAT written by Qing Yi and published by . This book was released on 2000 with total page 32 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Resources in Education written by and published by . This book was released on 1996-07 with total page 336 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book The Impact of Item Location Effects on Ability Estimation in CAT written by Mei Liu and published by . This book was released on 2006 with total page 40 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Effects of Test Length and Sample Size on the Estimates of Precision of Latent Ability Scores written by Linda L. Cook and published by . This book was released on 1979 with total page 35 pages. Available in PDF, EPUB and Kindle. Book excerpt: One of the most important advantages that accrue from the application of latent trait models is the possibility of specifying a target information curve and then selecting items from an item pool to produce a test with the features characterized by this curve. By proceeding in this manner, it is possible to develop a test that provides a pre-specified level of precision (Standard Error of Ability Estimate) at selected ability levels. One problem with this paradigm is that little is known about the precision of the standard error of ability estimates (SEE) under varying circumstances. The purpose of the research reported in this paper was to address three practical questions of importance and interest to test developers: 1) What are the effects of examinee sample size and test length on the precision of SEE Curves, 2) What effects do the statistical characteristics of an item pool have on the precision of SEE Curves, and 3) What is the relationship between test length and SEE Curves in typical item pools? Keywords: Latent trait theory, Psychological tests, Aptitude tests, Mathematical models. (SDW).

Download or read book Extension of the Item Pocket Method Allowing for Response Review and Revision to a Computerized Adaptive Test Using the Generalized Partial Credit Model written by Mishan G. B. Jensen and published by . This book was released on 2017 with total page 346 pages. Available in PDF, EPUB and Kindle. Book excerpt: Computerized Adaptive Testing (CAT) has increased in the last few decades, due in part to the increased use and availability of personal computers, but also partly due to the benefits of CATs. CATs provide increased measurement precision of ability estimates while decreasing the demand on examinees with shorter tests. This is accomplished by tailoring the test to each examinee and selecting items that are not too difficult or too easy based on the examinees’ interim ability estimate and responses to previous items. These benefits come at the cost of the flexibility to move through the test as an examinee would with a Paper and Pencil (P & P) test. The algorithms used in CATs for item selection and ability estimation require restrictions to response review and revision; however, a large portion of examinees desire options for review and revision of responses (Vispoel, Clough, Bleiler, Hendrickson, and Ihrig, 2002). Previous research has examined response review and revision in CATs with limited review and revision options and are limited to after all items had been administered. The development of the Item Pocket (IP) method (Han, 2013) has allowed for response review and revision during the test, relaxing the restrictions, while maintaining an acceptable level of measurement precision. This is achieved by creating an item pocket in which items are placed, which are excluded from use in the interim ability estimation and the item selection procedures. The initial simulation study was conducted by Han (2013) who investigated the use of the IP method using a dichotomously-scored fixed length test. The findings indicated that the IP method does not substantially decrease measurement precision and bias in the ability estimates were within acceptable ranges for operational tests. This simulation study extended the IP method to a CAT using polytomously-scored items using the Generalized Partial Credit model with exposure control and content balancing. The IP method was implemented in tests with three IP sizes (2, 3, and 4), two termination criteria (fixed and variable), two test lengths (15 and 20), and two item completion conditions (forced to answer and ignored) for items remaining in the IP at the end of the test. Additionally, four traditional CAT conditions, without implementing the IP method, were included in the design. Results found that the longer, 20 item IP method conditions using the forced answer method had higher measurement precision, with higher mean correlations between known and estimated theta, lower mean bias and RMSE, and measurement precision increased as IP size increased. The two item completion conditions (forced to answer and ignored) resulted in similar measurement precision. The variable length IP conditions resulted in comparable measurement precision as the corresponding fixed length IP conditions. The implications of the findings and the limitations with suggestions for future research are also discussed.

Download or read book Statistical Aspects Of Computerized Adaptive Testing written by Haskell Sie and published by . This book was released on 2014 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: In the past several decades, Computerized Adaptive Testing (CAT) has received much attention in educational and psychological research due to the efficiency in achieving the goal of assessment, whether it is to estimate the latent trait of test takers with high precision or to accurately classify them into one of several latent classes. In the latter case, the adaptive nature of CAT is used in educational testing to make inferences about the location of examinees' latent ability relative to one or more pre-specified cut-off points along the ability continuum. When there is only one cut-off point and two proficiency groups, this type of CAT is commonly referred to as Adaptive Mastery Testing (AMT). A well-known approach in AMT is to combine the Sequential Probability Ratio Test (SPRT) stopping rule with item selection to maximize Fisher information at the mastery threshold. In the first part of this dissertation, a new approach is proposed in which a time limit is defined for the test and examinees' response times are considered in both item selection and test termination. Item selection is performed by maximizing Fisher information per time unit, rather than Fisher information itself. The test is terminated once the SPRT makes a classification decision, the time limit is exceeded, or there is no remaining item that has a high enough probability of being answered before the time limit. In a simulation study, the new procedure showed a substantial reduction in average testing time while slightly improving classification accuracy compared to the original method. In addition, the new procedure reduced the percentage of examinees who exceeded the time limit. Another well-known stopping rule in AMT is to terminate the assessment once the examinee's two-sided ability confidence interval lies entirely above or below the cut score. The second part of this dissertation proposes new procedures that seek to improve such a variable-length stopping rule by coupling it with curtailment and stochastic curtailment. Under the new procedures, test termination can occur earlier if the probability is high enough that the current classification decision remains the same should the test continue. Computation of this probability utilizes normality of an asymptotically equivalent version of the maximum likelihood estimate (MLE) of ability. In two simulation studies, the new procedures showed a substantial reduction in average test length (ATL) while maintaining similar classification accuracy to the original stopping rule based on the ability confidence interval. In the last part of this dissertation, generalization to multidimensional CAT (MCAT) is examined. Research has shown that MCAT improves the precision of both subscores and overall scores compared to its unidimensional counterpart. Several studies have investigated the performance of MCAT in recovering examinees' multiple abilities depending on the item selection methods. None of these studies, however, considered an item pool containing a mixture of multiple-choice (MC) and constructed-response (CR) items. With many assessments currently containing such a mixture of item types that measure more than one trait, there is an obvious need to understand how different item selection methods choose different types of items depending on their dimensional loadings (simple-structure versus complex-structured) and location of maximum information. In a simulation study, performance of five MCAT item selection methods were compared using an item pool consisting of a mixture between MC and CR items for mixed-format assessments. Ability recovery as well as item preferences of each method (simple- versus complex-structured items and location of maximum information) were examined.

Download or read book Evaluating the Impacts of Item Exposure Procedures on Ability Estimates in CAT When Items are Disclosed written by Wei He and published by . This book was released on 2008 with total page 30 pages. Available in PDF, EPUB and Kindle. Book excerpt: Test security has been a concern for computerized adaptive tests (CAT) due to the nature of continuous testing. This concern becomes unprecedentedly severe with increasingly easy access to the World-Wide-Web where some examinees post on the internet their recollections of items they are administered, leaving future examinees with opportunities to get their scores inflated by having pre-knowledge of the items that appear in their tests. This study investigated, given some items in the item pool are disclosed, which item exposure control procedure results in minimum negative effect on ability estimate. The disclosed items in this study are defined as items released to the public or intentionally memorized by the test-takers and shared with future examinees and items to which examinees can respond correctly. The results from both simulation and statistical hypothesis testing are in consistent with Schnipke & Scrams (1999) in terms that the presence of disclosed items can inflate the final ability estimates in the context of CATs by an unacceptably high degree, especially for the low-ability examinees. Among all item exposure procedures under study, the a-stratified test with b-blocking, i.e., a design that stratifies the item pool but takes care of the relationship between item difficulty and discrimination parameters, performs the best. (Contains 18 tables, 3 figures, and 4 footnotes.).

Download or read book Effect of Person Cluster on Accuracy of Ability Estimation of Computerized Adaptive Testing in K 12 Education Assessment written by Shudong Wang and published by . This book was released on 2011 with total page 31 pages. Available in PDF, EPUB and Kindle. Book excerpt: The ability estimation procedure is one of the most important components in a computerized adaptive testing (CAT) system. Currently, all CATs that provide K-12 student scores are based on the item response theory (IRT) model(s); while such application directly violates the assumption of independent sample of a person in IRT models because ability estimation is mostly based on cluster (or correlated) educational data in which students usually are clustered in certain groups or settings (classrooms or schools). The consequences of such violations are commonly ignored. The purpose of this study is to investigate the effect of ignoring hierarchical data structures of students sample on the accuracy of ability estimation by using a regular Rasch model. Results show that ICCs have not only statistically significant effect on the accuracy of a person's ability estimation, but also large effect sizes. (Contains 13 tables and 7 figures.).

Download or read book Impact of Local Item Dependence on Item Response Theory Scoring in CAT written by Lynda M. Reese and published by . This book was released on 1999 with total page 30 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Variable length Computerized Adaptive Testing Adaptation of the A stratified Strategy in Item Selection with Content Balancing written by Yan Huo and published by . This book was released on 2010 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Variable-length computerized adaptive testing (CAT) can provide examinees with tailored test lengths. With the fixed standard error of measurement (SEM) termination rule, variable-length CAT can achieve predetermined measurement precision by using relatively shorter tests compared to fixed-length CAT. To explore the application of variable-length CAT, this dissertation proposes four variable-length item selection methods adapted from the a-stratified strategy (Chang & Ying, 1999). These methods are named 1) the circularly increasing a-stratified method (STR-Ca), 2) the circularly decreasing a-stratified method (STR-Cd), 3) the random a-stratified method (STR-R), and 4) the two-stage a-stratified variable-length method (STR+R). The general strategy of these four methods allows test items to be selected in a mixed-strata ordering fashion from all strata partitioned by different levels of the discrimination parameter. This flexibility can overcome the potential problem of unbalanced item usage across different strata caused by previous attempts of applying the original a-stratified method into variable-length CAT. Study 1 examines the STR-Ca, the STR-Cd, and the STR-R methods in fixed-length CAT situations and the results show that their performance is comparable to that of the original a-stratified method in the fixed-length simulations in terms of various criterion measures such as Bias, MSE, efficiency, and item exposure rates. Study 2 explores these four item selection methods under the variable-length situations and the results indicate that these four methods can achieve good ability estimation while maintaining balanced item usage in the variable-length CAT simulations. To extend the implementation of these four variable-length item selection methods into a more realistic testing situation with content balancing constraints, Study 3 proposes two two-phase content balancing control methods, the variable-length modified multinomial model (MMM) method and the content weighted item selection index method. They can be naturally incorporated with these four adapted a-stratified methods to realize variable-length CAT with content control. Lastly, the intent of Study 4 is to explore decision making tools regarding choices among several variable-length CAT designs. Two quantitative indices, the cost-effective ratio and the variable-fixed-fitness index, are developed and their applications are demonstrated with some hypothetical examples. Together, these study findings will advance the research and understanding of variable-length CAT, and will facilitate the application and adoption of variable-length CAT in real world testing.

Download or read book The Impact of Missing Data on Parameter Estimation in Computerized Adaptive Testing written by Xiaowen Liu and published by . This book was released on 2021 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Computer adaptive testing (CAT) targets items to student ability level and creates a missing data pattern that may not be ignorable, depending on the use of the testing data. The current study investigates the impact of missingness on parameter recovery in CAT by examining parameter estimation and the sampling distributions of parameter estimates in several testing conditions. Overall, recalibration of item parameters after an operational CAT was reasonable despite the missing data challenges. Under some conditions, a subset of item discrimination parameters were estimated to be negative, resulting in a discontinuity in the ability continuum. A cluster analysis revealed the extent to which post-CAT data segments students and items. The ignorability of missingness in process data in CAT was also explored. Results for the estimation of response time and accuracy models, t separately and jointly, showed the importance of including the observed responses relevant to the missing data mechanism.

Download or read book Bulletin of the Atomic Scientists written by and published by . This book was released on 1955-04 with total page 64 pages. Available in PDF, EPUB and Kindle. Book excerpt: The Bulletin of the Atomic Scientists is the premier public resource on scientific and technological developments that impact global security. Founded by Manhattan Project Scientists, the Bulletin's iconic "Doomsday Clock" stimulates solutions for a safer world.

Download or read book Bulletin of the Atomic Scientists written by and published by . This book was released on 1970-12 with total page 104 pages. Available in PDF, EPUB and Kindle. Book excerpt: The Bulletin of the Atomic Scientists is the premier public resource on scientific and technological developments that impact global security. Founded by Manhattan Project Scientists, the Bulletin's iconic "Doomsday Clock" stimulates solutions for a safer world.

Download or read book Elements of Adaptive Testing written by Wim J. van der Linden and published by Springer Science & Business Media. This book was released on 2010-03-10 with total page 433 pages. Available in PDF, EPUB and Kindle. Book excerpt: The arrival of the computer in educational and psychological testing has led to the current popularity of adaptive testing---a testing format in which the computer uses statistical information about the test items to automatically adapt their selection to a real-time update of the test taker’s ability estimate. This book covers such key features of adaptive testing as item selection and ability estimation, adaptive testing with multidimensional abilities, sequencing adaptive test batteries, multistage adaptive testing, item-pool design and maintenance, estimation of item and item-family parameters, item and person fit, as well as adaptive mastery and classification testing. It also shows how these features are used in the daily operations of several large-scale adaptive testing programs.

Download or read book Computerized Adaptive Testing Theory and Practice written by Wim J. van der Linden and published by Springer Science & Business Media. This book was released on 2000-07-31 with total page 327 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book offers a comprehensive introduction to the latest developments in the theory and practice of CAT. It can be used both as a basic reference and a valuable resource on test theory. It covers such topics as item selection and ability estimation, item pool development and maintenance, item calibration and model fit, and testlet-based adaptive testing, as well as the operational aspects of existing large-scale CAT programs.

Download or read book Resources in Education written by and published by . This book was released on 1996 with total page 1112 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Practical Considerations in Computer Based Testing written by Cynthia G. Parshall and published by Springer Science & Business Media. This book was released on 2012-12-06 with total page 233 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book introduces computer-based testing, addressing both nontechnical and technical considerations. The material is oriented toward practitioners and graduate students. The practical emphasis will be useful to measurement professionals who are or will be responsible for implementing a computerized testing program. The instructional information is also designed to be suitable for a one-semester graduate course in computerized testing in an educational measurement or quantitative methods program. While certain theoretical concepts are addressed, the focus of the book is on the applied nature of computerized testing. For this reason, the materials include such features as example applications, figures, and plots to illustrate critical points in the discussions. A wide range ofnontechnical issues need to be considered in implementing a computer-based testing program. Separate chapters are provided on test administration and development issues, examinee issues, software issues, and innovative item types. Test administration and delivery issues include the location of exam administration, selection of hardware and software, security considerations, scheduling of administration frequency and time limits, cost implications, and program support as well as approaches for addressing reliability, validity, comparability, and data analysis. Examinee issues include the influence ofexaminees' reactions to adaptive testing, the effect ofcomputer based task constraints, and the impact of examinees' prior computer experience. Software issues include usability studies and software evaluation as tools in selecting and developing appropriate software, based on the test program needs.