Download or read book Comparing Item Selection Methods in Computerized Adaptive Testing Using the Rating Scale Model written by Meredith Sibley Butterfield and published by . This book was released on 2016 with total page 346 pages. Available in PDF, EPUB and Kindle. Book excerpt: Computer Adaptive Testing (CAT), a form of computer-based testing that selects and administers items that match the examinee’s trait levels, can be shorter in length and maintain comparable or greater measurement precision than traditional fixed-length paper-and-pencil testing. Administration of computer-based patient reported outcome (PRO) measures has increased recently in the medical field. Because PRO measures often have small item pools, small numbers of items administered, and populations in poor health, the benefits of CATs are especially advantageous. In CAT, Maximum Fisher information (MFI) is the most commonly used item selection procedure since it is easy to use and computationally simple. However, its main drawback is the attenuation paradox. If the estimated trait level of the examinee is not the true trait level, the items selected will not maximize information at the true trait level and the measurement is less precise. To address this issue, alternative item selections methods have been proposed. In studies, these alternatives have not performed better than MFI. Recently, Gradual Maximum Information Ratio (GMIR) item selection method was proposed and previous findings suggest GMIR could be beneficial for a short CAT. This simulation study compared GMIR and MFI item selection methods under conditions specific to the constraints of the PRO measures. GMIR and MFI are compared under Andrich’s Rating Scale Model (ARSM) across two polytomous item pool sizes (41 and 82), two population latent trait distributions (normal and negatively skewed), and three combination maximum number of item and minimum standard error stopping rules (5/0.54, 7/0.46, 9/0.40). The conditions were fully crossed. Performance was evaluated in terms of descriptive statistics of the final trait estimates, measurement precision, conditional measurement precision, and administration efficiency. Results found GMIR had better measurement precision when the test length was 5 items, with higher mean correlations between known and estimated trait levels, smaller mean bias, and smaller mean RMSE. An effect of item pool size and population latent trait distribution was not found. Across item selection methods, measurement precision increased as the test length increase, but with diminishing returns from 7 to 9 items.
Download or read book A Comparison of Item Selection Procedures Using Different Ability Estimation Methods in Computerized Adaptive Testing Based on the Generalized Partial Credit Model written by Tsung-Han Ho and published by . This book was released on 2010 with total page 360 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Item Selection Methods in Polytomous Computerized Adaptive Testing written by Usama S. Ali and published by . This book was released on 2011 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Given the rapid advancement of computer technology, the importance of administering adaptive tests with polytomous items is in great need. With regard to the applicability of adaptive testing using polytomous IRT models, adaptive testing can use polytomous items of either rating scales, or in some testing situations of multiple choice. Additionally, the availability of computerized polytomous scoring of open-ended items enhances such applicability. This need promotes the research in polytomous adaptive testing (PAT). This dissertation is an e ort to focus on item selection methods, as a major component, in polytomous computerized adaptive testing. So, it consists of ve chapters that cover the following: Chapter 1 focuses on a thorough introduction to the item response theory (IRT) models and adaptive testing related to polytomous items. Such an important overview and introduction to basic concepts in test theory and mathematical models for polytomous items is needed for the ow of consequent chapters. Chapter 2 is devoted to the development of a central location index (LI) to uniquely represent the polytomous item with a scale value parameter using most commonly used polytomous models. The motivation and rationale to search for a central or an overall location parameter is twofold: a) the confusion of multiple and di erent parameterizations for a polytomous item even for the same model, and b) the unavailability of such single location parameter block the usage of certain item selection methods in adaptive testing. Two approaches are used to derive the proposed LIs, one is based on the item category response functions (ICRFs) and the other is based on the polytomous item response function (IRF). As a result, four LIs are proposed. Chapter 3 is particularly assigned to development of an item selection method based on the developed location index and primarily assess its performance in the PAT context relative to existing methods. This method belongs to the non-information based item selection methods and we referred it as Matching-LI method. The results support that this proposed method is promising and is capable to produce accurate ability estimates and successfully manage the item pool usage. Chapter 4 introduces new item selection methods taking in consideration the previous chapter's results. The new methods are the hybrid, stage-based information, polytomous a-strati cation methods. The first two methods try to merge more than one criterion for selecting items of each PAT (e.g., the hybrid method merges both the Matching-LI and maximum information (MI) methods). The last method uses Matching-LI method within each stratum. Chapter 5 provides discussion, conclusions, and limitations and future research directions with respect to important components of an adaptive testing program (i.e., item selection methods, item response models, item banks, and trait versus attribute estimation).
Download or read book Comparison of Ability Estimation and Item Selection Methods in Multidimensional Computerized Adaptive Testing written by Qi Diao and published by . This book was released on 2009 with total page 246 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Theoretical and Practical Advances in Computer based Educational Measurement written by Bernard P. Veldkamp and published by Springer. This book was released on 2019-07-05 with total page 399 pages. Available in PDF, EPUB and Kindle. Book excerpt: This open access book presents a large number of innovations in the world of operational testing. It brings together different but related areas and provides insight in their possibilities, their advantages and drawbacks. The book not only addresses improvements in the quality of educational measurement, innovations in (inter)national large scale assessments, but also several advances in psychometrics and improvements in computerized adaptive testing, and it also offers examples on the impact of new technology in assessment. Due to its nature, the book will appeal to a broad audience within the educational measurement community. It contributes to both theoretical knowledge and also pays attention to practical implementation of innovations in testing technology.
Download or read book Computerized Adaptive Testing written by David J. Weiss and published by Guilford Publications. This book was released on 2024-06-12 with total page 387 pages. Available in PDF, EPUB and Kindle. Book excerpt: "CAT is the use of artificial intelligence and machine learning to create a test unique to each person in real time. CAT: From Concept to Implementation is designed to take you from your initial decision to implement a CAT through each step in the process of fielding a CAT that will provide its benefits both to your examinees and to your organization. It describes how CATs are developed, and how they are implemented using current advanced psychometric methods"--
Download or read book Computerized Adaptive and Multistage Testing with R written by David Magis and published by Springer. This book was released on 2017-11-23 with total page 183 pages. Available in PDF, EPUB and Kindle. Book excerpt: The goal of this guide and manual is to provide a practical and brief overview of the theory on computerized adaptive testing (CAT) and multistage testing (MST) and to illustrate the methodologies and applications using R open source language and several data examples. Implementation relies on the R packages catR and mstR that have been already or are being developed by the first author (with the team) and that include some of the newest research algorithms on the topic. The book covers many topics along with the R-code: the basics of R, theoretical overview of CAT and MST, CAT designs, CAT assembly methodologies, CAT simulations, catR package, CAT applications, MST designs, IRT-based MST methodologies, tree-based MST methodologies, mstR package, and MST applications. CAT has been used in many large-scale assessments over recent decades, and MST has become very popular in recent years. R open source language also has become one of the most useful tools for applications in almost all fields, including business and education. Though very useful and popular, R is a difficult language to learn, with a steep learning curve. Given the obvious need for but with the complex implementation of CAT and MST, it is very difficult for users to simulate or implement CAT and MST. Until this manual, there has been no book for users to design and use CAT and MST easily and without expense; i.e., by using the free R software. All examples and illustrations are generated using predefined scripts in R language, available for free download from the book's website.
Download or read book A Comparison of Item selection Methods for Adaptive Tests with Content Constraints written by Wim J. van der Linden and published by . This book was released on 2005 with total page 32 pages. Available in PDF, EPUB and Kindle. Book excerpt: "In adaptive testing, items are selected for an individual test taker with the goal of administering a test that is, as closely as possible, tailored to the ability level of that test taker. The selection is sequential in that one item is selected at a time. At the same time, adaptive tests typically have to meet a large number of content constraints, and this requirement is solved more naturally by simultaneous item selection. In this project, the three main item-selection methods in adaptive testing for solving this dilemma were investigated: (1) the spiraling method (SM), which moves across content categories of items in the item pool in a manner that is proportional to the numbers of items needed from them during item selection, (2) the weighted-deviations method (WDM), which selects the items using a projection of a weighted sum of the attributes of the entire test, and (3) the shadow test approach (STA), which selects the items based on a projection of the actual items in the entire test. An empirical comparison among the methods was conducted for an adaptive version of the Law School Admission Test (LSAT)."--Publisher website.
Download or read book Handbook of Polytomous Item Response Theory Models written by Michael L. Nering and published by Routledge. This book was released on 2011-01-19 with total page 349 pages. Available in PDF, EPUB and Kindle. Book excerpt: This comprehensive Handbook focuses on the most used polytomous item response theory (IRT) models. These models help us understand the interaction between examinees and test questions where the questions have various response categories. The book reviews all of the major models and includes discussions about how and where the models originated, conceptually and in practical terms. Diverse perspectives on how these models can best be evaluated are also provided. Practical applications provide a realistic account of the issues practitioners face using these models. Disparate elements of the book are linked through editorial sidebars that connect common ideas across chapters, compare and reconcile differences in terminology, and explain variations in mathematical notation. These sidebars help to demonstrate the commonalities that exist across the field. By assembling this critical information, the editors hope to inspire others to use polytomous IRT models in their own research so they too can achieve the type of improved measurement that such models can provide. Part 1 examines the most commonly used polytomous IRT models, major issues that cut across these models, and a common notation for calculating functions for each model. An introduction to IRT software is also provided. Part 2 features distinct approaches to evaluating the effectiveness of polytomous IRT models in various measurement contexts. These chapters appraise evaluation procedures and fit tests and demonstrate how to implement these procedures using IRT software. The final section features groundbreaking applications. Here the goal is to provide solutions to technical problems to allow for the most effective use of these models in measuring educational, psychological, and social science abilities and traits. This section also addresses the major issues encountered when using polytomous IRT models in computerized adaptive testing. Equating test scores across different testing contexts is the focus of the last chapter. The various contexts include personality research, motor performance, health and quality of life indicators, attitudes, and educational achievement. Featuring contributions from the leading authorities, this handbook will appeal to measurement researchers, practitioners, and students who want to apply polytomous IRT models to their own research. It will be of particular interest to education and psychology assessment specialists who develop and use tests and measures in their work, especially researchers in clinical, educational, personality, social, and health psychology. This book also serves as a supplementary text in graduate courses on educational measurement, psychometrics, or item response theory.
Download or read book Multidimensional Item Response Theory written by M.D. Reckase and published by Springer Science & Business Media. This book was released on 2009-07-07 with total page 355 pages. Available in PDF, EPUB and Kindle. Book excerpt: First thorough treatment of multidimensional item response theory Description of methods is supported by numerous practical examples Describes procedures for multidimensional computerized adaptive testing
Download or read book Elements of Adaptive Testing written by Wim J. van der Linden and published by Springer Science & Business Media. This book was released on 2010-03-10 with total page 433 pages. Available in PDF, EPUB and Kindle. Book excerpt: The arrival of the computer in educational and psychological testing has led to the current popularity of adaptive testing---a testing format in which the computer uses statistical information about the test items to automatically adapt their selection to a real-time update of the test taker’s ability estimate. This book covers such key features of adaptive testing as item selection and ability estimation, adaptive testing with multidimensional abilities, sequencing adaptive test batteries, multistage adaptive testing, item-pool design and maintenance, estimation of item and item-family parameters, item and person fit, as well as adaptive mastery and classification testing. It also shows how these features are used in the daily operations of several large-scale adaptive testing programs.
Download or read book Item Selection Methods in Multidimensional Computerized Adaptive Testing Adopting Polytomously scored Items Under Multidimensional Generalized Partial Credit Model written by and published by . This book was released on 2012 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Language Testing in the 1990s written by J. Charles Alderson and published by . This book was released on 1991-01-01 with total page 256 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book A Comparison of Four Item Selection Methods for Severely Constrained CATs written by and published by . This book was released on 2014 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: This study compared four item-selection procedures developed for use with severely constrained computerized adaptive tests (CATs). Severely constrained CATs refer to those adaptive tests that seek to meet a complex set of constraints that are often not conclusive to each other (i.e., an item may contribute to the satisfaction of several constraints at the same time). The procedures examined in the study included the weighted deviation model (WDM), the weighted penalty model (WPM), the maximum priority index (MPI), and the shadow test approach (STA). In addition, two modified versions of the MPI procedure were introduced to deal with an edge case condition that results in the item selection procedure becoming dysfunctional during a test. The results suggest that the STA worked best among all candidate methods in terms of measurement accuracy and constraint management. For the other three heuristic approaches, they did not differ significantly in measurement accuracy and constraint management at the lower bound level. However, the WPM method appears to perform considerably better in overall constraint management than either the WDM or MPI method. Limitations and future research directions were also discussed.
Download or read book Extension of the Item Pocket Method Allowing for Response Review and Revision to a Computerized Adaptive Test Using the Generalized Partial Credit Model written by Mishan G. B. Jensen and published by . This book was released on 2017 with total page 346 pages. Available in PDF, EPUB and Kindle. Book excerpt: Computerized Adaptive Testing (CAT) has increased in the last few decades, due in part to the increased use and availability of personal computers, but also partly due to the benefits of CATs. CATs provide increased measurement precision of ability estimates while decreasing the demand on examinees with shorter tests. This is accomplished by tailoring the test to each examinee and selecting items that are not too difficult or too easy based on the examinees’ interim ability estimate and responses to previous items. These benefits come at the cost of the flexibility to move through the test as an examinee would with a Paper and Pencil (P & P) test. The algorithms used in CATs for item selection and ability estimation require restrictions to response review and revision; however, a large portion of examinees desire options for review and revision of responses (Vispoel, Clough, Bleiler, Hendrickson, and Ihrig, 2002). Previous research has examined response review and revision in CATs with limited review and revision options and are limited to after all items had been administered. The development of the Item Pocket (IP) method (Han, 2013) has allowed for response review and revision during the test, relaxing the restrictions, while maintaining an acceptable level of measurement precision. This is achieved by creating an item pocket in which items are placed, which are excluded from use in the interim ability estimation and the item selection procedures. The initial simulation study was conducted by Han (2013) who investigated the use of the IP method using a dichotomously-scored fixed length test. The findings indicated that the IP method does not substantially decrease measurement precision and bias in the ability estimates were within acceptable ranges for operational tests. This simulation study extended the IP method to a CAT using polytomously-scored items using the Generalized Partial Credit model with exposure control and content balancing. The IP method was implemented in tests with three IP sizes (2, 3, and 4), two termination criteria (fixed and variable), two test lengths (15 and 20), and two item completion conditions (forced to answer and ignored) for items remaining in the IP at the end of the test. Additionally, four traditional CAT conditions, without implementing the IP method, were included in the design. Results found that the longer, 20 item IP method conditions using the forced answer method had higher measurement precision, with higher mean correlations between known and estimated theta, lower mean bias and RMSE, and measurement precision increased as IP size increased. The two item completion conditions (forced to answer and ignored) resulted in similar measurement precision. The variable length IP conditions resulted in comparable measurement precision as the corresponding fixed length IP conditions. The implications of the findings and the limitations with suggestions for future research are also discussed.
Download or read book A Comparison of Item Selection Rules Including Precision Content and Exposure Considerations at the Early Stages of Computerized Adaptive Testing written by Shu-Ying Chen and published by . This book was released on 1999 with total page 444 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Computerized Adaptive Testing Theory and Practice written by Wim J. van der Linden and published by Springer Science & Business Media. This book was released on 2000-07-31 with total page 327 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book offers a comprehensive introduction to the latest developments in the theory and practice of CAT. It can be used both as a basic reference and a valuable resource on test theory. It covers such topics as item selection and ability estimation, item pool development and maintenance, item calibration and model fit, and testlet-based adaptive testing, as well as the operational aspects of existing large-scale CAT programs.
