Download or read book A Top down Approach for Optimally Designing Multistage adaptive Tests written by Hwanggyu Lim and published by . This book was released on 2019 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: In multistage-adaptive testing (MST), there are many interrelated design variables that impact the nature and quality of ability estimation. Previous research has identified general principles for the effective design of MSTs in terms of measurement performance. However, those principles are unlikely to apply uniformly to every testing context. The purpose of this dissertation is to propose a process of finding an MST design that has optimal measurement properties, given a specific set of test circumstances. To achieve this goal, an efficient strategy was introduced at each of three phases to discover the optimal design of the MST; constructing MSTs, systematically searching a design space of the MST, and evaluating the MST performance. For the first phase, a top-down approach was applied in this study. For the second phase, a way to systematically search the parameterized design space of an MST was used. For the third phase, a new analytical evaluation method for MST was proposed. In the dissertation, Study 1 proposed a new analytical evaluation method for MST. Using this new approach, measurement precision of ability estimation and classification accuracy could be derived analytically. The simulation results indicated that the new analytical method produced more exact measurement properties of an MST than the Monte Carlo simulation method. Therefore, the new analytical method would be the most efficient and competitive tool to asses measurement performance of an MST among other evaluation methods. Study 2 proposed a process to find a design of an MST that shows optimal measurement properties applying the three efficient strategies, given a specific set of testing context. The process consists of four important features: (1) setting a testing circumstance and MST design space, (2) systematically searching the MST design space using the top-down approach, (3) analytically evaluating measurement performance of an MST, and (4) computing objective functions. The suggested process was applied to a real item pool from a large-scale assessment. The results of the application study provided evidence that the process could be generalized to more complex and realistic test circumstances to create optimal designs of MST.
Download or read book Linear Models for Optimal Test Design written by Wim J. van der Linden and published by Springer Science & Business Media. This book was released on 2006-01-01 with total page 421 pages. Available in PDF, EPUB and Kindle. Book excerpt: Wim van der Linden was just given a lifetime achievement award by the National Council on Measurement in Education. There is no one more prominent in the area of educational testing. There are hundreds of computer-based credentialing exams in areas such as accounting, real estate, nursing, and securities, as well as the well-known admissions exams for college, graduate school, medical school, and law school - there is great need on the theory of testing. This book presents the statistical theory and practice behind constructing good tests e.g., how is the first test item selected, how are the next items selected, and when do you have enough items.
Download or read book Investigation of Optimal Design and Scoring for Adaptive Multi stage Testing written by Duanli Yan and published by . This book was released on 2010 with total page 161 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Investigating the Impact of a Mixed format Item Pool on Optimal Test Designs for Multistage Testing written by Ryoungsun Park and published by . This book was released on 2015 with total page 228 pages. Available in PDF, EPUB and Kindle. Book excerpt: The multistage testing (MST) has drawn increasing attention as a balanced format of adaptive testing that takes advantages of both fully-adaptive computerized adaptive testing (CAT) and paper-and-pencil (P\&P) tests. Most previous studies on MST have focused on purely dichotomous or polytomous item formats although the mixture of two item types (i.e., mixed-format) provides desirable psychometric properties by combining the strength of both item types. Given the dearth of studies investigating the characteristics of mixed-format MST, the current study conducted a simulation to identify important design factors impacting the measurement precision of mixed-format MST. The study considered several factors-namely, total points (40 and 60), MST structures (1-2-2 and 1-3-3), the proportion of polytomous items (10%, 30%, 50% and 70%), and the routing module design (purely dichotomous and a mixture of dichotomous and polytomous items) resulting in 32 total conditions. A total of 100 replications were performed, and 1,000 normally distributed examinees were generated in each replication. The performance of MST was evaluated in terms of the precision of ability estimation across the wide range of the scale. The study found that the longer test produced greater measurement precision while the 1-3-3 structure performed better than 1-2-2 structure. In addition, a larger proportion of polytomous items resulted in lower measurement precision through the reduced test information during the test construction. The interaction between the large proportion of polytomous items and the purely dichotomous routing module design was identified. Overall, the two factors of test length and the MST structure impacted the ability estimation, whereas the impact of the proportion of polytomous items and routing module design mirrored the item pool characteristic.
Download or read book Computerized Multistage Testing written by Duanli Yan and published by CRC Press. This book was released on 2016-04-19 with total page 532 pages. Available in PDF, EPUB and Kindle. Book excerpt: Unlike other forms of adaptive testing, multistage testing (MST) is highly suitable for testing educational achievement because it can be adapted to educational surveys and student testing. This volume provides the first unified source of information on the design, psychometrics, implementation, and operational use of MST. It shows how to apply theoretical statistical tools to testing in novel and useful ways. It also explains how to explicitly tie the assumptions made by each model to observable (or at least inferable) data conditions.
Download or read book Research for Practical Issues and Solutions in Computerized Multistage Testing written by Alina von Davier and published by Routledge. This book was released on 2025-01-03 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This volume presents a comprehensive collection of the latest research findings supporting the current and future implementations and applications of computerized multistage testing (MST). As a sequel to the widely acclaimed "Computerized Multistage Testing: Theory and Applications" (2014) by Yan, von Davier, and Lewis, this volume delves into the experiences, considerations, challenges, and lessons learned over the past years. It also offers practical approaches and solutions to the issues encountered. The topics covered include: purposeful MST designs, practical approaches for optimal design, assembly strategies for accuracy and efficiency, Hybrid designs, MST with natural language processing, practical routing considerations and methodologies, item calibration and proficiency estimation methods, routing and classification accuracy, added value of process data, prediction and evaluation of MST performance, cognitive diagnostic MST, differential item functioning, robustness of statistical methods, simulations, test security, the new digital large-scale SAT, software for practical assessment and simulations, AI impact, and the future of adaptive multistage testing. This volume is intended for students, faculty, researchers, practitioners, and education officers in the fields of educational measurement and evaluation, both in the United States and internationally.
Download or read book Optimal design of item pools for computerized adaptive tests written by Martha L. Stocking and published by . This book was released on 1996 with total page 25 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Multistage Adaptive Testing for a Large scale Classification Test written by and published by . This book was released on 2012 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book An Investigation of the Optimal Test Design for Multi stage Test Using the Generalized Partial Credit Model written by Ling-Yin Chen and published by . This book was released on 2010 with total page 296 pages. Available in PDF, EPUB and Kindle. Book excerpt: Although the design of Multistage testing (MST) has received increasing attention, previous studies mostly focused on comparison of the psychometric properties of MST with CAT and paper-and-pencil (P & P) test. Few studies have systematically examined the number of items in the routing test, the number of subtests in a stage, or the number of stages in a test design to achieve accurate measurement in MST. Given that none of the studies have identified an ideal MST test design using polytomously-scored items, the current study conducted a simulation to investigate the optimal design for MST using generalized partial credit model (GPCM). Eight different test designs were examined on ability estimation across two routing test lengths (short and long) and two total test lengths (short and long). The item pool and generated item responses were based on items calibrated from a national test consisting of 273 partial credit items. Across all test designs, the maximum information routing method was employed and the maximum likelihood estimation was used for ability estimation. Ten samples of 1,000 simulees were used to assess each test design. The performance of each test design was evaluated in terms of the precision of ability estimates, item exposure rate, item pool utilization, and item overlap. The study found that all test designs produced very similar results. Although there were some variations among the eight test structures in the ability estimates, results indicate that the performance overall of these eight test structures in achieving measurement precision did not substantially deviate from one another with regard to total test length and routing test length. However, results from the present study suggest that routing test length does have a significant effect on the number of non-convergent cases in MST tests. Short routing tests tended to result in more non-convergent cases, and the presence of fewer stage tests yielded more of such cases than structures with more stages. Overall, unlike previous findings, the results of the present study indicate that the MST test structure is less likely to be a factor impacting ability estimation when polytomously-scored items are used, based on GPCM.
Download or read book Enhancing Item Pool Utilization when Designing Multistage Computerized Adaptive Tests written by Lihong Yang and published by . This book was released on 2016 with total page 142 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Computerized Multistage Testing written by Duanli Yan and published by . This book was released on 2016 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Devising tests that evaluate a nation's educational standing and implement efficacious educational reforms requires a careful balance among the contributions of technology, psychometrics, test design, and the learning sciences. Unlike other forms of adaptive testing, multistage testing (MST) is highly suitable for testing educational achievement be.
Download or read book Computerized Adaptive Testing Theory and Practice written by Wim J. van der Linden and published by Springer Science & Business Media. This book was released on 2000-07-31 with total page 327 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book offers a comprehensive introduction to the latest developments in the theory and practice of CAT. It can be used both as a basic reference and a valuable resource on test theory. It covers such topics as item selection and ability estimation, item pool development and maintenance, item calibration and model fit, and testlet-based adaptive testing, as well as the operational aspects of existing large-scale CAT programs.
Download or read book Elements of Adaptive Testing written by Wim J. van der Linden and published by Springer Science & Business Media. This book was released on 2010-03-10 with total page 433 pages. Available in PDF, EPUB and Kindle. Book excerpt: The arrival of the computer in educational and psychological testing has led to the current popularity of adaptive testing---a testing format in which the computer uses statistical information about the test items to automatically adapt their selection to a real-time update of the test taker’s ability estimate. This book covers such key features of adaptive testing as item selection and ability estimation, adaptive testing with multidimensional abilities, sequencing adaptive test batteries, multistage adaptive testing, item-pool design and maintenance, estimation of item and item-family parameters, item and person fit, as well as adaptive mastery and classification testing. It also shows how these features are used in the daily operations of several large-scale adaptive testing programs.
Download or read book Item Response Theory written by Ronald K. Hambleton and published by Springer Science & Business Media. This book was released on 2013-11-11 with total page 340 pages. Available in PDF, EPUB and Kindle. Book excerpt: In the decade of the 1970s, item response theory became the dominant topic for study by measurement specialists. But, the genesis of item response theory (IRT) can be traced back to the mid-thirties and early forties. In fact, the term "Item Characteristic Curve," which is one of the main IRT concepts, can be attributed to Ledyard Tucker in 1946. Despite these early research efforts, interest in item response theory lay dormant until the late 1960s and took a backseat to the emerging development of strong true score theory. While true score theory developed rapidly and drew the attention of leading psychometricians, the problems and weaknesses inherent in its formulation began to raise concerns. Such problems as the lack of invariance of item parameters across examinee groups, and the inadequacy of classical test procedures to detect item bias or to provide a sound basis for measurement in "tailored testing," gave rise to a resurgence of interest in item response theory. Impetus for the development of item response theory as we now know it was provided by Frederic M. Lord through his pioneering works (Lord, 1952; 1953a, 1953b). The progress in the fifties was painstakingly slow due to the mathematical complexity of the topic and the nonexistence of computer programs.
Download or read book Modern Psychometrics with R written by Patrick Mair and published by Springer. This book was released on 2018-09-20 with total page 464 pages. Available in PDF, EPUB and Kindle. Book excerpt: This textbook describes the broadening methodology spectrum of psychological measurement in order to meet the statistical needs of a modern psychologist. The way statistics is used, and maybe even perceived, in psychology has drastically changed over the last few years; computationally as well as methodologically. R has taken the field of psychology by storm, to the point that it can now safely be considered the lingua franca for statistical data analysis in psychology. The goal of this book is to give the reader a starting point when analyzing data using a particular method, including advanced versions, and to hopefully motivate him or her to delve deeper into additional literature on the method. Beginning with one of the oldest psychometric model formulations, the true score model, Mair devotes the early chapters to exploring confirmatory factor analysis, modern test theory, and a sequence of multivariate exploratory method. Subsequent chapters present special techniques useful for modern psychological applications including correlation networks, sophisticated parametric clustering techniques, longitudinal measurements on a single participant, and functional magnetic resonance imaging (fMRI) data. In addition to using real-life data sets to demonstrate each method, the book also reports each method in three parts-- first describing when and why to apply it, then how to compute the method in R, and finally how to present, visualize, and interpret the results. Requiring a basic knowledge of statistical methods and R software, but written in a casual tone, this text is ideal for graduate students in psychology. Relevant courses include methods of scaling, latent variable modeling, psychometrics for graduate students in Psychology, and multivariate methods in the social sciences.
Download or read book Theoretical and Practical Advances in Computer based Educational Measurement written by Bernard P. Veldkamp and published by Springer. This book was released on 2019-07-05 with total page 399 pages. Available in PDF, EPUB and Kindle. Book excerpt: This open access book presents a large number of innovations in the world of operational testing. It brings together different but related areas and provides insight in their possibilities, their advantages and drawbacks. The book not only addresses improvements in the quality of educational measurement, innovations in (inter)national large scale assessments, but also several advances in psychometrics and improvements in computerized adaptive testing, and it also offers examples on the impact of new technology in assessment. Due to its nature, the book will appeal to a broad audience within the educational measurement community. It contributes to both theoretical knowledge and also pays attention to practical implementation of innovations in testing technology.
Download or read book Handbook of Test Development written by Suzanne Lane and published by Routledge. This book was released on 2015-10-08 with total page 676 pages. Available in PDF, EPUB and Kindle. Book excerpt: The second edition of the Handbook of Test Development provides graduate students and professionals with an up-to-date, research-oriented guide to the latest developments in the field. Including thirty-two chapters by well-known scholars and practitioners, it is divided into five sections, covering the foundations of test development, content definition, item development, test design and form assembly, and the processes of test administration, documentation, and evaluation. Keenly aware of developments in the field since the publication of the first edition, including changes in technology, the evolution of psychometric theory, and the increased demands for effective tests via educational policy, the editors of this edition include new chapters on assessing noncognitive skills, measuring growth and learning progressions, automated item generation and test assembly, and computerized scoring of constructed responses. The volume also includes expanded coverage of performance testing, validity, fairness, and numerous other topics. Edited by Suzanne Lane, Mark R. Raymond, and Thomas M. Haladyna, The Handbook of Test Development, 2nd edition, is based on the revised Standards for Educational and Psychological Testing, and is appropriate for graduate courses and seminars that deal with test development and usage, professional testing services and credentialing agencies, state and local boards of education, and academic libraries serving these groups.
