Robert Joseph Mislevy (known to friends and colleagues as Bob) was born June 28, 1950, and passed away at the age of 74 in Severna Park, Maryland, USA on May 22, 2025, with his daughters by his side. In his career, Bob had tremendous impact on research, practice, and philosophy. He was a past-president of the Psychometric Society, an elected member of the National Academy of Education, an elected fellow of the American Educational Research Association, and served on National Academies of Science committees concerning assessment, instruction, and psychology. Among other honors, he received the Award for Significant Contribution to Educational Measurement and Research Methodology from the American Educational Research Association, the E.F. Lindquist Award from the American Educational Research Association and ACT, the National Council on Measurement in Education’s Award for Exceptional Achievement in Educational Measurement (five times), and awards for career and lifetime contributions from the Psychometric Society and the National Council on Measurement in Education.

We introduce a novel regularization method for detecting differential item functioning (DIF) in two-parameter logistic (2PL) models. Existing regularization methods require choosing a reference group and using an $L_1$ penalty (LP) to shrink the item parameters of focal groups toward those of the reference. This approach has two key limitations: (1) shrinking all focal groups toward a reference is inherently unfair, as results are affected by the choice of reference and direct comparison among focal groups is unavailable and (2) the LP leads to biased estimates because it overly shrinks large nonzero parameters toward zero. These limitations are particularly problematic for intersectional DIF, where various identity aspects intersect to create multiple smaller groups. Our method addresses these issues by penalizing item parameter differences between all pairs of groups using a truncated LP, thereby treating groups equally and avoiding excessive penalization of large differences. Simulations demonstrate that the proposed method outperforms existing approaches by accurately identifying items exhibiting DIF even with multiple small groups. Application to two real-world datasets further illustrates its utility. We recommend this method as a more equitable and precise tool for DIF detection. The proposed method is available as D2PL_pair_em() in the R package VEMIRT (https://map-lab-uw.github.io/VEMIRT).

Process data, in particular, log data collected from a computerized test, documents the sequence of actions performed by an examinee in pursuit of solving a problem, affording an opportunity to understand test-taking behavioral patterns that account for demographic group differences in key outcomes of interest, for instance, final score on a cognitive item. Addressing this aim, this article proposes a latent class mediation analysis procedure. Using continuous process features extracted from action sequence data as indicators, latent classes underlying the test-taking behavior are identified in a latent class mediation model, where an examinee’s nominal latent class membership enters as the mediator between the observed grouping and outcome variables. A headlong search algorithm for selecting the subset of process features that maximizes the total indirect effect of the latent class mediator is implemented. The proposed procedure is validated with a series of simulations. An application to a large-scale assessment highlights how the proposed method can be used to explain performance gaps between students with learning disability and their typically developing peers on the National Assessment of Educational Progress (NAEP) math assessment.

Cognitive Diagnostic Models (CDMs) are popular discrete latent variable models in educational and psychological measurement. While existing CDMs mainly focus on binary or categorical responses, there is a growing need to extend them to model a wider range of response types, including but not limited to continuous and count-valued responses. Meanwhile, incorporating higher-order latent structures has become crucial for gaining deeper insights into cognitive processes. We propose a general modeling framework for higher-order CDMs for rich types of responses. Our framework features a highly flexible data layer that is adaptive to various response types and measurement models for CDMs. Importantly, we address a challenging exploratory estimation scenario where the item-attribute relationship, specified by the Q-matrix, is unknown and needs to be estimated along with other parameters. In the higher-order layer, we employ a probit-link with continuous latent traits to model the binary latent attributes, highlighting its benefits in terms of identifiability and computational efficiency. Theoretically, we propose transparent identifiability conditions for the exploratory setting. Computationally, we develop an efficient Monte Carlo Expectation–Maximization algorithm, which incorporates an efficient direct sampling scheme and requires significantly reduced simulated samples. Extensive simulation studies and a real data example demonstrate the effectiveness of our methodology.

Reliability analysis is one of the most conducted analyses in applied psychometrics. It entails the assessment of reliability of both item scores and scale scores using coefficients that estimate the reliability (e.g., Cronbach’s alpha), measurement precision (e.g., estimated standard error of measurement), or the contribution of individual items to the reliability (e.g., corrected item-total correlations). Most statistical software packages used in social and behavioral sciences offer these reliability coefficients, whereas standard errors are generally unavailable, which is a bit ironic for coefficients about measurement precision. This article provides analytic nonparametric standard errors for coefficients used in reliability analysis. As most scores used in behavioral sciences are discrete, standard errors are derived under the relatively unrestrictive multinomial sampling scheme. Tedious derivations are presented in appendices, and R functions for computing standard errors are available from the Open Science Framework. Bias and variance of standard errors, and coverage of the corresponding Wald-based confidence intervals are studied using simulated item scores. Bias and variance, and coverage are generally satisfactory for larger sample sizes, and parameter values are not close to the boundary of the parameter space.

The article proposes a novel item response theory model to handle continuous responses and sparse polytomous responses in psychological and educational measurement. The model extends the traditional two-parameter logistic model by incorporating a precision parameter, which, along with a beta distribution, forms an error component that accounts for the response continuity. Furthermore, transforming ordinal responses to a continuous scale enables the fitting of polytomous item responses while consistently applying three parameters per item for model parsimony. The model’s accuracy, stability, and computational efficiency in parameter estimation were examined. An empirical application demonstrated the model’s effectiveness in representing the characteristics of continuous item responses. Additionally, the model’s applicability to sparse polytomous data was supported by cross-validation results from another empirical dataset, which indicates that the model’s parsimony can enhance model-data fit compared to existing polytomous models.

In maximum likelihood factor analysis, we need to solve a complicated system of algebraic equations, known as the normal equation, to get maximum likelihood estimates (MLEs). Since this equation is difficult to solve analytically, its solutions are typically computed with continuous optimization methods, such as the Newton–Raphson method. With this procedure, however, the MLEs are dependent on initial values since the log-likelihood function is highly non-concave. Particularly, the estimates of unique variances can result in zero or negative, referred to as improper solutions; in this case, the MLE can be severely unstable. To delve into the issue of the instability, we algebraically compute all candidates for the MLE. We provide an algorithm based on algebraic computations that is carefully designed for maximum likelihood factor analysis. To be specific, Gröbner bases are employed, powerful tools to get simplified sub-problems for given systems of algebraic equations. Our algebraic algorithm provides the MLE independent of the initial values. While computationally demanding, our algebraic approach is applicable to small-scale problems and provides valuable insights into the characterization of improper solutions. For larger-scale problems, we provide numerical methods as practical alternatives to the algebraic approach. We perform numerical experiments to investigate the characteristics of the MLE with our two approaches.

Advances in deep learning and representation learning have transformed item factor analysis (IFA) in the item response theory (IRT) literature by enabling more efficient and accurate parameter estimation. Variational autoencoders (VAEs) are widely used to model high-dimensional latent variables in this context, but the limited expressiveness of their inference networks can still hinder performance. We introduce adversarial variational Bayes (AVB) and an importance-weighted extension (IWAVB) as more flexible inference algorithms for IFA. By combining VAEs with generative adversarial networks (GANs), AVB uses an auxiliary discriminator network to frame estimation as a two-player game and removes the restrictive standard normal assumption on the latent variables. Theoretically, AVB and IWAVB can achieve likelihoods that match or exceed those of VAEs and importance-weighted autoencoders (IWAEs). In exploratory analyses of empirical data, IWAVB attained higher likelihoods than IWAE, indicating greater expressiveness. In confirmatory simulations, IWAVB achieved comparable mean-square error in parameter recovery while consistently yielding higher likelihoods, and it clearly outperformed IWAE when the latent distribution was multimodal. These findings suggest that IWAVB can scale IFA to complex, large-scale, and potentially multimodal settings, supporting closer integration of psychometrics with modern multimodal data analysis.

Numerous applications in psychological research require that a data set is partitioned via the inverse of a clustering criterion. This anticlustering seeks for high similarity between groups (maximum diversity) or high pairwise dissimilarity within groups (maximum dispersion). Brusco et al. (2020) proposed a bicriterion heuristic (BILS) that simultaneously seeks for maximum diversity and dispersion, introducing the bicriterion approach for anticlustering. We investigate if the bicriterion approach can be improved using exact algorithms that guarantee globally optimal criterion values. Despite the theoretical computational intractability of anticlustering, we present a new exact algorithm for maximum dispersion that scales to quite large data sets ($N = 1,000$). However, a fully exact bicriterion approach was only feasible for small data sets (about $N = 30$). We therefore developed hybrid approaches that maintain optimal dispersion but use heuristics to maximize diversity on top of it. In a simulation study and an example application, we compared several hybrid approaches. An adaptation of BILS that initiates each iteration with a partition having optimal dispersion (BILS-Hybrid-All) performed best across a variety of data inputs. All of the methods developed here as well as the original BILS algorithm are available via the free and open-source R package anticlust.

Multivariate analysis of psychological variables using graphical models has become a standard analysis in the psychometric literature. Most cross-sectional measures are either binary or ordinal, and the methodology for inferring the structure of networks of binary and ordinal variables is developing rapidly. In practice, however, research questions often focus on whether and how networks differ between observed groups. While Bayes factor methods for inferring network structure are well established, a similar methodology for assessing group differences in networks of binary or ordinal variables is currently lacking. In this article, we extend the Bayesian framework for the analysis of ordinal Markov random fields, a network model for binary and ordinal variables, and develop Bayes factor tests for assessing parameter differences in the networks of two independent groups. The proposed methods are implemented in the R package bgms, and we use numerical illustrations to show that the implemented methods work correctly and how well the methods work compared to existing methods in situations resembling empirical research.

We propose an optimally regularized Bayesian estimator of multilevel latent variable models that aims to outperform traditional maximum likelihood (ML) estimation in mean squared error (MSE) performance. We focus on the between-group slope in a two-level model with a latent covariate. Our estimator combines prior information with data-driven insights for optimal parameter estimation. We present a “proof of concept” by computer simulations, involving varying numbers of groups, group sizes, and intraclass correlations (ICCs), which we conducted to compare the newly proposed estimator with ML. Additionally, we provide a step-by-step tutorial on applying the regularized Bayesian estimator to real-world data using our MultiLevelOptimalBayes package.

Encouragingly, our results show that our estimator offers improved MSE performance, especially in small samples with low ICCs. These findings suggest that the estimator can be an effective means for enhancing estimation accuracy.

When the row and column variables consist of the same category in a two-way contingency table, it is called a square contingency table. Since square contingency tables have an association structure due to the concentration of observed values near the main diagonal, a primary objective is to examine symmetric relationships and transitions between variables. Various models and measures have been proposed to analyze these structures to understand the changes between two variables’ behavior at two-time points or cohorts. This is necessary for a detailed investigation of individual categories and their interrelationships, such as shifts in brand preferences. We propose a novel approach to correspondence analysis (CA) for evaluating departures from symmetry in square contingency tables with nominal categories, using a modified divergence statistic. This approach ensures that well-known divergence statistics can also be visualized and regardless of the divergence statistics used, the CA plot consists of two principal axes with equal contribution rates. Notably, the scaling of the departures from symmetry provided by the modified divergence statistic is independent of sample size, allowing for meaningful comparisons and unification of results across different tables. Confidence regions are also constructed to enhance the accuracy of the CA plot.

A ranking pattern approach is proposed to build item response theory (IRT) models for forced-choice (FC) items. This new approach is an addition to the two existing approaches, sequential selection and Thurstone’s law of pairwise comparison. A new dominance IRT model, the multidimensional generalized partial preference model (MGPPM), is proposed for FC items with any number (greater than 1) of statements. The maximum marginal likelihood estimation using an expectation-maximization algorithm (MML-EM) and Markov chain Monte Carlo (MCMC) estimation are developed. A simulation study is conducted to show satisfactory parameter recovery on triplet and tetrad data. The relationships between the newly proposed approach/model and the existing approaches/models are described, and the MGPPM, Thurstonian IRT (TIRT) model, and Triplet-2PLM are compared when applied to simulated and real triplet data. The new approach offers more flexible IRT modeling than the other two approaches under different assumptions, and the MGPPM is more statistically elegant than the TIRT and Triple-2PLM.