Guide
Guide
A direct page for the workshop guide, bringing the main repository context into the site navigation.
Repository Overview
This project contains the full material for a hands-on workshop on latent-ability-aware evaluation in machine learning.
Main Message
- aggregate metrics are useful, but incomplete;
- even in supervised settings, local example difficulty matters;
- IRT makes ability, difficulty, and discrimination explicit;
- Beta4-IRT extends that logic to bounded responses;
- unsupervised disagreement becomes the bridge into CLAIRE;
- CLAIRE turns model agreement into a latent-evaluation workflow for clustering.
Main Sections
- Supervised evaluation as motivation, followed by Binary IRT, 1PL intuition, 2PL-IRT, and ICC interpretation.
- Beta4-IRT for bounded responses and synthetic recovery checks.
- CLAIRE as the final block, beginning with unsupervised evaluation and then moving into agreement-based latent analysis.
