Methodology

The rigor story behind every number the platform produces.

Method, not results: four commitments behind every number the platform produces.

Ask About the Methodology See the Full Platform

1. Measurement Before Prediction

A trait has to be measured well before it gets to predict anything.

A single score on a report actually rests on two different claims: that an instrument measures a stable, real trait consistently and honestly (a measurement model), and that the trait predicts something people care about, like job performance (a prediction model). It is tempting to test only the second claim by running everything against the outcome and call whatever correlates “validated.” That shortcut flatters weak instruments: a shaky measurement can look validated on the strength of a lucky correlation, while a well-measured trait that predicts a narrower set of outcomes can look weaker than it actually is.

FitSelect Insight keeps the two apart on purpose. A trait is validated as a measurement and checked for internal consistency, structure, and stability on its own terms, before anyone asks what it predicts. The prediction model is then validated separately, drawing on the measurement without pretending it is flawless: predictions are built to carry the measurement's own score uncertainty forward rather than treating a trait score as a fixed, error-free number. This separation makes it harder for weakness in one to be obscured by strength in the other.

Internal consistency, structure, and stability are a floor, not a finish line. A measurement model is also checked for how precisely it measures across the whole trait range (not just near the average), whether individual items behave the same way for different groups, where the construct's boundaries actually sit against neighboring traits, and whether the same structure replicates in a fresh sample.

A trait is validated as a measurement before it is used to predict anything.
The prediction model is validated separately, carrying the measurement's own uncertainty forward rather than treating it as error-free.
When a score misses, this separation makes it possible to say which part failed: the measurement, the prediction, or neither.

What we check, at the measurement stage:

Conditional precision across the trait range, not just average reliability.
Item functioning: whether individual items behave consistently across groups.
Measurement invariance and differential item functioning (DIF) across populations and occasions.
Construct boundaries: where this trait ends and a neighboring one begins.
Replication: whether the same structure holds up in a fresh sample.

2. Population-Referenced Scores

Scores are anchored to a fixed reference, not a shifting sample.

Most convenience norms drift: a percentile is measured against whoever happened to test during a given stretch of time, so the same raw score can translate to a different percentile next quarter even though nothing about the person changed. Population-referencing fixes the yardstick instead of letting the crowd redraw it. Every score is anchored to a stable, known reference population; preserving comparability over time then requires linking, equating, and explicit monitoring for drift and differential item functioning whenever items, forms, or populations change.

That stability is what makes candidates comparable over time and across roles: a benchmark set this year still means the same thing when the next cohort is measured against it, and a candidate who is re-tested later can be compared honestly to their earlier result instead of to a moving target.

A fixed reference population is necessary, but it is not sufficient by itself. Holding a score's meaning stable over time also requires linking and equating work whenever items or forms change, ongoing measurement-invariance monitoring across groups and occasions, and explicit checks for scale drift and differential item functioning. All of that is part of the calibration program. It is not assumed for free just because a reference population was set once.

Every score is anchored to a stable, known reference population, not a rolling convenience sample.
That stability is actively maintained, not assumed: linking and equating, invariance monitoring, and drift/DIF checks run alongside the fixed reference population.
Candidates, cohorts, and re-tests can be compared against a fixed yardstick instead of a moving one.

3. Versioned, Explicitly Activated Models

Scored once, defended everywhere. No silent updates.

Every scoring model carries an explicit version. Updates go through validation and then have to be explicitly activated before they touch a live session. A model is never quietly swapped in behind a score that has already been delivered. That means a report generated today and a report generated a year ago can each be traced back to the exact model version that produced them.

The practical benefit is full provenance: for any reported score, there is a documented path back to the response data, which model version scored it, and when that version was activated. If someone asks how a number was produced, there is a documented answer, not a reconstruction from memory.

Every model is versioned; nothing is updated silently behind a live score.
Updates are validated and then explicitly activated. Activation is a deliberate step, not an automatic push.
Any reported score can be traced back to its inputs, its model version, and when that version was activated.

4. Adversarial Verification

Consequential artifacts are checked by someone trying to break them.

Before a scoring model, a decision threshold, or any other artifact that a real decision will rest on is put into use, it goes through three steps. First, a sealed specification: the parameters and decision rules are frozen and recorded before anyone builds against them, so nothing can be quietly tuned after the fact to match a preferred result. Second, independent recomputation: decisive arithmetic is independently reproduced from those same frozen inputs and specifications, and any disagreement gets flagged rather than averaged away. The second pass does not always need to be blind to count, but it does need to be genuinely independent. Third, the artifact is independently checked and cleared for its stated use by someone whose job is specifically to try to break it, not to wave it through.

This is heavier scrutiny than routine review, and it is reserved for what decisions will actually rest on, such as a frozen scoring model, a released threshold, or a client-facing result. It is not applied as ceremony to every working document along the way. Signatures are reserved for genuine consensus gates, such as freezing a plan, a contract, or a release, rather than for routine work in between.

Sealed specification: parameters and decision rules are frozen and recorded before anyone builds against them.
Independent recomputation: decisive arithmetic is reproduced from frozen inputs, and disagreements are flagged, not smoothed over.
The artifact is independently checked and cleared for its stated use by someone whose role is specifically to try to break it, before it is used for a real decision.

5. The Broader Architecture

Eight techniques, each answering a different failure mode.

A broader modeling toolkit, each technique answering a specific failure mode.