Each endpoint, against its own past.
A check compares two endpoints at one moment. The drift log compares one endpoint at two moments.
§1The record
The first batch of answers we ever collected from an endpoint is kept as its baseline. Every batch collected later is held against it with the same test a check uses.
§2The three standings
- Steady
- The latest batch still matches the first.
- Watch
- The latest batch did not match. One odd batch in a hundred is expected by chance, so one alone is not called a change.
- Changed
- The last two batches both failed to match. The endpoint no longer answers like it used to.
§3What a change can be
- The maker released a new version under the same name.
- A gateway moved the model to a different host, or to a compressed copy.
- A hidden prompt was added or changed.
The log shows that an endpoint moved. It does not show which of these happened.
§4Where it stops
- It only sees the models we collect from, as often as we collect.
- A slow change spread over many small steps can stay under the threshold at every step. That is why the baseline is the first record, not the previous one.
→Keep reading
- Next
- When a run fails · the errors you may see
- Before this
- What spec check tests
- All pages
- The docs