Where it stops.
A detector that hides its blind spots is worse than none. These are ours.
§1What a verdict is not
- It gives evidence, not proof. Inconsistent means two endpoints do not behave alike. It does not say why, and it says nothing about intent.
- It is one moment. A verdict covers the requests that were sent. It says nothing about yesterday or tomorrow.
§2What looks the same
- Other causes leave the same mark. A hidden system prompt, a different version, changed sampling settings: each shifts the answers. The token counts help you tell a hidden prompt from the rest.
- The reference decides everything. If the side you trust is not the real model, the verdict is about the wrong thing.
§3What gets past it
- Compressed copies. In our own trial, a 4-bit copy of a model usually passed as consistent against the full version at the standard sample size. See the case.
- A prepared gateway. One that recognises these eight questions could send only those to the real model and everything else to a cheaper one.
→Keep reading
- Next
- Using OpenRouter · presets, pinning a host, rate limits
- Before this
- Reading a result
- All pages
- The docs