Model updates don't just improve — they redistribute capability. Some tasks get better; adjacent ones silently regress. Model-Drift Watch runs a fixed battery of production-shaped regression tasks against every major release, across providers, and publishes the diff — so you find out the same day, not from your customers.
| Workflow archetype | claude-sonnet-5 | gemini-3.5-flash | gpt-5.6-sol |
|---|---|---|---|
| Structured output (JSON) | 0/25 | 25/25 | 25/25 |
| Long-doc extraction | 18/25 | 25/25 | 24/25 |
| Instruction adherence | 25/25 | 20/25 | 24/25 |
| Multilingual | 14/25 | 25/25 | 25/25 |
| Tool/function calling | 25/25 | 25/25 | 25/25 |
| Agentic multi-step | 25/25 | 25/25 | 25/25 |
| Total | 107/150 · 71% | 145/150 · 97% | 148/150 · 99% |
Headline: every claude-sonnet-5 failure here is one behavior — it wraps JSON in a markdown code fence despite an explicit "no markdown" instruction (43/43 fails; the JSON inside is otherwise correct). gemini-3.5-flash and gpt-5.6-sol honor the instruction 100%. A deterministic checker catches it every time.
Methodology: N=5 trials per task, 30 tasks, pinned seed, temperature 0 (or provider default), deterministic checkers; 0 infrastructure errors across all 450 trials. This is the inaugural baseline each future model release is diffed against. Task categories are published; raw task inputs never are.
Free forever for the public scorecard. No spam; one email per model release.