SI models are up to 48% smarter at 3am. Six claims, opened.
Every figure on this page is a claim about a base. Here the claims sit in one column, each carrying its number, each opening onto its evidence, its arithmetic and its source. Collapsed is a reading order, not a withholding: nothing is off the page, and the receipt below never closes.
The claim index
Six lines. Each one is a disclosure.
-
Evidence
Artificial Analysis ran the same three evaluations against 26 endpoints of GLM-5.3. The best preserved 99.3% of the reference accuracy; the worst preserved 67.2%. Ten of the 26 were statistically below the reference — their 95% interval sits wholly below 100. Each endpoint was measured once, at its most favourable settings. The name on the invoice was the same for all 26.
The six models in the Endpoint Accuracy Index: endpoint count, index range, endpoints below reference. model endpoints index range below ref. DeepSeek V4.1 Flash 1017 22 97.5–101.4 0 GLM-5.3 26 67.2–99.3 10 GLM-5.3 Flash 22 78.6–104.2 5 gpt-oss-120b 16 78.2–104.0 5 Kimi K3 12 91.0–101.5 2 Muse Spark Open 5 93.4–99.9 2 Six models, 103 endpoints, 24 statistically below their reference. The headline uses the widest: GLM-5.3. The other five run from a 4.0% relative spread to 33.0%, a median of 11.5%; across all six the median is 22%. The plural in “SI models” is the measured set; the 48% is its worst case.
Arithmetic
(99.3 − 67.2) ÷ 67.2 = 47.8%
99.3 − 67.2 = 32.1 points, absolute
gap interval ±6.1 pts = √(4.3² + 4.3²) · 39–57% relativeSource
Artificial Analysis, “Endpoint Accuracy Index v1.1.” Measured 8 Oct–4 Nov 2026 · published 9 Nov 2026 · fetched 17 Nov 2026 · artificialanalysis.ai
Receipt row · 48% — and the row that follows it, ±6.1 pts. A cross-endpoint spread, not a clock measurement.
-
Evidence
Anthropic’s postmortem records a worst hour: “At the worst impacted hour on August 31, 16% of Sonnet 4 requests were affected” — up from 0.8%, after “a routine load balancing change.” Roughly 30% of Claude Code users hit a misrouted server at least once.
OpenRouter’s routing docs describe the mechanism: routing “learns how much load each provider can take before it returns 429s, and sends the rest to other providers.” At 3am, the best endpoint has room. At 3pm, the rest is what’s left.
Anthropic’s own model-ID documentation says what can move: “Model weights are fixed for a given ID, but the serving infrastructure around the model can change over time. This infrastructure includes components such as the request router, safety classifiers, and sampling logic.”
“We never reduce model quality due to demand, time of day, or server load.”
Anthropic · A postmortem of three recent issues · 17 September 2025The sentence is true about intent. It is not true about routing: the same document reports a 20× swing in exposure at the worst hour.
Arithmetic
0.8% → 16% at the worst hour · 16 ÷ 0.8 = 20×
Source
Anthropic, “A postmortem of three recent issues.” 17 Sep 2025 · anthropic.com
OpenRouter, “Provider Routing.” Fetched 17 Nov 2026 · openrouter.ai/docs
Anthropic, “Model IDs and versions.” Fetched 17 Nov 2026 · platform.claude.comReceipt row · 16%. The clock is the mechanism; the spread in claim 01 is the size.
-
Evidence
If 16% of requests draw the worst endpoint and 84% draw the best, the average is 94.2 — a 5.2% loss against the best. The tail is 48%.
The provider did not choose to degrade the model. The routing did.
Arithmetic
(84·99.3 + 16·67.2) ÷ 100 = 94.16
(99.3 − 94.16) ÷ 99.3 = 5.2%Source
Derived from the 16% exposure (Anthropic, 17 Sep 2025) and the 99.3 / 67.2 endpoints (Artificial Analysis, 8 Oct–4 Nov 2026). Derived 17 Nov 2026.
Receipt row · 5.2%. An expected loss, not a measured one.
-
Evidence
Margin Lab’s Codex tracker records a decline from 58.1% to 54.0% over 24 published days and 1,175 graded task runs, 25 Jan–20 Feb 2026. The first flyer printed this average, rounded to 8%.
The average is not the spread. A 7.6% mean across a window can sit on top of a 48% endpoint gap.
Arithmetic
(58.1 − 54.0) ÷ 54.0 = 7.6% · rounded to 8%
Source
Margin Lab, “Codex tracker.” 25 Jan–20 Feb 2026 · fetched 17 Nov 2026 · marginlab.ai/trackers/codex-historical-performance
Receipt row · 7.6%.
-
Evidence
Margin Lab runs 50 fixed tasks once a day at 06:00 UTC, with Wilson 95% intervals. Its largest same-model daily move is 64% → 50% on 22–23 May 2026 — 14 points, approximately 1.08× its own ±13-point daily bar, and flagged by its same-task check at p < 0.001.
It is the largest of 213 daily comparisons. A daily move is not an endpoint spread: it is the same model, the same tasks, one day apart.
Arithmetic
(64 − 50) ÷ 50 = 28.0%
14 points vs a ±13-point daily bar · 14 ÷ 13 ≈ 1.08×Source
Margin Lab, “Claude Code tracker” and “Methodology.” 50 fixed SWE-Bench-Pro tasks, one run per day at 06:00 UTC, Wilson 95% intervals; 22–23 May 2026 · fetched 17 Nov 2026 · marginlab.ai/trackers/claude-code
Receipt row · 28%.
-
Evidence
HLE-250 scored 7.6 against 63.6 under a 16,384-token output cap — 56 points, 88% of the uncapped base, measured 8 Oct–4 Nov 2026. The same index carries the number; it is the far edge of the same question.
Arithmetic
(63.6 − 7.6) ÷ 63.6 = 88.1%
63.6 − 7.6 = 56 points, absoluteSource
Artificial Analysis, “Endpoint Accuracy Index v1.1.” HLE-250, 16,384-token cap, measured 8 Oct–4 Nov 2026 · artificialanalysis.ai
Receipt row · 88%.
The receipt · always open
Every figure, its base, its sample, its date.
A percentage is a claim about a base. Every row names the base it divides by, the sample it divides, and the date the source published it. The six lines above point here; this table points at nothing else.
| figure | arithmetic | base | sample | date · source |
|---|---|---|---|---|
| 48% | (99.3 − 67.2) ÷ 67.2 = 47.8% | worst endpoint | 26 endpoints, one model | 8 Oct–4 Nov 2026 · Artificial Analysis |
| ±6.1 pts | √(4.3² + 4.3²) = 6.1 | gap interval (AA avg ±4.3) | 26 endpoints | same · 39–57% relative |
| 32.1 pts | 99.3 − 67.2 | 0–100 index | same 26 | same |
| 10 of 26 | count, 95% CI wholly < 100 | reference = 100 | 26 endpoints | same |
| 28% | (64 − 50) ÷ 50 = 28.0% | peak day (50) | 50 fixed tasks, n=50 | 22–23 May 2026 · Margin Lab |
| 16% | 0.8% → 16% at worst hour | Sonnet 4 requests | 31 Aug 2025 | 17 Sep 2025 · Anthropic |
| 5.2% | (84·99.3 + 16·67.2)/100 = 94.16; (99.3 − 94.16) ÷ 99.3 = 5.2% | best endpoint | 16% draw the worst | derived · 17 Nov 2026 |
| 7.6% | (58.1 − 54.0) ÷ 54.0 = 7.6% | peak base (54.0) | 24 days, 1,175 graded | 25 Jan–20 Feb 2026 · Margin Lab Codex |
| 88% | (63.6 − 7.6) ÷ 63.6 = 88.1% | uncapped endpoint | HLE-250, 16,384-token cap | 8 Oct–4 Nov 2026 · Artificial Analysis |
| 4.0–47.8% | per-model relative range, median 22% | each model’s worst | six models, 103 endpoints | 8 Oct–4 Nov 2026 · Artificial Analysis |
Figures current as of 17 November 2026. Rows are ordered as the claims are; nothing in this table is derived except the two marked derived.
Out of the index
Two sources the six claims do not spend.
The file carries two more measurements. They are evidence of the same shape — a tail where a fixed thing stops being fixed — and they are not part of the six figures above, so they are named here rather than folded in.
“The State of FP8 KV-Cache and Attention Quantization.” 128k needle-in-a-haystack: BF16 91%, FP8 before the accumulation fix 13%, after 89%. The tail is real; the fix is real.
Chen, Zaharia & Zou, “How Is ChatGPT’s Behavior Changing over Time?” GPT-4 84.0% → 51.1% on a balanced 1,000-item prime/composite set, March–June 2023 (revised version). Temporal drift, not a clock effect.
The limit
The one measurement nobody has run.
Not a controlled clock experiment. Nobody has published one model, one task set, one endpoint, at 03:00 and 15:00, with the serving endpoint logged. The 48% is the endpoint spread; the 16% is the load-hour exposure; the 5.2% is the expected loss if 16% of requests draw the worst endpoint. The clock experiment is the one measurement the industry still owes. When a provider runs it, this page gets a better number.
Not every model. Six models are in the index; the headline uses the widest, GLM-5.3. Not intelligence. Task success on three evaluations. Not a promise. A measured spread and a routing mechanism.
An index makes the shape of the argument visible: six claims, six figures, one base each. It does not make the argument bigger than its evidence, and the limit above is part of the index too.
Sources
Dated, and checkable.
Artificial Analysis, “Endpoint Accuracy Index v1.1.” 26 GLM-5.3 endpoints, 67.2–99.3; 10 below reference; HLE-250 7.6 vs 63.6 under a 16,384-token cap; six models, 103 endpoints, 24 below. Measured 8 Oct–4 Nov 2026. artificialanalysis.aipublished 9 Nov 2026 · fetched 17 Nov 2026
OpenRouter, “Provider Routing.” Routing “learns how much load each provider can take before it returns 429s, and sends the rest to other providers.” openrouter.ai/docsfetched 17 Nov 2026
Anthropic, “A postmortem of three recent issues.” “We never reduce model quality due to demand, time of day, or server load”; “At the worst impacted hour on August 31, 16% of Sonnet 4 requests were affected,” up from 0.8%, after “a routine load balancing change”; ~30% of Claude Code users hit a misrouted server. anthropic.com17 Sep 2025
Anthropic, “Model IDs and versions.” “Model weights are fixed for a given ID, but the serving infrastructure around the model can change over time. This infrastructure includes components such as the request router, safety classifiers, and sampling logic.” platform.claude.comfetched 17 Nov 2026
Margin Lab, “Claude Code tracker” and “Methodology.” 50 fixed SWE-Bench-Pro tasks, one run per day at 06:00 UTC, Wilson 95% intervals; the largest same-model daily move is 64% → 50% on 22–23 May 2026 — 14 points, just over its ±13-point daily bar, and flagged by its same-task check at p < 0.001. The corrected 58.1 → 54.0 window is the Codex tracker, 25 Jan–20 Feb 2026. marginlab.aifetched 17 Nov 2026
vLLM, “The State of FP8 KV-Cache and Attention Quantization.” 128k needle-in-a-haystack: BF16 91%, FP8 before the accumulation fix 13%, after 89%. The tail is real; the fix is real. vllm.ai22 Apr 2026
Chen, Zaharia & Zou, “How Is ChatGPT’s Behavior Changing over Time?” GPT-4 84.0% → 51.1% on a balanced 1,000-item prime/composite set, March–June 2023 (revised version). Temporal drift, not a clock effect. arXiv:2307.090092023
Figures current as of 17 November 2026.