StandardsAboutContact

The Weights

Breaking News

GLM-5.3 Cyber Risk: Wait, Don't Ban Open ModelsTeardown

GLM-5.3 Cyber Risk: Wait, Don't Ban Open Models

Wait on a blanket verdict: Anthropic's GLM-5.3 report shows the open-weight model nearly matches Claude Mythos Preview on ExploitBench, 50 of 410 exploits against 56, and its safeguards fall away once weights are public. But the tests were simulated, so the evidence supports controlled-release rules rather than a ban on open models.

Read More
Red Team

Small AI Models Near Self-Replication on Cheap Devices

The Anthropic Frontier Red Team study is a credible signal, not a live threat. In a simplified evaluation, GLM-5.2 was the only open-weight model to complete end-to-end replication, spreading to all eight test nodes in four of five trials. The authors call their results an upper bound because the cyber steps were trivial.

Read More
Teardown

Python 3.15 lazy imports: use it, with one caveat

Use it: the lazy imports in Python 3.15 (PEP 810) are worth adopting now for command-line tools and slow-starting apps, but not for libraries that must run on older Pythons without care. The PEP's own claim of 50-70% faster CLI startup is a design-document figure, not a benchmark we reproduced.

Read More
The Signal

Nemotron Hits Gold Level at IOI and IMO 2026

Nvidia's Nemotron family produced credible gold-level olympiad results, but the evidence is self-reported. Nemotron-3-Ultra-CC scored 535.4 of 600 in an unofficial IOI 2026 run, and a Nemotron 3 Ultra pipeline scored 30 of 42 at the IMO against a 29-point gold threshold, graded by official graders.

Read More

Featured Stories

Aleph Alpha's Kolibri: A Documented Apache 2.0 Release

Aleph Alpha's Kolibri: A Documented Apache 2.0 Release

Teardown

Kolibri is a credible open-weight German-English model, but its quality claims remain unverified. Aleph Alpha's 78-billion-parameter mixture-of-experts model activates about 3.46 billion parameters per token and ships under Apache 2.0. Its benchmark scores are the company's own, and we found no independent reproduction.

AI's Grid Load: IEA Puts Data Centres Under 10%

AI's Grid Load: IEA Puts Data Centres Under 10%

Red Team

The IEA base case puts data-centre demand at about 945 TWh by 2030, under 10% of demand growth, while MIT Technology Review calls rising demand, driven in part by AI, a top story. AI data centres are a real but minority driver of global electricity growth. Deployers should plan around local grid constraints, not global headlines.

MAI-Transcribe-2-Streaming Teardown: Wait

MAI-Transcribe-2-Streaming Teardown: Wait

Teardown

Wait. Microsoft's MAI-Transcribe-2-Streaming claims first partial transcripts in just over 100ms at $0.54 per audio hour, 1.5x the $0.36 per hour Artificial Analysis lists for the earlier batch MAI-Transcribe-1. The latency and its number-one ranking are vendor-reported, with no disclosed audio conditions, so test on your own calls first.

Claude Code Mods Run Unsandboxed With Your Permissions

Claude Code Mods Run Unsandboxed With Your Permissions

Teardown

Bottom line: Wait before installing third-party Mods; use first-party or self-written ones. Anthropic's documentation says a mod runs unsandboxed with your user permissions. It can read secrets, rewrite prompts and tool calls, and approve calls before you are asked. The deciding number is zero: no sandbox covers a process a mod starts.

Suno's Speech Beta Merges Narration and Music in One

Suno's Speech Beta Merges Narration and Music in One

The Fine Print

Suno's Speech is a real capability but an unproven product: it generates narration and matching music as one track, yet Suno's announcement and The Decoder's report show no training-data disclosure, licensing terms or benchmarks. Treat Speech as a prototyping tool for personal audio, not a safe source for commercial work until rights questions are answered.

AstaBrief-8B Teardown: Ai2's Open 8B Report Writer Posts

AstaBrief-8B Teardown: Ai2's Open 8B Report Writer Posts

Teardown

Use it, for cited research briefs where seconds matter. AstaBrief-8B, an Apache 2.0 Qwen3-8B fine-tune from Ai2, scores 87.0 on the ScholarQA-CS2 test set with 90.5% citation precision, and Fast mode averages 51.1 seconds per report versus 178.5 for Thinking mode. Citation recall of 78.2% means some claims arrive unsupported.

ServiceNow's AutoSynthData: Wait for Replication

ServiceNow's AutoSynthData: Wait for Replication

Teardown

Wait for independent replication before committing budget. ServiceNow CoreAI reports its AutoSynthData loop raised Gemma-4-26B-A4B-it by 7.2 Pass@1 points, a 35% relative gain, on hybrid enterprise tasks. The evidence is a vendor blog scored on a benchmark from the same organization, with no reported seeds or variance.

Ideogram 4.5 claims drift-free edits

Ideogram 4.5 claims drift-free edits

Ships or Slips

Ideogram 4.5 is unproven as a precise-edit model: the company's claims of cleaner multi-turn edits, native 2K output and pricing from 0.8 cents per image rest on its own page and demos, with no published benchmark. Treat it as a candidate to test on your own edit-drift suite, not a verified upgrade.

Photo Scrubber: A Local Face-Blur Tool Built by an AI

Photo Scrubber: A Local Face-Blur Tool Built by an AI

The Signal

Photo Scrubber is a Signal, not noise: a small, experimental browser tool that blurs faces and strips metadata locally. It runs Google's BlazeFace through MediaPipe's WebAssembly build. Its documented limits matter: Google tunes BlazeFace for phone-camera images, and the post publishes no accuracy figures.

Gemini 4 Argon ties GPT-6 Astra on Artificial Analysis

Gemini 4 Argon ties GPT-6 Astra on Artificial Analysis

The Signal

Gemini 4 Argon is a credible but not leading frontier model: it scores 53 on the Artificial Analysis Intelligence Index, level with GPT-6 Astra and behind Claude Opus 5.5 at 58. It uses more than twice Astra's output tokens per task, yet at promotional prices costs less per task. Access remains restricted.

AI Mushroom ID: Skip It — Best Model Still Calls Poisonous

AI Mushroom ID: Skip It — Best Model Still Calls Poisonous

Teardown

Bottom line: skip it for foraging safety. A Quesma benchmark tested 16 vision-language models on 1,040 photos of 55 edible and deadly mushroom species from the FungiTastic dataset. The top model, Google's Gemini 3.8 Flash, reached 65% first-guess accuracy and still called a poisonous mushroom edible in 11% of cases.

Nemotron 3 Diarization: Nvidia's Free 100M-Parameter

Nemotron 3 Diarization: Nvidia's Free 100M-Parameter

Teardown

Verdict: use it for meeting and call-center transcription, not as an identity control. Nvidia's free, 100M-parameter Nemotron 3 Diarization posts a 14.72% diarization error rate on VoiceArena's independent benchmark, beating the next system's 19.3%, but its speaker labels are anonymous, session-scoped guesses — Nvidia's Hugging Face blog post warns against treating any assignment as infallible.

Nvidia's SoL-Pi Is Real, Open-Source Code

Nvidia's SoL-Pi Is Real, Open-Source Code

Teardown

Bottom line: Wait. Nvidia's SoL-Pi (arXiv:2609.20519) is real, MIT-licensed, npm-installable code for the Pi coding-agent harness — not vaporware. It cuts EdgeBench tokens up to 49% at 93.7-94.3% score retention, but on Terminal-Bench 4 it solves only 15 of 63 tasks versus 18 for both Pi and Codex, a real capability regression its own headline numbers don't disclose.

GPT-6 Astra Hits 80% on an IKEA-Error Benchmark

GPT-6 Astra Hits 80% on an IKEA-Error Benchmark

Teardown

OpenAI's GPT-6 Astra scored 80 percent on Epoch AI's 60-photo Furniture Assembly Benchmark, nearly tripling Claude Opus 4.5's 28 percent from November 2025 and beating Claude Fable 5.1 (70 percent) and Claude Opus 5 (61 percent). The gain is real, but the test is tiny and Astra takes three minutes per photo — too slow for live assembly help.

Multiverse Computing's Ising-Optimization Pruning

Multiverse Computing's Ising-Optimization Pruning

Teardown

Bottom line: Wait. Multiverse Computing's block-removal method, detailed on Hugging Face and in arXiv preprint 2602.00161, lifts Llama-3.3-70B-Instruct from a 54.0-MMLU baseline to 76.9 at 50% depth compression — but the edge shrinks to parity at 8B scale, the low-energy-equals-good-model premise breaks after retraining, and zero independent reproductions of any number exist.

View More Posts

Sign up for the Newsletter

The week in the field, weighed — what ships and where it slips.

Today's briefLoad more