StandardsAboutContact

The Weights

Breaking News

Metis Reasoning: AlphaGo Idea, No Benchmarks YetTeardown

Metis Reasoning: AlphaGo Idea, No Benchmarks Yet

Wait. Metis Reasoning, a London startup founded by ex-DeepMind researcher Thore Graepel, proposes a knowledge ledger that separates facts from reasoning so conclusions can be checked against evidence. It has published no benchmarks and no working system. Until the company releases numbers, treat it as an architecture claim, not a hallucination fix.

Read More
The Signal

Sponsored 'Industrial Intelligence' Piece Lacks

Treat this sponsored article as vendor positioning, not evidence. Its two case-study results are presented without baselines, sample sizes or methods, so they cannot be checked. MIT Technology Review labels the piece as produced by Microsoft; its governance checklist is useful, but its performance figures are unverified claims.

Read More
The Signal

GPT-6.1 Terra: Half the Price, Real Gains, Open Questions

GPT-6.1 Terra is a real price cut with measurable gains, but independence is limited. Artificial Analysis reports prices of $1 input and $5 output per million tokens versus $2 and $12 for GPT-5.6 Terra, and a 6-point Intelligence Index gain at max effort. Signal for agent workloads, pending your own tests.

Read More
The Fine Print

TasteVal's 2027 self-improvement signal is thinner

Treat 2027 as a hypothesis, not a finding. TasteVal reports Opus 5.5 at a 2.3x compute multiplier over human experts, but the 95% interval runs 1.15 to 4.37, the tasks are unreleased, and the trend covers under a year. Jack Clark's shift, per The Decoder, rests on that thin evidence.

Read More

Featured Stories

Scalable AI Foundations: A Session With Little Evidence

Scalable AI Foundations: A Session With Little Evidence

The Signal

Noise, for now: MIT Technology Review's EmTech Future session on scalable AI names the right ingredients (data, engineering, platforms, operating models, governance) but, in the published summary, offers no benchmark, incident or named deployment to test. Treat it as a checklist, not evidence, until the full session shows measurable results.

Context-aware AI: a Celonis-sponsored pitch, not evidence

Context-aware AI: a Celonis-sponsored pitch, not evidence

The Signal

This 'case for context-aware AI' is a sponsored Celonis interview, not independent research. MIT Technology Review Insights issued it with no statistics or study, so the claim that agents need operational context is plausible but unmeasured. Treat it as a vendor pitch for the Celonis context model and demand a baseline evaluation.

VocabX's 77% Cost Cut Rests on One Grader

VocabX's 77% Cost Cut Rests on One Grader

Teardown

Wait. VocabX, Dharma-AI's vocabulary rebuild, cut measured inference cost 77% on one Qwen3-4B essay grader, and its bench score rose from 0.703 to 0.810. But the paper is unpublished, the bench is private, and no code or model is linked, so the result is a claim a team cannot yet reproduce.

Google Cloud's OpenAI Preview: Real

Google Cloud's OpenAI Preview: Real

The Fine Print

Google Cloud's OpenAI offering is confirmed but narrow. Google says GPT-6 Astra, GPT-6.1 Sol and GPT-6 Luna are in private preview for select enterprise customers in Model Garden, billed through Marketplace. No prices, regions or benchmarks are published, and zero data retention is still "coming soon." Treat it as binding soon, not binding now.

An AI kill switch is a testable control, not a moonshot

An AI kill switch is a testable control, not a moonshot

The Signal

An AI kill switch is a management control, not a moonshot. The NIST AI RMF Playbook already expects organizations to assign responsibility and mechanisms to deactivate AI systems that behave inconsistently with intended use. MIT Technology Review's letter groups an AI kill switch with fusion as ambition, but neither source measures how reliably shutdown works.

Stroke-Rehab Robot: Feasibility Shown, Recovery Unproven

Stroke-Rehab Robot: Feasibility Shown, Recovery Unproven

Teardown

The stroke-rehabilitation robot built by Johannes Lachner and Noah Geiger has demonstrated feasibility, not recovery. Its diffusion-based impedance learning approach is documented in a preprint, but the reported tests used healthy participants, and no stroke-patient outcomes are reported. Hospitals should treat it as research to watch until the Munich clinical study reports.

DeepInfer teardown: Wait on DeepSeek's inference engine

DeepInfer teardown: Wait on DeepSeek's inference engine

Teardown

Wait. DeepInfer, the Apache-2.0 inference framework DeepSeek open-sourced for DeepSeek-V4.1-Flash, is worth reading but not yet worth deploying: the release covers offline inference only, targets NVIDIA SM100 and Huawei Ascend 950 hardware, and omits the KV cache, disaggregation and load-balancing pieces a complete service needs.

Eleven Realtime v1 Max: Fast and Accurate, Still Wait

Eleven Realtime v1 Max: Fast and Accurate, Still Wait

Teardown

Wait. Eleven Realtime v1 Max is listed by Artificial Analysis at 99 percent on Big Bench Audio with 0.57 seconds to first audio, matching ElevenLabs' reported 98.8 percent. That is a strong third-party signal, but it covers one benchmark. Test ElevenLabs' models on your own calls before switching.

ALTK-Evolve Teardown: Wait, the Gain Costs 65% More

ALTK-Evolve Teardown: Wait, the Gain Costs 65% More

Teardown

Bottom line: Wait. ALTK-Evolve lifted gpt-4.1 retail Pass^4 on τ²-bench from 57.5% to 67.5% on 40 held-out tasks, but cost per conversation rose 65% and the samples are tiny. IBM Research reports the results itself, and independent replication is missing.

Clairic: Wait on Cloudflare's Extraction Model

Clairic: Wait on Cloudflare's Extraction Model

Teardown

Wait. Clairic, Cloudflare's preview extraction model, looks promising for field extraction, but the only evidence is Cloudflare's own test: a three-step pipeline finished in 19.3 seconds versus 61.7 for GLM-5.3 alone, at roughly half the cost. No accuracy benchmark or open weights exist yet, so pilot it, don't migrate.

AnyLanguageModel 1.0: A Portability Layer, Not a Speedup

AnyLanguageModel 1.0: A Portability Layer, Not a Speedup

The Signal

AnyLanguageModel 1.0 is a credible portability layer, not a performance upgrade: it lets Swift apps swap Apple's Foundation Models for MLX, llama.cpp, Core ML or hosted APIs by changing one import. Its own repo admits local backends cannot force tool calls, so agentic workloads need testing before migration.

Taste-Bench: Best Agent Scores 59.7% on Strategic Forks

Taste-Bench: Best Agent Scores 59.7% on Strategic Forks

Teardown

Wait before trusting agents with strategic direction. On Taste-Bench, GPT-5.6 Sol scored 59.7% on a test that counts a fork only when the model picks the winning path in both option orders, against a 25% chance level. Accuracy fell to 21% when the deciding clue arrived late, and extra reasoning did not help.

German Courts Treat Google's AI Answers as Its Own

German Courts Treat Google's AI Answers as Its Own

The Fine Print

Google is now answerable in German courts for what its AI search says. The Regional Court Aachen, in a 24 September 2026 injunction, applied the reasoning of the Regional Court Munich I: AI Overviews output is Google's own statement. Both rulings are first-instance, so the law is not settled.

llm-tools-datasette 0.1.1 patches a break from LLM 0.33

llm-tools-datasette 0.1.1 patches a break from LLM 0.33

The Signal

Noise for most readers, a small Signal for plugin maintainers: llm-tools-datasette 0.1.1 fixes a breakage after LLM 0.33 switched to httpx2. Simon Willison says plugins using httpx without depending on it directly broke, and this was one of them. The sources name no other affected plugin.

Coding agents take the puzzle, not the paycheck

Coding agents take the puzzle, not the paycheck

Ships or Slips

Coding agents take away the puzzle that made programming satisfying, but Jeremy Scheff's account does not show lost work: his business is having a strong year. Simon Willison reads the same shift as a gift for builders. It is a first-person essay, not data, so treat it as a morale signal.

View More Posts

Sign up for the Newsletter

The week in the field, weighed — what ships and where it slips.

Today's briefLoad more