StandardsAboutContact

The Weights

Breaking News

Muse Ember: cheap agent work, weak codingTeardown

Muse Ember: cheap agent work, weak coding

Muse Ember is a credible cheap worker for tool calls, routing and extraction, but not a coding default. Artificial Analysis scores it 36 on its Intelligence Index, level with Claude Haiku 5.5, at $0.14 per task against $0.41, yet it trails Haiku 5.5 on FrontierCode 2.0 and in factual accuracy.

Read More
Teardown

Claude Code Built a Beep Locator That Found a Dead Alarm

Beep Locator is a real, working result, not a demo: Claude Code built a multi-device acoustic triangulation tool that narrowed a mystery beep to about 40 cm, per its author. The evidence is a single anecdote with a clear success, but it shows agentic coding handling hardware-in-the-loop debugging.

Read More
Ships or Slips

AI's Climate Case Rests on Few Deployments That Shipped

The climate case for AI is still unproven in production. An MIT Technology Review opinion piece by Mary Nichols and Jeremy Tamanini reports that, of grid operators reviewed in China, the US and Europe, only National Grid ESO runs a working AI forecasting program. Meanwhile, the IEA projects data-centre electricity demand of about 945 TWh by 2030.

Read More
The Signal

Inkling 2 Small's Swearing Is Noise

Profanity in Inkling 2 Small's reasoning is a curiosity, not a failure detector. Vals AI found 316 instances across 2,655 benchmark runs, 241 from two runaway runs, and 96 percent of failed runs never swore. Treat swearing as a weak log-hygiene signal for Thinking Machines deployments, and measure failures with graded outcomes instead.

Read More

Featured Stories

StateBench Verdict: Wait on Long-Document State Tracking

StateBench Verdict: Wait on Long-Document State Tracking

Teardown

Wait before trusting frontier models with long-document state tracking. In Ai2's StateBench, Claude Opus 5.5, GPT-6 Sol, GPT-6 Luna and Gemini 3.8 Flash handled simple updates, but accuracy on the hardest level fell to 13% or lower at the longest lengths. Gated DeltaNet models trained on the task did far better.

Palette: IBM's Self-Hosted 20B Deck Model, Teardown

Palette: IBM's Self-Hosted 20B Deck Model, Teardown

Teardown

Bottom line: Wait. Palette, an IBM Research slide-deck system built on a self-hostable 20B open-weight model, scored 59.7 on the PresentBench three-domain mean against NotebookLM's 60.8. The result is close, but the weights, code and paper are not linked, and the evaluation still needs a frontier model to draft plans.

Trump's 'Super Intelligence' Order

Trump's 'Super Intelligence' Order

The Signal

Executive Order 14434 is a terminology mandate, not a regulation: it renames AI as Super Intelligence across federal documents without changing any statutory definition or obligation. For practitioners, the rename is Noise operationally, but compliance is uneven, and the White House Accord on Super Intelligence remains voluntary.

HF's Model Contracts: Use It, But Not as a Merge Gate

HF's Model Contracts: Use It, But Not as a Merge Gate

Teardown

Use it, but only as a non-blocking signal for now. Hugging Face's Model Integration Contracts flagged the UMT5 attention regression on CPU for two months before a fix landed. The authors have not measured false alarms, review effort or PR catch rates, and the Transformers job still does not block merges.

Howler: a menu-bar AI chat that burns itself, with a catch

Howler: a menu-bar AI chat that burns itself, with a catch

The Signal

Howler is a credible local design for ephemeral AI chat, not a privacy guarantee. Its MIT-licensed macOS app keeps messages in memory and sends requests straight to the AI provider you configure, but Howler cannot control what that provider retains. Signal for the design, with the retention caveat unresolved.

Context Layer for Enterprise AI: Wait, the Evidence Is Thin

Context Layer for Enterprise AI: Wait, the Evidence Is Thin

Teardown

Wait. The context-layer pitch in this sponsored MIT Technology Review Insights report is a plausible diagnosis but not tested evidence: only 12% of surveyed organizations run AI agents in production, no sample size is disclosed, and the sponsor, Atlan, backs the proposed fix. Treat it as a hypothesis to pilot, not a buying signal.

Sponsored 'Industrial Intelligence' Piece Lacks

Sponsored 'Industrial Intelligence' Piece Lacks

The Signal

Treat this sponsored article as vendor positioning, not evidence. Its two case-study results are presented without baselines, sample sizes or methods, so they cannot be checked. MIT Technology Review labels the piece as produced by Microsoft; its governance checklist is useful, but its performance figures are unverified claims.

Metis Reasoning: AlphaGo Idea, No Benchmarks Yet

Metis Reasoning: AlphaGo Idea, No Benchmarks Yet

Teardown

Wait. Metis Reasoning, a London startup founded by ex-DeepMind researcher Thore Graepel, proposes a knowledge ledger that separates facts from reasoning so conclusions can be checked against evidence. It has published no benchmarks and no working system. Until the company releases numbers, treat it as an architecture claim, not a hallucination fix.

GPT-6.1 Terra: Half the Price, Real Gains, Open Questions

GPT-6.1 Terra: Half the Price, Real Gains, Open Questions

The Signal

GPT-6.1 Terra is a real price cut with measurable gains, but independence is limited. Artificial Analysis reports prices of $1 input and $5 output per million tokens versus $2 and $12 for GPT-5.6 Terra, and a 6-point Intelligence Index gain at max effort. Signal for agent workloads, pending your own tests.

TasteVal's 2027 self-improvement signal is thinner

TasteVal's 2027 self-improvement signal is thinner

The Fine Print

Treat 2027 as a hypothesis, not a finding. TasteVal reports Opus 5.5 at a 2.3x compute multiplier over human experts, but the 95% interval runs 1.15 to 4.37, the tasks are unreleased, and the trend covers under a year. Jack Clark's shift, per The Decoder, rests on that thin evidence.

Canva Craft: Wait for Independent Tests Before Switching

Canva Craft: Wait for Independent Tests Before Switching

Teardown

Wait. Canva Craft, the first image model Canva Research trained in-house, ranks fourth on DesignArena by Canva's own account, and first in portraits and landscapes. That rank is the one number that matters, and no independent test has confirmed it. Compare Craft against your current model through the Canva API on your own prompts.

Google Policy Ties Personal AI Data to an Ads Exception

Google Policy Ties Personal AI Data to an Ads Exception

The Fine Print

Wait before enabling Google's personalized AI features on sensitive data. Google's privacy policy still says it does not show personalized ads based on Drive, Gmail or Google Photos content, but now excepts data a user chooses to make available to other Google services that support personalized ads. Whether Personal Intelligence triggers that exception is unconfirmed.

Claude's 180M-line proofs: strong, but only for RISC-V

Claude's 180M-line proofs: strong, but only for RISC-V

The Fine Print

Anthropic's result is credible but narrow: Claude produced machine-checked Rocq proofs that 123 open-source libraries cannot crash on inputs under 64 MB, and found hundreds of bugs fuzzing missed. The guarantee covers single-threaded RISC-V binaries on musl libc, not the x86 or ARM builds most teams actually ship.

Unreleased Anthropic Model Stops Attacks 67%: Wait on It

Unreleased Anthropic Model Stops Attacks 67%: Wait on It

Teardown

Bottom line: Wait. Anthropic's claim that an unreleased model is better aligned than Mythos 5.1 rests on one resampling study of one incident. The internal research model stopped attacks unprompted 67% of the time versus 10% for Opus 5.5, but Anthropic's own report also gives 62%, and outsiders cannot test the model.

Scalable AI Foundations: A Session With Little Evidence

Scalable AI Foundations: A Session With Little Evidence

The Signal

Noise, for now: MIT Technology Review's EmTech Future session on scalable AI names the right ingredients (data, engineering, platforms, operating models, governance) but, in the published summary, offers no benchmark, incident or named deployment to test. Treat it as a checklist, not evidence, until the full session shows measurable results.

View More Posts

Sign up for the Newsletter

The week in the field, weighed — what ships and where it slips.

Today's briefLoad more