StandardsAboutContact

The Weights

Breaking News

Rumble-2-Flash teardown: Use it, with one audio testTeardown

Rumble-2-Flash teardown: Use it, with one audio test

Use it, with a test on your own audio first. Rumble-2-Flash, a 0.5B Apache 2.0 speech model from Reson8, averages 6.60% word error rate on the English Open ASR Leaderboard and 7.86% on AppTek phone calls. Weak spots are Hindi at 14.11% and conversational speech at 13.14%.

Read More
The Fine Print

Vibe Desktop Ships Parallel Agents, but No Evals

Vibe Desktop is a packaging move, not a demonstrated capability gain. Mistral's Vibe Desktop app, per The Decoder, runs parallel agent tasks in Git worktrees on macOS and Windows, but neither source reports benchmark, latency or cost data. Teams should treat it as a coding-agent client to pilot, not evidence.

Read More
The Fine Print

Hugging Face Routes LLM Calls Through OpenRouter

Hugging Face's change is a net convenience with a real lock-in cost: language-model requests on Inference Providers now run through OpenRouter, and bring-your-own provider keys no longer work for them. The 46 percent capacity-error cut is OpenRouter's own measurement, so treat it as unverified until you test your own workload.

Read More
The Signal

Crosspoint Reader shows review, not code, is now scarce

Crosspoint Reader shows that LLM-assisted contributions shift open-source scarcity from code to review. Its lead, Justin Mitchell, told an Ubuntu Summit audience the project gets multiple submissions daily, including automated ones, and that judgement, trust and responsibility are now the limiting resources.

Read More

Featured Stories

BotPsy-0.8B: Big Tool-Use Gains, No Device Numbers

BotPsy-0.8B: Big Tool-Use Gains, No Device Numbers

Teardown

Wait. BotPsy-0.8B, a Qwen3.5-0.8B fine-tune from Tether AI Research, lifts BFCL v4 tool-calling from 23.4 to 44.0 and beats the larger Qwen3.5-2B-Think on that benchmark, but the release reports no latency, memory or device figures, and multi-step retrieval stays weak at 4.5 on BrowseComp-Plus.

Willison Tags 244 Vibe-Coded Tools: Noise for Security

Willison Tags 244 Vibe-Coded Tools: Noise for Security

The Signal

Noise for enterprise teams: Simon Willison used Fable 5.2 and tree-sitter to tag his 244 vibe-coded tools by JavaScript API, library and web API host. It is a self-reported personal-site feature we have not tested, with no security finding and no benchmark behind it, so it supports no conclusion about AI-generated code beyond what the author states.

Muse Ember: cheap agent work, weak coding

Muse Ember: cheap agent work, weak coding

Teardown

Muse Ember is a credible cheap worker for tool calls, routing and extraction, but not a coding default. Artificial Analysis scores it 36 on its Intelligence Index, level with Claude Haiku 5.5, at $0.14 per task against $0.41, yet it trails Haiku 5.5 on FrontierCode 2.0 and in factual accuracy.

Claude Code Built a Beep Locator That Found a Dead Alarm

Claude Code Built a Beep Locator That Found a Dead Alarm

Teardown

Beep Locator is a real, working result, not a demo: Claude Code built a multi-device acoustic triangulation tool that narrowed a mystery beep to about 40 cm, per its author. The evidence is a single anecdote with a clear success, but it shows agentic coding handling hardware-in-the-loop debugging.

AI's Climate Case Rests on Few Deployments That Shipped

AI's Climate Case Rests on Few Deployments That Shipped

Ships or Slips

The climate case for AI is still unproven in production. An MIT Technology Review opinion piece by Mary Nichols and Jeremy Tamanini reports that, of grid operators reviewed in China, the US and Europe, only National Grid ESO runs a working AI forecasting program. Meanwhile, the IEA projects data-centre electricity demand of about 945 TWh by 2030.

Inkling 2 Small's Swearing Is Noise

Inkling 2 Small's Swearing Is Noise

The Signal

Profanity in Inkling 2 Small's reasoning is a curiosity, not a failure detector. Vals AI found 316 instances across 2,655 benchmark runs, 241 from two runaway runs, and 96 percent of failed runs never swore. Treat swearing as a weak log-hygiene signal for Thinking Machines deployments, and measure failures with graded outcomes instead.

Perplexity Teammates: Persistent Agents With Real Access

Perplexity Teammates: Persistent Agents With Real Access

The Signal

Perplexity's Teammates are persistent agents that follow up on their own and keep memory across runs. Perplexity's announcement ties them to Max subscribers and lets them hold saved credentials and GitHub access. For security teams, that standing access is the real evaluation point. Perplexity has published no Teammates benchmark or red-team result.

GDPx: no AI agent tops 12% on real-work tasks

GDPx: no AI agent tops 12% on real-work tasks

The Signal

No model clears 12 percent on SemiAnalysis' GDPx, and the benchmark is better read as a map of agent weaknesses than a ranking. Claude Opus 5.5 leads at 12.0 percent, with Gemini 4 Carbon at 10.9 percent and a third of the cost per task, but small per-domain samples limit precision.

Meta's Muse Avatar Latency Claims Rest on Its Own Tests

Meta's Muse Avatar Latency Claims Rest on Its Own Tests

The Fine Print

Meta's own research post gives Muse Realtime Avatar numbers worth arguing with: about 870 ms latency to first byte and 12 concurrent sessions per GB200. But they are company-measured on one hardware stack, and Meta has not said the voice mode now reaching users runs on it.

SETI's Telescope Agents Work, but Prove Little So Far

SETI's Telescope Agents Work, but Prove Little So Far

Red Team

The Allen Telescope Array's agentic AI setup is a credible engineering upgrade but not yet a verified science result. The Crab pulsar detection shows the pipeline works on a known natural source, and the weeks-to-hours speedup comes from NVIDIA's account. No extraterrestrial signal has been found, and we found no independent paper on the agents.

Message Threads: 35% Median TTFT Cut Is Claude Code Data

Message Threads: 35% Median TTFT Cut Is Claude Code Data

The Signal

Message Threads, Anthropic's Messages API beta, cut median time to first token about 35 percent for 500,000 to one million token requests, by Anthropic's own account. The figure comes from Claude Code production usage, not a controlled benchmark. Token usage and billing stay unchanged, and threads are best-effort, so teams must handle expiry.

Palette: IBM's Self-Hosted 20B Deck Model, Teardown

Palette: IBM's Self-Hosted 20B Deck Model, Teardown

Teardown

Bottom line: Wait. Palette, an IBM Research slide-deck system built on a self-hostable 20B open-weight model, scored 59.7 on the PresentBench three-domain mean against NotebookLM's 60.8. The result is close, but the weights, code and paper are not linked, and the evaluation still needs a frontier model to draft plans.

StateBench Verdict: Wait on Long-Document State Tracking

StateBench Verdict: Wait on Long-Document State Tracking

Teardown

Wait before trusting frontier models with long-document state tracking. In Ai2's StateBench, Claude Opus 5.5, GPT-6 Sol, GPT-6 Luna and Gemini 3.8 Flash handled simple updates, but accuracy on the hardest level fell to 13% or lower at the longest lengths. Gated DeltaNet models trained on the task did far better.

Trump's 'Super Intelligence' Order

Trump's 'Super Intelligence' Order

The Signal

Executive Order 14434 is a terminology mandate, not a regulation: it renames AI as Super Intelligence across federal documents without changing any statutory definition or obligation. For practitioners, the rename is Noise operationally, but compliance is uneven, and the White House Accord on Super Intelligence remains voluntary.

HF's Model Contracts: Use It, But Not as a Merge Gate

HF's Model Contracts: Use It, But Not as a Merge Gate

Teardown

Use it, but only as a non-blocking signal for now. Hugging Face's Model Integration Contracts flagged the UMT5 attention regression on CPU for two months before a fix landed. The authors have not measured false alarms, review effort or PR catch rates, and the Transformers job still does not block merges.

View More Posts

Sign up for the Newsletter

The week in the field, weighed — what ships and where it slips.

Today's briefLoad more