
Nvidia's SoL-Pi Is Real, Open-Source Code
Nvidia's SoL-Pi harness optimizer is a genuine, MIT-licensed, npm-installable release, not a paper-only research result. But on Terminal-Bench 4, the paper's own numbers show it solving fewer tasks than the baseline it was built to beat.
Bottom line: Wait. Nvidia's SoL-Pi (arXiv:2609.20519) is real, MIT-licensed, npm-installable code for the Pi coding-agent harness — not vaporware. It cuts EdgeBench tokens up to 49% at 93.7-94.3% score retention, but on Terminal-Bench 4 it solves only 15 of 63 tasks versus 18 for both Pi and Codex, a real capability regression its own headline numbers don't disclose.
The Weights Desk · 5 min read- SoL-Pi ships as real, installable code — MIT-licensed, roughly 3.1k GitHub stars, npm-installable inside Nvidia's own 'Pi' coding-agent framework — not an unreleased research artifact, correcting an earlier assumption that no public release existed.
- On EdgeBench, SoL-Pi cuts tokens 49.0% and API cost 33.2% while retaining 93.7% of Pi's average score (42.0 vs. 44.8); transferred unmodified to Anthropic's Opus 5 it keeps 94.3% of score while cutting tokens 44.7%, per the paper's own tables.
- On Terminal-Bench 4's 63 CPU-only tasks, SoL-Pi solves only 15 — fewer than both the Pi baseline and Codex, which each solve 18 — even though its $14.07 cost per solved task beats Pi's $15.91 and Codex's $15.13.
- The four harness mechanisms — Action Fusion, ObservationPack, Online Context Compaction, and the Evidence-Preserving Reducer (which compresses build/test logs into a compact receipt using a cheaper model) — were found by testing roughly 150 candidate designs; the paper explicitly states this search scope 'does not establish a scaling law.'
- SoL-Pi is coupled to Nvidia's own Pi agent harness, not a universal patch for arbitrary coding agents, and its mechanisms were optimized on one backend (GPT-5.6 Sol) with only one other model (Opus 5) tested for transfer, where trigger rates ran lower.
Nvidia's SoL-Pi is not the unreleased research curiosity it might first appear to be: the code behind it is public, MIT-licensed, and installable today at github.com/NVlabs/SoL-Pi. The bottom line is still Wait, but not for the reason a surface read of the paper suggests — the real problem is that on Terminal-Bench 4, a genuine held-out benchmark, SoL-Pi solves fewer tasks than the baseline harness it was built to make cheaper.
What Nvidia actually released
SoL-Pi is a four-mechanism optimization layer — Action Fusion, ObservationPack (observation archiving), Online Context Compaction, and the Evidence-Preserving Reducer — bolted onto Nvidia's own 'Pi' coding-agent harness. The paper (arXiv:2609.20519) links a working GitHub repository, MIT-licensed with roughly 3.1k stars, installable via npm and requiring Node.js 22.19+ inside the Pi framework. It is not a universal patch for Codex or Claude Code; it only runs inside Pi.
The numbers that hold up
On EdgeBench, SoL-Pi cuts token usage 49.0% and API cost 33.2% against Pi, retaining 93.7% of Pi's average score (42.0 vs. 44.8), per the paper's Table 1. Transferred unmodified to Anthropic's Opus 5, it keeps 94.3% of score while cutting tokens 44.7% and cost 33.5% — though the paper notes trigger rates run lower on Opus 5, since the mechanisms were tuned exclusively on GPT-5.6 Sol trajectories.
The number the headline hides
On Terminal-Bench 4's 63 CPU-only tasks, SoL-Pi solves 15 — the Pi baseline and Codex each solve 18, per Table 3. SoL-Pi's $14.07 cost per solved task beats Pi's $15.91 and Codex's $15.13, but that framing obscures a real capability regression: three fewer tasks solved outright, not just a cost tradeoff. The 93.7-94.3% retention figure describes EdgeBench and the Opus 5 transfer test — it does not describe Terminal-Bench 4.
Who should use this
Teams already running Nvidia's Pi harness on GPT-5.6 Sol, working on EdgeBench-style tasks, can install SoL-Pi today and expect the paper's documented cost savings to roughly hold. Teams whose workloads look more like Terminal-Bench's CPU-only, longer-horizon tasks should test the solve-rate regression on their own suite first — the paper's own results show cheaper does not mean equally capable there. SoL-Pi also is not a drop-in for non-Pi agent stacks.
The bottom line
Wait — not because the code is unavailable, but because the paper's own hardest, most realistic benchmark shows a real capability regression that the marketed 93.7-94.3% retention number doesn't cover. SoL-Pi is a legitimate, freely installable engineering contribution with real EdgeBench gains, discovered through a genuinely large automated search (roughly 150 candidates, 3,000-plus runs). But it is Pi-specific, tuned on a single backend, and on Terminal-Bench 4 solves fewer tasks than both its own baseline and Codex. Test it on your own hardest benchmark, inside the Pi harness, before trusting the headline percentage.
- Is SoL-Pi actually available to use today?
- Yes. The code is public at github.com/NVlabs/SoL-Pi, MIT-licensed with roughly 3.1k stars, and installable via npm (`pi install git:github.com/NVlabs/SoL-Pi`) — it requires Nvidia's own Pi coding-agent framework and Node.js 22.19+, per the repository and the arXiv listing's linked project page.
- Does SoL-Pi match the baseline it's compared against?
- It depends on the benchmark. On EdgeBench it retains 93.7-94.3% of the Pi baseline's average score across configurations. But on Terminal-Bench 4, per the paper's Table 3, SoL-Pi solves only 15 of 63 tasks outright versus 18 for both Pi and Codex — a real task-count regression the retention framing doesn't capture.
- What is the Evidence-Preserving Reducer?
- The fourth of SoL-Pi's four harness mechanisms: it compresses build and test logs of at least 4 KiB, archiving the exact output while asking a lower-cost model (GPT-5.6 Luna, per the paper) to extract key evidence into a compact, deterministically verified receipt.
- Who should actually use this?
- Teams already running Nvidia's Pi harness on GPT-5.6 Sol, for EdgeBench-like coding tasks where the paper shows real cost savings. Teams whose workload looks more like Terminal-Bench's CPU-only tasks should weigh the disclosed 15-vs-18 solve-rate regression before adopting, and should not assume the harness is model-agnostic beyond Pi.