StandardsAboutContact
The Weights
Nvidia's SoL-Pi Is Real, Open-Source Code

Nvidia's SoL-Pi Is Real, Open-Source Code

Nvidia's SoL-Pi harness optimizer is a genuine, MIT-licensed, npm-installable release, not a paper-only research result. But on Terminal-Bench 4, the paper's own numbers show it solving fewer tasks than the baseline it was built to beat.

Bottom line: Wait. Nvidia's SoL-Pi (arXiv:2609.20519) is real, MIT-licensed, npm-installable code for the Pi coding-agent harness — not vaporware. It cuts EdgeBench tokens up to 49% at 93.7-94.3% score retention, but on Terminal-Bench 4 it solves only 15 of 63 tasks versus 18 for both Pi and Codex, a real capability regression its own headline numbers don't disclose.

The Weights Desk · 5 min read

Nvidia's SoL-Pi is not the unreleased research curiosity it might first appear to be: the code behind it is public, MIT-licensed, and installable today at github.com/NVlabs/SoL-Pi. The bottom line is still Wait, but not for the reason a surface read of the paper suggests — the real problem is that on Terminal-Bench 4, a genuine held-out benchmark, SoL-Pi solves fewer tasks than the baseline harness it was built to make cheaper.

What Nvidia actually released

SoL-Pi is a four-mechanism optimization layer — Action Fusion, ObservationPack (observation archiving), Online Context Compaction, and the Evidence-Preserving Reducer — bolted onto Nvidia's own 'Pi' coding-agent harness. The paper (arXiv:2609.20519) links a working GitHub repository, MIT-licensed with roughly 3.1k stars, installable via npm and requiring Node.js 22.19+ inside the Pi framework. It is not a universal patch for Codex or Claude Code; it only runs inside Pi.

The numbers that hold up

On EdgeBench, SoL-Pi cuts token usage 49.0% and API cost 33.2% against Pi, retaining 93.7% of Pi's average score (42.0 vs. 44.8), per the paper's Table 1. Transferred unmodified to Anthropic's Opus 5, it keeps 94.3% of score while cutting tokens 44.7% and cost 33.5% — though the paper notes trigger rates run lower on Opus 5, since the mechanisms were tuned exclusively on GPT-5.6 Sol trajectories.

The number the headline hides

On Terminal-Bench 4's 63 CPU-only tasks, SoL-Pi solves 15 — the Pi baseline and Codex each solve 18, per Table 3. SoL-Pi's $14.07 cost per solved task beats Pi's $15.91 and Codex's $15.13, but that framing obscures a real capability regression: three fewer tasks solved outright, not just a cost tradeoff. The 93.7-94.3% retention figure describes EdgeBench and the Opus 5 transfer test — it does not describe Terminal-Bench 4.

Who should use this

Teams already running Nvidia's Pi harness on GPT-5.6 Sol, working on EdgeBench-style tasks, can install SoL-Pi today and expect the paper's documented cost savings to roughly hold. Teams whose workloads look more like Terminal-Bench's CPU-only, longer-horizon tasks should test the solve-rate regression on their own suite first — the paper's own results show cheaper does not mean equally capable there. SoL-Pi also is not a drop-in for non-Pi agent stacks.

The bottom line

Wait — not because the code is unavailable, but because the paper's own hardest, most realistic benchmark shows a real capability regression that the marketed 93.7-94.3% retention number doesn't cover. SoL-Pi is a legitimate, freely installable engineering contribution with real EdgeBench gains, discovered through a genuinely large automated search (roughly 150 candidates, 3,000-plus runs). But it is Pi-specific, tuned on a single backend, and on Terminal-Bench 4 solves fewer tasks than both its own baseline and Codex. Test it on your own hardest benchmark, inside the Pi harness, before trusting the headline percentage.

Is SoL-Pi actually available to use today?
Yes. The code is public at github.com/NVlabs/SoL-Pi, MIT-licensed with roughly 3.1k stars, and installable via npm (`pi install git:github.com/NVlabs/SoL-Pi`) — it requires Nvidia's own Pi coding-agent framework and Node.js 22.19+, per the repository and the arXiv listing's linked project page.
Does SoL-Pi match the baseline it's compared against?
It depends on the benchmark. On EdgeBench it retains 93.7-94.3% of the Pi baseline's average score across configurations. But on Terminal-Bench 4, per the paper's Table 3, SoL-Pi solves only 15 of 63 tasks outright versus 18 for both Pi and Codex — a real task-count regression the retention framing doesn't capture.
What is the Evidence-Preserving Reducer?
The fourth of SoL-Pi's four harness mechanisms: it compresses build and test logs of at least 4 KiB, archiving the exact output while asking a lower-cost model (GPT-5.6 Luna, per the paper) to extract key evidence into a compact, deterministically verified receipt.
Who should actually use this?
Teams already running Nvidia's Pi harness on GPT-5.6 Sol, for EdgeBench-like coding tasks where the paper shows real cost savings. Teams whose workload looks more like Terminal-Bench's CPU-only tasks should weigh the disclosed 15-vs-18 solve-rate regression before adopting, and should not assume the harness is model-agnostic beyond Pi.
  1. SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness — arXiv
  2. NVlabs/SoL-Pi — GitHub
  3. Nvidia's SoL-Pi system cuts coding agent token usage nearly in half by optimizing the harness — The Decoder