SoL-Pi moves coding-agent efficiency into the harness

SoL-Pi is a new efficiency layer for the open-source Pi coding agent, released by researchers from NVIDIA, NTU and MIT on September 21. Its main shift is architectural: instead of changing the model or the Pi agent itself, it changes the harness around the agent. The authors report that, on the 51-task EdgeBench evaluation, the full SoL-Pi stack reduces token traffic by 44.7% to 49.0% and lowers API cost by about 33%, while retaining about 94% of Pi’s average score.

From model changes to harness changes

The work targets a familiar pressure point for coding agents: their repeated interaction with environments can generate long histories, repeated observations, and costly model calls. SoL-Pi does not claim to introduce a new coding model. It is described as an MIT-licensed extension that works with an unmodified Pi release. The reported tests used Pi 0.85.1 and Node.js 22.19 or later.

That distinction matters because efficiency work in coding agents can happen at several levels. A model provider can change the model. An agent developer can alter the agent’s planning or tool-use logic. A benchmark operator can measure end-to-end performance in a fixed environment. SoL-Pi sits in the second layer around the agent, but the reported numbers are still author-reported benchmark results, not independent measurements.

The researchers used an AI auto-research process to search for efficiency mechanisms. According to the paper, that process explored 152 directions across six families, 535 executable environments, more than 3,000 runs and over 60,000 agent-environment interactions. From that search, four mechanisms survived: Action Fusion, Online Context Compact, ObservationPack and an Evidence-Preserving Reducer.

The resulting system can be read as an attempt to reduce waste in the conversation between a coding agent and its environment. Rather than asking the underlying model to become cheaper or more concise by itself, SoL-Pi constrains, compresses or reorganizes what the model sees and does through the harness. The authors’ claim is therefore not just that Pi can be cheaper on a benchmark, but that some efficiency can be extracted without retraining or replacing the agent.

Four mechanisms around Pi

Action Fusion is one of the four mechanisms selected by the auto-research process. Based on the name and placement in the harness stack, its role is to reduce inefficient action patterns by combining or streamlining operations that would otherwise be handled separately. The verified material does not provide implementation detail beyond identifying it as one of the surviving mechanisms, so its precise internal rules should not be inferred beyond that.

Online Context Compact addresses the context carried through the agent’s run. Coding agents often accumulate information as they inspect files, run commands, observe outputs and revise plans. The mechanism’s stated place in SoL-Pi indicates that it compacts context during the run, with the aim of lowering token traffic while preserving enough information for the agent to continue solving tasks.

ObservationPack is another harness-side mechanism. Its name indicates attention to how observations from the environment are packaged before reaching the agent. In a coding-agent loop, raw observations can be verbose, repeated or poorly structured for the next model call. The mechanism is part of the authors’ reported stack for reducing token traffic, though the brief does not support a more specific description of its formatting choices.

The Evidence-Preserving Reducer is the fourth surviving mechanism. Its name highlights a central trade-off in compression for agentic systems: reducing text can remove information that later turns out to be needed. By preserving evidence, the reducer is positioned as a safeguard against compressing away the basis for decisions. The measured result reported by the authors is that the full stack retains about 94% of Pi’s average EdgeBench score while reducing token traffic and cost.

What the benchmark numbers show

On EdgeBench, the main author-reported result is a reduction in token traffic of 44.7% to 49.0% for the full SoL-Pi stack. The same evaluation reports about 33% lower API cost. Because API cost is tied to model calls and token use, the result is consistent with the system’s goal: reduce the amount of text and interaction overhead sent through the agent loop while keeping much of the benchmark score.

The retained performance figure is also important. The authors report that SoL-Pi keeps about 94% of Pi’s average score on the 51-task EdgeBench evaluation. That is not the same as saying performance is unchanged, and it is not an independent validation. It means that, under the authors’ reported conditions, the harness-side reductions came with a measured score loss relative to Pi, while still preserving most of the average score.

Terminal-Bench 4 gives a second reported data point with a different shape. There, SoL-Pi solved 15 tasks compared with 18 for Pi, while reducing total cost by 26.3%. This result shows the trade-off more plainly: fewer solved tasks in exchange for lower total cost. It also cautions against reading the EdgeBench cost reduction as a universal free gain across benchmarks.

Cross-model transfer is described as preliminary. That limits the generality of the claim. A harness method that works well with one agent and one tested release may not necessarily carry the same efficiency-performance balance to other model and agent combinations. The verified facts support the statement that SoL-Pi works with an unmodified Pi release and was tested with Pi 0.85.1, but they do not establish broad transfer across coding-agent systems.

Practical implications and limits

The practical implication is that coding-agent operators may have another place to look for efficiency: the environment harness. If repeated observations, verbose context and inefficient action patterns are major cost drivers, then a wrapper can reduce usage without changing the agent release itself. SoL-Pi’s MIT license and compatibility with unmodified Pi are relevant here because they make the approach a separable extension rather than a forked agent design.

At the same time, the evidence is bounded by the reported benchmarks. The EdgeBench figures are author-reported results on 51 tasks. The Terminal-Bench 4 figures are also reported by the authors and show a lower solved-task count than Pi. The auto-research process was extensive in the numbers provided, but it does not replace independent replication. The numbers demonstrate a reported efficiency-performance trade-off under specified tests; they do not prove that all coding-agent workloads will see the same cost reduction or the same retained score.

The broader point is methodological. SoL-Pi reframes part of coding-agent optimization as harness engineering: controlling actions, compacting context, packaging observations and reducing evidence without discarding the information needed to act. The reported one-third lower API cost on EdgeBench is the headline outcome, but the more durable question is whether such harness-side mechanisms can be measured consistently across agents, models and tasks without hiding failure modes behind compression.