Nvidia's SoL-Pi Cuts AI Agent Token Usage Nearly in Half
Every expensive AI agent is a negotiation between two systems: a powerful language model that wants to reason freely, and a harness that tells it what tools it can use, what context matters, and when to stop thinking.
A new paper from Nvidia researchers suggests that the fastest path to cheaper AI agents is not a better model at all. It is a better harness. And the best way to build that harness, the researchers found, is to let AI do it.
The system, called SoL-Pi, watches another agent's execution traces, proposes changes to the harness that connects the model to its tools, and tests those changes automatically. Across more than 3,000 runs and 60,000 agent-environment interactions, the approach cut token usage by up to 49 percent while retaining 93.7 percent of the original harness performance.
The invisible cost center
The longer AI agents work unsupervised, the more expensive they get. A single prediction that costs a fraction of a cent turns into long chains of reasoning, tool calls, and feedback loops. Token usage balloons along the way.
Most efficiency research has focused on the model itself: faster attention kernels, model compression through quantization, and swapping in cheaper models. The harness, which lives between the model and the environment, has received far less attention.
That oversight might be expensive. The harness controls how an agent reads state, runs actions, and processes feedback. Tool calls echo back long outputs. Context piles up with every planning step. Error logs are repeatedly sent through expensive models. These costs multiply on every task.
Optimizing the harness manually is difficult because tool usage, context management, verification, and abort logic are all tightly coupled. A change that saves tokens in one place can trigger errors elsewhere or push costs into a later phase.
Recursive optimization
SoL-Pi automates that work. A research agent watches another agent's traces, proposes changes to the harness, and tests them in prepared environments. Capability checks determine which candidates survive. Efficiency checks measure whether token usage actually dropped.
This is a form of recursive self-improvement: an AI system optimizing the infrastructure that an AI system runs on.
Across 535 executable environments, SoL-Pi explored 152 different directions. It generated 495 tasks from GitHub issue-pull-request pairs and 40 synthetic test cases. In total, the process ran more than 3,000 agent sessions and over 60,000 agent-environment interactions.
The researchers took care to avoid a common research pitfall: contaminating the evaluation data with the search data. They used a held-out set of 11 EdgeBench tasks for one-time validation of finished candidates, reserving the remaining 40 for the final evaluation. An automatic verification step catches cases where a cheaper harness produces wrong answers.
Four mechanisms that save tokens
The search produced four distinct mechanisms, each targeting a different source of token waste.
Action Fusion merges two consecutive steps into one. A code edit followed by a test run, for example, becomes a single action. This eliminates an entire language model call.
Online Context Compact runs after each planning step and trims accumulated context whenever it can do so without losing information needed for subsequent steps.
ObservationPack archives long tool outputs and drops in a short summary on later steps, rather than resending the full text each time.
Evidence-Preserving Reducer routes large error and test logs to a cheaper model that boils them down to the key findings. An automatic verification step catches any lost information.
The most efficient configuration combines all four mechanisms, achieving the 49 percent reduction at 93.7 percent of the original harness performance.
Running locally
The SoL-Pi approach is not tied to a specific model or benchmark. The paper's method for separating search from evaluation, and the four mechanisms it discovered, apply broadly to any coding agent that uses a harness layer.