Prime Intellect Releases Verifiers v1: Composable Tasksets, Harnesses, and Runtimes for Agentic RL Training and Evaluations

0 0

Prime Intellect launched verifiers 0.2.0. It previews a rewritten core, shipped under the new verifiers.v1 namespace. Modern evaluations now run coding agents with tools, compaction, and subagents. Accordingly, v1 rebuilds environments to run these agentic workloads at scale.

What is verifiers v1?

First, consider what verifiers is: Prime Intellect’s environment stack for agentic reinforcement learning and evaluations. Previously, an environment bundled its data, agent logic, and infrastructure together. In contrast, v1 breaks that bundle into three composable pieces.

A taskset defines the work: the data, tools, and scoring. A harness solves the task and produces a rollout. That harness can be a ReAct loop, a CLI agent, or your own. The rollout then runs inside a runtime, either local or in a sandbox. Because the pieces decouple, any taskset runs under any compatible harness.

How the Architecture Works?

With those pieces defined, the next question is how they communicate. The central piece is the verifiers-managed interception server. It sits between the agent’s runtime and the inference server. Specifically, it proxies requests to, and responses from, inference. Meanwhile, it records the trace, sets sampling parameters, and can rewrite tool responses. That rewriting helps mitigate reward hacks during training.

For scale, each server multiplexes a constant number of rollouts, defaulting to 32. A pool then scales elastically with observed concurrency. The server also owns a client that relays those requests. During evaluation, an EvalClient acts as a blind HTTP proxy. During training, a TrainClient wraps renderers for faithful token-in RL training.

Because harnesses speak different dialects, verifiers supports three as of now. These are OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages. A dialect adapter normalizes each wire format into canonical vf.types. Consequently, your scoring logic stays independent of the agent tested.

Run rollout</button>
<button id=”vf-reset” class=”vf-ghost”>Reset</button>
<span class=”vf-lab”>Harness dialect:</span>
<select id=”vf-dialect”>
<option value=”Chat”>OpenAI Chat Completions</option>
<option value=”Resp”>OpenAI Responses</option>
<option value=”Msg”>Anthropic Messages</option>
</select>
</div>

<div class=”vf-stage”>
<div class=”vf-row” style=”margin-bottom:14px”>
<div class=”vf-node vf-taskset” id=”n-taskset”>
<div class=”vf-nt”>Taskset</div>
<div class=”vf-nd”>what · data · tools · scoring</div>
</div>
</div>

<div class=”vf-runtime-wrap”>
<span class=”vf-runtime-tag”>RUNTIME · where (subprocess · Docker · sandbox)</span>
<div class=”vf-row” id=”vf-flow”>
<div class=”vf-node vf-harness” id=”n-harness”>
<div class=”vf-nt”>Harness</div>
<div class=”vf-nd”>how · Codex · Terminus 2 · ReAct</div>
</div>
<div class=”vf-arrow”>→</div>
<div class=”vf-node vf-intercept” id=”n-intercept”>
<div class=”vf-nt”>Interception Server</div>
<div class=”vf-nd”>proxy · records trace</div>
</div>
<div class=”vf-arrow”>→</div>
<div class=”vf-node vf-infer” id=”n-infer”>
<div class=”vf-nt”>Inference Server</div>
<div class=”vf-nd”>vLLM · model</div>
</div>
<div class=”vf-packet” id=”vf-packet”>req</div>
</div>
</div>

<div class=”vf-status” id=”vf-status”>Press “Run rollout” to send a request through the interception server.</div>
</div>

<div class=”vf-grid”>
<div class=”vf-panel”>
<h3>Trace · message graph (v1)</h3>
<div class=”vf-hint”>Each message is a unique node. Size grows linearly in turns.</div>
<div class=”vf-graph” id=”vf-graph”>
<div class=”vf-empty”>No messages recorded yet.</div>
</div>
</div>

<div class=”vf-panel”>
<h3>Trace size: v0 vs v1</h3>
<div class=”vf-hint”>Drag to change turns. v0 repeats prompt-completion pairs; v1 stores unique nodes.</div>
<div class=”vf-chart”>
<svg viewBox=”0 0 260 150″ id=”vf-svg”>
<line x1=”30″ y1=”130″ x2=”255″ y2=”130″ stroke=”#dfe6ef” stroke-width=”1.5″/>
<line x1=”30″ y1=”10″ x2=”30″ y2=”130″ stroke=”#dfe6ef” stroke-width=”1.5″/>
<path id=”vf-v0″ fill=”none” stroke=”#d1477a” stroke-width=”2.5″/>
<path id=”vf-v1″ fill=”none” stroke=”#0b8f8f” stroke-width=”2.5″/>
<text x=”140″ y=”147″ font-size=”9″ fill=”#94a3b8″ text-anchor=”middle”>turns →</text>
</svg>
</div>
<div class=”vf-legend”>
<span><i style=”background:#d1477a”></i> v0 · quadratic</span>
<span><i style=”background:#0b8f8f”></i> v1 · linear</span>
</div>
<div class=”vf-slider-row”>
<span>Turns</span>
<input type=”range” id=”vf-turns” min=”4″ max=”60″ value=”24″>
<span id=”vf-turns-val” style=”width:26px;text-align:right”>24</span>
</div>
</div>
</div>

<div class=”vf-foot”>
Illustrative demo of the verifiers v1 architecture · Built by <b>Marktechpost</b>
</div>
</div>

<script>
(function(){
var root=document.getElementById(“vfv1-demo”);
var packet=document.getElementById(“vf-packet”);
var status=document.getElementById(“vf-status”);
var graph=document.getElementById(“vf-graph”);
var runBtn=document.getElementById(“vf-run”);
var resetBtn=document.getElementById(“vf-reset”);
var dialectSel=document.getElementById(“vf-dialect”);
var nHarness=document.getElementById(“n-harness”);
var nIntercept=document.getElementById(“n-intercept”);
var nInfer=document.getElementById(“n-infer”);
var flow=document.getElementById(“vf-flow”);

var turn=0, running=false;
var msgs=[]; // recorded nodes
var dialectLabel={Chat:”Chat”,Resp:”Resp”,Msg:”Msg”};

function pos(el){ // center x relative to flow
var f=flow.getBoundingClientRect();
var r=el.getBoundingClientRect();
return (r.left – f.left) + r.width/2 – 32;
}
function clearActive(){ [nHarness,nIntercept,nInfer].forEach(function(n){n.classList.remove(“vf-active”);}); }

function movePacket(fromEl,toEl,ms,label,isResp){
return new Promise(function(res){
packet.textContent=label;
packet.classList.toggle(“vf-resp”,!!isResp);
packet.style.transition=”none”;
packet.style.left=pos(fromEl)+”px”;
packet.style.opacity=”1″;
void packet.offsetWidth;
packet.style.transition=”left “+ms+”ms cubic-bezier(.45,.05,.35,1)”;
packet.style.left=pos(toEl)+”px”;
setTimeout(res,ms);
});
}

function addNode(role,label,color){
if(msgs.length===0){ graph.innerHTML=””; }
var d=document.createElement(“div”);
d.className=”vf-msg”;
d.innerHTML='<span class=”vf-dot” style=”background:’+color+'”></span><code>’+label+'</code><span class=”vf-role”>’+role+'</span>’;
graph.appendChild(d);
graph.scrollTop=graph.scrollHeight;
msgs.push(label);
}

function sleep(ms){return new Promise(function(r){setTimeout(r,ms);});}

async function runTurn(){
if(running) return;
running=true; runBtn.disabled=true;
turn++;
var dl=dialectLabel[dialectSel.value];

// seed system + user on first turn
if(turn===1){
addNode(“system”,”S1″,”#6366f1″); await sleep(160);
addNode(“user”,”U1″,”#6366f1″);
}

clearActive();
nHarness.classList.add(“vf-active”);
status.textContent=”Harness builds a “+dl+” request…”;
await sleep(350);

// harness -> interception
nIntercept.classList.add(“vf-active”);
status.textContent=”Interception server proxies the request → inference.”;
await movePacket(nHarness,nInfer,850,dl+” req”);
clearActive(); nInfer.classList.add(“vf-active”);
status.textContent=”Inference server generates the reply (vLLM).”;
await sleep(350);

// inference -> interception (records) -> harness
nIntercept.classList.add(“vf-active”);
status.textContent=”Interception server records the trace, relays the response.”;
await movePacket(nInfer,nHarness,850,”resp”,true);
packet.style.opacity=”0″;
clearActive();

// record assistant node (+ occasional tool)
addNode(“assistant”,”A”+turn,”#0b8f8f”); await sleep(150);
if(turn%2===0){ addNode(“tool”,”T”+turn,”#e0a800″); }

status.textContent=”Turn “+turn+” recorded as a unique node in the message graph.”;
running=false; runBtn.disabled=false;
}

function reset(){
turn=0; msgs=[]; running=false; runBtn.disabled=false;
clearActive(); packet.style.opacity=”0″;
graph.innerHTML='<div class=”vf-empty”>No messages recorded yet.</div>’;
status.textContent=”Press “Run rollout” to send a request through the interception server.”;
}

runBtn.addEventListener(“click”,runTurn);
resetBtn.addEventListener(“click”,reset);

// —- v0 vs v1 growth chart —-
var v0=document.getElementById(“vf-v0”);
var v1=document.getElementById(“vf-v1”);
var turnsR=document.getElementById(“vf-turns”);
var turnsV=document.getElementById(“vf-turns-val”);

function drawChart(N){
var x0=30,x1=255,y0=130,y1=12,W=x1-x0,H=y0-y1;
var maxV0=N*N; // quadratic reference
function ptV0(i){var x=x0+(i/N)*W;var y=y0-((i*i)/maxV0)*H;return x+”,”+y;}
function ptV1(i){var x=x0+(i/N)*W;var y=y0-((i/N)*H);return x+”,”+y;} // linear
var p0=”M”,p1=”M”;
for(var i=0;i<=N;i++){ p0+=(i?” L”:””)+ptV0(i); p1+=(i?” L”:””)+ptV1(i); }
v0.setAttribute(“d”,p0); v1.setAttribute(“d”,p1);
}
turnsR.addEventListener(“input”,function(){ turnsV.textContent=turnsR.value; drawChart(+turnsR.value); });
drawChart(+turnsR.value);

// —- auto-resize for WordPress iframe embedding —-
function sendHeight(){
var h=document.getElementById(“vfv1-demo”).offsetHeight+40;
if(window.parent){ window.parent.postMessage({vfv1Height:h},”*”); }
}
window.addEventListener(“load”,sendHeight);
window.addEventListener(“resize”,sendHeight);
new MutationObserver(sendHeight).observe(document.getElementById(“vf-graph”),{childList:true});
setInterval(sendHeight,1200);
})();
</script>
</body>
</html>
“>

v0 vs v1: A Quick Comparison

These changes separate v1 from v0.

Use Cases with Examples

With the architecture clear, consider how teams use it. For example, you can run Nemotron 3 Ultra on Terminal-Bench 2 under Codex.

Similarly, teams can reuse Harbor datasets without rewriting reward logic. Prime Intellect ported Terminal Bench 2 into v1 with only a small class. In its internal testing, verifiers matched Harbor’s performance on the same tasks. Harbor is the first fully-supported third-party format; NeMo Gym and OpenEnv have alpha support.

On the training side, the same environments plug into prime-rl directly. In a length-penalty ablation, GLM-4.5-Air trained on ScaleSWE across six H200 nodes. That run took two days and evaluated on SWE-Bench-Verified, showing stable agentic training.

A Minimal Taskset and Launch

Each run starts from a taskset that defines data and scoring, independent of any harness:

import verifiers.v1 as vf

class AdditionData(vf.TaskData):
    answer: int

class AdditionTask(vf.Task[AdditionData]):
    @vf.reward
    async def exact_match(self, trace: vf.Trace) -> float:
        return float(trace.last_reply == str(self.data.answer))

class AdditionTaskset(vf.Taskset[AdditionTask, vf.TasksetConfig]):
    def load(self) -> list[AdditionTask]:
        return [
            AdditionTask(
                AdditionData(idx=i, prompt=f"What is {i} + {i}?", answer=2 * i),
                self.config.task,
            )
            for i in range(100)
        ]

__all__ = ["AdditionTaskset"]

Any taskset then runs under a chosen harness via TOML and the CLI:

model = "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B"

[taskset]
id = "primeintellect/terminal-bench-2"

[harness]
id = "codex"
version = "0.116.0"

uv run eval @ path/to/config.toml

Key Takeaways

• verifiers v1 splits an environment into a taskset (what), a harness (how), and a runtime (where).
• A verifiers-managed interception server proxies harness–inference requests and records traces on the fly.
• A linear message-graph trace replaces v0’s quadratic prompt-completion pairs, enabling long-horizon training.
• It ships with full prime-rl training support; the legacy code path is now frozen.
• Harbor datasets and harnesses like Codex and Terminus 2 work out of the box.

Check out the Technical details. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us

The post Prime Intellect Releases Verifiers v1: Composable Tasksets, Harnesses, and Runtimes for Agentic RL Training and Evaluations appeared first on MarkTechPost.

​MarkTechPost

About Post Author

Manoj Balakrishnan

[email protected]

https://annapoornainfo.com

Happy

0 0 %

Sad

0 0 %

Excited

0 0 %

Sleepy

0 0 %

Angry

0 0 %

Surprise

0 0 %