---
title: "The Verification Horizon: No Silver Bullet for Coding Agent Rewards"
created: 2026-07-02
type: raw
arxiv: "2606.26300"
authors: "Binghai Wang, Chenlong Zhang, Dayiheng Liu, Jiajun Zhang, Jiawei Chen, Mingze Li, Mouxiang Chen, Rongyao Fang, Siyuan Zhang, Xuwu Wang, Yuheng Jing, Zeyao Ma, Zeyu Cui (Qwen Team, Alibaba)"
venue: arXiv
date: 2026-06-24
---

# Raw: The Verification Horizon

A classical intuition holds that verifying a solution is easier than producing one. For today's coding agents, this intuition is being inverted: generating complex candidate solutions is no longer difficult — reliably verifying them has become the harder problem.

## Core Framework

Every verifier is a proxy for human intent, never the intent itself. Verification quality characterized along three dimensions:
- **Scalability**: can the signal be produced cheaply at training scale?
- **Faithfulness**: how much of true user intent does the signal reflect?
- **Robustness**: can the verifier hold across diverse inputs and optimization pressure?

Achieving all three simultaneously is the central challenge.

## Four Reward Constructions

1. **Test Verifier** (§2): execution-based test suites + agentic quality judge + behavior monitoring. Hacked resolved rate: 28.57% → 0.56%; Clean resolved: 40.22% → 60.53%.
2. **Interactive Judge** (§3): rubric-based static judge → agentic interactive judge with Playwright. Resists length exploitation.
3. **User Feedback Verifier** (§4): extract HIRS from interaction data → Span-KTO training. +13.3pp on Aone-bench.
4. **Automated Agent Verifier** (§5): autonomous evaluator for long-horizon tasks. Evaluator quality is metric-dependent.

## Core Claim

No fixed reward function can remain effective as policy capability continues to grow; verification must co-evolve with the generator.

Source: https://arxiv.org/abs/2606.26300