20260706:新增一些文章
This commit is contained in:
39
concepts/interactive-judge.md
Normal file
39
concepts/interactive-judge.md
Normal file
@@ -0,0 +1,39 @@
|
||||
---
|
||||
title: "交互式判断器(Interactive Judge)"
|
||||
created: 2026-07-02
|
||||
updated: 2026-07-02
|
||||
type: concept
|
||||
tags: [verification, frontend, judge, playwright, interaction]
|
||||
sources:
|
||||
- "[[verification-horizon-no-silver-bullet]]"
|
||||
---
|
||||
|
||||
# 交互式判断器(Interactive Judge)
|
||||
|
||||
用于前端任务的 Agent 判断器,通过模拟真实用户交互来评估生成页面,抵抗静态判断器的长度利用。
|
||||
|
||||
## 三阶段 Pipeline
|
||||
|
||||
1. **Action Planning**(单次前向传递):Action Planner 从任务规范和页面信息(accessibility tree、browser state、keyboard listeners)一次性生成完整 action list
|
||||
2. **Render Execution**:Playwright server 在真实浏览器中执行 action list,录制交互轨迹(screenshots、DOM changes、console output)
|
||||
3. **Scoring**:Judge model 根据 rubric checklist 评估交互轨迹 + 源码
|
||||
|
||||
## 关键优势
|
||||
|
||||
- **抵抗长度利用**:奖励来自运行时行为,而非源码长度 → 模型无法通过生成冗余 CSS/JS 骗分
|
||||
- **覆盖动态行为**:多页导航、动画、状态转换、表单验证 → 静态截图无法捕获
|
||||
- **效率**:单 pass 生成 action list(vs 迭代 agent loop),降低推理成本
|
||||
|
||||
## 对比静态判断器
|
||||
|
||||
| 特征 | Static Judge | Interactive Judge |
|
||||
|------|:---:|:---:|
|
||||
| 评估依据 | 源码 + 静态截图 | 运行时交互行为 |
|
||||
| 长度利用 | 易被利用 | 抵抗 |
|
||||
| 动态行为 | 无法覆盖 | 可覆盖 |
|
||||
| 成本 | 低 | 中 |
|
||||
|
||||
## 参<><E58F82>
|
||||
|
||||
- [[verification-horizon-no-silver-bullet|论文原文]]
|
||||
- [[rubric-based-evaluation|量规评估]]
|
||||
Reference in New Issue
Block a user