81 lines
3.5 KiB
Markdown
81 lines
3.5 KiB
Markdown
---
|
||
title: "The Rules of the Game: A Survey of Rubrics for Large Language Models"
|
||
created: 2026-06-27
|
||
updated: 2026-06-27
|
||
type: paper
|
||
source_url: "https://8421bcd.github.io/_pages/Rubrics_Survey.pdf"
|
||
github: "https://github.com/8421BCD/Rubrics_Survey"
|
||
authors:
|
||
- "Wenhan Liu"
|
||
- "Jiajie Jin"
|
||
- "Zhaoheng Huang"
|
||
- "Tongyu Wen"
|
||
- "Guanting Dong"
|
||
- "Ziliang Zhao"
|
||
- "Yutao Zhu"
|
||
- "Zhicheng Dou"
|
||
- "Ji-Rong Wen"
|
||
affiliation: "Renmin University of China"
|
||
date: "2026-05-22"
|
||
tags:
|
||
- rubric
|
||
- evaluation
|
||
- reward-modeling
|
||
- survey
|
||
- llm
|
||
---
|
||
|
||
# The Rules of the Game: A Survey of Rubrics for LLMs
|
||
|
||
## 核心问题
|
||
|
||
LLM 正从简单文本生成器进化为推理、决策、工具使用和长周期求解系统。当任务变得开放、高风险（深度研究、医疗诊断、agentic tool use），单一的 correctness 信号和 LLM judge 的偏好分已不足以评估——需要多维度标准。
|
||
|
||
**Rubrics 填补这个空缺**：将质量评估分解为显式的 [[rubrics-for-llms|结构化评分项目]]，逐项打分后聚合，同时提供透明、可控、可诊断的评估，并可转化为训练监督信号。
|
||
|
||
## 论文贡献
|
||
|
||
1. **首次全面综述** LLM 的 rubric-based 研究
|
||
2. **系统分类** rubric 构建方法为四大范式：[[rubric-construction|直接生成、对比生成、迭代精炼、在线协同演化]]
|
||
3. **全面回顾** rubric 在模型训练中的应用：[[rubric-based-reward-modeling|Policy model RL + Reward model training]]
|
||
4. **深度讨论** 开放挑战：[[reward-hacking|rubric reward hacking]]、泛化性、[[rubric-safety|rubric 安全]]、[[rubric-personalization|个性化 rubric]]、评估偏置
|
||
|
||
## 关键框架
|
||
|
||
### Rubric 形式化定义
|
||
R = {(dⱼ, wⱼ)}ᵏⱼ₌₁，逐项打分 cⱼ(x,y) ∈ [0,1]，[[rubric-aggregation|加权聚合]]为 S_R。
|
||
|
||
### 概念区分
|
||
Rubrics = **评估标准**（what） vs [[llm-as-a-judge|LLM-as-a-Judge]] = **评估者**（who） vs Reward Model = **输出分数方式**（how） vs [[rlvr-unified-framework|RLVR]] = **自动验证方式**。
|
||
|
||
### Rubric 构建四范式
|
||
|
||
| 范式 | 机制 | 代表作 |
|
||
|------|------|--------|
|
||
| Direct Generation | 从 query/answer 直接生成 | RaR, RLCF, CARMO |
|
||
| Contrastive Generation | 从偏好对提取区分标准 | OpenRubrics, CDRRM, MaMs |
|
||
| Iterative Refinement | 验证→分解→压缩循环 | RRD, RubricHub, CARO, OptimSyn |
|
||
| Online/Co-evolving | 训练中动态调整 | DR-Tulu, Rubric-ARM, OpenRS, SibylSense |
|
||
|
||
### Rubric 用于训练
|
||
|
||
- **Policy Model**: Standard RL / Advanced Reward Design / Rubrics as Policy Guidance
|
||
- **Reward Model**: Interpretability (R3, ArmoRM) / Reward Signals (METAJUDGE) / Data Construction (CROME)
|
||
|
||
### Rubric 用于评估
|
||
|
||
- **通用任务**: Reasoning, Deep Research, Agent, Alignment — [[rubric-driven-evaluation|多维度 benchmark]]
|
||
- **领域特定**: 医疗 QA, 多模态生成, 代码生成, 视频理解
|
||
|
||
## 开放问题
|
||
|
||
1. **Rubric Reward Hacking**: policy 学会利用 rubric 的盲点而非真正提升能力（参见 [[reward-hacking]]）
|
||
2. **泛化性**: rubric-based RM 跨任务/跨领域泛化弱
|
||
3. **评估偏置**: 措辞偏置、judge model 偏置、人类专家分歧
|
||
4. **个性化 Rubric**: 用户专属 vs 通用标准的张力（参见 [[rubric-personalization]]）
|
||
5. **Rubric 安全**: RIPD 攻击——rubric 可被操纵为攻击面（参见 [[rubric-safety]]）
|
||
|
||
## 来源
|
||
|
||
[原始存档](raw/papers/rubrics-survey-2026.md) | [GitHub 仓库](https://github.com/8421BCD/Rubrics_Survey) | [PDF](https://8421bcd.github.io/_pages/Rubrics_Survey.pdf)
|