54 lines
1.7 KiB
Markdown
54 lines
1.7 KiB
Markdown
---
|
||
title: "Rubric-Driven Evaluation"
|
||
created: 2026-06-27
|
||
updated: 2026-06-27
|
||
type: concept
|
||
tags:
|
||
- rubric
|
||
- evaluation
|
||
- benchmark
|
||
- llm
|
||
sources:
|
||
- "rubrics-survey-2026"
|
||
---
|
||
|
||
# Rubric-Driven Evaluation
|
||
|
||
## 定义
|
||
|
||
使用结构化的 rubric 作为评估框架，对 LLM 在各类任务上的表现进行**多维度、可解释**的定量评估。
|
||
|
||
## 通用任务评估 (General-Task)
|
||
|
||
### Reasoning Capability
|
||
- MathCheck, SedarEval, RefGrader, RubricCode, TRACE, ProfBench, MoReBench
|
||
- 重点：数学正确性、推理步骤有效性、逻辑一致性
|
||
|
||
### Deep Research & Open-ended Generation
|
||
- HelloBench, WritingBench, DeepResearchBench, DeepResearchBenchII, DEER, ResearchRubrics, PencilsDown
|
||
- 重点：证据支撑、主题覆盖度、事实一致性、结构清晰度
|
||
|
||
### General Agent Capability
|
||
- AgentBoard, AdaRubric, TRAJECT-Bench, MCP-Universe, MultiChallenge, Scribe, AstraBench, PaperBench
|
||
- 重点：子目标完成、工具选择正确性、轨迹级评估
|
||
|
||
### Alignment Evaluation
|
||
- FLASK, G-Eval, InFoBench, AdvancedIF, WildBench, JudgeBench, StrongREJECT
|
||
- 重点：指令遵循、安全合规、偏好对齐
|
||
|
||
## 领域特定任务评估 (Domain-Specific)
|
||
|
||
### Intermediate Trajectories（中间轨迹）
|
||
- PaperBench, MoReBench, ProfBench, RubricCode, Scribe, MedMT-Bench
|
||
- 评估模型的推理/规划**过程**而非仅最终输出
|
||
|
||
### Final Outputs（最终输出）
|
||
- HealthBench, DeepResearchBenchII, PRBench, TechImage-Bench, LongShotBench, PresentBench, SpeechL2, LLM-RUBRIC
|
||
- 涵盖医疗 QA、多模态生成、代码生成、视频理解等
|
||
|
||
## 参考
|
||
|
||
- [[rubrics-for-llms|Rubrics for LLMs]]
|
||
- [[rubrics-survey-2026|Rubrics Survey (2026)]]
|
||
- [[rubric-aggregation]]
|