title, created, updated, type, tags, sources
| title |
created |
updated |
type |
tags |
sources |
| Rubric-Driven Evaluation |
2026-06-27 |
2026-06-27 |
concept |
| rubric |
| evaluation |
| benchmark |
| llm |
|
|
Rubric-Driven Evaluation
定义
使用结构化的 rubric 作为评估框架，对 LLM 在各类任务上的表现进行多维度、可解释的定量评估。
通用任务评估 (General-Task)
Reasoning Capability
- MathCheck, SedarEval, RefGrader, RubricCode, TRACE, ProfBench, MoReBench
- 重点：数学正确性、推理步骤有效性、逻辑一致性
Deep Research & Open-ended Generation
- HelloBench, WritingBench, DeepResearchBench, DeepResearchBenchII, DEER, ResearchRubrics, PencilsDown
- 重点：证据支撑、主题覆盖度、事实一致性、结构清晰度
General Agent Capability
- AgentBoard, AdaRubric, TRAJECT-Bench, MCP-Universe, MultiChallenge, Scribe, AstraBench, PaperBench
- 重点：子目标完成、工具选择正确性、轨迹级评估
Alignment Evaluation
- FLASK, G-Eval, InFoBench, AdvancedIF, WildBench, JudgeBench, StrongREJECT
- 重点：指令遵循、安全合规、偏好对齐
领域特定任务评估 (Domain-Specific)
Intermediate Trajectories（中间轨迹）
- PaperBench, MoReBench, ProfBench, RubricCode, Scribe, MedMT-Bench
- 评估模型的推理/规划过程而非仅最终输出
Final Outputs（最终输出）
- HealthBench, DeepResearchBenchII, PRBench, TechImage-Bench, LongShotBench, PresentBench, SpeechL2, LLM-RUBRIC
- 涵盖医疗 QA、多模态生成、代码生成、视频理解等
参考
