42 lines
1.3 KiB
Markdown
42 lines
1.3 KiB
Markdown
---
|
||
title: "知识适应 (Knowledge Adaptation)"
|
||
created: 2026-05-21
|
||
type: concept
|
||
tags: ["continual-learning", "knowledge-injection"]
|
||
sources: ["[[when-large-multimodal-models-confront-evolving-knowledge]]"]
|
||
---
|
||
|
||
# 知识适应 (Knowledge Adaptation)
|
||
|
||
## 定义
|
||
|
||
知识适应是[[evolving-knowledge-injection|进化知识注入]]的首要目标，指 LMM 在接触新知识后，能在**未见过的评估问题**上准确泛化。
|
||
|
||
## 形式化
|
||
|
||
max E [ I(M*(i_q, x_q) = y_q) - I(M(i_q, x_q) = y_q) ]
|
||
|
||
即最大化注入后模型 M* 相对原始模型 M 在评估数据 D_Q 上的准确率增益。
|
||
|
||
## MMEVOKE 上的适应表现
|
||
|
||
| 方法 | LLaVA-v1.5 CEM | Qwen-VL-Chat CEM |
|
||
|------|---------------|-----------------|
|
||
| Vanilla（零样本） | 4.89% | 5.84% |
|
||
| Full-FT | 18.02% | 10.16% |
|
||
| LoRA | 15.23% | 6.95% |
|
||
| MM-RAG UniIR | 40.68% | 32.75% |
|
||
| Sufficient Context | 56.78% | 49.98% |
|
||
|
||
## 关键发现
|
||
|
||
1. **所有方法表现不佳**——即使最佳方法（Sufficient Context）也仅达 56.78%
|
||
2. **知识感知增强**可进一步提升适应能力
|
||
3. **知识适应 ≠ 数据记忆**——模型需要"内化"知识而非"背诵"数据
|
||
|
||
## 参见
|
||
|
||
- [[knowledge-retention|知识保留]]
|
||
- [[knowledge-aware-augmentation|知识感知增强]]
|
||
- [[sufficient-context-paradox|充分上下文悖论]]
|