20260601

raw/papers/kore-knowledge-injection.md

@@ -0,0 +1,39 @@
+---
+title: "KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Controls"
+authors:
+  - Kailin Jiang
+  - Hongbo Jiang
+  - Ning Jiang
+  - Zhi Gao
+  - Jinhe Bi
+  - Yuchen Ren
+  - Bin Li
+  - Yuntao Du
+  - Lei Liu
+  - Qing Li
+date: 2026
+arxiv: "2510.19316"
+venue: "ICML 2026"
+domain: "Multimodal Learning, Knowledge Injection, Continual Learning"
+type: paper
+source: "https://arxiv.org/abs/2510.19316"
+---
+
+# KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Controls
+
+**Authors**: Kailin Jiang, Hongbo Jiang, Ning Jiang, Zhi Gao, Jinhe Bi, Yuchen Ren, Bin Li, Yuntao Du, Lei Liu, Qing Li
+
+**Venue**: ICML 2026
+
+**arXiv**: 2510.19316
+
+## Abstract
+
+KORE is a synergistic method centered around Knowledge-Oriented Controls for injecting new knowledge into LMMs while preserving old knowledge. It implements a two-stage optimization: (1) KORE-AUGMENTATION converts individual knowledge items into structured multi-round dialogues and instruction tasks, building a "knowledge tree" that enables internalization; (2) KORE-CONSTRAINT stores previous knowledge in the covariance matrix of linear layer activations and initializes a LoRA adapter by projecting original weights into the matrix's null space, defining a fine-tuning direction that minimally interferes with previous knowledge.
+
+## Key Contributions
+
+1. **KORE-AUGMENTATION**: Structured knowledge augmentation pipeline — multi-round dialogues (trunk) + instruction tasks (branches) = knowledge tree
+2. **KORE-CONSTRAINT**: Null space projection via covariance matrix SVD — freezes adapter A in null space, fine-tunes only B
+3. **HARS metric**: Harmonized Adaptation-Retention Score for unified evaluation
+4. **State-of-the-art**: Outperforms 9 baselines on EVOKE benchmark across LLaVA-v1.5 (7B/13B) and Qwen2.5-VL (7B)