---
title: "KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Controls"
authors:
  - Kailin Jiang
  - Hongbo Jiang
  - Ning Jiang
  - Zhi Gao
  - Jinhe Bi
  - Yuchen Ren
  - Bin Li
  - Yuntao Du
  - Lei Liu
  - Qing Li
date: 2026
arxiv: "2510.19316"
venue: "ICML 2026"
domain: "Multimodal Learning, Knowledge Injection, Continual Learning"
type: paper
source: "https://arxiv.org/abs/2510.19316"
---

# KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Controls

**Authors**: Kailin Jiang, Hongbo Jiang, Ning Jiang, Zhi Gao, Jinhe Bi, Yuchen Ren, Bin Li, Yuntao Du, Lei Liu, Qing Li

**Venue**: ICML 2026

**arXiv**: 2510.19316

## Abstract

KORE is a synergistic method centered around Knowledge-Oriented Controls for injecting new knowledge into LMMs while preserving old knowledge. It implements a two-stage optimization: (1) KORE-AUGMENTATION converts individual knowledge items into structured multi-round dialogues and instruction tasks, building a "knowledge tree" that enables internalization; (2) KORE-CONSTRAINT stores previous knowledge in the covariance matrix of linear layer activations and initializes a LoRA adapter by projecting original weights into the matrix's null space, defining a fine-tuning direction that minimally interferes with previous knowledge.

## Key Contributions

1. **KORE-AUGMENTATION**: Structured knowledge augmentation pipeline — multi-round dialogues (trunk) + instruction tasks (branches) = knowledge tree
2. **KORE-CONSTRAINT**: Null space projection via covariance matrix SVD — freezes adapter A in null space, fine-tunes only B
3. **HARS metric**: Harmonized Adaptation-Retention Score for unified evaluation
4. **State-of-the-art**: Outperforms 9 baselines on EVOKE benchmark across LLaVA-v1.5 (7B/13B) and Qwen2.5-VL (7B)