53 lines
1.6 KiB
Markdown
53 lines
1.6 KiB
Markdown
---
|
||
title: "死方向 (Dead Direction)"
|
||
created: 2026-06-10
|
||
updated: 2026-06-10
|
||
type: concept
|
||
tags: ["singular-learning-theory", "information-geometry", "fisher-metric", "deep-learning-theory"]
|
||
sources: ["[[dead-directions-geometric-singular-learning]]"]
|
||
---
|
||
|
||
# 死方向 (Dead Direction)
|
||
|
||
**Dead Direction** 是 [[dead-directions-geometric-singular-learning|Shirodkar (2026)]] 提出的桥接原语：Fisher 信息度量退化方向上的单位向量，连接 [[singular-learning-theory|奇异学习理论]]和[[information-geometry|信息几何]]。
|
||
|
||
## 两大解读
|
||
|
||
| 框架 | 解读 |
|
||
|------|------|
|
||
| Amari 信息几何 | Fisher 度量 F 失去非退化性的方向 |
|
||
| Watanabe 奇异学习理论 | 解析奇异集 Sigma_T 的切向量，具有确定的 KL 阶 k |
|
||
|
||
两者命名**同一向量**——这是桥接的关键。
|
||
|
||
## 形式化定义
|
||
|
||
沿路径 theta(t) → 奇异集（t → 0），方向 u 满足：
|
||
|
||
```
|
||
u^T F(theta(t)) u → 0 as t → 0
|
||
```
|
||
|
||
## 核心定理（Theorem 2）
|
||
|
||
```
|
||
u^T F(theta(t)) u = Theta(t^{2(k-1)})
|
||
```
|
||
|
||
其中 k 是 KL 阶。最小 Fisher 特征值的衰减斜率直接读出 k：
|
||
- k=1（正则）：斜率 0
|
||
- k=2：斜率 2
|
||
- k=3：斜率 4
|
||
|
||
## 为什么重要
|
||
|
||
1. **无需广中平祐消解**：KL 阶在原始参数坐标中可计算
|
||
2. **连接两大传统**：Amari 的退化方向 = Watanabe 的切向量
|
||
3. **实践可操作**：可从单个 checkpoint 的梯度信息中提取
|
||
|
||
## 参考
|
||
- [[dead-directions-geometric-singular-learning|Dead Directions]]
|
||
- [[kl-order|KL Order]]
|
||
- [[fisher-information-metric|Fisher Information Metric]]
|
||
- [[singular-learning-theory|Singular Learning Theory]]
|