51 lines
1.7 KiB
Markdown
51 lines
1.7 KiB
Markdown
---
|
||
title: "奇异学习理论 (Singular Learning Theory)"
|
||
created: 2026-06-10
|
||
updated: 2026-06-10
|
||
type: concept
|
||
tags: ["bayesian-statistics", "algebraic-geometry", "model-selection", "generalization"]
|
||
sources: ["[[dead-directions-geometric-singular-learning]]"]
|
||
---
|
||
|
||
# 奇异学习理论 (Singular Learning Theory, SLT)
|
||
|
||
**Singular Learning Theory**（Watanabe, 2009）是处理**不可识别模型**（参数到分布的映射非单射）的贝叶斯渐近理论。
|
||
|
||
## 与传统统计理论的区别
|
||
|
||
| 传统理论 | SLT |
|
||
|---------|-----|
|
||
| 假设模型可识别 | 允许不可识别 |
|
||
| Fisher 信息矩阵满秩 | Fisher 矩阵退化 |
|
||
| BIC 等准则有效 | 需要修正准则 (WAIC, WBIC) |
|
||
| 正则渐近 | 奇异渐近 |
|
||
|
||
## 核心对象
|
||
|
||
### 奇异集 Sigma_T
|
||
```
|
||
Sigma_T = {theta : p_theta = p*}
|
||
```
|
||
所有完美拟合真实分布的参数集合——在过参数化网络中，这是一个连续流形而非孤立点。
|
||
|
||
### 广中平祐消解 (Hironaka Resolution)
|
||
通过坐标变换将奇异集"吹开"为简单交叉的乘积形式，在消解坐标中 KL 散度取法交形式。
|
||
|
||
### 实对数典范阈值 (RLCT)
|
||
[[real-log-canonical-threshold|lambda]] 是主导贝叶斯自由能渐近修正的不变量：
|
||
```
|
||
F_n = n*S_n + lambda*log n + ...
|
||
```
|
||
|
||
## SLT 与信息几何的鸿沟
|
||
|
||
- SLT 在"消解坐标"中工作（计算 lambda 需要做 blow-up）
|
||
- [[information-geometry|信息几何]]在"原始坐标"中工作（假设 Fisher 非退化）
|
||
- [[dead-direction|Dead Direction]] 桥接了两者
|
||
|
||
## 参考
|
||
- [[dead-directions-geometric-singular-learning|Dead Directions]]
|
||
- [[real-log-canonical-threshold|RLCT]]
|
||
- [[watanabe-triple|Watanabe's Triple]]
|
||
- [[information-geometry|Information Geometry]]
|