47 lines
1.4 KiB
Markdown
47 lines
1.4 KiB
Markdown
---
|
||
title: "Watanabe 三元组 (Watanabe's Triple)"
|
||
created: 2026-06-10
|
||
updated: 2026-06-10
|
||
type: concept
|
||
tags: ["singular-learning-theory", "bayesian-statistics", "asymptotics"]
|
||
sources: ["[[dead-directions-geometric-singular-learning]]"]
|
||
---
|
||
|
||
# Watanabe 三元组 (lambda, m, nu)
|
||
|
||
**Watanabe 三元组**完整刻画了奇异统计模型的贝叶斯渐近性质：
|
||
|
||
- **lambda**（[[real-log-canonical-threshold|RLCT]]）：主导自由能的 log n 修正
|
||
- **m**（重数 multiplicity）：log log n 项的系数
|
||
- **nu**（奇异波动 singular fluctuation）：泛化误差的渐近修正
|
||
|
||
## 公式
|
||
|
||
贝叶斯自由能：
|
||
```
|
||
F_n = n·S_n + lambda·log n - (m-1)·log log n + O(1)
|
||
```
|
||
|
||
泛化误差：
|
||
```
|
||
G_n = S + lambda/n + nu/n + o(1/n)
|
||
```
|
||
|
||
## Shirodkar 的贡献
|
||
|
||
[[dead-directions-geometric-singular-learning|Shirodkar (2026)]] 的核心突破：
|
||
|
||
1. **nu 的通用性**：对一维 dead direction，nu 在 KL 阶中通用确定
|
||
2. **单 checkpoint 读取**：从一次前向+反向传播计算 lambda, m, nu
|
||
3. **无需后验采样**：传统 SLT 需要 MCMC 采样 → 现在仅需梯度信息
|
||
|
||
## 实践意义
|
||
|
||
直接从训练轨迹（梯度流）读取 (lambda, m, nu) → 实时监控模型的泛化性质——这在之前需要完整的贝叶斯后验分析。
|
||
|
||
## 参考
|
||
- [[dead-directions-geometric-singular-learning|Dead Directions]]
|
||
- [[singular-learning-theory|Singular Learning Theory]]
|
||
- [[real-log-canonical-threshold|RLCT]]
|
||
- [[kl-order|KL Order]]
|