68 lines
2.5 KiB
Markdown
68 lines
2.5 KiB
Markdown
---
|
||
title: "Uncertainty Estimation and Generalization Bounds for Modern Deep Learning"
|
||
created: 2026-06-17
|
||
updated: 2026-06-17
|
||
type: paper
|
||
tags: [bayesian-deep-learning, generalization, uncertainty, pac-bayesian, gaussian-process]
|
||
sources: [raw/papers/ortega-phd-thesis-2026.md]
|
||
confidence: high
|
||
---
|
||
|
||
# 现代深度学习中的不确定性估计与泛化界
|
||
|
||
> Luis A. Ortega Andrés — PhD Thesis, Autonomous University of Madrid, 2026
|
||
> Supervisor: Daniel Hernández-Lobato | arXiv: [2606.13818](https://arxiv.org/abs/2606.13818)
|
||
|
||
## 核心问题
|
||
|
||
神经网络预测性能强大，但**泛化能力与不确定性量化**仍理解不完整。本论文从方法论和理论两个角度，在统一的概率视角下连接 Bayesian 推断、函数空间建模和大偏差理论。
|
||
|
||
## 方法论贡献
|
||
|
||
### Deep Variational Implicit Process ([[deep-variational-implicit-process|DVIP]])
|
||
|
||
- 将[[implicit-processes|隐式过程]]扩展到深度架构的可扩展 Bayesian 框架
|
||
- 建模**易采样但无显式密度**的函数分布
|
||
- 在深度高斯过程 1/10 的计算代价下达到竞争性能
|
||
|
||
### 后处理方法
|
||
|
||
| 方法 | 全称 | 机制 |
|
||
|------|------|------|
|
||
| [[variational-linearized-laplace-approximation|VaLLA]] | Variational Linearized Laplace | 变分 + 线性化 Laplace 后验 |
|
||
| [[fixed-mean-gaussian-process|FMGP]] | Fixed-Mean Gaussian Process | 冻结均值 + GP 协方差校准 |
|
||
|
||
两者均为预训练确定性网络**附加校准的不确定性估计**，桥接确定性与 Bayesian 深度学习。
|
||
|
||
## 理论贡献
|
||
|
||
### 统一泛化框架
|
||
|
||
在 [[pac-bayesian-bounds|PAC-Bayesian]] 和大偏差理论下连接三个泛化机制：
|
||
|
||
1. **多样性（Diversity）**：集成成员的函数独立性降低泛化误差
|
||
2. **光滑性（Smoothness）**：损失景观曲率放大经验损失的集中率函数
|
||
3. **随机性（Stochasticity）**：SGD 噪声作为隐式正则化 → 偏向平坦极小值
|
||
|
||
### PAC-Chernoff 界
|
||
|
||
- 在**插值区间**仍有意义（传统界在此失效）
|
||
- 提供对 [[double-descent|双下降]] 的定量、分布依赖解释
|
||
|
||
## 论文结构
|
||
|
||
| 章节 | 内容 |
|
||
|------|------|
|
||
| Ch 2 | Bayesian 推断基础 + GP + 泛化界 |
|
||
| Ch 3 | DVIP: 可扩展隐式过程 Bayesian 推断 |
|
||
| Ch 4 | VaLLA + FMGP: 后验不确定性校准 |
|
||
| Ch 5 | PAC-Bayes + 大偏差泛化框架 |
|
||
| Ch 6 | SGD 隐式正则化的概率分析 |
|
||
|
||
## 参考
|
||
|
||
- [[bayesian-deep-learning|Bayesian 深度学习]]
|
||
- [[deep-gaussian-process|深度高斯过程]]
|
||
- [[generalization-bounds|泛化界]]
|
||
- 来源：[原始存档](raw/papers/ortega-phd-thesis-2026.md)
|