20260601
This commit is contained in:
42
concepts/primitive-completeness.md
Normal file
42
concepts/primitive-completeness.md
Normal file
@@ -0,0 +1,42 @@
|
||||
---
|
||||
title: "Primitive Completeness (原语完备性)"
|
||||
created: 2026-05-26
|
||||
type: concept
|
||||
tags: ["bayesian-inference", "architecture", "transformers"]
|
||||
sources: ["agarwal-bayesian-attention-geometry"]
|
||||
---
|
||||
|
||||
# Primitive Completeness
|
||||
|
||||
> Transformer 在推理任务中的主导地位不是来自规模效应,而是架构的**原语完备性**——它是实现全部三种推理原语的最小架构。
|
||||
|
||||
## 定义
|
||||
|
||||
一个架构是**原语完备的**,当且仅当它能同时实现 [[belief-accumulation]]、[[belief-transport]] 和 [[random-access-binding]]。
|
||||
|
||||
## 完备性矩阵
|
||||
|
||||
| 架构 | 完备性 | 推理覆盖 |
|
||||
|------|:---:|---------|
|
||||
| Transformer | ✅ 完备 | 全部推理任务 |
|
||||
| Mamba | ❌ 缺失绑定 | 滤波优秀,联想回忆失败 |
|
||||
| LSTM | ❌ 缺失传输+绑定 | 仅静态推理 |
|
||||
| MLP | ❌ 全部缺失 | 无推理能力 |
|
||||
|
||||
## 核心论点
|
||||
|
||||
> The dominance of transformers in reasoning tasks arises not from scale alone, but from primitive completeness: they are the minimal architecture realizing the full set of inference primitives.
|
||||
|
||||
这是一个结构性论据:**规模不是原因,架构才是**。原语完备性解释了为什么更大的 LSTM 或 Mamba 仍无法弥合与 Transformer 在复杂推理上的差距——缺失的原语不能通过更多参数弥补。
|
||||
|
||||
## 设计启示
|
||||
|
||||
- 如需全部推理能力 → 需要注意力(或等价的随机访问机制)
|
||||
- 如任务仅需滤波 → Mamba SSM 可能更高效
|
||||
- LSTM 适合静态信念更新但无动态推理
|
||||
|
||||
## 相关页面
|
||||
|
||||
- [[inference-primitives]] — 三个原语的详细定义
|
||||
- [[bayesian-wind-tunnels]] — 验证原语完备性的实验方法
|
||||
- [[bayesian-attention-geometry]] — 原语在注意力头中的几何实现
|
||||
Reference in New Issue
Block a user