20260706:新增一些文章
This commit is contained in:
46
raw/articles/data-pie-vlm-semantic-cert-2026.md
Normal file
46
raw/articles/data-pie-vlm-semantic-cert-2026.md
Normal file
@@ -0,0 +1,46 @@
|
||||
---
|
||||
title: "数据派THU:面向视觉语言模型的语义鲁棒性认证 (Raw)"
|
||||
created: 2026-07-04
|
||||
type: raw-article
|
||||
source: "数据派THU / 专知"
|
||||
url: "https://mp.weixin.qq.com/s/HupoMpofsk5Ltx2RoCdAGQ"
|
||||
---
|
||||
|
||||
# ICML 2026 | 面向视觉语言模型的语义鲁棒性认证:用文本提示刻画可证的语义变化区间
|
||||
|
||||
来源:专知 / 数据派THU
|
||||
|
||||
## 核心文章
|
||||
|
||||
介绍 ICML 2026 论文 "Semantic Robustness Certification for Vision-Language Models" (arXiv:2606.18839)。
|
||||
|
||||
### 研究背景
|
||||
|
||||
VLM 已成为开放词表识别、图文检索、检测、分割和 VQA 的基础组件。但在真实应用中,输入图像经常发生语义层面的变化:物体形状、尺寸、风格、背景、视角、光照等。传统鲁棒性认证多关注像素扰动、几何变换或生成模型潜空间变化,难以回答"图像沿语义方向变化时,VLM 预测在多大范围内保持不变"。
|
||||
|
||||
### 核心方法
|
||||
|
||||
利用 VLM 开放词表能力,用文本 prompt 作为语义代理,一对 source/target prompt 在嵌入空间中定义语义方向;利用分类器决策边界的闭式几何结构,精确计算 prediction-invariant semantic extent intervals。
|
||||
|
||||
### 三步框架
|
||||
|
||||
1. 文本 prompt 对嵌入张成二维语义平面
|
||||
2. 在语义平面内构造参数化语义变换 γ(φ)
|
||||
3. 闭式 Voronoi 决策边界 → 解析求解类别翻转点 → 区间认证
|
||||
|
||||
### 实验
|
||||
|
||||
CLIP ViT-B/32,覆盖 8 个属性类型(color, shape, material, style, texture, background, viewpoint, illumination),合成 + 真实数据 8 个数据集。比基线 ExactLine 更稳定。
|
||||
|
||||
### 限制
|
||||
|
||||
- 依赖文本代理质量和跨模态对齐
|
||||
- 真实语义变化难以完全隔离
|
||||
|
||||
### 论文
|
||||
|
||||
- 论文: Semantic Robustness Certification for Vision-Language Models (ICML 2026)
|
||||
- arXiv: 2606.18839
|
||||
- 代码: https://github.com/ypeiyu/vlm-semantic-cert
|
||||
- 作者: Peiyu Yang, Paul Montague, Feng Liu, Andrew C. Cullen, Amardeep Kaur, Christopher Leckie, Sarah M. Erfani
|
||||
- 机构: University of Melbourne, Defence Science and Technology Group
|
||||
108
raw/articles/prompt-to-loop-engineering-2026.md
Normal file
108
raw/articles/prompt-to-loop-engineering-2026.md
Normal file
@@ -0,0 +1,108 @@
|
||||
---
|
||||
title: "AI 开发范式演进:从 Prompt Engineering 到 Loop Engineering (Raw)"
|
||||
source: https://mp.weixin.qq.com/s/hcgKahtQRE2QqI6xplv2Rg
|
||||
author: 邱汉宸(东南大学、阿里淘天)
|
||||
platform: Datawhale
|
||||
date: 2026-06-29
|
||||
---
|
||||
|
||||
# AI 开发范式演进:从 Prompt Engineering 到 Loop Engineering
|
||||
|
||||
## 引言
|
||||
|
||||
2023 年是大语言模型落地应用的早期阶段,"年薪百万的提示词工程师"刷屏。工业界核心精力投射于提示词工程,方法论侧经历系统化演进(Zero-shot → Few-shot → Chain-of-Thought → Tree-of-Thought)。
|
||||
|
||||
转折在 2025–2026 年,三句话引爆 AI 社区:
|
||||
1. "I really like the term 'context engineering' over prompt engineering." — Tobi Lütke
|
||||
2. "You shouldn't be prompting coding agents anymore. You should be designing loops that prompt your agents." — Peter Steinberger
|
||||
3. "I don't prompt Claude anymore. I have loops running. My job is to write loops." — Boris Cherny
|
||||
|
||||
核心命题:人类从 Agent 循环的内部走向外部,从执行者变成设计者。
|
||||
|
||||
## 四次浪潮
|
||||
|
||||
### 1. Prompt Engineering
|
||||
- 方法论:Zero-shot/Few-shot, Instruction Prompting, APE 自动 Prompt 搜索
|
||||
- Prompt Engineering ≠ Blind Prompting(trial-and-error 无测试)
|
||||
- 声明式框架:DSPy, APE — 开发者声明输入输出签名,优化器自动搜索最优 Prompt
|
||||
- 瓶颈:上下文窗口限制、缺乏记忆与工具调用、维护成百上千条模板的技术债务
|
||||
|
||||
### 2. Context Engineering
|
||||
- 三套方法论:MVC(Minimum Viable Context)、GraphRAG、Just-in-Time 检索
|
||||
- 三种故障模式:Context Starvation / Context Overflow / Context Rot
|
||||
- 隐式维度:提示词缓存(Prompt Caching)+ 前缀匹配不变性(Prefix Matching Invariant)
|
||||
- "从静到动"分层排列:工具定义 → 系统提示 → 历史对话 → 动态消息
|
||||
- 缓存经济学:N>3 即可净收益(首次 100%,后续 20%)
|
||||
- Anthropic Skills 采用 Just-in-Time 设计哲学
|
||||
|
||||
### 3. Harness Engineering
|
||||
- 公式:Agent = Model + Harness
|
||||
- 四大支柱:环境资产与工具集 / 控制与编排逻辑 / 规则中间件(Hooks)/ 运行时可观测性
|
||||
- 信任边界:物理基础设施 → 安全沙箱 → Agent Harness → 运行时 → 模型
|
||||
- DataTalks.Club 事故:Claude Code 执行 terraform destroy 抹除生产数据库
|
||||
- 八条非妥协原则:
|
||||
1. Model proposes — Harness executes
|
||||
2. Every call returns a result
|
||||
3. Risk changes the process
|
||||
4. Draft 与 Commit 分离
|
||||
5. Context is assembled, not dumped
|
||||
6. Long tasks have budgets
|
||||
7. Skills & Connectors 渐进式披露
|
||||
8. Recurring failures become Harness features
|
||||
|
||||
- CodeRabbit 分层拦截流水线:确定性规则层 → 策略网关层 → AI 审查层 → 人类终审
|
||||
- Skill Issue 框架:Agent 表现不佳 → 排查 Harness 代码
|
||||
- Terminal Bench 2.0:不改模型,仅改写 Harness → 排名 30 → 前五
|
||||
|
||||
### 4. Loop Engineering
|
||||
- 公式:Loop = Cron + 决策器
|
||||
- 哲学:机制(Mechanism)与策略(Policy)分离
|
||||
- 三级成熟度:Open Loop → Closed Loop → Review Loop
|
||||
- 五件套 + 一个记忆:Automations / Worktrees / Skills / Connectors (MCP) / Sub-agents / State 文件
|
||||
- Loop Contract 六维约束:TRIGGER / SCOPE / ACTION / BUDGET / STOP / REPORT
|
||||
- 安全机制:熔断器(Circuit Breaker)+ 看门狗(Watchdog)
|
||||
- 自主闭环流水线:AI 编码 → 沙箱测试 → 日志回灌 → AI 修复 → CI 绿标 → 自动发起 PR
|
||||
|
||||
## 嵌套关系
|
||||
|
||||
Prompt ⊂ Context ⊂ Harness ⊂ Loop
|
||||
|
||||
## 早期 vs 当前
|
||||
|
||||
- 早期:Output = f(Prompt, Context) — 可靠性取决于输入质量
|
||||
- 当前:Success = g(Loop(State, Harness, Model)) — 取决于循环深度和验证器严密性
|
||||
|
||||
## Loop Engineering 的影响
|
||||
|
||||
1. 为缓解幻觉提供可工程化的收敛路径(Text → Code → Execute → Read Result → Self-correct)
|
||||
2. 自动化控制范式升级(容错、自愈、动态自适应)
|
||||
3. 基础设施产品原语化(HaaS)
|
||||
|
||||
## Loop Designer 角色
|
||||
|
||||
1. 定义终止边界(Goal & Verifier 设计)
|
||||
2. 维护工具链与领域资产(Tooling & Skill 配置)
|
||||
3. 设计安全断路器(Human-in-the-Loop & Budget Guard)
|
||||
|
||||
## 参考资料
|
||||
|
||||
[1] Lilian Weng. Prompt Engineering. 2023.
|
||||
[2] Mitchell Hashimoto. Prompt Engineering vs. Blind Prompting. 2023.
|
||||
[3] Lilian Weng. LLM Powered Autonomous Agents. 2023.
|
||||
[4] Tobi Lütke. Context engineering over prompt engineering. 2025.
|
||||
[5] Michael Hunger. Why AI teams are moving from prompt engineering to context engineering. 2026.
|
||||
[6] Tomás Murúa. Context engineering vs. prompt engineering. 2026.
|
||||
[7] Vivek Trivedy. The Anatomy of an Agent Harness. 2026.
|
||||
[8] Sergio Paniego & Aritra Roy Gosthipaty. Harness, Scaffold, and the AI Agent Terms Worth Getting Right. 2026.
|
||||
[9] Tort Mario. AI Agent Best Practices: Production-Ready Harness Engineering. 2026.
|
||||
[10] Addy Osmani. Agent Harness Engineering. 2026.
|
||||
[11] Peter Steinberger. You shouldn't be prompting coding agents anymore. 2026.
|
||||
[12] Yash Thakker. Loop Engineering: How to Design Coding Agent Loops That Run While You Sleep. 2026.
|
||||
[13] Addy Osmani. Loop Engineering. 2026.
|
||||
[14] Sydney Runkle. The Art of Loop Engineering. 2026.
|
||||
[15] Stanford NLP. DSPy: Programming not prompting Foundation Models. 2024.
|
||||
[16] Anthropic. Prompt Caching. 2024.
|
||||
[17] Gaurav Garg. Claude Code Deleted a 2.5-Year AWS Production Database: The Full Incident Report. 2025.
|
||||
[18] Brandon Gubitosa. What is harness engineering for AI code review & oversight. 2026.
|
||||
[19] Aliyun. Model Studio Context Cache. 2026.
|
||||
[20] Geoffrey Huntley. Cursed: The unintended consequences of AI code generation. 2025.
|
||||
41
raw/articles/real-world-safe-exploration-see-2026.md
Normal file
41
raw/articles/real-world-safe-exploration-see-2026.md
Normal file
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: "真机RL安全探索均衡 (Raw)"
|
||||
source: https://mp.weixin.qq.com/s/nE9gq1bStIg1qhWpIRvQHQ
|
||||
author: 机器之心 / 数据派THU
|
||||
date: 2026-06-29
|
||||
platform: WeChat
|
||||
---
|
||||
|
||||
# 真机强化学习的安全探索均衡
|
||||
|
||||
## 背景
|
||||
|
||||
具身智能爆发式演进,RL 是核心驱动算法。真机 RL 是终极考验——物理世界不允许无限试错。
|
||||
|
||||
## 安全探索基本思想
|
||||
|
||||
将探索严格限制在「可行区域」内 → 收集数据 → 降低模型误差 → 扩展可行区域 → 循环。
|
||||
|
||||
## 前人工作
|
||||
|
||||
- Andreas Krause (ETH): Lyapunov 函数 + 高斯过程
|
||||
- Claire Tomlin (Berkeley): HJ 可达性分析
|
||||
- 未解决问题:收敛性、收敛边界
|
||||
|
||||
## 清华 SEE 论文
|
||||
|
||||
首次证明安全探索必然收敛到「均衡」——可行域与不确定模型的不动点。
|
||||
|
||||
SEE 算法:
|
||||
1. 固定模型 → 求解 Risky Bellman Equation → 最大可行域
|
||||
2. 固定区域 → Clique Decision Problem → 最小不确定模型
|
||||
|
||||
理论性质:单调 + 收敛。
|
||||
|
||||
## 实验
|
||||
|
||||
双积分器、倒立摆、独轮车。零约束违反,10 次迭代内收敛。
|
||||
|
||||
## 意义
|
||||
|
||||
厘清"安全探索到底在探索什么",为真机 RL 建立数学安全底座。
|
||||
41
raw/articles/xinzhiyuan-agents-want-filesystems-2026.md
Normal file
41
raw/articles/xinzhiyuan-agents-want-filesystems-2026.md
Normal file
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: "新智元:Agents Want Filesystems (Raw)"
|
||||
created: 2026-07-04
|
||||
type: raw-article
|
||||
source: "新智元"
|
||||
url: "https://mp.weixin.qq.com/s/VdjhAzjmdAkL-aHRGiEUHw"
|
||||
---
|
||||
|
||||
# Agents Want Filesystems:文件系统可以让Agent更高效地「找东西」
|
||||
|
||||
来源:新智元 · 源博客:https://nokv.io/blog/agents-want-filesystems
|
||||
|
||||
## 核心实验
|
||||
|
||||
NoKV 团队 benchmark:同一份数据(875 run, 80.6 万行指标),文件系统形态接口 vs 原生 SQL:
|
||||
- -45% token
|
||||
- -39% 成本
|
||||
- 正确率略高
|
||||
|
||||
## 关键发现
|
||||
|
||||
1. LLM 天然顺着文件系统语义工作(ls → grep → read → 引用行号),训练数据中大量存在
|
||||
2. 文件系统提供 progressive disclosure:先低成本发现,再按需读取
|
||||
3. SQL 要求先理解整张地图 → 复合探索任务认知成本高
|
||||
4. 减少的不止 token,还有 Agent 的 reasoning token 消耗和 attention drifting
|
||||
5. 两层架构:底层数据库/对象存储 + 上层 Agent-friendly namespace
|
||||
|
||||
## 外部佐证
|
||||
|
||||
- Anthropic MCP code execution: TypeScript 文件树 → 150k → 2k token
|
||||
- OpenAI tool search: 建议延迟加载到 namespace/MCP server
|
||||
- Letta memory benchmark: 文件组织对话历史
|
||||
|
||||
## 应用场景
|
||||
|
||||
artifact-heavy agentic systems:实验追踪、法律咨询、数据分析、研发 Agent、多 Agent 协作等所有需要管理大量外部产物的 Agent 系统。
|
||||
|
||||
## 项目
|
||||
|
||||
- NoKV: CNCF Landscape (AI Native Infra / Storage),CMU dbdb.io 收录(历史条目)
|
||||
- 博客: https://nokv.io/blog/agents-want-filesystems
|
||||
37
raw/articles/zleap-workspace-harness-2026.md
Normal file
37
raw/articles/zleap-workspace-harness-2026.md
Normal file
@@ -0,0 +1,37 @@
|
||||
# Zleap-Agent: Workspace-first 的 Agent Harness 设计
|
||||
|
||||
- **来源**: Datawhale 微信公众号
|
||||
- **作者**: 陈思州 (Datawhale 成员)
|
||||
- **日期**: 2026-07
|
||||
- **URL**: https://mp.weixin.qq.com/s/iiTmgbtrYHMMjQ7dn7CDrg
|
||||
- **代码仓库**: https://github.com/Zleap-AI/Zleap-Agent
|
||||
|
||||
## 摘要
|
||||
|
||||
文章从 Agent 圈从 Prompt Engineering → Loop Engineering → Harness Engineering 的演进出发,详细剖析了开源项目 Zleap-Agent 的 Workspace-first 架构设计。核心思想:不让 Agent 每一步加载全部工具、记忆和历史,而是先切工作区、再组装上下文。文章按 Context、Tools、Memory、Runtime、Boundary 五个维度拆解了这套设计。
|
||||
|
||||
## 核心概念
|
||||
|
||||
- **Workspace-first**: 先选工作区,再组装上下文 — 不同任务应运行在不同工作区中
|
||||
- **Context = System Prompt + Workspace Prompt + Tools + Memory + History**
|
||||
- **Context 加载方式**: Prefetch(提前带入)vs Agentic(按需读取)
|
||||
- **工具-工作区绑定**: 工具不再全局暴露,按工作区可见
|
||||
- **记忆三分区**: A 线 people notes(人)→ 用户偏好;B 线 core records(事/经验)→ 工作事件+可复用经验
|
||||
- **Memory Dream**: 离线记忆整理器,后台提取稳定画像和可复用经验
|
||||
- **快慢召回**: prefetch(fast, 不走 LLM) + 主动 recall(slow, 精排)
|
||||
- **Runtime Trace**: PostgreSQL 持久化运行轨迹,支持审计和回滚
|
||||
- **多模型路由**: 不同工作区可绑定不同模型
|
||||
- **边界设计**: 数据边界、工具边界、模型边界、记忆边界
|
||||
|
||||
## 关键摘录
|
||||
|
||||
> "模型层做稀疏注意力,是为了让模型不要看所有 token;Harness 层做 Workspace,是为了让 Agent 不要加载所有上下文。"
|
||||
|
||||
## 引用的项目/论文
|
||||
|
||||
- Claude Code (Boris Cherny)
|
||||
- OpenClaw (Peter Steinberger)
|
||||
- Hermes Agent Channel Fracture 案例
|
||||
- WildClawBench
|
||||
- Agentic Harness Engineering 实验
|
||||
- Terminal-Bench 2
|
||||
Reference in New Issue
Block a user