31 lines
2.2 KiB
Markdown
31 lines
2.2 KiB
Markdown
---
|
|
title: "LLMs Corrupt Your Documents When You Delegate"
|
|
created: 2026-05-14
|
|
type: paper
|
|
source: https://arxiv.org/abs/2604.15597
|
|
authors: ["Philippe Laban", "Tobias Schnabel", "Jennifer Neville"]
|
|
---
|
|
|
|
# LLMs Corrupt Your Documents When You Delegate
|
|
|
|
- **arXiv ID**: 2604.15597
|
|
- **Authors**: Philippe Laban, Tobias Schnabel, Jennifer Neville (Microsoft Research)
|
|
- **Categories**: cs.CL (Computation and Language), cs.HC (Human-Computer Interaction)
|
|
- **Published**: 2026-04-17
|
|
- **Repository**: microsoft/DELEGATE52
|
|
- **Dataset**: datasets/microsoft/DELEGATE52
|
|
|
|
## Abstract
|
|
|
|
Large Language Models (LLMs) are poised to disrupt knowledge work, with the emergence of delegated work as a new interaction paradigm (e.g., vibe coding). Delegation requires trust — the expectation that the LLM will faithfully execute the task without introducing errors into documents. We introduce DELEGATE-52 to study the readiness of AI systems in delegated workflows. DELEGATE-52 simulates long delegated workflows that require in-depth document editing across 52 professional domains, such as coding, crystallography, and music notation. Our large-scale experiment with 19 LLMs reveals that current models degrade documents during delegation: even frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) corrupt an average of 25% of document content by the end of long workflows, with other models failing more severely. Additional experiments reveal that agentic tool use does not improve performance on DELEGATE-52, and that degradation severity is exacerbated by document size, length of interaction, or presence of distractor files. Our analysis shows that current LLMs are unreliable delegates: they introduce sparse but severe errors that silently corrupt documents, compounding over long interaction.
|
|
|
|
## Key Metrics
|
|
|
|
- 19 LLMs tested across 6 model families
|
|
- 310 work environments across 52 professional domains
|
|
- Frontier models average ~25% degradation after 20 interactions
|
|
- All-model average ~50% degradation after 20 interactions
|
|
- Python is the only domain where most models (17/19) achieve "ready" status
|
|
- Critical failures account for ~80% of total degradation
|
|
- Agentic tool use incurs 2-5x input token overhead
|