22 lines
1.8 KiB
Markdown
22 lines
1.8 KiB
Markdown
# Video Generation Models are General-Purpose Vision Learners (GenCeption)
|
|
|
|
- **arXiv**: 2607.09024v1
|
|
- **Published**: 2026-07-10
|
|
- **Venue**: ECCV 2026
|
|
- **Categories**: cs.CV, cs.AI
|
|
- **Authors**: Letian Wang, Chuhan Zhang, Rishabh Kabra, Jasper Uijlings, Steven Waslander, Andrew Zisserman, Joao Carreira, Kaiming He, Misha Andriluka, Eduard Gabriel Bazavan, Andrei Zanfir, Cristian Sminchisescu
|
|
- **Affiliations**: Google DeepMind, University of Toronto, UCL, University of Oxford, MIT, Lund University
|
|
- **Project**: https://genception.github.io
|
|
|
|
## Abstract
|
|
|
|
The paper argues that large-scale text-to-video generation serves as a strong pre-training paradigm for computer vision — the visual equivalent of next-token prediction in NLP. GenCeption repurposes a pre-trained video generative diffusion backbone into a feed-forward perception model capable of diverse vision tasks steered by text instructions. It achieves SOTA across depth, surface normal, camera pose estimation, expression-referring segmentation, and 3D keypoint prediction, often matching or surpassing specialized models with 7x-500x less training data. Notably, a model trained exclusively on synthetic human videos generalizes to real-world footage and OOD categories (animals, robots).
|
|
|
|
## Key Contributions
|
|
|
|
1. **Video Generation as Universal Vision Pre-training**: satisfies three imperatives—spatio-temporal priors, vision-language alignment, and scalability
|
|
2. **Feed-Forward Repurposing**: iterative diffusion → single-step perception via conditioning on t=0 and negating velocity output
|
|
3. **Unified Task Representation**: all dense tasks in 3-channel RGB space; sparse tasks via learnable tokens
|
|
4. **Data Efficiency**: matches SOTA specialists with 7x-500x less training data
|
|
5. **Emergent Sim-to-Real Transfer**: synthetic-only training generalizes to real-world and OOD categories
|