1.6 KiB
source_url, ingested, sha256
| source_url | ingested | sha256 |
|---|---|---|
| https://arxiv.org/abs/2505.05522 | 2026-05-15 | d2d4c033d6f7dd91d6af9eeefa3878c8cee14ac8c1e7acb762b5772c1cf8d004 |
Continuous Thought Machines
Authors: Luke Darlow, Ciaran Regan, Sebastian Risi, Jeffrey Seely, Llion Jones (Sakana AI, Tokyo; University of Tsukuba; IT University of Copenhagen)
arXiv: 2505.05522v4 | Category: cs.LG | Venue: NeurIPS 2025
Abstract
Biological brains demonstrate complex neural activity, where neural dynamics are critical to how brains process information. Most artificial neural networks ignore the complexity of individual neurons. We challenge that paradigm. By incorporating neuron-level processing and synchronization, we reintroduce neural timing as a foundational element. We present the Continuous Thought Machine (CTM), a model designed to leverage neural dynamics as its core representation. The CTM has two innovations: (1) neuron-level temporal processing, where each neuron uses unique weight parameters to process incoming histories; and (2) neural synchronization as a latent representation. The CTM aims to strike a balance between neuron abstractions and biological realism.
We demonstrate the CTM's performance across a range of tasks, including solving 2D mazes, ImageNet-1K classification, parity computation, and more. Beyond displaying rich internal representations and offering a natural avenue for interpretation, the CTM can perform tasks that require complex sequential reasoning. The CTM can also leverage adaptive compute, where it can stop earlier for simpler tasks, or keep computing when faced with more challenging instances.