From Words to Worlds: Spatial Intelligence is AI’s Next Frontier

Spatial intelligence is presented as the next stage beyond language models. The essay says current AI handles abstract knowledge well, but still lacks grounded understanding of space, physics, and action. World models are proposed as the way to build that capability and expand AI into real and virtual environments.

It also says this work matters because it can support creativity, robotics, scientific discovery, healthcare, and education while keeping AI aligned with human goals. The author frames the effort as a continuation of the long AI project begun with Turing.

Reading notes

#

• Spatial intelligence is described as the capability that links imagination, perception, and action.
• LLMs are said to be strong with words but still ungrounded in the physical world.
• The essay argues that perception and action form a core loop in the evolution of intelligence.
• Spatial intelligence is said to support everyday physical tasks, creative work, and simulated environments.
• Examples such as Eratosthenes, the Spinning Jenny, and Watson and Crick are used to show how spatial reasoning has shaped discovery.
• Current AI systems are said to remain weak at estimating distance, orientation, size, navigation, shortcuts, and basic physics.
• World models are defined through three capabilities: generative, multimodal, and interactive.
• World models should generate worlds with semantic, geometric, physical, and dynamic consistency.
• World models should accept inputs such as images, videos, depth maps, text, gestures, and actions.
• World models should output the next state of the world given actions or goals.
• The essay says building world models will require new objectives, large-scale visual data, synthetic data, and new architectures.
• The author mentions RTFM and Marble as examples of ongoing work at World Labs.
• The essay argues that spatial intelligence should augment human capability rather than replace people.
• Creative uses include filmmaking, games, architecture, design, and immersive experiences.
• Robotics uses include training, collaboration, caregiving, and different embodied forms.
• Scientific, healthcare, and educational applications are described as longer-term areas with broad impact.