The Evolution of Mixed Reality Gaming
In 2016, as the sun dipped below the horizon, players like you were out exploring neighbourhoods, keenly searching for elusive Pokémon such as Snorlax in the popular game Pokémon Go. This mixed-reality experience captivated hundreds of millions, enabling players to engage with charming characters while unwittingly contributing to a technological evolution. The environmental scans conducted by players would eventually aid robots in navigating obstacles and interacting with their surroundings.
Niantic, the company behind Pokémon Go, made a significant pivot in its focus after selling the game the previous year. The firm has turned its attention to a pressing topic in Silicon Valley: world models. Brian McClendon, Chief Technology Officer at Niantic Spatial, emphasises their commitment to reconstructing the real world using a wealth of data. Unlike large language models (LLMs) that rely on text, world models utilise visual data to create digital environments that simulate physical laws.
Understanding World Models
World models operate fundamentally differently than language models. As Oliver Cameron, CEO of Odyssey, an AI lab specialising in this area, explains, “With a language model, you provide a sentence, and it predicts the next word. With a world model, you input a visual state—a picture—and it predicts the next visual state.” These representations serve as digital training grounds for various applications.
According to McClendon, world models allow robots to train without needing to physically occupy their intended environments. By capturing images of the locations where robots are expected to operate, a 3D model can be generated. Subsequently, the software can execute millions of simulations to refine the robot’s performance based on the specific parameters of that space.
Applications in Robotics and Autonomous Vehicles
This simulated training extends beyond humanoid robots; autonomous vehicles are also benefitting from world models. In the absence of human drivers, the software’s ability to react in fractions of a second is crucial. Jamie Shotton, Chief Scientist at Wayve, underscores the necessity for these models to understand physics, cause and effect, and how various objects and people respond to our actions.
Wayve’s generative world model is powered by images from dashcams, enabling the startup to recreate hazardous road conditions that would be impractical to test in real life.
The Broader Implications for Gaming and AI Development
Beyond practical applications, developers are also exploring how world models could reshape the gaming industry. Given the importance of physics and 3D rendering in video games, major corporations like Tencent are investing in their own world models. Anticipation is high for leading AI companies, including Google, to showcase their advancements in this field.
Significant financial backing is flowing into emerging tech names focusing on world models. Yann LeCun, former Chief AI Scientist at Meta, has raised over a billion dollars for his startup, Advanced Machines Intelligence. Additionally, renowned Stanford computer scientist Fei-Fei Li established World Labs in 2024, which AMD acquired this year for over $8 billion.
The Unseen Impact on Everyday Life
Even if you are determined to avoid autonomous vehicles and household robots, it is important to recognise that world models are already reshaping the job market in subtle ways. To create accurate simulations of our physical reality, AI labs require real people to record it. Earlier this year, I strapped an iPhone to my forehead and filmed hours of my daily activities, from folding laundry to washing dishes, selling the footage through an app for a modest fee. As more companies aim to enhance their world models, the demand for such data collection jobs may increase.
Reflecting on my journey from chasing virtual monsters in 2016 to monetising videos of mundane chores in 2026, it is evident how these systems are increasingly entwining with my physical existence. I can’t help but imagine a digital spectre of a humanoid robot trapped within a simulation of my bedroom, tirelessly folding laundry with metallic claws, improving with each iteration. I regret not tidying up first.