One of OpenAI's co-leads on its video generator, Sora, has left for Google

Tim Brooks, who was leading development on Sora with William Peebles, announced in a post on X that he's going to join Google DeepMind, Google's AI research division, to work on video generation technologies and "world simulators."

I had an amazing two years at OpenAI making Sora, Brooks wrote. "Thanks to all the passionate and kind people I worked with."

I'll be joining @GoogleDeepMind to work on video generation and world simulators! Can't wait to collaborate with such a talented team.

What a great two years at OpenAI building Sora! Thanks to all the passionate and kind people I worked with. Looking forward to the next chapter!

— Tim Brooks (@_tim_brooks) Oct 3, 2024
In response on X, Google DeepMind CEO Demis Hassabis welcomed Brooks, saying that he will help "make the long-standing dream of a world simulator a reality." The concept of world simulator is rather vague, and the term poorly defined; however, DeepMind has applied that to models like its recently released Genie, which can generate playable, action-controllable virtual worlds from synthesized images, real photos, and even sketches.

As DeepMind researchers described it in a 2023 paper: "Applications of a real-world simulator range from controllable content creation in games and movies to training embodied agents purely in simulation that can be directly deployed in the real world."

 Thrilled to welcome @_tim_brooks to @GoogleDeepMind. So excited to be working together to make the long-standing dream of a world simulator a reality!! https://t.co/dJrvXjMRBQ

 — Demis Hassabis (@demishassabis) October 3, 2024
Brooks was among the very first, and he helped launch the project at OpenAI in January of 2023. According to Brooks' LinkedIn profile, he claims to have led the research direction of the project and trained large models.

His departure comes at the same time when sources at The Information report that the unreleased Sora has technical issues that put it at a disadvantage relative to rival systems from Luma, Runway, and others. Sources made available by The Information said the original system presented last February had taken over 10 minutes of processing time before it could create a 1-minute video clip. OpenAI is training an enhanced version of Sora that could create clips in much less time, says the sources.
The company also has its own video generation model, Veo, which the tech giant unveiled this spring at its annual I/O developer conference and which will soon appear in YouTube Shorts, YouTube's short-form video format, to allow creators to generate backgrounds and six-second clips.

OpenAI, aside from the tech-specific hurdles, appears to have given up plenty of partnership ground to the video generation challengers over the last couple of months. Late last month, Runway announced that it would be partnering with Lionsgate, home of the "John Wick" franchise, to train a custom video model on Lionsgate's movie catalog. About a week later, Stability, which is developing its own family of video generation models, unveiled director James Cameron of "Avatar," "Terminator," and "Titanic" fame as an investor and member of its board of advisors.

The company reportedly met with filmmakers and studios in Hollywood earlier this year to demo Sora – ex-CTO Mira Murati attended Cannes – and has partnered with a number of independent directors and some brands to demonstrate the system's capabilities.

But no major collaboration has been announced.

Brooks — who, in a bizarre turn of events, is actually returning to Google after previously working on the Pixel smart phones for the company — is one of the high-profile resigantees from OpenAI.

In late September, CTO Mira Murati and chief research officer Bob McGrew and research VP Barret Zoph announced they would leave OpenAI. Research scientist Andrej Karpathy had resigned in February; months later, OpenAI co-founder and former chief scientist Ilya Sutskever quit, along with ex-safety leader Jan Leike. Co-founder John Schulman said in August he would leave OpenAI. And Greg Brockman, the company's president, is on sabbatical.