Tesla is Building a Foundation World Model for Their AI

Sunday - July 16 2023, 11:06 UTC - 1 year ago

Tesla is making an physical world “everything bagel” that helps them build a true AI FSD empire. This will be a base understanding of our physical work for driving, walking and interacting.

Dr Know it All points out that Ashok Elluswamy, Tesla’s Director of Autopilot software, said Tesla is building Foundational World Models for its AI. John Gibbs (Dr Know It All) believes this is Tesla converting the Full Self Driving software to have a large language model style view of the physical world. Instead of training to predict the next word, John believes Tesla is taking many short video clips of driving and blocking off a few seconds and having the system predict those few seconds.

Recently Phil Duan, Senior staff software engineer at Tesla, gave a talk at CVPR where he mostly rehashed what we already knew from AI Day 2 last year…. However toward the end of his presentation, he got into building Foundational World Models, and this is where things got interesting indeed. Tesla has provided some hints regarding the capabilities of the Foundation World Model including large language model style view of the physical world.

Tesla Occupancy Networks (TONs) can process data from cameras and LIDAR to detect objects and interactions. This technology seems to already being employed by Tesla for various use-cases. Tesla is also focusing efforts on creating FSM models for a wide variety of applications, from driving to walking and even interacting with people.

Tesla is also looking at creating what they refer to as an “Everything Bagel” which is an AI model that can do more traditional tasks that wouldn’t necessarily be associated with cars. Foundational World Models are designed to break past the current scope and limitations of AI FSD models and help create a more comprehensive vision of the physical world.

The possibilities are endless with this technology, and Tesla is one of the few companies who are actually foraying into this ambitious project. Tesla’s latest AI advancements may help them further distinguish their autonomous driving capabilities from other competitors in the space.