AI Predictions for Autonomous Vehicles: Waabi's Generative Model Using Lidar

Friday - March 15 2024, 17:50 UTC - 8 months ago

Waabi has developed a new generative AI model, Copilot4D, that predicts the movements of surrounding vehicles using lidar data. This model is being used in their autonomous trucks in Texas and could potentially reduce the need for extensive road-testing. Unlike camera-based models, lidar is considered essential for achieving Level 4 automation. Additionally, Waabi's lidar model is not used in their simulator to avoid teaching any potential hallucinations.

Self-driving technology has been steadily advancing in recent years, with many companies and researchers pushing the boundaries of what is possible. One such company is Waabi, which has recently announced the development of a new generative AI model called Copilot4D. This model is trained on massive amounts of lidar data and aims to predict the movements of surrounding vehicles in order to assist with decision-making in autonomous driving systems.

While most autonomous driving companies have relied on traditional machine learning methods for route planning and object detection, some are now exploring the potential benefits of generative AI. Waabi's competitor, Wayve, released a similar model last year that is trained on video data collected by their vehicles.

Waabi's approach with Copilot4D is similar to other image and video generators, such as OpenAI's DALL-E and Sora. The model takes in point cloud data from lidar sensors, which creates a 3D map of the car's surroundings, and breaks it into smaller chunks. It then uses this data to predict how all points will move, allowing it to generate predictions 5-10 seconds into the future.

What sets Waabi apart from its competitors is its focus on lidar rather than cameras. According to Waabi's CEO Raquel Urtasun, lidar is essential for achieving Level 4 automation, where the car is able to operate without human intervention. While cameras can provide visual information, they are not as reliable in measuring distances and understanding the geometry of the car's surroundings.

However, Waabi's lidar model is not used for training in their simulator. This is to ensure that any potential hallucinations generated by the model do not get mistakenly taught in the simulator. This safety precaution showcases the company's commitment to responsible and ethical development of autonomous driving technology.

While generative AI models are not new, Waabi's use of a generative lidar model on a commercial scale is groundbreaking. According to PhD student Bernard Adam Lange, who has experience with similar models, this is the first time he has seen a generative lidar model being used outside of a research lab.

Overall, Waabi's development of Copilot4D is an exciting step towards the future of autonomous driving. With the potential to reduce hours of road-testing and improve decision-making, generative AI models could play a crucial role in advancing the technology.