Unlocking the Potential of Language Models in Robotics: A Case Study with Prof. Angela Schoellig

Wednesday - May 1 2024, 14:38 UTC - 9 months ago

Prof. Angela Schoellig from TUM has successfully used language models to control swarms of drones, allowing for intuitive and efficient choreographies. The technology also addresses safety concerns with a built-in collision prevention filter. This has potential applications in entertainment, logistics, and search and rescue missions.

Language models have come a long way since their inception in the 2010s. These powerful AI systems are now being used for a variety of tasks, from generating text to answering questions. But could they also be used in robotics? Prof. Angela Schoellig from the Technical University of Munich (TUM) certainly thinks so. Her latest research focuses on using language models to control swarms of drones to perform choreographed movements.

Traditionally, controlling a swarm of drones has been a complex and time-consuming process. Engineers would have to manually program each drone with precise instructions for their movements. This approach is not only tedious but also limits the creative potential of drone performances. Prof. Schoellig believes that language models can change that.

By using large language models (LLMs) such as ChatGPT, Prof. Schoellig and her team were able to create a new way of controlling drones - through natural language commands. This means that instead of writing lines of code, users can simply give verbal or written instructions in their preferred language. The drones will then understand and carry out the commands, making the process more intuitive and efficient.

But that's not all. In addition to controlling the movements of the drones, Prof. Schoellig's team also incorporated a safety filter to prevent mid-air collisions. This is an important consideration when controlling swarms of drones, as even a small error in instructions could result in a catastrophic collision. With the help of language models, the drones are able to communicate with each other and make adjustments to their flight paths, avoiding potential collisions.

This groundbreaking research has countless potential applications, especially in the entertainment industry. Imagine a swarm of drones performing a synchronized light show to live music, all controlled by simple voice commands. The possibilities are endless.

But this technology also has practical uses. Prof. Schoellig envisions drones being used in logistics to transport goods in a more efficient and automated manner. They could also play a crucial role in search and rescue missions, where their ability to quickly and safely navigate through challenging terrains could save lives.

In conclusion, Prof. Angela Schoellig's work highlights the potential of combining language models with robotics. With further advancements and research, we could see a future where drones are controlled with ease and precision, opening up a whole new world of possibilities.