Unlocking Complex Reasoning with Natural Language: Three Frameworks for Neurosymbolic AI

Sunday - May 5 2024, 09:52 UTC - 1 year ago

Three neurosymbolic frameworks developed by MIT researchers use natural language as a source of context to build better abstractions for programming, AI planning, and robotic tasks. One system, LILO, combines a large language model with an algorithm to identify and document code abstractions, showing promising results and potential for more complex applications.

Large language models (LLMs) are becoming increasingly useful for programming and robotics tasks, but for more complicated reasoning problems, the gap between these systems and humans looms large. Without the ability to learn new concepts like humans do, these systems fail to form good abstractions—essentially, high-level representations of complex concepts that skip less-important details—and thus sputter when asked to do more sophisticated tasks.

Luckily, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers have found a treasure trove of abstractions within natural language. In three papers to be presented at the International Conference on Learning Representations this year, the group shows how our everyday words are a rich source of context for language models, helping them build better overarching representations for code synthesis, AI planning, and robotic navigation and manipulation. All three papers are also available on the arXiv preprint server.

The three separate frameworks build libraries of abstractions for their given task: LILO (library induction from language observations) can synthesize, compress, and document code; Ada (action domain acquisition) explores sequential decision-making for artificial intelligence agents; and LGA (language-guided abstraction) helps robots better understand their environments to develop more feasible plans. Each system is a neurosymbolic method, a type of AI that blends human-like neural networks and program-like logical components.

LILO: A neurosymbolic framework that codes .

Large language models can be used to quickly write solutions to small-scale coding tasks, but cannot yet architect entire software libraries like the ones written by human software engineers. To take their software development capabilities further, AI models need to refactor (cut down and combine) code into libraries of succinct, readable, and reusable programs.

Refactoring tools like the previously developed MIT-led Stitch algorithm can automatically identify abstractions, so, in a nod to the Disney movie "Lilo & Stitch," CSAIL researchers combined these algorithmic refactoring approaches with LLMs. Their neurosymbolic method LILO uses a standard LLM to write code, then pairs it with Stitch to find abstractions that are comprehensively documented in a library.

LILO's unique emphasis on natural language allows the system to do tasks that require human-like common sense knowledge, such as identifying and removing all vowels from a string of code and drawing a snowflake. In both cases, the CSAIL system outperformed standalone LLMs, as well as a previous library learning algorithm from MIT called DreamCoder, indicating its ability to build a deeper understanding of the words within prompts.

These encouraging results point to how LILO could assist with things like writing programs to manipulate documents like Excel spreadsheets, helping AI answer questions about visuals, and drawing 2D graphics.

"Language models prefer to work with functions that are named in natural language," says Gabe Grand, an MIT Ph.D. student in electrical engineering and computer science, CSAI member, and author on a paper for LILO. "We propose an algorithm that can mine functions and concepts directly from natural language text in seconds, supported by a library of natural language primitives that provide different ways to fill in the execution details." .