Generating Unique Recipes with AI Models

Saturday - May 13 2023, 22:04 UTC - 1 year ago

Two researchers at PeopleTec, a technology company based in Alabama, developed a computational model that can create complex recipes containing ingredients that users have available, after analyzing images of what is inside their fridge. They used image analysis and the GPT-4 language model when generating the recipes, and tested their approach with more than 2,000 images, developing a 100-page recipe book with novel recipes.

Artificial intelligence (AI) models have already made their way into a wide range of real-world settings, helping humans to tackle everyday problems faster and more efficiently. Recently, computer scientists have also been exploring their potential for assisting humans with creative tasks, such as coming up with interesting designs, poems and even recipes.Two researchers at PeopleTec, a technology company based in Alabama, developed a computational model that can create complex recipes containing ingredients that users have available, after analyzing images of what is inside their fridge.

Their approach, presented in a paper pre-published on arXiv, is based on models that can identify objects in images and GPT-4, the renowned large language model (LLM) developed by OpenAI."In 2020, a group of our AI researchers had a party game where one team would propose a list of ingredients and another would use a language model to dream up an original recipe," David Noever, one of the researchers who carried out the study, told Tech Xplore."As the ingredients got more interesting, the recipes got worse and worse, finally degenerating into nonsense like 'A Recipe for Hungarian Shoe Leather, serving 2 for breakfast'—basically, complete gibberish. But since 2020, both the image and language models have gotten so good, that now we could really build an application that would solve the practical cook's challenge—just look in your fridge right now, take some basic preconceived ideas about what one might want to cook today and generate a great novel recipe." .

A key objective of the recent work by Noever and his colleague Samantha Elizabeth Miller Noever was to highlight recent advances in the field of AI in a practical and useful way. To achieve recipe generation from imagery, they specifically used application programming interfaces (APIs) of models for image analysis as well as the text generator underpinning ChatGPT."The basic idea behind our work was to combine raw food and recipe ingredients using image analysis, then to ask a powerful language model to construct a plausible cooking recipe, including the expected title, proportion, and steps," Noever explained."One of the interesting twists of this language-image approach is to constrain the recipe style generator in different and often complex ways based on say minimizing the cost of the meal, changing the portion sizes, or accommodating dietary restrictions. The hard parts of that really fall on how good the language model is, which of course, has made tremendous breakthroughs in only the last few months." .

The researchers evaluated their computational approach in a series of tests, feeding it more than 2,000 images of open refrigerators with different ingredients inside them. Using these images, their model generated a 100-page recipe book, with interesting and unique recipes featuring the 30 top ingredients pictured in the input images.

"Several researchers had helped us by building pictures of the top 30 items to find in a refrigerator," Noever said. "We trained our own image models to give back a list of those in different settings. The novel part of our work was to continuously make the recipe fit to different constraints. Of course, there are billions of combinations available for even a small list of ingredient possibilities." .