The smart folks at MIT have come up with a new food-technology AI breakthrough that has some interesting applications when it comes to meal prep and curating recipes. Called Pic2recipe, the system identifies recipes based on images delivering a list of possible matches and their ingredients. While it is a long way from launch, Pic2recipe not only has value as a standalone app. It could also become a key component of guided cooking systems.
Pic2recipe uses computer vision, a technology long associated with video search. Using a dataset—in this case recipes from All Recipes and Food.com—images are annotated with information about the picture that could then be identified and correlated to the matching recipe. To accomplish this final step, a neural network is trained to find patterns and make connections between pictures and recipes. At this point, Pic2recipe has a database of more than one million recipes.
“In computer vision, food is mostly neglected because we don’t have the large-scale datasets needed to make predictions,” MIT’s Yusuf Aytar said in a recent interview. “But seemingly useless photos on social media can actually provide valuable insight into health habits and dietary preferences.”
A test of what looks to be a beta version of the Pic2recipe system was hit and miss. This enhanced search engine was unable to correctly match a salad and a vegetable stew to the correct recipe. On a third try, this a picture of homemade minestrone soup, Pic2recipe fared far better. Beef minestrone was fourth on the list with a 79% degree of confidence.
Moving forward, it’s easy to imagine Pic2recipe as an app that could feed into a guided recipe system. By showing the recipe ingredients, a user can make any modification necessary for a special diet. For example, the minestrone soup identified in the trial is actually vegan; the recipe shown in the search results can easily be modified by substituting ingredients. So, if you are in an Italian restaurant and snap a pic of your eggplant parmesan, once you make the recipe match, you can cut down on the salt and substitute a lower-fat cheese to make the dish healthier. Send that recipe to your guided cooking system and you are on your way to preparing it at home.
“This could potentially help people figure out what’s in their food when they don’t have explicit nutritional information,” Nick Hynes graduate student at MIT adds. “For example, if you know what ingredients went into a dish but not the amount, you can take a photo, enter the ingredients, and run the model to find a similar recipe with known quantities, and then use that information to approximate your own meal.”
