By now, most everyone has tried their hand at prompt engineering ChatGPT or another LLM to create a decent recipe.
But a decade and a half ago, well before the current craze of making recipes with generative AI, IBM was trying to figure out how to make Watson start cooking. The supercomputer-powered AI, which was probably the first real-world AI most of us knew by name, had just broken into the broader American consciousness after it had beaten human players Ken Jennings and Brad Rutter in a Jeopardy tournament. Now, IBM was looking for ways to showcase how the technology could help people be more creative, and they identified cooking and recipes as the next world to conquer.
Around this time, the Watson team teamed up with the Institute of Culinary Education (ICE) to help train Watson. James Briscione, who had won Chopped season 2 a couple of years before and was the ICE’s director of culinary research, remembers those early days when IBM computer scientists filed into his kitchen.
“The first day we set up, the Watson team came to the kitchens at ICE, walked in with a laptop, flipped it open, logged into an interface that IBM was hosting, and we started parsing datasets.”
This meant going through and looking at ingredient combinations based on cuisine style, dish type, and flavor profiles of different dishes, as well as breaking down each type of ingredient into the various flavor and aromatic compounds into building blocks, which allowed Watson to then process millions of flavor combinations and recommend them to ICE chefs. During the process, the Watson team made sure the human chefs remained as ana integral and necessary part of the AI feedback loop.
“For the majority of the project, it did not give us recipes, it gave us ingredient combinations,” said Briscione. “And then I did the work then to translate that into the recipe.”
Briscione said taking Watson’s combination suggestions and combining them into a recipe helped unlock the creativity of him and the other chefs.
“As a sort of a thought experiment, it was even more interesting because then we could take an ingredient output, I would take it and interpret that ingredient output one way. Another chef could take that exact same ingredient output and interpret it completely differently. So in inspiring creativity, it was really, really powerful.”
Nowadays, Briscione is applying what he’s learned to build a new company that helps train large language models to better understand food. He will discuss this new company at the Smart Kitchen Summit next week.
You can watch the entire interview and see the transcript below. .
Transcript
Michael Wolf: I’m excited to have James Briscione who is a chef I’ve been following for a while. James, you do so many things. You’re an author. You’re a Food Network personality. And you’re one of those rare chefs that have been dabbling with AI longer than pretty much most people even working with AI at all. So it’s exciting to have you. Thanks for coming.
James Briscione Yeah, Michael. Excited to chat here excited about SKS coming up in June. This will be a great event and can’t wait to get there.
Michael Wolf Yeah, we’re going to hear you on stage talking about your experiences and what you’re looking forward to with the integration of AI. But for those who don’t know you, tell us a little bit about your background and what you’ve done over your career.
James Briscione As you said, I’m a chef first. I started as a dishwasher at the age of 16, worked my way up to some of the top kitchens in the country. James Beard award winning kitchens that I was at the helm of. Four Star Fine Dining in New York City. Kind of did it all. With that really elevated fine dining background, I moved into education at the Institute of Culinary Education in Manhattan and really was in the right place at the right time when IBM came knocking and said, ‘we’ve got this crazy idea. We’ve got this thing called Watson, that just conquered Jeopardy. And now we want to see if it can help people. We know it can answer questions. We want to see if it can help people be more creative.’
And they thought about music, they thought about visual arts, but you know, felt those were too subjective and culinary arts was a very objective area for this. So when they came to meet with us, they met with all the instructors, kind of talked about the process of development and creating dishes, and how you work as a chef. Having just been the first two-time champion on the show Chopped on the Food Network, the way I sort of process and put together flavors and ingredients was exactly what they were trying to build with Watson. So that started about a four-year relationship working with the core team there at IBM to develop Chef Watson, which I now know was recipe generative AI. Almost 11 years ago, before we started building it, I had no idea what those words even meant. And AI was only something you saw in Will Smith movies.
Michael Wolf So those early days, you’re helping with Watson. Are they bringing you into a kitchen at IBM headquarters? What does that exactly mean? Are they monitoring you with cameras, or are you saying, ‘hey, these are what flavors are trying to tell a computer what a flavor is?’
James Briscione First, as we talked about it, I was still in that Chopped competition mode. So I was like, ‘if I’m going to cook against this computer, I’m going to kick its ass. I’m actually going to prove that this thing can’t do it better than a human. The first day we set up, they came to the kitchens at ICE (the Institute of Culinary Education), walked in with a laptop, flipped it open, logged into an interface that IBM was, was hosting, and we started parsing datasets and going through and generating ingredient combinations based on a number of different factors based on cuisine style. so original cuisine, a type of dish and, and, and a core ingredient to inform, the flavor profile of, of the dish. So we’d say Italian grilled lobster. And then it would generate trillions of possible ingredient combinations that could be used to create a dish that were typical Italian ingredients that kind of fit in with what it knew about a grilled lobster recipe or a grilling recipe and a lobster recipe overlay. And then use that lobster to as kind of the core flavor profile to then build sort of that flavor tree off of that core ingredient, which that process, that’s how I tend to think about creating a dish, but getting down to the molecular level, understanding all of the aromatic compounds in the food, how those flavors relate to one another, why they go well together. I never looked at information that way or understood it in that form. And it was mind blowing to process tens of thousands of aromatic compounds in every dish, just like that.
Michael Wolf So it was essentially building, I don’t know if the right word is ontology, but kind of trying to dissect food at a more atomic level and then understanding what the commonalities are. You know, saying ‘lobster often goes in these types of dishes’ or ‘Hey, maybe it works with these types of dishes.’ So really trying to create the data building blocks so Watson can then say, hey, here’s a unique flavor idea, recipe idea you may not have thought of with your small human brain.
James Briscione Exactly. And, you know, for the majority of the project, it did not give us recipes. It gave us ingredient combinations. And then like, you know, it was kind of, I did the work then to translate that into the recipe. But as sort of a thought experiment, it was even more interesting because then we could take an ingredient output, I would take it, and interpret that ingredient output one way. Another chef could take that exact same ingredient output and interpret it completely differently. So in inspiring creativity, it was really, really powerful. And actually, there were some cool examples of where we would take the same generation, go to separate sides of the kitchen, and come back in the middle with our finished dish. You couldn’t even tell that they started at the same place.
Michael Wolf You’ve watched over the past decade, this expansion of folks trying to use technology to understand the way we cook better. Those early days of watching Watson were pretty seminal and informative, and that was the first time I remember seeing articles, maybe in the New York Times, saying ‘Watson beat Jeopardy, now it’s trying to cook’. So as you’ve watched this evolve over the past decade, what have you been thinking about? And what have you learned maybe about AI and its intersection with food? Is it something now you’re more excited about than ever?
James Briscione 100% more excited than ever. I think the potential here to simplify, to streamline, which to me is kind of the ultimate promise of AI, to make our lives better, to organize and streamline. I think where obviously it gets tricky, is one, it’s new. So there’s going to be some inherent distrust of it. One bad recipe, one recipe that doesn’t work and people are going to bail on it as well.
Michael Wolf Right, right. We’ve all done those bad recipes with ChatGPT. Like that just sounds awful.
James Briscione Yeah, and you know, I mean, it’s going to be interesting to watch this landscape too now because the majority of what’s out there are just some, you know, some basic GPT wrappers. And if any of these copyright lawsuits get through, a lot of these datasets, these sources, start to dry up or become more restricted. So one thing I’m starting to work into is building a new dedicated model for recipe generation with nutrition and flavor inputs that really can optimize your food specifically for you. If you want to get down as far as the genome, I think that’s some functionality that is off in the future, but generally, as an active 44-year-old male who lives in a hot climate, AI can tell me exactly what I should be eating on a day-to-day basis to optimize me for what I do.
Michael Wolf That’s interesting. And I think the startup you’re working on is called CulinAI. And so that’s exactly it. And so is this something you’re building your own large language model or you’re building something that can integrate with maybe some of the other large language models? Tell us a little bit about it.
James Briscione Yeah, so, and I’m actually working with the original developer of Chef Watson. It’s kind of a hybrid model where we are going to be employing some large language models, but also some kind of dedicated pieces that would be unique to this model, particularly the flavor science and the nutrition data input. And then, really, kind of the secret sauce is in the selection because, again, we know that the large language models can generate lots of great things that look like good recipes, but training it to then go back through those and select out the ones that are actually right is where it all comes together.
Michael Wolf Well, I’m excited to hear more about that at Smart Kitchen Summit. You are someone who works in a professional kitchen. You’ve been on TV, won awards, you have your own restaurant. But there’s also the consumer, right? Someone who, like me, doesn’t know what they’re doing. And one of the reasons I got interested in the Smart Kitchen in the early days is because I thought that maybe technology can help me become a better cook. How do you think average everyday consumers who aren’t like you can use technology tools like AI to help them cook better?
James Briscione We talked about kind of one of the biggest benefits AI is to make our lives better, to simplify processes and personalization, right? And I think that’s really where it comes in to find the right information. Even just how to get your ingredients organized at the beginning of the week to set up for, hey, ‘here’s what I’m going to, here’s what I’m going to cook for the week’, building out a meal plan that utilizes all of the ingredients that you have so that you don’t, at the end of the week, have half a pint of cherry tomatoes, three quarters of a head of celery, two onions, and half a butternut squash. It’s all just sitting there because you bought it all because you had to have it for that recipe, and now it all is just kind of like laying to waste, and you leave it there until it’s time to finally throw it away. And I think some of those, I think a lot of those things are what discourage people or kind of keep people from cooking. So, AI tools that can teach you to approach that process the way I do as a chef of not just looking at, okay, here’s what I’m gonna do for dinner for Tuesday night, but okay, as I’m doing dinner for Tuesday night, here’s how we get lunch for Wednesday ready.
Michael Wolf Right, right.
James Briscione And another chunk of dinner for Thursday, all kind of set up and set aside so that that’s easier too. And I think a lot of those tools are some of the things we’re looking at building into CulinAI, and I think those are the pieces that I’m excited about.
Michael Wolf Well, I’m excited to hear you in Seattle in June at Smart Kitchen Summit. James, where can people find out more about you?
James Briscione Most social media platforms at James Briscione. That’s probably the best way to find me, LinkedIn, all of the typical places, just right under my name, I’m there. There’s not many Brisciones around, so.
Michael Wolf All right, man, we’ll see you in a bit. Yeah, there aren’t. That’s a great, unique name. All right, James, we’ll see you soon.
James Briscione All right.