data

 

I recently caught up with Victor Penev, CEO of Edemam, about his company’s effort to create a data layer for the Internet of Food. You can hear that conversation on the Smart Kitchen Show podcast here, or you can read the full transcript of the conversation below. The conversation has been edited slightly for readability.  

Michael Wolf: How are you doing, Victor?

Victor Penev: I’m doing very well. Good to be here, Michael.

Michael Wolf: Now you started your company back in 2011, and you’re one of the early companies I think really to go after this idea of trying to organize food data. Tell us about the original concept for Edamam.

Victor Penev: So the original concept actually started a little bit before 2011. I’ll give you just a little bit about my personal history. I’m a serial entrepreneur. I had a good exit at my last company. We built the largest Internet company in Bulgaria and I had taken a year off and I was looking to do something new. Eventually because I’m a passionate cook, and I cook every day in my life, I decided I’m going to do something at the cross section of food and technology.

I started looking around the space and very quickly came across one of the biggest problems I think that’s related to food is that people even looking 50, 100 years from now will still want to know what’s in their food and how it impacts their health and wellbeing. What I realized is that the information about food is not that readily available. It’s oftentimes contradictory, inaccurate, and so on and so forth. We decided we’re going to try and organize the world’s food knowledge and give it back to people, so they can make smarter food choices and live healthier and happier lives. That was kind of the original idea. Then that was probably 2010.

Then we looked at various technologies of how to approach it and eventually ended up with semantic technology. There was a very simple hypothesis. Semantic technology is one of those things that fail quite often. People try to boil the ocean with it, but we thought that food is a fairly contained domain without too much spillover what an ontology can do, so that semantic technology can actually work in the food space. I spent some of my own time and money before launching another one formally in building a little bit of technology just to make sure that semantic would work.

Officially I think I called it October 2011, but somewhere around that date, we launched the company. We initially started as a B2C company and now after a couple of years, we switched to a B2B model.

Michael Wolf: In those early days of being a B2C company, you guys not have any success. You guys garnered hundreds of thousands of downloads through your application. Talk a little bit about those days and why you switched to be a B2B company?

Victor Penev: It’s a very simple business decision, so initially we thought we’ll organize high-quality recipes. We’ll take them for all kinds of nutrition and calories. We’re just going to provide smart suggestions for what to eat and maybe even meal plans on a weekly basis, and so on and so forth. We did have about 800,000 folks that installed our app, both on iOS and Android. What we found out after a couple of years is that as we tried to create paid product that consumers will be paying for, there were very few takers. We realized consumers just take back anything that’s related to food in terms of data to be free, so we couldn’t figure it out. A few hundred thousand users don’t have enough of people to create advertising-supported model, and I didn’t personally believe in advertising-supported model.

That was the time when we realized, “Okay, we got to do something different.”

Then just at that exact time, a few catering companies started coming to us and said, “Can you do the nutrition for our recipes?”

Then we said, “Sure! We’ll charge you $20 a recipe.”

They said, “Okay.”

Then we looked at ourselves and said, “Okay, well somebody is willing to pay for what we have,” so we re-productized everything and launched as a B2B player. That’s what happened. That was probably end of 2013, beginning of 2014, and since then we’ve been doing a B2B model. We went through API, custom implementations of the API, as well as data licensing.

Michael Wolf: I want to get into that and talk a little bit about your customers and how that’s kind of grown overtime, but talk a little bit about this idea of building an ontology of food using semantic web and association to create this database. What was the goal there and what does that mean? Was there a lot of work early on to kind of create the categories in trying to figure out where to put things?

Victor Penev: The hard work is not actually building the ontology. Like I said, the food space is relatively self-contained and fairly easy and straightforward to organize. I think a lot of the difficulty around organizing data around food is that it’s fuzzy. It’s not well structured. It’s not like physics, or chemistry, or any other hard subject that you have taxonomy and you have ways of organizing and so on and so forth.

I mean food has been around since humanity existed, so people talk about food in all kinds of different ways. They have a lot of implied meanings. There’s a lot of cultural background around it. The difficulty around structuring food data is not just the ontology itself, but actually the layering and what we build a natural language understanding on top of it. The ability to capture any data in terms of what people say about food and then transform it into something that’s quantifiable ‑ nutrition in our case.

We went with semantics because we were looking 10, 20 years down the road to be able to provide smart suggestions to people what they should be eating that necessarily imply influencing. We want to be able to know things about the person, maybe they’re allergic to something, if they’re on a diet, if they have a heart condition, but also their biochemistry. They are like sensors that take real-time blood samples, what have you, and also know what their goals are in terms of fitness and health and start inferring from all the structure that we have, what will be the best meal for them to eat. That process is endless.

I mean new data can be added constantly. I think there’s a big new field coming on board, which is the microbiome, which will be probably in the next 10 years change drastically the notion of how we should be eating. Obviously, there are sensors that are trying to constantly measure what’s in your blood and that’s a new thing that will probably hit the market again in the next 10 years.

Our goal was to organize and structure all these data in a way that can do meaningful suggestions to people what they should eat and that required inferencing, that’s why semantic technology.

Michael Wolf: And over time, you accumulated a huge database. I think you said you have a database of about 1.7 million recipes and you’re working with companies like the New York Times, Epicurious. Talk about how you provide that information and then how it’s used by these companies.

Victor Penev: that’s one of our major use cases is companies that have lots of recipes. The New York Times and Epicurious are great clients, but we do the same thing also for catering companies, for restaurants, anyone that has a lot of recipes that need nutritional analysis. We really replace the human nutritionists so to speak because that’s the alternative for most of those companies. For some of them, it’s just not affordable like if you’re Epicurious, you have 300,000 or 400,000 recipes. Even hiring an army of nutritionists, it becomes very expensive and obviously a no-go proposition.

The way we work with all those companies is very simple. It’s an API integration based on their recipe in the format they have it. We process it on our end. We do the analysis. It takes less than 400 milliseconds per recipe to get analyzed and it’s not just cooking up ingredients to nutrients. We also take into account techniques such as what happens to the food if it’s fried, or marinated, or baked in salt, and so on and so forth, and we return back the data.

The data that we return to them has up to 70 different nutrients. It’s automatically tagged for about 40 most popular diets, so all the allergens, anything that is for example low-sodium, low-sugar, paleo, vegan, and so on and so forth, I mean any diet that you can imagine that has been the popular culture, we tag the recipes for.

We just return this data to them, and then after that they decide whether to display the data to the end-consumer. Some of those companies use it to improve their searchability and also for SEO purposes because that’s metadata that is very relevant to the content they have, so that’s how we work with them.

Michael Wolf: And so, when you look at the evolution of the connected kitchen, you guys have started to look at that space. Increasingly companies who were adding connectivity also were trying to add value on top of that. How would you envision yourself possibly working with a company that is making a device for the consumer and then the consumer wants to understand what they’re reading from the nutrients and health perspective?

Victor Penev: I mean there’s a couple of major use cases here. Again we’re coming from the perspective that people want to know what’s in their food and how that will impact their health and wellbeing, there are a couple of things that people can do. One is obviously find what they should be cooking, and that’s where our database of 1.7 million recipes comes in. They’re all nutritionally tagged and analyzed. You might be sitting in front of your smart fridge and a touchscreen or you might be talking to a virtual assistant that’s part Alexa or Cortana or whatever it is, and you might be saying, “Hey, I have broccoli in my fridge.”

We can actually know that you have broccoli in your fridge if the fridge is smart enough.

“I’m diabetic, and my husband is on a paleo diet, and my kids are allergic to peanuts. What can I do?”

We can suggest very high-quality meals that you can cook, and then from that point on, there is transactional capability to create a shopping list. They might be kind of what you mentioned earlier about the ability about guided cooking, so that particular aspect has a set of video instructions that take you through the cooking process, and so on and so forth. That is one use case.

The other use case, which probably is even simpler and more prevalent, would be people would be just cooking things and then finding out what’s in their food. It’s surprising to me that in this day and age, the majority of meals that people eat are home-cooked meals and there is no way for them to figure out the nutrition of those meals. Maybe you read a box of cereal and maybe you know what’s in a cup of milk but if you do anything a little bit more complicated, you start to track actually what you’re eating. You got to have to be very, very precise, take a lot of time doing it or kind of give up. That’s where we come in.

You can just in natural language speak, “This is what’s in that recipe and this is what I did with it,” and within a second, we will return the nutrition.

We can tell you, “Okay, well the [unintelligible 0:14:03] that you did is actually 700 calories per serving and it’s got that much salt and that much fat.” Then you can decide whether next time you’re going to cook it or modify the recipe, or maybe serve less of it, and so on and so forth.

Michael Wolf: This seems like the perfect Alexa Skill [laughter] I hear you talk about that. Have you guys talked about either through your partner or kind of have been the backend for an Alexa Skill that I can ask in making this, I have these ingredients, what is the calorie count?

Victor Penev: Yeah. I mean we’ve talked to Alexa from day 1 ever since Alexa was launched. Our challenge there was that we never figured out a business model, much like with the B2C space Alexa is a platform that says build an app and that app can be used by our consumers except there is no transaction. We don’t get paid by the consumers to do that and we know the B2C companies. We couldn’t figure out the business model on Alexa, but that is top of our minds.

We’re building for our nutrition research, which is a tool we sell to dieticians and nutritionists and restaurants, which leverages natural language. We are building voice recognition capabilities into mobile devices, and eventually we want to do in the kitchen as well. I will want to do it in every room actually, but we have to figure out the business model. In addition to Alexa, I know Microsoft is working on Cortana and they are pushing very hard in that direction.

If we figure out a way for a business to use our capability or somebody to sponsor an app that is voice-powered app for the Echo device or any other device that any company is putting out there that is powered by voice recognition, we’ll very quickly build it. It is very easy because we’ve done all the natural language, understanding the work upfront, and so for us, it’s just hooking up the voice recognition to that.

Michael Wolf: Couldn’t you basically build a white label skill that you then go to appliance company X or CPG Company Y said, “This is just you plug in. Here’s your skill. You put your skin on it, Maybe you add a few kind of cast components and then they create their own Alexa Skill with all this nutritional information?

Victor Penev: That’s a wonderful idea. The only thing I would correct with the idea is that I personally want to have the appliance manufacturer or the retailer to come to us and say, “We’ll pay for that for you to build that skill,” and then we’ll build the skill.

We scrapped the startup and we try not to put resources against something that is not going to have guaranteed revenue. That is the only thing, but I can definitely see ourselves working with Whirlpool, or Samsung, or Bosch, or any of those companies and be able to power that particular skill for them.

Michael Wolf: It’s still so hard to figure out what is the nutrition of this thing I’m making every night, and then you start to throw in all this different branch predictions. I’m going to fry it, I’m going to put it in an oven, I’m going to put olive oil on it. I mean there’s just so many and if you guys have the data, I mean I think we’re going to get to the point where consumers can access that information in a fairly quick labor. We’re not there yet, so it takes companies like you in combination with the consumer-facing brands whether that’s a hardware supplier, or apps, or whatever to do that.

Victor Penev: Uh-huh.

Michael Wolf: I definitely am in line with you. I think that’s going to happen. I think most consumers will want that.

Victor Penev: Absolutely, I think so, too. To my mind, that’s not a question of if but when and whether 2017 is going to be the year or we’re going to have to wait another year. That is the big question I think.

Michael Wolf: You mentioned a little bit about sensors and being able to kind of detect. Have you been observing what’s going on in that space? I think it’s an interesting space. We had a company called Nima at our event that does gluten sensing. I saw at CS this year finally the company is making the SCIO, which is making basically an infrared food scanner, which there’s been a lot of debate whether or not you can a low-cost infrared food scanner like the kind they’re doing. It’s usually that will be an interesting area as well. Have you guys looked into that that space?

Victor Penev: We looked into that space. I think like many other space in that area, that is still very early stages and it’s evolving. The challenge for all those companies, they serve a particular use case. If you are checking for gluten, there is a 0.8 percent of probes that have celiac disease. That’s a godsend product for you. The problem with most of those solutions is that they’re not serving the general public because to serve the general public, you have to do full chemical analysis of the food. You have to be able to say not just the content of gluten or if there is like a pathogen in it, but also to tell how much fat, or how much carbs and sugar, and how much vitamin A.

Right now, this has been done in chemical analysis labs and the largest one in Wisconsin is 1 million square foot, so it’s a lot of equipment that you have to fit essentially into a small device. Is that going to happen? I think so. It’s just going to take time to kind of get this million-square foot fitted into devices ‑

Michael Wolf: In your pocket?

Victor Penev: In your pocket, yeah, in your hand, or something like that [laughter]. I think that will happen. The other thing that’s interesting about sensors and I think that’s actually more evolved is it probably requires a lot more regulation and idea of program whatnot is those kinds of implantable sensors in the human body and/or stickers that constantly take blood samples in real-time, and so they track your biochemistry. To an extent, it’s not even that important what you eat; it’s important how what you eat impacts your body and your own blood chemistry.

It’s important to know how it impacts what’s in your food, so that you can make informed decisions whether you’re going to eat that or not eat it, but once you’ve eaten it, it’s interesting to understand how that impacts your blood chemistry and what corrective action you want to take if you need to take that corrective action. There are many people that monitor particular nutrients like that people with diabetes or kidney disease that is absolutely necessary. But for folks that are just checking calories or fat or sodium, that can be very useful, and so that’s a whole different set of sensors other than the ones that are analyzing food.

Michael Wolf: Speaking of sensors, in a way I think what Apple is doing with HealthKit is an interesting health layer. I think it would be interesting once you start to fuse the type of data you have with what for example they’re doing with HealthKit. Have you guys looked at integrating?

Victor Penev: Yeah. We’ve looked at obviously HealthKit. We looked at Fitbit. We’ve looked at every single platform out there that does health record management and personalized record management, and we’re very careful not to get into the space where we have to manage electronic health records. We need to be HIPAA compliant and whatnot. But obviously, food intake is an important thing.

I think for a lot of those companies, it’s an important thing. I think for a lot of those companies, they’re still trying to figure out who’s going to win the race on the device wearable, and the wearables for better or worse, are just too focused on sensors measuring energy output, how many steps, if I jump, if I’ve done 100 crunches and so on and so forth. The energy input, which is essentially food, is lagging behind, and part of the reason why it’s lagging behind is because it’s hard to do the energy input. Unless you find a way to do it automatically, which will be measuring the bloodstream of somebody, it will be ‑ people are not disciplined enough.

We made a conscious choice to hold off until we see enough of a use case of people being willing to input data through their mobile devices. The interface is still not there, and I think we probably have the most advanced interface with voice recognition with the accuracy. There’s a lot of people that do voice recognition but we have a very, very high accuracy in management and situational analysis. Even that is still more of a case where people that are health nuts or they have particular disease than the general public.

I think there’s going to be a watershed moment when Apple probably or one of the other companies in this space that’s really big. Apple and Fitbit look like are going to be the winners, but they go and say food is important to us now, so let’s build tools around food. I think when they start pushing it into their devices, that’s going to be the moment we’re going to jump on the bandwagon.

Again, we’re very careful. There’s a lot of trends we can put our resources against, so many different things, and we decided that this is not something that’s going to happen in 2017.

Michael Wolf: But I like that, the way you phrase that. They’re all geared today toward measuring energy output. Food really is the energy input, and I wasn’t necessarily suggesting – I guess I was a bit suggesting going into healthcare and competing with them but like a fusion of the capability and the data that you have with HealthKit data maybe in the consumer-facing app, maybe it’s on an iOS device, that would just be very powerful. It sounds like you’re thinking the same thing: you’re just kind of waiting for the right time.

Victor Penev: Yeah.

Michael Wolf: Maybe it’s one of your partners. Maybe it’s Apple using your data to do that.

Victor Penev: Yeah. I mean that’s exactly our play. We hope that eventually we’re going to plug into HealthKit. We’re going to plug into every single platform. We also integrate with Validic. I don’t know if you’re familiar with this company. But essentially, we’ll have data output that will go into personal profiles when people eat something and we’re starting to do that. But right now, we’re focused on providing these tools to dieticians, to restaurants, professionals that are paying for this service because they need it to run their business. That same thing is a B2C product very easily, which has dropped the price I think. We removed some of the feature that food service professionals or dieticians may need then. It becomes a B2C experience that can be plugged into HealthKit or anything else and just become an app that does that.

We do have something very similar with Samsung. We did maybe 2 or 3 years ago, a partnership with them for S-Health, which is essentially the equivalent of HealthKit. We were the first food app there and our recipes search based on nutrition with ability to log in the recipe account directly into S-Health, so with all the nutrition.

We’ve done that sometime ago and that was part of the experience of why we think the market is not ready yet. But we’re closely monitoring that market.

Michael Wolf: Take a step back and just kind of if you look 5 years from now, what does the kitchen look like with regards to nutrition information, all this kind of devices and data layers like yours? I mean will it have arrived at that point?

Victor Penev: Well, I don’t know if it’s going to be 5 years, but I’ll tell you what I think ultimately the situation is going to be, and I think the kitchen is moving maybe 2 years behind the smart car in terms of there was a lot of investment in the space and eventually started to become a reality and now it’s a question of somebody just putting the right regulation in place and the smart cars can become reality.

I think the smart kitchen is probably a couple of years behind, so maybe in 5 years it will happen because there was a lot of investment from big name companies into the space. I think that every single device in the kitchen will be connected. It’s the IoT dream that the devices in the kitchen don’t need to be connected to that many other things. They need to communicate with each other, and so the fridge and the stove, and the sous vide and your food processor only to have kind of have the same platform and be able to communicate with each other. If the fridge has onions, what can you be doing with those onions and have some kind of a communication to the oven where maybe you’ll be I don’t know putting them in the oven or whatnot and that will form a particular recipe, or a particular way or cooking, or they might be chopped in a food processor.

There’s got to be connection there, but I think every single device will have an interface. A device probably will have a touchscreen, will probably have voice recognition interface, or both. It will have probably some kind of a display to display to you important information. It might be a video that teaches you how to cook something that’s on top of your stove, but it might be like a shopping list that is displayed or recipe suggestion, which I hope we will be powering a new fridge, or even just a timer on your kitchen appliance, or in-built weight measurement that tells you how much of whatever you’re cooking with.

Those are the things, and I think that in addition to those interfaces, there’s going to be an overall software that runs them and the kitchen operating system and there’s going to be a data layer because this kitchen operating system an interface with a human will have with all devices will necessitate data, and it’s data about specialized nutrition but it can be data about cooking, about the provenance of the food, anything that might be related to your experience in the kitchen becoming much more seamless.

I’m going away from technology, but in kind of winning the kitchen back for the human. We used to enjoy being in the kitchen and sharing food and whatnot and we kind of went away with that with microwaves and TV dinners and whatnot. I think actually technology can bring us back to the kitchen and the joy of cooking because it’s going to make it a lot easier, food is going to be delicious every time you make it, and everybody will love it.

That’s kind of the vision I see and I hope us to be part of that solution specifically on the data layer with regard to foods and recipes and nutrition, so that we can help people make those smart food choices and eat better.

Michael Wolf: It sounds good. We’d love to have you out in Seattle, the Smart Kitchen Summit, to talk a little bit about it, so thanks for spending some time with me today.

Victor Penev: Thank you, and yes, we’re planning to be in Seattle.

Why don’t you subscribe to our free weekly newsletter to get great analysis like this in your inbox?