Close Ad

The Spoon

Daily news and analysis about the food tech revolution

augmented reality

Kittch Partners With Qualcomm for Augmented Reality-Powered Cooking

Why wear beer goggles when you can wear bing goggles?

Well, now you can, at least if you’re using Kittch’s new AR cooking mode to make chef Ming Tsai’s MingBings. Today Kittch, a culinary video community, announced they have teamed up with Qualcomm to integrate AR features into their cooking app, according to an announcement sent to The Spoon. The collaboration, done in partnership with technical design company Trigger, is being demoed this week at Augmented World Expo (AWE) in Santa Clara, California.

Kittch app users can access the Kittch AR cooking mode by plugging in AR glasses to a mobile phone and clicking the “view in kitchen” button. From there, users can follow interactive videos and recipes, set timers, and order ingredients via AR gesture control. The new feature was demoed by Ming Tsai and his MingBing recipe here.

Kittch & Qualcomm aren’t the first to show off AR-powered cooking for the home kitchen. Back in 2021, AR designer Lauren Cason went viral after spending an afternoon hacking together an AR cooking prototype to demonstrate to her father the power of augmented reality.

The Kittch demo shares some of the same features, including showing recipes and the ability to place visual timers in the designated places around the kitchen, but unlike Cason’s demo, the Kittch app lets you order ingredients.

It’s a cool feature for home cooks, but I’m still waiting for someone to create an AR app for the professional kitchen. A non-obtrusive AR tool made for busy cooks to monitor their different meals in process, access order info, check food inventory, and more would be a hugely valuable tool if done right. Get on it, AR people!

Amazon Now Lets You Buy Physical Goods in Virtual Worlds. Could It Work For Food?

This week, Amazon announced a new platform called Amazon Anywhere that enables the discovery and purchase of physical products from within virtual environments such as virtual and augmented reality and video games.

The platform, which the company showed off through an integration with an augmented reality pet game called Peridot (from the same company that made Pokemon Go), allows customers to buy physical products without leaving the game environment. Game players and VR explorers can see product details, images, availability, Amazon Prime eligibility, price, and estimated delivery date as they would on Amazon’s website. They tap the “buy” button and check out using the linked Amazon account without leaving the game, and from there, products will ship out and can be tracked and managed via the Amazon app or website.

Today in-game and virtual world purchases are limited to digital goods like currencies or digital characters, but Amazon’s new platform opens up a potentially interesting new way for players to buy physical products. The Peridot demo enables players to buy merch like t-shirts, hoodies, phone accessories, and throw pillows with game art on them, but what if shelf-stable food or food-related items were sold from within the virtual environment? Would emerging CPG brands, which often use DTC strategies early on, see this as a potential new channel to market?

While the idea is an intriguing one, the main problem with Amazon’s platform is it’s Amazon’s platform. Amazon is a relatively expensive place to purchase food, and smaller emerging DTC brands tend to prefer selling on their website using white-label e-commerce platforms like Shopify, WooCommerce/WordPress, Magento, and Squarespace until they finally graduate to retail.

However, in-world physical product purchases might get traction with bigger multichannel CPGs. Amazon tried to court big CPG brands early on with its IoT-powered Dash buttons, but eventually abandoned the project in 2019 (though they are still selling a Dash smart shelf). The company also tried to get a return on its massive investment in Alexa through sales of everyday consumables, but the division’s recent struggles show consumers, for the most part, still like to click buttons on a web page or an app to complete a purchase.

Which brings us back to Amazon Everywhere. The use of virtual or augmented worlds will grow in time, meaning Amazon’s early effort to build a platform could pay big dividends in the long run. Brands could tie products to stories or characters through experiences that would be pretty much impossible through more traditional advertising. With in-world purchases, they would be able to convert in an entirely new way.

While it’s too soon to tell if consumers will bite, I have no doubt Amazon will attempt to find out. My guess is we’ll also see other players like Facebook and Microsoft follow Amazon’s lead and build out VR and video game in-world purchase platforms for physical products as well, but for now, it looks like Amazon has got the jump on them.

Meta Wants Your AR-Powered Ray Bans to Remind You to Buy Milk

We’ve covered augmented reality tech that helps you cook in your home kitchen or visualize items in a food court, but what about reminding you to buy milk or eggs at the grocery store?

That’s a scenario Facebook parent Meta describes in a new patent awarded to the company for a smart assistant for augmented reality hardware as first reported by longtime tech and entertainment reporter (and former coworker of mine) Janko Roettgers on his blog Lowpass.

According to the patent, the Meta smart assistant app would be used on smart glasses (such as its Ray-Ban Stories wearables) through both reminders and geo-aware contextual information provided by GPS signals:

  • Another example shown in the application (and atop this article, if you’re reading it on the web) is a reminder to buy milk. The assistant then figures out that the best time to deliver this reminder is when the user goes to the store.
  • From the filing: “The assistant system may determine that the user is at a supermarket/grocery store based on … the user’s location information captured by GPS signals or visual signals captured by the smart AR/VR glasses” and then remind them to buy milk.

I’d find this type of application useful, but where I’d really like to see this go is to provide in-store navigation information (like help me find milk in a grocery store) or to provide additional contextual information as I compare items on a store shelf (which items has lower sugar or doesn’t have gluten, for example).

That would require in-store beacons and sensors, which we’re already starting to see make their way into stores to power contactless retail systems. This type of in-store contextual AR info would be even more useful for grocery store employees, as they restock shelves, check prices or check to see if an item is in stock.

Make Pie in the Metaverse? Designer Creates AR Kitchen Assistant With Snap Spectacles

One day when Lauren Cason was making a pie with her dad, it wasn’t long before talk turned to her job.

As an XR designer who helps big brands figure out how to use technology like augmented and virtual reality, Cason explained to her father how augmented reality glasses like the Snap Spectacles work. As she was talking, Cason had an epiphany: she could make a prototype to show him.

“One of the things that’s really cool about the Spectacles is that you can prototype stuff with them very quickly,” Cason said during an interview with The Spoon. “So we worked on the pie, and then I spent the afternoon putting the prototype together, and I made a video.”

The video, which you can see above, shows the view of someone working in the kitchen with an AR lens designed to help them cook. The lens puts relevant information such as cooking temperature and countdown timers over items cooked on the stove and in the oven. Cason also created the ability to pull up a recipe by touching a specific area within the cook’s view and even had a note from her dad appended to the dough roller reminding her not to over-roll the dough.

As soon as Cason put the video on Twitter, it blew up. People loved it, including chefs, who told her they would use something like this in their restaurants.

Lauren Cason

“They said, ‘I don’t know if I would really use this at home, but I would totally use this in my industrial kitchen,'” Cason said.

Cason also got helpful feedback on other aspects of the video. For example, in the first version, a recipe would pop up when the cook tapped on the microwave and a recipe showed up, but some felt the information would obscure the view of the oven as the cook looked around.

“Some folks gave me the feedback that it might be a safety issue if you can obscure the cooktop and the flame area with the recipe,” Cason said. “So I changed it.”

When Cason bakes cookies in the second prototype version of her AR lens, not only does the recipe stick in one place on the wall, but it also includes visual placement guidance on the cookie sheet to help the baker evenly space the cookie dough.

Talking to Cason, my mind began to envision all sorts of ways in which augmented reality could be used as a cooking assistant both in the consumer and professional kitchen alike. Our talk also confirmed what I already suspected: augmented reality, of all the broader technologies that are being grouped under the term metaverse, is the most mature and ready to be useful in real world situations like our kitchens.

When I asked Cason about the metaverse, she said that it’s an interesting time for both the technology and the term itself. However, she also expressed cautious skepticism, especially when it came to big companies like Facebook’s embracing the technology.

“One of my old bosses gave a really good talk about the term metaverse and he said that Facebook was ‘cookie licking’ right now.” Cason said. She explained that the term cookie-licking is when a company treats a new technology space like a plate of cookies, only instead of eating the cookie, Facebook is essentially just picking one up, licking it, and putting it back on the plate.

They’re saying “I’m not going to eat the cookie right now, but I don’t want anybody else to eat the cookie either,” Cason said.

For now, Cason doesn’t concern herself with what cookies Mark Zuckerberg is choosing to lick on the metaverse cookie sheet. Instead, she just plans on continuing to keep iterating on her prototype so her dad – and maybe even a professional chef or two – can bake cookies in the real world with a bit of help from AR.

With Food Scan, Snapchat is the Latest Company Trying to Create a ‘Shazam for Food’

Last week, Snapchat became the latest company to add an AI-powered food scanning and recognition feature to their product. Called Food Scan, the new feature enables Snapchat users to scan food items and get recipe suggestions and other information about the food.

Here’s how it works:

Snapchat users can scan food by opening up the AR bar within Snapchat from the main camera menu option. From there, they choose to scan and click a picture of the food item. Snapchat’s AI will process the image and suggest a recipe from partner Allrecipes, as well as serve up other information, such as a Wikipedia page, about with the item.

According to Snapchat, the feature has access to over 4 thousand recipes and can process up to 1500 ingredients. However, based on my own attempts, the product may need to add a few more items to the list to be helpful.

When I scanned Campbell’s Cream of Celery Soup, it offered up recipes for Campbell’s bacon soup. A scan of Adam’s peanut butter resulted in recipes for tahini. The closest match came from a scan of a navel orange, which resulted in a Wikipedia page for a mandarin orange and recipe recommendations for mandarin orange cake and mandarin orange salad.

Here at The Spoon, we’ve written a lot over the years about attempts by companies to create a ‘Shazam for food.’ Big tech companies such as Microsoft, Pinterest, and Google have been at work at this for some time, as have appliance brands like Samsung and Whirlpool. Part of the reason so many have dedicated resources to building augmented reality and AI products for food recognition is, quite simply, because food is one of the easiest product categories to recognize and create databases around. But it’s also because food recognition unlocks numerous commerce, health and nutrition tracking, and kitchen management scenarios if done right.

Snapchat’s new Food Scan feature, while pretty rudimentary at this point, clearly has designs on building potential revenue through shoppable recipes and product recommendations. However, I can also see it becoming a broader food-related augmented reality recommendation tool that suggests, similar to Alexa’s new What to Eat skill, restaurants, meal kits, and other potential monetizable recommendations.

Make Beats at the Breakfast Table with Reese’s Puffs AR Cereal Box Drum Machine

After seeing Mark Ronson’s “Watch the Sound” series on music, my 10-year-old son is now very into drum machines and beats. And while he would love a vintage Roland TR 808 to kick off his burgeoning music career, I think instead I’ll get him a box of Reese’s Puffs cereal.

Not that I think a bowl of Reese’s Puffs is the breakfast of champions, but a new promotional box for the cereal out now features an augmented reality drum machine on the back. I received a press release about the new cereal box beatmaker this morning. Usually when I get these types of emails, I immediately toss them. But as I looked at the information, it actually seemed like a pretty cool use of technology, so I went out and bought a box this morning (with apologies to my wife, who does not yet know there is a giant box of sugar cereal jammed into our pantry).

Here’s how it works. On the back of the box is a diagram of the RP-FX drum pad. You scan a special QR code with your mobile phone and it takes you to a special Reese’s web app that accesses your phone’s camera. Place cereal puffs wherever you like on the drum pad spaces and then hover your camera phone over the box. Using AR, the app “reads” where you placed your puffs and generates a beat accordingly. Move the puffs around and the beat changes.

It’s definitely not high fidelity or Pro Tools quality production, but it actually works surprisingly well. Once you have your beat just as you like, you can use the app to record it, so you can share it with friends or use it as the basis of your next club banger.

You can hear the one I whipped up this morning here (or, you know, wait a few months and it’ll be all over the radio).

As noted earlier, I typically shy away from covering promotional stunts like this. But as a parent and a fan of both music and technology, this promotion is actually worth, well, promoting. Besides, using a cereal box to build your own beat sure beats digging for a cheap plastic toy at the bottom of one.

Third Aurora to Bring Augmented Reality to Beer Labels

I’m not ashamed to admit that sometimes I solely pick out a can of beer based on the label artwork. Sometimes there isn’t enough information listed on the can to make an educated decision, so I let bright and colorful artwork catch my eye. Third Aurora, a tech company focused on augmented reality (AR) and machine learning, is gearing up to launch its platform for augmented reality beer labels so beer cans do provide more information. The platform is in the form of a mobile app called Beerscans, which will be made available for users in the upcoming months.

Matt Hallberg, one of the cofounders of Third Aurora, said that many breweries have interesting stories and are doing things behind the scenes, like focusing on sustainability, but the consumer isn’t able to get the essence of the brewery from just one can. A beer can only has space for a few brief sentences and consumers typically have to go on the brewery’s website to learn more.

Augmented Reality Packaging - Beverage Packaging Demonstration - Third Aurora

The Beerscans mobile app uses the phone’s camera, augmented reality and computer vision to scan a beer can, and the app recognizes the circumference of the can and the label. An augmented reality label then pops up and hovers over the can. The augmented reality label will share the story behind the brewery and beer, tasting notes of the beer, and other information that a brewery may want to share. Breweries interested in being a part of the Beerscans app must sign up through the Beerscans website and upload desired information through the website’s portal.

Previously, Third Aurora launched Winerytale, an app that is the same concept as Beerscans, except for wine bottles. Living Labels is another company that has partnered with a handful of wineries to provide AR labels including the popular 19 Crimes winery. Drink AR launched in 2020, and provides AR labels for wineries, breweries, and distilleries.

Third Aurora aims on launching Beerscans in about three months, and it will be available for free on both iOS and Android. Third Aurora is in the process of signing up breweries and wants to have at least 100 breweries signed up prior to the launch.

Augmented Reality Bites

This is the web version of our weekly restaurant tech newsletter. Sign up today to get updates on the rapidly changing nature of the food tech industry.

Virtual food hall, meet the augmented-reality restaurant menu. You’ll soon be best friends.

Hear me out.

Over these last few weeks, multiple news bites around virtual food halls have surfaced. These food halls are collections of restaurants that exist online and where meals are only available for delivery and pickup. They are in many ways a natural effect of the pandemic shutting down dining rooms and the restaurant biz going off-premises.

The latest one comes from Lunchbox. This week, the company integrated its online digital order platform into C3’s virtual restaurant brand ecosystem to bring a bunch of different delivery-only eateries under one virtual umbrella.

Being able to order a plant-based burger, chicken, and maybe a rice dish through a single digital interface sounds great until you zero in on that word digital. One of the potential problems with this new wave of virtual food halls is that customers will never have the chance to actually visit these restaurants in person. Your introduction to their food comes in the form of 2D thumbnails you have to scroll through on your phone and squint at to even get an inkling of what you’re about to order. If you’re familiar with the restaurant that’s less of an issue, but most virtual food halls and brands are new, and ordering from them is something of a culinary gamble.

Enter augmented reality (AR), a technology some say is the next great innovation for restaurant menus. Modern Restaurant Management ran a piece this week exploring the possibility of customers using their own smartphones to display 3D models of the food they are about to order. With AR, instead of a small, flat, 2D image, a user could “see” how the dish looks on their table, zoom in on it and view it from multiple angles to get a much better idea of what they’re about to buy.

I should note that the Modern Restaurant Management Post was authored by Mike Cadoux of augmented reality platform QReal. In other words, Cadoux’s has skin in the AR game.

But he makes a good point when it comes to thinking about AR in the context of the new off-premises reality in which restaurants now operate: “Early adoption of AR was hindered by the problem of getting the experience to the customer. People are loath to download apps, and delivery platforms had to service thousands of restaurants, most of which wouldn’t have access to 3D models. Now a restaurant or brand can push their own content to the customer. They would be wise to utilize all the smartphones capabilities and showcase their food with the next-generation of content.”

Spoon Editor Chris Albrecht actually spoke with Cadoux back in August, when QReal released a study with Oxford University’s Saïd Business School that found participants were more likely to order an item if they could view options in AR. “It’s like a test drive for a car,” Cadoux told The Spoon at the time. “Same way when you buy food, you want to think about what it’s like to eat it.”

The tech makes especially good sense for virtual food halls. As I said, these restaurants do not have dining rooms, so customers are relying solely on the digital realm to learn about the food. If, for the sake of argument, Lunchbox and C3 were to integrate AR into their ordering platform, they could better showcase the “fine dining” aspects of their food and in doing so make their meals more appetizing. Everyone else, from Zuul’s virtual-only sandwich chain to Steve Aoki’s pizza brand, could also reap the benefits of AR in the virtual restaurant realm.

AR is not yet mainstream, and its presence in the restaurant industry is still largely forthcoming. But since one pandemic year seems equal to five normal ones, an AR-powered food hall may be closer than we think.

Uber Engineer Says “No” to Uber’s Prop. 22

Californians, take note. One of the things those in the Golden State will vote on come November is Prop. 22, a $180 million ballot measure that would allow third-party delivery services to classify drivers as independent contractors. The measure would effectively override California’s Assembly Bill 5 (AB 5), which was signed into law last year and dictates that Uber, Grubhub, and other gig-economy companies must classify drivers and couriers as employees. 

Classifying them as independent contractors means delivery drivers would lack access to workers comp., paid sick leave, and other benefits W-2 employees receive. It goes without saying that a lot of folks are against Prop. 22. One of them is an employee at Uber.

Kurt Nelson, who’s been a software engineer at Uber since 2018, penned an op-ed at TechCrunch this week that argues drivers should be classified as employees. Nelson, who still makes deliveries for app-based companies in order to understand the gig economy, writes that Uber “refuses to obey” AB5 and instead prefers to “write a new set of rules for themselves” with Prop. 22. 

Among many other notable lines, there was also this gem about the gig economy: “I’ve met drivers who have to sleep in their cars, risk financial ruin over a single doctor’s appointment or go without life-saving medication. There’s no way around it. Uber’s Prop 22 is a multi-million effort to deny these workers their rights.”

You can read the piece in its entirety here. Uber has yet to make any public response to Nelson’s op-ed, so stay tuned.

Restaurant Tech ‘Round the Web

Kitchen United CEO Jim Collins has stepped down to “focus on personal endeavors,” according to Nation’s Restaurant News. Collins played a major role in turning KU into one of the leaders of the ghost kitchen space. Michael Montagano, KU’s former chief financial officer and treasurer, has been named CEO.

Mobile POS platform GoTab launched an integration with hospitality labor management system 7shifts. The combined offering gives restaurant owners/operators the ability to view sales and labor data from the same interface.

Meal prep software company Meallogix announced a partnership with DoorDash this week. A press release sent to The Spoon notes that the deal gives Meallogix’ customers the option of using the third-party delivery service to manage their routes for the last mile of delivery.

How Will AR and VR Change the Way We Eat? Jenny Dorsey Has Some Thoughts

Part chef, part entrepreneur, all innovator, Jenny Dorsey has become to go-to expert in the intersection of augmented and virtual reality. When Smart Kitchen Summit founder Michael Wolf spoke with her on our podcast last year, he called her “foremost authority on the nexus point between AR/VR and food.”

So of course we invited Dorsey to speak about it on stage at SKS. To whet your palate, we asked her a few questions to discover more about what exactly we have to look forward to in culinary future — virtual and otherwise.

Want to learn more? Make sure to get your tickets to SKS on October 8-9th to see Jenny Dorsey talk about how augmented and virtual reality will change the way we eat.

This interview has been edited for clarity and content.

Q: What drew you to explore AR and VR through food, something seemingly very separate and disconnected?
A: It is the strangest story. I went to acupuncture in the spring of 2017 totally confused about what I wanted to do with my life and art. I had this random idea pop into my head at acupuncture that I should focus on AR and VR…which I literally knew nothing about. I went home to my husband and he just said, “Okay, I support you…but I don’t know what you’re talking about.”

Fast forward a year, and I’ve been experimenting with different ways to merge these various things together. I’ve learned a lot about what doesn’t work (eating with headsets on) and what makes people prone to distraction (AR apps), but I also found some pretty awesome ways to communicate and strengthen my food through AR/VR. For instance, I hosted a tasting event in Nicaragua where we profiled three different types of Nicaraguan agricultural staples using 360° video, then served guests both headsets and the final tasting menu after they watched — and learned — the seed-to-harvest process of these ingredients. It was really educational, fun (for many, it was their first time in VR!) and the process added some extra meaning to the food and drink we prepared.

Our next big thing is a series called “Asian in America”, which explores the Asian American identity through a symbolic meal, paired with a stroke-by-stroke Tilt Brush recreation of each dish for viewers to watch, while listening to the symbolic explanations, before eating. (You can see more about both of those events over at Studio ATAO.)

Q: Tell us about your experimental pop-up series, Wednesdays.
A: Wednesdays started in January 2014 as a personal creative outlet while I was working in a restaurant and feeling pretty burned out. At the time, my then-boyfriend, now-husband was still in business school (where we met) and I remember us commiserating on how hard it was to get to really know people around us. He was interested in making cocktails, and we thought: why don’t we host a dinner party? We wanted to create an environment where people would be comfortable enough to be themselves and be vulnerable around others.

We hosted a beta-series of dinners with friends for the first month, then we started getting strangers coming to the table to eat, which prompted us to say “Hey, maybe we are onto something”. Fast forward 4 ½ years and we’ve hosted hundreds of dinners for thousands of guests across New York City and San Francisco, been written up in many major food media outlets, and usually sell out in 30 minutes or less!

We aren’t your average dinner party — we do ask a lot from our guests. There’s mandatory questions to answer before you even purchase your ticket (everything from “What’s your biggest failure and how has it motivated you?” to “Are you in the job you want? If not, how are you getting there?”), lots of bizarre things to eat and drink when you arrive (like bugs!) and direct, in-your-face realness from me, my husband and our team. There’s no small talk. It’s not for everyone, but for the people who follow us I think it’s really what they are looking for.

Q: What’s the coolest/craziest way you’ve seen technology changing the food system? Blow our mind!
A: I’m currently very interested in how blockchain could help the food system. Seeds & Chips just put out a call for blockchain influencing the egg supply chain, so I’m really excited to see what different companies come up with. I also spent some time at a winery last year and was amazed to see they have drones which tell them literally when and which plot of vineyards to pick for a certain Brix (sugar) count in that specific grape. That sort of detailed information would’ve taken constant field-walks to ascertain years ago.

There’s also technology that will calculate exactly how much food waste your restaurant generates in a week/month/year, AND a system that will turn that waste into compost. While technology has done a lot in terms of streamlining of our food system, I’m still waiting for it to solve some of the biggest issues we face today: a living wage, worker rights, consistency and training, preventing food waste, educating consumers, etc. — pieces that require more politics and facetime. Overall, we still have lots of work to do!

Q: How do you see AR/VR — and technology in general — shaping the future of food?
A: I still stand by the major points in my TechCrunch article from late last year. I think the biggest areas of impact will be food products (CPG) and how they are marketed — both experientially (through VR), but also packaging (through AR).

In terms of restaurants, I just wrote a piece about VR training, which I do think will be a fantastic and hugely influential piece of the technology — but it really needs to come down in price point first.

Overall, I think artists and creators are still getting acclimated to how this technology works and what they can do it with. I hope to see AR/VR become almost an expected point of interaction or engagement between food business (product, service or restaurant) and the customer as we continue finding artistry in it.

Q: What’s your desert island food or dish?
A: I feel I should say something cold, because I would be hot, but most likely I would be craving pho. LOL!

Coolhobo Brings Augmented Reality to Chinese Millennial Shoppers

While the name “Coolhobo” might conjure up an image of an old-timey drifter with a bindle and a sweet leather jacket, it’s actually a slight mispronunciation of the Chinese word for “carrot” (胡萝卜).

Based in Shenzen, China, Coolhobo is an early-stage startup that does augmented reality shopping and is aimed at Chinese Millennials. Though it could be used for any type of shopping, the company is starting with groceries. Coolhobo is creating a mobile app that allows shoppers to find desired products in a store, discover more information about them and share their experiences in a social setting to make shopping more fun.

As Coolhobo Co-Founder and CEO Loïc Kobes explained it to me, when the app is launched, users will be able to choose a product they are looking for (for example, a particular bread), and the app will tell them what store close by carries it. Once in the store, a cute Coolhobo virtual assistant pops up in the app to direct the user to the product’s location. Customers point their camera at the item and an array of floating information flitters about on-screen about the product, including nutrition and preparation information, as well as reviews from other customers.

Show is better than tell, so here’s a Coolhobo demo video that walks you through the experience:

AR Shopping by Coolhobo, Adding features and testing new UI

Kobes said that Chinese consumers really like imported products, but there isn’t a lot of information about them. Coolhobo provides that information, and can help customers choose between two similar imported items. But even once they choose something, “People need help to understand how to prepare it, how to cook it,” said Kobes.

Kobes described Coolhobo as a B2B and B2C company, and said he’s in partnership talks with a number of Chinese and international retailers. He said the app is scheduled to launch in June or July. Kobes took an earlier version of Coolhobo through the SOSV accelerator program, and is currently raising a seed round of funding.

According to Kobes, China is far ahead of the U.S. when it comes to shopping experiences, and, in particular, integrating mobile technology into them. This summer, we’ll see if Coolhobo’s augmented reality will be enough of a carrot for stores and consumers to sign up for the experience.

Podcast: A Chef’s Journey To The Intersection Of Virtual/Augmented Reality & Food

Ever since I saw Chewie and CP30 playing hologram chess in Star Wars as a kid, I’ve been intrigued by the idea of creating virtual images and worlds.

A generation later, I more fascinated than ever by what we now call augmented and virtual reality. I’m especially intrigued about where these new technologies intersect with food, and a week doesn’t go by where I read about an innovator creating a new way to enhance the shopping, restaurant or cooking experience with AR or VR.

Another person excited about this fast growing space is Jenny Dorsey. A year ago, the professional chef had an epiphany: she needed to become the foremost authority on nexus point between AR/VR and food.

On the podcast, I catch up with Jenny to hear how her journey to become the go-to expert in this exciting area is going and learn about some new and interesting ways that augmented and virtual reality are changing food.

You can listen to the podcast below, download here or find it on Apple podcasts.

Campbell’s Evaluating Augmented Reality and Other Tech As It Navigates Digital Transformation

While cans of Campbell’s Soup may not look much different today from when you or I grew up, don’t be fooled. Executives at the storied food brand are spending lots of time nowadays figuring out how to navigate a fast-changing market increasingly disrupted by e-commerce, IoT and other digital technologies.

So at last month’s Smart Kitchen Summit, we decided to ask Campbell’s Matt Pritchard about the company’s digital transformation. As Campbell’s VP of Digital Marketing, Pritchard is one of the executives responsible for charting the course of the company as it prepares for the future.

One thing the soup company exec made clear is that need for Campbell’s to stay close to the consumer no matter what the technology.

“What we got to figure out is how we deepen our consumer connection and how do we become more meaningful in their life.”

That connection, according to Pritchard comes across the entire meal journey, whether that’s during planning, shopping or making meals.

“Where we got to focus on is what is the desired consumer journey, where they are trying to get to, and how do we create integrated experiences to play after that.”

Some of the ways in which Campbell’s touches the consumer through digital are expected, such as recipe apps. But some ideas suggested by Pritchard hint at some interesting potential directions for the brand and CPG products more broadly.

“Some could be a communication challenge where we make clear we are in recipe collection apps. But it could quite easily be in the center store where we’ve got those rows and rows of cans of soup, how can we create an experience with augmented reality that brings the nutrition panel to life.”

We’ve seen a variety of retailers embrace augmented reality as of late as the technology matures, interest from the brands themselves could extend the reach of the technology from store shelves into the home and around the cooking experience.

And it’s in the kitchen where Campbell’s sees much of their future technology evolution.

“Being in the heart of the connected kitchen is the purpose of today and one of those things we need to figure out,” said Pritchard.

What would that look like? One possible way is through smarter inventory management and replenishment.

“A consumer goes home, put some of our products in the pantry, then they use those products,” said Pritchard. “We’ve got to figure out how do we know they used those products so we can help with a replenishment scenario. So, I’m looking at things like sensor companies because they can figure out when a product has been in a fridge or a pantry.”

So while the soup may stay the same, the company behind the red and white cans is busy figuring out how to stay connected with consumers as the food journey becomes increasingly digital.

You can watch the entire interview with Matt Pritchard below: