Close Ad

The Spoon

Daily news and analysis about the food tech revolution

Interfaces

Newsletter: The Spoon’s Food Tech 25 Is Here. So Is the Battle for the Drive-Thru.

This the post edition of our newsletter. To get the Weekly Spoon delivered to your inbox, subscribe here. 

One of my favorite things about tech is that it starts a lot of debate. Even within our small team here at The Spoon, we’re constantly on different pages about what’s groundbreaking and what’s just hype, whether something’s progressive or just invasive, how to spell the phrase “food tech.”

So when it came time to put together our annual Food Tech 25 list, which dropped yesterday, you can bet it took a whole lot of discussion to whittle the entire food industry down to just 25 companies.

As we always do, though, the Spoon team — Mike Wolf, Chris Albrecht, Catherine Lamb, and myself — managed to compile a list of companies we individually and collectively, believe are truly impacting the human relationship to food. That impact takes many forms, from the way Creator makes it possible for humans and robots to coexist in the kitchen to Yo-Kai’s vending machine of the future to Goodr’s efforts to use tech to keep food out of the trash and redistribute it to those in need.

I’m hoping readers enjoy this list, but I’m also hoping it sparks some healthy dispute, too. Who else should be on the list? For that matter, who shouldn’t, and why? We encourage you to email us with any additions, subtractions, rants and raves on the matter.

And, most important, congratulations to the companies who made it on this list!

Image via Unsplash.

Drive-Thru Tech Moves Into the Fast Lane

One area of food tech that’s going to raise many more questions over the next few years is the QSR drive-thru. Specifically, how AI is changing the drive-thru and what that means for both restaurant operators and customers.

We’ve been following closely the story behind McDonald’s acquisition of Dynamic Yield, a New Zealand-based AI company whose tech has already been rolled out to almost 1,000 Mickey D’s drive-thru lanes. Then, this week, Clinc, best known for its work in the financial sector, announced a new funding round that will allow the company to expand into other markets with QSR drive-thrus at the top of the list.

Clinc’s using AI-powered voice controls to facilitate more natural conversation between the customer and the ordering system in the hopes of making the drive-thru experience smoother and faster. Drive-thru order times are much longer than they used to be, and companies are betting AI will speed up the order process by making it more accurate and also making more personalized recommendations, like immediately suggesting a pastry to someone when they place their morning coffee order. There are even companies working on making those recommendations not just in real time but also based on existing customer data. One such company is 5thru, which does away with voice altogether by scanning your license plate number, which is attached to a profile stored with the restaurant and can make real-time recommendations based on your existing preferences and order history attached to that license plate number. Cue progressive-versus-creepy debate.

Join the Conversation at The Spoon’s New Food Tech Fireside Event

As much as we value the sound of our own voices over here, though, we actually want to hear more from readers on their thoughts around tech. That’s why we started a new online event, The Spoon’s Food Tech Firesides. Every month, we’ll hold a virtual sit down with one or two food industry innovators and invite the audience to join in the talk via written questions.

First up will be Tessa Price of WeWork Food Labs and Peter Bodenheimer from Food-X talking about food accelerators: what they are, what they’re not, and which companies and entrepreneurs should consider them as a path towards growth.

The event takes place May 30 at 10:00 a.m. PDT/1:00 p.m. EDT. Catch the full details here, and be sure to register early, as there’s limited space available.

May your week be filled with lively debate.

Onwards,

Jenn

Google Lens Could Make the Restaurant Experience Super Convenient — or Super Predictable

Yesterday at its I/O conference, Google announced new features to Google Lens, its image-recognition app for Android, and they’re all about improving the restaurant experience for customers.

Google Lens uses machine learning, computer vision, and a whole lot of data to interact with the world around you and answer your questions. For example, when Lens launched in 2017, it was touted as a way to instantly translate another language.

According to a blog post from Google published yesterday, the new features for Google Lens “provide more visual answers to visual questions.”

Like what’s popular on your restaurant menu right now. Within the Google Lens app, when a user snaps a photo of the menu and taps on an item, Lens automatically pulls up relevant information, like a photo item description, and reviews — data Google already has thanks to its Google Maps-Yelp integration.

Lens is basically automating something most of us have done at least once while out at a restaurant, which is see another table’s food arrive and, intrigued, ask them what it is. But asking Drew at the next table what she ordered only offers so much information. After all, Drew hasn’t tasted the food yet, and her preferences could be completely opposite of yours.

What Lens appears to be doing with this new feature is taking most of that guesswork out of the ordering process by not only matching a photo with its name and description, but also aggregating reviews, so a user can get a clearer sense of how the dish tastes. If nine people say the dish was super bland, those who prefer a little more kick to their meals might order something else.

It all sounds wonderfully convenient for us consumers. For example, it could be a valuable tool when you’re trying out a new type of cuisine and have no idea where to start.

What I wonder, though, is how this will affect menu planning for restaurants. On the one hand, it could provide valuable information for restaurants when it comes to figuring out what is and isn’t selling on the menu, so chefs and operators could better adjust their planning and inventory (potentially helping them avoid food waste and keep costs down).

But what this will do to the adventure of going out to eat? Part of the fun of the restaurant experience is the guesswork, which would be gone were we to rely too heavily on data-driven recommendations. This seems unlikely at higher-end restaurants and places designed for adventurous foodies, with robust appetites for the unknown. For all the places in between, though, too much knowledge might make the restaurant experience just a little too predictable.

Alibaba Wants to Help You Find Restaurants and Order Food From Your Car

E-commerce giant Alibaba announced today at the Shanghai Auto Show it is developing apps for connected cars that will allow drivers to find restaurants, get “in line” for a table, and even order food. As TechCrunch reports, users will also eventually be able to “complete a plethora of other tasks” with the same controls using voice, motion, or touch control” from their vehicle.

These so-called “lite” apps are smaller to build than regular standalone apps and come bundled with Alibaba’s all-in-one digital wallet Alipay, which is the most used digital-pay service in China. All apps included in connected cars must meet specific criteria around safety as set by the auto industry and will run on Alibaba’s operating system, AliOS.

In-car commerce, especially around restaurants and food ordering, isn’t yet a widespread phenomenon, but that could change, and quickly. In the states, Domino’s has been testing different versions of its Anyware digital-ordering platform in cars since 2014. And GM’s Marketplace in-car system lets users order and pay for food from chains like Dunkin’ and IHOP.

Cars, meanwhile, are only one of the ways Alibaba is adding disruption to the food industry. Last summer, it opened its automated Robot.he restaurant at the Hema supermarket. The restaurant uses a combination of apps, QR codes, and robots to provide what my colleague Chris Albrecht called “a futuristic dining experience.” The company also unveiled a delivery robot as well as an expanding smart locker, both of which are well-suited to food delivery.

As yet, there’s no timeframe for the in-car apps, though TechCrunch reported that Alibaba is “already planning for a launch.” When Alibaba relaunched AliOS in 2017, the company made it clear it was out to make car software, not the cars themselves. So this in-car commerce news is likely just the start of what will, for China, at least, be many, many in-car app offerings to come.

You’ll Soon Be Able to Order Domino’s Pizza From Your Car

Domino’s announced this morning it will launch its Anyware digital-ordering platform in cars in 2019. To do so, the pizza chain-turned-tech trailblazer has teamed up with Xevo, whose in-vehicle commerce technology is currently in about 25 million cars.

This is actually not the first time the Anyware platform has made its way into a car. In 2014, Domino’s worked with Ford Motors to bring voice order to the Ford Sync vehicle. That initiative was slurped up into Anyware when the latter launched in 2015 and is still available today.

With the new in-car app, customers use the car’s touchscreen to find their local store, order, and track the pizza. Voice-ordering will also be available. According to a press release, the feature will be automatically loaded onto cars with Xevo platform starting “in late 2019.” While the release didn’t state which car brands this includes, Xevo already works with Buick, Cadillac, Chevrolet, and GMC, so I’d expect models from those companies to be on the list. Xevo also partnered with Hyundai in 2018 to allow customers to order and pay for meals from Applebee’s, so this isn’t its first go at a QSR partnership.

For Domino’s, the Xevo partnership seems like another stop on Domino’s quest to seemingly try out every new technology it possibly can for delivering pizzas. The list of channels from which Domino’s customer can use the Anyware platform keeps growing: phones, smart watches, TVs, Alexa and Google Home devices, Slack, Facebook Messenger, and Domino’s own Zero Click app. The chain also delivers to HotSpots, which are “non-traditional” locations like beaches, parks, and probably even the zoo. And Domino’s launched a separate partnership with global addressing platform what3words earlier this year, to use the latter’s technology in countries and regions that lack a more straightforward address system.

At one point, Domino’s was the only pizza chain around trying out new technologies left and right, but times have changed. Pizza Hut recently partnered with FedEx to use its autonomous bot to deliver pizzas, and even has some weirder projects in the works, like the autonomous pizza factory on wheels the company unveiled in 2018. (It’s still a prototype.) Papa John’s, who has weathered a good deal of trouble in the last year, got a $200 million investment in February. The company hasn’t said yet what the money will go towards, but if it wants to keep up, a little tech innovation will probably be part of its plans.

The only bummer about the Domino’s-Xevo deal is that you still have to either pick the pizza up at a store or get home by the time the pie arrives. My guess is that will change quickly, and Domino’s will either integrate its HotSpots into car ordering or even use the what3words’ tech. You have to figure that, with its many tech initiatives and a platform called Anyware, Domino’s is aiming to eventually deliver everywhere.

MealMe’s Instagram-Meets-Yelp-Style App Helps You Decide What’s for Dinner

The average consumer nowadays will probably use multiple different apps to figure out where to eat on any given night: discovery, recommendations, booking a reservation or ordering food for delivery. As the folks at Hypepotamus recently noted, a new company, MealMe, wants to simplify the food lover’s food journey by bringing its many pieces into one single app.

Matthew Bouchner and Will Said, both students at Emory University, created the MealMe app as a kind of Instagram-like social network for food that also integrates table booking and ordering capabilities. The idea is to offer a one-stop-shop for all your food needs by aggregating the many disparate parts of the food journey into a single interface.

When a user opens the MealMe app, they see an AI-powered feed that combines Yelp’s data (location, cost, etc.) with user-generated data such as comments and likes, and pulls the most relevant results to that user. So if you live in Indianapolis and tend to order a lot of Thai food, the app will pull up restaurant results relevant to that criteria. The app also pulls in Yelp’s mapping feature, so you can see where your restaurant of choice is and your distance from it.

Image via MealMe.

Users can also snap photos of their meals and share them, much like they do on Instagram, and restaurants can do the same to promote specific items. When a user clicks on a photo, they are given the option to access services like Opentable or Postmates, to book a reservation or order delivery.

The visual aspect of this food journey is especially important to the MealMe folks. Research has shown that the visual cortex of the brain plays a much more significant role in decision making than was previously thought, and can impact whether or not we choose an item from a menu. So in addition to making the logistical side of “what’s for dinner” easier, MealMe also bills itself as a place for people who also just “love looking at food porn,” an activity that’s so popular it’s a bonafide industry.

In fact, multiple companies are now zeroing in on the visual aspect of the food industry and trying to turn it into a business. Wine n Dine, which like MealMe lets users order from within the app, is a food-porn-lover’s dream that shows a feed of pictures a la Instagram and even breaks pictures down into cuisine. And Pinon is another group of students, this time from the University of Massachusetts, who have created “smart” menus that show videos and interactive pictures of food items, and also lets users order from within the app.

Bouchner and Said have been at work on the MealMe app since 2018; it’s currently in beta, with a launch planned for later this year and is currently free to use. The company has said it makes money by taking commissions from delivery orders, offering targeted ad space, and selling premium data packages to restaurants and CPGs. Right now the company is currently bootstrapped and seeking a $120,000 seed round.

Drop Wades Into Kitchen Appliance Voice Control with Siri Integration

Kitchen tech company, Drop, announced today that it will be launching its first foray into voice control of kitchen appliances via its new Siri integration.

Voice control is a particularly interesting interface in the kitchen, where sticky fingers and loads of wet ingredients aren’t great for touching devices like iPads and smart screens. Being able to “talk” to your oven or your sink while knead-deep in dough, promises to make cooking more efficient.

To that end, Drop, which up to now has offered guided cooking recipes and remote control of select appliances via phones and tablets, has started the process of adding voice control. Interestingly, the first voice integration isn’t with the omnipresent Alexa, or even Google Assistant, it’s with Apple’s Siri. A Drop corporate blog post explained the decision, saying that the company had previously worked with and had good experiences with Apple and that Siri’s voice command recognition was superior to either Google or Alexa’s.

The blog post also explained that user and data privacy was a priority for Drop, with the company even invoking GDPR:

Also, unusually among big tech companies, Apple has maintained a firm standpoint on user privacy. Whereas assessing and analyzing user data from the cloud allows other companies (most notably Google and Facebook) to improve their AI capabilities, it does so at the expense of user privacy. With Apple, all processing of Siri shortcuts is implemented directly on the device, rather than by sending sound-bytes from our homes out to the cloud.

Security is something which has been, and will always be, of utmost importance to us at Drop. An example of this was achieving GDPR compliancy well ahead of schedule, going to great lengths to restrict data collection and access internally. We also have a rigid approach to building infrastructure and implementing and enforcing security measures.

In addition to rolling out on just one platform, at first Drop’s voice control will only make you coffee, and that coffee has to be made with Bosch Coffee Machines equipped with Home Connect technology. As you can see from the video below, once set up, you can just have to say “Hey Siri, make me an espresso” and it automatically fires up the Bosch coffee maker. Drop said it will be adding voice controls to more recipes that will work with more appliances.

Drop Test Lab: Making an espresso with Drop Recipes and the Bosch Coffee Machine

While it probably won’t go mainstream this year, voice control is becoming more central to the kitchen experience. GE Appliances and Electrolux expanded their Google Assistant capabilities last summer, LG’s Thinq appliances work with both Alexa and Google Assistant, Drop rival, Innit is working with Google, and Amazon built its own Alexa-powered microwave. Drop’s adoption of Siri is a nice feather in the cap for Apple’s assistant, and further evidence that voice control will soon become ubiquitous in our appliances and apps and throughout our homes.

Will Wearable Tech and Artificial Intelligence Help or Hinder Restaurants?

Restaurant tech company Presto dropped a couple new products today for front-of-house operations in the restaurant: Presto Wearables and Presto A.I. These join Presto’s tabletop terminal, on which guests can order, pay, report feedback, and contact the manager if needs be.

According to a press release, Presto Wearables are smartwatch-like devices that notify the servers wearing them of a customer’s needs (e.g., refill table six’s Diet Coke). In the event of a bigger issue, guests can also notify the manager via the wearable device. Presto A.I., meanwhile, does real-time data analytics and predictive modeling, both of which typically improve things like inventory management and labor costs.

Presto launched in 2008 after founder Rajat Suri — also a cofounder of Lyft — dropped out of MIT and spent a year waiting tables and testing a prototype device. The ensuing tabletop terminal, dubbed PrestoPrime, allows guests to order, pay, play games, and leave feedback, and is currently at major chains like Outback Steakhouse and Applebee’s.

One thing Suri highlighted was the “easy to use” aspect of the new offerings, which brings up a certain issue. Recently, I had a conversation with someone in the restaurant industry about the burden GMs now shoulder of having to not just manage a restaurant but also act as de facto IT person for the many devices and software systems now part of a restaurant’s operations. Which is to say, wearable tech and A.I. sound great in theory, but it’s too soon yet to tell whether these new offerings will be a blessing or another tech burden for GMs to wrestle with. Plus, rating a server via tablet has gotten an understandably bad rap; I can’t imagine expanding that capability to a device the server is wearing will make the idea any more palatable.

As for AI, it’s currently sweeping the restaurant industry, from voice ordering to more robust POS systems to facial recognition. Given that, Presto will definitely face competition in this area.

The company may stand out more on the wearables front, since that’s an area that hasn’t really entered the restaurant industry as of yet. Oracle’s Restaurant 2025 report from last year found that only about half of restaurant operators surveyed found the idea of wearable tech appealing; fewer than half of consumers surveyed said they would use the tech for things like ordering.
 
Presto’s device isn’t yet primed for ordering capabilities, but it does seem like the logical next step. And if Presto’s watch-like device can grab the same high-profile chains its tabletop device did, we may start to see more wearables on the dining room floor in future.

Lifesum Unveils a Google Assistant Version of Its Health-Tracking App

It being the time of year when most of us set some health-related goals, Lifesum picked an apt time to unveil its latest offering. Yesterday, the Swedish company officially unveiled a Google Assistant version of its nutrition app, which allows users to track meals, weight, and water intake using their voice instead of a phone or watch.

The Lifesum app, which has already amassed a large following for its Android and iOS versions, uses a combination of data and motivational psychology to help users track their food intake and meet goals around their health, like losing 10 pounds or learning to like broccoli. Upon signing up, users take a brief test to help them choose an appropriate diet plan (e.g., keto, high protein, etc.), then track protein, fat, and carb intake for each meal via the app. A weekly score tells the user how on-track their food consumption is with their set goals, and offers tips for improvement. At just over $3/month, it’s also a pretty good bargain for the number of features you get.

The obvious plus of combining an app like this with Google Assistant is that one need not pull out their phone and input every single meal or glass of water they want to track. You simply tell Assistant to talk to Lifesum when you want to log items. The Lifesum app will also issue challenges based on a location you give it (this is completely opt-in). If you’re in your kitchen, for example, it might tell you to hide sugary foods.

That said, Lifesum’s Google Assistant app isn’t yet as robust as we expect it to become in future. Right now it’s really more of a companion to the mobile versions of the app. You can tell Google Assistant to track meals, but to record the specific foods and other details, you’ll have to go into your phone later. So for now, at least, the Lifesum app simply acts as a placeholder. If you don’t have time to record all the details of your enormous supper directly after the meal, you can just say “track a large dinner,” then fill in the details once you have time. Also, since Google Assistant’s usefulness varies based on location, using the Lifesum app in certain scenarios, like a loud restaurant, would be difficult.

The global fitness app market is predicted to grow to $2 billion by 2023. Lifesum raised $10 million in 2016, and there’s a lot of other capital going around this space. Nutrition and fitness training app Freeletics closed a $45 million Series A funding round in December. MyFitnessPal, who was acquired by Under Armour back in 2015, is another popular food- and fitness-tracking app out there.

Lifesum for Google Assistant is only available in English at the moment. The company indicates it will eventually bring full functionality to the Assistant app, and welcomes suggestions for features from current users.

I’m not sure how useful this version of the app is right now, given its current limitations and the fact that it’s not that hard to take your phone out of your pocket and put some numbers in. Nonetheless, it’s nice to see some companies experimenting with how voice command can play a role in keeping track of what you eat and setting realistic goals around that.

MAC’D is Rolling Out eatsa’s Automation Concept in Its Stores

Fast-casual chain MAC’D announced yesterday it has partnered with eatsa to bring the latter’s tech platform into its restaurants. Doing so will automate much of the restaurant experience for MAC’D customers, who can build their own mac-n-cheese dishes via kiosk and pick them up in designated cubbies.

Back when it was still an actual restaurant chain, eatsa got a lot of press for its automated front-of-house experience: customers could order their meals via a touchscreen kiosk, then wait for the food to appear in a designated cubby a few minutes later. But in October of 2017, eatsa shuttered most of its locations and pivoted to a new approach: using its technology to power other restaurants. A partnership with Wow Bao was announced not long after, and obviously liking how it worked, Wow Bao opened a second automated location.

The eatsa system manages a restaurant’s entire digital ordering process. Besides the aforementioned kiosks and cubbies, the eatsa platform also synchs with the back of house, speeding up order fulfillment and also giving restaurant operators access to real-time customer data. The eatsa site claims this combination leads to “lightning-fast fulfillment times.”

As Michael Wolf wrote about eatsa, “this is the future of fast food and fast casual,” especially in large cities with busy lunch hours. But the other thing eatsa has going for it is that it’s, well, fun. The front-of-house kiosks are so low friction that they make choosing food and customizing an order quick and enjoyable, rather than intimidating and frustrating (as is sometimes the case with self-serve kiosks.) And who doesn’t want their food to appear seemingly magically in a futuristic cubby with their name on it?

MAC’D said via press release that it will use the full eatsa platform, which includes the kiosks, cubbies, status boards, back-of-house software, and a mobile app. The chain will roll that out first in its Polk St. store in San Francisco. Meanwhile, for those who still want the full (and quinoa-centric) eatsa experience, the company still operates a couple of stores in San Francisco.

The whole touchscreen-to-cubby concept will doubtless become old-hat at some point, which will normalize the eatsa-like dining venture. Getting there will be about more than just placing kiosks in the front of house, though. Restaurants — at least QSR’s and fast casual ones — will need to treat touchscreens, management software, and magical cubbies not as a bunch of new, disjointed technology, but as a unified suite that links the customer to their eating experience.

Video: Google’s Duplex AI Assistant Makes Restaurant Reservation

When Google debuted Duplex this past summer, the virtual assistant made headlines because of the eerily natural way in which the artificial intelligence (AI) interacted with humans. The company showed Duplex purportedly making a restaurant reservation that was complete with saying “ummms” and “ahhhs” as it was “thinking.”

Duplex is now being rolled out into the wild on “select” Google Pixel phones. Over the holiday break, Venturebeat reporter Emil Protalinksi recorded a video of Duplex making a reservation with a human host at Cafe Prague in San Francisco. Check it out below.

Google Duplex: AI assistant makes a restaurant reservation

Despite some overly-long pauses, the result is a very natural-sounding conversation between a human and a computer (even on speaker in a noisy restaurant!). It’s quite impressive, and if you were just hearing the audio, you’d be hard-pressed to tell it wasn’t a person.

Protalinzki’s Duplex call starts off saying that it is calling to make a reservation for a client, that the call was coming from Google and may be recorded. This is, no doubt, something Google added in response to the backlash it received when it first showed off Duplex. There was a lot of hand-wringing over the ethical issues surrounding an AI assistant “tricking” humans on the other end of the line.

Restaurant workers we spoke with at the time said they wouldn’t care if it was an AI calling to make a reservation… as long as it didn’t take any longer than dealing with a human. They don’t want a dumb AI gumming up their workflow, taking too long to figure out the answer to a simple question.

The VentureBeat video didn’t show some of the more complicated questions that an AI assistant will have to deal with; questions other than just what time and how many. For example, the restaurant host may ask if there are any food allergies or if there is a special event being celebrated.

And as I wrote earlier, with robots becoming more normal in restaurants, it’s only a matter of time before eateries adopt their own AI assistants. We are already seeing that type of technology being developed by companies like Clinc, which makes conversational AI assistants for QSR drive-throughs.

At some point reservations will happen in the background as your AI assistant talks to the restaurant’s AI assistant to make your reservations. Hopefully they won’t gossip about what I order.

Square Launches a New Platform to Make Tableside Payments Faster and Cheaper

Just a couple weeks after launching Square Terminal, its new all-in-one payment device, Square today announced via press release an update that integrates that hardware with its restaurant POS system, Square for Restaurants. By combining the two, Square hopes to speed up the process for guests when it’s time to pay the bill and save restaurants time and money.

Square Terminal — which any retailer can use by itself — allows businesses to process multiple payment methods with the same terminal. A single device can accept chip cards, those with a magnetic stripe, and payment from mobile devices via Apple Pay, Google Pay, and other NFC-enabled methods. Businesses are at an advantage because they pay one flat rate for each payment. They also avoid certain fees that come with other terminals, like a security fee.

Square Restaurants, meanwhile, centralizes restaurant operations into a single platform, so that tasks like table reservations, order taking, and menu updates can be managed from the same interface. The platform launched in May.

Beginning today, any restaurant using Square for Restaurants, whether sit-down, quick-service, or a mix, will be able to use a Terminal device to access all of its checks and let the guests pay tableside. The server only has to walk the device over to the table, where guests will swipe, insert, or tap, then enter a tip digitally into the machine.

“In a restaurant, one of the few times a diner looks at their watch is when they’re ready to pay and leave. Taking payments tableside with Square Terminal helps speed up that process,” Naama Tamir, owner of Lighthouse restaurant in Brooklyn, said in the press release.

It also helps cut down on potential confusion that can happen when credit cards travel from table to server station, or when guests ask for multiple checks on a single table and all those cards have to be kept track of. I remember one time, back in the pre-iPhone days, being asked to split the check on a fifteen-person table. Square’s new platform would have come in handy, as the process took upwards of 20 minutes and made everyone involved mad.

Square also handles PCI compliance, fraud prevention, check disputes, and multiple other payment issues restaurants have to contend with. Square terminal costs $399 per device, while Square for Restaurants is $60/month for a single restaurant location.

Will BSH’s PAI Usher In The Era of the Kitchen Projection Interface?

The idea of using your countertop as a touchscreen interface has been something big tech and kitchen appliance makers have been playing around with for much of the past decade.

First there was Whirlpool’s attempt in 2014:

IKEA served up the idea for its Kitchen 2025 concept a year later:

IKEA Concept Kitchen 2025

And Bosch has been showing off things like this coffee robot with a projection interface for a few years:

Spotted at #ces2017: coffee robot at the @Bosch booth.

A post shared by Michael Wolf (@michaelawolf) on

And this year it looks like the large German appliance conglomerate, BSH Appliances (the company behind the Bosch, Thermador and Gaggenau appliance brands, to name a few), is showing off what looks to be a more evolved version of the projection interface in PAI at IFA in Berlin.

PAI, which stands for ‘Projection and Interaction’, is a system that projects an image onto a flat surface to create a virtual interactive interface for the kitchen.  While the projector incorporates a camera, a speaker, a microphone, two USB ports, WiFi and Bluetooth antennas, the key technology here is a 3D sensor that detects minute movements of fingers on the surface.

According to project manager Markus Helminger, the PAI 3D sensor powers a projection interface that can “be perfectly operated even with dirty fingers and occupies no space on the work surface, so that consumers have enough space for cooking or baking.”

While other efforts at projection interfaces at trade shows have largely been to show off the concept with no concrete plans for commercialization, this time things look different with PAI. According to a German language post about PAI by on the BSH Kitchen Stories blog, they plan on rolling out PAI in February 2019 in China. While there’s no indication as to when we might see the technology in Europe or the US, my guess is we could see the technology in product rolled out in Europe as early as next year.

You can see a demo of the PAI interface (in German) below courtesy of Computer Bild TV:

Bosch PAI: Projektor für die Küche vorgestellt!

The story behind PAI is an interesting one. The technology spun out of development work Bosch was doing in ventilation. According a company spokesperson, researchers were looking to improve the user experience for cooks and “the developers wanted to create an assistance function for an extractor hood that would make it possible to project images from the hood and display recipes on the work surface. In an extensive UX study carried out by Bosch, this idea went down so well that the project was actually carried out.”

According to the company, consumer testers almost universally said they use tablet or smartphone when cooking or baking, but they didn’t like giving up the counter space to these devices required and, perhaps more importantly, they worried touching these devices with dirty hands. As the company worked on the concept more, they eventually decided to not incorporate it into a vent hood but to make the PAI a standalone projection system that allowed the consumer to place it where they desired on the counter.

The company has also integrated the PAI with its Kitchen Stories guided cooking system and its Home Connect platform, which opens up some intriguing possibilities. It’s not hard to envision a Kitchen Stories guided cook experience that shows step-by-step instructions projected onto the kitchen counter. With Home Connect, PAI could also project virtual start buttons, timers and other ways for the consumer to interact with their appliances.

With BSH Appliances – one of the world’s biggest appliance companies – taking projection interfaces seriously, my guess is we’ll likely see other big appliance brands push forward with their own projection interface commercialization efforts in the coming year and we’ll most likely see some of these teased at CES in Las Vegas next January.