Close Ad

The Spoon

Daily news and analysis about the food tech revolution

Interfaces

Weekly Spoon: Kitchen Projection Interfaces, Amazon Drone Patent & Innit Nabs Arçelik

This is the post version of our weekly newsletter. If you’d like to get the weekly Spoon in your inbox, you can subscribe here.

If you go to lots of trade shows like me, you know it takes time for innovation to make its way from the show floor into our living rooms.

We’ve all seen this with technologies like virtual reality and 3D food printing; only after years of development and iteration cycles do we get to the point where a product is ready for prime time.

And then there are technologies like projection interfaces that – up until now at least – seem like they’re stuck in development stasis. The idea of a projectable, anywhere surface interface has been discussed for close to a decade in the research and academic community and started showing up on trade show floors about five years ago. Despite this, the concept never seemed to go beyond an occasional product demo.

So last year I started to wonder why exactly the technology hadn’t reached consumers yet. After all, with the likes of Whirlpool, IKEA, and Bosch showing off jaw-dropping demos, it only made sense this technology would find its way to market at some point.

While there’s no clear answer, I narrowed it down to a couple of factors. First, the reality is the technology still needed some refinement to make it both consumer-ready and affordable. Second, appliance vendors often wait for big-tech to take the first leap, and from what I could tell none of the big-tech 5 (Google, Apple, Facebook, Microsoft, and Amazon) seemed particularly active with projection interfaces.

But now, it looks like one big company is committing to bringing a projection interface to market. BSH Appliances showed off a new product concept called PAI at IFA in Berlin this past week. PAI is a movable projector interface that is designed to go on a countertop in a kitchen.

I’m excited that BSH looks to be serious about bringing PAI to market. While some see voice as the dominant user interface of the future, abysmal usage rates of Alexa skills for commerce have shown us that voice in itself isn’t enough.  Consumers are visual, and most things we do in the kitchen are multimodal. Because of this, I believe the dynamic projection interface could a vibrant area in UI development over the next decade.

BSH Appliances wasn’t the only one to make news at IFA last week. Sharp showed up in Berlin with a fridge that featured a built-in vacuum sealer, prompting Chris Albrecht to wonder what exactly types of features he would want in his next fridge.

Smart kitchen platform startup Innit had a strong showing at IFA, popping up in a number appliance OEM booths, including that of a new partner in Arçelik.  The Turkish appliance conglomerate behind the Beko and Grundig brands showed off an Innit integration to power guided cooking as part of its HomeWhiz smart home platform.

It wasn’t all IFA this week. This week the Spoon scooped yet another Amazon patent in which the Seattle tech giant shows an innovative new method for delivery drones to generate power while in transit by harnessing both wind and kinetic energy.
After the news of a $30 million funding round for Farmer’s Fridge, I’m looking forward to a panel at Smart Kitchen Summit on the future of lunch. We’ll be discussing how new technologies, business models and delivery formats are all colliding to change what we eat every day, including at work.  You can read Chris’s piece on Farmer’s Fridge, a startup we’ve been covering for the past year.

Speaking of Smart Kitchen Summit, we’re less than five weeks away from our flagship show exploring the future of food and cooking. We have an amazing program planned, and not only will we have executives from big appliance brands like Whirlpool, GE and more, but we’ll also have startups from the smart kitchen, food robotics, restaurant tech, retail and more to discuss and showcase how each of these industries are changing to disrupt the consumer meal journey.  Make sure to get your tickets now and use the discount code NEWSLETTER (You can also use this link which has the code already applied).

That’s it for now. Have a great rest of the week.

Mike

P.S. Make sure to tune into this weekend’s episode of the Smart Kitchen Show podcast, where I have a great conversation with the CEO of the startup behind what is arguably the most successful consumer food robot ever made. You won’t want to miss it!

In the 09/06/2018 edition:

Ordermark Raises $9.5 Million for its Online Order Management Tools

By Chris Albrecht on Sep 06, 2018 10:23 am
Ordermark, a startup that helps restaurants unify and organize online orders, today announced that it has closed a $9.5 million Series A led by Nosara Capital. This brings the total amount raised by the company to $12.6 million.

Nima Peanut Sensor Now Available, Gluten Sensor Selling at Select CVS Stores

By Chris Albrecht on Sep 06, 2018 06:00 am
It’s back-to-school time, which means my son is once again eating lunches in a cafeteria. I doubt his school is unique, but it actually has separate tables for kids who bring in peanut butter sandwiches.

With its own Grocery Delivery Service, Walmart Grabs More Data

By Chris Albrecht on Sep 05, 2018 04:00 pm
Walmart is leaving no stone unturned when it comes to getting you your groceries. As of today, that includes testing out its own delivery service (h/t Food Dive). The retailing giant announced a pilot program for its new last-mile delivery service, dubbed Spark Delivery, which will deliver groceries directly to customers’ front door.

For Goodr’s Jasmine Crowe, Blockchain Is a Key Piece to the Food Waste Puzzle

By Catherine Lamb on Sep 05, 2018 02:00 pm
Food waste is generating quite a lot of interest as of late; but one buzzword that might give “food waste” a run for its money is blockchain.

Innit Adds Arçelik To Growing List of Appliance Partners

By Michael Wolf on Sep 05, 2018 12:00 pm
The smart kitchen was everywhere this year at IFA, Europe’s big appliance and tech expo, and one company that seemed to be on everyone’s dance card was Innit.

Farmer’s Fridge Stocks up with $30M

By Chris Albrecht on Sep 05, 2018 10:54 am
Farmer’s Fridge, the company which makes vending machines that dispense healthy meals such as salads and protein bowls, today announced that it has raised a new $30 million round of funding led by Innovation Endeavors. This brings the total amount raised by the company to $40 million.

Seltzer? Sous Vide? Smart Apps? What Cool Things do you Want in a Fridge?

By Chris Albrecht on Sep 05, 2018 07:33 am
For those old enough to remember, there is an episode from season two of The Simpsons where Homer designs a car for the average American. The result, as you can imagine was a hodge-podge monstrosity that featured bubble domes, three horns, shag carpeting and cost $82,000.

Bear Flag Robotics Raises $3.5 Million for Autonomous Tractor Tech

By Chris Albrecht on Sep 04, 2018 10:00 am
The common refrain from robotics companies is that they help with manual, repetitive tasks. And when you run a farm, there are plenty of manual, repetitive tasks, and Bear Flag Robotics raised $3.5 million seed funding right before the holiday weekend to help agricultural workers out with them.

Amazon Patent Points to In-Flight Recharging For Delivery Drones

By Michael Wolf on Sep 04, 2018 06:54 am
Ever since Jeff Bezos teased the idea of drone deliveries on 60 Minutes in 2013, the tech world has been abuzz with the idea. At the time Bezos said that the reality of drone deliveries wasn’t there yet, but he thought it just might be in 4-5 years.

FoodPlus Sells Surplus Food (in Slovenia) so it Doesn’t go to Waste

By Catherine Lamb on Sep 04, 2018 06:00 am
Based in Slovenia, FoodPlus began in 2015 when co-founder Dalibor Matijevic began searching for a way to cut down on food waste by redistributing surplus food. He developed a B2B platform for companies to buy and sell extra food at a super low cost — creating a new revenue stream and keeping food out of landfills.

Will BSH’s PAI Usher In The Era of the Kitchen Projection Interface?

The idea of using your countertop as a touchscreen interface has been something big tech and kitchen appliance makers have been playing around with for much of the past decade.

First there was Whirlpool’s attempt in 2014:

IKEA served up the idea for its Kitchen 2025 concept a year later:

IKEA Concept Kitchen 2025

And Bosch has been showing off things like this coffee robot with a projection interface for a few years:

Spotted at #ces2017: coffee robot at the @Bosch booth.

A post shared by Michael Wolf (@michaelawolf) on

And this year it looks like the large German appliance conglomerate, BSH Appliances (the company behind the Bosch, Thermador and Gaggenau appliance brands, to name a few), is showing off what looks to be a more evolved version of the projection interface in PAI at IFA in Berlin.

PAI, which stands for ‘Projection and Interaction’, is a system that projects an image onto a flat surface to create a virtual interactive interface for the kitchen.  While the projector incorporates a camera, a speaker, a microphone, two USB ports, WiFi and Bluetooth antennas, the key technology here is a 3D sensor that detects minute movements of fingers on the surface.

According to project manager Markus Helminger, the PAI 3D sensor powers a projection interface that can “be perfectly operated even with dirty fingers and occupies no space on the work surface, so that consumers have enough space for cooking or baking.”

While other efforts at projection interfaces at trade shows have largely been to show off the concept with no concrete plans for commercialization, this time things look different with PAI. According to a German language post about PAI by on the BSH Kitchen Stories blog, they plan on rolling out PAI in February 2019 in China. While there’s no indication as to when we might see the technology in Europe or the US, my guess is we could see the technology in product rolled out in Europe as early as next year.

You can see a demo of the PAI interface (in German) below courtesy of Computer Bild TV:

Bosch PAI: Projektor für die Küche vorgestellt!

The story behind PAI is an interesting one. The technology spun out of development work Bosch was doing in ventilation. According a company spokesperson, researchers were looking to improve the user experience for cooks and “the developers wanted to create an assistance function for an extractor hood that would make it possible to project images from the hood and display recipes on the work surface. In an extensive UX study carried out by Bosch, this idea went down so well that the project was actually carried out.”

According to the company, consumer testers almost universally said they use tablet or smartphone when cooking or baking, but they didn’t like giving up the counter space to these devices required and, perhaps more importantly, they worried touching these devices with dirty hands. As the company worked on the concept more, they eventually decided to not incorporate it into a vent hood but to make the PAI a standalone projection system that allowed the consumer to place it where they desired on the counter.

The company has also integrated the PAI with its Kitchen Stories guided cooking system and its Home Connect platform, which opens up some intriguing possibilities. It’s not hard to envision a Kitchen Stories guided cook experience that shows step-by-step instructions projected onto the kitchen counter. With Home Connect, PAI could also project virtual start buttons, timers and other ways for the consumer to interact with their appliances.

With BSH Appliances – one of the world’s biggest appliance companies – taking projection interfaces seriously, my guess is we’ll likely see other big appliance brands push forward with their own projection interface commercialization efforts in the coming year and we’ll most likely see some of these teased at CES in Las Vegas next January.

Why Synthetic Sensors Could Be The Future Of Smart Kitchen Monitoring

Will the smart kitchen of the future be stocked with arrays of distributed sensors or could a single suite of sensors, localized on a credit-card sized housing, plug into an outlet to imbue the kitchen with all the intelligence it needs? According to Carnegie Mellon researchers in the Future Interfaces Group, the latter concept is highly promising.

The Future Interfaces Group has developed a synthetic sensor-based device that can monitor multiple types of phenomena in a room, including sounds, vibration, light, heat, electromagnetic noise, and temperature. This device, featuring nine sensors, can determine whether a faucet’s left or right spigot is running, if the microwave door is open or how many paper towels have been dispensed.

“The idea is you can plug this in and immediately turn a room into a smart environment,” said Gierad Laput, a Ph.D. student in CMU’s Human-Computer Interaction Institute (HCII). “You don’t have to go out and buy expensive smart appliances, which probably can’t talk to each other anyway, or attach sensors to everything you want to monitor, which can be both hard to maintain and ugly. You just plug it into an outlet.”

Machine learning algorithms combine raw data feeds into powerful synthetic sensors that can identify a wide range of events and objects. For example, they can distinguish between a blender, coffee grinder, and mixer based on sounds and vibrations.

The CMU researchers discuss the technology in the following video and have been demonstrating it at recent conferences:

Synthetic Sensors: Towards General-Purpose Sensing

“Smart appliances are expensive and rarely talk to one another,” the researchers note. “We’ve explored an alternate, general purpose sensing approach where a single, highly capable sensor board can indirectly monitor an entire room. We started our research by taking an inventory of sensors used in commercial and academic systems. Our sensor board is plug-and-play, uses wall power and connects to our cloud over WiFi.”

CMU researchers are also expanding the types of data feeds that the sensors work with. For example, the sensors can infer human activity, such as when someone has left for work, and the sensors can be trained to recognize various phenomena, such as the cycling of heating and air conditioning units. In addition, the sensors can be trained to detect many popular devices and brands of kitchen products

Google, through the GIoTTo Expedition Project, has supported the CMU research, as has the David and Lucile Packard Foundation.  Google is also actively pursuing its Tensor Processing Units, or TPUs, which are similar in concept to the general purpose sensors from Carnegie Mellon.

The CMU sensing concepts are, of course, joining many other imaginative new ideas for sensors that could impact smart kitchens. For example, NeOse is a new device that connects to smartphones and databases and can recognize more than 50 types of odors. This smell-sensing device could detect when a food item is spoiled in a refrigerator, when food is being overcooked, and more.

Make sure to check out the Smart Kitchen Summit, the only event about the future of food, cooking, and the kitchen. Use the discount code SPOON to get 25% off of tickets. Also, make sure to subscribe to get The Spoon in your inbox. 

Projected Video Interfaces May Be The Future Of The Kitchen. Why Aren’t They Here Yet?

With all the talk about Alexa nowadays, you’d think the future has arrived and we have our interface: voice.

But before you give up all your iPads for Echo Dots, take a moment to consider how much you use touch interfaces on a daily basis. While voice will no doubt play a huge role in the future of the smart home, touch continues to proliferate. Not only are car makers adding interactive touch screens, restaurants and pretty much everywhere else we go is getting better touch interfaces, while products in our kitchen like refrigerators are getting better and better touch interfaces.

And now, touch is combining with gesture recognition in a new science-fiction spin on interfaces that is gaining favor among product designers. The ‘projected interface’ – where an image is projected onto a flat surface to make what is essentially an interactive touch screen through the use of machine vision – is a new product interface concept that has captured the imagination interface designers in the kitchen over the past few years.

Projected Interface Demos Are Everywhere

Below are a few high profile projected interface product demos rolled out over the past few years:

Whirlpool, the world’s largest appliance maker, started talking about their interactive cooktop three years ago at CES 2014. You can see in the video how the Whirlpool smart cook top utilizes projected video as an interface.

Over a year later, people were wowed by the projected interface used by IKEA in its 2025 future kitchen concept video:

IKEA Concept Kitchen 2025

At this year’s CES, Bosch seemed to embrace the concept of the projected interface, incorporating it into not only a demo of its coffee making robot:

Spotted at #ces2017: coffee robot at the @Bosch booth.

A video posted by Michael Wolf (@michaelawolf) on

And high-end German consumer electronics manufacturer Grundig has been showing off its VUX projection interface concept for the past year and a half:

So, why all this interest in projected video interfaces and, more importantly, why haven’t any of these “visions of the future” made it to market yet?

Why Projected Video Interfaces Are Inevitable

It’s clear why projected video is a popular concept for demos: it looks cool. The very idea of turning any flat surface into an interactive interface is one that, at least for now, wows the user. Of course, that could change over time if these interfaces become more commonplace, but isn’t that true of every new technology? For now, projected interfaces could be used in a variety of interesting ways that consumers would love.

As for appliance makers, projected interfaces present an exciting new way to create consumer experiences. Think about it: Product designers used to working with tiny screens would find a much bigger pallet in projected interfaces, allowing them to create more compelling experiences that open doors to more education, instruction, and marketing content. Projected interfaces would also allow them to refresh their product interface regularly, rather than being stuck with the same small, mono-color screen for the life of the appliance.

If It’s So Cool To Demo, Why Aren’t Projected Video Interfaces In Market Yet?

If the projected video interface is such a crowd pleaser, why does it only seem to show up in concept videos and on trade show floors in “kitchen of the future” concept demos?

Here are a few possible reasons:

It’s still early, and projected interfaces are not quite ready for prime time. Most of the demos we see are all created for highly controlled demo experiences.  It may be that making projected video interfaces work in a mass-market environment in the consumer kitchen isn’t as easy as they’ve made it look in a demo video.

Implementing projected video requires a complicated setup. The IKEA 2025 kitchen concept utilizes a “smart light” concept which would require the consumer to have a specialized projector and an intelligent camera that can then read the gestures of the consumer installed above a table.  The other demos also have separate projection systems installed above the work surface. This type of set up would mean these systems are likely expensive and require a new installation method to which appliance makers and their channels are not accustomed.

Appliance Makers Move Slow. While all of these demos make clear that the projected interface is a promising concept, it’s important to remember appliance makers don’t introduce radical new concepts quickly.

Still, I’m hopeful that maybe Whirlpool, Bosch or IKEA will introduce an actual product at next year’s CES using projected video rather than just a concept demo. I do suspect that some, if not all, of the big appliance makers, are working on productizing projected interface enabled products. I also suspect disrupters may be working on something in this area.

And I wouldn’t be surprised if the world’s most disruptive company, Amazon, has something up their sleeves. After all, I keep hearing that the online giant has a bunch of crazy new products in store for us this year, so who knows, maybe the “kitchen computer” of the future is a social robot that has a projection video interface built in to go with Alexa.

To get analysis like this and to stay up to date on the future of cooking and the kitchen, subscribe to our newsletter, the Weekly Spoon.