Close Ad

The Spoon

Daily news and analysis about the food tech revolution

Google Home

KloveChef Opens Up Voice-Guided Cooking Platform to Publishers

KloveChef, the voice-guided cooking startup cofounded by one of India’s biggest celebrity chefs in Sanjeev Kapoor, is opening up its platform this month to publishers wanting to add voice-guided cooking functionality to their recipes.

The new tool will allow anyone who has recipe content — chefs, cookbook authors, bloggers or food retailers — to upload their recipes to KloveChef’s platform via a web interface and it will convert them into a voice-guided recipes.

“We will democratize the interactive recipe creation and distribution,” said Bahubali Shété, KloveChef cofounder and CEO, in an interview with The Spoon.

Shété told me that recipe publishers will be able to use KloveChef to publish their recipes across a variety of voice platforms such as Amazon Alexa, Google Assistant, Google Home and Amazon Fire TV. To do so, they just copy the recipe URL or paste the full recipe into the web interface and KloveChef will convert into a voice-guided recipe.

Shété also said that publishers will have the option of letting users send their recipes posted on other web channels such as YouTube or Pinterest to their voice assistants for guided cooking.

KloveChef is opening up their voice platform after finding some success with their Alexa voice skill targeted primarily at home cooks in India. According to Shété, the guided cooking assistant has a total of 465,000 users and 100,000 monthly active users.

Shété says publishers can make money through KloveChef if the recipe is converted into a shopping list. The recipe-to-shopping list feature, which KloveChef has been testing through its app in India, currently has over 1 million recipes converted into shopping lists via voice search.

I have to admit, I like the idea of self-publishing recipes to voice platforms. It reminds me of the early days of ebooks, when authors would use technology from early pioneers like Smashwords to put their books into the world and on other popular platforms. Perhaps not all that surprisingly, just as like those early days of ebooks, recipe self-publishers are relying on Amazon to reach the end consumer, only instead of Kindle this time it’s Alexa.

It’s too soon to see how successful KloveChef will be in attracting cooks for its voice guided recipe assistant outside of India. In its home market, they’ve been able to leverage the large reach of Kapoor, while here in the states, Alexa tends to favor its featured partners such as Food Network or Tasty. KloveChef will have to compete with the algorithm-favored partners through attracting recipe publishers such as popular food bloggers or food retailers with built-in audiences to accrue a sizeable user base.

Looking forward, the company hopes to also attract users by making the platform better over time. One of the early features will be adapting guided cooking where users can speed up a recipe or slow it down depending on their experience. The company plans to release the new capabilities by mid-August.

Google’s Not-So-Secret Weapon in the Virtual Assistant Wars: Photos

If the kitchen is the heart of the home, then Google, not Amazon has the upper hand when it comes to building the smart kitchen assistant of choice. The secret weapon of the Google Home Hub isn’t a smarter AI, or better sound quality (it’s definitely not that), it’s pictures.

To understand why pictures are so important to winning in the smart assistant space, think of the iconic Kodak pitch scene in Mad Men. In it, Don Draper explains that the 1960s slide projector didn’t have a “wheel” as the executives described it, but rather a nostalgic carousel.

Mad Men - The Carousel (Higher Quality)

While it was far less dramatic (and fewer cigarettes were smoked), my wife recently explained that she had formed an “emotional connection” with the Google Home Hub, Google’s smart display. This immediately caught my attention because she was more frustrated than anything by Amazon’s Echo Show, but she genuinely loves Google Home Hub. That’s because it is a kind of time machine. Google Home Hub can access the thousands of pictures stored in my Google Photos account. Any time she’s in the kitchen, she sees a picture of our son’s fifth birthday party, or the moment he lost a tooth, or a vacation picture, and it makes her happy.

Google Home Hub is her (and my) carousel, and this type of emotional connection will only get stronger as we get older and the photos from today resurface years from now. We actually demoted our Echo Show to the living room even though we listen to a ton of music and the sound quality of the Home Hub is way worse. Being able to see our photos was more important than the fidelity of Steely Dan’s Aja.

Alexa cannot access my photos and I have no plans to store them on Amazon. I’m pretty deep in the Google Photo ecosystem and I’m not alone. As of May 2017, Google Photos had 500 million users. They make it so easy to upload from your phone that it’s hard to envision a scenario where I switch to Amazon Photos just so I could see them on an Echo Show.

But this isn’t just about warm and fuzzy feelings. There is big money at stake.

As my colleague Mike Wolf has written, kitchen screens are going to be a big deal. The combination of voice control and visuals allow you to quickly find out information, plan your day, access video entertainment, and even help guide you while you cook.

On a very base level, there are billions of dollars at stake from the device sales alone. Strategy Analytics reports that more than 12 million homes will own a smart display by the end of this year, and that number will jump to 100 million homes by 2023. Whoever can grab more marketshare, obviously, makes more direct sales money.

Then of course there is the additional revenue generating opportunity from photo storage. I’ve paid Google two bucks a month for four years for that privledge, and don’t see a time when I’ll stop (hopefully they won’t kill it).

And because we live in the times we do, dominant smart assistants also get access to all that data we generate by asking it questions, controlling our smart devices and playing us songs and videos. That data, in turn, helps perpetuate whomever’s dominance. Right now, Alexa is king in the smart speaker space, but its lead is slipping as Google gains marketshare.

Whoever dominates in the smart assistant space also helps shape the future of the connected kitchen as appliance makers look to incorporate new technologies that have a proven user base. Google was everywhere at CES last week, and more appliance makers are highlighting Google Home integration. GE Appliances showed off its mega Kitchen Hub 27 inch video touchscreen that mounts above your oven. It runs on Android and the company highlighted its Google Assistant integration. Elsewhere, KitchenAid joined other manufacturers like JBL and Lenovo in launching its own Google-powered smart display.

The smart assistant is the tip of the spear to accessing and controlling more of your future smart home. Though the battle has just begun, in the smart assistant arms race between Google and Amazon, it’s not necessarily the brains of the device that will prove to be the winner, but the one that wins over our hearts.

Surprise! Amazon Says it Sold a Lot of Amazon Devices (and Instant Pots, too!)

Amazon put out a news release today touting its record-breaking holiday shopping weekend. And, in what will come as a shock to absolutely no one, the company said the best-selling products across all categories sold on Amazon.com were Amazon devices like the Amazon Echo Dot, which Amazon just happened to put on sale… on Amazon.

Get where we’re going with this?

The retail giant is always vague on details, and this release proved no exception, saying only that it was the “Biggest holiday shopping weekend ever for Echo devices, with millions sold worldwide—all-new Echo Dot was the #1 selling product on Amazon globally, from any manufacturer, in any category.”

FWIW, last December, when Amazon released a similar batch of vague statistics, the company reported selling “tens of millions of Alexa-enabled devices” worldwide over the entire 2017 holiday season.

OK, obviously Amazon releasing glowing stats about Amazon devices sold at a discount is a total corporate puffery, and I am complicit in writing about it. But, as vague as these stats may be, they are important to consider as more people adopt smart kitchen tech. Amazon’s Alexa is locked in a battle with Google Home to be your preferred voice ecosystem. Dominating voice control could in turn determine which kitchen appliances you buy, or impact where you buy your groceries.

Hardware startups, software developers and appliance makers alike want to align with a winner when it comes to incorporating new smart tech into their products. Amazon can create its own dominance by dint of controlling one of the largest e-commerce companies on the planet. Consider that at last count, Amazon had 100 million Amazon Prime subscribers worldwide. That’s a huge user base to be potentially guided into buying an Amazon Echo device.

The more Echo devices are sold, the more people will want to use Alexa in their homes, which means more third-party support for even more devices and apps. The more third party devices and apps that incorporate Alexa means that Amazon is collecting even more of our data, which Amazon can then use to sell us more stuff, more of its own stuff (like groceries from Whole Foods) or even create more of its own devices like the Alexa microwave, which, now that we mention it, was not mentioned in the Amazon press release.

The point is, the more Amazon can flex its retail power, the more it can dominate the emerging world of smart assistants and voice control in our homes.

Alas, Alexa devices weren’t the only thing the company sold over the Thanksgiving to Cyber Monday shopping season. I throw this in here as just a bit of Amazon sales trivia, but Instant Pots continue to steamroll other kitchen appliances, with Amazon saying the Instant Pot DUO60 was also a top seller this past weekend. During the 2017 holiday season, the Instant Pot DUO80 was the top-selling kitchen item.

So take these numbers with the appropriate amount of salt, and be on the lookout for a release from Google touting its own Google Home Cyber Monday sales, followed by another holiday season recap from Amazon at the end of December.

SmartThinq-ing. LG Appliances to Work with Google and Alexa

In a Switzerland-like move of neutrality, LG announced today that its SmartThinq line of connected appliances will work with both Google Home Assistant and Alexa (hat tip to Digital Trends). This will allow users to control more than a hundred SmartThinq devices, including ranges, ovens and refrigerators, with their voice through Google Home or Amazon Echos.

LG had previously announced Alexa integration for its new line of Thinq smart refrigerators, and at the same time had stated its commitment to an Open Platform, Open Partnership philosophy, which included incorporating Google Assistant. Today’s announcement seems to make this openness official across all of LG’s SmartThinq devices.

This is a smart (pardon the pun) play for LG, as both Alexa and Google Home devices have been selling in the millions. By working with both Google and Alexa, LG can slide its smart home devices into whatever ecosystem people have chosen for themselves. Removing any barrier to adoption is good — especially when you are asking people to plunk down thousands of dollars for something like a smart fridge.

It’s also a contrast to Samsung, which has chosen to push its own smart assistant, Bixby, in its Family Hub fridges at the expense of working with outside voice services.

In the kitchen specifically, LG is creating a very open platform in a bid to capture early adopters. In addition to working with both Google and Alexa, LG announced at CES that it will work with both Innit and SideChef to provided guided cooking capabilities into its ovens and ranges.

Probably not so coincidentally, LG also announced its own Thinq Google Assistant Speaker today, giving LG its own entrant in the smart speaker market. And since it’s powered by Google, you can talk to it to control your other LG devices.

Podcast: The CES 2018 Smart Kitchen & Foodtech Wrapup Show

CES 2018 is in the books. It was a hectic week packed with smart kitchen news and showcases. Mike was on the floor in Las Vegas and reveals the big trends (voice activation everywhere!), the cool news stuff (guided cooking!), and the countertop dishwasher he calls “sexy.”

Take a listen for all the in-depth analysis you need. You can also subscribe to the Smart Kitchen Show in Apple Podcasts or download it here.

“Alexa, How Can You Be Used in Restaurants?”

There’s a good chance that an Amazon Alexa or Google Home device is on a holiday wish list of someone you know. Consumer Intelligence Research Partners estimates that 15 million Amazon Echo units have been sold across the U.S. (Amazon does not disclose sales figures). As of now, Alexa’s use in dining out is centered more around at-home, consumer experiences. A quick glance through the restaurant related Alexa skills show an emphasis on discovery, information and ordering. Find a nearby restaurant. Order a pizza. Etc.

But is there a bigger opportunity for Alexa and Google Home inside the restaurant?

According to the National Restaurant Association, there are more than 1 million restaurant locations in the U.S. generating $799 billion in sales. One million on its own isn’t huge, but with some creative thinking, you could easily envision multiple devices deployed per restaurant, plus all the data captured from in-store interactions and you can see restaurants becoming a front in the voice assistant battle worth fighting over.

As a fun thought experiment, I put together a few potential uses for Alexa inside dining out:

An interactive table alert. Instead of a dumb, inert buzzer that flashes and vibrates when your table is ready, modify an Echo Dot to be the messenger. Instead of bugging the host, patrons could ask Alexa how much longer the wait time is, and be alerted when they can be seated. If you wanted to get real adventurous, in the right setting, you could even do ongoing interactive trivia games to keep people entertained.

Informed ordering. With its touch screen, an Echo Show would be an excellent way to show menu items, explain more about ingredients, and highlight popular dishes. You could also enable ordering and payment for a more streamlined experience. In a cruel, horrible world, one can imagine restaurants offer a cheaper meal if customers allow ads to be displayed while they’re eating (please don’t do that).

Back of house. Alexa could be used for quick ordering of ingredients, equipment or other sundries which, of course, could be fulfilled by Amazon that day. It could also be used to alert employees about their break times and inform them of any news or specials.

Communication back home. By gathering real time data inside a restaurant, Alexa at home could provide better, more informed real-time decisions about where and when to eat at a particular establishment.

Having said all this, there are some real world limitations to this type of implementation:

There’s a pretty small needle to thread in terms of the types of restaurant voice control will work. Too loud and voice control is useless and you can’t hear Alexa talk. Too quiet and voice control is annoying for everyone else.

It’s hard to imagine restaurants buying and modding Echo devices or writing their own skills. However companies such as Toast or Square could weave Alexa into their platforms and embed them on customized devices that are sold into restaurants.

We are still in the early stages of voice assistants and how far into our lives they will go. But as they get better, restaurant owners may not want to wait until the holidays to get their own.

Have you seen Alexa or Google Home used in interesting ways inside restaurants? Or have a wild idea about how they could? Leave a comment below and let us know what you think.

Google Assistant Is Becoming A Guided Cooking Platform. We Talk To The Person Leading The Charge

Back in April, Google’s Emma Persky wrote a post telling the world that Google has been working on a recipe guidance capability for Google Assistant.

At last month’s Smart Kitchen Summit, she gave a little more detail on what exactly the company has been up to.

Persky, who runs point for Google Assistant’s cooking guidance team, told the Spoon’s Allen Weiner how much of the focus has been on building in contextual understanding of recipes.

“We can talk you through step by step how to cook a recipe and answer contextual questions about how to do that,” said Persky. “To do this, we have to build a deep model of what a recipe actually looks like. Have to take the text of a recipe and understand that text so we know what pasta is, we know what type of pasta you’re talking about, we know what the units are, we know the cooking temperature.”

“On the other side, we know what the user is saying. We have a whole bunch of machinery at Google that is able to understand what a user is saying and turn that into a machine question. We have a whole bunch of data about how different people ask these questions, which we use to build a model and understand these types of questions.”

When asked about how Google is utilizing company competencies like search and YouTube, Persky said that while there’s been significant work done here, there’s an opportunity to get better.

“We do a pretty good job now when you ask on your phone or desktop ‘how do I sauté an onion?’ we show you a nice video of how to sauté an onion.”

But, she said, “there’s a lot of opportunity for guided cooking feature to more deeply integrate with this, so when your recipe says sauté the onion, and you don’t know how to sauté the onion, we are able to return these types of video answers on on Google Home platform to help you become a better chef over time.”

Persky also discussed how she thought web content schemas could evolve to create a foundation for richer content through platforms like Google Assistant.

“When it comes to companies that have this kind (recipe) data available to them, there are a lot of opportunity for finding ways to increase the fidelity of the data that we have access to. At the moment we have schema.org markup, which is a good first pass. We don’t have a lot of detail and use machine learning to extract a lot of the context from that. And I think where there’s an opportunity to where a lot of people working on this stuff is to find ways to access more high fidelity data that we could offer to the users as an improved experience.”

When asked by Weiner how schema.org and other web markup languages could improve, she had this to say: “There’s a lot of work we can do to improve the quality of that markup. For example, right now the markup just has one block of text for all the instructions in the recipe, but actually if we could break that down and have a step by step, it would be easier for us to parse that out. Right now we have to apply machine learning across that to do that.”

It’s a really good conversation to understand what Google has been up to as they look to combine recipe content with their voice AI platform. You can watch the full conversation between Allen Weiner and Emma Persky below:

Ed note: Answers in this interview have been edited slightly for brevity/clarity

Sonos gets Alexa (and soon Google and Siri)

If you’re like me, Amazon’s Echo plays a central role in your kitchen. Player of music, answerer of questions, setter of timers, forecaster of weather. The Echo is super convenient, but the sound quality is definitely lacking.

Since it was announced last year, I’ve eagerly anticipated the marriage of Alexa’s voice control with the room-filling sound of Sonos Play speakers. Today, Sonos announced that Alexa integration is finally here, along with a new Sonos One speaker with Alexa built in — as well as forthcoming Google Assistant and Siri support.

For existing Sonos and Alexa owners, the Amazon connected assistant is available via a Sonos app beta update available today. With it, you’ll be able to control your Sonos speakers with your voice via the Echo and Echo Dot.

With the Sonos One (available October 24), Sonos is vying to become Switzerland of connected home assistants. The $199 speaker sports Alex integration out of the box, with support for Google Assistant and Siri (via iOS) coming in 2018.

This is probably the best position for Sonos to take as it feels the squeeze between cumbersome traditional high-end audio and the more convenient but lower fidelity smart home devices like the Echo. The timing is good too, given that today Google unveiled its beefy Google Home Max smart speaker and the upcoming Apple Homepod bills itself as having superior audio. Being agnostic to your assistant ecosystem of choice could give Sonos an edge and a chance to regain some lustre.

We had three Sonos speakers in our house, and almost immediately stopped using all of them once we got an Amazon Echo. Despite having worse sound, the device was in the kitchen, so it was more convenient to where our family mostly congregates. It’s such a high-class problem, but after experiencing Alexa’s voice control, running to get my phone to control my speaker felt like so much… work.

But was we played more music in the kitchen — whether to cook to, do homework by, or just throw an impromptu dance party — sound quality became more important. My pre-Sonos One solution was to plug an Echo Dot into a Sonos Play:5, but that takes up a lot of counter space, has dangling cords and can make it harder for Alexa to hear my voice. With its smaller footprint and newer technology, the Sonos One becomes an interesting proposition.

Chefling Is The Smart Kitchen Personal Assistant You Never Knew You Needed

The Smart Kitchen Summit Startup Showcase provides a platform for exciting startups, inventors, culinary makers and cutting-edge product companies to showcase what they are working on and let others experience it firsthand. Now in its third year, the Startup Showcase + PitchFest take place during SKS on October 10-11, 2017 in Seattle and is sponsored by the leading maker of soups and simple meals, beverages, snacks and packaged fresh foods, Campbell Soup Company. Campbell’s will provide a $10,000 cash prize to the winner, announced at live at SKS.

Created in 2016 by a group of Northwestern University grads, the Chefling app was designed to be the ultimate smart kitchen assistant. The app, available on both Android and iPhone aims to resemble life spent in the kitchen of an average family and comes with three main features: a home inventory management system that syncs across devices, a colorful shopping list and a smart cookbook that suggests recipes based on available ingredients.

The home inventory management system monitors what a user buys and then keeps an eye on freshness levels, based on purchases. It also syncs across every family member’s device so any quick trips to the grocery store are simplified. The shopping list feature allows users to organize and create a list that is not only easy to browse, but easy to share. The cookbook feature browses a user’s inventory using an algorithm that calculates recipe matches based on what is in the pantry.

Chefling also has Amazon Echo and Google Home skills so users can operate the app hands-free through voice control.

Chefling’s long-term goal is to bring the platform outside of the phone and into smart fridges, as well as incorporating advanced A.I. elements and image recognition technology to create an assistant that is truly integrated with the home kitchen.

Learn more about Chefling at http://www.chefling.net/.

Use this link to get 25% off to the Smart Kitchen Summit & see the startups in action!

Klove Offers Low-Cost Entry Point To The Smart Kitchen

The current state of the smart kitchen is still, well, kind of expensive. And that’s ok -- markets in their early days often produce products with high price tags as demand is still being developed and solutions still being realized. Intelligent ovens, smart fridges, connected tea infusers -- it all sounds like an amazing eutopia of high tech deliciousness. But these solutions aren’t making cooking easier for the masses -- at least not yet. But some areas of the smart kitchen are starting to produce at mainstream prices -- most specifically in the sous vide space with sub $100 machines available at big box retailers like Target.

And then there are startups like Klove. The concept behind the Klove stove top knob is pretty cool -- it’s a retrofit device that replaces your dumb stove knobs and adds a pretty crazy amount of intelligence into a little form factor.

The Klove smart knob acts as an entry-level guided cooking system -- assessing the state of heat on whatever dish you’re preparing and letting you know when to adjust and when you can walk away. Klove comes with a host of recipes to start with, so you don’t have to guess what to cook when you get started.

Klove -Just Talk and cook

With a companion app (because of course) and Google Home / Amazon Echo compatibility, the Klove smart knob also has some machine learning baked in and will adjust recipes based on your preferences over time. Sure it might say to scramble eggs for 5-7 minutes, but maybe you like yours runny. The knob will learn that over time and adjust its alerts accordingly. Like a little digital sous chef who remembers your favorite type of pancake. (Chocolate chip. It’s always chocolate chip.)

There’s even a safety feature built into the device. Try to leave home with your stove on -- the Klove app will alert you before you get out the door, ending the days of wondering “did I leave my stove on?” when you get to work. It will also alert you if you walk away from the stove for a minute or two and are needed to turn something up or down. If you’ve ever cooked something too long -- or had boiled water overflow and spill onto the stove top, you’ll probably find this feature helpful.

The best part is that Klove is only $29. Well, for now -- it’s available for pre-order on Indiegogo for $29 which is technically a price drop from the company’s first unsuccessful crowd funding campaign. But there seems to be some momentum this time around and the sub $40 price point is pretty attractive. If voice control is driving more interest in connected tech in the home, retrofit devices like Klove can help consumers see the value of technology to assist them in cooking better and easier at home.

Klove has had some momentum recently, having raised $250k from investors in a SEED round of funding and being named “The Next Big Thing” in food tech via the Nestle “Next Big Thing” startup competition in London. Klove isn’t the first company to create a retrofit smart knob for stoves -- Meld introduced its smart knob in 2015 and went a step further than Klove to be able to automatically control the temperature during cooking as opposed to notifying someone to turn the dial up or down when ready. Meld ended up cancelling its Kickstarter when it was acquired by Meyer Corp (owner of guided cooking brand Hestan Cue) after the campaign was successfully funded. Hestan took the learning and knowledge behind the Meld knob and used it to build its current guided cooking platform -- though the actual knob form factor never resurfaced.

Klove has about 4 weeks left to raise the initial $20k to get started on development -- and with a few smart knob competitors in the space, it will be interesting to see how they do. For now, you can grab a Klove for $29 as an early bird backer and expect to see the smart knob right around Christmas.

Is Nest’s New Face Recognition Cam A Sign It’s Waking From Its Slumber?

One of the great mysteries of the smart home world over the past few years has been the relative quiet of Nest, the one-time connected home star that burst onto the scene with the launch of its impressive Nest Learning Thermostat and, two years later, a smoke and carbon monoxide detector by the name of Nest Protect.

After Nest got acquired in early 2014, updates slowed to a crawl, and much of the news that did come out from the company during this time was bad. While there was occasional news about Works with Nest and Thread, you were just as likely to read about management dysfunction under Tony Fadell and product recalls. The only wholly new product line introduced into the Nest family during this time was the Nest Cam, a product that, in reality, owes more to Dropcam (another acquisition by Google) than to internal development from Nest.

But now there are indications the company might be waking up from its long slumber. In a recent story in The Verge, Nest product manager Maxine Verson hinted that the rest of 2017 should be busy for the company:

Verge writer Vlad Savov writes, “Veron tells me Nest’s relative silence in recent times is about to be a thing of the past. “I am very excited about the next six months,” he says with a grin, “we’ll talk again soon and you’ll understand why.”

Savov goes on the speculate that the next product might be a cheaper home thermostat. A welcome addition to be sure, but I think for those who witnessed Nest’s early days of innovation, a lower-cost version of an existing product is hardly something to get excited about.

The product I’m excited to see is Nest’s long-rumored home security system. My own sources have confirmed the existence of this long-gestating project and, given Google’s patent filings in the space, I think a Nest home security system could be truly differentiated. Add in the fact they just introduced a camera with facial recognition capabilities – an interesting potential component of a smart security system – and we may be getting close.

Another potential product is a video doorbell.  While the market is certainly crowded at this point, Nest’s brand name and recent development of a Pro group that supports home builders and integrators could help a Nest video doorbell get traction.

Whatever Nest does, chances are it’s been limited to a certain set of products by its parent company. Alphabet/Google has let non-Nest groups develop products in some of the most interesting areas – voice assistants and mesh Wi-Fi for example – while Nest has largely stuck with thermostats and cameras.

One thing is certain: the company’s new Nest Cam itself is a sign of progress. With it, the company has started to integrate image-based AI into its Nest cams, a potential indication that it – like Amazon – sees computer vision as one of the key new frontiers in the smart home.

And who knows? Maybe now – if the new camera and hints being dropped are any indication – maybe Nest truly has something new and interesting up its sleeves.

Make sure to subscribe to the Spoon newsletter to get it in your inbox. And don’t forget to check out Smart Kitchen Summit, the only event on the future of the food, cooking, and the kitchen. 

Rubin: Essential Home Will Be The “Bridge” Between Competing Smart Home Ecosystems

Last night Andy Rubin got on stage with Walt Mossberg at the Code Conference to discuss his new company.

The two spent a good chunk of the conversation talking about the Essential Phone, but when they finally got to the Essential Home, they didn’t disappoint.

I wrote yesterday about what we already knew about the Essential Home smart home product, but Andy’s discussion with Walt gave us a better understanding of the company’s strategy for the device.

Rubin and his team have (correctly) identified the main problem of the smart home as one of too many competing ecosystems. The main goal of the Essential Home is to solve for that.

Per Rubin: “One of the problems in the home is the UI problem. There are too many things you have to interact with in your home.”

While we often use that term UI to describe the various consumer interaction layers such as voice, touch or motion, Rubin is using the term more broadly here. He points to a fragmented smart home world with too many competing apps, smart home protocols, and technologies. And, as the guy behind Android, Rubin admitted that in many ways he helped create the problem.

“I feel somewhat responsible. One of things Android helped do make really easy to write a mobile app. the guy building your IoT doorbell, he’s going to write an app.”

According to Rubin, the problem with so many apps and technologies is each time a consumer walks through their smart home, they are walking through a series of competing apps and ecosystems.

“In certain ways,” said Rubin “the industry has recognized what the problem is, which is you don’t want to launch someone’s app when you walk up to your front door to unlock it, where they have their own UI, their own login credentials, and when you finally get through front door and its time to turn on your lights, do the same thing with the guy that built your light bulb.”

He’s right in saying the industry knows fragmentation is the main problem in consumer adoption. In our survey of over 100 smart home execs last December, the number one hurdle to adoption of smart home products identified by industry insiders is confusion over too many smart home platforms.

In other words, fragmentation. Or, as Rubin puts it, “a UI problem.”

Rubin said the solution to the problem of UI fragmentation is to bridge all of these competing ecosystems by working to integrate as many of them as possible together.

“You have to think of it as a UI problem,” said Rubin, “and you have to solve the UI for the home as an interoperability and integration issue. You can’t just support ten devices; you have to support one hundred thousand devices.”

That’s a lot of devices, but Rubin plans to get there by bridging the various ecosystems across the world of Apple, Amazon, Google and more. In other words, he doesn’t want to compete with the giants, but instead wants to connect them to one another.

“You can think of this as everyone is creating an island by creating their own ecosystem, so building bridges is the best way to describe what we’re doing. It has to talk to all these ecosystems, whether it’s Smart Things, HomeKit, or Google Home, or Thread or Weave.”

Rubin didn’t go into the specifics of how he plans to solve the fragmentation issue, other than to say they think they’ve found a way to do it. Whatever the approach is, it sounds like one built from the operating system on up with a focus on security.

“We had to build a new operating system so it can speak all those protocols and it can do it security and privately.”

That operating system is called Ambient OS. It will be part of the new Essential Home which is rumored to ship in late summer.

Make sure to subscribe to the Spoon newsletter to get it in your inbox. And don’t forget to check out Smart Kitchen Summit, the first and only event on the future of the connected kitchen and the future of cooking.