Over the past couple years, there’s been what can only be described as an intellectual property land grab in the world of computer vision as Google, Amazon and Microsoft file more patents in an effort to establish foundations from which to launch an innumerable amount of AI-driven products and services over the next decade or more.
But as it turns out, big appliance isn’t quite ready to cede the entirety of this fast-moving space to big tech, especially when it comes to their home turf, err, the home. Last week appliance giant Whirlpool was awarded what appears to be a a fairly broad patent for a computer vision system to track behavior of people and objects within a home and enable all sorts of potential scenarios such as auto-replenishment, guided cooking and more.
The patent, entitled “Interaction recognition and analysis system”, starts with fairly broad language to describe a system that utilizes image capture technology to identify the contents of a person’s hand and trigger a range of reactions from appliances within the home. From there, the descriptions and related diagrams get more specific, illustrating how the system could enable a variety of specific scenarios such as suggesting a recipe or ordering more laundry detergent from an online retailer.
Figure: Diagram of refrigerator utilizing interaction & analysis system
One such example is a description of how the system could be used in a fridge across a number of different “access regions” that correspond to the different storage areas within a fridge. The system would know the freezer drawer from the fresh food drawer and be able to monitor corresponding changes in frozen food or fresh food inventory.
In another example, the patent describes how the system could be used to monitor activity in the cooking cavity of a microwave or oven. It could also determine the level of doneness of the food. From there, it could initiate a specific timer or a series of cooking processes. In other words, the system could serve as a foundation for guided cooking.
Of course, we shouldn’t be all that surprised that Whirlpool would be delving into computer vision-powered services as suggested by this patent. At the last CES, the company showed off ingredient recognition capabilities of the new Yummly app and has been showing off concept demos utilizing computer vision-powered interfaces since CES 2014.
Still, this new patent is intriguing and I am very interested to see where Whirlpool takes this type of technology in the future. Because of the broadness of the patent, it could serve as the IP underpinning for what is essentially a kitchen operating system, where fridges, ovens and washing machines are not only in sync with the consumers that use them, but also with each other and all the various systems in the home. The end result could be a more sentient and anticipatory kitchen in place of the smart appliances operating independently of one another that occupy our homes today.
Sure, that’s a lot to infer from one patent, but that’s the direction Whirlpool, BSH and others have been heading in the last couple years. My guess is 2019 will see significant moves in this direction, starting in just a couple weeks at CES in Las Vegas.
Great Article!
My hope is that true empathy for the customer is at the heart of new products and capabilities that come from the “smarting” of the kitchen space Why does the customer need this? Just because technology has given us the HOW, doesn’t mean we know WHAT the customer needs.
Is it only about trying to own information about the customer that can be monetized? That’s a good question to be asked.