This week, Amazon gave a sneak peek at the new AI model that powers its Just Walk Out platform.
In a post written by Jon Jenkins, the VP of Just Walk Out (and, as Spoon readers may remember, the former founder of Meld and head of engineering for the Hestan Cue), we get a peek at the new AI model from Amazon. Jenkins writes the new technology is a “multi-modal foundation model for physical stores is a significant advancement in the evolution of checkout-free shopping.” He says the new model will increase the accuracy of Just Walk Out technology “even in complex shopping scenarios with variables such as camera obstructions, lighting conditions, and the behavior of other shoppers, while allowing us to simplify the system.”
The previous Just Walk Out AI model was built in 2018 using generative AI and machine learning models available at the time. This technology, while advanced for the time, could only power a sequential processing of key variables – shopper movement and location in the store, what they picked up, and the quantity of each item—each action processed one after another. This sequential processing worked in most simple scenarios, but complex scenarios like multiple shoppers accessing the same area at a given time, would lead to potential errors that would need to get sorted out at checkout.
The new system differs from the previous system in that it analyzes data from multiple sources—cameras, weight sensors, and other data—simultaneously rather than sequentially. It also uses “continuous self-learning and transformer technology, a type of neural network architecture that transforms inputs (sensor data, in the case of Just Walk Out) into outputs (receipts for checkout-free shopping).”
Jenkins writes that the new system will be better at navigating these complex situations that would result in potential errors with the previous system. He detailed a scenario where a shopper picks and puts down multiple varieties of yogurt, and while doing so, another customer reaches for the same item or the freezer door fogs up and obscures the cameras’ view. In this scenario, the new system processes inputs from various sources such as weight sensors on the fridge shelves and continuously learns from these inputs, eventually deciding which are most important in order to accurately sort out who took what.
The post also gave an update on the current installed base of Just Walk Out technology. According to Amazon, Just Walk Out is currently in 170 third-party locations, including airports, stadiums, universities, hospitals, among other locations. The system is installed in the US, UK, Australia, and Canada and the company plans to double the number of third-party stores with the technology in 2024.
Leave a Reply