For a while now, we’ve known the basic gist of how Amazon’s Just Walk Out technology works: A combination of computer vision, machine learning, and other sensor data helps enable a friction-free shopping experience in which customers pick items off the shelf and walk out the door without ever having to stop at a cash register.
But in a recent blog post by Amazon’s retail technology team, the company explained how it all worked in greater detail than we’ve seen in the past, including how the company has been using generative AI to train its Just Walk Out platform for long-tail cases that are rare but entirely possible in the unpredictable environment of retail.
According to Gérard Medioni, vice president and distinguished scientist at Amazon, the company uses a generative AI called a generative adversarial network (GAN) to create synthetic data for training Amazon’s Just Walk Out technology. The Just Walk Out team used datasets from millions of AI-generated synthetic images and video clips mimicking realistic, and sometimes rare, shopping scenarios, including variations in lighting, store layouts, and crowd sizes. According to Amazon, this training using generative AI-created synthetic visual data enables Just Walk Out to recognize and properly interpret millions of customer actions.
“When the customer exits, having an accurate account of their purchases is critical,” Medioni said.
The company also went into detail about how Just Walk Out and its Amazon One palm-based bioauthentication technology does – and don’t – work together. According to Amazon, the two systems operate independently of each other, keeping a person’s biometric information associated with their payment separate from Just Walk Out. When a shopper enters the store, the Just Walk Out system assigns the shopper a temporary numeric code, which serves as their unique digital signature for that shopping trip. When a shopper exits, the code disappears. When they come back, they get a new code.
Medioni says that Just Walk Out associates a person’s “pixels” to the one-time payment code assigned for that trip and the products they pick up off the shelf.
“Just Walk Out tech doesn’t collect any biometrics. All we need to know is where that person is on the floor, and where their hands are in relation to the store’s merchandise.”
According to Medioni, the system is sophisticated enough to track groups of shoppers assigned to a single payment instrument, and the system can create a single receipt for a group shopping trip.
“We had a tour bus that came in one day, and they had 90 people all paying with a single credit card,” Medioni adds. “Even if people leave the store separately and we can still keep track of the group’s purchases.”
While Amazon has shown mixed signals regarding its retail footprint, the company appears to remain interested in developing its technology platform for usage by other retailers. My guess is they’ll likely see some smaller retailers and non-grocers (like stadiums/sports venues) adopt the technology, but larger grocers will remain reticent to jump on board with technology developed by a competitor.
If you’re interested in how generative AI will change food retail, join us at the Spoon’s Food AI Summit on October 25th in Alameda!
Leave a Reply