Close Ad

The Spoon

Daily news and analysis about the food tech revolution

Just Walk Out

While Amazon Struggles With Futuristic Retail, Sam’s Club is Doubling Down on Computer-Vision To Aid Store Exits

An interesting set of stories came out over the past week, which showed how two retail giants are leveraging technology to help consumers get out of the store faster.

One one hand, you have Amazon halving its Go store count to 16 locations, shifting focus to licensing its “Just Walk Out” technology to third-party retailers.

On the flip side, Walmart’s bulk membership club Sam’s Club is investing more heavily to use computer vision to aid shoppers to get out of the store faster.

The key difference? Sam’s Club is using computer vision for receipt verification. Previously, members had to stop at the exit while associates manually checked receipts, causing bottlenecks. Now, cameras at the exit automatically scan carts and verify purchases, allowing members to walk out without interruption. AI works in the background to refine accuracy, while employees are freed up to assist shoppers rather than policing receipts.

Contrast this with Just Walk Out, which is Amazon’s effort to eliminate the checkout experience all together which, it appears for many people, is still too weird and feels a little to close to shoplifting.

As I wrote last year, Just Walk Out is “a radically tech-forward evolution of checkout, but one in which Amazon appears to have widely overestimated just how many people would use it and how easy it would be to implement. Self-checkout fits most shoppers’ needs when they are in a hurry, and there aren’t that many situations where consumers feel they need to skip checkout altogether.”

Amazon Gives a Peek at the New AI Model Powering Just Walk Out Platform

This week, Amazon gave a sneak peek at the new AI model that powers its Just Walk Out platform.

In a post written by Jon Jenkins, the VP of Just Walk Out (and, as Spoon readers may remember, the former founder of Meld and head of engineering for the Hestan Cue), we get a peek at the new AI model from Amazon. Jenkins writes the new technology is a “multi-modal foundation model for physical stores is a significant advancement in the evolution of checkout-free shopping.” He says the new model will increase the accuracy of Just Walk Out technology “even in complex shopping scenarios with variables such as camera obstructions, lighting conditions, and the behavior of other shoppers, while allowing us to simplify the system.”

The previous Just Walk Out AI model was built in 2018 using generative AI and machine learning models available at the time. This technology, while advanced for the time, could only power a sequential processing of key variables – shopper movement and location in the store, what they picked up, and the quantity of each item—each action processed one after another. This sequential processing worked in most simple scenarios, but complex scenarios like multiple shoppers accessing the same area at a given time, would lead to potential errors that would need to get sorted out at checkout.

The new system differs from the previous system in that it analyzes data from multiple sources—cameras, weight sensors, and other data—simultaneously rather than sequentially. It also uses “continuous self-learning and transformer technology, a type of neural network architecture that transforms inputs (sensor data, in the case of Just Walk Out) into outputs (receipts for checkout-free shopping).”

Jenkins writes that the new system will be better at navigating these complex situations that would result in potential errors with the previous system. He detailed a scenario where a shopper picks and puts down multiple varieties of yogurt, and while doing so, another customer reaches for the same item or the freezer door fogs up and obscures the cameras’ view. In this scenario, the new system processes inputs from various sources such as weight sensors on the fridge shelves and continuously learns from these inputs, eventually deciding which are most important in order to accurately sort out who took what.

The post also gave an update on the current installed base of Just Walk Out technology. According to Amazon, Just Walk Out is currently in 170 third-party locations, including airports, stadiums, universities, hospitals, among other locations. The system is installed in the US, UK, Australia, and Canada and the company plans to double the number of third-party stores with the technology in 2024. 

Big Tech Set Its Sights on Reinventing Checkout. Consumers Said ‘Not So Fast’

When it comes to technology and grocery shopping, one primary focus for grocery chains and technology providers in recent years has been the checkout experience.

Amazon and various other technology companies have been developing platforms to enable consumers to skip the checkout counter. These platforms aim to transform the shopping experience into something akin to walking into a giant pantry, loading up your cart, and then walking out without going through a checkout line.

Others (including Amazon) pushed technology into the shopping cart, enabling customers to check out products as they walked through the store, get coupons and ads for special deals, and learn more about items via a built-in touchscreen.

And then there’s online grocery shopping. After two decades of slow adoption by both grocers and shoppers, a pandemic forced every major grocery chain to invest heavily in enabling the easiest of all grocery buying options: letting us shop at home and have our groceries delivered to our door.

Meanwhile, everyday consumers continue to do things the way we’ve always done things. It’s a lazy Sunday, and you’re in no hurry? Get in line and chat it up with the cashier and bagger. Are you hurrying to return to work or arrive home in time for dinner? Jump into the self-checkout line and get out as soon as possible. Too busy to head to the grocery store at all? Order online and have stuff delivered to your home.

In other words, grocery shoppers are not a monolith. Most of us change our behavior depending on the current situation.

But what about Just Walk Out? It’s a radically tech-forward evolution of checkout, but one in which Amazon appears to have widely overestimated just how many people would use it and how easy it would be to implement. As I said in last week’s Food Tech News Show (FTNS), self-checkout fits most shoppers’ needs when they are in a hurry, and there aren’t that many situations where consumers feel they need to skip checkout altogether.

As for self-checkout, it definitely isn’t perfect and could be made a much better experience. As Scott Heimendinger said on the FTNS, self-checkout can sometimes be unnecessarily difficult, almost like plugging in a USB. Amazon and others should probably spend their time using technology to make self-checkout work better.

We love robots - FTNS

Target is doing something about self-checkout, changes which it claims will allow shoppers to get out quicker. According to the company, self-checkout lines with cameras were able to check out twice as fast as self-checkout lines without a camera. Of course, their motivation is mostly somewhat self-motivated, driven by the retailer’s desire to limit theft, so my guess is there’s a good chance they can bungle the rollout if it doesn’t deliver clear benefits and customers are feeling spied on.

All that said, while some shoppers may not like it, the combination of computer vision and self-checkout might be the future, particularly if it makes the self-checkout experience less painful than it currently is. Because of this, Amazon should look at repurposing its Just Walk Out into a self-checkout accelerator, not a platform for making shoppers feel like they are shoplifting. For now, however, they’re emphasizing the rollout of their Dash shopping carts, a solution that is unclear if shoppers are asking for. Others, like Instacart, are also betting big on as well. The company had a blog post touting their progress today, saying they plan to have ‘thousands’ of shopping carts deployed by the end of 2024.

Just Walk Out and other light-touch self-checkout will thrive in the near term in shopping contexts where a consumer needs one or two items and is in a hurry, such as airports and stadiums. One of the smartest implementations I’ve seen with self-checkout is at Costa Coffee at SeaTac airport, where they had a Mashgin AI-powered self-checkout station with a dedicated line for customers who just wanted drip coffee. In other words, a quick and low-touch checkout solution for a product with a high degree of certainty where customers are often in a hurry.

The bottom line is that everyday shoppers will continue to shop the way they’ve become accustomed to, choosing between three primary methods: full-service checkout, self-checkout, and delivery. More advanced technology should primarily focus on improving these existing modes. New technology that allows (or forces) consumers to change their behavior should only be used in scenarios that make sense.

Otherwise, consumers will reject it, and retailers will be forced to retrench, just like we saw last week with Amazon’s pullback of Just Walk Out.

Amazon Pulling ‘Just Walk Out’ from Amazon Fresh Grocery Stores

According to a story published in The Information, Amazon is planning to pull its Just Walk Out cashierless technology from its large-format grocery store, Amazon Fresh.

As part of the move, the company will begin to deploy its Dash smart shopping carts. Like Just Walk Out, the Dash carts have embedded computer vision, allowing customers to scan products as they put them in the cart.

“We’ve also heard from customers that while they enjoyed the benefit of skipping the checkout line with Just Walk Out, they also wanted the ability to easily find nearby products and deals, view their receipt as they shop, and know how much money they saved while shopping throughout the store,” Amazon spokesperson Jessica Martin told Chain Store Age. “To deliver even more convenience to our customers, we’re rolling out Amazon Dash Cart, our smart-shopping carts, which allows customers all these benefits including skipping the checkout line.”

That Amazon pulled it from Fresh stores (of which there are 44 locations, nearly half in California) isn’t the end of Just Walk Out. The company plans to continue using the technology in its small-format Amazon Go stores and stadiums (such as Lumen Field).

The move, which comes as part of a larger effort to redesign its Fresh stores, is yet another sign that the company’s retail and grocery strategy remains somewhat rudderless. The company paused its Fresh expansion in February of last year before deciding to apparently unpause it later in the year. It has also closed its Fresh pickup format early this year.

My guess is that part of the challenge is that the company seems confused about whether it’s a technology platform provider or one serious about pursuing its own grocery store business. The acquisition of Whole Foods signaled the company was serious about grocery, but since then, it has continued to expand its efforts to pioneer AI and computer vision formats while seemingly neglecting to evolve the Whole Foods stores as the grocery-buying public started craving more flexible solutions around things like curbside pickup.

This move seems to signal that Amazon’s grocery business is beginning to embrace the post-Bezos era (he’s largely stepped away from day-to-day and is focused on his rocket company), moving on from being a company that often pushed new and experimental concepts at the expense building businesses with more conventional, straightforward approaches. According to The Information, the company plans to “spruce up” its stores “across the board” as it prepares to expand Amazon Fresh locations later in 2024.

Amazon Details Usage of Generative AI-Created Synthetic Data to Train Just Walk Out Technology

For a while now, we’ve known the basic gist of how Amazon’s Just Walk Out technology works: A combination of computer vision, machine learning, and other sensor data helps enable a friction-free shopping experience in which customers pick items off the shelf and walk out the door without ever having to stop at a cash register.

But in a recent blog post by Amazon’s retail technology team, the company explained how it all worked in greater detail than we’ve seen in the past, including how the company has been using generative AI to train its Just Walk Out platform for long-tail cases that are rare but entirely possible in the unpredictable environment of retail.

According to Gérard Medioni, vice president and distinguished scientist at Amazon, the company uses a generative AI called a generative adversarial network (GAN) to create synthetic data for training Amazon’s Just Walk Out technology. The Just Walk Out team used datasets from millions of AI-generated synthetic images and video clips mimicking realistic, and sometimes rare, shopping scenarios, including variations in lighting, store layouts, and crowd sizes. According to Amazon, this training using generative AI-created synthetic visual data enables Just Walk Out to recognize and properly interpret millions of customer actions.

“When the customer exits, having an accurate account of their purchases is critical,” Medioni said.

The company also went into detail about how Just Walk Out and its Amazon One palm-based bioauthentication technology does – and don’t – work together. According to Amazon, the two systems operate independently of each other, keeping a person’s biometric information associated with their payment separate from Just Walk Out. When a shopper enters the store, the Just Walk Out system assigns the shopper a temporary numeric code, which serves as their unique digital signature for that shopping trip. When a shopper exits, the code disappears. When they come back, they get a new code.

Medioni says that Just Walk Out associates a person’s “pixels” to the one-time payment code assigned for that trip and the products they pick up off the shelf.

“Just Walk Out tech doesn’t collect any biometrics. All we need to know is where that person is on the floor, and where their hands are in relation to the store’s merchandise.”

According to Medioni, the system is sophisticated enough to track groups of shoppers assigned to a single payment instrument, and the system can create a single receipt for a group shopping trip.

“We had a tour bus that came in one day, and they had 90 people all paying with a single credit card,” Medioni adds. “Even if people leave the store separately and we can still keep track of the group’s purchases.”

While Amazon has shown mixed signals regarding its retail footprint, the company appears to remain interested in developing its technology platform for usage by other retailers. My guess is they’ll likely see some smaller retailers and non-grocers (like stadiums/sports venues) adopt the technology, but larger grocers will remain reticent to jump on board with technology developed by a competitor.

If you’re interested in how generative AI will change food retail, join us at the Spoon’s Food AI Summit on October 25th in Alameda!

Standard AI Bridges Gap Between Autonomous & Self-Checkout With Acquisition of Skip

San Francisco-based company Standard AI has announced the acquisition of self-checkout solution provider Skip. The deal will combine Standard AI’s autonomous retail platform with Skip’s self-checkout technology, which they say will allow retailers to resolve labor challenges with a path toward autonomous checkout. Standard AI will integrate Skip’s cloud-based point-of-sale system with its back office technology post-acquisition, which Standard AI says will streamline operations and provide greater control over pricing and promotions.

Standard AI, which has been deployed with retailers such as Circle K, uses machine vision and AI to enable autonomous storefronts. The company says its technology is deployed and customized in stores without the retailer having to change their existing formats and footprints.

The deal allows Standard AI to offer more options, allowing retailers to reach a wider swath of customers. An autonomous just-walk-out experience can be almost jarring if you’ve never done it before. With a self-checkout option, retailers can give those customers who are not quite ready to just pick up and walk out with their items an option with which they are more comfortable.

Standard AI says it plans to bring to market the world’s first platform that connects self-checkout with autonomous retail later this year, allowing retailers to have the option to use the platform with or without the self-checkout kiosks.

The deal signals what could be a period of consolidation of the retail checkout automation market. Nearly seven years after the launch of Amazon Go, the space has been filled with several startups that offer platforms leveraging machine vision, sensors, and other technologies that enable retailers to move away from more traditional checkout formats. With the tightening of venture and other forms of capital, my guess is we’ll see more combinations like this over the next 12-18 months.

You can watch the video explaining how Standard AI technology is integrated into stores below:

Standard AI's Platform

Amazon Selling its “Just Walk Out” Cashierless Tech to Other Retailers

Amazon is now selling the cashierless technology used in its Go stores to other retailers, Reuters reports. The business line is called “Just Walk Out,” and Amazon says it has already signed up several customers for the service.

The Just Walk Out technology is being used at a number of Amazon Go convenience stores across the country and the just-launched, larger Go Grocery store in Seattle. Using a combination of computer vision, shelf sensors and deep learning, Just Walk Out allows shoppers to do just that — grab an item and walk out, getting charged automatically upon exit.

There is one big difference in the way this technology is being offered to other retailers. Unlike Go stores, where you scan the Amazon Go app on your phone in order to enter a store, Just Walk Out for third party retailers will require shoppers to insert their credit card into a (Amazon branded) turnstile as they enter. The technology still monitors your shopping the same way, it would at an Amazon Go store, but if you need a receipt, there will be a kiosk for you to associate an email with that credit card.

As Reuters points out, this type of setup does bring up the question of who owns the customer data. If customers are handing over their email address, what type of relationship is it entering into with Amazon? Dilip Kumar, Amazon’s vice president of physical retail and technology told Reuters that Amazon saves the email and credit card information only for the purpose of charging the customer.

But that still leaves a ton of data around when customers shop, how often, what times, what they pick up in stores and what they put back, etc.. That’s a treasure trove of information that Amazon could use to feed its own algorithms and apply to its own real world retail game.

The actual news today isn’t actually that big of a surprise for industry watchers. CNBC reported last year that Amazon was looking to sell its cashierless tech to third parties to places like movie theaters.

But the big question is who will adopt this. It’s unlikely that grocery retailers like Albertsons or Kroger would be interested. They are already locked in a heated battle with Amazon as the e-commerce giant rolls out physical stores like the aforementioned Go Grocery, and its forthcoming chain of full-on supermarkets in Los Angeles. So there’s no real incentive to hand Amazon more money.

Plus, there are a lot of options for retailers looking to add cashierless checkout to their retail experiences. Trigo, Grabango, AiFi, Caper, and Zippin all offer cashierless solutions that either retrofit existing stores, or extend their brand with smaller, self-contained, standalone cashierless retail experiences.

Having said that, in many ways, Amazon could be for cashierless tech was IBM was for computing technology in the past. The saying used to go “no one gets fired for buying IBM,” and with Amazon’s size and track record, the Bezos behemoth could be the safe choice for smaller retailers to up their high-tech offerings.

Amazon Wants To Automate The Grocery Store

Last month, I wrote about Amazon’s attack on the middle of the grocery store as it installs millions of auto-replenishment buttons in our homes.  As it turns out, this may only be the beginning.

Today the company announced a new grocery store concept called Amazon Go. The first store, which is currently open only to Amazon employees, utilizes what the company calls “Just Walk Out” technology to eliminate the worst part about going to the grocery store: cashiers and checkout lines.

What is Just Walk Out? According to the company’s intro video, it’s a mix of “computer vision, deep learning algorithms and sensor fusion” that combine to eliminate the need for the old-fashioned checkout. While they don’t go into more detail than that, here’s what I think they’re doing:

Computer vision: This means cameras to both identify the person and what they put in their bag. This most likely includes facial recognition technology for shopper identification, so your face is now your ID.

Sensor fusion: Likely a combination of proximity sensors for the shopper and RFID tags for the products themselves. For the RFID tagging, it’s probably not unlike those conveyor belt sushi restaurants where they can tell when you pick something up, only here you’re not sitting at a table but moving around the store.

Deep learning algorithms: the most mysterious part of ‘Just Walk Out’, but likely also the most valuable to Amazon. That’s because deep learning probably means behavior tracking, where Amazon not only learns what you buy, but what you almost buy. Stores like b8ta are experimenting with understanding how long consumers linger and interact with different products, and I would bet Amazon will use proximity sensors, cameras and sensors on the devices themselves to better understand your interests.

If you’re a consumer, the benefits are obvious. Being able to just pick up a quick meal and walk out is something anyone who has waited in a long line during lunch hour would jump on immediately. While there’s a slight creepiness factor to the tracking technology and just how much Amazon will know about you once you walk in their store, I have a feeling most consumers will shrug their shoulders at the big-brotherness of it all. After all, it’s not like they’re shying away from putting an always-listening speaker in their homes.

Bottom line, if you’re a grocery store and aren’t worried about what Amazon is doing, you should be. With Just Walk Out, they are looking to utilize IoT, AI and mobile to extend their dominance from the online and in-home commerce world to the corner grocery store.

Introducing Amazon Go and the world’s most advanced shopping technology