Close Ad

The Spoon

Daily news and analysis about the food tech revolution

Mashgin

How Do You Train AI-Powered Checkout To Recognize A Product? In Vegas (& Elsewhere), You Throw It Like Dice

When it comes to training machine vision and AI-powered retail checkout systems, packaged goods and locally created food items are treated very differently.

That’s at least according to Mashgin, a maker of touchless checkout systems. Company spokesperson Toby Awalt said that’s because another store on the network has likely already added that bag of chips or candy bar to their 10 thousand plus item database.

Not so when it comes to locally made food items.

“CPG items, we have to do less and less because there’s enough overlap,” said Awalt, who gave us a walkthrough of the system at CES 2022. “But for dishes, we’ll do every time.”

According to Awalt, adding a new food menu item for a restaurant doesn’t take that since most cafeterias or restaurants only serve between 15 and 50 items.

“You can do that relatively quickly,” he said.

Still, a new packaged good has to be entered into the system now and then. Whenever that happens, the operator has to position the package in several different positions to give the system enough info to recognize the product whenever it shows up under the camera.

Mashgin’s Toby Awalt Rolls the Häagen-Dazs

“I actually do dice rolls with the product,” said Awalt, throwing a Häagen-Dazs ice cream bar onto the tray.

According to Mashgin, the company recommends the system capture 20 to 50 total positions of a product so it can recognize the product from various angles and also distinguish between different variations within the same product line (such as two different flavors of ice cream or potato chips).

You can watch a walkthrough of the Mashgin system below.

The Spoon checks out Mashgin's AI-Powered Checkout at CES 2022

Mashgin’s Cashierless Checkout Tech now in Use at Nearly 500 Locations

Mashgin, which makes a cashierless checkout kiosk, announced today that its technology is now in use in nearly 500 locations worldwide and that it has processed roughly $100 million in sales transactions.

Unlike other players in the cashierless checkout space that are relying on in-store camera installations or smart shopping carts, Mashgin’s solution uses a countertop device that can sit anywhere in a store. Shoppers place items on the device where cameras and computer vision identify all of the products and tally up the bill. Up until now, Mashgin accepted credit cards for payment, but the company also announced today an integration with Glory to enable contactless acceptance of cash payments as well.

I spoke with Jack Hogan, Vice President of Partnerships at Mashgin by video chat this week. He said the sweet spot for Mashgin’s technology is in places like sports stadiums and convenience stores — locations where people are grabbing 1 – 10 items and want to get in and out of the store quickly.

In addition to identifying consumer packaged goods, Mashgin’s technology also identifies plated foods in settings like cafeterias. Users place there tray of food on the Mashgin device and the cameras can identify everything on the plate and charge accordingly.

How does Mashgin touchless self-checkout change the checkout experience at Delek?

Cashierless checkout has been one of the big news stories of 2021, as the pandemic accelerated retailer’s plans for more contactless shopping experiences. Throughout the year we’ve seen a number of cashierless checkout store launches with startups around the world including Zippin, AiFi, Imagr, and Trigo.

Hogan said that even though the pandemic has pushed retailers into more human-free checkout solutions, that’s not the biggest concern from partners. “The number one thing people have said is speed,” Hogan said, “Get in and out as soon as possible.” With cashierless checkout, retailers are able to kill two birds with one stone, providing line-free speed with a contactless experience. Mashgin said that deployments of its checkout kiosks grew more than 100 percent in 2021 and can now be found in convenience stores, airports and sporting arenas such as Mile High Stadium and Madison Square Garden.

Though the cashierless checkout space is crowded, Mashgin’s most direct competitor at this point is Caper. In addition to making smart shopping carts, Caper debuted its own computer vision-powered countertop kiosk last year. The small footprint of devices like Caper’s and Mashgin’s could prove attractive to c-stores and other locations where consumers aren’t buying a lot of stuff and want to get in and out of the store quickly. A countertop device doesn’t require retrofitting a store with cameras or buying new shopping carts, and can be set up with easily at a checkout stand.

It’s not hard to imagine that as part of the growth of cashierless checkout over the coming years, we’ll be seeing more players offer a similar type of kiosk as well.

Between Faster Checkout and Robot Tater Tots, the Food Tech Game is Strong at Stadiums

If you want to see some of the coolest innovation in food technology, you need to get in the game. Well, at least get a ticket to the game because stadiums and arenas are fast becoming hotbeds for new ways to sell and get you your food quickly.

Speed is the name of the game and the impetus for most of this disruption happening at large sporting and entertainment venues. The faster attendees can order and get their food, the less time they spend away from the game or concert, and, ideally, the more stuff they buy.

The most recent example is the partnership Postmates announced last week with Yankee Stadium. The partnership, which is similar to one the delivery service has with Dodgers Stadium in LA, and allows attendees to order their food from their seats and pick it up at a designated Postmates Pickup point.

Over at Mile High Stadium in Denver, computer vision-powered checkout scanners have been installed throughout the stadium. Shoppers place their items on a scanner created by Mashgin, which automatically identifies what is being purchased, so individual items don’t need to be rung up one at a time.

But innovation isn’t just happening on the checkout side of the food stand; the way stadium food is being made is undergoing an upheaval as well. Robots in particular will play an increasingly important role in making food at large venues. The Dodgers have used Flippy the robot to fry up chicken tenders and tater tots. And back at Mile High Stadium, the Broncos installed a robo-bartender to pour and serve beer.

The point is that stadiums are perfect venues for a lot of the food technology and automation that we write about. They are large, high-traffic areas that provide a good test case for new, automated workflows and systems. The food being served isn’t highly customized, but rather made in bulk, and meant to be more consistent than artisanal. And really, while it’s fun to eat a hot dog at the ballpark, people are there for the game, so as long as they get their grub in a timely manner, they don’t really care how they get it. Seems like a win for everyone involved.