Close Ad

The Spoon

Daily news and analysis about the food tech revolution

Robotics, AI & Data

Sensei Ag Uses AI Platform and Hydroponic Technology to Grow Food

As the world’s population inches towards its estimated 10 billion people by 2050, finding more, not to mention more sustainable, ways to feed people becomes more and more important. High-tech, indoor agriculture is one solution getting a lot of attention lately, and recently, a new company joined the fast-growing sector. Sensei Ag is the brainchild of Oracle’s Larry Ellison and scientist Dr. David Agus, and the company’s goal is to grow more greens using hydroponics and AI.

Based on the small Hawaiian island of Lāna’i, Sensei Ag has built a 100,000 sq. ft. hydroponic pilot greenhouse that is expected to grow 1 million pounds of food per year. I spoke with SenseiAg CEO Sonia Lo by phone this week, and she described the company as an integrated solution to indoor farming that uses the best practices in computer vision, germination, and seeding to optimize indoor growing.

I asked Lo about how the company incorporates AI into their greenhouses. She said that their AI platform will act as a data engine that harnesses global grower knowledge, and will create an algorithm for the best-practices in indoor growing. She did not go into the specifics of their platform, but did mention that this would be made available to other growers, and it would be embedded into each part of their agricultural system. Sensei Ag also uses advanced cameras within their greenhouses to identify pests, pathogens, plant health, and uneven growth in crops. The company’s goal is to enable platforms within the greenhouse to make decisions on growing food autonomous of human intervention.

The COVID-19 pandemic, climate change, and a growing population has forced us to consider the possibility of global food insecurity. In response, companies like Phytoponics, Element Farms, and Gotham Greens all operate indoor farms that use hydroponic techniques to grow leafy greens. Meanwhile, companies like Verdeat, Rise Gardens, and Seedo offer at-home vertical farming products that allow you to leafy greens in your living room. 

Sensei Ag grows cherry tomatoes, basil, and butter lettuce, and Lo said that they will definitely be expanding the crops they grow. They are currently scouting for a location in California or Nevada for their flagship farm, which will be used as a template for rolling out future farms. 

Greenfield Robotics Uses Robots to Tackle Weeds

Greenfield Robotics is on a mission to help farmers grow food with fewer chemicals. Rather than using the traditional method of applying herbicides and tilling the ground to control weeds, Greenfield uses a fleet of lightweight robots to take on the task. 

I spoke with Clint Brauer, the CEO of Greenfield Robotics, by phone this week, and he said that the main purpose of Greenfield’s robots is to mow down aggressive broadleaf weeds, specifically the fast-growing pigweed. Greenfield’s robots currently operate in soybean fields, and the next crop will be milo (grain sorghum). 

The robots from Greenfield Robotics weigh only 140lbs, and look like thin, upright vacuums. The perk of creating a small robot is that it is able to go out even in muddy conditions to mow weeds. Brauer said that even after fields received 3 inches of rain, Greenfield’s robots are able to go out and do their job, while a spray rig would easily get stuck in the mud. 

These petite robots are also intelligent, and have the ability to sense depth and crop rows. They essentially function as miniature lawn mowers, eliminating weeds as they travel up and down crop rows. As Greenfield Robotics continues to grow, their goal is to use a fleet of 10 robots to knock out 100 acres in one day. 

Greenfield Robotics is not the only company embracing robots as a solution to using fewer chemicals in agriculture. Farmwise builds self-driving robots equipped with computer vision and AI to identify and eliminate weeds. In the UK, the Small Robot Company uses a multi robot approach to map, identify and use electricity to zap weeds.

Greenfield Robotics has raised $885,000 in capital so far, and is currently raising an angel round. The company has signed up 10 farms in the U.S. to use its robots during the 2020 growing season. 

Report: 80 Percent of Restaurant Jobs Could be Taken Over by Robots

More than 80 percent of restaurant jobs, including cooking, serving and prepping, could be potentially be taken over by automation, according to restaurant consulting firm Aaron Allen & Associates.

Pizza Magazine was first to report on this, writing:

Aaron Allen & Associates shared a graphic proposing that 82 percent of restaurant positions could be automated. The majority of them, or 51 percent, would be server positions. Fifty-seven percent of fast-food and counter workers (or 3.2 million) could be replaced, and the same goes for 38 percent of waiters and waitresses. Twenty-one percent of cooking and food prep positions also could be automated, the company asserts.

Factors that could drive this widespread adoption of automation include continued labor shortage issues for restaurants and the COVID-19 pandemic.

We write about food robots a lot here at The Spoon, and while that 82 percent number is certainly daunting, it’s not completely surprising. Prior to the pandemic, sufficient staffing was an issue for restaurants as many potential workers preferred driving for Uber or doing some other form of gig work that allowed them to set up their own hours.

Thanks to the pandemic, the U.S. is dealing with massive amounts of unemployment, so finding people to work may not be as big an issue in the short term (though there is a debate about workers making more money via the stimulus than at their job). But the bigger problem now is the number of restaurants closing down dine in operations or shuttering altogether, reducing the number of jobs in the industry overall.

COVID accelerated the push towards off-premises dining, which requires a different kind of staffing set up. You don’t need servers if there are no customers sitting at tables to serve. And if a dining room is open, there will be fewer people eating in it to accommodate social distancing.

But even then, COVID has us re-thinking the amount of human-to-human contact as we get our food. We won’t know what the lasting impact of the pandemic on our psyche will be, but there is a good chance we will be more wary of strangers and more cognizant of the number of people who touch our food.

This is another reason why we could see more robots in restaurants. Already, a number of companies like Bear Robotics, Keenon Robotics, and Pudu Technology make server robots that can autonomously shuttle food and empty dishes back and forth from the kitchen. Then there is Flippy from Miso Robotics, which can grill burgers and work the deep fryer. White Castle recently announced that it was piloting Flippy at one of its Chicago locations. There’s also Picnic robots, which can assemble 200 pizzas in an hour.

There have always been deep societal concerns around automation, especially within the restaurant industry, which in addition to be a career many people are passionate about, also serves as an accessible first job for lots of different people. Robots taking more than 80 percent of those jobs will have massive ramifications for the country, the labor force and our collective future.

Now the coronavirus is upending those conversations. There are still issues around equity and the ability for people to find work and training if a robot takes their job, but there is the added wrinkle of what is economical for restaurants to stay in business and what people are comfortable with in their dining experiences.

Yo-Kai Express Adding Boba Dessert to its Machine and Meal Kit

Pretty soon, you’ll be able to get more than just ramen from Yo-Kai Express vending machines and meal kits. Yo-Kai Founder and CEO, Andy Lin, shared on Linkedin today that his company will be adding a frosty dessert to its menus.

The Yo-Kai “signature snow ice,” Lin wrote, consists of “Himalayan salt whip cream with brown sugar boba black milk tea.” No specific timeframe was mentioned but Lin said the dessert would be available in Yo-Kai machines and home meal kits “soon.”

We reached out to Lin to fill in some details and will update the post when those arrive. Overall, this marks another bit of expansion for the vending machine startup.

When we first started covering Yo-Kai, its automated vending machines served hot ramen to customers in high-traffic areas like malls, airports and corporate campuses. But as the pandemic hit the U.S., those locations saw fewer and fewer people.

So, if people can’t come to you, you may as well go directly to them. In April of this year, Yo-Kai started selling ramen meal kits that ship directly to consumers’ doors (if you live on the West Coast).

Like the meal kits, the addition of a cool dessert isn’t too much of a surprise either. Once made, Yo-Kai’s bowls of ramen are frozen and stored that way inside the vending machine before being reheated. A similar, pre-made frosty snow ice dessert in a bowl seems like a pretty easy thing for the Yo-Kai machine to dispense as well.

It should also be noted that according to the Yo-Kai website, two of its machines are installed at Milk Tea labs in San Francisco and San Jose. So offering a boba/black milk tea doesn’t seem like that much of a stretch.

As I’ve written before, I’m all-in on vending machines (and wrote an extensive report on them for Spoon Plus Subscribers), in part because of their ability to offer really good food, from a very compact space around the clock. While I haven’t had the chance to test out Yo-Kai’s snow ice dessert, given how much I liked their ramen meal kit, it’s hard to imagine them making their first foray into desserts a bad one.

Phood Fights Food Waste with Scales, Computer Vision and AI

With the pandemic still raging, restaurants are struggling to stay in business. One way restaurants can help stave off permanent closure is to make sure whatever money they have now is being spent properly and not going to waste. One way to do that is to measure the food being used in meals… and the food going to waste.

Phood is a company that uses a combination of scales, computer vision and artificial intelligence (AI) to help restaurants, cafeterias and other eateries better understand and optimize how their food inventory is being used.

There are three parts to the Phood system: a scale, a camera and a software backend. Food is placed on the scale either before going into a dish (to see how much is being used to make meals) or afterwards (to see how much waste is being generated). There’s a camera mounted above the scale that uses AI to automatically identify what each food item is.

Phood’s system also integrates with a restaurant’s existing POS and inventory management software to track how much of a particular item is being used and who supplied it. Based on that information, restaurants can then realign both production and ordering to reduce waste. So if a cafeteria or restaurant winds up with too many leftover mixed vegetables, that point is highlighted in a Phood dashboard so the manager can take appropriate action (make less or order more).

On its website, Phood claims that its solution can help reduce food waste by 42 percent. I spoke with Phood Founder, Luc Dang, by phone this week and he said Phood can provide a cost savings of 8 – 10 percent. In the thin margin world of restaurants, those savings can go a long way.

Phood, which began using computer vision and AI in its product last year and has raised $100,000 in seed funding, isn’t the only company fighting food waste in this manner. Winnow, which raised $12 million last year, uses a similar scale, computer vision and AI approach. LeanPath does much the same thing to help change behavior in the kitchen (e.g., less wasteful chopping of veggies or trimming of meat).

During these unpredictable times when the future of just about every eatery hangs in the balance, using a tool like Phood could not only help close down food waste, but also play its part in helping keep restaurants open.

Postmates Now Delivering Dodger Stadium Food to LA Doors

Dodger fans now can change up “Take Me Out to the Ball Game” with some new lyrics:

Take a walk down my hallway
Take a seat on the couch
Postmates and Dodgers now deliver snacks
Just in time now that baseball is back

Baseball fans may be stuck at home, but if you’re rooting for the Dodgers in LA, you can still eat like you’re at the stadium. Postmates announced earlier this week that has partnered with Home Plates to offer home delivery of Dodger Stadium food.

From a Postmates blog post describing the program:

Home Plates will serve Dodger Stadium fan-favorites like Dodger Dogs, micheladas, and garlic fries to go along with snacks, salads, bar, and dessert options. In addition to premium Dodger Dogs, items specially produced by Home Plates include individual thin-crust Brooklyn-style pizza, carne asada helmet nachos, and Dodgers’ blue gelato. Also, check out the family party pack and catering options.

Right now, delivery is only available in the Hollywood and West Hollywood neighborhoods of Los Angeles, with plans to expand to other areas of the city. Hollywood and West Hollywood, of course, are also the neighborhoods where Postmates uses its Serve robot to make deliveries. So your Dodger Dog may be brought to you by a cute robot!

This isn’t the first deal between Postmates and the Dodgers. Last year, the two teamed up for online ordering and food pickup inside Dodger Stadium. But with stadiums shut down, this is the next best thing for those wanting more of a realistic baseball experience while watching the games at home.

Postmates was recently acquired by Uber for $2.65 billion, which could buy a lot of peanuts and cracker jacks.

I say this without judgment, but given the greasy nature of stadium food, I’m curious as to how well it will travel for delivery. Obviously, downing a Dodger Dog on your recliner at home is a lot different from eating one amongst tens of thousands of your fellow fans. But will the food even be that good outside a stadium? If you’re in LA and try this out, drop us a line and let us know if it was a home run or a strikeout.

Woodman’s Market Adds Badger Technologies’ Robots for Shelf Scanning

Badger Technologies announced yesterday that its shelf-scanning robots will be roaming the aisles of Woodman’s Markets throughout Wisconsin and Illinois come the end of the year.

According to a press release sent to The Spoon, the robots will “monitor product availability, verify prices and deliver precise location data for more than 100,000 items at each location.” The plan is for the robots to be present in all 18 Woodman’s locations by the end of 2020.

The last time we wrote about Badger was January of 2019, when Ahold Delhaize ordered 500 of them for its GIANT/MARTIN and Stop & Shop stores. Those robots however, were being brought on to detect spills and other messes. (The robots didn’t even clean those up, they just alerted staff when there was a mess.)

Unlike that previous use case, Woodman’s, is using Badger’s robots to help automate inventory management. Woodman’s is far from the only store using robots for this purpose. Giant Eagle and Schnuck’s stores use Simbe Robotics’ Tally robot. And Walmart expanded the use of Bossa Nova’s shelf-scanning robot to 300 locations last year.

Pre-COVID, robots in grocery stores were seen as a threat to human jobs. Retailers like Walmart typically defended the increased automation because robots can do the manual, repetitive tasks — like monitoring inventory on store shelves — faster and more accurately than a human. Badger claims that its robots can detect out-of-stock items with more than 95 percent accuracy, and incorrect or mispriced products with 90 percent accuracy. Additionally, robots can work all day without getting and don’t need to take breaks.

During and post-COVID, however, those labor arguments may be shunted to the side as retailers and customers alike seek to cut down person-to-person interactions. Robots don’t get sick, don’t cough and can be easily sanitized by wiping them down.

Another argument in favor of more robots could be more accurate, real-time inventory data for stores and customers. With a robot roaming the aisles providing more precise item stocking information, retailers can better manage product flow and provide more accurate inventory data for people shopping online (which is happening in record numbers).

Robots are having a bit of a moment, thanks to this pandemic. Elsewhere today, online grocer FreshDirect announced a robot-powered micro-fulfillment center that is being built out by Fabric. With the coronavirus surging and re-surging around the world, robots and automation are likely to continue their march into retail throughout the rest of this year and into next.

QCify Goes 3D for Quality Control and Fair Pricing in the Food Supply Chain

As the economy is barraged daily by some kind of pandemic-related bad news, many businesses remain closed (or serving far fewer customers), job losses continue to pile up and people all over are being more cautious about how much they spend.

The food supply chain is not immune from this belt-tightenting. Looking to save some money, food buyers may haggle more vigorously over what they pay per pound for something like almonds, costing the growers and processors money. This problem, Raf Peeters told me, is where QCify can help.

Peeters is the CEO of QCify (pronounced kew-sih-fye), which uses a combination of computer vision and artificial intelligence (AI) to perform quality control on food items at processing plants. Right now QCify inspects almonds and pistachios by running samples through a special machine that uses six cameras to capture a 3D image of each almond. The company’s AI then analyzes the image and grades the almonds based on USDA (or other) criteria such as size, color, insect damage, imperfections, etc.

All that data collected by the inspection machines are sent back to to QCify HQ, where it is incorporated into the company’s algorithms. Twice a year QCify then sends out updates to all of its installed machines, which means that even if a customer bought a QCify system a couple years ago, it will run the newest AI. “Customers feel like they have the latest and greatest,” Peeters told me by phone this week.

Qcify explainer video

The result of all this computer vision and machine learning is that nut processors can set a fair price for their wares, based on objective criteria (like the USDA grading). Right now, QCify works with almonds and pistachios, and has customers in both the U.S. and Australia. A buyer can’t argue over the quality of the almonds (and thereby demand a lower price) because the processor not only has the grade from the QCify system, but it can also produce the sample images to show exactly what quality the almonds or pistachios are in.

QCify isn’t the only company looking to remove biases from the food supply chain using computer vision and AI. AgShift and Intello Labs do much the same thing. Peeters said that QCify is different from the competition because its six-camera setup captures 3D images of the nuts, instead of just scanning the top an bottom of the food, which Peeters claims is what his competition does.

QCify was founded in 2015 and Peeters said they company has only raised an unspecified amount of angel investment money. The company sells the machines themselves and charges a monthly/annual subscription fee for updates and calibration. While he wouldn’t reveal pricing, Peeters said that customers can earn their money back within a year.

In these cash-strapped times, a faster ROI isn’t just peanuts, which, coincidentally is one of the next nut categories QCify is expanding into.

Dexterity Robotics Has a Light Touch for Food Packing (and $56M in Funding)

Dexterity Robotics unveiled its new full-stack robotics solution for industrial applications like warehouses and supply chain management yesterday. In addition to taking the wraps off its product, Dexterity also revealed that it has raised $56.2 million, including venture investments and debt from Kleiner Perkins, Lightspeed Venture Partners, Obvious Ventures, Pacific West Bank, B37 Ventures, Presidio (Sumitomo) Ventures, Blackhorn Ventures, Liquid 2 Ventures, and Stanford StartX.

Part of Dexterity’s pitch is that its grippers have enough sensitivity to pick up and pack soft, delicate items like bread. Jonathan Vanian over at Fortune got an early look at the Dexterity robots in action and wrote:

The robotic arms can figure out how much pressure is needed to apply to a particular object, which is helpful so they don’t smash a spongy material like bread when they grip it. The software also helps the robot decide to gently lower the bread and other objects into a crate instead of dropping it carelessly into a container, which Menon described as a sort of Achilles heel for some pick-and-place machines.

Food is actually a good application to help advance the entire field of robotics. In addition to much of it being soft and malleable like bread, food is often irregularly shaped (produce) and delicate (eggs). That’s why Sony partnered with Carnegie-Mellon to develop food robots, and why researchers at MIT and Harvard developed an origami robot. Soft Robotics has also developed special octopus-like rubber grippers to handle food with odd shapes and textures.

The goal with all this is robo-development is to automate the repetitive tasks like packing boxes in large warehouses. Through computer vision and a soft touch, as it were, robots can accurately sort and pack items for shipment around the clock, ideally bringing more consistency and speed to the supply chain.

With a $56.2 million warchest, we’ll have to see now if Dexterity has the agility to make an impact on the supply chain.

Flytrex Trialing Delivery by Drone in North Dakota and North Carolina

Restaurant meal delivery by drone always seems to be just over the horizon, as it were. While there are a number of tests from different companies going on around the world, delivery by drone just doesn’t quite feel like a real thing yet.

You wouldn’t get that from speaking with Yariv Bash, Co-Founder and CEO of Flytrex, a Tel Aviv-based drone startup. Bash is very optimistic about the future of drone delivery, obviously, and says his company will be powering drone deliveries for thousands of customers by early next year.

Flytrex has already done thousands of drone deliveries in Reykjavik, Iceland since 2018. Here in the U.S., at the King’s Walk golf course in North Dakota, it offers drone service now as well (“No more waiting around for the beverage cart.”), according to Flytrex’s website. The company is also currently prepping another pilot in North Carolina.

The Flytrex drone is a hex-copter capable of carrying a 6.5 pound payload (enough to feed a family of four, said Bash), up to 40 m.p.h., with a range of six miles. The system is really built for suburban sprawl, and uses a tether to lower deliveries down to people when it arrives at its destination. Bash said that while a human driver can typically only make two to three deliveries an hour, a Flytrex drone operator can make up to 15.

“But the interesting part is on the inside,” said Bash about his drones. “There are multiple levels of redundancy. It can sustain motor loss, battery loss, communication failure. If everything fails, there is an independent parachute.”

Safety measures like those are top of mind in any conversation about drone delivery. It’s one thing if a rover robotstops dead on the sidewalk, quite another if a drone stops working above your house (or head).

And while the COVID pandemic may be accelerating the adoption of other contactless forms of delivery (like robots), Bash said that all stakeholders are taking the safety of drones very seriously. “The FAA is not willing to lower the bar on safety,” Bash said “but it is willing to work a lot harder with you.”

While safety is a priority, Bash also said that other regulators are excited about drone delivery. “On the local governmental levels, so far, everyone is really happy and wants us to start operating in their areas,” he said.

Flytrex is just one of many companies in various states of drone delivery around the world. There’s Manna in Ireland, Zomato in India, and Fling in Thailand. Domestically, Google Wing has been making deliveries in Virginia, Deuce Drone will be making grocery deliveries in Mobile, AL, Uber is supposed to be testing drone delivery this summer in San Diego.

Bash told me that his company will expand service by partnering in new markets with smaller, existing aviation companies that currently run their own manned flight companies. Those companies are already used to complex nature of flight and the regulatory issues surrounding them.

Though he didn’t spell out many details, Flytrex will also be partnering with unnamed companies to build out a marketplace of restaurants that will offer a drone delivery option. Those restaurants will charge an unspecified delivery fee that will be split between Flytrex and its aviation partner.

So far Flytrex has raised $11.5 million in funding. “We expect to be servicing hundreds of clients in a few months and then thousands early next year,” Bash said.

If Bash and Co. can make it work, drones will finally be crossing that far off horizon.

Kiwi’s Delivery Robots Roll Onto the Streets of San Jose, CA

Kiwi announced today that its rover robots are now delivering food and other items around two areas in San Jose, CA. According to Kiwi, what makes this rollout unique is that the company has partnered with the city itself to help mitigate deployment and maintenance issues that come with robots scurrying around city sidewalks.

Starting today, a “squad” of 25 robots will be making deliveries in the downtown area and Buena Vista neighborhoods of San Jose. As many as 120 businesses will be participating in the program.

Kiwi is also working with the city of San Jose to help with the management of the roaming robots. Kiwi Co-Founder and CEO, Felipe Chávez, told me by phone last week that previously, if a robot got stuck or broke down (or caught fire), someone would have to notify the city, which would then notify the company operating the robot. The company would then have to dispatch someone to take care of the issue, and the whole process could take a long time.

Chávez said what Kiwi has done is basically plugged into a citywide system. All of Kiwi’s robots are now displayed on a city dashboard so if one accidentally stops and blocks a curb, officials knows exactly where it is and can alert Kiwi right away.

As part of today’s announcement, Kiwi also said that it now has partnerships with Shopify and Ordermark. This allows businesses and restaurants with accounts with either of those services to easily add Kiwi’s robots to their delivery rosters.

Robots have become a more common means of delivery over the last few months. The pandemic has pushed restaurants and other businesses to adopt contactless delivery procedures, which Kiwi provides. Users order from participating restaurants on their phones select robot delivery and use their phones to unlock the robot cargo compartment when it arrives.

Additionally, robot delivery is just $3.99 per order, which the restaurant can pass on to the consumer. Four bucks to bring a burrito to your feet isn’t that bad, and there are no pesky third-party commissions eating further into a restaurant’s margins (or adding that much to a customer’s bill).

With the pandemic still raging across the country and restaurants being forced to close down again, robotic delivery can also help keep restaurant sales going, which provide much needed sales tax revenue for cities.

The bigger story here, however, might be how Kiwi has pivoted and adapted. When we first covered Kiwi, the company was focused college campus delivery, and was creating its own marketplace of restaurants that required downloading the Kiwi app to use.

First off, colleges shut down, all but eliminating campuses as an avenue for delivery. Starship, which had a number of high-profile college partnerships, had to make a similar municipal pivot.

But Kiwi is a robotics company, so building robots and a consumer marketplace was a big undertaking. So at the beginning of the year the company shifted to its current B2B model. Kiwi has also since the beginning of this year had a partnership with Latin American delivery service Rappi. Since pivoting, Chávez said Kiwi has booked 50 times more revenue than all of last year.

San Jose is just the beginning for Kiwi, as Chavez said that the company has two more high-profile partnerships that he can’t announce yet and the company is in the process of building 500 robots. Kiwi will release them in “squads” of 25 to different locations, with two humans managing and maintaining each squad.

Speaking of squad goals, if you see a Kiwi bot making a food delivery in San Jose, snap a pic and send it to us.

The EPIC-KITCHENS Project is Building a Foundation For Artificial Intelligence in the Kitchen

From the post in 2018:

The ultimate goal behind EPIC-KITCHENS is to create an open dataset about kitchen-centric objects, behavior, and interactions upon which researchers across the world can then focus their deep-learning algorithms on in the hope of advancing artificial intelligence in the kitchen.

Since those early days, the project has continued to progress, recently releasing a newly expanded dataset and publishing the results from the second annual challenge. The first research challenge, completed in 2019, was focused researchers building models that can recognize actions in the kitchen. The recently completed challenge focused on action anticipation, where they asked researchers to predict what action would take place after one second of video.

Researchers who competed in the most research challenge include teams from a variety of universities spanning the globe from Cambridge to Georgia to Singapore as well as some corporate research labs such as the AI team from Facebook.

I recently caught up with the resesarch lead for EPIC-KITCHENS, Dr. Dima Damen from the University of Bristol in the United Kingdom, who told me that the various research teams competing used a variety of approaches to help make their systems better at recognizing and predicting actions based on the information from the video.

“There are some people who’ve used audio,” said Damen. “So they’ve used the audio from the video to identify something like opening the tap versus closing the tap. Traditionally, computer vision has relied on just images without like videos without sound.”

“There are some people who looked on a very big set of things, at what happened the past minute, because that’s helping them. And there are people who said, ‘no, I’ll focus on the objects, like where the hand is, where the object is, that’s a better approach.'”

For the next set of challenges, the group is providing a newly expanded set of data and asking them to focus on things such as “test of time”, where they ask if models trained two years ago still perform well and “scalabilty,” where they will have researchers look at whether more data is better.

Part of the expanded data will be a newly broadened dataset called EPIC-KITCHEN-100, where new footage brings the total number of hours of video captured to 100. According to Damen, the new video is from a cohort that included participants from both the previous study (half of the original 32 participants agreed to participate again) and 8 new participants.

According to Damen, by bringing back past participants, it will allow the computer models to better understand kitchen behavior by factoring in what happens with the passage of time, like in real life, but also better understanding how small changes can impact the results.

“It’s the natural progression, like how life will be,” said Damen. “The question is what happens to computer vision in the meanwhile? So it’s tiny tiny changes, right? It’s a slightly new camera, people might have moved home, and then we’re asking more questions that we believe would be interest to the community.”

Damen said she hopes that her technology can help build better technology and systems that could be of help to humans who need assistance.

“So there are new questions that are being asked which, interestingly, even the assistive technology community is not talking about. As in, if you want to help someone, sometimes you can guess what they’re doing, but many times you can’t.”

Spoon Plus Subscribers can read the full transcript of our conversation and watch my video interview with Daman below.