• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Skip to navigation
Close Ad

The Spoon

Daily news and analysis about the food tech revolution

  • Home
  • Podcasts
  • Events
  • Newsletter
  • Connect
    • Custom Events
    • Slack
    • RSS
    • Send us a Tip
  • Advertise
  • Consulting
  • About
The Spoon
  • Home
  • Podcasts
  • Newsletter
  • Events
  • Advertise
  • About

Eating Disorder Org’s AI Blunder is a Cautionary Tale About Embracing Tech for Fundamentally Human Roles

by Michael Wolf
May 31, 2023May 31, 2023Filed under:
  • AI
  • News
  • Click to share on Twitter (Opens in new window)
  • Click to share on LinkedIn (Opens in new window)
  • Click to share on Facebook (Opens in new window)
  • Click to share on Reddit (Opens in new window)
  • Click to email this to a friend (Opens in new window)

One of the ongoing debates in tech circles and beyond is how fast AI will replace humans in certain lines of work. One role where we’ve already seen organizations embrace the technology is in customer support, deploying AI-powered customer interfaces to act as the first line of contact to handle inbound queries and provide critical information to customers.

The only problem? Sometimes the information they provide is wrong and potentially harmful to an organization’s customers. To illustrate this, we need to look no further than this week’s news about efforts by the National Eating Disorder Association to use AI-powered chatbots to replace human workers for the organization’s call helpline. The group announced a chatbot named Tessa earlier this month after the helpline workers decided to unionize and, just a couple of weeks later, announced they would shut the chatbot down.

The quick about-face resulted from the chatbot giving out information that, according to NEDA, “was harmful and unrelated to the program.” This included giving weight loss advice to a body-positive activist named Sharon Maxwell, who has a history of eating disorders. Maxwell documented the interaction in which the bot told her to weigh herself daily and track her calories. In doing so, the bot went off-script since it was only supposed to walk users through the organization’s eating disorder prevention program and refer them to other resources.

While one has to question the decision-making of an organization that thought it could replace professionals trained to help those with sensitive health and mental wellness challenges, the example of NEDA is a cautionary tale for any organization eager to replace humans with AI. In the world of food and nutrition, AI can be a valuable tool to provide information to customers. However, the potential cost savings and efficiency the technology provides must be balanced against the need for a nuanced human understanding of the sensitive issues and the potential damage bad information could cause.

NEDA saw AI as a quick fix to what it saw as a nuisance in the form of real human workers and their pesky desire to organize a union to force change in the workplace. But unfortunately, in swapping out humans for a computer simulation of humans, the organization lost sight of the fact that serving their community requires a fundamentally human form of expression in empathy, something AI is famously bad at.

All forms of customer interaction are not created equal. An AI that asks if you want a drink with your burger at the drive-thru is probably going to be suitable in most scenarios, but even in those scenarios, it’s perhaps best to tightly guardrail the AI’s knowledge set and build in offramps to the system where customers can seamlessly be handed over to an actual human in case they have a specialized question or if there’s any potential for doing more harm than good during the interaction.


Related

Checkers To Roll Out AI-Powered Voice Tech to 267 Restaurant Drive-Thrus By End of 2022

This week restaurant chain Checkers & Rally's announced a deal with Presto, a maker of restaurant technology, in which the drive-thru focused chain will roll out Presto's AI-powered voice assistant technology chain-wide by the end of the year. The announcement comes after early trials showed a 98% order accuracy for…

Chipotle ‘Quietly’ Introducing AI Into More Stores

Chipotle has been implementing more artificial intelligence (AI) into its stores this year in the form of AI-powered voice assistants for phone orders, according to Nation’s Restaurant News. The chain has already implemented the technology into 1,800 locations so far this year, and plans to have the system in remaining…

Wendy’s Announces FreshAI, a Generative AI for Drive-Thrus Powered by Google Cloud

Today Wendy's announced it is working with Google Cloud to develop a generative AI solution for drive-thrus called Wendy's Fresh AI. The new solution, which is powered by Google Cloud's generative AI and large language model technology, will go into a pilot test next month at a Wendy's company-operated store…

Get the Spoon in your inbox

Just enter your email and we’ll take care of the rest:

Find us on some of these other platforms:

  • Apple Podcasts
  • Spotify
Tagged:
  • AI
  • food tech

Post navigation

Previous Post Serve Robotics Strikes Deal With Uber to Scale Up to Two Thousand Sidewalk Delivery Robots
Next Post Kittch Partners With Qualcomm for Augmented Reality-Powered Cooking

Reader Interactions

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Primary Sidebar

Get The Spoon in Your Inbox

The Spoon Podcast Network!

Feed your mind! Subscribe to one of our podcasts!

Report: Restaurant Tech Funding Drops to $1.3B in 2024, But AI & Automation Provide Glimmer of Hope
Don’t Forget to Tip Your Robot: Survey Shows Diners Not Quite Ready for AI to Replace Humans
A Week in Rome: Conclaves, Coffee, and Reflections on the Ethics of AI in Our Food System
How ReShape is Using AI to Accelerate Biotech Research
How Eva Goulbourne Turned Her ‘Party Trick’ Into a Career Building Sustainable Food Systems

Footer

  • About
  • Sponsor the Spoon
  • The Spoon Events
  • Spoon Plus

© 2016–2025 The Spoon. All rights reserved.

  • Facebook
  • Instagram
  • LinkedIn
  • RSS
  • Twitter
  • YouTube
loading Cancel
Post was not sent - check your email addresses!
Email check failed, please try again
Sorry, your blog cannot share posts by email.