Words To Grow By

WORDS TO GROW BY- MAY, 2025

Monthly Quote:

``The term 'Ethical AI' has no meaning -- it's a machine, it has no ethics. Ethics belong to the realm of human activities. Unless there is evidence that AI is sentient, we should never be using that term as it obfuscate the issues.'' Jan 25, 2024

Monthly Article:

Increasingly sophisticated AI systems can perform empathy, but their use in mental health care raises ethical questions

Published: March 31, 2024 8:45am EDTAuthor

A.T. KingsmithLecturer, Liberal Arts and Sciences, OCAD University

Disclosure statementA.T. Kingsmith does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.
PartnersView all partners

In a world where technology is increasingly intertwined with our feelings, emotion-AI harnesses advanced computing and machine learning to assess, simulate, and interact with human emotional states.
As emotion-AI systems become more adept at detecting and understanding emotions in real-time, the potential applications for mental health care are vast.
Some examples of AI applications include: screening tools in primary care settings, enhanced tele-therapy sessions and chatbots offering accessible 24/7 emotional support. These can act as bridges for anyone waiting for professional help and those hesitant to seek traditional therapy.
However, this turn to emotion-AI comes with a host of ethical, social and regulatory challenges around consent, transparency, liability and data security.
My research explores these potentials and challenges of emotion-AI in the context of the ongoing mental health crisis in the years since the COVID-19 pandemic.
Read more: COVID-19 and mental health: Feeling anguish is normal and is not a disorder
Do experts have something to add to public debate?We think so
When emotional AI is deployed for mental health care or companionship, it risks creating a superficial semblance of empathy that lacks the depth and authenticity of human connections.
What’s more, issues of accuracy and bias can flatten and oversimplify emotional diversity across cultures, reinforcing stereotypes and potentially causing harm to marginalized groups. This is particularly concerning in therapeutic settings, where understanding the full spectrum of a person’s emotional experience is crucial for effective treatment.
Age of emotional AIThe global emotion-AI market is projected to be worth US$13.8 billion by 2032. This growth is driven by the expanding application of emotion-AI across sectors ranging from public health care and education to transportation.
Advancements in machine learning and natural language processing allow for a more sophisticated analysis of people’s emotional cues using facial expressions, voice tones and textual data.

Products like Empath use emotion-AI to analyze moods and feelings. (Marco Verch/flickr), CC BYSince its release in early 2023, OpenAI’s generative-AI chatbot ChatGPT-4 has been leading the charge with human-like responses across a broad spectrum of topics and tasks. A recent study found that ChatGPT consistently scored higher on “emotional awareness” — identifying and describing emotions accurately — than general population averages.
While OpenAI dominates North American and European markets, Microsoft’s chatbot Xiaoice is more popular in the Asia-Pacific region. Launched in 2014 as a “social chatbot” aimed at establishing emotional connections with users, Xiaoice is capable of sustained empathetic engagement, remembering past interactions and personalizing conversations.
In the coming years, a mix of productivity and emotional connection will transform mental health care and redefine how we interact with AI on an emotional level.
Future risksThe rapid rise of emotion-AI raises profound ethical and philosophical questions about the nature of empathy and emotional intelligence in machines.
In The Atlas of AI, AI Scholar Kate Crawford questions the accuracy of systems that claim to read human emotions through digital cues. She raises concerns about the process of simplifying and decontextualizing human emotions.
Digital scholar Andrew McStay further explores the implications of attributing empathy to emotion-AI systems. In Automating Empathy, McStay warns of “synthetic empathy,” highlighting a key distinction between simulating a recognition of human emotions and truly experiencing empathy.
Additionally, emotion-AI’s ability to analyze emotional states opens avenues for surveillance, exploitation and manipulation. This raises questions about the boundaries of machine intervention in personal and emotional domains.

Without experiencing empathy, emotion-AI can only simulate it. (Shutterstock)Rethinking human-AI relationsThe widespread application of AI in therapy, counselling and emotional support holds the potential to revolutionize access to care and alleviate pressures on overworked and overburdened human practitioners. However, the personification of emotion-AI creates a paradox where humanizing AI might lead to the dehumanization of human beings themselves.
At the same time, attributing human-like qualities to AI risks making mental health care less interpersonal. The potential for AI chatbots to misinterpret cultural and individual emotional expressions could lead to misguided advice or support. This can further complicate or exacerbate mental health issues, especially where the nuances of human empathy are essential.
These tensions underscore the need for the careful, ethically informed integration of emotion-AI in mental health treatment and care.
These technologies need to complement, rather than substitute, the human elements of empathy, understanding and connection. This requires rethinking human-AI relations, particularly around empathy.
By ensuring the ethical development of emotion-AI, we can aspire to a future where technology enhances mental health without diminishing what it means to be human.

On 4/30/2025 from: https://theconversation.com/increasingly-sophisticated-ai-systems-can-perform-empathy-but-their-use-in-mental-health-care-raises-ethical-questions-225498