How do you measure empathy in a machine?

Why it is harder than it sounds

Human empathy is usually measured with self-report questionnaires such as the Interpersonal Reactivity Index, with ratings by observers, or through behavior. Self-report does not transfer. Ask a language model how much it cares and its answer tells you about the text it learned from, not about an inner state. Measuring empathy in a machine means looking outside it, at its responses and at their effects.

Those outside measures are easy to bias. In the best-known comparison of chatbot and physician replies, the chatbot’s answers were rated far more empathetic, but they were also about four times longer, so length and empathy are hard to separate in that result. Labels distort too. The same response is rated more empathic when people believe a human wrote it. A measure that ignores length or disclosure can end up measuring those instead.

Scoring what the system says

The most common approach is to have people rate a system’s replies. It is how the chatbot and physician comparison worked, and how a 2025 study found AI responses rated more compassionate than those of expert crisis responders. Ratings like these are fast and easy to compare, but the raters are not the person being comforted, and they judge a reply in isolation rather than a conversation.

Coding schemes make ratings more systematic. A framework from the University of Washington breaks empathic communication in text into three moves: emotional reactions, interpretations, and explorations. Each is scored as absent, weak, or strong. Its authors annotated 10,000 exchanges from online support platforms and trained a model to score new ones. In a later randomized trial with 300 peer supporters, that kind of scoring showed that AI feedback made their replies 19.6% more empathic, and 38.9% more for those who found supporting others hard.

Benchmarks test language models directly. EQ-Bench, for example, asks a model to predict how intensely the characters in a dialogue feel various emotions. In a 2025 comparison, six widely used models averaged 81% on five standard emotional intelligence tests, against a human average of 56%. These are tests of what a model knows about emotions. EQ-Bench’s own author reports that its scores correlate at 0.97 with a broad general-knowledge benchmark, which suggests it partly measures general capability. A systematic review of empathy studies on language models reached a similar view: the models show elements of cognitive empathy, recognizing emotions and offering support, while the evaluations lean heavily on subjective ratings.

Asking the person on the other end

Perceived empathy is measured with questionnaires completed after an interaction. For years researchers adapted scales written for human empathy, without knowing whether they held up for machines. The Perceived Empathy of Technology Scale, published at CHI in 2024, was built for the purpose. It uses ten statements, such as “The system considered my mental state” and “The system understood my goals,” was validated with several hundred participants, and is free to use.

In our work on sustainable choices, we developed a multidimensional measure of perceived AI empathy for voice assistants and found that people reliably detect empathic cues in them, along the same core dimensions as human empathy.

Perception is also where labels bite hardest. Because people discount empathy they believe comes from a machine, a study that tells some participants they are talking to an AI and tells others nothing is measuring the label as much as the agent. The disclosure has to be the same in every condition.

Watching what people do

The test that matters most is behavioral: does empathy change what people feel and do? In our studies the outcomes are things people decide or express in the moment. An empathic voice assistant reduced verbal aggression and raised satisfaction when a service failed. Empathy combined with a green appeal shifted food orders toward sustainable options. In earlier work by our colleague Fotis Efthymiou, a trembling synthetic voice made people more willing to donate and, in a field test, raised click-through on charity ads.

The person’s own voice is evidence too. We measure the emotion in participants’ speech as the conversation unfolds. In the service failure study, analyses of tone of voice and of the words people used showed more satisfaction, relief, and positivity in the empathic conditions. Measuring sentence by sentence also reveals turning points that a single questionnaire at the end cannot capture.

What a credible measurement looks like

Putting these together, a study that claims to measure AI empathy should be able to answer yes to five questions.

1. Does it compare an empathic version of the agent with one that differs only in empathy?
2. Did participants actually notice the difference, measured with a validated perceived empathy scale?
3. Were length, voice, task, and disclosure held constant across versions?
4. Does it measure what people did, not only what they said they felt?
5. Were emotions measured during the conversation as well as after it?

We design our experiments around these questions. Our research platform stores every message alongside the emotion measured in the speaker’s voice, so that what people did and how they sounded can be analyzed together. We are also developing the AI Empathy Index, a benchmark that will compare how leading AI assistants and companions show perspective-taking, emotional resonance, and prosocial motivation.

Common questions