On Measurement

The empathy test

Here’s a report on some “research” that “measures” the amount of empathy people have:

Looking at their test questions, it becomes immediately clear that the test is a composite of several different things. All together, these things are labeled “empathy” and the implication is that the more of it you have, the better. Before I get into why this does not qualify as research, I’ll elaborate about their test.

Here are four of the questions from the empathy test:

• I can easily tell if someone wants to enter a conversation.
• I can pick up if someone says one thing but means another.
• I am quick to spot when someone is feeling awkward or uncomfortable.
• I can sense if I’m intruding, even if the other person doesn’t tell me.

These four questions apparently measure the ability to read what someone else is thinking or feeling, or what their motivation is. Now, here is a set of different questions, which are apparently designed to measure the extent to which a person internalizes the emotions of others:

• I tend to get emotionally involved with a friend’s problems.
• It upsets me to see an animal in pain.

While there may be some correlation (people who can read others’ feelings may tend to internalize the others’ emotions), the two things are different, and people could have one without the other. The first is a skill that is mainly under conscious control, while the second is a behavior pattern that is less voluntary.

Another set of questions measures knowledge of the local culture:

• I find it hard to know what to do in a social situation. (Anyone visiting a foreign culture might experience this.)
• I find it difficult to judge if something is rude or polite.

Another set of questions measures the relative value placed on the protection of other people’s feelings compared to other virtues:

• If someone asked me if I liked their haircut, I would reply truthfully even if I didn’t like it. (exposes whether honesty or protection of feelings is more important.)
• In a conversation, I tend to focus on my own thoughts rather than on what the other person is thinking. (exposes whether clarity or connection/reputation is more important – possibly)
• I am able to make decisions without being influenced by other people’s feelings.
• If I see a stranger in a group, I think that it is up to them to make an effort to join in. (There are reasons other than lack of empathy that a person may believe this.)

Another set of questions measures the similarity of ones style of thought with another’s style of thought:

• I find it difficult to explain things to others.
• I find it easy to put myself in someone else’s shoes.

Why the empathy test does not constitute measurement

Some of the five specific categories that I identified above might be ways to measure something. But in order for a measure to be scientifically valid, it has to be a measure of something that exists, not something that is created by the test. It cannot simply be a measure of the length of the ruler used to measure.

The specific problem with the empathy test is that it combines several related kinds of qualitative people-skills, creates a numerical ranking system, labels the composite rank as the word “empathy”, and then circularly declares that empathy exists scientifically because now we can measure it.

The general problem of measurement

In every subject area that I’ve gotten deeply involved in, I run into variants of this same issue, where the measurement is confused with the thing. The saying “the map is not the territory” is another way of looking at it.

• The notion of intelligence has become synonymous (in some people’s vocabulary) with the ability to score high points on the tests devised to measure intelligence. Notice the circularity there? The IQ test, the empathy test, and some others, such as emotional IQ, all are aimed to define the very thing that they measure. And then they imply goals to parents and educators, who then aim to game the tests, as if the act of finding ways to score higher on the test makes a person more intelligent, empathic or whatever.
• Other human composite traits like personality types and neurotypes (autism for example) can be circular when used as the basis for “proving” other things. For example there have been news reports claiming links between autism and behaviors similar to those used as diagnostic criteria for autism. (How amazing!) Studies with this circularity include this, this, this, and this.
• The economic measures of GNP and GDP are measures of something – but it is easier to understand what the formulas are than to understand the thing being measured. The general consensus is that increasing those numbers is good for the economy. But what if people are starving and the GDP is going up? The economy is still in bad condition based on actual events, even while it is doing great based on a made up ranking system. I tried to capture the twisted nature of economic policy in this parable. This news article captures the irony of how bad things are “good” for the economy.
• The field of urban development and transport is fraught with non-concepts like mobility, sprawl, and quality of life. People invent measures for these things, then try to enact policies that affect the measures or that are in service of the ephemeral concepts. A common argument is that we need urban rail because it promotes compact development and fewer miles traveled. Perhaps it does, but those are intermediate goals, or proxy goals, not the end goals. If the urban rail decreases travel at the expense of increased total fuel costs, has it accomplished anything? In this report I devised a system for planning based on human goals.

From these examples, it seems likely to me that policy and research on just about any topic – education, industry, public safety, or health – is probably just as twisted up in the problems of measurement as it is in these areas where I have studied it to some depth. Perhaps this is part of the general propensity of humans to believe that a state of durable, physical existence is conferred upon that which has been socially constructed, as I have argued in this book. Or perhaps belief in measures is manufactured to create false objectivity in cases where the measure corresponds with some personal stake.

Advertisement

---