measurement

Episode #12: The Moral Hazards of Measurements

As long as measurements are abused as a tool of control, measuring will remain the weakest area in a manager’s performance. —Peter Drucker

The illusion of certainty in our measurements creates—to borrow an important concept from the insurance industry—a moral hazard.

Simply defined, people have an incentive to take more risks or act carelessly when they are insured. Fire insurance causes arson; unemployment insurance allows people to not be as diligent in finding a job; life insurance causes suicide, or worse, murder; auto insurance can cause reckless, etc.

Our current cult of calculation, perpetuated by the infamous McKinsey maxim—what you can measure you can manage—creates the same type of risk, offering today’s business executives the illusion of control and mastery of knowledge.

It allows them to substitute statistics for thinking. It gives them a false sense of security where there should exist more doubt.

If we want to peer into the unknown future, our measures need to be linked to a theory, otherwise we are simply predicting the past—since history is the only dimension for which numbers can provide precision. With theory, we can also avoid the moral hazards of measurements so prevalent in today’s society.

Seven Moral Hazards of Measurements

In the Handbook of International Trade and Development Statistics, 1988, per capita output in 1988 in East Germany—one year before the Berlin Wall was pushed over—was placed at roughly seven-eighths of the West German level.

But as any Berlin taxi driver crossing through Checkpoint Charlie after the fall of the Wall could have told you, the economy of East Germany was manifestly inferior to that of West Germany, yet somehow—due to the moral hazard of measurement—those in the know got it precisely wrong rather than approximately right.

Taking into account the following seven moral hazards of measures may assist executives in avoiding these types of errors.

Moral Hazard 1: We Can Count Consumers, But Not Individuals

Singer Joan Baez used to say it was easier for her to have a relationship with 100,000 people than with one person. Stalin’s famous remark that “one death is a tragedy, whereas a million is a statistic” illustrates the danger of lumping individuals into aggregate, amorphous lumps as if they did not have a soul.

Stanley Marcus, the son of one of the founders of Neiman-Marcus, led the store through the difficult Great Depression, and one point he was especially fond of making was there was no such thing as a market, only customers.

In 2003, General Motors sold 8.59 million vehicles, yet each was sold one at a time. The micro level, where the customer interacts with the seller, is inherently a flesh-and-blood transaction. As economist Herbert Stein always used to say, “There is nobody here but us people.”

In the final analysis, markets and consumers are statistical abstractions, whereas customers are human beings who want to be treated specially and individually.

Moral Hazard 2: You Change What You Measure

Scientists call it Heisenberg’s Uncertainty Principle, which applies to all measures: that the observer in a scientific experiment affects the result. Central bankers call it Goodhart’s law: Any target that is set quickly loses its meaning as it comes to be manipulated. People will always find ways to make their numerical targets, even if it leads them to ineffective or, sometimes, unethical behavior.

A further hazard lies in the fact that in order to count something it must stand still, which is why the first statisticians were called “statists.” But people don’t stand still; they are constantly moving, changing, growing.

Moral Hazard 3: Measures Crowd Out Intuition and Insight

Once a measure becomes entrenched as part of the conventional wisdom, it is usually impenetrable to logic, intuition, critical thinking, or better ways to do something. Poverty statistics are a perfect example, as everyone accepts them as a precise measure of those citizens living below what we consider an acceptable standard of living. But how was this measure developed? Where did it come from?

The poverty rate measures the income of the poor, not their consumption, which is a false talisman of someone’s standard of living. It is not what you earn, it is what you are capable of spending; thus consumption should be measured, which would take into account nonreported earnings, noncash subsidies, and other services provided.

It is also a national statistic, and does not take into account regional differences in the cost of living. Further complicating the error, during the Johnson administration’s war on poverty, it was decided the “poverty rate” would be set at an arbitrary three times the cost of the U.S. Department of Agriculture’s economy food plan, as explained by Nicholas Eberstadt in his book The Tyranny of Numbers: Mismeasurement and Misrule.

Using a consumption-based, rather than an income-based, poverty measure, Eberstadt has concluded elsewhere the rate drops from approximately 13 percent to between 2 and 3 percent. Quite a difference, and illustrative of how far off even old measurements can be and how firmly entrenched they remain despite being precisely wrong.

If you have ever been bribed off an oversold airplane—with a free flight voucher, upgrade, or airline money equivalent—you have economist Julian Simon (1932–1998) to thank. Until 1978, and before the airlines were deregulated, travelers were bumped off overbooked planes rather capriciously (the airlines preferred to bump old people and military personnel on the theory they would be least likely to complain) and this caused enormous amounts of customer complaints and ill will.

Worse yet, the problem fed upon itself, because passengers began to expect being bumped and so would book several flights under various names to ensure a seat on at least one. This caused the airlines to increase bookings even more to ensure decent load factors, which of course were measured very precisely.

Had the airlines changed the process and tested Simon’s idea sooner, the airlines and its customers both would have been better off. Simon did not analyze countless numbers and statistics, but used his intuition, grounded by the economist’s theory of human behavior being rational, to solve a quite vexing problem.

Daniel Boorstin, librarian of Congress, wrote: “The greatest obstacle to discovery is not ignorance—it is the illusion of knowledge.”

Moral Hazard 4: Measures Are Unreliable

A country’s per capita gross domestic product increases when a sheep is born but decreases when a child is; or how divorce actually increases the GDP since almost two of every commodity must now be purchased rather than just one.

Picasso once said, “Art is a lie that tells the truth.” It seems in some instances, measurements are truths that tell lies.

Another example of the unreliability of measures is illustrated by the consulting firm Bain & Company’s Web site, where it proudly proclaims: “Our clients outperform the market 4 to 1,” shown over a graph from 1980 to 2012 depicting the S&P 500 Index and Bain clients.

This is the equivalent of the rooster taking credit for the sunrise because he crows every morning. One expects this type of unscientific hyperbole from politicians, not management consultants. I would be willing to bet that Bain’s clients perform better than the S&P 500, thus have more money to spend on consultants.

Moral Hazard 5: The More We Measure the Less We Can Compare

Engage in this gedanken: You (or a loved one) need(s) heart surgery. You talk to nurses, friends, and other people you trust and respect, and two surgeons are consistently recommended to you. You go online to do some research on these two practitioners and discover their mortality rates (i.e., the risk of dying from surgery: surgeon A = 65 percent; surgeon B = 25 percent. Which surgeon would you choose?

I have conducted this gedanken in seminars attended by various educated professionals—who certainly have taken a statistic class or two—and, astonishingly, the overwhelming majority select surgeon B. When I ask why, they say because of the lower probability of death.

Perhaps they think they need to choose between the two without gathering other information. But that is not how I set up the thought experiment: I left it open as to whether they could ask further questions. Not many do.

But wouldn’t you want to know what type of patients the two doctors serve? What if surgeon A takes a disproportionate share of hard cases and thus has a higher failure rate? He or she just may be the better surgeon.

The point is, we simply do not know without gathering more information, both quantitative and qualitative, and making further judgments based on our own risk profile. Seeing the two numbers side by side seems, though, to give people a false sense of precision and, in this case, could lead to a deadly decision.

Moral Hazard 6: The More Intellectual the Capital, the Less You Can Measure It

Ideas only come from sentient beings, not inanimate objects or pets. Since 80 percent of any country’s wealth-creating capacity resides in its human capital, how could it be otherwise?

To complicate matters, a lot of that knowledge is tacit, which is hard to capture in spreadsheets and pie charts. We may be able to count the physical assets of a Google or a Microsoft, but traditional accounting pays no attention to its human capital, what has been labeled the “invisible balance sheet.”

Traditional book value accounting—assets minus liabilities equals equity—can only explain about one-fourth of the value of the market capitalization on the nation’s stock markets. Accountants call the difference between market value and book value goodwill; but that is just a label for their ignorance. In an intellectual capital economy, debits don’t equal credits, because value is subjective and flows from free minds, not tangible commodities.

Data, reason, and calculation can only produce conclusions; they do not inspire action. Good numbers are not the result of managing numbers. As David Boyle wrote in The Sum of Our Discontent (Cloth): Why Numbers Make Us Irrational: “Decisions by numbers are a bit like painting by numbers. They don’t make for great art.”

Moral Hazard 7: Measures Are Lagging

Imagine driving your car with your dashboard gauges informing you of last month’s speed, fuel level, temperature, oil pressure, RPMs, and the rest. This is precisely the status of accounting information: it is like walking into the future backward.

It is a lagging indicator—or at best coincident, assuming real-time accounting takes place. This type of information can only tell us where we have been, never where we are going. Auditors come in after the battle and bayonet the wounded; they are historians with lousy memories.

Summary

The Danish philosopher Søren Kierkegaard wrote: “Life is lived forward but understood backward.” Certainly measures help us reflect on past events and aid us in improving our theories. But they can never take the place of dreams, imagination, passion, and the soul of enterprise where entrepreneurs toil and struggle to create our future.

No measure is capable of capturing the richness of free minds operating in free markets dreaming of better ways to improve our future, and it is folly to believe otherwise. It may even lead us into moral hazards, or a world where we are so preoccupied about measuring past performance we do not take the time to dream about the future.

Other books and resources mentioned

Minding the Store, Stanley Marcus