The halo of Saint Matthew

For several years Fortune magazine saw Enron as “America’s most innovative company”, right up to its spectacular bankruptcy in 2001. A very similar case is La Polar, the retail chain, in Chile. In April 2010 Pablo Alcalde presented “La Polar’s successful strategy” at the eClass seminar run by La Tercera / UAI (here is the video).

It is very interesting to listen to some of Alcalde’s words, especially in light of what we now know, and also from the perspective of everything we have covered in this series of articles:

“[...] the amount of analysis we have done on our consumers. We have gone to their homes, we have seen how they live, we have done sociological studies, we know what they think, we know what they do on Sundays, what they do with their time, why they go to shopping malls, why they don’t go to others, what’s going on with Transantiago, what’s going on with their kids, what their aspirations are, how they have evolved, etc., etc.

[…] I define the mix based on consumer behavior, and according to that lineup I say I need five thousand square meters of sales floor, or six thousand square meters, and I know I am optimizing sales per square meter and per salesperson, which gives me a motivated salesperson.

[…] The customer feels welcome, because he never walks past a store where he can’t buy. That person, that family unit, sees La Polar as more than a department store, sees it as a life choice..

[…] What you have, in short, is trust, which is the hardest thing to achieve in life. When you get people to trust you, you have broken the sound barrier. I believe in La Polar, and since I believe in La Polar I’m going to buy, I’m going to pay, I take out my insurance, etc. And only then does a more spiritual bond with the customer emerge”.¹

The trade press and the business schools presented La Polar as a brilliant example of a successful strategy, an example of innovation and leadership. The executives of Enron and La Polar were held up as role models for other executives. Of course, La Polar, just like Enron, projected an image of success built on a fraud. When the trade press, universities or think tanks evaluate these cases, they fall for what psychologists call the Halo Effect.

The Halo Effect

The Halo Effect is a cognitive bias in which we extend our perception of one particular trait to other traits that are not necessarily related. For example, if a person is physically attractive we tend to think they are also intelligent. John Adams once remarked that George Washington was considered the natural choice for leader simply because he always turned out to be the tallest person in the room.

Our judgment of how John Ausonius looks will depend on whether we think he is a) a movie star, b) a prize-winning scientist or c) a serial killer.

The problem is that companies, or people, can project these images of high performance, create an aura (the halo) that skews the analysis made by third parties. We get carried away by the image, to put it simply.

In one experiment several groups of participants were asked to do the financial analysis of a fictitious firm. Afterwards each group was scored on its performance and asked to comment on how well it had worked as a team on various metrics, such as group cohesion, communication and motivation. The groups that received a higher score considered themselves to be indeed more cohesive, motivated and communicative, unlike the groups that received lower scores, which came up with the harshest self-criticism. The catch is that each group’s score was assigned at random by the experimenter; there was no difference in performance between the groups with the highest scores and those with the lowest. The appearance of superior results leads to the illusion of high functioning. Mind you, there are no outside observers here explaining the performance; it is the groups themselves evaluating themselves according to the result they supposedly got. If they tell you you’re doing well, you’ll believe you’re doing well; if they tell you you did badly, you’ll find every flaw in yourself.

The halo effect distorts your judgment of your own performance inside your own head. The evaluation is not determined by the quality of the process; rather, it is the way we evaluate the outcome that determines how the process gets evaluated.

There is nothing wrong with evaluating a process by its results, as long as those results are repeatable and we can run several evaluations, as a way to mitigate the halo effect. But often there are processes we can only try once, and in that case what we must do is focus our effort on evaluating and improving what we do while we are doing it. There are techniques for this, which we will present later on.

Talent and luck

The halo effect can also trap us in the negative direction. Consider the scandal caused by the huge compensation paid to financial traders, especially after a crisis. The outrage is not that executives get paid a lot of money, because we know they get paid a lot of money, but that they get paid so much money for such disastrous results.

Take the example of those executives who in 2009 qualified for their bonuses. Did they deserve those bonuses in the middle of the crisis? After all, they had not caused the crisis, so why should they be penalized for other people’s mistakes? In the words of an AIG executive at the time: “I earned that money, and I have nothing to do with the bad things that happened to AIG”². The argument executives make is that they are necessary, because in the end they are the ones who generate profitable business, and if they were not paid these bonuses they could move to the competition and take the big deals with them. But this could be the halo effect all over again.

Imagine this experiment. Every year we flip a coin. If it comes up heads we will call it a good year and you get a bonus of 10 million; if it comes up tails it is a bad year and you get nothing. The interesting thing about this game is that as an executive your expected value is to make 5 million a year, while for the employer the gains are offset by the losses: your contribution as an executive will be zero.

Surely the AIG executive, and several others, will say that their performance is not a matter of luck, that it is the product of hard work, of long hours, of their skills and experience. But their colleagues said the same thing before the 2008 meltdown. So why should we believe him any more than his colleagues?

Telling talent from luck is a big problem. And there are a lot of people who get results by pure luck. To be sure, and to avoid falling for the halo effect, we need a measure that goes directly to the person’s skills, rather than to results that may be determined by forces beyond individual control.

The Matthew Effect

In sports we get the chance to measure athletes’ talent repeatedly. Not only do we offset the halo effect by having several possible measurements, we also try to make sure that every time they step onto the field the conditions are even for all opponents, by enforcing the rules of the game. It would bother us if Djokovic walked into a match with an extra serve just for being the top player on the ATP tour, or if the team leading the league were given a two-goal head start in the final stages of the championship.

But life is characterized by what sociologists call the Matthew Effect, which comes from the parable of the talents, specifically from the Gospel according to Saint Matthew: “to him who has more, more will be given, and from him who has less, it will be taken away and given to the one who has more.”³ In the biblical context the talents are coins, they are money, so in a literal sense that verse is about wealth, hence the phrase “the rich get richer and the poor get poorer”. The word talent acquired its current meaning from this parable (the idea that we should do our best with the talents we have been given), so the verse takes on another meaning as well, one that matches the idea of the sociologist who coined the term, Robert Merton, since he was referring to success in general.

Early success in an individual’s career confers a certain structural advantage that makes subsequent successes more likely, regardless of intrinsic ability. In science, for example, researchers who land at top-tier universities have more opportunities to publish papers and get funding than those at second- or third-tier universities. The result is that two individuals who might be roughly comparable at the start of their careers can experience dramatically different levels of success after 5 or 10 years, with no difference other than having been hired by different institutions. And from there on things get more unequal. Because successful scientists tend to receive a disproportionate share of the credit for anything they are associated with, as when they write papers with unknown students, who may have done most of the work, or even had the central idea. In other words, when a scientist becomes a star he not only gets more funding, he also attracts more collaborators, which can lead to his getting credit beyond what he actually deserves⁴. Robert Merton’s original article talks about precisely this phenomenon in science.

But sociologists have established that the Matthew Effect shows up in other careers too. Early success brings you prominence and recognition, which gives you more opportunities to succeed, more resources to succeed with, and the likelihood that later successes will be attributed to you without your having had much to do with them. And isolating the cumulative effect of those successes from innate ability, or hard work, is very difficult.

If we take people with similar characteristics, their fortunes will diverge, according to Merton’s theory. It has been observed that college students who graduated in years of economic depression earn less on average than those who graduated in boom years. And not just during those years: the difference persists for decades.

We certainly don’t like these effects, because we aspire to a meritocratic society, one where the most talented people, or those who work the hardest, are the ones who succeed. It may be true that the incompetent rarely succeed, or that especially talented people are unlikely to suffer great failures. But few of us fall at those extremes. For most people, chance combined with accumulated advantage can mean that ordinary people end up doing quite well, or badly. Since every individual story is unique, we can always convince ourselves that the outcome is somehow the product of that person’s unique characteristics. The important conclusion is that when we evaluate a person’s success based on the measures society imposes (wealth, prizes, honorary titles) instead of their abilities, or their skill at getting things done, we are fooling ourselves. Always measure a person’s abilities by their actual performance under conditions that depend only on their own abilities, not by results that don’t depend on them, or on the random circumstances that put them in a position to accumulate advantages. Otherwise you will always be fooled by the combined effect of the Halo of Saint Matthew :wink: