Necessary conditions

Security in human systems we’re told will always, always last -- Sting, History Teach us Nothing

In all the previous articles the central idea was to show that much of what we call common sense is some form of circular reasoning, that many common-sense explanations are wrong, and that this is shown when they are confronted with experiments or facts derived from the real study of the behavior of human groups.

The Mona Lisa is the most famous painting in the world because it has the attributes of the Mona Lisa. A few people are capable of promoting a trend, or setting a fashion. Many common-sense explanations may well be true, but what they really do is tell us that what happened happened, and not something else. Since these explanations can only be built once the outcome is known, we are never sure whether they are really explanations or simple descriptions.

The problem is that, unlike other sciences, where we can test hypotheses in controlled experiments that we can repeat over and over, with historical events that is not possible. The experiment happens only once. History happens only once. The outcome of a historical event has a lot to do with the micro interactions between people, and as we have seen it is not possible to know these in detail, or their distribution, and the explanation stays hidden from the eyes of the observers.

Sampling bias

One of the reasons we can’t find satisfactory explanations for historical events is that we are bad at selecting the relevant information. The problem lies in the things we pay attention to. We tend to ignore the events that don’t happen, or we ignore things that happen very frequently. We single out the times we run into a friend at the airport, but not the times that doesn’t happen. The times we miss the bus, we remember or notice more than the many times the bus pulls up moments after we arrive. We notice a soccer player’s scoring streaks, but not the great majority of players who have an ordinary season. In short, we notice the interesting facts. Obviously; it’s only natural. But the problem is that we project that natural tendency onto our explanations of phenomena, ignoring the rest of the information.

If we want to explain how people get rich, or the key to some companies' success, we look at the millionaires, or the successful companies, to identify the attributes they share. That is what financial magazines and newspapers do when they prepare their profiles of star executives or prosperous businessmen. But it turns out that if we looked at the thousands of other cases of people who aren’t rich, or who haven’t had “success”, we would discover that they exhibit many of the same attributes.

“The only way to identify the attributes that set the successful apart from the unsuccessful is to consider both kinds, and look for the differences systematically. But since we are interested in success, it seems pointless, or uninteresting, to worry about the lack of success. So we infer that certain attributes are related to success when in fact they are equally related to failure.” -- Duncan Watts¹

This is sampling bias. To understand the problems this kind of distortion causes, let’s look at an example. In a major plane crash that occurred during landing, the investigating commission determined five factors that contributed to the accident: first, the plane’s radio was faulty, and failed at a critical stage of the approach; second, the pilot and the navigator were fatigued, having had few hours of sleep in the days before the accident; third, the fog was especially thick that night, and visibility was poor; fourth, some instructions from the tower were contradictory, which caused confusion, and it wasn’t clear which runway was to be used; fifth, as a result of all of the above, and of the confusing instructions, the ground operator got confused and, given the stressful situation, understood that a different runway was to be used and forgot to clear a vehicle that was in the middle of the landing area. The investigation concluded that the combination of these five elements was the cause of the accident.

Graphically, the investigating committee’s conclusion is this:

Later a psychologist analyzed the case and pointed out that although the explanation is reasonable, familiar and consistent with plane crashes in general, these same factors show up consistently in many similar situations and no accidents occur. Graphically this can be represented like this:

In other words, these factors are necessary conditions, but not sufficient ones, for producing an accident. Without them an accident is extremely unlikely, but their being present does not mean an accident will happen. Yet when the accident does occur, all the other arrows disappear, and the world collapses into a single explanation: the combination of these five factors causes the accident. That is not really an explanation; we are only reporting what happened.

If in figure 2 the combination of factors looks like a bad predictor of the outcome, when the accident occurs we move to the scenario of figure 1, in which the same factors become the perfect explanation of the accident. The investigation helps find the necessary conditions for the accident, which is a good thing, because it lets us take preventive measures to avoid accidents. But the temptation to use these conditions as sufficient conditions for an accident leaves our intuition in very poor shape for explaining accidents.

And the same thing happens with all kinds of rare events, as when an armed person walks into the metro and kills its passengers. We can find all the attributes that explain this person’s behavior: psychological profile, past, history, psychiatric problems, or alcohol, depression, etc. A pile of necessary conditions for this kind of attack to happen. But there are thousands of people with the same problems, and they don’t go around shooting people in the metro. We simply cannot explain the reasons that lead a person to act this way.