Frequently I catch myself concluding that a given personality trait is predictive of another. For example, that people who are outgoing are also domineering or that people who are shy are also unkind– you get the idea. Though I’m aware that these can only be proved through rigorous statistical investigation, unconsciously I use these poorly supported judgements to influence my valuations of people. It seems to be a human tendency to detect such relationships from stories, usually comprised of only a few characters but each with a wide breadth of personality traits. In his book Once upon a number, John Allen Paulos explains why we often make this mistake:
Stories and statistics offer us the complementary choices of knowing a lot about a few people or knowing a little about many people. The first option leads to the common observation that novels illuminate great truths of the human condition. Novels are multivalent and bursting with ironies, details, and metaphors, while social science and demographic statistics can seem simple-minded and repellingly earnest by comparison. We can easily delude ourselves, however, into thinking that more of a general nature is being revealed to us by a memoir, personal reminiscence, novel or short story than is truly the case. Biased and small samples are always major problems, of course, but my caveat arises from something more specific: the technical, uneuphonic statistical notion of an adjusted multiple correlation coefficient.
If the number of traits considered is large compared to the number of people being surveyed, there will appear to be more of a relationship among the traits than actually obtains.
.
.
.
Whenever the number of characteristics is a significant fraction of the number of people, the so-called multiple correlation among the characteristics will suggest spurious associations.
To tell us something useful, multiple correlation analysis must be based on a relatively large number of people and a much smaller number of characteristics. Yet the insights that come from stories and everyday life are precisely the opposite. We each know, in a full-bodied way relatively few people, and for these people the number of characteristics, relationships, characteristics of relationships, relationships of characteristics and so on that we are aware of is indeterminately large. Thus we tend to overestimate our general knowledge of others and are convinced of all sorts of associations that are simply bogus. By failing to adjust downward our multiple correlation coefficients, so to speak, we convince ourselves that we know all manner of stuff that just isn’t so.
Paulos, John Allen, Once upon a number: the hidden mathematical logic of stories. Basic Books (1998), pp. 26-37.