The mind sees what it wants to. It is an excellent detector for patterns in seemingly random data, but it also excels at making patterns where none exist. It's built to do that. It's how we learn. And that often gets us into trouble.
Dick Feynman had a very interesting teaching trick to illustrate this problem - he used it several times in different situations, ranging from his freshman physics lectures at Caltech to his lectures during trips after his Nobel Prize award. Feynman would suddenly interrupt himself in the middle of a statistics lecture, and excitedly say something like: "On my way to campus today, I saw a car with the licence plate XRT-375 in the parking lot - isn't that amazing? What are the odds of seeing that exact licence?" After letting the class wrestle with exactly what he was asking, he would make the point that there is a HUGE difference between calculating odds before the fact and after the fact. The chance of seeing that particular plate is simple to calculate: 1/26*1/26*1/26*1/10*1/10*1/10, or about one in eighteen million. And it really would be amazing if you picked a number out of the air, and then found it in the lot. However, Feyman's point was that having seen the plate first, it is unremarkable that you then ask the question about that particular number. The chance is unity. You can't use a set of data to make a hypothesis, and then turn around and use that same data to test the hypothesis!
This problem, using data to test hypotheses that has been previously used to make the hypotheses, is staggeringly common. Sometimes it is obvious, but more often, it is very subtle. In science, epidemiology suffers from this statistical mistake quite frequently. But this made me think about the field of history and its re-use of data, when one of those odd "coincidences" occurred to me.
I recently finished a book by Frederick Pohl, Earl Henry Sinclair. It makes the case for a trans-Atlantic voyage in 1398 by an Earl of the Orkney islands, Henry Sinclair. I love this kind of stuff that rattles established traditions (Columbus as having been the first European to see the Americas), but I couldn't help feeling uneasy about the style of Pohl's writing. Perhaps it was because I was reading a book published for the general public rather than a scholarly paper, but it didn't seem to have any balance to it. It was obvious that Pohl was an out and out fan of Sinclair's and he was having none of anyone's arguments about problems with the evidence. Not surprisingly, it turns out that there is a whole fan club.
Sinclair likely crossed from the Orkneys to Iceland, and may have crossed from there to Greenland at least once during his life. This much is accepted, as traffic between Northern Europe and these areas was common by this time and is well documented. The debate is about whether Sinclair actually travelled from Greenland on to Newfoundland, New Brunswick, and eventually down into Massachusetts. Other people doubtless did, since there are many tales of early explorers of the Eastern coasts of the Americas commonly finding Basque, Portuguese, and Irish fishing crews that claimed they had fished these easterly banks for several generations. The ships Sinclair would have used were much like Viking longships, except by the 14th century they had versions with multiple masts and covered decks. Anyway, on to the "coincidence."
Reading the Sunday section of my newspaper just after finishing Pohl's book, I came across an article about a summer folk festival on the Mall in Washington. Two people participating were going to sail a viking longship from the Orkneys to Washington. One of the people had the surname Sinclair. Isn't that amazing?
And of course, the answer is NO. Succumbing to the thought that this is extraordinary is to fall into Feynman's licence plate trap. What has happened is that you have primed your mind to a particular set of facts, and you will pay attention when they occur again in some combination over the next few months. I would have had the same odd feeling if one of the participants had the surname Pohl. I would have noticed anything to do with Micmac indians, because they too figured prominently in the story. I would have noticed anything to do with Scotland and Norway. And the point is also that I would have noticed a combination of circumstances like that for any one of the other books I was reading at the time. Those kinds of combinations jump out at you - not because something strange is afoot, but because our minds are built to do exactly that: pick out patterns that might mean something, or might be useful to our survival. Where you go from there with that feeling, that information is the critical part - whether you believe in mere coincidence or whether you turn to non-scientific explanations is completely a matter of conditioning. It is not scientific education, because there are plenty of scientists and even statisticians who will make irrational choices when faced with similar circumstances.
And what about bonnie Prince Henry? Well, of course there are impassioned debates about whether he really did make it across the Atlantic. There is an critical article by Alastair Hamilton here, that seems to tear apart any chance that the crossing actually ocurred. Hamilton's article too, has the flavor of complete conviction, this time in a more scholarly format. But it is still vitriolic, and nowhere near the balance I would hope could surround this type of debate. Orkneymen seem to favor Hamilton's view, as do many historians - and despite the presence of an outline of a 14th century knight holding a shield associated with Sinclair carved (OK, punched) into a Massachusetts stone ledge, I tend to agree with them. It strikes me as odd that the Orkneys, with such powerful oral traditions as the Orkeyinga Saga would have no trace of a story about an Earl who travelled to such a far off and strange land and who had later returned.
It struck me that both these sides, and indeed historians generally, are caught by their traditions and academic training in a perpetual Feynman licence plate trap. They try and explain given facts with a hypothesis, and then turn around immediately to use those same facts to prove the hypothesis.
But I don't see any easy way out for them either. They simply have to recognize that many of their methods are statistically invalid. And that's not an easy admission for an academic, especially a historian with a proud tradition invested in the particular way they do research.
