Tuesday, February 06, 2007

Testimonial Bias

This is a repost, and I don't generally re-post things; but I'd forgotten that I'd written and thought it interesting enough to put it up again.

Hume, in his Essay on Miracles:

The wise lend a very academic faith to every report which favours the passion of the reporter; whether it magnifies his country, his family, or himself, or in any other way strikes in with his natural inclinations and propensities. But what greater temptation than to appear a missionary, a prophet, an ambassador from heaven? Who would not encounter many dangers and difficulties, in order to attain so sublime a character? Or if, by the help of vanity and a heated imagination, a man has first made a convert of himself, and entered seriously into the delusion; who ever scruples to make use of pious frauds, in support of so holy and meritorious a cause?


The problem with this argument is the first premise: "The wise lend a very academic faith to every report which favours the passion of the reporter." The problem with this, as was soon noted in responses to Hume, is that this, despite an initial plausibility, turns out to be untenable. Boswell records the following conversation with Samuel Johnson, which touches on the problem:

We talked of denying Christianity. He said it was easy to be on the negative side. "If a man were now to deny that there is salt upon the table, you could not reduce him to an absurdity. I deny that Canada is taken, and I can support my assertion with pretty good arguments. The French are a much more numerous people than we; and it is not likely that they would allow us to take it.--'But the Ministry tells us so.'--True. But the Ministry have put us to an enormous expense, and it is their interest to persuade us that we have got something for our money.--'But we are told so by thousands of men who were at the taking of it.'--Ay, but these men have still more interest in deceiving us. They don't want you should think they have gone a fool's errand; and they don't want you should think that the French have beat them, but that they have beat the French. Now suppose you should go over and see if it is so, that would only satisfy yourself; for when you come home, we will not believe you. We will say you have been bribed.--Yet, for all these plausible objections, we believe that Canada is really ours. Such is the weight of common testimony."
[Boswell's London Journal. Pottle, ed. McGraw-Hill (New York: 1950) 301-302.]

A similar line of thought forms part of Richard Whately's biting satire on Hume's arguments against miracles, Historical Doubts Relative to Napoleon.

And we can back their arguments up with an additional consideration. If you haven't recognized it yet, journalism is largely about testimony: reporters don't generally identify facts directly, but find a source that will give testimonial evidence for whatever is being put forward. Now, it has also been recognized that journalists have an infuriating habit of reporting crackpot testimony alongside respectable testimony. I think if you look closely at this practice, you will see that what lies behind it is Hume's principle. When scientists give testimony about what the best explanation of a phenomenon is, this testimony is in conformity with the passions and interests of the scientist themselves; when this is combined with Hume's principle, it follows that we must lend an 'academic faith' to the reports of scientists, because of the potential for bias. So journalists go off and find some other testimony with a countervailing potential for bias, so as not to prevent a one-sided view of a matter that is clearly conformable to the interests of those giving testimony.

Hume's principle, in other words, is, if taken strictly, the stuff of quackery and conspiracy theory.

Why then does it initially seem so plausible? The reason, I think, was rightly recognized by many of the early critics of Hume (Campbell, Shepherd, etc.): we do use something like Hume's principle, when we have independent reason for thinking the testimony in this particular case to be distorted by the passions of the reporter. In other words, Hume mistakenly treats as a general principle what in fact is a rule of thumb for particular cases that meet certain conditions. For Hume's principle to have merit, each case of testimony must be considered on its own. The reason is that there is very little testimony that is not in conformity with the passions and the interests of the reporter in some way; and, indeed, for all we can say a priori, the passions and interests of the reporters may in this case be helping them to give a more accurate testimony. We have to look and see whether there is any reason to think the testimony in this particular case is genuinely distorted.