I've always been puzzled at the crudeness with which analytic philosophers often handle counterexamples, and was thinking about it after having come across a seriously egregious bungling of counterexamples in my reading recently. Counterexamples are pretty common, and most people manage to muddle through them in a reasonably tolerable way, but there seems to me to be a lack of sophistication, as evidence by the sheer puzzlement with which people regard the relatively simple case of Hume's missing shade of blue, and sometimes straightforward incompetence, when handling them. Part of the problem, I think, is that everyone starts using them early, and no one really bothers to have in hand a good account of how they work; I never was taught one, nor have I met many people who show any indication of having been taught one.
This is related, I think, to a failure to appreciate the ramifications of the usual accounts of validity. There are tricky details in handling them, but roughly on these accounts validity is truth-preservation, where truth-preservation has the following characteristics:
(a) if all of the premises are assigned the value TRUE, the conclusion is assigned the value TRUE
(b) if the conclusion is assigned the value FALSE, at least one of the premises is assigned the value FALSE
The important point here is that these are characteristics that can only be had under inference rules; and since inference rules can vary from domain to domain, validity varies from domain to domain. This isn't really surprising; truth-preservation does not work exactly the same way in an intuitionistic system that it does in a classical, or an affine, or a linear system. And there are homelier examples; in some domains inference from part to whole will be truth-preserving, and in others it will be the fallacy of composition. But this has ramifications for counterexamples used to show invalidity; counterexamples can only show arguments invalid for the domains in which they can be identified. In another domain the counterexample might not exist and the very same argument might be valid. And so it is always crucial to know what domains are involved. This is one reason why the missing shade of blue is not a problem for Hume's thesis, for instance; it's a counterexample that only occurs in a domain other than that in which Hume is interested (the one he is interested in being that which contains only events that happen naturally and regularly). There are other reasons, of course, but it is one of them. For every counterexample, we have to ask: is this counterexample from the right domain? Often it is, but sometimes one finds counterexamples being treated as unproblematic when it actually isn't clear that they apply.
Often counterexamples get their bite not from being mere counterexamples, but from an implicit suggestion that goes with the counterexample that the counterexample comes from a domain that's relevant to the argument or its premises. Counterexamples work, in other words, on the condition that they are relevant (when accurately described, of course -- the fact that something can seem to be a counterexample only when misdescribed is what allows for what Lakatos calls 'monster adjustment', where one simply redescribes the example in a way that eliminates its appearance as a counterexample).
UPDATE: In the comments David Corfield reminds me of this discussion of mathematical difficulties and counterexamples at "n-Category Cafe".
