Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.
Which is more probable?
1. Linda is a bank teller.
2. Linda is a bank teller and is active in the feminist movement.
Most people will pick #2. But in standard probability theory, the probability of #2, being more specific than #1, will always be less than or equal to the probability of the more general #1. Making this mistake in contexts requiring standard probability theory is what is known as the conjunction fallacy.
I very deliberately put it in those terms, because I think people are often quite sloppy in discussing it. What is it that makes this a fallacy? Contrary to what some people seem to assume, the notion of probability in standard probability theory is not a particularly natural one; in fact, it is (in comparison with many other notions of probability) very artificial, and took an immense amount of thought and effort to develop even at a fairly advanced stage in the history of mathematics. Its importance derives not from its being our root notion of probability (or likelihood, or any other such terminology), but from its being a derivative notion of probability that is both very mathematically tractable and of fairly extensive utility. But it's clear enough that when we talk about probability in colloquial life we are generally talking not about probability theory but about analogical and presumptive reasoning, which are very different. For instance, we usually mean by 'probable' nothing more than 'fitting the appropriate profile well'. Thus what makes this a fallacy is not using a notion of probability in which #2 is more probable than #1, but in confusing this notion of probability with that of standard probability theory, or (perhaps more commonly) using this looser notion of probability where standard probability theory would be more appropriate. People do, in fact, make this error, so it is a genuine fallacy; and, in fact, we should expect this given that the notion of probability used in standard probability theory is so unnatural. But we should be careful not to characterize it in such a way as to suggest that the less mathematical notions of probability are themselves fallacious, because they are involved in different kinds of reasoning, and each kind of reasoning must be examined on its own merits. One of the points of the 1983 paper by Tversky & Kahneman was the distinction between "extensional" and "intuitive" probability inferences. They argued that their cases showed "people's affinity for nonextensional reasoning". However, even they sometimes slipped into talking about the latter form of reasoning as if it were a degenerate form of reasoning -- "seductive," as they called it, introducing a perhaps ineliminable "incoherence" into our reasoning, where "incoherence" means simply "not satisfying the constraints of probability theory" -- when in fact all the conjunction fallacy shows is that this reasoning is inappropriate in cases where standard probability theory should be used. There is no reason to think that these kinds of reasoning lack a proper domain, nor is there reason to think that in their proper domain the constraints of probability theory are even relevant to them.
(Of course, in practice we also have to keep in mind interference from implicatures. That is, if we are not careful #1 can easily be read, in the context of #2, as meaning "Linda is a bank teller (and not a feminist)." This is a pretty standard way of reading things -- if A is contrasted with (A & B), it is very natural to take the original A as meaning A alone, that is, (A & not-B). And, while reading this way will often, perhaps usually, allow us to interpret what we are reading correctly, this implicature also sometimes confuses us, and we act as if it existed even where it would make no sense for it to exist.)
UPDATE: By happy happenstance it turns out that Robin Hanson pointed to a paper on the conjunction fallacy that argues that incentives and group interaction can both massively reduce the extent to which people commit the fallacy, and that therefore there are certain areas (economics, for instance) where it may not be as serious an issue as some have speculated it might be.
