Compare:
"Why do I have free will?"
"Why do I think I have free will?"
The nice thing about the second question is that it is guaranteed to have a real answer, whether or not there is any such thing as free will. Asking "Why do I have free will?" or "Do I have free will?" sends you off thinking about tiny details of the laws of physics, so distant from the macroscopic level that you couldn't begin to see them with the naked eye. And you're asking "Why is X the case?" where X may not be coherent, let alone the case.
"Why do I think I have free will?", in contrast, is guaranteed answerable. You do, in fact, believe you have free will. This belief seems far more solid and graspable than the ephemerality of free will. And there is, in fact, some nice solid chain of cognitive cause and effect leading up to this belief.
But this seems to me to make assumptions that don't stand scrutiny. It will no doubt often be the case that the second question is safer, due to the relative ease with which we access our thoughts compared to the relative difficulty of accessing other things that happen (when, in fact, access to thoughts is easier than access to other events -- a good case can be made that the one is not always easier than the other), but the one is no more guaranteed than the other. The belief may or may not be more "solid and graspable" than what the belief is about; but it may not exist at all: on reviewing the matter, for instance, you may find that really you don't believe it at all -- it's just that verbally you weren't distinguishing it adequately from something else that you do believe, and hadn't done the work to show it. Likewise, if X is not coherent, it becomes problematic to affirm that I think X, since there is no coherent X to think. So here, too, we can ask "Why do I think X is the case?" when thinking X may not even be coherent, much less the case.
Moreover, it's a bit misleading to say that there is a "solid chain of cognitive cause and effect". What we really find when we look at the causes of any belief is not a "solid chain" but a complicated and continually shifting net, in which some nodes remain relatively stable and easily discernible, while others are not stable, or not easily discernible, or are neither. A great deal of cognitive science over the past few decades has been devoted to exploring this unstable and difficult-to-discern aspects of human thinking, with often interesting results (one thinks of studies of blindsight or of motivated reasoning). It is laughable, from the cognitive science perspective, to say that beliefs sit solidly in your mind, as Yudkowsky does later. If I move from asking the question, "Why do black holes exist?" to "Why do I think black holes exist?" I am not moving to a question that is more easily answered; I am moving to a question that is extraordinarily difficult to answer completely, due to the sheer complexity of the phenomenon. All I can do is pick out a few of the more obvious threads that (one hopes) are sufficient for practical purposes. Indeed, one could argue that the former question is clearly the simpler and more easily answered question; it doesn't drag in any weirdness about human thinking but lets you focus on two points: the coherence of the notion of black holes and the evidence that they exist.
So, short of a dubious quasi-Cartesian assumption that self-knowledge is relatively easy, there appears to be no general improvement in moving from the first type of question to the second. Whether there is so in the case of free will has to be determined not, as Yudkowski attempts to do, on general principles, but on the particular character of the subject of the question. What Yudkowsky really seems to mean is just that we step back from the question of why something happens to the question of what evidence we have that it happens at all (although he says things that are not strictly consistent with this, so I'm not wholly sure) -- which may be salutary advice on occasion, but hardly because the latter is "guaranteed answerable". And, of course, it might just be salutary in this case to step back from the question, "Why is the first question, in sending us off to consider minute details of physics, less useful than the second question, which is guaranteed answerable?" in order to ask the question, "What actual evidence do I have either that the first question requires us to get into minute details of physics or that the second is guaranteed answerable?"
Similarly, with regard to what Yudkowsky says later, one might step back to ask whether the second question is really a "cognitive science question" and therefore "answerable"; rather than a bit of folk psychology that has practical use but falls short of what is required for serious cog. sci. inquiry, much less for cognitive science answers.
