By any reasonable test, David Hume should be safe. (He’s not yet listed on toppletheracists.org.) Hume’s racism was no more than was sadly normal at the time and it had nothing to do with what made his philosophy great.
This is a shining example of philosophers and academics giving themselves free passes that have not been earned even on their own principles. Baggini had previously argued that Nelson, whose life overlaps Hume's (he was around 18 when Hume died), fails the test because, "Nelson was not just a racist in a racist world, but a defender of racism against contemporaries who were challenging it." Hume's racism was not "no more than was sadly normal at the time"; it was highly controversial at the time. Beattie's Essay on Truth, which was very popular and well-regarded at the time, spends pages and pages attacking Hume on precisely this point, and was occasionally lauded on precisely this point. Baggini says that Hume's racism "had nothing to do with what made his philosophy great", but this has been a matter of controversy among Hume scholars interested in the question for a long time. The most famous argument that Hume's comments on race are intimately tied to the particular form of empiricism he espouses is still that of Eric Morton's 2002, "Race and Racism in the Works of David Hume", but it has by no means been the only one. And certainly that "bigot" Beattie, as Hume called him, thought that it was a significant and seriously important element in Hume's philosophy. Why does Baggini's judgment count more than Morton's or Beattie's?
Suppose we grant, though, that it was not "what made his philosophy great". Was it a part of his historical work? Hume's name was made as much in his historical work as in his philosophical work. Was it part of his essay work? Hume's essays were highly influential in a number of ways, and certainly Hume didn't write them in order not to exert influence. When people think of Lord Nelson's achievements, they are not generally thinking of his politics but of his military career; but Baggini still counts his politics. The only reason Baggini is not doing the same to Hume is that Baggini wants to support the monument-breaking but wants Hume not to be a target, despite Hume's explicit comments.
But the real question, of course, is not statues. Books are monuments to their author; they are far more widely available than statues; they are far more effective and influential, since books, unlike statues, have a long history of making converts and bolstering support for positions; they are themselves tied far more closely to ideas. So the real question of whether Baggini is serious comes down to the results of applying his three-pronged test,
(1) Is the achievement for which they are being celebrated closely tied to their sins?
(2) Were they significantly worse relative to others in their period?
(3) How recent was the offense?
to the banning and burning of books. Should we support, as the "moral high ground", the banning and burning of Hume's Essays, which are monuments to Hume that make explicitly racist comments, and which were written to convince people (admittedly among many other things) of those comments, and which were worse relative to others in Hume's day on this very point? And if Nelson doesn't get off on being too distant in time, surely neither does Hume. It seems like Hume's Essays do not pass a test like that, and so fall down into "the abyss of iniquity". There are obvious reasons why a philosopher would oppose the precedent of banning and burning books of philosophy; but if the argument is purely a matter of moral principle, there is absolutely no reason to exempt philosophy books from the same test as public statues. Any statue or picture of Heidegger is pocket-change in influence and effect compared to Heidegger's Being and Time.
As I've said, of course, moral principle is not particularly relevant, and attempts like Baggini's to moralize monument-breaking are doomed to collapse into either incoherence or question-begging; moral principles may be involved in monument-breaking, but only indirectly by way of contributing to people's self-image in the act itself. They don't give a reason for toppling statues in particular; there is a vast range of other actions you might do on exactly the same moral principles. This is why, contrary to Baggini's argument, the slippery slope is a worry: if you treat it as a purely moral question, as Baggini still does, you can't just decide when to get off the ride. Moral principles are not taxi-cabs.
Monument-breaking, like monument-building, is an attempt to send a message about who has power, about what kind of society is in place. Our society has tended to treat book-burning and monument-breaking as both bad acts, and has tended in the past to punish them by opprobium, the sanction of public opinion, as Mill would have it, and occasionally by harsh and strict enforcement of property and anti-vandalism laws, because both book-burning and monument-breaking are anti-liberal. Allowing them lets stand a message that you do not have to justify yourself before liberal processes (like convincing a democratic government) or institutions, that the liberal regime does not inherit the good of past generations while rising above their often serious defects (indeed, it implies that liberal regimes are complicit with them in their defects, as has been explicitly stated by some people supporting the monument-breaking), that liberal institutions and procedures can be bypassed on these matters for moral reasons to which political liberalism must itself be held account and judged guilty.
Since Baggini wants to allow monument-breaking, trying to maintain himself in simultaneous superposition between the liberal and the anti-liberal position, he needs to explain how far he is willing to go with book-burning, with the removal of books from curricula and libraries, and the like. After all, he tells us it is a matter of moral high ground; so he should stop treating it like something that can be managed by tactical maneuvering.