The phrase the 'banality of evil' was coined by philosopher Hannah Arendt after witnessing the trial of high-ranking Nazi Adolf Eichmann who seemed, at least to Arendt, to be the most mundane of individuals whose evil acts were driven by the requirements of the state and orders from above.
It was Eichmann, not Arendt, who said Eichmann was only following orders. Arendt didn't think Eichmann's acts were driven by the requirements of the state and orders from above; her point was that in a sense nothing drove him to it at all. The banality of evil wasn't that it was done out of conformity; it was that it was done without a second thought. As she puts it, evil is extreme but never radical: Eichmann was not a mere lackey, he was wicked -- but wickedness turned out not to be Lucifer from Paradise Lost, not an evil genius out to destroy the good, but a plain, rather mediocre person who spent an immense amount of effort and ingenuity on whatever task was at hand, even if it was mass murder, as if he were planning a party or trying to balance the books. Evildoing was an "authentic inability to think," not from stupidity but from refusal to be more than superficial. The shocking thing about people like Eichmann for Arendt was not that they followed orders, but that they saw little major difference between manufacturing food and manufacturing Jewish corpses -- and what difference they did see was old cliché, party line, and recycled prejudice. Arendt's conclusion was that evildoers cannot -- will not -- seriously reflect on the things that are important because they cannot -- will not -- recognize that they are important, or more worthy of reflection than anything else. The wickedness of Eichmann's acts could not be traced to deep and terrible motives; Eichmann's actions had no deep motives at all, being made possible entirely by a refusal to use the sound judgment required to realize that what he was doing was wrong and not right as he facilely assumed. Eichmann did the terrible things he did not because of his hatred of Jews (which Arendt doesn't deny existed), which in another person might have amounted to nothing, but because of a deliberate refusal to use the imaginative capacities that make the suffering of others real to us. The problem with Eichmann was not that he was unaware of the consequences of his actions but that he didn't seriously question his assumption that his actions were right (indeed, put himself in a mental position where he couldn't possibly have done so). Eichmann was wicked because he had entirely abandoned thoughtfulness, the precondition for the hard work of doing good; and the banality of evil lies in evil's inability to be aware of itself as evil.
Also, it's important to note (with regard to something else said in the article discussed by the "Mind Hacks" post) that Arendt's judgment wasn't based only on what she personally saw at the trial; she researched Eichmann's testimony, and the investigation into his wrongdoing, quite thoroughly. She used what she saw at the trial not as the sole foundation of her argument but more as a way of summing up her point.
