There has been quite a bit of discussion about peer review and its disappointments in the past few years, some of it good, some of it bad. (For a recent example of a good discussion, see Adam Mastroianni, The rise and fall of peer review.) However, there is an error that commonly arises in many of these discussions, whether good or bad, and it is the view that peer review was ever supposed to indicate a high quality of research or argument, or that it was supposed to 'catch bad research', or anything like it. Even a little common sense would show the absurdity of this -- how could it possibly do any such thing, given that it is not even structured in a way that could give reasonable guarantees of such a result? If you want a filter to indicate very good research, or even to eliminate bad research, you would very obviously have to design it to work in a very different way from the way peer review has ever worked.
When you publish formally in a peer-reviewed journal, the point is not that you have reached the top tier of research quality -- very few formally published works are that -- nor is that you have even reached some kind of satisficing threshold. Publication is itself never supposed to be the end of the process, but a particular kind of beginning to a public process of discussion and further examination. The point of pre-publication peer review was always to make sure that published papers had done certain things (exactly what might vary according to the field) that would facilitate further discussion and examination of the research or argument presented in the paper. To the extent that it served as a filter at all, it was because journals used it to narrow down the number of papers they received; but even so, the whole point was to narrow it down to papers that had done what was needed and appropriate to make further discussion, rebuttal, refutation, etc., easier.
I find it odd that so many academics don't seem to have ever learned this. But I'm often surprised in this way; I think there was a period of time during which early Boomer-era academics simply assumed that new graduate students would pick up all the details by osmosis, so nobody actually learned the reasons for why things are done the way they are done. (Certainly, looking back on my own graduate experience, I think it was indeed common for faculty to just assume that graduate students would pick up most things by proximity.) The result is that we get a lot of cargo-cultism about academic matters that is, frankly, more dangerous than any problem with peer review as such.
It is not at all surprising that peer review fails to guarantee quality of research; if that were the standard, it was always going to fail to do that. Nonetheless, I do think that the argument that pre-publication peer review (which is not very old) has largely failed to do what it actually is supposed to do -- facilitate discussion and response -- is a very good one. Perhaps some of the problem is the cargo-cultism -- it's certainly the case that it's a rare pre-publication reviewer whose assessment is a judgment of whether the paper has done what would be useful for public discussion -- but I think it's also the case that having had the system in place for a while, we've discovered a lot of non-obvious ways in which it tends to be self-defeating. It is slow, for one thing. It clogs up communications very badly; in many fields (like philosophy) it regularly leads journals having only out-of-date papers because the actual discussion, through other channels, has already gone well beyond the paper by the time the paper is even OK'd for publication. It is also tedious, requiring academics to do a large amount of mostly uncredited professional service for it even to work as it currently does. It is very inconsistent, and it often leads to authors having to modify papers in ways that are arbitrary or pointless. There's no real way for peer reviewers themselves to be held accountable to standards that are in any way consistent across the entire field. Just considering the idea of pre-publication peer review, none of these were obvious, but they recur again and again, no matter what anyone tries to do to reform the process. There is indeed a very good argument that pre-publication peer review is a failed idea, starting from peer review's actual purpose.
We also don't live in the twentieth century any more, a fact that some academics seem to have difficulty with. There is no longer any need to keep paper costs down for journals -- even if you have a paper journal, you can have a web-only set of papers -- and journals themselves are no longer the easiest way to keep up with research in many fields. After a review to make sure it fits the house format and perhaps a few other very basic standards (e.g., providing the raw data for scientific research papers), pretty much everything can be published, and a system of post-publication peer review can do far better than any system of pre-publication peer review at facilitating discussion and examination.
