The philosopher Neil Levy’s recent book about peer review is more than a book about peer review. Its first half is about bullshit, interpretive charity, and the hermeneutic circle.
Levy tackles a critique of the journal system that turns not on overburdened editors and reviewers or predatory publishers but on
the fact that papers subsequently regarded as groundbreaking were rejected multiple times before finally finding a home, and [on] empirical evidence that accepted papers do not fare particularly well when resubmitted with cosmetic changes.
Does peer review do a bad job of tracking the quality of research—assuming there is such a thing?
Against this inference, Levy contends that it’s reasonable to extend interpretive charity on the basis of circumstantial features of a text: who the author is, and where it was published, for instance.
The fact that three high-profile journals rejected work that subsequently appeared in an influential and important book is at best weak evidence that these journals aren’t tracking quality (or even that on these occasions they failed to track quality), because the reviewers did not have the same evidence available to them as we do, looking back.
In other words, anonymous readers—for good reason—make their judgements on the basis of artificially limited evidence. They are more fallible than we will later be, with the benefit of data on impact, influence, and authorship. It’s unfair to complain that their false negative rate is higher than ours, since it could not be otherwise. If there is evidence that they do a bad job with the evidence they have, the fact that they make what come to be seen as serious mistakes does not establish it.
Which is a fair point, I think—but we don’t need twenty pages on bullshit to make it. Levy hints at more expansive and more playful purposes in his opening paragraph:
Kant identified three questions that philosophy must address: What can I know? What must I do? What may I hope? Today, a fourth question preoccupies many philosophers: Why was my paper rejected?
The implicit answer in what follows is that your paper was rejected because the reader perceived it as bullshit—with two provisos:
the relevant sort of bullshit isn’t Harry Frankfurt’s, i.e. speech that is indifferent to the truth, but Gerry Cohen’s or Gordon Pennycook’s, i.e. speech that can’t be clarified because it is nonsense masked as meaningful assertion
this sort of bullshit comes by degree, as does our willingness to interpret it charitably, which can make the difference not just between sense and nonsense but “between ‘reject’ and ‘revise and resubmit.’”
Cohen’s prime example of (alleged) bullshit comes from Étienne Balibar:
This is precisely the first meaning we can give to the idea of dialectic: a logic or form of explanation specifically adapted to the determinant intervention of class struggle in the very fabric of history.
“But,” Levy asks, “is Balibar’s work really unclarifiable?” In his reply to Cohen, Frankfurt offers this sensible paraphrase:
The most distinctive point of dialectical explanations is precisely that they are supposed to be particularly helpful in illuminating how class struggle has determined the course of history.
Reporting sceptically on the authors of a letter that called on Cambridge University to deny Jacques Derrida an honorary degree, Levy writes:
Their assessment of his work might have been based on casual acquaintance with it: reading excerpts or maybe an essay. Perhaps some of the signatories thought that intellectual responsibility required them to dip into Derrida in this kind of way before signing the letter. Perhaps some of them went deeper, reading several essays, a whole book, or even several books. … But it takes serious and sustained engagement to be in that position with regard to a philosopher working in a different tradition; more serious and sustained engagement than is commonly thought.
The upshot is that, despite his defence of false negatives in peer review, Levy makes a case against Reviewer #2; his argument, aptly, deconstructs itself. We are not nearly charitable enough—and with sufficient generosity, we can make good sense of texts we first found unintelligible, or vacuous, or utterly confused.
I think Levy’s right. I’ve come to value interpretive charity, and patience, more than ever; and I’m increasingly uncomfortable calling bullshit. If an idea is both obscure and fashionable, I’m inclined to think that the emperor must have clothes, even if they are not my style. That’s how it is to be middle-aged.
There’s an episode of Barry Lam’s wonderful podcast, Hi-Phi Nation, in which he quizzes bystanders on whether out-of-context quotes were produced by actual philosophers or online bullshit generators. I was one of his marks—I turn up about 3 minutes into the clip below, at 30:08, along with Cian Dorr—and I thought every quote was philosophical: no bullshit.
About 7 minutes into the clip, at 34:13, I defend my propensity to find meaning almost anywhere by appeal to interpretive charity. You might object that this makes me little better than Reviewer #2: we have symmetrical vices, his generating false negatives, mine false positives. There’s a condition, apophenia, in which people see patterns or connections between random things. Isn’t that where my “charity” is headed?
But there’s a deep asymmetry here. An interpretive false negative spurns meaningful philosophy as garbage. That’s a genuine loss. An interpretive false positive finds meaningful philosophy that isn’t really there. But if it’s meaningful philosophy, who cares? It may be cogent, powerful, original—even if it’s an artifact of apophenia. That’s pure gain. The value of philosophical ideas does not depend upon their author, or upon their having one. The truth may come to us in a misreading or a dream; and wherever it originally came from, it is ours.
Hello Kieran,
It appears both your comment and Neil Levy's book (which I haven't yet read) have a focus on philosophy work when discussing peer review. Beyond these higher level issues, I believe there are more clearer and more fundamental issues inherent in peer review. It's a volunteer work most of us do out of custom, and it comes with very little accountability due to the widely adopted anonymous peer review system.
I have written about the issue as a part of broader issues in academic publishing. In case you're interested, you can take a look at here : https://mahmutruzi.substack.com/p/addressing-and-resolving-critical
Link for Open Access version of the book:
https://www.cambridge.org/core/books/philosophy-bullshit-and-peer-review/F54B9C195549B08D69311EF385DF3D56