The Philosophical Gourmet Report

The Philosophical Gourmet Report is the most widely known ranking of graduate programs in philosophy. It is also extremely controversial. After a threatened boycott in the fall of 2014 over alleged unprofessional and uncollegial behavior by its editor, Brian Leiter, the 2014-2015 PGR experienced a significant loss in the number of its evaluators. Two striking examples: first, the total number of evaluators in the Philosophy of Language was down by almost 50% from 2011; second, the PGR could not find a sufficient number of evaluators to evaluate in Feminist Philosophy in 2014, and instead combined results from 2011 and 2014, and still had only twelve evaluators. It should be noted that these losses were sustained even after it was announced that Berit Brogaard had agreed to serve as a co-editor for 2014-2015, and that Brian Leiter would step down as an editor after the 2014-2015 PGR was complete. In addition to the boycott statement over Professor Leiter’s actions, there was also a call in the fall of 2014 by other philosophers not to participate in the evaluation process because of deep problems with the PGR itself.

There are many criticisms of the PGR. It is not scientific. It is methodologically flawed. It has marginalized large swathes of the profession, those that have been viewed unfavorably by Brian Leiter. And it helps sustain the underrepesentation of women and people of color in the profession. On this site you will find links to articles and essays identifying the various problems with the PGR. It is an excellent example of why the APA does not endorse rankings in philosophy.

The PGR was the invention of one person, Brian Leiter. As a graduate student in the 1990’s he realized that there was a market for a ranking of graduate programs in philosophy. He developed one, and circulated it in photocopy. By the late 1990s it was posted on the web. Leiter also understood that there was a market for a website that could function as an aggregator for goings-on in the profession, so he created a blog to report (and comment on) these items. Leiter cornered the rankings market, and then relocated elements of the original PGR—news about faculty moves, tenurings, job offers etc.—to his blog, effectively making the two web publications extensions of each other. The rankings needed the information that the blog supplied, and the blog drew on the influence and notoriety of the rankings. Once the blog was off the ground, Leiter could sell space on his blog to advertisers, using the traffic, drummed up in part by the PGR, as a draw. Although Professor Leiter was supposed to have stepped down as co-editor of the PGR by the beginning of 2015, he appears to retain the copyright, and his blog is still linked to the PGR. The organization and commentary on the PGR still bear the stamp of his personal tastes regarding what is properly called philosophy.

Readers are invited to submit items for inclusion in the collection of articles and items below. Some of the titles appear with excerpts, and some are listed under more than one category.

….

Overview of the Controversy

An Open Letter to Prospective Evaluators for the 2014-2015 Philosophical Gourmet Report–An Update (Aboulafia)

Brian Leiter’s Continuing Influence on the Philosophical Gourmet Report: The Past as Future (Aboulafia)

Rank-and-Yank: What’s Next for the Philosophy Rankings Game? (Aboulafia)

The Rise and Fall of the Philosophical Gourmet Report (Ben Alpers)

…I think that the rise and fall of the PGR will be a wonderful future topic for U.S. intellectual historians. The appeal of such a ranked list tells us interesting things about the field of philosophy in the late 20^th and early 21^st centuries, as do the specific philosophical preferences of Leiter and the PGR. And the timing of the PGR’s collapse is also interesting, as it comes in the midst of a moment in which the philosophical profession is doing serious soul-searching over the problem of (often gendered) bullying within it. And these are just some of the interesting – and important – aspects of this story.

Discussions of Philosophy Rankings (A posting of items by Richard Heck)

Further contributions to the intellectual history of the PGR (John Protevi)

Archive of the Meltdown (Many links to posts on the controversy surrounding Brian Leiter and the PGR, fall 2014, Leigh Johnson.)

1979 and 2014 (Bharath Vallabha)

….

Methodological and Procedural Problems with the PGR

The New Philosophical Gourmet Report Indicts Itself (Aboulafia)

The Philosophical Gourmet Report, by the Experts (Aboulafia)

The Incovenience of Rigor: The Pseudo-Science of the PGR (Aboulafia)

The Dog Ate My (Philosophical Gourmet) Report (Aboulafia)

Appearance and Reality in The Philosophical Gourmet Report: Why the Discrepancy Matters to the Profession of Philosophy (Bruya)

(From the article’s abstract.) The article demonstrates that the actual value of the PGR, in its current form, is not nearly as high as it is assumed to be and that the PGR is, in fact, detrimental to the profession. The article lists and explains five objections to the methods and methodology of the report. Taken together, the objections demonstrate that the report is severely flawed, failing to provide the information it purports to and damaging the profession overall.

Our Naked Emperor: The Philosophical Gourmet Report (Zachery Ernest)

It is my contention that the Report is not merely unsound as a ranking system and detrimental to the profession; it is obviously unsound as a ranking system and obviously detrimental to the profession. Indeed, its flaws are so obvious that it would seem to be unnecessary to discuss them. However, the Report is also an institution unto itself. It is so deeply entrenched into the profession of academic philosophy that otherwise highly intelligent and critical professionals seem to have developed a blind spot to it. Indeed, the Report’s flaws are so obvious and so severe that I find it embarrassing to be influenced by it, even unwillingly.

About the Philosophical Gourmet Report (Richard Heck)

Another, and in some ways more serious, worry concerns the influence the Report has upon the profession as a whole. Partly as a result of the factors just mentioned, the overall rankings in the Report are biased towards certain areas of philosophy at the expense of others. The most famous such bias is that against continental philosophy. I don’t much care for that style of philosophy myself, but it isn’t transparently obvious why Leiter’s oft-expressed and very intense distaste for much of what goes on in certain “continental” departments should be permitted to surface so strongly in the rankings. Other biases are less obvious but every bit as real. It is well understood in the profession that hiring someone pretty good who works in philosophy of mind will have more influence on a department’s overall ranking than will hiring someone much better who works on logic, let alone on ancient or medieval philosophy. I have been told that this fact has actually influenced hiring decisions—told, that is, by people who were present at meetings where such decisions were made.

Ranking Philosophy Programs (Noelle McAfee)

Philosophy Rankings (Noelle McAfee)

I ask any administrator who is taking seriously the Leiter report to confer with the statisticians in his or her own university to get an objective measure of the soundness of these rankings’ methodologies.

Rankings Exercises in Philosophy and Implicit Bias, Section Three (Jennifer Saul)

It is very important that university administrators be made aware of the possible discriminatory effects of using likely Gourmet Report rankings as a method for making hiring decisions. Given the way that reputations are created and perpetuated, the workings of implicit bias, and methodology of the Gourmet Guide, it is highly likely that members of stigmatized groups will have less impressive reputations than they actually deserve. Using a reputational guide like the Gourmet Report as a guide to hiring, then, is likely to have detrimental effects on members of stigmatized groups. Worse yet, one of the best ways to follow the advice “hire someone who will improve our Gourmet ranking” would be to deliberately discriminate. Because of this, it cannot in good conscience be used as a guide to hiring. It would be worth investigating the implications of this legally as well.

Manufactured Assent: The Philosophical Gourmet Report’s Sampling Problem (Gregory Wheeler)

More on the Educational Imbalance within the PGR Evaluator Pool (Gregory Wheeler)

To this one might counter that the rater pool is, despite appearances, an accurate representation of the larger community of research active philosophers. As I mentioned in my previous post, this position is hard to reconcile with objective measures of research activity, like the one that Jon Kvanvig put together at Certain Doubts. But we don’t even need to bother wading into the controversies surrounding bibliometrics to clear this objection. Instead, simply look at the PGR itself and consider whether its own judgment of research excellence across its own 33 areas of specialization is reflected in the composition of the PGR evaluator pool. Put simply, does the PGR walk the PGR talk? The answer is, No, not really.

Two Reasons for Abolishing the PGR (Gregory Wheeler)

The PGR assumes that departments of philosophy are things which can be sensibly ordered, even though there is little cross-specialty agreement within its own highly biased evaluator pool. The health of the field may well turn on embracing this lack of consensus rather than trying to paper over it. . . . The Philosophical Gourmet Report is philosophy’s bubble. It is no longer so much a question of whether it will pop, but when. (Although I am not holding my breath.) And when it does pop, when the collective scales fall from our eyes and forehead smacks are heard in faculty lounges across the land, all of us, but particularly our elders, will owe Richard Heck an apology for not having had his back 15 years ago. He was right.

Public Facebook Post on why we should abandon the PGR (Jessica Wilson)

[R]anking systems encourage premature dogmatism, whereby the favoured topics, approaches and methodology of those working at the top-ranked institutions (either absolutely, or relative to a given specialty) takes on the sheen of ‘to be accepted’ by others. Hence it is that so many talented philosophers of the past generation have spent their valuable time working within frameworks whose foundational presuppositions are clearly and immediately questionable, while more plausible and illuminating approaches to the topics at issue are neglected….

….

Exclusionary Dimensions of the PGR

A Portrait of the 2014 Philosophical Gourmet Report by the Numbers (Aboulafia)

The Halo Culture: Taking the Rankings Challenge (Aboulafia)

Rankings Exercises in Philosophy and Implicit Bias, Section Three (Jennifer Saul)

The Function of the Philosophical Gourmet Report (Bharath Vallabha)

Not only does PGR retain much of the traditional structures, it in fact implicitly enables the internet to be used to expand the reaches of the traditional structures and to further marginalize less powerful departments. Even as it enables a few students from less prestigious departments to make it into the upper echelons of the profession, PGR worsens the situation for most departments by rendering them voiceless in the public understanding of the profession. . . .

What is lost, and covered over, is the idea that there could be different forms of philosophical local knowledge, that different departments think of philosophy in very different terms, with their own subcultures of the projects to be pursued, forms of argumentation, characteristic limitations and so on. PGR fosters the sense that across “the English-Speaking World” there is a single, shared sense of doing philosophy, and that this shared sense of philosophy works as the measuring stick by which all departments in the “English-Speaking World” can be evaluated.

PGR’s Supposed Altruism (Bharath Vallabha)

….