May R. Berenbaum

PNAS August 20, 2019 116 (34) 16659-16662

Figure: May R. Berenbaum.

When I was named the new Editor-in-Chief of PNAS in October 2018, I received hearty congratulations from colleagues from a wide range of disciplines, befitting the intended audience of this venerable journal. The appointment is not my first experience serving as Editor-in-Chief; in 2017, I stepped down after 21 years as Editor-in-Chief of the Annual Review of Entomology (ARE). Although ARE is known in entomological circles as the most highly cited journal in the field, even my entomological colleagues readily acknowledge that PNAS is considerably more influential. By one widely used metric, however, the change could be seen as a step down; the impact factor of the last volume of ARE I edited, 13.860, was higher than the impact factor, 9.504, of the journal whose editorial ranks I had just joined.

The irony of the comparison is not lost on me, and I’m hardly the only person who finds journal impact factors (JIF) troubling, even when their values are not calculated to the third decimal point. I’m not even the first person to express concerns about JIF within the pages of PNAS. Four years ago, former Editor-in-Chief Inder Verma (1) cautioned the scientific community against relying on JIF as a surrogate for the quality of any individual article. Before that, Alan Fersht (2), in introducing an alternative metric to readers, reported that, of 39 different bibliometric scales of ranking journals, the impact factor was “at the periphery” (p. 6883). The first to criticize JIF may well have been the person who invented the metric. Although legendary information scientist Eugene Garfield created the concept of science citation analysis in 1955 (3), he did not expand the idea of using “journal impact factor” to evaluate the relative “importance” of a journal until 1972 (4). His concept was to use an “average citation rate per published article” to rank relative journal impact, calculating the average by dividing the number of times a particular journal was cited in a given year by the number of articles it published during a previous specified time period (which today is typically the preceding two years).

At that time, Garfield thought the metric would be of value to librarians managing their journal collections, scientists prioritizing which journals to read, editors evaluating their journal’s performance, and science policy analysts aiming to identify new research fronts. By 1998, however, in an article subtitled “Is the tail now wagging the dog?,” Garfield (5) noted that his tool had been expropriated for the “frequent . . . misuse of citations for the evaluation of individual research performance. . . . While there are countless legitimate applications of citation data, in the hands of uninformed users, unfortunately, there is the definite potential for abuse” (p. 68).

Less than 10 years later, Garfield (6) bemoaned the proliferation of those abuses, mostly perpetrated by funding agencies who “often wish to bypass the work involved in obtaining actual citation counts for individual articles and authors” and administrators who decide to “estimate the future impact of a recently published paper by incorporating the impact factor for the journal in which the paper is published” (p. 92). Given the typically long right-tail skew in calculating average impact factor, whereby the majority of citations received by a journal might be earned by a minority of articles, he considered using the impact factor as a predictor of the future impact of a paper “a rather dubious practice.”

See more: https://www.pnas.org/content/116/34/16659