An Academic Scandal: The Importance of Peer Review

By January 12, 2003 No Comments

A constant danger for activists, regardless of the worth of their cause, is an eagerness to ‘over-egg the pudding’ when making a point. Michael Bellesiles’ 2000 book Arming America: the Origins of a National Gun Culture provides a case in point. Academics and scientists are supposed to be reputable seekers after the truth and, while free to engage in activism, should be careful with respect to how they cook their book.

The Bellesiles scandal is attracting media attention in the US, but it has barely shown up on Canadian radar. Michael Bellesiles, an historian at Emory University in Atlanta, won the prestigious Bancroft Prize in 2001, but he is now scrambling to defend himself against charges of “fraud” and “intentional deception.” He is still under suspicion as the department continues to investigate the charges.

Despite the fact that this scandal concerns American history, it raises an important challenge for Canadian academics. How could allegedly fraudulent research, not just get published, but win prestigious academic prizes? The rise and fall of Bellesiles opens a window on a more general problem, the fragility of peer review. According to Professor Emeritus David Bordua, at the University of Illinois, the real scandal concerns the willing gullibility of ideological reviewers and academic historians. The Bellesiles case represents a monumental failure of peer review. When his work was first published, reviewers hailed the book apparently without bothering to evaluate the research. The book even won the Bancroft Prize without anyone on this prestigious panel bothering to check his footnotes thoroughly. Isn’t this something academics are supposed to do even with run-of-the-mill undergraduate papers?

In 2000, Bellesiles was being fawned upon by academic and popular critics alike for his book Arming America: the Origins of a National Gun Culture. Peter Onuf, the author of Jefferson’s Empire,” opined that “Michael A. Bellesiles moves to the front rank of American historians with this deeply researched, brilliantly argued, energetically written, and timely book.” Robert J. Spitzer, author of The Politics of Gun Control, claimed the book was, “Meticulously, even extravagantly, researched.” Michael Zuckerman, the author of Peaceable Kingdoms even said, “This is stunning history, brilliantly argued. It throws into a cocked hat our most cherished assumptions about guns and gun culture in early America.” Today, these critics are silently eating crow.

What was all the fuss about? In his book, Bellesiles claimed to have examined over 11,000 probate records between 1765 and 1850 and found a surprisingly small number of firearms. He concluded that firearm ownership was less widespread than previously believed, so that, before the 1860s, a widespread “gun culture” didn’t exist. More provocatively, he claimed that onset of the US Civil War, and the rapid growth of the arms industry, gave birth to the distinctive American gun culture. If true, this would undercut the myth of the American “minuteman” who plays an important role in American political theory.

This is heady stuff. To many historians and social critics, this study supported their prejudices against firearms and their distrust of industrial corporations. Without checking too closely, many rushed to applaud Bellesiles as their new hero.

Unfortunately for Bellesiles, cracks soon began to appear in his argument. First, legal researchers challenged his sources. When these scholars checked the sources they couldn’t confirm his findings. For example, when Professor James Lindgren, Professor of Law at Northwestern University, tried to check the records, he found glaring discrepancies. Bellesiles records either weren’t where he said they would be or, if found, they differed substantially from what Bellesiles said. Other scholars also checked; all found it impossible to get Bellesiles to share his sources.

Worse was in store. When the records were checked further, many of the records Bellesiles claimed existed couldn’t be found. When asked, he had various excuses, each more implausible than the ones he’d used earlier. All however were variations of ‘the dog ate my homework.’ Bellesiles told professor Lindgren that he’d kept his notes on yellow foolscap and had no records on disk. This is surprising, but then he said his notes had been destroyed when there’d been a flood in his office. However, professor emeritus Jerome Sternstein at Brooklyn College wasn’t impressed. Particularly troubling, archives where Bellesiles claimed to have taken extensive notes had no record he’d ever been there. His goose was cooked when he was confronted with claiming to have studied records that were lost in the great San Francisco earthquake and fire of 1906.

His academic critics have now painstakingly shown that Bellesiles made up or grossly misrepresented much of his data. The academics who nominated him for the Bancroft Prize are no longer supporting him. The American Society of Criminology scheduled a panel to examine the scandal, but no one could be found to defend him. The National Endowment of the Humanities has acted to remove its name from Bellesiles’ fellowship. After a six-month investigation, Emory University has now placed him on leave pending their decision about the charges of academic dishonesty.

Academics such as professor Bordua argue that the ease with which his book was accepted by academics is even more shocking than Bellesiles’ deficient scholarship. How could anyone be awarded the Bancroft Prize without any serious effort being made to corroborate his research? The real scandal is not that a professor would conduct fraudulent work, but that his peers accepted it. The most plausible explanation is that ideologues believe the end justifies the means.

In Bellesiles’ case, historians eagerly allowed themselves to be hoodwinked. The charitable interpretation is that academics, like anyone else, are not as critical of arguments that support their prejudices as they are with those that are less comforting. This is just human nature. But, academics are supposed to be professionals. Every discipline has set up methodological standards to guard against such human failings. In order to be published, academic books and articles must survive rigorous criticism in the peer-review process. If we do not honorably adhere to the highest disciplinary standards, our research loses credibility.

A less charitable interpretation is that academics are more cynical. Historians may have willingly supported Bellesiles’ outrageous claims, even knowing they were implausible, because doing so furthered their political objectives. Historians tend to consider themselves “progressive,” and so they share a dislike of firearms. Thus, they willingly flocked to support Bellesiles in order to promote their ideological agenda.

Some academics defend such cynicism, arguing that there is no such thing as truth. Everything is subjective. Unfortunately, this standard too easily promotes propaganda. Academic prizes are seen as rewards for your friends or fellow travelers. As one European intellectual told me, “The law only exists to be applied to your enemies.” I believe such a cynical approach undermines faith in all academic research.

Don’t misunderstand me. This problem isn’t limited to history. Nor is it limited to liberals. The Bellesiles case illustrates how crucial peer review is to all academics – Canadian or American, historians or scientists. If peer review can fail in history, it can fail anywhere. Recently, the Journal of the American Medical Association admitted that there are serious problems with peer review in medical publications. All academics must work to keep the peer review process healthy and act vigorously to root out unprofessional conduct.

There is no happy ending to this story yet. Peer review did eventually uncover the gross distortions, but this took years longer than it should. Bellesiles’ critics continue to grow as the evidence against him mounts relentlessly. Unfortunately, the American Historical Association does not appear to be listening. It is too soon to know if the flaws in the peer-review process that failed with Bellesiles have been recognized, let alone fixed.

Leave a Reply