News FocusScientific Misconduct

Cleaning Up the Paper Trail

See allHide authors and affiliations

Science  07 Apr 2006:
Vol. 312, Issue 5770, pp. 38-43
DOI: 10.1126/science.312.5770.38

Once an investigation is completed and the publicity dies down, what happens to fraudulent or suspect papers? In many cases, not much

What now?

Several papers by Korean cloner Woo Suk Hwang and his team have already been retracted, but who will validate or remove the rest?

CREDIT: KIM KYUNG-HOON/REUTERS

Last October, after a 14-month investigation, immunologist Luk van Parijs was fired from the Massachusetts Institute of Technology (MIT) in Cambridge. The school alleged that he had confessed to faking data in one published paper, several unpublished manuscripts, and grant applications.

Van Parijs's academic future may be shot to pieces. But his scientific past, so far, is intact: roughly 40 papers stretching back to 1994, many of them in the blossoming field of RNA interference. None has yet been publicly labeled fraudulent or retracted. MIT has not said which paper it found to be problematic. Other investigations are continuing.

“One of the biggest problems in these fraud things,” says Kathleen Case, publisher at the American Association for Cancer Research (AACR) in Philadelphia, Pennsylvania, is that “the investigations get finished, the wrist-slapping [ensues]. And the last thing people think of is the journals.” AACR publications ran three of van Parijs's papers.

An examination by Science of more than a dozen fraud or suspected fraud cases spanning 20 years reveals uneven and often chaotic efforts to correct the scientific literature. Every case has its own peculiarities. Whether wayward authors confess to fraud; whether investigations are launched at all, and if they are, whether their scope is broad or narrow; whether fraud findings are clearly communicated to journals—each of these helps determine how thorough a mop-up ensues.

Large-scale fraud cases are rare. But scientists whose work is challenged have often co-authored dozens or even hundreds of papers. How their legacy is handled may determine whether work by innocent co-authors, particularly young scientists, is wrongly tainted. But debates rage over how comprehensive fraud investigations need to be—whether, for example, they ought to examine a scientist's entire body of work regardless of expense.

And then there are the journals, keepers of the historical record. Journal editors often stress—and universities and funders agree—that publications are in no position to investigate fraud. The burden, they say, should be on institutions and funding agencies; they have the money and staff to convene sweeping inquiries and demand raw data. Traditionally, journals wait for the results of inquiries to steer decisions on problem papers. Some act only if a retraction has been requested by a paper's authors—preferably all of them. But authors accused or suspected of fraud often don't agree to a retraction. Editors must then make a potentially career-wrecking decision, with varying degrees of guidance.

Even papers that investigators have found fraudulent can linger in the scientific record for years. In one case, findings of a fraud investigation in Germany were not translated into English. In another, some journals declined to correct obesity papers that a U.S. agency's exhaustive inquiry had deemed partly fake.

Fear of being sued lies behind inaction in some cases, especially when there has been no clear-cut finding of fraud. Some journals, however, are becoming more assertive, contacting investigators and settling on their own middle ground in nebulous cases. “All the participants are making up the rules as we go along,” says Barbara Cohen, executive editor of the Public Library of Science (PLoS) Publications.

Mopping up

A researcher is found guilty of fraud. A black mark is splashed across certain published papers, and it's recommended that they be withdrawn. What happens next?

“The responsibility is very much on the shoulders of those who know [of fraud] to correct the record as speedily as possible,” says Francis Collins, director of the National Human Genome Research Institute in Bethesda, Maryland. In the mid-1990s, one of his graduate students, Amitav Hajra, confessed to faking data on leukemia projects. In that case, all the authors requested that three papers be retracted and two others corrected. The journals responded within months.

For the journals, a confession followed by author unanimity to pull a paper is a best-case scenario. “The official rule for journals is that the authors must do the retracting,” says AACR's Case. A retraction on these terms sharply reduces the legal risk that journals will be accused of tainting a scientist's reputation by retracting a paper without his or her consent.

What to do when an alleged fraudster doesn't confess is fuzzier. “More and more, … the authors dig in their heels and try to salvage some of their reputation,” says Case. When this happens, journals often rely on the findings of investigators.

And here, they often hesitate.

Last March, for example, the Office of Research Integrity (ORI), which was formed in 1989 to investigate misconduct cases involving funds from the National Institutes of Health (NIH) and certain other federal agencies, broke bad news to 10 publications: a paper they had published was fraudulent.

The news was not wholly unexpected. Eric Poehlman, an obesity and aging researcher at the University of Vermont in Burlington, had left the school after a whistleblower brought concerns of research inconsistencies to university officials. ORI oversaw its biggest inquiry ever, covering 10 papers co-authored by Poehlman and 15 of his NIH grant applications. All 10 papers, they determined, contained fabricated data and ought to be retracted or corrected.

An ORI finding, many journal editors say, gives publications ironclad backing to withdraw a paper even if an author doesn't cooperate. But ORI officials weren't happy with the journals' response in the Poehlman case. By last September, 6 months after the office issued its report, six of 10 journals had published retractions or corrections, supplied by Poehlman as part of his agreement with government officials. Two more followed. But two journals have not acted at all, according to ORI officials and journal records. (Poehlman has pleaded guilty to making false statements on a federal grant application and is awaiting sentencing.)

The spotty response in the Poehlman case echoes another from 2 decades ago. In the mid-1980s, Paul Friedman, a radiologist and then-associate dean of the University of California, San Diego, spent 15 months overseeing an investigation of 135 publications by a colleague, Robert Slutsky, who was accused of widespread fraud. Of the 60 publications judged fraudulent or questionable, Science found retractions for 18. “The journals responded very variably,” says Friedman.

Journal editors, however, say the situation is rarely black and white. In the Poehlman case, the two journals that haven't carried out ORI's recommendations are outside the United States. Their editors may be less familiar with ORI, although ORI officials have no evidence that this explains their inaction.

Lengthy inquiries and garbled communication can also complicate removing tainted papers. Both were on vivid display in the case of cancer specialist Friedhelm Herrmann, who worked in Berlin, Freiburg, and Ulm, Germany.

PHOTO CREDIT: RAJ CHAWLA/UNIVERSITY OF VERMONT MEDICAL PHOTOGRAPHY

In 1998, Ulf Rapp, a cancer researcher at the University of Würzburg in Germany agreed, with some trepidation, to lead an inquiry into Herrmann's work set up by the DFG, Germany's main science funding agency. Rapp and his helpers painstakingly examined hundreds of autoradiograms, images that reflect RNA and protein production. In paper after paper, they uncovered autoradiograms that had been manipulated, flipped upside down, or recycled from earlier experiments. The investigators identified 29 “falsification-beset” publications; 28, they concluded, “should be withdrawn,” and in one case, a correction “would be sufficient.” Another 65, nine of them book chapters, were deemed “strongly suspicious.”

The investigation took more than 2 years, during which editors came and went at many journals. “What may have been on one person's radar isn't going to show up on another person's,” says John Hawley, executive director of the American Society for Clinical Investigation, which publishes the Journal of Clinical Investigation. JCI issued four retractions of articles co-authored by Herrmann, but three appeared in 2003, years after retractions elsewhere. (The fourth was retracted in 1998 at the behest of two authors.) JCI, says Hawley, was unaware that some other journals were retracting Herrmann's work. “But of course,” he says, “we will become aware of something eventually.”

In the Herrmann case, the DFG notified some journals directly, says DFG spokesperson Eva-Maria Streier, although none with whom Science spoke could recall having heard from the agency. The DFG “got cold feet,” says Rapp, because Herrmann vigorously denied fraud and threatened to sue. The DFG posted Rapp's report on its Web site, but, Rapp recalls, it wasn't easy to find there.

Streier responds that the DFG distributed the report at a press conference and posted it online. The list of papers was in English, she says, although she acknowledges that blurbs summarizing the fraud status of those publications went untranslated.

At the journals, the result was chaos. AACR journals had published six papers co-authored by Herrmann that Rapp's inquiry deemed fraudulent or suspicious. Case, who never saw the DFG report, contacted Rapp. He recommended the papers be withdrawn. But “this is one guy saying you probably should,” says Case. All six papers are still in the literature.

“I think we got a preliminary report … that was written in German” but couldn't glean a clear message from it, says Richard Dodenhoff, journals director of the American Society for Pharmacology and Experimental Therapeutics, which publishes Molecular Pharmacology. Like other editors, he says he traditionally waits for guidance from investigators. “Normally, we rely on some official body to tell us that there's something wrong,” says Dodenhoff, adding that the journal was never informed that its papers should be pulled. The journal has three publications on the “falsification-beset” list and one on the “strongly suspicious” list. None was retracted.

Other journals reacted differently. Blood, a popular venue of Herrmann's, was well aware of the investigation early on. The editor at the time, Kenneth Kaushansky, a hematologist at the University of California, San Diego, retracted eight articles by Herrmann, all of those it knew to be problematic, over protests from the journal's attorney that the corresponding author—in several cases, Herrmann himself—had not given permission.

Finally, fraud investigators propose another reason why journals sometimes might be reluctant to pull a paper: Retractions may rank low on the priority scale and can breed bad blood between the journal and researchers. They can also reflect poorly on a publication. “You don't want to make waves,” says Friedman. The journals with which he corresponded in the Slutsky case ranged from pleasantly collegial to downright defensive.

Rapp's contact with the journals left a bitter taste in his mouth. Most ignored his notes and faxes, he says, or “wrote back very nasty letters.” Several, he says, remained determined to get permission from every author on the paper. The reaction “did surprise me. … It seemed to me we were helping those guys. They had a rotten egg in their basket. We gave them a chance to clear it up.”

Casting the net

If a critical question for journals is whether to retract papers, the typical bind for investigators is how wide a net to cast after a paper is alleged to be fraudulent. The spectrum of opinions here is vast. An investigation's breadth lays the groundwork for how much might be corrected, and how much collateral damage—papers by a suspect researcher left unexamined by investigators—will remain.

Some investigators are driven by sheer curiosity and a desire to get at the truth. Uncovering some fraud leaves the nagging question: “You wonder, is any of his previous work, and there was a lot of it, invalid?” says Friedman, explaining the investigation of Slutsky, which examined every one of the radiologist's papers.

But is it worth parsing a 10- or 20-year-old paper when this saps time and money from university faculty members, government officials, and journal editors? Some scientists and journals favor digging into every paper by a known or suspected fraudster, whereas others believe that narrower inquiries suffice. Often, there is no easy way to measure the value of catching fakery in the scientific literature.

For people like Steven Shirey, that value is incalculable. Last July, the Washington, D.C., geologist had a 2-cm by 2-cm square of skin excised from his tongue. The experience was traumatic enough, but Shirey felt he had no choice: Genetic testing had revealed that a lesion found there was aneuploid, meaning it had an abnormal number of chromosomes. A Norwegian oncologist, Jon Sudbø of the University of Oslo, had found that 84% of people with aneuploid lesions go on to develop a deadly form of oral cancer. The work appeared in 2001 and 2004 in The New England Journal of Medicine (NEJM).

“Based on Sudbø's article, I thought I had a death sentence,” Shirey says, adding that he would have signed up for a preventive chemotherapy trial had one been available.

In January, after a whistleblower raised questions about data in a recently published Lancet paper, Sudbø admitted through his attorney to faking signature findings on aneuploid lesions in the 2004 NEJM paper. The 2001 paper is under suspicion after journal editors found that it contains a pair of duplicate images. Anders Ekbom, a surgeon at the Karolinska Institute in Stockholm, is leading an investigation into all of Sudbø's 38 research papers. He hopes to complete it later this spring.

Shirey knows now that one of the publications that guided him as a patient contained faked data (although its conclusions may turn out to be correct). But his story is a cautionary tale of the devastating impact fraudulent findings can have. Of all types of fraud, fakery in clinical research tends to engender the greatest sense of urgency among universities and investigators. In its breadth, the Sudbø inquiry mirrors unusually comprehensive inquiries in two other fraud cases, those of Herrmann and Poehlman. All encompass clinical research that can directly affect patients—and in some instances, already has.

Even in cases like these, though, there's disagreement over the hours, money, and sweat worth pouring into an investigation. ORI prefers extensive inquiries and frequently prods institutions to expand them, says Alan Price, ORI's associate director for investigative oversight. But there comes a point at which even ORI feels it has to stop. Investigating 100 papers “is a horrendous investment,” says Price. The inquiry into 10 papers and 15 grant applications of Poehlman's took the better part of 2 years. Poehlman's name is on 204 papers in all. “At some point,” says Price, “you have to say, ‘Maybe this implies that his work cannot be trusted.’”

But Poehlman's employer, the University of Vermont College of Medicine, isn't satisfied. After ORI's findings on Poehlman were released a year ago, “we were left with nothing said about the vast majority of his 200-odd papers,” says Russell Tracy, senior associate dean for research and academic affairs. The school, says Tracy, felt an obligation to those who participated in Poehlman's menopause studies, “and responsibility to the people of the country whose clinical care might be impacted.” Pressed in part by faculty members whose names appear on papers alongside Poehlman's, Vermont is asking geographically dispersed co-authors whether they can vouch for the data in an additional 125 papers or so—all those published since 1995.

The investigation into Herrmann and his colleagues, who had published many studies of drug effects on cancer cells, was even more exhaustive. DFG asked Rapp to examine more than 600 publications. Herrmann was a co-author on 347 of them. “My main goal was to clear up the literature—that's a community service,” says Rapp. The work with clinical implications left him especially queasy. “Just imagine someone in New Zealand reads this paper and says, ‘That's cool, I can do this with my patients,'” says Rapp.

In other fields, investigators may worry less about dissecting a scientist's multiyear oeuvre. The name of Jan Hendrik Schön, a Bell Labs physicist at the center of one of the most notorious fraud cases in the physics world, appeared on more than 90 papers. After whistleblowers alerted Bell Labs in the spring of 2002 to an identical figure in two papers, Bell Labs launched an investigation. But it chose to examine only papers about which concerns were being raised, 25 in all.

“Our committee was put together to investigate allegations of scientific misconduct, not to go look for it,” says Malcolm Beasley, a physicist at Stanford University in California who headed the 4-month inquiry. That arrangement was driven in part by the numbers of co-authors involved. Bell Labs was also under heavy pressure to assess Schön's fakery quickly. Beasley's committee found that 17 papers contained fake data; all were retracted, along with an additional 14 that had been based on the suspect work.

Both Beasley and Lydia Sohn, one of the Bell Labs whistleblowers who is now at the University of California, Berkeley, believe that because so many of the papers examined were fraudulent, the rest can safely be considered tainted. “The burden of proof shifts under these circumstances,” says Beasley. Adds Sohn: “Pretty much people have written off” anything by Schön.

PHOTO CREDIT: S. PFRÄNGER/INSTITUT FUER MEDIZINISCHE STRAHLENKUNDE UND ZELLFORSCHUNG (MSZ)

“Over some measure of time, the community … has a way of self-correcting,” agrees Samuel Kaplan, a microbiologist at the University of Texas, Houston, and chair of the American Society for Microbiology (ASM) publications board. ASM journals published one paper by Herrmann, one by MIT's van Parijs, and one by Hajra, Collins's student. (The latter's work was retracted.) Referring generally to older and clearly fraudulent research and speaking for himself, Kaplan argues that “if those papers are hanging out there, they're probably not going to do any harm. We could go ahead and say in a journal in 2006, a paper published in 1997 is suspect—but no one's going to do that.”

Collateral damage

Between clear fraud and clean data lies a vast sea of gray. Inevitably, investigators label some papers “suspicious” but not definitively fake. University investigations fail to confirm fraud without ruling it out. Some fraud inquiries sought by worried journals never launch.

“We don't condemn people for crimes that we couldn't prove,” says PLoS's Cohen, pointing out the difficulties of retracting a suspect article. But at the same time, suspicions of fraud that are never established put journals in an uncomfortable position. These uncertainties are “the Achilles' heel of the process,” says Kaplan, and no one seems to know how to handle them.

Legal concerns weigh heavily. “I have not seen actual litigation, but I have seen threats of it,” says Debra Parrish, an attorney at Parrish Law Offices in Pittsburgh, Pennsylvania. Threats are sometimes enough to discourage journals from taking action. Dodenhoff, who oversees Molecular Pharmacology, recalls an instance when the journal stepped away from alerting readers to suspicious findings for fear of a lawsuit. “We're fairly small, we don't have a lot of money, and frankly we backed off,” he says.

Attorneys say that journals' legal fears are sometimes overblown. “If there has been a complete due process investigation, … a journal can rely on a report,” says Barbara Mishkin, a partner at Hogan & Hartson in Washington, D.C. But retractions in suspicious cases are “going to be a close call for the journal editor,” she says. In theory, publications could be sued for libel or defamation, explains Mishkin, or even contract interference, if the retraction torpedoes a scientist's chance for a job for which he'd been under consideration.

Occasionally, journals take calculated risks, retracting a paper even absent misconduct findings or author unanimity. In late 2004, Cell and the Proceedings of the National Academy of Sciences (PNAS) retracted papers co-authored by Meena Chandok, a plant biologist who had been working at the Boyce Thompson Institute (BTI) for Plant Research in Ithaca, New York. According to a lawsuit later filed by Chandok, her supervisor, Daniel Klessig, “accused her of falsifying her research data” and requested the retractions. BTI also began conducting an investigation.

PNAS's executive editor, Diane Sullenberger, said in an e-mail message that the journal attempted to get all the authors to agree to the retraction but couldn't locate Chandok. It pulled the paper anyway. The retraction mentioned that Chandok had not signed off on it, a stance Sullenberger says the journal felt comfortable with.

In the end, BTI's president David Stern said in a memorandum, the allegation was “not substantiated,” although he noted “numerous disputes on factual issues.” Chandok sued Klessig last August for defamation that her lawsuit claims has “significantly damaged” her scientific reputation. The journals were not included in the suit.

If the authors don't initiate retractions, publications usually wait for the findings of an investigation before assessing their options. Doing otherwise carries grave risks, editors believe. “Our lawyers have told us that we [would be] wide open for a lawsuit,” says Kaplan.

But what happens when an investigation doesn't cover a particular journal's papers, or isn't undertaken at all? Since stem cell scientist Woo Suk Hwang, formerly of Seoul National University, was accused last fall of one of the most brazen frauds in recent memory, multiple investigations have swung into gear in South Korea and the United States. It's not yet clear how many of the dozens of papers by Hwang and his collaborators at MizMedi Hospital and Hanyang University in Seoul the investigators will be examining.

That's prompting some editors to take matters into their own hands. At Stem Cells, executive editor Martin Murphy is angst-ridden over a paper the journal published in 2004, whose nine authors include seven Hwang collaborators. Desperate to learn whether the paper is fraudulent, he has contacted officials at Seoul National University, MizMedi Hospital, Hanyang University, and the University of Pittsburgh, where two of the scientists had been working. No Korean investigation has scrutinized the paper. A Pittsburgh dean, he says, told him that “since no one was of employ at Pittsburgh at the time the papers were published, it's outside our review.”

Murphy's concern is twofold: correcting the literature in his journal, and shielding the innocent. “You really have to protect those folks who were swept along,” he says. Unable to identify the blameless, Stem Cells has for now banned articles by any of the nine authors. On 2 April, Stem Cells issued an editorial retraction of the paper.

Sometimes investigations simply don't happen or quickly peter out. “Papers from outside Northern Europe, Australia, North America, some countries where regulatory bodies are extremely reluctant to get involved, we usually have to give up in the end,” says Harvey Marcovitch, chair of the Committee on Publication Ethics, a U.K. group that advises journals on ethics matters. Geographic patterns aside, this can happen anywhere.

In 2000, for example, the British Medical Journal (BMJ) rejected a paper by Ranjit Kumar Chandra, a nutrition researcher then at Memorial University in Newfoundland, Canada. The work examined how a multivitamin improved the memory of older individuals. But a BMJreviewer had concerns about its statistical analyses, says Richard Smith, then the journal's editor. Concerned, Smith contacted Memorial.

Unbeknown to him, the university had been handling complaints against Chandra since the early 1990s, according to a statement now posted on its Web site. But “repeatedly Dr. Chandra avoided fulfilling” the university's requests for his data, the statement notes, at one point claiming “that the data had been stolen.” Christopher Loomis, vice president of research at Memorial, says that absent Chandra's cooperation, the school couldn't reach a definitive outcome. “We ended up with a ‘he said, she said’ situation,” he says.

Meanwhile, in 2001 Chandra published in Nutrition the paper originally submitted to BMJ, says Smith, who contacted Michael Meguid, the journal's editor, to let him know. Nutrition took the rare step of launching its own investigation, says Meguid. Meguid asked outside scientists to examine the study's data, and in 2005 the paper was retracted. The Nutrition retraction notice described statistical concerns and added that “Chandra failed to declare that he … has a financial stake” in a vitamin supplement formula the study was testing.

PHOTO CREDIT: LINDA A. CICERO/STANFORD NEWS SERVICE

Chandra has published some 200 articles in a career that dates back more than 30 years. The Nutrition paper was the only one for which Science found a retraction.

In general, in the absence of firm investigative findings or unanimous author requests for a retraction, journals do not retract a paper. Increasingly, however, they are seeking a middle ground to accommodate the fuzziness they keep encountering. The “Expression of Concern,” an editorial acknowledging worries about a paper without formally pulling it, has been popping up more and more. In the Sudbø case, for example, The New England Journal of Medicine quickly published an expression of concern about the aneuploid papers after noting duplicate images in one, although Sudbø had not personally and publicly confessed to faking them and the investigation was just beginning.

A creative approach was taken by a senior scientist trying to clear his name after the Max Planck Institute for Plant Breeding Research in Cologne, Germany, concluded that his lab technician had falsified experiments. In 1999, plant biologist Jeff Schell of Max Planck published a paper in The Plant Journal detailing eight papers co-authored by the technician that he and his colleagues stated could not be replicated. Two other papers on which the technician was an author had already been retracted.

Schell's paper was handled delicately by the journal and published like any other new finding, says Irene Hames, now managing editor of The Plant Journal. As a condition of publication, Schell sent letters of correction to each of the journals that had run the articles in question. “I think the journal is the primary point of enforcement” against fraud, says Harry Klee, The Plant Journal's current editor. “In the end, it's our process that got that work into publication.”

But that attitude doesn't seem to be widely shared. Eight years after undertaking his mammoth investigation of Herrmann, Rapp's tone is flat and dispirited. “It was a waste of time,” he says now. Science found that just 13 of the 29 “falsification-beset” papers were retracted. Six others in the suspect category, out of 56 papers, were also pulled, and two corrected. Problematic papers identified by Rapp's investigation still litter the scientific record. Their influence on ongoing research is anyone's guess.

View Abstract

Stay Connected to Science

Navigate This Article