The Old File-Drawer Problem

See allHide authors and affiliations

Science  23 Jul 2004:
Vol. 305, Issue 5683, pp. 451
DOI: 10.1126/science.305.5683.451

There is welcome news about an old problem. For years, we've been getting only part of the story on clinical drug trials. The successful ones get published and touted, but others that didn't work out so well may never see the light of day. New developments, however, promise a long-awaited exposure of the negative results.

Most scientific studies that examine a possible threat or benefit to the public health are repeated, sometimes by several different investigators. When high economic stakes are involved, someone is usually interested enough to perform a meta-analysis, pooling the results of all the published studies to test for significance. That's true for clinical trials, toxicity tests, and other studies designed to assess human risks. So far, so good. But a thoughtful statistician can spoil the fun: “Look, journals and scientists like positive results and get disappointed by negative results. So there's a problem—all the unpublished negative results lurking in those file drawers!” Thus, the fly in the meta-analysis ointment: It's likely that aggregated results from published papers constitute a biased sample.

How does the old problem of selection bias relate to new events in the world of clinical trials? Well, pharmaceutical companies often gather favored medical specialists to evaluate, and even tout, the value of a particular drug treatment. Results of clinical trials favorable to the drug and its sponsor are collected, sometimes in the form of a symposium volume or even as a supplement to a specialty medical journal. The U.S. Food and Drug Administration (FDA) has attempted to regulate such sponsored publications in the same way as it regulates advertising in medical or lay journals.


But that weapon has been blunted. In a recent lawsuit, the Washington Legal Foundation challenged the FDA's authority to regulate the promotion of drugs for use “off label”—that is, for treating symptoms or diseases for which the drug has not been proven effective by the FDA. The court ruled that the FDA had to permit drug company-sponsored advertisements for off-label use, as long as they were directed at physicians and not consumers. The legal environment now appears to favor the “commercial free speech” doctrine; a position, ironically, that was supported by the FDA's chief counsel on behalf of drug company clients before he came to the agency. The FDA, not surprisingly, is now more hesitant about undertaking enforcement activity against claims made by pharmaceutical firms.

The difficulty is that positive claims are sometimes made against a background of unrevealed negative results. A clinical trial that fails to show effectiveness (or indicates safety problems) is required to be submitted to the FDA along with the positive results. But these may not be released, because their proprietary nature is recognized under the law. Back in 1977, an advisory body to the Secretary of Health, Education and Welfare pleaded for public access to these data. Subsequently, consumer groups such as Public Citizen have repeatedly sued to liberate them, with only limited success. In 1997, a new U.S. federal law required companies to register trials at a government database, later called ClinicalTrials.gov. But the FDA has lacked the authority to enforce compliance, and the database remains incomplete.

Now a rescue is beginning to take shape. Revisions to the U.S. law regarding the government- enforced registry are now under discussion. The International Committee of Medical Journal Editors is also considering a proposal whereby journals that publish the results of clinical trials would require sponsors to deposit trial-related results in a national registry as a condition of publication. The American Medical Association agrees with this and has recommended that institutional review boards make registration a condition for allowing any trial to proceed. The World Health Organization also plans to propose an international registry of drug trials to national health ministers later this year.

These new proposals are promising, and some thanks surely go to the innovative and very public Attorney General of New York, Eliot Spitzer, who made an amazing discovery last month: The First Amendment is not a defense against fraud! He has sued the pharmaceutical complany GlaxoSmithKline for holding back data that may have made the antidepressant drug Paxil look less effective than the successful trials being advertised. That neatly finesses the legal constraints on the FDA, whose scientists should be cheering. And it might finally empty some of those old file drawers.

Stay Connected to Science

Navigate This Article