News FocusScience and the Stimulus

Medicine Under the Microscope

See allHide authors and affiliations

Science  27 Nov 2009:
Vol. 326, Issue 5957, pp. 1183-1185
DOI: 10.1126/science.326.5957.1183

A billion dollars in stimulus money will go to evaluating which medical treatments and procedures work best.

Long-standing advice to all women in the United States that they should get a mammogram every year after age 40 was overturned last week. After combing through medical data and analyzing six models of disease progression, an expert panel declared that regular mammograms, which expose women to excess radiation and may lead to unnecessary surgery, are more harmful than helpful to someone younger than 50. Even after 50, the panel said, a mammogram every other year would be sufficient. The American Cancer Society and the American College of Radiology immediately rejected the new advice—as did Health and Human Services (HHS) Secretary Kathleen Sebelius the next day—recommending that women follow the familiar old rule.

Such turmoil in the medical ranks could become more common if an evidence-rich kind of analysis backed by the economic stimulus bill takes off. It's known as comparative effectiveness research (CER), and it received a windfall of $1.1 billion in February under the American Recovery and Reinvestment Act. CER employs methods similar to the approach that produced the new advice on mammography.

CER draws data from many sources to reach an evidence-based judgment on the value (or lack of value) of medical techniques and strategies. The process must be rigorous, according to a definition of CER hammered out in June by a federal coordinating group ( Its scope is broad, ranging from comparing drugs in a clinical trial to studying behavior-modification methods to dissecting the impact of health policies. The aim is the same, however: to survey a patient's choices and determine which course works best. The results of a CER study are somewhat like a consumer's guide (see sidebar, p. 1184)—and often as confusing.

Obama Administration officials are enthusiastic about CER and pushed for its inclusion in the stimulus bill as a way to “bend the curve” of health care spending. By this, they mean that it can spot ineffective and overused medical procedures. Armed with such information, they argue, patients and doctors will become more discriminating. In time, they hope, this will slow the growth of U.S. health care spending, which now stands at about $2.4 trillion per year, or one-sixth of the economy.

CER proponents also hope to use the approach to analyze “real world” medical problems. This phrase suggests something different from classic randomized clinical trials, which generally exclude patients who don't fit a specific profile. CER studies, in contrast, may gather records from small clinics and observational studies, taking in people of all ages, including those with complex and overlapping medical problems. Proponents say this could deliver more practical information, more rapidly, than do randomized trials.

“You can see why we are happy, proud, and excited,” says Carolyn Clancy, director of the Agency for Healthcare Research and Quality (AHRQ), which is at the core of the government's push for improving medicine. Her agency has been getting by since 2005 with about $30 million a year for CER, she says. Now the stimulus windfall has “bumped up” the CER portion to $300 million. The National Institutes of Health (NIH) and the secretary of HHS, working with AHRQ, will parcel out the remaining $800 million in stimulus money (see table).

The plan is controversial. Some patient advocacy groups and physicians charge that CER studies could eventually be used to guide insurance and Medicare payments or to overrule a physician's judgment about what is best for the patient. CER has been denounced on blogs as a precursor to “medical rationing.”

Key decisions about CER could be affected by the roiling debate. It's not clear, for example, whether the billion-dollar launch of CER in 2009 will be followed by sustained federal support. Nor has it been decided specifically how the government will oversee this research. “There have been open differences about how to frame the research—with some people wanting a more restricted scope and others trying to avoid burdensome restrictions,” says Steven Pearson, president of the Institute for Clinical and Economic Review at Massachusetts General Hospital in Boston.


U.S. health costs have been rising steadily and taking up an increasing share of the economy; leaders hope CER can help “bend the curve” downward.


Spending the windfall

With their pockets full of cash, the three agencies with CER stimulus funding began to plunk down commitments this fall. Clancy says the top priority at AHRQ is to pay for “evidence generation,” funding investigators to learn what happens to patients with similar ailments who undergo different treatments. Linked to this, Clancy says, is a push to build infrastructure by investing in clinical databases, sharpening analytical methods, and creating networks to disseminate findings.

NIH was first out of the blocks. It has awarded 165 CER grants, totaling $350 million of the $400 million to be spent ( Like others, NIH is funding work on topics from a list of 100 questions drawn up in June by a panel of experts at the National Academies' Institute of Medicine. ( NIH Director Francis Collins noted in October that much of this research is familiar: “Eighty-eight of the 100 priorities in the plan are already covered” by NIH-funded investigators, he said in a talk at AAAS (which publishes Science).

Typical of the NIH winners is a $2 million project that will zero in on the use of advanced medical imaging, a controversial topic. New machines are being deployed rapidly, says Bruce Hillner, a physician-researcher at Virginia Commonwealth University in Richmond who's involved in the project. X-ray computed tomography, magnetic resonance imaging, and positron emission tomography (PET) are now used for many purposes, from checking out nonspecific “belly pain” to detailed monitoring of cancer as it progresses or recedes. “For more than a decade, the use of these technologies has increased 15 to 20% every year,” Hillner says. Studies have shown that specific uses have great value, but indiscriminate use may have penalties, such as triggering “futile surgery”—life-threatening operations that lead to no benefit. Furthermore, the imaging boom isn't driven strictly by medical evidence; it may also be fueled by “self-referral” by physicians who have a stake in imaging services.

Worth it?

An important question for comparative effectiveness research is whether medical scans are used appropriately.


Hillner is co–principal investigator at a repository, the National Oncologic PET Registry (NOPR), that has gathered data from U.S. cancer-related PET scans since 2006. He's teaming up with Dartmouth Medical School researchers William Black and Anna Tosteson, who lead a new collaboration to review cancer imaging. Hillner says he plans to “work backwards” from the NOPR database, which was designed for this purpose, to see if physicians followed the cancer management plan they described when they sought reimbursement for a PET scan. Among the questions he'll be asking are: Did the plan change? What happened to the patient? How did PET affect surgery decisions, including futile surgeries? Did physicians generally use multiple overlapping technologies, or did they use only PET?

Like NIH, which is putting nearly all of its money into research, AHRQ will invest $100 million of its $300 million in investigator-initiated clinical studies called CHOICE awards. It plans to make 10 of these at roughly $10 million apiece, running for 3 years each. For these, the White House has waived the rule that stimulus money be spent within 2 years. “They are big grants” for the field, says Tosteson.

AHRQ hasn't announced the CHOICE winners as of yet. But its detailed spending plan, required under the Recovery Act, includes $48 million for patient registries, $25 million for evidence synthesis, $29.5 million for dissemination of results, and $20 million for training. (HHS hasn't released a list of grants or a spending plan.)

How will findings be used?

The plan to boost CER ran into flak early this year when opponents said studies might be used to cut back on specific treatments in an attempt to lower costs and ration medical care. Critics included well-known physicians, patient advocacy groups, and members of Congress.

Evidence, please.

Carolyn Clancy directs AHRQ, which funds research on what works in medicine.


One surprising blast came from the respected hematologist and The New Yorker writer Jerome Groopman and his wife, Pamela Hartzband, both of Harvard Medical School in Boston. Their 31 August editorial in The Wall Street Journal suggested that a national effort to define the best practices in medicine would end up describing only averages and not what's good for a specific patient. They warned that economic incentives intended to nudge doctors toward following national rules—as some would like to do—would mean that “federal bureaucrats are directing health decisions.”

Advocacy groups have been outspoken, too. Tony Coelho, a former Democratic congressman from California who heads the new Partnership to Improve Patient Care, has kept up a steady drumbeat of warnings about the need to circumscribe CER. The group argues that CER is fine if used to inform patients and doctors, but it should not be used for “making centralized coverage and payment decisions or recommendations.” Coelho's group is supported by the Biotechnology Industry Organization, the Pharmaceutical Research and Manufacturers of America, and the American Association of Neurological Surgeons, among others.

In response to such concerns, Congress decreed in a conference report on the stimulus bill that CER should not examine cost issues. And members of Congress specifically banned using such research to “mandate” changes in Medicare. The future use of CER hangs in part on whether this ban is continued. “Even without cost-effectiveness information,” says Harvard historian of science Jeremy Greene, an expert in this subject, “I think CE information is essential to improving the quality of our health care system.”

The health care reform bills now before Congress may also determine how CER is managed. The House of Representatives–passed bill and the version drafted by Senate Democrats both seek to reduce friction over CER findings by ensuring that the government consults with stakeholders—patients, doctors, and industry representatives—before making big investments in CER or acting on research findings. But the House and Senate proposals differ on who calls the shots. The House plan would rely essentially on AHRQ to set research policies, with input from advisers. The Senate approach would vest authority in an independent, nongovernment corporation.

Congress is treating the subject with caution precisely because CER promises a radical change in the way medical practices are evaluated, bringing a lot of new data to bear on decisions. And as the brouhaha over screening for breast cancer shows, new evidence isn't necessarily welcome.

Navigate This Article