News FocusThe War in Afghanistan

Counting the Dead in Afghanistan

Science  11 Mar 2011:
Vol. 331, Issue 6022, pp. 1256-1260
DOI: 10.1126/science.331.6022.1256

A military data set of civilian casualties, provided exclusively to Science, indicates that the war has become more lethal to the Afghan population, largely because of indiscriminate insurgent attacks.

Dangerous protectors.

As the conflict in Afghanistan drags on, civilian casualties are increasing.

CREDIT: AFP/GETTY IMAGES

KANDAHAR, AFGHANISTAN—By day, you don't see the war. The wind throws up a red haze that obscures everything. Bearded Afghan men in colorful vests whip their donkeys, lugging vegetables and firewood. Unless you spot an armored convoy picking its way across the rugged landscape or hear a fighter jet scream overhead, you couldn't even guess the century. But once night falls, a brilliant cluster of lights appears, as if a new city has sprouted in the valley several kilometers south of the provincial capital. This is Kandahar Airfield. What used to be a derelict airport littered with debris from the Soviet era has grown into a massive base for the International Security Assistance Force (ISAF).

Inside its outer maze of concrete walls and machine gun nests is a world of surreal contrasts. Thousands of ISAF soldiers clog the streets, some hurrying to dinner at the end of a grueling workday, others just starting their night shifts. The conversations are in English, French, German, Dutch, and other languages from the 48 nations that have contributed troops to the ISAF coalition. There is no safety beyond the walls, so all recreation must happen here. The Americans brought in a string of restaurants, including T.G.I. Friday's. The Canadians built a full-sized, open-air hockey rink—the floor is concrete rather than ice—and they play in full gear under the glare of stadium lighting. Tonight, men and women in uniform amble along a wooden promenade, licking ice cream cones and watching a volleyball game.

The spell is broken as sirens blare and everyone dives for the ground. Once per day on average, insurgents manage to send small rockets flying over the walls. The Canadians in the hockey rink famously play through these attacks, each team refusing to flinch. But newcomers on the base can't help but flinch constantly—every jet overhead sounds like an incoming rocket. It turns out to be a drill this time. But an hour later, a dozen missiles erupt from the base's launchers, breaking the speed of sound in seconds and fading over the horizon like cigarette embers. Somewhere to the west people are dying.

What began as an invasion to capture Osama bin Laden in 2001 has become a full-blown occupation and counterinsurgency. As the war in Afghanistan grinds toward the decade mark—the longest in U.S. history—public support is waning. When he testifies before the U.S. Congress next week, ISAF Commander Gen. David Petraeus will face some difficult questions. Most contentious is the issue of civilian casualties. Media reports of surging violence give the impression of a country slipping out of control, but ISAF has reported steady progress. “There is so much rhetoric flying around and none of it can be tested,” says Neil Johnson, a physicist at the University of Miami in Florida who studies the dynamics of warfare. “What we need is hard, reliable data.”

A deluge of data arrived last year from WikiLeaks, the organization that has made public thousands of classified documents. It includes the raw observation of casualties by soldiers on the ground in Afghanistan, but the many unknowns surrounding those reports have left researchers puzzled about how to interpret them. A few independent organizations, including the United Nations, have published their own reports on civilian casualties in Afghanistan, but only for illustrating broad trends. The data underlying their reports have never been released.

Data Visualization

Embedded Image

See an interactive visualization of the casualty data, created by G. M. Brower. (Requires Google Chrome 9, Safari 10.6 or Firefox 4 browser.)

For the first time, those data are now publicly available. In January, ISAF provided Science with a database of civilian casualties called CIVCAS. It is the military's internal record of the death and injury of Afghan civilians, broken down by month, region, weaponry, and perpetrator. By its reckoning, 2537 civilians were killed and 5594 were wounded over the past 2 years, with 12% of those casualties attributed to ISAF forces and the rest to insurgents. The death toll is 93% identical to that in the WikiLeaks data, revealing those raw field observations to be far more reliable than researchers had suspected.

In February, after learning that the military was releasing these data, both the UN and an Afghan human rights organization agreed to release versions of their own civilian casualty data to Science. They show twice as many civilians killed over the same period, including 393 deaths by air strikes that were not counted in the military database. ISAF officials acknowledge the gap. “The civilian casualties reported by the UN have always been higher than those reported by ISAF,” says U.S. Navy Rear Adm. Gregory Smith, the director of communications for NATO based in Kabul. “But the trends have been very consistent.”

Science assembled a team of experts to analyze the released data sets. They conclude that while the war has grown deadlier for Afghan civilians over the past 2 years—up to 20% more civilians were killed in 2010 compared with the year before—ISAF has become a safer fighting force. The majority of deaths, and nearly all of the recent increase, are attributed to indiscriminate attacks by insurgents rather than ISAF soldiers. In spite of a troop surge and the launch of new operations against the Taliban last year, the data provided by the UN show a 26% drop in civilian deaths caused by military forces. And both the UN and ISAF data sets show a drop in deaths due to air strikes last year, by 50% and 10%, respectively.

All of these data, as well as other information never before released, are now online at http://scim.ag/afghandata. Taken together, they provide the clearest picture yet of the human cost of the war.

Counting bodies

Millions of people have died in modern wars, but the exact number is anyone's guess. Historians estimate that at least 10 million people were killed in World War I and at least 50 million in World War II. Although records exist for the number of soldiers who went to war and never came home, most civilian deaths went uncounted. The military did not systematically track casualties beyond their own troops.

That changed in the late 1960s, when Americans opened their newspapers to find an official count of the people dying in the Vietnam War. Each week, the U.S. military released those numbers to the media, dividing the weekly casualties between U.S. soldiers, U.S. allies, and “Communists killed.” One of the readers was Michael Sutherland, then a statistics Ph.D. student at Harvard University. “The military had a goal of achieving a 10-to-1 kill ratio,” he says. “The idea was that if we were killing 10 of their guys for every one of ours, we were winning.” Sutherland, who was eligible for the draft and had friends already serving in Vietnam, started collecting and analyzing those weekly numbers.

The kill ratio was indeed approaching 10-to-1, but he discovered other patterns. It was clear that at least some of the data were fabricated. The frequency of the numbers' last digits was skewed, with far too few zeros and fives compared with chance. The reason? “If you're making up numbers, you never say that 150 Communists were killed,” Sutherland says. “Instead, you use 147 or 152.” By comparing the trends in deaths, he also found that U.S. soldiers were clearly the ones fighting the battles rather than the South Vietnamese allies. “We were told that we were over there only as advisers,” he says. Even the official number of troops deployed to Vietnam looked fishy. “We were being manipulated by the government through the media,” says Sutherland, now a statistician at the University of Massachusetts, Amherst.

Forty years later, with wars raging in Iraq and Afghanistan, the military has become far more cautious with casualty statistics. The new approach was described by U.S. Army Gen. Tommy Franks. “We don't do body counts,” he told reporters in Afghanistan in 2002. Although the deaths of coalition soldiers in Iraq and Afghanistan are available—for example, at www.icasualties.org—an “information vacuum” has surrounded civilian casualties, says Michael Spagat, an economist at Royal Holloway, University of London. So researchers trying to measure the human cost of the wars have had to turn to other data sources.

One source is the media. The Iraq Body Count (IBC) Web site has tallied Iraqi civilian casualties—over 100,000 and counting—from media reports since the U.S.-led invasion in March 2003. However, this method can provide only a lower limit to the true number.

Another source is the civilian population itself. Several research teams have used household surveys to estimate casualties in Iraq. A 2006 survey published in The Lancet claiming a civilian death toll of 600,000 has been widely criticized (Science, 6 March 2009, p. 1278). A larger survey in Iraq led by the World Health Organization came to a figure close to 150,000 for the same period.

But in Afghanistan, these methods may be impossible. The country's size and population are close to those of Iraq—both have 30 million people in an area comparable to the size of France—but that is where the similarities end. Whereas Iraq is a flat country with a well-educated and mostly urban population, Afghanistan is a nightmare for field research. Most Afghans live in small villages nestled within rugged and poorly connected river valleys. Their inaccessibility thwarts surveys, and a lack of journalists on the ground makes rigorous media-based casualty counts “extremely difficult,” says John Sloboda, the director of IBC based in London.

These difficulties have not stopped some organizations from publishing estimates of the death toll in Afghanistan. The Red Cross has monitored the flow of casualties into hospitals, for example, and the UN has collected statistics through its regional offices. Although these organizations have published estimates periodically, none has given researchers access to their data to make an independent assessment.

Of course, the organization in the best position to directly record civilian casualties is the military itself, with nearly 150,000 observers on the ground witnessing the violence every day. But it seemed that the military kept no record of those observations—that is, until last year when WikiLeaks showed otherwise.

From WikiLeaks to CIVCAS release

Starting in July 2010, the largest leak of secret military information in history went public. WikiLeaks gave media outlets 92,000 internal military documents related to the war in Afghanistan and 400,000 from Iraq, allegedly provided by a low-ranking U.S. soldier who is now in custody. The documents include the raw operational reports from troops on the ground between January 2004 and December 2009. In Iraq over that period, soldiers reported a total of 79,000 civilian deaths, 15,000 of which the media missed (Science, 29 October 2010, p. 575). Afghanistan has been spared Iraq's sectarian violence. Over the same 6-year period, the leaked documents note 4024 Afghan civilian deaths.

The information vacuum was breached, but researchers have been wary of using the data. The military's raw operational reporting was not intended for research, so any errors, biases, and inconsistencies they might contain are unknown. Did investigations confirm those casualties? How many bodies identified as “combatants” later turned out to be civilians, and how many of the wounded subsequently died from their injuries? Without those answers, the data do little to dispel the fog of uncertainty.

Late last summer, a confidential source within ISAF informed me that the military was curating a database of civilian casualties. He described a dedicated military team that investigates civilian casualties and analyzes trends in the final tally to help ISAF reduce the number. In a series of e-mail exchanges with Science, ISAF officials confirmed that such a tracking system does exist and that its output is an internal database of civilian casualties called CIVCAS.

In October 2010, ISAF hosted me in Kabul and Kandahar as an embedded reporter. I was given access to military personnel at every level of the civilian casualty–tracking system, from the collection and quality-checking of CIVCAS data to the analysis that leads to new combat directives. I was also able to tour medical facilities and interview medical personnel (see p. 1261). What I was not allowed to do was take the data with me. ISAF officials were concerned that sensitive information associated with civilian casualty data—such as the tactics and movements of troops—could be revealed. But after 3 months of negotiation, ISAF agreed to give the entire CIVCAS database to Science for public release.

“Our database is 100% transparent,” says Smith. “Ultimately, this is a war being fought here in Afghanistan on behalf of the Afghan people.” Along with the death rate of soldiers, he says, this is “the most significant data set in identifying whether or not you're making progress.”

In a series of confidential meetings over the past year, ISAF has provided the UN and some human rights organizations full access to CIVCAS. “The other organizations will share their compiled data but not their raw data,” says Smith. “That's really to protect their own access, freedom, and independence, which is certainly something we recognize and respect.”

After learning that ISAF was releasing CIVCAS, some of those organizations provided Science with data as well. The United Nations Assistance Mission in Afghanistan (UNAMA) provided 3 years of their monthly casualty statistics, including a detailed accounting of deaths by air strikes since January 2009. The Kabul-based organization Afghanistan Rights Monitor (ARM) provided Science with the highest-resolution data of all, describing individual incidents during the first half of 2010. “Providing these data is a great public service,” says Sutherland, who is undertaking the first statistical analysis.

Science is making all these available at http://scim.ag/afghandata. ISAF has released additional sets of information to help researchers analyze the casualty data, including precise monthly troop deployment numbers over the past 3 years. Perhaps the most important is a document describing the standard operating procedures of ISAF's civilian casualty–tracking system. “This is crucial,” says Johnson. “It's not enough to see the numbers; you need to know how they are collected.”

From deaths to data

Nestled deep inside Kandahar Airfield is an inner bastion, like the nucleus of a cell. Stacked white trailers have been converted into a village of temporary offices. At the center is the Combined Joint Operations Center. Dozens of officers sit in an auditorium facing a wall of giant screens. Some show maps of the combat space, and others show live video feeds from unmanned drones. Since operations were launched last year to push the Taliban out of their original homeland around Kandahar, this has become the nerve center of the war.

“We have a casualty,” says an Australian officer sitting in the top row. (She requested that her name not be used because she is not an authorized ISAF spokesperson.) On her computer is a window of spooling text called JChat—it looks like an Internet chat session—providing a real-time view of the events of the war. One line of text is red, indicating a request for medical evacuation. It is one of many in this region, with about a dozen evacuations requested per day.

“It's an Afghan girl wounded by shrapnel,” she says, deciphering the string of acronyms in JChat. The officer is not concerned about who caused the injury. Her job is to save the girl's life by identifying the most efficient path to a doctor. She plots the grid references on a map, and a series of overlapping circles appear around the nearest helicopters. The path is computed and the orders are given. Help is on the way.

Meanwhile, a parallel stream of data has already started to flow. “This is the start of a process that we call Consequence Management,” says Col. Martin Bricknell, a senior U.K. military doctor and the medical director for southern Afghanistan. The soldiers on the ground with the injured girl have already radioed in a First Impression Report, describing the deaths and injuries that occurred. Once they return to base, they will give their commander a more detailed account. Within 9 days, ISAF Headquarters is expecting a CIVCAS assessment report. Besides reviewing the known facts of the case, the purpose is to improve future operations, Bricknell says. “Is there anything that we can learn from that so we can reduce casualties next time?”

At that point, the ISAF leadership will decide whether to investigate further. “We are absolutely determined to [track] the consequences of the conflict to the civilian population,” Bricknell says. “Therefore we take every allegation of a civilian casualty very seriously.” If local government officials give a different account of the event, for example, an incident assessment team will be assembled and sent into the field. For the most serious incidents, officials from the UN or other organizations join the team.

Casualties in time and space.

The seasonal rhythms and shifting battlefields of the war emerge in this view of the 8131 Afghan civilians killed or injured over the past 2 years, recorded in a military database called CIVCAS. (No data were available for the first 5 months of 2010 in the Southwest region.)

CREDIT: GEORGE MICHAEL BROWER

The CIVCAS database tracks all of these deaths and injuries. Between January 2009 and December 2010, it logged a total of 2537 civilians killed and 5594 wounded. About 80% of the deaths and injuries are attributed to insurgents. (The CIVCAS data go back to January 2008, but insurgent-caused casualties were not tracked until 2009.)

Throughout the war, critics have accused ISAF of undercounting civilian casualties, particularly those caused by their own soldiers. Just last month, a battle in Kunar province on Afghanistan's eastern border with Pakistan generated conflicting accounts. According to villagers, ISAF killed 65 civilians, including 50 women and children. According to ISAF, only insurgents were killed.

The data provided by UNAMA do show far more casualties than those from ISAF. For 2009 and 2010, its data include 5191 civilian deaths, over 70% of them caused by “antigovernment elements,” 20% by “pro-government forces,” and the rest undetermined. Compared with CIVCAS, they attribute nearly three times the number of civilian deaths caused by military forces, only a small portion of which are Afghan national rather than ISAF forces. One of the most significant discrepancies comes from the 529 civilians that UNAMA claims were killed by “air attacks” in 2009 and 2010. CIVCAS shows only 136 civilians killed by jets and helicopters over that period.

Asked whether casualty reporting by soldiers might be biased, British Army Lt. Col. George Wilson, who oversees ISAF Consequence Management, demurs. “No, I think the converse,” he says. “I genuinely think we get honest reporting from the ground.” According to Wilson and other ISAF sources, the cause of the disparity in body counts is methodology.

“Raw numbers will never be the same, and there's a good reason for that,” Smith says. “We do not have a presence in all 34 provinces,” he says, and therefore CIVCAS does not track all casualties. “We only count that which we see.” In some cases, such as alleged casualties from air strikes, “we can do a tremendous amount of forensics, … [but] seldom do we see the actual bodies. Sometimes we have access to someone who was wounded, but not always,” he says. “You can do overflights of cemeteries to determine any fresh grave sites and confirm or deny the numbers involved.” He adds that other organizations trying to track civilian casualties face the same challenges.

Smith does not question the accuracy of UNAMA's body count. “The UN has a much broader mandate” to track civilian casualties, he says, “and the resources to do that.” Rather than creating a definitive record for history, the purpose of CIVCAS, he says, is “marking progress.”

A safer force

Over the past month, Sutherland, Spagat, Johnson, and other experts have analyzed CIVCAS and the other civilian casualty data sets at Science's request. We have also built a timeline of the past 2 years of the war that compares those data with the casualties reported in the media, available at http://scim.ag/afghandata.

By all accounts, the war has grown deadlier for Afghan civilians. The CIVCAS data show a 19% increase in the total number of civilians killed in 2010 compared with the year before, and the UN data show a 15% jump. But at the same time, there are signs that ISAF has become a safer fighting force, treading more lightly on local populations.

Although the overall death toll in Afghanistan has risen, the increase was not wrought by soldiers. Over 90% of last year's spike in CIVCAS, and the entirety of that in the UN data, is attributed to insurgents. IED explosions have continued to cause the majority of civilian deaths. By contrast, 2010 saw a 26% drop in the number of civilians killed by soldiers in the UN data. In their own data, ISAF admitted to killing 12% more civilians compared with the year before, while wounding 20% fewer. This happened in the context of the largest military offensive in years and a surge that doubled the number of troops in Afghanistan to 140,000 last year. The lack of a corresponding spike in military-caused casualties is surprising.

This is especially true of air strikes, by far the most dangerous military activity for civilians. In the data provided by the UN, air strikes killed 171 civilians in 2010, half as many as in 2009. CIVCAS, which separately defines casualties caused by jets and helicopters, shows 11% fewer deaths. There are also subtler signs of progress in the data.

In April last year, ISAF units received a new directive concerning “escalation of force”: the shooting of civilians due to communication breakdowns. Most of these incidents occur at the hundreds of checkpoints that ISAF operates across the country. In the course of analyzing their casualty data, ISAF commanders noticed a trend. “What became very clear to me is that all the [civilian] fatalities occurred between the 100-meter point and the 0-meter point” approaching a checkpoint, Wilson says. Beyond that range, a flare usually suffices to warn drivers to slow down. But if a vehicle has not slowed down yet, Wilson says, “this is where as a soldier you suddenly think, ‘I'm about to die because this vehicle is going to drive in here and detonate.’” The problem, he says, is that soldiers in that situation had no nonlethal options.

The new directive gave soldiers more options for warning drivers at a distance, Wilson says. These include laser dazzlers, paint ball guns, and even chalk bullets. “If you fire them at a vehicle,” he says, “they will ping off and make such a loud noise that, if they're a genuinely innocent person, they'll get the message.” The CIVCAS data put numbers to that narrative. Deaths due to escalation of force dropped by 50% in the 8 months after the April 2010 directive went out, compared with the same 8 months in 2009.

While they applaud the release of these data sets, researchers are grappling with their limitations. “One problem is that organizations are all using different definitions in their data,” Johnson says. For example, whereas the UN data have separate categories for casualties caused by “mortar and rocket fire” and “shooting,” ISAF pools all ground-based battle casualties into “direct fire” and “indirect fire.” The different organizations also divide the country along slightly different lines for coding the location of casualties. “They should be trading notes,” Johnson says, “at least so their data can be easily compared.”

Such limitations would be solved if all data for civilian casualties were released at the level of individual events rather than aggregated monthly. “Ultimately, that is the only way you can verify them,” says IBC's Sloboda. The data provided to Science by ARM come closest to this level of resolution, but so far, they only cover the first half of last year.

“Counting seems like such a simple thing,” Sutherland says. “But it is the only way to see the actual effect of the war.”

Subjects

Navigate This Article