The real science behind CR's chicken tests

When we planned our recent tests for bacteria in chickens, we had no reason to suspect that food safety would be in the news as our report appeared this month. Unfortunately, a spate of recent events, including the Taco Bell E. coli outbreak, has kept the issue in the headlines. As "Fast Food Nation" author Eric Schlosser said in The New York Times earlier this week, "our food can be much safer than it is right now."

Consumer Reports' tests found that eight out of every 10 chickens tested harbored dangerous bacteria (campylobacter and/or salmonella) that sicken millions every year. While the U.S. Department of Agriculture (USDA) routinely tests for salmonella,  it does not test for campylobacter. Commenting on CR's study and its implications, a USDA spokesperson's response to a Reuters reporter was disappointing, coming from the agency with primary responsibility for meat safety. The spokesperson dismissed our findings as “junk science” and went on to say that “there is virtually nothing or any conclusion that anyone could draw from 500 samples.”

What the USDA’s comments fail to recognize is the careful statistical design that we use in studies like this to ensure to the greatest extent possible that our results accurately characterize the products that consumers are purchasing across the country. For this study, we used a national product retrieval service to help us purchase the 525 chickens at retail outlets nationwide. This included creating a randomized “purchasing design” so that shoppers could be dispatched in the 23 states we surveyed. Each shopper was given a mission: Anonymously purchase specified brands of chicken on specific days with a maximum of two birds per brand to be purchased during each shopping visit (The shopping was conducted over a 6-week period). After testing was completed, statistical analyses were done to ensure that there were no biases influencing the results, such as the potential effects of any particular region or store or day of the week that the samples were purchased. Studies appropriately labeled “junk science” typically do not use such standard statistical methods.

As a basis for their attack on CR’s findings, government and industry critics point to a recent study done by researchers from the National Chicken Council and the USDA Agricultural Research Service.  This joint industry-government study, which had several serious flaws in its design, showed a 26% rate of campylobacter contamination. Tested birds came from 13 processing plants that had volunteered to participate in the study -- plants that could potentially be cleaner than those that did not volunteer. The chickens were taken in batches off the processing line, a method that would miss any contamination occurring further down the line, and therefore the sample did  not fully represent the quality of chickens bought at retail. Further, the study used a less sensitive method to detect campylobacter, which would be expected to find fewer contaminated samples than our more sensitive test.

Perhaps the most interesting footnote to this back and forth is that throughout our long history of testing chickens for these pathogens, our results have been reasonably consistent with results of USDA’s regular monitoring for salmonella and with their baseline tests for campylobacter. The USDA reported a 15 percent incidence for salmonella in the second half of 2005, and between 11 and 12 percent in the first three quarters of 2006, virtually identical to our own results this year, which found a 15 percent incidence. The USDA baseline campylobacter study in 1994 and 1995 found a contaminant incidence of 88 percent for that pathogen, which was higher than the 63 percent we reported in 1998, but much closer to the 81 percent we reported this week.

It’s critical that in all of this, however, we don’t engage so deeply in this argument that we lose sight of our ultimate goal of improving the safety of the chicken in the marketplace. Consumers Union has long urged the USDA to develop a standard for campylobacter, so that both the industry and government can be made fully accountable. And on a broader level, we support legislation that would establish a single food-safety agency that would require regular inspections and have the power to recall contaminated food — a power that the USDA now lacks.

And finally, the bottom line for consumers is not to chicken out from eating an important source of protein, but rather to practice vigilance in the kitchen by cooking their chicken thoroughly and exercising good hygiene to avoid cross contamination.