Several recent scientific papers have concluded that only limited data are available to judge whether national Salmonella control programs have had an impact on human illness rates. There is no limit, however, on the supply of speculative interpretation that is available, including a common style of analysis that can be described as unscientific thinking.
A 2003 study by the usually reliable National Research Council observed, “The overall decline in salmonellosis would be even greater except for the concurrent increase in infections due to S. Newport (up 32 percent).” The fact is, however, that there always seem to be some serotypes that are increasing in human salmonellosis and others that are decreasing, so discounting the increases may not be a realistic way to judge the overall situation.
Concerning E. coli O157:H7, the same study observed, “The fact that no sustained decline has been observed yet may mean that the established zero tolerance for this pathogen does not offer protection, or perhaps it was effective and blunted what otherwise would have been an increase.” The National Research Council also found that the problem of limited data is not really a problem: “It may not be statistically valid to compare the two data sets [Baseline and HACCP]; however, because of the vast number of data sets collected, a decrease in Salmonella-positive samples can be clearly observed since the implementation of the Salmonella standard.”
Hypothesis must be falsifiable to be scientific
A hypothesis has to be falsifiable to be considered scientific; that is, there must be reasonably obtainable data that might cause rejection of the hypothesis. The possibility that HACCP for raw meat and poultry has reduced human illness may not be falsifiable because of unscientific thinking.
The Food Safety and Inspection Service said last year, “It is possible that reduction of salmonellosis due to one food product such as chicken could be negated by the increase in salmonellosis due to another product or unrelated vehicle, such as produce, thus causing the stable case rate in recent years.” The thinking on the part of FSIS seems to be that an improvement has to be there and, therefore, must be covered up by an increase in a product inspected by a different agency.
A recent peer-reviewed journal paper compared hospitalization rates of elderly patients with salmonellosis before and after implementation of HACCP. There was no overall improvement, with fewer cases in some regions, more cases in some, and no change in most. The authors observed, however, “It is possible that the hospitalization rate would have been even higher if no regulation were introduced at all.”
Scientific proof or wishful thinking?
Several journal papers that studied E. coli as an indicator of Salmonella reported non-significant differences in Salmonella prevalence when carcasses were sorted into groups based on E. coli counts, but still described E. coli as a good indicator or said that Salmonella prevalence was lower despite the result of statistical testing.
One of the early reviewers of the original HACCP rule wrote, “FSIS often expressed a ‘feeling’ that by the mere institution of regulatory measures, food safety benefits would ensue because plants would be compelled to begin microbial testing.” The Modernization of Poultry Slaughter Inspection proposal from earlier this year stated, ”FSIS’s experience with using post-chill testing for generic E. coli … has led the Agency to conclude that such testing is not the most effective way to prevent contamination from occurring,” so FSIS still has that feeling. Proven controls must be available to deal with microbiological contamination, which cannot be prevented with testing or regulations or standards in the absence of controls.
More unscientific thinking
Another example of unscientific thinking can be found in the FoodNet Annual Report for 2002. HACCP in raw meat and poultry was given credit as a contributing factor in a significant reduction in human illness caused by Salmonella typhimurium, but HACCP was not blamed for significant increases in illnesses caused by three other serotypes (Newport, Javiana and Heidelberg). HACCP was also recognized as a contributing factor in reductions in cases of Campylobacter, Listeria, and Yersinia, although HACCP samples from raw meat and poultry were not being tested for those bacteria.
HACCP has to be judged objectively on its merits, not by what we wish that it might have done. Perhaps calling attention to these examples can help put an end to unscientific thinking and lead to more realistic assessments of what HACCP has accomplished.