The Food Safety and Inspection Service says that the Hazard Analysis and Critical Control Point system is the foundation of its “strategic, data-driven inspection program,” but scheduling of HACCP verification sampling sometimes drives the Salmonella data. A limited amount of quarterly Salmonella information is available for analysis, but it shows the limitations of the Salmonella prevalence data that the service uses to justify policy.
Over the 14 years of quarterly HACCP Salmonella data for chickens (1998 to 2001 combined, 2002 to 2004 combined, and annual data for 2005 to 2011), the second quarter had the lowest average Salmonella prevalence. Both high and low yearly Salmonella percentages occurred in every quarter at least once, so despite well-known trends, it was impossible to know in advance when highs and lows would occur. The average high-to-low ratio was about 1.5 (if the low quarter had a prevalence of 10 percent, the high quarter had a prevalence of about 15 percent), so there were substantial differences between quarters. The annual prevalence figures were calculated as the total number of positive samples as a percentage of all samples that were tested for Salmonella during the year.
The percentage of Salmonella-positive samples in each quarter varied as follows:
- 6.7 percent to 18.3 percent for January to March
- 5.2 percent to 19.7 percent for April to June
- 7.1 percent to 15.1 percent for July to September
- 4.3 percent to 15.3 percent for October to December
Most frequently sampled quarters
The most frequently sampled quarter was always the second or third of the calendar year, with every quarter represented at least once in the least-sampled category. The most-sampled quarter averaged about 3.3 mores samples than the least-sampled quarter over the 14 years, a substantial departure from the expected pattern if sampling were scheduled more randomly. In other words, Salmonella prevalence in the most sampled quarter was weighted 3.3 times more in the annual average than Salmonella prevalence in the least sampled quarter.
The percentage of annual samples taken in each quarter over the 14 years varied as follows:
- 11.6 percent to 34.6 percent for January to March
- 10.9 percent to 49.3 percent for April to June
- 9.6 percent to 45.3 percent for July to September
- 9.0 percent to 31.4 percent for October to December
Actual variation is underestimated here because only combined numbers are available for 1998 to 2001 and 2002 to 2004. To the degree that variations in Salmonella prevalence interacted with the variations in numbers of samples taken, the numbers presented by the Food Safety and Inspection Service as estimates of annual Salmonella prevalence in chicken HACCP are not reliable enough to justify comparison of prevalence in different years or attempted correlation with human cases of salmonellosis.
Seasonal variation not accounted for in prevalence estimates
Figures 1, 2 and 3 show the quarterly percentages of Salmonella-positive samples on the left side and the number of annual samples per quarter on the right side. Figure 1 shows cumulative numbers from the beginning of HACCP in 1998 to the end of 2001. The pattern of Salmonella positives shows the known seasonal variation in carriage by chickens in the U.S., with lower percentages usually earlier in the year and higher percentages in late summer and early fall. More than 81 percent of the HACCP samples were taken from January to June in that four-year period. Chicken production and Salmonella prevalence are year-round occurrences, so the resulting annual figures for Salmonella prevalence reported by the Food Safety and Inspection Service are heavily weighted toward the most frequently sampled parts of the year, with chickens sampled 5.2 times more frequently in April to June than in October to December. The resulting numbers are not an accurate estimate of Salmonella occurrence in chickens throughout the year.
Figures 2 and 3 show quarterly Salmonella prevalence and sampling numbers for 2006 and 2010, with slightly different patterns for Salmonella prevalence and quite different sampling patterns compared to 1998-2001. About 71 percent of HACCP samples were taken in the second half of 2006, with 3.7 times more samples taken in July to September than in January to March. By chance, the Salmonella prevalence was similar in the most frequently and least frequently sampled quarters in 2006, but a scientific sampling plan would not allow luck to influence whether annual prevalence was determined accurately. In 2010, about 72 percent of HACCP samples were taken in the six months from April to September, with 4.3 times more samples taken in April to June than in January to March. This sampling schedule does not seem likely to generate an accurate estimate of Salmonella prevalence for any sampling year.
HACCP verification samples not valid estimates of national prevalence
The Food Safety and Inspection Service has listed the annual numbers for Salmonella prevalence in chicken HACCP samples for 2002, 2003 and 2004 as 11.5 percent, 12.8 percent and 13.5 percent, respectively, a sequence described by the service as an increasing trend. The differences in sampling patterns and seasonal variation in Salmonella shown in HACCP data mean that the service cannot reliably estimate Salmonella prevalence in poultry within 1.3 percent or 0.7 percent, the differences between those years. The prevalence estimates for those years should not have been described as a trend. The Food Safety and Inspection Service has stated many times that HACCP Salmonella verification samples are not valid estimates of national Salmonella prevalence, but it and others frequently use the annual HACCP prevalence numbers for exactly that purpose. Besides the national statistical picture, Salmonella testing has regulatory consequences for the poultry industry. The probability that a plant will pass a test set and the inspection category the plant is placed in will be influenced by the seasonal variation in Salmonella during the time that HACCP samples are being taken and any weighting that is applied if an unusual percentage of samples is taken in one part of the year.
Salmonella verification data for the last 14 years shows that the Food Safety and Inspection Service was correct in warning against using HACCP sampling results as an indicator of national Salmonella prevalence or trends. Unfortunately, the service itself has not always followed that advice.