Issue 15 - September 2003 : CDC’S FRAMEWORK FOR EVALUATING SYNDROMIC SURVEILLANCE SYSTEMS

___________________________________________________________________________________________________

CDC’S FRAMEWORK FOR EVALUATING SYNDROMIC SURVEILLANCE SYSTEMS ______________________________________________________________________________________

The Centers for Disease Control and Prevention presented a draft framework for evaluating syndromic surveillance systems at the 1st annual National Syndromic Surveillance Conference in New York City in October 2002. Originally drafted by the CDC’s Dr. Daniel Sosin, it was presented as a work in progress, not to be cited or distributed.

In preparation for the 2nd National Syndromic Surveillance Conference in NYC October 23-24, 2003, the draft was updated with input from several known experts in the field of syndromic surveillance, and placed on the web. (1) [Technical note - view and print this document with Microsoft Word only]. Dr Sosin invites discussion and debate of the concepts presented in his Framework. He plans formal publication in the Spring 2004.

The Framework begins with the definition of syndromic surveillance. It emphasizes: timely and complete data collection through automated electronic data interchange; refined signal detection to recognized outbreaks (alarms); and capture of data about people and disease that might not have been possible with more traditional methods. (1)

Using this definition as a basis for understanding what syndromic surveillance is about, the Framework then describes a comprehensive outline which can be used to evaluate any system. The four basic components to the evaluation include: System Description, Outbreak Detection, Experience, and Conclusions and Recommendations. Here’s a brief description of each category.

System Description…(1)

1) What is the purpose of the system? Is the purpose long-term monitoring, or just for short-term, high risk situations (e.g. the Olympic Games)?

2) Who are the stakeholders for this system? The stakeholders are all the people who would have to cooperate for it to work, and all the people who would want to see the data that is produced.

3) How does the system work? Everyone should understand the complexity and resources needed to operate the system.

Outbreak Detection…(1)

1) How early in the outbreak or disease process can the event be detected? That is, what is the overall speed from exposure to the initiation of a public health response?

2) How well does the system perform in outbreak detection? This question may involve determining the stability of the background occurrence of cases, eliminating noise, and the usefulness of the statistical algorithms. Is the input data quality adequate to support valid output?

Experience…(1)

1) Is the system useful? That is, does it contribute to the early detection of outbreaks and allow for more effective intervention.

2) Is the system flexible? Can the system change as needs change?

3) Will the stakeholders accept the system? Is the time involved in participating too great? Are there privacy issues?

4) Can the system be duplicated in another setting? That is, is the system simple enough that anyone can learn it, or does it require many “person-dependent” steps?

5) Is the system stable? Would program updates, coding changes, or other necessary maintenance cause too much downtime?

6) What is the cost? All costs should be considered, including direct costs to purchase software and hardware, for running the system once purchased, and for follow up interventions indicated by the data.

Conclusions and Recommendations… (1)

1) What are the strengths and weaknesses of this system? An overall evaluation is critical when comparing multiple systems.

Certainly one of the most valuable portions of this document is the Appendix A. This “Operations Checklist” takes the theoretical framework that CDC has spent more than a year developing, and organizes it into bullet points to use when determining which syndromic surveillance system best meets your needs. There is no syndromic surveillance system today that matches the “ideal” described by this framework, and this group makes no attempt to estimate how much such a system should cost. Still, hospitals and health authorities who must make informed decisions in the near future may want to compare potential systems using this comprehensive evaluation structure.

(1) http://www.cdc.gov/epo/dphsi/files/Draft6_Framework_for_Evaluating_Syndromic_Surveillance_Systems.doc

Back to the top