Routine immunization reporting and data quality

Transcription

1 Routine immunization reporting and data quality 4th Annual Global Immunization Meeting February, 2009 Marta Gacic-Dobo, WHO/IVB/EPI 1

2 lobal DTP3 Immunization coverage (81% in 2007) % coverage rce: WHO/UNICEF coverage estimates, , as of August 2008, 193 WHO Member States. 2

3 Measuring immunization coverage Administrative method Surveys Immunization Specific survey Expanded Programme on Immunization (EPI) 30 cluster survey Multi indicator household surveys UNICEF Multiple Indicator Cluster Survey (MICS) Demographic and Health Survey (DHS) 3

4 Administrative coverage data Limitations: Number of doses administered through routine services - Inclusion of vaccination conducted - outside the target group - during campaigns - Incomplete reporting - Transcription or calculation errors Number of population in target group - Projected based on old census data - Incomplete reporting - Different sources used at different administrative levels (HF, districts, nationa 4

5 Survey coverage data Limitations: Number of children in the sample vaccinated - Immunization card availability - Reliance on recall - Interviewer interaction - Length or complexity of the questionnaire may compromise the accuracy of the response Number of children in the sample - Representativeness of sample - Availability of sampling frame - Household/child selection 5

6 Advantages and disadvantages of administrative and survey methods Administrative method Advantages: Based on data necessary for service provision Timely management monitoring tool Provides data at local level Disadvantage: May not include private sector Surveys Advantages: Estimate of immunization coverage can be obtained if the denominator is unknown Vaccinations given by the private sector reflected. Provides additional information on social economical status of reached and unreached children Disadvantage: Provide information on the previous birth year s cohort Cost Results may not be available at local level 6

7 Official Government Estimate Opportunity for countries to correct known limitations Private sector Denominator issue Incomplete reporting Use survey data as an official coverage estimate Reported in WHO publications and web sites as "reported estimates" 7

8 elation between DTP3 administrative coverage & government official estimates, administrative (%) % 40government official estimate < administrative coverage 20 N=1354 data points 75% government official estimate = administrative coverage 14% government official estimate > administrative coverage In countries made at least 5% adjustment government official estimate (%) 8

9 omparison of DTP3 coverage from officia overnment reports and surveys, survey coverage (%) N=499 data points 18% reported 100 coverage < survey coverage reported coverage (%) MICS DHS EPI Cluster other 30% reported coverage > survey coverage 9

10 omparison of DTP3 coverage from officia overnment reports and surveys, survey coverage (%) N=135 data points 11% reported 100 coverage < survey coverage reported coverage (%) MICS DHS EPI Cluster other 39% reported coverage > survey coverage 10

11 WHO/UNICEF coverage estimates Annual review of coverage data National reports to WHO & UNICEF - JRF Administrative coverage data Country official estimates Published and grey literature DHS, MICS, other surveys Additional information Stock out information Data quality audit results UNICEF supply division data Expert opinion/local knowledge (consultation with RO/country focal point...) WHO/UNICEF estimates of national immunization coverage 11

12 Estimation rules Estimate = reported data if reported data are: Consistent with quality survey results (+/- 10% points) Consistent across years (no sudden, unexplained changes). Consistent between vaccines (DTP3 = OPV3). No other data are available. Vaccination coverage 100% not achievable If data are inconsistent select most "likely" value. Include private sector where data are available Update previous estimates if new data becomes available 12

13 Country profile of immunization coverage Vaccine shortage due to frozen vaccines Card retention rate of 38% ttp:// ttp:// 13

14 Relation between reported and estimated DTP3 coverage, 1990, 2000, and 2007 number of countries 14% 82% 76% 73% WHO/UNICEF goal is to have all reported data and coverage estimates to be the same 10% 11% 16% 15% 3% reported coverage raised reported coverage lowered reported = estimated coverage 14

15 What could be done to improve immunization data quality? Improve the quality of the empirical data Countries to validate and improve routine monitoring systems; both numerators and denominators Surveys or data quality self-assessments (DQS) Regular use of data to monitor and improve programme performance Increase efforts to capture private sector delivery of immunization. Improve the WHO/UNICEF estimation process Widen consultation process More objective & replicable (Formal knowledge representation and reasoning) Develop uncertainty ranges around estimates 15

16 Thank You

17 lobal DTP3 coverage estimates from different sources, reported: 90% ; WHO/UNICEF: 81%; IHME: 74% coverage (%) s Official coverage WHO/UNICEF estimated coverage IHME estimates (data points estmimated from IHME graph) 17

18 Immunization coverage data collection JRF (started 1998) WHO/UNICEF Joint Reporting Form on immunization for the period January-December, YYYY Annual process Sent to all Member States Filled in by MoH Core set of question with dditional regional modifications Global level (JRF) 18

19 Survey types Immunization Specific survey Expanded Programme on Immunization (EPI) 30 cluster survey designed specifically for measuring immunization coverage simple to administer and easy to conduct frequently conducted by national EPI staff have a precision plus or minus 10% points at 50% coverage reports and datasets often difficult to obtain Multi indicator household surveys UNICEF Multiple Indicator Cluster Survey (MICS) Demographic and Health Survey (DHS) more extensive surveys covering a variety of indicators, have a more rigorous design, higher degree of precision Survey reports and datasets are available in public domain more expensive, logistically more complex and the questionnaire is longer and more difficult to administer. 19