PEEIs Quality Report December 08 PEEIS QUARTERLY QUALITY REPORT 2 ND QUARTER 2008 Author: Gianluigi Mazzi Eurostat unit D1 Key Indicators for European Policies gianluigi.mazzi@ec.europa.eu 1
CONTENTS Introduction......3 Section 1: Definition of Quality Dimensions and Measures...4 Se ction 2: Quality Indicators for the Euro area PEEIs......10 Tab 1.1 Synthesis of quality measurements euro area (final)...10 1.1 Availability and Length of the Euro area PEEIs... 11 1.2 Accuracy and reliability of the Euro area PEEIs...13 1.3 Timeliness......14 1.4 Accessibility of PEEIs for the Euro area...16 Section 3: Quality Indicators for the PEEIs...17 2.1 Industrial Production Index...17 Glossary...23 2
INTRODUCTION Eurostat s mission is to provide the European Union with a high-quality statistical information service. Accordingly, quality considerations play a central role in statistical collection, compilation, analysis, and dissemination. The development of an improvement policy for short-term economic statistics has been driven mainly by a specific quality concern: timeliness of EU/Euro area aggregates was considered to be largely insufficient, most notably in comparison with indicators for the US. This made the Statistical Programme Committee (SPC) adopt the list of Principal European Economic Indicators (PEEIs), in conjunction with an improvement programme setting clear objectives with milestones. The purpose of this Report is to describe and produce some quantitative quality measures to be applied to the PEEIs, which can assist in the regular monitoring of quality features and improvements in PEEIs data. In other words, the purpose is to present a framework for data quality assessment over time, as well producing quality measures of PEEIs. The results should help data producers to better explain the quality of their data, and help users to better understand the quality issues and evaluate the impact of using the PEEIs dataset in their analysis. Even if a regular production of quality indicators is needed to reveal trends and the impact of specific events, the target here is not to fix a unique set of measures, but to identify a structure for the qualitative assessment of PEEIs that could be enhanced both for identifying the various aspects of data quality and for responding to series specificities in fact, PEEIs cover data from different statistical domains and with different characteristics. However, the quality measures have been selected according to the dimensions identified in Eurostat s definition of quality 1, definitions which are wellestablished and easy to interpret. In addition, the measures are designed to be well-matched with the IMF s Data Quality Assessment Framework (DQAF). This report has been structured as follows: Section 1 defines the quality measures, by drawing up the classes applicable to each quality dimension. Section 2 presents the results obtained by applying the measures to the PEEIs data for the euro area included in the Euro-IND historical database. Section 3 applies the measures to both the national data and European aggregates in the Euro-IND historical database for specific series in this issue the industrial production index (IPI) is analysed. 1 Eurostat (2003a), Definition of quality in statistics, Document n Eurostat/A4/ Quality/03/General/Definition. Eurostat (2003b), Standard Quality Report, Document n Eurostat/A4/ Quality/03/General/Standard Report. 3
SECTION 1: DEFINITION OF QUALITY DIMENSIONS AND MEASURES The quantitative measures are calculated from the information drawn from the historical Euro-IND database, which contains the extractions carried out on a daily basis (Monday to Friday excluding holidays) from the data stored in the NewCronos database of Eurostat. Data are available from the beginning of 2001. The database has been cleaned or corrected for errors which may occur in daily updating procedures. Eurostat Unit D1 is in charge of storing on a daily basis the statistical information contained in the Euro-IND database for a very wide set of short-term statistics, including PEEIs. All data are extracted from the Euro-IND database. The concept of data quality takes on many aspects but could be essentially reduced to users satisfaction: what users basically need is statistics available on a timely basis as well as reliable and accurate; documentation makes their use easier. As already mentioned, the PEEIs comprise data with different characteristics which have been collected in consideration of their significance for economic analysi s. When defining the scoreboard for monitoring the quality of PEEIs, a major concern has been to select measures able to take in the dimensions of quality applicable to all PEEIs statistics. The list of indicators to be considered has been bound by data characteristics and the aim to define a common set; of course for each PEEI this list could be extended by further investigations of the real problems in the statistical production processes. In fact, the quantitative quality measures have been defined in a way suitable for comparing countries data and series. The quality dimensions identified in Eurostat s 2 and the IMF s DQAF serve as a framework to this report. These are: Relevance is the degree to which statistics meet current and potential users needs ; in particular in this report we refer to the completeness in terms of users expectations. Completeness is the extent to which all statistics that are needed are available. Accuracy in the general statistical sense denotes the closeness of computations or estimates to the exact or true values. Timeliness of information reflects the length of time between its availability and the event or phenomenon it describes. Clarity refers to the data s information environment whether data are accompanied by appropriate metadata, illustrations such as graphs and maps, whether information on their quality is also available (including limitation in use ) and the extent to which additional assistance is provided by the NSI. On the basis of the quality dimenstions, the following measures have been defined and used in this Report: 2 Eurostat 2005 "Assessment of quality in statistics"doc.eurostat/a4/quality/03/ Glossary. 4
PEEIs Quality Report December 08 Quality measure Blue cells Not filled cells Yellow cells 1. Availability The indicator is available in seasonally-adjusted form 2. Coverage Length of the indicator satisfying the target of 5, 10, 15 years The indicator is not available in seasonallyadjusted form Length of the indicator less than the target of 5, 10, 15 years 3. Number of observations Number of observations (1 year period) 4. Number of revised observations Number of different observations per period (1 year) 5. Maximum revisions Maximum difference between the lowest and highest value per period (1 year) 6. Coefficient of variation Countries with minimum variability of revisions (on 1- year period) 7. Timeliness (average) Average timeliness 8. Timeliness (last release) Countries with increased timeliness in the last release with respect to the average Countries with reduced timeliness in the last release with respect to the average 9. Accessibility Metadata are available Metadata are not available 9
SECTION 2: QUALITY INDICATORS FOR THE EURO AREA PEEIS Tab 1.1 Synthesis of quality measurements euro area (final) COUNTRY AVAIL. COVERAGE (AVERAGE) WITH RESPECT TO TARGET OF 15 Y 10 Y 5 Y N. OF OBS. MAX. N. OF REV. O BS. MAX. REVISION COEFF. OF DELAY BETWEEN REF. PERIOD AND FIRST DAY STORE IN VARIATION DATA-BASE AVG. AVG. LAST GDP in volume 53 53 115426 0.408 45 45 Pr. final cons. in 53 52 74300.2 0.762 45 45 Investments in 53 52 18389.8 0.505 45 45 External trade 108 107 1674.9 14.465 48 46 Current account- 112 108 9253 7.249 72 72 Inflation items) 149 2 0.01 0.000 17 15 Unempl. rate tot 184 106 0.6 0.154 31 30 Unempl. rate - 15-184 167 1.5 0.417 31 30 RELEASE Unempl. rate - + 24 184 176 0.7 0.350 31 30 Labour Cost Index 49 42 0.7 0.137 74 74 Employment 53 51 588 0.087 75 74 Ind. producer 328 18 0.12 0.000 34 34 Ind. production 220 191 1.36 0.008 43 44 Industrial new 160 154 3.49 0.030 53 52 Construction 220 217 10.8 0.044 49 49 Retail trade 160 148 2.68 0.013 35 35 Government 13 9 17988.8 1.599 109 109 Gen. Gov.gross debt 13 4 9102.6 0.022 109 109 Econ. Sentiment 226 1 0.01 0.000 0 0 3 months Interest 282 115 0.7 0.056 : 8 Long t. Gov. bond 221 39 0.13 0.019 : 11 Euro-dollar 82 0 0 0.000 na na AVAIL. OF METADATA 10
1.1 AVAILABILITY AND LENGTH OF THE EURO AREA PEEIS Behind any consideration about quality dimensions, the most immediate need of users is the availability of series and especially the availability of long series. In this section we present the availability of PEEIs and regard as a benchmark the availability of at least 15 years, by considering this period long enough for normally including two business cycles. Figure 1.1: Length and starting date of PEEIs for the euro area (final) ESI 3-m. Interest rate Long-t. gov. bond yields Construction prod. Ind.new orders Empl. Gen. gov. gross debt Unempl. - Total Unempl.- 15-24 Unempl.- +24 years GDP in volume Private cons. Investments Inflation Labour Cost Index External trade bal. Current account Gov. deficit/surplus Ind. producer prices Ret.trade defl.turnover Industrial production /$ exchange rate (Days) 0 5 10 15 20 25 30 Start Date Economic Sentiment indicator 3 months Interest rate Long term government bond yields Construction production Industrial new orders Employment Unemployment rate - Total Unemployment rate - 15-24 years Unemployment rate - +24 years GDP in volume Private final cons. in volume Investments in volume Inflation (HICP all items) Labour Cost Index External trade balance Current account- Total Government deficit/surplus Industrial producer prices Retail trade deflated turnover Industrial production General government gross debt Euro-dollar exchange rate 1985m01 1990m01 1990m01 1990m01 1991m01 1991q02 1993m01 1993m01 1993m01 1995q02 1995q02 1995q02 1996m02 1996q02 1999m01 1999m01 2000a00 2001m01 2001m01 2001m01 1991a00 2002m01 less than 5 years between 5 and 10 years between 10 and 15 years more than 15 years 11
Table 1.1: Length of PEEIs for the euro area (final) INDICATOR NO SA OR WDA DATA AVAILABLE LENGTH LESS THAN 5 YEARS LENGTH BETWEEN 5 AND 10 YEARS LENGTH BETWEEN 10 AND 15 YEARS LENGTH GREATER THAN 15 YEARS GDP in volume Private final consumption in volume Investments in volume External trade balance Current account- Total Inflation (HICP all items) Unemployment rate Total Unemployment rate - 15-24 years Unemployment rate - + 24 years Labour Cost Index Employment Industrial producer prices Industrial production Industrial new orders Construction production Retail trade deflated turnover Government deficit/surplus General government gross debt Economic Sentiment indicator 3 months Interest rate Long term government bond yields Euro-dollar exchange rate 12
1.2 ACCURACY AND RELIABILITY OF THE EURO AREA PEEIS Another important aspect of quality is the users expectation for statistics that are accurate and reliable. However, measurement of this dimension of quality is not straightforward. In fact, users tend to consider reliable the most recently published data and, even if the idea is commonly accepted that revisions include the latest basic information and hence increase the accuracy of data, there is a complex interrelation and even a trade-off between the revisions of data and the perception of accuracy and reliability of data. Fig. 1.2: Average number of changes in the release period of the PEEIs for the euro area (final) External trade balance Construction production Industrial new orders Investments in volume Pr. final cons. in volume Employment Retail trade deflated turnover Unempl. rate - 15-24 years Ind. production Labour Cost Index GDP in volume Unempl. rate tot Current account- Total Unempl. rate - + 24 years 3 months Interest rate Gen. Gov.gross debt Long t. Gov. bond yields Government deficit/surplus Ind. producer prices Inflation items) Econ. Sentiment Ind. Euro-dollar exchange rate Average variation rate 14.465 0.044 0.030 0.762 0.505 0.087 0.013 0.417 0.008 0.137 0.408 0.154 7.249 0.350 0.056 0.022 0.019 1.599 0.000 0.000 0.000 0.000 0 20 40 60 80 100 13
1.3 TIMELINESS A key element in the users perception of the quality of data is timeliness: data need to be available on a timely basis. Measurement of timeliness is fairly complex since it relies on the definition of timely basis and, for any user, this concept corresponds to as soon as possible. Nevertheless, for quality measurement we have to set a benchmark and a definition of timeliness. In this report we opted for defining it as the time lag between the reference time point and the time for publication of the statistics. In particular with reference to the PEEIs, we have considered the period of time between the end of the reference period and the date of appearance in the EuroIND database as well as that of publication as indicated in the GES calendar. When taking into account not only the latest release but a longer period (2007 onwards), we indicate the results of the latest release with respect to the average timeliness over the reference period. The average timeliness has been calculated on the basis of the GES calendar. (Days) 120 110 100 90 80 70 60 50 40 30 20 10 0 Last release Average Fig.1.3: Average timeliness of PEEIs for the euro area (final) 14
Fig.1.4: Timeliness of PEEIs for the euro area (final) Long term government bond yields Inflation (HICP all items) Unemployment rate - above 24 years Unemployment rate - 15-24 years Unemployment rate - Total Industrial producer prices Retail trade deflated turnover 3 months Interest rate Industrial production Investments in volume Private final consumption in volume GDP in volume External trade balance Construction production Industrial new orders Current account- Total Labour Cost Index Employment General government gross debt Government deficit/surplus 109 109 0 20 40 60 80 100 120 (Days) PEEIs target na 17 30 30 30 35 28 na 40 90 90 45 46 45 50 90 70 45 90 90 15
1.4 ACCESSIBILITY OF PEEIS FOR THE EURO AREA As indicator of accessibility we have indicated the availability of metadata, in particular whether metadata is available in the base page of the Eurostat site. Tab.1.2 Availability of metadata (final) INDICATOR EUROSTAT METADATA BASE PAGE SUMMARY METHOD. GDP Private final consumption Investments External trade balance Current account Inflation Unemployment rate Labour Cost Index Employment Industrial producer prices Industrial production Industrial new orders Construction production Retail trade deflated turnover Government deficit/surplus General government gross debt Economic Sentiment indicator 3 months Interest rate Long term gov. bond yields 16
Euro-dollar exchange rate SECTION 3: QUALITY INDICATORS FOR THE PEEIS 2.1 INDUSTRIAL PRODUCTION INDEX The industrial production index shows the output and activity of the industry sector. It measures changes in the volume of output on a monthly basis. Data are compiled according to the Statistical classification of economic activities in the European Community, (NACE Rev.1.1, Eurostat 1996). Industrial production is compiled as a "fixed base year Laspeyres type volume-index". The current base year is 2000 (Index 2000=100). The index is presented in calendar and seasonally adjusted form. Growth rates with respect to the previous month (M/M-1) are calculated from calendar and seasonally adjusted figures while growth rates with respect to the same month of the previous year (M/M-12) are calculated from calendar adjusted figures. Tab 2.1.1 Synthesis of quality measurements Industrial Production Index (final) COUNTRY AVAIL. COVERAGE (AVERAGE) WITH 15 Y RESPECT TO TARGET OF 10 Y 5 Y N. OF OBS. AVG. N. OF REV. OBS. MAX. MAX. REVISION AVG. VARIATION RATE AVG. DELAY BETWEEN REF. PER. AND FIRST DAY STORE IN DATA-BASE EU-27 220 193 0.99 0.005 43 44 EA 220 191 1.36 0.008 43 44 BE 216 130 1.65 0.029 42 54 BG 100 95 6.83 0.115 40 41 CZ 124 119 2.81 0.111 43 42 DK 280 36 4.1 0.035 37 37 DE 340 136 1.1 0.013 38 37 EE 124 119 9.9 0.246 32 30 EL 160 148 1.11 0.045 40 39 ES 340 287 0.75 0.014 36 36 FR 220 153 0.7 0.033 40 41 IE 340 70 8.9 0.059 47 39 IT 220 95 1.2 0.032 41 41 CY 112 108 8.07 0.266 54 53 LV 148 130 12.51 0.159 36 34 LT 124 116 30.5 0.124 22 22 LU 340 328 13.2 0.085 68 63 HU 123 62 2 0.058 43 36 MT NL 172 145 4.6 0.115 40 37 AT 100 71 5.8 0.142 56 55 PL 161 145 2.3 0.039 21 21 PT 160 69 4.2 0.030 30 30 RO 220 204 5.62 0.087 37 37 SI 124 100 2.1 0.106 42 41 SK 124 113 5.7 0.319 39 40 FI 220 186 12.1 0.108 30 29 SE 220 167 1.7 0.025 39 41 AVG. LAST RELEASE AVAIL. OF METADATA 17
UK 340 26 0.48 0.004 38 37 18
Figure2.1.2 Industrial Production Index - Length and starting date of PEEIs for the euro area (final) 0 5 10 15 20 Starting Date 25 30 (Days) DK 1985m01 DK EL 1990m01 EL FR 1990m01 FR LT 1990m01 LT HU 1991m01 HU SE 1991q02 SE CZ 1993m01 CZ EU27 1993m01 EA EU27 1993m01 BE 1995q02 EA ES 1995q02 BE IE 1995q02 ES PT 1996m02 IE SK 1996q02 PT FI 1999m01 SK UK 1999m01 FI DE 2000a00 UK EE 2001m01 DE AT 2001m01 EE PL 2001m01 AT NL 1991a00 PL BG 2002m01 NL LV 2002m02 LU BG 2002m03 RO 2002m04 LV SI 2002m05 LU IT 2002m06 RO CY 2002m07 SI IT less than 5 CY years between 5 and 10 years between 10 and 15 years more than 15 years 19
Figure 2.1.2 Average number of changes in the release period of the PEEIs for the euro area (final) LU ES GR CZ BE LV CY SK BG EE SI FR FI IT RO HU AT LT EU27 DK DE SE NL PL IE PT UK (Days) 0 50 100 150 200 250 LU ES GR CZ BE LV CY SK BG EE SI FR FI IT RO HU AT LT EU27 DK DE SE Average Coefficient of variation 0.085 0.014 0.045 0.111 0.029 0.159 0.266 0.319 0.115 0.246 0.106 0.033 0.108 0.032 0.087 0.058 0.142 0.124 0.005 0.035 0.013 0.025 NL 0.115 PL 0.039 IE 0.059 PT 0.030 UK 0.004 20
21
Figure 2.1.3 Timeliness (final) LU AT CY IE EA CZ HU SI BE IT FR GR SE NL BG SK UK DE DK RO LV ES EE PT FI LT PL (Days) 0 10 20 30 40 50 60 70 80 22
GLOSSARY Timeliness: Punctuality of time schedule of effective publication, and Average time between the end of the reference period and the date of the first results, Computations carried out on releases from January 2007 to June 2008. Revisions: Changes based upon additional or improved information. Metadata: Data that describes information about either online or offline data. Information that characterizes the who, what, where, and how related to data collection. Currently the format used is the SDDS format of IMF. PEEIs: Eurostat compiles European Union and Euro area infra-annual economic statistics relevant for short-term economic analysis. Among these, a list of 19 indicators, called Principal European Economic Indicators (PEEIs), has been identified by the key users as of prime importance for the conduction of monetary and economic policy of Euro zone. These indicators are mainly released through the Euroindicators website: http://ec.europa.eu/eurostat/euroindicators. PEEIs target: the PEEIs target concern the decrease of the delay between the releases of the different PEEIs. (According to COM (2002) 661 final.) Seasonally-adjusted and working-day adjusted series: Seasonally-adjusted time series are series that have been adjusted for seasonal variations, including trading-day (working-day) effects and other regular calendar variations if present. Reference period: The period of time for which data are collected IMF Data Quality Assessment Framework (DQAF): The generic DQAF current July 2003 version was introduced at the Fifth Review of the Fund's Data Standards Initiatives. The DQAF identifies qualityrelated features of governance of statistical systems, statistical processes, and statistical products. It is rooted in the UN Fundamental Principles of Official Statistics and grew out of the Special Data Dissemination Standard (SDDS) and General Data Dissemination System (GDDS), the IMF s initiatives on data dissemination: http://dsbb.imf.org/applications/web/dqrs/dqrsdqaf. Eurostat s definition of quality: With the adoption of the European Statistics Code of Practice, Eurostat and the statistical authorities of the EU Member States have committed themselves to an encompassing approach towards high quality statistics. It builds upon a common European Statistical System definition of quality in statistics and targets all relevant areas from the institutional environment, the statistical production processes to our output: European official statistics. More information available on the Eurostat website at: http://ec.europa.eu/eurostat Number of observations: number of observations in the last version of the series. Number of revised observations: number of revised observations in the version of the series. Maximum revision: maximum revision of an observation (between 2 successive versions), in absolute value. 23
Variation rate: variation rate of an observation between 2 successive versions, e.g. 2007m01 between the versions of 1st and 15 th October 2008, in absolute value Period: from 1 st January 2007 to 30 th June 2008. 24