A new index calculation system in Finland Mervi Virtanen, Olli-Pekka Penttilä and Timo Koskimäki

Transcription

1 A new index calculation system in Finland Mervi Virtanen, Olli-Pekka Penttilä and Timo Koskimäki

2 Contents The problem Solution: The New Index System Reflections and outcome Annex: For those interested in details-how is it compiled in practice)

3 The Problem Finnish Customs foreign trade unit price and volume indices: monthly publications since the 1920 s Compiled using various techniques throughout the years Statistics Finland also publishes import- and export price indices, based on traditional price index methodology The current Customs system (as of ~year 2002): A set of CPA 4-digit unit value series as a starting point Initial selection of the CPA series based on the importance and stability of the series A set of CN8-unit value series selected to represent each CPAcategory stability criteria In price index terminology, price observation is CN-8 unit value. About per cent of trade value used for index compilation, the rest is deleted because of volatility Annually chained Laspeyres-type price index, weights always from year T-1

4 The Problem The problems of the current system Timeliness: due to manual validation, publication schedule ~t+90 days Acceptable schedule ~t+60 days, i.e. the same as for first detailed trade statistics results Target 1: improve timeliness by 30 days by making the production more automatic Quality adjustment: Heterogenity of the elementary aggregates: Quality adjustment inappropriate because elementary aggregates are unit value series Requires manual checking and deletion of volatile elementary series prior to index compilation Target 2: improve homogeniety by using more disaggregate elementary aggregates Target 3: cut down signifigantly the resources used for the system (from current 0,5 person years to 0,05 person years)

5 The Solution Basic ideas: Price observations are unit prices in categories formed as combination of CN8-class and enterprise, not an unit value for the whole CN8 as in the old system Price relatives compiled in CN8*enterprise classes Base price: average of year T-1 monthly unit prices in the CN8*enterprise category Price relative: Current price/base price Elementary aggregate compiled in CPA 4-categories as a weighted arithmetic average of acceptable price relatives, current period weights Number (and relative weight) of enterprises may vary between different months depending the availability of data

6 Basic ideas: The Solution Automatic detection of evident outliers Unit price decrease more than 50 per cent or increase of more than 100 per cent year-on-year ->observation deleted, interpreted as a different product despite the same CN Thresholds can be adapted for inidividual CPA/CN-series CPA 4 elementary aggregate indices aggregated to total index using CPA 3-digit categories year T-1 weights (Laspeyres)

7 The solution Compilation practices: Missing elementary aggregate indices CPA 3-series with no observations in current month replaced by CPA 3-index t-1 multiplied by change in the total index for month t. The final index is compiled using month-onmonth-changes that are chained over one year Weights always year T-1; annual chaining in December/January

8 Reflection The new system has so far met expectations results comparable with the old Customs UVI, more automated and transparent production The new system utilises significantly more data than the old one (and also significantly more than StatFi index) StatFI index sample ~3 400 price observations monthly, Customs old index sample ~1 300 unit value series monthly (the unit value series contain around value/quantity observations, summmed up to those unit value series) Customs new index sample ~ price observations monthly Customs new index covers ~90 per cent of the turnover of the total external trade, the old index covered ~60 70 per cent A number of things still to do prior to production(data warehousing for index-series, volume indices, analytical tables et.c.) päiväys 8

9 Reflection There is still some discrepancy between Customs and StatFi export and import price indices May result from different quality adjustment methods, different chaining system or different samples StatFi index chained every five years, Customs index annually StatFi product/enterprise sample smaller than Customs StatFi products strictly homogenious (tight specification of the product to be followed up), Customs new CN*enterprise categories still more heterogenous Issue will be explored more when the production system is finalised päiväys 9

10 The outcome Comparing indices of Import _1 2010_3 2010_5 2010_7 2010_9 2010_ _1 2011_3 2011_5 2011_7 2011_9 2011_ _1 2012_3 2012_5 NewCustoms_Import StatFinland_Import OldCustoms_Import päiväys 10

11 The outcome Comparing indices of Export NewCustoms_Export StatFinland_Export OldCustoms_Export _1 2010_3 2010_5 2010_7 2010_9 2010_ _1 2011_3 2011_5 2011_7 2011_9 2011_ _1 2012_3 2012_ päiväys 11

12 Compilation in practice The compilation process flow and formuli used päiväys 12

13 Index calculation process DW:dataware cn flow kauptil stat_num country time value quantity1 quantity2 number DW:Notific stattnum companycode time number SUMM:Index-year cn cpa2008 companycode flow group of countries time value quantity1 quantity2 Yearly changes (basic data) Data-spesific tables Basic and current year Item register (data-spesific) cpa4-level sumvalue of basic year &DATA = EUimport/export and external import/export I:&DATA_year price_pt +others I:&DATA_preyr price_pt +others Data warehouse Sum to cn-companylevel I:Basic- &DATA_ preyr cpa2008 cn business code price_p0 I:Total&DAT_y ear cn cpa2008 business code time value quantity1 quantity2 price_pt price_p0 DW:Cn-cpa2008- item cn-code cn-cpa2008code cn-beginn_day cn-level_code I:Par_cpa4_&DATA_ year cpa2008 cpa4 I:Sum_cpa4&DATA_ preyr cpa2008 cn company code yearsum_value_cpa päiväys 13

14 Formula of most detailed level (CPA4) Chained Laspeyres index I CPA4 = n i=1 ((p it q it * (p it / p i0 )) n i=1 (p it q it ) (company*cn= i = 1,2,,n) päiväys 14

15 Most detailed level unit price index and general unit price index Calculated basicprice p0 of new companies (2 month) and updated the table of basicprices price_pt_pre/cpa4_ind I:Basic_&DATA_preyr I CPA = n i=1 ((p it q it * (p jet / p j0 )) n i=1 (p it q it ) (comp*cn= i = 1,2,,n). Current weights Summed to cpa4-level Total export and import indeces calculated on weighted average EU import + external import, EU export+ external export I:&DATA_2_year_month sum_value_cpa4 sum_value*priceratio cpa4_ind_pre I::total&DATA _year I::cpa4&DATA_year_PRE I:Cpa4_index&DATA_year_month cpa2008 sum_value_cpa4 sum_value* Priceratio cpa4_ind I:Cpa4_index&DATA_year_ month cpa2008 sum_value_cpa4 sum_value* Priceratio cpa4_ind I:Karsittu&aineisto_ vvvv_kk cn cpa2008 lytun vvkk tarvoeur paljous1 paljous2 hinta_pt hinta_p0 hintasuhde arvo*hintasuhde I:General_ind_&DATA_year_month Yleis_IND_vvvv_kk Calculated monthly Interval to reject pricerartio 0.5 < priceratio > 2.0 I:S um _cp a4 &D AT A_ edv v General index is calculated with basic weights Other tables: I:Abnormal_&DATA_year_month I:Abnormalt_oil_&DATA_year_month I:Missing_&DATA_year_month I:New_&DATA_year_PRE_month päiväys 15

16 Formula of CPA 3-digit level unit price index I kk/(cpa3) = n i=1 ((p i0 q i0 ) * I icpa4 ) n i=1 (p i0 q i0 ) (CPA= i = 1,2,,n) päiväys 16

17 CPA 3-digit level indices, imputation and chained monthly changes I kk = n i=1 ((p i0 q i0 ) * I icpa ) n i=1 (p i0 q i0 ) (CPA= i = 1,2,,n) Summed to cpa3-level I:Cpa4_index_&DATA_year_month I:Cpa3_index&DATA_year_ month cpa2008 sum_value_cpa4 yearsum_value_cpa3 cpa3_ind_month Combine all months in one table and impute the missing cpa3- index value from previous month s same cpa3-index + change of general index I:Cpa3_index&DATA_year cpa3 sum_value_cpa4 yearsum_value_cpa3 cpa3_ind_month (all months) Calculated monthly changes. January compared to previous year s re-calculated index of desember = Chain changechain_preyr_pre.* change&year_mth I:Cpa3_ind_imp_&DATA_year cpa3 yearsum_value_cpa3 cpa3_ind_mth (all month) I:Sum_cpa4&DATA_PREyr I:General_change_&DATA_ye ar_pre_mth General_IND_year_PRE General_IND_year_mth muutos I:General_indexi&DATA_yr_m th Calculated the change of general index I:General_ind2_&DATA_year_month General_IND2_year_mth I:General_ind2_&DATA_year_PRE_mth General_IND2_year_PRE General_IND2_year_mth I:General_changeCHAIN_&DATAyear general_ind2_year_mth change_mth changechain _year_mth (all months) I:MthchangeCHAIN_&DATA_year cpa3 change_mth changechain_year_mth (all month) Other tables: I:Cpa3_julk_muut_&aineisto_vvvv I:Kkket_kaikki_vuodet_&aineisto I:Yleis_muuket_julk_&aineisto_vvvv I:Yleis_muukset_kaikki_v_&aineisto päiväys päiväys Aihe 17

18 CPA 2-digit level and CPA toplevel indices Calculated on the same way as CPA3, without imputation. No general index Summed to CPA2-level I:Cpa2_index&DATA_yr_mt h cpa2 sum_value*cpa3 yearsum_value_cpa2 cpa2_ind_mth I:Cpa3_ind_impu_&DATA_year_mth Combine all months in one table Monthly changes calculated on the same way as CPA3 I:Cpa2_indeksit&DATA_year cpa2 sum_value*cpa3 yearsum_value_cpa2 cpa2_ind_kk (all months) I:Mth_cpa2_changchain_&DATA_year cpa2 cpa2_ind_mth (all months) change_mth changechain_year_mth DW:par_cpa2008code CPA2008_CPAKD CPA2008_topclass CPA2008_name_FI I:Cpa1_index&DATA_year_mth cpa_topclass sum_value_cpa2 yearsum_value_cpa1 cpa2_ind_mth I:Cpa1_index&DATA_year cpa_topclas cpa1_ind_mth (all month) I:Mth_cpa2_chang_&DATA_year_mth cpa3 change_mth changechain_year_mth I:Mth_cpa1_chagchain_&DATA_year cpa3 change_mth changechain_year_mth (all month) I:Mth_cpa1_chang_&DATA_yr_mth cpa3 change_mth changechain_year_mth päiväys 18

19 Unit price indices on import: total/extra/ intra General index of Import Total import Extra import Intra import _1 2010_3 2010_5 2010_7 2010_9 2010_ _1 2011_3 2011_5 2011_7 2011_9 2011_ _1 2012_3 2012_ päiväys 19

20 Unit price indices on export: total/extra/intra General index of Export Total export Extra export Intra export _1 2010_3 2010_5 2010_7 2010_9 2010_ _1 2011_3 2011_5 2011_7 2011_9 2011_ _1 2012_3 2012_ päiväys 20