BALANCE OF PAYMENTS WORKING GROUP

Transcription

1 EUROPEAN COMMISSION EUROSTAT Directorate C: National accounts, prices and key indicators Unit C5: Integrated Global Accounts and Balance of Payments 4 April 2016 BP/16/06 BALANCE OF PAYMENTS WORKING GROUP APRIL 2016 Deutsche Bundesbank Regional Office in Bavaria Munich Branch Leopoldstrasse München Starting: Wednesday, 20 April 2016 at Ending: Thursday, 21 April 2016 at Agenda item 6 ESS.VIP.SIMSTAT: Exchange of micro-data on intra-eu trade between Member States Document available on CIRCABC here (user must be logged in CIRCABC) 1

2 European Commission Eurostat Directorate B: Methodology, corporate statistical and IT solutions Unit B-3: IT for statistical production Directorate G: Global Business Statistics Unit G-5: Goods production and International Trade Luxembourg, 29 March 2016 ESS.VIP.SIMSTAT Project for exchange of micro-data on intra-eu trade FINAL REPORT 1

3 Table of Contents 1. EXECUTIVE SUMMARY OVERVIEW OF THE PROJECT Background ESS.VIP.SIMSTAT EXECUTION OF THE PROJECT Project governance structure and organisation of work Tasks carried out during the project Organisation of the pilot exchange Analysis scheme for exchanged micro-data Spreadsheets based on ESSnet Indicators based on ESSnet Procedure for preparing the final report to the ESSC Limitations of the pilot exercise STATISTICAL ANALYSIS OF THE EXCHANGED MICRO-DATA Indicators of coverage Indicators of match between national imports and mirror exports Indicators reflecting gain in information Triangular trade between Member States Mixed flow indicator Comparison of behavioural pattern of revisions Validity and consistency of mirror data Comparison based on spreadsheet Conclusions of the statistical analysis ANALYSIS OF THE PROCESS OF DATA EXCHANGE CONCLUSIONS AND RECOMMENDATIONS Annex 1: Overview of Member States involvement Annex 2: ESS.VIP.SIMSTAT governance structure Annex 3: The process of micro-data exchange Annex 4: Timetable for pilot micro-data exchange Annex 5: Overview of all tables and indicators

4 Annex 6: Overview of consultations Annex 7: Procedure for evaluation of the pilot exchange of micro-data Annex 8: Values of indicators of coverage Annex 9: Indicators of match between national imports and mirror exports Annex 10: Indicators reflecting gain in information Annex 11: Indicators reflecting gain in information concerning traders below threshold Annex 12: Values of the Mixed Flow Indicator (I 1 MF )

5 1. EXECUTIVE SUMMARY The current data collection system for compiling intra-eu trade statistics (Intrastat) is characterised by a dedicated business survey where data are collected directly from the traders. The system has been in place since 1993 and there have been a number of changes made to the collection process over the years. These changes have included a reduction in the total number of mandatory data fields in the survey and a reduction in the number of businesses required to complete the survey. In November 2011, the European Council called upon the European Statistical System (ESS) to take effective measures ensuring a substantial reduction of the response burden by redeveloping Intrastat, while maintaining at the same time, a sound level of quality. Eurostat s response was the SIMSTAT (Single Market Statistics) proposal. SIMSTAT proposes the creation of an additional data source by making the exchange of micro-data (trader level data) on intra-eu exports among Member States compulsory. Each transaction reported in one Member State may serve as a data source for two Member States: for compiling the intra-eu exports of the exporting Member State, and for compiling and/or verifying the intra-eu imports of the partner Member State. In May 2012, the European Statistical System Committee (ESSC) launched a project to investigate the feasibility of the SIMSTAT approach. The purpose of the project was on the one hand to analyse the comparability and usability of the exchanged micro-data, and on the other hand to explore the technical viability of exchanging big volumes of data in a secure way. At the November 2012 ESSC meeting, the project was included in the ESS.VIP programme. The SIMSTAT project was launched in June Twenty Member States 1 agreed to participate in the project and exchange micro-data on intra-eu exports on a voluntary basis. The pilot exchange of micro-data itself was preceded by an important preparatory phase, resulting in agreements on a number of statistical issues, confidentiality arrangements, development and installation of necessary IT infrastructure and connection to a secure transmission network. During the second and third quarter of 2015, the participating Member States exchanged micro-data on intra-eu exports for the reference period January August 2015, according to a pre-agreed timetable. During the period April-October 2015, the Member States analysed the exchanged micro-data by comparing them with the collected data on intra-eu imports with the aim to assess the quality, completeness, comparability and re-usability of the exchanged micro-data. Member States also evaluated the process of exchanging micro-data from an IT infrastructure point of view. The overall results showed that: The mirror exports data could be used effectively as a full or partial substitution of the nationally collected imports data when the VAT numbers of importers were collected and exchanged. Member States involved in the project indicated that a period of a few years would be needed to achieve the full benefit of the exchanged data. 1 AT, BG, CZ, DE, DK, EE, EL, FI, FR, HR, IT, LT, LU, LV, MT, PL, PT, RO, SI and SK. UK did not participate in the pilot exchange, but received data from FR and IT on a bilateral basis. 4

6 The use of mirror data for compiling intra-eu imports statistics could reduce the administrative burden on Intrastat reporters on the intra-eu imports side. The pilot exercise proved that from an IT point of view the secure exchange of microdata was feasible. Some enhancements to the system would be required to ensure a fully automated exchange process. The results from the pilot exercise were obtained using defined laboratory conditions in which intrinsic limitations were applied; this should be taken into consideration when generalising these results outside of this environment. On the basis of the analysis carried out by the Member States some more detailed conclusions could be drawn: Successes Generally, a good coverage of traders was achieved through using the mirror data and a gain in additional information was found for aggregated trade values and to a lesser extent for the numbers of Intrastat reporters. In order to ensure the usability of mirror micro-data, the mandatory collection and exchange of the partner ID number (VAT number of the importer) is essential. However, some of the Member States who simulated partner ID number also showed very good results. The quality of mirror data was found to be good and Member States could benefit from the exchanged micro-data by improving the coverage of their intra-eu imports and thus the quality of their data with respect to accuracy. The comparability of data between neighbouring Member States was often considerably higher than average. "Small" Member States could identify a high share of their trade in value in the microdata received from so-called big Member States. Data validity and consistency, except for some minor shortcomings, were fairly good. It could be concluded that many validity errors were simply due to unclear data specification and failures in the implementation of the prescribed guidelines. The analysis carried out by individual Member States highlighted a potential alternative method of calculating intra-eu imports statistics using a mixture of nationally collected data and the received mirror data in order to ensure a smooth transition and potentially mitigate a break in time series. Further elaboration at detailed level (product and partner country) would be needed. The use of the mirror exports micro-data in compilation of imports statistics meant a gain in the number commodity codes reported compared with imports compiled only from nationally collected data. There was also data for a higher number of Intrastat reporters - although the gain in value from these extra Intrastat reporters was relatively small. The IT infrastructure worked well and the technical support was of high-quality. 5

7 Risks Problems regarding data completeness, coverage, data consistency at detailed level and breaks in time series were detected. Although the difference between imports and mirror exports at total level was not very large, there were considerable differences for individual Member States at commodity code (CN8) level. In a number of cases Member States could identify important differences between the declared values of matched traders and commodities. Further enhancements of the IT system will be needed to help fully automate the microdata exchange. Experience gained during the pilot exercise Participating Member States raised the following issues as important for the smooth functioning of SIMSTAT in a real production environment: The reference guidelines need to be precisely defined and implemented by all Member States, Exchange of data on specific goods and movements, metadata and information about the existing simplification options would be needed, New IT investments in Eurostat and the Member States would be required to fully realise the benefits of the SIMSTAT approach, Harmonisation of methodological practices would be necessary to define common statistical treatments appropriate to specific flows like, triangular trade and goods for processing. The ESS.VIP.SIMSTAT project was successfully conducted and all its objectives were achieved thanks to the close collaboration among the ESS partners, commitment of the participating Member States and clear guidance given by the ESSC and the Vision Implementation Group (VIG). The pilot exercise showed that micro-data exchange has the potential to allow Member States to begin to replace the collection of their imports data. The possibility offered to reduce the asymmetries between the data on intra-eu imports and mirror exports at a detailed level can lead to an improvement in the quality of intra-eu trade data and thus of other related macroeconomic statistics (e.g. Balance of Payments, National Accounts). Regular micro-data exchange between Member States also creates an opportunity for realising the ESS vision for changing the production method of European statistics within the ESS. 6

8 2. OVERVIEW OF THE PROJECT 2.1. Background Traditionally, statistics on international trade in goods are based on the data collected by national customs administrations. The establishment of the Single Market on 1 January 1993 abolished the barriers of movements of goods within the European Union (EU). It removed customs formalities between Member States and led to a subsequent loss of intra-eu trade statistics data sources. To compensate for this loss, a new data collection system called Intrastat was introduced by Council Regulation (EEC) No 3330/91. The objective was to ensure that the statistics would still be able to provide an accurate and detailed view of the trade between Member States, in order to respond to various policy needs. The Intrastat system can be characterised as follows: Direct data collection from traders: A dedicated enterprise survey where data are collected directly from the traders liable to Intrastat reporting 2. Close linkage with the VAT system: The Intrastat legislation requires tax authorities to regularly provide national statistical authorities with information on intra-eu supplies and acquisitions. A threshold system to reduce overall burden on enterprises: Member States are required to set up cut-off thresholds in annual values of intra-eu trade, below which traders are exempted from Intrastat reporting. The Intrastat system replaced customs declarations as a source of data for statistics on intra- EU trade. Although this led to a noticeable reduction of burden, the burden associated with Intrastat was still considered high. The simplification approach, which has been followed until now, was based on reducing the minimum coverage requirement (exempting more enterprises from Intrastat reporting by raising the exemption thresholds) in the Intrastat legislation. This has meant a decrease of the minimum coverage rates from the initial 99% to the current 97% for intra-eu exports and 93% for intra-eu imports. There are, however, obvious limits with this approach because of the necessary trade-off between the burden reduction and the level of data quality. A burden reduction obtained by decreasing the coverage rate cannot be achieved without the use of other data sources or by increasing the amount of data that has to be estimated. Although it would still be possible to compile reliable statistics at a more aggregated level for macro-economic purposes, there would be a loss of accuracy at the more detailed level, and no new ways addressing the problem of asymmetries either. In November 2011, the European Council concluded that the reduction of the burden achieved so far was not sufficient and called upon the ESS to: take the steps needed to address current and future user needs, take effective measures ensuring a substantial reduction of the response burden by redeveloping Intrastat, maintain, at the same time, a sound level of quality. 2 A joint collection system for Intrastat and VIES data is applied in some Member States. 7

9 2.2. ESS.VIP.SIMSTAT Since the traditional way of simplifying the system by reducing coverage rates could no longer satisfy the requested sizeable reduction in administrative burden, a new statistical architecture was necessary to redevelop Intrastat. Eurostat proposed SIMSTAT, a concept which consists of three complementary measures: creation of an additional data source on the intra-eu imports side by making the exchange of micro-data (trader level data) on intra-eu exports among Member States compulsory, enhancement of the quality of statistical information on the intra-eu export side, simplification of the reporting requirements for Member States on the intra-eu import side. SIMSTAT adopts the principle that data within the ESS need not be collected more than once. The overall aim of the project is to exchange micro-data on intra-eu exports of goods between Member States. Thus, each transaction collected in one Member State may serve as a data source for two Member States: for compiling the intra-eu exports of the exporting Member State, and for compiling and/or verifying the intra-eu imports of the partner Member State. This means that Member States would no longer be obliged to collect Intrastat data from importers according to a minimum coverage requirement. Instead Member States may choose to use part or all of the exchanged micro-data on intra-eu exports that has been collected by other Member States and made available to the importing Member State to compile their intra-eu imports statistics. The use of exchanged micro-data has the potential to answer the continuing calls for the reduction of burden on enterprises with this additional data source allowing Member States to begin to replace their collection of imports data. In addition, the possibility offered to reduce asymmetries between data on imports and mirror exports can lead to improvements in the quality of intra-eu trade data and thus of other related macroeconomic statistics (e.g. Balance of Payments, National Accounts). Finally, the regular micro-data exchange between Member States creates another opportunity for realising the ESS vision for changing the production method of intra-eu statistics within the ESS. The SIMSTAT proposal allows Member States to replace partly or fully their survey on intra- EU imports, but does not oblige them to do so. It gives the opportunity of gradually replacing the nationally collected intra-eu imports data with other sources, possibly over a period of a few years. The approach described will thus progressively reach the political target of a "sizeable" difference in the burden on enterprises, while maintaining an adequate level of quality. In May 2012, at its 13 th meeting, the ESSC discussed the SIMSTAT proposal. The ESSC "agreed on the main principles of SIMSTAT and supported the launching of the project to test the feasibility of micro-data sharing" 3. The project had the following objectives: 3 ESSC 2012/13/5/EN 8

10 clarify relevant statistical, organizational and technical issues for establishing micro-data sharing, set up the necessary IT infrastructure and transmission network, carry out a pilot exchange of micro-data between Member States on a voluntary basis, analyse the results of the pilot, and submit a report describing the work done, to the ESSC in May At the time of the launch of the project, the ESSC underlined that this was a "decision on starting the project, not a final decision on how to deal with the reform of Intrastat". The ESSC also stated that "at the end of the process it might not be possible to reach the 50% burden reduction, as described in the background documents", and asked to handle the wording in any future communication "with care" 4. At the November 2012 ESSC meeting, the project was included in the ESS.VIP programme. In May 2014, the ESSC agreed on ESS Vision 2020 as the main reference framework to guide the strategic decisions at the ESS level in the coming years. As one of the first steps towards the implementation of Vision 2020 all ESS.VIPs, including ESS.VIP.SIMSTAT, were evaluated in order to investigate their alignment with the Vision The evaluation concluded that the SIMSTAT project was in line with the Vision 2020 insofar as its objectives contributed to meeting some of the challenges listed in the Vision 2020 and could also be clearly positioned in relation to the five key areas for delivering the Vision However, the evaluation also identified a number of elements that had to be further elaborated. Some of them were intrinsic to the project like detailed project plan, decision making gateways and updated risk analysis. Others, like alternatives for the re-design of Intrastat, cost-benefit analysis and the assessment of administrative burden reduction potential as well as legal and methodological issues, went beyond the scope and mandate of the ESS.VIP.SIMSTAT project as set up in As a response to this evaluation, Eurostat presented two separate documents at the November 2014 ESSC meeting: one, elaborating all the ESS.VIP.SIMSTAT related issues 5 and a second detailing the business case for a new project named Re-design of Intrastat, which addressed the above mentioned issues which were beyond the scope of the ESS.VIP.SIMSTAT project (in particular, administrative burden reduction potential and cost- benefit analysis of possible options for redeveloping Intrastat) 6. The ESSC were content with the additional information provided for ESS.VIP.SIMSTAT and welcomed the proposal of the new project concerning the re-design of Intrastat 7 aiming at enabling a sound evidence based decision on the future of Intrastat. The SIMSTAT project was launched in June 2012 with a joint session of the first meeting of the Project Steering Committee and the Project Task Force. The project schedule covered the period from June 2012 to May The three main phases of the project were: 4 ESSC 2012/13/EN 5 ESSC 2014/23/2c1.1/EN 6 ESSC 2014/23/2c1.2/EN 7 ESSC 2014/23/EN 9

11 Phase 1: Feasibility study for micro-data exchange (June 2012 Q2 2013); Phase 2: Development of the system for micro-data exchange (Q Q2 2015); Phase 3: Pilot testing of the system and reporting (Q Q2 2016). One of the main objectives of the first phase was to analyse various technical options for exchanging micro-data, considering their costs, benefits and potentials to fulfil pre-defined technical, functional and statistical requirements. In parallel, in depth analysis of relevant statistical questions, like timeliness, confidentiality and data quality etc. were also carried out. The objectives of the second phase were to develop the IT system, which was selected in phase one and to find solutions to all relevant statistical issues. Moreover, the organisation of the pilot exchange, the procedure to analyse the exchanged data, to document the results and to draft the reports were agreed upon during phase two. The third phase concentrated on the voluntary pilot exchange of micro-data, analysis of the exchanged data according to the agreed procedure, documenting the results and drafting the reports. Member States who participated in the pilot exchange were: AT, BG, CZ, DE, DK, EE, EL, FI, FR, HR, IT, LT, LU, LV, MT, PL, PT, RO, SI and SK (see also Annex 1). The final report was scheduled to be submitted to the May 2016 ESSC meeting. 10

12 3. EXECUTION OF THE PROJECT 3.1. Project governance structure and organisation of work The project governance structure adopted was (see also Annex 2 of this document): Project Steering Committee (PSC): Responsible for high-level management from the business and IT points of view. It defined the main orientations of the project, adopted the detailed project planning, monitored the project progress, reviewed the finalisation of the project and approved the final report before its presentation to the ESSC. Project Task Force (PTF): Responsible for the technical execution of the project, assisted Eurostat in the preparation and implementation of the project including the elaboration and monitoring of the detailed project planning. They helped Eurostat in reviewing the project documentation and the reports and provided their opinion on the acceptance of the project deliverables. ESS.VIP.IT Task Force: Responsible for providing information about IT systems of the Member States, helped Eurostat and the Project Task Force to handle IT issues during the feasibility phase, to implement the IT system for micro-data exchange and to monitor the testing phases. They also helped to prepare the pilot exchange of micro-data, reviewed the IT related project documentation and provided their opinion on the acceptance of IT project deliverables, including the results of the pilot exchange. Eurostat core project team: Responsible for the project management, coordination and implementation on a day-to-day basis. They drafted the project planning, the specifications, the tender documentation, the acceptance criteria and other documents of a statistical and IT nature. In addition, they elaborated and reviewed project documentation and reports. Finally, they organised the meetings of the Project and the IT Task Forces and the Project Steering Committee and prepared inputs for these meetings. ESSnet on preparation for exchange of micro-data on intra-eu trade between Member States (ESSnet1): This multi-beneficiary grant agreement in the form of an ESSnet consisted of 18 partners from 17 Member States. The ESSnet dealt with the following issues: setting up confidentiality arrangements, establishing rules for micro-data validation, designing the exchange timetable, elaborating the data compilation guidelines, elaborating the quality evaluation of the pilot results, and producing the technical documentation and instructions on technical issues. The grant agreement entered into force in September 2013 and the action finished in September ESSnet on implementation of trial exchange of micro-data on intra-eu trade between Member States (ESSnet2): This multi-beneficiary grant agreement started in October Its overall objectives were to practically implement the results of the previous ESSnet project, test the newly established infrastructure and tools, carry out the exchange of micro-data in a coordinated manner, analyse the exchanged data, and assess the functioning and effectiveness of the micro-data exchange. The ESSnet consisted of all Member States participating in the pilot exchange (and the UK). It concluded its work in December

13 3.2. Tasks carried out during the project Phases one and two of the project were dedicated to conduct the following tasks: a) Content of exchanged micro-data: The purpose of the pilot micro-data exchange was to exchange data, collected on Intrastat declarations, between participating Member States. There was a need to agree on the data elements to be exchanged during the pilot exercise. This content, definition and structure of all data elements (type, length, format, etc.) were defined in a reference guideline (DocMet400-SIMSTAT). b) Exchange of importer s VAT number: One important feature of the proposed SIMSTAT system was the exchange of the identification number of the trading partner in the Member State of import. Since the current legislation does not require this data element, it was agreed to exchange real data, if available, and simulated data based on VIES if it was not. An appropriate method for simulating partner ID number was developed by the Project Task Force. c) Timeliness and integration into the production process: The timetable for micro-data exchange was established in such a way that the current national obligation of transmitting data to Eurostat, to other main national users, like National Accounts and Balance of Payments statistics, and the national release calendar would not be affected. d) Data validation and quality issues: In order to ensure the quality of data the Member States agreed to a set of data validation rules to be applied by all the countries sending micro-data. The types of validations that should be done by the different participants in the micro-data exchange (sending Member State, Hub and receiving Member State) were defined and implemented. e) Confidentiality issues of micro-data exchange: The Member States participating in the pilot exchange and Eurostat signed a multilateral "Data Sharing Agreement (confidentiality charter)" ensuring that they would treat the data received from other Member States in accordance with a set of common confidentiality standards. The content of the confidentiality charter was developed by the ESSnet1. f) Data analysis: The subject of how to compare and document the match between nationally collected intra-eu imports data with the corresponding mirror exports data to evaluate their re-usability were extensively discussed by the Member States. Based on the proposals from ESSnet1 and ESSnet2, a common procedure to analyse the exchanged micro-data and to document the results was established. Pre-defined tables and indicators based on both collected national intra-eu imports data and mirror exports data were agreed. Templates for the tables and tools to calculate the indicators were provided to all participating Member States (for details see section 3.4). g) Technical feasibility study: The purpose of the feasibility study was to propose an IT infrastructure capable of exchanging micro-data on intra-eu exports between the Member States which would meet a set of statistical and technical requirements. The feasibility study investigated the current state of IT environments in the Member States and examined possible solutions for data exchange with detailed functional requirements for each solution. It also investigated which transmission network for the micro-data exchange would be used. The Project Steering Committee approved the proposal recommended in the feasibility study, i.e. implementing a centralised Hub-based system 12

14 where all Member States would exchange the micro-data through a Hub. It was also proposed to use the Common Communication Network (CCN) as the transmission channel. h) Development of the SIMSTAT Hub: The Hub was the central information system responsible for receiving, validating, applying transformation functions and sending data and reports to the Member States. It was developed by an external contractor and installed in a secure environment of the European Commission. i) The transmission network: All participating Member States and Eurostat were connected to the CCN gateways, which enabled them to communicate (send and receive data) via the secure CCN network. This was done in cooperation with the Directorate General for Taxation and Customs of the European Commission (DG TAXUD), who maintains the network. j) SIMSTAT communication component: The SIMSTAT communication component was developed and installed in all participating Member States as a stand-alone application. It facilitated the micro-data exchange between the SIMSTAT portal and the SIMSTAT Hub by making use of the configured underlying network (CCN network). k) Service desk: A service desk was established in order to provide assistance to the users of the SIMSTAT portal where issues were reported during the transmission of micro-data. In order to facilitate their participation in the pilot micro-data exchange, Eurostat offered different types of grants to the Member States: i) Individual IT grants for preparation for pilot micro-data exchange: Eurostat offered individual IT grants to 17 Member States. The grant agreements addressed the following issues: construction of a SIMSTAT database for the pilot exchange, setting up the appropriate IT infrastructure for data processing and exchange, tools for data validation and analysis. The actions started in January 2014 for a duration of 15 months. A second round of grants was started in January 2015, again for a duration of 15 months. ii) Individual grants to study methodological issues: To respond to the needs expressed by the ESSC and the Intrastat Committee (current ITGS Steering Group), Eurostat offered individual grants addressing the following themes: assessment on the burden aspects related to implementing SIMSTAT, analysis on the quality aspects related to SIMSTAT, analysis and enhanced use of administrative data, and reduction of asymmetries. The actions started in January 2014 with a duration of 15 months. iii) Individual grants to establish connectivity to the CCN network: To prepare the IT infrastructure for enabling the micro-data exchange between Member States, Eurostat offered individual grants to enable connection to the transmission platform CCN Organisation of the pilot exchange The third and the final phase of the project focused on the exchange of micro-data, the analysis of the exchanged data, the documentation of the results and the reporting to the ESSC. 13

15 The SIMSTAT pilot exchange of micro-data started in April 2015 and continued until October This section gives a short overview of the organisation of the exchange by describing the business and technical processes. A graphical presentation of the process is shown in Annex 3. The presented organisation of the pilot exchange was approved by the Project Steering Committee in its meeting on 27 March a) Sending Member State: After Intrastat data collection, each Member State extracted microdata on their intra-eu exports according to the structure described in the reference guidelines DocMet400-SIMSTAT. The micro-data file consisted of 31 data elements. Besides the usual data elements which Member States currently collect on Intrastat declarations either as mandatory or optional data, the micro-data file included two additional elements: partner ID number (mandatory) and country of origin (optional). Currently the partner ID number is collected in France, Italy, Malta, Portugal (mandatory for the top 100 biggest exporters and some voluntary declarations) and Romania. Moreover, Luxembourg collects it on a voluntary basis and about 30% of the Intrastat reporters transmit this information. The rest of the Member States were asked to simulate the partner ID number, based on the VAT/VIES data using the method proposed and agreed by the Project Task Force (see also section 3.6). Member States sending the micro-data file were responsible for ensuring good quality of data by carrying out appropriate validations. The validation rules, adopted by the PTF, were to be applied by all the Member States before sending the micro-data file according to an established timetable (see Annex 4). b) SIMSTAT Hub: The Hub acted as the central dispatch point for micro-data exchange between Member States. Upon the reception of the micro-data file from the sending Member State, the following automated processing was performed on the received micro-data: decryption, unzipping, splitting, validation, currency conversion, encryption and sending. The Hub executed some validity checks (whether valid codes were used, correct formats were used, etc.) in line with the agreed rules and an error report was automatically generated both for the sending and the receiving Member States. c) Receiving Member State: The Hub made available the micro-data files to the receiving Member States. They connected to the SIMSTAT portal, downloaded the files and imported them into their production database. It should be noted that the Member States received only those data where they were the trading partners; they did not receive the whole pool of micro-data. That is, only trade data measured in country X that concerns exports to country Y are exchanged with country Y and not with all other countries. In short, none of the Member States have access to micro-data concerning trade between two other Member States Analysis scheme for exchanged micro-data A common procedure to analyse the exchanged micro-data and to document the results was approved by the Project Steering Committee in March While defining the assessment criteria, a differentiation was made between the process of the exchange of micro-data and the product of this exchange, i.e. the exchanged micro-data themselves. Consequently, separate assessment criteria for the process and the product were defined. 14

16 The process consisted of a series of actions taken in order to get the selected set of micro-data transferred from the sender's environment to the receiver's environment via the Hub, using the CCN network and ultimately integrate them in the production environment of the receiver. The functioning of the whole system was evaluated during the entire period of the pilot phase of the SIMSTAT project, for the exchange of monthly data for the reference period January 2015 August The micro-data, which were exchanged between the Member States, were defined as the product. As the main purpose of SIMSTAT was to make available to all Member States an additional source of data for use in compiling their imports statistics, it was important to assess this data, in terms of their quality, completeness, comparability and re-usability. It is important to note that the comparison between the nationally collected imports and exchanged micro-data on exports will be affected by the lack of partner ID number in the micro-data exchanged and the existing asymmetries in the data collected by Member States. The pilot exchange gave the opportunity to carry out thorough analysis of the two data sets (collected and received) and to investigate the re-usability of the exchanged micro-data Spreadsheets based on ESSnet1 The tables proposed by ESSnet1 were used to give an overall assessment of the usability of the micro-data as a possible additional source for nationally collected imports data. The analyses focused on how well the mirror exports micro-data matched the national data and how good the coverage would be using the mirror micro-data. The Member States were asked to analyse the degree of match of micro-data at the level of the traders (Intrastat reporters) with the help of the following standard tables, using the microdata for the period Nationally collected imports and mirror exports micro-data were linked at the level of year/country/intrastat reporter/partner ID number. The analyses included: Table 1: Number of traders in nationally collected Intrastat imports and in mirror exports micro-data Table 2: Difference between nationally collected Intrastat imports and in mirror exports micro-data at Intrastat reporter level Table 3: Distribution of difference between nationally collected Intrastat imports and in mirror exports micro-data at Intrastat reporter level Table 4: Important traders (Intrastat reporters) in national data collection (Top 50% coverage) Table 5: Important traders (Intrastat reporters) according to mirror exports data (Top 25) Table 6: Important product codes (CN8) in national data collection (Top 25). The templates of the tables with explanations and practical instructions were delivered to all participating Member States. A description of the tables is provided in Annex 5 of this document. 15

17 Indicators based on ESSnet2 In addition to the analyses proposed in ESSNet1, summary indicators were calculated to facilitate the understanding of this large data set. While an overview of the indicators is given below, their detailed description is provided in Annex 5 of this document. a) Indicators of Coverage, Match and Gain Coverage: Shows the share (in percentage) of national Intrastat reporters and their trade values covered by the exchanged mirror micro-data. Match: Shows how good the match is between collected Intrastat imports and mirror exports data in terms of Intrastat reporters and their traded values. Gain in information: Shows if Member States will gain or lose information in terms of Intrastat reporters and trade value because of using mirror exports data. These indicators were calculated each month for the reference period March-August 2015, at t+60 (i.e. 60 days after the reference month); as well as for the whole period January-August 2015 by November 10, Information on traders below threshold: Shows what information would be available in the mirror exports data about traders which are below national exemption threshold. The above mentioned indicator was calculated once, before 10 November 2015, on the January-August 2015 data. b) Indicator for Triangular Trade This indicator shows the extent of triangular trade (i.e. when the country mentioned in the partner ID number is not the same as the country of destination). It was calculated once, before November 10, 2015, on January-August 2015 data. This indicator can be calculated only by those Member States that collect the partner ID number. c) Mixed Flow Indicators In SIMSTAT it would be up to the Member States to decide to what extent they would use the exchanged micro-data. These indicators show which mix of nationally collected imports data and the mirror exports data gives the best match and the maximum gain/impact. These indicators were calculated once, before July 15, 2015, on January-May 2015 data. d) Indicator of revisions Convergence time: This indicator shows how quickly initial values for a given reference month converge to a stable final value, both for nationally collected imports and mirror exports, i.e. after how many periods the absolute revision rates are less than 1%. Member States were advised to calculate this indicator for March 2015 data as this was the first monthly data to be exchanged and would offer the best opportunity to evaluate the evolution of the revised data. They were also advised to calculate the indictors for April 2015 data in the event the March 2015 data was affected by initial data transmission problems. 16

18 3.5. Procedure for preparing the final report to the ESSC This section describes the procedure for preparing the final report to the ESSC on the pilot micro-data exchange. Member States' report: Each Member State, who took part in the pilot data exchange, analysed the performance of the new IT infrastructure, the efficiency of the new production process involving the micro-data exchange, the usefulness of the new data source and the quality of data with the use of the exchanged micro-data. Each participant in the pilot exchange gave written feedback on the above-mentioned elements and compared and analysed the collected imports data with the mirror exports data by following recommendations described in section 3.4. The results of these analyses were documented by the Member States in the form of a report, using an agreed template. All the country reports are available in circabc under: d4a1cbaea053. ESSnet co-ordinator's report: The participating Member States transmitted their reports to the coordinator of the ESSnet2. The coordinator prepared a synthesis of the country reports, providing a comprehensive assessment of the pilot exercise, including both qualitative and quantitative evaluation. The country reports as well as the co-ordinator s draft report were presented and discussed at the final meeting of the ESSnet2 on November They are also available in the above mentioned circabc site. Final report on ESS.VIP.SIMSTAT: Eurostat prepared this consolidated report on the whole SIMSTAT project covering all the aspects and including the inputs received from the Member States and the co-ordinator of ESSnet2. The report was initially discussed in the SIMSTAT Project Task Force and the related IT Task Force. Thereafter it was presented to the SIMSTAT Project Steering Committee. In addition, Eurostat organised a series of consultations, reporting the outcomes of the project to various stakeholders (see Annex 6). The report was discussed with the Vision Implementation Network on 15 March A graphical presentation of the reporting procedure is given in Annex 7 of this document Limitations of the pilot exercise Although ESS.VIP.SIMSTAT offered a good opportunity to evaluate the feasibility of the SIMSTAT principles (exchange of micro-data, their re-usability for compiling intra-eu imports statistics and improving data quality), it has to be kept in mind that the pilot exercise was carried out under so-called laboratory conditions which are subject to a number of intrinsic limitations. Firstly, as not all Member States participated in the pilot exchange, it was not possible to compile the complete intra-eu imports statistics using the mirror data. Consequently, while the quality of the mirror micro-data was analysed, the Member States were not in a position to investigate the intra-eu totals. Secondly, most of the participating Member States do not collect the partner ID number (VAT number of the importer) from exporters reporting Intrastat data. In those cases, the partner ID number was simulated using a prescribed algorithm using VAT/VIES data. This approximation could not guarantee that the real trading partner (importer) was correctly 17

19 identified. Consequently, some comparisons of the collected data with the mirror data should be treated with caution. The situation concerning the collection or simulation of the partner ID numbers was as follows: FR, IT, RO, MT: collect systematically on a mandatory basis and exchanged them. PT: LU DE: Others: collects from top 100 biggest exporters on a mandatory basis (+ some voluntary declarations), the remaining partner ID numbers were neither simulated nor exchanged. collects around one-third of the partner ID numbers on a voluntary basis, the remaining partner ID numbers were neither simulated nor exchanged. does not collect and did not simulate. DE undertook random allocation of partner ID numbers to the exchanged transactions. Simulated according to the prescribed algorithm. Thirdly, data collected outside the Intrastat data collection system were not exchanged. The list of such data components (like estimates of missing intra-eu trade, intra-eu trade data coming from customs declarations, data on electricity, gas and other intra-eu trade data related to specific goods and movements not collected on Intrastat declaration, below threshold estimates, etc.) varies from one Member State to another. 18

20 4. STATISTICAL ANALYSIS OF THE EXCHANGED MICRO-DATA As mentioned in sections and 3.4.2, the statistical analysis of the exchanged micro-data was carried out using a pre-defined set of indicators and spreadsheets. Values of all the indicators calculated by the participating Member States for agreed time periods can be found in the circabc under: 46a6-a101-a5f7d6b5cd2b Indicators of coverage Two indicators were defined to measure the coverage of the nationally collected Intrastat data achieved with the mirror data received through the pilot exercise: Indicator I 1 : Coverage in terms of the number of Intrastat reporters, shows the percentage share of national Intrastat reporters covered by the mirror data received from a partner Member State. It is defined as the ratio of the number of Intrastat reporters identified in both national and mirror collections to the total number of Intrastat reporters in the national collection; Indicator I' 1 : Coverage in terms of the traded values, defined as the ratio of the trade value of the Intrastat reporters identified in both datasets (as recorded by national Intrastat reporters) to the total traded value according to national collection. The above mentioned indicators show the share of national Intrastat reporters and their trade value covered by the exchanged mirror micro-data. The indicators can be read from two different points of view from the view point of the receiving Member States (how good are the data received from all other partner Member States for compiling the national statistics) and from the view point of the sending Member States (to what extent the data sent to all other Member States are useful for their partners). The indicators calculated for the data covering the period January-August 2015 are presented in Annex 8 of this document. The coverage of national datasets by the mirror datasets is generally satisfactory, especially in terms of trade values (I' 1 ) and to a lesser extent in terms of number of Intrastat reporters (I 1 ). Indicator I 1 is in most of the cases between 0.5 and 0.9 (with the exception of some specific combinations of Member States, see Chart 1). This means that generally a reasonable share of national Intrastat reporters were identifiable in the data received from other Member States. The indicator I' 1 shows an even better result with most of the values between 0.8 and close to 1.0 (again with some specific exceptions, see Chart 1). The values are quite often above 0.9, which shows that the important enterprises with a high share of trade in the national collection could be found in both databases. Chart 1 shows the values of the indicator I' 1 from the point of view of the receiving Member States (x-axis). So, if one takes the example of AT receiving data from 19 other Member States, one can see that between 80% and 99% of AT s bilateral trade is covered by the mirror data received from 15 other Member States. Only in the mirror data from 4 Member States is the coverage is lower. 19

21 The distribution for all other Member States is very similar. The fact that most of the Member States are concentrated in the upper part of the chart shows that the majority of participating Member States are able to find a good coverage of their bilateral trade (at aggregated trade value level) in the mirror data received from other Member States. Chart 1: Coverage of national trade in values by mirror data (receiver s point of view) * The results are distorted, because around 70% of the partner ID numbers were unavailable Indicators I 1 and I' 1 show that the coverage is better when the data comes from Member States where the VAT numbers of importers (partner ID number) are collected rather than simulated, in particular data from FR and IT, that have systematically collected the partner ID number for a number of years, as well as from RO who started collecting partner ID number in January It has to be noted that the coverage is very good even for some countries that do not collect partner ID number. For example, data from AT, CZ, DE, DK, EL and SI were found highly useful by a majority of countries. Chart 2, on the other hand, shows the performance of the sending Member States (x-axis) in terms of coverage. It shows clearly that AT, BG, CZ, DE, FI, FR, IT, PL and RO are very good sending countries, as data sent by these countries reflect high coverage rates when compared with the data collected by the receiving countries. 20

22 Chart 2: Coverage offered by mirror data (sender s point of view) * The results are distorted, because around 70% of the partner ID numbers were unavailable Benefits of small Member States: At the time of inception of the SIMSTAT project, several "small" Member States expressed their concern that the coverage of their data in the mirror datasets of "big" Member States would be rather small and as a result they would lose considerable amount of information using exchanged micro-data. The relatively high exemption threshold in the big countries was usually mentioned as the possible reason. The indicators I 1 and I' 1 show that this concern was not justified. Chart 1 shows that Member States like LU, MT, PT or SI could identify a high share of their Intrastat reporters and an even higher share of the trade in value in the data received from socalled "big" Member States like IT, FR or DE. Table 1 shows that between 91% and 99% of the trade of these four "small" Member States with the above mentioned three "big" Member States is covered by the data received from them. Table 1: Coverage of "small" Member States' data by "big" Member States data Sending Receiving Member States Member States LU MT PT SI DE FR IT

23 As mentioned in section 3.6, LU collects around one-third of the partner ID numbers, the remaining partner ID numbers were not simulated. Consequently, around two-thirds of the traders were indicated as unknown. For this reason the data sent by LU quite often could not be compared by the receiving Member States. This led to the unsatisfactory level of coverage which does not say anything about the quality of LU data. In the case of PT the collection of partner ID was only mandatory for the Top 100 biggest exporters (despite some voluntary declarations), while the remaining partner ID numbers were neither simulated nor provided, which, with some additional effect of triangular trade, explains the low coverage. Trade between neighbouring Member States: The analysis of the data exchanged during the pilot exercise revealed another interesting result. The comparability of data between neighbouring Member States was often very high. As Table 2 shows, for example, BG could find a very good coverage of its imports from EL (96%) and RO (96%) in the datasets received from these two countries. Other examples are good coverage of CZ data in the mirror data from SK, of DE data in the data from CZ and in LT data when compared to data from LV. These results are rather encouraging as neighbouring countries are usually main trading partners. Moreover, long-term buyer-seller relationships and good knowledge of the traders over the border may also explain these findings. Table 2: Coverage in mirror data from neighbouring Member States Receiving MSs BG CZ DE EE FI HR LT LV PL Sending MSs EL=0.96; RO=0.96 SK=0.91 CZ=0.95 FI=0.95; LT=0.96; LV=0.91 EE=0.94 SI=0.91 LV=0.95; PL=0.94 LT=0.97 CZ=0.93; SK= Indicators of match between national imports and mirror exports Indicator I 2 shows whether the trade value of those Intrastat reporters which were identified in both national and mirror datasets were similar. It is calculated as the ratio of traded value of common Intrastat reporters present in both datasets. Values close to 1 mean they are very similar. In this sense the indicator reflects how well the match of national and mirror datasets is, for Intrastat reporters present in both data collections. However, similar total values may conceal considerable differences in the trade value of individual Intrastat reporters. For this reason this indicator should be considered in combination with the correlation of the trade value of common Intrastat reporters in national and mirror datasets. The match is considered to be good if both I 2 and the correlation coefficient (CC) are near to 1; if I 2 is far from 1 or CC near to 0, the match is bad. The indicators I 2 and CC calculated for the data covering the period January-August 2015 are presented in Annex 9 of this document. 22

24 In order to classify the match obtained by the participating Member States, the following categorization has been introduced. The match is considered to be: good (++): if 0.8 I and 0.75 CC 1.0. satisfactory (+): if 0.6 I 2 < 0.8 or 1.2 < I and 0.5 CC 0.75 bad (-): otherwise. Table 3 shows that AT, CZ, DE, IT and LV could achieve fairly good match with mirror data received from most of the sending Member States. On the other hand, data sent by BG, CZ, EL, IT, LT and SK were considered to have good match by most of the receiving Member States. It should be noted that most of the Member States reported a rather bad match with the data received from DE, LU, MT and PL. As mentioned in section 3.6, DE undertook a random allocation of partner ID, LU on the other hand collected and exchanged around one-third of the partner ID. These facts might explain the reason for a poorer match. Table 3: Degree of match between national imports and mirror exports Receiver AT BG CZ DE DK EE EL FI FR HR IT LT LU LV MT PL PT RO SI SK AT BG CZ DE DK EE EL FI FR HR IT LT LU LV MT PL PT RO SI Sender 4.3. Indicators reflecting gain in information The indicators I 3 and I 4 show whether the Member States gain or lose information in terms of Intrastat reporters (I 3 ) and trade value (I 4 ) through using mirror exports data. I 3 is defined as the number of Intrastat reporters identified only in the mirror dataset as a share of the total national Intrastat reporters. I 4 on the other hand reflects the trade value of those Intrastat reporters as share of the total trade in value reported in the national collection. The values of the indicators I 3 and I 4, calculated for the data covering the period January-August 2015, are presented in Annex 10 of this document. As regards the level of gain in additional information through the use of mirror data, the above mentioned indicators displayed remarkably good results for many Member States. In particular, small Member States, like EE, EL, LU, LV, MT and SI could find a considerable amount of additional information (in trade values Indicator I 4 ) in the mirror data received mainly from other small Member States. 23

25 Part of the gain may relate to those traders who were below the national exemption threshold. Indicator I' 3 shows this type of gain in terms of the number of Intrastat reporters, while the indicator I' 4 relates to the trade in values. Although indicator I' 3 shows the possibility of gaining considerably in terms of the number of Intrastat reporters, the gain in terms of trade in values (indicator I' 4 ) is comparatively lower because of the relatively small volumes of those transactions. The indicators I' 3 and I' 4, calculated for the data covering the period January- August 2015, are presented in Annex 11. In some cases, the mirror data contained un-identified Intrastat reporters due to unsuccessful simulation of partner ID number. This was one of the reasons for a high value of indicator I 4. This is partly system inherent resulting from some of the known differences between the VAT and Intrastat system. These findings show the importance of collection of partner ID number for improving the level of match between the nationally collected data and the exchanged mirror data Triangular trade between Member States The extent of triangular trade was revealed by the pilot exercise. That is the case when the commodity and the invoice are not sent to the same entity (e.g. the commodity is sent to a manufacturing subsidiary, and the invoice to a commercial subsidiary). This information was only reported for the trade with Member States collecting the partner ID numbers. Comparable indicators were calculated for the data coming from FR, IT, LU, and MT. This type of flow is detected in the record when the country specification in the partner ID number was different than the Member State of destination. Indicator I 5 reports the share of trade (in percent) identified as triangular trade (vis-á-vis the above mentioned four Member States) in the national total trade. Values of this indicator are reported in the Table 4. Table 4: Extent of triangular trade in the national total trade (in %) Receiving FR IT LU MT MSs AT BG CZ DE DK EE EL FI FR HR IT LT LU LV MT PL PT RO SI SK

26 So, for example, for SI, the value traded by Intrastat reporters involved in triangular trade with FR represents 6% of the total imports of Slovenia from France. In other words, 6% of the French goods shipped by French exporters to Slovenian importers were in fact invoiced to a country other than Slovenia. In some cases, triangular trade represents an important value and is a major cause of asymmetries Mixed flow indicator The SIMSTAT approach leaves it up to the Member States to decide to what extent they will use the exchanged micro-data. The mixed flow defines a method of compiling the monthly trade statistics by using a mixture of both national and mirror data. The relevant indicator (I 1 MF ) is defined as: I 1 MF = total collected traded value calculated with mixed flow system for Intrastat reporters identified in both national and mirror datasets / total collected traded value calculated with actual system for Intrastat reporters identified in both national and mirror datasets. Here, total means total of collected data under Intrastat system, without estimation or additional data outside the scope of Intrastat, and for the aggregated flow of all countries for the period January-May A proper application of this procedure is possible only if all traders are identified in the mirror data collection and if the match is good at the trader level. In such a situation, the mixed flow method allows for a smooth transition by gradually increasing the share of the mirror data in the national statistics. In the case of the pilot exercise some traders remained unidentified in the mirror collection. Depending on the shares of national and mirror data sources used in the computation of the statistics, the coverage, the risk of double counting unidentified flows and the match between the mirror and the (remaining) national data collection vary. The level of late or non-response has also important effect. Moreover, the absence of DE data (because of missing real partner ID numbers) as well as absence of data from other important Member States not participating in the pilot exercise, and quite a large number of un-identified partner ID numbers in case of some of the Member States, created a bias in the calculation of the mixed flow indicator. For these reasons the values of the indicator, calculated by the participating Member States have to be treated with some reservation. Moreover, more investigations will be necessary to explore the possible mixture of the two data sources at commodity code level. The values of the indicators, calculated by the Member States are presented in Annex Comparison of behavioural pattern of revisions Both nationally collected and exchanged mirror data undergo revisions (data corrections) and integrations (late responses). The pilot exercise was used to compare the stability of the two datasets and their convergence to a final value. From the point of view of the receiving Member State, this required a longitudinal observation of how an initial estimate for April 2015 converged to a stable final value in the two different data collection systems (national and mirror). 25

27 Fifteen Member States used an Excel tool proposed in ESSnet2 and provided related revision indicators and graphs. Table 5 presents the convergence time (number of periods after the first data exchange required to converge to a final value) for the nationally collected data and the received mirror data as well as the difference between them. The differences indicate the increase or decrease in the required convergence time which would be expected when moving from the nationally collected data to a SIMSTAT data production system. The table shows indicators for 14 Member States only, since one Member State s results reflected national specificities in data processing and could not be compared with others. Table 5: Main revision indicators for 14 MSs AT BG DE DK EE EL FI IT LT LU LV MT PL SI National convergence time 1 3M 5D 4M 3M 3M >4M 2M 15D 3M >4M 3M 3M 3M 3M Mirrorconvergencen time 2 4M 4M 3M 4M 3M >4M 3M 3M 4M 4M 3M >4M 2M 4M Difference 3 +1M +4M -1M +1M M +2M +1M -1M 0 >1M -1M +1M 1: Number of months (M) or days (D) needed for the nationally collected data to converge to a final value 2: Number of months (M) or days (D) needed for the mirror data to converge to a final value 3: 2-1 In most cases, the increase or decrease in convergence time in the SIMSTAT system is moderate (+1 month for 5 MSs, -1 month for 3 MSs and 0 month for 3 MSs). Only in the case of BG is the SIMSTAT convergence time much higher than the national figure (difference of +4 months). In case of IT, the SIMSTAT convergence process is slower (difference of +2 months), whilst MT also experienced an increase of more than 1 month. Table 6 compares the revisions for the reference month of April 2015 at T+5M with respect to the measure of trade values for both nationally collected data and the received mirror data at T+35 days. So, for example in case of AT, the nationally collected total trade for the reference period April 2015 (measured at T+35 days) was revised 5 months later by 1900 million. Using the mirror data received from other Member States would lead to a revision of 200 million. It should be mentioned that the relatively high difference in AT figures results from the deadlines of collecting authorities (for VAT it is T+45); VAT is essential on the intra-euimport side for the control of completeness. For Member States like DK, EE, FI, LT, LV, MT and PL the two data sources differed only minimally in terms of revisions. Table 6: Amount of revisions for the reference month April 2015 at T+5M compared to T+35D, in million Euro in Million Euro AT BG DE DK EE EL FI IT LT LU LV MT PL SI National revision at T+5M Mirror revision at T+5M It can be concluded that in most cases, convergence to a stable value is similar for both nationally collected data and the exchanged mirror data. In case of mirror data, the convergence is rather smooth and the initial values converged smoothly (i.e. without spikes ) to the final value. 26

28 4.7. Validity and consistency of mirror data The assessment of the quality of mirror data in terms of validity and consistency varies among the participating Member States. Some of them reported on the good overall quality of received data, while others reported about the need for higher standardization in both validation practices and in the coding and reporting of variables. Some Member States (in particular CZ,) did not experience any problems regarding the validity and consistency of the received data, some (e.g. BG, FI) stated that the quality of mirror data and collected data, in terms of number of encountered errors, was similar to that in national data, while some others (e.g. EE) mentioned that the number of errors in mirror data was much higher than in the nationally collected data. The most common validity problems faced by the Member States were: Invalid Insert/Delete sequence: This was detected by some Member States (e.g. BG, EE, FI, IT) in the datasets received from a few other Member States. This was caused by not strictly respecting the instructions described in the data reference guidelines DocMet400- SIMSTAT by some countries as well as by some other technical issues. Invalid Commodity codes: This was mainly caused either by failures in implementing the DocMet400-SIMSTAT (cases of missing leading zeroes) or due to misclassification. Such incidences were reported by AT, DE, PL, PT and SK. In some other cases the received CN codes which were not valid according to the receiving Member States were in fact associated to specific simplification measures granted to Intrastat reporters by the sending Member States. Invalid Nature of transaction codes and missing or incorrect Supplementary units: reported by AT, EE, EL, PL, PT, RO, SI and SK. Use of simplified CN codes by some Member States: The receiving Member States (e.g. AT and FI) were not aware of the simplified codes used by some other Member States and could not process those transactions. Invalid importer VAT number: Failures in the structure of the VAT number as well as unknown numbers when checked against VIES data, reported by AT, DE, EL, PL, SI and SK. Problems of under-coverage or over-coverage: Some Member States detected under or over-coverage whilst using the various tools, these included time series analysis, the ratio mirror data/collected data for April 2015 or comparison of data using spreadsheets. Some of the receiving Member States applied their national consistency checks to the received mirror data and compared the number and the impact of the potential outliers found in the mirror data with those found in the nationally collected data. Some othe r Member States based their evaluation on the number of observed records which were outside the acceptance intervals (for example, RO reported that the number of warnings related to the unit value and number of outliers for unit weight in the received mirror data was quite big when the acceptance level based on RO national data was used). However, it should be noted that although a larger number of outliers was detected in the mirror data, in most cases the Member States reported only a moderate impact of the potential error on the total value of their trade. 27

29 It can be concluded that: Stricter respect of the specifications described in the document DocMet400-SIMSTAT (which has to be reviewed and formulated more accurately) by all Member States is necessary since many validity errors were simply due to failures in specifications and in its implementation. Data consistency failures occurred more frequently in mirror data than in collected data and this seems to suggest the need for more harmonization in validation practices. Data completeness, coverage problems and time series breaks have been detected. In some cases they seem to result from applying different methodological approaches in sending and receiving Member States. More in-depth analysis should be performed on cases with relevant impact Comparison based on spreadsheet The comparison based on the spreadsheet tables 1-6, (as described in section 3.4.1) showed good results at a more aggregated level. The mirror exports data coverage rate (value of mirror exports / value of national imports) confirms the generally good coverage achieved with mirror data. Coverage rates were quite often above 100%. The reason seems to be a universally better recording of exports as compared to imports and a higher concentration of bigger Intrastat reporters in exports, in contrast to imports where a certain number of smaller traders seems to be cut off from data collection by the higher exemption threshold for intra- EU exports. The quality of mirror data, in general, was found to be good and Member States could benefit from the exchanged data by improving the quality and coverage of their Intrastat imports. Most of the Member States mentioned that the usability of mirror data would increase if the partner ID number was collected and exchanged. Mirror data would also be a useful additional data source for enterprises characterised as below threshold trade and late response. A number of Member States indicated that the total value of their national imports based on the nationally collected data was lower than the values from the mirror exports data. Though, a few Member States found the opposite to be true. Although at the total level, the exchanged data does well to mirror the nationally collected data, at the more detailed level there are considerable discrepancies which will have an impact on the time series of specific products and trade areas should the mirror data be used,. These differences were not limited to commodity codes, but were also present at HS2 level. For the Top 25 most important commodity codes (CN8) in national data collection compared with the correspondent values from mirror data, it was possible to see that a higher number of Intrastat reporters were reported in the mirror data in comparison with the national data. Also the number of lines was higher in the mirror data. On the whole, there was an increase in the number of commodity codes reported in imports when using the mirror exports data. This follows from the higher coverage rate in exports resulting in more enterprises and transactions, which are exempted in imports. Although the level of match was good for CN8 in terms of number of codes appearing in both datasets, there was a considerable discrepancy at the value level. The differences on CN8- level were investigated by looking at the 25 most important codes for each Member State. 28

30 Some of the largest differences were clearly related to different approaches in compiling trade data for specific goods and movements such as vessels and aircraft. Another observed cause for asymmetries was the differences in classifying products. Some Member States reported that there were cases with large differences at the CN8-level, but when added up to the 4-digit level the differences were reduced. This may be due to the simulation process of the partner ID numbers, as the mirror trade might had been allocated to the wrong Intrastat reporter (i.e. the case of one to many). In particular, the Member States detected asymmetries on some products traded under processing. Differences in the methodological approach in defining taxable amount might be the reason behind this. The findings from the analysis of the tables are in line with those resulting from the indicators mentioned before Conclusions of the statistical analysis On the basis of the computed indicators, analysis of the tables and the investigations on revisions carried out during the pilot exchange of micro-data, it can be said that mirror exports data can be a very valuable additional data source for the compilation of Intrastat imports. Generally, a good coverage and a gain in additional information were achieved through using the mirror data for aggregated trade value results. Data validity and consistency, except for some minor shortcomings, were mostly good. The consistency of the indicators over a longer time frame confirmed the stability of the results. Matching national data with the mirror data at the Intrastat reporter level revealed much better results with those Member States who collected partner ID number. However, some of the other Member States who simulated partner ID number also had very good results. The match for some of the Member States though was insufficient, particularly for the number of common Intrastat reporters in both datasets, and to a lesser extent for the Intrastat reporters values. The results were confirmed by the respective summary indicators, the tables and also for the matching of the top 25 traders and the top 25 commodity codes. The convergence time, defined as the time needed for monthly figures to converge to stable final results, were quite similar both for national imports and for mirror exports. The Member States could identify considerable differences in values aggregated by commodity codes at CN8-level. It has to be considered however, as these exchanged monthly figures are still preliminary, they certainly reflect the usual asymmetries which might be partly resolved at a later stage with the release of final figures. The results also indicate that there might be methodological differences in data collection and processing across the Member States. It is worth mentioning that the analysis could not clarify if the identified discrepancies and asymmetries were systematically caused by the lack of data quality or rather by the methodological differences between the Member States. 29

31 The use of mirror data as an additional data source for Intrastat imports could enable the Member States to raise their national Intrastat threshold on imports, this has the potential to reduce the administrative burden on importing enterprises. The Re-design project, mentioned in page 9, will give information on the impact of the collection of partner ID number and the subsequent disaggregation of the declared exports data on the total burden reduction. Derived from the experience gained during the pilot exercise, the participating Member States raised the following issues as important for a smooth and efficient functioning of the SIMSTAT in a real production environment: The reference guidelines as described in the document DocMet400-SIMSTAT need to be practical, unambiguous and strictly implemented by all Member States in a future scenario of micro-data exchange, in order to guarantee the correctness of data, to achieve comparable results and to allow an automatic integration of exchanged data into the national databases. Correct use of Unique Record Identifier (URI) and of Insert/Delete sequence of the transactions is necessary. Exchange of data on specific goods and movements and metadata between the Member States would be required. Information about the existing simplification options (car-/aircraft-parts, simplification threshold, etc.) should also be exchanged. Member States also indicated that they would need a few years to fully implement and realise the benefits of the SIMSTAT approach. In summary, the results of the pilot exercise are promising as they show that the mirror data could be used as a valid substitution for at least part of the nationally collected data when the VAT numbers of importers are collected systematically. The use of mirror data opens up the possibility to limit the data collection on the imports side and thereby reduce the reporting burden on importing enterprises. The total burden reduction potential of the SIMSTAT approach is being investigated within the scope of the Re-design project. 30

32 5. ANALYSIS OF THE PROCESS OF DATA EXCHANGE Following a feasibility study and after thorough discussion at the Project Task Force and the related IT Task Force, the SIMSTAT Project Steering Committee decided to pursue a centralised data Hub model for exchanging micro-data between the Member States. This means that instead of bilateral exchanges (all countries sending data to and receiving data from all other countries), the participating Member States would send their micro-data file to a Hub, which, after some agreed processing (split of the global files into files for individual destination countries, agreed validation checks, currency conversion, etc.), would redistribute the files to the destination countries (see Chart 3). Consequently, the Member States could see only those files which were addressed to them and not the whole pool of micro-data. The SIMSTAT infrastructure comprises the hosting of the Hub and the communication component, together with the communication infrastructure created for the micro-data exchange. The Hub is hosted in a secure infrastructure segregated from the standard infrastructure of the European Commission. To exchange data with the Hub, the participants installed the communication component in their premises. The Common Communication Network (CCN) was chosen as transmission channel for the micro-data exchange. It is a highly sophisticated and secure network owned by DG TAXUD that is already in use for exchanging sensitive customs and taxation data. Chart 3: Centralised data Hub model During the pilot exercise (April - October 2015), the traffic exchange was monitored by Eurostat and DG TAXUD, both in terms of volume of data exchanged with the Hub and in terms of frequency of this exchange. The SIMSTAT communication component was installed in the participating Member States to connect to the national CCN gateway and facilitate the micro-data exchange. Installation and configuration was done by the Member States with the support of Eurostat and DG TAXUD between January and March All the installations and configurations were done in time to start with the pilot exchange in April The software was delivered and installed without any noteworthy issues. 31

33 During the period from 20 April 2015 to 9 October 2015 the 20 participating Member States exchanged micro-data for the reference period January 2013-August 2015 (32 months). Throughout this period the Member States transmitted 839 files to the Hub. After splitting the received files by destination Member States and conducting agreed validation checks, the Hub sent files back to the Member States. In terms of the volume, the Member States sent 17.4 Gigabytes (zipped) to the Hub and received 23.1 Gigabytes (zipped) from the Hub. This corresponds to 154 GB and 283 GB of unzipped files respectively. The CCN network performed without any disruptions, no delays were identified during the transmission of the files. Chart 4 shows the breakdown of the traffic volume sent by the Member States to the Hub (outgoing) in Gigabytes (zipped) and the traffic received by the Member States from the Hub (incoming). Chart 4: Total volume of data in Gigabytes (zipped) sent (outgoing) and received (incoming) by the Member States during the pilot exercise It is worth mentioning that the volume of data transferred through the Hub during the pilot exercise was higher than the total volume of data transferred from the Member States to Eurostat through EDAMIS in all statistical domains for data production. Table 7 shows that in 86% of all cases monthly micro-data for the reference period March - August 2015 were transmitted within the time interval (t+20 t+35). Only in a few cases the files were sent after t+35. Given the monthly frequency of the trade in goods statistics and tight deadlines for providing inputs to National Accounts and Balance of Payments, timeliness of the exchange of micro-data will be a crucial factor. 32

34 Table 7: Number of times files were transmitted within the following time intervals Reference period: March to August 2015 MS between t+20 and t+30 between t+31 and t+35 after t+35 AT 2 4 BG CZ DE 1 5 DK EE 1 5 EL 6 FI FR 6 HR 3 3 IT 5 1 LT 4 2 LU 4 2 LV 4 2 MT PL PT 4 2 RO 3 3 SI 5 1 SK All Member States stated that the pilot exercise clearly demonstrated the technical feasibility of exchanging micro-data using this IT infrastructure. The system fulfilled its task to transfer large data files among the Member States in a secure way even though the volume of the exchanged data was larger than it was estimated before the exercise. Despite some minor shortcomings, the current system worked effectively and efficiently within the scope of the pilot exercise. The accessibility and the performance of the CCN network, the national gateway and the communication component were highly satisfactory. Overall, the availability and the functioning of the portal (data Hub) were good. Consequently, the data exchange process worked well. All received files were correct and delivered in a timely manner. Some initial problems (missing currency conversions in some cases, problems with time stamp) were solved in the course of the exchange. Moreover, during the exercise some additional improvements were achieved. At end of the project phase the performance of the portal had improved considerably offering enhanced user-friendliness. Nevertheless, Member States mentioned some deficiencies of the system and proposed concrete measures for improvement that would ensure a smooth functioning in a real production environment. The most commonly expressed concern was related to the fact that the current system of uploading and downloading files was rather burdensome, time consuming and required a very precise follow-up. 33

35 The workload associated with downloading the large number of files was significant; particularly before the major update allowing bulk downloads, was installed. Moreover the initial lack of proper labelling of data files allowing a better identification of different versions, the lack of a notification system informing about the availability of new files for download, the problems associated with the browser updates and the incompatibility of the applet with the latest java versions were mentioned as shortcoming to be addressed. For the future production version of the system, the Member States proposed the provision of an application programming interface in order to connect the communication component to the database. This functionality would avoid the expensive use of the manual intervention and fully automate the data exchange process. The new functions of the communication component could include a command driven, automated file transfer and decoding, a notification centre for new incoming files and an additional functionality returning the status of the communication component (e.g. down, running) as well as the status of the connection to the CCN gateway. Moreover, some Member States said they would prefer receiving only one aggregated file each month, instead of multiple files broken down by sending Member States and reference periods. However, this proposal needs to be further analysed, to avoid delays in data transmission. The Member States asked for an improvement in the user-friendliness of the whole technical part of the process in order to make it more easily manageable. In order to guarantee a smooth and efficient functioning of the whole process in a monthly production environment, Member States said it would be necessary to respect agreed rules specified in the reference guidelines, which have to be adapted to a possible real SIMSTAT system environment. Overall, the Member States found that the data exchange system worked well, the technical support from the national CCN administrators as well as from Eurostat was of high-quality, and the information about maintenance and upgrades of the portal was in most cases received in time. All in all, the pilot exercise fulfilled its purpose and proved clearly that from a technical point of view the exchange of micro-data is feasible on a monthly basis, however, it will need improvement if a SIMSTAT system is to be implemented in a real production environment. 34

36 6. CONCLUSIONS AND RECOMMENDATIONS SIMSTAT was a challenging project involving the exchange of a large volume of micro-data between 20 Member States. The project required engagement, commitment and a high quality of work from all participating Member States. The actual pilot exchange of micro-data was preceded by an important preparatory phase (during the years ) resulting in agreements on a number of statistical questions, confidentiality arrangements, installation of necessary IT infrastructure and connection to a secure transmission network (for details see section 3.2). During the second and third quarter of 2015, the participating Member States successfully exchanged micro-data on intra-eu exports for the reference period January 2013-August The total volume of data sent and received by the Member States was 154 GB and 283 GB respectively (see chapter 5). During the last quarter of 2015, Member States analysed the exchanged micro-data (mirror exports) by comparing them with the collected data on imports and using pre-defined indicators and tables (see section 3.4) with the aim to assess the quality, completeness, comparability and re-usability of the exchanged data. Member States also evaluated the process of exchanging micro-data from an organisational and IT infrastructure point of view. Overall conclusions drawn after these four years of intensive work: The mirror exports data could be used effectively as a full or partial substitution of the nationally collected imports data when the VAT numbers of importers were collected and exchanged. Member States involved in the project indicated that a period of a few years would be needed to achieve the full benefit of the exchanged data. The use of mirror data for compiling intra-eu imports statistics could reduce the administrative burden on Intrastat reporters on the imports side. The pilot exercise proved that from an IT point of view the secure exchange of microdata was feasible. Some enhancements to the system would be required to ensure a fully automated exchange process. The results from the pilot exercise were obtained using defined laboratory conditions in which intrinsic limitations were applied; this should be taken into consideration when generalising these results outside of this environment. On the basis of the computed indicators, analysis of the tables and the investigations on revisions carried out during the pilot exchange of micro-data, some more detailed conclusions could be drawn: Successes Generally, a good coverage of aggregated trade values and to a lesser extent of the numbers of Intrastat reporters was achieved through using the mirror data. Better recording of exports (higher minimum coverage rate) and usually a higher concentration 35

37 of bigger Intrastat reporters in exports in contrast to many exempted traders in the imports collection have contributed to these results (section 4.1). Matching national data with partners data on Intrastat reporter level revealed much better results with those Member States who collected partner ID number. However, some of the other Member States who simulated partner ID number also showed very good results reflecting the obvious successful application of the simulation algorithm (section 4.2). The quality of mirror data was found to be good and Member States could benefit from the exchanged data by improving the coverage of their Intrastat imports. Most of the Member States mentioned that the usability of the mirror micro-data would increase if the partner ID number was collected and exchanged. Mirror data would be a useful source of additional data, especially in the case of enterprises characterised by below threshold trade and late response. The comparability of data between neighbouring Member States was often considerably high. This is encouraging because neighbouring countries are also quite often main trading partners. Long-term buyer-seller relationship and good knowledge of the traders across the border may explain this (section 4.1). "Small" Member States like LU, MT, PT or SI could identify a high share of their Intrastat reporters and even much higher share of the trade in value in the data received from so-called big Member States like IT, FR or DE (section 4.1). As regards the level of gain in additional information through the use of partner data, the analysis showed remarkably good results for many Member States. Part of the gain in information may relate to those traders which could be identified in the mirror datasets but were not included in the national collection because they were below the national exemption threshold. Although this information could enrich the current data availab le for traders below the exemption threshold, the gain in terms of value was relatively lower because of the relatively small trade values for those traders (section 4.3). The time needed for monthly figures to converge to stable final results, were quite similar both for national imports and for mirror exports (section 4.6). Data validity and consistency, except for some minor shortcomings, were fairly good. It could be concluded that many validity errors were simply due to unclear data specification and failures in the implementation of the prescribed guidelines (section 4.7). The analysis carried out by individual Member States highlighted a potential alternative method of calculating intra-eu imports statistics using a mixture of nationally collected data and received mirror data in order to ensure a smooth transition and mitigate break in time series. Further elaboration at detailed level (product and partner country) would be needed. The pilot exchange revealed a higher number of Intrastat reporters available in mirror data in comparison with national data. Also the number of lines was higher in mirror data. The use of the mirror exports micro-data in compilation of imports statistics meant a gain in the number commodity codes reported compared with imports compiled only from nationally collected data. There was also data for a higher number of Intrastat reporters (although the gain in value from these extra Intrastat reporters was relatively small). This 36

38 follows from the higher coverage rate for the intra-eu exports flow, resulting in more enterprises and transactions, which are exempted in imports. The Member States found that the IT infrastructure for the data exchange system worked well and the technical support from the national CCN administrators as well as from Eurostat was of high quality. Risks Although the difference between imports and mirror exports at total level was not very high, there were considerable differences for individual Member States at both HS2 and commodity code level. Member States could identify considerable differences between the declared values of matched traders and commodities used to classify the goods by product codes at CN8- level. The differences on CN8-level were investigated by taking a closer look into 25 most important codes for each Member State. Some of the largest differences were related to different approaches in compiling trade data for specific goods and movements such as vessels or aircraft as well as for some products traded under processing. The match for some of the Member States was insufficient, particularly for the number of traders, but less so for the traded values. Problems regarding data completeness, coverage, consistency at detailed level and asymmetries have been detected. In some cases they seem to result from diverging methodological approaches in the sending and receiving Member States. For a real SIMSTAT production environment the timeliness of data exchange will play a critical role. A strict respect of the deadlines will be necessary. Further enhancements of the IT system will be needed to help fully automate the microdata exchange. These requirements will necessitate further IT investments both from Eurostat and the Member States. Experience gained during the pilot exercise The participating Member States raised the following issues as important for a smooth and efficient functioning of SIMSTAT in a real production environment: The reference guidelines need to be practical, unambiguous and strictly implemented by all Member States in a future scenario of micro-data exchange, in order to guarantee the correctness of data and to allow an automatic integration of exchanged data in national databases. Data on specific goods and movements, metadata and information about simplification options and metadata should also be exchanged. Further methodological harmonisation would be necessary to define common statistical treatments appropriate for specific flows, like triangular trade, goods for processing, etc. For the future production version of the IT system, the Member States proposed some improvements, mainly to minimise manual intervention and fully automate the data 37

39 exchange process. In general, the Member States said there was a need to improve the user-friendliness of the technical part of the process in order to make it more easily manageable. Member States who have been involved in the project said that more preparatory work, both from statistical and IT perspectives, would be necessary before implementing SIMSTAT in real production environment. Although the risks of the ESS.VIP.SIMSTAT project could be successfully mitigated, the risks related to the implementation of the SIMSTAT approach in real production will need to be addressed. Depending on the decision of the ESSC on the future orientation for Intrastat, one should take the following issues into consideration: Agreement on statistical confidentiality by securing mutual trust between statistical authorities, preserving public trust in their protection of confidential data and ensuring respondent and political support by means of appropriate regulations Clear rules on who has the right to ask for confidentiality, when confidentiality has to be granted, in which cases a receiving Member State has to also keep the corresponding information on the imports side confidential and any secondary suppressions Setting up of a mechanism in order to prevent and punish misuse of confidential partner data Definition of the core principles governing micro-data exchange Establishment of common and verifiable IT security standards and infrastructure for confidential data protection. Adjustment of the IT infrastructure to real life production condition Design and implementation of tailored communication actions in order to understand stakeholder views and convey the right messages on the benefits and the ESS experience of the exchange of micro-data Work towards a positive perception of the exchange of micro-data by policymakers, survey respondents and users of European statistics Necessity to review and reinforce the relevant legal frameworks. The project was successfully conducted and all its objectives were achieved thanks to the close collaboration among the ESS partners, commitment of the participating Member States and clear guidance given by the ESSC and the Vision Implementation Group. Regular micro-data exchange between Member States creates an opportunity for realising the ESS vision for changing the production method of European statistics within the ESS. 38

40 Annex 1: Overview of Member States involvement Member States Project Steering Committee Project Task Force IT Task Force Pilot exchange ESSnet1 ESSnet2 AT X X X X X X BE X BG X X X X X CY X CZ X X X X X DE X X X X X X DK X X X X X EE X X X X X EL X X X X X X ES FI X X X X X X FR X X X X X X HR X X X X HU X IE X IT X X X X X X LT X X X X LU X X X X X LV X X X X MT X X X X X NL X X PL X X X X X PT X X X X X RO X X X X X SI X X X X X SK X X X X X X SE X UK X X X X X Total

41 Annex 2: ESS.VIP.SIMSTAT governance structure 40