T HE EUROPEAN COURT OF AUDITORS D EFINITION & T REATMENT OF DAS ERRORS

T HE EUROPEAN COURT OF AUDITORS D EFINITION & T REATMENT OF DAS ERRORS E N G L II S H

Introduction 4 Error definition & classification concerning the different DAS Sources 5 General situation 5 Weaknesses addressed by the new guidelines for DAS errors 6 Basic definitions and principles to be applied in the context of substantive testing 8 Reference framework for determining legality and regularity of underlying transactions 8 Underlying transactions 9 Errors 9 Classification and treatment of errors 10 The starting point 10 The different steps to classify an error 12 Step 1: Analysis of whether legal requirements are affected 12 Errors affecting conditions for payments 12 Errors affecting other compliance issues 14 Step 2: Analysis of whether errors are quantifiable 15 Quantification as key element for the assessment of the financial impact of errors 15 Quantifiable errors 15 Non-quantifiable errors 16

Step 3: Analysis of whether errors are serious, limited or insignificant 17 Differentiation of quantifiable errors 17 Differentiation of non-quantifiable errors 17 Step 4: Analysis of whether errors are systematic 18 Criteria for determining the systematic or non-systematic character of errors 18 Professional judgement 18 Step 5: Analysis of the overall impact of errors 18 General principles 18 Preparation of indicators 19

Introduction Pursuant to Article 287 of the Treaty on the Functioning of the European Union (TFEU), the Court of Auditors provides the European Parliament and the Council with a Statement of Assurance concerning the reliability of the accounts and the legality and regularity of the underlying transactions (the DAS). The aim of the work on the reliability of the accounts is to obtain sufficient evidence to conclude on the extent to which revenue, expenditure, assets and liabilities have been properly registered and that the annual accounts of the European Union faithfully reflect the financial position at the end of the year, and the results of their operations and cash flows for the year then ended. The aim of the work on the legality and regularity of the underlying transactions is to gather sufficient evidence of a direct and indirect nature so that the Court can reach a conclusion, with reasonable assurance, about whether transactions were in accordance with the intentions of the legislator* and the budget authority, and amounts were correctly calculated. The essential question to be answered is: What constitutes an error affecting the audit opinion, and therefore needs to be taken into consideration in the framework of substantive testing, in order to come to an informed and balanced professional judgement. * As reflected in the various types of EU legislation, i.e. regulations, directives, decisions, etc. The audit work on the reliability of the accounts was based, since the first DAS concerning the financial year 1994, on a stable methodology and treatment of errors which followed international standards and proved its value. Necessary adaptations over time, for example due to the introduction of accruals based accounting, were introduced in the respective audit plans. 4

Error definition & classification concerning the different DAS Sources General situation The Practical guidelines for DAS audit work after 2005 adopted by the Court in the beginning of 2006 provide a framework for the definition and treatment of errors/deficiencies concerning supervisory and control systems and management representations contained in the Directors- General's annual activity reports and declarations and the Commission's synthesis report. Furthermore, they provide guidance on the use of the work of other auditors. The diagram below illustrates the interaction of the different DAS sources as well as the iterative process and the interrelation of errors: Diagram on the interrelation of the main DAS sources As illustrated in the diagram below, in the framework of substantive testing it needs to be considered whether the quantifiable or non-quantifiable error or other compliance issue is by its origin purely isolated (error in the sample) or also system related (systemic/systematic). This assessment may also serve as an input to corroborate the analysis of the supervisory and control systems. In general, the impact of errors caused by systems weaknesses is bigger than in the case of isolated accidental errors. Therefore, the system related element of errors requires further assessment in order to estimate as precisely as possible the financial consequences concerning the total population and to identify the necessary corrective actions to remedy the weaknesses in the supervisory and control system. Analysis of supervisory and control systems Interrelation Substantive testing of transactions AARs & Declarations Other auditors work Decision tree Decision tree System deficiencies Mutual corroboration Errors and other compliance issues (quantifiable and non-quantifiable) System related Origin of errors Isolated 5

Error definition & classification concerning the different DAS Sources Weaknesses addressed by the new guidelines for DAS errors Experience showed that the previous guidelines needed revision. Especially the distinction between substantive errors and formal errors was sometimes misleading and difficult to implement. Previous guidelines Some legality/regularity errors have measurable effects on the amount of the underlying transactions financed from the EU budget. These errors were called substantive. Other legality/regularity errors consist of violations of the regulatory and control mechanisms but do not have a measurable effect on the amount of the underlying transactions financed from the EU budget. These errors were called formal. In fact, while the rate of substantive errors found in the sample was applied to the entire population, for formal errors only the frequency was taken into consideration. This clearly had as an effect that formal errors, although being often very serious (e.g. non-respect of call for tender procedures, non-respect of obligatory minimum control), were generally considered as formalities and therefore less important than substantive errors (even if the latter only have a marginal financial impact). Furthermore, following strengthened eligibility rules requiring the retention of clear and complete documentary evidence, the absence of accounting information had sometimes to be considered as 100% substantive error under the previous guidelines. However, in certain cases this treatment probably gave an unrealistic indication about (the potential impact of) the problems (e.g. evidence exists that an action took place, but is not cost-efficient to apply alternative procedures in order to quantify the eligible expenditure, in particular concerning overheads). The auditor must exercise professional judgement about the seriousness of missing information. Moreover, new elements are introduced in the legal framework or have gained in importance, which need to be addressed, although they do not have a direct impact on the payments made. In fact, they are important in the context of legality and regularity of underlying transactions and have therefore to be taken into consideration. 6

Error definition & classification concerning the different DAS Sources A further weakness consisted in the fact that the previous guidelines did not give clear indications concerning the definition of insignificant or trivial errors. Thus, often the entirety of rules was included in the questionnaires applied for the audits on-the-spot, without taking into consideration that a possible infringement will have no or only a limited (potential) financial impact, and is therefore not relevant in the framework of the DAS. A clear focus on essential issues will help to concentrate on the most critical problems and contribute to a reduction of the DAS workload. Taking into consideration the analysis above, the revised guidelines clarify the definition and treatment of errors, which are identified on the basis of substantive testing of representative samples covering the different policy areas. 7

Basic definitions and principles to be applied in the context of substantive testing Reference framework for determining legality and regularity of underlying transactions The purpose of the part of the Statement of Assurance relating to the legality and regularity is to inform the recipients of the DAS whether the Court has reasonable assurance that the transactions underlying the annual accounts are in accordance with the intentions of the legislator and the budget authority. In view of the difficulty within the EU sphere of marking out a precise boundary between the concepts of legality and regularity, and given that the consequences of noncompliance with these rules are exactly the same within the DAS framework, no distinction is made between these two concepts. The Court verifies that the transactions are both legal and regular. A judgment about the seriousness of errors (a percentage or an absolute amount stakeholders are willing to accept) is a priori required and needs to be allocated to both the underlying transactions and to the overall population. See also ISA 320, paragraph 7 For consistency reasons, the same limits apply for both the individual transactions (being the elementary reporting entity in substantive testing) and the relevant population (being the reporting entity to draw overall audit conclusions and the audit opinion). For instance, if all transactions containing errors individually remain below the 2%- ceiling the overall error rate for the sample also remains below this ceiling. In other words, an adverse opinion is only possible if errors above 2% are identified. It is therefore necessary to make a distinction in order to be able to assess whether risk is adequately managed in specific areas. In principle, the compliance with all Community provisions of a regulatory or contractual nature (Treaty, Regulation, Directive, Decision, financial protocol, financial agreement, contract, etc.) and with necessary national provisions can be relevant to a judgement on the legality and regularity of a transaction. National provisions of this type are, in principle, formally approved or provided for by Community decisions incorporating them, de facto, into the framework of valid Community provisions. The auditor should however reach a balanced judgement on whether a transaction should be considered as legal and regular, i.e. fulfilling the intentions of the legislator and the budget authority, and correctly calculated. Official memoranda and internal organisational directives do not form part of the regulatory framework. Should national provisions not conform to Community regulations, this constitutes an error. 8

Basic definitions and principles to be applied in the context of substantive testing Underlying transactions Underlying transactions refer to revenue and expenditure operations, which have been or should have been registered in the EU s annual accounts. All transactions of this type are to be taken into consideration, from the level of the central management of the EU institutions to the level of the final beneficiaries or those providing the EU with own resources. This includes analysis of every act, procedure, process or document, of a legal, administrative, financial or banking nature, as well as any physical events or factors, which underlie them to the extent necessary to form a robust judgement. Errors Transactions (or parts thereof) and/or actions linked to them which have not been carried out in accordance with the legal and regulatory provisions applicable are considered as errors. other compliance issues which have no direct impact on the payment made but imply a financial risk and/or could lead to financial corrections to be borne by Member States or fines to be paid by final beneficiaries (e.g. nonobservance of management and control procedures or best practices imposed by EU provisions). Fraud by its very nature is not necessarily detected by means of usual audit procedures carried out in accordance with generally accepted auditing standards. The Court is therefore not able to give an assurance that cases of this type do not exist. Cases of (presumed) intentional irregularities that are detected are treated as DAS errors and communicated to OLAF. Errors that are detected and corrected on the initiative of the managing body before the closure of the accounts for the financial year and independently of the checks carried out by the Court, are not taken into account in the DAS, since they demonstrate the efficient working of the EU system and no longer affect the accounts of the financial year. A distinction is to be made between legality/regularity errors affecting: the conditions for payments directly (e.g. non-respect of eligibility rules or of obligations concerning recoveries); and 9

Classification and treatment of errors The starting point In the context of the audit of the legality and regularity of underlying transactions, errors are relevant for the DAS if they, individually or aggregated with other errors, would reasonably affect the decisions of the addressees of the audit opinion. The Court s policy concerning the relevance of errors acknowledges that it would be unrealistic to assume that no errors occur in practice and, consequently, a degree of tolerance regarding the appropriate level of accuracy is to be considered acceptable. See also the Court's opinion No 2/2004 on the single audit model (and a proposal for a Community internal control framework), OJ C 107, 30.04.2004, p. 1-20. Following the decision tree below, errors detected during DAS audits should therefore be analysed in a five-step-process in order to determine whether and to what extent they are relevant for the Statement of Assurance (left hand side of the dotted line) or not (right hand side of the dotted line). The errors under the latter category are not relevant to form the DAS opinion (non-significant or trivial errors) and are not to be reported under normal course of events. Most error types formerly defined as simple formal errors belong to this category (such as short and inconsequential delays in payments or absence of non-essential documents). Previous guidelines The degree of seriousness was determined by the importance of the principle, which had been infringed and by the gravity of the facts and the actual or potential consequences. The decision tree has to be applied for each anomaly, which is identified during the audit work directly linked to a sample. Errors, which have not been identified based on audits directly linked to a representative sample (but during supplementary work carried in the framework of special reports etc.) are to be considered as known errors. For these known errors, the main part of the decision tree can be applied by analogy. However, the assessment of their financial impact has to follow a different pattern (see the last two paragraphs below). If, in an audit of the legality and regularity of the underlying transactions, the auditor is unable to determine whether non-compliance has occurred because of limitations imposed by the circumstances rather than by the entity, the auditor should not consider the effect as an error ( non-opinion ) See also ISA 200 and ISA 700 as regards disclaimers of opinion. 10

Classification and treatment of errors Decision tree for classification of errors detected during substantive testing Error in connection with samples 1 Condition for payment affected? NO Other compliance issues concerned bearing the risk of financial corrections or penalties? NO YES Error quantifiable? YES YES Error rate higher than reporting threshold? (>0.5% de minimis clause) NO Error material by nature or context? NO NO YES YES Exceeding the threshold of 2%? Is the whole payment or an important part of the payment affected? NO Is a significant part of the payment affected? NO YES NO YES YES Serious error ( 2%) Limited error ( 0.5% < 2%) Serious error Limited error Insignificant error Precise error rate to be taken into consideration when calculating the most likely error 2, 3 To be taken into consideration when determining the frequency of nonquantifiable errors affecting the whole or an important part of the payment 2 To be taken into consideration when determining the frequency of nonquantifiable errors affecting a significant part of the payment 2 1 2 3 DAS relevant errors in the context of substantive testing DAS errors not normally reported Known errors are treated in the same way, but not extrapolated. In case that the error is systematic, this should be taken into consideration when assessing the financial impact of the deficiency If the most likely error is below the materiality threshold, professional judgement is necessary which takes into account the level of quantifiable errors and the frequency of non-quantifiable serious errors. 11

The different steps to classify an error Step 1: Analysis of whether legal requirements are affected Errors affecting conditions for payments At national / regional/ intermediate level, a wrong interpretation or application of the respective legal basis may lead in individual cases to wrong amounts paid, incorrect cofinancing rates, deductions not permitted from payments, transfer of funds to erroneous beneficiaries, etc. The first part of step 1 of the decision tree focuses on the question of whether the error identified affects the conditions for payment. In general, payment conditions are concerned when elementary legal requirements of the respective aid scheme or contract are affected. Most errors types formerly defined as substantive errors belong to this category (such as non-respect of eligibility criteria or non-deduction of prior payments). For practical reasons, the assessment can be based on the following question: Should the amount paid and/or the contracted body have been different if the correct procedures have been followed? An answer has to be provided for all levels of the audit trail. At Commission level, this refers to errors as regards the existence of a legal basis (regulation or directive), financing decision, legal and budgetary commitment, regular validation and/or authorisation of the payment etc. If an error substantially affects the first stages of the financial management cycle (e.g. irregular legal and/or budgetary commitment), this will have a direct impact on all following stages 12

The different steps to classify an error At final beneficiary/recipient level the condition for payment requires in principle the compliance with eligibility and reality criteria: The project, activity or beneficiary should be eligible by nature (in most cases projects or beneficiaries have to fulfil specific eligibility conditions as for example age, size of activity, minimum quality requirements, minimum professional qualifications, etc). Recoverable VAT, other taxes and charges, administrative costs, depreciation and costs not directly linked to the activity are in general ineligible. In the area of Agriculture, Statutory Management Requirements (SMR) and Good Agricultural and Environmental Conditions (GAEC) have to be respected. In case of non-compliance with those rules, payments to the final beneficiaries may be reduced. It is stipulated in the relevant Commission regulation (1122/2009) that compliance requirements are not regarded as eligibility criteria, but as they in substance affect the payment, the Court regards such infringement as errors affecting conditions for payment. The activities should be carried out or expenditure should be incurred within the eligibility period. Moreover, eligible expenditure should be justified by appropriate supporting documents (e.g. receipted invoices or other accounting documents of equivalent probative value). The activity performed or the project implemented should physically exist (number of trainees following the training course, area of land, number of livestock owned, length of road constructed, etc.) and the declared expenditure has actually been incurred. 13

The different steps to classify an error In the context of checking whether the conditions for payment are affected or not, it is necessary to stress that final beneficiaries/recipients often declare higher physical quantities or more eligible expenditure than required, i.e. they create a buffer above the trigger point necessary to release a payment. In these cases, errors found are deducted first from the buffer of eligible physical quantities or expenditure declared. Only if the error amount exceeds the buffer, the conditions for payment are affected and an error needs to be reported (error = trigger point minus remaining eligible physical quantities or expenditure). As far as errors concerning negative amounts are concerned, a distinction must be made between: those which arise from the final beneficiary's underestimating the amount to which he was entitled, which are not taken into consideration in the DAS; and Errors affecting other compliance issues The second part of step 1 of the decision tree focuses on the question of whether other compliance issues are concerned. These other compliance issues are specific conditions to be fulfilled, which result from related policy objectives and related legal obligations or systems requirements. However, the non-compliance with these requirements does not mean that the condition for payment is affected but it implies a financial risk and/or could lead to financial corrections to be borne by Member States or fines to be paid by final recipients of grants. Thus, the non-observance of other compliance issues may be considered as material by nature or context, although it has no direct financial impact on the payment itself. In principle, most of the former serious formal errors fall under this category (such as non-respect of publicity requirements or non respect of obligatory minimum control). Previous guidelines The degree of seriousness was determined by the importance of the principle which had been infringed and by the gravity of the facts and the actual or potential consequences. those which are the fault of the Commission or those bodies responsible for the implementation of the actions, which are taken into consideration in the DAS. 14

The different steps to classify an error concrete examples for (possible) errors can be derived from strengthened rules in the area of shared management i.e. infringements relating to publicity or system requirements in the area of Structural Actions, which represent a financial risk and/or could lead to financial corrections to be paid by Member States. Possible decisions concerning the extent to which these issues are covered in the context of the DAS should follow the procedure as set out in the section below. In many cases, it depends on the professional judgment of the competent audit unit to decide whether a criterion is considered to be a condition for payment or has to be treated as an other compliance issue. If the error detected does neither affect any condition for payment nor any other compliance issues, the error is considered as a insignificant/trivial error and has not to be reported. Step 2: Analysis of whether errors are quantifiable Quantification as key element for the assessment of the financial impact of errors Step 2 of the analysis asks whether the error found is quantifiable or not. The general evaluation of the financial impact of errors for the purpose of the Statement of Assurance is based on a precise quantification of each error detected. However, in some cases the quantification of an error is not possible or only at prohibitive costs. Therefore, errors affecting payment conditions should only be quantified if reasonable assurance exists, or can be obtained on the basis of cost-efficient procedures, concerning the amount at stake. This is of particular importance, since, when extrapolating the results of a MUS-type sample, even the smallest error will be of some significance (see step 5 below). Quantifiable errors Quantifiable errors are those which have a direct and measurable financial impact on the amount of the underlying transactions financed from the EU budget (e.g. reimbursement errors resulting from ineligible items included in cost statements or serious non-compliance with public procurement rules ( eligibility errors ), quantities declared not corresponding to reality ( accuracy errors ), cost not supported by an invoice or some other document of probative value ( occurrence error )). By definition, errors affecting other compliance issues do not fall under this category because they have no direct impact on the payment in question. 15

The different steps to classify an error The percentage error and the monetary value of the quantifiable error discovered must be calculated in relation to the recorded value of the transaction at the level concerned. The quantification of the error concerns the EU contribution. In cases of co-financing (mainly with the Member States), a distinction must be made between the part financed by the EU and the part financed by third parties. Quantification depends on a comparison between the actual value of the transaction and the value to which it would have amounted if it had been conducted in accordance with applicable provisions. The difference thus calculated is expressed as a percentage under-estimation or over-estimation of the value of the transaction recorded. In case of quantification of cross-compliance errors, the national system for the reduction of payment is to be used to determine the percentage of errors. Non-quantifiable errors Errors which concern the non-observance of a condition for payment or other compliance issue, but which do not have a direct financial impact on the amount of an underlying transaction (e.g. less serious non-compliance with public procurement rules not frustrating the objectives of the procedure, non respect of deadlines set by the regulations, delays in the transfer of EU funds to final beneficiaries, absence of or irregularity in the authorisation or validation process of the payment); Errors which concern the non-observance of a condition for payment that may have an impact on the amount of underlying transaction, but this effect is not measurable (e.g. absence or late availability of cost-benefit analysis for a project, bank guarantee missing or insufficient); Errors which concern the non-observance of a condition for payment having a direct financial impact on the amount of underlying transactions but where it is not possible to quantify the exact amount of the error due to a lack of information (e.g. inappropriate supporting documentation for expenditure declared in cases where evidence exists that an eligible action took place). Within the group of nonquantifiable errors, a distinction should be made between: For non-quantifiable errors, it is by definition not necessary to establish the percentage error but merely to indicate the amount of the transaction concerned. Should the non-quantifiable error only affect a specific part of the underlying transaction (e.g. several reports are missing from a list of beneficiaries who have to submit implementation reports), the amount concerned serves to determine whether the error is serious or limited (see step 3). 16

The different steps to classify an error Step 3: Analysis of whether errors are serious, limited or insignificant Differentiation of quantifiable errors Step 3 of the analysis in case of quantifiable errors leads to a differentiation in three categories. It is the Court's policy that the materiality by value threshold is set at 2% of the value that most reasonably reflects the level of financial activity or subject of the audit. Furthermore, when preparing the 1995 DAS, the Court introduced a de minimis rule below which, unless they resulted from a deliberate act, quantifiable errors would not be considered and would therefore not be recorded for the DAS. These thresholds were set at the following levels: a rate of 0,5% for errors which can be extrapolated (because they were identified on the basis of substantive testing of representative samples); an amount of 10 000 Euro for known errors. All quantifiable errors equal to 0,5% or more of the value of the transaction concerned will be taken into account in the extrapolation of the (most likely) error (see step 5 Preparation of indicators ). However as the Court intends to continue to comment on the extent to which the errors it identifies are limited, i.e. between 0,5% and 2% of the value of the transactions concerned ( limited errors ) or serious, i.e. errors representing more than 2% ( serious errors ) these categories are recorded separately. Errors below 0,5% are considered to be insignificant or trivial. Therefore they are not to be recorded for the DAS. The de minimis rule of 0,5% should not be applied for individual errors exceeding one million euro. Differing ceilings in terms of absolute amounts may be fixed on the basis of financial volume and policy area concerned. Differentiation of non-quantifiable errors Step 3 of the analysis in case of nonquantifiable errors involves considerations other than simple value. These qualitative assessments are based on the concepts of material by nature and material by context. Material by nature is related with inherent characteristics and concerns such issues where there may be specific disclosure requirements or high political or public interest. Material by context concerns items that are material by their circumstance. The degree of seriousness is determined by the importance of the principles which have been infringed, and by the gravity of the facts and the actual or potential consequences. 17

The different steps to classify an error If the whole or an important part (above 10%) of the payment is affected by the problem, the error in question is defined as being serious. If the deficiency concerns a significant part of the payment (above 2,5% and up to 10%), the error in question is defined as being limited. If the problem affects less than 2,5% of the payment, the error is defined as insignificant or trivial and not to be recorded for the DAS. When an error is defined as being systematic, its causes must be identified. In addition, if it is at all possible to estimate the impact of the error for the financial year under review, the total value of the population affected by the error must be indicated as precisely as possible excluding the known value of the error in the transaction sampled. Professional judgement Step 4: Analysis of whether errors are systematic Criteria for determining the systematic or non-systematic character of errors Step 4 of the analysis asks (implicitly) for the systematic or non-systematic character of errors detected. In this context, it is necessary to clarify whether the error is isolated (e.g. an arithmetical inaccuracy caused by human error) or systematic (e.g. entry by a member of staff of a wrong factor in official instructions or in a database used to calculate a number of payments). In some cases it will be difficult to make this distinction (e.g. an arithmetical error repeated several times by an employee). The threshold below which an error is considered to be an isolated one and above which it is considered to be systematic depends on the auditor s professional judgement. Additional work (analysis of other transactions) will often be necessary to enable the auditor to conclude that the error is systematic. Step 5: Analysis of the overall impact of errors General principles The assessment of the overall impact of errors will be determined by qualitative and quantitative aspects. The qualitative evaluation has to take into consideration the importance of the rules, which have been infringed, and the gravity of the facts and the actual or potential consequences. The quantitative assessment is in particular, based on indicators, such as frequency of errors and (most likely) error rate. 18

The different steps to classify an error Preparation of indicators Quantifiable errors identified during the audits of representative samples are the basis for the extrapolation of the (most likely) error rate. Extrapolation will consist of multiplying the cumulative total level of errors detected in the sample by the value corresponding to the sampling interval, which, in principle, means applying to the entire population the rate of error found in the sample examined. Errors identified during supplementary work outside of the scope of representative samples ( known errors ) are not projected to the entire population. They are taken into consideration on the basis of the absolute amounts or numbers of errors involved. The sampling interval is obtained by dividing the financial volume of the population to be audited by the number of transactions contained in the sample. For non-quantifiable errors identified during the audits of representative samples, the frequency of serious and limited errors detected in the sample will be applied to the entire population. When presenting statistical information on the overall situation of serious and limited errors, the numbers of nonquantifiable and quantifiable errors are aggregated. 19