CASEN 2011, ECLAC clarifications 1 1. Background on the National Socioeconomic Survey (CASEN) 2011 The National Socioeconomic Survey (CASEN), is carried out in order to accomplish the following objectives: Provide regular information on demographic, education, health, housing, work and income aspects affecting the status of households and of the population, especially the population living in poverty and those groups which have been designated as priorities for the purposes of social policy. In particular, CASEN is intended to estimate the magnitude of poverty and income distribution; identify lacks and the needs of the population in the areas mentioned; and assess the gaps separating the various social segments and geographical areas. Evaluate the impact of social policy: to estimate the coverage, targeting and distribution of fiscal expenditure for the main social programmes of national reach aimed at households according to their level of income and the impact of such expenditure on household incomes and their distribution. The CASEN survey examines households living in private dwellings located throughout Chile, with the exception of very isolated or inaccessible areas, and the individuals who make up these households. As well as the nationwide level, CASEN examines regions and the urban and rural dimensions. The CASEN survey has been conducted by the Ministry of Social Development every two or every three years since 1985. So far, surveys have been taken in 1985, 1987, 1990, 1992, 1994, 1996, 1998, 2000, 2003, 2006, 2009 and 2011 (www.mideplan.cl). 1 Prepared by the Statistics Division of the Economic Commission for Latin America and the Caribbean (ECLAC). 1
Responsibility for defining the thematic scope of the survey, designing and setting the questions included in the questionnaire, conducting the actual data capture and processing the information resides with the Ministry of Social Development of Chile. The Ministry is also responsible for issuing the results. The Ministry tenders out the fieldwork and the sample design. On this occasion (CASEN 2011) data capture was conducted by the Microdata Center of the Faculty of Economics and Business of the University of Chile. The National Institute of Statistics (INE) was responsible for designing and selecting the sample; mapping the sample and preparing the sample expansion factors. The role of ECLAC The role of ECLAC is confined to the following: 2 1. Correcting and adjusting income variables, specifically measuring and correcting measurement errors which occur in the process of data collection, with regard to labour income, social security benefits, capital income and imputed rent. (MDS-ECLAC agreement, Consultancy for the process of correction and adjustment of income for the twelfth National Socioeconomic Survey, CASEN 2011) 2. Constructing the income totals corresponding to each recipient and each household, by aggregating the different types of income captured in the questionnaire, which have already been duly adjusted. The Ministry of Social Development (MDS) then uses these income aggregates to estimate indicators of poverty and income distribution. The part of the process of the CASEN survey in which ECLAC is involved begins with the receipt of the CASEN database, on which the Ministry has already conducted a consistency analysis. ECLAC immediately performs a 2 A fresh agreement between ECLAC and the Ministry of Social Development is drawn up on each occasion. 2
new consistency analysis of the database, centred on the income variables, and conveys possible inconsistencies to the Ministry, so that the Ministry can make the necessary adjustments. The tasks carried out after the database has been purged of inconsistencies are as follows: 1. Correction for non-response: income is imputed for individuals who report having an income from wages or salaries, independent work or pensions and retirement benefits, but who do not report the amount thereof. 2. Income figures are adjusted for underreporting, based on a comparison of the amounts arising from CASEN and those shown in the national accounts. 3. Computation of aggregates of personal and household incomes. ECLAC then returns the CASEN micro-database to the Ministry with the adjusted income and aggregates for each observation, with a fully transparent account of the methodology employed, 3 and there ends its involvement. The Ministry then conducts the estimates and releases the poverty indicators and other data arising from the survey. The main criterion guiding the work of ECLAC is to make the income estimates as comparable over time as possible. It must be clarified that the survey questionnaires are usually altered between periods, sometimes to reflect changes observed in the type and characteristics of existing income sources, and sometimes to improve the capture of income sources. 3 As part of the agreement, ECLAC prepared and conveys to the Ministry a detailed technical note containing the methodology. 3
2. Differences in the poverty estimated conducted by the Ministry of Social Development and ECLAC starting in 2009 From 2009 on, the poverty estimates conducted by the Ministry of Social Development do not coincide with those which ECLAC prepares regularly for Chile in the framework of its poverty calculations for a wide array of countries in the region. This situation has arisen because of differences in the way the poverty line has been updated since 2007. Until that year, both the indigence line and the poverty line were updated according to changes in food prices. From 2007 onwards, however, ECLAC has calculated the variation in the indigence line using the variation in the consumer price index (CPI) for foodstuffs, whereas the part of the poverty line that corresponds to expenditure on non-food products is updated using the variation in the corresponding CPI. As noted in Social Panorama of Latin America 2008 (p. 45-46), the reason for changing the criterion for updating poverty lines was the significant increase in food prices observed across the region in 2007 and 2008, which far exceeded inflation for other goods and services. In fact, in the case of Chile, in the three-year period 2006-2009 the CPI for food rose by 32.4% but the CPI for other goods rose only 6.4%. A similar pattern, albeit with smaller differences, occurred between 2009 and 2011, when the rises were, respectively, 13.1% and 2.9%. Although ECLAC and the Ministry for Social Development employ the same methodologies for the rest of the parameters, this difference in the method for updating the poverty line generated divergences in the results starting with the calculations for 2009, as may be seen below: 4
CHILE: EVOLUTION OF POVERTY AND INDIGENCE, 1987 2011 (Percentage of the population) Source: Economic Commission for Latin America and the Caribbean (ECLAC), on the basis of official figures. a MSD: Ministry of Social Development of Chile. 3. Changes made by the Ministry of Social Development in the 2011 CASEN survey Three main points must be considered in relation to the 2011 version of the CASEN survey and the construction of a poverty estimate that is comparable with the historical series. First, the survey s statistical design is based on the new master sampling frame developed by the National Institute of Statistics (INE), which has differences with respect to that used in the previous rounds of CASEN. The survey also uses different criteria for sample selection and size, especially, in the latter case, with respect to its distribution by region of Chile. As is usual in these cases, the institution responsible for the survey, i.e. the Ministry of Social Development, should report on the evaluation of both the 5
probable differential effect of those changes on the value of the estimators, and the sampling errors associated with the survey s main variables. Second, the Ministry of Social Development decided to use two independent subsamples for the fieldwork for the 2011 CASEN survey. The first was conducted in the field between 17 October 2011 and 21 November 2011, with 27,770 interviews. The second was taken between 22 November 2011 and 22 January 2012 and included 59,084 interviews. In order to maintain comparability with the historical series of poverty estimates, the results processed by ECLAC and released to date use only the second subsample, since it is the only one in which both the period of fieldwork and the reference period for income are comparable with the previous rounds of the survey. The decision to use a longer fieldwork period and to divide the sample into two representative subsamples was taken entirely by the Ministry of Social Development. Third, the questionnaire for CASEN 2011 contained certain modifications with respect to the measurement of income. As a result, it was necessary to analyse the behaviour of the variables and make some assumptions for certain income flows in order to generate aggregates comparable with those of 2009 and earlier years. Two types of income are particularly significant in this respect. These have to do with the questions on Asignación Social ( social benefit ) and on the labour income of unpaid, unemployed and inactive family members. (a) Asignación social (y23) The CASEN 2011 survey included a question (y23) on income from what is known as Asignación social ( social benefit ). This is a new subsidy with four possible components: a baseline grant, a grant for complying with certain counterpart commitments (e.g. children s health check-ups), a grant for working women and a special annual grant (in November). This benefit was in the process of being defined when the questionnaire was being 6
designed, and one of its components (the annual grant) was in fact set up after the questionnaire construction was finalized. As a result, the reporting in the questionnaire makes no separate provision for the monthly element and the annual element paid during the month when the survey was taken (the survey enquired only into the aggregate amount received during the month). In addition, the format of the payment voucher, which the survey instructions directed enumerators to request, did not identify retroactive payments. It identified only the total amount received by the family group, with no breakdown by type of benefit or period to which it corresponded. In the set of grants that make up the family benefit, the annual grant paid in November 2011 and only that grant should be divided into 12 monthly payments, in order to compute only that part registered as disposable monthly income. This is the way the Bonos de Apoyo a la Familia ( family support grants ) paid in March and August 2009 were dealt with and it is also the procedure applied to other regular income which is received once or twice per year, such as gratuities, performance bonuses, payments for rental of agricultural property, etc. However, given the problem with the questionnaire design, ECLAC was unable to apply this procedure, leaving only the following alternatives: (a) To break down into 12 months the whole value reported for the social benefit, which would have underestimated the actual monthly income (and hence overestimated poverty); (b) Not to break down the value reported, which would have produced the opposite bias, that is an overestimation of income (and hence an underestimation of poverty); (c) A more drastic alternative, since the variable could not be properly computed, was to exclude it completely from the aggregate household income, thereby maximizing income underestimation. 7
A fourth alternative was also considered at the suggestion of the Ministry of Social Development: attempting to separate the various grants reported by survey respondents by using algorithms to exogenously impute each value as a function of the characteristics of households that report receiving any component of the benefit. To this end, the Ministry designed imputation models, which ECLAC applied. Unfortunately, the results were not consistent with the data reported, since they gave an overall amount far higher than (more than double) that reported in the survey. Almost no values coincided with those actually reported, and the differences were in general quite broad in both directions, especially those above the amount reported. The possibility of applying this sort of procedure was therefore discounted: it would have meant replacing the amount reported by households almost completely. ECLAC has never proceeded in this manner in the past in correcting the response errors that commonly arise in connection with the various social benefits (even if not corrected, they are detected, measured and reported to the user). 4 Moreover, given that as noted on this occasion both the questionnaire and the fieldwork manual instructed enumerators to request the payment voucher, ECLAC finally considered that the most appropriate course of action was to respect the sum reported and not impute other values. In short, the set of grants that make up the social benefit includes the special annual payment made in November 2011, which, for the purposes of poverty measurement, should have been divided by 12 in order to impute the portion corresponding to a single month. Unfortunately, this was not possible, because problems with the survey questionnaire design prevented distinction between the different components of the social benefit. The only remaining option was to select one of the 4 In fact, on this occasion the survey covered only 50% of actual beneficiaries of the social benefit. 8
alternatives described above, and the course chosen was not to divide the whole social benefit variable into monthly amounts but to leave the value reported by respondents unaltered. As noted, this course of action meant accepting a degree of overestimation of monthly income, and therefore an underestimation of reported poverty. As a reference, if the social benefit variable is excluded, poverty would be 0.15 percentage point higher than was actually reported and indigence would be 0.1 percentage point higher, while if the total social benefit variable is divided by 12, the difference would be 0.13 and 0.1 percentage point, respectively. So, although it is true that, with the limitations described, the solution chosen was not ideal (and nor were any of the other alternatives), its impact on the poverty measurement was small. 5 (b) Question on certain types of labour income (y11) In 2011 a question was included for the first time on labour income received (the previous month and in the previous three months) for work, activities or business conducted that month by individuals who reported being unemployed, inactive or unpaid family workers (y11). This question is intended to capture information on income from work performed by individuals to whom other questions on labour income do not apply since they were not employed in the reference week, and which corresponds to activities carried out the previous month but before the reference week. Whether it is appropriate to include this variable from the 2011 round in the computation of income merits some discussion, in terms of maintaining comparability with measurements taken in earlier years, particularly in 2009. 5 In any event, this situation leaves valuable lessons for improving the design of enquiry into this income category (which will be known in the future as Ethnical Family Income benefit) for the next round of the CASEN survey. 9
In the first place, in both 2009 and 2011 all persons aged 12 and over were asked about income received the previous month in the form of payment for occasional work (y13.1 and y14a respectively) 6 and from work carried out before last month (y13.2 and y14b). For the first of these questions (payment for occasional work), the 2009 questionnaire gave no specific instructions, but the 2011 questionnaire instructed enumerators to exclude income reported under y6 and y10 (income from a secondary waged or independent occupation), as well as income reported under the new y11a. 7 The fact that in 2009 the question was posed only to those not employed (see footnote 3) but in 2011 it was posed to all those aged 12 and over, led to 41,881 employed persons reporting income from occasional work in 2011. 8 Apparently, then, the presumably unmeasured component in previous years corresponds to income reported in y11a, as (non-occasional) work done the previous month that had ended and for which the individual had been paid before the week preceding the interview, in which week these people reported being unemployed, inactive or unpaid family workers. However, the survey itself does not provide information to support a distinction between the regular and occasional components of income. Nevertheless, informed conjecture may be made on the basis of variables that act as a proxy for the regularity or otherwise of the employment or work whose income is reported under this question. For example, 85% of those reporting having income under variable y11a (last month) report having 6 Strictly speaking, in 2009 the scope of application of this question is rather ambiguous, since while the questionnaire apparently applies it to all those aged 12 and over, the enumerator s instructions state that it should be posed only to persons not employed, and this was the criterion finally used in the consistency tests. 7 The 2011 fieldwork manual (p. 80) states that income from occasional work carried out during the last month and not reported elsewhere in the survey should be included here. If the person is employed and has already reported all earnings from labour income of any kind, this space should be left blank. 8 In 2009 this income may have been reported under income from secondary occupations. 10
worked before, while almost 80% report having received this type of income not only last month but also the month before, and over 70% the month before that as well (i.e. for the last three months). 9 Although this evidence is not conclusive, it does indicate that most of this income is not coming from strictly occasional work. For that reason, ECLAC initially preferred to exclude this variable from the income aggregate used to measure poverty in 2011. A second line of analysis concerns the fact that all the versions of the survey have contained a residual question entitled Other income (question y17.3 in CASEN 2009 and y18d in CASEN 2011). The purpose of this question is to capture sources of household income that have not been referred to directly during the interview. With respect to this variable, information provided by the Ministry of Social Development (Note on Inclusion of y11 in poverty calculations) stated that: experiences documented during the pretest of the 2011 CASEN survey like the experiences of the 2011 CASEN contractor (Microdata Center of the University of Chile) indicate that the items commonly mentioned by interviewees under this question are: Lottery game winnings Inheritances Tax rebates Labour earnings from last month by those who, although not classified as employed during the reference week, did receive work pay during the last month (for work performed last month). 9 This is in addition to the fact that the number reporting income from occasional work (y14a) which, by instruction, excluded those reporting income under y11a, showed virtually no variation between 2009 and 2011, while the overall amounts reported rose by almost 9%. 11
The first three sources of income do not correspond to current household income and enumerators were therefore instructed not to report them, but were explicitly instructed that the fourth component should be reported. This is an easy distinction to make if the interviewee asks the enumerator whether these items should be reported or not. The problem is that interviewees often simply report a sum without indicating whether it corresponds to an excluded item. With a view to resolving this difficulty, the Ministry of Social Development opted for the following course of action: Eliminate the erroneous inclusion of tax rebates. A very common doubt is whether or not to report tax rebates. Accordingly, it was decided to include the question explicitly (y18c) in order to ensure that it would not be included in reported other income (y18d). 10 Include the question on labour income from last month for those not classified as employed in the reference week. Another very common question refers to labour income received by individuals who were not employed during the reference week but did receive income last month for (non-occasional) work done last month. In light of these considerations, the Ministry concluded (in its Note on Inclusion of y11 in poverty calculations) that until the 2009 CASEN, enumerators were instructed to capture this type of income in the question other income (y17.3) provided that it did not correspond to occasional work (in which case it was to be reported under y13.1). 10 Strictly speaking, now that this calculation is possible, tax rebates should be excluded anyway, not so much on the basis of the conceptual argument given here but because the rebate introduces an asymmetry with respect to individuals whose income is reduced through effective payment of global complementary tax, which is not captured in this survey. 12
According to this interpretation, the correct course would be to include the y11 variable in the income aggregate used to measure poverty in 2011. 11 In sum, given the difficulty of establishing clearly whether or not comparability between the 2009 and 2011 income measures is best served by including y11 in aggregate household income for 2011, and the fact that y11 undoubtedly does represent a type of labour income received in the reference month, it was finally decided to include it in the calculation of total household income in the last year. 12 Evaluation of the impact of this decision showed that exclusion of y11 from household income calculations would have yielded a poverty rate 0.6 percentage point higher than the officially reported rate and an indigence rate 0.4 percentage point higher. Conclusions 1. The tasks commended to ECLAC were to correct and adjust the income variables. 2. This objective was fulfilled, guided by the aim of achieving the highest possible degree of comparability of income estimates over time. This is a key objective for the purpose of generating poverty measurements that are consistent with the historical series Chile has built over the past two decades. 11 In the framework of this hypothesis it is notable, however, that while the item other income records a drop in overall amounts from CLP 2.152 billion to CLP 1.477 billion and in the number of recipients from 69,934 to 28,602 between 2009 and 2011, the variable corresponding to the labour income of unpaid, unemployed and inactive family members totalled CLP 24.861 billion and 205,503 recipients, much more than the growth expected from an income flow which was supposedly already contained in the original variable. 12 The Ministry also indicated that on previous occasions a similar approach had been adopted in the event of changes to the questionnaire design that improved capture of particular items of income, without necessarily adding new concepts to previous measurements. 13
3. It may be concluded that the income measurement methodology employed in the 2011 CASEN basically followed the same criteria and procedures as previous versions of CASEN. This ensures the consistency and comparability of the results across the various years of the survey at least in that regard and subject to the considerations and precautions set forth here. 4. It must be realized that the precision of poverty measurements of the type carried out in the CASEN survey is influenced both by the fact that the data capture is based on a sample (sampling errors) and by necessary assumptions and decisions regarding the treatment of the data during the estimation process. All this confers a degree of variability on the estimates, which must be borne in mind when releasing and interpreting the results. 14