Section on Survey Research Methods

Transcription

1 Using the Statistics of Incoe Division s Saple Data to Reduce Measureent and Processing Error in Sall Area Estiates Produced fro Adinistrative Tax Records Kiberly Henry, Partha Lahiri, and Robin Fisher Internal Revenue Service, P.O. Box 268, Washington DC Abstract: For developing public policies and research purposes, incoe-related statistics are frequently needed for different sall geographic regions (sall areas). The large Individual Returns Transaction File constructed by the Internal Revenue Service (IRS) is generally used for these purposes. Previous research based on the Statistics of Incoe (SOI) Division s For 4 saple, a large national saple of cleaned adinistrative tax records, suggests that the IRS data is subject to various kinds of easureent errors. Thus, sall-area estiates based on IRS data, though free fro the usual sapling errors encountered in a typical sall area estiation proble, are subject to nonsapling errors that do not affect tax liability. On the other hand, the SOI saple estiates do not suffer fro the nonsapling error, but they are subject to large sapling variability for sall doains. We deonstrate how SOI saple data can be used to reduce the nonsapling errors of IRS-based sall area estiates using an epirical best prediction approach to ipleent our proposed hierarchical odeling. Key words: Survey sapling, Adinistrative Records, Indirect Estiators. Introduction: Sall Area Estiation Using IRS Data and Its Associated Nonsapling Errors The approxiately 33 illion tax records on the Internal Revenue Service s (IRS) Individual Returns Transaction File have several uses to ultiple governent agencies. In particular, these data serve as the sapling frae for the Statistics of Incoe (SOI) Division of IRS, as well as a source of population data for other tabulations. For exaple, SOI publishes tabulated onetary aounts and the associated nuber of returns by and Adjusted Gross Incoe (AGI) categories using these data (Table 2 in each Spring issue of the SOI Bulletin). Also, the U.S. Census Bureau copiles the data to the county level for such uses as estiating county-to-county igration patterns (e.g., Gross 25) and auxiliary inforation in the Sall Area Incoe and Poverty Estiation Progra s (SAIPE) odels to estiate the nuber of children in poverty within each U.S. county. These population data, based on adinistrative tax records for the U.S. tax filing population, are not errorfree. While estiates fro these data are free fro sapling error, the data contain various nonsapling errors, as discovered in prior SOI research coparing return records in the transaction file to records for the sae returns in SOI s augented and edited For 4 saple. Only those ites necessary for coputer processing of a tax return are retained on the transaction file, as opposed to ites that ight be needed for other purposes, such as auditing. Measureent errors exist between the IRS and SOI data values due to different data editing rules. For revenue processing purposes, IRS does not spend scarce resources correcting errors that do not affect tax liability in the approxiately 3 illion tax return records it processes each year. Since tax liability is correct, this approach does no har to IRS s tax collection ission or to taxpayers, but it can adversely affect the usability of the data for statistical purposes. SOI s transcription and editing staff receive extensive training, and the saple of approxiately 23, returns is augented with additional ites fro the return, and ore closely onitored and checked for data consistency. Errors occur particularly for variables that are indirectly related to tax liability, such as and Local Incoe Taxes deducted on Schedule A. They were also discovered for variables such as Taxable Interest and Business Incoe/Loss fro Sole Proprietors (as reported on Schedule C) in the Tax Year 23 IRS data. To correct these errors, SOI had to delay its publication of Table 2 for several onths. Other liitations in the IRS data include a saller aount of inforation being available, copared to SOI s saple, and data are often provided to SOI in tabular for, with onetary aounts rounded to thousands and certain high incoe taxpayers are oitted. In order to iprove on design-based estiators, several indirect and odel-based ethods have been proposed in the literature. These iproved estiation procedures essentially use iplicit or explicit odels which borrow strength fro related resources such as adinistrative and census records and previous survey data. In order to estiate per-capita incoe for sall areas (population less than,), Fay and Herriot (979) used an epirical Bayes ethod that cobines the U.S. Current Population Survey data with various adinistrative and census records. In order to incorporate both the sapling and odel errors, Fay and Herriot (979) used a two-level odel which can be either viewed as a Bayesian odel or a ixed regression odel. Their epirical Bayes estiator [also an epirical best linear unbiased predictor (EBLUP)] perfored better than the direct survey estiator and a synthetic estiator used earlier by the U.S. Census Bureau. In an EBLUP approach, the best linear unbiased predictor (BLUP) of the sall-area ean is first produced 3

2 and the unknown variance coponent(s) is (are) estiated by a standard ethod [e.g., axiu likelihood (ML), residual axiu likelihood (), ANOVA, etc.]. The resultant predictor, i.e., the BLUP with estiated variance coponent(s), is known as an EBLUP of the true sall-area ean. A challenging proble in an EBLUP approach is to obtain a reliable easure of uncertainty of an EBLUP that captures all sources of variability. We do not attept to cite all the papers that use the Fay-Herriot odel or its extension; such references can be found in Rao (23) and Jiang and Lahiri (26). 2.: The SOI Saple 2. Data Description SOI draws annual saples of tax returns to produce richer and cleaner data for population estiation and tax odeling purposes. Stratification for the finite population of tax returns for SOI s Tax Year 24 Individual saple used the following categories:. Nontaxable returns with adjusted gross incoe or expanded incoe of $2, or ore. 2. High cobined business or far total receipts of $5,, or ore. 3. Presence/absence of special fors or schedules (For 2555, For 6, For 4 Schedule C, and For 4 Schedule F). Stratu assignent priority was based on the order in which a return et one of these categories. For exaple, if a return et () and (2), it fell into strata based on (). Within category (3), stratification also used size of total gross positive or negative incoe and an indicator of the return s usefulness for tax policy odeling purposes (Scali and Testa 26). The positive/negative incoe values in strata boundaries were indexed for inflation between 99 and the current tax year (Hostetter et al. 99). These criteria resulted in 26 strata. Each tax return in the target population was assigned to a stratu based on these criteria, then subjected to sapling in a two-step procedure. Within each stratu, a.5 percent stratified siple rando saple, called the Continuous Work History Saple (CWHS), was selected (Weber 24). For returns not selected for this saple, a Bernoulli saple was independently selected fro each stratu, with sapling rates fro.5 to percent. SOI s data capture and cleaning procedures resulted in a saple of 2,778 returns, including 65,948 CWHS returns, fro an estiated population of 33,89,982 returns. We placed the 34,484 returns that SOI sapled with certainty into one certainty stratu, since they represented a census of tax returns. Thus, without loss of generality, we exclude this stratu fro the population and develop our estiation ethod to estiate totals fro all other strata. To estiate the entire population total, we siply add the total fro the certainty strata to our estiate for the reaining population. 2.2: Sall Areas and Variables of Interest The reduced dataset for this analysis was created by first separating SOI s Tax Year 24 (i.e., incoe reported in 25 that was earned in 24) saple into the certainty and non-certainty units. For both sets, the weighted saple data were tabulated to the level, where included the 5 U.S. s, Washington DC, and an other category that included returns filed by civilians and ilitary individuals living abroad in U.S. possessions and territories, Puerto Rico, etc. We selected six variables, which can be grouped into two categories: variables that are ore or less susceptible to errors in the IRS data. They are listed, with their location on the For 4 and a brief description, in Table below. Susceptible to Error Table : Variable Naes, Tax For Location, and Description, by Variable of Interest Variable 24 Tax For Location Description a Adjusted Gross Incoe Line 36 Incoe reported fro the calculation of total incoe (Line 22) (pp. 7-8). Less Taxable Interest Incoe Line 8a Taxable aount of interest fro bonds, savings, etc. (p. 42). More Earned Incoe Tax Credit Real E Taxes and Local Incoe Taxes and Local General Sales Taxes a: page nubers fro IRS 26. Line 65a Line 6, Schedule A Line 5a, Schedule A Line 5b, Schedule A Taxpayer credit for lower-incoe working individuals (pp ). Aount of non-business related real e taxes paid (p. 37). An iteized deduction of the /local incoe taxes withheld fro taxpayers 24 salary (p. 44). Deducted and local general sales tax (instead of and local incoe tax deduction, p. 39). 32

3 3. Direct Estiators Let y k be the value of the characteristic of interest for the kth tax return, k U, the finite population of tax returns. We are interested in estiating the finite population total: Y = y. k U k Let s denote the saple of tax returns drawn fro the population of tax returns, s d s denote the part of the saple that belongs to the doain d of interest, and w k denote the sapling weight for the kth sapled tax return, k s. The sapling weight is siply the inverse of the inclusion probability and represents a certain nuber of population units in the finite population. In our case, we have epse sapling within each stratu, i.e., the sapling weights are the sae for all the sapled units belonging to the sae stratu. The weights vary across strata, due to disproportional allocation of the saple into different strata. Our doain cuts across the design strata, so weights of different sapled units inside a doain are generally different. Let Yd = yk k Ud denote the population total for the dth doain (excluding the tax returns belonging to the certainty stratu). Since N d is known fro the IRS records, our proble is equivalent to estiating the finite population ean for doain d: Yd = Yd / Nd. We can consider the following design-based direct estiator of Y d : y w y w. = () dw k s k k k d k sd If the doain d is large, then a reliable design-based estiate of the sapling variance of y dw can be obtained using the Taylor linearization technique using software like SUDDAN. 4. EBLUP Estiators In this section, we shall obtain an epirical best linear unbiased estiator of Y d under the following area level odel due to Fay and Herriot (979): For d =, L,, assue Level : ydw ~ ind N( Yd, Dd ); T Level 2: Yd ~ ind N ( xdβψ, ), where D d is the estiated sapling variance of y dw and x d is a p vector of known auxiliary variables based on the IRS data. Under the Fay-Herriot odel, the best predictor (BP) of Yd is given by: ˆ BP T Y = ( B ) y + B x β, (2) where d d dw d d D B d d = Dd + ψ. Note that the BP can be otivated without the norality assuption. If ψ is known, then β is estiated by the weighted least squares estiator: ˆ( ) T = xdxd xdydw. d= Dd ψ + d= Dd + ψ βψ Replacing β by ˆ( β ψ ), we obtain the following epirical best predictor of Y d : ˆ EBP T Y ( ) ˆ d = Bd ydw + Bdxdβ ( ψ ). (3) Note that Y ˆ d EBP is also the best linear unbiased predictor (BLUP) of Yd under the following linear ixed odel: T ydw = xd β + vd + ed, where the sapling errors { ed } and the rando effects { vd } are uncorrelated, with vd ~(, ψ ) and ed ~(, D d). When both β and ψ are unknown, we propose the following epirical best linear unbiased predictor (EBLUP) of Y d : ˆ EBLUP ˆ ˆ T Y = ( B ) y + B x ˆ βψ ( ˆ), (4) d d dw d d where ˆ D B d d =. Dd + ψˆ In this paper, we consider the residual axiu likelihood (), Prasad-Rao siple ethod-of-oents (), and Fay-Herriot s ethod-of-oents () estiators of ψ. We define the ean square prediction error of Y ˆ d EBLUP as ˆEBLUP ( ) 2 ( ˆEBLUP MSPE Y ) = E Y Y, d d d where the expectation is taken over the joint distribution of ydw and Y d under the Fay-Herriot odel. A naïve MSPE estiator is obtained by estiating the MSPE of the BLUP and is given by: N sped = gi( ψˆ) + g2i( ψˆ), (5) where g ˆ d( ψˆ) = Bdψˆ 2, g ˆ 2d( ψ ˆ) = Bdhdd, and h T T dd x d j j d ˆ x x = x j= D j + ψ. Intuitively, this naïve MSPE estiator is likely to underestiate the true MSPE since it fails to incorporate the additional uncertainty due to the estiation of ψ. In fact, Prasad and Rao (99) showed that the order of this underestiation is O ( ) under the following regularity conditions: 33

4 (.),, L, ; r < DL Dd DU < d = r d hdd = O (.2) sup ( ). Interestingly, the naïve MSPE estiator even underestiates the true MSPE of the BLUP, the order of underestiation being O ( ). A second-order unbiased estiator of MSPE is given by: DL sped = gd( ψˆ) + g2d( ψˆ) + 2 g3d( ψˆ) g4d( ψˆ), (6) 2 Bˆ where 3 ( ˆ d ) var( ˆ 2 g d ψ = ψ ) and g ˆ 4d( ψˆ) = Bdbias( ψˆ). Dd + ψˆ Here bias( ψˆ ) and var( ψˆ ) are the asyptotic bias and variance estiates of ψ ˆ, respectively. For exaple, ˆ ˆ 2 var( ψ) = 2 ( Dd + ψ) d = ˆ 2 ˆ 2 var( ψ ) = 2 ( Dd + ψ ) d = 2 ˆ ˆ var( ψ) = 2 ( Dd + ψ) d = ( ˆ ) for the ethod, for the ethod, and for the ethod; bias ψ = for the and ethods, and 2 2 d ˆ 2 (D ) (D d ˆ + ψ + ψ) d= d= bias( ψˆ ) = 3 (D d ψˆ + ) d = for the Fay-Herriot ethod. See Datta and Lahiri (2) and Datta, Rao and Sith (25) for details. 5. Results: Descriptive Plots Figure A. contains plots of ydw versus the corresponding ean calculated fro the IRS tabular data. The ain sources of error in the IRS eans are the easureent and processing errors. On the other hand, the SOI eans are priarily subject to the sapling error. The agnitude of these errors varies depending on the variable, but the effect of both errors is that the points are further fro the reference line drawn in each plot. However, a strong linear relationship is observed between these eans for each variable, particularly for variables that are considered less likely affected by IRS errors. The relationship is weaker for and Local Incoe Taxes and and Local General Sales Taxes, where IRS data have ore error. The and Local Incoe Taxes variable also has an apparent outlier in one s saple ean (TN). Figure A.2 contains descriptive plots of the shrinkage factors for each variable and analysis ethod, sorted by the size of D d. For each variable, as the estiates of D d increase, the shrinkage factor increases, which iplies that ore weight is given to the IRS ean in Y ˆ d EBLUP. All three ethods yield zero estiate of ψ, for Taxable Interest Incoe, which produces an estiate of for the shrinkage factors for all areas. This is undesirable since in such a situation the EBLUP estiate is identical to the regression synthetic estiate, which does not directly use the SOI data. Figure A.3 contains plots of the percentage relative differences of the IRS totals, to various alternatives, for all six variables. For all variables, the s were sorted by the size of the estiated coefficient of variation (CV) of the direct estiate for the total. As the CV of the direct estiate increased, the direct estiate was further fro the IRS-based total, shown by the points being further fro zero on the right side of each plot. Figure A.4 shows the percent relative gain of EBLUP estiates over that of the direct estiates for each variable. That is: ( ) ( ˆ EBLUP CV ydw CV Yd ) % Rel Gain = CV ( ydw) For Adjusted Gross Incoe, the results had the largest percent relative gains, except for the two largest s (CA and NY), where the direct estiates were ore precise. This was due to the fact that in this case EBLUP is identical to the regression synthetic estiate, since no weight was given to the direct estiate (as the shrinkage factor was equal to ). All of the estiates were ore precise than the direct, shown by positive gains for all s, while the were less precise for the five s with the lowest CV s. As expected, all EBLUP s gains in precision increased as the CV of the direct estiate increased. For Taxable Interest Incoe, there were also soe large gains in precision. Precision gains below (or loss above ) were truncated, which occurred for nearly a third of the estiates and the and estiates for California. However, the and ethods generally perfored well for this variable. For Earned Incoe Tax Credit, we obtained close positive percent relative gains for all s except Other, where the perfored best. For Real E Taxes, all three EBLUP s perfored well; and had higher gains in precision in s where the direct estiates had saller CV s than, but all three perfored equally well (and better than the direct estiates) for s where the direct estiates had higher CV s. For and Local Incoe Taxes, we see that only the perfored well; the and ethods produced negative percent gains in precision for all s except the outlier point noted in A., which had a uch higher gain in precision. Thus, these ethods appear to be 34

5 sensitive to outliers: they adjusted our outlier, but at the expense of the other s. For and Local General Sales Taxes, we see lower (but positive) gains in precision that only slightly increased as the CV of the direct estiate increased. Thus, when the relationship shown in A. is uch weaker, due to easureent and processing error in the IRS data, we see lower gains in precision in the EBLUP s. 6. Conclusions, Liitations, and Future Considerations We attept to iprove upon population-based estiates that are subject to nonsapling error and saple-based estiates subject to sapling error. In general, our EBLUP s see to produce preferable results, obtained by exploiting relationships between the saple and population variable eans. This was deonstrated by high gains in precision and ore stability in the estiates. However, for four out of six of the tax return variables we exained, at least one of the,, and ethods used to estiate ψ produced shrinkage factors equal to one for all s. This proble ay be due to using unreliable design-based direct variance estiates for the D d. The ethods also appear to be sensitive when there are outliers and perforance is lower when the relationship is weaker. In order to overcoe soe of these probles and to ake inferences ore flexible, we plan to consider a hierarchical Bayes ethod in the future. Starting in Tax Year 25, SOI s individual tax return saple is expected to increase by approxiately 65, non-certainty returns. This new saple will be useful to iprove on the estiates. We can also use this new saple to develop a robust evaluation criterion to copare different odel-based ethods. REFERENCES Survey Research Methods Section, Aerican Statistical Association, pp Internal Revenue Service, (26), Explanation of Ters, Statistics of Incoe 24 Individual Incoe Tax Returns, Internal Revenue Service, Publication 34, pp Jiang, J., and Lahiri, P. (26). Mixed odel prediction and sall area estiation (with discussions).test, 5,, -96. Lahiri, P. (2), Model Selection, IMS Lecture Notes/Monograph, Volue 38. Lahiri, P. (23b), A review of epirical best linear unbiased prediction for the Fay-Herriot sall-area odel, The Philippine Statistician, 52, -5. Prasad, N.G.N., and Rao, J.N.K. (99). The Estiation of Mean Squared Error of Sall Area Estiators. Journal of Aerican Statistical Association 85, Rao, J.N.K. (23). Sall Area Estiation, John Wiley & Sons: New York. Sarndal, C.E., and Hidiroglou, M.A. (989), Sall Doain Estiation: A Conditional Analysis, Journal of the Aerican Statistical Association, 84, pp Scali, J. and Testa, V. (26), Statistics of Incoe 24 Individual Incoe Tax Returns, Internal Revenue Service, Publication 34, pp U.S. Census Bureau, Sall Area Incoe and Poverty Estiation Division (SAIPE). Weber, M., (24), The Statistics of Incoe Continuous Work History Saple Individual Tax Return Panel, Proceedings of the Section on Survey Research Methods, Aerican Statistical Association. Datta, G.S. and Lahiri, P. (2). A unified easure of uncertainty of estiated best linear unbiased predictors in sall area estiation probles. Statistica Sinica,, Fay, R. E., and Herriot, R. A. (979). Estiates of Incoe for Sall Places: an Application of Jaes-Stein Procedure to Census Data. Journal of Aerican Statistical Association 74, Gross, E. (25). Internal Revenue Service Area-To-Area Migration Data: Strengths, Liitations, and Current Trends Proceedings of the Section on Governent Statistics, Aerican Statistical Association. Hostetter, S., Czajka, J. L., Schir, A. L., and O Conor, K. (99), Choosing the Appropriate Incoe Classifier for Econoic Tax Modeling, Proceedings of the Section on 35

6 A.. Mean Variable Plots, by Variable of Interest 7 Adjusted Gross Incoe,2 Taxable Interest Incoe 7 Earned Incoe Tax Credit 6, , 2,5 2,,5, Real E Taxes ,,2 & Local Incoe Taxes Deducted & Local General Sales Taxes Section on Survey Research Methods 5-6 5,,5 2, 2,5 3,

7 A.2. Shrinkage Factors, by Variable of Interest Adjusted Gross Incoe Taxable Interest Incoe Earned Incoe Tax Credit CA FL OH IL NC IN VA NJ WI SC AL KY MD CO IA MS KS WV NM NE ME SD HI VT DE WY Real E Taxes CA OH TX IN VA NC IL PA MO SC MA OK CT WI KS WA AZ RI ME NE MT NM VT HI SD DC and Local Incoe Taxes Shrinkage Factors (sorted by D).3 CA PA IL OH MA MI MN CO CT Other IA AZ OR TN UT NE AL HI SC LA ND WV AR NM SD DE and Local General Sales Taxes LA WV NC KY AZ FL SC TX MO CO CA MI MN NY IL MD IA NV WI SD WY NE CT ME DC NH Other CA PA IN NY KY OH AZ GA WI WA CT MO WV IA AR UT MN AL ID KS RI NE SD ND MT A.3. Percent Relative Differences Between Various Estiates and IRTF Totals, by Variable of Interest Adjusted Gross Incoe Taxable Interest Incoe Direct Hajek 2% 5% Direct Hajek=== TX FL WY IL CA PA NV AZ MO NY VA NC SC GA OK MA IA WV KS UT NE CT MT ID DE HI 2% 5% Earned Incoe Tax Credit Direct Hajek Section on Survey Research Methods % Rel Diff to IRS Total 4-4 % Rel Diff to IRTF Total % 5% % -5% -% % Rel Diff to IRTF Total % 5% % -5% -% -5% -5% -8 CA FL IL OH M I MA NC MD M N WI CT CO OR SC NV IA KS MS NE WV Other RI DE MT VT AK -2% CA NY IL OH PA VA CO IN M O TN M D WI AZ OK NV AL WV M T OR DE NE ID ND SD VT AK -2% CA FL GA IL M I TN AL NJ M S KY OK WA M A WI OR NM UT WV CT ID SD NH DE HI Other VT

8 % Rel Diff to IRS Total A.3. Percent Relative Differences Between Various Estiates and IRTF Totals, by Variable of Interest (cont d) Real E Taxes and Local Incoe Taxes 2.% Direct Hajek Direct Hajek % Rel Diff to IRTF Total.% -2.% Rel Diff to IRTF Total -% -2% -4% -6% and Local General Sales Taxes 4% 2% Direct Hajek % GA NC DE IN KS IL NM LA DC CA M N HI NJ ID NE M I SC PA UT M S WI WV FL WA AK TX -2% -4% -6% -8% -2 NY CA TX PA M I VA M D GA CT CO M O OR KY UT OK AL NH NE NM HI M E WV DE DC AK WY -4.% CA NC IL NJ NY M I GA WA SC OR OH IA M A CT NV LA NE AL M S NH VT WV Other WY DC RI -8% -2% % 38 5% % % A.4. Percent Relative Gain in EBLUP Estiates over the Coefficients of Variation of SOI Saple-based Estiates, by Variable of Interest Adjusted Gross Incoe Taxable Interest Incoe Earned Incoe Tax Credit CA NY IL GA VA AZ SC MA MD OR MS NV NM ID SD RI DE DC Real E Taxes % 5% % % CA OH TX IN VA NC IL PA M O SC M A OK CT WI KS WA AZ RI M E NE M T NM VT HI SD DC and Local Incoe Taxes % 5% % % CA FL OH VA M N WI Other M O OR KY NE OK SC WY WV M S SD and Local General Sales Taxes Section on Survey Research Methods 5% 5% 5% % LA WV NC KY AZ FL SC TX M O CO CA M I M N NY IL M D IA NV WI SD WY NE CT M E DC NH % % TX FL WY IL CA PA NV AZ M O NY VA NC SC GA OK M A IA WV KS UT NE CT M T ID DE HI Other TX IN FL OH SC WI AK MO MD AR OR AL OK RI NH ND TN