Family Resources Survey and related series Don Burke Family Resources Survey Surveys Branch Department for Work and Pensions
What we are going to cover The Family Resources Survey Overview Users and uses Quality assurance Improving timeliness Challenges and opportunities What s new on the FRS FRS derived series Households Below Average Income New approach to measuring uncertainty Pensioners Incomes Series Income-Related Benefits: Estimates of Take-Up Latest results 2
The Family Resources Survey 3
Family Resources Survey overview Cross-sectional survey of income and living conditions running since 1992 Designed for DWP policy needs In-depth face to face interviews with a target 20,000 households across the UK All adults (around 34k) are interviewed and average interview time is around 70 minutes Stratified, clustered, probability sampling (GB) Continuous fieldwork across each financial year (April- March) with annual questionnaire review Fieldwork carried out by ONS and NatCen in GB and NISRA in NI 4
Key strengths of the FRS Breath and depth of information which is not available in combination (or at all) from administrative or other survey sources High quality data much more detailed editing and quality assurance of income and benefits data compared to other surveys Large sample size allowing analysis at regional level and for particular subgroups Stable design and long time series 5
FRS contains detailed information on: Household composition and circumstances Individual income from all sources including benefits, tax credits, investments and pensions Housing tenure and housing costs Caring needs and responsibilities Disability Education Childcare Child Maintenance Material deprivation Well-being 6
Who uses the FRS? Outside Government International Eurostat EU-SILC (up to 2016) Pensions Policy Institute New Policy Institute Institute for Fiscal Studies ( IFS ) University of East Anglia ( UEA ) UK Data Service ( UKDS ) University of York London School of Economics ( LSE ) HMRC ONS Northern Ireland (Dept. for Communities) Scottish Government Welsh Government Institute for Social and Economic Research ( ISER ) NHS Scotland DCLG DfE Home Office MOJ HMT Households Below Average Income Pensioners Incomes Take-Up Statistics Policy Simulation GLA Housing Policy Disability Policy Pensions policy Government (OGD) Children, families and disadvantage DWP Family Resources Survey 7
The FRS is the single most critical DWP research project Social welfare expenditure 2015-16: DWP 175bln + HMRC (Tax Credits 29bln + Child Benefit 11bln)) = 215bln 33% of all UK Government expenditure FRS is a key input for DWP Policy Simulation Model DWP Pensim2 estimating the effects of pension reform IGOTM - Treasury and HM Revenue modelling of the effects of tax and welfare changes on households 8
What decisions has the FRS influenced? Universal Credit Personal Independence Payments Benefit uprating policy Spare Room Subsidy Workplace pension reform Single tier and workplace pension reforms HMT and HMRC costing and assessment of policies including High Income Child Benefit Charge and the Tax- Free Childcare Scheme. Assessing the impact of policies on particular groups e.g. disabled, ethnic minorities, pensioners etc. 9
FRS microdata quality is key for policy simulation purposes Quality has always been prioritised over timeliness Focus in face to face interviews on capturing detailed accurate, unbiased information data on income, tax and benefits and household characteristics With respondents encouraged to consult payslips, bills etc. as much as possible Focus on detailed editing of benefits data, income, savings, investments and other monetary amounts For many years it has taken 15 months from the end of fieldwork to publication 10
FRS end-to-end process Annual questionnaire development and testing Fieldwork Post interview checks: in Blaise (ONS/NatCen) Initial SAS-based editing (ONS) Data loading and validation on DWP systems Benefit editing Period codes Outliers Credibility checks Derived variables and imputation Further Benefit editing Mop ups Re run derived variables Further credibility checks Re run derived variables Final data checks Comparisons FRS data quality assurance Initial FRS dataset HBAI production and QA Final FRS and HBAI datasets 11
FRS 2014-15 production timetable Consortium FRS Team FRS Users Jun-13 Jul-13 Questionnaire Questionnaire Questionnaire Aug-13 Development Development Development Sep-13 Oct-13 Programme Nov-13 changes to Dec-13 questionnaire Jan-14 Feb-14 Questionnaire Questionnaire Mar-14 Testing Testing Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Questionnaire Oct-14 in the field Nov-14 Dec-14 Jan-15 Feb-15 Data converted Mar-15 and delivered Apr-15 to DWP May-15 Jun-15 Jul-15 Process data Aug-15 6 month data QA Sep-15 Oct-15 Nov-15 Dec-15 12 month data QA Jan-16 Feb-16 Mar-16 Production of reports Production of reports Apr-16 May-16 Jun-16 Report / dataset release Report / dataset release 12
Improving timeliness processing review FRS end to end processing review was carried out, starting in 2014, using Lean Six Sigma process improvement methodology Lean Six Sigma combines the Lean focus on reducing waste and improving process flow with the Six Sigma focus on improving quality through reducing error and getting things right first time LSS provided a systematic approach to analysing FRS processes using the DMAIC review framework: Define, Measure, Analyse, Improve, Control Benefits have come through this year with FRS, HBAI and PI 2015-16 publications brought forward by 3 months Potential for further improvements in the next few years by continuing the LSS review cycle 13
Improving timeliness sources of improvement Ending fieldwork slightly early (no March cases reissued in April) Scheduling processing work as early as possible after fieldwork ends at ONS and scheduling work in quarterly batches at DWP Adding questionnaire checks to reduce post interview editing; Eliminating redundant checks Deciding on the optimal place for editing to be carried out (ONS or DWP) and reorganising work accordingly Improvements to SAS checking programmes and improvements to the DWP SAS interface functionality (improving speed) 14
FRS 2015-16 production timetable Consortium FRS Team FRS Users Jun-14 Jul-14 Questionnaire Questionnaire Questionnaire Aug-14 Development Development Development Sep-14 Oct-14 Programme Nov-14 changes to Dec-14 questionnaire Jan-15 Feb-15 Questionnaire Questionnaire Mar-15 Testing Testing Apr-15 May-15 Jun-15 Jul-15 Aug-15 Sep-15 Questionnaire Oct-15 in the field Nov-15 Dec-15 Jan-16 Feb-16 Data converted Mar-16 and delivered 6 month data QA Apr-16 to DWP May-16 Jun-16 Jul-16 Process data Aug-16 Sep-16 12 month data QA Oct-16 Nov-16 Dec-16 Jan-17 Production of reports Production of reports Feb-17 Mar-17 Report / dataset release Report / dataset release Apr-17 May-17 Jun-17 15
Challenges and opportunities Challenges: - Falling response rates (60% in 2013-14 to 54% in 2016-17) means we are struggling to achieve the target of 20,000 participating households - New incentive strategy was introduced in 2015-16, with a 10 unconditional Post Office voucher included with the advance letter to respondents, replacing the long-standing book of 6 First Class stamps - With response rates still falling, we are considering trialling alternatives - From the start of 2016 we have increased the issued sample - Opportunities: - We are currently developing plans for research into the potential for making use of administrative data to enhance the FRS both DWP benefits data and HMRC tax return data particularly data from the new Real Time Information systems. 16
What s new on the FRS FRS 2015-16 (published March 2017) - A set of questions on social and cultural participation replacing additional questions on material deprivation (to meet UK requirements for EU-SILC) - New questions on hourly rates of pay - Question on how pension wealth is drawn down during retirement. - Improvements to questions on the High Income Child Benefit Charge FRS 2016-17 (for Feb-March 2018 publication) - A section on access to services (replacing social and cultural participation) - Questions probing reasons for pay rates below the NMW/NLW - Improvements to the capture of child maintenance arrangements 17
What s new on the FRS FRS 2017-18 (for Feb-March 2019 publication) - Most questions specifically added for EU-SILC purposes since 2012 removed (178 items in total) - Revisions to childcare section, harmonising with CEYSP, covering 30 hours free childcare and tax-free childcare initiatives. - Extensive revisions to pensions section, to cover pension wealth drawdown - Self-employment duration - Experimental questions on expenditure (following the example of USoc innovation panel capturing expenditure with a one shot question) Numerous other minor updates and changes to the questionnaire are made each year to reflect changing categories, definitions etc. and in response to interviewer and processor feedback 18
FRS derived series Households Below Average Income Pensioners Incomes Series Income-Related Benefits: Estimates of Take-Up 19
Households Below Average Income (HBAI) HBAI uses FRS data to measure the income distribution Income is measured as total weekly household income from all sources (including child income) after tax, national insurance and other deductions Adjustments are made to take account of inflation and problems with measuring the high-end of the distribution Income is equivalised i.e. adjusted to make it comparable across households of different size and composition. For example, equivalisation adjusts the income of a single person upwards, so their income can be compared directly to the standard of living for a couple 20
Measuring uncertainty for HBAI estimates Up to 2011-12 confidence intervals for HBAI estimates were calculated using an estimating function approach. From 2012-13 to 2014-15, bootstrapping techniques have been used. The Institute for Fiscal Studies (IFS) were commissioned to develop the DWP methodology further to account as fully as possible for the specific features of the FRS sampling design and grossing. This new refined approach has been used for the HBAI 2015-16 estimates 21
Measuring uncertainty for HBAI estimates The new methodology produces: GB resamples simulating the FRS stratified, cluster sampling of GB households. NI resamples simulating the FRS stratified sampling of NI households. A unique set of grossing factors for each GB and NI resample, replicating the original HBAI grossing process, to produce lower and upper confidence intervals. Accounting for: Cluster sampling widens confidence intervals for most estimates, reflecting that this feature makes survey estimates less precise. Post-sample grossing to population totals narrows confidence intervals for estimates sensitive to incomes towards the very top of the income distribution, as specific control totals are set for high income individuals. 22
Pensioner Income Series (PI) PI uses HBAI data to analyse pensioner incomes by source and over time Examines how pensioners incomes differ from those of working age and variations in income between different types of pensioners. Changes in the economy and to the benefit system have meant that pensioners average weekly incomes have changed a lot over the past two decades. These statistics look at these changes 23
Income-Related Benefits: Estimates of Takeup Uses FRS data matched to administrative records to produce estimates of take-up for the main incomerelated benefits: Pension Credit Housing Benefit Jobseeker s Allowance (Income-based) and Income Support and Employment and Support Allowance (Income-related) Take-up refers to the receipt of benefits someone is entitled to and we estimate take-up in two ways by caseload and expenditure 24
Publication of 2015/16 data and results FRS, HBAI and PI results for 2015-16 were published on Thursday 16 March are available here: https://www.gov.uk/government/collections/family-resources-survey--2 The datasets are available on UKDS website: https://discover.ukdataservice.ac.uk/series/?sn=200017 Estimates of take-up for 2015-16 are due later this year We welcome all feedback 25
26
27
28
29
Welfare Reform and Work Act 2016 DWP is required under the act to by the end of each financial year on four income-related measures, three of which are reported in HBAI: Children living in households with equivalised net income less than 60% of median equivalised household income; Children living in households with equivalised net income less than 70% of median equivalised household income and who experience material deprivation; Children living in households with equivalised net income less than 60% of median equivalised household income, adjusted to take account of changes in the value of money since 2010/11; The fourth measure required under the act is: Children living in households where equivalised net income has been less than 60% of median equivalised net household income in at least 3 of the last 4 survey periods. We published experimental estimates of this measure using USoc data at the same time as the FRS, HBAI and PI last March. 30
Thank you 31