WIOA COMMON PERFORMANCE MEASURES 1 WORKSHOP GOALS 2 Learn about the six WIOA common measures Understand how you can enhance performance KEY DEFINITIONS 3 Participant Date of Participation Qualifying Service Exit Date Exit Quarter 1
PARTICIPANT 4 Eligible for Indian Programs Receives a WIOA Service 4 DATE OF PARTICIPATION 5 Eligible Received a WIOA service If both checked, it s the date the participant received the 1 st WIOA service. 5 QUALIFYING SERVICE 6 Qualifying services include (among other things) Initial and comprehensive assessments Career counseling Development of employability plan Individualized assistance with job search Prevocational services & work experience Training Does not include Eligibility determination Self services Providing general information Follow up services after exit 6 2
NEW NAMES FOR SOME SERVICES 7 WIA CORE WIA Intensive WIOA Career Services WIA core & intensive services combined into WIOA career services DATE OF EXIT 8 The last day on which the individual received a service funded by WIOA or a partner program. 8 EXIT 9 BearTracks will automatically exit a participant who has not received a WIOA service within 90 days and is not scheduled for future services. 9 3
QUARTERS 10 Quarter 1 Quarter 2 Quarter 3 Quarter 4 JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 10 Quarters QUARTERS 11 Most performance measures are calculated based on the quarter in which a participant exits, and the performance outcomes are due in different quarters. 11 EXAMPLE QUARTERS 12 EXIT EXIT JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 1 st Quarter 2 nd Quarter after Exit after Exit JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 3 rd Quarter 4 th Quarter after Exit after Exit 12 4
GLOBAL EXCLUSIONS 13 Participants are excluded from the common measures for the following reasons: Medical condition expected to last more than 90 days causes the person to exit the program and precludes employment Incarcerated Hospitalized or in other treatment facility Deceased Reservists called to active duty 13 14 THE MEASURES WIOA MEASURES 15 Adult Measures 1. Employed in the 2 nd quarter after exit 2. Employed in the 4 th quarter after exit 3. Median earnings in the 2 nd quarter after exit 4. Credential attainment 5. Skill gains 6. Service to employers 7. Other measures as determined 15 5
MEASURE #1: 2ND QUARTER EMPLOYMENT RATE 16 The percent of exiters who were employed anytime in the second quarter after the exit quarter. 16 2ND QUARTER EMPLOYMENT RATE EXAMPLE 17 Exit/ Job Loses Job Gets Job JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 17 2ND QUARTER EMPLOYMENT RATE EXAMPLE 18 Exit/ Job Loses Job Gets Job JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 2 nd Quarter after exit 18 6
2ND QUARTER EMPLOYMENT RATE EXAMPLE 19 Exit Gets Job Loses Job JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 2nd Quarter after exit 19 MEASURE #2: 4TH QUARTER EMPLOYMENT RATE 20 The percent of exiters who were employed anytime in the fourth quarter after the exit quarter. 20 4TH QUARTER EMPLOYMENT RATE EXAMPLE 21 EXIT/ Loses Job Job JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Gets Job JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 4 th Quarter after Exit 21 7
MEASURE #3: MEDIAN EARNINGS 22 The median quarterly earnings of those employed in the second quarter after the exit quarter. MEDIAN EARNINGS EXAMPLE 23 JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Enters EXIT JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Post-Program JANEarnings FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 23 CALCULATING A MEDIAN 24 Earnings in 2 nd Quarter Person 1 $1,500 Person 2 $2,200 Person 3 $2,700 Person 4 $3,000 Person 5 $3,500 Person 6 $6,200 Person 7 $8,200 Sum $27,300 Average = 27300 = $3,900 7 } Half of people below midpoint Midpoint } Half of people above midpoint Median= $3,000 (the earnings of the person at the midpoint) 24 8
MEDIAN MINIMIZES THE EFFECT OF EXTREME VALUES 25 Earnings in 2 nd Quarter Person 1 $1,500 Person 2 $2,200 Person 3 $2,700 Person 4 $3,000 Person 5 $3,500 Person 6 $6,200 Person 7 $14,000 Sum $33,100 Average= 33100 = $4,729 7 } Half of people below midpoint Midpoint } Half of people above midpoint Median= $3,000 (the midpoint) 25 CALCULATING A MEDIAN WITH AN EVEN NUMBER OF PARTICIPANTS 26 Earnings in 2 nd Quarter Person 1 $1,500 Person 2 $2,200 Person 3 $2,700 Person 4 $3,000 Person 5 $3,200 Person 6 $3,500 Person 7 $6,200 Person 8 $8,200 } Median= (3000 + 3200) =$3,100 2 Middle two people 26 MEASURE #4: CREDENTIAL RATE 27 Percent of exiters who obtain a post secondary credential or high school diploma or equivalent within one year after exit. Those obtaining a high school diploma or equivalent count as a success only if they are employed or in education or training leading to a recognized post secondary credential within the year. 9
CREDENTIALS 28 Credentials are: Awarded in recognition of an individual s attainment of measurable technical or occupational skills necessary to obtain employment or advance within an occupation. Based on standards developed or endorsed by employers. 28 CREDENTIAL EXAMPLES 29 Educational Degrees and Diplomas Industry Recognized Certificates Registered Apprenticeships Occupational Licenses GEDs High School Diplomas Associate and Bachelor Degrees Cisco Certified Network Associate Microsoft Office Specialist Carpenter s Helper Chef Certified Phlebotomist Accounting Clerk Certificate Gaming License CREDENTIALS 30 Credentials do not include: Work readiness certificates Credentials awarded by grantees, state, or local workforce boards. Attainment of general skills related to safety, hygiene, etc., even if required to qualify for employment (example: OSHA 10) Certificates of completion 30 10
CREDENTIAL RATE EXAMPLE 31 Enter Training Finish Training Licensing Exam EXIT JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Within One Year After Exit JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Within One Year After Exit 31 CREDENTIAL RATE EXAMPLE 32 Enroll; EXIT JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC in H.S. Diploma Job No Job JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC Job JAN SEP FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 32 CREDENTIAL RATE 33 Who is in the base? Exiters (except global exclusions) who, while a participant: Were in postsecondary education or training (other than OJT or customized training), or Were in secondary school (without already having a diploma) Who counts as a success? Those who, during participation or within one year after exit Obtain a postsecondary credential, or Obtain a secondary school diploma or equivalent, but only if they are also employed or enrolled in postsecondary education leading to a credential anytime within the year after exit 33 11
MEASURE #5: SKILL GAIN 34 Percent of participants who, during the course of the program year, are in education or training programs that lead to a recognized postsecondary credential or employment and who are achieving measurable skill gains, defined as documented academic, technical, occupational, or other forms of progress, towards such a credential or employment SKILL GAIN KEY ELEMENTS 35 Designed to measure interim progress of participants, as well as those who exited during the reporting period Key Elements Base includes those who, during the reporting period, were in education or training programs that lead to a recognized postsecondary credential or employment Includes those in secondary school Includes those in OJT Does not include work experience (this is not training, according to WIOA) 35 SKILL GAIN KEY ELEMENTS 36 Key Elements Each participant can count as a success only once during a reporting period, even if the individual has more than one skill gain Participants can count in multiple reporting periods if they have at least one skill gain in each period 36 12
SKILL GAIN INDICATORS 37 Indicators of Skill Gains Documentation of skill gain includes: Increasing at least one educational functional level for those in programs below the postsecondary level Attainment of high school diploma or GED Making satisfactory progress in school (e.g., earning sufficient credit hours) Secondary school: transcript or report care for 1 semester Postsecondary: at least 12 credit hours in a semester or, for part time students, 12 credit hours over 2 consecutive semesters 37 SKILL GAIN INDICATORS 38 Indicators of Skill Gains Documentation of skill gain includes: Making satisfactory progress towards measurable standards as attested by an employer or training provider Passing an exam designed to measure occupational or technical skills 38 MEASURE #6: SERVING EMPLOYERS 39 Retention with the Same Employer: Percent of exiters employed in the 2nd quarter after exit who are employed with the same employer in the 4th quarter after exit. 13
SERVING EMPLOYERS CALCULATION 40 Number employed in the 4 th quarter with the same employer as in the 2 nd quarter Number employed in the 2 nd quarter after exit quarter Would be calculated strictly through UI wage matching Does not require additional data collection by the grantee Could not be calculated until the 4th quarter after exit MEASURE #7: INA SPECIFIC MEASURE 41 Rationale Ability to effectively serve native peoples and communities has always been understood as a critical part of grant responsibility Logistics In addition to the required measures, ETA, in consultation with the Advisory Council, must develop a set of performance indicators and standards that are applicable to the INA program. Discussion ETA hasn t made progress on these measures yet This is a statutory requirement, so work on this should start soon 41 42 MEASUREMENT ISSUES 14
WHY USE CALENDAR QUARTERS? 43 Performance is measured using Unemployment Insurance (UI) wage records. Use of UI data requires a lag of 2 quarters before data become available UI records are reported quarterly, and include earnings for the quarter. They usually cannot identify the date of employment, hours of employment, or hourly wages. 43 HOW EMPLOYMENT OUTCOMES ARE MEASURED 44 Both wage matching and grantee data can be used Wage matching o Grantees submit SSNs of participants in the SPIR o DOL matches against Unemployment Insurance (UI) wage files and federal employment database, which contain employment records for (nearly) everyone employed in the U.S. Supplemental data (provided by grantees) o Also as part of the SPIR, grantees indicate which participants are employed according to the grantee's own records 44 TYPES OF SUPPLEMENTAL DATA 45 Documentation for supplemental data (data provided by the grantee) include: Pay stubs Letter from an employer Follow up survey of the participant Detailed case notes signed by the counselor If supplemental data is being used, the same source must be used for measuring employment and wages 15
WHAT DIFFERENCE DOES EACH DATA SOURCE MAKE? 46 PY 15 Performance on the WIA Measures Grantee Data Only UI Data Only Both EER 41.5% 54.9% 66.9%* Retention 75.2% 73.9% 78.8%* Avg Earnings $10,562 $9,722* na * This is the official performance in WIA 47 OBSERVATIONS & NEXT STEPS OBSERVATIONS 48 Statutory Language Limits DINAP s Options Legislation requires use of common measures for the INA program Some Important Flexibility Grantees can seek waivers Advisory Council can help develop INAP specific measures 16
IMPLICATIONS FOR REPORTING 49 Changed reporting forms: 9084 & 9085 will be replaced PIRL replaces the SPIR New forms have already been approved by DOL and OMB NEW DATA SYSTEM 50 Bear Tracks is being replaced Eliminate concepts of core and intensive services New outcomes need to be reported Some changes to info on participants background characteristics for the PIRL (but not many) May need to provide data on participants, not just exiters Will be a cloud based system Individual level data system being developed for both CSP and Supplemental Youth Program SIGNIFICANT CHALLENGES 51 There are significant challenges and unknowns It is more challenging to time exits to maximize success Measuring as far out as 4 quarters after exit will be difficult Measuring skills gains will be challenging Measures for the Supplemental Youth Program are especially challenging given the limited funding for program services Substantial uncertainty remains - Phase in and timing - Employer measures and INA specific measure(s) - New data system - Flexibility to request waivers and waiver process 17
WHAT TO EXPECT 52 PY 2017 is likely to be a transition year No standards set, but baseline data collection likely to begin Bear Tracks SPIR will be examined for what it reveals - Data on employment in the 2 nd quarter - Wages - Credential attainment SSNs can be sent for UI matching to measure new outcomes Advisory Council likely to begin work on new measures DINAP likely to issue guidance on transition and phase-in during the summer Work on new data system to proceed WHAT TO DO? 53 What to do in the meantime Aggressively enter data in Bear Tracks - Credentials attained - Supplemental data on employment - SSNs, if appropriate (UI wage matching is very helpful) Make better use of follow-up services - Think about allocating budget for follow-up services - Supportive services and counseling are allowed for up to 12 months after exit (see Final Rule 684.310(d)) Keep the focus on serving your customers and community Support your Advisory Council - Make your challenges and needs known - Seek as much flexibility as allowable under the law RESOURCES 54 Final Rule for ETA (issued 8/19/16) Provides regulations for ETA only WIOA provisions Regulations for the INA program at Section 684 Available from the Federal Register at https://www.gpo.gov/fdsys/pkg/fr 2016 08 19/pdf/2016 15975.pdf Final Joint Rule (issued 8/19/15) Describes performance standards requirements (see Section 677) Available from the Federal Register at https://www.gpo.gov/fdsys/pkg/fr 2016 08 19/pdf/2016 15977.pdf Approved Information Collection Request Describes proposed new individual level record layout (to replace the SPIR) and new aggregate report Available at https://doleta.gov/performance/reporting/eta_default.cfm 18
CONTACT INFORMATION 55 Laura Aron Senior Associate Social Policy Research Associates 510-788-2476 Laura@spra.com 19
Activity Irene, age twenty-seven, is looking for a job to support herself and her two children. She has held fast food jobs in the past and, while not well-paying, she enjoyed working the cash register and making change. She s always been into numbers. Irene left high school when she was 16 without a degree. You found that she was eligible for the program on August 28 th and then she returned on August 30 th for skills and interests assessments. On September 26 th she entered the accounting clerk certificate program where she can earn her GED at the same time as earning a short-term stackable certificate. After completing the program in two quarters, she earned her certificate and her GED. You helped her find an accounting clerk position at the local insurance company and she started work on March 14 th. After she started her job, you called her once a week for the first month to check-in. These were considered follow-up services, so she exited from the program on March 14 th. You contacted her once a quarter for the following year and each time she was employed, although she changed took a similar position with another employer for more pay on September 30 th. 1. What quarter did Irene exit? 2. Are there advantages/disadvantages of exiting a participant on March 14 th vs. April 14 th? 3. Will Irene earn a positive on the 2 nd Quarter Employment Rate measure? 4. Will Irene earn a positive on the 4 th Quarter Employment Rate measure? 5. Which months of earnings will count for the Median Earnings measure? 6. Irene earned a GED and an accounting clerk certificate. Can you count this twice for the Credential measure? What about for the Skill Gain measure? Do you have enough information? If not, what do you need to know? 7. Is Irene included in the Retention with the Same Employer measure? Social Policy Research Associates