2014 Managed Vision Care Patient Experience Study

Transcription

1 2014 Managed Vision Care Patient Experience Study RESEARCH 1

2 Measure, Understand and Improve Your Members Experiences We Can Help Measure with eyecarescore a validated, copyrighted survey tool that asks actionable questions about identified key drivers. Results are available in real-time dashboard reports with comparisons to industry averages and time trends. Understand your members, what they experienced, and how those experiences compare to prior periods, and to the industry. Improve in areas identified as statistically significant, and important to driving a better overall experience. Call for deeper insights / / % 75% 50% 25% 0% 71% 64% 57% 57% 57% / /-0.28 Surveys Percentile Rank 86% 86% 86% 71% 71% / % 79% Gap Analysis / / / / / / % 79% 71% 71% / / % 64% 64% Sample Ave / / % 71% Performance Dashboard Reports Plan Out of Pocket Cost Cross Tabs under $35,000 $35,001 - $75,000 Claims Convenience Ease Exam Coverage Courtesy Locations Provider types Calls Info CL Informed Variety Maximize Frame Variety Lens Frame Lens Variety Coverage Coverage CL Coverage Importance Income $75,001 - $120,000 over $120,000 $0 - No cost 12% 22% 38% 28% $1 to $5/mo 25% 38% 19% 19% $6 to $10/mo 11% 37% 33% 19% $11 to $15/mo 8% 41% 35% 16% > $15/mo 13% 29% 41% 17% Performance - Importance contact: Ron Krefman, OD rkrefman@.com tel:

3 Table of Contents Introduction Data Usage Policy 5 Objectives 6 Methods Question Development 8 Fielding the Survey 9 Plans in the Report 10 Indexes 11 Additional Questions 12 Sample Averages 13 How to Read the Charts 14 Performance Charts 15 Quad Charts 16 Aggregate Results Patient Demographics 18 Advocacy, etc. 19 Years Membership 20 Out of Pocket Costs 21 Plan Utilization 22 Health Reform and Pediatric 23 Plan Absence 24 Plan Payment Type 25 Index Box scores 26 Rating Box Scores 27 Year Over Year Index Scores 28 Year Over Year Rating Scores 29 Advocates vs. Critics 30 Distribution of Advocates 31 Touchpoint Quad Chart 32 Coverage Quad Chart 33 Provider Quad Chart 34 Specific Plan Results Member Profile 36 Overall Scores 37 Years Membership 38 Out of Pocket Costs 39 Touchpoints 40 Coverage 41 Providers 42 All Plans Averages Tables Appendix Questionnaire Validity Literature Search Focus Groups Expert Review Cognitive Testing Field Testing 3

4 Introduction RESEARCH 4

5 Data Usage Policy You may not, without Jobson s prior written consent, reproduce or display any part of the Study or any data from the Study, for free or otherwise: In any document or on any Web site that is going to be distributed to or seen anyone outside the company other than a Confidential Advisor. In any way that reasonably could be expected to deprive Jobson of an opportunity to sell another copy of the Study. 5

6 Objectives The purpose of this Managed Vision Care Patient Experience report is to: Sample consumers who had recent experiences with a vision plan Analyze key measures of the patient experience with their vision plan Benchmark performance with valid, statistically meaningful data Report performance and provide feedback to the industry 6

7 Methods RESEARCH 7

8 Questionnaire The people experiencing interactions with a plan and utilizing their benefits are the best source of information about the consumer experience. Through extensive testing, a survey instrument was developed to ask the best questions that describe quality in the vision plan consumer experience. The question selection was based upon input from major stakeholders: consumers/patients, ECPs, and vision plan industry experts. Multiple qualitative and quantitative survey science methodologies and rounds of extensive field testing were employed to insure question validity and survey reliability. The steps included literature search, consumer focus groups, expert review, cognitive testing, multiple rounds of field testing, and statistical and psychometric analysis of the data. See APPENDIX for more details on the questionnaire development. 8

9 Fielding During first and second quarter of 2014 By Jobson Research (as part of VisionWatch) VisionWatch is an on-going study of a statistically balanced sample of 100,000 US residents designed to be representative of the U.S. population 18 years of age and older. Safeguards were instituted to ensure that respondents participated in the survey only once in order to eliminate bias that could result from multiple session participation. Surveyed 19,934 respondents in total 9

10 Vision Plans 2014 Data from 19,934 respondents was analyzed if the respondent: 1. had a managed care vision plan (10,031), and 2. had an eye exam within the preceding 12 months (7,072), and 3. the managed plan was a vision plan (2,328), and 4. the vision plan was used within the preceding 12 months (1,567), and 5. the named plan was rated by 15 or more respondents (1,417) 19,934 Respondents 10,031 MVC Plan 7,072 recent exam 2,328 Vision Plan 1,567 Used Vision Plan 1, Respondents Plans Included in this report (>15 respondents) Plan Name # Respondents Aetna 38 Anthem 31 Avesis 25 Cigna 44 Davis 114 EyeMed 278 Guardian 15 Humana 27 Kaiser 25 MetLife 29 NVA 25 Spectera 71 Superior 32 VBA 30 VSP

11 Indexes Index Scores - Questions are grouped by similarities in concepts being rated, and groupings are referred to as an Index. Indexes are known to have greater statistical reliability than a single measure, and give a single summary metric about performance. Index scores are reported both as a single composite average score, or shown as average of each box score for the index. The three indexes are: Touchpoints, Coverage, and Providers. Touchpoints Index - Ease of finding info - Advice to maximize benefits - Communicating what s covered - Reaching live agent - Courtesy of agents Coverage Index - Frame coverage amount - Lens coverage amount - Contacts coverage amount - Exam coverage amount - Variety of covered frames - Variety of covered lenses - Variety of covered contacts Providers Index - ECP type - Location choice - Claim turnaround - Provider Convenience (hrs, lab) - Ease of using plan at ECP 11

12 Additional Questionnaire Items Demographics - Gender - Age - Region - Education - Income - Ethnicity - Hispanic Outcomes - Advocacy - Satisfaction - Trust - Plan Importance Years Membership - vs Advocacy - vs Plan Cost - vs Income Plan Use - Visit reason - Plan used for - Referrals from visit - Plan link Out of Pocket Costs - Monthly for Plan - Eyeglasses - Contact Lenses - Exams Plan Attributes - Who pays? (self vs. employer) - Type of plan - Plan Importance 12

13 Sample Averages Plan A Plan B Plan C Average of Averages = Sample Average (Equally weighted among plans) Unequal sample sizes exist among the 15 plans. To assess plan level performance, equal weighting was applied by calculating the mean score for each question, for each plan. Thus, each plan contributed equally to the average of averages, referred to as the Sample Average. 13

14 How to Read the Charts Average - Or, mean, is the most commonly used measure of central tendency and provides a simple single statistic representing the typical rating given by respondents. Average scores are reported for questions and indexes. Box Scores - Additional information can be gathered by looking at the box scores which show the frequency distribution, or percent of responses for each rating scale point. Presented as stacked columns, they show the percent of best scores (Percent of 5 s ) at the top of each column, descending down (% 4 s, % 3 s, etc.). To make box scores easy to read, you can focus on simple patterns in the data, like top-box scores. However, its important to consider the frequency of responses in the bottom boxes, as well. 14

15 How To Read The Performance Charts Performance The average score (mean) is the most commonly used measure of central tendency and provides a simple single statistic representing the typical rating given by respondents. A plan s performance on a particular driver is it s average score on the 5 point scale. Average scores are reported for questions and indexes. Index The combined drivers of the customer experience are aggregated to create three indexes: Touchpoints, Coverage and Providers. Each 2014 index score is reported along with the percentile ranking (i.e., the percentage of scores that are equal to or less than that score). The 2014 score is compared to the 2013 score and to the aggregate score for all plans (2014 Benchmark). Differences in scores and statistical significance are reported. Drivers Scatter plots are used to compare a plan s performance against the aggregate. A plan s Mean (average) score is shown as a circle; the aggregate average score of all plans are shown as a square (benchmark), and the maximum and minimum scores by competitive plans are bars at each end of the line. If the Plan s score is significantly different from the average score of all plans at the 95% confidence level, the circle will have a ring around it. Index Score Percentile Rank % Difference Benchmark Significant Difference (95% Confidence) Min (Other Plan) Mean (All Plans) Score (This Plan, Not Sig. Different vs. Mean) Max (Other Plan) Score (This Plan, Sig. Different vs. Mean) Plan Score (significant difference from Mean)* Plan Score (non significant difference from Mean) Mean (all Plans) Min, Max (competitor Plans) 15

16 How to Read the Performance/Importance Quad Charts Performance - Importance Each driver s importance is calculated as the correlation coefficient between that driver and the Likelihood to Recommend the plan (i.e., Advocacy ). The correlation coefficient between each driver and the recommend outcome are reported on the vertical axis, labeled Importance. More important drivers have a higher correlation (stronger association). All are positive correlations and range between 0.25 (moderate importance) to 0.75 (very strong importance). The mid-point of 0.50 indicates a strong importance. The Performance reported on the 5 point questionnaire scale is plotted on the horizontal axis, with the midpoint centered around the aggregate mean. The midpoints of the performance and importance statistics divide the chart into quadrants: More Important/Under Performing occupying the upper left quadrant; Less Important/Under Performing in the lower left quadrant; Less Important/Out Performing describes the lower right quadrant; and More Important/Out Performing in the upper right quadrant. 16

17 Aggregate Results RESEARCH 17

18 Patient Demographics Aggregate (All Plans) Gender Male Female 47% 53% Typical Patient Female, Age 25-60, South Region, College Educated, Income $35k -$120k, Caucasian, Not Hispanic Age Region or older 5% 8% 14% 15% 9% 10% 12% 13% 8% 6% Midwest Northeast South West 19% 22% 38% 22% Education High school <4 year college 4 year college Post graduate degree 12% 34% 35% 19% Income Less than $35k $35k to $75k $75k to $120k More than $120k 8% 38% 37% 17% Ethnicity American Indian Asian Black Caucasian Other 0% 6% 10% 83% 2% Hispanic Hispanic Not Hispanic 7% 93% 18

19 Advocacy, Importance, Satisfaction and Trust Aggregate (All Plans) Neutral x% Critic x% x% Advocate x% x% Net Advocacy Score Exceptional Excellent Very Good Good Not Good Year Over Year Advocacy Scores No Significant Change 95% Confidence Level How likely are you to recommend your plan to friends or family? Response options: 1. Unlikely; 2. Likely; 3. Very Likely; 4. Extremely Likely; 5. Definitely Advocates =4 or 5, Critics = 1 or 2, and Neutrals = 3. The Net Advocacy is the difference between the percentage of Advocates and Critics. Vision Plan Dental Plan Health Plan 7% 21% 20% Plan Importance 89% 74% 75% x% Vision Plan is Somewhat, or Very Important 0% 20% 40% 60% 80% 100% Not at all Not Very Neutral Somewhat Very Year Over Year Satisfaction, Trust 2.88 Satisfaction Satisfaction: No Significant Change Trust: Significantly Lagging 95% Confidence Level Trust 19

20 Years Membership Aggregate (All Plans) Years Membership Advocacy by Years Membership Less than 1 year 1 to 3 years 4 to 6 years x% Have been members 1-3 years Less than 1 year 1 to 3 years 4 to 6 years 2% 11% 7% 7% 19% 11% Advocate Critic 7 to 9 years 7 to 9 years 3% 6% Neutral 10 or more years 10 or more years 5% 8% 0% 10% 20% 30% 40% 50% 0% 20% 40% 60% 80% 100% Eyeglass Out of Pocket Cost by Years Membership Plan Cost by Years Membership Less than 1 year 1 to 3 years 4 to 6 years 7 to 9 years 10 or more years 0% 20% 40% 60% 80% 100% $0 (100% Covered) Less than $100 $100 to $200 $201 to $300 $301 to $400 $401 to $600 More than $600 Less than 1 year 1 to 3 years 4 to 6 years 7 to 9 years 10 or more years 0% 20% 40% 60% 80% 100% $0 (No cost to me) $1 to $5 per month $6 to $10 per month $11 to $15 per month More than $15 per month 20

21 Out of Pocket Costs Aggregate (All Plans) Plan Cost Out of Pocket Services Cost Out of Pocket $0 (No cost to me) 14% 77% Pay > $6/month $0 (100% Covered) $1 to $5 per month 10% Less than $25 $6 to $10 per month 29% $26 to $50 55% Pay < $25 $11 to $15 per month 27% $51 to $100 More than $15 per month 21% More than $100 0% 10% 20% 30% 40% 0% 5% 10% 15% 20% 25% 30% Eyeglass Cost Out of Pocket Contact Cost Out of Pocket $0 (100% Covered) Less than $100 46% Pay < $100 $0 (100% Covered) Less than $100 63% Pay < $100 $100 to $200 $100 to $200 $201 to $300 $201 to $300 $301 to $400 $301 to $400 $401 to $600 $401 to $600 More than $600 More than $600 0% 5% 10% 15% 20% 25% 30% 35% 0% 10% 20% 30% 40% 50% 60% 21

22 Plan Utilization Aggregate (All Plans) Visit Reason Plan Used For Eye Exam Exam New Glasses Contacts Follow-Up Eye Problem 64% Visited for Exam Glasses Contacts Eye Treatment Meds 87% Used Plan for Exam New Eye Problem Refractive Surgery Other Other 0% 10% 20% 30% 40% 50% 60% 70% 0% 20% 40% 60% 80% 100% Plan Linked To Referred for Additional Care? 100% Stand Alone Medical plan Dental plan Medicare or Medicaid 45% Stand Alone 90% 80% 70% 60% 50% 40% 30% 20% 12% Referred for Additional Care 10% 0% 10% 20% 30% 40% 50% 0% No Yes 22

23 Health Care Reform and Pediatric Coverage Aggregate (All Plans) Plan Changes Last 12 Months Decrease Increase No Change Coverage Cost x% x% x% Exam Benefits x% x% x% Eyewear Benefits x% x% x% Child s Vision Coverage Medical Plan x% Different Vision Plan from Parent x% None x% Same Vision Plan as Parent x% Child Plan Changes Switched Vision Plans x% Enrolled for 1st time x% No Change x% Last 12 Months More saw an xxx in Cost Benefits x% Same Plan as Parent x% New Enrollment Children's Benefit Used Medical Eye Visit x% Contacts x% Glasses x% Exam x% x% Used Plan for Glasses 23

24 Plan Absence Aggregate (All Plans) Don t need You had a plan, why no longer? 8% 22% Employer Did Not Offer a Vision Plan Chose not to participate 14% Not Eligible 15% Between Jobs 19% Employer doesn t offer 22% Other 23% 0% 5% 10% 15% 20% 25% 24

25 Plan Payment and Type Aggregate (All Plans) Who Pays for your Plan? Split with my employer I pay 100% myself My employer pays 100% No cost (free membership) 54% split payment Don t know 0% 10% 20% 30% 40% 50% 60% Flex Discount Insurance Type Used for Most Recent Eye Exam Other Not sure 25% Vision Plan State or federal program Vision Medical 0% 5% 10% 15% 20% 25% 30% 35% 25

26 Index Box Scores Aggregate (All Plans) x% Top 2 Boxes x% Top 2 Boxes x% Top 2 Boxes 24% 23% 24% Exceptional 28% 27% 35% Excellent 23% 22% Very Good 27% 20% 21% 12% 4% 6% 2% Touchpoint Index Coverage Index Provider Index Good Not Good Ease of finding info Advice to maximize benefit Communicating what s covered Reaching live agent Courtesy of agents Frame coverage amount Lens coverage amount Contacts coverage amount Exam coverage amount Variety of covered frames Variety of covered lenses Variety of covered contacts ECP type Location choice Claim turnaround Convenience (hrs, lab. etc.) Ease of using plan at ECP 26

27 Rating Box Scores Aggregate (All Plans) x% Top 2 Boxes Agent Courtesy x% Top 2 Boxes Exam Coverage x% Top 2 Boxes Provider Ease 100% 90% Exceptional 80% 70% 60% 50% 40% 30% 20% 10% 0% Excellent Very Good Good Not Good Touchpoints Coverage Providers 27

28 Year Over Year Index Scores Aggregate (All Plans) Exceptional Significant Change Excellent Very Good Good 2 Not Good 1 Touchpoint Index Coverage Index Provider Index Ease of finding info Advice to maximize benefit Communicating what s covered Reaching live agent Courtesy of agents Frame coverage amount Lens coverage amount Contacts coverage amount Exam coverage amount Variety of covered frames Variety of covered lenses Variety of covered contacts ECP type Location choice Claim turnaround Convenience (hrs, lab. etc.) Ease of using plan at ECP 28

29 Year Over Year Rating Scores Aggregate (All Plans) Significant Change Exceptional 4 Excellent Very Good Good 2 Not Good 1 Touchpoints Coverage Providers 29

30 Index Scores: Advocates vs. Critics Aggregate (All Plans) Advocate Critic Touchpoint Coverage Provider Advocates are those who rate their likelihood to recommend their plan with a 4 or a 5, while Critics are those who are less likelihood to recommend, based upon scores given as 1 or a 2. 30

31 Distribution of Advocates Critics Aggregate (All Plans) Advocate Critic Neutral Age Income or older Less than $35k $35k to $75k $75k to $120k More than $120k 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Plan Cost Eyeglass Out of Pocket $0 (No cost to me) $1 to $5 per month $6 to $10 per month $11 to $15 per month More than $15 per month $0 (100% Covered) Less than $100 $100 to $200 Over $200 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% 31

32 Touchpoint Performance/Importance Aggregate (All Plans) XXX and XXX are Important Yet Underperforming 0.80 More Important Under Performing More Important Out Performing Importance Easy Info Max Benefit Informed Coverage Agent Access Agent Courtesy Less Important Under Performing Less Important Out Performing Avg 3.90 Performance 32

33 Coverage Performance/Importance Aggregate (All Plans) XXX and XXX are Important Yet Underperforming 0.80 More Important Under Performing More Important Out Performing Importance Frame Coverage Lens Coverage Contact Coverage Exam Coverage Frame Variety Lens Variety Contact Variety Less Important Under Performing Less Important Out Performing Avg Performance 33

34 Provider Performance/Importance Aggregate (All Plans) 0.80 More Important Under Performing XXX and XXX are Important Yet Underperforming More Important Out Performing Importance Provider Types Provider Locations Provider Conveniences Claims Provider Ease Less Important Under Performing Less Important Out Performing Avg 4.20 Performance 34

35 Specific Plan Results RESEARCH 35

36 SAMPLE PLAN Member Profile Sample Plan Members are: xxx xxx xxx xxx Age Aetna All Plans Average < Age 40 x% x% 40 and Older x% x% Income < $75k x% x% > $75k x% x% Education < 4 Yrs College x% x% 4 or More Yrs College x% x% Hispanic Yes x% x% No x% x% Importance of Having a Plan Dental x% x% Vision x% x% Medical x% x% (% Very Important + % Somewhat Important) 36

37 SAMPLE PLAN Overall Scores Recommend Satisfaction Trust x% x% x% Plan Score (significant difference from Mean)* Plan Score (non significant difference from Mean) Mean (all Plans) Min, Max (competitor Plans) % Percentile Rank Net Advocacy (Recommend) Year Over Year Neutral x% Advocate x% -x% x% Net Promoter Score Critic x% 1 Recommend Satisfaction Trust 37

38 SAMPLE PLAN Years Membership Years Membership Advocacy by Years Membership Less than 1 year 1 to 3 years 4 to 6 years 7 to 9 years 10 or more years Less than 1 year 1 to 3 years 4 to 6 years 7 to 9 years 10 or more years Unlikely Likely Very Likely Extremely Likely Definitely 0% 10% 20% 30% 40% 50% 0% 20% 40% 60% 80% 100% Income by Years Membership Plan Cost by Years Membership Less than 1 year 1 to 3 years 4 to 6 years 7 to 9 years 10 or more years Less than $35k $35k to $75k $75k to $120k More than $120k Less than 1 year 1 to 3 years 4 to 6 years 7 to 9 years 10 or more years $0 (No cost to me) $1 to $5 per month $6 to $10 per month $11 to $15 per month More than $15 per month 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% 38

39 SAMPLE PLAN Out of Pocket Costs Plan Cost Services $0 (No cost to 16% $0 (100% Covered) 28% $1 to $5 per 6% Less than $25 31% $6 to $10 per 16% $26 to $50 22% $11 to $15 per 35% $51 to $100 9% More than $15 26% More than $100 9% Glasses Contacts $0 (100% Covered) 21% $0 (100% Covered) 18% Less than $100 24% Less than $100 45% $100 to $200 31% $100 to $200 27% $201 to $300 14% $201 to $300 9% >$300 10% >$300 0% 39

40 SAMPLE PLAN Touchpoints Sample Plan Index Score Percentile Rank 2014 X.XX X% Difference 2013 X.XX X.XX 2014 Benchmark X.XX X.XX No Significant Difference (95% Confidence) x% Percentile Rank Index XXX vs Index XXX to 2014 Benchmark Performance: Touchpoint Drivers Easy Info Max Benefit Informed Coverage Agent Access Agent Courtesy Plan Score (significant difference from Mean)* Plan Score (non significant difference from Mean) Mean (all Plans) Min, Max (competitor Plans) XXX to Benchmarks Importance/Performance: Touchpoint Drivers Importance More Important Under Performing Less Important Under Performing 4 2 More Important Out Performing 3 Less Important Out Performing Avg 4.40 Performance Easy Info Max Benefit Informed Coverage Agent Access Agent Courtesy XXX XXX are most important and both are out-performing 40

41 SAMPLE PLAN Coverage Sample Plan Index Score Percentile Rank 2014 X.XX X% Difference 2013 X.XX X.XX 2014 Benchmark X.XX X.XX No Significant Difference (95% Confidence) x% Percentile Rank Index XXX vs Index XXX to 2014 Benchmark Performance: Coverage Drivers Frame Coverage Lens Coverage Contact Coverage Exam Coverage Frame Variety Lens Variety Contact Variety Plan Score (significant difference from Mean)* Plan Score (non significant difference from Mean) Mean (all Plans) Min, Max (competitor Plans) XXX to Benchmarks Importance/Performance: Coverage Drivers Importance More Important Under Performing More Important Out Performing 3 2 Frame Coverage Lens Coverage Contact Coverage Exam Coverage Frame Variety Lens Variety Contact Variety Less Important Under Performing Less Important Out Performing Avg 4.80 Performance Frame XXX XXX Coverage, are most Lens important Variety are and most both are important out-performing and are out performing 41

42 SAMPLE PLAN Performance: Provider Drivers Provider Types Providers Sample Plan Index Score Percentile Rank 2014 X.XX X% Difference 2013 X.XX X.XX 2014 Benchmark X.XX X.XX No Significant Difference (95% Confidence) x% Percentile Rank Index XXX vs Index XXX to 2014 Benchmark Provider Locations Provider Conveniences Claims Provider Ease Plan Score (significant difference from Mean)* Plan Score (non significant difference from Mean) Mean (all Plans) Min, Max (competitor Plans) XXX to Benchmarks Importance/Performance: Provider Drivers 0.80 More Important Under Performing More Important Out Performing Importance Provider Types Provider Locations Provider Conveniences Claims Provider Ease Less Important Under Performing Less Important Out Performing Avg 4.70 Performance XXX XXX are most important and both are out-performing 42

43 All Plans Averages Tables 43

44 All Plans Touchpoints Full Report (All Plans) Easy Info Max Benefit Informed Coverage Agent Access Agent Courtesy Touchpoint Index Aetna Anthem Avesis Cigna Davis EyeMed Guardian Humana Kaiser MetLife NVA Spectera Superior VBA VSP Sample Mean

45 All Plans Coverage Full Report (All Plans) Frame Coverage Lens Coverage Contact Coverage Exam Coverage Frame Variety Lens Variety Contact Variety Coverage Index Aetna Anthem Avesis Cigna Davis EyeMed Guardian Humana Kaiser MetLife NVA Spectera Superior VBA VSP Sample Mean

46 All Plans Providers Full Report (All Plans) Provider Types Provider Locations Provider Conveniences Claims Provider Ease Provider Index Aetna Anthem Avesis Cigna Davis EyeMed Guardian Humana Kaiser MetLife NVA Spectera Superior VBA VSP Sample Mean

47 All Plans Overall Outcomes Full Report (All Plans) Recommend Satisfaction Trust Aetna Anthem Avesis Cigna Davis EyeMed Guardian Humana Kaiser MetLife NVA Spectera Superior VBA VSP Sample Mean

48 Appendix 48

49 Questionnaire Validity: Literature Search To find valid drivers of plan performance, the touchpoints of the eyecare patient s journey were explored through an extensive literature review. PubMed MEDLINE searches using appropriate medical subject headings (MeSH) search terms were conducted to identify all potentially relevant articles published in the English language. PubMed references and suggestions, Google searches for relevant scholarly works from the social sciences, and Google searches for non-peer reviewed related content expanded the search. The search results were reviewed for questions or touchpoints relevant to vision plan performance. 116 possible questions were found. Number QUESTION TOPIC Questions in Lit. Experience (Satisfaction, Recommend, etc.) 10 Access Choices Access Ease 15 Network 11 Choices Coverage 12 Cost 13 Variety 17 Communication Care Claims Handling 9 Courtesy 6 Information 23 Grand Total

50 Questionnaire Validity: Focus Groups To refine the question list, the identified touchpoints were subjected to content validity assessment through web based, asynchronous patient focus group discussions asking patients what they considered to be key drivers in judging their eyecare experience. [1] The focus group moderator guide included discussion points and the dialogue corroborated and enhanced the literature findings on important touchpoints of the eyecare experience with the vision plan. Click here for more information about Key Drivers. 50

51 Questionnaire Validity: Expert Review To further narrow the list of questions, the Delphi method, used to solve an array of problems in healthcare, was selected for its efficiency and efficacy in gaining consensus among experts as to the most important drivers of quality regarding consumer experiences with their vision plan. [1] 52 potential experts were identified from their relevant work in academia, committees, clinical practice, managed care organizations, ophthalmic retailers, optometry journals, eyewear manufacturers, presentations or publications. 26 of the experts completed a questionnaire asking them to rate the importance of the literature and consumer identified touchpoints of the eyecare plan experience on a 5 point Likert scale. Importance criterion was set at mean score of 4 or higher ( important or very important ) and consensus 75% or higher among experts, with rounds of questioning were to be repeated until at least 5 items met the criteria for each of the major categories. The ability to upgrade contact lenses, coverage for non-routine medical eyecare, number of network doctors, coverage for out of network doctor, and coverage for laser vision correction were judged not to meet the consensus for importance by 59% of the experts, (mean % CI ±0.358). Information on overall wellness was also deemed unimportant. 51

52 Questionnaire Validity: Cognitive Review Cognitive interview testing was used to identify and address errors in the questionnaire, provide insights into assessments and interpretations, understandings or misunderstandings, recall, confusion, redundancy, and ambiguity as well as response scale misfit. The survey used a labeled 5 point scale [1], and the overall dimension was tested with multiple scales. Cognitive interviews were conducted by specialists at The Survey Research Laboratory (SRL) which conducts some 10,000 interviews annually, researching survey methodology including healthcare and consumer satisfaction. IRB approval was obtained and subjects were at least 18 years old, and in the past 6 months: 1) had an eye examination or eyecare visit, 2) utilized their vision plan benefits and 3) procured new eyeglasses or contact lenses. Scripted cognitive probes were completed with 10 qualifying, consenting patients. The respondents were diverse in education, race and income. The interviews lasted an average 46 minutes. Rephrasing was recommended on some questions, and redundancies or incongruities led to recommendations to drop other questions. [2] The questionnaire reading statistics showed one sentence per paragraph, 7.4 words per sentence and 4.7 characters per word, with Flesch Reading Ease score of 68., considered easily understandable, and Flesch-Kincaid Grade Level score of 5.5. Cognitive interviews revealed that the questionnaire was easy to read and the majority of questions were easily understood. In the few instances where question confusion or redundancy occurred, changes or dropping of questions were suggested by Survey Research Laboratory experts. Interviewing subjects about navigating and comprehending the questionnaire identified the preferred scale and improved the utility of the questionnaire. 52

53 Questionnaire Validity: Field Test The questionnaire was fielded to evaluate the consistency and reliability of the data, as well as the validity of inferences being made from the data. Testing and analysis was conducted adhering to the Declaration of Helsinki principles, and using anonymity and data collector neutrality. The questionnaire was distributed to respondents recruited from on-line panelists who passed screening questions to determine their eligibility: had an eyecare visit, used insurance and obtained new eyewear all within the past six months. Analyses of the questions comprising the dimensions Touchpoints and Coverage lead to the reduction or rewording of questions based on standardized fit statistics, correlations, and principal component analysis of standardized residuals. These statistical indicators of item quality were used in conjunction with content considerations and a goal of maintaining a reasonable level of internal consistency (i.e., minimum of.80) in determining whether to remove or reword and re-evaluate questions. Eight week test-retest reliability showed stability in the data. The predictive validity evidence supported the questions as good indicators of plan quality as the relationships between desired Experience (overall satisfaction and likely to recommend) and the relevant questions exhibited high correlations. Internal Consistency (criterion >0.8) Test-retest Reliability (criterion >0.5) Predictive Validity (criterion >0.5) Touchpoints Coverage

54 Measure, Understand and Improve Your Members Experiences We Can Help Measure with eyecarescore a validated, copyrighted survey tool that asks actionable questions about identified key drivers. Results are available in real-time dashboard reports with comparisons to industry averages and time trends. Understand your members, what they experienced, and how those experiences compare to prior periods, and to the industry. Improve in areas identified as statistically significant, and important to driving a better overall experience. Call for deeper insights / / % 75% 50% 25% 0% 71% 64% 57% 57% 57% / /-0.28 Surveys Percentile Rank 86% 86% 86% 71% 71% / % 79% Gap Analysis / / / / / / % 79% 71% 71% / / % 64% 64% Sample Ave / / % 71% Performance Dashboard Reports Plan Out of Pocket Cost Cross Tabs under $35,000 $35,001 - $75,000 Claims Convenience Ease Exam Coverage Courtesy Locations Provider types Calls Info CL Informed Variety Maximize Frame Variety Lens Frame Lens Variety Coverage Coverage CL Coverage Importance Income $75,001 - $120,000 over $120,000 $0 - No cost 12% 22% 38% 28% $1 to $5/mo 25% 38% 19% 19% $6 to $10/mo 11% 37% 33% 19% $11 to $15/mo 8% 41% 35% 16% > $15/mo 13% 29% 41% 17% Performance - Importance contact: Ron Krefman, OD rkrefman@.com tel: