Engaging Pension Plan Participants Using Text Mining to better Understand Participants Thomas Post (Maastricht University and Netspar)

Transcription

1 Engaging Pension Plan Participants Using Text Mining to better Understand Participants Thomas Post (Maastricht University and Netspar)

2 What is the Problem? Don t know their expected pension benefits 56% 53% Don t know their type of pension scheme Don t read their annual pension statement 60% 43% Find pension information not understandable Don t understand what indexation is 36%

3 Why do we care?

4 How do we care?

5 How do we care? Emotions/Text Mining Life events Peer effects Framing Retirement Belief Model Visualization Future Self Planner Interactivity

6 Research Goal: Understand Participants (non) Behavior Emotions & things that matter Research methods Survey/experiments Interviews + survey Revealed preferences Text mining, Dimensions for method choice Proximity of measurement to event / hindsight bias Researcher assumed issues vs. real issues Data availability / Big Data Accuracy Ease of interpreting and communicating results Costs,.

7 Text Mining Knowledge Discovery from Text Some magic occurs Customer product reviews, s, call/chat data, essays, articles, annual reports, analyst reports, Facebook/blog/forum posts Product dimensions & ratings, emotions, tense, topics, segments, relations, (dynamic) networks,

8 Some State of the Art Examples: Emotional Content Ludwig, de Ruyter, Friedman, Brüggen, Wetzels, Pfann (2013) More Than Words: The Influence of Affective Content and Linguistic Style Matches in Online Reviews on Conversion Rates. Journal of Marketing, 77: Input: 18,955 Amazon book reviews (641 unique books)

9 Some State of the Art Examples: Topic Models Tirunillai and Tellis (2014) Mining Marketing Meaning from Online Chatter: Strategic Brand Analysis of Big Data Using Latent Dirichlet Allocation. Journal of Marketing Research, 51: Input: 350,000 consumer reviews

10 Some State of the Art Examples: Topic Models Hanley and Hoberg, Dynamic Interpretation of Emerging Systemic Risks, working paper & Hoberg and Maksimovic (2015), Redefining Financial Constraints: a Text-Based Analysis, Review of Financial Studies 28: Input :10 k forms Output: topics that predict 2008 bank failures As of 2006 Better and earlier than VIX measures Non financial firms experiencing equity or debt financial constraints Resulting in investment cuts and covenant violations Better than traditional measures

11 Application to pension fund participant communication (work in progress) Aim Text mine participant communication and chat data Currently 2016 NEST member survey data 2,155 respondents Closed (categories) and open (free text) questions Why (not) registered? Why increased contributions? Next Compare closed responses with Open / human coded Open / algorithm coded (topic model)

12 Disclaimer Please interpret the following results With a (big) grain of salt (I will explain why)

13 Why registered / login for the first time? Closed vs. open / human Closed questions Welcome pack/ /message from NEST 67% To register and make sure the log in details I had were correct 51% For reassurance that everything was working 39% To see how much was in my pension 39% To find out more information about my pension 35% To check that the contributions that I / my employer was making were correct 35% The company I work for provided me with the link to the NEST website 32% To see what tools were available on the NEST website 28% To increase my regular contributions 23% To choose who would get my pension when I died 20% Open questions (90 categories!!) Check payments so far (value of pension / contributions) 24% Setting up and check account (information correct / personal details) 21% Check how it works / what it was all about / options available 12% Increase pension contribution 8% Received an /letter/message from Nest or company 6% No meaningful response 5% Life event Job (new or lost) 2% Check value of pension and increase contribution (combined) 2% Concerned about future 1% Check information correct and the value of the pot (combined) 1%

14 Topic Model - Input To find out how it worked and how much money was in my fund To see if I could see the value of my pension change of personal details and circumstances/ checking all details correct Started new employment, automatic into scheme, had scheme with other employment and wanted to try to put two pensions together to try to make pot bigger Too little and short for text mining

15 Topic Model Output Top 5 topics (rank and top words) 1 check wanted works account worked make checking logged details pot payments pension balance scheme working log correct find 2 pension nest employer job started paying add save plan saving joined scheme left account previous pensions continue pot 3 set money account monthly direct pay site track company payment curiosity system work website top debit info view add 4 contributions employer retirement made additional worth extra correctly savings remember put future details online registered debit mail properly review 5 change amount increase funds saved needed year tax start current order invested age personal auto employers financial making checking topic message by NEST topic? increase contribution topic?

16 With short text and small number of documents Human beats algorithm (but 3 days vs. 5 mins) Open vs closed? Clear differences in results Which are better?

17 What we can do with (slightly) richer text Please write an essay about your thoughts and feelings about retirement with a minimum of 200 words in the box below. Please be specific: what do you think about retirement, how do you feel about this topic? Write down the first things that come to your mind when thinking about retirement. 157 essays

18 Emotional Rating

19 Top 5 Topics 1 time life money live lot hope enjoy things spend family travel good retire nice health friends world grandchildren children family and friends 2 retirement life work working age feel topic important pension start thinking retired make job free care future thing important topic people retired stay living part longer person day physical hobbies feelings standard difficult important aspect sort state family regularly pension sufficient future amount population due years system save personal ageing increase worries government major contribute general actively scared garden role projects enjoying countries close mixed lost waiting extent challenged visiting master studies friend rest books wealth prepare? ageing societies, worry do nice things

20 Summary and Conclusion Text mining: powerful method to understand participants Emotions Motivations/topics Given enough text Easy to implement Can be automized And integrated in business processes Differences between participants Emotions and topics over time And ultimately Adapt communication, products,. To increase participant engagement

21 Contact Dr. Thomas Post Assistant Professor of Finance Department of Finance School of Business and Economics Maastricht University

22 Additional Material

23 Why made additional contributions? Closed vs. open / human Closed questions Saw pension value / statement and worried not saving enough 40% Saw deductions on payslip and thought I needed to save more 30% Starting a new job 25% nothing in particular 20% Seeing someone struggle to make ends meet during retirement 15% payrise, bonus, inheritance 15% welcome pack from NEST 12% Seeing a discussion about pensions / retirement planning on an online forum 12% Seeing an article in a newspaper/website about pensions/retirement planning 11% Open questions (71 categories!!) To increase pension and have money after retirement 15% employer contribution / own contribution very small / low and concerned about money in pot 12% Availability of spare/extra money / pay raise / can afford to pay more 10% No specific answer / unclassified 10% just to save some additional money 10% Thinking / concern for the future/ getting a sense of security / worried 8% Started a new job Life event 3% getting closer to retirement age 3% Tax reasons 3% keep contributing after leaving job 3%

24 Topic Model: Input To increase the amount of money in my pension by paying in any extra I have Had some spare funds available at time and following financial advice The money that was going in from my employer was small it seemed like a sensible idea I have always wanted to save for a pension but usually I struggle to make ends meet. As my living expenses had been paid on this new temporary job I thought I could manage to put something aside Too little and short for text mining

25 Topic Model Output Top 5 topics (rank and top words) 1 pension wanted increase pot future nest small tax add contribute income paying start end bit contributing worth year months 2 contributions employer make work additional time top regular current conversation wage continue moved final efficient great prompted finally reason 3 retirement saving savings age started put save continue thinking family cash funds coming whilst percentage lack start secure grow 4 money contribution extra account spare save salary build due payment longer wages minimum afford level pensions realised part knew 5 amount felt low made needed nest retire employed put top worried larger afford future invest idea change worked monthly increase pot topic spare/extra money topic? worry topic?