BIASES OVER BIASED INFORMATION STRUCTURES:

Transcription

1 BIASES OVER BIASED INFORMATION STRUCTURES: Confirmation, Contradiction and Certainty Seeking Behavior in the Laboratory Gary Charness Ryan Oprea Sevgi Yuksel UCSB - UCSB UCSB October 2017

2 MOTIVATION News sources are often biased. Media outlets select, discuss, and present facts differently, and do so in ways that systematically favor one side or the other. Concerns of bias increase as there are more news sources. There is a worry that people self-select into different information bubbles. Voters partially internalize this bias. Most believe most news sources to be politically biased.

3 RESEARCH QUESTION We study (in a lab experiment), how agents learn in environments where information sources are known to be biased. Our goal is to study: How agents select news sources in settings when different sources are characterized by different biases? How agents interpret biased news sources? Are agents able to take into account the bias in the information source when they update their beliefs?

4 RESEARCH QUESTION We study (in a lab experiment), how agents learn in environments where information sources are known to be biased. Our goal is to study: How agents select news sources in settings when different sources are characterized by different biases? How agents interpret biased news sources? Are agents able to take into account the bias in the information source when they update their beliefs?

5 BROADER CONTEXT These questions have clear policy implications in the context of political information and media bias. Understanding the economic forces that determine media content is thus of first-order importance. (Gentzkow et al. 2014) But our interest is more general. Consider, a consumer acquiring information about the quality of a product. a board member acquiring information about investment options. a patient acquiring information about different treatment options.

6 RESEARCH QUESTION We study (in a lab experiment), how agents choose among biased information structures. Features of the experimental design: Abstract setting, no attachment to prior. Removing motivated reasoning. Bias of the information sources are transparent. Removing quality concerns. Vary whether confirmation seeking behavior is optimal. Theoretically neutral.

7 RESEARCH QUESTION We study (in a lab experiment), how agents choose among biased information structures. Features of the experimental design: Abstract setting, no attachment to prior. Removing motivated reasoning. Bias of the information sources are transparent. Removing quality concerns. Vary whether confirmation seeking behavior is optimal. Theoretically neutral.

8 LITERATURE REVIEW Non-instrumental demand for information: Eliaz and Schotter (2010), Moebius et al. (2013), Golman and Loewenstein (2015), Ambuehl and Li (2015) Timing/concentration of information: Zimmermann (2014), Falk and Zimmermann (2014), Holder (2016), Masatlioglu et al. (2016) Non-bayesian interpretation of information: Eil and Rao (2011), Falk and Zimmermann (2016), Charness and Levin (2005), Charness et al. (2007), Jin et al. (2016), Enke and Zimmermann (2017), Enke (2017), and Frechette et al. (2017) Emergence of bias and Motivated Reasoning: Brocas et al. (2011), Mullainathan and Shleifer (2005), Rabin and Schrag (1999)

9 THEORETICAL FRAMEWORK Unobserved state of the world ϑ Θ := {L, R}. An information structure is mapping σ from Θ S. Let s S := {l, n, r}. Prior p is the probability that ϑ = R. Agent tries to match the state: a Θ. u(a, θ) = { 1 if a = ϑ 0 if a ϑ

10 DEFINING RELATIVE BIAS Definition σ is biased to the right of σ if (i) σ and σ are consistent, and (ii) distribution of posteriors shift to the right (FOSD) when σ is replaced by σ without the agent s awareness.

11 BIAS VIA DISTORTION σ A l r ϑ = L 1 0 ϑ = R 1 λ λ σ B l r ϑ = L λ 1 λ ϑ = R 0 1 Observation 1 σ B is biased to the right of σ A.

12 BIAS VIA DISTORTION σ A l r ϑ = L 1 0 ϑ = R 1 λ λ σ B l r ϑ = L λ 1 λ ϑ = R 0 1 Observation 1 σ B is biased to the right of σ A.

13 BIAS VIA DISTORTION σ A l r ϑ = L 1 0 ϑ = R 1 λ λ σ B l r ϑ = L λ 1 λ ϑ = R 0 1 Observation 2 For p > 0.5, the agent strictly prefers σ B to σ A.

14 BIAS VIA DISTORTION σ A l r ϑ = L 1 0 ϑ = R 1 λ λ σ B l r ϑ = L λ 1 λ ϑ = R 0 1 Observation 2 For p > 0.5, the agent strictly prefers σ B to σ A.

15 BIAS VIA FILTERING CASE 1: σ A l n r ϑ = L λ h 1 λ h 0 ϑ = R 0 1 λ l λ l σ B l n r ϑ = L λ l 1 λ l 0 ϑ = R 0 1 λ h λ h Observation 3 σ B is biased to the right of σ A.

16 BIAS VIA FILTERING CASE 1: σ A l n r ϑ = L λ h 1 λ h 0 ϑ = R 0 1 λ l λ l σ B l n r ϑ = L λ l 1 λ l 0 ϑ = R 0 1 λ h λ h Observation 3 σ B is biased to the right of σ A.

17 BIAS VIA FILTERING CASE 1: σ A l n r ϑ = L λ h 1 λ h 0 ϑ = R 0 1 λ l λ l σ B l n r ϑ = L λ l 1 λ l 0 ϑ = R 0 1 λ h λ h Observation 4 For any p > 0.5, the agent strictly prefers σ A to σ B.

18 BIAS VIA FILTERING CASE 1: σ A l n r ϑ = L λ h 1 λ h 0 ϑ = R 0 1 λ l λ l σ B l n r ϑ = L λ l 1 λ l 0 ϑ = R 0 1 λ h λ h Observation 4 For any p > 0.5, the agent strictly prefers σ A to σ B.

19 BIAS VIA FILTERING CASE 1: σ A l n r ϑ = L λ h 1 λ h 0 ϑ = R 0 1 λ l λ l σ B l n r ϑ = L λ l 1 λ l 0 ϑ = R 0 1 λ h λ h

20 BIAS VIA FILTERING CASE 1: σ A l r ϑ = L λ h 1 λ h ϑ = R 0 1 σ B l ϑ = L λ l 1 λ l ϑ = R 0 1 r

21 BIAS VIA FILTERING CASE 2: σ A l n r ϑ = L λ h 1 λ h 0 ϑ = R 0 1 λ l λ l σ B l n r ϑ = L λ l 1 λ l 0 ϑ = R 0 1 λ h λ h

22 BIAS VIA FILTERING CASE 2: σ A l r ϑ = L λ h 1 λ h ϑ = R 0 1 σ B l r ϑ = L 1 0 ϑ = R 1 λ h λ h

23 SUMMARY When information structures are symmetrically biased, And bias is created via distortion: Optimal to choose information structure biased towards prior. And bias is created via filtering: Optimal to choose information structure biased against prior.

24 POTENTIAL TYPES Optimal: Chooses the optimal information structure. Confirmation seeking: Chooses information structure biased towards prior. Contradiction seeking: Chooses information structure biased against prior. Certainty seeking: Chooses information structure to maximize fully-revealing signals.

25 POTENTIAL TYPES Optimal: Chooses the optimal information structure. Confirmation seeking: Chooses information structure biased towards prior. Contradiction seeking: Chooses information structure biased against prior. Certainty seeking: Chooses information structure to maximize fully-revealing signals.

26 DESIGN OVERVIEW: 10 rounds of information choice Feedback Survey Bias in the set information sources choices vary: Distortion, Filtering + Blackwell

27 DESIGN In each round, subjects are asked to guess the color of a ball drawn from a known urn. 1. ADVISOR CHOICE: Two (computerized) advisors which provide information on the color of the ball are presented. Subjects choose which advisor they would like to receive advice from. 2. GUESS: Conditional on the advisor choice, using the strategy method, subjects make a guess on the color of the ball. 3. BELIEFS: Conditional on the advisor choice, using the strategy method, subjects state the likelihood that their guesses are correct.

28 DESIGN In each round, subjects are asked to guess the color of a ball drawn from a known urn. 1. ADVISOR CHOICE: Two (computerized) advisors which provide information on the color of the ball are presented. Subjects choose which advisor they would like to receive advice from. 2. GUESS: Conditional on the advisor choice, using the strategy method, subjects make a guess on the color of the ball. 3. BELIEFS: Conditional on the advisor choice, using the strategy method, subjects state the likelihood that their guesses are correct.

29 DESIGN In each round, subjects are asked to guess the color of a ball drawn from a known urn. 1. ADVISOR CHOICE: Two (computerized) advisors which provide information on the color of the ball are presented. Subjects choose which advisor they would like to receive advice from. 2. GUESS: Conditional on the advisor choice, using the strategy method, subjects make a guess on the color of the ball. 3. BELIEFS: Conditional on the advisor choice, using the strategy method, subjects state the likelihood that their guesses are correct.

30 DESIGN INCENTIVES: $ 11 for show up + filling out the survey. $0, or $10 for earnings from a randomly selected round. Earnings depend on either accuracy of their guess, or a random lottery. Responses to belief questions determine the cutoff (crossover mehcanism). Elicitation method first described by Karni (2009) does not rely risk neutrality to be incentive-compatible.

31 BIAS THROUGH DISTORTION σ A l r ϑ = L 1 0 ϑ = R σ B l r ϑ = L ϑ = R 0 1 Four rounds with p = 0.7, 0.8 varying also the direction of the prior.

32 BIAS THROUGH FILTERING σ A l n r ϑ = L ϑ = R σ B l n r ϑ = L ϑ = R Four rounds with p = 0.7, 0.8 varying also the direction of the prior.

33 BLACKWELL RANKED ADVISORS σ A l r ϑ = L ϑ = R 0 1 σ B l r ϑ = L ϑ = R 0 1 Two rounds with p = 0.7, 0.3.

34 OVERVIEW Data is from 74 subjects in 4 sessions at UCSB - May-June Sessions computerized using z-tree lasted 70min. Detailed instructions with examples read out loud. Questions presented in four different orders. Distortion and Filtering questions presented in blocks. Half observed Distortion first. Half observed prior 0.7 (vs 0.8) first. 60 second delay between questions. Earnings were either $11 or $21.

35 OPTIMALITY Result 1: In the aggregate, when information structures are biased and cannot be Blackwell ranked, subjects fail to choose the optimal information structure at a rate much better than random.

36 OPTIMALITY OF ADVISOR CHOICE Frequency of choosing optimal advisor Distortion Filtering Blackwell

37 OPTIMALITY OF ADVISOR CHOICE Frequency of choosing optimal advisor Distortion Filtering Blackwell

38 CONSEQUENCES Result 2: Failing to choose the optimal information structure has significantly impact on guessing patters. Guessing accuracy is much lower in these cases, and likelihood of guessing against prior significantly higher.

39 CONSEQUENCE OF ADVISOR CHOICE Guessing accuracy relative to prior Probability of guessing against prior Optimal A. Other A Optimal A. Other A. Distortion Filtering

40 PATTERNS IN ADVISOR CHOICE Result 3: The fraction of subjects displaying confirmation seeking behavior is as large as the fraction showing optimal behavior. A smaller fraction show contradiction and certainty seeking behavior.

41 PATTERNS IN ADVISOR CHOICE Random Benchmark CONTRADICTION OPTIMAL 4 3 Filtering CERTAINTY CONFIRMATION Distortion

42 PATTERNS IN ADVISOR CHOICE Random Benchmark CONTRADICTION OPTIMAL 4 3 Filtering CERTAINTY CONFIRMATION Distortion

43 PATTERNS IN ADVISOR CHOICE Noisy Optimal Benchmark CONTRADICTION OPTIMAL 4 3 Filtering CERTAINTY CONFIRMATION Distortion

44 PATTERNS IN ADVISOR CHOICE Data CONTRADICTION OPTIMAL 4 3 Filtering CERTAINTY CONFIRMATION Distortion

45 CONTRADICTION CERTAINTY OPTIMAL CONFIRMATION CONTRADICTION CERTAINTY OPTIMAL CONFIRMATION CONTRADICTION CERTAINTY OPTIMAL CONFIRMATION PATTERNS IN ADVISOR CHOICE Random Benchmark Noisy Optimal Benchmark Data Filtering Filtering Filtering Distortion Distortion Distortion

46 TYPE DISTRIBUTION Classification method Type share among classified subjects (%) Perfect r.s. 1 error r.s. 2 errors r.s. Optimal Confirmation Contradiction Certainty Share classified

47 TYPE DISTRIBUTION Classification method Type share among classified subjects (%) Perfect r.s. 1 error r.s. 2 errors r.s. Optimal Confirmation Contradiction Certainty Share classified

48 TYPE DISTRIBUTION Classification method Type share among classified subjects (%) Perfect r.s. 1 error r.s. 2 errors r.s. Optimal Confirmation Contradiction Certainty Share classified

49 TYPE DISTRIBUTION Classification method Type share among classified subjects (%) Perfect r.s. 1 error r.s. 2 errors r.s. Optimal Confirmation Contradiction Certainty Share classified

50 TYPE DISTRIBUTION Classification method Type share among classified subjects (%) Perfect r.s. 1 error r.s. 2 errors r.s. Optimal Confirmation Contradiction Certainty Share classified

51 TYPE DISTRIBUTION Classification method Type share among classified subjects (%) Perfect r.s. 1 error r.s. 2 errors r.s. Optimal Confirmation Contradiction Certainty Share classified

52 HOW UNLIKELY IS IT TO OBSERVE THESE PATTERNS? Apply classification method to a large random sample. Simulate 10,000 data sets each consisting of 74 random subjects to generate benchmark type distribution. Estimate a mixture model. (ω O, ω Cf, ω Ct, ω Ce ) denotes the share of types. κ denotes implementation noise. All other choices are assumed to be random. Estimate parameters on 74 8 decisions.

53 COMPARISON TO A RANDOM SAMPLE Classification method Type share among classified subjects (%) Perfect r.s. 1 error r.s. 2 errors r.s. Optimal Confirmation Contradiction Certainty Share classified

54 COMPARISON TO MIXTURE MODEL ESTIMATION RESULTS Classification method Type share among classified subjects (%) Perfect 1 error 2 errors Mixture-model Optimal Confirmation Contradiction Certainty Share classified κ = 0.15 Consistent with error rate in Blackwell questions (0.10). Implies in expectation 1.2 mistakes per subject.

55 OPTIMAL TYPES VS. OTHERS Result 4: Subjects displaying confirmation, contradiction or certainty seeking behavior perform significantly worse at the guessing task.

56 TYPES Accuracy of guess Accuracy of guess relative to relative to Types prior conditional bayesian Optimal Confirmation Contradiction Certainty Untyped Variable compares expected accuracy of guess to prior. Optimal benchmark is 17.5%.

57 TYPES Accuracy of guess Accuracy of guess relative to relative to Types prior conditional bayesian Optimal Confirmation Contradiction Certainty Untyped Variable compares expected accuracy of guess to that of a bayesian agent conditional on chosen advisor. Optimal benchmark is 0%.

58 TYPES AND UPDATING Result 5: Subjects displaying confirmation seeking behavior are no worse at Bayesian updating than those displaying optimal behavior. We find no evidence linking confirmation seeking behavior to problems with Bayesian updating. Result 6: Subjects displaying contradiction and certainty seeking behavior are worse at Bayesian updating than those displaying optimal or confirmation seeking behavior.

59 TYPES Types Bias in expected posterior Optimal -8.9 Confirmation -6.1 Contradiction Certainty Untyped Variable compares expected stated posterior to prior. Optimal benchmark is 0%.

60 CONCLUSION AND NEXT STEPS We find strong evidence of confirmation and contradiction seeking behavior in the laboratory. In a context where traditional explanations for such behavior are removed. Confirmation seeking behavior does not appear to be correlated with how signals are later interpreted. Survey results suggest non-optimal choices over information structures to be linked to cognitive ability and political affiliation. We are in the process of gathering additional data using MTurk with comprehensive survey questions on media habits and political attitudes.

61 ADVISOR CHOICE back

62 GUESS back

63 BELIEFS back

64 TYPES Optimal (Sub 1012 ) 1 Correct Choice Filtering Distortion Pr: Rd: Q:

65 TYPES Optimal (Sub 1012 ) 2 Advisor Choice Filtering, Correct Filtering, Observed Distortion, Correct Distortion, Observed 1 Pr: Rd: Q:

66 TYPES Confirmation (Sub 1019 ) 1 Correct Choice Filtering Distortion Pr: Rd: Q:

67 TYPES Confirmation (Sub 1019 ) 2 Advisor Choice Filtering, Correct Filtering, Observed Distortion, Correct Distortion, Observed 1 Pr: Rd: Q:

68 TYPES Certainty (Sub 1005 ) 1 Correct Choice Filtering Distortion Pr: Rd: Q:

69 TYPES Certainty (Sub 1005 ) 2 Advisor Choice Filtering, Correct Filtering, Observed Distortion, Correct Distortion, Observed 1 Pr: Rd: Q:

70 TYPES Contradiction (Sub 1006 ) 1 Correct Choice Filtering Distortion Pr: Rd: Q:

71 TYPES Contradiction (Sub 1006 ) 2 Advisor Choice Filtering, Correct Filtering, Observed Distortion, Correct Distortion, Observed 1 Pr: Rd: Q:

72 TYPES Contradiction (Sub 1006 ) Optimal (Sub 1012 ) 1 1 Correct Choice Filtering Distortion Correct Choice Pr: Rd: Q: Pr: Rd: Q: Certainty (Sub 1005 ) Confirmation (Sub 1019 ) 1 1 Correct Choice Correct Choice Pr: Rd: Q: Pr: Rd: Q:

73 TYPES Contradiction (Sub 1006 ) Optimal (Sub 1012 ) 2 2 Advisor Choice Filtering, Correct Filtering, Observed Distortion, Correct Distortion, Observed Advisor Choice 1 1 Pr: Rd: Q: Pr: Rd: Q: Certainty (Sub 1005 ) Confirmation (Sub 1019 ) 2 2 Advisor Choice Advisor Choice 1 1 Pr: Rd: Q: Pr: Rd: Q: