A Multi-Agent Prediction Market based on Partially Observable Stochastic Game

Transcription

1 based on Partially C-MANTIC Research Group Computer Science Department University of Nebraska at Omaha, USA ICEC / 37

2 Problem: Traders behavior in a prediction market and its impact on the prediction market s behavior Solution: A multi-agent system that formalizes the strategic behavior and decision making by market s participants based on a partially observable stochastic game validation: Comparison with other trading approaches 1 / 37

3 A Prediction market is a market-based mechanism used to - combine the opinions on a future event from different people and - forecast the possible outcome of the event based on the aggregated opinion 2 / 37

4 3 / 37

5 4 / 37

6 5 / 37

7 6 / 37

8 7 / 37

9 8 / 37

10 9 / 37

11 10 / 37

12 11 / 37

13 12 / 37

14 13 / 37

15 Main Features A prediction market is run for a real-life unknown event Each event has a finite duration Each event s outcome has a security associated with it Traders buy and sell the securities based on their beliefs about the outcome of the event Traders beliefs are expressed as probabilities Market maker aggregates the probabilities from all the traders into a single probability, market price Market price of a security represents the probability of the outcome of an event associated with that security happening Traders get paid according to their reported beliefs 14 / 37

16 15 / 37 Software trading agents perform calculations and trade on behalf of human traders Provides testbed for modeling different strategic behaviors of traders through simulations with trading agents

17 Do s Work? Yes, evidence from real markets, laboratory experiments, and theory I.E.M. beat political polls 451/596 [Forsythe 1999, Berg 2001, Pennock 2002] HP market beat sales forecast 6/8 [Plott 2000] Sports betting markets provide accurate forecasts of game outcomes [Debnath 2003, Schmidt 2002] Market games work [Pennock 2001] Laboratory experiments confirm information aggregation [Forsythe 1990, Plott 1997, Chen 2001] Theory of Rational Expectations [Lucas 1972, Grossman 1981] and more / 37

18 Addressed 17 / 37 Develop a formal, game-theoretic model of the trading agent behavior in prediction markets including - impact of information from external sources on trading agent decisions/behavior, - a solution concept for calculating the equilibrium strategies of the trading agents Research Questions: How does different traders behaviors affect market prices? What trading strategies give the highest utilities to the traders? How can prediction markets incentivize traders to participate and report their beliefs truthfully in order to achieve a higher prediction accuracy?

19 Our Solution A partially observable stochastic game with information ()-based model of the trading agent behavior A correlated equilibrium (CE)-based solution to determine equilibrium strategy in the representation 18 / 37

20 Partially with Information () 19 / 37

21 Partially with Information () 19 / 37 - Logarithmic Market Scoring Rule (LMSR) [Hanson 2007] gives formula to calculate aggregate market price from the outstanding quantity of a security

22 Partially with Information () - Belief state is updated using a Bayesian model of past beliefs, past actions and current observation 20 / 37

23 Partially with Information () - Information signal can be { 1, 0, 1} representing positive, neutral, or negative information 21 / 37

24 Partially with Information () 22 / 37

25 Two Agent Trading Scenario with 1 security Example Actions = to buy, sell, or hold one security Market state is denoted by q C(q) is the cost function calculated by LMSR, that reflects the total money collected by the market maker Can construct a normal form game to capture the decision problem for each agent 23 / 37

26 Correlated Equilibrium (CE) Set up Let N be the set of trading agents Let A i be the set of actions for agent i N Let Φ i be the set of mixed strategy profiles defined over A i Joint strategy space is then Φ = N i=1 Φ i Let φ Φ be a strategy profile and φ i denote agent i s component in φ Let u i (φ) be the utility of agent i from joint strategy φ 24 / 37

27 Correlated Equilibrium (CE) Correlated Equilibrium is a probability distribution p on Φ such that for all agents i and all strategies φ i, φ i if all agents follow a strategy profile φ, agent i has no incentive to play another strategy φ i instead The Chicken-Dare Game Assume there is a trusted third party that draws a strategy for each player and announces it to each player separately Players agree to follow the strategy suggested by the third party Players get higher payoffs when use CE than NE 25 / 37

28 Correlated Equilibrium (CE) Algorithm CE algorithm applied to our model is based on the method proposed by Papadimitriou [2008] CE algorithm of the gives the action (buy or sell; quantity) for each agent at each time step Time complexity of our CE algorithm is O( N Φ i 2 ) 26 / 37

29 Correlated Equilibrium (CE) with Risk Preferences The beliefs and risk preferences of traders are directly correlated [Kadane 1988, Dimitrov 2008] Use constant relative risk averse (CRRA) utility function To find CE in the market with risk-averse trading agents: - Find the set of all Pareto optimal strategy profiles - Check whether a Pareto optimal strategy profile satisfies CE constraints 27 / 37

30 Correlated Equilibrium (CE) with Risk Preferences A strategy profile φ P is Pareto optimal if there does not exist another strategy profile φ such that u i (φ ) u i (φ P ) i N with at least one inequality strict Pareto optimal strategy is found by maximizing weighted utilities and solving the maximization problem using Lagrangian method Proposition If p is a correlated equilibrium and φ P is a Pareto optimal strategy profile calculated by p in a prediction market with risk averse agents, then the strategy profile φ P is incentive compatible, that is each agent is best off reporting truthfully 28 / 37

31 Set up Default values for market related parameters came from Iowa Electronic Marketplace (IEM) movie market Assume there is one event with two outcomes Consider only disjoint events Report utilities and market price for the security corresponding to the outcome of the event being 1 29 / 37

32 Set up 30 / 37 We use the following strategies for comparison of the trading agents and market s behavior 1 ZI (Zero Intelligence) - each agent submits randomly calculated quantity to buy or sell 2 ZIP (Zero Intelligence Plus) - each agent selects a quantity to buy or sell that satisfies a particular level of profit 3 CP (by Preist and Tol) - each agent adjusts its quantity to buy or sell based on past prices and tries to choose that quantity so that it is competitive among other agents 4 GD (by Gjerstad and Dickhaut) - each agent maintains a history of past transactions and chooses the quantity to buy or sell that maximizes its expected utility 5 DP (Dynamic Programming solution for POSG game) - each agent uses dynamic programming solution to find the best quantity to buy or sell [Hansen 2004]

33 Risk-neutral Agents 31 / 37 (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with trading agents using CE strategy result in more accurate market prices (c) Trading agents using CE strategy obtain 38% more utility than the agents following the next best performing strategy DP

34 Risk-neutral Agents 31 / 37 (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with trading agents using CE strategy result in more accurate market prices (c) Trading agents using CE strategy obtain 38% more utility than the agents following the next best performing strategy DP

35 Risk-neutral Agents 31 / 37 (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with trading agents using CE strategy result in more accurate market prices (c) Trading agents using CE strategy obtain 38% more utility than the agents following the next best performing strategy DP

36 Risk-neutral Agents (a) Trading agents using CE strategy obtain more utility than trading agents using ZIP strategy (b) Trading agents using CE strategy obtain more utility than trading agents using CP strategy 32 / 37

37 Risk-neutral Agents (a) Trading agents using CE strategy obtain more utility than trading agents using ZIP strategy (b) Trading agents using CE strategy obtain more utility than trading agents using CP strategy 32 / 37

38 Risk-neutral Agents (c) Trading agents using CE strategy obtain more utility than trading agents using GD strategy (d) Trading agents using CE strategy obtain more utility than trading agents using DP strategy 33 / 37

39 Risk-neutral Agents (c) Trading agents using CE strategy obtain more utility than trading agents using GD strategy (d) Trading agents using CE strategy obtain more utility than trading agents using DP strategy 33 / 37

40 Risk-averse Agents 34 / 37 (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with risk-averse trading agents using CE strategy result in more accurate market prices (c) Risk-averse trading agents using CE strategy obtain 41% more utility than the agents following the next best performing strategy DP

41 Risk-averse Agents 34 / 37 (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with risk-averse trading agents using CE strategy result in more accurate market prices (c) Risk-averse trading agents using CE strategy obtain 41% more utility than the agents following the next best performing strategy DP

42 Risk-averse Agents 34 / 37 (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with risk-averse trading agents using CE strategy result in more accurate market prices (c) Risk-averse trading agents using CE strategy obtain 41% more utility than the agents following the next best performing strategy DP

43 Conclusions and In this work we have: - described a novel multi-agent based representation of the prediction market using POSG - developed a CE solution for solving POSG - empirically compared different agent behavior strategies in the prediction market - showed how CE can be obtained in the prediction market with risk averse agents 35 / 37

44 Conclusions and 35 / 37 In this work we have: - described a novel multi-agent based representation of the prediction market using POSG - developed a CE solution for solving POSG - empirically compared different agent behavior strategies in the prediction market - showed how CE can be obtained in the prediction market with risk averse agents In the future we plan to: Conduct experiments in an n-agent scenario using richer commercial data sets Investigate the dynamics evolving from multiple prediction markets that interact with each other

45 References 36 / 37 1 Y. Chen, D. Pennock. Utility Framework for Bounded-Loss Market Maker. Proc. of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI 2007), pages 49-56, E. Hansen, D. Bernstein, S. Zilberstein. Dynamic programming for partially observable stochastic games. In Proceedings of the 19th National Conference on Artificial Intelligence, pages , R. Hanson. Logarithmic Market scoring rules for Modular Combinatorial Information Aggregation. Journal of s, 1(1):3-15, Iowa Electronic Marketplace. URL: 5 C. Papadimitriou, T. Roughgarden. Computing correlated equilibria in multi-player games. Journal of ACM, 55(3):1-29, 2008.

46 Thank You! Questions? C-MANTIC Research Group 37 / 37