The Efficient Markets Hypothesis Does Not Hold When Securities Valuation is Computationally Hard

Transcription

1 The Efficient Markets Hypothesis Does Not Hold When Securities Valuation is Computationally Hard By Shireen Tang and Shijie Huang and Elizabeth Bowman and Nitin Yadav and Carsten Murawski and Peter Bossaerts Draft: February 27, 2017 We study the Efficient Markets Hypothesis (EMH) in a setting where information heterogeneity emerges because securities valuation requires solving an NP-hard problem. We demonstrate experimentally that the quality of prices deteriorates substantially as computational complexity increases. Participants whose valuations are closer to true values earn more from trading. Participants improved their individual valuations by learning from market data, and their individual valuations on average were better than those reflected in market prices. These results are in sharp contrast with findings in experiments where correct valuation requires averaging of private information. They suggest that EMH only holds in very specific circumstances. Keywords: Efficient Markets Hypothesis, Computational Complexity, Financial Markets, Grossman-Stiglitz Paradox, Hirshleifer Effect, Intellectual Discovery, Patents, Prediction Markets All authors are of the Brain, Mind & Markets Laboratory, Department of Finance, Faculty of Business and Economics, The University of Melbourne, Victoria 3010, Australia. Corresponding author: Peter Bossaerts; peter.bossaerts@unimelb.edu.au. Bossaerts designed the experiment, Tang run experiment and conducted the statistical analysis; Bossaerts, Murawski and Tang wrote the manuscript; Huang, Bowman and Yadav contributed to software design and data acquisition. Financial support from a R@map chair at The University of Melbourne (Bossaerts) is gratefully acknowledged. The experiment reported here was approved by the University of Melbourne Human Research Ethics Committee (Ethics ID: ) and was conducted in accordance with the World Medical Association Declaration of Helsinki. All participants provided written informed consent. 1 Electronic copy available at:

2 2 XXX XXXX The Efficient Markets Hypothesis (EMH) is one of the foundations of modern asset pricing theory. Markets are called efficient when security prices fully reflect all available information (Fama, 1991, p. 1575). Theoretical analyses of this statement usually envisage a setting where the value of securities is a function of information that is distributed among market participants, and correct valuation requires aggregation of the different information sets (Grossman and Stiglitz, 1976). 1 In the rational expectations equilibrium of such an economy (the Radner perfect foresight equilibrium; Radner, 1972), prices perfectly reveal values, and hence, security prices fully reflect all available information. Starting with Plott and Sunder (1982, 1988), controlled experiments have confirmed this prediction in the case where security payoffs are pure common-value, or, if not, private payoff schedules are known, and markets are organised as centralised double auctions. 2 It has been pointed out that EMH cannot always hold. If information is costly to obtain, then no agent has any incentive to acquire information; everyone should just wait for prices to settle, at which point information could readily be obtained from prices. Informed agents, that is, those agents who acquired information, would not be able to recuperate the cost of their information acquisition (Grossman and Stiglitz, 1976). Sunder (1992) shows that this Grossman Stiglitz Paradox (GS Paradox) is not only of theoretical concern. He verified the prediction experimentally: participants did not acquire information if information came at a fixed cost; alternatively, the price of a fixed set of signals decreased to zero when the signals were auctioned. Here, we consider a fundamentally different setting. Correct valuations will not depend on aggregating bits of information, and in fact there will not even be 1 The market microstructure literature sometimes takes a different approach, where better-informed compete on the basis of the same information bit. See, e.g., Holden and Subrahmanyam (1992). There, the issue is whether prices eventually reveal that information bit, and how. See Bossaerts, Frydman and Ledyard (2014) for an experimental investigation. 2 The ability of decentralised markets to aggregate dispersed information is hotly disputed, with Wolinsky (1990) and Duffie and Manso (2007) taking opposite views. See Asparouhova and Bossaerts (2017) for experimental evidence. Electronic copy available at:

3 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 3 uncertainty. Instead, information heterogeneity emerges naturally because correct security valuation requires market participants to solve a problem of high computational complexity. We follow computer science convention and define computational complexity as the amount of computational resources required to solve a given problem (Arora et al., 2010). 3 Problems that require resources (usually in terms of time) beyond those available are referred to as computationally intractable (Cook, 1983). Here, we study a particular type of computational problem, and ask whether it is intractable for markets to solve. More precisely, our setting is one where the values of securities depend on the solution of different instances of the 0-1 knapsack problem (KP), a combinatorial optimisation problem (Kellerer, Pferschy and Pisinger, 2004). In the KP, the decision-maker is asked to find the sub-set of items of different values and weights that maximises the total value of the knapsack, subject to a weight constraint. The KP is computationally hard. There is no known algorithm that both finds the solution and is efficient, that is, can compute the solution in polynomial time. 4 Intuitively, it is easy to verify that a particular set of items achieves a given total value, but it is hard to find the set of items with the highest total value. 5 The KP permeates economic life, from low-level cognition (optimal inattention; Sims, 2006) to high-level cognition (portfolio optimisation; Kellerer, Pferschy and Pisinger, 2004). We therefore argue that it provides a realistic setting to test whether EMH holds in the presence of high computational complexity. Like computers, humans struggle to find solutions to instances of the KP; they differ in 3 Note that recent analyses of complexity in finance and economics either take a less formal approach (e.g., Skreta and Veldkamp, 2009), or model complexity in probabilistic terms (e.g., Spiegler, 2016). 4 In computational complexity theory, an algorithm is called efficient if the rate at which the amount of time to compute the solution grows as the size of the computational problem increases, is upper-bounded by a polynomial. The KP can be solved with dynamic programming. This algorithm is polynomial in the number of items; however, the memory required to implement dynamic programming grows exponentially. Therefore, dynamic programming substitutes exponentially growing time to address memory for exponentially growing time to compute. Experiments demonstrate that humans do not appear to use dynamic programming when solving the KP, which is not surprising: human working memory is very small (Murawski and Bossaerts, 2016). 5 Technically, the version of the KP used here is the optimisation version of the problem. The corresponding decision version of the problem is NP-complete. The optimisation version is at least as hard as the corresponding decision version.

4 4 XXX XXXX their ability to find solutions; and they display substantial heterogeneity in solution approaches (Murawski and Bossaerts, 2016). Therefore, the KP naturally generates a situation of information heterogeneity as a consequence of high computational complexity. Our experiment was organised as follows. We endowed participants with shares of several securities (10 or 12), each of which corresponded to an item in a given instance of the KP. All securities lived for a single period, after which they paid a liquidating dividend. The dividend equalled one dollar if the corresponding item was in the solution of the instance of the KP; and zero otherwise. After markets opened, participants traded the securities in a computerised continuous openbook system (a version of the continuous double auction where infra-marginal orders are kept in the system until cancelled). 6 All participants were provided with the same information about the instance to be solved, that is, each items value and weight, and the total capacity of the knapsack. They had access to a computer program where they could try out candidate solutions. 7 In the following, we report results that address the following questions. (i) Do security prices reveal the solution of the KP instance? That is, were securities corresponding to items in the optimal knapsack priced at $1, and others at $0? (ii) Did informed traders, defined as participants who found the solution of the instance, make money? 8 (iii) Do uninformed traders, defined as participants who did not find the solution, attempt to read information from prices? In particular, were uninformed traders able to improve their proposed solutions as a result of events in the marketplace? The remainder of the paper is organised as follows. In the next section, we summarise the experimental paradigm. We then present the results. In the third section, we discuss the implications of our findings, in particular: how they 6 We used the software Flex-E-Markets ( 7 The program is part of the ULEEF GAMES suite and and can be accessed at 8 It should be pointed out that, while some participants at times may have found the optimal solution, they might not have realised that it was indeed the optimal solution Murawski and Bossaerts (2016). This, of course, is a consequence of the nature of the KP.

5 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 5 relate to the Grossman-Stiglitz Paradox and the Hirshleifer Effect; how they shed new light on past tests of EMH on historical field data; how markets may be a mechanism to incentivise intellectual discovery; why prediction markets may sometimes fail; and what the findings mean for the Church-Turing thesis in the theory of computation (Church, 1934; Turing, 1936). We close with a concluding section. I. Experimental Design Participants Participants were recruited from the University of Melbourne community, in four experimental sessions with 18 (one session) or 20 (three sessions) participants per session. To be eligible, participants had to be current students of the University of Melbourne aged between 18 to 30 years old with normal or correctedto-normal vision. The final sample included a total of 78 participants (age range: 18 to 26 years, mean age = 22, standard deviation = 4, gender: 44 male, 34 female). The study was approved by the University of Melbourne Human Research Ethics Committee (Ethics ID: ) and was conducted in accordance with the World Medical Association Declaration of Helsinki. All participants provided written informed consent. Task Participants attempted five instances of the 0-1 knapsack problem, while simultaneously trading in an online marketplace. In each instance, participants selected items with given values and weights from a set, in order to maximise the total combined value within the weight constraint for the selection. Formally, participants were asked to solve the following maximisation problem: max x i I x i v i i=1 s.t. I x i w i C and x i {0, 1}, i=1

6 6 XXX XXXX where i, w, v and C denote the item number, item weight, item value and knapsack capacity, respectively. The number of items in an instance varied between 10 and 12. Instances were taken from two prior studies (Meloso, Copic and Bossaerts, 2009; Murawski and Bossaerts, 2016). The sequence of instances in an experimental session was counterbalanced across sessions. Parameters for the five instances are provided in Table 1. The KP instances were made available electronically on a computer interface where participants could try out different solutions. 9 The software recorded every move of an item into and out of the knapsack. Each item in an instance mapped into a security in the online marketplace. Therefore, between 10 to 12 markets were available in each instance. Exchange was organised using the continuous double-sided open book system, like most electronic stock markets globally. Trading was done on the online experimental markets platform Flex-E-Markets. 10 Participants traded for 15 minutes, or in later rounds, less. The user interfaces of the two systems, knapsack solver and online marketplace, are shown in Fig. 1. Sessions started with a brief motivation and a reading of the instructions, after which participants were given ample time to familiarise themselves with problem interface, online trading platform, and how to exploit, through trading, knowledge acquired when attempting to solve a practice instance. Motivation, instructions and practice took a total of one hour. After a break, we ran the (five) rounds that counted for final earnings. In total, a session took between two and two-and-a-half hours. Participant instructions including the timeline of a typical experimental session can be found in the Supplementary Online Material (SOM). We recorded price data of every order and trade in the marketplace, as well as every move into or out of the knapsack from each participant. Timestamps were 9 The application is part of a game suite called ULEEF GAMES; it can be acessed at 10 See

7 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 7 Figure 1. Knapsack Problem Interface and Online Trading Platform Note: Left: KS instance solution interface, where participants were shown the instance, and could attempt solutions by moving items from the OUT panel (right) to the IN KNAPSACK panel (left). Capacity, capacity used, and KS value were displayed on top (left). Right: Online markets platform. Each of the 12 colour-coded markets corresponded to an item in the KS, and promised to pay one dollar if the item was in the optimal solution. The book of limit orders was shown in the middle, listed by price level, and colour-coded (blue: bids; red: asks). To the right is the order form, where participants could submit buy and sell limit orders, or cancel previous orders.

8 8 XXX XXXX synchronised between the KP solver and the online market. 11 Participant incentives Participants took positions in the items they believed to be in the optimal knapsack by buying shares of the corresponding security. They could also sell shares corresponding to items they believed not to be in the solution; short sales were not permitted, however. In every instance, participants were endowed with $25 in cash holdings (Australian dollars, approximately eighteen U.S. dollars), and 12 shares randomly allocated to securities. The price range of a share was bounded between $0 and $1. Final earnings consisted of: (i) liquidating dividends for the shares held at market close; (ii) any change in cash holdings between the beginning and end of trading. Earnings were cumulative across instances. Additionally, participants received a fixed reward ($2) for submitting a proposed solution through the KP instance rendering software, as well as a show-up fee of $5. Initial Allocations We designed initial allocations of securities to induce trade, by concentrating individual endowments in particular markets. While initial allocations were randomised, they were fair in the sense that all participants received the same number of shares in correct items across the five instances. Initial allocations were such that $31.20 in liquidating dividends were paid per participant on average. Participants were not told that they had fair initial allocations. We imposed fairness in the belief that earnings would suffer from the Hirshleifer Effect if EMH were to emerge (see Discussion for more details). Although the concepts of risk and risk aversion have yet to be defined precisely in the context of computational complexity, we intuited that uncertainty 11 The marketplace server became unreachable during the second round in the final session, so we have no trade data for that round. This denial of service originated with the server service provider, and hence was beyond our control.

9 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 9 about the solution to a KP instance would induce participants to diversify holdings across multiple securities. As a result, participants would trade not only because of perception of superior information. We eliminated aggregate risk by ensuring that there were an equal numbers of shares across securities. This was meant to avoid price distortions that could arise from differences in relative supplies. Additionally, payment for submission of a solution through the KP interface was fixed and independent of whether the submission was correct. This made it impossible for participants to hedge between trading in the marketplace and submission of solutions through the KP interface. Informed and Uninformed Traders All participants were given the same information about the KP instance in a session. Thus, there was no information heterogeneity and no information asymmetry ex ante. However, we did not provide participants with the solutions, and since our instances were hard while performance was variable (the proportion of participants who submitted the correct solution varied between 6.4% and 60.3%; see SOM for details), information heterogeneity (and asymmetry) arose spontaneously as participants started to search for the correct solution. We define a participant who submitted the correct solution as an informed trader, and one who did not, as an uninformed trader. 12 II. Results Descriptive Statistics Each of the 78 participants solved five instances of the KP (390 attempts in total). We first looked at computational performance of participants, that is, participants ability to find the optimal solutions of instances. To this end, we 12 As mentioned before, the nature of the KP is such that even informed traders may not have been aware that they found the solution.

10 10 XXX XXXX examined the proportion of participants that were able to solve an instance. Overall, 37.2% of attempts were correct. Performance varied both by participant (min = 0, max = 1, SD = 0.26) and experimental session (min = 0.33, max = 0.47, SD = 0.06). Next, we investigated whether computational performance in an instance was related to the instance s complexity. We measured instance complexity with Sahni-k. This metric increases with both the number of computational steps and the amount of memory required to solve an instance. Intuitively, Sahni-k is equal to the number of items that have to be selected into the knapsack before the knapsack can be filled up using the greedy algorithm to find the solution. The greedy algorithm fills the knapsack by selecting items in decreasing order of the ratio of value over weight until none of the remaining items fits into the knapsack. If Sahni-k equals 0, the greedy algorithm generates the solution of the instance. If k is greater than 0, the Sahni algorithm generates all feasible k-element subsets, fills up the knapsack using the greedy algorithm and finds the set of items with the highest total value. The Sahni-k of the instances in this study varied from 0 to 4; they are listed in Table The proportion of participants who solved the instance correctly decreased from 60.3% when Sahni-k was equal to 0, to 6.4% when Sahni-k was equal to 4. To test the negative relation between computational performance and Sahni-k, we estimated a mixed-effects model with a binary variable set to 1 if an attempt was correct as dependent variable (0 otherwise), a fixed effect for Sahni-k and random effects (varying intercepts) for participant and experimental session. We found a significant main effect of Sahni-k (β = 0.578, p < 0.001). The pattern confirms the validity of Sahni-k as a measure of instance difficulty for humans, first documented in Meloso, Copic and Bossaerts (2009) and Murawski and Bossaerts 13 Note that the number of sets the algorithm considers is given by the binomial coefficient with n equal to the number of items in the instance and k equal to Sahni-k. Note that the number of subsets being considered increases with k and quickly grows beyond the capacity of human working memory. For an instance with 12 items, the number of sets considered if k is equal to 1, is 1 whereas if k is equal to 4, the number of sets considered equals 495.

11 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 11 (2016). 14 Importantly, the presence of markets does not invalidate the correlation between Sahni-k and computational performance (see SOM for further details). We used the number of items participants moved into and out of the knapsack as a proxy for effort. The mean total number of moves in an instance was 24.7 (min = 3, max = 123, SD = 19.5; descriptive statistics can be found in SOM). To test whether the number of moves depended on instance complexity, we related the number of moves a participant made, to Sahni-k of the instance (mixedeffects model with a fixed effect for Sahni-k and random effects for participant and session). We found a positive effect of Sahni-k on the number of item moves (β = 1.649, p < 0.05). This means that participants expended more effort on harder instances. This finding is consistent with Murawski and Bossaerts (2016), who also found a positive correlation between proxies of instance difficulty and effort, using a larger number of instances than in the present study. The mean number of trades per session was (min = 91, max = 202, SD = 28.0). The mean number of trades varied by both session (min = 117.4, max = 161.6, SD = 18.9) and instance (min = 127.2, max = 165.7, SD = 17.0). The number of trades did not vary with instance difficulty as measured with Sahni-k (one-way ANOVA, F (4, 14) = 1.27, p > 0.1). The number of trades in items in the optimal solution did not differ from the number of trades in items that were not in the optimal solution (two-sample t-test, t(52) = 0.811, p > 0.1). Fig. 2 plots the evolution of trade prices in the third round of the first session. Each security is indicated by the weight and value of the corresponding item (weight value) and whether the item is in the optimal knapsack. Notice how prices do not monotonously decrease towards zero (indicating that the corresponding item is OUT ) or one (the corresponding item is IN ). Shares in item , for instance, bounce back and forth between a minimum of 25 cents and a maximum of 95 cents. By the end, they were trading at 60 cents. The shares expired 14 For a comparison of Sahni-k as a measure of instance complexity with other measures, see Murawski and Bossaerts (2016).

12 12 XXX XXXX worthless because the corresponding item was not in the optimal knapsack (more descriptive statistics on prices, including range of prices per item, stratified by instance difficulty, are available in the SOM). First Session, Instance _28 (1)IN 0.9 Traded Price _9 OUT 77_3 (2) OUT 184_25 IN 229_31 IN 219_24 OUT 66_10 IN 184_28 (2) IN 129_15 OUT 72_1 OUT _14 OUT 0 06:47:31 06:48:58 06:50:24 06:51:50 06:53:17 06:54:43 06:56:10 Time (hr:min:sec) Figure 2. Evolution of Traded Prices in One Instance. Note: Shown are time series of transaction prices for the 3rd instance in the first session. Series are identified by the weight and value of the corresponding item (weight value) and whether the item was in the optimal knapsack ( IN, in which case the shares paid one dollar; when OUT, they expired worthless). Finally, participants earned $6.32 on average per instance (min = 1.66, max = 11.41, SD = 1.92) and on average in total (min = 8.30, max = 49.25, SD = 9.59) from trading in the market. 15 To test whether the earnings from trading depended on instance complexity, we related earnings of a participant in an instance, to Sahni-k of the instance (mixed-effects model with a fixed effect for Sahni-k and random effects for participant and session on the intercept). We did not find a systematic relation between Sahni-k and earnings from trading (β = 0.177, p > 0.1) Instance earnings could be negative because participants were allowed to buy, and hence spend cash, on shares that eventually did not pay a dividend; the change in cash was subtracted from total dividends earned on final share holdings. Earnings cumulated across instances. At the end of the experimental session, participants were paid the minimum of $25 and cumulative earnings plus sign-up reward of $5 plus per-instance submission reward of $2. 16 Mean earnings were highest in the instance with Sahni-k equal to 0 and lowest in the instance with Sahni-k = 2. Mean earnings amounted to $8.41, $5.13, $5.03, $5.93 and $7.08 for Sahni-k 0 to 4,

13 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 13 In the remainder of this section, we report several tests of EMH. We first consider the question whether security prices revealed security values, that is, whether they revealed the optimal solution of an instance. This is a direct test of EMH. We then examine whether participants who found the optimal solution were able to profit from their information. This is an indirect test of EMH. Finally, we examine whether uninformed traders benefitted from the information revealed by security prices. Issue 1: Did Securities Prices Reveal The Optimal Solution? To evaluate whether prices correctly reflected the solution of an instance, we constructed a market performance metric as the distance, in item space, of market solutions implied by trade prices from the optimal solution. Distance in item space is measured as a score which is incremented by one point if a correct item was in the submitted knapsack or an incorrect item was left out of the knapsack. The score is subsequently scaled by dividing by the total number of items in the knapsack. To construct market solutions, we interpreted the last traded price of an item as the market s belief that the item belonged in the optimal solution. We then bootstrapped a market knapsack by drawing without replacement items based on these beliefs and filling the knapsack until capacity was reached. We computed the performance score of this market knapsack. We repeated this procedure 10,000 times. We then averaged the resulting market performance scores across the bootstraps. If prices correctly valued securities, and hence, correctly revealed an instance s solution, we would draw only from IN items, and hence, obtain a perfect performance metric. Information revealed in market prices allowed us to reach a performance score of 78% in case of the instance with Sahni-k equal to 0. This score decreased respectively. Mean maximum payoff was $21.05 in the instance with Sahni-k equal to 0, compared to $12.50, $13.70, $16.25, and $11.95 for instances with Sahni-k equal to 1 to 4, respectively. Notice the non-monotonicity of earnings. This is related to the way we allocated shares, which, as mentioned before, attempted to balance incentives to trade and fairness.

14 14 XXX XXXX monotonically to 68%, 64%, 57% and 52% as Sahni-k increased from 1 to 4. We conclude that market prices were never efficient in the sense of EMH as they never revealed the optimal solution with perfect accuracy. In the next step, we examined whether the market did better in solving the instances than individual participants. To this end, we compared the performance of the market knapsack to the performance of the knapsacks submitted by individual participants. To do so, we constructed performance scores for individuals the way we did for the market. We found that the market knapsack performed worse than the knapsack submitted by the average participant for every level of difficulty (Fig. 3). In three out of five instances, the market s score was significantly worse (two-sample t test with unequal variances; Bonferroni-Holm family-wise error correction at the p = 0.05 level). Performance of the market decreased with difficulty (slope = 0.06, p < 0.001), at the same rate as for individual participants. Note that except for the most difficult instance (Sahni-k = 4), there was always at least one participant who found the optimal solution (in the instance where Sahni-k was equal to 0, the majority of participants found the solution). This means that the information to value securities correctly was available among participants. Yet market prices never allowed us to construct an average market knapsack with a performance better than the average submitted by participants. EMH failed so badly that the market did not even do as well as the average participant. Issue 2: Do Informed Traders Make Money? To perform our second test of EMH, we correlated individual computational performance in an instance with earnings (in Australian dollars) from trading in the marketplace. We computed the former as distance in item space of submitted knapsack from the optimal solution, using the score computed as described above. The mean payoff in an instance for informed participants (those who found the

15 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 15 Figure 3. Performance Of Instance Solutions Generated From Last Trade Prices Compared To Average Individual Submissions. Note: Market performance score, measured as distance in item space of market knapsack obtained from simulations based on last trade prices (blue), against corresponding average score of individual participants (red), across all sessions, stratified by instance difficulty (Sahni-k). Error bars cover +/- 1 standard error (Market: across sessions; Individuals: across individuals in all sessions). Dashed lines are corresponding regression lines. Asterisks in boxes indicate that the average individual score is significantly higher than market score at p = 0.05 significance level, corrected for multiple comparisons.

16 16 XXX XXXX solution of the instance) was $8.21 (min = 0.85, max = 21.05, SD = 3.51), compared to $5.11 (min = 10.55, max = 13.7, SD = 3.93) of uninformed participants. Moreover, in every instance, the highest payoff among all participants was earned by an informed participant. To test whether informed traders were able to make money, we related computational performance, as measured by the score described above, to earnings from trading. A higher score implies that the participant had more correct items in the submitted knapsack and more incorrect items out of the submitted knapsack, and as such, the participant knew the correct value of more securities than someone with a lower score. We estimated a generalised linear mixed-effects model with earnings in a session as dependent variable, a fixed effect for score (computational performance) and random intercepts for participant and instance. We found a significant effect of score (β = 7.011, p < 0.001). A ten percentage point increase in computational performance score was associated with additional earnings of 70 cents (see Fig. 4, top panel). Considering overall session earnings per participant and computational performance, we found that they were highly correlated: an increase in performance score of 10 percentage points increased earnings in the marketplace by almost $4 on average (Fig. 4, bottom panel). Note that only two participants received a perfect score, implying that only two participants solved all five KS instances correctly. To summarise, our comparison of earnings of informed against uninformed traders, and the analysis of the effect of superior computational performance both suggest that EMH did not hold: better-informed traders made significantly more profit; and participants whose submitted knapsacks were closer to the optimal solution (in item space) generated higher earnings from trading.

17 Payoff ($) VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY r: Sahni-k 0 Sahni-k 1 Sahni-k 2 Sahni-k 3 Sahni-k coeff: (p < 0.001) Performance score (scaled to 1) 60 r: Total session payoff ($) coeff: 37.1 (p < 0.001) Performance score (scaled to 1) Figure 4. Individual Trading Payoff Against Performance Score Of Submitted Knapsack. Note: Individual payoff from trading, per instance (top; colour-coded by difficulty) and per session (bottom), against average performance score for the submitted solutions; performance score is measured as distance in item space of the submitted knapsack; see text for details; per-instance observations are stratified by Sahni-k. Slope coefficient per instance (mixed-effects regression; see text) equals 7.01 (p < 0.001); Pearson correlation r = 30.3%. Slope coefficient per participants equals 37.1 (p < 0.001); Pearson correlation r = 42.2%.

18 18 XXX XXXX Issue 3: Do Uninformed Traders React To Information Reflected In Prices? In the rational expectations equilibrium used to provide empirical content to EMH, uninformed traders are assumed to know the mapping from states to prices and use this mapping to infer states from observed prices. In this setting, the term state refers to an expectation based on all the information available in the economy; here, it should be interpreted as the correct solution to the KS instance. We investigated whether uninformed traders indeed read information from prices. Specifically, we study to what extent trade induced uninformed traders (those that did not submit the optimal solution) to re-visit and improve their knapsack, moving it closer to optimum in item space. Overall, descriptive statistics of moves of correct and incorrect items in and out of knapsacks can be found in SOM. Prior research has shown that an important reason why humans may not find the optimum is because they tend not to re-consider incorrect items that they put into the knapsack early on (Murawski and Bossaerts, 2016). Poor episodic memory may explain this reluctance to re-visit early moves. Here, we ask whether trade in such items made it more likely that uninformed traders took them out. To test whether participants improved their knapsack based on information available in the market, we regressed the probability of removing an incorrect item that was included early on, onto the price of, and the trading volume in, the corresponding security, as well as their interaction using a generalised linear model with a logit link function. 17 We consider an item to be included early on if the participant moved it into the knapsack within the first two minutes of trading. We found that there was a significant main effect of price (β = 3.491, p < 0.001). This means a 10% decrease in price of a security was associated with a 11% increase in the probability of removing the corresponding item from the knapsack. The effect was stronger when the security corresponding to the 17 Model selection analysis based on the Bayesian Information Criterion suggested that a model with random effects for participant, instance difficulty (Sahni-k) and session fitted worse than a fixedparameter regression.

19 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 19 item was traded more heavily (interaction term of price and trading volume, β = , p < 0.05). However, there was no main effect of trading volume (β = 7.125, p = 0.068). The negative relation between security price and probability of removal is displayed in Fig. 5 (univeriate LS fit with hyperbolic link function; see caption). Figure 5. Impact Of Trade Prices On Removal Of Incorrect Items Note: Incorrect items chosen into knapsack within two minutes of trading: Fraction of those items that are eventually removed, as a function of average trade price. Red line: least squares fit of β/(1 + x) 2 where x = price level, β = 0.85 (p < 0.01). These findings suggest that trade, combined with low prices, induced participants to re-consider incorrect inclusions of items, thus improving overall computational performance, and hence, securities valuation. We did not find an effect on KS choice from trade in high-priced securities.

20 20 XXX XXXX We conjecture that short-sale restrictions contributed to the asymmetry between high-priced and low-priced securities: if an item was deemed to be overpriced, participants could only sell shares that they already owned, and hence could not put more pressure on prices. III. Discussion According to EMH, markets are called efficient when security prices fully reflect all available information (Fama, 1991, p. 1575). As a result, traders with more or better information cannot profit from it. We report results from a markets experiment aimed at testing EMH where heterogeneous information emerges spontaneously due to high computational complexity. We found that prices did not reveal security values, and that informational efficiency deteriorated as instance complexity increased. Market prices could not even be used to construct candidate solutions (knapsacks) that were closer to the optimal solution than those submitted by the average participant. We also document that informed traders (those who submitted the optimal solution) earned significantly more from trading on their information, and that uninformed traders (those who did not submit the correct solution) earned more the closer their submission was to the solution. Closeness was measured in terms of the number of correct items in the submission and the number of incorrect items left out of the submission. Finally, we discovered that prices, in conjunction with trade, fed back into problem solving: heavily traded securities with low prices induced uninformed traders to re-consider incorrect items they had put in their knapsack early on. Consequently, while EMH did not hold, one core principle of the traditional theory behind EMH was upheld, namely, that uninformed traders read information from prices. Still, this principle is too narrow, because uninformed traders watched volume, in addition to prices, to improve their valuations. Tests of EMH in principle require that profits be adjusted for risk (Fama, 1991). Controlled experiments, like in the pioneering studies (Plott and Sunder, 1982,

21 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY ) and in our investigation, do not suffer from this shortcoming. We did not have to extract risk-adjusted returns because all participants were equal ex ante, in terms of allocations, knowledge and attitudes towards risk, uncertainty and complexity. We hasten to add that notions like risk and ambiguity aversion are ill-defined in our setting since there is neither risk nor uncertainty, and the true meaning of complexity aversion, to our knowledge, is yet to be explored. To conceptualise computational complexity, we used standard notions from computer science. These notions are based on a theoretical computational model (Turing machine), and it was not a priori obvious that they would extend to human computing. Recent evidence suggests that computational complexity theory does extend to decision-making by humans (Murawski and Bossaerts, 2016), and the findings reported here corroborate this notion. Therefore, evidence is mounting that complexity theory is universal, in support of the Church-Turing thesis (Church, 1934; Turing, 1936). In the present study, we show that computational complexity theory extends even to markets: we found that prices violated EMH more when valuation had higher computational complexity, where complexity was measured in terms of Sahni-k. This metric, one possible measure of computational resources required to solve an instance, tracks human performance in solving KP instances (Murawski and Bossaerts, 2016). Our finding confirms earlier experiments on the use of markets as a mechanism to solve computationally hard problems (Meloso, Copic and Bossaerts, 2009). We divided our participants into a group of informed and uninformed traders, based on whether they submitted the optimal solution. It is important to realise that informed traders may actually not have been aware that they knew the solution. The nature of the KP is such that the only way to ascertain that a candidate solution is the optimal solution, is to solve the instance. Evidence from prior studies suggests that even when participants submitted the optimal solution, they were often not aware that their solution was indeed optimal (Meloso,

22 22 XXX XXXX Copic and Bossaerts, 2009; Murawski and Bossaerts, 2016). 18 Given that our experiment only had participants per trading session, it could be argued that our results are due to insufficient competition or liquidity. However, other experiments have shown that EMH can be obtained in settings with fewer participants (Plott and Sunder, 1982, 1988). In those experiments, valuation only required simple aggregation of heterogeneous information (through averaging). We show that EMH does not hold when security valuation is computationally hard. In the context of market efficiency, the Grossman-Stiglitz (GS) Paradox is the biggest obstacle to informational efficiency (Grossman and Stiglitz, 1980). The GS Paradox refers to the inability of agents to recuperate the cost of information gathering when market prices satisfy EMH. Our experiments clearly show that better-informed traders earned more, and hence, could have recuperated costs if we had charged them. As such, the GS Paradox does not extend to a situation where information heterogeneity stems from high computational complexity. Importantly, we found that uninformed traders learned from prices. This is in sharp contrast with Asparouhova et al. (2015). There, correct security valuation required one to solve a computationally simple but highly non-intuitive Bayesian problem (variations on the so-called Monty Hall problem). When faced with prices that were at odds with their beliefs, participants did not learn from prices, but instead retreated from exposure to risk by trading to portfolios with valuations that did not depend on the Bayesian problem at hand. In contrast, here, we report that uninformed traders got cues from more heavily traded, low-priced securities, nudging them to re-consider whether to delete the corresponding items from their knapsack. Feedback from trading to problem solving may explain why earlier experiments showed that markets can be used to help people solve KP instances. Meloso, 18 It is highly unlikely that humans are able to implement the algorithms to compute the optimal solution, such as dynamic programming or brute-force search (Murawski and Bossaerts, 2016).

23 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 23 Copic and Bossaerts (2009) report that more participants managed to find the correct solution of KP instances when incentives were similar to those in our experiment, namely, one had to trade to make money. The benchmark incentive scheme in that study was one were only the first to submit the optimal solution would be paid a fixed prize. This prize was significant: it amounted to the sum total of dividends paid in the markets treatment. Fewer participants found the correct solution under the prize treatment. Prices in our experiment were extremely noisy. Yet participants learned from prices. Eventually, they got ahead of the market: they constructed knapsacks that were better on average than those revealed by market prices. We conclude that prices are not a reliable metric for how well the average market participant fares. It confirms an emerging finding in experimental finance, namely, that participants may be well off even if prices are wrong (Asparouhova et al., 2016). We discovered that markets mitigate one of the strongest biases that keep individuals from discovering the correct solution of KP instances, namely, hesitation to re-consider items that were added to the knapsack early on (Murawski and Bossaerts, 2016). In the experiment reported here, pricing, in conjunction with trade, made participants re-visit parts of the solution that they had constructed within the first two minutes of trading. It is interesting to note that this explanation differs from the conjecture in Meloso, Copic and Bossaerts (2009) as to why markets cause better problem-solving. There, it was suggested that success in a prize system hinged on the belief that one is good enough to sometimes be the best, a belief that most people would never entertain. In contrast, in markets, one merely has to believe to be better than the median, something the majority does believe, a situation known as the overconfidence bias (Kahneman and Tversky, 1977). As such, participants worked harder in the markets treatment. Here, we show that there was an effect beyond mere market participation, namely, trading activity provided valuable cues that improved individual problem solving. Altogether, our findings demonstrate that markets may provide more powerful

24 24 XXX XXXX incentives to solve complex problems than a prize system. The prize system in Meloso, Copic and Bossaerts (2009) is analogous to the current patent system, and hence, our findings are relevant for the debate on the desirability of patents as a way to promote innovation. Indeed, intellectual discovery can be thought of as the solution of a combinatorial optimisation problem such as the KP (Boldrin and Levine, 2002). A recent empirical study corroborates this claim, by showing that patents filed with the U.S. Patent Office between 1790 and 2010 were mostly for inventions that combined existing technologies in novel ways rather than opening up fundamentally new avenues of exploration (Youn et al., 2015). Our findings suggest that markets may promote innovation better than a prize system. To illustrate this notion, consider markets in Li, Na, Mg and other chemicals (we are using standard chemical abbreviations). They are potential components of future battery technology. To determine which chemicals markets to invest in requires one to assess which component, or maybe combination of components, will be required for the best battery technology. This casts intellectual discovery squarely in terms of the KP and the markets we designed to profit from finding the optimum. Inventors are induced to participate in the marketplace, and earn money by buying those components that they believe are in the best battery technology, while selling others. If EMH held, inventors would not be able to make money from their knowledge. Other incentives, like prizes or patents, would be needed to incentivise discovery. However, we found that EMH does not hold when valuation depends on solving a KP. Hence, markets can provide incentives to innovate. This conclusion is important for economic history as well. It is generally accepted that the patent system provided the main impetus for technological advances over the last century and a half (Khan and Sokoloff, 2001). However, at the same time markets penetrated all parts of life, and it may very well have been that markets were the major facilitator of innovation rather than patents. This conjecture is consistent with historical evidence that technological advances

25 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 25 can be far bigger during epochs with share trading but without patents than in epochs with only patents (Nuvolari, 2004). The failure of prices to reveal all available information resolves another problem with EMH, namely, the Hirshleifer Effect (Hirshleifer, 1971). This is the detrimental effect on welfare that agents experience when they happen to be endowed with assets that prices reveal to be of little value. In the battery example above, if Mg emerges as useless for the best technology and this is immediately revealed in prices, agents endowed with Mg will have poor terms of trade from the beginning. They would have wished to be able to take out insurance before initial endowments were revealed. The Hirshleifer Effect is to be counted with in traditional experiments on EMH, where bits of information are spread across participants, but when averaged, provide the best estimate of the value of the securities at hand. Participants end up with vastly different total payoffs (and will complain) unless the experimenter deliberately rigs the initial endowments to generate equal ex ante earnings. For more discussion, see, e.g., Asparouhova and Bossaerts (2017). Here, too, we assigned initial allocations in fair ways, to guard against participant dissatisfaction with our experiment in case EMH were to obtain and the Hirshleifer Effect were to bite. Because EMH did not obtain, participants did not complain, and with hindsight, we could have dispensed with the complication of ensuring fair initial endowments. Our findings allow one to put into perspective the evidence on EMH from historical analyses of field data. Fama (1991, 1998) surveys a vast body of studies that appears to suggest that EMH holds, because anomalies could be attributed to chance (sometimes the null will be rejected) or to methodological errors. Behavioural finance scholars reject this conclusion, starting with Bondt and Thaler (1985), who claimed that long-run price reversals are caused by overreaction. Among others, Hirshleifer (2001) summarises the evidence against EMH. Our experiment should shed new light on the EMH controversy. When heterogeneous

26 26 XXX XXXX information emerges because agents hold dispersed information from which security prices can easily be computed, markets can be expected to satisfy EMH. But it is obvious that not all real-world situations can be described as such. Instead, correct valuation may be computationally hard, at which point the conclusions from our experiment become important, and violations of EMH are to be expected. In the traditional instantiation of EMH, information is dispersed and correct valuation is feasible and merely requires averaging this information. Computer scientists would describe this as a situation of finite sample complexity (Valiant, 1984) (combined with low computational complexity). There, one is with high chance close to the correct valuation even with a finite sample, a notion referred to as probably approximately correct (Valiant, 1984). This contrasts with hard computational problems like the KP, where a finite number of random attempts will not necessarily get one closer to the solution. With this problem categorisation in mind, our conjecture could be translated as follows: high computational complexity causes violations of EMH, while EMH obtains in situations of finite sample complexity (and low computational complexity; or when valuation depends only on simple but non-intuitive arguments, as discussed before; see Asparouhova et al., 2015). We leave it to future work to verify this conjecture. To illustrate how price evolution under high computational complexity could be dramatically different from that under finite sample complexity (and low computational complexity), we simulated markets and assumed prices gradually incorporated the accumulated information. First, we considered finite sample complexity. There, repeated sampling gets one closer to the truth: the chance that one deviates too far from knowing the truth decreases with sample size. Imagine there are ten securities that pay a liquidating dividend of 1 or 0 dollars, with equal chance. Each period ( trial ), the representative agent receives a signal, drawn from a Bernoulli distribution with p = 0.70 if the final dividend is 1, and with p = 0.30 otherwise. The agent uses Bayes law to update beliefs about the final

27 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 27 payoff. Assuming a quadratic loss function, security values equal the posterior mean, starting from an unconditional estimate of 0.5. The top panel of Fig. 6 shows how valuations of the securities quickly separate between those that will end up paying 1 and those that expire worthless. As time progresses, the chance of one valuation to veer off (say, from close to 1 down to 0) is drastically reduced. That is, even for finite samples (finite number of signals), the valuation estimate is probably approximately correct. This is exactly what is meant with finite sample complexity. The bottom panel of Fig. 6 illustrates high computational complexity. There, security values depend on the solution of an instance of the 0-1 knapsack problem. The instance is #8 in Murawski and Bossaerts (2016). We assume that the representative agent constructs estimates of the security values as follows. Each trial, the agent randomly tries a subset of the items that fills the knapsack to capacity. The agent then computes the value of the knapsack and compares to the maximum value obtained in previous trials. If the new value is less than the previous maximal value, then signals of the value of the securities equal 1 for those that were in the previous solution, and 0 otherwise. If the new value is higher than the previous maximal value, then signals of the value of the securities equal 1 for the in securities in the current trial, and zero otherwise. In analogy with Bayesian updating (but rather arbitrarily see below), the agent updates security valuations in trial t by weighting the valuation in the previous trial t 1 by (t 1)/t and the signal in the present trial by 1/t. The right panel of Fig. 6 shows how the resulting valuations evolve over time. Unlike under finite sample complexity, even after 30 trials, it is still unclear which item is in the optimal solution (which security will pay one dollar). Worse, valuations can readily move from high to low and v.v. even in later trials. Since there are only a finite, albeit enormous, number of possible capacity-filled knapsacks, the algorithm eventually finds the optimal solution. This will happen when, just by chance, the best knapsack is drawn. (In the present situation, there are 82 pos-

28 28 XXX XXXX 1 Values are Value estimate Trials Optimal: 2,5, Value estimate Trials Figure 6. Simulations Of Market Prices Under Finite Sample Complexity (Top) And Computational Complexity (Bottom). Note: Top: Evolution of prices of ten securities in a situation of finite sample complexity; as sampling increases over time, the probability of being approximately correct ( PAC ) increases. This is the situation in traditional theoretical analyses of EMH. Terminal values of the 10 securities are listed on top, in ascending order of security number. Bottom: Evolution of prices of ten securities in a situation of a computationally hard valuation problem; as sampling increases over time, the probability of being correct even approximately does not increase gradually. Only the securities with numbers listed on top ( Optimal ) pay a liquidating dividend of 1; all others expire worthless. See text for details.

29 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 29 sible capacity-filled knapsacks, so the chance of drawing the best knapsack in any trial equals 1/82; this implies, among others, that the chance that the algorithm finds the optimal knapsack within 30 trials is about 5%; it takes approximately four hundred trials to bring this chance up to 95%.) There is no suggestion that markets actually produce prices as in the figure. For one thing, random sampling of at-capacity knapsacks is inefficient: far more efficient procedures can be used; they are discussed in, among others, Murawski and Bossaerts (2016). Yet, the similarity between the price evolution depicted on the right side of Fig. 6 and that obtained in our experiment (see Fig. 2) is unmistakable. Much theory still needs to be developed. Matters could be complicated if there is more than one solution to a KS instance. Such was the case in our fifth instance (see Table 1), though one solution (the one we used to define securities payoffs in the experiment) dominated in terms of weight (total weight was less) and complexity (the Sahni-k measure of the alternative equalled 6). Often, there are multiple solutions that produce the same value and weight, and require equivalent computational resources. It is not immediately obvious how markets would price those. In the tradition of Savage (Savage, 1972), one could argue that, if the solution to the knapsack problem is unknown, it should be treated as a random variable, endow it with a prior (possibly uninformed) distribution, sample, and update the prior using Bayes law. This approach is infeasible, however, because it requires one to specify the likelihood (of observing sampled knapsack values given the true solution) and this, in turn, requires one to have solved the KP in the first place. Computationally hard problems cannot be solved effectively using the approach that is at the basis of traditional analyses of EMH, namely, the Bayesian approach. The evidence in favour of EMH from earlier laboratory studies has led to the emergence of a new type of information aggregation device in the field, namely, prediction markets. These markets are purposely designed to aggregate the infor-

30 30 XXX XXXX mation that is out there, i.e., to harvest the wisdom of the crowds. Successful field implementations abound (Arrow et al., 2008), but instances have emerged where market prices did not predict the outcome, such as the Brexit vote in the U.K. or the presidential election in the U.S. in Fundamentally, the power of prediction markets depends on the validity of EMH. Our research suggests that prediction markets will serve their purpose if the situation is one of finite sample complexity and low computational complexity. IV. Conclusion We provide evidence that EMH does not obtain when heterogeneous information emerges because valuation requires the solution to a computationally hard problem. Prices reflect valuations that are worse than even those of the average participant. Participants who were closer to the correct valuation made more money through trading. And those participants who were further away, combined volume data with price data to improve their value estimates. The framework of our analysis, computational complexity theory, is substantially different from the one that has been the context for formal analysis of EMH. In computer science, the latter would be categorised as finite sample complexity. Past experiments have shown that the theory (of EMH) holds well under finite sample complexity, to the point that there are no incentives for anyone to gather information when it is costly to do so. Computational complexity is high in many real-world decision-making contexts. Even if one is in a setting of finite sample complexity, mere computation of the aggregates may already be intractable computationally. 19 Consequently, we could argue that our findings are relevant for most realistic cases; EMH can be expected to obtain only in specific, and possibly rare, circumstances. Much needs to be done in order to better understand our new framework. In the 19 Computer scientists have started to add a computational requirement to the definition of finite sample complexity: one is to be Probably Approximately Correct (PAC) in finite samples only in ways that can easily be computed.

31 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 31 first place, we need a model of asset pricing. The model underlying our simulation (see Fig. 6) is rudimentary, though it already generates price behaviour that is not unlike in our experiments (compare with Fig. 2). Our framework casts new light on old issues. It provides a completely novel way of interpreting the empirical record of EMH in field data. It invites economists to re-visit incentives in intellectual discovery. And it forces one to re-think the wisdom of the crowds and the promise of prediction markets. REFERENCES Arora, Sanjeev, Boaz Barak, Markus Brunnermeier, and Rong Ge Computational complexity and information asymmetry in financial products Arrow, Kenneth J, Robert Forsythe, Michael Gorham, Robert Hahn, Robin Hanson, John O Ledyard, Saul Levmore, Robert Litan, Paul Milgrom, Forrest D Nelson, et al The promise of prediction markets. Science, 320(5878): 877. Asparouhova, Elena, and Peter Bossaerts Experiments on Percolation of Information in Dark Markets. Economic Journal, in press. Asparouhova, Elena, Peter Bossaerts, Jon Eguia, and William Zame Asset pricing and asymmetric reasoning. Journal of Political Economy, 123(1): Asparouhova, Elena, Peter Bossaerts, Nilanjan Roy, and William Zame Lucas in the Laboratory. Journal of Finance, 71: Boldrin, Michele, and David Levine The Case against Intellectual Property. The American Economic Review, 92(2): Bondt, Werner F. M. De, and Richard Thaler Does the Stock Market Overreact? The Journal of Finance, 40(3): pp

32 32 XXX XXXX Bossaerts, Peter, Cary Frydman, and John Ledyard The Speed of Information Revelation and Eventual Price Quality in Markets with Insiders: Comparing Two Theories*. Review of Finance, 18(1): Church, Alonzo The Richard Paradox. The American Mathematical Monthly, 41(6): Cook, Stephen A An overview of computational complexity. Communications of the ACM, 26(6): Duffie, Darrell, and Gustavo Manso Information percolation in large markets. The American economic review, 97(2): Fama, Eugene Market Efficiency, long-term returns, and behavioral finance. Journal of Financial Economics, 49: Fama, Eugene F Efficient Capital Markets: II. The Journal of Finance, 46(5): pp Grossman, Sanford, and Joseph Stiglitz On the Impossibility of Informationally Efficient Markets. The American Economic Review, 70(3): Grossman, Sanford J., and Joseph E. Stiglitz Information and Competitive Price Systems. The American Economic Review, 66(2): Hirshleifer, David Investor Psychology and Asset Pricing. The Journal of Finance, 56(4): Hirshleifer, J The private and social value of information and the reward to inventive activity. American Economic Review, 61(4): Holden, Craig W, and Avanidhar Subrahmanyam Long-lived private information and imperfect competition. The Journal of Finance, 47(1):

33 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 33 Kahneman, Daniel, and Amos Tversky Intuitive prediction: Biases and corrective procedures. DTIC Document. Kellerer, Hans, Ulrich Pferschy, and David Pisinger Knapsack Problems. Springer Science & Business Media. Khan, B Zorina, and Kenneth L Sokoloff History lessons: the early development of intellectual property institutions in the United States. The Journal of Economic Perspectives, 15(3): Meloso, Debrah, Jernej Copic, and Peter Bossaerts Promoting Intellectual Discovery: Patents vs. Markets. Science, 323: Murawski, Carsten, and Peter Bossaerts How Humans Solve Complex Problems: The Case of the Knapsack Problem. Scientific Reports, 6. Nuvolari, Alessandro Collective Invention During the British Industrial Revolution:The Case of the Cornish Pumping Engine. Cambridge Journal of Economics, 28: Plott, Charles, and Shyam Sunder Rational Expectations and the Aggregation of Diverse Information in Laboratory Security Markets. Econometrica, 56(5): Plott, Charles R, and Shyam Sunder Efficiency of experimental security markets with insider information: An application of rational-expectations models. The Journal of Political Economy, Radner, Roy Existence of Equilibrium of Plans, Prices, and Price Expectations in a Sequence of Markets. Econometrica, 40(2): Savage, Leonard J The foundations of statistics. Courier Corporation. Sims, Christopher A Rational inattention: Beyond the linear-quadratic case. The American Economic Review, 96(2):

34 34 XXX XXXX Skreta, Vasiliki, and Laura Veldkamp Ratings shopping and asset complexity: A theory of ratings inflation. Journal of Monetary Economics, 56(5): Spiegler, Ran Competition and Obfuscation. Annual Review of Economics, 8(1). Sunder, Shyam Market for information: Experimental evidence. Econometrica: Journal of the Econometric Society, Turing, A M On computable numbers, with an application to the Entscheidungsproblem. J of Math, Valiant, Leslie G A theory of the learnable. Communications of the ACM, 27(11): Wolinsky, Asher Information revelation in a market with pairwise meetings. Econometrica: Journal of the Econometric Society, Youn, H., D. Strumsky, L. M. A. Bettencourt, and J. Lobo Invention as a combinatorial process: evidence from US patents. Journal of the Royal Society Interface, 12:

35 VOL. XXX NO. XX EMH AND COMPUTATIONAL COMPLEXITY 35 Table 1 Five KS Instances Used In The Experiment. 1 (k = 1) C = 1, 900 Items v w Density Solution IN IN OUT IN OUT OUT IN OUT OUT OUT 2 (k = 3) C = 1, 044 Items v w Density Solution IN OUT IN IN OUT OUT OUT OUT IN IN 3 (k = 2) C = 850 Items v w Density Solution IN IN IN OUT IN IN OUT OUT OUT OUT Items v 3 1 w Density Solution OUT OUT 4 (k = 0) C = 1500 Items v w Density Solution IN IN IN IN IN IN IN OUT OUT OUT 5 (k = 4) C = 1300 Items v w Density Solution IN IN OUT OUT IN IN IN IN IN OUT Items v w Density Solution IN OUT Note: Available items, capacity C (maximum total weight of the knapsack), and Sahni-k k, for each of the instances used in the experiment. Items ordered by descending density. Density is defined as the ratio of value v to weight w. IN means the item is in the solution; OUT means it is not. Problem 5 has a second solution (not indicated), with a higher weight and higher Sahni-k however.

36 FOR ONLINE PUBLICATION Supplementary Online Material A Further Results Table I: Summary table of average payoffs. Average payoff $ SD ($) Per instance (all participants) Per instance (for a perfect score) Per instance (for a non-perfect score) Table II: Summary table of average performance. Average performance % SD (%) Of participants in all instances Of markets in all instances Table III: Summary table of the number of participants with a perfect performance score of 1 by Sahni-k instance difficulty. Number of participants that solved the instance Sahni-k Number of participants Proportion of participants (%)

37 Figure I: Relationship between the average economic payoff from trading and Sahni-k instance difficulty (all solutions). 18 Relationship between economic payoff and Sahni-k (all solutions) 16 Average payoff: $ Payoff ($) Sahni-k The average economic payoff from trading at every level of Sahni-k instance difficulty, whether solved for the optimal solution or not. The average payoff for any given instance across all levels of instance difficulty was $

38 Table IV: Summary table of total trades in the marketplace by instance (and Sahni-k difficulty). Total number of trades in the marketplace per instance Sahni-k Session B Session C Session D Session E NA 138 Average Table V: Summary table of the average number of trades in a market by item type. Average number of trades per item in each instance Item type Correct items Incorrect items Session B Session C Session D Session E Average

39 Figure II: Price variation per item ordered by relative efficiency ratio for Sahni-k 0 (all session). Variation in price per item (all Sessions: Sahni-k 0) Price ($) IN IN IN IN IN IN IN OUT OUT OUT Item in marketplace ordered by descending efficiency ratio The above figure shows the relationship between item prices and their respective efficiency ratios, and the price variation; all data from all sessions. Items are arranged in descending order of value-to-weight ratio, and indicate whether they belong to the optimal knapsack solution or not. 4

40 Figure III: Price variation per item ordered by relative efficiency ratio for Sahni-k 1 (by instance). Variation in price per item (all Sessions: Sahni-k 1) Price ($) IN IN OUT IN OUT OUT IN OUT OUT OUT Item in marketplace ordered by descending efficiency ratio The above figure shows the relationship between item prices and their respective efficiency ratios, and the price variation; all data from all sessions. Items are arranged in descending order of value-to-weight ratio, and indicate whether they belong to the optimal knapsack solution or not. 5

41 Figure IV: Price variation per item ordered by relative efficiency ratio for Sahni-k 2 (by instance). Variation in price per item (all Sessions: Sahni-k 2) Price ($) IN IN IN OUT IN IN OUT OUT OUT OUT OUT OUT Item in marketplace ordered by descending efficiency ratio The above figure shows the relationship between item prices and their respective efficiency ratios, and the price variation; all data from all sessions. Items are arranged in descending order of value-to-weight ratio, and indicate whether they belong to the optimal knapsack solution or not. 6

42 Figure V: Price variation per item ordered by relative efficiency ratio for Sahni-k 3 (by instance). Variation in price per item (all Sessions: Sahni-k 3) Price ($) IN OUT IN IN OUT OUT OUT OUT IN IN Item in marketplace ordered by descending efficiency ratio The above figure shows the relationship between item prices and their respective efficiency ratios, and the price variation; all data from all sessions. Items are arranged in descending order of value-to-weight ratio, and indicate whether they belong to the optimal knapsack solution or not. 7

43 Figure VI: Price variation per item ordered by relative efficiency ratio for Sahni-k 4 (by instance). Variation in price per item (all Sessions: Sahni-k 4) Price ($) IN IN IN OUT IN OUT OUT IN OUT IN IN OUT Item in marketplace ordered by descending efficiency ratio The above figure shows the relationship between item prices and their respective efficiency ratios, and the price variation; all data from all sessions. Items are arranged in descending order of value-to-weight ratio, and indicate whether they belong to the optimal knapsack solution or not. 8

44 Table VI: Summary table of the average number of item moves by a participant per instance. Average number of moves per participant by instance Sahni-k Session B Session C Session D Session E NA 27 Average Table VII: Summary table of the average number of moves per item by a participant in an instance by item type. Average number of moves per item in each instance Item type Correct items Incorrect items Session B 2 3 Session C 3 3 Session D 3 3 Session E 3 3 Average 3 3 9

45 II Instructions 10

46 11

47 12

48 III Timeline Sample timeline for Session 4 (KPD): Practice Replication 1 Replication 2 Replication 3 Replication 4 Replication 5 Practice Marketplace = KPD- Practice ~55 Marketplace = KPD Marketplace = KPD Marketplace = KPD Marketplace = KPD Marketplace = KPD The above timeline allocates approximately 55 minutes for practice, and instructing participants how to trade and test knapsack solutions. Afterwards, participants engage in five consecutive instances allocated approximately 15 minutes each. 13