Machine Learning Markets

Transcription

1 Machne Learnng Marets Amos Storey School of Informatcs, Unversty of Ednburgh Abstract Predcton marets show consderable promse for developng flexble mechansms for machne learnng. Here, machne learnng marets for multvarate systems are defned, and a utltybased framewor s establshed for ther analyss. Ths dffers from the usual approach of defnng statc bettng functons. It s shown that such marets can mplement model combnaton methods used n machne learnng, such as product of expert and mxture of expert approaches as equlbrum prcng models, by varyng agent utlty functons. They can also mplement models composed of local potentals, and message passng methods. Predcton marets also allow for more flexble combnatons, by combnng multple dfferent utlty functons. Conversely, the maret mechansms mplement nference n the relevant probablstc models. Ths means that maret mechansm can be utlzed for mplementng parallelzed model buldng and nference for probablstc modellng. 1 Introducton One ntrgung feature of the hstory of machne learnng, s that despte ts ubqutous methods, ts mmedate mportance n a data rch world, and the desre for automaton, the machne learnng endeavour typcally nvolves taclng each new problem through the ndvdual craftng of a soluton by experenced practtoners. The practtoners often use compostonal structures to buld machne learnng models, but despte the large number of dfferent models, the number of dfferent compostonal approaches s qute small. Almost all fall n to one or other of the followng categores: Model averagng e.g. Bayesan model averagng, boostng [9, 20]. Appearng n Proceedngs of the 14 th Internatonal Conference on Artfcal Intellgence and Statstcs (AISTATS) 2011, Fort Lauderdale, FL, USA. Volume 15 of JMLR: W&CP 15. Copyrght 2011 by the authors. Mxtures e.g. mxture of experts [13], mxture models, topc models [12, 5] and Drchlet process mxtures. Products/Factors e.g. Marov random felds, Condtonal random felds, product of experts [11], Boltzmann machnes, belef networs. Mxngs e.g. Independent Component Analyss based models and Gaussanzaton [21]. Although hybrd modellng has an extensve hstory, t s stll the case that ndvdual models are usually a composton of multple homogenous elements rather than nhomogeneous ones. Despte ths, the results of the Netflx challenge [3] suggest these ndvdually desgned results seem to be outperformed by combnatons of dfferng methods, pooled usng farly smple poolng (e.g. model averagng) mechansms. Extendng machne learnng methods to more and more complcated scenaros wll requre ncreasng the flexblty of the modellng approaches. It may well be desrable to buld models from nhomogeneous unts as standard, and experment wth more flexble compostonal methods. In ths paper we suggest that machne learnng marets play a role n ths. If they are to contrbute n ths way then we must establsh that they can extend machne learnng methods. At the very least such marets must be able to mplement current model structures and perform nference n those structures. A frm probablstc nterpretaton s mportant, and any new approach should allow both the freedom for ndvdual model buldng and the sutable combnaton of methods. In the paper we show that gven a set of agents, the maret equlbrum can mplement a number of standard componental probablstc model formalsms. Hence maret dynamcs provde a mechansm for probablstc nference n those models. Machne Learnng Marets are predcton marets nvolvng ndvdual machne learnng agents, each wth a utlty functon and a probablstc belef about the doman to be modelled. The goods n ths maret represent bets on outcomes of ndvdual system states, and a no-arbtrage assumpton means that the prce of goods can be nterpreted probablstcally. The approach of ths predcton maret goes beyond the smple sngle bnary state predctors that are common n consderaton of real predcton marets. 716

2 Machne Learnng Marets Rather we consder predcton marets on large jont spaces and multple varables at one tme. We establsh that some of the ey model combnaton methods lsted above can be drectly mplemented by groups of agents, and nference n those models s obtaned by maret dynamcs. Dfferent model combnaton methods are obtaned va dfferent utlty functons for the ndvdual agents. We also show that t s possble to consder agents wth nche belefs about a small subset of the varables n the system, and show that we can derve typcal factor graph (.e. product) representatons from such systems. As the agents are autonomous enttes actng n a maret, the methods outlned here are very amenable to parallelsm. We show that by restrctng the goods avalable n the maret, the maret dynamcs can be represented as message-passng mechansms between prcng and stocholdng, whch are an to the messages between varables and factors n message passng on factor graphs. Most mportantly the homogenety of the agents whch form the above models can be relaxed n favour of many dfferent forms of agents wth varyng utlty functons, wthout any change n the overall structure of the system. As a result a whole spectrum of dfferent model combnaton procedures can be mplemented here. The focus of ths paper s the examnaton of model combnaton methods for agents that have already learnt ther belefs. We leave the examnaton of learnng n these marets to future wor. However ths examnaton of combnaton methods s also of moot phlosophcal nterest. Subjectve Bayesan methods for handlng the updatng of ndvdual belef are well establshed. However the ssue of ratonally combnng the posteror belefs of dfferent agents to form a consensus belef s a long-standng and unsolved ssue n Bayesan phlosophy [10]. Though we do not pretend that the approach descrbed here s a soluton to ths, we do suggest that t s one way the problem could be consdered, and ndeed relates to the consderatons gven by other authors [19, 18, 14]. Fnally we note that one further ssue that s not dscussed n any detal n ths paper, but s nevertheless mportant. The flexblty of the maret structure allows any agent to produce new dervatve stocs whch can be added to the maret, and can be traded on le any other. Ths effectvely allows for the generaton of new features. In the maret economc analyss, we wll tae a neoclasscal perspectve and prmarly utlze a compettve equlbrum assumpton. Ths assumpton s merely for llustraton of the consequences of such equlbra, rather than suggestng that s precsely how such a maret would operate. Because we wll be utlzng concave utlty functons, we now the maret system wll have a unque fxed pont [1]. 2 Prevous Wor Here we summarze the man prevous wor n machne learnng and predcton marets. The number of papers drectly establshng maret mechansms for mplementng exstng probablstc machne learnng methods s small. One mportant paper s [15], where the authors do consder a predcton maret to form model combnaton for machne learnng. They only consder predctons regardng a sngle multnomal varable, and ther agents do not have utltes, but are nstead endowed wth bettng functons. These bettng functons may not be dervable from any sutable utlty, and ndeed requre that the amount bet s proportonal to the total wealth, a constrant not seen by some of the utlty functons used here. Even so ths paper establshes that artfcal predcton marets can provde useful mechansms for combnng classfers. We beleve the power of predcton marets can go well beyond ths and can be a powerful tool for the overall machne learnng endeavour. More recently other learnng methods have been related to predcton marets [7]. However here the focus s on cost functon based marets, where a global maret maer s defned and the global cost functon for the maret s specfed. The relatonshp between a global cost functon The case we consder here s a more general maret condton wth no global maret marer and ndependent agents, defned by ther respectve utlty functons. Each agent follows a standard utlty maxmzng procedure: ths s fully parallel. We also consder the case where goods can correspond to only a lmted number of the (usually exponental number of) margnal outcomes, whch has not been dscussed htherto. The potental of predcton marets has long been taled about [2, 17, 24]. In [8] the authors compared a number of dfferent mechansms for expert aggregaton ncludng a smple predcton maret approach. Dfferent maret desgns have dfferent features, and ensurng good predcton maret desgn wth suffcent fludty [6] wll be crtcal for effcently reachng equlbrum. In [22] the authors examne the statstcal propertes of maret agent models, whereas n [16] the authors consder predcton marets n the context of Bayesan learnng. 3 Predcton Marets A maret provdes a basc process for the exchange of goods between dfferent agents. We defne a basc maret as follows. Defnton 1 (Maret) A maret s a mechansm for the exchange of goods. The maret tself s neutral wth respect to the goods, or the trades. As such the maret tself cannot acqure or owe goods, and hence s subject to the maret constrant that the total number of goods sold s equal to the total number of goods bought. Any currency s smply another good n the context of a basc maret. However 717

3 Amos Storey we wll assume that n ths context there s an agreed currency: all partcpants n the maret are happy to use that currency for the purposes of trade. Wthn ths defnton, there are many potental forms and mechansms for mplementng a maret. For the sae of smplcty wll only consder charge-free marets n the context of ths paper. We defne a poston n ths maret as the stoc holdng (.e. number of each of the goods owned) of any ndvdual. We assume that agents can hold a short poston (.e. debt or negatve holdng) n a stoc. A postve holdng n a stoc s called a long poston. Suppose we have a maret where one type of good beng traded s a bet, and the other s a currency. In ths context we can defne a bet as a good that pays a fxed amount (taen to be 1 Grubnc 1 wthout loss of generalty) dependent on a partcular outcome of a future occurrence, and pays nothng otherwse. Marets consstng of trades of ths form of good are called predcton marets 2. Defnton 2 (Predcton Maret) A predcton maret (for the purposes of ths paper) s a maret wth an agreed currency and where the remanng goods are bets wth a fxed return on a partcular outcome of a future occurrence (and a zero return otherwse). Indvduals may choose to create those goods for sale,.e. produce a bet and sell t at a prce. Ths s equvalent to a short poston n that bet. We wll also mae the assumpton that the agents n the maret vew the currency as a rs-free asset, n that they are happy to defne utlty functons n terms of that currency. 4 Defntons We start wth a basc defnton of the terms we wll be usng, followed by examples of how these wll actually be used n practce. Suppose we have a sample space Ω of all possble outcomes of the set of relevant (future) occurrences. The elements of Ω are called events, and one and only one of those events wll be the actual outcome. Suppose we also have a σ-feld F on Ω. We enumerate a set of maret goods by = 1, 2,... N G, each assocated wth a set m F to be bets that pay out 1 Grubnc f the outcome s n m. We enumerate a set of agents = 1, 2,... N A. Each agent can buy or sell any of the maret goods. Hence each agent has a poston vector (or stoc holdng) s n all the goods avalable. s s the total number of tems agent has of good. s < 0 ndcates a short poston n that good. Note that s ths paper s s not the total amount nvested n tem : that depends also on the costs of the good. 1 The currency of Elbona s, naturally, respected worldwde. 2 More specfcally ths s sometmes called a wnner-taes-all maret: see e.g. [24, p2]. Each agent also has an assocated utlty functon U (W ) defned n terms of the currency, denotng the utlty to the agent of a wealth of W Grubncs. Each agent wll also have a belef, that s a probablty measure P defned on (Ω, F). We can also consder agents who have belefs defned on subspaces (Ω, F ) of the probablty space (Ω, F). We call these local belefs, and ths wll be approprate for example where we consder dstrbutons of many random varables, and these sub-felds are the σ-felds nduced by certan subsets of those random varables. In practce we wll wor wth random varables, and hence the underlyng σ-felds wll be mplct. We wll consder the cases of a sngle multclass random varable, and a dscrete multvarate random varable. We wll defne specfc maret goods n each nstance. Although many dfferent utlty functons are possble, three wll be partcularly mportant n ths paper. These are now gven. 4.1 Varous Utlty Functons Lnear debt-free utlty The frst utlty functon we wll consder s U S (x) = x f x > 0 and otherwse. (1) (where S denotes straght). Ths utlty functon prevents an agent from gong nto debt but otherwse s lnear. Ths utlty s not strctly concave (concave utlty functons result n equlbrum solutons) Logarthmc utlty The second utlty s concave, and taes a logarthmc form. Ths too does not allow debt, but has decreasng utlty gans for ncreasng wealth. U L (x) = log x (2) where L s for logarthm Exponental decayng negatve utlty The thrd utlty functon that we consder s U E (x) = exp( x) (3) where E stands for exponental. Ths utlty s upper bounded, and allows for unlmted assets and unlmted debts. The effectve dsutlty of debt s exponentally growng, whereas the benefts of ever ncreasng assets becomes margnal. It s a concave utlty functon and t has one analytc property that has a smplfyng effect: exp(w x) = exp(w ) exp( x) (4) whch says that decson regardng a change n wealth x are ndependent of current wealth W. For such a utlty functon, decsons do not depend on the wealth or budget of that ndvdual. As a result n marets where all agents have exponentally decayng negatve utlty, the wealth of the agents s rrelevant and can be 718

4 Machne Learnng Marets removed from the equaton. Ths utlty functon s commonly just called an exponental utlty. 4.2 Maret Structure The observant reader wll notce the cost of goods has not yet been mentoned. Ths s because the cost of goods s a functon of the agents tradng preferences, and the process of trade, and so t s dependent on the maret structure. It s perfectly possble to have maret structures whereby the cost of goods can be dfferent for dfferent agents. However we wll mae the assumpton that we have a maret that allows all traders to trade a gven good at any tme at a gven cost. Hence there s assocated wth each good a cost c whch s the prce the good s currently tradng at. We wll also mae a no-arbtrage assumpton regardng the maret and the agents n the maret. That s, we wll assume that t s not possble for any agent to mae proftable rs free trades n a set of assets. For example f some set of goods formed a jontly certan bet, but the total prce for those goods was less than one, then an agent could buy one of each of those goods and guarantee a net postve return when the bets are fnalzed. Ths s an arbtrage opportunty. If such opportuntes ever arse, traders would mmedately trade on those opportuntes so that they qucly dsappear. Any ndvdual or group who maes hmself or herself open to arbtrage tradng wll qucly lose money, and hence wll adjust hs or her poston. One of the common features of predcton marets s the assocaton of the cost of goods n a worng maret wth the probabltes of the outcomes assocated wth those goods. There are a number of good theoretcal reasons why ths assocaton s vald, and t s related to bettng nterpretatons of Bayesan nference (see e.g. [4]). For space reasons we are not able to elaborate ths here, except to note that the no-arbtrage assumpton ensures that for = 1, 2,... N G enumeratng a set of goods assocated wth mutually exclusve jontly certan events, we have N G c = 1 (5) =1 matchng the sum-to-1 assumpton for probabltes. More generally the no-arbtrage assumpton ensures that, f the maret goods are bets on a collecton of tems that form a σ-feld, then the costs are a probablty measure on that σ-feld. It s ths assocaton of prce wth probablty that maes predcton marets a useful tool for machne learnng. We wll consder equlbrum marets n ths paper. Here, maret equlbrum s defned by a prce and an allocaton such that no trader has any ncentve to trade and there s no excess demand of any good. The problem of maret equlbra was frst formulated n Economcs by Walras [23] n The exstence of such a maret equlbrum was establshed by Arrow and Debreu [1] usng analyss of the fxed-pont of the system. 5 General Formulaton Let W denote the current wealth of agent. Let the cost of goods be denoted by the cost vector c = (c 1, c 2,..., c NG ) T. Then the ratonal agent wll choose a utlty maxmzng poston s = (s 1, s 2,..., s ) T n each of the goods he or she has an opnon about (that s those n S = { m F }). However because the outcome s uncertan the actual utlty of holdng the goods s a weghted sum of the utlty assocated wth each possble outcome, weghted by the agent s belef about the probablty of that outcome. Ths s wrtten as s = s (W, c) = arg max P (j)u (W s T c + s r(, j)) s j Ω subject to s = 0 f / S. Here, r(, j) s the return of a bet on good n case of outcome j and s 1 f j m and zero otherwse. s (W, c) s the buyng functon for agent, and states how the agent would choose to act (gven no other constrants) n a maret wth costs c. There s n general no guarantee that the maxmum of ths utlty has a unque argument, and hence n general there wll not be a unque buyng functon; addtonal rs free trades may be possble resultng n dfferent purchase quanttes whle mantanng the same utlty. In an equlbrum maret (f t exsts), the agents are able to jontly act optmally gven the costs c. Hence the maret constrants N A s (W, c) = 0 (7) =1 are satsfed for some buyng functons, and can be solved to get the equlbrum costs. We wll llustrate that for certan utltes these equlbrum condtons mrror nown model combnaton procedures n machne learnng. However n a non-equlbrum stuaton, maret or aucton dynamcs can also be defned. One possble dynamc s that each good comes up for aucton at a tme. Then all nterested agents bd for those goods by gvng ther buyng functons for that good gven ther holdng of other goods. Costs are decded that best satsfy those bddng functons. The varous bds are satsfed, and we move on to the next good etc. We wll llustrate that ths maret dynamcs has much n common wth message passng schemes n probablstc nference. 5.1 Dscrete-state Marets Suppose we have a predcton maret, consstng of the purchase of bets that pay out 1 Grubnc on the future occurrence of one of N G mutually exclusve, jontly certan outcomes. In ths case the maret goods are just bets on the (6) 719

5 Amos Storey ndvdual events. Note that a multvarate dscrete dstrbuton can be represented ths way by enumeratng all the possble jont states. Let W denote the current wealth of agent. Let the cost of goods be denoted by the cost vector c = (c 1, c 2,..., c NG ) T. Then the ratonal agent wll choose a utlty maxmzng poston s = (s 1, s 2,..., s ) T n each of the goods. Ths s wrtten as s = s (W, c) = arg max P ()U (W s c + s ) (8) s where P () denotes the belef of agent about the probablty of the event occurrng (or more accurately the event assocated wth good occurrng). We collect these nto a vector p = (P (1), P (2),..., P (K)) T. Every agent has an opportunty for fnancally-neutral rsfree trades, due to the arbtrage-free assumpton, by buyng (or sellng) one unt of every stoc. The utlty assocate wth havng holdng s and s + α1 s dentcal: there are varous equvalent postons that are produced by rs free purchases or sales. If we also ntroduce an addtonal neutral agent that only maes these rs free trades, buyng/sellng one of each tem (and hence never has any dfference n hs/her return from the zero poston), then we only need to specfy the poston of each agent up to these utlty-equvalent classes. We ntroduce a standardzaton constrant to ensures each equvalence set s now represented by a sngle poston s that satsfes the constrant. Note ths does not mean that the agents have to obey the standardzaton constrant: that s rrelevant. It just means that any solutons we obtan that do obey the standardzaton constrant wll be a sutable representaton for all the other equvalent postons. In the analyss we wll use the most convenent constrant for any gven problem. If the constrant holds for s = 0 for all agents, then the maret constrant (7) wll also hold at equlbrum. One useful constrant s to set s T c to zero for each agent (each agent could buy/sell one of each stoc wth no change n utlty untl ths constrant were satsfed). Another possble constrant s that we choose to set the stoc holdng of all agents of stoc = N G to zero. Alternatvely one could requre that the mnmum stoc holdng for an agent n at least one stoc was zero Case 1: Lnear debt-free utlty gves weghted medan model combnaton In the case of a lnear debt-free utlty, we use the standardzaton constrant mn s = 0, whch s useful n that t ensures that the actve agents only mae long postons (the remanng rs free short poston s held by the neutral agent), and so we can wrte (8) as s = arg max s s T (p c) s.t. mn s = 0 and s T c s < W. (9) where the condtons are those mposed by the standardzaton constrant and the debt free constrant. Due to the standardzaton constrant, the last condton s T c s < W s satsfed f and only f s T c < W, whch smply states that the maxmum stae s the whole wealth. Ths leads to s = arg max s s T (p c) s.t. mn s = 0 and s T c < W. (10) Ths s optmzed by stang the whole wealth W on the good whch maxmzes P () c c. (11) In the bnary case wth many players of equvalent wealth, the equlbrum for the sngle cost c wll be the medan of the agents P (1) values as that wll balance the total long and short postons n the one good; wth varyng wealth, each agent s P (1) value wll be weghted by ts wealth before computaton of the medan. Note that the fact that the lnear debt free utlty s not strctly convex means that there s not necessarly a unque equlbrum, whch s clear from ths soluton as the medan s not unquely defned for even numbers of equally wealthy agents. In mult-class settngs ths utlty results n marets that choose costs to balance agents purchases across all the stocs, though the exact formalsm s not as smple as n the bnary settng as t s dependent on the number of agents nvolved Case 2: Logarthmc utlty gves weghted mean model combnaton Wth logarthmc utlty, U L (W, c, s ) = P () log(w s T c + s ), (12) the maret constrants (7) can also be solved. We use the standardzaton constrant s T c = 0 to buld a Lagrangan L = P () log(w + s ) + λ s T c (13) wth Lagrange multpler λ. By equatng the dervatves of ths Lagrangan to zero we get L = P () + λ c = 0 (14) s W + s whch we solve to get the buyng functon s = W (P () c ). (15) c Solvng for the maret constrant (7) gves the equlbrum cost c = W P () W (16) 720

6 Machne Learnng Marets whch sets the costs to be the wealth weghted mean of the agents belefs. Note that ths s a lnear aggregaton of classfers, an to methods used n boostng algorthms [9, 20] and model averagng approaches. If the wealth has been acheved through past performance, then the classfers are effectvely weghted by ther performance n prevous crcumstances, and so ths also relates to a mxture of experts approach. Note that n [15], the formula (16) was obtan va presumng constant bettng functons (the proporton of wealth bet as a functon of cost). In realty constant bettng functons are unrealstc n utlty terms as they would mply always bettng the same amount on the same goods rrespectve of prce. Here we show that a constant bettng functon s not necessary for a weghted mxture prcng scenaro. Instead we have derved a utlty consstent buyng functon that has the same prcng propertes Exponental decayng negatve utlty gves product model combnaton We can also consder the mult-class maret wth the exponental decayng negatve utlty U E The utlty for agent s wrtten as U E (W, c, s ) = P () exp( W + s T c s ) (17) Once agan, we use the standardzaton constrant s T c = 0 to buld a Lagrangan for ths of the form L = P () exp( W s ) λ exp( W )s T c (18) wth Lagrange multpler expressed as λ exp( W ) for convenence. By equatng the dervatves of ths Lagrangan to zero we get L = exp( W )P () exp( s ) λ exp( W )c = 0 s whch we solve to get the buyng functon (19) s = log P () log c log λ (20) where λ s set to ensure s T c = 0. We can now solve for (7) gvng N A c P () 1/N A (21) =1 whch sets the costs to be the geometrc mean of the agents belefs, and s a product model wth the potentals for each product scaled accordng to the number of agents. Hence the exponental decayng negatve utlty mplements a product combnaton. By wrtng Φ () = (1/N A ) log P () we have ) ( c = 1 Z exp Φ () (22) where Z s a normalzaton constant. We can see that each agent mplements a separate contrbutng potental to the overall maret dstrbuton. 5.2 Interm Summary We have shown how, n a maret for a sngle multclass outcome, many of the standard aggregaton methods used for constructng componental machne learnng models are reproducble as a result of dfferent agent utlty functons. Weghted medan of experts, weghted mxture of experts, and product of expert models can all occur, smply by changng the utlty functons nvolved. Furthermore, as each agent acts ndependently, the maret mechansm provdes a well defned approach for mxng agents wth dfferent utlty functons together. As a result we can obtan ntermedates between product dstrbutons and mxture dstrbutons. In the rest of ths paper we generalze the methods for multvarate settngs: here there are J dfferent varables, and the goods are bets on the jont state of all these varables. Hence N G s exponental n J. Frst we consder the case of local agents, and show how ths relates to factor graphs, and usng a dfferent form, for methods of combnng multple margnal belefs. Fnally we show that, when only a lmted number of goods are avalable, maret dynamcs can mplement message passng mechansms to obtan the equlbrum. 6 Nche Agents In economc marets, agents do not usually try to comprehend the complete jont system. Rather, ndvdual agents establsh nches that they attempt to explot. In general, gven some maret prcng, an agent may beleve that the real value dffers, n some lmted way, from the overall maret prce decded collectvely by all the agents, and each agent, learns and represents hs or her belefs relatve to the maret, and enters the maret to explot (n hs or her opnon) that dfference. There may also stll be agents wth drect opnon (not expressed relatve to the maret prce) n that maretplace. We consder, for purposes of llustraton, a maret consstng of a sngle agent wth a drect opnon, and a number of agents wth opnons relatve to the maret prce. The belef of each of these agents can now be represented as probablty dstrbuton that s a factoral devaton from that maret prce: P () = 1 Z F ()c. (23) In a multvarate settng those devatons wll generally occur n only a few random varables that the agents are nowledgeable about: the relatve belefs of an agent due to varatons n other varables wll match the dstrbuton establshed by the maret as a whole. For example suppose y = (y 1, y 2,..., y J ) T denotes the fnal outcome of a multvarate occurrence, where each element y j s, for the sae of notatonal smplcty, assumed to be bnary. There 721

7 Amos Storey are then N G = 2 J possble goods, each a bet on some outcome y. If agent only had devant opnons from the general consensus regardng varables n set S, we would wrte F (y) = F (y S ), where we use the superscrpt notaton y S to denote the vector derved from restrctng the vector y to just the elements wth ndex n S. The set S would be called a clque. Colloqually speang agent s happy to agree wth the consensus opnon regardng the varables (s)he has no nowledge about. Let = 1, 2,..., N G enumerate all the dfferent y: y 1, y 2,..., y NG. Then we use F () to represent F (y ) etc. Once agan, let c represent the cost vector (now of length N G = 2 J, one term for each good). We can wrte out the utlty for such a set of agents as N G U (W, c, s ) = F ()c U (W s T c + s ) (24) =1 (the utlty only needs to be defned up to a constant for optmzaton purposes). The utlty for the sngle agent wth a drect opnon can be wrtten N G U 0 (W, c, s ) = P 0 ()U 0 (W 0 s T 0 c + s 0 ) (25) =1 In ths case an exponental decay negatve utlty results n buyng functons s = log F () log λ (26) wth Lagrange multpler λ and s 0 = log P 0 () log c log λ 0 (27) The maret constrant then gves an equlbrum prce equaton of c P 0 () F () (28) whch s a product of local clque factors, along wth a global factor, whch represents some base dstrbuton and could be unform. Hence the use of agents that declare ther belefs relatve to the maret produces models of the form of a varous local clque potentals. The equlbrum prcng for ths maret represents a jont probablty dstrbuton for a the standard factor model, wth each agent representng a factor over a clque of the varables. We have shown that for a certan maret structure and certan utlty representaton, the maret precsely mplements a very common form of probablstc graphcal model. Any factor graph can be represented as a maret of ths form, and the equlbrum prcng of the maret represents the probabltes assocated wth that factor graph. 6.1 Margnal Agents The nche agents descrbed above are nterestng n terms of the model they mplement. However, the majorty of agent models wll not result n equlbrum costs that can be smply expressed. Nevertheless, such agents could stll be very valuable, as maret dynamcs wll establsh equlbra that are vald probablty dstrbutons and may satsfy desrable crtera. One of the problems wth the nche agents s that they rely on the consensus opnon of the other agents, and express ther belefs as perturbatons from that opnon. That can open the agent up to tang rss entrely on the bass of the opnon of others. Another approach s that agents may wsh to purchase bets that are rs free n the varables they have no opnon about. In large systems any agent may only have nowledge about a small subsecton of that system e.g. a few varables. The agent does not wsh to mae assumptons about the other varables. Dfferent agents may well want to purchase bets on dfferent subsets of those varables, due to ther ndfference regardng the others. We start the analyss by consderng goods coverng all possble multvarate states, and note that n order to mae a bet on a restrcted number of states, an agent need only purchase multple equal bets coverng all the optons of the remanng states. Let y = (y 1, y 2,..., y J ) denote the fnal outcome of a multvarate occurrence, where each y j s, for the sae of notatonal smplcty, assumed to be bnary. Let S J denote the collecton {1, 2,..., J} of all the varable ndces. The maret goods consst of bets on a payout for each y, and hence we use the y to label the goods, and wrte c(y) for the cost of a bet on outcome y, and s (y) for the amount of good y agent has. Each agent also has a belef, but now the belefs can be restrcted to a subset S of the varables. Once agan we call the sets S clques. We wll use the shorthand y to denote y S where that does not cause confuson. The ratonal agent wll choose a utlty maxmzng poston s ) wrtten as (ys s (y ) = arg max P (y ) s (y ) y U (W s (y )c(y ) + s(y )) (29) y where c(y ) = y (y ) S =y c(y) s the sum of the costs of all the goods needed to produce a bet on the margnal outcome y. Once agan we wll consder an exponental decayng negatve utlty. We ntroduce the standardzaton constrant y s (y )c(y ) = 0 and optmse the agents utlty wth respect to ths constrant to get the agent s buyng functon. Equatng the dervatve of the Lagrangan wth respect to s (y ) to zero, we get the buyng functon s(y ) = log P (y ) log c(y ) λ (30) where a purchase of goods y conssts of an equal purchase of all goods y consstent wth y on set S. We have no smple representaton for the costs of all the goods n an equlbrum maret of agents wth these buyng functons 722

8 Machne Learnng Marets (the number of goods s now exponental n the number of varables, and the buyng functons depend on the costs for many dfferent goods). However the maret wll stll mplement ths dstrbuton as an equlbrum of the dynamcal system that defnes a partcular choce of maret dynamcs. The maret wll then provde a mechansm for combnng a number of margnal belefs about a system. 6.2 Message passng Marets consstng of an exponental number of goods are practcally nfeasble. It wll become mpossble to eep trac of or even represent the prce of such a large number of goods. As a result the maret s lely to consst only of a reduced set of the possble goods. Just as t s lely that agents wll only have opnons on a small set of goods, so the maret as a whole wll only nvolve trades on a smaller set of goods than all those that are possble. Agan let y = (y 1, y 2,..., y J ) denote the fnal outcome of a multvarate occurrence, where each y j s, for the sae of notatonal smplcty, assumed to be bnary. Let S J denote the collecton {1, 2,..., J} of all the varable ndces. The maret goods are bets on the outcome y j = 1 for each j. The total number of goods s N G = J, and so S J = S NG. Each good s ndexed by some chosen from the set S NG. An agent wll, once agan, have the probablstc belef P (y). Gven some maret cost c(y), and an exponental decayng negatve utlty, the agent wll have an expected utlty of U (W, c, s ) = y P (y) exp( W + s T (c y)). (31) Suppose agent has been communcated all the costs for all the goods (for a margnal/nche agent ths would only need to be all the costs n the clque). Then that agent s able to optmze ts poston n those goods to obtan a prce condtonal optmal value s. We can then communcate that poston n the followng way: Consder a trade n a sngle good. Gven the agents optmzed poston, and gven the current prces c, we defne A (y ) by A (y ) = P (y y ) exp((s ) T (c y )) y (32) where the superscrpt notaton denotes the vector wth the th term removed. Then we can wrte U (W, c, s ) as y A (y )P (y ) exp( W + s (c y )) (33) where s s the holdng n stoc. Tang dervatves of ths expected utlty wth respect to s, and equatng to zero, we get y (c y ) exp(s (c y ))A (y )P (y ) = 0 (34) where P s the margnal belef about y. Hence we can wrte the optmzed poston n good, s as s (c ) = log 1 c + log A (1)P (1) (35) c A (0)P (0) condtoned on the nowledge of the postons n the other goods. Gven ths buyng functon, the equlbrum constrant gves c (y ) A (y ) 1/N P (y ) 1/N (36) Ths means we can compute the prce for a bet on varable gven the prce of everythng else, so long as we have computed the messages A for all the agents. The new cost then gets passed to all the agents so they can update ther messages A resultng n new buyng functons. Though ths apples generally, t s not useful unless the A are straghtforward to compute. Ths s only the case for nche or margnal agents. In those stuatons (32) nvolves only a sum over the local clque and hence s computable n tme exponental n the clque sze. 7 Dscusson In ths paper we establsh the flexblty of machne learnng marets for representng, through maret prces, dfferent forms of compostonal machne learnng model. We show that many of the compostonal structures typcally used n machne learnng, the localzed representatons, and the nferental mechansms such as message passng schemes can be nterpreted n terms of machne learnng marets. Put smply, certan probablstc machne learnng models can be redefned as sets of ndependent agents wth partcular utlty functons. Any choce of convergent maret dynamcs can then be vewed as an nference approach. In ths way the propagaton of cost nformaton and purchase nformaton can be seen as messages that are passed between ndependent agents, much as message passng schemes wor between nodes n a graph. The beneft of ths approach s that s allows for consderably more versatle models to be set up, by usng multple agents wth dfferent utlty functons. These agents can functon ndependently and need no nformaton about what other agents are dong save for the prces they are wllng to sell maret goods for. Ths approach has sgnfcant long term appeal: t allows for mmedate ntegraton of multple dfferent types of agents as well as a natural, large scale parallel process for nference. Acnowledgements The author would le to than Jono Mlln and Krzysztof Geras for very helpful comments for the fnal verson of the paper, and Jono for presentng the paper at the conference. 723

9 Amos Storey References [1] K.J. Arrow and G. Debreu. Exstence of an equlbrum for a compettve economy. Econometrca: Journal of the Econometrc Socety, 22(3): , [2] K.J. Arrow, R. Forsythe, M. Gorham, R. Hahn, R. Hanson, J.O. Ledyard, S. Levmore, R. Ltan, P. Mlgrom, and F.D Nelson. The promse of predcton marets. Scence, 320:877, [3] R.M. Bell and Y. Koren. Lessons from the Netflx prze challenge. ACM SIGKDD Exploratons Newsletter, 9(2):75 79, [4] J.M. Bernardo and A.F.M. Smth. Bayesan theory. Wley, [5] D.M. Ble, A. Ng, and M.I. Jordan. Latent Drchlet allocaton. Journal of Machne Learnng Research, 3: , [6] A. Brahma, S. Das, M. Magdon-Ismal, Z. Lang, L. He, J. Wu, R. Chcheportche, J.P. Bouchaud, S. Du, and Y. Lvne et al. Comparng predcton maret structures, wth an applcaton to maret mang. arxv: , [7] Y. Chen and J.W.Vaughan. A new understandng of predcton marets va no-regret learnng. In Proceedngs of the 11th ACM conference on Electronc commerce, [8] V. Dan, O. Madan, D. Pennoc, S. Sangha, and B. Galebach. An emprcal comparson of algorthms for aggregatng expert predctons. In Proceedngs of the Conference on Uncertanty n Artfcal Intellgence (UAI). Cteseer. Cteseer, [9] J. Fredman, T.Haste, and R.Tbshram. Addtve logstc regresson: A statstcal vew of boostng. Annals of Statstcs, 28(2): , [10] K. Greene. Collectve belef models for representng consensus and dvergence n communtes of Bayesan decson-maers. PhD thess, Unversty of New Mexco. Dept. of Computer Scence, [11] Geoffrey E. Hnton. Tranng products of experts by mnmzng contrastve dvergence. Neural Computaton, 14: , [12] T. Hofmann. Probablstc latent semantc analyss. In Uncertanty n Artfcal Intellgence 1999, [13] R.A. Jacobs, M.I. Jordan, S.J. Nowlan, and G.E. Hnton. Adaptve mxtures of local experts. Neural Computaton, 3:79 87, [14] E. Joun and C. Napp. Aggregaton of heterogeneous belefs. Journal of Mathematcal Economcs, 42(6): , [15] N. Lay and A. Barbu. Supervsed aggregaton of classfers usng artfcal predcton marets. In Proceedngs of ICML, [16] D.S. Lee and E. Morett. Bayesan learnng and the prcng of new nformaton: Evdence from predcton marets. Amercan Economc Revew, 99(2): , [17] C.F Mans. Interpretng the predctons of predcton marets. Economcs Letters, 91:425429, [18] M. Ottavan and P.N. Sørensen. Aggregaton of nformaton and belefs n predcton marets. FRU Worng Papers, [19] D.M. Pennoc and M.P. Wellman. Representng aggregate belef through the compettve equlbrum of a securtes maret. In Proceedngs of the Thrteenth Conference on Uncertanty n Artfcal Intellgence, pages Cteseer, [20] R.E. Schapre. The boostng approach to machne learnng: An overvew. In Lecture Notes n Statstcs, pages Sprnger Verlag, [21] S.S.Chen and R.A.Gopnath. Gaussanzaton. In Advances n Neural Informaton PRocessng Systems (NIPS1999), [22] J.J. Tseng, C.H. Ln, C.T. Ln, S.C. Wang, and S.P. L. Statstcal propertes of agent-based models n marets wth contnuous double aucton mechansm. Physca A: Statstcal Mechancs and ts Applcatons, 389(8): , [23] L. Walras and W. Jaffé. Elements of Pure Economcs (1874). Translaton. Allen & Unwn London, [24] J. Wolfers and E. Ztzewtz. Predcton marets. Journal of Economc Perspectves, 1: ,