A Heuristic Bidding Strategy for Buying Multiple Goods in Multiple English Auctions

Transcription

1 A Heurstc Bddng Strategy for Buyng Multple Goods n Multple Englsh Auctons MINGHUA HE, NICHOLAS R. JENNINGS, and ADAM PRÜGEL-BENNETT Unversty of Southampton Ths paper presents the desgn, mplementaton, and evaluaton of a novel bddng algorthm that a software agent can use to obtan multple goods from multple overlappng Englsh auctons. Specfcally, an Earlest Closest Frst heurstc algorthm s proposed that uses neurofuzzy technques to predct the expected closng prces of the auctons and to adapt the agent s bddng strategy to reflect the type of envronment n whch t s stuated. Ths algorthm frst dentfes the set of auctons that are most lkely to gve the agent the best return and then, accordng to ts atttude to rsk, t bds n some other auctons that have approxmately smlar expected returns, but whch fnsh earler than those n the best return set. We show through emprcal evaluaton aganst a number of methods proposed n the multple aucton lterature that our bddng strategy performs effectvely and robustly n a wde range of scenaros. Categores and Subject Descrptors: I.2.11 [Artfcal Intellgence]: Dstrbuted Artfcal Intellgence Intellgent agents; K.4.4 [Computers and Socety]: Electronc Commerce General Terms: Algorthms, Desgn, Expermentaton Addtonal Key Words and Phrases: Intellgent agents, onlne auctons, multple Englsh auctons, bddng strategy, e-commerce 1. INTRODUCTION Onlne auctons are ncreasngly beng used for a varety of e-commerce applcatons [He et al. 2003]. Ths wdespread adopton means there are nvarably multple auctons sellng the desred good or servce at the same tme (for example, on ebay alone, there are typcally over 13, 000 auctons for dgtal cameras at any one tme). Moreover, t s frequently the case that there s a need to buy multple goods from these multple auctons. For example, buyng a flght from one aucton and bookng a correspondng hotel from another or buyng a car from a varety of cars for sale onlne. Gven the fact that there s a huge search space, t s mportant to provde automated tools that can montor relevant auctons, compare and make trade-offs between the offerngs, decde whch M. He current afflaton s Unversty of Wales, Bangor. Authors address: School of Electroncs and Computer Scence, Unversty of Southampton, Southampton SO17 1BJ, UK; emal: nrj@ecs.soton.ac.uk. Permsson to make dgtal or hard copes of part or all of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or drect commercal advantage and that copes show ths notce on the frst page or ntal screen of a dsplay along wth the full ctaton. Copyrghts for components of ths work owned by others than ACM must be honored. Abstractng wth credt s permtted. To copy otherwse, to republsh, to post on servers, to redstrbute to lsts, or to use any component of ths work n other works requres pror specfc permsson and/or a fee. Permssons may be requested from Publcatons Dept., ACM, Inc., 2 Penn Plaza, Sute 701, New York, NY USA, fax +1 (212) , or permssons@acm.org. C 2006 ACM /06/ $5.00 ACM Transactons on Internet Technology, Vol. 6, No. 4, November 2006, Pages

2 466 M. He et al. auctons to bd n, and determne what bds to place n the chosen auctons n order to obtan the best deals [Kephart 2002]. Software whch can autonomously and flexbly acheve these tasks on behalf of a partcular user s here termed a software agent [Jennngs 2001]. Aganst ths background, ths artcle develops a heurstc bddng algorthm that a software agent can use to bd across multple Englsh auctons (wth varyng start and end tmes) n order to obtan multple ndependent goods. 1 In ths context, an Englsh aucton s one n whch a sngle good s on offer and the auctoneer starts wth hs reserve (mnmum acceptable) prce and solcts successvely hgher (publc) bds from the bdders and the last bdder remanng n the aucton s the wnner. We focus on Englsh auctons because, although there are mllons of dfferent types of aucton [Wurman et al. 2001] that can be used for e-commerce, the Englsh aucton s by far the most common [Luckng-Reley 2000]. In our case, each such Englsh aucton s assumed to sell a sngle unt of the desred good and ths good may be descrbed by multple attrbutes 2 (for example, n auctons sellng flghts, the goods can be descrbed by ther dates of departure and return, by ther carrer, and the class of tcket beng bought). We only consder the case where the agent buys multple ndependent goods from such auctons; thus the falure to obtan one nstance of the good does not nfluence the avalablty or desrablty of other goods. Moreover, bds for these goods must be at least h(a) pounds sterlng (hereafter shortened to pound) larger than the prevous prce to be vald. Then f an agent bds successfully, t becomes the actve agent that s holdng the bd. It may, of course, be subsequently outbd. Auctons respond to any bd before they close and ther good s allocated to the actve bd holder when the aucton closes. In a stand-alone Englsh aucton where there s not a hard deadlne, the bddng strategy s smple. The agent s domnant strategy (the best thng to do, rrespectve of what the others do) s to bd the smallest allowable mnmum amount more than the current hghest bd and stop when the user s valuaton s reached. By adherng to ths domnant strategy, the good s always allocated to the bdder wth the hghest valuaton. However, when there are multple auctons runnng at the same tme, bddng s much more complex and has much greater uncertanty. Ths s because () the auctons have varyng start and end tmes (thus comparsons need to be made between auctons that are nearng completon (and probably have a hgh ask prce) and auctons that are near the begnnng (and probably have a low ask prce); () the partcpatng agents are lkely to adopt a varety of dfferent strateges (for example, some may bd aggressvely n the earler auctons n order to ensure they get the desred goods, whle others may wat untl later auctons to see f they can obtan bargans), and () the agents are lkely to have dfferent sets of 1 The goods are actually substtutes n the sense that f the agent fals to buy an tem n one of the auctons, t can bd n the other auctons f they are stll runnng. Moreover, we assume the agent s tasked wth obtanng multple such goods from a number of dfferent auctons. These multple goods are ndependent n the sense that the falure to buy one of them does not nfluence the purchase of the others. 2 The goods have multple attrbutes, but the buyer agents bd only on prce. Thus, our work dffers from the case where agents bd on multple attrbutes (as dscussed n Secton 6).

3 A Heurstc Bddng Strategy for Buyng Multple Goods 467 auctons that they consder bddng n (drven by ther own deadlnes), thus the set of agents n a gven aucton wll vary and so, consequently, wll the supply/demand. In some varants of the Englsh aucton, there s a strct deadlne for when the aucton wll fnsh and ths promotes a strategy of tryng to place a bd at the last moment so that no other agent has a chance to place a hgher bd (ths s called snpng [Rust et al. 1991]). To counteract such end effects, our auctons have a soft deadlne; that s, they do not close untl a fxed perod after the last bd s placed (as per Yahoo!Auctons and Aucton Unverse). Ths means snpng s not effectve and the auctons are akn to the standard Englsh one. Moreover, we consder the case where the agent wants to purchase multple goods from the ongong auctons because ths s a more general case than just purchasng a sngle tem. In practce, an agent may requre multple goods ether because a sngle owner wants multple tems or because multple owners have combned forces n a form of group buyng [Tsvetovat and Sycara 2000]. In the latter case, the agent also needs to allocate the goods t has purchased or s currently holdng to the varous customers n order to maxmse ts return. Gven ths context, we have developed (for the frst tme) a heurstc bddng algorthm that buys multple unts of the desred good from the avalable auctons. Ths algorthm operates n the followng way. It calculates what t beleves are the best set of auctons to bd n (t does ths by predctng the auctons closng prces usng a fuzzy neural network, allocatng the goods to the customers t s actng on behalf of, and then calculatng the satsfacton degree of the allocaton). However the predcton of ths best set of auctons s hghly uncertan because t depends on the strateges, profles, and reservaton prces of an arbtrary set of agents. Therefore rather than just bddng n ths set, an agent could also decde to bd n other auctons that are lkely to have broadly the same outcome because, by dong so, ts chances of obtanng the goods are ncreased (more places to buy from) and the lkely satsfacton degree compared wth what s beleved to be the best ones s stll reasonably hgh. In more detal, our algorthm adopts the heurstc of bddng n the expanded set of auctons (the best set, plus those that have a broadly smlar satsfacton degree) n the order of ncreasng aucton end tme. Thus we term t an Earlest Closest Frst (ECF) algorthm. Moreover, as the goods are composed of multple attrbutes, the agent may have to make trade-offs between them n ts bddng n order to best satsfy the user s preferences. Thus, for example, a user may deally wsh to fly out on a Saturday, return the followng Wednesday and fly wth Brtsh Arways, but would be wllng to accept (for a lower prce) flght dates of Frday and Wednesday wth Quantas (a BA partner). To allow such flexblty (or mprecson), we choose to model preferences usng fuzzy sets (snce they have a proven track record for such tasks ([Luo et al. 2003, Zadesh 1965, 1975]). Specfcally, the agent s preferences are prvate nformaton that nclude () valuatons v for the good (expressed as fuzzy sets); () the ratngs for dfferent values of the good s attrbutes (expressed as fuzzy sets); and () the weghts whch balance the valuaton and the other attrbutes. By means of an example, consder the case of a student who wants to buy a flght tcket to New York. She prefers to buy a cheap tcket (cheap s a fuzzy term). Thus

4 468 M. He et al. the lower the prce, the hgher her degree of satsfacton. Ideally she wants to depart on Saturday, but t s acceptable to go on Frday or Sunday. Here, the date can be denoted as a trangular fuzzy number [He et al. 2003], where Saturday has the hghest satsfacton degree, and Frday and Sunday have lower ones. The arlnes she lkes can also be a fuzzy number where the membershps of the fuzzy set lke are gven by her satsfacton on the arlnes. Fnally, she can express the relatve mportance of the attrbutes of prce, date, and arlne by assgnng them the approprate weghts. To cope wth the uncertanty nherent n the mult-aucton context and to make trade-offs between the dfferent varants of the goods avalable s a complex decson makng problem. Ideally the closng prce of the auctons would be known frst n order to calculate the satsfacton degree of a bd. However, snce the closng prce s only known after the aucton s closed, t s mportant for the agent to make predctons about the lkely closng prces of the varous auctons. By so dong, the agent can determne whether t should place a bd at the current moment or t should delay because better deals mght subsequently become avalable. In our prevous work, we successfully used adaptve fuzzy nference methods for ths task n contnuous double auctons (CDA) [He et al. 2003] and the Tradng Agent Competton (TAC) [He and Jennngs 2004]. However, n both cases, the parameter adaptaton of the fuzzy rules was somewhat lmted. For example, our agent for the CDA can only adapt ts parameters n a sngle drecton of change (for example, all the parameters are bgger n a compettve envronment). However ths s napproprate for the mult-aucton context because each parameter n the strategy should be adjusted accordng to ts actual drecton of change (for example, the center of the membershp functon for the fuzzy set medum may need to go up, and the correspondng wdth may need to go down to reflect the fact that ths fuzzy set should cover a smaller range of hgher values). To rectfy ths, we explot fuzzy neural networks (FNNs) [Jang 1993] snce these can do the fuzzy reasonng and, through learnng, can adjust the parameters of the fuzzy terms and the consequent output as the auctons progress. Ths adaptaton enables the agent s bddng behavor to better reflect the current state of ts envronment (hereafter we call ths strategy FNN and the agent usng ths strategy an FNN Agent 3 ). We choose FNN for a number of reasons (1) As wth a regular neural network, t can adapt ts parameters to better ft the prevalng stuaton whch cannot readly be determned a pror. (2) In contrast to a regular neural network, t s easer to put pror knowledge about the doman nto the system (va the fuzzy rules), and ths speeds up the learnng process. (3) An FNN s easer to nterpret than a standard neural network (because the output s obtaned based on rules), and so t s easer to understand how and why the output s as t s. The work descrbed n ths artcle advances the state of the art n the followng ways. Frst, we develop for the frst tme an agent bddng algorthm (ECF) 3 To clarfy, an agent that uses the ECF bddng strategy needs to predct the auctons closng prces. In our case, ths s acheved va a FNN (although other methods could be used n the future). Indeed, n Secton 5, we do use an alternatve method for makng ths predcton, but retan the same ECF strategy as per the FNN agent benchmarkng purposes.

5 A Heurstc Bddng Strategy for Buyng Multple Goods 469 for obtanng multple goods from multple overlappng Englsh auctons. We beleve ths strategy s usable n a wde varety of e-commerce settngs because t s based on readly observable nformaton that can be found n many aucton settngs and because the underlyng heurstc of bddng n auctons that have broadly smlar expected returns s also wdely applcable. Second, fuzzy neural networks are developed that can make predctons about the closng prces and adapt the parameters n the neural network through both offlne and onlne learnng to sut the envronment the agent s stuated n. Through emprcal evaluaton the agent that uses ECF combned wth the FNN s shown to be effectve n a wde range of stuatons. Fnally, by explotng a fuzzy set representaton of the user s preferences, the strategy s able to make tradeoffs between the varous attrbutes of the goods the agent purchases and can cope wth the nherent mprecson/flexblty that often characterses a user s preferences. The rest of ths artcle s structured as follows. Secton 2 descrbes the ECF bddng algorthm, and Secton 3 descrbes the FNN mplementaton. Secton 4 gves an example of a flght aucton scenaro n whch the operatonal effectveness of our algorthm s evaluated. Secton 5 actually provdes the systematc emprcal evaluaton of the strategy and benchmarks t aganst a number of strateges that have been proposed n the lterature. Secton 6 dscusses the related work n multple aucton bddng, mult-attrbute auctons, and fuzzybased bddng methods. Fnally, Secton 7 concludes and presents avenues for future research. 2. THE EARLIEST CLOSEST FIRST BIDDING ALGORITHM Ths secton detals the ECF algorthm. Frst, however, we ntroduce some specfc terms n our aucton context and then the algorthm s descrbed. There are multple auctons n the market, each sellng one unt of good. There are three states for each aucton () watng: before ts start tme, nothng happens n ths aucton; () runnng: the aucton s open for bds; and () closed: the aucton fnshes when the market tme s bgger than the aucton s closng tme and there have been no actve bds n the aucton for a fxed perod. Each agent ams to buy multple goods, thus t consders bddng f and only f the sum of the bds t holds (that s, those auctons n whch t s the actve bdder) and owns (closed auctons n whch the agent won) s less than the number of good t desres. 4 If t decdes to bd, the agent needs to determne whch auctons t should bd n. To do so, t frst determnes the auctons that best satsfy the user s preferences (calculaton s detaled n Secton 3.1) gven ts expectaton about the closng prces of each aucton. Then, rather than placng a bd n the selected auctons mmedately, t bds n auctons that close earler than the selected auctons and have an evaluaton close (a fuzzy term) to that of the selected auctons. The ntuton here s that, gven the sgnfcant degrees of uncertanty that exst, precse calculatons about the closng prces are smply not relable and an aucton that appears slghtly less promsng may well 4 The case where the agent bds n more auctons than the number of goods t wants s not consdered here.

6 470 M. He et al. Fg. 1. The Earlest Closest Frst bddng algorthm. turn out to be comparable or even better. Gven ths, the agent should consder bddng n auctons that have broadly smlar expected returns so as to ncrease ts chances of obtanng the tem (by partcpatng n more auctons), whle ensurng the lkely return s one of the hghest. Thus, for each good t desres, f there are such close auctons, the agent wll bd n the selected auctons n order of ncreasng closng tme that s, bd frst n the one that s gong to close frst, then n the one that wll close next, and so on (hence the name Earlest Closest Frst). The degree of closeness that s requred to trgger bddng s captured by the threshold (λ [0, 1]). Thus f the dfference s wthn λ, the agent wll bd n the aucton. In ths sense, the choce of λ represents the rsk atttude of the user. If λ s hgh, the agent can be vewed as beng rsk averse because t bds n many more auctons n order to maxmse ts chance of gettng the good (although t s lkely to get a less satsfactory set of goods because t may accept a hgher ask prce). If λ s low, the agent s takng a greater rsk because t s tryng to obtan a hgh degree of satsfacton (but by not bddng n as many auctons, t has a lower chance of actually beng successful). If λ s somewhere n between, the agent s strkng a balance between the two postons. In more detal, the decson makng algorthm ECF s gven n Fgure 1. In ths algorthm, n actve, n own, and n demand are the number of goods the agent holds, owns, and desres, respectvely. An explanaton of the algorthm s key functons s as follows.

7 A Heurstc Bddng Strategy for Buyng Multple Goods 471 The functon of AuctonRunnng (lne 1) returns true f there are stll avalable auctons to bd n, false otherwse. The functon of update() (lne 2) returns changes n the auctons snce they were last montored. Such changes nclude whether the agent s holdng an actve bd or has obtaned the good, the updated ask prce of each aucton, the transacton prce for any auctons that recently closed, and the number of auctons left to bd n. The functon of predct() (lne 3) predcts all aucton s closng prces gven the current market stuaton and the hstory transacton prces. The agent uses a FNN to predct the closng prces n ths artcle (see Secton 3.1). The functon of allocate() (lne 4) allocates all the goods the agent owns and possbly owns to ts user accordng to ther preferences. The later ncludes the goods the agent holds n watng or runnng auctons. The way that we assgn the aucton to the user s through an assgnment algorthm dscussed n Secton 3.4. The functon of tobd(g) (lnes 6, 7 and 16) returns the aucton ID of the aucton n whch good g s beleved to have the hghest degree of satsfacton gven the predcton of the closng prces. Ths s the output of the allocaton n allocate(). If there s a best aucton for buyng good g gven the allocaton, the returned value wll be the aucton ID; otherwse, the agent wll bd n any aucton n whch the current satsfacton degree s postve. The functon of RunnngAuctons() (lne 12) returns a lst L of all the currently runnng auctons n ascendng order of ther end tmes. The functon of evaluate(a, g) (lnes 7 and 15) returns the evaluaton of aucton a gven the agents preference for good g at ts current ask prce. Ths evaluaton balances both prce and the other attrbutes of the goods usng a fuzzy aggregaton method (see Secton 3.3). The functon of chkthreshold() (lne 17) returns the threshold parameter for the agent gven the current stuaton of the aucton market. The threshold λ s determned by the number of auctons that have a postve satsfacton degree for the agent n the market at that partcular moment n tme. Here the general rule for choosng λ s: the more such auctons there are, the smaller λ should be. Ths captures the ntuton that, f there are many chances for the agent to wn the good, t can have a hgher threshold so that t wll have a hgher satsfacton degree for the purchased goods (and vce versa). The experments n Secton 5.4 show the effect of dfferent λs on the performance of the agent. The functon of bd(a) (lnes 19 and 23) places a bd n aucton a. The prce to place s the ask current prce of the aucton plus the bd step h(a). To realze ths algorthm, a number of predcton technques are needed (lne 3). Here we use fuzzy neural networks (for the reasons outlned n Secton 1) and ther applcaton s dscussed n Sectons 3.1 and 3.2. Moreover, an evaluaton method s needed for rankng the varous auctons (Secton 3.3), and a good allocaton method s needed to decde whch auctons should be assgned to whch user (Secton 3.4). However, n other applcatons, a

8 472 M. He et al. Fg. 2. Reference prce (p ref ) calculaton for the FNN. getrelprce() returns the average transacton prce of all the auctons (wth the same attrbutes as aucton ) that have closed snce the agent started bddng. getavgprce() returns the hstorcal average transacton prce n the hstory for auctons (wth the same attrbutes as aucton ). range of other technologes could be used to acheve the objectves of the ECF algorthm. 3. THE FNN IMPLEMENTATION Ths secton detals the FNN mplementaton of the ECF algorthm. Frst, we descrbe how the FNN s structured and how t operates to obtan the predcted closng prce. Second, the learnng algorthm of the FNN s descrbed. Thrd, the way of evaluatng auctons s ntroduced. Fnally, the allocaton method that allocates the goods to the user s gven. 3.1 FNN Predcton To reason about the expected closng prce of each aucton, the FNN agent consders a per aucton reference prce (p ref ), the order n whch the auctons are due to close (o aucton ), and the number of alternatve auctons (n aucton ) where the smlar tem s avalable. Here the reference prce represents a lkely value at whch the aucton wll close for that partcular varant of the good [He et al. 2003]. It s computed by consderng the transacton prces of auctons that have prevously sold the specfed good and the average transacton prce n the hstory records for the specfed good (see functon getrefprce() n Fgure 2). The FNN agent records such data examples and removes some of the oldest data to ensure t only learns based on the latest data. After each game fnshes, the agent adjusts the parameters to better reflect the prevalng crcumstances (see Secton 3.2). To predct the closng prces of the auctons, fuzzy reasonng s used. Through analyzng our expermental data, we found that the reference prce, aucton s closng order, and the number of substtute auctons are closely correlated wth the actual closng prces. 5 Thus, fuzzy rules (defned n Table I) are desgned to capture the relaton among these factors. In partcular, p ref s expressed usng the fuzzy lngustc terms hgh, medum and low; o aucton s expressed by early, medum and late; and n aucton s expressed by bg, medum and small. The consequent output s expressed as the nteger numbers very hgh, hgh, medum and low. Thus, the FNN agent takes three nputs (p ref, o aucton, n aucton ) and has one output (the expected aucton closng prce p close ). Accordng to the rule base just 5 In the future, we may consder addng other attrbutes, but our current analyss ndcates these parameters are the man determnants of outcome.

9 A Heurstc Bddng Strategy for Buyng Multple Goods 473 R 1 : R 2 : R 3 : R 4 : R 5 : R 6 : R 7 : R 8 : R 9 : R 10 : R 11 : R 12 : R 13 : R 14 : R 15 : R 16 : R 17 : R 18 : R 19 : R 20 : R 21 : Table I. The FNN Agent s Rule Base If p ref s hgh and o aucton s early and n aucton s small then p close s very hgh. If p ref s hgh and o aucton s early and n aucton s medum then p close s very hgh. If p ref s hgh and o aucton s early and n aucton s bg then p close s hgh. If p ref s hgh and o aucton s medum and n aucton s medum then p close s hgh. If p ref s hgh and o aucton s medum and n aucton s bg then p close s medum. If p ref s hgh and o aucton s late and n aucton s medum then p close s medum. If p ref s hgh and o aucton s late and n aucton s bg then p close s low. If p ref s medum and o aucton s early and n aucton s small then p close s very hgh. If p ref s medum and o aucton s early and n aucton s medum then p close s hgh. If p ref s medum and o aucton s early and n aucton s bg then p close s medum. If p ref s medum and o aucton s medum and n aucton s medum then p close s medum. If p ref s medum and o aucton s medum and n aucton s bg then p close s low. If p ref s medum and o aucton s late and n aucton s medum then p close s low. If p ref s medum and o aucton s late and n aucton s bg then p close s very low. If p ref s low and o aucton s early and n aucton s small then p close s hgh. If p ref s low and o aucton s early and n aucton s medum then p close s medum. If p ref s low and o aucton s early and n aucton s bg then p close s low. If p ref s low and o aucton s medum and n aucton s medum then p close s low. If p ref s low and o aucton s medum and n aucton s bg then p close s very low. If p ref s low and o aucton s late and n aucton s medum then p close s very low. If p ref s low and o aucton s late and n aucton s bg then p close s very low. Fg. 3. Overvew of the FNN archtecture. mentoned, we developed a FNN wth 5 layers (as shown n Fgure 3). The nput varables correspond to the nodes n layer 1. The nodes n layer 2 correspond to ndvdual rules for reasonng about the closng prces of the auctons. Nodes n layer 3 calculate the relatve mportance (weght) of each of these rules. The nodes n layer 4 combne the output of each rule nto the overall output. Fnally, the node n layer 5 sums up all the outputs n layer 4 and gves the predcted

10 474 M. He et al. aucton closng prce. In more detal: Layer 1. Each node n ths layer generates the membershp degrees of a lngustc label for each nput varable (for example, reference prce s low or there are a large number of substtute auctons n the market). Specfcally, the th node performs the followng (fuzzfcaton) operaton: O (1) = μ A (x) = e (x c ) 2 2δ 2 + γ, (1) where O (1) s the output of layer 1 (that s, the membershp degree wth respect to the correspondng fuzzy sets), x s the nput to the th node, and A s the lngustc value (hgh, medum, low, etc.) assocated wth ths node. The set of parameters (c, δ ) determnes the shape of the membershp functon. 6 These parameters can be adapted by learnng (as we wll explan n Secton 3.2). γ s a very small number (here we choose ) that avods the output n layer 1 from becomng zero. Layer 2. Each node n ths layer calculates the frng strength (the mmmum of all the nputs, and t s n the range of [0, 1]) of each rule (n Table I) va the multplcaton operaton: O (2) = w = j S (1){μ A j }, (2) where S (1) s the set of nodes n layer 1 whch feed nto node n layer 2, and w s the output of ths node (that s, the strength of the correspondng rule). Layer 3. The th node of ths layer calculates the rato of the th rule s frng strength to the sum of all rules frng strengths: O (3) = w = w j S w, (3) (2) j where S (2) s the set of nodes n layer 2. Ths rato ndcates the relatve mportance of each rule. Layer 4. The th output of the node s calculated by: O (4) = r w j, (4) j S (3) where j S (3) s the set of nodes n layer 3 that feed nto node. The output of ths layer combnes all the outputs of the rules that have the same consequent output. Layer 5. The sngle node n ths layer aggregates the overall output of the FNN (that s, p close ) as the summaton of all ncomng sgnals: O (5) = r, (5) S (4) j S (3) where j S (4) s the set of nodes n layer 4. w j 6 Here we assume that ths s a Gaussan functon. Ths s because t has nonzero dervatves throughout the unverse of dscourse and s therefore easy to mplement. Also, the dervatves of a Gaussan functon are contnuous and smooth, thus, t can produce good tranng performance.

11 A Heurstc Bddng Strategy for Buyng Multple Goods 475 Gven the nature of ths decson makng task, t s mportant that the varous parameters of the FNN algorthm ft the prevalng context as accurately as possble. Ths s acheved through combned learnng (see Secton 3.2 for more detals). To start, a number of smulated games 7 are used to set the ntal parameters of the FNN. After ths, the agent can be used n an operatonal settng to actually purchase goods. The agent keeps track of the varous auctons and records the latest data whch s then combned wth the old data. The combned data set s fed nto the FNN as new tranng examples. However, these examples are weghted more hghly than older ones and so enable the agent to better reflect prevalng crcumstances (but wthout beng completely reactonary to the last set of changes). To llustrate the operaton of ths archtecture consder the followng example. Let the reference prce for the next aucton to bd n be p ref = 20, and assume that t wll be the fourth one to close o aucton = 4. Suppose there are 8 substtute auctons. Thus (20, 4, 8) s fed nto the FNN. Let these values be assgned the followng membershp degrees of the followng fuzzy sets: medum (20) = 0.8, late (4) = 0.7 and bg (8) = 0.5 (from (1)). These values are then the respectve outputs of nodes 2, 6 and 7 n layer 1. Then, takng R 14 as an example, the output of node R 14 n layer 2 s = 0.28 (from (2)). Then, suppose the sum of all the frng strengths n layer 2 s 3. The weght of R 14 (output of w 14 ) wll be 0.28/3 = (from (3)). Thus, R 14 contrbutes among all the rules. After ths, n layer 4, there are 3 rules that have the same consequent output, whch s very low, that s, R 14, R 19, R 20, and R 21 (from Fgure 3). Let r very low = 12, w 19 = 0.1, w 20 = 0.1, and w 21 = 0.1. In whch case, the output of node very low s 12 ( ) = (from (4)). Fnally, the output of layer 5 s the sum of all the outputs of layer 4 (usng Equaton (5)) one of whch wll be the comng from the very low node. Fnally, to guarantee the predcted closng prce s alway hgher than the current ask prce, the predcted aucton closng prce wll be P close = max{p close, p now }, (6) where p close s the output of the FNN and p now s the current ask prce of the aucton. 3.2 FNN Learnng Gven the tranng data x ( = 1, 2, 3), the desred output value Y, and the fuzzy logc rules (from Table I), the parameters of the membershp functons for the FNN s nput varables are adjusted by supervsed learnng. Here the 7 The ntal parameters of the FNN can ether be set drectly by the user or can be set by playng smulated games. A smulated game s played by strateges or humans n a test envronment where the varous agents compete but money does not actually change hands. In the cases where such games are not avalable, users can montor real aucton Web stes (such as ebay) and collect relevant data (snce the tradng hstory of most aucton stes s readly avalable). However, f the user s confdent about ther parameter settngs, the agent can be put drectly nto practce wthout gong through the smulated games.

12 476 M. He et al. goal s to mnmze the error (E) functon for all the tranngs E = 1( Y j O (5) ) 2, j (7) 2 j where Y j s the actual closng prce of pattern 8 j, and O (5) j s the predcted closng prce of the FNN for pattern j. For each set of tranng data, startng at the nput nodes, a forward pass s used to compute the actvty levels of all the nodes n the network. Then startng at the output nodes, a backward pass s used to compute E for all the hdden nodes. In our FNN agent, the O parameters that get adjusted durng learnng are the consequent output of each rule (r n layer 4) and the center and wdth of the Gaussan membershp functons for each of the fuzzy terms (c and δ n layer 1). For the parameters of r, the learnng rule of the FNN agent s based on standard gradent descent optmzaton [Rumelhart et al. 1986] ( r(t + 1) = r(t) + η E ), (8) r where η s the learnng rate (η [0.001, 0.01]). Thus, the learnng rule for adjustng the parameters of r n layer 4 and (c, δ )nlayer1 E r = E O (5) O (5) O (4) O (4) r = (O (5) Y ) j S (3) j S (3) w j, (9) Hence r s updated by: r (t + 1) = r (t) η(o (5) Y ) w j. (10) For parameters c and s, the conjugate gradent algorthm [Johansson et al. 1990] s used snce t s shown to be faster than the standard gradent descent optmzaton [Rumelhart et al. 1986]. Here suppose α s the parameter we are nterested n and the gradent of teraton t of the learnng s g t (t > 1), then the new search drecton s to combne the new steepest descent drecton wth the prevous one, that s, p t = g t + β t p t 1, (11) where by usng Fletcher-Reeves [Fletcher and Reeves 1964] update, β t = g t T g t gt 1 T g. (12) t 1 Thus, the adaptve rule of c n layer 1 s as follows (where S (2) means the set of nodes n layer 2 that are connected wth node n layer 1): E c = m S (2) ( E O (2) m O (2) m O (1) O (1) c 8 Here a pattern s a tranng example (for example, for the prevous example, a pattern mght be {{20, 4, 8},10.722}, where gven the nputs {20, 4, 8} the actual output s a prce of ). )

13 where A Heurstc Bddng Strategy for Buyng Multple Goods 477 = ( E O (3) k m S (2) E O (3) k k S (3) m O (3) k = O m (2) O (2) m O (1) O (1) c So the adaptve rule of c s: ) O (3) k O m (2) O (2) m O (1) O (1), (13) c = (O (5) 1 Y )r k ; (14) { k w k w m ( k w f k = m, k) 2 w m (15) ( k w otherwse; k) 2 = w m ; (16) O (1) = e (x c ) 2 2δ 2 (x c ). (17) c (t + 1) = c (t) + ηp t, (18) where p t = g t + β t p t 1 and g t = E c. Smlarly, from Equatons (14), (15), and (16), the adaptve rule of δ s derved as: E = ( ) E O (3) k O m (2) O (1), (19) δ O (3) k O m (2) O (1) δ where m S (2) k S (2) m O (1) δ = e (x c ) 2 2δ 2 δ 2 (x c ) 2. (20) Hence the adaptve rule of δ becomes δ (t + 1) = δ (t) ηp t, (21) where p t = g t + β t p t 1 and g t = E δ. 3.3 Evaluatng the Auctons Gven the expected aucton closng prces, the agent needs to make a decson about whch auctons to bd n. 9 For ease of expresson, we present ths evaluaton functon for the case where only prce and one other attrbute of the good are consdered (but the concepts are equally applcable for arbtrary numbers of attrbutes). Gven the user s preference on prce and other attrbutes (as defned n Secton 1), the evaluatons of the varous factors need to be ntegrated. In fuzzy theory, the process of combnng such ndvdual ratngs for δ 3 9 Such an evaluaton functon s used to evaluate the bddng strategy that consders more than one of the good s attrbutes n makng ts bddng choce. Thus, for example, all the benchmark strateges n Secton 5 explot such a functon.

14 478 M. He et al. an alternatve nto an overall ratng s referred to as aggregaton [Yager 1994]. Now let w p and w q be, respectvely, the weght of prce and the other attrbute that the agent s concerned wth, and u p be the evaluaton wth respect to prce, and u q the evaluaton wth respect to the other attrbute. Intutvely, the role of the aggregaton operator s to balance u p and u q and obtan an overall evaluaton u p,q somewhere between the two values. There are three man aggregaton operators that are commonly used and each of them has dfferent semantcs (conformng to dfferent user objectves) Weghted average operator u p,q = u p w p + u q w q. (22) Usng ths operator means that even f one of the evaluatons s very low, the overall output can stll be reasonably hgh. For example, f the user does not lke the tme of the flght but t s very cheap, the overall evaluaton can stll be hgh. Weghted Ensten operator [Luo et al. 2003] u p u p,q = u q 1 + (1 u p )(1 (23) u q ), where u p = (u p 1)w p, and u q = (u q 1)w q. Ths equaton ensures that f one evaluaton s not satsfed (that s, u p = 0oru q = 0), the overall evaluaton s 0. Intutvely, ths corresponds to the stuaton where both evaluatons must be satsfed more or less. For example, even f the flght tcket s free, the user cannot accept t snce travelng after a specfc date s totally useless (for example, he has a very mport meetng at a specfc date). Weghted unnorm operator [Yager and Rybalov 1996] (1 τ)u p u p,q = u q (1 τ)u p u q + τ(1 u p )(1 (24) u q ), where u p = (u p 1)w p max{w p,w q } +1, and u q = (u q 1)w q max{w p,w q +1, τ (0, 1) s the unt element } of ths operator. The unt element can be regarded as a threshold: f both the evaluatons are above the threshold, the overall evaluaton s enhanced; f both are less than the threshold, the overall evaluaton s weakened; f there s a conflct between the two evaluatons, the overall evaluaton s a compromse. For example, f the user lkes the date and prce, the overall evaluaton s very hgh; f the user hates the date and prce, the overall evaluaton s even lower; and f the user lkes the date but hates the prce, then some ntermedate value s chosen. Snce these operators are all plausble means of aggregatng prce and the other attrbutes, and none s necessarly superor n all cases, we need to emprcally evaluate the mpact of these operators on the performance of the agents. Ths we do n Secton Goods Allocaton The agent needs to allocate the goods t owns and potentally owns to ts customers n order to maxmze the overall satsfacton degree (f there s a sngle

15 A Heurstc Bddng Strategy for Buyng Multple Goods 479 customer, then ths s a trval stage). Thus good allocaton takes place each tme the agent accesses the market and when the market nformaton has been updated. The goods are allocated to the agent s users optmally so as to maxmze the sum of the users satsfacton. Here ths allocaton process s regarded as an assgnment problem whch we solve usng a shortest augmentng path algorthm [Jonker and Volgenant 1987] (ths s commonly used to solve assgnment problems because of ts stablty and effcency). In more detal, suppose D s a d d square, 10 s 11 s 1d D =, s d1 s dd where d s the number of auctons, and s j s the satsfacton degree of the user s j th requrement for aucton a : 11 evaluate(a, g j )f n demand, s j = 99 f owns or holds a good n a and j > n demand 0 otherwse, where 99 s used to avod the owned goods beng allocated to the dummy nodes. Gven ths nput, suppose an allocaton X s x 11 x 1d X =, x d1 x dd where each row and each column has only a sngle 1 (that s, each aucton s only allocated to one user). Thus, d x j = 1, j =1 d x j = 1, =1 and the objectve functon to mnmze s d d (s j x j ), j =1 =1 where x j = 0or1. Usng ths method, the agent can decde how to allocate the goods t owns and holds to ts users optmally gven the ask prce or the predcted prce of the auctons. Ths assgnment method s also used to calculate the performance of the agents at the end of the game. 10 Here a square s necessary n order to use the algorthm. The row represents the aucton, and the column represents the goods the agent desres. To use the algorthm, some dummy nodes may need to be added to make a square so that the row number s equal to the column number. 11 Our problem here s a maxmzaton problem, but n order to use the shortest augmentng path algorthm, we need to put a mnus before the evaluaton value.

16 480 M. He et al. 4. FLIGHT AUCTION SCENARIO Ths secton provdes an ntutve scenaro 12 n whch the operaton of our ECF algorthm can be exemplfed and ts performance emprcally assessed (see Secton 5). Here we consder how to model the user s preferences as fuzzy sets, outlne the envronmental settng for realzng the scenaro, and present the tranng results for the FNN agent. In more detal, there are a number of arlnes sellng flght tckets through auctons. Each aucton s sellng one flght tcket. Each software agent s actng on behalf of one customer, and they are nformed of the customer s preferences about prces and travel dates. 13 The am of the agent s to obtan the goods that maxmze the sum of ts users satsfacton. 4.1 Users Preference Settngs We descrbe the valuaton v of a customer for a flght tcket as a trapezod shape fuzzy number (l bottom, l top, r top, r bottom ), where l bottom = 0 and l top = 0, and r top and r bottom are the values where the satsfacton starts to decrease and where t becomes 0. In ths case, the hgher the prce of the good, the lower the satsfacton degree. When the prce ncreases to the valuaton of the agent, the satsfacton degree s 0. The travel date q s represented as a trangular fuzzy number 14 (l q, c q, r q ), where c q s the preferred date and l q and r q are the left and rght lmts, respectvely. 15 By way of llustraton, suppose a customer s valuaton for the tcket s about 300 pounds and she wants to travel on or about the 15th of December. These preferences are expressed as fuzzy sets by the respectve membershp functons μ P and μ Q gven n Equatons (25) and (26) and are shown graphcally n Fgures 4 and 5. 1 f x 100, 300 x μ P (x) = f 100 < x < 300, (25) f x 300. y 12 f 12 y 15, 3 μ Q ( y) = 18 y f 15 y 18, 3 0 f y 12 or y 18. (26) 12 We choose (for reasons of famlarty) a flght aucton scenaro where an agent s tryng to buy multple flght tckets on behalf of a user. Ths s a problem that we often meet n real lfe and t fts the requrements of our context as outlned n Secton 1. It has also been used n the Tradng Agent Competton whch s an nternatonal forum for benchmarkng bddng strateges (see Secton 6 for more detals). 13 For reasons of smplcty, we focus on the two attrbute case. However, the prncple s smlar wth more attrbutes. 14 Any knd of fuzzy number can be used here, for example, trapezod or bell-shaped fuzzy numbers. We choose a trangular one smply because t s the most commonly used. 15 If a user has a crsp preference, for example, he has to travel on the 15th of December, the smlarty degree of the 15th s 1 and 0 otherwse.

17 A Heurstc Bddng Strategy for Buyng Multple Goods 481 Fg. 4. Customer s preference about prce. Fg. 5. Customer s preference about travel date. 4.2 Expermental Settngs The experments am to cover a broad range of scenaros. All the parameters about the envronment are assgned at the begnnng of the game. Here we suppose that all auctons start at a prce of 100, and all have a bd ncrement of 10 pounds. Also a day n the game equals ζ = 5 seconds of real tme 16 ; an aucton s startng tme t start s randomly chosen from a unformly dstrbuted range (0, (q 5)ζ ). Ths ensures all the auctons start at least fve days before the travel date 17 ; aucton s end tme s randomly chosen from a unformly dstrbuted range (t start + 2ζ,(q 3)ζ ). Ths guarantees that the auctons close at least three days before the travel date 18 ; each agent s assgned to a customer whch has n requrements, n [1, 8]; 16 The length of one day can be shorter f t can be guaranteed that all the agents have tme to respond n the market. 17 We choose fve to ensure the agent has a reasonable tme to transact the tcket before the travel date. 18 We choose three to ensure that the start tme of the aucton s before the end tme, and there s a reasonably long tme for the aucton.

18 482 M. He et al. Fg. 6. Learnng curve: root mean square error versus tme. all the agents start bddng at the begnnng of the game 19 ; aucton s flght date q s chosen randomly from a unformly dstrbuted range (11, 19); the valuaton of the goods for a customer are randomly chosen from a unformly dstrbuted range (170, 370); a customer s preferred travel date s randomly chosen from a unformly dstrbuted range (12, 18) The FNN Agent s Learnng Algorthm As dscussed n Secton 3.2, the agent engages n a perod of offlne learnng n order to provde ntal parameters for the FNN agent. In more detal, Fgure 6 shows the tranng and testng curves of the root mean square error wth respect to the number of tranng epochs. 21 The errors are computed for both the tranng set and the test set. After each game s played, the new game data (test data) and the latest game data before ths game (tranng set) are nput to the FNN for learnng. After 100 tranng epochs, t can be seen that the error between the target output and the actual output reaches ts lowest pont and so the parameters settngs of ths pont are those used when the agent s made operatonal. Specfcally, Fgures 7 and 8 show, respectvely, the comparson of the FNN parameters before and after tranng. As can be seen, the parameters for each of the three nputs are adjusted from the orgnal settngs defned by the feld experts. 4.4 Parameter Adaptaton n Dfferent Envronments Ths secton compares the parameter adaptaton n two envronments where the supply s hgh (25 auctons) and low (15 auctons). Specfcally, Fgures 9 19 Ths s because we want to evaluate all types of agents farly. If some agents start bddng late, they wll be at a dsadvantage compared wth those who start bddng early. 20 Ths range s smaller than the range of the auctons flght dates because ths preferred travel date s a fuzzy number. Thus when defuzzfed, t wll actually cover the full range of the flght s dates. 21 The reference prce and ask prce are scaled by dvdng by 10 durng learnng. However, ths does not affect the result n any way. In the real-world scenaro, the prce can be any number.

19 A Heurstc Bddng Strategy for Buyng Multple Goods 483 Fg. 7. Comparng antecedent membershp functons (MFs) before (dashed lne) and after (sold lne) offlne learnng. Fg. 8. Comparng consequent membershp functons before (dashed lne) and after (sold lne) offlne learnng. to 12 show how the parameters are adjusted dfferently n dfferent envronments. In Fgure 10, when supply s hgh, the closng prces tend to be low and, thus, the consequent parameters are lower than the ntal ones. In contrast, n Fgure 12, when supply s low, the closng prces tend to be hgh, and the consequent parameters are hgher than the ntal ones.

20 484 M. He et al. Fg. 9. Comparng antecedent membershp functons (MFs) before (dashed lne) and after (sold lne) offlne learnng n-hgh supply envronment. Fg. 10. Comparng consequent membershp functons before (dashed lne) and after (sold lne) offlne learnng n a hgh supply envronment.

21 A Heurstc Bddng Strategy for Buyng Multple Goods 485 Fg. 11. Comparng antecedent membershp functons (MFs) before (dashed lne) and after (sold lne) offlne learnng n a low supply envronment. Fg. 12. Comparng consequent membershp functons before (dashed lne) and after (sold lne) offlne learnng n a low supply envronment. 5. EMPIRICAL EVALUATION Ths secton evaluates the FNN agent by comparng t n a varety of envronments wth other agents that use bddng strateges proposed n the lterature. In partcular, we are nterested n assessng the performance of each knd of agent n dfferent envronments. There are three man groups of experments, and there are a number of sessons whch correspond to experments wth

22 486 M. He et al. dfferent settngs (as per Secton 4.2). For each sesson, at least 200 games 22 are played among the agents. Snce the number of agents n each experment vares, the performance ρ K of a partcular type K of agent (for example, FNN) s calculated as the average satsfacton degree per agent of the knd K, that s: nk ρ K =, (27) n K where n K s the number of type K agents n the same game, u () j means the evaluaton of customer j, and m s the number of customers of agent. Snce most of the extant mult-aucton bddng strateges are concerned solely wth prce (see Secton 6), we had to extend them to deal wth bddng for goods that are characterzed by multple attrbutes. Thus, n all cases, the agents used the aggregaton operators specfed n Secton 3.3 n order to make trade-offs between prce and travel date. To deal wth multple goods, the allocaton functon can decde whch user bds n whch aucton. The specfc benchmark strateges we used are the followng. Greedy (GRD) Strategy (adapted from Byde [2001a]). Whle the sum of the number of goods held and owned s less than what the agent needs, bd n auctons where the aucton s current satsfacton has the hghest evaluaton (as defned n Secton 3.3); Fxed Aucton (FIX) Strategy (adapted from Byde et al. [2002]). Select at the begnnng of the game the auctons n whch bds wll be placed, and then only bd n these auctons. The auctons chosen here are those where the sum of the users satsfacton s hghest for date (at ths tme none of them have a value for prce). The agent contnues bddng n ts selected aucton untl the prce satsfacton degree equals zero, n whch case t wll swtch to another aucton (untl all those n the fxed set have been tred). Average (AVG) Strategy. AVG also uses the ECF algorthm, but t uses a much smpler predcton functon based on the past hstory transacton prces to predct the closng prces (that s, to mplement the predct() functon n Fgure 1). In more detal, t calculates the average closng prces of all the auctons for each knd of good from the recent games. Suppose n the latest N games, there are m auctons wth attrbute, then the predcted closng prce of an aucton wth attrbute s m j m p () close = j m, where p () j s the real closng prce of aucton j wth attrbute. 5.1 Varyng Agent Populatons Ths experment ams to compare the performance of the dfferent types of agents when there are varyng numbers of the other agent types n the populaton (here the populaton sze s fxed). In ths experment, we studed three 22 A t-test showed that 200 games are suffcent to gve a sgnfcant rankng among the agents. A p value of p < 0.05 s reported for all the experments. u () j p () j

23 A Heurstc Bddng Strategy for Buyng Multple Goods 487 envronments: when supply s low (15 auctons), medum (20 auctons), and hgh (25 auctons). When the weghted average operator s used and the weght rato s w p : w q = 1 : 1, Fgure 13 shows the results when there are fxed numbers of each type of agent n a sesson (a), and when one type domnates numercally (b) to (e). 23 From ths, t can be seen that the FNN agents perform better than other agents n all the cases consdered. We attrbute ths success to ther ablty to be able to select the auctons to bd n accordng to the relatvely correct predcton on the closng prces of the auctons. In more detal, the FNN agent s better than GRD agents. Ths s because the GRD agent endeavors to make a transacton whenever t can. Its man shortcomng s that t only consders ongong auctons (t gnores those that have not yet started and so fals to consder the full set of potental purchasng opportuntes when makng bddng decsons). Thus, t sometmes buys a good at the user s valuaton prce when, f t had wated, t mght well fnd subsequent auctons wth lower closng prces. The FIX agent performs the worst because t only bds n auctons where t knows a pror that t can get hgh satsfacton on the flght date. Ths leads to a poor overall performance because t msses auctons that have a hgh evaluaton on prce but a lower one on date. As s shown n Fgure 13, FIX agents have the smallest transacton numbers. It can also be seen from all the subfgures n Fgure 13 that, when the supply s hgh, all the agents have a hgher performance value than when there s a low supply. Ths s as expected because, n general, the auctons close at a lower prce when the supply s hgh (because there s less comptton). In Fgure 13(b), GRD agents domnate the market, and they often make the transacton prce of some auctons very hgh. Moreover, t can be seen from the performance n Fgure 13(b), when compared wth other fgures n Fgure 13, that GRD agents have a relatvely worse performance value. When FIX agents domnate the market (Fgure 13(d)), all the other agents have a hgher performance value compared to the other cases n Fgure 13. Ths s because the FIX agents only bd n a small number of auctons. Thus other auctons have less competton and, consequently, lower prces. 5.2 Varyng Aggregaton Operators Ths experment studes the mpact on the dfferent types of agents of the dfferent ways of tradng-off the prce and travel date 24 (see Fgure 14). To do ths, the number of auctons s generated randomly n the range of [15, 25] and the number of agents s 8. We also fx the weght of w p : w q = 1 : 1 for each 23 In (a), there are equal numbers of each agent, and we have 4 knds of agents. There are 4 agents and each agent desres 8 unts of goods. In (b) to (e), there are 2 of one knd of agent and 1 of the other three knds. Thus, there are 5 agents n total, and each agent desres 7 unts of goods. 24 We do not beleve t s approprate to compare the performance between the dfferent aggregaton operators. Ths s because the users ntenton n choosng the operators reflects dfferent objectves whch are, n turn, reflected n the dfferent semantcs of the operators. The weghted average operator s used to balance all the evaluatons; the weghted Ensten operator to satsfy all the evaluatons more or less; and the weghted unnorm operator to compromse postve and negatve evaluatons.