On-demand, Spot, or Both: Dynamic Resource Allocation for Executing Batch Jobs in the Cloud

Transcription

1 On-demnd, Spot, or Both: Dynmic Resource Alloction for Executing Btch Jobs in the Cloud Ishi Menche Microsoft Reserch Ohd Shmir Weizmnn Institute Nvendu Jin Microsoft Reserch Abstrct Cloud computing provides n ttrctive computing prdigm in which computtionl resources re rented on-demnd to users with zero cpitl nd mintennce costs. Cloud providers offer different pricing options to meet computing requirements of wide vriety of pplictions. An ttrctive option for btch computing is spot-instnces, which llows users to plce bids for spre computing instnces nd rent them t (often) substntilly lower price compred to the fixed on-demnd price. However, this rises three min chllenges for users: how mny instnces to rent t ny time? wht type (on-demnd, spot, or both)? nd wht bid vlue to use for spot instnces? In prticulr, renting on-demnd risks high costs while renting spot instnces risks job interruption nd delyed completion when the spot mrket price exceeds the bid. This pper introduces n online lerning lgorithm for resource lloction to ddress this fundmentl trdeoff between computtion cost nd performnce. Our lgorithm dynmiclly dpts resource lloction by lerning from its performnce on prior job executions while incorporting history of spot prices nd worklod chrcteristics. We provide theoreticl bounds on its performnce nd prove tht the verge regret of our pproch (compred to the best policy in hindsight) vnishes to zero with time. Evlution on trces from lrge dtcenter cluster shows tht our lgorithm outperforms greedy lloction heuristics nd quickly converges to smll set of best performing policies. Introduction This pper presents n online lerning pproch tht lloctes resources for executing btch jobs on cloud pltforms by dptively mnging the trdeoff between the cost of renting compute instnces nd the user-centric utility of finishing jobs by their specified due dtes. Cloud computing is revolutionizing computing s ser- Figure : The vrition in Amzon EC2 spot mrket prices for lrge computing instnces in the US Est-cost region: Linux (left) nd Windows (right). The fixed on-demnd price for Linux nd Windows instnces is.34 nd.48, respectively. vice due to its cost-efficiency nd flexibility. By llowing multiplexing of lrge resources pools mong users, the cloud enbles gility the bility to dynmiclly scle-out nd scle-in ppliction instnces cross hosting servers. Mjor cloud computing providers include Amzon EC2, Microsoft s Windows Azure, Google AppEngine, nd IBM s Smrt Business cloud offerings. The common cloud pricing schemes re (i) reserved, (ii) on-demnd, nd (iii) spot. Reserved instnces offer users to mke one-time pyment for reserving instnces over -3 yers nd then receive discounted hourly pricing on usge. On-demnd instnces llow users to py for instnces by the hour without ny long-term commitment. Spot instnces, offered by Amzon EC2, llow users to bid for spre instnces nd to run them s long s their bid price is bove the spot mrket price. For btch pplictions with flexibility on when they cn run (e.g., Monte Crlo simultions, softwre testing, imge processing, web crwling), renting spot instnces cn significntly reduce the execution costs. Indeed, severl enterprises clim to sve 5%-66% in computing costs by using spot instnces over on-demnd instnces, or their combintion [3]. Reserved instnces re most beneficil for hosting long running services (e.g., web pplictions), nd my

2 lso be used for btch jobs, especilly if future lod cn be predicted [9]. The focus of this work, however, is on mnging the choice between on-demnd nd spot instnces, which re suitble for btch jobs tht perform computtion for bounded period. Customers fce fundmentl chllenge of how to combine on-demnd nd spot instnces to execute their jobs. On one hnd, lwys renting on-demnd incurs high costs. On the other hnd, spot instnces with low bid price risks high dely before the job gets strted (till the bid is ccepted), or frequent interruption during its execution (when the spot mrket price exceeds the bid). Figure shows the vrition in Amzon EC2 spot prices for their US est cost region for Linux nd Windows instnces of type lrge. We observe tht spot mrket prices exhibit significnt fluctution, nd t times exceed even the ondemnd price. For btch jobs requiring strict completion dedlines, this fluctution cn directly impct the result qulity. For exmple, web serch requires frequent crwling nd updte of serch index s the freshness of this dt ffects the end-user experience, product purchses, nd dvertisement revenues [2]. Unfortuntely, most customers resort to simple heuristics to ddress these issues while renting computing instnces; we exemplify this observtion by nlyzing severl cse studies, reported on the Amzon EC2 website [3]. Litmus [6] offers testing tools to mrketing professionls for their web site designs nd emil cmpigns. Its heuristic for resource lloction is to first lunch spot instnces nd then on-demnd instnces if spot instnces do not get llocted within 2 minutes. Their bid price is set to be bove the on-demnd price to improve the probbility of their bid getting ccepted. Similrly, BrowserMob [7], strtup tht provides website lod testing nd monitoring services, ttempts to lunch spot instnces first t low bid price. If instnces do not lunch within 7 minutes, it switches to ondemnd. Other compnies mnully ssign dely sensitive jobs to on-demnd instnces, nd dely-tolernt ones to spot instnces. In generl, these schemes do not provide ny pyoff gurntees or how fr do they operte from the optiml cost vs. performnce point. Further, s expected, these pproches re limited in terms of explored policies, which ccount for only smll portion of the stte spce. Note tht strwmn of simply witing for the spot instnces t the lowest price nd purchsing in bulk risks delyed job completion, insufficient resources (due to limit on spot instnces nd job prllelism constrints), or both. Therefore, given fluctuting nd unpredictble spot prices (Fig. ), users do not hve n effective wy of reinforcing the better performing policies. In this pper, we propose n online lerning pproch for utomted resource lloction for btch pplictions, which blnces the fundmentl trdeoff between cloud computing costs nd job due dtes. Intuitively, given set of jobs nd resource lloction policies, our lgorithm continuously djusts per-policy weights bsed on their performnce on job executions, in order to reinforce best performing policies. In ddition, the lerning method tkes into ccount prior history of spot prices nd chrcteristics of input jobs to dpt policy weights. Finlly, to prevent overfitting to only smll set of policies, our pproch llows defining brod rnge of prmeterized policy combintions (bsed on discussion with users nd cloud opertors) such s () rent on-demnd, spot instnces, or both; (b) vry spot bid prices in predefined rnge; nd (c) choose bid vlue bsed on pst spot mrket prices. Note tht these policy combintions re illustrtive, not comprehensive, in the sense tht dditionl prmeterized fmilies of policies cn be defined nd integrted into our frmework. Likewise, our lerning pproch cn incorporte other resource lloction prmeters being provided by cloud pltforms e.g., Virtul Mchine (VM) instnce type, dtcenter/region. Our proposed lgorithm is bsed on mchine lerning pproches (e.g., [8]), which im to lern good performing policies given set of cndidte policies. While these schemes provide performnce gurntees with respect to the optiml policy in hindsight, they re not pplicble s-is to our problem. In prticulr, they require pyoff vlue per execution step to mesure how well policy is performing nd to tune the lerning process. However, in btch computing, the performnce of policy cn only be clculted fter the job hs completed. Thus, these schemes do not explicitly ddress the issue of dely in getting feedbck on how well prticulr policy performed in executing jobs. Our online lerning lgorithm hndles bounded dely nd provides forml gurntees on its performnce which scles with the mount of dely nd the totl number of jobs to be processed. We evlute our lgorithms vi simultions on job trce from dtcenter cluster nd Amzon EC2 spot mrket prices. We show tht our pproch outperforms greedy resource lloction heuristics in terms of totl pyoff in prticulr, the verge regret of our pproch (compred to the best policy in hindsight) vnishes to zero with time. Further, it provides fst convergence while only using smll mount of trining dt. Finlly, our lgorithm enbles interpreting the lloction strtegy of the output policies, llowing users to pply them directly in prctice. 2 Bckground nd System Model In this section we first provide bckground on the online lerning frmework nd then describe the problem setup nd the prmeterized set of policies for resource 2

3 lloction. Regret-minimizing online lerning. Our online lerning frmework is bsed on the substntil body of work on lerning lgorithms tht mke repeted decisions while iming to minimize regret. The regret of n lgorithm is defined s the difference between the cumultive performnce of the sequence of its decisions nd the cumultive performnce of the best fixed decision in hindsight. We present only brief overview of these lgorithms due to spce constrints. In generl, n online decision problem cn be formulted s repeted gme between lerner (or decision mker) nd the environment. The gme proceeds in rounds. In ech round j, the environment (possibly controlled by n dversry) ssigns rewrd f j () to ech possible ction, which is not reveled beforehnd to the lerner. The lerner then chooses one of the ctions j, possibly in rndomized mnner. The verge pyoff of n ction is the verge of rewrds J J j= f j() over the time horizon J, nd the lerner s verge pyoff is the verge received rewrd J J j= f j( j ) over the time horizon. The verge regret of the lerner is defined s mx J J j= f j() J J i= f j( j ), nmely the difference between the verge pyoff of the best ction nd the lerner s sequence of ctions. The gol of the lerner is to minimize the verge regret, nd pproch the verge gin of the best ction. Severl lerning lgorithms hve been proposed tht pproch zero verge regret s the time horizon J pproches infinity, even ginst fully dptive dversry [8]. Our problem of llocting between on-demnd nd spot instnces cn be cst s problem of repeted decision mking in which the resource lloction lgorithm must decide in repeted fshion over which policies to use for meeting job due dtes while minimizing job execution costs. However, our problem lso differs from stndrd online lerning, in tht the pyoff of ech policy is not reveled immeditely fter it is chosen, but only fter some dely (due to the time it tkes to process job). This requires us to develop modified online lgorithm nd nlysis. Problem Setup. Our problem setup focuses on single enterprise whose btch jobs rrive over time. Jobs my rrive t ny point in time, however job rrivl is monitored every fixed time intervl of L minutes e.g., L = 5. For simplicity, we ssume tht ech hour is evenly divided into fixed number of such time intervls (nmely, 6/L). We refer to this fixed time intervl s time slot (or slot); the time slots re indexed by t =,2,... Jobs. Ech job j is chrcterized by five prmeters: (i) Arrivl slot A j : If job j rrives t time [L(t ),Lt ), then A j = t. (ii) Due dte d j N (mesured in hours): If the job is not completed fter d j time units since its rrivl A j, it becomes invlid nd further execution yields zero vlue. (iii) Job size z j (mesured in CPU instnce hours to be executed): Note tht for mny btch jobs such s prmeter sweep pplictions nd softwre testing, z j is known in dvnce. Otherwise, smll bounded over-estimte of z j suffices. (iv) Prllelism constrint c j : The mximl degree of prllelism i.e., the upper bound on number of instnces tht cn be simultneously ssigned to the job. (v) Vlue function: V j : N R +, which is monotoniclly non-incresing function with V j (τ)= τ > d j. Thus, job j is described by the tuple{a j,d j,z j,c j,v j }. The job j is sid to be ctive t time slot τ if less thn d j hours hve pssed since its rrivl A j, nd the totl instnce hours ssigned so fr re less thn z j. Alloction updtes. Ech job j is llocted computing instnces during its execution. Given the existing cloud pricing model of chrging bsed on hourly boundries, the instnce lloction of ech ctive job is updted every hour. The i-th lloction updte for job j is formlly defined s triplet of the form (o i j,si j,bi j ). oi j denotes the number of ssigned on-demnd instnces; s i j denotes the number of ssigned spot instnces nd b i j denotes their bid vlues. The prllelism constrint trnsltes to o i j + si j c j. Note tht NOP decision i.e., llocting zero resources to job, is hndled by setting o i j nd si j to zero. Spot instnces. The spot instnces ssigned to job operte until the spot mrket price exceeds the bid price. However, s Figure shows, the spot prices my chnge unpredictbly implying tht spot instnces cn get terminted t ny time. Formlly, consider some job j; let us normlize the hour intervl to the closed intervl [,]. Let y i j [,] be the point in time in which the spot price exceeded the i-th bid for job j; formlly, y i j = inf y [,]{p s (y)>b i j }, where p s( ) is the spot price, nd y i j if the spot price does not exceed the bid. Then the cost of utilizing spot instnces for job j for its i-th lloction is given by s i j ˆpi j, where ˆpi j = yi j p j(y)dy, nd the totl mount of work crried out for this job by spot instnces is s i j yi j (with the exception of the time slot in which the job is completed, for which the totl mount of work is smller). Note tht under spot pricing, the instnce is chrged for the full hour even if the job finishes erlier. However, if the instnce is terminted due to mrket price exceeding the bid, the user is not chrged for the lst prtil hour of execution. Further, we ssume tht the cloud pltform provides dvnce notifiction of the instnce revoction in this scenrio. Finlly, s in [23] studies dynmic checkpointing strtegies for scenrios where customers might incur substntil overheds due to out-of-bid sitution. For simplicity, we do not model such scenrios in this pper. However, we note tht the techniques developed in [23] re complementry, nd cn be pplied in conjunction to our online lerning 3

4 Amzon EC2, our model llows spot instnces to be persistent, in the sense tht the user s bid will keep being submitted fter ech instnce termintion, until the job gets completed or the user cncels it. On-Demnd instnces. The price for n on-demnd instnce is fixed nd is denoted by p (per-unit per timeintervl). As bove, the instnce hour is pid entirely, even if the job finishes before the end of the hourly intervl. Utility. The utility for user is defined s the difference between the overll vlue obtined from executing ll its jobs nd the totl costs pid for their execution. Formlly, let T j be the number of hours for which job j is executed (ctul durtion is rounded up to the next hour). Note tht if the job did not complete by its lifetime d j, we set T j = d j + nd lloction T j j =(,,). The utility for job j is given by: U j ( j,...,t j j )= V j (T j ) T j i= { ˆp i j s i j + p oi j} () The overll user utility is then simply the sum of job utilities: U() = j U j ( j,...,t j j ). The objective of our online lerning lgorithm is to mximize the totl user utility. For simplicity, we restrict ttention to dedline vlue functions, which re vlue functions of the form V j (i)= v j, for ll i [,...,d j ] nd V j (i) = otherwise, i.e., completing job j by its due dte hs fixed positive vlue [2]. Note tht our lerning pproch cn be esily extended to hndle generl vlue functions. Remrk. We mke n implicit ssumption tht user immeditely gets the mount of instnces it requests if the price is right (i.e., if it pys the required price for on-demnd instnces, or if its bid is higher thn mrket price for spot instnces. In prctice, however, user might exhibit delys in getting ll the required instnces, especilly if it requires lrge mount of simultneous instnces. While we could semlessly incorporte such delys into our model nd solution frmework, we ignore this spect here in order to keep the exposition simple. Resource Alloction Policies. Our lgorithmic frmework llows defining brod rnge of policies for llocting resources to jobs nd the objective of our online lerning lgorithm is to pproch the performnce of the best policy in hindsight. We describe the prmeterized set of policies in this section, nd present the lerning lgorithm to dpt these policies, in detil in Section 3. For ech ctive job, policy tkes s input the job specifiction nd (possibly) history of spot prices, nd outputs n lloction. Formlly, policy π is mpping of the form π : J R + R + R n + A, which for every ctive job j t time τ tkes s input: frmework. (i) the job specifiction of j: {A j,d j,z j,c j,v j } (ii) the remining work for the job z τ j (iii) the totl execution cost C j incurred for j up to time τ (nmely, Cj τ = τ t =A j s t j ˆpt j + p ot j, nd (iv) history sequence p s ( ) of pst spot prices. In return, the policy outputs n lloction. As expected, the set of possible policies define n explosively lrge stte spce. In prticulr, we must crefully hndle ll possible instnce types (spot, on-demnd, both, or NOP), different spot bid prices, nd their exponentil number of combintions in ll possible job execution sttes. Of course, no pproch cn do n exhustive serch of the policy stte spce in n efficient mnner. Therefore, our frmework follows best-effort pproch to tckle this problem by exploring s mny policies s possible in the prcticl operting rnge e.g., spot bid price close to zero hs very low probbility of being ccepted; similrly, bidding is futile when the spot mrket price is bove the on-demnd price. We ddress this issue in detil in Section 3. An elegnt wy to generte this prcticl set of policies is to describe them by smll number of control prmeters so tht ny prticulr choice of prmeters defines single policy. We consider two bsic fmilies of prmeterized policies, which represent different wys to incorporte the trdeoff between on-demnd instnces nd spot-instnces: () Dedline-Centric. This fmily of policies is prmeterized by dedline threshold M. If the job s dedline is more thn M time units wy, the job ttempts llocting only spot-instnces. Otherwise (i.e., dedline is getting closer), it uses only on-demnd instnces. Further, it rejects jobs if they become nonprofitble (i.e., cost incurred exceeds utility vlue) or if it cnnot finish on time (since dedline vlue function V j will become zero). (2) Rte-Centric. This fmily of policies is prmeterized by fixed rte σ of llocting on-demnd instnces per round. In ech round, the policy ttempts to ssign c j instnces to job j s follows: it requests σ c j instnces on-demnd (for simplicity, we ignore rounding issues) t price p. It lso requests ( σ) c j spot instnces, using bid price strtegy which will be described shortly. The policy monitors the mount of job processed so fr, nd if there is risk of not completing the job by its due dte, it switches to ondemnd only. As bove, it rejects jobs if they become non-profitble or if it cnnot finish on time. A pseudocode implementing this intuition is presented in Algorithm. The pseudo-code for the dedline-centric fmily is similr nd thus omitted for brevity. We next describe two different methods to set the bids for the spot instnces. Ech of the policies bove cn 4

5 use ech of the methods described below: (i) Fixed bid. A fixed bid vlue b is used throughout. (ii) Vrible bid. The bid price is chosen dptively bsed on pst spot mrket prices (which mkes sense s long s the prices re not too fluctuting nd unpredictble). The vrible bid method is prmeterized by weight γ nd sfety prmeter ε to hndle smll price vritions. At ech round, the bid price for spot instnces is set s the weighted verge of pst spot prices (where the effective horizon is determined by the weight γ) plus ε. For brevity, we shll often use the terms fixed-bid policies or vrible-bid policies, to indicte tht policy (either dedline-centric or rte-centric) uses the fixed-bid method or the vrible-bid method, respectively. Observe tht vrible bid policies represent one simple lterntive for exploiting the knowledge bout pst spot prices. The design of more sophisticted policies tht utilize price history, such s policies tht incorporte potentil sesonlity vrition, is left s n interesting direction for future work. ALGORITHM : Rtio-centric Policy Prmeters (with Fixed-Bid method): On-demnd rte σ [,]; bid b R + Prmeters (with Vrible-Bid method): On-demnd rte σ [,]; weight γ [,]; sfety prmeter ε R + Input: Job prmeters {d j,z j,c j,v j } If c j d j < z j or p σ z j > v j, drop job //Job too lrge or expensive to hndle profitbly for Time slot t in which the job is ctive do If job is done, return Let m be the number of remining time slots till job dedline (including the current one) Let r be the remining job size Let q be the cost incurred so fr in treting the job // Check if more on-demnd instnces needed to ensure timely job completion if(σ + m )min{r,c j }<r then // Check if running job just with on-demnd is still worthwhile if p r+ q<v j then Request min{r,c j } on-demnd instnces else Drop job end if else Request σ min{r,c j } on-demnd instnces Request ( σ) min{r,c j } spot instnces t price: Fixed-Bid method: Bid Price b Vrible-Bid method: Z Z = y γτ y dy is normliztion constnt end if end for y p s(y)γ τ y dy+ε, where Note tht these policy sets include, s specil cses, some simple heuristics tht re used in prctice [3]; for exmple, heuristics tht plce fixed bid or choose bid t rndom ccording to some distribution (both with the option of switching to on-demnd instnces t some point). These heuristics (nd similr others) cn be implemented by fixing the weights given to the different policies (e.g., to implement policy which selects the bid uniformly t rndom, set equl weights for policies tht use the fixed-bid method nd zero weights for the policies tht use the vrible-bid method). The lerning pproch which we describe below is nturlly more flexible nd powerful, s it dpts the weights of the different policies bsed on performnce. More generlly, we emphsize tht our frmework cn certinly include dditionl fmilies of prmeterized policies, while our focus on the bove two fmilies is for simplicity nd proof of concept. In ddition, our lerning pproch cn incorporte other prmeters for resource lloction tht re provided by cloud pltforms e.g., VM instnce type, dtcenter/region. At the sme time, some of these prmeters my be set priori bsed on user constrints e.g., n extr-lrge instnce my be fixed to ccommodte lrge working sets of n ppliction in memory, nd dtcenter my be fixed due to ppliction dt stored in tht loction. 3 The Online Lerning Algorithm In this section we first give n overview of the lgorithm, nd then describe how the lgorithm is derived nd provide theoreticl gurntees on its performnce. Algorithm Overview. The lerning lgorithm pseudocode is presented s Algorithm 2. The lgorithm works by mintining distribution over the set of lloction policies (described in Section 2). When job rrives, it picks policy t rndom ccording to tht distribution, nd uses tht policy to hndle the job. After the job finishes execution, the performnce of ech policy on tht job is evluted, nd its probbility weight is modified in ccordnce with its performnce. The updte is such tht high-performing policies (s mesured by f j (π)) re ssigned reltively higher weight thn low-performing policies. The multiplictive form of the updte ensures strong theoreticl gurntees (s shown lter) nd prcticl performnce. The rte of modifiction is controlled by step-size prmeter η j, which slowly decys throughout the lgorithm s run. Our lgorithm lso uses prmeter d defined s n upper bound on the number of jobs tht rrive during ny single job s execution. Intuitively, d is mesure of the dely incurred between choosing which policy to tret given job, till we cn evlute its performnce on tht job. Thus, d is closely relted to job lifetimes d j defined in Section 2. Note tht while d j is mesured in time units (e.g., hours), d mesures the number of new jobs rriv- 5

6 ing during given job s execution. We gin emphsize tht this dely is wht sets our setting prt from stndrd online lerning, where the feedbck on ech policy s performnce is immedite, nd necessittes modified lgorithm nd nlysis. The running time of the lgorithm scles linerly with the number of policies nd thus our frmework cn del with (polynomilly) lrge sets of policies. It should be mentioned tht there exist online lerning techniques which cn efficiently hndle exponentilly lrge policy sets by tking the set structure into ccount (e.g. [8], Chpter 5). Incorporting these techniques here remins n interesting direction for future work. We ssume, without loss of generlity, tht the pyoff for ech job is bounded in the rnge [,]. If this does not hold, then one cn simply feed the lgorithm with normlized vlues of the pyoffs f i ( j). In prctice, it is enough for the pyoffs to be on the order of ± on verge for the lgorithm to work well, s shown in our experiments in Section 4. ALGORITHM 2: Online Lerning Algorithm Input: Set of n policies π prmeterized by{,..., n}, upper bound d on jobs lifetime Initilize w =(/n,/n,...,/n) for j =,...,J do Receive job j Pick policy π with probbility w j,π, nd pply to job j if j d then w j+ := w j else η j := 2log(n)/d( j d) for π =,...,n do Compute f j (π) to be the utility for job j d, ssuming we used policy π w j+,π := w j,π exp ( η j f j (π) ) end for for π =,...,n do w j+,π := w j+,π / n r= w j+,r end for end if end for Derivtion of the Algorithm.Next we provide forml derivtion of the lgorithm s well s theoreticl gurntees. The setting of our lerning frmework cn be bstrcted s follows: we divide time into rounds such tht round j strts when job j rrives. At ech such round, we mke some choice on how to del with the rriving job. The choice is mde by picking policy π j from fixed set of n policies, which will be prmeterized by {,..., n}. However, initilly, we do not know the utility of our policy choice s future spot prices re unknown. We cn eventully compute this utility in retrospect, but only fter d rounds hve elpsed nd the relevnt spot prices re reveled. Let f j (π j d ) denote the utility function of the policy choice π j d mde in round j d. Note tht ccording to our model, this function cn be evluted given the spot prices till round j. Thus, f j(π j d ) is our totl pyoff from ll the jobs we hndled. We mesure the lgorithm s performnce in terms of verge regret with respect to ny fixed choice in hindsight, i.e., mx π J f j (π) J f j (π j d ). Generlly speking, online lerning lgorithms ttempt to minimize this regret, nd ensure tht s J increses the verge regret converges to, hence the lgorithm s performnce converges to tht of the single best policy in hindsight. A crucil dvntge of online lerning is tht this cn be ttined without ny sttisticl ssumptions on the job chrcteristics or the price fluctutions. When d =, this problem reduces to the stndrd setting of online lerning, where we immeditely obtin feedbck on the chosen policy s performnce. However, s discussed in Section, this setting does not pply here becuse the function f j does not depend on the lerner s current policy choice π j, but rther on its choice t n erlier round, π j d. Hence, there is dely between the lgorithm s decision nd feedbck on the decision s outcome. Our lgorithm is bsed on the following rndomized pproch. The lerner first picks n n-dimensionl distribution vector w = (/n,...,/n), whose entries re indexed by the policies π. At every round j, the lerner chooses policy π j {,...,n} with probbility w j,π j. If j d, the lerner lets w j+ = w j. Otherwise it updtes the distribution ccording to w j+,π = w j,π exp(η j f j (π)) n π= w j,i exp(η j f j (i)), where η j is step-size prmeter. Agin, this form of updte puts more weight to higher-performing policies, s mesured by f j (π). Theoreticl Gurntees. The following result quntifies the regret of the lgorithm, s well s the (theoreticlly optiml) choice of the step-size prmeter η j. This theorem shows tht the verge regret of the lgorithm scles with the jobs lifetime bound d, nd decys to zero with the number of jobs J. Specificlly, s J increses, the performnce of our lgorithm converges to tht of the best-performing policy in hindsight. This behvior is to be expected from lerning lgorithm, nd crucilly, occurs without ny sttisticl ssumptions on the jobs chrcteristics or the price fluctutions. The performnce lso depends - but very wekly - on the size 6

7 n of our set of policies. From mchine lerning perspective, the result shows tht the multiplictive-updte mechnism tht we build upon cn indeed be dpted to delyed feedbck setting, by dpting the step-size to the dely bound, thus retining its simplicity nd sclbility. Theorem. Suppose (without loss of generlity) tht f j for ll j =,...,J is bounded in [,]. For the lgorithm described bove, suppose we pick η j = log(n)/2d( j d). Then for ny δ (,), it holds with probbility t lest δ over the lgorithm s rndomness tht mx π J J f j (π) j= J J 2d log(n/δ) f j (π j d ) 9. j= J To prove Theorem, we will use the following two lemms: Lemm. Consider the sequence of distribution vectors w +d,...,w defined by w +d =(/n,...,/n) nd π {,...,n}, w j+,π = w j,π exp(η j f j (π)) n π= w j,π exp(η j f j (π)), where η j = log(n)/( j d) for some (,2]. Then it holds tht n J log(n) mx f j (π) w j,π f j (π) 4. π {,...,n} π= Proof. For ny j =,...,J, let g j (π)= f j (π). Then the updte step specified in the lemm cn be equivlently written s π {,...,n}, w j+,π = w j,π exp( η j g j (π)) n π= w j,π exp( η j g j (π)). The initiliztion of w +d nd the updte step specified bove is identicl to the exponentilly-weighted verge forecster of [8], lso known s the Hedge lgorithm [9]. Using the proof of Theorem 2.3 from [8] (see pg. 2), we hve tht for ny prmeter, if we pick η j = log(n)/ j, then n π= w j,π g j (π) min π {,...,n} g j (π) T log(n) (J+ )log(n) log(n) Since (J+ )log(n)/ J log(n)/+ log(n)/, the expression bove cn be upper bounded by T log(n) J log(n) log(n) T log(n) J log(n) Since (,2], this is t most 4 J log(n)/, so we get n π= w j,π g j (π) min π {,...,n} g j (π) 4 J log(n)/. The result stted in the lemm follows by re-substituting g j (π) = f j (π), nd using the fct tht π w j,π =. Lemm 2. Let,..., n [,] nd η > be fixed. For ny distribution vector w in the n-simplex, if we define w to be the new distribution vector π {,...,n}, w π = w π exp( η π ) n r= w r exp( η π ), Then w w 4min{,η}. Proof. If η > /2, the bound is trivil, since for ny two distribution vectors w,w, it holds tht w w 2. Thus, let us ssume tht η /2. We hve w w = = n π= w π n w π w n π = w w π= π= ( ) exp( η π ) n r= w. r exp( η π ) Since w =, we cn pply Hölder s inequlity, nd upper bound the bove by mx π exp( η π ) n r= w r exp( η π ). (2) Using the inequlity x exp( x) for ll x, we hve tht exp( η π ) η π n r= w r exp( η π ), η π so we cn upper bound Eq. (2) by mx π η π = mx η π 2η mx π η π 2η, π π using our ssumption tht π for ll π, nd tht η π η /2. Using these lemms, we re now redy to prove Theorem : Proof. Our gol is to upper bound f j (π) f j (π j d ) (3) for ny fixed π. We note tht π,...,π J re independent rndom vribles, since their rndomness stems only 7

8 from the independent smpling of ech π j from ech w j. Thus, the regret cn be seen s function over J independent rndom vribles. Moreover, for ny choice of π,...,π J, if we replce π j by ny other π j, the regret expression will chnge by t most. Invoking Mc- Dirmid s inequlity [8], which cptures how close to their expecttion re such stble rndom functions, it follows tht Eq. (3) is t most E [ ] f j (π) f j (π j d ) + log(/δ)j (4) with probbility t lest δ. We now turn to bound the expecttion in Eq. (4). We hve [ ] E f j (π) f j (π j d ) = f j (π) E π j d w j d [ f j (π j d )]. (5) On the other hnd, by Lemm, we hve tht for ny fixed π, J log(n) f j (π) E π w j [ f j (π)] 4. (6) where we ssume η j = log(n)/ j. Thus, the min component of the proof is to upper bound ( ) E π w j [ f j (π)] E π j d w j d [ f j (π j d )] = n π= (w j d,π w j,π ) f j (π). (7) By Hölder s inequlity nd the tringle inequlity, this is t most w j d w j d i= w j i+ w j i which by Lemm 2, is t most 4 d i= min{,η j i}, where we tke η j i = if j i < +d (this refers to the distribution vectors w,...,w d, which don t chnge, nd hence the norm of their difference is zero). This in turn cn be bounded by 4d 4min{,η j } = 4d { min, } log(n). j d Combining this upper bound on Eq. (7) with Eq. (6), we cn upper bound Eq. (5) by { J log(n) 4 + 4d min, J log(n) 4 + 4d J log(n) 4 + 8d J log(n). log(n) j d } log(n) j d Picking = /2d, we get tht Eq. (5) is t most 8 2d log(n)j. Combining this with Eq. (4), nd noting it is probbilistic upper bound on Eq. (3), we get tht f j (π) f j (π j d ) 8 2d log(n)j+ log(/δ)j 8 2d log(n/δ)j+ log(n/δ)j (8 2d+ ) log(n/δ)j 9 2d log(n/δ)j. Dividing by J, nd noting tht the inequlity holds simultneously with respect to ny fixed π, the theorem follows. 4 Evlution In this section we evlute the performnce of our lerning lgorithm vi simultions on synthetic job dt s well s rel dtset from lrge btch computing cluster. The benefits of using synthetic dtsets is tht it llows the flexibility to evlute our pproch under wide rnge of worklods. Before continuing, we would like to emphsize tht the contribution of our pper is beyond the design of prticulr sets of policies - there re mny other policies which cn potentilly be designed for our tsk. Wht we provide is met-lgorithm which cn work on ny possible policy set, nd in our experiments we intend to exemplify this on plusible policy sets which cn be esily understood nd interpreted. Throughout this section, the prmeters of the different policies re set such tht the entire rnge of plusible policies is covered (with limittion of discretiztion). For exmple, the spot-price time series in Section 4.2 rnges between.2 nd.68 (see Fig. 6()). Accordingly, we llow the fixed bids b to rnge between.5 nd.7 with 5 cents resolution. Higher thn.7 bids perform exctly s the.7 bid, hence cn be excluded; bids of. or lower will lwys be rejected, hence cn be excluded s well. 4. Simultions on Synthetic Dt Setup: For ll the experiments on synthetic dt, we use the following setup. Job rrivls re generted ccord- 8

9 ing to Poisson distribution with men minutes; job size z j (in instnce-hours) is chosen uniformly nd independently t rndom up to mximum size of, nd the prllelism constrint c j ws fixed t 2 instncehours. Job vlues scle with the job size nd the instnce prices. More precisely, we generte the vlue s x p z j, where x is uniform rndom vrible in[.5, 2], nd p is the on-demnd price. Similrly, job dedlines lso scle with size nd re chosen to be x z j /c j, where x is uniformly rndom on [,2]. As discussed in Section 3, the on-demnd nd spot prices re normlized (divided by ) to ensure tht the verge pyoff per job is on the order of ±. The on-demnd price is.25 per hour, while spot prices re updted every 5 minutes (the wy we generte spot prices vries cross experiments). Resource lloction policies. We generte prmeterized set of policies. Specificlly, we use 24 dedlinecentric policies, nd sme number of rte-centric policies. These policy set uses six vlues for M (M {,...,5}) nd σ (σ {,.2,.4,.6,.8,}), respectively. For either policy set, we hve policies tht use the fixed-bid method (b {.,.5,.2,.25}), nd policies tht use the vrible-bid method (weight γ {,.2,.4,.6,.8}, nd sfety prmeter ε {,.2,.4,.6,.8,.}). Simultion results: Experiment. In the first experiment, we compre the totl pyoff cross k jobs of ll the 48 policies to our lgorithm. Spot prices re chosen independently nd rndomly s.5 +.5x, where x is stndrd Gussin rndom vrible (negtive vlues were clipped to ). The results presented below pertin to single run of the lgorithm, s they were virtully identicl cross independent runs. Figure 2 shows the totl pyoff for the 48 policies for this dtset. The first 24 policies re rte-centric policies, while the remining 24 re dedline-centric policies. The performnce of our lgorithm is mrked using dshed line. As cn be seen, our lgorithm performs close to the best policies in hindsight. Further, it is interesting to note tht we hve both dedline-centric nd rte-centric policies mong the best policies, indicting tht one needs to consider both sets s cndidte policies. We perform three dditionl experiment with similr setup to the bove, in order to obtin insights on the properties nd inner-working of the lgorithm. To be ble to dive deeper into the nlysis, we use only the 24 rtecentric policies. The only element tht we modify cross experiments is the sttisticl properties of the spot-prices sequence. Experiment 2. Spot prices re generted s bove, except tht we use. s their men (opposed to.2 bove). After executing jobs, our lgorithm performs close to tht of the best policy s it ssigns probbility close Totl Pyoff (x e5) Figure 2: Totl pyoff for processing 2k jobs cross ech of the 48 resource lloction policies (while lgorithm s pyoff is shown s dshed blck line). The first 24 policies re rte-centric, nd the lst 24 policies re dedline-centric Distribution fter 5 Jobs Distribution fter 25 Jobs Distribution fter Jobs Distribution fter 5 Jobs Figure 3: Evlution under sttionry spot-price distribution (men spot price of.): ssigned per policy fter executing 5,, 25 nd 5 jobs. to for tht policy, while outperforming 99 out of totl 24 policies. Further, its verge regret is only.3 s opposed to 7.5 on verge cross ll policies. Note tht the upper bound on the dely in this experiment is d = 66, i.e., up to 66 jobs re being processed while single job finishes execution. This shows tht our pproch cn hndle significnt dely in getting feedbck, while still performing close to the best policy. In this experiment, the best policy in hindsight uses fixed-bid of.25. This cn be explined by considering the prmeters of our simultion: since the on-demnd price is.25 nd the spot price is lwys reltively lower, bid of.25 lwys yields lloction of spot instnces for the entire hour. This result lso highlights the esy interprettion of the resource lloction strtegy of the best policy. Figure 3 shows the probbility ssignment for ech policy over time by our lgorithm fter executing 5,, 25 nd 5 jobs. We observe tht s the number of processed jobs increse, our lgorithm provides performnce close to the best policy in hindsight. Experiment 3. In the next experiment, the spot prices is set s bove for the first % of the jobs, nd then the 9

10 Totl Pyoff (x,) Totl Pyoff (x,) Figure 4: Evlution under non-sttionry distribution (men spot price of.2): () Totl pyoff for executing k jobs cross ech of the 24 policies (while lgorithm s pyoff is shown s dshed blck line) nd (b) the finl probbility ssigned per policy by our lerning lgorithm. Figure 5: Evlution under highly dynmic distribution (hourly spot prices lternte between.3 nd zero): () Totl pyoff for processing k jobs cross ech of the 24 policies (lgorithm s pyoff is shown s dshed blck line), nd (b) the finl probbility ssigned per policy by our lerning lgorithm men is incresed to.2 (rther thn.) during the execution of the lst 9% jobs. This setup corresponds to non-sttionry distribution: lerning lgorithm which simply ttempts to find the best policy t the beginning nd stick to it, will be severely penlized when the dynmics of spot prices chnge. Figure 4 shows the evlution results. We observe tht our online lgorithm is ble to dpt to chnging dynmics nd converges to probbility weight distribution different from the previous setting; Overll, our lgorithm ttins n verge regret of only.5, s opposed to 4.8 on verge cross 24 bseline policies. Note tht in this setting, the best policies re those which rely purely on on-demnd instnces insted of spot instnces. This is expected becuse the spot prices tend to be only slightly lower thn the on-demnd price, nd their dynmic voltility mke them unttrctive in comprison. This result demonstrtes tht there re indeed scenrios where the dilemm between choosing ondemnd vs. spot instnces is importnt nd cn significntly impct performnce, nd tht no single instnce type is lwys suitble. Experiment 4. This time we set the spot price to lternte between.3 for one hour nd then zero in the next. This vrition is fvorble for vrible-bid policies with smll γ, which use smll history of spot prices to determine their next bid. Such policies quickly dpt when the spot price drops. In contrst, fixed-bid policies nd vriblebid policies with lrge γ suffer, s their bid price is not sufficiently dptive. Figure 5 shows the results. We find tht the group of highest-pyoff policies re those for which γ = i.e., they use the lst spot price to choose bid for the current round, nd thus quickly dpt to chnging spot prices. Further, our lgorithm quickly detects nd dpts to the best policies in this setting. The verge regret obtined by our lgorithm is.8 compred to 4.5 on verge for our bseline policies. Moreover, the lgorithm s overll performnce is better thn 92 out of 24 policies. spot price Time (Hr) Totl Pyoff (x e9) Figure 6: Evlution on rel dtset: () Amzon EC2 spot pricing dt (subset of dt from Figure ) for Linux instnces of type lrge. The fixed on-demnd price is.34; (b) Totl pyoff for processing 2k jobs cross ech of the 54 resource lloction policies (while lgorithm s pyoff is shown s dshed blck line) 4.2 Evlution on Rel Dtsets Setup: Worklod dt. We use job trces from lrge btch computing cluster for two dys consisting of bout 6 MpReduce jobs. Ech MpReduce job comprises multiple phses of execution where the next phse cn strt only fter ll tsks in the previous phse hve completed. The trce includes the runtime of the job in server CPU hours (totcpuhours), the totl number of servers llocted to it (totservers) nd the mximum number of servers llocted to job per phse (mxserversperphse). Since our job model differs from the MpReduce model in terms of phse dependency, we construct the prllelism constrint from the trce s follows: since the verge running time of server is totcpuhours totservers, we set the prllelism bound c j for ech job to be c j = mxserversperphse totcpuhours totservers. Note tht this bound is in terms of CPU hours s required. Since the dedline vlues per job re not specified, we use the job completion time s its dedline. For ssigning vlues per job, we generte them using the sme pproch s for synthetic dtsets. Specificlly, we ssign rndom vlue for ech job j equl to its totl size (in CPU hours) times the on-demnd price times B = (α + N j ) where α = 5 nd N j [,] is drwn uniformly t rndom. The job trce is replicted to generte 2k jobs. Spot Prices. We use subset of the historicl spot

11 price from Amzon EC2 s shown in Figure for lrge Linux instnces. Figure 6() shows the selected smple of spot price history showing significnt price vrition over time. Intuitively, we expect tht overll tht policies tht use lrge rtio of spot instnces will perform better since on verge, the spot price is bout hlf of the on-demnd price. Resource Alloction Prices. We generted totl of 54 policies, hlf rte-centric nd hlf dedline-centric. In ech hlf, the first 72 re fixed-bid policies (i.e. policies tht use the fixed-bid method) in incresing order of (on-demnd rte, bid price). The remining 8 vrible-bid policies re in incresing order of (ondemnd rte, weight, sfety prmeter). The possible vlues for the different prmeters re s described for the synthetic dt experiments, with the exception tht we llow more options for the fixed bid price, b {.5,.2,.25,.3,.35,.4,.45,.5,.55,.6,.65,.7}. Evluting our online lgorithm on the rel trce poses severl new chllenges compred to the synthetic dtsets in Section 4.. First, jobs sizes nd hence their vlues re highly vrible, to the effect tht the difference in size between smll nd lrge jobs cn be of six orders of mgnitude. Second, spot prices cn exhibit high vribility, or lterntively be lmost stble towrds the end s exemplified in Figure 6(). Simultion results: Figure 6(b) shows the results for typicl run of this experiment. Notbly, the pyoff of our lgorithm outperforms the performnce of most of individul policies, nd obtins comprble performnce to the best individul policies (which re subset of the rte-centric policies). We repeted the experiment 2 times, nd obtined the following results: The verge regret per job for our lerning lgorithm is 27 ± 43, while the verge regret cross policies is 7654± Note tht the verge regret of our lgorithm is round 34 times better (on verge) thn the verge regret cross policies. Figure 7 shows the evolution of policy weights over time for typicl run, until converging to finl policy weights (fter hndling the entire 2 jobs). We observe tht our lgorithm evolves from preferring reltively lrge subset of both dedline-centric nd rtecentric policies (t round 5 jobs) to preferring only rte-centric policies, both fixed-bid nd vrible-bid (t round 2 jobs). Eventully, the lgorithm converges to single rte-centric policy with fixed bid. This behvior cn be explined bsed on spot pricing dt in Figure 6(): Due to initilly high vribility in spot prices, our lgorithm lterntes between fixed-bid policies nd vrible-bid policies, which try to lern from pst prices. However, since the prices show little vribility for the remining two thirds of the dt, the lgorithm progressively dpts its weight for the fixed-bid policy, which is 6 x Distribution fter 5 jobs Distribution fter 3 jobs Distribution fter jobs Distribution fter 5 jobs Figure 7: Evlution on rel dtset: The probbility ssigned per policy by our lerning lgorithm fter processing 5,, 3 nd 5 jobs. The lgorithm converges to single policy (fixed-bid rte-centric policy) mrked by n rrow. commensurte with the lmost stble pricing curve. 5 Relted literture While there exist other potentil pproches to our problem, we considered n online lerning pproch due to its lck of ny stochstic ssumptions, its online (rther thn offline) nture, its cpbility to work on rbitrry policy sets, nd its bility to dpt to delyed feedbck. The ide of pplying online lerning lgorithms for sequentil decision-mking tsks is well known ([9]), nd there re quite few ppers which study vrious engineering pplictions (e.g., [, 5,, 5]). However, these efforts do not del with the problem of delyed feedbck s it violtes the stndrd frmework of online lerning. The issue of dely hs been previously considered (see [4] nd references therein), but re either not in the context of the online techniques we re using, or propose lessprcticl solutions such s running mny multiple copies of the lgorithm in prllel. In ny cse, we re not wre of ny prior study of dely-tolernt online lerning procedures for our ppliction domin. The lunch of commercil cloud computing offerings hs motivted the systems reserch community to investigte how to exploit this mrket for efficient resource lloction nd cost reductions. Some solution concepts re borrowed from erlier works on executing jobs in multiple grids (e.g., [2] nd references therein). However, new techniques re required in the cloud computing context, which directly incorporte cost considertions nd vriety of instnce renting options. The hve been numerous works in this context deling with different provider nd customers scenrios. One brnch of

12 ppers consider the uto-scling problem, where n ppliction owner hs to decide on the right number nd type of VMs to be purchsed, nd dynmiclly dpt resources s function of chnging worklod conditions (see, e.g., [7, 6] nd references therein). We focus the reminder of our literture survey on cloud resource mngement ppers tht include spot instnces s one of the lloction options. Some ppers focus on building sttisticl models for spot prices which cn be then used to decide when to purchse EC2 spot instnces (see, e.g., [3, ]). Similrly, [24] exmines the sttisticl properties of customer worklod with the objective of helping the cloud determine how much resources to llocte for spot instnces. In the context of lrge-scle btch pplictions, [4] proposes probbilistic model for bidding in spot prices while tking into ccount job termintion probbilities. However, [4] focuses on pre-computtion of fixed (nondptive) bid, which is determined greedily bsed on existing mrket conditions; moreover, the suggested frmework does not support n utomtic selection between on-demnd nd spot instnces. [22] uses genetic lgorithm to quickly pproximte the preto-set of mkespn nd cost for bg of tsks; ech underlying resource configurtion consists of different mix of on-demnd nd spot instnces. The setting in [22] is fundmentlly different thn ours, since [22] optimizes globl mkespn objective, while we ssume tht jobs hve individul dedlines. Finlly, [2] proposes ner-optiml bidding strtegies for cloud service brokers tht utilize the spot instnce mrket to reduce the computtionl cost while mximizing the profit. Our work differs from [2] in two min spects. First, unlike [2], our online lerning frmework does not require ny distributionl ssumptions on the spot price evolution (or the job model). Second, our model my ssocite different vlue nd dedline for ech job, wheres in [2] the vlue is only function of job size, nd dedlines re not explicitly treted. 6 Conclusion In this pper we design nd evlute n online lerning lgorithm for utomted nd dptive resource lloction for executing btch jobs over cloud computing pltforms. Our bsic model cn be extended to solve other resource lloction problems in cloud domins such s renting smll vs. medium vs. lrge instnces, choosing computing regions, nd different bundling options in terms of CPU, memory, network nd storge. We expect tht the lerning frmework developed here would be useful in ddressing these extensions. An interesting direction for future reserch is incorporting reserved instnces, for long-term hndling of multiple jobs. This mkes the lgorithm stteful, in the sense tht its ctions ffect the pyoffs of policies chosen in the future. This does not ccord with our current theoreticl frmework, but my be hndled using different tools from competitive nlysis. Acknowledgements. We thnk our shepherd Alexndru Iosup nd the ICAC reviewers for the useful feedbck. References [] AGMON BEN-YEHUDA, O., BEN-YEHUDA, M., SCHUSTER, A., AND TSAFRIR, D. Deconstructing Amzon EC2 spot instnce pricing. ACM Trnsctions on Economics nd Computtion, 3 (23), 6. [2] ALIZADEH, M., GREENBERG, A., MALTZ, D., PADHYE, J., PATEL, P., PRABHAKAR, B., SEN- GUPTA, S., AND SRIDHARAN, M. Dt center TCP (DCTCP). In ACM SIGCOMM Computer Communiction Review (2), vol. 4, ACM, pp [3] AWS Cse Studies. solutions/cse-studies/. [4] ANDRZEJAK, A., KONDO, D., AND YI, S. Decision model for cloud computing under sl constrints. In MASCOTS (2). [5] ARI, I., AMER, A., GRAMACY, R., MILLER, E., BRANDT, S., AND LONG, D. Acme: Adptive cching using multiple experts. In WDAS (22). [6] AZAR, Y., BEN-AROYA, N., DEVANUR, N. R., AND JAIN, N. Cloud scheduling with setup cost. In Proceedings of the 25th ACM symposium on Prllelism in lgorithms nd rchitectures (23), ACM, pp [7] Browsermob. solutions/cse-studies/browsermob. [8] CESA-BIANCHI, N., AND LUGOSI, G. Prediction, lerning, nd gmes. Cmbridge University Press, 26. [9] FREUND, Y., AND SCHAPIRE, R. E. A decisiontheoretic generliztion of on-line lerning nd n ppliction to boosting. J. Comput. Syst. Sci. 55, (997), [] GRAMACY, R., WARMUTH, M., BRANDT, S., AND ARI, I. Adptive cching by refetching. In NIPS (22). 2