IN this work, we aim to design real-time dynamic pricing

Transcription

1 Pricig for the Optimal Coordiatio of Opportuistic Agets Ozgur Dalkilic, Atilla Eryilmaz, ad Xiaoju Li Abstract We cosider a system where a load aggregator (LA) serves a large umber of small-sized, ecoomically-drive cosumers with deferrable demad, as evisioed i smart electricity grid ad data etworks. I these systems, cosumers ca behave opportuistically by deferrig their demad i respose to the prices, to obtai ecoomic gais. However, if ot cotrolled properly, such opportuistic behavior ca be detrimetal to the system by creatig aggregate effects that lead to udesirable fluctuatios i price ad total load. To avoid the uwated effects of demad-side flexibilities ad to reap system-wide beefits from them, we propose two ovel real-time dyamic pricig algorithms. The first algorithm commuicates idividual prices to cosumers by addig small radom perturbatios to a commo price. The secod algorithm itroduces a secodary price that pealizes the chage i users cosumptio i time. The commo feature of both algorithms is creatig differetiatio amog cosumers ad thus regulatig the aggregate load. We coduct comprehesive umerical ivestigatios ad show that both the LA ad the cosumers ecoomically beefit uder the proposed pricig schemes. I. INTRODUCTION IN this work, we aim to desig real-time dyamic pricig strategies for large systems where the demads possess various types of flexibilities. Demad-side flexibilities arise i systems such as smart electricity grids ad cloud computig services. I these systems, flexibilities ca materialize i various forms icludig, but ot limited to, shiftig or deferrig service i time, givig itermittet service, ad cotrollig the service amout. Cosumers, who are iheretly self-iterested ad ecoomically-drive, will aturally wat to alter their cosumptio behavior to take advatage of these flexibilities. Such cosumer behavior iduced by demad-side flexibilities brigs both opportuities ad challeges, ad ecessitates the desig of ovel maagemet techiques. We cosider a system where a large umber of small self-iterested cosumers with flexible demad are served by a load aggregator (LA). Specifically, we use the retaillevel smart electricity grid as a example. The cosumers ca be households with smart electrical devices, or small maufacturers, whereas the LA ca be a electricity retailer. The type of demad-side flexibility that is cosidered i this paper is the ability to defer demad i time. For istace, smart air coditioers ad washig machies ca operate i this fashio. We assume that the cosumers are ot uder the direct cotrol of the LA, i.e. they idepedetly decide This work is fuded by the DTRA grat HDTRA ; ad the NSF grats: CCSS-EARS , CNS-NeTS , CMMI-SMOR , CNS-WiFiUS , CCF ad ECCS The work of A. Eryilmaz was also supported by the the QNRF Grat NPRP O. Dalkilic ad A. Eryilmaz ({dalkilic.1, eryilmaz.2}@osu.edu) are with the Electrical ad Computer Egieerig Departmet at the Ohio State Uiversity, Columbus; ad X. Li (lix@ec.purdue.edu) is with the School of Electrical ad Computer Egieerig at Purdue Uiversity, West Lafayette. o their cosumptio based o their idividual ecoomic iterests ad service requiremets. Furthermore, i cotrast to a offlie formulatio of the problem where future demad is assumed to be kow i advace, we cosider the realtime cotrol problem for the LA, where future demad ca exhibit ucertai dyamics. We further ote that the model i this paper ca be applied to various scearios such as a cloud computig ceter servig customers with computatioal tasks. From the LA s perspective, demad-side flexibilities ca be utilized to the advatage of the system operatio. For example, i a smart electricity grid [1], or i a computer etwork [2], cosumer demad ca be deferred to a later time to cut peak load ad to reduce service ad maiteace costs. O the other had, from cosumer s perspective, flexibilities ca be exploited to obtai ecoomic beefits by reducig paymets [3]. Towards this ed, this work aims to desig real-time pricig schemes to be implemeted by a LA that icetivize ecoomically-drive agets to defer their flexible demad so that system-wide beefits ca be obtaied. However, pricig-based dyamic cotrol of self-iterested users also raises challeges. Uder a time-depedet pricig scheme, cosumers will likely defer their service to the periods of time with lower price i a opportuistic maer. Ideed, works such as [4], [5] establish optimality of threshold-based cosumptio policies, that have this opportuistic flavor, uder differet flexibility ad cost structures. However, aggregate respose of a large cosumer base employig such threshold policies ca potetially lead to highly- ad abruptly-fluctuatig total load ad price (as will be demostrated i Sectio III). I most systems, this volatile behavior is udesirable, because it icreases service costs, puts stress o the etwork, ad edagers the stability of the ifrastructure [6]. Thus, i desigig ew pricig mechaisms, we aim to mitigate such effects of opportuistic behavior of flexible cosumers. Previous related work o demad-side maagemet, which address the aforemetioed opportuities ad challeges related to demad-side flexibilities, ad their fudametal differeces from this work are listed below. I [7] [9], the welfare maximizatio problem of a LA is studied uder a utility maximizatio framework at day-ahead ad real-time time-scales. However, cosumers are assumed to have strictly cocave utility fuctios, which result i smoother user behavior, ad hece do ot capture the above-metioed volatility ad fluctuatio exhibited from the opportuistic decisio makig of self-iterested users with deferrable demad. Game theoretic approaches are discussed i [10], [11] for icetivizig timeshiftig of eergy cosumptio, but the resultig mechaisms require the kowledge of all cosumers demad ad utility which is urealistic whe the system is large. Furthermore, [12] cosiders a LA s problem of reewable supply itegratio via load schedulig, ad formulate a Markov Decisio

2 problem. However, the solutio requires the precise kowledge of the probabilistic distributio of future ucertaity. Thus, a key differece of this work from the aforemetioed literature is that we cosider the opportuistic decisio-makig of self-iterested cosumers, with deferrable demad, without relyig o assumptios of strictly-cocave utility fuctios or kow probabilistic distributios. As i [4] [6], we directly capture the behavior of such users through threshold policies, which however have bee foud to lead to system-level volatility. We the explicitly address such volatility through two ew pricig mechaisms. Specifically, ote that our prelimiary ivestigatios (Sectio III) show that cosumptio decisios of flexible cosumers uder a threshold policy get sychroized whe all cosumers face a commo price sigal. Motivated by this observatio, the key idea i desigig our ew pricig algorithms is to create differetiatio either i price or i agets iteral states. Our first algorithm creates iformatio asymmetry amog users by sedig idividual prices to users obtaied by creatig small perturbatios aroud a commo price (Sectio IV). This scheme is appropriate whe there are a large umber of cosumers, ad it preserves log-term fairess amog users although each user sees slightly differet prices at each time period. O the other had, the secod algorithm itroduces heterogeeity amog users by imposig a commo secodary price for the chage i cosumptio of each user i time (Sectio V). This scheme is appropriate for both small ad large systems, ad it does ot differetiate users based o the price they see. Techically, both of the proposed algorithms solve variatios of the same cost miimizatio problem, which are obtaied by augmetig the objective of the origial problem with covex terms. Furthermore, our results (Sectio VI) covey the promiet message that itroducig differetiatio amog opportuistically behavig agets alleviates the detrimetal effects of the feedback loop betwee aggregate load ad price. I particular, uder the proposed algorithms (i) high volatility ad istability problems are alleviated; (ii) a flatter load patter, which is less costly to supply, is achieved; ad (iii) flexible cosumers obtai ecoomic beefits. II. SYSTEM MODEL AND PROBLEM FORMULATION We cosider a system where a load aggregator (LA) serves a large umber of small cosumers. I the followig, we preset a geeric real-time model ad itroduce the system participats. The, we focus o the smart electricity grid ad formulate the cotrol ad pricig problem. A. System Model The system is operated over discrete time periods, t = 0, 1,..., ad at each time period the participats make their cotrol decisios. The system comprises a LA ad a large umber of cosumers as depicted i Figure 1. The LA sets the real-time prices for its cosumers, ad esures that cosumers loads are served upo their request. The goal of the LA is to maximize its profit. O the other had, cosumers seek to satisfy their demad with the aim of makig the lowest paymet for cosumptio. There are two types of cosumers i the system. Flexible cosumers have deferrable demad, i.e. they ca delay their cosumptio. Iflexible cosumers, however, caot delay their cosumptio ad must serve their demad at the time the demad is realized. The system model described is a closed-loop feedback stochastic dyamical system. Cosumers react to the price geerated by the LA i real-time, ad the LA adjusts the price based o the total load. Next, we preset the participats ad the operatio of the system i detail, usig electricity system as a specific example. Load Aggregator (LA) Supply Cost: C ( s(t) ) s(t) Supply Price: p (t) x ( t) x (t) Load Cosumer, =1,,N Waitig Queue a (t) Demad Fig. 1. The system model depictig the participats ad their iteractios. 1) Load Aggregator (LA): The LA serves its customers by procurig electricity via purchasig from a wholesale market or a distributor. The procuremet of s watts of power icurs cost C(s) to the LA. Note that the cost fuctio C ecapsulates the paymets for purchasig electricity as well as maiteace ad capital costs [7], [8], [10], [11], [13], [14]. We assume that C : R + R + is a cotiuously differetiable ad icreasig fuctio of s. We also assume that C(s) cc > 0 for all s 0, ad hece C is strogly covex ad Ċ is ivertible. The LA iteds to coordiate its customers by settig the real-time prices at each time period, i.e. p(t) for t = 0, 1,.... The price is geerated ex-ate, meaig that the amout of cosumptio is ukow at the time the price is set. Based o the price it sets, the LA receives the paymet ω(t) p(t)s(t). Hece, the LA s goal is to maximize its profit ω(t) C(s(t)). Furthermore, the LA does ot have the kowledge of cosumer valuatios ad their cotrol strategies. 2) Cosumers: There are N flexible ad N i iflexible cosumers. At period t, cosumer geerates demad a (t). a (t) is a radom variable that is assumed to be idepedet amog cosumers ad i.i.d. over time. The average demad geeratio rate is λ, ad E [a (t)] = λ for all t. We assume that demad is bouded such that a (t) [0, A max ]. The eergy cosumptio, amely load 1, by user at period t is deoted by x (t) [0, x ]. For iflexible cosumers, x (t) = a (t) because the realized demad must be served immediately. We defie S i (t) N i =1 x (t), with mea λ S = N i =1 λ, to be the total load of iflexible users. For flexible cosumers, the amout of electric eergy cosumptio is ot ecessarily equal to the amout of realized demad; Realized demad ca be deferred ad served later as load. The waitig queue for flexible cosumer s deferrable demad at time t is q (t) ad its evolutio is give by q (t + 1) = [q (t) + a (t) x (t)] + (1) where [z] + max{z, 0}. These queues are required to be stable, otherwise the delay experieced by the demad will 1 To be precise: Demad is exterally geerated accordig to a, but ca be delayed. Load is the actual cosumptio at each time period.

3 approach ifiity. The goal of flexible cosumer is to miimize its paymet, r (t) p(t)x (t), uder the queue stability costrait. Iflexible cosumers do ot have such objective sice they do ot have cotrol o their load. B. Problem Formulatio I the paper, we use boldface letters to deote vectors, e.g. x = (x 1,..., x N ) is the N dimesioal vector of the scalar quatities x for = 1,..., N. We use {.} to deote a set of quatities whose size should be uderstood from cotext. The optimizatio problem we cosider is the LA s cost miimizatio problem: mi {x(t)}{s(t)} s.t. T 1 lim E [C (s(t))] (2) t=0 N x (t) + S i (t) = s(t), t = 0, 1,... (3) =1 1 lim T E [x (t)] λ,. (4) t=0 I problem (2), the objective is the time-averaged expected cost of electricity procuremet. Costrait (3) esures that the cosumer load is served completely, costrait (4) esures that the flexible cosumers experiece fiite delay. Istead of Problem (2), we will cosider the followig static (oe time-period) problem: mi x,s C (s) s.t. N x + λ S s (5) =1 λ x,. It ca be show that the optimum objective value of (5) is a lower boud for that of (2) due to the covexity of C. Hece, by providig a solutio to problem (5), which satisfies the costrait (4) ad matches supply to load, we ca achieve a objective value that is close to the optimum value of problem (2). Problem (5) is easy to solve ad various iterative algorithms ca be developed to achieve the optimum solutio. However, such algorithms may dictate udesirable cotrol-rules o the cosumer side that do ot alig with flexible cosumers objective of miimizig their paymets. O the other had, as we will show i Sectio III-B, allowig flexible cosumers to fully exhibit their opportuistic behavior may cause istability ad iefficiecy by geeratig abrupt chages ad fluctuatios i power cosumptio. Therefore, our goal is to desig cotrol ad real-time pricig schemes that will give flexible cosumers the freedom to opportuistically cosume electricity for their ow iterest, ad that will also achieve the miimum or closeto-miimum electricity procuremet cost. III. FLEXIBLE CONSUMER BEHAVIOR AND BENCHMARK REAL-TIME PRICING SCHEMES I the followig, we will first characterize the flexible cosumer behavior, ad the discuss its impact o the system performace. To demostrate the detrimetal effects of cosumer-side flexibility, we preset two simple ad ituitive real-time pricig schemes that will also serve as bechmarks whe assessig our ow pricig schemes performace. A. Flexible Cosumer Behavior I our model, cosumers are price-takig; At period t, each cosumer receives a price p(t) for cosumig uit amout of power, ad the decides o his load. Thus, the optimizatio problem faced by a flexible cosumer ca be formulated as mi {x (t)} s.t. T 1 lim E [p(t)x (t)] (6) 1 lim t=0 T E [x (t)] λ,. t=0 From a sigle cosumer s perspective, his idividual load decisios have egligible effect o the future prices whe the umber of users is large. Hece, we assume that p(t) is exogeous; It is idepedet of x (t) i problem (6). Uder this assumptio, the followig policy achieves the optimal value of (6) i the asymptotic regime where the desig parameter κ > 0 approaches 0: x (t) = x 1{p(t) κ q (t)}. (7) We ote that this threshold policy ad similar threshold-based policies have bee show to be asymptotically optimal whe the prices are exogeous [4], [15], [16]. The policy i (7) results i a opportuistic behavior. Users cosume electricity oly whe price is below a certai threshold, ad whe they cosume, they demad their maximum load x to take full advatage of the low price. However, this behavior, whe aggregated over a large cosumer base, will cause very high (low) load whe price is low (high). Thus, as we will see subsequetly, the resultig load patter will ot be flat ad will be costly to supply. Furthermore, supply ad price will be highly fluctuatig sice the price is adjusted i real-time by the LA as a respose to the chages i load. B. Bechmark Real-time Pricig Schemes i) Scheme I (Real-time Pricig With Zero Flexible Cosumer Peetratio): I this scheme, all cosumers are iflexible, so they do ot have the ability to defer their loads; Arrivig demad is served immediately, i.e. x (t) = a (t) for all. O the other had, the LA uses x (t) as the predictio of the load o the ext time period, ad sets the price to the total margial procuremet cost, i.e. p(t + 1) = Ċ (s(t)) subject to s(t) = x (t). This choice of price maximizes the LA s profit assumig that the load predictio is accurate. To summarize, Scheme 1 is give as follows: Scheme 1. At time t: Cosumer sets x (t) = a (t). The LA computes: p(t + 1) = Ċ (s(t)), s.t. s(t) = x (t) + S i (t)

4 We ote that Scheme 1 serves as a baselie setup, which will be useful i assessig both the advatages ad disadvatages of cosumer-side flexibility. ii) Scheme II (Gradual Real-time Price Update uder Flexible Cosumer Presece): Uder his scheme, a percetage of users have flexible demad. We assume that these users implemet the threshold policy (7). Due to (7), we expect the aggregate load to become either very large or too small, sice the cosumers use the maximum amout x m or othig based o the commo price. Thus, i order to prevet fluctuatios i price i respose to the total load, this scheme iteratively updates the price istead of settig it to the margial cost of the total load. Scheme 2, which updates the price based o the dual of problem (5), is preseted below. Scheme 2. At time t: Cosumer computes (1) ad x (t) = x 1 {p(t) κ q (t)} (8) The LA must meet the real load x (t)+s i (t). Further, it computes s(t) = Ċ (p(t)), ad updates the price: [ p(t + 1) = p(t) + κ s ( x (t) + S i (t) s(t))] + (9) Uder Scheme 2, although the price exhibits relatively small oscillatios due to the dampeig effect of κ s, the total load abruptly fluctuates as see i Figure 2. Figure 2 also depicts the same amout of total load uder Scheme 1. Note that although Scheme 1 does ot have the fluctuatio problem as Scheme 2, it does ot take advatage of the demad flexibilities either. Uder the presece of demad flexibilities, the key problem appears to be that the customers, who implemet the threshold policy (7), respod to a commo price i a sychroous maer. I the followig sectios, we will propose pricig schemes that will resolve this sychroizatio problem by itroducig differetiatio amog cosumers. Load (MW) x 10 4 Scheme 1 ad 2: Load vs. Time Scheme 1: Total Load (Iflexible) Scheme 2: Iflexible Load Scheme 2: Total Load Time (Hour) Fig. 2. Load uder Scheme 1 ad 2: Flexible cosumers receive Poisso distributed demad arrivals, ad their load costitute 5% of the total load. IV. RANDOMIZED PRICING (RP) ALGORITHM I this sectio, we propose to employ radomized pricig to overcome the deficiecies of the bechmark schemes i Sectio III-B. Radomized pricig has previously bee employed i ecoomic models for differet purposes, icludig: hidig iformatio from cosumers ad competitors (see [17] ad refereces therei); ad profit maximizatio (datig back to [18]). I this work, we propose radomized pricig (cf. Algorithm RP) for the purpose of mitigatig volatility ad istability problems from opportuistic behavior of flexible cosumers, while also guarateeig fairess i terms of opreferetial treatmet of cosumers (see Propositio 1). The uderlyig motivatio i the desig of our RP Algorithm is twofold. First, we cosider updatig the commo price icremetally so that sudde chages i load do ot directly traslate to large fluctuatios i price. Secod, i order to prevet flexible cosumers load decisios from aligig together (which creates peaks ad valleys i the aggregate load), we differetiate the price over the cosumer base. I particular, each cosumer receives a idividual price that is radomly differetiated from the commo price. The real-time radomized pricig (RP) algorithm is give below. I Algorithm RP, idividual prices are geerated by addig to the commo price i.i.d. radom oise ɛ (t), which has the CDF (Cumulative Distributio Fuctio) F ɛ. The additive perturbatios ca have a arbitrary distributio as log as F ɛ satisfies Assumptio 1, which esures that F ɛ has a iverse fuctio. Assumptio 1. F ɛ is cotiuous o its domai ad strictly icreasig from 0 to 1 o a iterval [ ɛ mi, ɛ max]. Algorithm RP Radomized Pricig Algorithm At iteratio t: Cosumer receives a idividual price p (t). The, it computes its queue as i (1) ad load as x (t) = x 1{p (t) κ q (t)} (10) The LA must meet the real load x (t) + S i (t). Further, it computes s(t) = Ċ (p(t)), ad updates the commo price: [ ( N + p(t + 1) = p(t) + α x (t) + S i (t) s(t))]. =1 The, the LA geerates idividual prices that are commuicated to each cosumer separately: p (t + 1) = p(t + 1) + ɛ (t + 1) where ɛ (t) are i.i.d. radom variables over time ad cosumers with the CDF F ɛ. Uder Algorithm RP, users pay for their cosumptio at the idividual price that is privately commuicated to them by the LA. Sice this price is geerated by addig a radom disturbace to the commo price, the reveue obtaied at each time period will be differet from the reveue aticipated by the LA. Hece, it is ot surprisig that RP does ot achieve the optimal solutio to problem (2). Istead, we will show that RP achieves the optimal solutio to a welfare maximizatio problem that is closely related to the origial problem. The basic idea is that commuicatig radomized prices to cosumers iduces a utility-fuctio based decisio at the cosumer side. To demostrate this, we first preset a cotiuous-time fluid approximatio of RP which will also be istrumetal i aalyzig its optimality ad covergece. The, we preset the aalysis of RP i discrete-time i the ext subsectio.

5 A. Cotiuous-time Fluid Approximatio Model ad Utility- Maximizatio-Based Formulatio I this sectio we derive a cotiuous-time fluid approximatio for algorithm RP [19]. The, we relate the model to a utility maximizatio problem with modified cosumer utility fuctios iduced by price radomizatio. The aggregate flexible cosumer load is the sum of N biary variables, i.e. X(t) x 1{ɛ (t) κ q (t) p(t)}. Moreover, coditioed o p(t) ad q (t), each x (t) is idepedet sice ɛ are idepedet. Applyig the Law of Large Numbers based o this assumptio, we obtai the followig expressio for the aggregate load X(t) x F ɛ (κ q (t) p(t)). (11) The above expressio is the mea behavior for the aggregate load, ad whe the umber of users is large it will well approximate the dyamics of the load. We defie u (x) x F ɛ ( x), ad write x (t) u (p(t) κ q (t)), which approximates the mea behavior of idividual users. Next, we preset a cotiuous-time approximatio to RP. Algorithm RP-C Cotiuous-time Approximatio to RP x (t) = u (p(t) κ q (t)) (12) s(t) = Ċ (p(t)) (13) { q(t)>0, or λ x (t) if λ q (t) = x (t) 0 0 otherwise ( N ) α =1 x (t) + λ S s(t) p(t)>0, or ṗ(t) = if N =1 x(t)+λ S s(t) 0 0 otherwise I RP-C, cosumer loads are computed via the smooth fuctios u. Sice F ɛ is cotiuous ad strictly icreasig o [ ɛ mi, ɛ max], u is cotiuous ad strictly decreasig, ad has a iverse u with the domai [0, x ]. We defie fuctio U such that U (x) u (x) o (0, x ), which exists sice u (x) is cotiuous, ad hece itegrable. Explicitly, U (x) u (x)dx, for x [0, x ] (14) Note that U is strictly cocave o [0, x ], i.e. there exists c u > 0 such that Ü(z) c u < 0 for all z ad. Havig defied the fuctios U, we cosider the followig social welfare maximizatio problem mi x,s s.t. C(s) N U (x ) (15) =1 N x + λ S s (16) =1 λ x, (17) I (15), U ca be iterpreted as a cosumer utility fuctio. Problem (15) is quite similar to problem (5) oly with a chage i the objective fuctio, where the utility of cosumptio is ameded. Defie p to be the dual variable correspodig to (16), ad q to be the dual variables correspodig to (17). Let (ˆx, ŝ, ˆp, ˆq) be the optimal primal-dual solutio to problem (15). The ext theorem shows that RP-C coverges to the optimal solutio of (15)-(17). Theorem 1. The cotiuous-time approximatio algorithm RP-C coverges to the optimal solutio (ˆx, ŝ, ˆp, ˆq) of Problem (15). Proof. See Appedix A. Note that Theorem 1 gives isights o the average behavior of RP as we will see i Sectio VI. For the covergece ad performace result o RP, we provide the discrete-time aalysis of RP i the followig. B. Discrete-time Aalysis of Algorithm RP I the previous sectio, we observed that the cotiuoustime approximatio of RP coverges to the optimal solutio of (15), which is closely related to the origial problem (5) via the distributio of the price perturbatios ɛ (t). I this sectio, we provide covergece results for RP i discrete time. The followig theorem shows that, uder Algorithm RP, s(t) ad the price p(t) get arbitrarily close to the correspodig optimum values ŝ ad ˆp of the modified problem (15). Theorem 2. Uder Algorithm RP, we have T 1 lim E [ (s(t) ŝ) 2] B (18) c c 1 lim T E [ ( x (t) ˆx ) 2] B (19) c u where B is a costat that depeds o the step size α ad user parameter κ, ad x (t) u (p(t) κ q (t)). Proof. See Appedix B. We have the followig observatios o Theorem 2: i) The Effect of Price Radomizatio: Theorem 2 shows that the commo price p(t) ad the average cosumer load x (t) get closer to the optimum values of problem (15), as c c ad c u become large. Note that c u is defied as Ü(x) c u ad we had U (x) = u (x). Thus, the larger c u is, the less steep the CDF F ɛ is. For istace, for ɛ (t) U ( ɛ, ɛ), as ɛ gets larger c u gets larger as well. Hece, icreasig the amout of radomess o the commo price decreases the bouds i (18) ad (19). O the other had, we should ote that B is directly proportioal to the secod momet of ɛ (t); icreasig the amout of radomess, icreases B. Due to this two-sided effect of radomizatio o covergece results, the distributio for the disturbaces added to the commo price should be carefully chose as we will observe i Sectio VI. ii) Step Size α ad Load Trackig Capability: The results show that α should be sufficietly small for better covergece

6 because the boudig term B depeds o α. O the other had, too small a value of α may affect how the algorithm tracks the chages i the iflexible load S i (t). Specifically, if α is chose to be too small, the price ca lag behid S i (t), ad cosequetly the flexible load ca miss the valleys i the daily patter. We ote that this trade-off betwee covergece rate ad trackig capability is commo amog iterative algorithms. iii) Number of Flexible Users N ad Fluctuatios i Total Load: Theorem 2, i particular (19), suggests that x(t) will be close to ˆx. Note that x (t) u (p(t) κ q (t)) ad ˆx = u (ˆp κ ˆq ). Thus, as α 0 we get (p(t) κ q (t)) (ˆp κ ˆq ). I this regime, the cosumptio of idividual customer is give by x (t) = x 1{ɛ (t) κ q (t) p(t)} x 1{ɛ (t) κ ˆq ˆp}. Hece x (t) are approximately idepedet radom variables. The, we ivoke Kolmogorov s Strog Law of Large Numbers to argue that, as N, we have with probability 1 1 N x (t) 1 x F ɛ (κ ˆq ˆp) 0 (20) N As a result, we ca deduce that as the umber of flexible users, N, icreases, total flexible load teds to remai close to its average value. C. Fairess We coclude this sectio with a discussio o the fairess of Algorithm RP. Although each user receives a radomized versio of the price p (t), the radomizatio is performed i a ubiased ad idepedet maer based o a commo price p(t). As a result, o user receives more preferetial treatmet i its idividual price. I this sese, the prices see by the users are still fair. This fairess property ca be stated rigorously as follows. Cosider a arbitrary time-iterval [t 1, t 2 ]. For user t2 t=t 1 p (t) as the average 1, defie p (t 1, t 2 ) = t 2 t 1+1 price see by user durig this time iterval. Let F(t) deote the σ-algebra geerated by all radom variables at or before time t. Further, for ay a > 0, let I(a) = max θ 0 {θa log[m ɛ (θ)m ɛ ( θ)]}, where M ɛ (θ) E[exp(θɛ (t)] is the momet geeratig fuctio of the i.i.d. radom variable ɛ (t). It is easy to verify that I(a) > 0 for all a > 0 (see, e.g., [20, p27]). Propositio 1 (Fairess Property of RP). The followig properties hold for all t 1 t 2 ad for ay two users 1, 2 : (i) E[p 1 (t 1, t 2 ) F(t 1 1)] = E[p 2 (t 1, t 2 ) F(t 1 1)], where the expectatio is take with respect to the radomizatio itroduced by all radom variables ɛ (t). (ii) P[ p 1 (t 1, t 2 ) p 2 (t 1, t 2 ) a] 2e (t2 t1+1)i(a). As we ca see from the above propositio, over ay time iterval [t 1, t 2 ], the average price see by ay two users 1 ad 2 will be the same i expectatio, idepedetly of what happes before t 1. Further, the probability that their average prices over this time-iterval differ by more tha a value a > 0 will decrease expoetially to zero as the legth of the timeiterval icreases. The proof follows directly from the Markov iequality, ad is omitted due to space costraits. V. CHANGE-OF-USE PRICING (COUP) ALGORITHM I this sectio, we take a differet approach ad propose a ew pricig scheme. The key idea is to pealize large variatios i each cosumer s load by itroducig a secodary price. I particular, cosumers are charged for a extra pealty based o the amout of chage i their loads betwee cosecutive time periods, while they still pay for their cosumptio at each time period at the primary price. Pricig the chage i load ca be iterpreted as aother sort of differetiatio amog users. I this case, the secodary price itroduces heterogeeity amog cosumptio decisios of users. Ituitively, users will prefer chagig their cosumptio more gradually depedig o their iteral states istead of cosumig either the maximum x or 0. We will show that our pricig algorithm coordiates users cosumptio decisios i a asychroous maer such that chages i users loads cacel out to create a total load that is flat. Uder the ew algorithm COUP, at each time t the commo price p(t) ad the secodary price γ are aouced. The paymet at time t for a cosumer with load x (t) is p(t)x (t) + γ(x (t) x (t 1)) 2. Here, the secod term is the ew compoet that icurs a pealty (uiform across users ad costat over time) o the chage of load. Ituitively, this pealty ecourages the users to smooth out their loads, ad reduces the potetial volatility. Havig discussed the ew pricig scheme, we preset the ew pricig mechaism i Algorithm COUP below. Algorithm COUP Chage-of-Use Pricig Algorithm At iteratio t: Cosumer receives the commo price p(t) ad the pealty price γ. The, it computes its queue as i (1) ad load as [ x (t) = x (t 1) + 1 ] + 2γ (κ q (t) p(t)) (21) The LA must meet the real load x (t) + S i (t). Further, it computes s(t) = Ċ (p(t)), ad updates the price: [ ( N )] + p(t + 1) = p(t) + α x (t) + S i (t) s(t) =1 I COUP, (21) correspods to the solutio of the followig optimizatio problem give the user s cosumptio i the previous period t 1: mi x { (p κ q )x + γ(x x (t 1)) 2}. (22) Drawig direct compariso to RP ad the threshold rule (10), we observe that without the secod term, (22) is similar to the problem that a cosumer solves uder RP. Hece, the secod term ca be see as the additio due to the pealty o the chage i cosumptio. Furthermore, the price update rule i COUP is still the same as that i RP. Followig a similar method as i the aalysis of RP, we will show ext that the cotiuous-time approximatio of COUP achieves the optimal objective value of the origial problem (5) by solvig a closely

7 related welfare maximizatio problem. I particular, the ew welfare maximizatio problem differs from the origial oe i its objective, which ivolves the augmetatio of a proximal term to the origial objective due to the pealty term we itroduced i the pricig mechaism. A. The Cotiuous-Time Fluid Approximatio Model ad Welfare-Maximizatio-Based Formulatio The cotiuous-time fluid approximatio model for COUP is straightforward to obtai ad it is preseted i Algorithm COUP-C. Similar to what we oted before for the discrete-time algorithms, COUP-C differs from RP-C oly i the descriptio of user cosumptio x (t). Algorithm COUP-C Cotiuous-time Approximatio to COUP { 1 x (t)>0, or κ q (t) p(t) 0 2γ ẋ (t) = (κ q (t) p(t)) if 0 otherwise { q(t)>0, or λ x (t) if λ q (t) = x (t) 0 0 otherwise (23) (24) s(t) = Ċ (p(t)) (25) ( N ) α =1 x (t) + λ S s(t) p(t)>0, or ṗ(t) = if ( N (26) =1 x(t)+λ S s(t))>0 0 otherwise Next, we will show that COUP-C coverges to a statioary regime where it achieves the optimal objective of the origial problem (5). To this ed, we augmet the objective of problem (5) with a additioal cost term motivated by the proximal optimizatio algorithm [21]. The resultig welfaremaximizatio problem is mi x,y,s s.t. C (s) + γ N (x y ) 2 (27) =1 N x + λ S s (28) =1 λ x,, (29) where γ is a positive costat ad y R are auxiliary variables. It is easy to see that if x ad s are the optimal solutio to problem (5), the x = x, y = x, ad s = s are trivially the optimal solutio to problem (27). However, the quadratic term i (27) makes the problem strictly covex i x, which helps to alleviate volatility as we will see shortly. As i other proximal optimizatio algorithms [21], at each iteratio we first fix y (t) ad optimize the objective of (27) over x. Let the correspodig optimal solutio be x (t). We the set y (t + 1) = x (t) ad cotiue with the ext iteratio. By settig y (t + 1) = x (t), the quadratic term i (27) becomes γ N =1 (x (t) x (t 1)) 2, which pealizes the differece i load betwee periods t ad t 1. We cosider the Lagragia fuctio for problem (27) for fixed y = x (t 1), ad obtai the dual fuctio as D(p, q) = N =1 mi x { (p κ q )x + γ(x y ) 2} + mi s 0 {C(s) ps} + pλ S + N λ κ q, (30) =1 where p 0 ad κ q [κ q 0, = 1, 2,..., N] are the dual variables correspodig to the costraits (28) ad (29), respectively. The first optimizatio i (30) is the users optimizatio problem (22) whose solutio gives the cosumptio update rule (21) of COUP. The secod optimizatio i (30) is the profit maximizatio for the LA. Furthermore, ispectig the dual problem reveals that COUP-C correspods to the dual algorithm for problem (27). Specifically, the primal variables x, s ad the dual variables p, q are updated at each iteratio first with y kept fixed, ad the y is updated at the ed of each iteratio by settig y(t+1) = x(t). Thus, y(t) is dropped from the algorithm descriptio ad is replaced with x(t 1). Havig established the relatio betwee COUP-C ad problems (5) ad (27), we ca study the covergece ad optimality of COUP-C. Before doig so, we give the defiitio of the statioary poit for the sum of variables. Defiitio 1. Defie Φ(t) ( x (t), q (t), p(t), s(t)). Φ (X, Q, p, s ) is a statioary poit of COUP ad COUP-C i the sum sese, if Φ(t 0 ) = Φ for some t 0 < ad Φ(t) = Φ(t 0 ) for all t > t 0. Note that, Φ may ot achieve the optimal objective of problem (27) sice we use x (t 1) i place of y (t). However, if Φ satisfies p = Ċ(s ), s = X + λ S, X = λ, the Φ achieves the optimal objective of problem (5). The ext theorem shows that the system of equatios give i COUP-C coverges to the statioary state as described i Defiitio 1, where Φ achieves the optimal objective of (5). Theorem 3. I the system characterized by Algorithm COUP- C, Φ(t) coverges to a statioary poit Φ, which achieves the optimal objective value of problem (5). Proof. See Appedix C. Theorem 3 shows that the aggregate flexible load, x (t) coverges to X. Thus, the mai observatio that we draw from the theorem is that the oscillatios i the price ad the total load become arbitrarily small uder COUP-C. B. Discrete-time Aalysis of Algorithm COUP This sectio presets the aalysis of COUP i discrete-time. I particular, Theorem 4 shows that s(t) ad the price p(t) get arbitrarily close to s ad p, respectively, the optimum values of the origial problem (5). Theorem 4. Uder Algorithm COUP, we have T 1 lim E [ (s(t) ŝ) 2] B (31) c c

8 1 lim T [ E (x (t) x (t 1)) 2] B γ (32) where B is a costat that depeds o the step size α ad user parameter κ. Proof. See Appedix D. Theorem 4 further states that the differeces i idividual cosumers loads betwee cosecutive time slots decrease with icreasig pealty factor γ. Thus, as aticipated, sufficietly large values of γ dampes the fluctuatios i the total flexible load ad the price. Furthermore, as i the case of RP ad Theorem 2, we ca make similar observatios regardig the step size α ad the algorithm s load trackig capability. VI. PERFORMANCE AND NUMERICAL RESULTS We ow provide umerical results that demostrate the desirable features of the proposed algorithms. The performace metrics that we cosider are the paymets made by the cosumers ad the cost of geeratio. I terms of these metrics, we compare the performace of RP ad COUP to the bechmark schemes. I the rest of this sectio, the followig simulatio setup is used uless stated otherwise: LA cost is set to be C(s) = s2 2. Flexible cosumers receive Poisso distributed radom arrivals. I RP, ɛ (t) U ( ɛ, ɛ) for all t. I COUP, κ = 1 ad α = First, we preset the algorithms behavior over time. I Figure 3, load evolutios obtaied by ruig RP ad COUP are plotted for two days. There are 1000 flexible cosumers ad their total average load is set to be 5% of the total load. Historical metered load data from PJM is used as iflexible load [22]. I Figure 3, we observe a waterfillig behavior that results i a smoother load patter (c.f. Figure 2); Flexible users cosume electricity whe the iflexible demad is low (i.e. whe the average price is low) ad fill the valleys i the daily patter. The effect of price differetiatio by radomizatio amog flexible users shows up as the radom zigzag patter for RP. O the other had, total load is smoother uder COUP as deduced from (32) i Theorem 4. Load (MW) Iflexible Load RP: Total Load COUP: Total Load RP ad COUP: Load vs. Time Time (Hour) Fig. 3. Waterfillig behavior of the aggregate grid load. I RP, ɛ is set to be 1% of the average price. I COUP, γ is set to be 1% of the average price. A. Cosumer ad LA Paymets The paymet made by cosumer i RP-C is give by [ { r (t) = p(t)x (t) + x m E ɛ 1 ɛ U(x }] (t)) which follows from (12). O the other had, the LA aticipates the paymet w(t) = p(t)s(t), sice it sets p(t) based o the computed value of s(t). At the equilibrium of Algorithm RP- C, the supplier ad total cosumer paymets are give by ˆr = ˆp ˆx + [ { x m E ɛ 1 ɛ U(ˆx }] ) (33) ˆω = ˆpŝ = ˆp ˆx (34) where (34) follows from the KKT coditio i (39). O the other had, uder COUP-C, a idividual cosumer s paymet does ot achieve a costat equilibrium value due to the sum covergece result i Theorem 3. Istead, cosumer paymet has the followig time-varyig limit whe the system is i the statioary regime give i Defiitio 1: r (t) = ˆpx (t) + γ (x (t) x (t 1)) 2 (35) Therefore, total cosumer paymet is also time-varyig, ad is give by r(t) = ˆpX + γ (x (t) x (t 1)) 2 (36) Besides, supplier paymet uder COUP-C achieves a costat value i the statioary state as it does uder RP-C ˆω = ˆpŝ = ˆpX (37) Comparig (33) to (34), ad (36) to (37), we observe that the amout of paymet received from flexible cosumers ad the amout of aticipated paymet computed by the supplier do ot ecessarily match. The differeces betwee cosumer ad supplier paymets are give by the secod terms i (33) ad (36). We call this differece the LA deficit. Note that the LA deficit is always positive for COUP-C because of the secodary price γ, whereas uder RP-C, the deficit ca be either egative or positive depedig o the distributio of ɛ. Naturally, oe wats to make the LA deficit as close as possible to 0 so that the system actually clears i terms of paymets. As a example, for algorithm RP-C, cosider the case where λ = λ, x = x for all, ad ɛ s have the idetical uiform distributio over the iterval [ɛ, ɛ + a], i.e. F ɛ (x) = x ɛ a. The, settig ɛ = aλ x esures that the deficit is 0. O the other had, for COUP-C, icreasig γ decreases the chages i idividual cosumer loads due to Theorem 4, ad cosequetly the secodary term i (36) decreases. I fact, our simulatios show that the LA deficit is fairly small for both RP ad COUP. For RP, aively settig ɛ U( ɛ, ɛ), where ɛ is approximately 1% of the average price, esures that the deficit is o larger tha 0.5% of the paymet aticipated by the LA. O the other had for COUP, settig κ to 1 ad γ to 1% of the average price icreases the cosumers paymets by oly 0.01% while achievig the desired flat load. B. Impact of Flexible Cosumer Peetratio o Supply Cost ad Paymets Next, we demostrate the impact of icreasig peetratio of flexible cosumers o the supply cost ad the flexible cosumers paymets i Figure 4. We vary the umber of flexible cosumers i the system while keepig the total load costat. I Figure 4, the arrows idicate the directio that the

9 peetratio of flexible cosumers icreases. We observe that a flexible cosumer s paymet is greatly reduced compared to the case where it has to serve its demad immediately (i.e. Scheme 1). Furthermore, compared to the amout that they pay uder Scheme 2, flexible cosumers pay less uder RP, ad they pay similar or slightly higher uder COUP. Thus, we ca coclude that cosumers sigificatly beefit from havig flexible demad ad they will be willig to participate i the ew pricig mechaisms to further reduce their paymets. Aother observatio is that paymets icrease as the umber of flexible cosumers icreases. This is because lower price periods are filled with flexible load, ad cosequetly prices i these periods are ot as low as before. As a result, the flexible cosumers do ot have as much opportuity to take advatage of prices. represetative umerical results to illustrate their differet behavior. I the followig radomized-delay (RD) schemes, the LA sets the commo price p(t) based o the total load, ad the cosumers make their cosumptio decisios, x (t), cosiderig the price ad the amout of their waitig tasks. The, the LA adds aother delay amout d (t), which is a radom variable that is idepedetly geerated for each piece of demad from a commo distributio. Each cosumer has to defer its decided cosumptio amout, x (t), for d (t) time slots. Cosequetly, the decisio x (t) appears as load i the system at time t + d (t). We ivestigated two ways of implemetig RD: o-preemptive, where the cosumers are ot allowed to make further cosumptio decisios util their last task is served; ad preemptive, where they are allowed to do so. Supply Cost Supply Cost vs. Paymet from Flexible Cosumers Icreasig Flexibility Scheme 1 Scheme 2 Alg RP Alg COUP Paymet from Cosumers Fig. 4. Algorithms RP ad COUP perform better with icreasig flexible cosumer peetratio i the system. I RP, ɛ (t) U ( ɛ, ɛ), where ɛ is set to be 1% of the average of the optimal commo price. I COUP, γ is set to be 10% of the average of the commo price. Figure 4 demostrates that for Scheme 2 there are two regimes i terms of the supply cost. I the first regime, where flexible load is less tha 20% of the total load, icreasig flexibility reduces the cost, eve though Scheme 2 already starts to exhibit abrupt fluctuatios i price ad total load. A reaso for the decrease i cost is the reduced peak load with the icreased umber of flexible cosumers. We also ote that our formulatio is based o covex cost structure, hece it may ot capture efficietly the effect of abrupt fluctuatios such as the stress o the etwork ad maiteace costs. However, i the secod regime where flexible load is higher tha 20%, fluctuatios i load become too large ad they are reflected directly i cost uder Scheme 2. As far as RP ad COUP are cocered, they elimiate the fluctuatios, ad thus always lead to strictly lower supply cost as the peetratio of flexible cosumers icreases. Due to this reaso, the advatage of the proposed algorithms become more apparet whe the flexible cosumer peetratio icreases beyod 20% as see i Figure 4. C. Performace with Radomized-Delay (RD) Schemes We ote that the idea i Algorithm RP, i.e., of differetiatig the users to elimiate the aligmet of their load decisios, ca also be applied i other dimesios. For istace, the LA may istead delay the cosumers service by a radom delay. While a full aalysis of such radomized-delay schemes is beyod the scope of this paper, below we briefly preset some Load (MW) Load (MW) Load (MW) Radom Delay (8 slots): Load vs. Time Iflexible Load Radom Delay: Total Load Time (Hour) (a) 8-miute radom delay, o-preemptive Radom Delay (8 slots): Load vs. Time Iflexible Load Radom Delay: Total Load Time (Hour) (b) 8-miute radom delay, preemptive Radom Delay (32 slots): Load vs. Time Iflexible Load Radom Delay: Total Load Time (Hour) (c) 32-miute radom delay, preemptive Fig. 5. Load evolutio uder two versios of radomized-delay schemes: (a) o-preemptive; (b) ad (c): preemptive. Fig. 5(a) ad (b) show the performace of both schemes with a radom delay of withi 8 miutes (each slot takes 1 miute). We fid that the first scheme (o-preemptive) is ot effective i elimiatig the fluctuatios of the total load (Fig. 5(a)). I this case, users will begi to get service at every opportuity due to the excessive backlog they receive while waitig for the radom amout of delay, prevetig the valley-fillig behavior. The secod scheme (preemptive) of radomized delay ca exhibit valley-fillig behavior, but the amout of delay must be carefully chose. If the delay is ot sufficietly high, the load exhibits highly fluctuatig behavior (Fig. 5(b)). Whe the delay is large (e.g., 32 miutes), the preemptive RD scheme starts to produce valley-fillig behavior (Fig. 5(c)). However, we believe that i this case the excessive delay may

10 distort the users perceptio of service quality. Note that the key idea for pricig-based demad-respose is to let users choose their time-of-service based o both the price sigals ad their ow prefereces (e.g., i terms of how much they are willig to wait). Whe a ucotrolled, potetially substatial, amout of radom delay is added by the LA, the users o loger have a accurate sese of the overall delay experieced by their demad. This distortio of service experiece could potetially make it less desirable for the users to take iformed actios as to whe to use service. I cotrast, there is o such distortio i our proposed radomized pricig (RP) scheme: the users are always i complete cotrol as to whe they ca choose to have their demad served. Thus, how to desig RD schemes to achieve comparable performace as RP schemes remais a ope problem, which may warrat a separate idepth study o its ow. VII. CONCLUSIONS We proposed two ovel real-time dyamic pricig schemes that attempt to solve the volatility problem i a system where ecoomically-drive cosumers have the flexibility to defer their demad. We demostrated the destabilizig effect of opportuistic cosumer behavior o the load ad the price, whe covetioal real-time pricig methods are employed. We propose two ew pricig schemes to address this problem. I our first pricig scheme, idividual cosumers receive differet prices that are created by addig small radom perturbatios to a commo price. O the other had, the secod algorithm sets a secodary price for all cosumers, alog with the commo price for cosumptio. The secodary price pealizes abrupt chages i idividual users cosumptio. The uderlyig idea i both proposed algorithms is to create differetiatio amog cosumers so that their aggregate behavior is averaged out over the cosumer base. The proposed pricig schemes are simple to implemet sice they do ot require ay kowledge o cosumer strategies, ad they ca be employed i various systems other tha the smart grid where demad has time flexibilities. Furthermore, i the paper, we umerically demostrated that self-iterested cosumers ecoomically beefit from deferrig their demad while the supply cost for the LA is kept low. APPENDIX A PROOF OF THEOREM 1 The covergece of the cotiuous-time algorithm RP-C ca be established by usig techiques i [23], [24]. To that ed, usig the KKT coditios for problem (15), it ca be show that the followig Lyapuov fuctios is strictly decreasig: V (t) = 1 2α (p(t) ˆp) APPENDIX B PROOF OF THEOREM 2 κ (q (t) ˆq ) 2. (38) First, ote the KKT coditios for problem (15): ŝ = ˆx + λ S, ˆp = Ċ(ŝ), (39) ˆx = U (ˆp κ ˆq ), ˆx λ, (40) ˆq (λ ˆx ) = 0, (41) To establish the covergece of RP, we cosider the followig Lyapuov fuctio ad its 1-slot drift: V (t) = 1 2α (p(t) ˆp)2 + 1 κ (q (t) ˆq ) 2. (42) 2 After ( algebraic maipulatios, ad usig the fact that 2 [y] + z) (y z) 2 for z 0, the boud o the drift V (t + 1) V (t) is obtaied as ( 2 V (t + 1) V (t) α x (t) + S i (t) s(t)) (43) 2 ( ) + x (t) + S i (t) s(t) (p(t) ˆp) (44) + + κ 2 (a (t) x (t)) 2 (45) κ (a (t) x (t))(q (t) ˆq ) (46) Notig that S i (t), a (t), ad ɛ (t) are idepedet of the rest of the variables, we take the expectatio of the drift boud w.r.t. their distributios coditioal o p(t) ad q (t). After takig the expectatio, we ote that (43) ad (45) ca be bouded as follows: ( 2 E α x (t) + S i (t) s(t)) 2 p(t), q (t) B 1 E [ (47) ] 2 (a (t) x (t)) 2 p(t), q (t) B 2 (48) κ where 0 < B 1, B 2 <, due to the fact that first ad secod momets of the aforemetioed variables are bouded. Furthermore, we have E [S i (t) p(t), q (t)] = λ S, E [a (t) p(t), q (t)] = λ, ad x (t) E [x (t) p(t), q (t)] = E [x 1{ɛ (t) κ q (t) p(t)} p(t), q (t)] = x F ɛ (κ q (t) p(t)) = u (p(t) κ q (t)) We also defie the coditioal expected drift as V (t) E [V (t + 1) V (t) p(t), q (t)]. Usig the bouds (47) ad (48), ad the above expected values, we obtai the boud o V (t) as ( ) V (t) B 1 + B 2 + x (t) + λ S s(t) (p(t) ˆp) + κ (λ x (t))(q (t) ˆq ) Addig ad subtractig ˆx ad ŝ, usig ŝ = ˆx +λ S, ad defiig B 3 B 1 + B 2 we obtai ( ) V (t) B 3 + (p(t) ˆp) ( x (t) ˆx ) + ŝ s(t)

11 + κ (q (t) ˆq )(λ ˆx + ˆx x (t)) = B 3 + (p(t) ˆp)(ŝ s(t)) (49) + + κ (q (t) ˆq )(λ ˆx ) (50) ( x (t) ˆx ) (p(t) κ q (t) (ˆp κ ˆq )) (51) Now, we treat each term i the above expressio separately. For the secod term i (49), we apply the mea value theorem ad use the strog covexity of C to obtai ) (p(t) ˆp)(ŝ s(t)) = (Ċ(s(t)) Ċ(ŝ) (s(t) ŝ) = C(z)(s(t) ŝ) 2 c c (s(t) ŝ) 2 where c c > 0 is such that C(z) cc > 0 for all z. Furthermore, (50) is upper bouded by 0: From complemetary slackess coditio give i (41), ˆq (λ ˆx ) = 0 for all. Also, q (t)(λ ˆx ) 0 due to dual ad primal feasibility. Hece, (q (t) ˆq )(λ ˆx ) 0 for all, ad (50) is upper bouded by 0. Cosider (51) i the drift boud. Takig the iverse of the fuctio u, we get p(t) κ q (t) = u ( x (t)) = U( x (t)). Furthermore, ˆp κ ˆq = U(ˆx ) from (40). Pluggig the above expressios i (51), applyig the mea value theorem, ad usig the strog cocavity of U, we obtai ( x (t) ˆx ) (p(t) κ q (t) (ˆp κ ˆq )) = ( ( x (t) ˆx ) U( x (t)) U(ˆx ) ) = ( x (t) ˆx)Ü(z)( x (t) ˆx) c u ( x (t) ˆx) 2 where c u > 0 is such that Ü(z) c u < 0 for all z ad. Usig the bouds that we obtaied for the terms (49), (50), ad (51) we obtai V (t) B 3 c c (s(t) ŝ) 2 c u ( x (t) ˆx ) 2 (52) The, we take the expectatio of the above drift expressio w.r.t. p(t) ad q (t), write it for t = 0,..., T 1, add both sides of the iequalities, divide by T, ad take the limit as t to obtai c u lim T E [ ( x (t) ˆx ) 2] c c + lim T E [ (s(t) ŝ) 2] B 3 (53) APPENDIX C PROOF OF THEOREM 3 The covergece of the cotiuous-time algorithm COUP- C ca be established by usig techiques i [25]. We defie Θ(t) (x(t), q(t), p(t), s(t)) to be the system state at t. Usig the KKT coditios for problem (27), we cosider the followig Lyapuov fuctio: V (Θ(t)) = N 2α (p(t) p ) 2 ( ) 2 ( ) 2 + γ x (t) X + κ q (t) Q (54) 2 where Φ is give i Defiitio 1 ad achieves the optimal objective of (5). Usig similar techiques to the oes i [23], [24], it ca be show that (54) is strictly decreasig, which establishes the theorem s result. APPENDIX D PROOF OF THEOREM 4 Note the KKT coditios for problem (27): ŝ = ˆx + λ S, ˆp = Ċ(ŝ), (55) ˆx = ŷ, ˆx λ, (56) ˆq (λ ˆx ) = 0, ˆq = ˆp, (57) To establish the covergece result for COUP, we cosider the followig Lyapuov fuctio ad its 1-slot drift: V (t) = 1 2α (p(t) ˆp)2 + 1 ( κ q (t) ˆq ) 2 2 κ + γ (x (t 1) ˆx ) 2 (58) Note that the above fuctio is very similar to the oe we used i Appedix B i the proof of Theorem 2, except with the editio of the last term. Therefore, followig similar steps take i Appedix B, we obtai the followig boud o the coditioal expected drift as V (t) E [V (t + 1) V (t) p(t), q (t), x (t 1)]: V (t) B 3 + (p(t) ˆp)(ŝ s(t)) (59) + ( κ q (t) ˆq ) (λ ˆx ) (60) κ + (x (t) ˆx ) (p(t) κ q (t) (ˆp ˆq )) (61) + 1 (κ q (t) p(t)) 2 (62) 4γ + (x (t 1) ˆx ) (κ q (t) p(t)) (63) Now, we treat each term i the above expressio separately. First, ote that, i (61), (ˆp ˆq ) = 0 due to (57). For the secod term i (59), applyig the mea value theorem ad usig the strog covexity of C, we obtai ) (p(t) ˆp)(ŝ s(t)) = (Ċ(s(t)) Ċ(ŝ) (s(t) ŝ) = C(z)(s(t) ŝ) 2 c c (s(t) ŝ) 2. Furthermore, (60) is upper bouded by 0 by usig the KKT coditios give i (55)-(57)

12 Usig the bouds that we obtaied for the terms (59), (60), ad combiig (61) ad (63) together, we obtai V (t) B 3 c c (s(t) ŝ) (κ q (t) p(t)) 2 4γ + (x (t) x (t 1)) (p(t) κ q (t)). (64) Rearragig the terms ad usig the update rule for x (t), we obtai V (t) B 3 c c (s(t) ŝ) 2 γ (x (t) x (t 1))) 2. The, we take the expectatio of the above drift expressio w.r.t. p(t), q (t), ad x (t 1), write it for t = 0,..., T 1, add both sides of the iequalities, divide by T, ad take the limit as t to obtai γ lim T c c + lim E [(x (t) x (t 1)) 2] T E [ (s(t) ŝ) 2] B 3. (65) REFERENCES [1] The Brattle Group. (2007) Quatifyig demad respose beefits i PJM. [2] C. Joe-Wog, S. Ha, ad M. Chiag, Time-depedet broadbad pricig: Feasibility ad beefits, i st Iteratioal Coferece o Distributed Computig Systems (ICDCS), IEEE, 2011, pp [3] A. J. Coejo, J. M. Morales, ad L. Barigo, Real-time demad respose model, IEEE Tras. o Smart Grid, 1:3, pp , [4] M. J. Neely, A. S. Tehrai, ad A. G. Dimakis, Efficiet algorithms for reewable eergy allocatio to delay tolerat cosumers, i First IEEE It. Cof. o Smart Grid Commuicatios, pp , [5] D. Materassi, M. Roozbehai, ad M. A. Dahleh, Equilibrium price distributios i eergy markets with shiftable demad, i IEEE 51st Aual Coferece o Decisio ad Cotrol, pp , [6] M. Roozbehai, M. A. Dahleh, ad S. K. Mitter, Volatility of power grids uder real-time pricig, IEEE Trasactios o Power Systems, vol. 27, o. 4, pp , [7] L. Jiag ad S. Low, Real-time demad respose with ucertai reewable eergy i smart grid, i 49th IEEE Aual Allerto Cof. o Commuicatio, Cotrol, ad Computig, pp , 2011 [8], Multi-period optimal eergy procuremet ad demad respose i smart grid with ucertai supply, i 50th IEEE Coferece o Decisio ad Cotrol ad Europea Cotrol Coferece, pp , [9] L. Che, N. Li, S. H. Low, ad J. C. Doyle, Two market models for demad respose i power etworks, IEEE SmartGridComm, vol. 10, pp , [10] A.-H. Mohseia-Rad, V. W. Wog, J. Jatskevich, R. Schober, ad A. Leo-Garcia, Autoomous demad-side maagemet based o game-theoretic eergy cosumptio schedulig for the future smart grid, IEEE Tras. o Smart Grid, vol. 1, o. 3, pp , [11] A.-H. Mohseia-Rad, V. W. Wog, J. Jatskevich, ad R. Schober, Optimal ad autoomous icetive-based eergy cosumptio schedulig algorithm for smart grid, i IEEE Iovative Smart Grid Techologies (ISGT), pp. 1 6, [12] M. He, S. Murugesa, ad J. Zhag, Multiple timescale dispatch ad schedulig for stochastic reliability i smart grids with wid geeratio itegratio, i Proceedigs of IEEE INFOCOM, pp , [13] C. Joe-Wog, S. Se, S. Ha, ad M. Chiag, Optimized day-ahead pricig for smart grids with device-specific schedulig flexibility, IEEE Joural o Selected Areas i Comm., 30:6, pp , [14] O. Dalkilic, O. Cadoga, ad A. Eryilmaz, Pricig algorithms for the day-ahead electricity market with flexible cosumer participatio, i IEEE Ifocom Workshops, pp , [15] S. Che, P. Siha, ad N. B. Shroff, Schedulig heterogeeous delay tolerat tasks i smart grid with reewable eergy, i IEEE 51st Aual Cof. o Decisio ad Cotrol (CDC), pp , [16] T. T. Kim ad H. V. Poor, Schedulig power cosumptio with price ucertaity, IEEE Tras. o Smart Grid, 2:3, pp , [17] M. Baye, Maagerial Ecoomics ad Busiess Strategy, 6th Editio. McGraw-Hill/Irwi, [18] S. Salop, The oisy moopolist: Imperfect iformatio, price dispersio ad price discrimiatio, Review of Ecoomic Studies, vol. 44, pp , [19] J. G. Dai, O positive harris recurrece of multiclass queueig etworks: a uified approach via fluid limit models, The Aals of Applied Probability, pp , [20] A. Dembo ad O. Zeitoui, Large Deviatios Techiques ad Applicatios, Spriger, [21] X. Li ad N. B. Shroff, Utility maximizatio for commuicatio etworks with multipath routig, IEEE Trasactios o Automatic Cotrol, vol. 51, o. 5, pp , [22] PJM. (2014) Historical metered load data. [Olie]. Available: http: // [23] A. Cherukuri, E. Mallada, ad J. Cortés, Asymptotic covergece of costraied primal dual dyamics, Systems & Cotrol Letters, vol. 87, pp , [24] S. Mey, Cotrol techiques for complex etworks. Cambridge Uiversity Press, [25] H. K. Khalil, Noliear systems. Pretice hall Upper Saddle River, 2002, vol. 3. Ozgur Dalkilic received his B.S. ad M.S. degrees i Electrical ad Electroics Egieerig from Bogazici Uiversity, Istabul, i 2006 ad 2009, respectively. Ozgur s life was tragically cut short due to a accidet i early 2016 as he was about to successfully complete his Ph. D. studies at the Electrical ad Computer Egieerig Departmet of The Ohio State Uiversity. His kidess will be remembered by all his loved oes, ad his brilliace is documeted i this article. Atilla Eryilmaz (S 00 / M 06) received his M.S. ad Ph.D. degrees i Electrical ad Computer Egieerig from the Uiversity of Illiois at Urbaa- Champaig i 2001 ad 2005, respectively. Betwee 2005 ad 2007, he worked as a Postdoctoral Associate at the Laboratory for Iformatio ad Decisio Systems at the Massachusetts Istitute of Techology. He is curretly a Associate Professor of Electrical ad Computer Egieerig at The Ohio State Uiversity. Dr. Eryilmaz s research iterests iclude desig ad aalysis for commuicatio etworks, optimal cotrol of stochastic etworks, optimizatio theory, distributed algorithms, pricig i etworked systems, ad iformatio theory. He received the NSF-CAREER Award i 2010 ad two Lumley Research Awards for Research Excellece i 2010 ad He is a co-author of the 2012 IEEE WiOpt Coferece Best Studet Paper, ad the 2016 IEEE Ifocom Best Paper. He has served as TPC co-chair of IEEE WiOpt i 2014 ad of ACM Mobihoc i 2017, ad is a Associate Editor of IEEE/ACM Trasactios o Networkig sice Xiaoju Li (S 02 / M 05 / SM 12) received his B.S. from Zhogsha Uiversity,Guagzhou, Chia, i 1994, ad his M.S. ad Ph.D. degrees from Purdue Uiversity, West Lafayette, Idiaa, i 2000 ad 2005, respectively. He is curretly a Associate Professor of Electrical ad Computer Egieerig at Purdue Uiversity. Dr. Li s research iterests are i the aalysis, cotrol ad optimizatio of wireless ad wirelie commuicatio etworks. He received the IEEE INFOCOM 2008 best paper award ad 2005 best paper of the year award from Joural of Commuicatios ad Networks. His paper was also oe of two ruerup papers for the best-paper award at IEEE INFOCOM He received the NSF CAREER award i He was the Workshop co-chair for IEEE GLOBECOM 2007, the Pael co-chair for WICON 2008, the TPC co-chair for ACM MobiHoc 2009, ad the Mii-Coferece co-chair for IEEE INFOCOM He is curretly servig as a Area Editor for (Elsevier) Computer Networks joural, ad has served as a Guest Editor for (Elsevier) Ad Hoc Networks joural, ad a Associate Editor for IEEE/ACM Trasactios o Networkig.