Statistical Cost Sharing

Transcription

1 Statstcal Cost Sharng Erc Balkansk Harvard Unversty Umar Syed Google NYC Serge Vasslvtsk Google NYC Abstract We study the cost sharng problem for cooperatve games n stuatons where the cost functon C s not avalable va oracle queres, but must nstead be learned from samples drawn from a dstrbuton, represented as tuples (S, C(S)), for dfferent subsets S of players. We formalze ths approach, whch we call STATISTICAL COST SHARING, and consder the computaton of the core and the Shapley value. Expandng on the work by Balcan et al. [015], we gve precse sample complexty bounds for computng cost shares that satsfy the core property wth hgh probablty for any functon wth a non-empty core. For the Shapley value, whch has never been studed n ths settng, we show that for submodular cost functons wth bounded curvature κ t can be approxmated from samples from the unform dstrbuton to a 1 κ factor, and that the bound s tght. We then defne statstcal analogues of the Shapley axoms, and derve a noton of statstcal Shapley value and that these can be approxmated arbtrarly well from samples from any dstrbuton and for any functon. 1 Introducton The cost sharng problem asks for an equtable way to splt the cost of a servce among all of the partcpants. Formally, there s a cost functon C defned over all subsets S N of a ground set of elements, or players, and the objectve s to farly dvde the cost of the ground set C(N) among the players. Unlke tradtonal learnng problems, the goal here s not to predct the cost of the servce, but rather learn whch ways of dvdng the cost among the players are equtable. Cost sharng s central to cooperatve game theory, and there s a rch lterature developng the key concepts and prncples to reason about ths topc. Two popular cost sharng concepts are the core [Glles, 1959], where no group of players has an ncentve to devate, and the Shapley value [Shapley, 1953], whch s the unque vector of cost shares satsfyng four natural axoms. Whle both the core and the Shapley value are easy to defne, computng them poses addtonal challenges. One obstacle s that the computaton of the cost shares requres knowledge of costs n myrad dfferent scenaros. For example, computng the exact Shapley value requres one to look at the margnal contrbuton of a player over all possble subsets of others. Recent work [Lben-Nowell et al., 01] shows that one can fnd approxmate Shapley values for a restrcted subset of cost functons by lookng at the costs for polynomally many specfcally chosen subsets. In practce, however, another roadblock emerges: one cannot smply query for the cost of an arbtrary subset. Rather, the subsets are passvely observed, and the costs of unobserved subsets are smply unknown. We share the opnon of Balcan et al. [016] that the man dffculty wth usng cost sharng methods n concrete applcatons s the nformaton needed to compute them. 31st Conference on Neural Informaton Processng Systems (NIPS 017), Long Beach, CA, USA.

2 Concretely, consder the followng cost sharng applcatons. Attrbutng Battery Consumpton on Moble Devces. A modern moble phone or tablet s typcally runnng a number of dstnct apps concurrently. In addton to foreground processes, a lot of actvty may be happenng n the background: emal clents may be fetchng new mal, GPS may be actve for geo-fencng applcatons, messagng apps are pollng for new notfcatons, and so on. All of these actvtes consume power; the queston s how much of the total battery consumpton should be attrbuted to each app? Ths problem s non-trval because the operatng system nduces cooperaton between apps to save battery power. For example there s no need to actvate the GPS sensor twce f two dfferent apps request the current locaton almost smultaneously. Understandng Black Box Learnng Deep neural networks are prototypcal examples of black box learnng, and t s almost mpossble to tease out the contrbuton of a partcular feature to the fnal output. Partcularly n stuatons where the features are bnary, cooperatve game theory gves a formal way to analyze and derve these contrbutons. Whle one can evaluate the objectve functon on any subset of features, deep networks are notorous for performng poorly on certan out of sample examples [Goodfellow et al., 014, Szegedy et al., 013], whch may lead to msleadng conclusons when usng tradtonal cost sharng methods. We model these cost sharng questons as follows. Let N be the set of possble players (apps or features), and for a subset S N, let C(S) denote the cost of S. Ths cost represents the total power consumed over a standard perod of tme, or the rewards obtaned by the learner. We are gven ordered pars (S 1, C(S 1 )), (S, C(S )),..., (S m, C(S m )), where each S N s drawn ndependently from some dstrbuton D. The problem of STATISTICAL COST SHARING asks to look for reasonable cost sharng strateges n ths settng. 1.1 Our results We buld on the approach from Balcan et al. [015], whch studed STATISTICAL COST SHARING n the context of the core, and assume that only partal data about the cost functon s observed. The authors showed that cost shares that are lkely to respect the core property can be obtaned for certan restrcted classes of functons. Our man result s an algorthm that generalzes these results for all games where the core s non-empty and we derve sample complexty bounds showng exactly the number of samples requred to compute cost shares (Theorems 1 and ). Whle the man approach of Balcan et al. [015] reled on frst learnng the cost functon and then computng cost shares, we show how to proceed drectly, computng cost shares wthout explctly learnng a good estmate of the cost functon. Ths hgh level dea was ndependently dscovered by Balcan et al. [016]; our approach here greatly mproves the sample complexty bounds, culmnatng n a result logarthmc n the number of players. We also show that approxmately satsfyng the core wth probablty one s mpossble n general (Theorem 3). We then focus on the Shapley value, whch has never been studed n the STATISTICAL COST SHARING context. We obtan a tght 1 κ multplcatve approxmaton of the Shapley values for submodular functons wth bounded curvature κ over the unform dstrbuton (Theorems 4 and 11), but show that they cannot be approxmated by a bounded factor n general, even for the restrcted class of coverage functons, whch are learnable, over the unform dstrbuton (Theorem 5). We also ntroduce a new cost sharng method called data-dependent Shapley value whch s the unque soluton (Theorem 6) satsfyng four natural axoms resemblng the Shapley axoms (Defnton 7), and whch can be approxmated arbtrarly well from samples for any bounded functon and any dstrbuton (Theorem 7). 1. Related work There are two avenues of work whch we buld upon. The frst s the noton of cost sharng n cooperatve games, frst ntroduced by Von Neumann and Morgenstern [1944]. We consder the Shapley value and the core, two popular soluton concepts for cost-sharng n cooperatve games. The Shapley value [Shapley, 1953] s studed n algorthmc mechansm desgn [Anshelevch et al., 008, Balkansk and Snger, 015, Fegenbaum et al., 000, Mouln, 1999]. For applcatons of the Shapley value, see the surveys by Roth [1988] and Wnter [00]. A nave computaton of the Shapley value of a cooperatve game would take exponental tme; recently, methods for effcently approxmatng

3 the Shapley value have been suggested [Bachrach et al., 010, Fatma et al., 008, Lben-Nowell et al., 01, Mann, 1960] for some restrcted settngs. The core, ntroduced by Glles [1959], s another well-studed soluton concept for cooperatve games. Bondareva [1963] and Shapley [1967] characterzed when the core s non-empty. The core has been studed n the context of multple combnatoral games, such as faclty locaton Goemans and Skutella [004] and maxmum flow Deng et al. [1999]. In cases wth no solutons n the core or when t s computatonally hard to fnd one, the balance property has been relaxed to hold approxmately [Devanur et al., 005, Immorlca et al., 008]. In applcatons where players submt bds, cross-monotone cost sharng, a concept stronger than the core that satsfes the group strategy proofness property, has attracted a lot of attenton [Immorlca et al., 008, Jan and Vazran, 00, Mouln and Shenker, 001, Pál and Tardos, 003]. We note that these applcatons are suffcently dfferent from the ones we are studyng n ths work. The second s the recent work n econometrcs and computatonal economcs that ams to estmate crtcal concepts drectly from a lmted data set, and reason about the sample complexty of the computatonal problems. Specfcally, n all of the above papers, the algorthm must be able to query or compute C(S) for an arbtrary set S N. In our work, we are nstead gven a collecton of samples from some dstrbuton; mportantly the algorthm does not know C(S) for sets S that were not sampled. Ths approach was frst ntroduced by Balcan et al. [015], who showed how to compute an approxmate core for some famles of games. Ther man technque s to frst learn the cost functon C from samples and then to use the learned functon to compute cost shares. The authors also showed that there exst games that are not PAC-learnable but that have an approxmate core that can be computed. Independently, n recent follow up work, the authors showed how to extend ther approach to compute a probably approxmate core for all games wth a non-empty core, and gave weak sample complexty bounds [Balcan et al., 016]. We mprove upon ther bounds, showng that a logarthmc number of samples suffces when the spread of the cost functon s bounded. Prelmnares A cooperatve game s defned by an ordered par (N, C), where N s the ground set of elements, also called players, and C : N R 0 s the cost functon mappng each coalton S N to ts cost, C(S). The ground set of sze n = N s called the grand coalton and we denote the elements by N = {1,..., n} = [n]. We assume that C( ) = 0, C(S) 0 for all S N, and that max S C(S) s bounded by a polynomal n n, whch are standard assumptons. We wll slghtly abuse notaton and use C() nstead of C({}) for N when t s clear from the context. We recall three specfc classes of functons. Submodular functons exhbt the property of dmnshng returns: C S () C T () for all S T N and N where C S () s the margnal contrbuton of element to set S,.e., C S () = C(S {}) C(S). Coverage functons are the canoncal example of submodular functons. A functon s coverage f t can be wrtten as C(S) = S T where T U for some unverse U. Fnally, we also consder the smple class of addtve functons, such that C(S) = S C(). A cost allocaton s a vector ψ R n where ψ s the share of element. We call a cost allocaton ψ balanced f N ψ = C(N). Gven a cooperatve game (N, C) the goal n the cost sharng lterature s to fnd desrable" balanced cost allocatons. Most proposals take an axomatc approach, defnng a set of axoms that a cost allocaton should satsfy. These lead to the concepts of Shapley value and the core, whch we defne next. A useful tool to descrbe and compute these cost sharng concepts s permutatons. We denote by σ a unformly random permutaton of N and by S σ< the players before N n permutaton σ..1 The core The core s a balanced cost allocaton where no player has an ncentve to devate from the grand coalton for any subset of players the sum of ther shares does not cover ther collectve cost. Defnton 1. A cost allocaton ψ s n the core of functon C f the followng propertes are satsfed: Balance: N ψ = C(N), Core property: for all S N, S ψ C(S). 3

4 The core s a natural cost sharng concept. For example, n the battery blame scenaro t translates to the followng assurance: No matter what other apps are runnng concurrently, an app s never blamed for more battery consumpton than f t were runnng alone. Gven that app developers are typcally busness compettors, and that a moble devce s battery s a very scarce resource, such a guarantee can rather neatly avod a great deal of fnger-pontng. Unfortunately, for a gven cost functon C the core may not exst (we say the core s empty), or there may be multple (or even nfntely many) cost allocatons n the core. For submodular functons C, the core s guaranteed to exst and one allocaton n the core can be computed n polynomal tme. Specfcally, for any permutaton σ, the cost allocaton ψ such that ψ = C(S σ< {}) C(S σ< ) s n the core.. The Shapley value The Shapley value provdes an alternatve cost sharng method. For a game (N, C) we denote t by φ C, droppng the superscrpt when t s clear from the context. Whle the Shapley value may not satsfy the core property, t satsfes the followng four axoms: Balance: N φ = C(N). Symmetry: For all, j N, f C(S {}) = C(S {j}) for all S N \ {, j} then φ = φ j. Zero element: For all N, f C(S {}) = C(S) for all S N then φ = 0. Addtvty: For two games (N, C 1 ) and (N, C ) wth the same players, but dfferent cost functons C 1 and C, let φ 1 and φ be the respectve cost allocatons. Consder a new game (N, C 1 + C ), and let φ be the cost allocaton for ths game. Then for all elements, N, φ = φ1 + φ. Each of these axoms s natural: balance ensures that the cost of the grand coalton s dstrbuted among all of the players. Symmetry states that two dentcal players should have equal shares. Zero element verfes that a player that adds zero cost to any coalton should have zero share. Fnally, addtvty just confrms that costs combne n a lnear manner. It s surprsng that the set of cost allocatons that satsfes all four axoms s unque. Moreover, the Shapley value φ can be wrtten as the followng summaton: φ = E σ [C(S σ< {}) C(S σ< )] = S N\{} S!(n S 1)! (C(S {}) C(S)). n! Ths expresson s the expected margnal contrbuton C(S {}) C(S) of over a set of players S who arrved before n a random permutaton of N. As the summaton s over exponentally many terms, the Shapley value generally cannot be computed exactly n polynomal tme. However, several samplng approaches have been suggested to approxmate the Shapley value for specfc classes of functons Bachrach et al. [010], Fatma et al. [008], Lben-Nowell et al. [01], Mann [1960]..3 Statstcal cost sharng Wth the sole excepton of Balcan et al. [015], prevous work n cost-sharng crtcally assumes that the algorthm s gven oracle access to C,.e., t can query, or determne, the cost C(S) for any S N. In ths paper, we am to (approxmately) compute the Shapley value and other cost allocatons from samples, wthout oracle access to C, and wth a number of samples that s polynomal n n. Defnton. Consder a cooperatve game wth players N and cost functon C. In the STATISTICAL COST SHARING problem we are gven pars (S 1, C(S 1 )), (S, C(S )),..., (S m, C(S m )) where each S s drawn..d. from a dstrbuton D over N. The goal s to fnd a cost allocaton ψ R n. In what follows we wll often refer to an ndvdual (S, C(S)) par as a sample. It s temptng to reduce STATISTICAL COST SHARING to classcal cost sharng by smply collectng enough samples to use known algorthms. For example, Lben-Nowell et al. [01] showed how to approxmate the Shapley value wth polynomally many queres C(S). However, f the dstrbuton D s not algned wth these specfc queres, whch s the case even for the unform dstrbuton, emulatng these 4

5 algorthms n our settng requres exponentally many samples. Balcan et al. [015] showed how to nstead frst learn an approxmaton to C from the gven samples and then compute cost shares for the learned functon, but ther results hold only for a lmted number of games and cost functons C. We show that a more powerful approach s to compute cost shares drectly from the data, wthout explctly learnng the cost functon frst. 3 Approxmatng the Core from Samples In ths secton, we consder the problem of fndng cost allocatons from samples that satsfy relaxatons of the core. A natural approach to ths problem s to frst learn the underlyng model, C, from the data and to then compute a cost allocaton for the learned functon. As shown n Balcan et al. [015], ths approach works f C s PAC-learnable, but there exst functons C that are not PAClearnable and for whch a cost allocaton that approxmately satsfes the core can stll be computed. The man result of ths secton shows that a cost allocaton that approxmates the core property can be computed from samples for any functon wth a non-empty core. We frst gve a sample complexty bound that s lnear n the number n of players, a result whch was ndependently dscovered by Balcan et al. [016]. Wth a more ntrcate analyss, we then mprove ths sample complexty to be logarthmc n n, but at the cost of a weaker relaxaton. Our algorthm, whch runs n polynomal tme, drectly computes a cost allocaton that emprcally satsfes the core property,.e., t satsfes the core property on all of the samples. We argue, by leveragng VC-dmenson and Rademacher complexty-based generalzaton bounds, that the same cost allocaton wll lkely satsfy the core property on newly drawn samples as well. We also propose a stronger noton of the approxmate core, and prove that t cannot be computed by any algorthm. Ths hardness result s nformaton theoretc and s not due to runnng tme lmtatons. The proofs n ths secton are deferred to Appendx B. We begn by defnng three notons of the approxmate core: the probably approxmately stable (Balcan et al. [016]), mostly approxmately stable, and probably mostly approxmately stable cores. Defnton 3. Gven δ, ɛ > 0, a cost allocaton ψ such that N ψ = C(N) s n the probably approxmately stable core f Pr S D [ S ψ C(S) ] 1 δ for all D (see Balcan et al. [015]), the mostly approxmately stable core over D f (1 ɛ) S ψ C(S) for all S N, the probably mostly approxmately stable core f Pr S D [ (1 ɛ) S ψ C(S) ] 1 δ for all D, For each of these notons, our goal s to effcently compute a cost allocaton n the approxmate core, n the followng sense. Defnton 4. A cost allocaton ψ s effcently computable for the class of functons C over dstrbuton D, f for all C C and any, δ, ɛ > 0, gven C(N) and m = poly(n, 1 /, 1 /δ, 1 /ɛ) samples (S j, C(S j )) wth each S j drawn..d. from dstrbuton D, there exsts an algorthm that computes ψ wth probablty at least 1 over both the samples and the choces of the algorthm. We refer to the number of samples requred to compute approxmate cores as the sample complexty of the algorthm. We frst present our result for computng a probably approxmately stable core wth sample complexty that s lnear n the number of players, whch was also ndependently dscovered by Balcan et al. [016]. Theorem 1. The class of functons wth a non-empty core has cost shares n the probably approxmately stable core that are effcently computable. The sample complexty s ( ) n + log(1/ ) O. δ The full proof of Theorem 1 s n Appendx B, and can be summarzed as follows: We defne a class of halfspaces whch contans the core. Snce we assume that C has a non-empty core, there exsts a cost allocaton ψ n ths class of halfspaces that satsfes both the core property on all the samples and the balance property. Gven a set of samples, such a cost allocaton can be computed wth a smple lnear program. We then use the VC-dmenson of the class of halfspaces to show that the performance on the samples generalzes well to the performance on the dstrbuton D. 5

6 We next show that the sample complexty dependence on n can be mproved from lnear to logarthmc f we relax the goal from computng a cost allocaton n the probably approxmately stable core to computng one n the probably mostly approxmately stable core nstead. The sample complexty of our algorthm also depends on the spread of the functon C, defned as max S C(S) mn S C(S) (we assume mn S C(S) > 0). Theorem. The class of functons wth a non-empty core has cost allocatons n the probably mostly approxmately stable core that are effcently computable wth sample complexty ( ) 1 ɛ ( 18τ(C) log(n) + 8τ(C) log(/ ) ) ( (τ(c) ) = O (log n + log(1/ ))). ɛδ ɛδ where τ(c) = max S C(S) mn S C(S) s the spread of C. The full proof of Theorem s n Appendx B. Its man steps are: 1. We fnd a cost allocaton whch satsfes the core property on all samples, restrctng the search to cost allocatons wth bounded l 1 -norm. Such a cost allocaton can be found effcently snce the space of such cost allocatons s convex.. The analyss begns by boundng the l 1 -norm of any vector n the core (Lemma 3). Combned wth the assumpton that the core s non-empty, ths mples that a cost allocaton ψ satsfyng the prevous condtons exsts. 3. Let [x] + denote the functon x max(x, 0). Consder the followng loss" functon: [ S ψ ] 1 C(S) Ths loss functon s convenent snce t s equal to 0 f and only f the core property s satsfed for S and t s 1-Lpschtz, whch s used n the next step. 4. Next, we bound the dfference between the emprcal loss and the expected loss for all ψ wth a known result usng the Rademacher complexty of lnear predctors wth low l 1 norm over ρ-lpschtz loss functons (Theorem 10). 5. Fnally, gven ψ whch approxmately satsfes the core property n expectaton, we show that ψ s n the probably mostly approxmately stable core by Markov s nequalty (Lemma 4). Snce we obtaned a probably mostly approxmately stable core, a natural queston s f t s possble to compute cost allocatons that are mostly approxmately stable over natural dstrbutons. The answer s negatve n general: even for the restrcted class of monotone submodular functons, whch always have a soluton n the core, the core cannot be mostly approxmated from samples, even over the unform dstrbuton. The full proof of ths mpossblty theorem s n Appendx B. Theorem 3. Cost allocatons ψ n the (1/ + ɛ)-mostly approxmately stable core,.e., such that for all S, ( ) 1 + ɛ ψ C(S), S cannot be computed for monotone submodular functons over the unform dstrbuton, for any constant ɛ > 0. 4 Approxmatng the Shapley Value from Samples We turn our attenton to the STATISTICAL COST SHARING problem n the context of the Shapley value. Snce the Shapley value exsts and s unque for all functons, a natural relaxaton s to smply approxmate ths value from samples. The dstrbutons we consder n ths secton are the unform dstrbuton, and more generally product dstrbutons, whch are the standard dstrbutons studed n the learnng lterature for combnatoral functons Balcan and Harvey [011], Balcan et al. [01], Feldman and Kothar [014], Feldman and Vondrak [014]. It s easy to see that we need some restrctons on the dstrbuton D (for example, f the empty set f drawn wth probablty one, the Shapley value cannot be approxmated). + 6

7 For submodular functons wth bounded curvature, we prove approxmaton bounds when samples are drawn from the unform or a bounded product dstrbuton, and also show that the bound for the unform dstrbuton s tght. However, we show that the Shapley value cannot be approxmated from samples even for coverage functons (whch are a specal case of submodular functons) and the unform dstrbuton. Snce coverage functons are learnable from samples, ths mples the counter-ntutve observaton that learnablty does not mply that the Shapley value s approxmable from samples. We defer the full proofs to Appendx C. Defnton 5. An algorthm α-approxmates, α (0, 1], the Shapley value of cost functons C over dstrbuton D, f, for all C C and all δ > 0, gven poly(n, 1 /δ, 1 /1 α) samples from D, t computes Shapley value estmates φ C such that αφ φ 1 α φ for all N such that φ 1/ poly(n) 1 wth probablty at least 1 δ over both the samples and the choces made by the algorthm. We consder submodular functons wth bounded curvature, a common assumpton n the submodular maxmzaton lterature Iyer and Blmes [013], Iyer et al. [013], Svrdenko et al. [015], Vondrák [010]. Intutvely, the curvature of a submodular functon bounds by how much the margnal contrbuton of an element can decrease. Ths property s useful snce the Shapley value of an element can be wrtten as a weghted sum of ts margnal contrbutons over all sets. Defnton 6. A monotone submodular functon C has curvature κ [0, 1] f C N\{} () (1 κ)c() for all N. Ths curvature s bounded f κ < 1. An mmedate consequence of ths defnton s that C S () (1 κ)c T () for all S, T such that S T by monotoncty and submodularty. The man tool used s estmates ṽ of expected margnal contrbutons v = E S D S [C S ()] where ṽ = avg(s ) avg(s ) s the dfference between the average value of samples contanng and the average value of samples not contanng. Theorem 4. Monotone submodular functons wth bounded curvature κ have Shapley value that s 1 κ ɛ approxmable from samples over the unform dstrbuton, whch s tght, and 1 κ ɛ approxmable over any bounded product dstrbuton for any constant ɛ > 0. Consder the algorthm whch computes φ = ṽ. Note that φ = E σ [C(A σ< {}) C(A σ< )] (1 κ)v > 1 κ 1+ɛ ṽ > (1 κ ɛ)ṽ where the frst nequalty s by curvature and the second by Lemma 5 whch shows that the estmates ṽ of v are arbtrarly good. The other drecton follows smlarly. The 1 κ result s the man techncal component of the upper bound. We descrbe two man steps: 1. The expected margnal contrbuton E S U S, S =j [C S ()] of to a unformly random set S of sze j s decreasng n j, whch s by submodularty.. Snce a unformly random set has sze concentrated close to n/, ths mples that roughly half of the terms n the summaton φ = ( n 1 j=0 E S U j S[C S ()])/n are greater than v and the other half of the terms are smaller. For the tght lower bound, we show that there exsts two functons that cannot be dstngushed from samples w.h.p. and that have an element wth Shapley value whch dffers by an α factor. We show that the Shapley value of coverage (and submodular) functons are not approxmable from samples n general, even though coverage functons are PMAC-learnable ( Balcan and Harvey [011]) from samples over any dstrbuton Badandyuru et al. [01]. Theorem 5. There exsts no constant α > 0 such that coverage functons have Shapley value that s α-approxmable from samples over the unform dstrbuton. 5 Data Dependent Shapley Value The general mpossblty result for computng the Shapley value from samples arses from the fact that the concept was geared towards the query model, where the algorthm can ask for the cost of any set S N. In ths secton, we develop an analogue that s dstrbuton-dependent. We denote t by φ C,D wth respect to both C and D. We defne four natural dstrbuton-dependent axoms resemblng 1 See Appendx C for general defnton. 7

8 the Shapley value axoms, and then prove that our proposed value s the unque soluton satsfyng them. Ths value can be approxmated arbtrarly well n the statstcal model for all functons. The proofs are deferred to Appendx D. We start by statng the four axoms. Defnton 7. The data-dependent axoms for cost sharng functons are: Balance: N φd = E S D [C(S)], Symmetry: for all and j, f Pr S D [ S {, j} = 1] = 0 then φ D = φ D j, Zero element: for all, f Pr S D [ S] = 0 then φ D = 0, Addtvty: for all, f D 1, D, α, and β such that α + β = 1, φ αd1+βd where Pr [S αd 1 + βd ] = α Pr [S D 1 ] + β Pr [S D ]. = αφ D1 + βφ D The smlarty to the orgnal Shapley value axoms s readly apparent. The man dstncton s that we expect these to hold wth regard to D, whch captures the frequency wth whch dfferent coaltons S occur. Interpretng the axoms one by one, the balance property ensures that the expected cost s always accounted for. The symmetry axom states that f two elements always occur together, they should have the same share, snce they are ndstngushable. If an element s never observed, then t should have zero share. Fnally costs should combne n a lnear manner accordng to the dstrbuton. The data-dependent Shapley value s φ D := S : S Pr [S D] C(S). S Informally, for all set S, the cost C(S) s dvded equally between all elements n S and s weghted wth the probablty that S occurs accordng to D. The man appeal of ths cost allocaton s the followng theorem. Theorem 6. The data-dependent Shapley value s the unque value satsfyng the four data-dependent axoms. The data-dependent Shapley value can be approxmated from samples wth the followng emprcal data-dependent Shapley value: φ D = 1 C(S j ). m S j S j : S j These estmates are arbtrarly good wth arbtrarly hgh probablty. Theorem 7. The emprcal data-dependent Shapley value approxmates the data-dependent Shapley value arbtrarly well,.e., φ D φ D < ɛ wth poly(n, 1/ɛ, 1/δ) samples and wth probablty at least 1 δ for any δ, ɛ > 0. 6 Dscusson and Future Work We follow a recent lne of work that studes classcal algorthmc problems from a statstcal perspectve, where the nput s restrcted to a collecton of samples. Our results fall nto two categores, we gve results for approxmatng the Shapley value and the core and propose new cost sharng concepts that are talored for the statstcal framework. We use technques from multple felds that encompass statstcal machne learnng, combnatoral optmzaton, and, of course, cost sharng. The cost sharng lterature beng very rch, the number of drectons for future work are consderable. Obvous avenues nclude studyng other cost sharng methods n ths statstcal framework, consderng other classes of functons to approxmate known methods, and mprovng the sample complexty of prevous algorthms. More conceptually, an exctng modelng queston arses when desgnng desrable" axoms from data. Tradtonally these axoms only depended on the cost functon, whereas n ths model they can depend on both the cost functon and the dstrbuton, provdng an nterestng nterplay. 8

9 References Ellot Anshelevch, Anrban Dasgupta, Jon Klenberg, Eva Tardos, Tom Wexler, and Tm Roughgarden. The prce of stablty for network desgn wth far cost allocaton. SIAM Journal on Computng, 38(4): , 008. Yoram Bachrach, Evangelos Markaks, Ezra Resnck, Arel D Procacca, Jeffrey S Rosenschen, and Amn Saber. Approxmatng power ndces: theoretcal and emprcal analyss. Autonomous Agents and Mult-Agent Systems, 0():105 1, 010. Ashwnkumar Badandyuru, Shahar Dobznsk, Hu Fu, Robert Klenberg, Noam Nsan, and Tm Roughgarden. Sketchng valuaton functons. In Proceedngs of the twenty-thrd annual ACM- SIAM symposum on Dscrete Algorthms, pages Socety for Industral and Appled Mathematcs, 01. Mara-Florna Balcan and Ncholas JA Harvey. Learnng submodular functons. In Proceedngs of the forty-thrd annual ACM symposum on Theory of computng, pages ACM, 011. Mara-Florna Balcan, Florn Constantn, Satoru Iwata, and Le Wang. Learnng valuaton functons. In COLT, volume 3, pages 4 1, 01. Mara-Florna Balcan, Arel D. Procacca, and Yar Zck. Learnng cooperatve games. In Proceedngs of the Twenty-Fourth Internatonal Jont Conference on Artfcal Intellgence, IJCAI 015, Buenos Ares, Argentna, July 5-31, 015, pages , 015. Mara-Florna Balcan, Arel D Procacca, and Yar Zck. Learnng cooperatve games. arxv preprnt arxv: v, 016. Erc Balkansk and Yaron Snger. Mechansms for far attrbuton. In Proceedngs of the Sxteenth ACM Conference on Economcs and Computaton, pages ACM, 015. Olga N Bondareva. Some applcatons of lnear programmng methods to the theory of cooperatve games. Problemy kbernetk, 10: , Xaote Deng, Toshhde Ibarak, and Hrosh Nagamoch. Algorthmc aspects of the core of combnatoral optmzaton games. Mathematcs of Operatons Research, 4(3): , Nkhl R Devanur, Mlena Mhal, and Vjay V Vazran. Strategyproof cost-sharng mechansms for set cover and faclty locaton games. Decson Support Systems, 39(1):11, 005. Shaheen S Fatma, Mchael Wooldrdge, and Ncholas R Jennngs. A lnear approxmaton method for the shapley value. Artfcal Intellgence, 17(14): , 008. Joan Fegenbaum, Chrstos Papadmtrou, and Scott Shenker. Sharng the cost of multcast transmssons (prelmnary verson). In Proceedngs of the thrty-second annual ACM symposum on Theory of computng, pages ACM, 000. Vtaly Feldman and Pravesh Kothar. Learnng coverage functons and prvate release of margnals. In COLT, pages , 014. Vtaly Feldman and Jan Vondrak. Optmal bounds on approxmaton of submodular and xos functons by juntas. In Informaton Theory and Applcatons Workshop (ITA), 014, pages IEEE, 014. Donald B Glles. Solutons to general non-zero-sum games. Contrbutons to the Theory of Games, 4(40):47 85, Mchel X Goemans and Martn Skutella. Cooperatve faclty locaton games. Journal of Algorthms, 50():194 14, 004. Ian J. Goodfellow, Jonathon Shlens, and Chrstan Szegedy. Explanng and harnessng adversaral examples. CoRR, abs/ , 014. URL Ncole Immorlca, Mohammad Mahdan, and Vahab S Mrrokn. Lmtatons of cross-monotonc cost-sharng schemes. ACM Transactons on Algorthms (TALG), 4():4,

10 Rshabh K Iyer and Jeff A Blmes. Submodular optmzaton wth submodular cover and submodular knapsack constrants. In Advances n Neural Informaton Processng Systems, pages , 013. Rshabh K Iyer, Stefane Jegelka, and Jeff A Blmes. Curvature and optmal algorthms for learnng and mnmzng submodular functons. In Advances n Neural Informaton Processng Systems, pages , 013. Kamal Jan and Vjay V Vazran. Equtable cost allocatons va prmal-dual-type algorthms. In Proceedngs of the thry-fourth annual ACM symposum on Theory of computng, pages ACM, 00. Davd Lben-Nowell, Alexa Sharp, Tom Wexler, and Kevn Woods. Computng shapley value n supermodular coaltonal games. In Internatonal Computng and Combnatorcs Conference, pages Sprnger, 01. Irwn Mann. Values of large games, IV: Evaluatng the electoral college by Montecarlo technques. Rand Corporaton, Hervé Mouln. Incremental cost sharng: Characterzaton by coalton strategy-proofness. Socal Choce and Welfare, 16():79 30, Hervé Mouln and Scott Shenker. Strategyproof sharng of submodular costs: budget balance versus effcency. Economc Theory, 18(3): , 001. Martn Pál and Éva Tardos. Group strategy proof mechansms va prmal-dual algorthms. In Foundatons of Computer Scence, 003. Proceedngs. 44th Annual IEEE Symposum on, pages IEEE, 003. Alvn E Roth. The Shapley value: essays n honor of Lloyd S. Shapley. Cambrdge Unversty Press, Sha Shalev-Shwartz and Sha Ben-Davd. algorthms Understandng machne learnng: From theory to Lloyd S Shapley. On balanced sets and cores. Naval research logstcs quarterly, 14(4): , LS Shapley. A value for n-person games Maxm Svrdenko, Jan Vondrák, and Justn Ward. Optmal approxmaton for submodular and supermodular optmzaton wth bounded curvature. In Proceedngs of the Twenty-Sxth Annual ACM-SIAM Symposum on Dscrete Algorthms, pages SIAM, 015. Chrstan Szegedy, Wojcech Zaremba, Ilya Sutskever, Joan Bruna, Dumtru Erhan, Ian J. Goodfellow, and Rob Fergus. Intrgung propertes of neural networks. CoRR, abs/ , 013. URL John Von Neumann and Oskar Morgenstern. Theory of games and economc behavor Jan Vondrák. Submodularty and curvature: the optmal algorthm. RIMS Kokyuroku Bessatsu B, 3: 53 66, 010. Eyal Wnter. The shapley value. Handbook of game theory wth economc applcatons, 3:05 054,

11 Appendx A Concentraton Bounds Lemma 1 (Chernoff Bound). Let X 1,..., X n be ndependent ndcator random varables wth values n {0, 1}. Let X = n =1 X and µ = E[X]. For 0 < δ < 1, Pr [X (1 δ)µ] e µδ / and Pr [X (1 + δ)µ] e µδ /3. Lemma (Hoeffdng s nequalty). Let X 1,..., X n be ndependent random varables wth values n [0, b]. Let X = 1 m m =1 X and µ = E[X]. Then for every 0 < ɛ < 1, Pr [ X E[X] ɛ] e mɛ /b. B Mssng Defntons and Analyss from Secton 3 We frst show the result wth lnear sample complexty, then the result wth logarthmc sample complexty, and fnally the mpossblty result. B.1 Lnear sample complexty for probably approxmately stable core We frst state the generalzaton error obtaned for a class of functons wth VC-dmenson d. Theorem 8 (Shalev-Shwartz and Ben-Davd [014], Theorem 6.8). Let H be a hypothess class of functons from a doman X to { 1, 1} and f : X { 1, 1} be some correct" functon. Assume that H has VC-dmenson d. Then, there s an absolute constant c such that wth m c(d + log(1/ ))/δ..d. samples x 1,..., x m D, Pr [h(x) f(x)] 1 m 1 x D m h(x ) f(x ) δ =1 for all h H, wth probablty 1 over the samples. We use a specal case of the class of halfspaces, for whch we know the VC-dmenson. Theorem 9 (Shalev-Shwartz and Ben-Davd [014], Theorem 9.). The class of functons {x sgn(w x) : w R n } has VC-dmenson n. We frst defne a class of functons that contans the core, and prove that t has low VC-dmenson. Gven a sample S, defne x S such that x S = 1 S for [n] and x S n+1 = C(S). Note that sgn ( n =1 ψ x S n+1) xs = 1 f the core property s satsfed for sample S. We now bound the VC-dmenson of ths hypothess class of functons nduced by cost allocatons ψ. Corollary 1. The class of functons H core = {x sgn( n =1 ψ x x n+1 ) : ψ R n, ψ = C(N)} has VC-dmenson at most n + 1. Proof. We combne the observaton that {x sgn( n =1 w x x n+1 ) : w R n, w = C(N)} {x sgn(w x) : w R n+1 } wth the well-known fact that the VC-dmenson of H s at most the VC-dmenson of H for H H. It remans to show how to optmze over functons n ths class. Theorem 1. The class of functons wth a non-empty core has cost shares n the probably approxmately stable core that are effcently computable. The sample complexty s ( ) n + log(1/ ) O. δ Proof. Let ψ be a cost allocaton whch satsfes both the core property on all the samples and the balance property,.e., S ψ C(S) for all samples S and N ψ = C(N). Note that such a 11

12 cost allocaton exsts snce we assume that C has a non-empty core. Gven the set of samples, t can be computed wth a smple lnear program. We argue that ψ s probably approxmately stable. Defne h(x) = sgn ( n =1 ψ x S n+1) xs and f(x) = 1 for all x. Snce the core property s 1 m satsfed on all the samples, m =1 1 h(x) f(x) = 0. Thus, by Theorem 8, [ ] [ ( n ) ] Pr ψ C(S) = 1 Pr sgn ψ x S S D x S S x S n+1 1 :S D =1 [ ( = 1 Pr h x S ) f ( x S)] x S :S D [ ( = 1 Pr h x S ) f ( x S)] 1 m 1 x S :S D m h(x S ) f(x S ) 1 δ wth O((n + log(1/ ))/δ) samples. B. Logarthmc sample complexty for probably mostly approxmately stable cores The followng result follows from the Rademacher complexty of lnear classes. Theorem 10 (Shalev-Shwartz and Ben-Davd [014], Theorem 6.15). Suppose that D s a dstrbuton over X R such that wth probablty 1 we have that x R. Let H = {w R d : w 1 B} and let l : H (X R) R be a loss functon of the form l(w, (x, y)) = φ(w x, y) such that for all y R, a φ(a, y) s an ρ-lpschtz functon and such that max a [ BR,BR] φ(a, y) c. Then, for all w H and any (0, 1), wth probablty of at least 1 over m..d. samples (x 1, y 1 ),..., (x m, y m ) from D, E [l(w, (x, y))] 1 m log(d) log(/ ) l(w, (x, y )) + ρbr + c. (x,y) D m m m =1 We frst bound the l 1 norm of vectors n the core to bound the space of lnear functons that we search over. Lemma 3. Assume that ψ s a vector n the core, then ψ 1 max S C(S). Proof. Fx some vector ψ n the core. Let A be the set of elements such that ψ 0 and B be the remanng elements. Frst note that ψ C(A) max C(S) S A where the frst nequalty s by the core property. Next, note that 0 C(N) = A =1 ψ + ψ max C(S) + ψ S B B where the equalty s by the balance property, so B ψ max S C(S). Thus, ψ 1 = A ψ B ψ max S C(S) + max C(S). S We can thus focus on bounded cost allocatons ψ H where { } H := ψ : ψ R n, ψ 1 max C(S). S The next lemma shows that f the core property approxmately holds n expectaton, then t s lkely to approxmately hold. 1

13 Lemma 4. For any 0 < ɛ, δ < 1 and cost allocaton ψ, [ [ S E ψ ] ] [ 1 ɛδ S D C(S) + 1 ɛ Pr (1 ɛ) ] ψ C(S) S D S 1 δ. Proof. For any a > 0 and nonnegatve random varable X, by Markov s nequalty we have Pr[X a] 1 E[X]/a. By lettng a = ɛ/(1 ɛ), X = [ ( S ψ )/C(S) 1 ], and observng that + [ S ψ ] 1 ɛ C(S) + 1 ɛ S ψ 1 ɛ C(S) 1 ɛ (1 ɛ) ψ C(S), S we obtan Pr S D [ (1 ɛ) S ψ C(S) ] 1 δ. Combnng Theorem 10, Lemma 3, and Lemma 4, we obtan the man result. Theorem. The class of functons wth a non-empty core has cost allocatons n the probably mostly approxmately stable core that are effcently computable wth sample complexty ( ) 1 ɛ ( 18τ(C) log(n) + 8τ(C) log(/ ) ) ( (τ(c) ) = O (log n + log(1/ ))). ɛδ ɛδ where τ(c) = max S C(S) mn S C(S) s the spread of C. Proof. Fx C C. Suppose we are gven m samples from D. We pck ψ H such that core property holds on all the samples and such that the balance property holds ( N ψ = C(N)). Ths cost allocaton ψ can be found effcently snce the collecton of such ψ s a convex set. By the assumpton that C has at least one vector n the core and by Lemma 3, such a ψ exsts. Gven S D, defne x S such that x S = 1 S /C(S). Fx y = 1. Defne the loss functon l as follows, l ( ψ, ( x S, y )) := [ ψ x S y ] [ = S ψ ] 1 + C(S) We wsh to use Theorem 10 wth R = 1/ mn S C(S), B = max S C(S), φ(a, y) = [a y] +, ρ = 1, and c = τ(c). We verfy that all the condtons hold. Frst note that wthout loss of generalty, samples where S = can be gnored, so x S 1/ mn S C(S) for all S. Next, ψ 1 max S C(S) for ψ H by defnton of H. The loss functon l s of the form l(ψ, (x, y)) = φ(ψ x, y) = [ψ x y] + such that a φ(a, y) = [a y] + s an 1-Lpschtz functon and such that max a [ BR,BR] φ(a, y) max S C(S) / mn S C(S) = τ(c). In addton, note that 1 m m l ( ψ, ( x S, 1 )) = 0 snce the core property holds on all the samples. Thus, by Theorem 10, [ S E ψ S D C(S) =1 ] [ ( 1 = E l ψ, ( x S, 1 ))] log(n) log(/ ) 4τ(C) + τ(c). + x S :S D m m Choose any ɛ, δ > 0. If the number of samples m s chosen as n the statement of the theorem, then the rghthand sde of the above nequalty wll be less than ɛδ 1 ɛ. Thus by Lemma 4, [ ] whch completes the proof. Pr S D (1 ɛ) S ψ C(S) 1 δ, + 13

14 B.3 Hardness of mostly approxmately stable core We now gve the mssng analyss for the mpossblty result of approxmatng the core. Theorem 3. Cost allocatons ψ n the (1/ + ɛ)-mostly approxmately stable core,.e., such that for all S, ( ) 1 + ɛ ψ C(S), S cannot be computed for monotone submodular functons over the unform dstrbuton, for any constant ɛ > 0. Proof. The ground set of elements s parttoned n ɛ 1 sets A 1,... A ɛ 1 of sze ɛn for some small constant ɛ > 0. Let C = {C A : [ɛ 1 ]} where ( C A (S) = (N \ A) S + mn A S, (1 + ɛ) ɛn ). The expected number of elements of A n a sample S from the unform dstrbuton s A / = ɛn/, so by the Chernoff bound [ Pr A S (1 + ɛ) nɛ ] e ɛ n 6, Thus, by a unon bound, C A (S) = S over all and all samples S wth probablty 1 O(e n ) and we henceforth assume ths s the case. It s therefore mpossble to learn any nformaton about the partton A 1,... A ɛ 1 from samples. Any cost allocaton ψ computed by an algorthm gven samples from C A s thus ndependent of. Next, consder such a cost allocaton ψ ndependent of satsfyng the balance property. There exsts A such that j A ψ j > (1 ɛ)nɛ snce j N ψ j = C(N) > (1 ɛ)n by the balance property. In addton, C A (A ) = (1 + ɛ)ɛn/. We obtan j A ψ j > (1 ɛ)nɛ = (1 ɛ) (1 + ɛ) CA (A ). Thus, the core property s volated by a 1/ + ɛ factor for set A and functon C A, and for any constant ɛ > 0 by pckng ɛ suffcently small. C Mssng Analyss from Secton 4 We gve a complete defnton of approxmate Shapley values. Defnton 8. An algorthm α-approxmates, α (0, 1], the Shapley value of a famly of cost functons C over dstrbuton D, f, for all C C and all δ > 0, gven poly(n, 1 /δ, 1 /1 α) samples from D, t computes Shapley value estmates φ C such that for postve bounded Shapley value, f φ 1/ poly(n), then αφ φ 1 α φ ; negatve bounded Shapley value, f φ 1/ poly(n), then 1 α φ φ αφ ; small Shapley value, f φ < 1/ poly(n), then φ φ = o(1). for all N wth probablty at least 1 δ over both the samples and the choces made by the algorthm. Next, we show that the estmates ṽ of the expected margnal contrbuton v = E S D S [C S ()] are arbtrarly good. Recall that S and S are the collectons of samples contanng element and not contanng t respectvely and that avg(s) = ( S S C(S))/ S s the average value of the samples n S. Let v = C() and ṽ = avg(s ) avg(s ). 14

15 Lemma 5. The expected margnal contrbuton of an element to a random set from a bounded product dstrbuton D not contanng s estmated arbtrarly well by ṽ,.e., for all N and gven poly(n, 1 /δ, 1 /ɛ) samples, (1 ɛ)v ṽ (1 + ɛ)v f v 1/ poly(n) v ṽ ɛ f v < 1/ poly(n) (1 + ɛ)v ṽ (1 ɛ)v f v 1/ poly(n) wth probablty at least 1 δ for any δ > 0. Proof. Note that v = E [C S()] = S D S E S D S [C(S )] E [C(S)] = E [C(S)] E [C(S)]. S D S S D S S D S where the second equalty s snce D s a product dstrbuton. We also have that E[avg(S )] = E S D S [C(S)] and E[avg(S )] = E S D S [C(S)]. Snce margnal probabltes of the product dstrbutons are assumed to be bounded from below and above by 1/ poly(n) and 1 1/ poly(n) respectvely, S = m/ poly(n) and S = m/ poly(n) for all by Chernoff bound. In addton, max S C(S) s assumed to be bounded by poly(n). So by Hoeffdng s nequalty, for 0 < ɛ < /v and Thus, Pr ( avg(s ) E Pr ( avg(s ) E S D S [C(S)] S D S [C(S)] ) v ɛ/ ) v ɛ/ Pr( ṽ v v ɛ) e m( v ɛ) poly(n) e m( v ɛ) poly(n), e m( v ɛ) poly(n). and, ether (1 ɛ)v ṽ (1 + ɛ)v f v > 0 or (1 + ɛ)v ṽ (1 ɛ)v f v < 0, wth probablty at least 1 e m( v ɛ) poly(n). If v < 1/ poly(n), we obtan v ṽ < ɛ wth a smlar analyss wthout any assumpton on v. Otherwse, the bounds on the estmaton hold wth probablty at least 1 e mɛ poly(n). Theorem 4. Monotone submodular functons wth bounded curvature κ have Shapley value that s 1 κ ɛ approxmable from samples over the unform dstrbuton, whch s tght, and 1 κ ɛ approxmable over any bounded product dstrbuton for any constant ɛ > 0. It remans to mprove the bound from 1 κ ɛ to 1 κ ɛ for the unform dstrbuton. Let φ = κ 1 κ ṽ be the estmated Shapley value. Denote by U j the unform dstrbuton over all sets of sze j, so φ = E σ [C Aσ< ()] = 1 n n 1 j=0 E [C S()]. S U j S The man dea to mprove the loss from 1 κ to 1 κ s to observe that v can be a factor 1 κ away from the contrbuton E S Ujl S[C S ()] of j to sets of low szes j l L := (1 ɛ ) n/ or 1 κ away from ts contrbuton E S Ujh S[C S ()] to sets of hgh szes j h H := (1 + ɛ ) n/, but not both, otherwse the curvature property would be volated as llustrated n Fgure 1. The followng lemma shows that E S Uj S[C S ()] s decreasng n j by submodularty. Lemma 6. Let C be a submodular functon, then for all j {0,..., n 1} and all N, E [C S()] E [C S()] S U j S S U j+1 S 15

16 Fgure 1: The expected margnal contrbuton E S Uj S[C S ()] of an element to a set of sze j as a functon of j. The curvature property mples that any two ponts are at most a factor 1 κ from each other. Lemma 6 shows that t s decreasng. Lemma 7 shows that the expected margnal contrbuton v of to a unformly random set s approxmately between E S UL S[C S ()] and E S UH S[C S ()]. The Shapley value of s the average value of ths expected margnal contrbuton over all ntegers j {0,..., n 1}. Proof. By submodularty, S: S =j, S C S () S: S =j, S In addton, observe that by countng n two ways, S: S =j, S S {} By combnng the two prevous observatons, 1 n j 1 C S { }() = (j + 1) E [C 1 S()] = S U j S {S : S = j, S} 1 = ( n 1 j j + 1 ) n j 1 S {} S: S =j+1, S S: S =j, S S: S =j+1, S 1 {S : S = j + 1, S} = E S U j+1 S [C S()] C S { }(). C S () C S () S: S =j+1, S C S (). C S () The next lemma shows that for j slghtly lower than n/, the expected margnal contrbuton v of element to a random set cannot be much larger than E S Uj S[C S ()], and smlarly for j slghtly larger than n/, t cannot be much smaller. Lemma 7. Let C be a submodular functon, then for all N, (1 + ɛn ) e 6 E 1 κ [C S()] v ( 1 e ) ɛn 6 E [C S()]. S U L S S U H S 16

17 Proof. By Chernoff bound, L S and S H wth probablty at least 1 e ɛn 6 each for S drawn from the unform dstrbuton. Denote the unform dstrbuton over all sets by U. So, v = n 1 Pr S U S j=0 H Pr S U S j=0 Pr S U S [ S = j] E [C S()] S U j S [ S = j] E [C S()] S U j S [ S H] E [C S()] Lemma 6 S U H S (1 e ɛn 6 ) E [C S()]. S U H S Smlarly, v = n 1 Pr S U S j=0 Pr S U S e ɛn 6 [ S = j] E [C S()] S U j S [ S < L] C() + Pr S U S 1 1 κ E [C S()] + S U L S [ S L] E [C S()] Lemma 6 S U L S E [C S()] S U L S curvature We are now ready to prove Theorem 4: n 1 φ = 1 E n [C S()] S U j S j=0 = 1 H 1 n 1 + ɛ j=0 E [C S()] + S U j S n 1 =H E [C S()] S U j S C() + 1 E S U H S [C S()] Lemma ɛ (1 κ) v 1 + (1 e ɛ n 6 ) v curvature and Lemma 7 ( ) κ (1 κ) + c 1ɛ v ( ) κ (1 κ) + c ɛ ṽ Lemma 5 ( ) 1 = φ 1 κ c3 ɛ defnton of φ 17

18 for some constants c 1, c, c 3 and let ɛ = ɛ/c 3 to obtan the desred result for any ɛ. Smlarly, φ = 1 n L E [C S()] + 1 S U j S n j=0 1 ɛ 1 ɛ ( 1 + ɛ n e 6 1 κ ( κ n 1 j=l+1 E [C S()] S U j S E [C S()] + 1 (C(N) C(N \ {}) Lemma 6 S U L S ) v + 1 κ v Lemma 7 and curvature ) c 1 ɛ v ( ) κ c ɛ ṽ Lemma 5 = ( 1 κ c 3 ɛ ) φ defnton of φ We show that ths approxmaton s optmal. We begn wth a general lemma to derve nformaton theoretc napproxmablty results for the Shapley value. Ths lemma shows that f there exsts two functons n C that cannot be dstngushed from samples wth hgh probablty and that have an element wth Shapley value whch dffers by an α factor, then C does not have a Shapley value that s α-approxmable from samples. Lemma 8. Consder a famly of cost functons C, a constant α (0, 1), and assume there exst C 1, C C such that: Indstngushable from samples. Wth probablty 1 O(e βn ) over S D for some constant β > 0, C 1 (S) = C (S). Gap n Shapley value. There exsts N such that φ C1 < α φ C. Then, C does not have Shapley value that s α-approxmable from samples over D. Proof. By a unon bound, gven m sets S 1,..., S m drawn..d. from D wth m polynomal n n, C 1 (S j ) = C (S j ) for all S j wth probablty 1 O(e βn ). Let C = C 1 or C = C wth probablty 1/ each. Assume the algorthm s gven m samples such that C 1 (S j ) = C (S j ) and consder ts (possbly randomzed) choce φ. Note that φ s ndependent of the randomzaton of C snce C 1 and C are ndstngushable to the algorthm. Snce φ C1 /φ C < α, φ s at least a factor α away from φ C wth probablty at least 1/ over the choces of the algorthm and C. Label the cost functons so that φ s at least a factor α away from φ C1 wth probablty at least 1/ over the choces of the algorthm. Thus, wth δ = 1/4, there exsts no algorthm such that for all C {C 1, C }, α φ C over both the samples and the choces of the algorthm. We obtan the napproxmablty result by constructng two such functons. φ C 1 α φc wth probablty at least 3/4 Theorem 11. For every κ < 1, there exsts a hypothess class of submodular functons wth curvature κ that have Shapley value that s not 1 κ + ɛ-approxmable from samples over the unform dstrbuton, for every constant ɛ > 0. Proof. We show that an element can have Shapley value that dffers by a factor 1 κ for two functons C 1 and C wth curvature κ that are ndstngushable from samples. Combnng wth Lemma 8, we then obtan the negatve result. These two functons have a smpler defnton va ther 18