Universal Multiparty Data Exchange and Secret Key Agreement

Transcription

1 Unversal Multparty Data Exchange and Secret Key Agreement Hmanshu Tyag Shun Watanabe 1 arxv: v2 [cs.it] 23 Jan 2017 Abstract Multple partes observng correlated data seek to recover each other s data and attan omnscence. To that end, they communcate nteractvely over a noseless broadcast channel each bt transmtted over ths channel s receved by all the partes. We gve a unversal nteractve communcaton protocol, termed the recursve data exchange protocol (RDE), whch attans omnscence for any sequence of data observed by the partes and provde an ndvdual sequence guarantee of performance. As a by-product, for observatons of length n, we show the unversal rate optmalty of RDE up to an O(n 1/2 log n) term n a generatve settng where the data sequence s ndependent and dentcally dstrbuted (n tme). Furthermore, drawng on the dualty between omnscence and secret key agreement due to Csszár and Narayan, we obtan a unversal protocol for generatng a multparty secret key of rate at most O(n 1/2 log n) less than the maxmum rate possble. A key feature of RDE s ts recursve structure whereby when a subset A of partes recover each-other s data, the rates appear as f the partes have been executng the protocol n an alternatve model where the partes n A are collocated. I. INTRODUCTION An m party omnscence protocol s an nteractve communcaton protocol that enables m partes to recover each other s data. The communcaton s error-free and s n a broadcast mode wheren the transmsson of each party s receved by all the other partes. Such protocols were frst consdered n [14] n a two-party setup, where bounds for the number of bts communcated on average and n the worst-case were derved for the case when no error s allowed. The m party verson, and the omnscence termnology, was proposed n [12] where the collectve observatons of the partes was assumed to be an Department of Electrcal Communcaton Engneerng, Indan Insttute of Scence, Bangalore , Inda. Emal: htyag@ece.sc.ernet.n. Department of Computer and Informaton Scences, Tokyo Unversty of Agrculture and Technology, Tokyo , Japan. Emal: shunwata@cc.tuat.ac.jp.

2 2 ndependent and dentcally dstrbuted (IID) sequence generated from a known dstrbuton 1 P X1 X m. It was shown n [12] that a smultaneous communcaton protocol based on sendng random hash bts of approprate rates attans the optmal sum-rate R (P X1 X m ). A common feature of these pror works s that the protocol reles on the knowledge of the underlyng dstrbuton P X1 X m. Note that the protocol proposed n [12] reles on the classc multtermnal source codng scheme gven n [10]. Thus, t nherts the followng unversalty feature from that scheme: If for 1 m the th party communcates rate R, the protocol attans omnscence for any source dstrbuton P X1 X m for whch the rate vector (R 1,..., R m ) les n the omnscence rate-regon correspondng to P X1 X m. Nevertheless, ths provdes no guarantee of unversal optmalty for the sum-rate (R R m ) for an arbtrary source P X1 X m. A nave protocol entals usng the frst n samples to estmate the entropes nvolved and then applyng the optmal protocol of [12] wth rates satsyng the entropy constrants. Specfcally, by usng the estmator for entropy proposed n [29], we can estmate the entropy to wthn an approxmaton error of O(1/ n ) usng n samples, where the constants mpled by O depend on the support sze of the dstrbuton. Ths results n an unversally sum-rate optmalty protocol, but for observatons of length n, the overall excess rate of communcaton over the optmal rate s O(n /n+1/ n ), whch s at best O(n 1/3 ). Furthermore, there s no guarantee of performance for ths protocol for a fxed sequence (x 1,..., x m ) observed by the partes. In ths paper, we present a protocol for omnscence, termed the recursve data exchange protocol (RDE), that s unversal and works for ndvdual sequences of data n the sprt of [31], namely t attans omnscence wth probablty close to 1 for every specfc data sequence. For a gven sequence (x 1,..., x m ) of data consstng of n length observatons, RDE attans an excess communcaton rate of O(n 1/2 ) over R (P x1 x m ) where P x1 x m denotes the jont type of the observatons. As a consequence, we show that for the generatve model where the data of the partes s IID, RDE s unversally sumrate optmal wth an excess rate of O(n 1/2 log n). Note that even for the case when the underlyng dstrbuton s known, the optmal rate can only be acheved asymptotcally and an excess rate s often needed. In partcular, for 2 m = 2, the precse leadng asymptotc term n excess worst-case rate was establshed n [25] and was shown to be O(n 1/2 ). An nterestng applcaton of RDE appears n secret key (SK) agreement [17], [1], [12]. Specfcally, 1 Throughout we shall restrct to fnte random varables and use the phrase probablty dstrbuton nterchangeably wth probablty mass functon (pmf). 2 For m > 2, a varant of RDE s shown n [26] to attan the optmal second-order asymptotc term, whch s O(n 1/2 ), for worst-case rates when the dstrbuton s known.

3 3 Csszár and Narayan showed n [12] that an optmum-rate SK can be generated by frst attanng omnscence and then extractng secure bts from the recovered data. We follow the same procedure here wth RDE n place of the omnscence protocol of [12] and obtan a unversal SK of rate at most O(n 1/2 log n) less than the optmal average and the worst-case rate. Note that for the case m = 2 wth known dstrbuton, the precse leadng asymptotc term n the gap to optmal worst-case rate was establshed n [15] and was shown to be O(n 1/2 ). Therefore, for multparty data exchange as well as SK agreement RDE can roughly attan the worst-case performance for the case of known dstrbutons, wthout requrng the knowledge of the dstrbuton. Also, for average rate, the unversal O(n 1/2 log n) gap to optmal rates attaned by RDE s to our knowledge the best-known. It was shown n [30] that nteracton enables an ACK NACK based unversal varable-length codng scheme for the Slepan-Wolf problem, where only party 1 needs to send ts data to party 2. Our protocol, too, s nteractve n a smlar sprt, but t reles on carefully ncreasng the rate of communcaton for each party. Note that whle for m = 2 a smple extenson of the protocol n [30] works for the data exchange problem as well, ths s not the case when m > 2. For m > 2, the order n whch the partes communcate must be carefully chosen. We gve a very smple crteron for choosng ths order of communcaton and show that the resultng protocol s unversally rate-optmal. Specfcally, the encoders n RDE send random hash bts correspondng ther nputs, whle the decoders, whch use a varant of mnmum entropy decodng, try to decode the observatons of any subset of communcatng partes. A key feature of RDE s ts recursve structure whereby when a subset A of partes recover each-other s data, the rates appear as f the partes have been executng the protocol n an alternatve model where the partes n A are collocated from the start. To enable ths, the partes communcate n the order of the entropes of ther emprcal types, wth the hghest entropy party communcatng frst, followed by the next hghest entropy party, and so on. The delay n communcaton between the partes s chosen to ensure that for every par of communcatng partes, the dfference of ther rates of communcaton, at any nstance, s equal to the dfference of the entropes of ther margnal types. We follow ths polcy and ncrease the rate n steps untl a subset of partes can attan local omnscence,.e., recover each other s data. Our encoders are easy to mplement, but the decoders are theoretcal constructs whch use type classes to form a lst of guesses for the data of other partes. Furthermore, snce we try to decode the data of every possble subset of communcatng partes, the complexty of our decoder s exponental n m. Nevertheless, we beleve that RDE s a steppng-stone towards a practcal protocol for the multparty data exchange problem.

4 4 There s a rch lterature relatng to the problems consdered here. Followng the semnal work of Slepan and Wolf [23], whch ntroduced fxed-length dstrbuted source codng for two partes, unversal error-exponents for the multparty extenson of ths problem were consdered n [11], [9], [20]. For the case of two partes, unversal varable length protocols wth optmal average rate were proposed n [13], [30]. In partcular, the protocol used n [30] has excess rate less than 3 O(n 1/2 ), whch s the best-known. A related protocol was used n [25] n a sngle-shot setup whch, when appled to IID observatons wth a known dstrbuton, was shown to be of optmal worst-case length even up to the second-order asymptotc term. A slght varant of the data exchange or omnscence problem, whch assumes the data of the partes to be elements of a fnte feld and requres exact recovery usng lnear communcaton, has been consdered n [22], [24], [7], [18], [19]. Whle RDE doesn t drectly relate to these works, we propose t as an alternatve approach for ensurng data exchange n these settngs. The remander of ths paper s organzed as follows: The next secton contans the formal descrpton of the omnscence problem. We frst descrbe an dealzed verson of RDE, RDE d, n Secton III where we assume that the rates can be contnuously ncreased and an deal decoder s avalable. We also llustrate the workng of RDE d wth examples. Ideal assumptons are removed n the subsequent secton whch contans a complete descrpton of RDE and our man results about ts performance. The SK agreement problem and our unversal SK agreement protocol based on RDE are descrbed n Secton V. All the proofs are gven n Secton VI. Our proofs rely on techncal propertes of the formula for mnmum communcaton for omnscence. Some of these propertes are new and maybe of ndependent nterest. Notatons. We start by recallng the standard notatons: We consder dscrete random varables X takng values n a fnte set X and wth pmf P X. Denote the set {1,..., m} of all partes by M. For random varables (X : M) and A M, X A denotes the collecton (X : A). Also, X n A denotes the sequence of IID random varables {X A,t } n t=1, where X A,t = (X,t : A). Smlarly, X A denotes the product set A X and X n = X 1 X n. For gven dstrbutons P and Q, ther varatonal dstance s denoted by P Q = 1 2 x P(x) Q(x). Whle our protocols are conceptually smple, the analyss s notatonally heavy and reles on some bespoke notatons. For easy reference, we summarze all nonstandard notatons used n ths paper n Table I. We often need to thnk of a subset of partes as a sngle party and use natural extensons of our notatons to ndcate such cases. For nstance, for a partton σ of A M or of M, the notaton ( Rσ (A σ ) : 1 σ ) extends R (A) gven n Table I 3 For m = 2 even RDE has excess rate less than O(n 1/2 ). The extra O( log n) factor for a general m appears snce the optmal sum-rate may not be a concave functon of P X1 X m for m > 2, and we take recourse to a Taylor approxmaton of the sum-rate functon.

5 5 and denotes the soluton ( R 1,..., R σ ) for equatons R j = H (X A X σ ), j 1 σ. Note that we have abused the subscrpt notaton, wth dfferent connotatons n dfferent contexts. For nstance, we use the notaton A σ for a partton σ of A, whch represents the set A as a collecton of elements σ σ. However, the specfc connotaton should be clear from the context. Notaton Descrpton Σ(A) Set of all nontrval parttons of A σ Number of parts n the partton σ σ f (A) The fnest partton {{} : A} of A σ B (A), B A The partton {{A \ B}, {} : B} of A R A Sum rate A R R CO (A) Set of all vectors (R : A) s.t. R B H(X B X A\B ), B A R CO (A) Set of all vectors (R : A) s.t. R B H(X B X A\B )+ B, B A R CO (A) Mnmum of R A over all R R CO (A) 1 σ H σ (A), σ Σ(A) σ 1 =1 H (X A X σ ) R (A), A Soluton of j R j = H(X A X ), A A σ, σ Σ(A) {A σ : 1 σ }, where A σ = σ treated as a sngle party π Maxmum number of bts communcated n any executon of the protocol π π av Expected value of the number of bts communcated n an executon of the protocol π TABLE I: Summary of notatons used n the paper. II. OMNISCIENCE We begn wth the descrpton of the problem for IID observatons. Specfcally, partes n a set M = {1,..., m} observe an IID sequence XM n = (X M1,..., X Mn ), wth the th party observng {X t } n t=1 and X Mt = (X t : M) P XM denotng the collectve data at the tth tme nstance. The partes have access to shared publc randomness (publc cons) U such that U s ndependent jontly of XM n. Furthermore, the th party, M, has access to prvate randomness (prvate cons) U such that U M, U, and XM n are mutually ndependent. Thus, the th party observes (Xn, U, U).

6 6 For smplcty, we restrct our exposton to tree-protocols (cf. [16]) descrbed below. A tree-protocol π for M conssts of a bnary tree, termed the protocol-tree, wth the vertces labeled by the elements of M. The protocol starts at the root and proceeds towards the leaves. When the protocol s at vertex v wth label v, party v communcates a bt b v based on ts local observatons (X n v, U v, U). The protocol proceeds to the left- or the rght-chld of v, respectvely, f b v s 0 or 1. The protocol termnates when t reaches a leaf, at whch pont each party produces an output based on ts local observatons and the bts communcated durng the protocol, namely the transcrpt Π = π(xm n, U M, U). Note that for treeprotocols the set of possble transcrpts s prefx-free. Also, note that the output s not ncluded n the transcrpt of the protocol, but s computed locally at each party. The lterature on dstrbuted functon computaton often focuses on Boolean functons and ncludes the 1-bt output as a part of the protocol transcrpt (cf. [16]). Ths results n a neglgble 1-bt loss n communcaton. However, ncludng the output n the transcrpt n our setup makes the data exchange problem trval snce the optmal protocol shall ental each party declarng ts observaton. Fgure 1 shows an example of a protocol tree for m = 3. The label of each node represents the party whch determnes the communcated bt at that node; the fnal boxes represent the termnaton of the protocol, at whch pont an output s produced by each party Fg. 1: A multparty protocol tree. The (worst-case) length π of a protocol π s the maxmum number of bts that are transmtted n any executon of the protocol and equals the depth of the protocol-tree. Also, the average length π av s gven by the expected value of the number of bts transmtted n an executon of the protocol π. In the omnscence problem, the partes engage n nteractve communcaton to recover each other s data. A protocol π consttutes an ɛ-omnscence protocol f, at the end of the protocol, the th party can

7 7 output an estmate X = X (X n, U, U, Π) XM n such that ( ) P X = XM n : M 1 ɛ. Defnton 1 (Communcaton for omnscence). Gven IID observatons wth a common dstrbuton P XM as above, for 0 ɛ < 1, a rate R 0 s an ɛ-achevable omnscence rate f there exsts an ɛ-omnscence protocol π wth length π less than nr, for all n suffcently large. The nfmum over all ɛ-achevable omnscence rates s denoted by R ɛ (P XM ). The mnmum rate of communcaton for omnscence R(P XM ) s gven by R(P XM ) = lm ɛ 0 R ɛ (P XM ). The mnmum average rate of communcaton for omnscence R av (P XM ) s defned smlarly by replacng length π wth average length π av. The fundamental quantty R(P XM ) was characterzed n [12] as { m R(P XM ) = mn R : R H(X B X B c), =1 B B M }. (1) Followng [12], the collecton of all rate vectors R = (R 1,..., R m ) satsfyng the constrants n (1), termed the CO regon, wll be denoted by R CO (M P XM ), and the mnmum sum-rate by R CO (M P XM ). When the dstrbuton P XM s clear from the context, we shall omt t from the notaton and smply use R CO (M) and R CO (M). Whle the result n [12] was shown to hold only for R(P XM ), the same characterzaton holds for R av (P XM ) as well. Indeed, note that the set of dstnct transcrpts of a tree protocol π s prefx-free. Therefore, the lengths of these transcrpts satsfy Kraft s nequalty, and so, H(Π) π av. By proceedng exactly as n [12], we can see that R av (P XM ) R CO (M P XM ). On the other hand, clearly R av (P XM ) R(P XM ) = R CO (M P XM ), whereby for every dstrbuton P XM, we have R av (P XM ) = R CO (M P XM ). An alternatve expresson for R CO (P XM ) was obtaned n [12] by lookng at ts dual form. In fact, by leveragng on the complementary slackness property, [3], [5] showed that the optmzaton n the dual

8 8 form can be restrcted to the parttons of M and showed that 4 R CO (M P XM ) = where Σ(M) denotes the set of parttons of M, and, for each σ Σ(M), H σ (M P XM ) = max H σ(m P XM ), (2) σ Σ(M) 1 σ 1 σ H (X M X σ ). (3) Note that the fact that R CO (M P XM ) s lower bounded by the rght-sde of (2) was shown earler n [12]. RDE drectly acheves the rght-sde of (2), thereby provdng an alternatve, operatonal proof for the =1 tghtness of ths lower bound for R CO (M P XM ) from [12]. Whle there can be several maxmzers of H σ, there exsts a maxmzng partton whch s a further partton of any other maxmzng partton [4, Theorem 5.2], the fnest maxmzng partton; we shall call ths fnest maxmzer of H σ n (2) the fnest domnant partton (FDP), whch was called fundamental parton n [4]. The fnest partton σ f (M) := {{}, M} plays a partcularly mportant role n RDE. Note that when the fnest partton s FDP, the optmal rate assgnment s unquely gven by the soluton R = (R 1,..., R m) of M\{j} R = H(X M X j ), j = 1,..., m. (4) III. UNIVERSAL PROTOCOL FOR OMNISCIENCE UNDER IDEAL ASSUMPTIONS We gve a unversal protocol for omnscence, whch, when a sequence x M s observed, wll transmt communcaton of rate no more than R CO (M P xm ). To present the man dea behnd RDE, we frst descrbe t assumng the followng deal assumptons. Specfcally, we make two assumptons: (a) Contnuous rate assumpton: Communcaton-rate, defned as the total number of bts of communcaton up to a certan tme dvded by n, can be ncreased contnuously n tme 5 ; and (b) Ideal decoder assumpton: We assume the avalablty of an error-free, deal decoder DEC d whch correctly decodes a sequence once suffcent communcaton has been sent and declares a NACK 4 An alternatve proof of (2) was provded n [4] by usng technques from submodular optmzaton. 5 Clearly, ths does not hold n practce snce the number of bts of communcaton can be ncreased only n steps of dscrete szes. The contnuous rate assumpton allows us to examne, loosely speakng, the flud lmt behavor of RDE.

9 9 otherwse. 6 A standard unversal decoder used n source codng s the mnmum entropy decoder whch, gven sdenformaton y and an nr-bt 7 random hash 8 of X n, searches for the unque sequence x such that the jont type P X Y = P xy satsfes H ( X Y ) R and the hash of x matches the receved hash bts. The decoder that we prescrbe n the next secton works on a smlar prncple except that t searches for any possble subset of sequences t can decode wth the current rate. To avod the addtonal complcatons due to decodng error, we frst assume the avalablty of an deal decoder DEC d whch enables omnscence for all partes j A as soon as the rate receved from the partes n A s suffcent. That s, the deal decoder guarantees that each party A can recover the correct sequence x A f the rates of communcaton R = (R : A) satsfy R R CO (A P xa ). Furthermore, f R / R CO (A P xa ), the deal decoder does not mstakenly output a wrong sequence x A, but declares a NACK nstead. Protocol 1 summarzes our assumed deal decoder DEC d. Protocol 1: Ideal decoder DEC d (j, σ, R) Input: An ndex 1 j m, a partton σ Σ(M), a rate vector R = (R 1,..., R m ). Output: An ACK message (ACK, A) or a NACK message 1) For σ such that j σ, search for the maxmal set A M such that σ A and (R l : l A) R CO (A P xa ), and reveal x A to party j. 2) f If such an A was found n Step 1 then return (ACK, A). else return NACK. Wth ths deal decoder at our dsposal, under the contnuous rates assumpton, fndng a unversal protocol s tantamount to fndng a polcy for ncreasng the rates (R 1,..., R m ) such that when the rate vector enters R CO (M P xm ) for the frst tme, the sum-rate s R CO (M P xm ). Note that ntally the margnal types P x are avalable to each party and can be transmtted usng O(log n) bts, snce there are only polynomally many types. Also, f a subset A attans local omnscence n the mddle of the 6 In analyss of the deal protocol, we do not account for the rate needed to send NACKs. In practce, each NACK symbol counts for a bt of communcaton and the sze of dscrete ncrements must be chosen carefully to render the rate needed to send NACKs neglgble. 7 nr s requred to be an nteger. When ths s not the case, we smply use nr bts n place of nr. Ths conventon wll be used throughout ths paper and wll be accounted for n our analyss. 8 A random hash of X n s a bt sequence produced by a functon f : X n {0, 1} nr whch s chosen randomly (usng publc randomness) from a class of functons satsfyng the 2-unversal property [2]. For nstance, the class of all functons satsfes the 2-unversal property and, therefore, standard random bnnng (cf. [8]) produces a random hash.

10 10 R 2 H(X 2 X 1) t 1 H(X 1) H(X 2) H(X 1 X 2) R 1 Fg. 2: Illustraton of protocol for m = 2. The transton pont t 1 depends only on the margnal types P x1 and P x2. protocol, any j A upon recoverng x A can transmt P xa n O(log n) bts to all the partes, who n turn can use t to compute H(P xa ). As an llustraton, consder the smple case when m = 2. Partes frst share P x1 and P x2 ; suppose H(P x1 ) H(P x2 ). Then, party 1 starts communcatng and ncreases ts rate R 1 at slope 9 1. When the rate R 1 reaches H(P x1 ) H(P x2 ), party 2 starts communcatng at slope 1 as well. Throughout the protocol, each party s tryng to decode the other usng the deal decoder DEC d and they keep on communcatng as long as the deal decoders output NACKs. The partes wll decode each other as soon as (R 1, R 2 ) enters R CO ({1, 2} P x1,x 2 ),.e., when R 1 H(X 1 X 2 ) and R 2 H(X 2 X 1 ), where (X 1, X 2 ) P x1,x 2. Note that once both partes start communcatng, the dfference R 1 R 2 s mantaned as H(X 1 ) H(X 2 ). Thus, when (R 1, R 2 ) enters R CO ({1, 2}), t holds that R 1 = H(X 1 X 2 ) and R 2 = H(X 2 X 1 ); the red lne n Fgure 2 llustrates 10 ths evoluton of rates. RDE extends the dea above to a general m. We desgn RDE so that the frst subset A whch attans local omnscence does so by usng communcaton only from the partes n A and of sum rate R A = H σf (A)(A P xa ) = A R (A); (5) see (13) n Lemma 7 gven n Secton VI below for the second equalty. To that end, we note (see Lemma 7 for a proof) that for every A R (A) R j (A) = H(X ) H(X j ). (6) 9 The slope s defned as the dervatve of rate wth respect to the tme under the contnuous rate assumpton. 10 It s also possble to proceed along the blue lne for the m = 2 case. However, ts extenson to a general m s not clear.

11 11 A key pont here s that for P xm ths dfference can be computed usng only the margnal types P x and P xj. RDE ensures that for every par (, j) of communcatng partes, the rate of communcaton R R (A) = R j R j (A), whch by (6) n turn can be ensured f the constant dfference property, namely R R j = H(X ) H(X j ), (7) s mantaned throughout the protocol for every par of communcatng partes. Thus, all communcatng partes reach the rate R (A) at the same tme. Specfcally, we frst arrange partes n decreasng order of the entropy of the emprcal dstrbuton of ther local observatons, whch are shared n O(log n)-bts. Assumng H(P x1 ) H(P x2 ) H(P xm ), party 1 starts communcatng, and the th party starts communcatng when R 1 H(P x1 ) H(P x ). Ths ensures the constant dfference property (7) for every par (, j) of communcatng partes. For notatonal convenence, we assgn 1 to R when the th party has not started communcatng; the rate vector (0, 1, 1,..., 1) ndcates that party 1 starts communcatng and every one else remans quet. When a subset A attans local omnscence, we decrease the rate-slope for each party A to 1/ A, thereby ensurng that collectvely partes n A ncrease the rate of communcaton R A at slope 1. Note that snce partes n A have recovered x A, any one party A can compute the type P xa and transmt t usng O(log n) bts. Our man observaton s that at ths pont the rates appear as f the partes n A were collocated to begn wth and have been executng the protocol as a sngle party. In partcular, R A R j = H(X A ) H(X j ) for any communcatng party j outsde A. The second crucal observaton s that for the frst subset A whch attans local omnscence, (R (A) : A) R CO (A). Snce by (5) A R (A) s a lower bound for R CO (A), the partes n A cannot attan local omnscence before they communcate at sum-rate A R (A). Further, RDE ensures that all partes n A reach the rate R (A) at the same tme. Thus, the partes n A must have communcated at sum-rate R A = A R (A) = H σf (A)(A P xa ) (8) when they attan local omnscence. As the protocol proceeds, subsets of partes keep attanng local omnscence and start behavng as a sngle party. Proceedng recursvely, t follows that when all partes attan omnscence, the rate of communcaton must equal H σ (M P xm ) for some σ Σ(M), whch n vew of (2) s no more than R CO (M P xm ) and must be optmal n the lmt as n.

12 12 Protocol 2: OMN d (σ, H, R) Input: A partton σ Σ(M) wth σ = k, an entropy estmate vector H = (H σ : 1 k), a rate vector R = (R 1,..., R m ); we assume that H s sorted,.e., H σ1 H σ2 H σk. Output: A rate vector R out, a famly of subsets O that have attaned omnscence. 1) Intalze s := max{ : R σ 0}. 2) All partes j such that j σ for some 1 s ncrease ther rates R j at slope 1/ σ. 3) f There exsts > s such that R σ1 H σ1 H σ then set R j = 0 for all j σ, and set s = max{ : R σ 0}. 4) For all j such that j σ for some 1 s, execute DEC d (j, σ, R), whch outputs NACK or (ACK, A j ). 5) f All partes send a NACK then return to Step 2. else Identfy the omnscence famly Set R out = R and return (R, O). O = {B M : all j B returned (ACK, B)}. To help the reader buld heurstcs for the complete protocol and ts analyss, we provde a sketch of the analyss for the deal stuaton and consder the deal verson RDE d. The formal proofs for the deal case closely follow those for the results for the actual protocol and have been omtted. As mentoned, RDE d proceeds recursvely by ncreasng the rates wth fxed slopes untl a subset attans omnscence, at whch pont the slopes are changed so that the partes n an omnscence attanng subset behave as f they are collocated. We descrbe the one-step omnscence protocol OMN d n Protocol 2. The protocol takes as nput a partton σ such that partes n any one part are behavng as collocated partes, a vector H = (H σ, 1 σ ) consstng of estmates of entropy for margnal dstrbuton of partes n any part of σ, and a rate vector R = (R 1,..., R m ) of rates of communcaton sent by all the partes up to ths pont. Note that a vald rate vector should reflect that partes n any one part have communcated enough to attan local omnscence. Also, snce we shall recursvely call OMN d, the only rate vectors OMN d encounters are those whch can arse by ncreasng the rates n the manner of RDE. We call the collecton of rate vectors satsfyng the two condtons above (σ, H)-vald. Formally, Defnton 2. For σ Σ(M) wth σ = k and H = (H σ1,..., H σk ) wth H σ1 H σ2 H σk, a rate vector (R 1,..., R m ) s (σ, H)-vald f (R j, j σ ) R CO (σ ), s.t. σ 2,

13 13 and (R σ, 1 k) can be obtaned by startng wth (0, 1, 1,..., 1) and ncrementng the rates as n Protocol 2 when the partes n each part σ are collocated,.e., each part σ starts ncreasng ts rate at slope 1 once R σ1 H σ1 H σ. As mentoned earler, nstead of ntalzng all rates wth 0 n RDE, and n the defnton of a vald rate vector, we dstngush between rate 0 and rate 1 for a techncal reason. A rate of 1 ndcates that the party s not partcpatng n the protocol yet and wll not even attempt to decode. In contrast, a 0 rate ndcates that the party has not yet communcated any bts, but has started decodng and wll ncrement ts communcaton rate n each step from here on. The result below shows a recursve property of OMN d that renders RDE unversally rate-optmal. Specfcally, t shows that f R s (σ, H)-vald then, when OMN d (σ, H, R) termnates, the output rate vector s (σ out, H out )-vald where σ out s a sub-partton of σ whch s obtaned by combnng the parts that have acheved local omnscence; H out s the correspondng estmate for entropes of the margnals of parts of σ out. Furthermore, for every set A that attans local omnscence, the sum-rate R A at the end of OMN d s exactly H σf (A σ)(a σ ). 11 Theorem 1. For σ Σ(M) wth σ = k and H = (H σ1,..., H σk ) wth H σ1 H σ2 H σk, let R n = (R n 1,..., Rn m ) be (σ, H)-vald. Then, f OMN d (σ, H, R n ) s executed, the fnal rates R out and the omnscence famly O satsfy the followng: 1) Every A O conssts of parts of σ,.e., A = c l=1 σ l for some { 1,..., c } {1,..., σ }, and the sum-rate R out A satsfes R out A = H {σ1 σ c } (A P xa ). 2) Let σ out Σ(M) be the partton obtaned by combnng the parts n σ that belong to the same A n O. Let H σ out s (σ out, H out )-vald. denote the entropy of the type of x σ out. Then, wth H out = ( H σ out, 1 σ out ), R out In fact, Theorem 1 s a specal case of Theorem 2, and the proof of the former follows from that of the latter gven below. However, we provde a bref sketch of the proof of Theorem 1 here to hghlght 11 When A = c l=1σ l, by our conventon H σf (A σ)(a σ) = H {σ1 σ c } (A P xa ).

14 14 the key deas and, also, to clarfy the techncal proof of Theorem 2. Proof sketch. For smplcty, assume that σ conssts of sngletons,.e., σ = σ f (M). The man component of our proof s the followng clam: Clam: The partes n a subset A attan local omnscence exactly when each R, A, reaches R (A). As mentoned before, all communcatng partes A reach R (A) smultaneously, and the partes n A cannot attan local omnscence before ths happens. The proof of the clam follows from Lemma 11 gven n Secton VI, snce no subset of A has attaned local omnscence before A. The theorem follows. Indeed, the frst asserton holds by (8). For the second asserton, we need to show that for two subsets A and B n O, R out A Rout B = H(X A) H(X B ). The complete proof consders varous cases dependng on f A (or B) contans a party wth nonnegatve R n. We llustrate the proof for a case when there exst A and j B wth R n, Rj n 0. Snce R n s vald for σ = σ f (M) and the communcatng partes mantan the dfference of ther rates, t follows from the clam above that R out A R out B = RA\{} out Rout B\{j} + Rout Rj out = RA\{} out Rout B\{j} + Rn Rj n = RA\{} out Rout B\{j} + H(X ) H(X j ) = Rl (A) Rk (B) + H(X ) H(X j ) l A\{} k B\{j} = H(X A X ) H(X B X j ) + H(X ) H(X j ) = H(X A ) H(X B ). Other cases can be handled smlarly. Therefore, R out s vald for σ out. Thus, f we proceed by recursvely callng OMN d, each tme wth (σ out, H out, R out ) obtaned from the prevous call, we shall ultmately attan omnscence usng the sum rate H σ (M) for some partton σ. Snce H σ (M) s a lower bound for R CO (M) by (2), ths rate must be optmal. We summarze the overall deal protocol n Protocol 3. Remark 1. Recently, t was shown n [6] that f a set A corresponds to a part n the partton that attans the maxmum n (2), then omnscence can be attaned n such a manner that the partes n A can attan omnscence along the way from the communcaton of the partes n A. RDE explctly has ths feature and attans omnscence for each part of the maxmzng partton along the way.

15 15 Protocol 3: RDE d : The recursve data exchange protocol under deal condtons 1) Intalze σ = σ f (M), R = (0, 1, 1,..., 1), k = σ. 2) whle k > 1 do () For 1 k, a party j σ computes P xσ and broadcasts t. Each party computes H σ = H ( ) P xσ, 1 k. () Let H be the sorted verson of (H σ : 1 k),.e., assume H σ1 H σ2 H σk. Call OMN d (σ, H, R). Let (R out, O) be ts output. () Let σ out = {σ : σ σ s.t. σ A A O} {A : A O}. Update R = R out, σ = σ out, and k = σ out. We conclude ths secton wth a few llustratve examples to demonstrate the workng of the deal verson RDE d. The frst example s for m = 3 and exhbts a case where σ f (M) s the FDP. Example 1. Let X 1 Ber(1/2), X 3 Ber(q), and X 2 = X 1 X 3. In ths case, R CO ({1, 2, 3}) s gven by rate vectors satsfyng the followng lnear constrants: R 1 + R 2 1, R 2 + R 3 h(q), R 1 + R 3 h(q). When 1 2 < h(q) 1, the fnest partton s the FDP, and R CO ({1, 2, 3}) = H {1 2 3} = 1 + 2h(q). 2 The CO regon s depcted n Fgure 3. As can be seen from the fgure, R CO ({1, 2, 3}) s acheved by the unque rate assgnment R = (1/2, 1/2, (2h(q) 1)/2). In RDE d, partes 1 and 2 communcate frst and ncrease ther rates at slope 1 untl R 1 = R 2 = H(X 1 ) H(X 3 ) = H(X 2 ) H(X 3 ) = 1 h(q). At ths pont, party 3 starts communcatng and all the partes ncrease ther rates at slope 1. Owng to the ntal lead of R 1 and R 2 over R 3, all the partes reach R smultaneously. When H σ s maxmzed by a partton σ other than the fnest partton σ f (M), as RDE d proceeds, the partes n parts of σ attan local omnscence, along the way, before all the partes attan omnscence. Consder the followng example, agan for m = 3. Example 2. Let W 1, W 2 Ber(1/2) and V 1, V 2 Ber(q) for some 0 < q < 1 2, and let X 1 = (W 1, W 2 ), X 2 = (W 1 V 1, W 2 ), and X 3 = W 2 V 2. In ths case, the partton {12 3} s the FDP, H {12 3} = 1+3h(q), and RDE d proceeds as follows: Partes 1 and 2 start ncrease ther rates at slope 1. When ther rates

16 16 Fg. 3: Illustraton of R CO ({1, 2, 3}) for Example 1. reach h(q), they attan local omnscence. At ths pont they start ncreasng ther rates at slope 1/2 and contnue dong so untl R 1 + R 2 reaches H(X 1, X 2 ) H(X 3 ) = 1 + h(q). Now, party 3 starts communcatng at slope 1. When all the partes reach ((1 + 2h(q))/2, (1 + 2h(q))/2, h(q)), they attan omnscence. Note that {1, 2} attan local omnscence even before 3 starts communcatng, llustratng the recursve structure of RDE d wheren a subset attanng local omnscence start behavng as f the partes n t were collocated to begn wth. In fact, ths recursve property holds even when only a subset of communcatng partes attans omnscence, as our fnal example wth m = 4 llustrates. The stuaton for m = 4 captures the typcal case for our general analyss establshng the recursve nature of the protocol at stuatons smlar to that llustrated by the pont t 3 n Fgure 4 consttutes the man step n our analyss. Example 3. Let W 1, W 2, W 3 Ber(1/2) and V 1, V 2 Ber(q) for some 0 < q < 1 2, and let X 1 = (W 1, W 2 ), X 2 = (W 1 V 1, W 2 ), X 3 = W 2 V 2, and X 4 = W 3. Note that the observatons of subset {1, 2, 3} are exactly as n Example 2. In ths case, the partton {123 4} s the FDP, H {123 4} = 3+2h(q), and RDE d proceeds as n Fgure 4. At t 1, partes 1 and 2 attan local omnscence and change the slopes of R 1 and R 2 to 1/2. At t 2, partes 3 and 4 start communcatng. At t 3, partes n {1, 2, 3} attan local omnscence and change ther slope to 1/3. Note that up to t 3 the evoluton of (R 1, R 2, R 3 ) s exactly the same as that n Example 2. Also, at t 3 the rate dfference (R 1 + R 2 + R 3 R 4 ) equals H(X 1, X 2, X 3 ) H(X 4 ) = 1 + 2h(q). Thus, after t 3 the rate par (R 1 + R 2 + R 3, R 4 ) behaves as f the partes n {1, 2, 3} were collocated to begn wth. Fnally, all partes attan omnscence at t 4.

17 17 R R1, R2 R3 5+4h(q) 6 1 R4 1+2h(q) 2 1+h(q) 2 1+2h(q) 3 h(q) t1 t2 t3 t4 t Fg. 4: The evoluton of rates for Example 3. IV. UNIVERSAL PROTOCOL FOR OMNISCIENCE: FULL DESCRIPTION Movng now to the real world, rates must be ncreased n dscrete ncrements and a postve decodng error probablty must be tolerated. To that end, the partes ncrementally transmt ndependent hash bts, n at a tme. The deal decoder of the prevous secton s replaced wth a typcal decoder DEC(j, σ, R) whch searches for the maxmal set A such that there exsts a unque sequence x A that contans the current rate vector n ts CO regon and s consstent wth the local observaton and the receved hash values. In fact, nstead of workng wth the orgnal CO regon R CO (A), we use the more restrctve regon R CO (A) consstng of vectors (R, A) such that R B H(X B X A\B ) + B, B A. The complete decoder s descrbed n Protocol 4. Protocol 4: DEC(j, σ, R) Input: An ndex 1 j m, a partton σ Σ(M), a rate vector R = (R 1,..., R m ) Output: A NACK message, an ACK message (ACK, A), or an error message ERR. 1) For σ such that j σ, fnd the maxmal set A M such that σ A and there exsts a unque sequence ˆx A such that the hashes of ˆx A match all the prevously receved hashes from partes n A \ {j} and the jont type P XA of ˆx A satsfes the followng: () P Xj = P xj, and () (R : A) R ( ) CO A PXA. 2) f there s a unque maxmal A found n Step 1 then return (ACK, A). else f there s no sequence found n Step 1 for any set A then return NACK. else f there are multple As found or multple sequences ˆx A are found for any A n Step 1 then return ERR.

18 18 Protocol 5: OMN(σ, α, H, R) Input: A partton σ Σ(M) wth σ = k, an α N, an entropy estmate vector H = (H σ : 1 k), a rate vector R = (R 1,..., R m ); we assume that H s sorted,.e., H σ1 H σ2 H σk Output: A rate vector R out, a famly of subsets O that have attaned omnscence. 1) Intalze s := max{ : R σ 0}. 2) All partes j such that j σ for some 1 s send n / σ random hash bts. Update R j R j + / σ. 3) f There exsts > s such that R σ1 H σ1 H σ + α then set R j = 0 for all j σ, and set s = max{ : R σ 0}. 4) For all j such that j σ for some 1 s, execute DEC(j, σ, R), whch outputs NACK, (ACK, A j ), or ERR. 5) f All partes send a NACK then return to Step 2. else f No party declares an ERR and some partes declare an ACK, then Identfy the omnscence famly f O s nonempty then Set R out = R, and return (R, O). else declare an error. else declare an error. O = {B M : all j B returned (ACK, B)}. Note that the decoder declares (ACK, A) f t can fnd a unque maxmal set A and a unque sequence x A, declares NACK f t fnds no such set, or an ERR otherwse. In fact, an error may occur even when t s not detected,.e., when ERR s not transmtted. However, we can dentfy an event E (descrbed formally n Secton VI-B) of small probablty such that under E c the real decoder DEC behaves exactly lke DEC d, but wth R CO (A) replaced wth R CO (A). Therefore, omnscence can be acheved n a smlar manner as the deal protocol of the prevous secton. The man component of RDE s the one step omnscence protocol OMN descrbed n Protocol 5, whch uses DEC for decodng. Protocol OMN proceeds very much lke the deal protocol except that a new party starts communcatng when R 1 H(P x1 ) H(P x ) + α, where α N s an ncreasng threshold parameter whch s updated as the protocol proceeds. Throughout the protocol, a rate R = 1 ndcates that the th party s not yet transmttng and only partes wth R 0 communcate. The decoder tres to attan omnscence only among the communcatng partes. The deal protocol of the prevous secton works due to ts recursve structure whereby when a subset A attans local omnscence, the rate vector appears as f the partes n A have been collocated from the start. Moreover, the frst subset to attan local omnscence does so by usng a communcaton of

19 19 rate H σf (A). Both these propertes were captured by Theorem 1. The result below establshes a smlar recursve property of OMN. However, the defnton of valdty needs to be modfed from Defnton 2 n place of the operatonal defnton n the deal case, we use the more techncal defnton below whch captures all the key features that we need. Defnton 3. For α N, σ Σ(M) wth σ = k and H = (H σ1,..., H σk ), a rate vector (R 1,..., R m ) s (σ, H, α)-vald f, for s = max{ : R σ 0}, the followng condtons hold: () (Approxmate constant dfference) For 1, j s, R σ R σj H σ H σj + α ; () (Noncommuncatng partes) R σ1 < H σ1 H σs+1 + α ; (9) () (Combned partes) 1 k such that σ 2, (R j : j σ ) R CO (σ ) ; (10) (v) (Separate parts) for all A {1,..., k} wth A 2, ( ) (R j : j σ, A) / R CO σ. The constant dfference condton s crucal for ensurng the recursve nature of RDE under deal condtons. In general, snce the rates must be ncremented n dscrete steps, the approxmate verson n Condton () has been ntroduced n the place of the orgnal constant dfference condton. For noncommuncatng partes, Condton () must be satsfed so that Condton () s mantaned for those partes n future rounds when they start communcatng. Condton () ensures that the current rates are enough for partes n each part to attan local omnscence, whle Condton (v) ensures that σ s the maxmal partton such that the partes n each part can attan local omnscence at current rates. The followng theorem captures our key observaton about OMN; ts proof s gven n Secton VI-B. A Theorem 2. For α N, σ Σ(M) wth σ = k and H = (H σ1,..., H σk ) wth H σ1 H σ2 H σk, let R n = (R n 1,..., Rn m ) be (σ, H, α)-vald. Then, f OMN(σ, α, H, R n ) s executed and error E (defned n Secton VI-B) does not occur, the fnal rates R out and the omnscence famly O satsfy the followng:

20 20 (I) For every A O, t holds that a) A conssts of parts of σ,.e., A = c l=1 σ l for some { 1,..., c }, and b) denotng by A σ the set {σ 1,..., σ c }, we have R σ l (A σ ) 2α R out σ l R σ l (A σ ) + (m + 2α), 1 l c. (II) Let σ out Σ(M) be the partton obtaned by combnng the parts n σ that belong to the same A n O. Let H σ out denote the entropy of the type of x σ out. Then, wth H out = ( H σ out, 1 σ out ), R out s (σ out, H out, c mα)-vald, where c m s a constant dependng only on m. We are now n a poston to descrbe RDE. We begn by callng OMN wth σ = σ f (M), α = 1, the sorted entropy estmates H computed from margnal emprcal dstrbutons P x, and the rate vector R = (0, 1,..., 1) ndcatng that party 1 starts communcatng and every one else remans quet. Note that R s (σ, H, 1)-vald. A new party starts communcatng when R 1 H 1 H +. If no error occurs, OMN wll termnate when a subset A attans omnscence. In vew of Theorem 2, at ths pont R A should be close to H σf (A) (A P xa ) and the rates wll be (σ out, H out, c mα)-vald. Thus, we are n a smlar stuaton as the frst call to OMN except that α must be replaced by c mα and the partes n a sngle part of σ out are behavng as f they are collocated. The protocol proceeds by callng OMN agan wth these updated parameters. Note that under E c, any party j A for A O can correctly compute P xa and transmt t usng O(log n) bts. Proceedng recursvely n ths manner, the protocol stops when partes n M attan omnscence, whch by Theorem 2 can only happen when the sum-rate R M s close to H σ (M P xm ) for some partton σ of M. Thus, omnscence wll be attaned n communcaton of rate roughly less than R CO (M P xm ). We formally descrbe RDE n Protocol 6 and summarse ts performance n Theorem 3. We close wth the followng result clamng the unversal rate optmalty of RDE for every IID dstrbuton. Proof s a smple consequence of Theorem 2 and s gven n Secton VI. Note that whle Protocol 6 s a varable length protocol, ts fxed length varant can be obtaned smply by abortng the protocol once the total number of bts communcated crosses nr. Theorem 3. There exst constants C > 0, = 1,..., 4 dependng only on m and a polynomal p(n)

21 21 Protocol 6: RDE: The recursve data exchange protocol 1) Intalze σ = σ f (M), R = (0, 1, 1,..., 1), k = σ, α = 1. 2) whle k > 1 do () For 1 k, a party j σ computes P xσ and broadcasts t. Each party computes H σ = H ( ) P xσ, 1 k. () Let H be the sorted verson of (H σ : 1 k),.e., assume H σ1 H σ2 H σk. Call OMN(σ, α, H, R). f There s no error declared then let (R out, O) be ts output. else Termnate. () Let σ out = {σ : σ σ s.t. σ A A O} {A : A O}. Update R = R out, σ = σ out, k = σ out, and α c mα. dependng on X, M, such that for every > 0 and every sequence x M, the probablty of error for Protocol 6 s bounded above by C 1 ( log XM ) + m p(n)2 n. Furthermore, f an error does not occur, the number of bts communcated by the protocol for nput x M s bounded above by nr CO (M P xm ) + nc 2 + C 3 ( log XM ) + m + C 4 log n. (11) Corollary 4. For = 1 n and every dstrbuton P XM, Protocol 6 has a probablty of error ɛ n vanshng to 0 as n and average length π av less than 12 nr CO (M P XM ) + O( n log n). Furthermore, for a fxed R > 0, the fxed-length varant of Protocol 6 has probablty of error ɛ n vanshng to 0 as n for all dstrbutons P XM that satsfy ( ) R > R CO (M P XM ) + O n 1 log n. V. UNIVERSAL SECRET KEY AGREEMENT Closely related to the omnscence problem s the SK agreement problem where the partes seek to generate shared random bts whch are almost ndependent of the communcaton used to generate them. Specfcally, an (ɛ, δ)-sk agreement protocol conssts of an nteractve communcaton protocol 12 The constant mpled by O( n log n) depends on P XM ; see (43) below.

22 22 π wth publc randomness U, prvate randomness U at Party, and wth the output of the th party K = K (X n, U, U, Π) such that there exsts a K-valued random varable K satsfyng the recoverablty condton P (K = K, M) 1 ɛ, and the secrecy condton 13 P KΠU P unf P ΠU δ, where P unf denotes the unform dstrbuton on K. Defnton 4 (Secret key capacty). For ɛ, δ [0, 1), a rate R 0 s an (ɛ, δ)-achevable SK rate f there exsts a K (n) -valued (ɛ, δ)-sk wth log K (n) nr for all n suffcently large. The supremum over all (ɛ, δ)-achevable SK rates s called the (ɛ, δ)-sk capacty, denoted C ɛ,δ (M P XM ). The SK capacty for P XM s gven by C(M P XM ) = lm C ɛ,δ(m P XM ). ɛ+δ 0 Theorem 5 ([12]). Gven a dstrbuton P XM, C (M P XM ) = H (X M ) R CO (M P XM ). In fact, t was shown n [27], [28] that a strong converse holds and C ɛ,δ (M P XM ) = C(M P XM ) for all ɛ + δ < 1. The achevablty of rate H (X M ) R CO (M P XM ) was shown n [12] by establshng a connecton between SK agreement and omnscence. In partcular, a SK achevng capacty was generated by frst communcatng at rate R CO (M P XM ) to attan omnscence, and then extractng a SK from XM n whch s almost ndependent of the communcaton used for omnscence. Followng the same methodology, we provde a unversal SK agreement protocol whch bulds upon the unversal omnscence protocol of the prevous secton. We consder a slght generalzaton of the defnton of SK above, whch admts varable length SKs. An (ɛ, δ)-sk K and ts estmates K 1,..., K m now take values n K = {0, 1}, the set of fnte length bnary sequences. The recoverablty condton remans as before. However, the secrecy condton needs to be modfed. Specfcally, denotng by T the random length of K, whch we assume to be avalable 13 We assume that the publc randomness U s avalable to the eavesdropper.

23 23 to the eavesdropper, the secrecy condton now requres P T (t) PKΠU T =t P unf,t P ΠU T =t δ, t where P unf,t denotes the unform dstrbuton on {0, 1} t. The average achevable rate and average SK capacty are defned as above wth the worst-case length log K replaced by the average length E[T ]. Instead of ntroducng a new notaton for average SK capacty, we note that t equals C (M P XM ) and, wth an abuse of notaton, use C (M P XM ) to denote both the SK capacty and the average SK capacty. Indeed, the achevablty s the same as above snce a fxed length SK consttutes a varable length SK. For the converse, denotng ɛ t := 1 P (K = K, M T = t) and δ t := PKΠU T =t P unf,t P ΠU T =t, t follows by applyng the converse proof of [12] for each fxed value T = t that t n C (M P X M ) + g 1 (ɛ t ) + g 2 (δ t ), where g 1 and g 2 are concave, ncreasng functons satsfyng g (x) 0 as x 0. Thus, E[T ] n C (M P X M ) + E[g 1 (ɛ T ) + g 2 (δ T )] C (M P XM ) + g 1 (E[ɛ T ]) + g 2 (E[δ T ]) C (M P XM ) + g 1 (ɛ) + g 2 (δ), where the last two nequaltes hold snce g, = 1, 2, are concave and ncreasng. We present a unversal SK agreement protocol that generates a SK of average length nc(m P XM ) O( n log n) wthout the knowledge of the underlyng dstrbuton P XM. Specfcally, frst the partes use Protocol 6 wth = 1/ n to recover XM n. If no error occurs and the recovered sequence s x M, by Theorem 3 the number of bts communcated s no more than l(x M ) = nr CO (M P xm ) + O( n). We extract a SK from recovered x n M by randomly hashng14 t to roughly nh(p xm ) l(x M ) values. Formal descrpton of the protocol s gven n Protocol 7; the length of the SK s tuned to the secrecy parameter δ. 14 The random hash can be replaced by a randomly selected member of a 2-unversal hash famly.

24 24 Protocol 7: A unversal SK agreement protocol Input: Step sze parameter and secrecy parameter δ 1) Partes execute Protocol 6 wth step-sze. 2) f Protocol 6 completes wthout declarng an error then Protocol 6. Each party M forms an estmate K of the SK as follows: () Denotng by P () the type of the estmate x () M of Xn M at Party and by ɛ n the maxmum error probablty of Protocol 6, set l ( P ()) to be the quantty n (11) for P () and k ( P ()) = nh ( P ()) l ( P ()) X M log(n + 1) 2 log () generate K by randomly hashng x () M to k ( P ()) bts. else Declare an error. 1 δ 2ɛ n + 2; Theorem 6. For = 1 n, 0 < δ < 1, and every dstrbuton P XM, Protocol 7 generates a varable length (ɛ n, δ)-sk wth ɛ n vanshng to 0 as n and average length greater than nc(m P XM ) O( n log n). (12) VI. TECHNICAL RESULTS AND PROOFS Ths secton contans the proofs of our results. We begn by notng a few propertes of the mathematcal quanttes nvolved n our proofs. A. Propertes of CO regon and related quanttes Wth a general subset A M n the role of M, we defne the notatons R (A) and H σ(a), σ Σ(A) n a smlar manner as n (4) and (3), respectvely. Our frst lemma notes some smple propertes of H σ (A) and R (A). Lemma 7. For A M and σ Σ(A), the followng relatons hold between R (A) and H σ(a): R (A) = H σf (A); (13) A R (A) = H σf (A) H(X A X ), A; (14) [ ] R (A) H(X A X A\B ) = B H σf (A) H σb (A), (15) B where the fnal equalty holds for every B A, wth the shorthand σ B for the partton σ B (A) Σ(A) gven by {{A \ B}, {} : B}.

25 25 Furthermore, R (A) satsfes the followng propertes: Rj (A) R (A) = H(X A X ), A; (16) j A R (A) R j (A) = H(X ) H(X j ),, j A. (17) Fnally, for A M and σ Σ(A), smlar results holds for R σ (A σ ), 1 σ, wth A σ n place of A. Proof: Snce (R (A) : A) s the soluton of R j = H(X A X ), A, j A j by takng the summaton of all the constrants and by dvdng by A 1, we have (13). Then, by subtractng the constrant for from (13), we have (14). From (14), for every B A t holds that R (A) = B H σf (A) H(X A X ). (18) B B Also, [ ] H σb (A) = 1 H(X A X ) + H(X A X B A\B ), B whch s equvalent to H(X A X ) = B H σb (A) H(X A X A\B ). B Combnng ths wth (18), we have (15). By takng the dfference of (13) and (14), we have (16); (17) also follows from (14). The fnal statement s proved exactly n the same manner by regardng X σ as a sngle random varable. Next, we prove another useful relaton between H and R showng that the dfference B R (A) H σf (B) must have the same sgn as H σb (A) H σf (A), where B denotes A\B and, as before, we have used the shorthand σ B for the partton σ B (A) of A. Lemma 8. For every B A M wth B = A\B, R (A) = H σf (B) + B B [ ] H σb (A) H σf (A). (19) B 1 B