Markov Chain Monte Carlo Algorithms for Lattice Gaussian Sampling

Transcription

1 Markov Chain Monte Carlo Algoriths for Lattice Gaussian Sapling Zheng Wang and Cong Ling Departent of EEE Iperial College London London, SW7 2AZ, United Kingdo Eail: z.wang10, Guillaue Hanrot École Norale Supérieure de Lyon LIP (CNRS, ENS Lyon, UCBL, INRIA) 46 Allée d Italie, Lyon Cedex 07, France Eail:Guillaue.Hanrot@ens-lyon.fr arxiv: v1 [cs.it] 7 May 2014 Abstract To be considered for an IEEE Jack Keil Wolf ISIT Student Paper Award. Sapling fro a lattice Gaussian distribution is eerging as an iportant proble in various areas such as coding and cryptography. The default sapling algorith Klein s algorith yields a distribution close to the lattice Gaussian only if the standard deviation is sufficiently large. In this paper, we propose the Markov chain Monte Carlo (MCMC) ethod for lattice Gaussian sapling when this condition is not satisfied. In particular, we present a sapling algorith based on Gibbs sapling, which converges to the target lattice Gaussian distribution for any value of the standard deviation. To iprove the convergence rate, a ore efficient algorith referred to as Gibbs-Klein sapling is proposed, which saples by using Klein s algorith. We show that Gibbs-Klein sapling yields a distribution close to the target lattice Gaussian, under a less stringent condition than that of the original Klein algorith. I. INTRODUCTION The lattice Gaussian distribution is eerging as a coon thee in various areas. In atheatics, Banaszczyk [1] firstly used it to prove the transference theores of lattices. In coding, it iics Shannon s Gaussian rando coding technique, yet perits lattice decoding. Forney applied the lattice Gaussian distribution to obtain the full shaping gain in lattice coding [2] (see also [3]). Recently, it has been used to achieve the capacity of the Gaussian channel [4] and to approach the secrecy capacity of the Gaussian wiretap channel [5], respectively. Sapling fro the lattice Gaussian has also been used in lattice decoding for the ulti-input ulti-output syste [6], [7]. In cryptography, lattice Gaussians have becoe a central tool in the construction of any priitives. Micciancio and Regev used it to propose lattice-based cryptosystes based on the worst-case hardness assuptions [8], and recently, it has underpinned the fully-hooorphic encryption for cloud coputing [9]. The key fact is again that a vector distributed as a lattice Gaussian centered at c with a sall standard deviation is typically very close to c. To illustrate why this ight be useful in cryptography, note that if one knows a short basis of the lattice, one can efficiently produce such a vector [10], while disclosing no inforation on the short basis since the lattice Gaussian distribution does not depend on the particular basis. Thus, in both coding and cryptography, efficient sapling algoriths for the lattice Gaussian as well as a good understanding on how the coplexity depends on the standard deviation is an iportant issue. However, in contrast to sapling fro the continuous Gaussian distribution, it is not at all straightforward to saple fro a discrete Gaussian distribution over a lattice. At present, the default sapling algorith for lattices is due to Klein, originally proposed for boundeddistance decoding [11] (see also [12], [13] for variations and [4] for an algorith for lattices of Construction A). It was shown in [10] that Klein s algorith saples within a negligible statistical distance fro the lattice Gaussian distribution only if the standard deviationσ ω( log n) ax 1 i n b i, where n is the lattice diension and b i s are the Gra- Schidt vectors of the lattice basis. Unfortunately, such a requireent of σ can be excessive, rendering Klein s algorith inapplicable to any cases of interest. Markov chain Monte Carlo (MCMC) ethods attept to saple fro the target distribution of interest by building a Markov chain, which randoly generate the next saple conditioned on the previous saples. As a ajor algorith of MCMC, Gibbs sapling [14] constructs a Markov chain which gradually converges to the target distribution by only considering univariate sapling at each step. In this paper, we introduce the Gibbs algorith into lattice Gaussian sapling and propose a ore efficient -based algorith naed as Gibbs-Klein sapling. In contrast to conventional ed sapling which is coputationally ore deanding, the proposed algorith takes advantages of Klein s algorith as a building. The proposed algoriths are applicable in the scenario σ < ω( log n) ax 1 i n b i. To the best of our knowledge, this is the first tie that MCMC ethods are used in lattice Gaussian distributions. Different fro previous works on Gibbs sapling for signal detection of finite constellations [15] [17], here we are concerned with countably infinite state spaces and with siulating Gaussian distributions over a lattice. It is worth pointing out that although the underlying Markov chain converges to the stationary distribution for all values of σ, the convergence is expected to becoe very slow when σ becoes sall, since for very sall σ we would solve the closest vector proble (CVP) and shortest vector proble (SVP) with high

2 Algorith 1 Klein s Algorith Input: B, σ, c Output: Bx Λ 1: let B QR and c Q T c 2: for i n,..., 1 do 3: let α i σ r and x i,i i c i 4: saple x i fro D Z,αi, x i 5: end for 6: return Bx n ji+1 ri,jxj probability. The rest of this paper is organized as follows. Section II introduces lattice Gaussian distributions and briefly reviews Klein s algorith. In Section III, the conventional Gibbs and the new Gibbs-Klein sapling algoriths are proposed for lattice Gaussians, followed by a theoretical analysis in Section IV. Section V presents the siulation results. II. LATTICE GAUSSIAN DISTRIBUTIONS Let B [b 1,...,b n ] R n consist of n linearly independent vectors. The n-diensional lattice Λ based on B is defined by Λ L(B) {Bx : x Z n }, (1) where B is known as the lattice basis. We define the Gaussian function centered at c R n for standard deviation σ > 0 as ρ σ,c (z) e z c 2 2σ 2, (2) for all z R n. Then, the discrete Gaussian distribution over Λ is defined as D Λ,σ,c (x) ρ σ,c(bx) ρ σ,c (Λ) 2σ 2 Bx c 2 x Z 1 n e 2σ 2 Bx c 2 (3) for all Bx Λ, where ρ σ,c (Λ) Bx Λ ρ σ,c(bx). An intuition of D Λ,σ,c (x) suggests that the closer lattice point Bx is to c, the higher probability it will be sapled. Thus, lattice Gaussian sapling can be applied to solve the CVP, and Klein s algorith was originally proposed for decoding [11]. As a randoized version of Babai s nearest-plane algorith (i.e., successive interference cancellation), Klein s algorith obtains a vector by sequentially sapling fro a 1-diensional conditional Gaussian distribution. As shown in Algorith 1, its operation has polynoial coplexity O(n 2 ) excluding QR decoposition. The paraeter σ is key to the distribution produced by Klein s algorith. Klein suggested σ in i b i / log n and this was followed/adapted in [6], [7]. In this case, Klein s algorith only yields a distribution that is lower-bounded by the Gaussian distribution. On the other hand, it was deonstrated in [10] that Klein s algorith actually saples fro D Λ,σ,c within a negligible statistical distance if σ ω( log n) ax 1 i n b i. (4) However, Gaussian sapling algoriths are lacking for the range σ < ω( log n) ax i b i. III. MCMC FOR LATTICE GAUSSIAN In this section, we introduce the concept of MCMC into lattice Gaussian sapling for the range of σ where Klein s algorith cannot reach. We further propose a ore efficient sapling algorith naed as Gibbs-Klein sapling to iprove the convergence rate. A. Gibbs Sapling for Lattice Gaussian Lattice Gaussian distribution D Λ,σ,c with σ < ω( log n) ax i b i can be seen as a coplex target distribution lacking direct sapling ethods. MCMC akes use of the conditional distribution as a tractable alternative to work with. Here we apply the Gibbs algorith to saple fro the original joint distribution D Λ,σ,c. Gibbs sapling eploys 1-diensional conditional distributions to construct the Markov chain [14], where all other variables in the distribution are unchanged in each step. In this way, we saple n rando variables fro the corresponding n univariate conditionals in a certain order instead of directly generating an n-diensional vector. Saples drawn fro the target joint distribution will be generated when the Markov chain reaches the stationary distribution. Specifically, in Gibbs sapling, each coordinate of x is sapled fro the following 1-diensional conditional distribution P(x t+1 i x t [ i] ) 2σ 2 Bxt+1 c 2 x t+1 i Z e 1 2σ 2 Bxt+1 c 2, (5) where 1 i n denotes the coordinate index of x, x t [ i] [xt 1,...,x t i 1,xt i+1,...,xt n] T, and t is the tie index of the Markov chain. It is noteworthy that there are any scan schees in Gibbs sapling and we apply the rando-scan in this paper, which eans the index i is randoly chosen at each step. The extension to other scan strategies is possible. By repeating such a procedure, an underlying Markov chain x t+1 [x t 1,...,xt i 1,xt+1 i,x t i+1,...,xt n ]T is induced, whose transition probability between two adjacent states is defined by the univariate Gibbs sapler, P(x t ;x t+1 ) P(x t+1 i x t [ i]). (6) Clearly, every two adjacent states of x differ fro each other by only one coordinate and it is easy to see that D Λ,σ,c stays invariant under such transitions. Algorith 2 gives the operation of Gibbs sapling for lattice Gaussian distributions. The initial rando variable x 0 can be chosen fro Z n arbitrarily or fro the output of a suboptial algorith, while the tie bound T is large enough to reach the stationary distribution D Λ,σ,c. With the transition probabilities (6), we ay for the infinite transition atrix P, whose (i,j)-th entry P(s i ;s j ) represents the probability of transferring to state s j fro the previous state s i. Denote by P t the transition atrix after t steps. We group in the following theore standard results about Gibbs sapling [18].

3 Algorith 2 Gibbs sapling for lattice Gaussian Input: B,σ,c,x 0 Output: x D Λ,σ,c as T 1: for t 1,..., T do 2: randoly choose coordinate indexifro{1,2,...,n} 3: saple x i fro P(x t i xt 1 4: update x t [x t 1 1,...,x t 1 i 1,x i,x t 1 i+1,...,xt 1 5: if Markov chain has reached stationarity then 6: output x t 7: end if 8: end for [ i] ) n ] T Proposition 1. Given the invariant distribution D Λ,σ,c, the Markov chain induced by the Gibbs sapler is irreducible, aperiodic and reversible (hence positive recurrent), and converges to the stationary distribution in the total variation (TV) distance as t : li t Pt (x; ) D Λ,σ,c TV 0, (7) for all states x Z n, where P t (x; ) denotes the row of P t corresponding to initial state x. According to Proposition 1, if tie perits to reach the stationary distribution, the proposed Gibbs sapler will draw saples fro D Λ,σ,c no atter what value σ takes, which eans the obstacle encountered by Klein s algorith is overcoe. B. Gibbs-Klein Sapling for Lattice Gaussian Although the afore-entioned Gibbs sapler will converge to the stationary distribution eventually, the way it functions by individually sapling only one coponent each tie leads to slow convergence. Especially, for lattice bases whose coponents are highly correlated with each other, the Markov chain induced by the standard Gibbs sapling can be trapped for a long tie. To hasten convergence of the Markov chain, a new sapling algorith cobining Gibbs and Klein algoriths is proposed in the sequel. The idea of ed sapling is to saple a of coponents of x at each step [19]. Intuitively, this will lead to a faster convergence rate, which is already shown in [14]. However, sapling a is generally ore costly than coponentwise sapling. We propose to use Klein s algorith for sapling; this leads to the Gibbs-Klein. At each step of the Markov chain, the proposed Gibbs-Klein sapling randoly picks up a of coponents of x to update. For convenience, an n n perutation atrix E is applied before ing so that the s are updated in a fixed order. Specifically, if E is rando, then Gibbs-Klein sapling on randoly chosen coponents will be equivalent to saple consecutive coponents of z in a fixed order, where z E 1 x and B BE. For siplicity, we always consider the fored by the first coponents of z, naely z [z 1,...,z ] T. After QR-decoposition B QR Algorith 3 Gibbs-Klein sapling for lattice Gaussian Input: B,σ,c,,x 0 ; Output: x fro a distribution close to D Λ,σ,c as T 1: for t 1,..., T do 2: randoly generate a perutation atrix E 3: Let B BE and z E 1 x 4: Let B QR and c Q T c 5: for i,..., 1 do 6: let α i σ 7: let z t 1 i c i ji+1 ri,jzt j n 8: saple z t i fro D Z,α i, z t 1 i 9: end for 10: update z t [z t ;zt 1 [ ] ]T 11: return x t Ez t j r +1 i,j zt 1 j 12: if Markov chain has reached stationarity then 13: output x t 14: end if 15: end for and calculating c Q T c, z i in the is sapled fro the following 1-diensional distribution with the backward order fro z to z 1 : i z t [ i] ) D Z,α i, z t i, (8) where α i σ r, i,i zt [ i] [zi+1 t+1,...,zt+1,zt +1,...,zt n ]T and z i t c i ji+1 ri,jzt+1 j n j +1 r i,j zt j. Algorith 3 gives the proposed Gibbs-Klein sapling, where z t+1 [z t+1 ;zt [ ] ] is obtained after each step, and zt [ ] [z+1,...,z t n] t T. The ipleentation given in Algorith 3 is not so efficient due to repeated QR decopositions; Optiizing for better efficiency will be pursued in the future. Note that the extension to other scan strategies is also possible. IV. ANALYSIS OF GIBBS-KLEIN SAMPLING In this section, we show that the proposed Gibbs-Klein sapling algorith can induce a reversible Markov chain within a negligible error. Fro (8) and by induction, the sapling probability of z t+1 conditioned on zt [ ] is given by zt [ ] ) i1 +1 i zt [ (+1 i)]). (9) The following lea gives a closed-for expression of this conditional probability within a negligible error and the proof follows [10]. Lea 1. For a given invariant distribution D Λ,σ,c, the transition probability zt [ ]) of Gibbs-Klein algorith is within negligible statistical distance of the following distribution D 2σ 2 Bz t+1 c 2 (10) z t+1 Z 2σ 2 Bz t+1 c 2

4 if σ ω( log) ax 1 i, where z t+1 [z t+1 ;zt [ ] ]. Proof: According to (8) and (9), we have i1 i1 zt [ ] ) i1 D Z,α+1 i, z t +1 i (zt+1 +1 i ) 2σ 2 i1(c +1 i j+1 i r+1 i,jzt+1 j ) 2 1 z t+1 +1 i Ze 2σ 2 (c +1 i j+1 i r+1 i,jzt+1 j ) 2 2σ 2 c rzt z t+1 +1 i Ze 2σ 2 (r +1 i,+1 iz t+1 +1 i c+1 i+ j+2 i r+1 i,jzt+1 j ) 2 ρ L(r),σ,c (z t+1 ) i1 ρ σ(r +1 i,+1 i Z+ξ), (11) where c i c i n j +1 r i,j zt, c [c j 1,...,c ] T, ξ j+2 i r +1 i,jzj t+1 c i+i and r is the segent of R with r 1,1 to r, in the diagonal. Clearly, the effect of the subvector z t [ ] is hidden in c i. In [20], it has been deonstrated that if σ > η ε (L(r)), then i1 ρ (( ) σ( Z+ξ) 1 ε i1 ρ,1] (12) σ( Z) 1+ε which eans i1 ρ σ( Z + ξ) can be substituted by i1 ρ σ( Z) within negligible errors when ε is sufficiently sall. As shown in [10], η ε (Λ) with negligible ε is upper bounded as η ε (Λ) ω( log n) ax 1 i n b i. Therefore, if σ ω( log ) ax 1 i, zt [ ]) shown in (11) can be rewritten as zt [ ] ) ρ L(r),σ,c(z t+1 ) i1 ρ σ( Z), (13) where represents equality up to a negligible error. Because the denoinator is independent of z t+1, zt [ ] and c, it can be viewed as a constant and the output has a lattice Gaussian distribution D L(r),σ,c (z t+1 ). Then we arrive at the following proposition. Proposition 2. Suppose σ ω( log) ax 1 i bi at each step so that the negligible statistical distance is absorbed by nuerical errors. Then, within nuerical errors, the Markov chain induced by the Gibbs-Klein sapler is irreducible, aperiodic and reversible (hence positive recurrent) and converges to the stationary distribution in the total variation distance as t : for all states x Z n. li t Pt (x; ) D Λ,σ,c TV 0 (14) Proof: Let s i and s j be two adjacent states in Gibbs- Klein sapling. For size, every two adjacent states in Gibbs-Klein sapling differ fro each other by at ost coponents. For convenience, we express the as s i [x (i),x [ ] ] and s j [x (j),x [ ] ], (15) where x (i) and x (j) denote the coponents belonging to s i and s j, respectively. Then, the transition probability of Gibbs-Klein sapling is P(s i ;s j ) P(x t+1 s j x t s i ) P(x t (i) xt+1 (j) xt [ ] ) (a) P(x t+1 (j) xt [ ] ) 2σ 2 Bsj c 2 x t+1 Z 2σ 2 Bxt+1 c 2, (16) where (a) is due to the fact that x t+1 is sapled only conditioned on x t [ ]. To show the Markov chain is irreducible, we note that given a state s one can attain with positive probability in one step any state s which shares > (n ) coponents with s. Now, if s and s have, say, d < n coponents in coon, there is always a positive probability that after each step they get exactly one ore coponent in coon. So we can go in n d steps fro one to the other. But as soon as > 2, we can assue that at the first step we get two ore coponents in coon, and then one at each further step, so we can go with positive probability in n d 1 steps. On the other hand, it is clear to see that the nuber of steps required to ove between any two states (can be the sae state) is arbitrary without any liitation to be a ultiple of soe integer. Put another way, the chain is not forced into soe cycle with fixed period between certain states. Therefore, the Markov chain is aperiodic. As for reversibility, it is no hard to check that the following relationship holds D Λ,σ,c (s i )P(s i ;s j ) D Λ,σ,c (s j )P(s j ;s i ) (17) with the sae expression 2σ 2 Bsi c 2 x Z n 2σ 2 Bx c 2 2σ 2 Bsj c 2 x t+1 Z 2σ 2 Bxt+1 c 2, (18) within negligible errors. Thus, the conclusion follows, copleting the proof. The advantages of Gibbs-Klein sapling are two-fold: copared with the conventional Gibbs sapling which only processes a single variate each tie, it is ore efficient to saple ultiple variates in a, iproving the convergence rate; on the other hand, it overcoes the liitation of Klein s sapling which requires large values of σ and extends lattice Gaussian sapling to the ore general case. V. SIMULATION RESULTS In this section, the perforances of various sapling schees are exeplified in the context of MIMO decoding. Specifically, we exaine the decoding error probabilities to assess the convergence rates. By sapling fro D Λ,σ,c, the closest lattice point will be returned with the highest probability, which iplies an effective approach to lattice decoding.

5 Bit Error Rate ZF Gibbs Sapling Gibbs Klein Sapling,2 Gibbs Klein Sapling,4 Klein Sapling ML Nuber of Iterations Fig. 1. Bit error rate versus the nuber of iterations for the uncoded 4 4 MIMO syste using 16-QAM. Fig. 1 depicts the bit error rates (BER) of different Gibbs saplers in a 4 4 uncoded MIMO syste with 16-QAM. This corresponds to lattice diension n 8. The perforances of zero-forcing (ZF) and axiu-likelihood (ML) decoding are also shown as bencharks. We assue a flat fading environent with fixed SNR (E b /N 0 15 db). The channel atrix H consists of uncorrelated coplex Gaussian fading gains with unit variance. Hx can be viewed as a lattice point in lattice Λ L(H) and detecting the transitted signal x corresponds to solving the CVP. Due to the finite constellation size, the ipleentation for discrete Gaussian sapling given in [6] is followed. Klein chose σ in 1 i n b i / log n and derived polynoial coplexityo(n Bx c 2 /in i b i 2 ) for his algorith to find the closest lattice point when it is not far fro c [11]. His derivation is essentially based on the assuption of a Gaussian distribution. However, we now know this choice of σ does not satisfy the soothing condition and thus his sapler does not really produce Gaussian saples [10]. Here, we follow Klein s choice of σ and apply the proposed Gibbs and Gibbs-Klein saplers to produce Gaussian saples fro the lattice. For a fair coparison, when the size is, we run sapling for n/ ties, and count this as a full iteration. This corresponds to one run of Klein s original algorith which saples n coponents. As shown in Fig. 1, the decoding perforance of all the sapling schees iprove with the nuber of iterations. With the sae nuber of iterations (hence the sae coplexity), the decoding perforance iproves with the size, which iplies a faster convergence rate. REFERENCES [1] W. Banaszczyk, New bounds in soe transference theores in the geoetry of nubers, Math. Ann., vol. 296, pp , [2] G. Forney and L.-F. Wei, Multidiensional constellations Part II: Voronoi constellations, IEEE J. Sel. Areas Coun., vol. 7, no. 6, pp , Aug [3] F. R. Kschischang and S. Pasupathy, Optial nonunifor signaling for Gaussian channels, IEEE Trans. Infor. Theory, vol. 39, pp , May [4] C. Ling and J.-C. Belfiore, Achieiving the AWGN channel capacity with lattice Gaussian coding, subitted to IEEE Trans. Infor. Theory, Mar. 2012, revised, Nov [Online]. Available: [5] C. Ling, L. Luzzi, J.-C. Belfiore, and D. Stehlé, Seantically secure lattice codes for the Gaussian wiretap channel, subitted to IEEE Trans. Infor. Theory, Oct. 2012, revised, Oct [Online]. Available: [6] S. Liu, C. Ling, and D. Stehle, Decoding by sapling: a randoized lattice algorith for bounded distance decoding, IEEE Trans. Infor. Theory, vol. 57, pp , Sep [7] Z. Wang and C. Ling, Decoding by sapling - part II: derandoization and soft-output decoding, IEEE Trans. Coun., vol. 61, no. 11, pp , Nov [8] D. Micciancio and O. Regev, Worst-case to average-case reuctions based on Gaussian easures, in Proc. Ann. Syp. Found. Coputer Science, Roe, Italy, Oct. 2004, pp [9] C. Gentry, A. Sahai, and B. Waters, Hooorphic encryption fro learning with errors: Conceptually-sipler, asyptotically-faster, attribute-based, in CRYPTO, [10] C. Gentry, C. Peikert, and V. Vaikuntanathan, Trapfoors for hard lattices and new cryptographic constructions, in Proc. 40th Ann. ACM Syp. Theory of Coput., Victoria, Canada, 2008, pp [11] P. Klein, Finding the closest lattice vector when it is unusually close, in ACM-SIAM Syp. Discr. Algoriths, 2000, pp [12] C. Peikert, An efficient and parallel Gaussian sapler for lattices, in CRYPTO, 2010, pp [13] Z. Brakerski, A. Langlois, C. Peikert, O. Regev, and D. Stehlé, Classical hardness of learning with errors, in STOC, 2013, pp [14] J. S. Liu, Monte Carlo Strategies in Scientific Coputing, New York: Springer-Verlag, [15] B. Farhang-Boroujeny, H. Zhu, and Z. Shi, Markov chain Monte Carlo algoriths for CDMA and MIMO counication systes, IEEE Trans. Signal Process., vol. 54, no. 5, pp , [16] B. Hassibi, M. Hansen, A. G. Diakis, H. A. J. Alshaary, and W. Xu, Optiized Markov chain Monte Carlo for signal detection in MIMO systes: an analysis of stationary distribution and ixing tie, [Online]. Available: [17] R. Chen, J. Liu, and X. Wang, Convergence analyses and coparisons of Markov chain Monte Carlo algoriths in digital counications, IEEE Trans. on Signal Process., vol. 50, no. 2, pp , [18] D. A. Levin, Y. Peres, and E. L. Wiler, Markov Chains and Mixing Tie, Aerican Matheatical Society, [19] G. O. Roberts and S. K. Sahu, Updating schees, correlation structure, ing and paraeterization for Gibbs sapler, J. Roy. Statist. Soc. Series B, 59(2): , [20] O. Regev, On lattice, learning with errors, rando linear codes, and cryptography, J. ACM, vol. 56, no. 6, pp. 34:1 34:40, ACKNOWLEDGMENT The authors would like to thank Daien Stehlé for helpful discussions.