Partial Lanczos SVD methods for R

Transcription

1 Partial Lanczos SVD methods for R Bryan Lewis 1, adapted from the work of Jim Baglama 2 and Lothar Reichel 3 1 REvolution Computing 2 University of Rhode Island 3 Kent State University UseR 2009

2 Outline SVD and partial SVD Partial Lanczos bidiagonalization The irlba package

3 SVD Let A R l n, l n. A = n j=1 σ j u j v T j, { vj T v k = uj T 1 if j = k, u k = 0 o.w., u j R l, v j R n, j = 1, 2,..., n, and σ 1 σ 2 σ n 0.

4 Partial SVD Let k < n. Ã k := k j=1 σ j u j v T j

5 Partial Lanczos bi-diagonalization Start with a given vector p 1. Compute m steps of the Lanczos process: AP m = Q m B m A T Q m = P m Bm T + r m em, T B m R m m, P m R n m, Q m R l m, P T mp m = Q T mq m = I m, r m R n, P T mr m = 0, P m = [p 1, p 2,..., p m ].

6 Approximating partial SVD with partial Lanczos bi-diagonalization A T AP m = A T Q m B m = P m BmB T m + r m emb T m,

7 Approximating partial SVD with partial Lanczos bi-diagonalization A T AP m = A T Q m B m = P m BmB T m + r m emb T m, AA T Q m = AP m B T m + Ar m e T m, = Q m B m B T m + Ar m e T m.

8 Approximating partial SVD with partial Lanczos bi-diagonalization Let B m = m σj B uj B j=1 ( v B j ) T.

9 Approximating partial SVD with partial Lanczos bi-diagonalization Let B m = m σj B uj B j=1 ( v B j ) T. ( ) i.e., B m vj B = σj B uj B, and Bmu T j b = σj B vj B. Define:

10 Approximating partial SVD with partial Lanczos bi-diagonalization Let B m = m σj B uj B j=1 ( v B j ) T. ( ) i.e., B m vj B = σj B uj B, and Bmu T j b = σj B vj B. Define: σ j := σ B j, ũ j := Q m u B j, ṽ j := P m v B j.

11 Partial SVD approximation of A Aṽ j = AP m vj B = Q m B m vj B = σ B j Q m u B j = σ j ũ j,

12 Partial SVD approximation of A Aṽ j = AP m vj B = Q m B m vj B = σ B j Q m u B j = σ j ũ j, A T ũ j = A T Q m u B j = P m B T mu B j + r m e T mu B j = σ B j P m v B j + r m e T mu B j = σ j ṽ j + r m e T mu B j.

13 Augment and restart 1. Compute the Lanczos process up to step m. 2. Compute k < m approximate singular vectors. 3. Orthogonalize against the approximate singular vectors to get a new starting vector. 4. Continue the Lanczos process with the new starting vector for m more steps. 5. Check for convergence tolerance and exit if met. 6. GOTO 1.

14 artial Lanczos SVD methods for R Sketch of the augmented process... P k+1 := [ṽ 1, ṽ 2,..., ṽ k, p m+1 ],

15 artial Lanczos SVD methods for R Sketch of the augmented process... P k+1 := [ṽ 1, ṽ 2,..., ṽ k, p m+1 ], A P k+1 = [ σ 1 ũ 1, σ 1 ũ 2,..., σ k ũ k, Ap m+1 ]

16 artial Lanczos SVD methods for R Sketch of the augmented process... P k+1 := [ṽ 1, ṽ 2,..., ṽ k, p m+1 ], A P k+1 = [ σ 1 ũ 1, σ 1 ũ 2,..., σ k ũ k, Ap m+1 ] Orthogonalize Ap m+1 against {ũ j } k j=1 : Ap m+1 = k j=1 ρ jũj + r k.

17 artial Lanczos SVD methods for R Sketch of the augmented process... P k+1 := [ṽ 1, ṽ 2,..., ṽ k, p m+1 ], A P k+1 = [ σ 1 ũ 1, σ 1 ũ 2,..., σ k ũ k, Ap m+1 ] Orthogonalize Ap m+1 against {ũ j } k j=1 : Ap m+1 = k j=1 ρ jũj + r k. Q k+1 := [ũ 1, ũ 2,..., ũ k, r k / r k ], σ 1 ρ 1 σ 2 ρ 2 B k+1 :=... ρk. r k

18 artial Lanczos SVD methods for R Sketch of the augmented process... P k+1 := [ṽ 1, ṽ 2,..., ṽ k, p m+1 ], A P k+1 = [ σ 1 ũ 1, σ 1 ũ 2,..., σ k ũ k, Ap m+1 ] Orthogonalize Ap m+1 against {ũ j } k j=1 : Ap m+1 = k j=1 ρ jũj + r k. Q k+1 := [ũ 1, ũ 2,..., ũ k, r k / r k ], σ 1 ρ 1 σ 2 ρ 2 B k+1 :=... ρk. r k A P k+1 = Q k+1 Bk+1.

19 The irlba package Usage: irlba (A, nu = 5, nv = 5, adjust = 3, aug = "ritz", sigma = "ls", maxit = 1000, reorth = 1, tol = 1e-06, V = NULL)

20 Small example > A<-matrix (rnorm(5000*5000),5000,5000) > require (irlba) > system.time (L<-irlba (A,nu=5,nv=5)) user system elapsed > gc() used (Mb)... max used (Mb) Ncells Vcells

21 artial Lanczos SVD methods for R Small example (continued) > system.time (S<-svd(A,nu=5,nv=5)) user system elapsed > gc() used (Mb)... max used (Mb) Ncells Vcells

22 Small example (continued) > system.time (S<-svd(A,nu=5,nv=5)) user system elapsed > gc() used (Mb)... max used (Mb) Ncells Vcells > sqrt (crossprod(s$d[1:5]-l$d)/crossprod(s$d[1:5])) [,1] [1,] e-12

23 Large examples (live demo) The R implementation of IRLBA supports: Dense real/complex in-process matrices (normal R matrices) Sparse real in-process matrices (Matrix) Dense, real in- or out-of-process huge matrices with bigmemory + bigalgebra

24 References jbaglama 3. reichel 4. blewis 5.