Stochastic Proximal Algorithms with Applications to Online Image Recovery

Transcription

1 1/24 Stochastic Proximal Algorithms with Applications to Online Image Recovery Patrick Louis Combettes 1 and Jean-Christophe Pesquet 2 1 Mathematics Department, North Carolina State University, Raleigh, USA 2 Center for Visual Computing, CentraleSupelec, University Paris-Saclay, Grande Voie des Vignes, Châtenay-Malabry, France S 3 Seminar - 24 March 2017

2 Outline 2/24 1. Introduction 2. Stochastic Forward-Backward 3. Monotone Inclusion Problems 4. Primal-Dual Extension 5. Application 6. Conclusion

3 Context 3/24 Need for fast optimization methods over the last decade Why?

4 Context 3/24 Need for fast optimization methods over the last decade Why? Interest in nonsmooth cost functions (sparsity) Need for optimal processing of massive datasets (big data) large number of variables (inverse problems) large number of observations (machine learning) Use of more sophisticated data structures (graph signal processing)

5 Variational formulation 4/24 GOAL: where minimize x H f(x) + h(x), H: signal space (real Hilbert space) f Γ 0 (H): class of convex lower-semicontinuous functions from H to ], + ] with a nonempty domain h: H R: differentiable convex function such that h is ϑ 1 -Lipschitz continuous with ϑ ]0, + [ F = Argmin(f + h) assumed to be nonempty.

6 Algorithm 5/24 CLASSICAL SOLUTION [Combettes and Wajs ] FORWARD-BACKWARD ALGORITHM ( n N) x n+1 = x n + λ n ( proxγnf (x n γ n h(x n )) x n ), where λ n ]0, 1], γ n ]0, 2ϑ[, and prox γnf of γ n f [Moreau ]: is the proximity operator prox γnf : x argmin y H f(y) + 1 2γ n x y 2. SPECIAL CASES: projected gradient method, iterative soft thresholding, Landweber algorithm,...

7 Algorithm 5/24 CLASSICAL SOLUTION [Combettes and Wajs ] FORWARD-BACKWARD ALGORITHM ( n N) x n+1 = x n + λ n ( proxγnf (x n γ n h(x n )) x n ), In the context of online processing and machine learning, what to do if h and f are not known exactly?

8 Proposed Solution 6/24 STOCHASTIC FB ALGORITHM ( n N) x n+1 = x n + λ n ( proxγnf n (x n γ n u n ) + a n x n ), where λ n ]0, 1] and γ n ]0, 2ϑ[

9 Proposed Solution 6/24 STOCHASTIC FB ALGORITHM ( n N) x n+1 = x n + λ n ( proxγnf n (x n γ n u n ) + a n x n ), where λ n ]0, 1] and γ n ]0, 2ϑ[ f n Γ 0 (H): approximation to f

10 Proposed Solution 6/24 STOCHASTIC FB ALGORITHM ( n N) x n+1 = x n + λ n ( proxγnf n (x n γ n u n ) + a n x n ), where λ n ]0, 1] and γ n ]0, 2ϑ[ f n Γ 0 (H): approximation to f u n second-order random variable: approximation to h(x n )

11 Proposed Solution 6/24 STOCHASTIC FB ALGORITHM ( n N) x n+1 = x n + λ n ( proxγnf n (x n γ n u n ) + a n x n ), where λ n ]0, 1] and γ n ]0, 2ϑ[ f n Γ 0 (H): approximation to f u n second-order random variable: approximation to h(x n ) a n second-order random variable: possible additional error term.

12 Assumptions Let X = (X n ) n N be a sequence of sigma-algebras such that ( n N) σ(x 0,..., x n ) X n X n+1. where σ(x 0,..., x n ) is the smallest σ-algebra generated by x 0,..., x n. l + (X ): set of sequences of [0, + [-valued random variables (ξ n ) n N such that ( n N) ξ n is X n -measurable and { l 1 +(X ) = (ξ n ) n N l + (X ) } ξ n < + P-a.s. n N l + (X ) = { (ξ n ) n N l + (X ) sup n N } ξ n < + P-a.s.. 7/24

13 Assumptions Let X = (X n ) n N be a sequence of sigma-algebras such that ( n N) σ(x 0,..., x n ) X n X n+1. where σ(x 0,..., x n ) is the smallest σ-algebra generated by x 0,..., x n. Assumptions on the gradient approximation: n N λn E(u n X n ) h(x n ) < +. For every z F, there exist sequences (τ n ) n N l +, (ζ n (z)) n N l + (X ) such that n N λn ζ n (z) < + and ( n N) E( u n E(u n X n ) 2 X n ) τ n h(x n ) h(z) 2 + ζ n (z). 7/24

14 Assumptions Let X = (X n ) n N be a sequence of sigma-algebras such that ( n N) σ(x 0,..., x n ) X n X n+1. where σ(x 0,..., x n ) is the smallest σ-algebra generated by x 0,..., x n. Assumptions on the prox approximation: There exist sequences (α n ) n N and (β n ) n N in [0, + [ such that n N λn α n < +, n N λ nβ n < +, and ( n N)( x H) prox γnf n x prox γnf x α n x +β n. n N λ n E( an 2 X n ) < +. 7/24

15 Assumptions Let X = (X n ) n N be a sequence of sigma-algebras such that ( n N) σ(x 0,..., x n ) X n X n+1. where σ(x 0,..., x n ) is the smallest σ-algebra generated by x 0,..., x n. Assumptions on the algorithm parameters: inf n N γ n > 0, sup n N τ n < +, and sup n N (1 + τ n )γ n < 2ϑ. Either inf n N λ n > 0 or [ γ n γ, n N τ n < +, and n N λ n = + ]. 7/24

16 Convergence Result 8/24 Under the previous assumptions, the sequence (x n ) n N generated by the algorithm converges weakly a.s. to an F-valued random variable. REMARKS: Related works: [Rosasco et al , Atchadé et al ] Result valid for non vanishing step sizes (γ n ) n N. We do not need to assume that ( n N) E(u n X n ) = h(x n ). Proof based on properties of stochastic quasi-fejér sequences [Combettes and Pesquet 2015, 2016].

17 Stochastic Quasi-Fejér Sequences Let φ: [0, + [ [0, + [, φ(t) + as t + Deterministic definition: A sequence (x n ) n N in H is Fejér monotone with respect to F if for every z F, 9/24 ( n N) φ( x n+1 z ) φ( x n z )

18 Stochastic Quasi-Fejér Sequences Let φ: [0, + [ [0, + [, φ(t) + as t + Stochastic definition 1: A sequence (x n ) n N of H-valued random variables is stochastically Fejér monotone with respect to F if, for every z F, ( n N) E(φ( x n+1 z ) X n ) φ( x n z ) 9/24

19 Stochastic Quasi-Fejér Sequences Let φ: [0, + [ [0, + [, φ(t) + as t + Stochastic definition 2: A sequence (x n ) n N of H-valued random variables is stochastically quasi-fejér monotone with respect to F if, for every z F, there exist (χ n (z)) n N l 1 +(X ), (ϑ n (z)) n N l + (X ), and (η n (z)) n N l 1 +(X ) such that ( n N) E(φ( x n+1 z ) X n )+ϑ n (z) (1+χ n (z))φ( x n z )+η n (z) 9/24

20 9/24 Stochastic Quasi-Fejér Sequences Let φ: [0, + [ [0, + [, φ(t) + as t + Stochastic definition 2: A sequence (x n ) n N of H-valued random variables is stochastically quasi-fejér monotone with respect to F if, for every z F, there exist (χ n (z)) n N l 1 +(X ), (ϑ n (z)) n N l + (X ), and (η n (z)) n N l 1 +(X ) such that ( n N) E(φ( x n+1 z ) X n )+ϑ n (z) (1+χ n (z))φ( x n z )+η n (z) Suppose (x n ) n N is stochastically quasi-fejér monotone w.r.t. F. Then ( z F) [ n N ϑ n(z) < + P-a.s. ] [W(x n ) n N F P-a.s.] [(x n ) n N converges weakly P-a.s. to an F-valued random variable]. W(x n ) n N : set of weak sequential cluster points of (x n ) n N.

21 10/24 More General Problem GOAL: where Find x H such that 0 Ax + Bx, A: H 2 H : maximally monotone operator, i.e. (x, u) gra A ( (y, v) gra A) x y u v 0. If A is maximally monotone, then its resolvent J A = (Id + A) 1 is a firmly nonexpansive operator from H to H.

22 10/24 More General Problem GOAL: where Find x H such that 0 Ax + Bx, A: H 2 H : maximally monotone operator, i.e. (x, u) gra A ( (y, v) gra A) x y u v 0. B: H H: ϑ-cocoercive operator, with ϑ ]0, + [, i.e. ( x H)( y H) x y Bx By ϑ Bx By 2, F = zer (A + B) assumed to be nonempty. EXAMPLE: A = f with f Γ 0 (H) and B = h with h convex with a ϑ 1 -Lipschitzian gradient.

23 Proposed Solution 11/24 STOCHASTIC FB ALGORITHM ( n N) x n+1 = x n + λ n ( JγnA n (x n γ n u n ) + a n x n ), where λ n ]0, 1] and γ n ]0, 2ϑ[

24 Proposed Solution 11/24 STOCHASTIC FB ALGORITHM ( n N) x n+1 = x n + λ n ( JγnA n (x n γ n u n ) + a n x n ), where λ n ]0, 1] and γ n ]0, 2ϑ[ J γna n : resolvent of a maximally monotone operator γ n A n : H 2 H approximating γ n A

25 Proposed Solution 11/24 STOCHASTIC FB ALGORITHM ( n N) x n+1 = x n + λ n ( JγnA n (x n γ n u n ) + a n x n ), where λ n ]0, 1] and γ n ]0, 2ϑ[ J γna n : resolvent of a maximally monotone operator γ n A n : H 2 H approximating γ n A u n second-order random variable: approximation to Bx n

26 Proposed Solution 11/24 where STOCHASTIC FB ALGORITHM ( n N) x n+1 = x n + λ n ( JγnA n (x n γ n u n ) + a n x n ), λ n ]0, 1] and γ n ]0, 2ϑ[ J γna n : resolvent of a maximally monotone operator γ n A n : H 2 H approximating γ n A u n second-order random variable: approximation to Bx n a n second-order random variable: possible additional error term

27 Convergence Conditions Let X = (X n ) n N be a sequence of sigma-algebras such that ( n N) σ(x 0,..., x n ) X n X n+1. where σ(x 0,..., x n ) is the smallest σ-algebra generated by x 0,..., x n. Assumptions on the approximation to the cocoercive operator: 12/24 n N λn E(u n X n ) Bx n < +. For every z F, there exist sequences (τ n ) n N l +, (ζ n (z)) n N l + (X ) such that n N λn ζ n (z) < + and ( n N) E( u n E(u n X n ) 2 X n ) τ n Bx n Bz 2 + ζ n (z).

28 Convergence Conditions Let X = (X n ) n N be a sequence of sigma-algebras such that ( n N) σ(x 0,..., x n ) X n X n+1. where σ(x 0,..., x n ) is the smallest σ-algebra generated by x 0,..., x n. Assumptions on the resolvent approximation: There exist sequences (α n ) n N and (β n ) n N in [0, + [ such that n N λn α n < +, n N λ nβ n < +, and ( n N)( x H) J γna n x J γnax α n x + β n. n N λ n E( an 2 X n ) < +. 12/24

29 Convergence Conditions Let X = (X n ) n N be a sequence of sigma-algebras such that ( n N) σ(x 0,..., x n ) X n X n+1. where σ(x 0,..., x n ) is the smallest σ-algebra generated by x 0,..., x n. Assumptions on the algorithm parameters: inf n N γ n > 0, sup n N τ n < +, and sup n N (1 + τ n )γ n < 2ϑ. Either inf n N λ n > 0 or [ γ n γ, n N τ n < +, and n N λ n = + ]. 12/24

30 Convergence Result 13/24 Under the previous assumptions, the sequence (x n ) n N generated by the algorithm converges weakly a.s. to an F-valued random variable. In addition if A or B is demiregular at every z F, then the sequence (x n ) n N generated by the algorithm converges strongly a.s. to an F-valued random variable. A is demiregular at x dom A if, for every sequence (x n, u n ) n N in gra A and every u Ax such that x n x and u n u, we have x n x. Example: A strongly monotone, i.e. there exists α ]0, + [ such that A αid is monotone.

31 Primal-Dual Splitting 14/24 GOAL: where minimize x H H: real Hilbert space f Γ 0 (H) f(x) + q g k (L k x) + h(x) k=1 h: H R: differentiable convex function with ϑ 1 -Lipschitz continuous gradient g k Γ 0 (G k ) with G k real Hilbert space L k : bounded linear operator from H to G k x H such that 0 f(x) + q k=1 L k g k(l k x) + h(x).

32 15/24 Reformulation Let K = H G with G = G 1 G q g : G ], + ] : v q k=1 g k(v k ) L: H G: x ( L k x ) 1 k q A: K 2 K : (x, v) ( f(x) + L v ) ( Lx + g (v) ) B: K K: (x, v) ( h(x), 0 ) V : K K: (x, v) ( ρ 1 x L v, Lx + U 1 v ) with U = Diag(σ 1 Id,..., σ q Id ) with (ρ, σ 1,..., σ q ) ]0, + [ q+1 and ρ q k=1 σ k L k 2 < 1. In the renormed space (K, V ), V 1 A is maximally monotone and V 1 B is cocoercive. In addition, finding a zero of the sum of these operators is equivalent to finding a pair of primal-dual solutions.

33 Resulting Algorithm where STOCHASTIC PRIMAL-DUAL ALGORITHM for n = 0, 1,... ( q ) y n = prox ρfn (x ) n ρ L k v k,n + u n + b n k=1 x n+1 = x n + λ n (y n x n ) for k = 1,..., q wk,n = prox σk gk( vk,n + σ k L k (2y n x n ) ) + c k,n v k,n+1 = v k,n + λ n (w k,n v k,n ). λ n ]0, 1] with n N λ n = + and ( ρ 1 q k=1 σ k L k 2) ϑ > 1/2 16/24

34 Resulting Algorithm where STOCHASTIC PRIMAL-DUAL ALGORITHM for n = 0, 1,... ( q ) y n = prox ρfn (x ) n ρ L k v k,n + u n + b n k=1 x n+1 = x n + λ n (y n x n ) for k = 1,..., q wk,n = prox σk gk( vk,n + σ k L k (2y n x n ) ) + c k,n v k,n+1 = v k,n + λ n (w k,n v k,n ). f n Γ 0 (H): approximation to f 16/24

35 Resulting Algorithm where STOCHASTIC PRIMAL-DUAL ALGORITHM for n = 0, 1,... ( q ) y n = prox ρfn (x ) n ρ L k v k,n + u n + b n k=1 x n+1 = x n + λ n (y n x n ) for k = 1,..., q wk,n = prox σk gk( vk,n + σ k L k (2y n x n ) ) + c k,n v k,n+1 = v k,n + λ n (w k,n v k,n ). u n second-order random variable: approximation to h(x n ) 16/24

36 Resulting Algorithm where STOCHASTIC PRIMAL-DUAL ALGORITHM for n = 0, 1,... ( q ) y n = prox ρfn (x ) n ρ L k v k,n + u n + b n k=1 x n+1 = x n + λ n (y n x n ) for k = 1,..., q wk,n = prox σk gk( vk,n + σ k L k (2y n x n ) ) + c k,n v k,n+1 = v k,n + λ n (w k,n v k,n ). b n and c n second-order random variables: possible additional error terms 16/24

37 Resulting Algorithm STOCHASTIC PRIMAL-DUAL ALGORITHM for n = 0, 1,... ( q ) y n = prox ρfn (x ) n ρ L k v k,n + u n + b n k=1 x n+1 = x n + λ n (y n x n ) for k = 1,..., q wk,n = prox σk gk( vk,n + σ k L k (2y n x n ) ) + c k,n v k,n+1 = v k,n + λ n (w k,n v k,n ). 17/24 REMARKS: Extension of the deterministic algorithms in [Esser et al 2010] [Chambolle and Pock 2011] [Vũ 2013] [Condat 2013] Parallel structure No inversion of operators related to (L k ) 1 k q required.

38 18/24 Assumptions Let X = (X n ) n N be a sequence of sigma-algebras such that ( n N) σ ( ) x n, v n 0 n n X n X n+1. Assumptions on the gradient approximation: n N λn E(u n X n ) h(x n ) < +. For every z F, there exists (ζ n (z)) n N l + (X ) such that n N λn ζ n (z) < + and ( n N) E( u n E(u n X n ) 2 X n ) τ n h(x n ) h(z) 2 + ζ n (z).

39 18/24 Assumptions Let X = (X n ) n N be a sequence of sigma-algebras such that ( n N) σ ( ) x n, v n 0 n n X n X n+1. Assumptions on the prox approximations: There exist sequences (α n ) n N and (β n ) n N in [0, + [ such that n N λn α n < +, n N λ nβ n < +, and ( n N)( x H) prox γnf n x prox γnf x α n x +β n. n N λ n E( bn 2 X n ) < + and n N λ n E( cn 2 X n ) < +.

40 Convergence Result F: set of solutions to the primal problem F : set of solutions to the dual problem 19/24 Under the previous assumptions, the sequence (x n ) n N converges weakly a.s. to an F-valued random variable and the sequence (v n ) n N converges weakly a.s. to an F -valued random variable.

41 Online Image Recovery 20/24 OBSERVATION MODEL ( n N) z n = K n x + e n, where x H = R N : unknown image K n : R M N -valued random matrix e n : R M -valued random noise vector. OBJECTIVE recover x from (K n, z n ) n N.

42 21/24 Application of Primal-Dual Algorithm FORMULATION Mean square error criterion ( x R N ) h(x) = 1 2 E K 0x z 0 2, assuming that (K n, z n ) n N are identically distributed Statistics of (K n, z n ) n N learnt online Approximation to h(x n ): u n = 1 m n+1 1 Kn m (K n x n z n ) n+1 n =0 where (m n ) n N is strictly increasing sequence in N f and g 1 L 1 (q = 1): regularization terms

43 Application of Primal-Dual Algorithm Mean square error criterion ( x R N ) h(x) = 1 2 E K 0x z 0 2, assuming that (K n, z n ) n N are identically distributed Statistics of (K n, z n ) n N learnt online Approximation to h(x n ): u n = 1 m n+1 1 Kn m (K n x n z n ) n+1 n =0 where (m n ) n N is strictly increasing sequence in N recursive computation: u n = R n x n c n with R n = 1 m n+1 1 m n+1 n =0 K n K n = 21/24 m n R n m n+1 1 Kn m n+1 m K n. n+1 n =m n

44 Application of Primal-Dual Algorithm 21/24 CONDITIONS FOR CONVERGENCE (K n, e n ) n N is an i.i.d. sequence such that E K 0 4 < + and E e 0 4 < +. Approximation to h(x n ): u n = 1 m n+1 1 Kn m (K n x n z n ) n+1 n =0 where (m n ) n N is strictly increasing sequence in N that m n = O(n 1+δ ) with δ ]0, + [. λ n = O(n κ ), where κ ]1 δ, 1] [0, 1]. f n f and the domain of f is bounded. b n 0 and c n 0. such

45 22/24 Simulation example Grayscale image of size with pixel values in [0, 255] Stochastic blur (uniform i.i.d. subsampling of a uniform 5 5 blur performed in the discrete Fourier domain with 70% frequency bins set to zero). Additive white N (0, 5 2 ) noise. f = ι [0,255] N Parameter choice: and g 1 L 1 = isotropic total variation. ( n N) { m n = n 1.1 λ n = (1 + (n/500) 0.95 ) 1.

46 Simulation example 22/24 Original image x Restored image (SNR = 28.1 db) Degraded image 1 (SNR = 0.14 db) Degraded image 2 (SNR = 12.0 db)

47 Simulation example 22/ x n x versus the iteration number n.

48 Conclusion 23/24 Investigation of stochastic variants of Forward-Backward and Primal-Dual proximal algorithms. Stochastic approximations to both smooth and non smooth convex functions. Extension to monotone inclusion problems. Theoretical guaranties of convergence. Novel application to online image recovery.

49 Some references P. L. Combettes and V. R. Wajs, Signal recovery by proximal forward-backward splitting Multiscale Model. Simul., vol. 4, pp , P. L. Combettes and J.-C. Pesquet Proximal splitting methods in signal processing in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, H. H. Bauschke, R. Burachik, P. L. Combettes, V. Elser, D. R. Luke, and H. Wolkowicz editors. Springer-Verlag, New York, pp , P. Combettes and J.-C. Pesquet Stochastic quasi-fejér block-coordinate fixed point iterations with random sweeping SIAM Journal on Optimization, vol. 25, no. 2, pp , July N. Komodakis and J.-C. Pesquet Playing with duality: An overview of recent primal-dual approaches for solving large-scale optimization problems IEEE Signal Processing Magazine, vol. 32, no 6, pp , Nov P. L. Combettes and J.-C. Pesquet Stochastic approximations and perturbations in forward-backward splitting for monotone operators Pure and Applied Functional Analysis, vol. 1, no 1, pp , Jan /24