arxiv: v1 [cs.cv] 11 Nov 2015

Transcription

1 A Continuous Max-Flow Approah to Cyli Field Reonstrution John S.H. Baxter a,b, Jonathan MLeod a,b, and Terry M. Peters a,b a Robarts Researh Institute, London, Ontario, Canada; b Western University, London, Ontario, Canada arxiv: v1 [s.cv] 11 Nov 2015 ABSTRACT Reonstrution of an image from noisy data using Markov Random Field theory has been explored by both the graph-uts and ontinuous -flow ommunity in the form of the Potts and Ishikawa models. However, neither model takes into aount the partiular yli topology of speifi intensity types suh as the hue in natural olour images, or the phase in omplex valued MRI. This paper presents yli ontinuous -flow image reonstrution whih models the intensity being reonstruted as having a fundamentally yli topology. This model omplements the Ishikawa model in that it is designed with image reonstrution in d, having the topology of the intensity spae inherent in the model while being readily extendable to an arbitrary intensity resolution. Keywords: Max-flow image reonstrution, optimal reonstrution, phase reonstrution, hue reonstrution 1. INTRODUCTION Markov random fields, speifially in the form of the Ishikawa model, 1 have previously been used to address the issue of reonstruting an image from noisy data using probabilisti priors and imum a posteriori probability optimization. The benefit of suh frameworks is their submodularity and therefore global optimizability via graph uts. 2 The primary improvement of Ishikawa models over previous graph ut approahes in reonstruting an image 3 or depth field 4 is that the topology of the intensity spae, an ordered linear topology, is fundamentally built into the funtional. Continuous -flow approahes have traditionally been seen as ontinuous analogs to earlier disrete models. Along these lines, a ontinuous Ishikawa model 5 was proposed whih maintains the same linear topology over the intensity spae but using a ontinuous, rather than disrete domain. This is espeially ruial in image reonstrution as the mitigation of disrete metrifiation artifats ahieved by the ontinuous formulation 6 has an immediate visual effet on the reonstrution. Certain types of fields, however, do not have ordered linear intensities. For example, hue in HSL and HSV olour spaes or phase in a omplex-valued image represents a yli, rather than linear, topology. These require a speialized reonstrution optimizer whih, as the Ishikawa model was to linear intensity models, sensitive to the intensity s yli topology. Although this ould in theory be aomplished through a direted ayli graph ontinous -flow (DAGMF model as desribed by Baxter et al., 7 the quadrati growth of said models is undesirable, espeially for medial appliations suh as phase proessing in suseptibility weighted magneti resonane imaging 8 in whih high intensity resolution is neessary. 2. CONTRIBUTIONS The main ontribution of this work is the development of a ontinuous -flow model desribing yli field image reonstrution, that is, reonstrution of an image in whih the image intensity is defined over a yli topology rather than a linear topology as explored by Bae et al. 5 This reonstrution algorithm is better suited for the reonstrution of yli fields suh as olour/hue in natural images or phase in omplex-valued magneti resonane images. Send orrespondene to J.S.H.B.: jbaxter@robarts.a 1

2 3. PREVIOUS WORK Previous work by Yuan et al. 6 has addressed the ontinuous binary -ut problem: E(u = (D s (xu(x + D t (x(1 u(x + S(x u(x dx s.t. u(x {0, 1} as well as the onvex relaxed ontinuous Potts Model: 9 E(u = (D L (xu L (x + S(x u L (x dx L s.t. u L (x 0 and u L (x = 1. L Both of these tehniques both used a ontinuous -flow model with augmented Lagrangian multipliers. In the ase of the onvex-relaxed ontinuous Potts model, the soure flow had infinite apaity and the osts in the funtional orresponded with onstraints on the sink flows. Bae et al. 5 extended the work on the ontinuous binary -ut problem to the ontinuous Ishikawa model: E(u = N (D L (xu L (x + S(x u L (x dx L=0 s.t. u L (x {0, 1} and u L+1 (x u L (x using similar variational methods but a tiered ontinuous graph analogous to that used by Ishikawa 1 in the disrete ase, that is, with finite apaities on intermediate flows between labels. Both ontinuous Potts and Ishikawa models are important to onsider as they were the first -flow models to be used in image reonstrution. However, they are not topologially suitable for yli fields beause the Potts model disallows any notion of label topology, and the Ishikawa model is defined only over linear orderings. Additionally, yli orderings are non-hierarhial, making them inaessible to generalized hierarhial -flow solvers 10 and direted ayli graph -flow models 7 of these orderings require quadrati time to handle linear inreases in the intensity resolution. 4. CYCLIC TOPOLOGY IN CONTINUOUS MAX-FLOW SEGMENTATION 4.1 Continuous Max-Flow Model The desired optimization model for yli field reonstrution would be: ( E = D θ(x (x + S θ(x (x θ(x dx (1 θ(x in whih θ(x is defined over a yli manifold to prevent modulation errors. One an re-express the reonstruted intensity θ(x with respet to a series of fuzzy indiator funtions: u θ (x 0 and u θ (x δ(x iff θ(x θ and π u θ (xdθ = 1 Using these indiator funtions, the data term an be re-written as: π D θ(x (xdx = D θ (xu θ (xdxdθ 2

3 and the smoothness term an be re-written as: S θ(x (x (x θ(x dx = π S θ (x (θ,x u θ (x dxdθ yielding the alternative imization formulation: π E = (D θ (xu θ (x + S θ (x u θ (x dxdθ u 0 u θ (x 1 = π u θ (xdθ and u θ (x 0 (2 The benefit of this new formulation via indiator funtions is that it resembles the previously explored ontinuous Potts model 9 with the exeption that the gradient magnitude operator is defined over the Cartesian produt of the spatial domain x with a yli intensity domain, θ [, π. Thus, the spatial domain augmented with the θ diretion forms a high dimensional ylinder as shown in Figure Primal Formulation As in the previous work, the primal formulation will be expressed as a -flow optimization, speifially: p S (xdx (3 subjet to the apaity onstraints: and the flow onservation onstraint: p,q,λ p θ (x D θ (x, q θ (x S θ (x 0 = G θ (x = div q θ (x + p θ (x p S (x. q θ (x, the spatial flow now has an additional dimension, the θ diretion. The θ diretion is equipped with a yli topology and therefore is not suseptible to modulation artifats. Similar to u θ (x, q θ (x makes use of the spatial domain augmented with the yli domain of θ as shown in Figure Primal-Dual Formulation The primal model an be onverted to a primal-dual model through the use of Lagrangian multipliers on the flow onservation onstraint G θ (x = div q θ (x + p θ (x p S (x = 0. This yields the equation: ( π p S (xdx + u θ (xg θ (xdxdθ u p,q (4 p θ (x D θ (x q θ (x S θ (x To ensure that this funtion meets the riteria of the i theorem, we must ensure that it is onvex with respet to u, onsidering p, q to be fixed, and onave with respet to p, q with u fixed. 11 For the first, it is suffiient to note that if p, q are fixed, then G is fixed as well, meaning that (4 is linear and therefore onvex with respet to u. It should also be noted that G is a linear funtion of p, q, implying the linearity (and onavity of (4 with respet to p, q. Thus, a saddle point must exist and the formulations are equivalent. 11 3

4 Figure 1: Topology used in yli -flow in whih the spatial domain, is one dimensional. The resulting topology has one more dimensional than whih enodes the intensity θ for the spatial flows, q θ (x, and indiator funtions, u θ (x. The gradient magnitude and divergene operators over q θ (x and u θ (x inlude both the spatial and the θ dimensions. 4.4 Dual Formulation To show the equivalene of the primal-dual formulation (Eq. (4 and the desired dual formulation (Eq. (1, we must show that the original formula an be reonstruted from the definition of the saddle-point. First, onsider the primal-dual formulation with the sink flows, p θ (x isolated: u p θ (x D θ (x u θ (xp θ (xdx = u θ (x 0 u θ (xd θ (xdx (5 whih implies that u θ (x 0 as if u θ (x < 0, the imization is unbounded as p θ (x. This reonstruts the data term portion of the original formulation using the fuzzy labeling funtion. The soure flow, p S in Eq. 4 an be isolated as: ( p S (xdx u p S (x π u θ (xp S (xdx = 0 (6 at the saddle point defined by 1 = π u θ(xdθ. This reonstruts the labeling onstraint, that is, that only one intensity value is assigned to eah voxel. Considering oupling the spatial domain with the yli domain of θ via a simple Cartesian produt with members taking the form z = [ x θ ]. In this oupled spae, the imization of the spatial flow funtions an be 4

5 expressed 12 as: u θ 0 q θ S θ (x θ [π,π] u θ (x div q θ (xdz = = = = θ [π,π] θ [π,π] θ [π,π] δ θ [π,π] (div(u θ (xq θ (x q θ (x u θ (x dz div(u θ (xq θ (xdx u θ (xq θ (x dsdθ q θ (x u θ (xdz θ [π,π] θ [π,π] u θ (x div q θ (xdz = q θ (x u θ (xdz q θ S θ (x q θ S θ (x ( = S θ(x θ [π,π] u θ (x u θ(x u θ (xdz = S θ (x u θ (x dz θ [π,π] θ [π,π] u θ (x div q θ (xdz = u θ 0 = u θ 0 θ [π,π] S θ (x u θ (x dz S θ(x (x θ(x dx. q θ (x u θ (xdz q θ (x u θ (xdz This yields the smoothness omponent in Eq. (1 thus showing the equivalene of the two models. 4.5 Solution to Primal-Dual Formulation The optimization problem being addressed is the augmented form of Eq. (4: ( π π G 2 θ p S (xdx + u θ (xg θ (xdxdθ (x dxdθ u p,q 2 p θ (x D θ (x q θ (x S θ (x (7 (8 in whih is a non-negative onstant whih has no effet on Eq. (4 when the onstraint Gθ(x = 0 holds, but serves as an additional penalty. Optimizing Eq. (8 an be ahieved by perforg the following steps iteratively, optimizing Eq. (8 with respet to eah individual variable: 1. Maximize (8 over q θ : q θ (x Proj qθ (x S θ (x (q θ (x + τ (div q θ (x + p θ (x p S (x u θ (x/ whih is a Chambolle projetion iteration with desent parameter τ > Note that both the divergene and gradient operators are evaluated over both the linear spatial domain and the yli θ domain. 2. Maximize (4 over the sink flows, p θ, by: p θ (x {D θ (x, p S (x div q θ (x + u θ (x/} 3. Maximize (4 over the soure flow p S analytially by: p S (x 1 ( π 1/ + (p θ (x + div q θ (x u θ (x/ dθ 2π 4. Minimize (4 over u θ analytially by: u θ (x u θ (x (div q θ (x p S (x + p θ (x The speifi augmented Lagrangian based algorithm used is: 5

6 x, θ, u θ (x = 1 /2π ; while not onverged do (x, θ, q θ (x Proj qθ (x S θ (x (q θ + τ (div q θ (x + p θ (x p S (x u θ (x/; (x, θ, p θ (x {D θ (x, p S (x div q θ (x + u θ (x/}; p S (x 1 2π (1/ + π (p θ(x + div q θ (x u θ (x/dθ; (x, θ, u θ (x u θ (x (div q θ (x p S (x + p θ (x; end 4.6 Pseudo-Flow Formulation The approah taken here for the use of proximal Bregman or pseudo-flow methods is derived from Baxter et al. 14 The orresponding non-smooth pseudo-flow model an be written as: (D θ(x + div q θ (x dx (9 q θ (x S θ (x θ [,π] whih an be derived from Eq (4 by: ( p S (x + u θ (x p S (x,q(x,p θ (x π s.t. p θ (x D θ (x, q θ (x S θ (x ( π = p S (x u θ (x p S (x,q(x,p θ (x s.t. p θ (x D θ (x, q θ (x S θ (x π = u θ (x (div q θ (x + p θ (x dθdx u θ (x q θ (x,p θ (x π s.t. u θ (xdθ = 1, p θ (x D θ (x, q θ (x S θ (x u θ (x (div q θ (x p S (x + p θ (x dθdx π u θ (xp S (xdθ + u θ (x (div q θ (x + p θ (x dθ dx (as the equivalene of equations (1 and (4 guarantees the onstraint π u θ(xdθ = 1 π = u θ (x (div q θ (x + D θ (x dθdx u θ (x q(x π s.t. u θ (xdθ = 1, u θ (x 0, q θ (x S θ (x (as the equivalene of equations (1 and (4 guarantees the onstraint u θ (x 0 and pθ (x D θ (x u θ (xp θ (x = D θ (x if u θ (x is non-negative π = u θ (x (div q θ (x + D θ (x dx q θ (x u θ (x π s.t. = q θ (x S θ (x u θ (xdθ = 1, u θ (x 0, q θ (x S θ (x (D θ (x + div q θ (x dx θ (sine uθ (x π u θ(x (D θ (x + div q θ (x = θ (D θ (x + div q θ (x when π u θ(xdθ = 1 and u θ (x 0 Now that a pseudo-flow representation is developed, we an take advantage of Bregman proximal projetions 15 to optimize this formula. Consider the distane between labeling funtions u θ (x and v θ (x as: π d g (u, v = (u θ (x ln(u θ (x/v θ (x u θ (x + v θ (x dθdx (10 6

7 whih an be verified to be a Bregman distane (when u θ (x [0, 1] using the entropy funtion: π g(u = (u θ (x ln u θ (x u θ (x dθdx. If we onsider a feasible labeling, v θ (x, we an find another proximal labeling, u θ (x, whih has a lower energy by addressing the optimization: ( u θ (x = arg u θ (x 0, u θ (x (D θ (x + div q θ (x dx + d g (u, v (11 π u θ(xdθ=1 q θ (x S θ (x where is a positive onstant. Using a Lagrangian multiplier on the onstraint π u θ(xdθ = 1, we an solve for u(x analytially as: ( v θ (x exp u θ (x = D θ(x+div q θ (x π v θ (x exp ( D θ (x+div q θ (x dθ (12 noting that this answer fulfills the onstraint u θ (x 0 provided the same holds for v θ (x. By letting the distane weighting parameter approah 0, u θ (x 0 if D θ (x + div q θ (x θ (D θ (x + div q θ (x. Using that fat, ( u θ (x (D θ (x + div q θ (x dx + d g (u, v u θ (x 0, π u θ(xdθ=1 q θ (x S θ (x q θ (x S θ (x (D θ(x + div q θ (x as 0 θ [,π] illustrating that the Bregman proximal method is a smoothed version of the non-smooth equation Eq ( Solution to Pseudo-Flow Formulation By taking the gradient of equation (11 with respet to div q θ (x (with u θ (x substituted by equation (12 one an derive the appropriate Chambolle iteration sheme 13 for imizing (11 with respet to the spatial flow variables q θ (x: ( ( ( q θ (x Proj qθ (x S θ (x q θ τ v θ (x exp D θ(x + div q θ (x (13 where τ is a positive gradient desent parameter. Thus, the proximal Bregman, or pseudo-flow, algorithm relies on the alternation between the analyti optimization of the label funtions, u θ (x through Eq. (12 and the speified Chambolle iteration. The proximal Bregman based algorithm proposed is therefore: x, θ, u θ (x = 1 /2π ; while not onverged do end (x, θ, q θ (x Proj qθ (x S θ (x ( (x, θ, u θ (x u θ (x exp (x, θ, u θ (x u θ(x π u θ(xdθ ; ( ( ( q θ τ u θ (x exp ; D θ(x+div q θ (x D θ(x+div q θ (x ; 7

8 5. DISCUSSION AND CONCLUSIONS In this paper, we present two algorithms for addressing ontinuous -flow image reonstrution in whih the intensity being reonstruted is fundamentally organized along a yli, as opposed to linear, topology. Suh a reonstrution method is more effiient than that developed from previous ontinuous -flow methods apable of inorporating said topology, speifially DAGMF, 7 by displaying linear rather than quadrati growth in omplexity in terms of the intensity resolution. This solver has been implemented using GPGPU aeleration due to its inherent parallelizability. This solver is available open-soure for both two- and three-dimensional images at Future work in ontinuous -flow based image reonstrution inludes the robust automati detetion of gradient diretion whih may allow for anisotropi smoothness terms to be employed. These terms ould be used to estimate level sets in the image, maintaining them while allowing for faster or less-onstrained gradients in the diretion orthogonal to said level set. ACKNOWLEDGMENTS The authors would like to aknowledge Zahra Hosseini and Dr. Martin Rajhl for their invaluable disussion and editing. REFERENCES 1. H. Ishikawa, Exat optimization for markov random fields with onvex priors, Pattern Analysis and Mahine Intelligene, IEEE Transations on 25(10, pp , V. Kolmogorov and R. Zabin, What energy funtions an be imized via graph uts?, Pattern Analysis and Mahine Intelligene, IEEE Transations on 26(2, pp , Y. Boykov, O. Veksler, and R. Zabih, Fast approximate energy imization via graph uts, Pattern Analysis and Mahine Intelligene, IEEE Transations on 23(11, pp , V. Kolmogorov and R. Zabih, Multi-amera sene reonstrution via graph uts, in Computer VisionECCV 2002, pp , Springer, E. Bae, J. Yuan, X.-C. Tai, and Y. Boykov, A fast ontinuous -flow approah to non-onvex multilabeling problems, in Effiient Algorithms for Global Optimization Methods in Computer Vision, pp , Springer, J. Yuan, E. Bae, and X.-C. Tai, A study on ontinuous -flow and -ut approahes, in Computer Vision and Pattern Reognition (CVPR, 2010 IEEE Conferene on, pp , IEEE, J. S. Baxter, M. Rajhl, J. Yuan, and T. M. Peters, A ontinuous -flow approah to multi-labeling problems under arbitrary region regularization, arxiv preprint arxiv: , E. M. Haake, Y. Xu, Y.-C. N. Cheng, and J. R. Reihenbah, Suseptibility weighted imaging (swi, Magneti Resonane in Mediine 52(3, pp , J. Yuan, E. Bae, X.-C. Tai, and Y. Boykov, A ontinuous -flow approah to potts model, in Computer Vision ECCV 2010, pp , Springer, J. S. Baxter, M. Rajhl, J. Yuan, and T. M. Peters, A ontinuous -flow approah to general hierarhial multi-labeling problems, arxiv preprint arxiv: , I. Ekeland and R. Temam, Convex analysis and 9 variational problems, E. Giusti, Minimal surfaes and funtions of bounded variation, no. 80, Springer Siene & Business Media, A. Chambolle, An algorithm for total variation imization and appliations, Journal of Mathematial imaging and vision 20(1-2, pp , J. S. Baxter, M. Rajhl, J. Yuan, and T. M. Peters, A proximal bregman projetion approah to ontinuous -flow problems using entropi distanes, arxiv preprint arxiv: , L. M. Bregman, The relaxation method of finding the ommon point of onvex sets and its appliation to the solution of problems in onvex programg, USSR omputational mathematis and mathematial physis 7(3, pp ,