On multivariate Multi-Resolution Analysis, using generalized (non homogeneous) polyharmonic splines. or: A way for deriving RBF and associated MRA

Transcription

1 MAIA conference Erice (Italy), September 6, 3 On multivariate Multi-Resolution Analysis, using generalized (non homogeneous) polyharmonic splines or: A way for deriving RBF and associated MRA Christophe Rabut, Mira Bozzini and Milvia Rossini University of Toulouse (INSA ; IMT, IREM, MAIAA), France. Some known tools... using Fourier Transform. Extension of polyharmonic splines, and associated MRA

2 Part : Some known tools... using Fourier Transform

3 A word on (odd degree) polynomial splines Definition: σ m = Argmin i [:n], f(xi )=y i IR (f (m) (x)) dx = σ m (x) = λ i x x i m + p m (x) i=:n with q IP m, λ iq(x i ) = and p m IP m i=:n Radial basis functions : writing u m (x) = (m!) x m σ m (x) = µ i u m (x x i ) + q m (x) i=:n (same constraints on µ and q m ) Derivative and Fourier Transform: E(u m ) := ( ) m u (m) m = Dirac û m (ω) = ω m 3

4 ... and on associated functions Basic spline, or B-spline : ϕ m = ( ) m δ m u m ϕm (ω) = sin ω m ω m Lagrangian spline, or L-spline : L m () = ; j ZZ\{}, L m (j) = L m = j Z L m (ω) = L m (j/) L m ( j) ; ϕ m = l Z ϕ m (ω) ϕ m (ω πl) = j Z ϕm (ω) j Z ϕ m (j) exp( i j ω) = ϕ m (j) L m ( j) l Z ω m (ω πl) m Semi-orthogonal wavelet: ψ m = ( ) m D m L m (to be normalized) ψ m (ω) = ω m L m (ω) = l Z ω m (ω πl) 4m Orthogonal wavelet: ψ (ω) = ( l Z ψ(ω) ψ( ω πl )) = ω m (ω πl) 4m l Z 4

5 Linear case B-spline L-spline psi-spline function B ; norm(b) =.48 function L ; norm(l) =.4 function psi ; norm(psi) = κ = ( ) ; γ = ( ) ; κ = ( ) ; γ = ( ) ; κ = ( ) ; γ = ( ) ; psi-ortho psi-hat psi-ortho-hat 3.5 function psi ortho ; norm(psi ortho ) = function psi hat function psi ortho hat κ = ( ) ; γ = ( ) ; 3 3 κ = ( ) ; γ = ( ) ; 3 3 κ = ( ) ; γ = ( ) ; 5

6 6

7 Cubic case B-spline L-spline psi-spline function B ; norm(b) =.346 function L ; norm(l) =.467 function psi ; norm(psi) = κ = ( ) ; γ = ( ) ; κ = ( ) ; γ = ( ) ; κ = ( ) ; γ = ( ) ; psi-ortho psi-hat psi-ortho-hat 3 function psi ortho ; norm(psi ortho ) = 6 function psi hat function psi ortho hat κ = ( ) ; γ = ( ) ; 3 3 κ = ( ) ; γ = ( ) ; 3 3 κ = ( ) ; γ = ( ) ; 7

8 8

9 Use of the associated functions (translation invariant spaces) ŷ is defined by ŷ(ω) = j Z y j exp( i j ω). B-spline approximation of vector y : (or of points P j for B-spline curve) σ m = j Z y j ϕ m ( j) σ = ŷ ϕ m. Interpolating spline of vector y : (or P j instead of y j for interpolating spline curve) σ m = y j L m ( j) j Z σ = ŷ L. Wavelet decomposition of some f L (IR): f = j Z l Z (f, ψ ( l j)) ψ ( l j) (f, ψ ( l j)) = l exp(i l j) ( f, ψ ( l ) 9

10 Spline under tension Definition and Fourier transform σ m,k = Argmin i Z, f(xi )=y i IR (f (m) (x)) dx+ρ IR (f (k) (x)) dx (say k < m) E ρ (u) := ( ) m D m u + ( ) k ρ D k u = Dirac ; û ρ (ω) = ω m + ρ ω k ϕ ρ (ω) = sinm ω + ρ sin k ω ω m + ρ ω k L ρ (ω) = ψ ρ (ω) = ω m L ρ(ω) + ρ ω k L ρ(ω) = l Z (ω m + ρ ω k ) ( (ω πl) m + ρ (ω πl) k) ( ω m + ρ ω k) ( (ω πl) m + ρ (ω πl) k) ψρ (ω) = l Z l Z ( ω m + ρ ω k) ((ω πl) m + ρ (ω πl) k )

11 function B ; norm(b) = κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; Tension splines (linear-cubic) B-spline L-spline psi-spline function L ; norm(l) = κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; function psi ; norm(psi) = κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; psi-ortho psi-hat psi-ortho-hat function psi ortho ; norm(psi ortho ) = 8 function psi hat function psi ortho hat κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; 3 3 κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; 3 3 κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ;

12

13 Tension splines (cubic-quintic) B-spline L-spline psi-spline function B ; norm(b) = function L ; norm(l) = function psi ; norm(psi) = κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; psi-ortho psi-hat psi-ortho-hat 3 function psi ortho ; norm(psi ortho ) = 7 x 4 function psi hat.4 function psi ortho hat κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; 3 3 κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; 3 3 κ = ( ) ; ( ) ; ( 3 ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; 3

14 4

15 Definition and Fourier transform Fractional splines Let s [..) and m IN such that α := m + s > d/ σ α = Argmin i [:n], f(xi )=y i IR (f (α) (x)) dx = Argmin i [:n], f(xi )=y i IR (ω ) s ( F(f (m) (ω) ) dω E α (u α ) := ( ) α D α u α = Dirac ; û α (ω) = (ω ) α u α (x) = c α x α u α (x) = c α x α ln x if α is not an even integer number. if α is an even integer number. (c α is some known real valued constant) They too are in some place between order (linear) and order (cubic) splines, but are different from splines under tension. B-spline : ϕα (ω) = sin ω ω α 5

16 Fractional splines (linear-cubic) B-spline L-spline psi-spline function B ; norm(b) = function L ; norm(l) = function psi ; norm(psi) = κ = ( ) ; ( ) ; ( ) ; ( ) ; γ = ( ) ; (.3 ) ; (.6 ) ; ( ) ; κ = ( ) ; ( ) ; ( ) ; ( ) ; γ = ( ) ; (.3 ) ; (.6 ) ; ( ) ; κ = ( ) ; ( ) ; ( ) ; ( ) ; γ = ( ) ; (.3 ) ; (.6 ) ; ( ) ; psi-ortho psi-hat psi-ortho-hat 3.5 function psi ortho ; norm(psi ortho ) = 6 function psi hat function psi ortho hat κ = ( ) ; ( ) ; ( ) ; ( ) ; γ = ( ) ; (.3 ) ; (.6 ) ; ( ) ; 3 3 κ = ( ) ; ( ) ; ( ) ; ( ) ; γ = ( ) ; (.3 ) ; (.6 ) ; ( ) ; 3 3 κ = ( ) ; ( ) ; ( ) ; ( ) ; γ = ( ) ; (.3 ) ; (.6 ) ; ( ) ; 6

17 7

18 comparison tension B-splines versus fractional B-splines.8 function B ; norm(b) = κ = (.84 ) ; ( ) ; ( ) ; ( ) ; γ = ( ) ; (.3 ) ; ( ) ; (.6 ) ; 8

19 comparison tension L-splines versus fractional L-splines function L ; norm(l) = κ = (.84 ) ; ( ) ; ( ) ; ( ) ; γ = ( ) ; (.3 ) ; ( ) ; (.6 ) ; 9

20 (d-dimensional, real order) polyharmonic splines Definition and expression Let s [..) and m IN such that α := m + s > / σ α = Argmin i [:n], f(xi )=y i IR d D α f(x) dx = Argmin i [:n], f(xi )=y i IR d ω s F(D m f)(ω) dx = σ α (x) = λ i u α (x x i ) + p m (x) i=:n with q IP m, where E α (u α ) := ( ) m α u α = Dirac i=:n λ iq(x i ) = and p m IP m u α (x) = c α x α d if α d is not an even integer number. = c α x α d ln x if α d is an even integer number. Radial basis functions (no other known writing)

21 (d-dimensional, real order) polyharmonic splines Derivative and Fourier Transform α u α = Dirac û α (ω) = ω α Polyharmonic B-spline: ϕ α = α u α ϕα (ω) = sin ω ω α Lagrangian spline, or L-spline : L α (ω) = ω α ( ω πl ) α l Z d Wavelets: ψ α = α L α ψ α (ω) = ( ω ) α L α (ω) = ( ω ) α ( ω πl ) α ψ α (ω) = ( ω ) α (( ω πl ) α l Z d l Z d

22 Biharmonic ( thin plate ) B-spline Elementary Thin Plate B Spline 3 Elementary Thin Plate B Spline

23 Biharmonic ( thin plate ) L and ψ-spline Lagrangean function polyharmonic wavelet

24 Biharmonic ( thin plate ) ψ and ψ -spline 4

25 (d-dimensional) scaled Whittle-Matérn-Sobolev kernel Idea Regularize the operator m of polyharmonic splines via a regularization of the differential operator: instead of ( ) m, use the operator E = ( + κ I) m, κ being some real number( scaled is for κ ). Its Fourier transform is clearly E(ω) = ( ω + κ ) m. Associated kernel (u function) u m,κ (x) = m (m )! κd/ m x m d/ K m d/ (κ x ) where K m d/ is the order m d/ Bessel function of the third kind (note the mythic m d/ constraint, actually here for ensuring u m being a continuous function and the differential operator having a meaning). Same type of functions generated. 5

26 A generalisation of Whittle-Matérn kernel (/) Mira Bozzini, Milvia Rossini, Robert Schaback, Idea: Mix different values of κ for each scaled iterated Laplacian operator: use E = m j= ( + κ ji) instead of ( + κ I) m, We get: E κ (ω) = m j= ( ω + κ j) The fundamental solution meets ûκ(ω) = j= ( ω + κ j) and u κ is a continuous radial kernel, and is a convolution product of the Kernels u,κj whose Fourier transform is ( ω + κ j ) (no computable by this way!). However we have the following explicit value u κ (x) = m ( ) m [κ /,..., κ x m/] z z m d/ To prove it we use the m-th divided difference relation : ( ) m m j= (s + t j) = [t,..., t m ] z (s + z) 6 K d/ ( x z).

27 Form of some u-functions 7 κ = ( ), (3 7), ( 3 4)

28 A generalisation of Whittle-Matérn kernel (/) Associated B-spline Lagrangian function ϕ κ (ω) = L κ (ω) = m j= l Z d sin ω + κ j ω + κ j m j= ( ω + κ j) m j= ( ω πl + κ j) Wavelets: ψ κ = ψ κ (ω) = ψ κ (ω) = m m j= ( + κ j I) j= ( ω + κ j) m m ( l Z d j= ( ω + κ j) L [κ κ] L [κ κ] (ω) = j= ( ω πl + κ j ) l Z d m j= ( ω + κ j) m j= ( ω πl + κ j 8

29 Proof of the expression of the pre-wavelet Theorem Let E = m ( + j= κ ji). Let L be the Lagrangian E -function, and let ψ = E(L ) (so ψ(ω) = Ê(ω) L (ω)). Let ψ e = ψ( +e) = (E(L ))( +e) where e {, } d \ d. Then ψ e is orthogonal to any cardinal E-function. So ψ e is a pre-wavelet (semi-orthogonal wavelet). Proof Let σ = j Z d λ j u( j) be a cardinal E-function (u is such that E(u) = Dirac). Then : (σ, ψ e ) = (σ, E(L ( +e) ) = (E(σ), L ( +e) ) = ( j Z d λ j E(u( j)), L ( +e) ) = ( j Z d λ j Dirac j, L ( +e) ) = j Z d λ j L (j + e) =. Actually an even more general property (quite general E, and also for scattered data Lagrangean function L and pre-wavelet ψ = E(L )). 9

30 Part : A proposal for a global extension of polyharmonic splines: functions generated by the differential operator E = m i= ( + κ ji) α j 3

31 The proposed generalization of polyharmonic splines Idea: We want both tension and continuous choice of the order. So we choose polyharmonic splines - regularized by the (extended) Whittle-Matérn coefficients (for tension), - generalized by real exponents, (for continuous choice of the order and of the shape). So : E κ,α = m i= ( + κ ji) α j Since the differential operator is a mix of the ( + κ ji) α j, the obtained functions will be a kind of mix of the fractional regularized polyharmonic splines. We get (with the condition m α j > d/) j= E κ,α (ω) = m i= ( ω + κ j) α j The fundamental u solution is a radial function which meets û κ,α (ω) = m j= ( ω + κ j) α j 3

32 Associated B-function Lagrangian function Wavelets: ψ κ,α = ψ κ,α (ω) = ψκ,α(ω) = m m Associated functions ϕ κ,α (ω) = L κ,α (ω) = E κ,α (sin ω) E κ,α (ω) l Z d j= ( + κ j I) α j j= ( ω + κ j) α j l Z (Ê(ω)) d Ê(ω πl) = m = m j= ( sin ω + κ j) α j m j= ( ω + κ j) α j m i= ( ω + κ j) α j j= ( ω πl + κ j) α j L [κ κ;α α] L [κ κ α α] (ω) = l Z d m l Z d m j= ( ω + κ j) α j m j= ( ω πl + κ j) α j j= ( ω + κ j) α j m j= ( ω πl + κ j ) α j 3

33 Examples of B- and L- functions B-function L-function κ = ( ) κ = (3 7) 33

34 34

35 Examples of ψ- and ψ functions ψ-function ψ function κ = ( ) κ = (3 7) 35

36 Minimization property, when all α j are integer Let γ be such that m γ i x i = m (x + i= j= κ j). γ i is positive and is the sum of all product of m i different κ j. In particular : γ = m j= κ j, γ = m i= j i κ j, γ m = m j= κ j, γ m = ; γ j >. Let H κ : (f, g) κ = IR d m j= γ j (D j f D j g) and f κ = IR d m j= γ j D j f Theorem Let A be a set of elements in IR d, and (y a ) a A some associated real numbers. If A is an infinite set, suppose furthermore that there exists f H κ such that a A, f(a) = y a. Then the set of all functions f in H κ meeting a A, f(a) = y a has a unique element, denoted σ A,κ with minimal (semi-)norm f κ. Besides, there exist real numbers (λ a ) a A such that σ A,y meets the relation E(σ A,y ) = λ aδ a, so σ A,y = λ au κ (x a) + p l, a A a A where l is the number of κ j being and p l is some polynomial in IP l. 36

37 Role and influence of κ : tension coefficients Theorem (direct consequence of previous theorem) Let σ A be in the form σ A = a A λ au κ ( a) + p l (), and let y = (y a ) a A such that y a = σ A (a). Then σ A minimizes f κ on all the functions f H κ such that a A, f(a) = y a. Discussion m Since f κ = IR d γ j D j f, we see that any function in the form () j= minimizes a quantity which is a linear combination of various polyharmonic seminorms, the weights of the linear combination being the γ j, which are connected to the κ j by the relation m γ i x i = m (x + i= j= κ j). Since it is known that a polyharmonic spline is all the more oscillating than its order is high, we can interpret γ j as tension coefficients, giving all the more tension than j is small. Intuition of the impact of a vector κ is quite easy when all but one κ j are zero, but is not so easy otherwise. 37

38 Role and influence of real valued α j s Very easy when only one α j is non integer and when the associated κ j is zero : we have real order polyharmonic splines under tension, with tension parameters as explained above for integer α j s. As said before, we know that real order polyharmonic splines are somewhere between the polyharmonic spline of the integer part of the order and of the integer part plus. Otherwise, we need work directly on the Fourier transform of E, and the minimized quantity is less intuitive. There is still work to do to improve intuition about that. 38

39 Two remarks On B-functions Note that for (one dimensional) polynomial splines, the B-spline is non-negative. This is NOT true for many other B-functions, as some tension B-splines, polyharmonic B-splines and all their d-dimensional extensions. We claim that the important property is NOT non-negativity, but positive definite functions, as they all are since ϕ is non-negative (in all extensions we showed here). On the strategy Starting on the differential operator and/or its Fourier transform is an interesting way to work on, and gives other possibilities, depending on what we want to obtain. An operator involving iterated Laplacian operator will give radial basis functions. Of course appropriate properties for having a MRA have to be checked. For example if we want something between quintic and linear we can use κ = ( ρ ρ ) (which gives E = D 6 + (ρ + ρ )D 4 + ρ ρ D in one dimension) and so get Ê(ω) = ω6 + (ρ + ρ )ω 4 ρ ρ ω (or equivalent with and ω in many dimensions), but we also can try to minimize IR (f (3) (x)) +ρ (f (x)) dx, which gives E = D 6 + ρ D and Ê(ω) = ω6 ρ ω (or equivalent with and ω in many dimensions), which is not included in the presented extension (not done!). 39

40 Computational considerations Extensive use of the fast Fourier transform All the functions are given by their Fourier transform. The whole computation is done via FFT and IFFT. For the basis functions (ϕ, ψ, ψ, L: sampling the Fourier transform of the function, and then IFFT of the so-obtained (d-dimensional) vector. the longest computation is for computing the sample of the Fourier transform of the functions. For spline functions in the form y j B( j h), we compute ŷ(ω) (via the FFT), i Z d and then we compute IFFT(ŷ B). Note that this is no more complicated for any type of extension presented here. As usual, there are some side effects for boundly supported vectors or functions. 4

41 Extension to scattered data analysis Surprisingly this is possible, still by using this type of kernel. However everything is not yet done. The main difficulty is to choose the appropriate points close to the point where we want to discretize the derivatives. But we can define the basis functions (one for each centre!) via the Fourier transform, and go on pretty well. However this need still work theoretical as well as numerical... Besides there are some true normalization problems 4

42 A classical test : Lena We use the associated filters on one Lena s eye : 4

43 and obtain : κ = ( ) κ = (3 7) 43

44 Enjoy your reseach, Enjoy your life,...and take care! 44

45 Some curves for κ = [;.5; ; ] function B ; norm(b) = function L ; norm(l) = κ = ( ) ; (.5 ) ; ( ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; κ = ( ) ; (.5 ) ; ( ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; 4 function psi ; norm(psi) = 4 function psi ortho ; norm(psi ortho ) = κ = ( ) ; (.5 ) ; ( ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; κ = ( ) ; (.5 ) ; ( ) ; ( ) ; γ = ( ) ; ( ) ; ( ) ; ( ) ; 45