Statistica Siica 009: Supplemet 1 L p -WAVELET REGRESSION WITH CORRELATED ERRORS AND INVERSE PROBLEMS Rafa l Kulik ad Marc Raimodo Uiversity of Ottawa ad Uiversity of Sydey Supplemetary material This ote cotais the proofs of Theorems 3.1 ad 3.. The proofs are based o the maxiset theorem from Kerkyacharia ad Picard 000. The steps are similar to those of Johstoe, Kerkyacharia, Picard ad Raimodo 004. The techical ovelties appear i momet bouds ad large deviatio results for wavelet coefficiets ˆβ κ D := 1 ψ κ u i Y i, ˆβ C κ := which we establish uder LRD assumptio. ψ κ tdy t. S1 Maxiset Theorem The followig theorem is borrowed from Kerkyacharia ad Picard 000. We refer to sectio S3 for the defiitio of Temlyakov property. First, we itroduce some otatio: µ will deote the measure such that for j IN, k IN, l q, µ = µ{j, k} = σ j ψ j,k p p = σ p p j j 1 ψ p p, { f, sup λ q µ{j, k : β j,k > σ j λ} < λ>0 }. S1. Theorem S1.1 Let p > 1, 0 < q < p, { ψ j,k, j 1, k = 0, 1,..., j } be a periodised wavelet basis of L I ad σ j be a positive sequece such that the heteroscedastic basis σ j ψ j,k satisfies Temlyakov property. Suppose that Λ is a set of pairs j, k ad c is a determiistic sequece tedig to zero with sup µ{λ } c p <. S1.3
RAFA L KULIK AND MARC RAIMONDO If for ay ad ay pair κ = j, k Λ, we have E ˆβ κ β κ p C σ j c p S1.4 P ˆβ κ β κ η σ j c / C c p c 4 S1.5 for some positive costats η ad C the, the wavelet based estimator ˆf = κ Λ ˆβκ ψ κ II{ ˆβ κ η σ j c } S1.6 is such that, for all positive itegers, if ad oly if : E ˆ f f p p C c p q, f l q, µ, ad, S1.7 sup c q p f β κ ψ κ p p <. S1.8 κ Λ This theorem idetifies the Maxiset of a geeral wavelet estimator of the form S1.6. This is doe by usig coditios S1.7 ad S1.8 for a appropriate choice of q. I the proof of the theorems we will choose q accordig to the dese or sparse regime by settig: q = q d := αp s + α, whe s α p π 1 S1.9 q = q s := αp 1 s 1 π + α, whe s < α p π 1. S1.10 S Momet bouds ad large deviatio estimates S.1 FBM model Here ˆβ κ = ˆβ C κ. I what follows C deotes a geeric costat which does ot depeds o but may chage from lie to lie. Recall that ˆβ C κ = β κ + ε α σ j z κ, where, as i Wag 1996, σj = Var ψ κ tdb H t ad z κ are weakly correlated Gaussia radom variables with variace 1 ad σ j = τ j1 α/. It follows that E ˆβ κ = β κ ad Var ˆβ κ = Var ε α ψ κ tdb H t = α j1 α τ Cσj c.
Wavelet regressio with correlated errors 3 Sice the rv s ˆβ κ β κ are Gaussia higher momets boud S1.4 follows from the previous iequality. Similarly, Pr ˆβ κ β κ > ησ j c / exp log η C c p c 4 8 provided η > 8α p. Which proves S1.5. S. Discrete model Here ˆβ κ = ˆβ D κ. Write ˆβ κ β κ = ˆβ κ E ˆβ κ + E ˆβ κ β κ = 1 1 X i ψ κ u i + fu i ψ κ u i β κ. The mai tool to derive rates of covergece is the followig lemma. To establish momets bouds we do ot assume that X i s are Gaussia. These estimates may be of idepedet iterest. Lemma S.1 For each fixed j ad k, ad p > 1, E ˆβ κ β κ j1 α α τd, S.11 E ˆβ p κ β κ = O αp/ jp1 α/. S.1 If moreover X i s are Gaussia, the for all λ > 1, Pr ˆβ κ β κ > λ α/ j1 α/ λ exp λ α j1 α τd. S.13 To prove this lemma we will replace β κ with ˆβ κ ad use E ˆβ κ β κ = O 1. Note that this just the distace betwee the itegral fxψ κ x dx ad the Riema-Stjeltjes sum. Proof: Note that ψκu i = j ψ j i 1 k = j ψ j xdx+o = +o. S.14 0 Bearig i mid that VarX i = EX1 = 1 we have: E ˆβ κ E ˆβ 1 κ = Var X i ψ κ u i = 1 ψκu i + ψ κ u i ψ κ z l CovX i, X l. i l
4 RAFA L KULIK AND MARC RAIMONDO By S.14 above, the first part is of order 1 + o 1. For the secod part we have ψ κ u i ψ κ z l CovX i, X l i l = j i l α ψ j i k ψ j l k i l = L j α i l α ψ j i k ψ j l k, i l which behaves asymptotically as j1 α α τ D. Further, the first part domiates the secod oe if ad oly if j >, which is ot possible. Thus S.11 follows. To prove S.1, let b r = a i r ψ κ u i, r = 1,...,, i=r b r = a i r ψ κ u i, r =,..., 0. Also, ote that by S.11, v := Var ɛ r b r = b r = Var X i ψ κ u i ad thus v / α j1 α τ D 1 as. Note ow that each Gaussia sequece ca be represeted as X i = a m ɛ i m, i 1, m=0 S.15 where a m is a regularly varyig sequece with idex α + 1/ ad {ɛ i, i 1} is a cetered sequece of i.i.d. radom variables. Via Rosethal iequality, for p E ˆβ κ E ˆβ p 1 p κ = E X i ψ κ u i = p p E a m ɛ i m ψ κ u i = p p E ɛ r b r m=0
Wavelet regressio with correlated errors 5 p b r p/ + p b r p p p/ br + p sup b r p b r r = p O α j1 α p/ + p p/ 1 O α j1 α = O αp/ jp/1 α + 1 α p/ j1 α. The secod term is egligible for all j such that j. To prove S.13 ote that X i ψ κ u i N 0, v. Thus, Pr ˆβ κ E ˆβ κ > λ C v λ exp λ v. ad the result follows by S.15. Cosequetly, E ˆβ p κ β κ = O αp/ jp/1 α = Oσ p j cp ad takig λ = λ j = ησ j c, Pr ˆβ κ β κ > ησ j c / exp log η 8 = Oc p provided η > 8pα. The similar argumet applies to 1 < p <. I this case we require η > 16α. S3 Temlyakov property As see i Johstoe, Kerkyacharia, Picard ad Raimodo 004, appedix A, the basis σ j ψ j,k. satisfies Temlyakov property as soo as j σj C sup j σj, Λ ad Λ Λ jp/ σ p j C sup jp/ σ p j, 1 p <, Λ which is clearly satisfied whe σ j = τ j1 α.
6 RAFA L KULIK AND MARC RAIMONDO S4 Fie resolutio tuig Here we check that coditio S1.3 is satisfied. Usig S1., µλ = j 1 j j 1 k=0 µj, k = j µj, k = τ p j j p p1 α 1 = O j 1, j j 1 j j 1 which with the choice of j 1 ad p > 1 yields µλ c p = α log log 1 αp which shows that coditio S1.3 is satisfied. = O c p 1 log log α 1 = o1, S5 Besov embeddig ad Maxiset coditio S5.1 Part I For both the dese S1.9 ad sparse S1.10 regime, we look for a Besov scale δ such that B δ π,r l q,. As usual we ote that it is easier to work with l q µ = f L p : f = β jk q σ q σ j ψ j,k p p <, j j,k A j where A j is a set of cardiality proportioal to j. Sice σ j ψ j,k p p = σp j j p 1 = αp j 1, we see that f l q µ if j 0 j αp +1 αq j 1 k=0 β j,k q = [ jq j 0 The latter coditio holds whe f B δ q,q for δ = α ] αp q + 1 1 j 1 q q k=0 β j,k q < +. p q 1. S5.16 Now depedig o whether we are i the dese S1.9 or sparse phase S1.10 we look for s ad π such that Bπ,r s Bq,q. δ S5.17 The dese phase. By defiitio S1.9 of q = q d we have π q d. Hece S5.17 follows from S5.1 as log as s δ = α p q 1 which is always true uder the
Wavelet regressio with correlated errors 7 dese regime where q = q d. Note that δ = α p q d 1 = s, thus automatically δ > 0. The sparse phase. Take q = q s ad δ = α p 1 = α sp p +1 π qs αp. We cosider two cases. If π > q s we use the embeddig S5.1. We have to check that s > α sp p +1 π αp which is equivalet to s < α p π 1, which is true i the sparse case. Further, we must guaratee that δ > 0 which leads to the two coditios i p > /α ad s > 1 π 1 p or ii p < /α ad s < 1 π 1 p. However, the last oe is ot relevat sice we have s > 1 π. Thus we established S5.17 for q s < π < q d. If π < q s we itroduce a ew Besov scale s ad idex q = q s such that s 1 π = s 1 q, s = α I this case, S5. ad S5.16 esures that B s π,r B s q,q l q µ, p q 1. S5.18 as had to be proved. sparse regime. Solvig S5.18 yields defiitio S1.10 of q uder the S5. Part II First we look for a Besov scale δ such that for ay f B δ p,r the maxiset coditio S1.8 is satisfied. We have c q p f β κ Ψ κ p p = c q p j1δp f B δ p,r = O κ Λ Thus coditio S1.7 holds for ay f B δ p,r if c q p+δp log α log δp. δ = 1 1 q p. S5.19 Now we look for s ad π such that B s π,r B δ p,r. S5.0 To aswer this questio, we will use two differet types of Besov embeddig, depedig o whether π p or π < p. We recall that B s π,r B s p,r, provided that π p, ad s s. S5.1 B s π,r B s p,r, provided that π < p, ad s 1 π = s 1 p. S5.
8 RAFA L KULIK AND MARC RAIMONDO The case π p. We ote that i this case we are always i the dese phase sice s must be o-egative. Here we use S5.1 with s = δ at S5.19. Hece we see that S5.0 holds as log as s 1 1 q p. Usig defiitio S1.9 of q = q d this will happe whe s 1 α. The dese case whe π < p. Here we itroduce a ew Besov scale s such that s 1 π = s 1 p ad use embeddig S5.. For S5.0 to hold i the dese case we eed s δ for q = q d at S1.9, we obtai the followig coditio: s s + α + 1 π 1 p. The sparse case whe π < p. Here we itroduce a ew Besov scale s such that s 1 π = s 1 p ad use embeddig S5.. For S5.0 to hold i the sparse case we eed s δ for q = q s at S1.10, we obtai the followig coditio: which is always true sice s > 1 π. s > 1 π α S6 Recall Fial step γ = αsp s + α, if s α p 1, S6.3 π γ = αps 1 π + 1 p s 1 π + α, if 1 π < s < α p 1. S6.4 π The proofs are a direct applicatio of Theorem S1.1 with our choice of σ j, c ad η. Combiig results of sectios S3,..., S5. we see that all the assumptios Theorem S1.1 are satisfied. Usig the embeddig results of Sectio S5.1 we derive rate expoet S6.3 for ay f Bπ,r s from defiitio S1.9 of q whe s α p π 1. Fially we derive rate expoet S6.4 for ay f Bs π,r usig defiitio S1.10 of q whe 1 π 1 p < s < α p π 1.