arxiv: v3 [math.st] 3 May 2016

Transcription

1 Bump detectio i heterogeeous Gaussia regressio arxiv:54.739v3 [math.st] 3 May 6 Farida Eikeeva farida.eikeeva@math.uiv-poitiers.fr Laboratoire de Mathématiques et Applicatios, Uiversité de Poitiers, Frace ad Istitute for Iformatio Trasmissio Problems, Russia Academy of Sciece, Moscow, Russia Axel Muk, Frak Werer muk@math.ui-goettige.de, frak.werer@mpibpc.mpg.de Felix Berstei Istitute for Mathematical Statistics i the Biosciece, Uiversity of Göttige ad ad Max Plack Istitute for Biophysical Chemistry, Göttige, Germay We aalyze the effect of a heterogeeous variace o bump detectio i a Gaussia regressio model. To this ed we allow for a simultaeous bump i the variace ad specify its impact o the difficulty to detect the ull sigal agaist a sigle bump with kow sigal stregth. This is doe by calculatig lower ad upper bouds, both based o the likelihood ratio. Lower ad upper bouds together lead to explicit characterizatios of the detectio boudary i several subregimes depedig o the asymptotic behavior of the bump heights i mea ad variace. I particular, we explicitly idetify those regimes, where the additioal iformatio about a simultaeous bump i variace eases the detectio problem for the sigal. This effect is made explicit i the costat ad / or the rate, appearig i the detectio boudary. We also discuss the case of a ukow bump height ad provide a adaptive test ad some upper bouds i that case. Keywords: miimax testig theory, heterogeeous Gaussia regressio, chage poit detectio. AMS classificatio umbers: Primary 6G8, 6G, Secodary 6G5, 6H5.. Itroductio Assume that we observe radom variables Y = Y,..., Y through the regressio model i i Y i = µ + λ Z i, i =,...,, i.i.d where Z i N, are observatioal errors, µ deotes the mea fuctio ad λ is the variace fuctio sampled at equidistat poits i/, say. The aim of this paper is to aalyze the effect of a simultaeous chage i the variace o the difficulty to detect µ agaist a mea of the form { if x I, µ x = I x = otherwise, Correspodig author

2 i.e. a bump with height > o a iterval I [, ] as locatio. The assumptio that > is posed oly for simplicity here, if is egative or µ is cosidered, similar results ca be obtaied aalogously. Further, i model we assume a simultaeous chage of the variace by λ x = σ + σ I x, x [, ] 3 with a baselie variace σ >. Note that the additioal chage i the variace σ > may oly occur whe the mea has a bump of size o I. We assume throughout this paper that σ ad σ are kow, ad that also the legth I of I is kow. Note that as the positio of I is ukow, both fuctios µ ad λ are partially ukow as well. Cocerig we will later distiguish betwee the cases that is kow to which we will provide sharp results ad ukow. The model arisig from 3 with the above described parameters will be called the heterogeeous bump regressio HBR model, which is illustrated i Figure. The aim of this paper is to provide some first isights for precise uderstadig of the effect of the heterogeeity o detectio of the sigal bump. σ + σ λ σ µ Y i I Figure : The HBR model: Data together with parameters I,, σ ad σ i 3 from the HBR. Here = 4, σ =, σ = 4 ad = 5... Applicatios The HBR model may serve as a prototypical simplificatio of more complex situatios, where may bumps occur ad have to be idetified. I fact, i may practical situatios it is to be expected that the variace chages whe a chage of the mea is preset; see e.g. [38] for a discussio of this i the cotext of CGH array aalysis. Aother example arises i ecoometrics ad fiacial aalysis, where it has bee argued that price ad volatility do jump together [5, 3]. The HBR model ca also be viewed as a heterogeeous extesio of the eedle i a haystack problem for I, i.e. to idetify s small cluster of o-zero compoets i a high-dimesioal multivariate vector [, 6, 7,, 3]. Fially we metio the detectio of opeig ad closig states i io chael recordigs see [4] ad the refereces therei, whe the so-called ope chael oise is preset which arises for large chaels, e.g. poris [43, 48]. A small segmet of a curret recordig of a pori i plaar lipid bilayers performed i the Steiem lab Istitute of Orgaic ad Biomolecular Chemistry

3 Uiversity of Göttige is depicted i Figure. We fid that the observatios fluctuate aroud two mai levels, which correspod to the ope ad closed state. It is also apparet that the variace chages together with the mea. Curret i pa Time i secods Y i Figure : Curret recordigs of a pori i picoampere pa sampled every.4 millisecods... Detailed settig ad scope of this paper Let us ow formalize the cosidered testig problem. Let I [, ], N a give sequece of itervals. W.l.o.g. assume i the followig that l := / I oly takes values i the itegers, i.e. l N for all N ad defie The for fixed N we wat to test A := { [j I, j I ] j l }. 4 H : µ, λ σ, σ > fixed agaist H : I A s.t. µ = I, λ = σ + σ I, σ >. 5 Throughout the most of this paper we assume that the parameters, σ, σ ad I are kow, oly the positio I of the bump is ukow. Note, that if the locatio I of the bump was also kow ad ot oly its width I, the the whole problem reduces to a hypothesis test with simple alterative, i.e. the Neyma-Pearso test will be optimal. This situatio may be viewed as a variatio of the celebrated Behres-Fisher problem see e.g. [37] to the situatio whe the variace chages with a mea chage. The requiremet that is kow will be relaxed later o. The mai cotributio of this paper is to precisely determie the improvemet comig from the additioal chage i variace by determiig the detectio boudary for 5. We stress that this issue has ot bee addressed before to the authors best kowledge, ad it has bee completely uclear if ad to what exted the additioal chage i variace adds iformatio to the model so far. We will provide some partial aswers to this. The testig problem 5 is i fact a simplificatio of the practically more relevat problem where the iterval I i the alterative is ot ecessarily a elemet of A, but ca be arbitrary i [, ] with the predefied legth I. Furthermore the parameters I, σ ad σ will also be ukow i most practical situatios, although ofte for the i Sectio. metioed chael recordigs the variaces ad the coductace level ca be pre-estimated with high accuracy, ad hece regarded as practically kow. I fact, lack of kowledge of all these quatities leads to a much more complex problem ad raises already a remarkable challege for desigig a efficiet test, which is far above the scope of this paper. Istead, we aim for aalyzig the amout of iformatio cotaied i the most simple HBR model compared to the homogeeous case, ad therefore we restrict ourselves to the simplified but still quite complicated situatio 5. Note that our lower 3

4 bouds for the detectio boudary are also lower bouds for the geeral situatio although we speculate that the ot sharp. Our results are achieved o the oe had by modifyig a techique from Dümbge & Spokoiy [] providig lower bouds i the homogeeous model. We will geeralize this to the case of a o-cetral chi-squared likelihood ratio which appears due to the chage i variace. O the other had we will aalyze the likelihood ratio test for 5 which the provides upper bouds. Doig so we use a geeralized deviatio iequality for the weighted sum of o-cetral chi-squared radom variables from [35] see Appedix A for a discussio of this ad related iequalities. The authors there also study a heterogeeous regressio problem, but i a settig which is ot related to bump detectio..3. Aalytical tools ad literature review Miimax testig theory. I this paper, the aalysis of the HBR model is addressed i the cotext of miimax testig theory, ad we refer e.g. to a series of papers by Igster [9] ad to Tsybakov [5] for the geeral methodology. Followig this paradigm we aim for determiig the detectio boudary which marks the borderlie betwee asymptotically detectable ad udetectable sigals of type. I the followig we cosider tests Φ : R {, }, where Φ Y = meas that we accept the hypothesis H ad Φ Y = meas that we reject the hypothesis. We deote the probability of the type I error by ᾱ Φ, σ := E H [Φ Y ] = P H Φ Y = ad say that the sequece of tests Φ has asymptotic level α, uder H, if lim sup ᾱ Φ, σ α. The type II error is deoted by β Φ,, I, σ, σ := sup P µ= I, λ I =σ +σ Φ Y = I A ad lim sup β Φ,, I, σ, σ is the asymptotic type II error of the sequece of tests Φ uder hypothesis H ad alterative H determied by the parameters, I, σ ad σ. Below we will skip the depedece o the parameters to ease the presetatio. Obviously, a fixed bump ca be detected always with the sum of type I ad II errors tedig to as. Thus we are iterested i asymptotically vaishig bumps, hece we wat to ivestigate which asymptotic behavior of, σ ad I implies the followig twofold coditio for a fixed α > cf. [34]: a lower detectio boud: let H cosist of faster vaishig bumps. The for ay test with lim sup ᾱ Φ α it holds lim if β Φ α. b upper detectio boud: let H cosist of slower vaishig bumps. The there is a sequece of tests with lim sup ᾱ Φ α ad lim sup β Φ α. Typically the terms faster ad slower are measured by chagig the costat appearig i the detectio boudary by a ±ε term. If upper ad lower bouds coicide asymptotically, we speak of the asymptotic detectio boudary, cf. 6 below. I fact we will see that for our specific situatio the obtaied asymptotic behavior, σ ad I does ot deped o the specific choice of α,. Cosequetly, we determie coditios for H ad H to be either cosistetly distiguishable or asymptotically idistiguishable i the spirit of Igster cf. [9]. 4

5 Literature review. Motivated by examples from biology, medicie, ecoomics ad other applied scieces, methods for detectig chages or bumps i a sequece of homogeeous observatios have bee studied extesively i statistics ad related fields. We refer to Csörgo & Horváth [9], Siegmud [45], Carlstei et al. [6] or Frick et al. [4] for a survey. Estimatig a piecewise costat fuctio i this settig cosists o the oe had i a model selectio problem amely estimatig the umber of chage poits ad o the other had i a classical estimatio problem estimatig the locatios ad amplitudes. For the former, we refer to Yao [5], who proposed a method to estimate the total umber of chage poits i the data by Schwarz criterio, ad to Bai & Perro [4], who ivestigate a Wald-type test. For a more geeral approach to treat the chage-poit problem from a model selectio perspective we refer to Birgé & Massart [9], ad for a combied approach with cofidece statemets ad estimatio, see Frick, Muk & Sielig [4]. Cocerig the latter, Siegmud & Vekatrama [46] used a geeralized likelihood ratio to estimate the locatio of a sigle chage poit i the data. The overall estimatio problem has e.g. bee tackled by Boyse et al. [] via l -pealized least-squares ad by Harchaoui & Lévy-Leduc [6] via a TV-pealized least-squares approach. Siegmud, Yakir & Zhag [47] proposed a specialized estimator for the situatio of may aliged sequeces whe a chage occurs i a give fractio of those. Recet developmets iclude efficiet algorithms for the overall estimatio problem based o accelerated dyamic programmig ad optimal partitioig by Killick, Fearhead & Eckley [33] ad [4], o pealized or averaged log likelihood ratio statistics by Rivera & Walther [4], or Bayesia techiques as proposed by Du, Kao & Kou []. We also metio the recet paper by Goldeshluger, Juditski & Nemirovski [5], which has bee discussed i this cotext [39]. The special questio of miimax testig i such a settig has also bee addressed by several authors. We metio the semial paper by Dümbge & Spokoiy [] ad also Dümbge & Walther [3] who studied multiscale testig i a more geeral setup, ad [4] for jump detectio i chage poit regressio. Furthermore, we refer to Che & Walther [8] for a optimality cosideratio of the tests itroduced i [4], ad to a rigorous miimax approach for detectig sparse jump locatios by Jeg, Cai & Li [3]. Nevertheless, all these papers oly address the case of a homogeeous variace, i.e. λ σ. There, it is well-kow that the detectio boudary is determied by the equatios I = σ log I, 6 ad the terms faster ad slower i the above defiitio are expressed by replacig σ by ± ε with a sequece ε N such that ε, ε log I as for details see [4]. I the heterogeeous model 3 ad the coected testig problem 5 maily two situatios have to be distiguished, see also [36]. The first situatio emerges whe it is ot kow whether the variace chages but it might. The as i 3 we explicitly admit σ = ad the variace is a uisace parameter rather tha a iformative part of the model for idetificatio of the bump i the mea. Hece, o improvemet of the boud 6 for the case of homogeeous variace is to be expected. The estimatio of the correspodig parameters has also bee cosidered i the literature. Huag & Chag [8] cosider a least-squares type estimator, ad Brau, Brau & Müller [3] combie quasilikelihood methods with Schwarz criterio for the model selectio step. We also metio Arlot & Celisse [3] for a approach based o cross-valiatio ad Boutahar [] for a CUSUM-type test i a slightly differet setup. Eve though, a rigorous miimax testig theory remais elusive ad will be postpoed to further ivestigatios. The secod situatio, which we treat i this paper, is differet: We will always assume σ > as i the examples i Sectio., which potetially provides extra iformatio o the locatio of the bump ad detectio of a chage might be improved. This is ot the case i the situatio discussed above, as the possibility of a chage i variace ca oly complicate the testig procedure, whereas σ > gives rise to a secod source of iformatio the variace for sigal detectio. The cetral questio is: Does the detectio boudary improve due to the additioal iformatio ad if so, how much? 5

6 Relatio to ihomogeeous mixture models. Obviously, the HBR model is related to a Gaussia mixture model Y i ε N, + εn, σ, i =,...,, 7 which has bee itroduced to aalyze heterogeeity i a liear model with a differet focus, see e.g. [, 4, 5]. For ε β < β < differet asymptotic regimes occur. If β > /, the o-ull effects i the model are sparse, which leads to a differet behavior as if they are dese β /. Although it is ot possible to relate 7 with the HBR model i geeral, it is isightful to highlight some distictios ad commoalities i certai scearios for the homogeeous case see the discussio part of [4]. A mai differece to our model is that the o-ull effects do ot have to be clustered o a iterval I. It is exactly this clusterig, which provides the additioal amout of iformatio due to the variace heterogeeity i the HBR model. A further differece is that the defiitio of a i.i.d. mixture as i 7 itrisically relates variace ad expectatio of the Y i..4. Orgaizatio of the paper The paper is orgaized as follows: I the followig Sectio we preset our fidigs of the detectio boudary. The methodology ad the correspodig geeral results are stated i Sectio 3. Afterwards we derive lower bouds i Sectio 4 ad upper bouds i Sectio 5. There we also discuss upper bouds for the case that is ukow ad provide a likelihood ratio test which adapts to this situatio. I Sectio 6 we preset some simulatios ad compare the fiite sample power of the o-adaptive test ad the adaptive test. Fially we will discuss some ope questios i Sectio 7. To ease the presetatio all proofs will be postpoed to the Appedix.. Overview of results Throughout the followig we eed otatios for asymptotic iequalities. For two sequeces a N ad b N, we say that a is asymptotically less or equal to b ad write a b if there exists N N such that a b for all N. Similarly we defie a b. If a /b c as for some c R \ {} we write a b. If c = we write a b. We use the termiology to say that a ad b have the same asymptotic rate but probably differet costat if a b, ad that they have the same rate ad costat if a b. Cosequetly, the relatio i 6 determiig the detectio boudary i the homogeeous case becomes I σ log I. The otatios ad coicide with stadard otatios. The defiitios of ad ca be see as extesios of the classical miimax otatios by Igster [9] or Dümbge & Spokoiy [], where the ull hypothesis is tested agaist the complemet of a ball aroud with varyig radius ρ = ρ. The results are typically preseted by statig that if ρ teds to faster tha the boudary fuctio ρ, or if their ratio coverges to some costat <, the the ull ad the alterative hypotheses are idistiguishable. I our settig, the asymptotic coditios are defied i terms of sums rather tha ratios. Thus we itroduced ad to have similar otatios. I the HBR model, our sigal has a chage i mea determied by ad also a chage i its variace, which will be described by the parameter κ := σ σ >. I fact, we will see that the detectio boudary is effectively determied by the ratio of κ ad, which leads to three differet regimes: 6

7 DMR Domiat mea regime. If κ vaishes faster tha, the mea will asymptotically domiate the testig problem. I this case, the additioal iformatio will asymptotically vaish too fast, so that we expect the same udetectable sigals as for the homogeeous case. The domiat mea regime cosists of all cases i which κ /,. ER Equilibrium regime. If κ has a similar asymptotic behavior as, we expect a gai from the additioal iformatio. The equilibrium regime cosists of all cases i which κ, ad c := lim κ / /σ = lim σ / σ satisfies < c <. DVR Domiat variace regime. If vaishes faster tha κ, the variace will asymptotically domiate the testig problem. I this case, we expect the same detectio boudary as for the case of testig for a jump i variace oly. The domiat variace regime cosists of all cases i which κ ad κ /,. With this otatio, we ca collect the most importat results of this paper i the followig Table. The correspodig detectio boudaries i differet regimes are also illustrated i Figure 3. rate Table : Mai results of the paper. costat lower boud upper boud kow ukow DMR I log I I log I ER I κ log I DVR I κ log I ε + ε + ε Thm. 4. Thm. 5. Thm. 5.4 c+ +c ε +c + ε σ +3c +c + ε Thm. 4. Thm. 5. Thm. 5.4 c c +c ε +c + ε c c+ +3c +c + ε cf. 4 aalog to 4 aalog to 4 ε + ε ε Thm. 4.3 Thm. 5.3 Thm. 5.4 Here ε is ay sequece such that ε, ε log I. The secod colum depicts the rates obtaied i the differet regimes, the colums three to five give the costats i differet situatios. ER is stated twice, because the results are show w.r.t. the differet rates from DMR ad DVR respectively. Exemplary, the lower boud etry i the DMR deotes that sigals are o loger detectable if I σ ε log I. Vice versa, the upper boud etry i the DMR meas that sigals are detectable as soo as I σ + ε log I, o matter if is kow or eeds to be estimated from the data. kow. It ca readily be see from Table that the lower ad the upper bouds with kow coicide up to the ±ε term i all regimes. This directly implies that the detectio boudaries are determied by these costats, which we will describe i more detail ow. 7

8 DMR: A compariso of the lower ad upper bouds i Table yields that the detectio boudary is give by I σ log I. 8 Hece, the detectio boudary i the domiat mea regime coicides with the detectio boudary i the homogeeous model cf. 6. More precisely, if the additioal iformatio κ about a jump i the variace vaishes faster tha the jump i mea, we caot profit from this iformatio asymptotically. ER: It follows similarly that the detectio boudary is give by I σ + c log I, 9 where c = lim σ/. The characterizatio 9 shows that we do always profit from the additioal iformatio i the ER case as +c < wheever c >. Compared to the homogeeous model or the domiat mea regime the costat improves but the rate stays the same. Note, that the improvemet ca be quite substatial, e.g. if c =, which amouts to the same magitude of σ ad, the additioal bump i the variace leads to a reductio of 33% sample size to achieve the same power compared to homogeeous bump detectio. If c =, we obtai a reductio of 66%. As ad κ are of the same order i the ER, we may replace by κ i 9 ad obtai the equivalet characterizatio I κ c + c log I. This formulatio allows for a compariso with the DVR below ad is also depicted i Table, secod ER row. DVR: We fid from etries three ad four i the DVR row of Table that the detectio boudary is give by I κ log I. If the jump i variace asymptotically domiates the jump i mea, we exactly obtai the detectio boudary for a jump i variace oly. This is a atural aalog to 8 ad coicides with the fidigs i the literature, cf. [9, Sect..8.7.]. Fially ote that if c i teds to, we ed up with. I this spirit a costat c < ca be also see as a improvemet over the pure DVR where the rate stays the same but the costat decreases, which is a aalog to the compariso of the DMR ad the ER, see above. A summary of the obtaied detectio boudary statemets is illustrated i Figure 3. ukow. I case that is ukow, it has to be estimated from the data Y ad the correspodig likelihood ratio test will be itroduced i Sectio 4.4. The chage i the cosidered statistics leads to differet upper bouds. It is readily see from Table that the obtaied upper bouds do ot coicide with the lower bouds i all regimes. DMR: It follows from the last etry i the DMR row that 8 also characterizes the adaptive detectio boudary for ukow. ER: If the parameter is ukow, it ca be see from the last etry i the ER row that it is uclear if the lower boud remais sharp. We do ot have a adaptive lower boud which would allow for a explicit statemet here. DVR: I this regime it is agai uclear if the lower boud is also sharp i case that is ukow agai we do ot have a adaptive oe, but we cosider it likely that the obtaied upper boud + 3 cf. the last etry i the DVR row is sharp. 8

9 DMR ER DVR C, cf. 9 I κ C l I c = c c c = I C l I C, cf. Figure 3: Illustratio of the detectio boudaries 8, 9, ad i the differet regimes. The dashed gree lies illustrate the phase trasitios betwee the DMR, ER ad ER, DVR respectively, whereas the dashed blue lie depicts the phase trasitio betwee the rates for DVR ad DMR. Note, that o the left-had side the x-axis correspods to c, whereas o the right-had side x- ad y-axis are iverted ad the x-axis belogs to /c. This loss of a costat ca be iterpreted as the price for adaptatio ad is quatified by the ratio r c of 33 ad 5, 34 ad 6 ad 35 ad 8 respectively. The ratios are give by DMR, c =, +c c+ +3c r c = +c ER, < c <, + 3 DVR, c =, which are displayed i Figure 4. Note that r is a cotiuous fuctio. Remarkably, the price for adaptatio is ever larger the, as r attais its maximum at c =, such that r =. As c, r teds to + 3/, as c, it teds to, meaig that adaptatio has o cost at all i this situatio. Note, that this is i lie with fidigs for the homogeeous Gaussia model [4]. r c + 3 c Figure 4: The price of adaptatio r plotted agaist the parameter c <. If we relax the regimes ER ad DVR by allowig for σ σ >, thigs become more complicated. We will prove lower bouds for those cases as well, which iclude logarithmic terms i 9

10 κ = lim κ cf. Theorems 4. ad 4.3 ad values of C i b ad b respectively. Our geeral methodology cf. Theorem 3.3 will also allow for upper bouds i these situatios cf. 7 ad 9, but it is clear that they caot be optimal for the followig simple reaso: The uderlyig deviatio iequality does ot iclude logarithmic terms i κ, either do ay from the literature cf. Appedix A. 3. Geeral methodological results Fixig I i 5, we obtai the likelihood ratio by straightforward calculatios as L,I, κ ; Y = κ + I exp Yi + Y i + κ = κ + I exp κ + κ σ κ i: i I i: i I Y i + κ I κ. For the geeral testig problem 5 where oly the size I is kow but the locatio I ot, the likelihood ratio fuctio is the give by sup I A L, I, κ ; Y. 3.. Lower bouds Let Φ be ay test with asymptotic sigificace level α uder the ull-hypothesis µ, λ σ. We will ow determie classes S := S N of bump fuctios, such that Φ is ot able to differetiate betwee the ull hypothesis µ, λ σ ad fuctios µ, λ S with type II error α. To this ed, we costruct a sequece S such that ᾱ Φ α + o β Φ α o, 3 which is equivalet. For such classes S we say that S is udetectable. I the followig, the sequece S will always be characterized by asymptotic requiremets o, κ ad I. To keep the otatio as simple as possible, oly the asymptotic requiremets will be stated below, meaig that the sequece S S = { µ = I, λ = σ + σ I, I, κ satisfy the specified requiremets } is udetectable. To prove lower bouds we use the followig estimate, which has bee employed i [, 4] as well: Lemma 3.. Assume that 3 hold true ad A is give by 4. If Φ is a test with asymptotic level α, uder the ull-hypothesis µ, i.e. ᾱ Φ α + o, the [ ] β Φ α E µ,λ L, I, κ, Y + o, 4 l I A as. Recall that we wat to costruct S such that the right-had side of 4 teds to. This ca be achieved as i [, Lemma 6.] by the weak law of large umbers. By cotrollig the momets of L, I, κ, Y, this leads to the followig theorem characterizig udetectable sets S: Theorem 3.. Assume the HBR model is valid with kow parameters, κ ad I. The sequece S determiig the asymptotic behavior of, κ ad I is udetectable, if I ad there exists a sequece δ >, satisfyig δ < /κ such that for, I σ + δ δ δ κ δ I log + κ I log δ κ + δ log I. 5

11 3.. Upper bouds To costruct upper bouds we will ow cosider the likelihood ratio test. Recall that I is kow, but the true locatio of the bump is ukow. Motivated by it is determied by T,κ, I, Y := sup I A i: i I Y i + κ 6 as test statistic. For simplicity of presetatio, we will just write T for the statistic i 6 i the followig ad drop the depedece o the parameters. Furthermore let { if T Y > c α,, Φ Y := 7 else be the correspodig test where c α, R is determied by the test level. I the followig we will be able to aalyze the likelihood ratio test 7 i DMR, ER ad DVR with the help of Lemma A.. For the relaxed situatios where κ we will prove lower bouds icludig logarithmic terms i Theorems 4. ad 4.3. Upper bouds icludig logarithmic terms caot be obtaied by the deviatio iequalities i Lemma A., ad we furthermore foud o deviatio iequality icludig logarithmic terms i the literature. Eve worse we are ot i positio to prove such a iequality here, ad thus we will ot be able to provide upper bouds which coicide with the lower oes i the relaxed regimes. Now we are able to preset the mai theorem of this Sectio: Theorem 3.3. Assume the HBR model with I, σ > holds true ad let H, H be as i 5. Let α, be a give sigificace level ad c α, = I + I log α I + I + κ4 log. 8 κ4 α I Assume furthermore that, I ad κ satisfy the followig coditio: I κ κ I κ log + κ log I + I κ α 4 + log α I + + κ I κ κ log α 9 The the test 7 with the statistic give T defied i 6 ad the threshold 8 satisfies lim sup ᾱ Φ α ad lim sup β Φ α. This theorem allows us to aalyze the upper bouds obtaied by the likelihood ratio test i the regimes DMR, ER ad DVR. Remark 3.4. Suppose for a secod that ot oly the width I, but also the locatio I of the bump is kow. I this case, the alterative becomes simple, ad the aalyzed likelihood ratio test will be optimal as it is the Neyma-Pearso test. It ca readily be see from the proofs that all log I -terms vaish i this situatio, whereas the other expressios stay the same. Thus our aalysis will also determie the detectio boudary i this case.

12 4. Lower bouds for the detectio boudary We will ow determie lower bouds i the three differet regimes by aalyzig Domiat mea regime DMR As metioed before, we expect the same lower bouds as for the homogeeous situatio here. Theorem 4.. Assume the HBR model with I ad let ε be ay sequece such that ε, ε log I.. If σ/ ad σ = o ε as, the the sequece S with I ε log I is udetectable.. If σ / where σ = σ + o ε ad / = o ε as, the the sequece S with is udetectable. 4.. Equilibrium regime ER Now let us cosider the case that σ ad are asymptotically of the same order. Cosequetly κ ad will be of the same order. I this situatio we expect a gai by the additioal iformatio comig from σ >. I fact, the followig Theorem states that the costat i the detectio boudary chages, but the detectio rate stays the same: Theorem 4.. Assume the HBR model with I ad let ε be ay sequece such that ε, ε log I.. Let σ = cσ + o ε, c > ad σ = o ε as. The the sequece S with as is udetectable. I C ε log I, C := σ + c a. If σ = σ + o ε ad = σ cσ + o ε as, the with κ := σ/σ the sequece S with I C ε log I, C := κ c κ + c log + κ as is udetectable. b I the first case, for c =, we have C = σ, which correspods to o chage i the variace ad hece reduces to the homogeeous model. But if c >, we always obtai C < σ, more precisely we improve by the multiplicative factor +c <. κ As C = c κ + c log + κ / is ot a multiple of, the gai i the secod case b is ot that obvious. But usig log + κ κ with equality if ad oly if κ = implies that C σ with equality if ad oly if κ =. Fially, if σ, the by a Taylor expasio it follows that κ κ c + c log + κ = + c 4 + O σ 6, σ, ad hece b will reduce to a i this situatio.

13 4.3. Domiat variace regime DVR Now let us cosider the case that the jump i the mea vaishes faster tha the jump i the variace σ. I this situatio we agai expect a further gai by the additioal iformatio. Somewhat surprisigly, we will eve obtai a gai i the detectio rate compared to the ER i. Theorem 4.3. Assume the HBR model with I ad let ε be ay sequece such that ε, ε log I.. If σ = σθ with sequeces, σ, θ as where σ = o ε ad θ = o ε, the the sequece S with I σ ε log I θ a is udetectable.. If σ = σ + o ε ad = o ε as, the the sequece S with is udetectable. I C ε log I, C := κ log + κ b First ote that a ca be equivaletly formulated as I κ ε log I, 3 which is also meaigful if ad hece determies the detectio boudary if oly a jump i the variace with homogeeous mea µ occurs. We wat to emphasize that the expoet of κ i 3 seems atural, as testig the variace for a chage is equivalet to testig the mea of the trasformed quatities Y i I i/ see [9, Sect..8.7.]. Cosequetly, the detectio rate improves compared to the DMR, as we assume κ /. Note that we ca also rewrite a i terms of the κ -rate, which gives the equivalet expressio I κ C ε log I, C := c + c. 4 This makes the ER comparable with the DVR. As expected C, ad thus we see a clear improvemet i the costat over 3. If c, the C teds to which coicides with 3. Agai, if κ it ca be see by a Taylor expasio that b reduces to a. Furthermore, b ca be see as the atural extesio of b as c Discussio The coditio σ = o ε seems to be uavoidable as soo as a Taylor expasio cf. 49c is about to be used. I several assumptios we require a covergece behavior of the form + o ε. Note that this caot readily be replaced by + o, because the detectio boudary is well-defied oly up to a term of order ε. Thus if the covergece behavior is slower, at least the costats will eed to chage depedig o the specific covergece behavior. 5. Upper bouds for the detectio boudary After determiig lower bouds for the detectio boudary, we ow aim for provig that those are sharp i the followig sese: There exists a level α test which detects with asymptotic type II error α bump fuctios havig asymptotic behavior of the form, ad where C ε o the right-had side is replaced by C + ε. 3

14 5.. Domiat mea regime DMR Let us first cosider the case σ / ad σ = o ε as. The followig theorem states that our lower bouds are optimal by exploitig the likelihood ratio test: Theorem 5.. Assume the HBR model with I ad let H, H be defied by 5. Suppose ε > is ay sequece such that ε log I as ad α [, ] is a give sigificace level. Furthermore, let. either σ / ad σ = o ε. or σ / ad σ = σ + o ε as. If I + ε log I, 5 the the test 7 with T defied i 6 ad the threshold 8 satisfies 5.. Equilibrium regime ER lim sup ᾱ Φ α ad lim sup β Φ α. Theorem 5.. Assume the HBR model with I ad let H, H be defied by 5. Suppose ε > is ay sequece such that ε log I as ad α [, ] is a give sigificace level. Furthermore let σ = cσ + o ε with c > ad σ = o ε as ad I C + ε log I, C := σ + c. 6 The the test 7 with T defied i 6 ad the threshold 8 satisfies lim sup ᾱ Φ α ad lim sup β Φ α. Comparig Theorems 4. ad 5. oe may ote that our upper bouds so far do ot hadle the case where κ does ot ted to. I fact we ca also obtai a upper boud from 9 for σ = σ + o ε ad = σ cσ + o ε as, i.e. I C + ε log I, C := κ + 4κ c + κ4 c + + c + κ + κ c + c + κ4 c. 7 Obviously, this boud does ot coicide with the lower bouds from Theorem 4.. This is due to the fact that our tail estimates for the χ distributio does ot iclude ay logarithmic terms, but our lower bouds do. Thus it is impossible to obtai the same bouds usig Lemma A. with our techique Domiat variace regime DVR Theorem 5.3. Assume the HBR model with I ad let H, H be defied by 5. Suppose ε > is ay sequece such that ε log I as ad α [, ] is a give sigificace level. Furthermore let σ = σθ with sequeces, σ, θ as where σ = o ε, θ = o ε ad I σ + ε log I θ 8 The the test 7 with T defied i 6 ad the threshold 8 satisfies lim sup ᾱ Φ α ad lim sup β Φ α. 4

15 Note agai that we ca also obtai upper bouds i case that σ = σ + o ε ad = o ε as, i.e. I C + ε log I, C := κ + + κ, 9 which agai caot coicide with the lower bouds from Theorem 4.3 for the same reaso as i the equilibrium regime Adaptatio I this Sectio we will discuss a adaptive versio of the likelihood ratio test for which does ot eed to be kow. I this case it is atural to replace by its empirical versio ˆ := I i: Y i. This leads to the margial likelihood ratio i I L I, κ ; Y = κ + I exp κ I κ i + + κ + ˆ ad to its correspodig test statistic T κ, I, Y := sup κ = κ + I exp κ + A A κ κ + i: i A i: i I Y i: i I Y i ˆ + I Y i ˆ + I σ i: i A Y i ˆ. 3 Agai we will drop the depedece o the parameters subsequetly ad write T := T κ, I, for the statistic i 3 to ease the presetatio. Let Φ Y := { if T Y > c α,, else 3 be the correspodig adaptive test where c α, R is determied by the test level α. A aalysis of the likelihood ratio test 3 with T as i 3 ca be carried out similarly to the o-adaptive case. The mai differece is that we ow eed Lemma A. with k =. Thus, the followig result ca be prove: Theorem 5.4. Assume the HBR model with I ad let H, H be defied by 5. Suppose ε > is ay sequece such that ε log I as ad α [, ] is a give sigificace level. Defie the correspodig threshold by [ ] c,α := κ κ + I ++ κ 4 κ + I + log + log 3 α I α I Now suppose that we are i oe of the followig three situatios: DMR: κ / ad / = o ε as ad I + ε log I. 33 ER: σ = cσ + o ε with c > ad σ = o ε as ad I C + ε log I, C := σ c + + 3c + c. 34 5

16 DVR: σ = σθ with sequeces, σ, θ as where σ = o ε, θ = o ε ad furthermore I + 3 σ + ε log I θ. 35 I ay of these cases the test 3 with the statistic give T defied i 3 ad the threshold 3 satisfies lim sup ᾱ Φ α ad lim sup β Φ α. The above theorem states upper bouds which are adaptive to the height of the bump. It is remarkable that the adaptive upper boud coicides with the o-adaptive oe cf. Theorem 5. i the domiat mea regime, where i the two other regimes the rates coicide but we lose a costat. 6. Simulatios I this Sectio we will examie the fiite sample properties of the test discussed i Sectio 5 by meas of a simulatio study. We therefore implemeted the o-adaptive likelihood ratio test Φ cf. 7 with the threshold c α, as i 8 as well as the adaptive likelihood ratio test Φ cf. 3 with c α, as i 3. For these tests the levels ᾱ Φ, ᾱ Φ ad powers β Φ, β Φ have bee examied by 4 simulatio rus for α =.5 with differet parameters σ,, κ, ad I, respectively. All derived detectio boudaries ivolve the ratio I / log / I. For the simulatios below, we therefore fixed this ratio ad the cosidered three situatios, amely small sample size I =., = 89, medium sample size I =.5, = 57 ad large sample size I =.5, = 53. I all three situatios we computed the power for σ =, κ {.,.,...,.} ad {.,.,...,.7}. The correspodig results are show i Figure 6. Power loss caused by adaptivity. By comparig the costats i Theorems ad Theorem 5.4 it ca be see as discussed i Sectio that the adaptive test is asymptotically at least as powerful as o-adaptive test if ad κ are multiplied by a factor of. By comparig subplots 5b ad 5a, 5d ad 5c ad 5f ad 5e respectively, it ca be see roughly that this also holds true i the simulatios. Iterestigly, the shape of the fuctio r from Figure 4 becomes also visible as a small belly o the lower right i the adaptive large sample situatio subplot 5f, i.e. has iitially to icrease with κ to reach the same power. Improvemet by heterogeeity. Let us cosider a exemplary situatio from the medium sample size case, i.e. = 57 ad I =.5. From subplot 5c it ca be see that the empirical power for =. icreases dramatically whe we alter κ from κ = to κ =.5. For κ = we use the stadard likelihood ratio test for compariso, which is based o the statistic T I Y = sup I A i: i I Y i ad uses the exact quatiles of N, I ad the uio boud to compute the threshold c α,. A compariso of the two situatios is show i Figure 6. Remarkably, the o-adaptive test at level 5% is able to detect the bump whe κ =.5, but ot whe κ = for the depicted data. Note that the empirical power of the test i the heterogeeous situatio is 47.%, whereas the oe of the test i the homogeeous situatio is oly.5% ote that this is larger tha the power depicted i Figure 6 for κ =. due to the used exact quatiles for the homogeeous test. 6

17 .8.8 κ κ a o-adaptive test power, small sample case b adaptive test power, small sample case.8.8 κ κ c o-adaptive test power, medium sample case d adaptive test power, medium sample case.8.8 κ κ e o-adaptive test power, large sample case f adaptive test power, large sample case Figure 5: Power of o-adaptive kow mea ad adaptive ukow tests plotted for vs. κ i three differet situatios small sample size I =., = 89, medium sample size I =.5, = 57, ad large sample size I =.5, = 53, each simulated by 4 Mote Carlo experimets. 7. Coclusios ad future work Summary. We obtai asymptotic miimax testig rates for the problem of detectig a bump with ukow locatio I i a heterogeeous Gaussia regressio model, the HBR model. The lower boud coditio 5 guides us for a distictio ito three possible regimes i the HBR model. It allows to quatify the ifluece of the presece of a additioal iformatio about the chage i the variace o the detectio rate. Both cases of kow ad ukow bump height 7

18 4 Y i µ λ a homogeeous bump detectio - the likelihood ratio test caot detect a bump 4 Y i µ λ b heterogeeous bump detectio - the likelihood ratio test detects a bump Figure 6: Compariso of homogeeous ad heterogeeous bump detectio. We simulated data from the model oce with λ homogeeous ad oce with λ x = +.5 I x heterogeeous for the same errors Z i, i. Here = 57, I = [.5,.55] ad =.. Eve though the observatios look quite similar i both situatios, the o-adaptive test 7 with level 5% is able to detect a bump i the heterogeeous situatio, but ot i the homogeeous situatio. are cosidered. I three possible regimes of the HBR model the exact costats of the asymptotic testig risk are obtaied for the case of kow. I the case of adaptatio to a ukow the obtaied rates remai the same as for the case of kow. The optimal costat coicides i the DMR with the case for kow, else it is ukow see Table for details. The obtaied results are based o o-asymptotic deviatio iequalities for the type I ad type II errors of testig. We would like to stress that the thresholds 8 ad 3 of both tests are o-asymptotic i the sese that they guaratee the o-asymptotic sigificace level α. This allows to apply the proposed tests eve for fiite samples. We also provide o-asymptotic upper detectio boud coditios 9 ad 53. Lemma 3. o the lower boud estimate ca be easily prove for the case of a o-asymptotic type I error as well. Ope issues ad extesios. Our results are icomplete cocerig the situatio if κ κ > i the equilibrium ad the domiat variace regime. We applied the same techiques to prove lower bouds i this case, ad we cojecture that these lower bouds are optimal, i.e. determie the detectio boudary. We were ot able to prove coicidig upper bouds due to limitatios i the used deviatio iequalities. To do so we would eed deviatio iequalities for χ radom variables ivolvig the logarithmic terms from the momets, but such a result is beyod the scope of this paper. Furthermore, we oly ivestigated adaptatio to, but it would also be iterestig to see what happes if the baselie variace σ or / ad σ is ukow. By usig the empirical variace o I as a estimator, this would lead to a likelihood ratio test with a test statistic ivolvig fourth powers of Gaussia radom variables. For the aalysis good tail iequalities for such radom variables are o demad. Additioally, oe could also ask for adaptatio w.r.t. I ad correspodig lower bouds, but 8

19 this is far beyod the scope of this paper ad requires further research. See, however, [4] for the homogeeous case. Aother importat ope questio is cocered with the situatio of multiple jumps. I the homogeeous case this has bee addressed i [4], ad upper bouds with costats 4 for a bouded ad for a ubouded umber of chage poits have bee prove. Eve though it seems likely that these costats are optimal, o correspodig lower bouds have bee prove so far. I the preset HBR model this is a ope questio as well. As stressed i the Itroductio we curretly oly deal with the HBR model, i.e. that a jump i mea also eforces a jump i variace κ >. But ofte it is more reasoable to cosider the situatio that wheever a jump i mea happes, there ca also be a jump i variace, but ot coversely. This would be modeled by lettig κ [,. For simplicity oe could restrict oeself to κ {, κ, }. Still it is uclear what the detectio boudary should be i this situatio. I fact it is ot eve clear if the additioal ucertaity leads to a loss of iformatio. We believe that lower bouds ca be costructed i the same way ad it is likely that they stay the same, but the calculatio of upper bouds seems quite more ivolved. Ackowledgemets AM wats to thak A. Goldeshluger for helpful discussios. The authors are grateful for several helpful commets of two referees ad the associate editor which lead to a improvemet i clarity ad dictio. AM ad FW gratefully ackowledge fiacial support by the Germa Research Foudatio DFG through subproject A7 of CRC 755, ad AM also ackowledges support through FOR 96, subprojects B, B3, ad CRC 83 subproject Z. We also thak A. Bartsch, O. Schütte ad C. Steiem Istitute of Orgaic ad Biomolecular Chemistry, Uiversity of Göttige for providig the io chael data. A. A chi-squared deviatio iequality For the aalysis of the likelihood ratio test ad the adaptive versio for ukow i Subsectio 5.4 a specific deviatio iequality for the weighted sum of two o-cetral χ distributed radom variables is required. Recall that X χ d a with o-cetrality parameter a ad d degrees of freedom, if X = d j= ξ j + a j where ξ j i.i.d. N, ad a = d a j. I this case E [X] = d + a ad V [X] = d + a. I the followig we cosider the weighted sum of k o-cetral chi-squared variables Z = k i= b ix i, where b i ad X i χ d i a i are idepedet with di N, a i, i =,..., k. Note that k k E [Z] = b i d i + a i ad V [Z] = b i d i + a i. i= Lemma A.. Let Z = k i= b ix i, where b i ad X i χ d i a i are idepedet with di N, a i, i =,..., k. Let b = max i k b i. The the followig deviatio bouds hold true for all x > : P Z E [Z] V [Z] x exp x, 36a P Z > E [Z] + V [Z] x + b x exp x. 36b The above Lemma is a reformulatio of [35, Lemma ]. i= j= 9

20 I the followig we will briefly discuss other deviatio iequalities for χ distributed radom variables [7, 8,, 7, 4, 49] ad their coectios to 36 from the perspective of our purpose. At first we metio a iequality by Birgé [8, Lemma 8.], which deals with k = ad coicides with the above result i that case. Also related to 36 is Prop. 6 i Rohde & Dümbge [4], P Z > E [Z] + V [Z] x + 4 b x + b x exp x, x >. 37 This differs from our boud 36b by the additioal 4 b x term i the square root, revealig 36b as strictly sharper. Other deviatio results were prove for secod order forms f w s = s T B T Bs + w T s, s R d of Gaussia vectors with B R d d, w R d. To relate this with our situatio, let d = k i= d i, choose ay a j,i R such that d i j= a j,i = a i for all i k ad set B := diag b d, b d,..., b k dk, w := b a,, b a,,..., b a d,, b a,,..., b a d,, b 3 a,3,... T. Here m =,..., R m. Let Tr A be the trace of a matrix A. The Tr B T B = Tr B = k i= b id i, Tr B 4 = k i= b i d i, B = b ad w T w = k i= b i a i. This yields f w ζ = Z k b i a i = Z E [Z] + Tr B, ζ N, I d 38 i= with Z as i Lemma A.. A deviatio iequality for the case w ca be foud i the moograph by Be-Tal et al. [7, Prop. 4.5.], which provides i our settig the boud P f w ζ > TrB + B x + TrB 4 + w T wx + B x exp x 39 for all x >, where B deotes the spectral orm of B give by the square root of the largest eigevalue. Takig 38 ito accout, this yields k P Z > E [Z] + b i d i + a i x + b x + b x exp t, t > i= for Z as i Lemma A.. As 8 k i= b i di + a i > V [Z], this iequality is strictly weaker tha the Dümbge-Rohde estimate 37 i the preset settig. For the case w = correspodig to cetered χ radom variables or a i = for i k i our otatio, Lemma A. reduces to the deviatio iequality i Hsu et al. [7]: P f ζ > TrB + TrB 4 x + B x exp x, x >. 4 The situatio w = has also bee cosidered by Spokoiy & Zhilova [49], where the followig iequality is prove: { P f ζ > TrB + max } TrB 4 x, 6 B x exp x, x > 4 Comparig 4 with 4 gives that 4 is sharper for moderate deviatios, i.e. if 3 Tr B 4 B x 4 Tr B 4 B, for large ad small deviatios it is weaker. We are actually iterested i large deviatios, ad furthermore the restrictio a i = makes 4 ot applicable for our purpose.

21 B. Proofs of Sectio 3 Proof of Lemma 3.. We follow [], as. β Φ α if µ,λ S E µ,λ [Φ Y ] α if E µ= I I,λ A =σ +σ [Φ Y ] α I [ ] E l µ= I,λ =σ +σ [Φ Y ] α I I A E l µ= I,λ =σ +σ [Φ Y E I µ [Φ Y ]] + o I A [ ] =E µ,λ L, I, κ, Y Φ Y + o l I A [ ] E µ,λ L, I, κ, Y + o, l I A Proof of Theorem 3.. By Lemma 3. we have to prove that the right-had side of 4 teds to uder the give coditios. Suppose that lim E [ µ,λ σ { L,I,κ,Y ξl } L, I, κ, Y ] = 4 for ay ξ >. The the weak law of large umbers for triagular arrays the coditio 4 implies the coditios imposed i [44, Ex..] is applicable as I ad hece l, i.e. m m i= Z i i probability with m = l ad Z i = L, [i I, i I ], κ, Y. As Z i > we have [ ] m E Z i = m E [ Z i ] = m E [Z i ] = for all m N, m m m i= i= which the implies m m i= Z i i expectatio, i.e. 4 teds to. The oly thig left to prove is that 4 holds true uder the imposed coditios. To show this we will use the momets of L = L, I, κ, Y uder the ull hypothesis. Note that uder the ull hypothesis µ it holds Y i i.i.d. N, σ ad thus due to idepedece for η > i= E µ,λ [L η ] = + κ I η E µ,λ exp η = + κ I η E µ,λ = + κ I η i: i A,j E [ exp κ i: i A,j Yi + Y i + κ exp η κ Yi + Y i + κ σ i: i A,j η κ X + σ X ] + κ = [ + κ I η E exp η κ X + σ X ] I + κ, 43

22 where X N,. Thus we eed to calculate the expectatio of exp η κ X + σ X + κ = exp η exp ηs X + λ 44 κ where we abbreviated s = κ + κ 45 λ = σ κ. 46 The right-had side i 44 correspods to the Laplace trasform of a o-cetral χ distributed radom variable give by [ E exp t X + λ ] λ exp t t = 47 t if λ R, t < /, otherwise the Laplace trasform does ot exist. Note that t = ηs < / if ad oly if η < + /κ. I this case the expectatio is give by [ E exp η κ X + σ X ] [ + κ = exp η E exp ηs X + λ ] κ + κ = η η exp + η κ + η. κ By 43 this implies E µ,λ [L η ] = + κ I η = + κ η I + η κ I + κ exp + η κ η η I exp + η. κ η η σ + η κ To prove 4 we ow ote for ay such that l /ξ as L > we have E µ,λ [ { L ξl } L ] = ξl P µ,λ L x + dx + ξl P µ,λ L ξl + ξl P µ,λ L x dx + ξl P µ,λ L ξl =E µ,λ [ {L ξl }L ] I [ E µ,λ L +δ] ξl δ I + δ δ = exp δκ δ I log + κ I log δκ + δ log I δ log ξ. 48 Now, for 48 to ted to, the expoet must coverge to. Cosequetly, the weak law of large umbers is applicable ad fially 4 teds to if 5 holds true.

23 Proof of Theorem 3.3. Recall that the test statistic is give by with the ier part T Y = SA := σ sup SA, A A i: i I Y i + κ We readily see that S A obeys the followig distributios Uder H : SA χ I I, σ κ4 Uder H : SI + κ χ I + κ Here I i the alterative deotes the true positio of the jump. Usig 36b with k =, b =, d = I ad a = I we fid κ4 P H sup SI > c α, I A I P H SI > c α, For the type II error deote = I P α. χ I I σ κ4. I > c α, σ κ4 y,α = + κ I + + κ I + κ I + + κ I κ4 log κ4 α. The we calculate usig 36a with k =, b = + κ, d = I ad a = + κ I κ4 that sup P B B A sup A A SA < y,α sup B A if P B SA < y,α A A sup P B SB < y,α B A =P + κ χ I + κ I < y,α α. Thus the claim is prove if y,α c,α, i.e. + κ I + + κ I I + I σ κ4 σ κ4 log α I + σ κ4 + κ I + + κ I I + log. κ4 α I σ κ4 log α But it ca be easily see by rearragig terms ad multiplyig by κ that this is equivalet to 9. 3

24 C. Proofs of Sectio 4 If δ κ ad / or κ, Taylor s formula ca be used to simplify some of the terms i 5, which we will use i the followig: + δ δ δ κ = δ + δ + κ + δ 3 κ + κ + κ 4 δ 4 + O κ 6 δ 4, 49a log δ κ = δ κ + δ κ 4 + O κ 6 δ 3, 49b δ log + κ = δ κ + δ κ 4 + O κ 6 δ. 49c Proof of Theorem 4.. Note that δ < κ is satisfied for free here as the right-had side diverges or is costat ad δ. Furthermore i both cases the assumptio I = O log I is satisfied as well. Thus we ca apply Theorem 3... Uder the assumptios of this part of the theorem, we may use 49a 49c ad isert κ = c /σ + o ε to fid that 5 is satisfied if δ [ I σ ] + c + log I + δ I + c + o δ ε I. 5 ε ad usig a we obtai Here we used that = o ε. The with δ := σ δ [ I σ ] + c + log I = ε log I C ε C = σ C ε log I σ C ε 3 log I ad furthermore δ I + c + C ε log I ε log I C ε C + C =σ = σ + c ε + O ε 3 log I. It ca be readily see that C := + c > for all c. Thus uder a we have [ I ] δ + c + log I + δ I + c = C ε + o ε log I as. Thus 5 is satisfied.. Here oly 49a ad 49b ca be applied. Thus 5 is satisfied if δ [ I σ log + κ ] + κ + o ε + log I ] [ +δ I + κ + κ4 4 + o ε + O δ 3 I 5 4

25 Isertig the assumptio = σ cσ + o ε = κ c + o ε, b ad δ := ηε with η > to be chose later we fid δ [ I log + κ ] + κ + o ε + log I =ηε log I [ C ε κ κ c + c log + κ ] + o ε =ηc ε log I + o ad [ δ I + κ ] + κ4 4 + o ε =η ε log I η C C ε κ 4 + ηc ε log I + o + O δ 3 I + κ c + κ4 4 κ =η ε log I η C C 4 + κ c + κ4 4 if we choose e.g. η = C 3 κ 4 c + κ + κ4 4 + o + o. Thus 5 is satisfied. Proof of Theorem 4... This follows directly from Theorem 4. with c = c.. Similarly as i the proof of Theorem 4. we have to show that 5 is satisfied. Therefore ote that with δ := ηε with η > to be chose later we have δ [ I log + κ ] + κ + o ε + log I [ =ηε log I ε + κ log + κ ] + o ε = ησ ε log I + o by the assumptio that / = o ε. Furthermore [ δ I + κ + κ4 4 σ =η ε log I ] + ησ ε log I + o η σ + κ + o if we choose e.g. η = + κ. This proves the claim. Proof of Theorem 4.3. Note agai that δ < κ is satisfied for free here as the right-had side diverges or is costat ad δ. Furthermore i both cases the assumptio I = O log I is satisfied as well. Thus we ca apply Theorem 3... Uder the assumptios of this part of the theorem, we may agai use 49a-49c to fid that 5 is satisfied if I δ + I κ4 4 + log I +δ I + I κ4 +o δ ε log I 4 5 5