Huber smooth M-estimator. Mâra Vçliòa, Jânis Valeinis. University of Latvia. Sigulda,

Transcription

1 University of Latvia Sigulda,

2 Contents M-estimators Huber estimator Smooth M-estimator Empirical likelihood method for M-estimators

3 Introduction Aim: robust estimation of location parameter Huber M-estimator (1964) - well known robust location estimator Owen (1988) introduced empirical likelihood method, also applicable to M-estimators Hampel (2011) proposed a smoothed version of Huber estimator Work in progress Two sample problem: empirical likelihood based method for a difference of smoothed Huber estimators (Valeinis, Velina, Luta: abstract for ICORS 2011 conference)

4 M-estimators M-estimator Let X 1, X 2,..., X n iid, X 1 F. An M-estimator T n is defined as a solution of n ρ(x i, t) = ρ(x, t)df n (x), (1) i=1 for a specific funtion ρ where F n is the empirical CDF. If ρ is differentiable in t, then (1) is minimized by the solution of where ψ(x, t) = tρ(x, t). n ψ(x i, t) = 0, i=1

5 M-estimators Examples Mean. ψ(x, t) = x t gives T n = X. ML estimator. ψ(x, θ) = d dθ log f(x, θ) for a class of density functions f(x, θ), gives T n is the root of likelihood equation ( n ) d dθ log f(x i, θ) = 0. i=1 Median. ψ(x, t) = ψ 0 (x t), ψ 0 (z) = k sgn(z), k > 0.

6 Huber estimator Huber estimator for location parameter µ Huber (1964) combined examples of mean and median. Let F have a symmetric density f µ,σ (x) = 1 σ f ( x µ σ assume σ = 1. Then M-estimator for the location parameter µ is defined as n ( ) Xi t ψ = 0. (2) σ i=1 ), Huber M-estimator is defined by the function ψ in (2): k, x k ψ k (x) = x, k x k k, x k. (3)

7 Huber estimator Huber's motivaton: Unrestricted ψ-functions have undesired properties (unstable to outliers); Cosider the limiting values of k in ψ k and their respective M-estimators: If k, then ψ k is mean; If k 0, then ψ k is median. k is a tuning constant determining the degree of robustness. Huber estimator has minimax assymptotic variance for class of distribution functions (1 ɛ)φ(x) + ɛh(x), where φ is pdf of N(0, 1) and h is a symmetric density.

8 Huber estimator Scaled estimator of location In reality σ is not known, thus a robust estimate of σ should be used. A common choice is MAD. MAD S n = MAD = median( X i median(x i ) ). Robust estimator is acquired, even in presence of outliers (up to 50% of the sample).

9 Smooth M-estimator Smoothed M-estimator (Hampel, 2011) For a general ψ-function of an M-estimator define ψ(x) = ψ(x + u)dq n (u), (4) where Q n may be chosen as a the distribution of the initial M-estimator Q n can be approximated by N(0, V/n), where V is assymptotic variance of the M-estimator. Need to specify distribution under which the assymptotic variance is computed. The smoothing prinicple can be applied to ψ functions already smooth.

10 Smooth M-estimator Smoothed Huber estimator The ψ-function of the smoothed Huber estimator defined by ψ = ψ k can be written in closed form as ( ) ( ( )) x k x + k ψ k (x) = kφ k 1 Φ σ n σ n ( ) ( )) x + k x k +x Φ Φ σ n σ n ( ) ( )) x + k x k +σ n (φ φ, (5) where σ n = V/n, and Φ and φ denote the cdf and pdf of N(0, 1). σ n σ n

11 Smooth M-estimator Example (a) (a) ψ function of Huber M-estimate; (b) ψ function of smoothed Huber M-estimate. k=1.35. (b)

12 Empirical likelihood method for M-estimators Empirical likelihood method for M-estimators Owen (1988) showed that EL method can be applied to certain M-estimators, including Huber estimator. Nonparametric Wilk's theorem applies thus EL based confidence intervals for Huber estimate can be obtained. Tsao, Zhu (2001) showed that EL based confidence intervals preserves robustness.

13 Empirical likelihood method for M-estimators EL confidence bands for Huber estimator Empirical likelihood ratio for parameter t R(t) = sup{ n ω i n n ω i ψ(x i, t) = 0, ω i 0, ω i = 1} i=1 i=1 i=1 is maximized by ω i (λ), where ω i (λ) = {n(1 + λz i )} 1, and Z i = ψ(x i, t) and λ follows from n 1 Zi /(1 + λz i ) = 0.

14 Empirical likelihood method for M-estimators 2lnL EL vid.vert EL Huber 2lnL EL vid.vert EL Huber Figure: EL -2*ln,(a) N(0, 3) (b) 0.95 N(0, 3) N(20, 3)

15 Empirical likelihood method for M-estimators Simulation results for one sample problem Table: Huber estimation for location parameter and its EL confidence bands, alpha=0.05 N(0, 3) 0.95 N(0, 3) N(20, 3) sample len estimate len estimate n=50 EL.huber EL.huber EL.huber EL.huber EL.mean EL.mean EL.mean 3.14 EL.mean t-test mean t-test mean z-test huber z-test huber Bootstrap Bootstrap n=20 EL.huber EL.huber EL.huber EL.huber EL.mean EL.mean EL.mean EL.mean t-test mean t-test mean z-test huber z-test huber Bootstrap Bootstrap n=10 EL.huber EL.huber EL.huber EL.huber EL.mean EL.mean EL.mean 9.68 EL.mean t-test mean t-test mean z-test 1.24 huber z-test 1.24 huber Bootstrap Bootstrap 9.74

16 Empirical likelihood method for M-estimators Two sample EL problem Consider empirical likelihood-based method for the difference of smoothed Huber estimators. Given two independent samples X and Y with distribution functions F 1 and F 2, respectively, we have two unbiased estimating functions: E F1 w 1 (X, θ 0, ) = 0, E F2 w 2 (Y, θ 0, ) = 0, where is the parameter of interest and θ 0 is a nuisance parameter. Specifically, = θ 1 θ 0 and ( ) ( ) X θ0 Y + θ0 w 1 (X, θ 0, ) = ψ w 2 (Y, θ 0, ) = ψ, ˆσ 1 ˆσ 2 where ˆσ 1 and ˆσ 2 are scale estimators, and ψ corresponds to the smoothed Huber estimator.

17 Empirical likelihood method for M-estimators Simulation results for two sample problem Conisder two models: Y 1 (1 ɛ)gamma(α = 5; σ = 1) + ɛuniform[0; 50] Y 2 Gamma(α = 1; σ = 5) Table: Coverage accuray and average confidence interval lengths based on 1000 replicates, n 1 = n 2 = 50 t.int EL.hub1 EL.hub2 Boot1 Boot2 acc ave acc len acc len acc len acc len σ = σ = σ = σ = σ =

18 Empirical likelihood method for M-estimators Thank you for your attention!