The Inverse Gaussian Process as a Degradation Model

Transcription

1 1 / 47 The Inverse Gaussian Process as a Degradation Model Nan Chen Department of Industrial & Systems Engineering, National University of Singapore, Singapore NCTS Industrial Statistics Research Group Seminar Dec 7, 2012

2 2 / 47 Outline 1 Introduction Degradation Definition 2 Justification and links Limiting compound Poisson process Connection with Wiener process 3 IG Process with Heterogeneity Covariate models Random effects models Model selection and testing 4 Case Study Other related works

3 3 / 47 Degradation Degradation Small and accumulative damages which eventually make the product not functional.

4 Degradation Degradation models Characterizing the degradation levels over time ln[y(t)/y(0)] Percentage increases in current ^4 Cycles General path model Stochastic process model ^3 Hours 4 / 47

5 5 / 47 Degradation State of the art Wiener process Independent and normally distributed increment Well studied (Doksum and Hoyland 1992; Doksum and Normand 1995; Padgett and Tomlinson 2004; Wang 2010; Peng and Tseng 2009) Not monotone degradation path Gamma process Independent and Gamma distributed increment Monotone degradation path Limited choice of random effects model (Lawless and Crowder 2004)

6 Definition Inverse Gaussian process 1 Definition The IG process {Y (t), t 0} is defined as the stochastic process satisfying: Y (t) has independent increments, i.e., Y (t 2 ) Y (t 1 ) and Y (s 2 ) Y (s 1 ) are independent t 2 > t 1 s 2 > s 1 ; Y (t) Y (s) follows an IG distribution IG ( Λ(t) Λ(s), λ[λ(t) Λ(s)] 2), t > s 0, where Λ(t) is a monotone increasing function and IG(a, b), a, b > 0, denotes the IG distribution. 1 Ye, Z. S., and Chen, N. Technometrics, accepted. 6 / 47

7 7 / 47 Definition First passage time Monotone increasing sample path P(T D < t) = P(Y (t) > D) = 1 F IG ( D; Λ(t), λ Λ 2 (t) ) The CDF of the first passage time is [ ] [ ] λ λ P(T D < t) = Φ (Λ(t) D) e 2λΛ(t) Φ (Λ(t) + D) D D The density of the first passage time is f TD (t) = Λ (t) + { [ ] [ ] λ λ(λ(t) D) λ(λ(t) + D) D φ 2λe 2λΛ(t) Φ D D [ λ D e2λλ(t) φ ]} λ(λ(t) + D), D

8 8 / 47 Definition Birnbaum-Saunders distribution When λ Λ(t) is large, Y (t) is approximately normal with mean Λ(t) and variance Λ(t)/λ. the CDF of T D can be approximated as [ ] [ D Λ(t) λ D ] λ P(T D < t) 1 Φ = Φ Λ(t) Λ(t)/λ Λ(t) A Birnbaum-Saunders type distribution (Tang and Chang 1995), which is used to model the fatigue failure times caused by crack growth passing a critical value (Desmond 1985)

9 Limiting compound Poisson process Compound Poisson process Many degradation phenomena can be considered as accumulations of additive and irreversible damage caused by a sequence of external random (Esary and Marshall 1973; Desmond 1985; Singpurwalla 1995; van Noortwijk 2009) The shocks arrival process may be approximated by a Poisson process (Singpurwalla and Wilson 1998; Lawless and Crowder 2010) each shock causes a small and random amount of damage to the system 9 / 47

10 10 / 47 Limiting compound Poisson process Compound Poisson process

11 Limiting compound Poisson process Limiting process The discrete process is difficult to analyze 1 F(t) = k=0 P k exp( λt)(λt) k k! where P k is the probability surviving the first k shocks, involves k th convolution. Limiting approximation is useful Gamma process: when λ and jump size decreases to zero in certain proportion (Lawless and Crowder 2004) 11 / 47

12 Limiting compound Poisson process Limiting compound Poisson process Theorem Consider the compound Poisson process {C n (t), t > 0} with arrival rate ν n and jump size distribution G n (dx) = Π n (dx)/ν n, where ( λ ν n = 2πx exp λx ) dx 2µ 2 Π n (dx) = 1/n ( λ 2πx exp λx ) I 2µ 2 [1/n, ) (x)dx. When n, {C n (t), t > 0} converges to the stationary IG process {Y (t), t 0} with marginal distribution Y (t) IG(µt, λt 2 ). 12 / 47

13 13 / 47 Connection with Wiener process Inverse Gaussian distribution For a Wiener process with linear drift, W (t) = βt + σb(t) At time t, the location follows Gaussian distribution W (t) N(βt, σ 2 t) Conversely, the first time the process reaches location x follows inverse Gaussian distribution ( x T (x) IG β, x ) 2 σ 2

14 Connection with Wiener process Inverse variate of the Wiener process Considering a monotone increasing threshold function Λ(x), the first passage time process crossing Λ(x) becomes the IG process indexed by x. Λ(x j ) Λ(x i ) IG(Λ(x)/β, Λ(x) 2 /σ 2 ) 14 / 47

15 15 / 47 Connection with Wiener process Knowledge transfer Because of this close relationship, many models developed for Wiener process can be modified for IG process. Wiener process with random drift (Crowder and Lawless 2007; Peng and Tseng 2009) Wiener process with time transformation

16 16 / 47 Heterogeneity Units may experience different rate of degradation dues to Random effects factors Internal defects Unobservable or unaccountable random factors Covariates Usage difference Environmental conditions, like temperature, humidity

17 17 / 47 Covariate models Examples Heterogeneous degradation rate

18 Covariate models Covariate models Covariate may influence the rate, volatility, time scale, etc, which motivates following covariate models Y (t) IG(µΛ(t), λλ 2 (t)) Rate heterogeneity µ i = h(s i ); Volatility heterogeneity λ i = h(s i ) Rate and volatility heterogeneity µ i = h(s i ), λ = g(s i ) Time scale transformation Λ(t; s i ) Λ(h(s i ) t) 18 / 47

19 Covariate models Covariate models Some commonly used link functions include: Power law function: h(s) = ξ 0 s α ; Arrhenius function: h(s) = ξ 0 e α/s ; Inverse-logit function: h(s) = ξ 0 + αe s /(1 + e s ); Exponential function: h(s) = ξ 0 e αs, 19 / 47

20 Covariate models Parameter estimation Using maximum likelihood estimation: l(θ) = ln λ 2 N n i + i=1 n N i [ln Λ ij λ(y ] ij/h(s i ) Λ ij ) 2 2y ij i=1 j=1 i is the unit index n i : number of observations from unit i y ij : degradation increment from t i,j 1 to t ij Λ ij : increment of the shape function Λ(t) 20 / 47

21 Random effects models Random effects models From the basic degradation model Y (t) IG(µΛ(t), λλ 2 (t)) consider some of the parameters as random variables µ is random means the hidden effects cause different degradation rate In theory, the distribution of the random effects could be quite arbitrary Limited choice if we want faster computation and estimation 21 / 47

22 22 / 47 Random effects models Random drift model The drift parameter µ is random. Remember the Wiener process with random drift: W (t) = βt + σb(t), β N(ω, κ 2 ) The induced IG process is IG(Λ(x)/β, Λ(x) 2 /σ 2 ) Here β can still be normally distributed. Using the previous notation, and only keeping the positive parts µ 1 T N (ω, κ 2 )

23 Random effects models Random drift model Given the initial distribution parameter ω, κ 2, and the degradation data of unit i, Y i = [y i1, y i2,, y i,ni ], the conditional distribution of µ i is [µ 1 i Y i ] T N ( ω i, κ 2 i ), where κ i = λy i (t i,ni ) + κ 2 and ω i = (λλ(t i,ni ) + ωκ 2 )/ κ 2 i. Easy computation and updating Preserve the Markov property Conjugate distribution, the same form for future prediction 23 / 47

24 24 / 47 Random effects models Marginal distribution of random drift model Since the random effects are not observable, we often integrate them out to obtain the marginal distribution of Y i f IG (Y i ) = 1 Φ( κ i ω i ) 1 Φ( κω) κ n i λλ2 ij exp κ2 i ω i 2 κ 2 ω 2 n i Λ 2 ij λ κ i 2 j=1 2πy 3 ij 2y ij j=1 Can be used for parameter estimation and degradation prediction Can be simplified if µ 1 N(ω, κ 2 ) instead of µ 1 T N (ω, κ 2 ), or when ωκ is large

25 25 / 47 Random effects models Estimation using EM Consider µ = [µ 1, µ 2,, µ N ] as the unobserved data E-Step u i E( 1 Y, Θ (k) φ( ω i κ i ) ) = ω i + µ i [1 Φ( ω i κ i )] κ i v i E( 1 µ 2 Y, Θ (k) ) = ω i ω i i M-Step Θ (k+1) = arg max Θ n N i i=1 j=1 [ N + i=1 φ( ω i κ i ) [1 Φ( ω i κ i )] κ i + [ 1 κ ] i ω i φ( κ i ω i ) 1 1 Φ( κ i ω i ) κ 2, i [ ln λ 2 λ ] 2 (v i y ij 2u i Λ ij + Λ 2 ij /y ij ) + ln Λ ij ) ln κ ln(1 Φ( ωκ)) κ2 2 (v i 2u i ω + ω 2 ) ],

26 26 / 47 Random effects models Random volatility model When the hidden effects does not change the degradation rate, but change the volatility (variance) of the increment Conditional distribution λ Γ(γ, δ) [λ i Y i ] Γ( γ i, δ i ) where γ i = γ + n i j=1 (y ij µλ ij ) 2 /(2µ 2 y ij ) and δ i = n i /2 + δ. Marginal distribution f IG (Y i ) = Γ( δ i ) Γ(δ) γ δ γ δ i i n i j=1 Λ 2 ij 2πy 3 ij

27 27 / 47 Random effects models Random drift-volatility model Sometimes higher drift couples with larger variation Y (t) IG(µΛ(t), λµ 2 Λ 2 (t)) and µ T N (ω, κ 2 ) Conditional distribution g(µ i Y i ) = 1 n i λ(y ij θ i Λ ij ) C µn i exp 2 κ2 (θ i ω) 2 I [0, ) (θ i ) 2y ij 2 j=1 Not in a conjugate form The moment E(µ k i Y i), k 1 can be computed efficiently

28 28 / 47 Random effects models Estimation performance RD model RV model RDV model True values λ = 1/3 ω = 1/2 κ = 4 Bias MSE Bias MSE Bias MSE Bia (50, 40) 1.1e-2 5.8e-2-1.3e-2 7.9e (100, 80) 3.7e-3 3.4e-2-7.0e-3 4.5e True Value µ = 1/2 δ = 1 γ = 3 Bias MSE Bias MSE Bias MSE Bia (50, 40) 8.5e-3 4.2e-2 6.8e-2 2.2e-1 4.8e (100, 80) 5.5e-3 2.4e-2 2.5e-2 1.4e-1 2.1e-1 6.4e True Value λ = 2/3 ω = 1/2 κ = 4 Bias MSE Bias MSE Bias MSE Bia (50, 40) -3.4e-2 1.2e-2-1.2e-2 5.2e-2 1.9e-1 6.0e (100, 80) 2.4e-2 7.2e-3-1.1e-2 3.7e-2 1.2e-1 4.0e-1 1.0

29 29 / 47 Model selection and testing Goodness of fit tests How do we know if the IG process (or its variants) fit the data well? If X IG(a, b), then b(x a) 2 /(a 2 X) χ 2 1. We can use the transformed degradation increment for GOF test In simple models, we have ζ ij ˆλ(y ij ˆΛ ij ) 2 y ij approx χ 2 1

30 30 / 47 Model selection and testing Increment based GOF When the random effects are present, use the conditional mean of the random effect parameter in the test statistic ˆζ ij = ˆλ(ˆµ 1 i y ij ˆΛ ij ) 2 y ij approx χ 2 1 where ˆµ 1 i = E(µ 1 i ˆθ, Y i ). Tests for other random effects variants or covariate models can be similarly developed

31 Model selection and testing Examples of GOF plot Sample Quantile Fixed Drift Shape D&S Theoretical Quantile Sample Quantile Fixed Drift Shape D&S Theoretical Quantile 31 / 47

32 Model selection and testing Failure time based GOF If the degradation model is appropriate, the projected failure time distribution should agree with the empirical failure time data 1.0 Fixed Drift Shape D&S Sample Quantile Theoretical Quantile Fixed Drift Shape D&S 1.0 Sample Quantile / 47

33 33 / 47 Model selection and testing Other quantitative criteria AIC AIC = 2l(ˆθ) + 2k Root mean square prediction error (RMSPE) [ e ij = Y i (t ij ) E Y i (t ij ) ] ˆθ i, Y i (t ik ); k < j, j = 1, 2,, n i We use leave one out for the cross validation, and RMSPE = N n i N eij 2/ i=1 j=1 i=1 n i

34 GaAs laser degradation data Percentage increases in current ^3 Hours 34 / 47

35 35 / 47 Parameter estimation Simple Model RD Model Estimation CI.lower CI.upper Estimation CI.lower CI.upper µ ω λ κ q λ q RV Model RDV Model Estimation CI.lower CI.upper Estimation CI.lower CI.upper δ ω γ κ µ λ q q Consistent heterogeneity in degradation rate Negligible heterogeneity in volatility

36 Model selection Simple RD RV RDV Sample Quantile Theoretical Quantile Sample Quantile Simple RD RV RDV Theoretical Quantile 36 / 47

37 Conclusion A new class of degradation model Close relationship with Wiener process and compound Poisson process Easy and versatile in handling random effects and covariates Nonparametric extensions More powerful goodness-of-fit test 37 / 47

38 Other related works Semiparametric Estimation of Gamma Processes 2 Estimating a nonparametric shape function of the Gamma process when the data are not aligned press 2 Ye, Z.S., Xie, M., Tang, L.C., and Chen, N. Technometrics, in 38 / 47

39 39 / 47 Other related works EM framework Conventional solutions are either not dependable or inefficient.

40 Other related works Random effects model The random effects Gamma process where µ Γ(γ, δ) Y (t) Γ(µ, Λ(t)) The missing data is composed of two parts The realization of the random effects µ 1, µ 2,, µ N The observations at the pseudo observation points Y i,j E(µ i Y, Θ (k) ), E(ln µ i Y, Θ (k) ), E(ln y i,j Y, Θ (k) ) 40 / 47

41 Other related works Two phase degradation model Two (or more) phases in degradation Random change points 3 Chen, N. and Tsui, K.L. IIE Transactions, in press 41 / 47

42 Other related works Bayesian model Mathematical Model L ij = ln[s(t ij ) δ] = { ai1 + b i1 t ij + σ i1 ɛ ij, t ij γ i a i2 + b i2 (t ij γ i ) + σ i2 ɛ ij, t ij > γ i All parameters β im = [a im, b im ], σ im, γ i are random. Empirical Bayes Approximate P(γ L) by γ, the posterior mode P(L θ, γ) π(θ γ) π(γ) P(θ L) = P(θ, γ L)dγ = dγ P(L) P(L θ, γ) π(θ γ). P(L γ) 42 / 47

43 Other related works Remaining useful life Consider a countable set of possible failure times Improved approach P(R τ > T k τ Lτ) = P(L(T 1 ) K, L(T 2 ) K,, L(T k ) K L = MT k (K; ṽ 2, X µ 2, Σ 2 2) residual life time residual life time Simulation replications Simulation replications 43 / 47

44 Other related works RUL Prediction Predicting RUL during the first phase When γ τ. Consider the uncertain locations of the change point. tau T1 T2 Tk k s=1 Ts T s 1 P(L Ts K, L Ts+1 K,, L Tk K L, γ) π(γ γ > γ)dγ + T k π(γ γ > γ)dγ, The RUL becomes a mixture of multivariate t distribution. 44 / 47

45 45 / 47 Other related works Bearing example For a in-service unit, the RUL distribution predicted at two different time points. probability distribution probability distribution residual life (a) Prediction at an earlier time residual life (b) Prediction at a later time

46 Other related works Reference I Crowder, M. and Lawless, J. (2007), On a Scheme for Predictive Maintenance, European Journal of Operational Research, 176, Desmond, A. (1985), Stochastic Models of Failure in Random Environments, Canadian Journal of Statistics, 13, Doksum, K. A. and Hoyland, A. (1992), Models for Variable-Stress Accelerated Life Testing Experiments Based on Wiener Processes and the Inverse Gaussian Distribution, Technometrics, 34, Doksum, K. A. and Normand, S. L. T. (1995), Gaussian Models for Degradation Processes-Part I: Methods for the Analysis of Biomarker Data, Lifetime Data Analysis, 1, Esary, J. and Marshall, A. (1973), Shock Models and Wear Processes, The Annals of Probability, 1, Lawless, J. and Crowder, M. (2004), Covariates and Random Effects in a Gamma Process Model with Application to Degradation and Failure, Lifetime Data Analysis, 10, (2010), Models and Estimation for Systems with Recurrent Events and Usage Processes, Lifetime Data Analysis, 16, Padgett, W. and Tomlinson, M. A. (2004), Inference from Accelerated Degradation and Failure Data Based on Gaussian Process Models, Lifetime Data Analysis, 10, Peng, C. Y. and Tseng, S. T. (2009), Mis-Specification Analysis of Linear Degradation Models, IEEE Transactions on Reliability, 58, Singpurwalla, N. D. (1995), Survival in Dynamic Environments, Statistical Science, 10, Singpurwalla, N. D. and Wilson, S. (1998), Failure Models Indexed by Two Scales, Advances in Applied Probability, 30, Tang, L. C. and Chang, D. S. (1995), Reliability Prediction Using Nondestructive Accelerated-Degradation Data: Case Study on Power Supplies, IEEE Transactions on Reliability, 44, van Noortwijk, J. M. (2009), A Survey of the Application of Gamma Processes in Maintenance, Reliability Engineering & System Safety, 94, Wang, X. (2010), Wiener Processes with Random Effects for Degradation Data, Journal of Multivariate Analysis, 101, / 47

47 47 / 47 Other related works Thanks! Questions?