Statistical analysis of the inverse problem

Transcription

1 Statistical analysis of the inverse problem Hongyuan Cao and Jayanta Kumar Pal SAMSI SAMSI/CRSC Undergraduate Workshop at NCSU May 24, 2007

2 Outline 1 Introduction 2 Preliminary analysis Estimation of parameters Residual analysis

3 Introduction Outline 1 Introduction 2 Preliminary analysis Estimation of parameters Residual analysis

4 Introduction Estimation problem We discuss the statistical procedure to solve the inverse problem. We use the spring model : d 2 y(t) dt 2 + C dy(t) dt + Ky(t) = 0. We have observations : (t 1, y 1 ),..., (t m, y m ). The goal is to estimate the unknown parameters C and K.

5 Introduction Estimation problem We discuss the statistical procedure to solve the inverse problem. We use the spring model : d 2 y(t) dt 2 + C dy(t) dt + Ky(t) = 0. We have observations : (t 1, y 1 ),..., (t m, y m ). The goal is to estimate the unknown parameters C and K.

6 Introduction Estimation problem We discuss the statistical procedure to solve the inverse problem. We use the spring model : d 2 y(t) dt 2 + C dy(t) dt + Ky(t) = 0. We have observations : (t 1, y 1 ),..., (t m, y m ). The goal is to estimate the unknown parameters C and K.

7 Introduction Differential equation For any C, K, the differential equation has a solution given initial value y(0) = y 0. We call that particular solution y(t; C, K). Notice the dependence on C and K! For fixed C, K it is just a function of t, as we would have liked.

8 Introduction Differential equation For any C, K, the differential equation has a solution given initial value y(0) = y 0. We call that particular solution y(t; C, K). Notice the dependence on C and K! For fixed C, K it is just a function of t, as we would have liked.

9 Introduction Cost function : least squares Fix C and K. We define the cost function L(C, K) = m (y i y(t i ; C, K)) 2 i=1 We seek to minimize this cost function L(C, K) over all possible values of C and K. The least squares estimator : Ĉ, ˆK.

10 Introduction Cost function : least squares Fix C and K. We define the cost function L(C, K) = m (y i y(t i ; C, K)) 2 i=1 We seek to minimize this cost function L(C, K) over all possible values of C and K. The least squares estimator : Ĉ, ˆK.

11 Introduction Non-linear regression This is a non-linear regression model. Assumption : we have equal variance measurement errors. Model y i = y(t i ; C, K) + ɛ i ɛ i are independent, identically distributed Normal errors with mean zero and variance σ 2. This is non-linear, since the solution to the ODE is a function involving exponentials and trigonometric forms.

12 Introduction Non-linear regression This is a non-linear regression model. Assumption : we have equal variance measurement errors. Model y i = y(t i ; C, K) + ɛ i ɛ i are independent, identically distributed Normal errors with mean zero and variance σ 2. This is non-linear, since the solution to the ODE is a function involving exponentials and trigonometric forms.

13 Introduction Linear and non-linear regression Examples : Linear model y = β 0 + β 1 x + β 2 x 2 y = β 0 + β 1 cos(x) + β 2 sin(x) + β 3 exp( x) Example : non-linear model y = exp( β 1 x) + α cos(β 2 x) The non-linearity is in terms of the parameters, not the other variables.

14 Introduction Linear and non-linear regression Examples : Linear model y = β 0 + β 1 x + β 2 x 2 y = β 0 + β 1 cos(x) + β 2 sin(x) + β 3 exp( x) Example : non-linear model y = exp( β 1 x) + α cos(β 2 x) The non-linearity is in terms of the parameters, not the other variables.

15 Outline 1 Introduction 2 Preliminary analysis Estimation of parameters Residual analysis

16 Preliminary analysis MATLAB commands I saved all my MATLAB commands in the file costminanalysis.m

17 Preliminary analysis Preliminary analysis Extracting the data from the.mat file U = load( data1 ) time = U.trace_x ; disp = U.trace_y(4,:); N = length(time); Plotting the data plot(time, disp); xlabel( t ); ylabel( y(t) );

18 Preliminary analysis Preliminary analysis Extracting the data from the.mat file U = load( data1 ) time = U.trace_x ; disp = U.trace_y(4,:); N = length(time); Plotting the data plot(time, disp); xlabel( t ); ylabel( y(t) );

19 Preliminary analysis Plot of y(t) vs t 8 x y(t) t

20 Preliminary analysis Cleaning the data for our purpose Observe that the data has some pattern at the beginning, that we will truncate. We will start at the point where we have maximum amplitude and the data dampens out gradually. Also, the variable disp is in row vector form. We change it to column vector by transposing. disp = disp ; [maxd,index] = max(disp); time = time(index:n)-time(index); disp = disp(index:n);

21 Preliminary analysis Cleaning the data for our purpose Observe that the data has some pattern at the beginning, that we will truncate. We will start at the point where we have maximum amplitude and the data dampens out gradually. Also, the variable disp is in row vector form. We change it to column vector by transposing. disp = disp ; [maxd,index] = max(disp); time = time(index:n)-time(index); disp = disp(index:n);

22 Preliminary analysis Centering the data We also just center the data so that the mean is 0, which is necessary for the analysis. disp = disp - mean(disp); We plot the new data again.

23 Preliminary analysis Plot of y(t) 8 x y(t) t

24 Estimation of parameters Solving a differential equation We write ẏ = dy dt, and ÿ = d2 y. So, the ODE of dt 2 spring-mass-dashpot becomes ÿ + Cẏ + Ky = 0 Or alternately, we write in a vector form, i.e. ( ) ( ) d y1 y2 = dt y 2 Ky 1 Cy 2

25 Estimation of parameters Solving a differential equation We write ẏ = dy dt, and ÿ = d2 y. So, the ODE of dt 2 spring-mass-dashpot becomes ÿ + Cẏ + Ky = 0 Or alternately, we write in a vector form, i.e. ( ) ( ) d y1 y2 = dt y 2 Ky 1 Cy 2

26 Estimation of parameters MATLAB ODE solver Here we find the function y(t; C, K) which solves the equation ÿ + Cẏ + Ky = 0 under the initial condition y(0) = y 0. The routine ode23 solves the equation system z = ode_model(t,z) when we call it. First, we code the ODE system in MATLAB function dy = ode_model(t,y,c,k) dy = zeros(2,1); dy(1) = y(2); dy(2) = -K*y(1) - C*y(2) Save this file in the working directory as ode_model.m.

27 Estimation of parameters MATLAB ODE solver Here we find the function y(t; C, K) which solves the equation ÿ + Cẏ + Ky = 0 under the initial condition y(0) = y 0. The routine ode23 solves the equation system z = ode_model(t,z) when we call it. First, we code the ODE system in MATLAB function dy = ode_model(t,y,c,k) dy = zeros(2,1); dy(1) = y(2); dy(2) = -K*y(1) - C*y(2) Save this file in the working directory as ode_model.m.

28 Estimation of parameters Cost function of the vibrating beam problem Next, we solve the ODE using the MATLAB routine, using the initial values, and also calculate the cost L(C, K) as defined before. The following function takes as an input the parameter values C and K with the data, and computes the cost L(C, K) = m (y i y(t i ; C, K)) 2 i=1 as defined before.

29 Estimation of parameters The MATLAB function to compute L(C, K) First we define a function to compute L(C, K). function [cost,d_model]=cost_beam(ck,time,disp) x0 = [disp(1) 0]; C = CK(1); K = CK(2); [t,x] = ode23(@ode_model, time, x0, [], C, K); d_model = x(:,1); cost = sum((d_model-disp).^2); Save this file as cost_beam.m

30 Estimation of parameters Plot of the function L(C, K) Let us plot the function over a wide range of values for C and K. C0 = [0:.05:2]; K0 = [1000:2:2000]; LCK = zeros(length(c0), length(k0)); for i = 1:length(C0) for j = 1:length(K0) CK0 = [C0(i);K0(j)]; [c,d] = cost_beam(ck0,time,disp); LCK(i,j) = c; end end surf(c0,k0, LCK ) xlabel( C ) ylabel( K )

31 Estimation of parameters Plot of L(C, K) 1.2 x K C

32 Estimation of parameters Optimization routine We use the MATLAB routine to minimize the cost function L(C, K) over a wide range of values of C and K. We have to give some initialization values. C0 = 1; K0 = 1500; CK0 = [C0;K0]; [CK,cost]=fminsearch(@cost_beam,CK0,[],time,disp C = CK(1) K = CK(2) Finally, we get the estimators Ĉ and ˆK. Ĉ =.9052, ˆK =

33 Estimation of parameters Optimization routine We use the MATLAB routine to minimize the cost function L(C, K) over a wide range of values of C and K. We have to give some initialization values. C0 = 1; K0 = 1500; CK0 = [C0;K0]; [CK,cost]=fminsearch(@cost_beam,CK0,[],time,disp C = CK(1) K = CK(2) Finally, we get the estimators Ĉ and ˆK. Ĉ =.9052, ˆK =

34 Estimation of parameters Optimization routine We use the MATLAB routine to minimize the cost function L(C, K) over a wide range of values of C and K. We have to give some initialization values. C0 = 1; K0 = 1500; CK0 = [C0;K0]; [CK,cost]=fminsearch(@cost_beam,CK0,[],time,disp C = CK(1) K = CK(2) Finally, we get the estimators Ĉ and ˆK. Ĉ =.9052, ˆK =

35 Estimation of parameters Estimation of σ 2 Recall that we have the model y i = y(t i ; C, K) + ɛ i ɛ i are independent, identically distributed Normal errors with mean zero and variance σ 2. An estimate of σ 2 is 1 m 2 m (y i y(t i, Ĉ, ˆK)) 2 i=1

36 Estimation of parameters Estimation of σ 2 Recall that we have the model y i = y(t i ; C, K) + ɛ i ɛ i are independent, identically distributed Normal errors with mean zero and variance σ 2. An estimate of σ 2 is 1 m 2 m (y i y(t i, Ĉ, ˆK)) 2 i=1

37 Estimation of parameters To get the residuals The MATLAB code is [c, d_model] =cost_beam(ck, time,disp); d_res = disp - d_model; m = length(disp); sigma2 = sum(d_res.^2)/m; We save the residuals for exploratory analysis. We estimate ˆσ = from the data

38 Residual analysis Plotting the residuals We start plotting the residuals to see whether they have any pattern. Residual vs Time plot(time,d_res) xlabel( time ) ylabel( residuals )

39 Residual analysis Residuals against the time 8 x residuals time The residuals show some pattern, and the independence assumption seems tenuous!!

40 Residual analysis Fitted vs residuals Next, we check for the homoscedasticity assumption : that is, whether the residuals have the same variance or not. Residual vs Fitted plot(d_model,d_res, k. ) xlabel( fitted ) ylabel( residuals )

41 Residual analysis Checking homoscedasticity 8 x residuals fitted x 10 5 Here also, some pattern, though less evident, can be observed.

42 Residual analysis Quantile-quantile plot Finally, we check the normality assumptions. We use quantile-quantile plot for that. We plot the sorted residuals against the quantiles for the normal distribution. We expect the points stay close to a straight line. qqplot(d_res) ylabel( residuals )

43 Residual analysis Q-Q plot 8 x 10 5 QQ Plot of Sample Data versus Standard Normal residuals Standard Normal Quantiles The assumption of normality is not that strong too!!!

44 Residual analysis Improvement for the model? The spring model does not seem so appropriate for the data. Residual shows dependence structure. Variances may not be equal. Normality assumption may not hold. We may have the same underlying model, but with different assumptions for the error structure. Or we may have an altogether different model. Is there a possible improvement on the physical spring model we use here?

45 Residual analysis Improvement for the model? The spring model does not seem so appropriate for the data. Residual shows dependence structure. Variances may not be equal. Normality assumption may not hold. We may have the same underlying model, but with different assumptions for the error structure. Or we may have an altogether different model. Is there a possible improvement on the physical spring model we use here?

46 Residual analysis Improvement for the model? The spring model does not seem so appropriate for the data. Residual shows dependence structure. Variances may not be equal. Normality assumption may not hold. We may have the same underlying model, but with different assumptions for the error structure. Or we may have an altogether different model. Is there a possible improvement on the physical spring model we use here?