Derivative Approximation by Finite Differences

Size: px

Start display at page:

Download "Derivative Approximation by Finite Differences"

Joanna Hunt
6 years ago
Views:

1 Derivative Approximation by Finite Differences David Eberly, Geometric Tools, Redmond WA This work is licensed under the Creative Commons Attribution 4 International License To view a copy of this license, visit or send a letter to Creative Commons, PO Box 866, Mountain View, CA 9442, USA Created: May 3, 2 Last Modified: September 2, 26 Contents Introduction 2 2 Derivatives of Univariate Functions 2 3 Derivatives of Bivariate Functions 7 4 Derivatives of Multivariate Functions 8

2 Introduction This document shows how to approximate derivatives of univariate functions F (x by finite differences Given a small value h >, the d-th order derivative satisfies the following equation where the integer order of error p > may be selected as desired, h d d! F (d (x i max ii min C i F (x + ih + O(h d+p ( for some choice of extreme indices i min and i max and for some choice of coefficients C i The equation becomes an approximation by throwing away the O(h d+p term The vector C (C imin,, C imax is called the template or convolution mask for the approximation Approximations for the derivatives of multivariate functions are constructed as tensor products of templates for univariate functions 2 Derivatives of Univariate Functions Recall from calculus that the following approximations are valid for the derivative of F (x A forward difference approximation is F F (x + h F (x (x + O(h (2 h a backward difference approximation is F (x and a centered difference approximation is F (x F (x F (x h h F (x + h F (x h + O(h (3 + O(h 2 (4 The approximations are obtained by throwing away the error terms indicated by the O notation The order of the error for each of these approximations is easily seen from formal expansions as Taylor series about the value x, and F (x + h F (x + hf (x + h2 2! F (x + F (x h F (x hf (x + h2 2! F (x + n n! F (n (x (5 ( n hn n! F (n (x (6 where F (n (x denotes the n-th order derivative of F Subtracting F (x from both sides of equation (5 and then dividing by h leads to the forward difference F (x (F (x+h F (x/h+o(h Subtracting F (x from both sides of equation (6 and then dividing by h leads to the backward difference F (x (F (x F (x h/h + O(h Both approximations have error O(h The centered difference is obtained by subtracting equation (6 from equation (5 and then dividing by to obtain (F (x + h F (x h/( + O(h 2 n 2

3 Higher order approximations to the first derivative can be obtained by using more Taylor series, more terms in the Taylor series, and appropriately weighting the various expansions in a sum For example, ( n F (x + F (n (x, F (x ( n (n F (n (x (7 n! n! n lead to a forward difference approximation with second order error, F (x n F (x + + 4F (x + h 3F (x to a backward difference approximation with second order error, F (x 3F (x 4F (x h + F (x and to a centered difference approximation with fourth order error, F (x F (x + + 8F (x + h 8F (x h + F (x + O(h 2 (8 + O(h 2 (9 + O(h 4 ( Higher-order derivatives can be approximated in the same way For example, a forward difference approximation to F (x is F F (x + 2F (x + h + F (x (x h 2 + O(h ( and centered difference approximations are and F (x F (x F (x + h 2F (x + F (x h h 2 + O(h 2 (2 F (x + + 6F (x + h 3F (x + 6F (x h F (x 2 + O(h 4 (3 Each of these formulas is easily verified by expanding the F (x + ih terms in a formal Taylor series and computing the weighted sums on the right-hand sides However, of greater interest is to select the order of derivative d and the order of error p and determine the weights C i for the sum in equation ( A formal Taylor series for F (x + ih is Replacing this in equation ( yields F (x + ih n i n hn n! F (n (x (4 h d d! F (d (x i max ii min C i n ( imax n ii min ( d+p imax n ii min d+p n in hn n! F (n (x + O(h d+p n! F (n (x + O(h d+p ( imax ii min n! F (n (x n! F (n (x + O(h d+p (5 Multiplying by d!/h d, the desired approximation is ( F (d (x d! d+p imax h d n ii min n! F (n (x + O(h p d! h d 3 d+p n ( imax ii min n! F (n (x (6

4 Treating the approximation in equation(5 as an equality, the only term in the sum on the right-hand side of the approximation that contains (h d /d!f (d (x occurs when n d, so the coefficient of that term must be The other terms must vanish for there to be equality, so the coefficients of those terms must be ; therefore, it is necessary that i max ii min, n d + p and n d, n d This is a set of d+p linear equations in i max i min + unknowns If we constrain the number of unknowns to be d+p, the linear system has a unique solution A forward difference approximation occurs if we set i min and i max d + p A backward difference approximation occurs if we set i max and i min (d + p A centered difference approximation occurs if we set i max i min (d + p /2 where it appears that d + p is necessarily an odd number As it turns out, p can be chosen to be even regardless of the parity of d and i max (d + p /2 Table indicates the choices for d and p, the type of approximation (forward, backward, centered, and the corresponding equation number, (7 Table Choices for the parameters for the example approximations presented previously equation d p type i min i max (2 forward (3 backward - (4 2 centered - (8 2 forward 2 (9 2 backward -2 ( 4 centered -2 2 ( 2 forward 2 (2 2 2 centered - (3 2 4 centered -2 2 Example Approximate F (3 (x with a forward difference with error O(h, so d 3 and p We need i min and i max 3 The linear system from equation (7 is C 2 3 C 4 9 C C 3 4

5 and has solution (C, C,, C 3 (, 3, 3, /6 Equation ( becomes F (3 (x F (x + 3F (x + h 3F (x + + F (x + 3h h 3 + O(h Approximate F (3 (x with a centered difference with error O(h 2, so d 3 and p 2 We need i max i min 2 The linear system from equation (7 is C 2 C C C and has solution (C 2, C, C, C, (, 2,, 2, /2 Equation ( becomes F (3 (x F (x + 2F (x h 2F (x + h + F (x O(h 2 Finally, approximate with a centered difference with error O(h 4, so d 3 and p 4 We need i max i min 3 The linear system from equation (7 is C 3 C 2 C C C C 3 and has solution (C 3, C 2, C, C, C,, C 3 (, 8, 3,, 3, 8, /48 Equation ( becomes F (3 (x F (x 3h 8F (x + 3F (x h 3F (x + h + 8F (x + F (x + 3h 8h 3 + O(h 4 Example 2 Approximate F (4 (x with a forward difference with error O(h, so d 4 and p We need 5

6 i min and i max 4 The linear system from equation (7 is C C C 3 C 4 and has solution (C, C,, C 3, C 4 (, 4, 6, 4, /24 Equation ( becomes F (4 (x F (x 4F (x + h + 6F (x + 4F (x + 3h + F (x + 4h h 4 + O(h Approximate F (4 (x with a centered difference with error O(h 2, so d 4 and p 2 We need i max i min 2 The linear system from equation (7 is C 2 C C C and has solution (C 2, C, C, C, (, 4, 6, 4, /24 Equation ( becomes F (4 (x F (x 4F (x h + 6F (x 4F (x + h + F (x + h 4 + O(h 2 Finally, approximate with a centered difference with error O(h 4, so d 4 and p 4 We need i max i min 3 The linear system from equation (7 is C 3 C 2 C C C C 3 and has solution (C 3, C 2, C, C, C,, C 3 (, 2, 39, 56, 39, 2, /44 Equation ( becomes F (4 (x F (x 3h + 2F (x 39F (x h + 56F (x 39F (x + h + 2F (x + F (x + 3h 6h 4 + O(h 4 6

7 3 Derivatives of Bivariate Functions For functions with more variables, the partial derivatives can be approximated by grouping together all of the same variables and applying the univariate approximation for that group For example, if F (x, y is our function, then some partial derivative approximations are f x (x, y f y (x, y f xx (x, y f yy (x, y f xy (x, y F (x+h,y F (x h,y F (x,y+k F (x,y k 2k F (x+h,y 2F (x,y+f (x h,y h 2 F (x,y+k 2F (x,y+f (x,y k k 2 F (x+h,y+k F (x+h,y k F (x h,y+k+f (x h,y k 4hk (8 Each of these can be verified in the limit, the x-derivatives by taking the limit as h approaches zero, the y-derivatives by taking the limit as k approaches zero, and the mixed second-order derivative by taking the limit as both h and k approach zero The derivatives F x, F y, F xx, and F yy just use the univariate approximation formulas The mixed derivative requires slightly more work The important observation is that the approximation for F xy is obtained by applying the x-derivative approximation for F x, then applying the y-derivative approximation to the previous approximation That is, f xy (x, y F (x+h,y F (x h,y F (x+h,y+k F (x h,y+k F (x+h,y k F (x h,y k 2k F (x+h,y+k F (x+h,y k F (x h,y+k+f (x h,y k 4hk (9 The approximation implied by equation ( may be written as h m m! d m dx m F (x i max ii min C (m i F (x + ih (2 The inclusion of the superscript on the C coefficients is to emphasize that those coefficients are constructed for each order m For bivariate functions, we can use the natural extension of equation (2 by applying the approximation in x first, then applying the approximation in y to that approximation, just as in our example of F xy, k n n! n h m y n m! m x m F (x, y kn n! n y n imax ii min C (m i F (x + ih, y i max ii min jmax i max ii min jmax jj min C (m i jj min C (m,n i,j C (n j F (x + ih, y + jk F (x + ih, y + jk (2 where the last equality defines C (m,n i,j C (m i C (n j (22 The coefficients for the bivariate approximation are just the tensor product of the coefficients for each of the univariate approximations 7

8 4 Derivatives of Multivariate Functions The approximation concept extends to any number of variables Let (x,, x n be those variables and let F (x,, x n be the function to approximate The approximation is ( h m m! where m x m hmn n m n! mn F (x x mn,, x n i max i i min C (m,,mn i max n i ni min n (i,,i n C (m i is a tensor product of the coefficients of the n univariate approximations C (m,,mn (i,,i n F (x + i h,, x n + i n (23 C (mn i n (24 8

The Intermediate Value Theorem states that if a function g is continuous, then for any number M satisfying. g(x 1 ) M g(x 2 )

The Intermediate Value Theorem states that if a function g is continuous, then for any number M satisfying. g(x 1 ) M g(x 2 ) APPM/MATH 450 Problem Set 5 s This assignment is due by 4pm on Friday, October 25th. You may either turn it in to me in class or in the box outside my office door (ECOT 235). Minimal credit will be given