lecture 31: The Secant Method: Prototypical Quasi-Newton Method

Transcription

1 169 lecture 31: The Secant Method: Prototypical Quasi-Newton Method Newton s method is fast if one has a good initial guess x 0 Even then, it can be inconvenient and expensive to compute the derivatives f 0 (x ) at each iteration The final root finding algorithm we consider is the secant method, a ind of quasi-newton method based on an approximation of f 0 It can be thought of as a hybrid between Newton s method and regula falsi 44 The Secant Method Throughout this semester, we have seen how derivatives can be approximated using finite differences, for example, f 0 (x) f (x + h) f h (x) for some small h (Recall that too-small h will give a bogus answer due to rounding errors, so some caution is needed; see Section 36) What if we replace f 0 (x ) in Newton s method with this sort of approximation? The natural algorithm that emerges is the secant method, x +1 = x f (x ) x x 1 f (x ) f (x 1 ) = x 1 f (x ) x f (x 1 ) f (x ) f (x 1 ) 0 x 0 x 1 x Note the similarity between this formula and the regula falsi iteration: c = a f (b ) b f (a ) f (b ) f (a ) f Both methods approximate f by a line that joins two points on the graph of f, but the secant method requires no initial bracet for the root Instead, the user simply provides two starting points x 0 and x 1 with no stipulation about the signs of f (x 0 ) and f (x 1 ) As a consequence, there is no guarantee that the method will converge: as in Newton s method, a poor initial guess can lead to divergence Do we recover the convergence behavior of Newton s method? Not quite but it is still better than the linear convergence exhibited by the bisection and regula falsi methods (provided it does converge) The convergence theory brings in a new technique we have not see before, where the error e = x x is presumed to reduce according to a generic convergence behavior, e +1 Me r, and this ansatz is used to derive values of M and r Begin by writing the linear interpolant to f at x and x 1 in the Newton form See Dahlquist and Björc, Numerical Methods, Section 643 (47) p(x) = f (x )+ x x x 1 x f (x 1 ) f (x )

2 170 Once again we can put the interpolation error formula (Theorem 13) to good use Assuming that f C (IR), for any x IR one can write f (x) p(x) = (x x )(x x 1 ), where x falls within the extremes of x, x, and x 1 Since f (x )=0, we can thus write 0 = p(x )+ (x x )(x x 1 ) Defining e j := x j x as usual, this last equation is 0 = p(x )+ Substituting formula (47) for p gives e e 1 (48) 0 = f (x )+ x x f (x x 1 x 1 ) f (x ) + e e 1 Now recall that, by design, the secant method pics x +1 as the zero of p, ie, 0 = p(x +1 ) (49) = f (x )+ x +1 x x 1 x f (x 1 ) f (x ) Subtract (48) from (49) to obtain (410) 0 = x +1 x x 1 x f (x 1 ) f (x ) f (x = e 1 ) f (x ) +1 e x 1 x e 1 We can clean up this last formula by realizing that e e 1 p f f (x 1 ) f (x ) x 1 x is the slope of the linear interpolant p It is also a difference quotient for f, and so, by the Mean Value Theorem, there exists h [x, x 1 ] where the slope of f matches that of the interpolant: 15 x 0 h x 1 f 0 (h) = f (x 1) f (x ) x 1 x Substituting this formula into (410) gives 0 = f 0 (h)e +1 e e 1,

3 171 which can be solved for (411) e +1 = f 0 (h) e e 1 As x and x 1 converge to x, the values for x and h must also converge toward x, justifying the approximation Contrast this formula with the error recurrence for Newton s method in (47): e +1 = f 0 (x ) e (41) e +1 Ce e 1 for asymptotic error constant C = f 00 (x ) f 0 (x ), presuming, as usual, that x is a simple root, so f 0 (x ) 6= 0 It remains to convert the approximation (41) into some order of convergence As x converges, we expect e 1 to be larger than e, so e e 1 in the secant method is probably not as small as the e term that appeared in the analogous formula for Newton s method Can we quantify how much smaller? Suppose the error obeys the approximation (413) e j+1 Me r j for some constants M and r Then implies that e Me r 1 e 1 M 1/r e 1/r, and so the error approximation (41) suggests (414) e +1 Ce e 1 CM 1/r e 1+1/r This equation must agree with the the error form (413) with j = : (415) e +1 Me r Equating the approximations (414) and (415) gives Matching the constants then gives CM 1/r e 1+1/r = Me r M = C r/(r+1), while matching the rates gives 1 + 1/r = r, ie, r r 1 = 0

4 17 Solve this quadratic to get r = 1 ± p 5 p The negative sign choice gives r = (1 5)/ = 06180, which does not correspond to a convergent process (If e < 1, then e > 1) Thus, if the process converges according to e +1 apple Me r, the r must correspond to the positive root, r = 1 + p 5 the golden ratio It follows that := j = 16180, e +1 applem e j for a constant M > 0 Note that j <, so, in the region of asymptotic convergence (x close to x ), one step of the secant method will mae a bit less progress to the root than one step of Newton s method Though you may regret that the secant method does not recover the quadratic convergence of Newton s method, tae solace in the fact that the secant method requires only one function evaluation f (x ) at each iteration, as opposed to Newton s method, which requires f (x ) and f 0 (x ) Typically the derivative is more expensive to compute than the function itself Assuming that evaluating f (x ) and f 0 (x ) requires the same amount of effort, then we can compute two steps of the secant method for roughly the same cost as a one step of Newton s method These two steps of the secant method combine to give an improved convergence rate: Of course, for the secant method one stores the f (x 1 ) value computed during the previous iteration e + apple M e +1 j apple M M e j j apple M 1+j e j, where j = 1 (3 + p 5) 6 > Hence, in terms of computing time, the secant method can actually be more efficient than Newton s method Figure 45 compares the convergence of the secant method to Newton s method for the function f (x) =x, which we can use to compute x = p as in Figure 45 This example starts with the (bad) initial guess x 0 = 10 To ensure that the secant method is not hampered by a bad value of x 1, this experiments uses the same x 1 value computed using Newton s method After these two initial steps, both methods steadily converge, but Newton s method taes fewer iterations, in agreement with the theory derived in this and the last lecture Table 4 shows the iterates x and magnitude of the errors e for both methods This discussion is drawn from 33 of Kincaid and Cheney, Numerical Analysis, 3rd ed

5 173 error, x x Newton s method secant method Figure 45: The convergence of Newton s method and the secant method for f (x) =x Both methods start with x 0 = 1, and the secant method uses the same x 1 that was generated by the first step of Newton s method x (Newton) e (Newton) x (secant) e (secant) Table 4: Comparison of iterates from Newton s method and the secant method for finding a zero of f (x) = x