Golden-Section Search for Optimization in One Dimension

Transcription

1 Golden-Section Search for Optimization in One Dimension Golden-section search for maximization (or minimization) is similar to the bisection method for root finding. That is, it does not use the derivatives (like, for example, Newton s method) to find the optimization. Similar to bisection, it is an iterative technique that starts with a search range and narrows down the range until an approximate optimum value is found. The idea behind the golden-section method is based on the relative values of F (x) for different values x to narrow down the search space. Compare this with the bisection method, where the idea was to use the sign change narrow down the search space. Assume that the function is unimodal (only one peak at x ) in the interval x l to x u (see Figure 1). The range x l to x u will be used as the initial search range. x l x u Figure 1: Initial search range in golden-section The steps of the golden-section method are as follows. 1. Select x l to x u for the initial search range.. Select two intermediate values x 1 and x as x 1 = x l + d (1) x = x u d () The values of x 1 and x are such that x 1 >x. The ideal (golden) value of d is given by (justification is given later) (x u x l ) (3) 3. Evaluate F (x 1 )andf (x ) and based on their relative values, update (narrow down) the search range. If F (x 1 ) >F(x ), we have two possibilities as shown in Figure. As can be seen from Figure, the only conclusion we can make when x 1 > x and F (x 1 ) >F(x ) is that x >x. That is, for the next iteration, the lower limit on the search range can be updated as l = x. Similarly, if If F (x ) >F(x 1 ), we have two possibilities as shown in Figure 3. 1

2 OR x x1 x x 1 Figure : Possibilities when x 1 >x and F (x 1 ) >F(x ) OR x x 1 x x 1 Figure 3: Possibilities when x 1 >x but F (x ) >F(x 1 ) In this case, we can only conclude that x <x 1. That is, u = x 1 for the new iteration. To summarize, update the values of x l and x u, based on the relative values of F (x 1 )and F (x ), as follows: If F (x 1 ) >F(x ) (case 1), update the search ranges as If F (x ) >F(x 1 ) (case ), update the search ranges as l = x (4) u = x u (no change) (5) l = x l (no change) (6) u = x 1 (7) 4. Continue with the new values of x l and x u until the the optimum point is found (terminate when the optimum point or the optimum value does not change significantly from one iteration to the next). Note that at the end of each iteration, the optimum value is either x 1 or x (depending on the value of F (x 1 )andf (x )). The choice of d Anyvaluecanbeusedford as long as x 1 >x. However, if d is selected appropriately, the number of function evaluations (for different values x 1 and x ) can be minimized. As we saw earlier, either x 1 (case ) or x (case 1) in one iteration becomes the new x u or x l for the next iteration. If we chose the values of x 1 and x such that the unused value (x 1 in case 1 and x in case ) is also used in the next iteration, the number of function evaluations can be minimized. To achieve this, d should selected such that = x 1 in case 1 or 1 = x in case. See an illustration of this in Figure 4. From the figure, we have l 0 = + l (8)

3 Figure 4: The selection of d to minimize the number of function evaluations To minimize the number of function evaluations, we need Combining these two Writing l l1 = r, l 0 = l (9) + l = l () 1+r = 1 r (11) That is, r + r 1 = 0 (1) the roots of which are given by 1 4( 1) r = 1 ± = = (13) which explains the choice of d given in (3). The algorithm can be terminated when x u x l (1 r) x opt 0% <ɛ t (14) where x opt is either x 1 or x depending on which one gives the optimum value for the function and ɛ t is the desired threshold (e.g., 1%). 3

4 The value of d is chosen to minimize the number of function evaluations (which can be expensive in practical problems). However, note that after each iteration the search range is reduced by a factor of (compare this with the.5 reduction in bisection). Thus, the golden-section method converges rather slowly. Example: Find the maximum of the function f(x) =sinx x with x l = 0 and x u = 4 as the starting search range. Iteration 1: (x u x l )= (4 0) =.47 (15) x 1 = x l =.47 (16) x = x u 4.47 = 1.58 (17) f(x ) = f(1.58) = 1.58 sin(1.58) = (18) f(x 1 ) = f(.47) =.47 sin(.47) = 0.63 (19) Since f(x ) >f(x 1 ), the values of x l and x u are updated as x u = x 1 =.47 (0) x l = x l = 0 (1) At the end of iteration 1, the optimum value of x is x =1.58 (since f(x ) >f(x 1 )). Iteration : (x u x l )= (.47 0) = 1.58 () x 1 = x l = 1.58 (3) x = x u = (4) f(x ) = f(0.944) = sin(0.944) = (5) f(x 1 ) = f(1.58) = (no need to evaluate again) (6) Note that the value of x 1 in the second iteration is the same as x in the first iteration. In this case, there is no need to evaluate f(x 1 ) in the second iteration (the value of f(x )canbe stored from iteration 1) Since f(x ) <f(x 1 ), the values of x l and x u are updated as x l = x =0.944 (7) x u = x u =.47 (8) 4

5 Also note that the range of x l to x u has been shortened. optimum value of x is x 1 =1.58 (since f(x ) <f(x 1 )). At the end of iteration, the This can be continued until the optimum value is found subject to the convergence condition given earlier. Figure 5 shows the progress of the algorithm in subsequent iterations. At the end of iteration 8, the optimum value of x is given by x =1.447 and the maximum value of the function is f(x) = Resources: Figure 5: Progress of the Golden-section method 1. Steven Chapra and Raymond Canale, Numerical Methods for Engineers, Fourth Edition, McGraw-Hill, 00 (see Section 13.1, 13.3)