Modelling Anti-Terrorist Surveillance Systems from a Queueing Perspective

Transcription

1 Systems from a Queueing Perspective September 7, 2012

2 Problem A surveillance resource must observe several areas, searching for potential adversaries.

3 Problem A surveillance resource must observe several areas, searching for potential adversaries. An adversary wishes to enter one of these areas, complete his objective, and then escape unnoticed.

4 Problem A surveillance resource must observe several areas, searching for potential adversaries. An adversary wishes to enter one of these areas, complete his objective, and then escape unnoticed. Our aim is to find policies for the surveillance resource that maximise its chance of catching the adversary

5 Problem A surveillance resource must observe several areas, searching for potential adversaries. An adversary wishes to enter one of these areas, complete his objective, and then escape unnoticed. Our aim is to find policies for the surveillance resource that maximise its chance of catching the adversary

6 Problem

7 Motivation A linear program can be formulated to find the optimal policy for a two area system.

8 Motivation A linear program can be formulated to find the optimal policy for a two area system. This is impractical for larger numbers of areas, as the problem quickly becomes very computationally demanding.

9 Motivation A linear program can be formulated to find the optimal policy for a two area system. This is impractical for larger numbers of areas, as the problem quickly becomes very computationally demanding. We aim to construct heuristic methods that can be applied to systems with many areas by trialling them on a two area system.

10 Motivation A linear program can be formulated to find the optimal policy for a two area system. This is impractical for larger numbers of areas, as the problem quickly becomes very computationally demanding. We aim to construct heuristic methods that can be applied to systems with many areas by trialling them on a two area system. We can improve our policies by forming a two person zero sum game, allowing the surveillance resorce to employ a mixture of successful strategies, with the payoffs given by the probability of catching the adversary under that policy. The value of the game shows the success of the policies.

11 Queues

12 Queues We treat each of these areas as a separate queue.

13 Queues We treat each of these areas as a separate queue. At any time, one of three things can happen:

14 Queues We treat each of these areas as a separate queue. At any time, one of three things can happen: A customer joins the queue, representing someone entering the area

15 Queues We treat each of these areas as a separate queue. At any time, one of three things can happen: A customer joins the queue, representing someone entering the area A customer is served at the front of the queue, representing the surveillance resource, or server, observing someone

16 Queues We treat each of these areas as a separate queue. At any time, one of three things can happen: A customer joins the queue, representing someone entering the area A customer is served at the front of the queue, representing the surveillance resource, or server, observing someone A customer abandons the queue, representing someone leaving the area

17 Queues We treat each of these areas as a separate queue. At any time, one of three things can happen: A customer joins the queue, representing someone entering the area A customer is served at the front of the queue, representing the surveillance resource, or server, observing someone A customer abandons the queue, representing someone leaving the area We assume the times between these actions occuring are distributed exponentially, with rate parameters λ, µ and θ respectively.

18 Queues cont.

19 Queues cont. As all events are distributed exponentially, these queues are continuous time Markov Chains, the state of which is given by the number of customers in the queue.

20 Steady State

21 Steady State To find the optimal strategy for the server, we must calculate the adversary s options.

22 Steady State To find the optimal strategy for the server, we must calculate the adversary s options. We assume that the adversary does not arrive atypically early, so the adversary sees the queues in their steady states.

23 Steady State To find the optimal strategy for the server, we must calculate the adversary s options. We assume that the adversary does not arrive atypically early, so the adversary sees the queues in their steady states. We calculate the steady state distribution for a single queue with constant service rate by assuming the Markov Chain is in stochastic balance, and solving the difference equation: (µ + θ(n + 1))π n+1 + λπ n 1 = (λ + µ + nθ)π n (µ + θ)π 1 = λπ 0 π n = 1 n 0

24 Steady State

25 Steady State This gives us π 0 (λ, µ, θ) = ) 1 (µ + mθ) ( n λ n n=0 m=1 ( k π k (λ, µ, θ) = λ k π 0 (λ, µ, θ) (µ + mθ) m=1 1 ) 1

26 Completion Rates

27 Completion Rates We measure the performance of a policy by the completion rates in the queues.

28 Completion Rates We measure the performance of a policy by the completion rates in the queues. CRi = µ i(1 π (i) 0 ) λ i

29 Completion Rates We measure the performance of a policy by the completion rates in the queues. CRi = µ i(1 π (i) 0 ) λ i This gives us the probability of the server catching the adversary in that queue, before the adversary abandons.

30 Assumptions The server and the adversary know the value of the parameters

31 Assumptions The server and the adversary know the value of the parameters The adversary behaves like a regular customer

32 Assumptions The server and the adversary know the value of the parameters The adversary behaves like a regular customer The server and adversary know the value of the completion rates under a given service policy

33 Assumptions The server and the adversary know the value of the parameters The adversary behaves like a regular customer The server and adversary know the value of the completion rates under a given service policy A customer cannot be served twice

34 Model

35 Model The server picks one queue and ignores the other.

36 Model The server picks one queue and ignores the other. This gives game matrix Server Policy Q1 Q2 Adversary Q1 CR1 0 Policy Q2 0 CR2

37 We solve this game by finding a strategy that gives the best worse-case scenario: ( ) CR2 x = CR1 + CR2, CR1 CR1 + CR2

38 We solve this game by finding a strategy that gives the best worse-case scenario: ( ) CR2 x = CR1 + CR2, CR1 CR1 + CR2 The value of the game is v = (CR1)(CR2) CR1 + CR2

39 We solve this game by finding a strategy that gives the best worse-case scenario: ( ) CR2 x = CR1 + CR2, CR1 CR1 + CR2 The value of the game is v = (CR1)(CR2) CR1 + CR2 Using λ 1 = 3 µ 1 = 6 θ 1 = 5 λ 2 = 6 µ 2 = 3 θ 2 = 2 as an illustrative set of parameters, we get x = (0.493, 0.561) v =

40 Two Queues Probabilistic Service We allow the server to serve both queues, choosing to serve queue 1 with probability p. Thus the service rate is pµ

41 Two Queues Probabilistic Service We allow the server to serve both queues, choosing to serve queue 1 with probability p. Thus the service rate is pµ A rational adversary will choose the queue with the lowest completion rate

42 Two Queues Probabilistic Service We allow the server to serve both queues, choosing to serve queue 1 with probability p. Thus the service rate is pµ A rational adversary will choose the queue with the lowest completion rate The server wants to choose the optimal value of p, which maximises the minimum completion rate of the two queues.

43 Two Queues Minimum Completion Rate Against Value of p for Illustrative Parameters Completion Rates Against Value of p Minimum Completion Rate Completion Rate CR1 CR p p

44 Two Queues The optimal p is the value that makes the completion rates for the queues identical, which we can prove to be the case. Optimal p=0.38 Max value of minimum CR= CR1(0.38)=CR2(1-0.38)=0.2639

45 Two Queues Game Structure We can add a further level of randomisation by creating a game of policies with different p values, applicable for instance if the server is limited in the values of p it can choose

46 Two Queues Game Structure We can add a further level of randomisation by creating a game of policies with different p values, applicable for instance if the server is limited in the values of p it can choose e.g.: Server Policy, p= Adversary Q1 CR1 (0.1) CR1 (0.2)... CR1 (0.9) Policy Q2 CR2 (0.1) CR2 (0.2)... CR2 (0.9)

47 Two Queues Linear Programming We can solve larger games like this by formulating them as linear programming problems first.

48 Two Queues Linear Programming We can solve larger games like this by formulating them as linear programming problems first. Let the strategy of the server be x, let thevalue of the game be v, and let the matrix of completion rates be A = (a ij )

49 Two Queues Linear Programming We can solve larger games like this by formulating them as linear programming problems first. Let the strategy of the server be x, let thevalue of the game be v, and let the matrix of completion rates be A = (a ij ) We want to maximise over the x j min i 9 x j a ij j

50 Two Queues Linear Programming We can solve larger games like this by formulating them as linear programming problems first. Let the strategy of the server be x, let thevalue of the game be v, and let the matrix of completion rates be A = (a ij ) We want to maximise over the x j min i 9 x j a ij j This is equivalent to the linear program min y i s.t. Ay 1, y 0 where y i = x i y v i

51 Two Queues The optimal mixed strategy for this game with illustrative parameters as above is

52 Two Queues The optimal mixed strategy for this game with illustrative parameters as above is Choose p=0.3 with probability Choose p=0.4 with probability This gives value of game as

53 Two Queues The optimal mixed strategy for this game with illustrative parameters as above is Choose p=0.3 with probability Choose p=0.4 with probability This gives value of game as Clearly this is no improvement on the earlier choice of optimal p.

54 Three Queues We can apply this model to a three queue system, to see if the result concerning completion rates still holds

55 Three Queues We can apply this model to a three queue system, to see if the result concerning completion rates still holds Here we use illustrative parameters λ 1 = 5, µ 1 = 3, θ 1 = 2, λ 2 = 5, µ 2 = 8, θ 2 = 5, λ 3 = 7, µ 3 = 6, θ 3 = 4.

56 Three Queues We can apply this model to a three queue system, to see if the result concerning completion rates still holds Here we use illustrative parameters λ 1 = 5, µ 1 = 3, θ 1 = 2, λ 2 = 5, µ 2 = 8, θ 2 = 5, λ 3 = 7, µ 3 = 6, θ 3 = 4. Maximising the minimum completion rate over both p and q we find Optimal p= Optimal q= Maximum minimum completion rate=0.2085

57 Three Queues We can apply this model to a three queue system, to see if the result concerning completion rates still holds Here we use illustrative parameters λ 1 = 5, µ 1 = 3, θ 1 = 2, λ 2 = 5, µ 2 = 8, θ 2 = 5, λ 3 = 7, µ 3 = 6, θ 3 = 4. Maximising the minimum completion rate over both p and q we find Optimal p= Optimal q= Maximum minimum completion rate= Using this p and q we can calculate the 3 completion rates, finding CR1=CR2=CR3=

58 Three Queues Completion Rates for 3 Queue System CR1 CR2 CR3 1 Completion Rate X: 0.25 Y: 0.4 Z: p 0 0 q

59 Three Queues

60 Three Queues

61 Dynamic Policy

62 Dynamic Policy All the policies we have considered so far have been static policies. These can waste time compared to dynamic policies

63 Dynamic Policy All the policies we have considered so far have been static policies. These can waste time compared to dynamic policies We now assume that the server, but not the adversary, knows the state of the system at all times.

64 Dynamic Policy All the policies we have considered so far have been static policies. These can waste time compared to dynamic policies We now assume that the server, but not the adversary, knows the state of the system at all times. Based on the methods laid out in Glazebrook et al. (2004) we can construct a dynamic policy:

65

66 Let n k be the state of queue k.

67 Let n k be the state of queue k. 1 Assign a reward, r k, earned upon a service, to each queue.

68 Let n k be the state of queue k. 1 Assign a reward, r k, earned upon a service, to each queue. 2 Let the server follow a policy similar to model 2, but pick p k that maximises reward earned. This gives a static policy.

69 Let n k be the state of queue k. 1 Assign a reward, r k, earned upon a service, to each queue. 2 Let the server follow a policy similar to model 2, but pick p k that maximises reward earned. This gives a static policy. 3 We calculate an index for each possible state of the system, for queue k, by the formula: I k (n k ) = r k µ k [1 µ k p k (µ k p k + n k θ k ) 1 π 0 (λ k, µ k p k, θ k )(π 0 (λ k, µ k p k + n k θ k, θ k )) 1 ].

70 Let n k be the state of queue k. 1 Assign a reward, r k, earned upon a service, to each queue. 2 Let the server follow a policy similar to model 2, but pick p k that maximises reward earned. This gives a static policy. 3 We calculate an index for each possible state of the system, for queue k, by the formula: I k (n k ) = r k µ k [1 µ k p k (µ k p k + n k θ k ) 1 π 0 (λ k, µ k p k, θ k )(π 0 (λ k, µ k p k + n k θ k, θ k )) 1 ]. 4 The queue with the highest index is served.

71 Let n k be the state of queue k. 1 Assign a reward, r k, earned upon a service, to each queue. 2 Let the server follow a policy similar to model 2, but pick p k that maximises reward earned. This gives a static policy. 3 We calculate an index for each possible state of the system, for queue k, by the formula: I k (n k ) = r k µ k [1 µ k p k (µ k p k + n k θ k ) 1 π 0 (λ k, µ k p k, θ k )(π 0 (λ k, µ k p k + n k θ k, θ k )) 1 ]. 4 The queue with the highest index is served. This index formula is calculated using results from regeneration and renewal theory.

72 Results

73 Results To test this policy, we randomly generate 10 policies by assigning uniformly randomly generated rewards to the two queues, in the range [0, 10].

74 Results To test this policy, we randomly generate 10 policies by assigning uniformly randomly generated rewards to the two queues, in the range [0, 10]. We then used discrete event simulation to simulate the two queue system with the server operating by a given policy, and calculated the completion rates for each of the 10 policies, given by CRi = Total number of services in queue i Total number of arrivals in queue i

75 Results To test this policy, we randomly generate 10 policies by assigning uniformly randomly generated rewards to the two queues, in the range [0, 10]. We then used discrete event simulation to simulate the two queue system with the server operating by a given policy, and calculated the completion rates for each of the 10 policies, given by CRi = Total number of services in queue i Total number of arrivals in queue i We put these policies into a game, and solve as a linear program.

76 Server Policy r r Adversary Q Policy Q

77 Server Policy r r Adversary Q Policy Q Value=0.3421

78 Server Policy r r Adversary Q Policy Q Value= Theoretical optimal value over all possible policies=

79 Server Policy r r Adversary Q Policy Q Value= Theoretical optimal value over all possible policies=

80 Conclusions

81 Conclusions In a model where the server decides which queue to serve probabilistically the optimal p value is that which makes completion rates equal, and this result seems to hold for larger numbers of queues, at the intersection of all completion rates.

82 Conclusions In a model where the server decides which queue to serve probabilistically the optimal p value is that which makes completion rates equal, and this result seems to hold for larger numbers of queues, at the intersection of all completion rates. The dynamic policy game performs better than any other model investigated, with a small sub-optimality gap. This model can be applied to systems with more queues to investigate their behaviour.

83 Any Questions?