Modelling Anti-Terrorist Surveillance Systems from a Queueing Perspective
|
|
- Margery Brown
- 5 years ago
- Views:
Transcription
1 Systems from a Queueing Perspective September 7, 2012
2 Problem A surveillance resource must observe several areas, searching for potential adversaries.
3 Problem A surveillance resource must observe several areas, searching for potential adversaries. An adversary wishes to enter one of these areas, complete his objective, and then escape unnoticed.
4 Problem A surveillance resource must observe several areas, searching for potential adversaries. An adversary wishes to enter one of these areas, complete his objective, and then escape unnoticed. Our aim is to find policies for the surveillance resource that maximise its chance of catching the adversary
5 Problem A surveillance resource must observe several areas, searching for potential adversaries. An adversary wishes to enter one of these areas, complete his objective, and then escape unnoticed. Our aim is to find policies for the surveillance resource that maximise its chance of catching the adversary
6 Problem
7 Motivation A linear program can be formulated to find the optimal policy for a two area system.
8 Motivation A linear program can be formulated to find the optimal policy for a two area system. This is impractical for larger numbers of areas, as the problem quickly becomes very computationally demanding.
9 Motivation A linear program can be formulated to find the optimal policy for a two area system. This is impractical for larger numbers of areas, as the problem quickly becomes very computationally demanding. We aim to construct heuristic methods that can be applied to systems with many areas by trialling them on a two area system.
10 Motivation A linear program can be formulated to find the optimal policy for a two area system. This is impractical for larger numbers of areas, as the problem quickly becomes very computationally demanding. We aim to construct heuristic methods that can be applied to systems with many areas by trialling them on a two area system. We can improve our policies by forming a two person zero sum game, allowing the surveillance resorce to employ a mixture of successful strategies, with the payoffs given by the probability of catching the adversary under that policy. The value of the game shows the success of the policies.
11 Queues
12 Queues We treat each of these areas as a separate queue.
13 Queues We treat each of these areas as a separate queue. At any time, one of three things can happen:
14 Queues We treat each of these areas as a separate queue. At any time, one of three things can happen: A customer joins the queue, representing someone entering the area
15 Queues We treat each of these areas as a separate queue. At any time, one of three things can happen: A customer joins the queue, representing someone entering the area A customer is served at the front of the queue, representing the surveillance resource, or server, observing someone
16 Queues We treat each of these areas as a separate queue. At any time, one of three things can happen: A customer joins the queue, representing someone entering the area A customer is served at the front of the queue, representing the surveillance resource, or server, observing someone A customer abandons the queue, representing someone leaving the area
17 Queues We treat each of these areas as a separate queue. At any time, one of three things can happen: A customer joins the queue, representing someone entering the area A customer is served at the front of the queue, representing the surveillance resource, or server, observing someone A customer abandons the queue, representing someone leaving the area We assume the times between these actions occuring are distributed exponentially, with rate parameters λ, µ and θ respectively.
18 Queues cont.
19 Queues cont. As all events are distributed exponentially, these queues are continuous time Markov Chains, the state of which is given by the number of customers in the queue.
20 Steady State
21 Steady State To find the optimal strategy for the server, we must calculate the adversary s options.
22 Steady State To find the optimal strategy for the server, we must calculate the adversary s options. We assume that the adversary does not arrive atypically early, so the adversary sees the queues in their steady states.
23 Steady State To find the optimal strategy for the server, we must calculate the adversary s options. We assume that the adversary does not arrive atypically early, so the adversary sees the queues in their steady states. We calculate the steady state distribution for a single queue with constant service rate by assuming the Markov Chain is in stochastic balance, and solving the difference equation: (µ + θ(n + 1))π n+1 + λπ n 1 = (λ + µ + nθ)π n (µ + θ)π 1 = λπ 0 π n = 1 n 0
24 Steady State
25 Steady State This gives us π 0 (λ, µ, θ) = ) 1 (µ + mθ) ( n λ n n=0 m=1 ( k π k (λ, µ, θ) = λ k π 0 (λ, µ, θ) (µ + mθ) m=1 1 ) 1
26 Completion Rates
27 Completion Rates We measure the performance of a policy by the completion rates in the queues.
28 Completion Rates We measure the performance of a policy by the completion rates in the queues. CRi = µ i(1 π (i) 0 ) λ i
29 Completion Rates We measure the performance of a policy by the completion rates in the queues. CRi = µ i(1 π (i) 0 ) λ i This gives us the probability of the server catching the adversary in that queue, before the adversary abandons.
30 Assumptions The server and the adversary know the value of the parameters
31 Assumptions The server and the adversary know the value of the parameters The adversary behaves like a regular customer
32 Assumptions The server and the adversary know the value of the parameters The adversary behaves like a regular customer The server and adversary know the value of the completion rates under a given service policy
33 Assumptions The server and the adversary know the value of the parameters The adversary behaves like a regular customer The server and adversary know the value of the completion rates under a given service policy A customer cannot be served twice
34 Model
35 Model The server picks one queue and ignores the other.
36 Model The server picks one queue and ignores the other. This gives game matrix Server Policy Q1 Q2 Adversary Q1 CR1 0 Policy Q2 0 CR2
37 We solve this game by finding a strategy that gives the best worse-case scenario: ( ) CR2 x = CR1 + CR2, CR1 CR1 + CR2
38 We solve this game by finding a strategy that gives the best worse-case scenario: ( ) CR2 x = CR1 + CR2, CR1 CR1 + CR2 The value of the game is v = (CR1)(CR2) CR1 + CR2
39 We solve this game by finding a strategy that gives the best worse-case scenario: ( ) CR2 x = CR1 + CR2, CR1 CR1 + CR2 The value of the game is v = (CR1)(CR2) CR1 + CR2 Using λ 1 = 3 µ 1 = 6 θ 1 = 5 λ 2 = 6 µ 2 = 3 θ 2 = 2 as an illustrative set of parameters, we get x = (0.493, 0.561) v =
40 Two Queues Probabilistic Service We allow the server to serve both queues, choosing to serve queue 1 with probability p. Thus the service rate is pµ
41 Two Queues Probabilistic Service We allow the server to serve both queues, choosing to serve queue 1 with probability p. Thus the service rate is pµ A rational adversary will choose the queue with the lowest completion rate
42 Two Queues Probabilistic Service We allow the server to serve both queues, choosing to serve queue 1 with probability p. Thus the service rate is pµ A rational adversary will choose the queue with the lowest completion rate The server wants to choose the optimal value of p, which maximises the minimum completion rate of the two queues.
43 Two Queues Minimum Completion Rate Against Value of p for Illustrative Parameters Completion Rates Against Value of p Minimum Completion Rate Completion Rate CR1 CR p p
44 Two Queues The optimal p is the value that makes the completion rates for the queues identical, which we can prove to be the case. Optimal p=0.38 Max value of minimum CR= CR1(0.38)=CR2(1-0.38)=0.2639
45 Two Queues Game Structure We can add a further level of randomisation by creating a game of policies with different p values, applicable for instance if the server is limited in the values of p it can choose
46 Two Queues Game Structure We can add a further level of randomisation by creating a game of policies with different p values, applicable for instance if the server is limited in the values of p it can choose e.g.: Server Policy, p= Adversary Q1 CR1 (0.1) CR1 (0.2)... CR1 (0.9) Policy Q2 CR2 (0.1) CR2 (0.2)... CR2 (0.9)
47 Two Queues Linear Programming We can solve larger games like this by formulating them as linear programming problems first.
48 Two Queues Linear Programming We can solve larger games like this by formulating them as linear programming problems first. Let the strategy of the server be x, let thevalue of the game be v, and let the matrix of completion rates be A = (a ij )
49 Two Queues Linear Programming We can solve larger games like this by formulating them as linear programming problems first. Let the strategy of the server be x, let thevalue of the game be v, and let the matrix of completion rates be A = (a ij ) We want to maximise over the x j min i 9 x j a ij j
50 Two Queues Linear Programming We can solve larger games like this by formulating them as linear programming problems first. Let the strategy of the server be x, let thevalue of the game be v, and let the matrix of completion rates be A = (a ij ) We want to maximise over the x j min i 9 x j a ij j This is equivalent to the linear program min y i s.t. Ay 1, y 0 where y i = x i y v i
51 Two Queues The optimal mixed strategy for this game with illustrative parameters as above is
52 Two Queues The optimal mixed strategy for this game with illustrative parameters as above is Choose p=0.3 with probability Choose p=0.4 with probability This gives value of game as
53 Two Queues The optimal mixed strategy for this game with illustrative parameters as above is Choose p=0.3 with probability Choose p=0.4 with probability This gives value of game as Clearly this is no improvement on the earlier choice of optimal p.
54 Three Queues We can apply this model to a three queue system, to see if the result concerning completion rates still holds
55 Three Queues We can apply this model to a three queue system, to see if the result concerning completion rates still holds Here we use illustrative parameters λ 1 = 5, µ 1 = 3, θ 1 = 2, λ 2 = 5, µ 2 = 8, θ 2 = 5, λ 3 = 7, µ 3 = 6, θ 3 = 4.
56 Three Queues We can apply this model to a three queue system, to see if the result concerning completion rates still holds Here we use illustrative parameters λ 1 = 5, µ 1 = 3, θ 1 = 2, λ 2 = 5, µ 2 = 8, θ 2 = 5, λ 3 = 7, µ 3 = 6, θ 3 = 4. Maximising the minimum completion rate over both p and q we find Optimal p= Optimal q= Maximum minimum completion rate=0.2085
57 Three Queues We can apply this model to a three queue system, to see if the result concerning completion rates still holds Here we use illustrative parameters λ 1 = 5, µ 1 = 3, θ 1 = 2, λ 2 = 5, µ 2 = 8, θ 2 = 5, λ 3 = 7, µ 3 = 6, θ 3 = 4. Maximising the minimum completion rate over both p and q we find Optimal p= Optimal q= Maximum minimum completion rate= Using this p and q we can calculate the 3 completion rates, finding CR1=CR2=CR3=
58 Three Queues Completion Rates for 3 Queue System CR1 CR2 CR3 1 Completion Rate X: 0.25 Y: 0.4 Z: p 0 0 q
59 Three Queues
60 Three Queues
61 Dynamic Policy
62 Dynamic Policy All the policies we have considered so far have been static policies. These can waste time compared to dynamic policies
63 Dynamic Policy All the policies we have considered so far have been static policies. These can waste time compared to dynamic policies We now assume that the server, but not the adversary, knows the state of the system at all times.
64 Dynamic Policy All the policies we have considered so far have been static policies. These can waste time compared to dynamic policies We now assume that the server, but not the adversary, knows the state of the system at all times. Based on the methods laid out in Glazebrook et al. (2004) we can construct a dynamic policy:
65
66 Let n k be the state of queue k.
67 Let n k be the state of queue k. 1 Assign a reward, r k, earned upon a service, to each queue.
68 Let n k be the state of queue k. 1 Assign a reward, r k, earned upon a service, to each queue. 2 Let the server follow a policy similar to model 2, but pick p k that maximises reward earned. This gives a static policy.
69 Let n k be the state of queue k. 1 Assign a reward, r k, earned upon a service, to each queue. 2 Let the server follow a policy similar to model 2, but pick p k that maximises reward earned. This gives a static policy. 3 We calculate an index for each possible state of the system, for queue k, by the formula: I k (n k ) = r k µ k [1 µ k p k (µ k p k + n k θ k ) 1 π 0 (λ k, µ k p k, θ k )(π 0 (λ k, µ k p k + n k θ k, θ k )) 1 ].
70 Let n k be the state of queue k. 1 Assign a reward, r k, earned upon a service, to each queue. 2 Let the server follow a policy similar to model 2, but pick p k that maximises reward earned. This gives a static policy. 3 We calculate an index for each possible state of the system, for queue k, by the formula: I k (n k ) = r k µ k [1 µ k p k (µ k p k + n k θ k ) 1 π 0 (λ k, µ k p k, θ k )(π 0 (λ k, µ k p k + n k θ k, θ k )) 1 ]. 4 The queue with the highest index is served.
71 Let n k be the state of queue k. 1 Assign a reward, r k, earned upon a service, to each queue. 2 Let the server follow a policy similar to model 2, but pick p k that maximises reward earned. This gives a static policy. 3 We calculate an index for each possible state of the system, for queue k, by the formula: I k (n k ) = r k µ k [1 µ k p k (µ k p k + n k θ k ) 1 π 0 (λ k, µ k p k, θ k )(π 0 (λ k, µ k p k + n k θ k, θ k )) 1 ]. 4 The queue with the highest index is served. This index formula is calculated using results from regeneration and renewal theory.
72 Results
73 Results To test this policy, we randomly generate 10 policies by assigning uniformly randomly generated rewards to the two queues, in the range [0, 10].
74 Results To test this policy, we randomly generate 10 policies by assigning uniformly randomly generated rewards to the two queues, in the range [0, 10]. We then used discrete event simulation to simulate the two queue system with the server operating by a given policy, and calculated the completion rates for each of the 10 policies, given by CRi = Total number of services in queue i Total number of arrivals in queue i
75 Results To test this policy, we randomly generate 10 policies by assigning uniformly randomly generated rewards to the two queues, in the range [0, 10]. We then used discrete event simulation to simulate the two queue system with the server operating by a given policy, and calculated the completion rates for each of the 10 policies, given by CRi = Total number of services in queue i Total number of arrivals in queue i We put these policies into a game, and solve as a linear program.
76 Server Policy r r Adversary Q Policy Q
77 Server Policy r r Adversary Q Policy Q Value=0.3421
78 Server Policy r r Adversary Q Policy Q Value= Theoretical optimal value over all possible policies=
79 Server Policy r r Adversary Q Policy Q Value= Theoretical optimal value over all possible policies=
80 Conclusions
81 Conclusions In a model where the server decides which queue to serve probabilistically the optimal p value is that which makes completion rates equal, and this result seems to hold for larger numbers of queues, at the intersection of all completion rates.
82 Conclusions In a model where the server decides which queue to serve probabilistically the optimal p value is that which makes completion rates equal, and this result seems to hold for larger numbers of queues, at the intersection of all completion rates. The dynamic policy game performs better than any other model investigated, with a small sub-optimality gap. This model can be applied to systems with more queues to investigate their behaviour.
83 Any Questions?
Chapter 10 Inventory Theory
Chapter 10 Inventory Theory 10.1. (a) Find the smallest n such that g(n) 0. g(1) = 3 g(2) =2 n = 2 (b) Find the smallest n such that g(n) 0. g(1) = 1 25 1 64 g(2) = 1 4 1 25 g(3) =1 1 4 g(4) = 1 16 1
More informationLecture 5. 1 Online Learning. 1.1 Learning Setup (Perspective of Universe) CSCI699: Topics in Learning & Game Theory
CSCI699: Topics in Learning & Game Theory Lecturer: Shaddin Dughmi Lecture 5 Scribes: Umang Gupta & Anastasia Voloshinov In this lecture, we will give a brief introduction to online learning and then go
More informationEE266 Homework 5 Solutions
EE, Spring 15-1 Professor S. Lall EE Homework 5 Solutions 1. A refined inventory model. In this problem we consider an inventory model that is more refined than the one you ve seen in the lectures. The
More informationReinforcement Learning and Simulation-Based Search
Reinforcement Learning and Simulation-Based Search David Silver Outline 1 Reinforcement Learning 2 3 Planning Under Uncertainty Reinforcement Learning Markov Decision Process Definition A Markov Decision
More informationGame Theory Tutorial 3 Answers
Game Theory Tutorial 3 Answers Exercise 1 (Duality Theory) Find the dual problem of the following L.P. problem: max x 0 = 3x 1 + 2x 2 s.t. 5x 1 + 2x 2 10 4x 1 + 6x 2 24 x 1 + x 2 1 (1) x 1 + 3x 2 = 9 x
More informationReasoning with Uncertainty
Reasoning with Uncertainty Markov Decision Models Manfred Huber 2015 1 Markov Decision Process Models Markov models represent the behavior of a random process, including its internal state and the externally
More informationSection 3.1: Discrete Event Simulation
Section 3.1: Discrete Event Simulation Discrete-Event Simulation: A First Course c 2006 Pearson Ed., Inc. 0-13-142917-5 Discrete-Event Simulation: A First Course Section 3.1: Discrete Event Simulation
More informationFinal exam solutions
EE365 Stochastic Control / MS&E251 Stochastic Decision Models Profs. S. Lall, S. Boyd June 5 6 or June 6 7, 2013 Final exam solutions This is a 24 hour take-home final. Please turn it in to one of the
More informationAppendix A: Introduction to Queueing Theory
Appendix A: Introduction to Queueing Theory Queueing theory is an advanced mathematical modeling technique that can estimate waiting times. Imagine customers who wait in a checkout line at a grocery store.
More informationMarkov Decision Processes
Markov Decision Processes Robert Platt Northeastern University Some images and slides are used from: 1. CS188 UC Berkeley 2. AIMA 3. Chris Amato Stochastic domains So far, we have studied search Can use
More informationHandout 8: Introduction to Stochastic Dynamic Programming. 2 Examples of Stochastic Dynamic Programming Problems
SEEM 3470: Dynamic Optimization and Applications 2013 14 Second Term Handout 8: Introduction to Stochastic Dynamic Programming Instructor: Shiqian Ma March 10, 2014 Suggested Reading: Chapter 1 of Bertsekas,
More informationDECISION MAKING. Decision making under conditions of uncertainty
DECISION MAKING Decision making under conditions of uncertainty Set of States of nature: S 1,..., S j,..., S n Set of decision alternatives: d 1,...,d i,...,d m The outcome of the decision C ij depends
More informationX(t) = B(t), t 0,
IEOR 4106: Introduction to Operations Research: Stochastic Models Spring 2007, Professor Whitt, Final Exam Chapters 4-7 and 10 in Ross, Wednesday, May 9, 1:10pm-4:00pm Open Book: but only the Ross textbook,
More informationMarkov Decision Processes (MDPs) CS 486/686 Introduction to AI University of Waterloo
Markov Decision Processes (MDPs) CS 486/686 Introduction to AI University of Waterloo Outline Sequential Decision Processes Markov chains Highlight Markov property Discounted rewards Value iteration Markov
More informationLecture 7: Bayesian approach to MAB - Gittins index
Advanced Topics in Machine Learning and Algorithmic Game Theory Lecture 7: Bayesian approach to MAB - Gittins index Lecturer: Yishay Mansour Scribe: Mariano Schain 7.1 Introduction In the Bayesian approach
More informationIntelligent Systems (AI-2)
Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 9 Sep, 28, 2016 Slide 1 CPSC 422, Lecture 9 An MDP Approach to Multi-Category Patient Scheduling in a Diagnostic Facility Adapted from: Matthew
More informationAn Approximation Algorithm for Capacity Allocation over a Single Flight Leg with Fare-Locking
An Approximation Algorithm for Capacity Allocation over a Single Flight Leg with Fare-Locking Mika Sumida School of Operations Research and Information Engineering, Cornell University, Ithaca, New York
More informationMATH3075/3975 FINANCIAL MATHEMATICS TUTORIAL PROBLEMS
MATH307/37 FINANCIAL MATHEMATICS TUTORIAL PROBLEMS School of Mathematics and Statistics Semester, 04 Tutorial problems should be used to test your mathematical skills and understanding of the lecture material.
More informationCEC login. Student Details Name SOLUTIONS
Student Details Name SOLUTIONS CEC login Instructions You have roughly 1 minute per point, so schedule your time accordingly. There is only one correct answer per question. Good luck! Question 1. Searching
More information004: Macroeconomic Theory
004: Macroeconomic Theory Lecture 13 Mausumi Das Lecture Notes, DSE October 17, 2014 Das (Lecture Notes, DSE) Macro October 17, 2014 1 / 18 Micro Foundation of the Consumption Function: Limitation of the
More information1. For two independent lives now age 30 and 34, you are given:
Society of Actuaries Course 3 Exam Fall 2003 **BEGINNING OF EXAMINATION** 1. For two independent lives now age 30 and 34, you are given: x q x 30 0.1 31 0.2 32 0.3 33 0.4 34 0.5 35 0.6 36 0.7 37 0.8 Calculate
More informationColumn generation to solve planning problems
Column generation to solve planning problems ALGORITMe Han Hoogeveen 1 Continuous Knapsack problem We are given n items with integral weight a j ; integral value c j. B is a given integer. Goal: Find a
More informationHeuristics in Rostering for Call Centres
Heuristics in Rostering for Call Centres Shane G. Henderson, Andrew J. Mason Department of Engineering Science University of Auckland Auckland, New Zealand sg.henderson@auckland.ac.nz, a.mason@auckland.ac.nz
More informationLesson Plan for Simulation with Spreadsheets (8/31/11 & 9/7/11)
Jeremy Tejada ISE 441 - Introduction to Simulation Learning Outcomes: Lesson Plan for Simulation with Spreadsheets (8/31/11 & 9/7/11) 1. Students will be able to list and define the different components
More informationBEHAVIOUR OF PASSAGE TIME FOR A QUEUEING NETWORK MODEL WITH FEEDBACK: A SIMULATION STUDY
IJMMS 24:24, 1267 1278 PII. S1611712426287 http://ijmms.hindawi.com Hindawi Publishing Corp. BEHAVIOUR OF PASSAGE TIME FOR A QUEUEING NETWORK MODEL WITH FEEDBACK: A SIMULATION STUDY BIDYUT K. MEDYA Received
More informationDo all of Part One (1 pt. each), one from Part Two (15 pts.), and four from Part Three (15 pts. each) <><><><><> PART ONE <><><><><>
56:171 Operations Research Final Exam - December 13, 1989 Instructor: D.L. Bricker Do all of Part One (1 pt. each), one from Part Two (15 pts.), and four from
More informationDefinition 4.1. In a stochastic process T is called a stopping time if you can tell when it happens.
102 OPTIMAL STOPPING TIME 4. Optimal Stopping Time 4.1. Definitions. On the first day I explained the basic problem using one example in the book. On the second day I explained how the solution to the
More informationOptimal Scheduling Policy Determination in HSDPA Networks
Optimal Scheduling Policy Determination in HSDPA Networks Hussein Al-Zubaidy, Jerome Talim, Ioannis Lambadaris SCE-Carleton University 1125 Colonel By Drive, Ottawa, ON, Canada Email: {hussein, jtalim,
More informationEcon 8602, Fall 2017 Homework 2
Econ 8602, Fall 2017 Homework 2 Due Tues Oct 3. Question 1 Consider the following model of entry. There are two firms. There are two entry scenarios in each period. With probability only one firm is able
More informationLog-Robust Portfolio Management
Log-Robust Portfolio Management Dr. Aurélie Thiele Lehigh University Joint work with Elcin Cetinkaya and Ban Kawas Research partially supported by the National Science Foundation Grant CMMI-0757983 Dr.
More informationarxiv: v1 [math.pr] 6 Apr 2015
Analysis of the Optimal Resource Allocation for a Tandem Queueing System arxiv:1504.01248v1 [math.pr] 6 Apr 2015 Liu Zaiming, Chen Gang, Wu Jinbiao School of Mathematics and Statistics, Central South University,
More informationAdaptive Experiments for Policy Choice. March 8, 2019
Adaptive Experiments for Policy Choice Maximilian Kasy Anja Sautmann March 8, 2019 Introduction The goal of many experiments is to inform policy choices: 1. Job search assistance for refugees: Treatments:
More informationStatistics and Their Distributions
Statistics and Their Distributions Deriving Sampling Distributions Example A certain system consists of two identical components. The life time of each component is supposed to have an expentional distribution
More informationEindhoven University of Technology BACHELOR. Price directed control of bike sharing systems. van der Schoot, Femke A.
Eindhoven University of Technology BACHELOR Price directed control of bike sharing systems van der Schoot, Femke A. Award date: 2017 Link to publication Disclaimer This document contains a student thesis
More informationComparative Study between Linear and Graphical Methods in Solving Optimization Problems
Comparative Study between Linear and Graphical Methods in Solving Optimization Problems Mona M Abd El-Kareem Abstract The main target of this paper is to establish a comparative study between the performance
More informationAssembly systems with non-exponential machines: Throughput and bottlenecks
Nonlinear Analysis 69 (2008) 911 917 www.elsevier.com/locate/na Assembly systems with non-exponential machines: Throughput and bottlenecks ShiNung Ching, Semyon M. Meerkov, Liang Zhang Department of Electrical
More informationPricing Problems under the Markov Chain Choice Model
Pricing Problems under the Markov Chain Choice Model James Dong School of Operations Research and Information Engineering, Cornell University, Ithaca, New York 14853, USA jd748@cornell.edu A. Serdar Simsek
More informationDynamic vs. static decision strategies in adversarial reasoning
Dynamic vs. static decision strategies in adversarial reasoning David A. Pelta 1 Ronald R. Yager 2 1. Models of Decision and Optimization Research Group Department of Computer Science and A.I., University
More informationA simple wealth model
Quantitative Macroeconomics Raül Santaeulàlia-Llopis, MOVE-UAB and Barcelona GSE Homework 5, due Thu Nov 1 I A simple wealth model Consider the sequential problem of a household that maximizes over streams
More informationSOLVING ROBUST SUPPLY CHAIN PROBLEMS
SOLVING ROBUST SUPPLY CHAIN PROBLEMS Daniel Bienstock Nuri Sercan Özbay Columbia University, New York November 13, 2005 Project with Lucent Technologies Optimize the inventory buffer levels in a complicated
More informationHandout 4: Deterministic Systems and the Shortest Path Problem
SEEM 3470: Dynamic Optimization and Applications 2013 14 Second Term Handout 4: Deterministic Systems and the Shortest Path Problem Instructor: Shiqian Ma January 27, 2014 Suggested Reading: Bertsekas
More informationMulti-armed bandit problems
Multi-armed bandit problems Stochastic Decision Theory (2WB12) Arnoud den Boer 13 March 2013 Set-up 13 and 14 March: Lectures. 20 and 21 March: Paper presentations (Four groups, 45 min per group). Before
More informationMartingale Pricing Theory in Discrete-Time and Discrete-Space Models
IEOR E4707: Foundations of Financial Engineering c 206 by Martin Haugh Martingale Pricing Theory in Discrete-Time and Discrete-Space Models These notes develop the theory of martingale pricing in a discrete-time,
More informationStochastic Differential Equations in Finance and Monte Carlo Simulations
Stochastic Differential Equations in Finance and Department of Statistics and Modelling Science University of Strathclyde Glasgow, G1 1XH China 2009 Outline Stochastic Modelling in Asset Prices 1 Stochastic
More informationDiscrete-Event Simulation
Discrete-Event Simulation Lawrence M. Leemis and Stephen K. Park, Discrete-Event Simul A First Course, Prentice Hall, 2006 Hui Chen Computer Science Virginia State University Petersburg, Virginia February
More informationDynamic Portfolio Execution Detailed Proofs
Dynamic Portfolio Execution Detailed Proofs Gerry Tsoukalas, Jiang Wang, Kay Giesecke March 16, 2014 1 Proofs Lemma 1 (Temporary Price Impact) A buy order of size x being executed against i s ask-side
More informationLecture 17: More on Markov Decision Processes. Reinforcement learning
Lecture 17: More on Markov Decision Processes. Reinforcement learning Learning a model: maximum likelihood Learning a value function directly Monte Carlo Temporal-difference (TD) learning COMP-424, Lecture
More informationYao s Minimax Principle
Complexity of algorithms The complexity of an algorithm is usually measured with respect to the size of the input, where size may for example refer to the length of a binary word describing the input,
More informationUnobserved Heterogeneity Revisited
Unobserved Heterogeneity Revisited Robert A. Miller Dynamic Discrete Choice March 2018 Miller (Dynamic Discrete Choice) cemmap 7 March 2018 1 / 24 Distributional Assumptions about the Unobserved Variables
More informationEquity correlations implied by index options: estimation and model uncertainty analysis
1/18 : estimation and model analysis, EDHEC Business School (joint work with Rama COT) Modeling and managing financial risks Paris, 10 13 January 2011 2/18 Outline 1 2 of multi-asset models Solution to
More informationNon-Deterministic Search
Non-Deterministic Search MDP s 1 Non-Deterministic Search How do you plan (search) when your actions might fail? In general case, how do you plan, when the actions have multiple possible outcomes? 2 Example:
More informationMarkov Decision Processes. CS 486/686: Introduction to Artificial Intelligence
Markov Decision Processes CS 486/686: Introduction to Artificial Intelligence 1 Outline Markov Chains Discounted Rewards Markov Decision Processes (MDP) - Value Iteration - Policy Iteration 2 Markov Chains
More informationBonus-malus systems 6.1 INTRODUCTION
6 Bonus-malus systems 6.1 INTRODUCTION This chapter deals with the theory behind bonus-malus methods for automobile insurance. This is an important branch of non-life insurance, in many countries even
More informationMengdi Wang. July 3rd, Laboratory for Information and Decision Systems, M.I.T.
Practice July 3rd, 2012 Laboratory for Information and Decision Systems, M.I.T. 1 2 Infinite-Horizon DP Minimize over policies the objective cost function J π (x 0 ) = lim N E w k,k=0,1,... DP π = {µ 0,µ
More informationSocially-Optimal Design of Crowdsourcing Platforms with Reputation Update Errors
Socially-Optimal Design of Crowdsourcing Platforms with Reputation Update Errors 1 Yuanzhang Xiao, Yu Zhang, and Mihaela van der Schaar Abstract Crowdsourcing systems (e.g. Yahoo! Answers and Amazon Mechanical
More informationUnemployment Insurance
Unemployment Insurance Seyed Ali Madanizadeh Sharif U. of Tech. May 23, 2014 Seyed Ali Madanizadeh (Sharif U. of Tech.) Unemployment Insurance May 23, 2014 1 / 35 Introduction Unemployment Insurance The
More informationECON 815. A Basic New Keynesian Model II
ECON 815 A Basic New Keynesian Model II Winter 2015 Queen s University ECON 815 1 Unemployment vs. Inflation 12 10 Unemployment 8 6 4 2 0 1 1.5 2 2.5 3 3.5 4 4.5 5 Core Inflation 14 12 10 Unemployment
More informationDynamic Resource Allocation for Spot Markets in Cloud Computi
Dynamic Resource Allocation for Spot Markets in Cloud Computing Environments Qi Zhang 1, Quanyan Zhu 2, Raouf Boutaba 1,3 1 David. R. Cheriton School of Computer Science University of Waterloo 2 Department
More informationCMSC 858F: Algorithmic Game Theory Fall 2010 Introduction to Algorithmic Game Theory
CMSC 858F: Algorithmic Game Theory Fall 2010 Introduction to Algorithmic Game Theory Instructor: Mohammad T. Hajiaghayi Scribe: Hyoungtae Cho October 13, 2010 1 Overview In this lecture, we introduce the
More informationUniversity of Groningen. Inventory Control for Multi-location Rental Systems van der Heide, Gerlach
University of Groningen Inventory Control for Multi-location Rental Systems van der Heide, Gerlach IMPORTANT NOTE: You are advised to consult the publisher's version publisher's PDF) if you wish to cite
More informationReal Options and Game Theory in Incomplete Markets
Real Options and Game Theory in Incomplete Markets M. Grasselli Mathematics and Statistics McMaster University IMPA - June 28, 2006 Strategic Decision Making Suppose we want to assign monetary values to
More informationPractical Hedging: From Theory to Practice. OSU Financial Mathematics Seminar May 5, 2008
Practical Hedging: From Theory to Practice OSU Financial Mathematics Seminar May 5, 008 Background Dynamic replication is a risk management technique used to mitigate market risk We hope to spend a certain
More informationOverview: Representation Techniques
1 Overview: Representation Techniques Week 6 Representations for classical planning problems deterministic environment; complete information Week 7 Logic programs for problem representations including
More informationLecture 5 January 30
EE 223: Stochastic Estimation and Control Spring 2007 Lecture 5 January 30 Lecturer: Venkat Anantharam Scribe: aryam Kamgarpour 5.1 Secretary Problem The problem set-up is explained in Lecture 4. We review
More information4.5 Comparing Exponential Functions
4.5 Comparing Exponential Functions So far we have talked in detail about both linear and exponential functions. In this section we ll compare exponential functions to other exponential functions and also
More informationPOMDPs: Partially Observable Markov Decision Processes Advanced AI
POMDPs: Partially Observable Markov Decision Processes Advanced AI Wolfram Burgard Types of Planning Problems Classical Planning State observable Action Model Deterministic, accurate MDPs observable stochastic
More informationExam M Fall 2005 PRELIMINARY ANSWER KEY
Exam M Fall 005 PRELIMINARY ANSWER KEY Question # Answer Question # Answer 1 C 1 E C B 3 C 3 E 4 D 4 E 5 C 5 C 6 B 6 E 7 A 7 E 8 D 8 D 9 B 9 A 10 A 30 D 11 A 31 A 1 A 3 A 13 D 33 B 14 C 34 C 15 A 35 A
More informationMaking Decisions. CS 3793 Artificial Intelligence Making Decisions 1
Making Decisions CS 3793 Artificial Intelligence Making Decisions 1 Planning under uncertainty should address: The world is nondeterministic. Actions are not certain to succeed. Many events are outside
More informationSTATE UNIVERSITY OF NEW YORK AT ALBANY Department of Economics. Ph. D. Preliminary Examination: Macroeconomics Fall, 2009
STATE UNIVERSITY OF NEW YORK AT ALBANY Department of Economics Ph. D. Preliminary Examination: Macroeconomics Fall, 2009 Instructions: Read the questions carefully and make sure to show your work. You
More information17 MAKING COMPLEX DECISIONS
267 17 MAKING COMPLEX DECISIONS The agent s utility now depends on a sequence of decisions In the following 4 3grid environment the agent makes a decision to move (U, R, D, L) at each time step When the
More informationReinforcement learning and Markov Decision Processes (MDPs) (B) Avrim Blum
Reinforcement learning and Markov Decision Processes (MDPs) 15-859(B) Avrim Blum RL and MDPs General scenario: We are an agent in some state. Have observations, perform actions, get rewards. (See lights,
More informationDesign of Information Sharing Mechanisms
Design of Information Sharing Mechanisms Krishnamurthy Iyer ORIE, Cornell University Oct 2018, IMA Based on joint work with David Lingenbrink, Cornell University Motivation Many instances in the service
More informationWhen one firm considers changing its price or output level, it must make assumptions about the reactions of its rivals.
Chapter 3 Oligopoly Oligopoly is an industry where there are relatively few sellers. The product may be standardized (steel) or differentiated (automobiles). The firms have a high degree of interdependence.
More informationLecture IV Portfolio management: Efficient portfolios. Introduction to Finance Mathematics Fall Financial mathematics
Lecture IV Portfolio management: Efficient portfolios. Introduction to Finance Mathematics Fall 2014 Reduce the risk, one asset Let us warm up by doing an exercise. We consider an investment with σ 1 =
More informationChapter 7 Pricing with Market Power SOLUTIONS TO EXERCISES
Firms, Prices & Markets Timothy Van Zandt August 2012 Chapter 7 Pricing with Market Power SOLUTIONS TO EXERCISES Exercise 7.1. Suppose you produce minivans at a constant marginal cost of $15K and your
More informationOptimal Search for Parameters in Monte Carlo Simulation for Derivative Pricing
Optimal Search for Parameters in Monte Carlo Simulation for Derivative Pricing Prof. Chuan-Ju Wang Department of Computer Science University of Taipei Joint work with Prof. Ming-Yang Kao March 28, 2014
More informationA selection of MAS learning techniques based on RL
A selection of MAS learning techniques based on RL Ann Nowé 14/11/12 Herhaling titel van presentatie 1 Content Single stage setting Common interest (Claus & Boutilier, Kapetanakis&Kudenko) Conflicting
More informationStochastic Games and Bayesian Games
Stochastic Games and Bayesian Games CPSC 532L Lecture 10 Stochastic Games and Bayesian Games CPSC 532L Lecture 10, Slide 1 Lecture Overview 1 Recap 2 Stochastic Games 3 Bayesian Games Stochastic Games
More informationEC3115 Monetary Economics
EC3115 :: L.12 : Time inconsistency and inflation bias Almaty, KZ :: 20 January 2016 EC3115 Monetary Economics Lecture 12: Time inconsistency and inflation bias Anuar D. Ushbayev International School of
More informationAnswers to Microeconomics Prelim of August 24, In practice, firms often price their products by marking up a fixed percentage over (average)
Answers to Microeconomics Prelim of August 24, 2016 1. In practice, firms often price their products by marking up a fixed percentage over (average) cost. To investigate the consequences of markup pricing,
More informationModelling Credit Spread Behaviour. FIRST Credit, Insurance and Risk. Angelo Arvanitis, Jon Gregory, Jean-Paul Laurent
Modelling Credit Spread Behaviour Insurance and Angelo Arvanitis, Jon Gregory, Jean-Paul Laurent ICBI Counterparty & Default Forum 29 September 1999, Paris Overview Part I Need for Credit Models Part II
More informationDecision making under uncertainty
Decision making under uncertainty 1 Outline 1. Components of decision making 2. Criteria for decision making 3. Utility theory 4. Decision trees 5. Posterior probabilities using Bayes rule 6. The Monty
More information004: Macroeconomic Theory
004: Macroeconomic Theory Lecture 14 Mausumi Das Lecture Notes, DSE October 21, 2014 Das (Lecture Notes, DSE) Macro October 21, 2014 1 / 20 Theories of Economic Growth We now move on to a different dynamics
More informationSequential Decision Making
Sequential Decision Making Dynamic programming Christos Dimitrakakis Intelligent Autonomous Systems, IvI, University of Amsterdam, The Netherlands March 18, 2008 Introduction Some examples Dynamic programming
More informationInvestment strategies and risk management for participating life insurance contracts
1/20 Investment strategies and risk for participating life insurance contracts and Steven Haberman Cass Business School AFIR Colloquium Munich, September 2009 2/20 & Motivation Motivation New supervisory
More informationCSCI 1951-G Optimization Methods in Finance Part 07: Portfolio Optimization
CSCI 1951-G Optimization Methods in Finance Part 07: Portfolio Optimization March 9 16, 2018 1 / 19 The portfolio optimization problem How to best allocate our money to n risky assets S 1,..., S n with
More informationFinancial Optimization ISE 347/447. Lecture 15. Dr. Ted Ralphs
Financial Optimization ISE 347/447 Lecture 15 Dr. Ted Ralphs ISE 347/447 Lecture 15 1 Reading for This Lecture C&T Chapter 12 ISE 347/447 Lecture 15 2 Stock Market Indices A stock market index is a statistic
More informationADVANCED MACROECONOMIC TECHNIQUES NOTE 7b
316-406 ADVANCED MACROECONOMIC TECHNIQUES NOTE 7b Chris Edmond hcpedmond@unimelb.edu.aui Aiyagari s model Arguably the most popular example of a simple incomplete markets model is due to Rao Aiyagari (1994,
More informationReinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration
Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration Piyush Rai CS5350/6350: Machine Learning November 29, 2011 Reinforcement Learning Supervised Learning: Uses explicit supervision
More informationValuing Capacity Investment Decisions: Binomial vs. Markov Models
Valuing Capacity Investment Decisions: Binomial vs. Markov Models Dalila B. M. M. Fontes 1 and Fernando A. C. C. Fontes 2 1 LIACC, Faculdade de Economia da Universidade do Porto Rua Dr. Roberto Frias,
More informationExpectimax and other Games
Expectimax and other Games 2018/01/30 Chapter 5 in R&N 3rd Ø Announcement: q Slides for this lecture are here: http://www.public.asu.edu/~yzhan442/teaching/cse471/lectures/games.pdf q Project 2 released,
More informationPortfolio Management Under Epistemic Uncertainty Using Stochastic Dominance and Information-Gap Theory
Portfolio Management Under Epistemic Uncertainty Using Stochastic Dominance and Information-Gap Theory D. Berleant, L. Andrieu, J.-P. Argaud, F. Barjon, M.-P. Cheong, M. Dancre, G. Sheble, and C.-C. Teoh
More informationDynamic Pricing with Varying Cost
Dynamic Pricing with Varying Cost L. Jeff Hong College of Business City University of Hong Kong Joint work with Ying Zhong and Guangwu Liu Outline 1 Introduction 2 Problem Formulation 3 Pricing Policy
More informationTUTORIAL KIT OMEGA SEMESTER PROGRAMME: BANKING AND FINANCE
TUTORIAL KIT OMEGA SEMESTER PROGRAMME: BANKING AND FINANCE COURSE: BFN 425 QUANTITATIVE TECHNIQUE FOR FINANCIAL DECISIONS i DISCLAIMER The contents of this document are intended for practice and leaning
More informationBSc (Hons) Software Engineering BSc (Hons) Computer Science with Network Security
BSc (Hons) Software Engineering BSc (Hons) Computer Science with Network Security Cohorts BCNS/ 06 / Full Time & BSE/ 06 / Full Time Resit Examinations for 2008-2009 / Semester 1 Examinations for 2008-2009
More informationSTATE UNIVERSITY OF NEW YORK AT ALBANY Department of Economics. Ph. D. Comprehensive Examination: Macroeconomics Fall, 2010
STATE UNIVERSITY OF NEW YORK AT ALBANY Department of Economics Ph. D. Comprehensive Examination: Macroeconomics Fall, 2010 Section 1. (Suggested Time: 45 Minutes) For 3 of the following 6 statements, state
More informationBudget Management In GSP (2018)
Budget Management In GSP (2018) Yahoo! March 18, 2018 Miguel March 18, 2018 1 / 26 Today s Presentation: Budget Management Strategies in Repeated auctions, Balseiro, Kim, and Mahdian, WWW2017 Learning
More informationAM 121: Intro to Optimization Models and Methods
AM 121: Intro to Optimization Models and Methods Lecture 18: Markov Decision Processes Yiling Chen and David Parkes Lesson Plan Markov decision processes Policies and Value functions Solving: average reward,
More informationCS 6300 Artificial Intelligence Spring 2018
Expectimax Search CS 6300 Artificial Intelligence Spring 2018 Tucker Hermans thermans@cs.utah.edu Many slides courtesy of Pieter Abbeel and Dan Klein Expectimax Search Trees What if we don t know what
More informationDynamic Pricing in Ridesharing Platforms
Dynamic Pricing in Ridesharing Platforms A Queueing Approach Sid Banerjee Ramesh Johari Carlos Riquelme Cornell Stanford Stanford rjohari@stanford.edu With thanks to Chris Pouliot, Chris Sholley, and Lyft
More information