Optimization Methods. Lecture 16: Dynamic Programming
|
|
- Robert Potter
- 5 years ago
- Views:
Transcription
1 Optimization Methods Lecture 16: Dynamic Programming
2 1 Outline 1. The knapsack problem Slide 1. The traveling salesman problem 3. The general DP framework 4. Bellman equation 5. Optimal inventory control 6. Optimal trading 7. Multiplying matrices The Knapsack problem nx Dene maximize subject to nx j=1 j=1 C i (w) = maximize c j x j w j x j K xj f0; 1g ix j=1 c j x j Slide subject to ix w j x j w j=1 x j f0; 1g.1 A DP Algorithm C i (w): the maximum value that can be accumulated using some of the rst i items subject to the constraint that the total accumulated weight is equal to w Slide 3 Recursion C i+1 (w) = max C i (w); C i (w w i+1 ) + c i+1 By considering all states of the form (i; w) with w K, algorithm has complexity O(nK) 1
3 3 The TSP G = (V; A) directed graph with n nodes c ij cost of arc (i; j) Approach: choice of a tour as a sequence of choices We start at node 1; then, at each stage, we choose which node to visit next. After a number of stages, we have visited a subset S of V and we are at a current node k S 3.1 A DP algorithm C(S; k) be the minimum cost over all paths that start at node 1, visit all nodes in the set S exactly once, and end up at node k (S; k) a state; this state can be reached from any state of the form fkg; m, with m S n fkg, at a transition cost of c mk Recursion C(S; k) = min C S n fkg; m + c mk ; k S msnfkg C f1g; 1 = 0: S n Slide 4 Slide 5 Length of an optimal tour is Complexity: O n min C f1; : : : ; ng; k k n operations + c k1 4 Guidelines for constructing DP Algorithms View the choice of a feasible solution as a sequence of decisions occurring in stages, and so that the total cost is the sum of the costs of individual decisions. Dene the state as a summary of all relevant past decisions. Determine which state transitions are possible. Let the cost of each state transition be the cost of the corresponding decision. Write a recursion on the optimal cost from the origin state to a destination state. The most crucial step is usually the denition of a suitable state. Slide 6
4 ! 5 The general DP framework Discrete time dynamic system described by state x k, k indexes time. u k control to be selected at time k. u k U k (x k ). Slide 7 w k randomness at time k N time horizon Dynamics: x k+1 = f k (x k ; u k ; w k ) Cost function: additive over time N1 X E g N (x N ) + g k (x k ; u k ; w k ) k=0 5.1 Inventory Control x k stock available at the beginning of the kth period u k stock ordered at the beginning of the kth period w k demand duirng the kth period with given probability distribution. Excess demand is backloged and lled as soon as additional inventory is available. Dynamics x k+1 = x k + u k w k Cost! N1 X E R(x N ) + (r(x k ) + cu k ) 6 The DP Algorithm k=0 Dene J k (x k ) to be the expected optimal cost starting from stage k at state x k. Bellman's principle of optimality Slide 8 Slide 9 J N (x N ) = g N (x N ) J k (x k ) = min E wk g k (x k ; u k ; w k ) + J k+1 (f k (x k ; u k ; w k )) u ku k(x k) Optimal expected cost for the overall problem: J 0 (x 0 ). 3
5 7 Inventory Control If r(x k ) = ax k, w k N( k ; k ), then Slide 10 u k = c k x k + d k ; J k (x k ) = b k x k + f k x k + e k If r(x k ) = p max(0; x k ) + h max(0; x k ), then there exist S k : S k x k if x k < S k u k = 0 if x k S k 8 Optimal trading S shares of a stock to be bought within a horizon T. t = 1; ; : : :; T discrete trading periods. Slide 11 Control: S t number of shares acquired in period t at price P t, t = 1; ; : : : ; T T X Objective: min E P t S t Dynamics: t=1 where > 0, t N(0; 1) T X s:t: S t = S t=1 P t = P t1 + S t + t 8.1 DP ingredients State: (P t1 ; W t ) Slide 1 P t1 price realized at the previous period W t # of shares remaining to be purchased Control: S t number of shares purchased at time t Randomness: t Objective: min E P T P t S t=1 t Dynamics: P t = P t1 + S t + t W t = W t1 S t1 ; W 1 = S; W T +1 = 0 Note that W T +1 = 0 is equivalent to the constraint that S must be executed by period T Slide 13 4
6 8. The Bellman Equation Slide 14 J t (P t1 ; W t ) = min E t P t S t + J t+1 (P t ; W t+1 ) S t Since W T +1 = 0 ) S T = W T J T (P T 1 ; W T ) = min E T [P T W T ] = (P T 1 + W T )W T S T 8.3 Solution J T 1 (P T ; W T 1 ) = = min E T 1 P T 1 S T 1 + J T (P T 1 ; W T ) S T1 = min E T 1 (P T + S T 1 + T 1 )S T 1 + S T1 J T P T + S T 1 + T 1 ; W T 1 S T 1 Slide 15 S T 1 = W T 1 3 J T 1 (P T ; W T 1 ) = W T 1 (P T + W T 1 ); 4 Continuing in this fashion, Slide 16 S T k = W T k k + 1 J T k (P T k1 ; W T k ) = k + W T k (P T k1 + W T k ) (k + 1) S 1 = S T S 1 J 1 (P 0 ; W 1 ) = P 0 S T S 1 = S = = S T = S T 5
7 8.4 Dierent Dynamics Slide 17 P t = P t1 + S t + X t + t ; > 0 X t = X t1 + t ; X 1 = 1 ; (1; 1) where t N(0; ) and t N(0; ) 8.5 Solution Slide 18 W T k b k1 S T k = + X T k k + 1 a k1 J T k (P T k1 ; X T k ; W T k ) = P T k1 W T k + a k W T k + for k = 0; 1; : : :; T 1, where: 1 a k = 1 + ; ; a 0 = k + 1 b k X T k W T k + c k X T k + d k b k1 b k = + ; b 0 = a k1 b k1 4a k1 c k = c k1 ; c 0 = 0 d k = d k1 + c k1 ; d 0 = 0 : 9 Matrix multiplication Matrices: M k : n k n k+1 Slide 19 Objective: Find M 1 M M N Example: M 1 M M 3 ; M 1 : 1 10, M : 10 1, M 3 : M 1 (M M 3 ) 00 multiplications; (M 1 M )M 3 0 multiplictions. What is the optimal order for performing the multiplication? Slide 0 6
8 m(i; j) optimal number of scalar multiplications for multiplying M i : : : M j. m(i; i) = 0 For i < j: m(i; j) = min (m(i; k) + m(k + 1; j) + n i n k+1 n j+1 ) ik<j 7
9 MIT OpenCourseWare J / 6.55J Optimization Methods Fall 009 For information about citing these materials or our Terms of Use, visit: - 1
IEOR E4004: Introduction to OR: Deterministic Models
IEOR E4004: Introduction to OR: Deterministic Models 1 Dynamic Programming Following is a summary of the problems we discussed in class. (We do not include the discussion on the container problem or the
More informationDynamic Programming (DP) Massimo Paolucci University of Genova
Dynamic Programming (DP) Massimo Paolucci University of Genova DP cannot be applied to each kind of problem In particular, it is a solution method for problems defined over stages For each stage a subproblem
More information6.231 DYNAMIC PROGRAMMING LECTURE 5 LECTURE OUTLINE
6.231 DYNAMIC PROGRAMMING LECTURE 5 LECTURE OUTLINE Stopping problems Scheduling problems Minimax Control 1 PURE STOPPING PROBLEMS Two possible controls: Stop (incur a one-time stopping cost, and move
More information6.231 DYNAMIC PROGRAMMING LECTURE 10 LECTURE OUTLINE
6.231 DYNAMIC PROGRAMMING LECTURE 10 LECTURE OUTLINE Rollout algorithms Cost improvement property Discrete deterministic problems Approximations of rollout algorithms Discretization of continuous time
More information6.231 DYNAMIC PROGRAMMING LECTURE 8 LECTURE OUTLINE
6.231 DYNAMIC PROGRAMMING LECTURE 8 LECTURE OUTLINE Suboptimal control Cost approximation methods: Classification Certainty equivalent control: An example Limited lookahead policies Performance bounds
More informationDynamic Programming cont. We repeat: The Dynamic Programming Template has three parts.
Page 1 Dynamic Programming cont. We repeat: The Dynamic Programming Template has three parts. Subproblems Sometimes this is enough if the algorithm and its complexity is obvious. Recursion Algorithm Must
More informationHandout 4: Deterministic Systems and the Shortest Path Problem
SEEM 3470: Dynamic Optimization and Applications 2013 14 Second Term Handout 4: Deterministic Systems and the Shortest Path Problem Instructor: Shiqian Ma January 27, 2014 Suggested Reading: Bertsekas
More informationLecture Notes 1
4.45 Lecture Notes Guido Lorenzoni Fall 2009 A portfolio problem To set the stage, consider a simple nite horizon problem. A risk averse agent can invest in two assets: riskless asset (bond) pays gross
More informationStochastic Optimal Control
Stochastic Optimal Control Lecturer: Eilyan Bitar, Cornell ECE Scribe: Kevin Kircher, Cornell MAE These notes summarize some of the material from ECE 5555 (Stochastic Systems) at Cornell in the fall of
More informationLecture 10: The knapsack problem
Optimization Methods in Finance (EPFL, Fall 2010) Lecture 10: The knapsack problem 24.11.2010 Lecturer: Prof. Friedrich Eisenbrand Scribe: Anu Harjula The knapsack problem The Knapsack problem is a problem
More informationIntroduction to Dynamic Programming
Introduction to Dynamic Programming http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html Acknowledgement: this slides is based on Prof. Mengdi Wang s and Prof. Dimitri Bertsekas lecture notes Outline 2/65 1
More informationMultistage Stochastic Programming
IE 495 Lecture 21 Multistage Stochastic Programming Prof. Jeff Linderoth April 16, 2003 April 16, 2002 Stochastic Programming Lecture 21 Slide 1 Outline HW Fixes Multistage Stochastic Programming Modeling
More informationDynamic Programming: An overview. 1 Preliminaries: The basic principle underlying dynamic programming
Dynamic Programming: An overview These notes summarize some key properties of the Dynamic Programming principle to optimize a function or cost that depends on an interval or stages. This plays a key role
More information6.262: Discrete Stochastic Processes 3/2/11. Lecture 9: Markov rewards and dynamic prog.
6.262: Discrete Stochastic Processes 3/2/11 Lecture 9: Marov rewards and dynamic prog. Outline: Review plus of eigenvalues and eigenvectors Rewards for Marov chains Expected first-passage-times Aggregate
More informationJune 11, Dynamic Programming( Weighted Interval Scheduling)
Dynamic Programming( Weighted Interval Scheduling) June 11, 2014 Problem Statement: 1 We have a resource and many people request to use the resource for periods of time (an interval of time) 2 Each interval
More information6.231 DYNAMIC PROGRAMMING LECTURE 3 LECTURE OUTLINE
6.21 DYNAMIC PROGRAMMING LECTURE LECTURE OUTLINE Deterministic finite-state DP problems Backward shortest path algorithm Forward shortest path algorithm Shortest path examples Alternative shortest path
More informationHandout 8: Introduction to Stochastic Dynamic Programming. 2 Examples of Stochastic Dynamic Programming Problems
SEEM 3470: Dynamic Optimization and Applications 2013 14 Second Term Handout 8: Introduction to Stochastic Dynamic Programming Instructor: Shiqian Ma March 10, 2014 Suggested Reading: Chapter 1 of Bertsekas,
More informationChapter 21. Dynamic Programming CONTENTS 21.1 A SHORTEST-ROUTE PROBLEM 21.2 DYNAMIC PROGRAMMING NOTATION
Chapter 21 Dynamic Programming CONTENTS 21.1 A SHORTEST-ROUTE PROBLEM 21.2 DYNAMIC PROGRAMMING NOTATION 21.3 THE KNAPSACK PROBLEM 21.4 A PRODUCTION AND INVENTORY CONTROL PROBLEM 23_ch21_ptg01_Web.indd
More informationAdvanced Operations Research Prof. G. Srinivasan Dept of Management Studies Indian Institute of Technology, Madras
Advanced Operations Research Prof. G. Srinivasan Dept of Management Studies Indian Institute of Technology, Madras Lecture 23 Minimum Cost Flow Problem In this lecture, we will discuss the minimum cost
More informationReinforcement Learning and Optimal Control. Chapter 1 Exact Dynamic Programming DRAFT
Reinforcement Learning and Optimal Control by Dimitri P. Bertsekas Massachusetts Institute of Technology Chapter 1 Exact Dynamic Programming DRAFT This is Chapter 1 of the draft textbook Reinforcement
More informationUNIT 2. Greedy Method GENERAL METHOD
UNIT 2 GENERAL METHOD Greedy Method Greedy is the most straight forward design technique. Most of the problems have n inputs and require us to obtain a subset that satisfies some constraints. Any subset
More informationCE 191: Civil and Environmental Engineering Systems Analysis. LEC 15 : DP Examples
CE 191: Civil and Environmental Engineering Systems Analysis LEC 15 : DP Examples Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 2014 Prof. Moura UC Berkeley
More information6.231 DYNAMIC PROGRAMMING LECTURE 3 LECTURE OUTLINE
6.21 DYNAMIC PROGRAMMING LECTURE LECTURE OUTLINE Deterministic finite-state DP problems Backward shortest path algorithm Forward shortest path algorithm Shortest path examples Alternative shortest path
More informationChapter wise Question bank
GOVERNMENT ENGINEERING COLLEGE - MODASA Chapter wise Question bank Subject Name Analysis and Design of Algorithm Semester Department 5 th Term ODD 2015 Information Technology / Computer Engineering Chapter
More informationProblem Set 2: Answers
Economics 623 J.R.Walker Page 1 Problem Set 2: Answers The problem set came from Michael A. Trick, Senior Associate Dean, Education and Professor Tepper School of Business, Carnegie Mellon University.
More informationOptimization Methods in Management Science
Problem Set Rules: Optimization Methods in Management Science MIT 15.053, Spring 2013 Problem Set 6, Due: Thursday April 11th, 2013 1. Each student should hand in an individual problem set. 2. Discussing
More informationMS-E2114 Investment Science Exercise 4/2016, Solutions
Capital budgeting problems can be solved based on, for example, the benet-cost ratio (that is, present value of benets per present value of the costs) or the net present value (the present value of benets
More informationDecision Theory: Value Iteration
Decision Theory: Value Iteration CPSC 322 Decision Theory 4 Textbook 9.5 Decision Theory: Value Iteration CPSC 322 Decision Theory 4, Slide 1 Lecture Overview 1 Recap 2 Policies 3 Value Iteration Decision
More informationOptimization Methods. Lecture 7: Sensitivity Analysis
5.093 Optimization Methods Lecture 7: Sensitivity Analysis Motivation. Questions z = min s.t. c x Ax = b Slide How does z depend globally on c? on b? How does z change locally if either b, c, A change?
More informationCHAPTER 5: DYNAMIC PROGRAMMING
CHAPTER 5: DYNAMIC PROGRAMMING Overview This chapter discusses dynamic programming, a method to solve optimization problems that involve a dynamical process. This is in contrast to our previous discussions
More informationCMPSCI 311: Introduction to Algorithms Second Midterm Practice Exam SOLUTIONS
CMPSCI 311: Introduction to Algorithms Second Midterm Practice Exam SOLUTIONS November 17, 2016. Name: ID: Instructions: Answer the questions directly on the exam pages. Show all your work for each question.
More information0/1 knapsack problem knapsack problem
1 (1) 0/1 knapsack problem. A thief robbing a safe finds it filled with N types of items of varying size and value, but has only a small knapsack of capacity M to use to carry the goods. More precisely,
More informationMore Advanced Single Machine Models. University at Buffalo IE661 Scheduling Theory 1
More Advanced Single Machine Models University at Buffalo IE661 Scheduling Theory 1 Total Earliness And Tardiness Non-regular performance measures Ej + Tj Early jobs (Set j 1 ) and Late jobs (Set j 2 )
More informationEE365: Risk Averse Control
EE365: Risk Averse Control Risk averse optimization Exponential risk aversion Risk averse control 1 Outline Risk averse optimization Exponential risk aversion Risk averse control Risk averse optimization
More informationTopics in Computational Sustainability CS 325 Spring 2016
Topics in Computational Sustainability CS 325 Spring 2016 Note to other teachers and users of these slides. Andrew would be delighted if you found this source material useful in giving your own lectures.
More informationLecture 5 January 30
EE 223: Stochastic Estimation and Control Spring 2007 Lecture 5 January 30 Lecturer: Venkat Anantharam Scribe: aryam Kamgarpour 5.1 Secretary Problem The problem set-up is explained in Lecture 4. We review
More informationLecture outline W.B.Powell 1
Lecture outline What is a policy? Policy function approximations (PFAs) Cost function approximations (CFAs) alue function approximations (FAs) Lookahead policies Finding good policies Optimizing continuous
More information56:171 Operations Research Midterm Exam Solutions October 22, 1993
56:171 O.R. Midterm Exam Solutions page 1 56:171 Operations Research Midterm Exam Solutions October 22, 1993 (A.) /: Indicate by "+" ="true" or "o" ="false" : 1. A "dummy" activity in CPM has duration
More informationEC316a: Advanced Scientific Computation, Fall Discrete time, continuous state dynamic models: solution methods
EC316a: Advanced Scientific Computation, Fall 2003 Notes Section 4 Discrete time, continuous state dynamic models: solution methods We consider now solution methods for discrete time models in which decisions
More informationSequential Decision Making
Sequential Decision Making Dynamic programming Christos Dimitrakakis Intelligent Autonomous Systems, IvI, University of Amsterdam, The Netherlands March 18, 2008 Introduction Some examples Dynamic programming
More informationDynamic Portfolio Choice II
Dynamic Portfolio Choice II Dynamic Programming Leonid Kogan MIT, Sloan 15.450, Fall 2010 c Leonid Kogan ( MIT, Sloan ) Dynamic Portfolio Choice II 15.450, Fall 2010 1 / 35 Outline 1 Introduction to Dynamic
More informationLEC 13 : Introduction to Dynamic Programming
CE 191: Civl and Environmental Engineering Systems Analysis LEC 13 : Introduction to Dynamic Programming Professor Scott Moura Civl & Environmental Engineering University of California, Berkeley Fall 2013
More informationSupply Chains: Planning with Dynamic Demand
Department of Industrial Engineering Supply Chains: Planning with Dynamic Demand Jayant Rajgopal, Ph.D., P.E. Department of Industrial Engineering University of Pittsburgh Pittsburgh, PA 15261 PRODUCTION
More information1) S = {s}; 2) for each u V {s} do 3) dist[u] = cost(s, u); 4) Insert u into a 2-3 tree Q with dist[u] as the key; 5) for i = 1 to n 1 do 6) Identify
CSE 3500 Algorithms and Complexity Fall 2016 Lecture 17: October 25, 2016 Dijkstra s Algorithm Dijkstra s algorithm for the SSSP problem generates the shortest paths in nondecreasing order of the shortest
More informationMS-E2114 Investment Science Lecture 4: Applied interest rate analysis
MS-E2114 Investment Science Lecture 4: Applied interest rate analysis A. Salo, T. Seeve Systems Analysis Laboratory Department of System Analysis and Mathematics Aalto University, School of Science Overview
More informationLecture 17: More on Markov Decision Processes. Reinforcement learning
Lecture 17: More on Markov Decision Processes. Reinforcement learning Learning a model: maximum likelihood Learning a value function directly Monte Carlo Temporal-difference (TD) learning COMP-424, Lecture
More informationAdvanced Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras
Advanced Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Lecture 21 Successive Shortest Path Problem In this lecture, we continue our discussion
More informationTHE TRAVELING SALESMAN PROBLEM FOR MOVING POINTS ON A LINE
THE TRAVELING SALESMAN PROBLEM FOR MOVING POINTS ON A LINE GÜNTER ROTE Abstract. A salesperson wants to visit each of n objects that move on a line at given constant speeds in the shortest possible time,
More information56:171 Operations Research Midterm Exam Solutions Fall 1994
56:171 Operations Research Midterm Exam Solutions Fall 1994 Possible Score A. True/False & Multiple Choice 30 B. Sensitivity analysis (LINDO) 20 C.1. Transportation 15 C.2. Decision Tree 15 C.3. Simplex
More informationThe Optimization Process: An example of portfolio optimization
ISyE 6669: Deterministic Optimization The Optimization Process: An example of portfolio optimization Shabbir Ahmed Fall 2002 1 Introduction Optimization can be roughly defined as a quantitative approach
More informationHomework solutions, Chapter 8
Homework solutions, Chapter 8 NOTE: We might think of 8.1 as being a section devoted to setting up the networks and 8.2 as solving them, but only 8.2 has a homework section. Section 8.2 2. Use Dijkstra
More informationLecture 2: Making Good Sequences of Decisions Given a Model of World. CS234: RL Emma Brunskill Winter 2018
Lecture 2: Making Good Sequences of Decisions Given a Model of World CS234: RL Emma Brunskill Winter 218 Human in the loop exoskeleton work from Steve Collins lab Class Structure Last Time: Introduction
More informationOptimization Methods in Management Science
Optimization Methods in Management Science MIT 15.053, Spring 013 Problem Set (Second Group of Students) Students with first letter of surnames I Z Due: February 1, 013 Problem Set Rules: 1. Each student
More informationHomework #4. CMSC351 - Spring 2013 PRINT Name : Due: Thu Apr 16 th at the start of class
Homework #4 CMSC351 - Spring 2013 PRINT Name : Due: Thu Apr 16 th at the start of class o Grades depend on neatness and clarity. o Write your answers with enough detail about your approach and concepts
More informationCS 188: Artificial Intelligence
CS 188: Artificial Intelligence Markov Decision Processes Dan Klein, Pieter Abbeel University of California, Berkeley Non-Deterministic Search 1 Example: Grid World A maze-like problem The agent lives
More informationPrice Discrimination As Portfolio Diversification. Abstract
Price Discrimination As Portfolio Diversification Parikshit Ghosh Indian Statistical Institute Abstract A seller seeking to sell an indivisible object can post (possibly different) prices to each of n
More informationMS-E2114 Investment Science Exercise 10/2016, Solutions
A simple and versatile model of asset dynamics is the binomial lattice. In this model, the asset price is multiplied by either factor u (up) or d (down) in each period, according to probabilities p and
More informationThe exam is closed book, closed calculator, and closed notes except your one-page crib sheet.
CS 188 Spring 2015 Introduction to Artificial Intelligence Midterm 1 You have approximately 2 hours and 50 minutes. The exam is closed book, closed calculator, and closed notes except your one-page crib
More informationRevenue Management Under the Markov Chain Choice Model
Revenue Management Under the Markov Chain Choice Model Jacob B. Feldman School of Operations Research and Information Engineering, Cornell University, Ithaca, New York 14853, USA jbf232@cornell.edu Huseyin
More informationMarkov Decision Processes
Markov Decision Processes Robert Platt Northeastern University Some images and slides are used from: 1. CS188 UC Berkeley 2. AIMA 3. Chris Amato Stochastic domains So far, we have studied search Can use
More information56:171 Operations Research Midterm Exam Solutions October 19, 1994
56:171 Operations Research Midterm Exam Solutions October 19, 1994 Possible Score A. True/False & Multiple Choice 30 B. Sensitivity analysis (LINDO) 20 C.1. Transportation 15 C.2. Decision Tree 15 C.3.
More informationLecture 1: Lucas Model and Asset Pricing
Lecture 1: Lucas Model and Asset Pricing Economics 714, Spring 2018 1 Asset Pricing 1.1 Lucas (1978) Asset Pricing Model We assume that there are a large number of identical agents, modeled as a representative
More informationIntroduction to Operations Research
Introduction to Operations Research Unit 1: Linear Programming Terminology and formulations LP through an example Terminology Additional Example 1 Additional example 2 A shop can make two types of sweets
More informationLecture 2 Dynamic Equilibrium Models: Three and More (Finite) Periods
Lecture 2 Dynamic Equilibrium Models: Three and More (Finite) Periods. Introduction In ECON 50, we discussed the structure of two-period dynamic general equilibrium models, some solution methods, and their
More informationDeterministic Dynamic Programming
Deterministic Dynamic Programming Dynamic programming is a technique that can be used to solve many optimization problems. In most applications, dynamic programming obtains solutions by working backward
More informationCS 188 Fall Introduction to Artificial Intelligence Midterm 1. ˆ You have approximately 2 hours and 50 minutes.
CS 188 Fall 2013 Introduction to Artificial Intelligence Midterm 1 ˆ You have approximately 2 hours and 50 minutes. ˆ The exam is closed book, closed notes except your one-page crib sheet. ˆ Please use
More informationMath Models of OR: More on Equipment Replacement
Math Models of OR: More on Equipment Replacement John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA December 2018 Mitchell More on Equipment Replacement 1 / 9 Equipment replacement
More informationOn solving multistage stochastic programs with coherent risk measures
On solving multistage stochastic programs with coherent risk measures Andy Philpott Vitor de Matos y Erlon Finardi z August 13, 2012 Abstract We consider a class of multistage stochastic linear programs
More informationCS134: Networks Spring Random Variables and Independence. 1.2 Probability Distribution Function (PDF) Number of heads Probability 2 0.
CS134: Networks Spring 2017 Prof. Yaron Singer Section 0 1 Probability 1.1 Random Variables and Independence A real-valued random variable is a variable that can take each of a set of possible values in
More informationProject Planning. Jesper Larsen. Department of Management Engineering Technical University of Denmark
Project Planning jesla@man.dtu.dk Department of Management Engineering Technical University of Denmark 1 Project Management Project Management is a set of techniques that helps management manage large-scale
More information1 Answers to the Sept 08 macro prelim - Long Questions
Answers to the Sept 08 macro prelim - Long Questions. Suppose that a representative consumer receives an endowment of a non-storable consumption good. The endowment evolves exogenously according to ln
More informationCS599: Algorithm Design in Strategic Settings Fall 2012 Lecture 4: Prior-Free Single-Parameter Mechanism Design. Instructor: Shaddin Dughmi
CS599: Algorithm Design in Strategic Settings Fall 2012 Lecture 4: Prior-Free Single-Parameter Mechanism Design Instructor: Shaddin Dughmi Administrivia HW out, due Friday 10/5 Very hard (I think) Discuss
More informationDynamic Programming and Reinforcement Learning
Dynamic Programming and Reinforcement Learning Daniel Russo Columbia Business School Decision Risk and Operations Division Fall, 2017 Daniel Russo (Columbia) Fall 2017 1 / 34 Supervised Machine Learning
More informationMaking Decisions. CS 3793 Artificial Intelligence Making Decisions 1
Making Decisions CS 3793 Artificial Intelligence Making Decisions 1 Planning under uncertainty should address: The world is nondeterministic. Actions are not certain to succeed. Many events are outside
More informationOptimal energy management and stochastic decomposition
Optimal energy management and stochastic decomposition F. Pacaud P. Carpentier J.P. Chancelier M. De Lara JuMP-dev workshop, 2018 ENPC ParisTech ENSTA ParisTech Efficacity 1/23 Motivation We consider a
More informationCS 188: Artificial Intelligence. Outline
C 188: Artificial Intelligence Markov Decision Processes (MDPs) Pieter Abbeel UC Berkeley ome slides adapted from Dan Klein 1 Outline Markov Decision Processes (MDPs) Formalism Value iteration In essence
More informationLecture 5. Varian, Ch. 8; MWG, Chs. 3.E, 3.G, and 3.H. 1 Summary of Lectures 1, 2, and 3: Production theory and duality
Lecture 5 Varian, Ch. 8; MWG, Chs. 3.E, 3.G, and 3.H Summary of Lectures, 2, and 3: Production theory and duality 2 Summary of Lecture 4: Consumption theory 2. Preference orders 2.2 The utility function
More information14.54 International Trade Lecture 3: Preferences and Demand
14.54 International Trade Lecture 3: Preferences and Demand 14.54 Week 2 Fall 2016 14.54 (Week 2) Preferences and Demand Fall 2016 1 / 29 Today s Plan 1 2 Utility maximization 1 2 3 4 Budget set Preferences
More informationYao s Minimax Principle
Complexity of algorithms The complexity of an algorithm is usually measured with respect to the size of the input, where size may for example refer to the length of a binary word describing the input,
More informationDynamic Appointment Scheduling in Healthcare
Brigham Young University BYU ScholarsArchive All Theses and Dissertations 2011-12-05 Dynamic Appointment Scheduling in Healthcare McKay N. Heasley Brigham Young University - Provo Follow this and additional
More informationMartingale Pricing Theory in Discrete-Time and Discrete-Space Models
IEOR E4707: Foundations of Financial Engineering c 206 by Martin Haugh Martingale Pricing Theory in Discrete-Time and Discrete-Space Models These notes develop the theory of martingale pricing in a discrete-time,
More information91.420/543: Artificial Intelligence UMass Lowell CS Fall 2010
91.420/543: Artificial Intelligence UMass Lowell CS Fall 2010 Lecture 17 & 18: Markov Decision Processes Oct 12 13, 2010 A subset of Lecture 9 slides from Dan Klein UC Berkeley Many slides over the course
More informationCredit Value Adjustment (Payo-at-Maturity contracts, Equity Swaps, and Interest Rate Swaps)
Credit Value Adjustment (Payo-at-Maturity contracts, Equity Swaps, and Interest Rate Swaps) Dr. Yuri Yashkir Dr. Olga Yashkir July 30, 2013 Abstract Credit Value Adjustment estimators for several nancial
More informationAN ALGORITHM FOR FINDING SHORTEST ROUTES FROM ALL SOURCE NODES TO A GIVEN DESTINATION IN GENERAL NETWORKS*
526 AN ALGORITHM FOR FINDING SHORTEST ROUTES FROM ALL SOURCE NODES TO A GIVEN DESTINATION IN GENERAL NETWORKS* By JIN Y. YEN (University of California, Berkeley) Summary. This paper presents an algorithm
More informationMarkov Decision Processes
Markov Decision Processes Robert Platt Northeastern University Some images and slides are used from: 1. CS188 UC Berkeley 2. RN, AIMA Stochastic domains Image: Berkeley CS188 course notes (downloaded Summer
More informationCS 188: Artificial Intelligence Fall 2011
CS 188: Artificial Intelligence Fall 2011 Lecture 9: MDPs 9/22/2011 Dan Klein UC Berkeley Many slides over the course adapted from either Stuart Russell or Andrew Moore 2 Grid World The agent lives in
More informationWhat is Greedy Approach? Control abstraction for Greedy Method. Three important activities
0-0-07 What is Greedy Approach? Suppose that a problem can be solved by a sequence of decisions. The greedy method has that each decision is locally optimal. These locally optimal solutions will finally
More informationChapter 15: Dynamic Programming
Chapter 15: Dynamic Programming Dynamic programming is a general approach to making a sequence of interrelated decisions in an optimum way. While we can describe the general characteristics, the details
More informationHigh Frequency Trading Strategy Based on Prex Trees
High Frequency Trading Strategy Based on Prex Trees Yijia Zhou, 05592862, Financial Mathematics, Stanford University December 11, 2010 1 Introduction 1.1 Goal I am an M.S. Finanical Mathematics student
More informationCS 188: Artificial Intelligence
CS 188: Artificial Intelligence Markov Decision Processes Dan Klein, Pieter Abbeel University of California, Berkeley Non Deterministic Search Example: Grid World A maze like problem The agent lives in
More informationRobust Dual Dynamic Programming
1 / 18 Robust Dual Dynamic Programming Angelos Georghiou, Angelos Tsoukalas, Wolfram Wiesemann American University of Beirut Olayan School of Business 31 May 217 2 / 18 Inspired by SDDP Stochastic optimization
More informationCS 188: Artificial Intelligence Spring Announcements
CS 188: Artificial Intelligence Spring 2011 Lecture 9: MDPs 2/16/2011 Pieter Abbeel UC Berkeley Many slides over the course adapted from either Dan Klein, Stuart Russell or Andrew Moore 1 Announcements
More informationCS 343: Artificial Intelligence
CS 343: Artificial Intelligence Markov Decision Processes II Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC
More informationCSE 417 Dynamic Programming (pt 2) Look at the Last Element
CSE 417 Dynamic Programming (pt 2) Look at the Last Element Reminders > HW4 is due on Friday start early! if you run into problems loading data (date parsing), try running java with Duser.country=US Duser.language=en
More informationCS 188 Fall Introduction to Artificial Intelligence Midterm 1. ˆ You have approximately 2 hours and 50 minutes.
CS 188 Fall 2013 Introduction to Artificial Intelligence Midterm 1 ˆ You have approximately 2 hours and 50 minutes. ˆ The exam is closed book, closed notes except your one-page crib sheet. ˆ Please use
More informationarxiv: v1 [q-fin.rm] 1 Jan 2017
Net Stable Funding Ratio: Impact on Funding Value Adjustment Medya Siadat 1 and Ola Hammarlid 2 arxiv:1701.00540v1 [q-fin.rm] 1 Jan 2017 1 SEB, Stockholm, Sweden medya.siadat@seb.se 2 Swedbank, Stockholm,
More informationJessie Jumpshot. Creating Value with Contingent Contracts
Jessie Jumpshot Creating Value with Contingent Contracts 1 BATNAS and Reservation Prices Jessie must get a TOTAL DEAL in expected monetary value at or in excess of alternative deal worth $2.1 M Salary
More informationEcon 172A - Slides from Lecture 7
Econ 172A Sobel Econ 172A - Slides from Lecture 7 Joel Sobel October 18, 2012 Announcements Be prepared for midterm room/seating assignments. Quiz 2 on October 25, 2012. (Duality, up to, but not including
More informationMarkov Decision Processes: Making Decision in the Presence of Uncertainty. (some of) R&N R&N
Markov Decision Processes: Making Decision in the Presence of Uncertainty (some of) R&N 16.1-16.6 R&N 17.1-17.4 Different Aspects of Machine Learning Supervised learning Classification - concept learning
More informationUGM Crash Course: Conditional Inference and Cutset Conditioning
UGM Crash Course: Conditional Inference and Cutset Conditioning Julie Nutini August 19 th, 2015 1 / 25 Conditional UGM 2 / 25 We know the value of one or more random variables i.e., we have observations,
More information