Speeding Up Exact Solutions of Interactive Dynamic Influence Diagrams Using Action Equivalence
|
|
|
- Jessie Wheeler
- 5 years ago
- Views:
Transcription
1 1 / 28 Speeding Up Exact Solutions of Interactive Dynamic Influence Diagrams Using Action Equivalence Yifeng Zeng Aalborg University, Denmark Prashant Doshi University of Georgia, USA
2 2 / 28 Outline Outline Motivation Background Interactive Influence Diagrams (I - ID) Interactive Dynamic Influence Diagrams (I - DID) Exact Method Behavioral Equivalence Action Equivalence Theoretical Results Experimental Results
3 3 / 28 Motivation State of the Art Exact solutions of I-DIDs up to 4 time horizons on standard problems (Doshi et al., JAAMAS 09) Multiagent Tiger example: S =2, Ω =6, A =3 Multiagent Machine example: S =4, Ω =2, A =4 Approximate solutions using model clustering (Zeng and Doshi, AAAI 07) Solves I-DIDs up to 8 time horizons Improved approximation using discriminative model updates (Doshi and Zeng, AAMAS 09) Solves I-DIDs up to time horizons Critical need for exact solutions over longer time horizons Provides optimal benchmarks for approximations
4 4 / 28 Background Interactive Influence Diagram (I - ID) (Doshi et al., JAAMAS 09) A generic level l I-ID for agent i situated with one other agent j Model Node: M j,l 1 Models of agent j at level l 1 Policy link: dashed arrow Distribution over agent j s actions given its models Beliefs on M j,l 1 : P(M j,l 1 s)
![5 / 28 Background Model Node Details of the Model Node Members of the model node Different chance nodes: solutions of models m j,l 1 Mod[Mj ] represents the](/docs-images/87/95409310/images/5-0.jpg "different models of agent j CPT of the chance node A j is a multiplexer Assumes the distribution of each of the action nodes (A 1 j, A2 j ) depending on the value of")
5 5 / 28 Background Model Node Details of the Model Node Members of the model node Different chance nodes: solutions of models m j,l 1 Mod[Mj ] represents the different models of agent j CPT of the chance node A j is a multiplexer Assumes the distribution of each of the action nodes (A 1 j, A2 j ) depending on the value of Mod[M j ]
6 6 / 28 Background Interactive Dynamic Influence Diagrams (I - DID) Interactive Dynamic Influence Diagram (I-DID) (Doshi et al., JAAMAS 09) A generic two time-slice level l I-DID for agent i Ri Ri A i t Aj t A i t+1 Aj t+1 S t S t+1 M j,l-1 t M j,l-1 t+1 O i t O i t+1
7 7 / 28 Background Model Update ink Semantics of Model Update ink The CPT of Mod[M t+1 j ] is an indicator function: τ(bj,l 1 t, at j, ot+1 j, b t+1 j,l 1 ) S t Mj,l-1 t Aj t Mj,l-1 t+1 Aj t+1 S t+1 Mod[Mj t+1 ] Mod[Mj t ] Oj t+1 Ai t mj,l-1t+1,1 Aj 1 mj,l-1 t,1 Aj 1 Oj 1 mj,l-1t+1,2 Aj 2 mj,l-1 t,2 Aj 2 Oj 2 mj,l-1t+1,3 Aj 3 mj,l-1t+1,4 Aj 4
8 8 / 28 Background Exact Solutions Computational Complexity Primary complexity of solving I-DIDs is due to the exponential number of models that must be solved over time At time step t: M 0 ( A Ω ) t Nested modeling adds to the complexity Multiple agents N+1 agent setting: (NM) l models (M is bounded number of models at each level)
9 9 / 28 Behavioral Equivalence Behavioral Equivalence Two models are behaviorally equivalent if they prescribe the same policy across the whole time horizon E.g., Expanded models at t=1 below exhibit same behaviors Actions (node labels): = isten O = Open left door = Open right door Observations (edge labels): G = Growl from left door = Growl from right door mj,l-1 1 mj,l-1 2 O t=0 G G t=1 G G t=3 G G
10 10 / 28 Behavioral Equivalence Build the policy graph m j,l-1 1 m j,l-1 2 O m j,l-1 3 G G G O G G O Merge (a)
11 11 / 28 Behavioral Equivalence Build the policy graph mj,l-1 1 mj,l-1 2 mj,l-1 3 mj,l-1 1 mj,l-1 2 mj,l-1 3 O time t=0 O G G G G G Merge G O O G G G G O O Merge (a) (b)
12 12 / 28 Behavioral Equivalence Build the policy graph mj,l-1 1 mj,l-1 2 mj,l-1 3 mj,l-1 1 mj,l-1 2 mj,l-1 3 O time t=0 O G G G G G Merge G O O G G G G O O Merge (a) mj,l-1 1 mj,l-1 2 mj,l-1 3 O time t=0 (b) G G O G G O (c)
13 13 / 28 Action Equivalence Action Equivalence Two models are actionally equivalent if they prescribe identical actions at a particular time step They differ in the entire behavior over the whole time horizon Actions (node labels): = isten O = Open left door = Open right door Observations (edge labels): G = Growl from left door = Growl from right door mj,l-1 1 mj,l-1 2 mj,l-1 3 G O G O time t=0 G G O
14 14 / 28 Action Equivalence Aggregate Action Equivalence Models Each class contains models prescribing an identical action at a particular time step E.g.: M t=0 j,l 1 = {Mt=0,1 j,l 1, Mt=0,2 j,l 1 } O time t=0 G G O G G O
15 Action Equivalence Model Update Issue The CPT of Mod[M t+1 j,l 1 ], is no longer deterministic (an indicator function) but probabilistic Modify indicator function: τ(bj,l 1 t, at j, ot+1 j, b t+1 j,l 1 ) O time t=0 G G O G G O (a) time t= O 15 / 28
16 16 / 28 Action Equivalence Revise CPT Pr(M t+1,p j,l 1 Mt,q j,l 1, at j, ot+1 j ) = m t+1 = = = j,l 1 Mt+1,p j,l 1 m t+1 j,l 1 Mt+1,p j,l 1 Pr(m t+1 j,l 1 Mt,q j,l 1, at j, ot+1 j ) Pr(m t+1 j,l 1,Mt,q j,l 1,at j,ot+1 j ) Pr(M t,q j,l 1,at j,ot+1 ) j M t+1,p Pr(m t+1 j,l 1,Mt,q j,l 1 mt j,l 1,at j,ot+1 )Pr(m j j,l 1 t at j,ot+1 ) j j,l 1 M t,q Pr(mj,l 1 t at j,ot+1 ) j j,l 1 m t+1 j,l 1 Mt+1,p j,l 1,mt j,l 1 Mt,q j,l 1 m t j,l 1 Mt,q j,l 1 b i (m t j,l 1 )τ(bt j,l 1,at j,ot+1 j,b t+1 j,l 1 ) b i (m t j,l 1 ) The probability is the proportion of the probability mass of individual models in a class
17 17 / 28 Action Equivalence Example O time t=0 G G O G G O (a)
18 18 / 28 Action Equivalence Example O time t= O G G G G O O G G G G O O (a) (b)
19 19 / 28 Action Equivalence Example O time t= O G G G G O O G G G G O O (a) (b) O time t=0,0.54 G,0.54,0.46,1.0 G,0.46 O G G O (c)
20 20 / 28 Action Equivalence Example O time t= O G G G G O O G G G G O O (a) (b) O time t= O,0.54 G,0.54,0.46,1.0 G,0.46,0.54,0.46 G,0.54, 1.0 G,0.46 O O G G,0.27,1.0 G,0.79,0.73 G,0.21, 1.0 O O (c) (d)
21 21 / 28 Theoretical Discussions Computational Savings Exact method: At most M 0 j,l 1 ( A j Ω j ) t models Behavioral equivalence: The number of behavioral equivalence classes over time Grow over time Action equivalence: A j model classes
22 Theoretical Discussions Complexity and Optimality - Behavioral Equivalence Proposition (Complexity of tree merge) Worst-case complexity of the procedure for merging policy trees to form a policy graph is O(( Ω j T 1 ) M0 j,l 1 ) where T is the horizon. Proposition (Optimality of Solutions) Model update using the policy graph preserves the I-DIDs solution. 22 / 28
23 23 / 28 Theoretical Discussions Optimality - Action Equivalence Proposition (Optimality) The predictive distribution over j s actions on aggregating model space due to action equivalence is preserved. I-DID solution optimality is preserved.
24 24 / 28 Experimental Results Comparison Experimental Results Exact method (Exact), action equivalence (Exact-AE), and behavioral equivalence (Exact-BE) Two standard domains: Multi-agent Tiger and Machine Maintenance Problems Average rewards obtained in the same amount of time Computational savings Model space Timing
25 Experimental Results Comparison Multiagent Tiger problem 11 evel 1 10 Average Reward Exact-BE Exact-AE Time(s) Exact-AE Exact-BE Exact Model classes evel Horizon Average Reward Exact-AE Exact-BE Exact Time(s) 25 / 28
26 Experimental Results Comparison Multiagent Machine problem 0.85 evel Average Reward Exact-BE Exact-AE Time(s) Exact-AE Exact-BE Exact Model Classes evel Horizon Average Reward Exact-AE Exact-BE Exact Time(s) 26 / 28
27 27 / 28 Experimental Results Comparison Run Times over Horizons evel 2 T Time (s) Exact-AE Exact-BE Exact Tiger MM Table: Aggregation using action equivalence scales significantly better to larger horizons. All experiments are run on a WinXP platform with a dual processor Xeon 2.0GHz and 2GB memory.
28 28 / 28 Summary Conclusions and Future Work Summary Using behavioral equivalence significantly reduces the model space Using action equivalence provides further reduction while preserving the solution optimality Further work Approximation using action equivalence Application on more scalable multiagent problems
Markov Decision Processes
Markov Decision Processes Ryan P. Adams COS 324 Elements of Machine Learning Princeton University We now turn to a new aspect of machine learning, in which agents take actions and become active in their
Robust Dual Dynamic Programming
1 / 18 Robust Dual Dynamic Programming Angelos Georghiou, Angelos Tsoukalas, Wolfram Wiesemann American University of Beirut Olayan School of Business 31 May 217 2 / 18 Inspired by SDDP Stochastic optimization
V. Lesser CS683 F2004
The value of information Lecture 15: Uncertainty - 6 Example 1: You consider buying a program to manage your finances that costs $100. There is a prior probability of 0.7 that the program is suitable in
Making Decisions. CS 3793 Artificial Intelligence Making Decisions 1
Making Decisions CS 3793 Artificial Intelligence Making Decisions 1 Planning under uncertainty should address: The world is nondeterministic. Actions are not certain to succeed. Many events are outside
Assessing Solvency by Brute Force is Computationally Tractable
O T Y H E H U N I V E R S I T F G Assessing Solvency by Brute Force is Computationally Tractable (Applying High Performance Computing to Actuarial Calculations) E D I N B U R M.Tucker@epcc.ed.ac.uk Assessing
Decision Trees with Minimum Average Depth for Sorting Eight Elements
Decision Trees with Minimum Average Depth for Sorting Eight Elements Hassan AbouEisha, Igor Chikalov, Mikhail Moshkov Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah
Introduction to Blockchain Technology
Introduction to Blockchain Technology Current Trends in Artificial Intelligence Volker Strobel PhD student @ IRIDIA 23 February 2017 Part I: Bitcoin: Idea, Basics, Technology Part II: Altcoins, Use cases,
A Multi-Stage Stochastic Programming Model for Managing Risk-Optimal Electricity Portfolios. Stochastic Programming and Electricity Risk Management
A Multi-Stage Stochastic Programming Model for Managing Risk-Optimal Electricity Portfolios SLIDE 1 Outline Multi-stage stochastic programming modeling Setting - Electricity portfolio management Electricity
Sequential Coalition Formation for Uncertain Environments
Sequential Coalition Formation for Uncertain Environments Hosam Hanna Computer Sciences Department GREYC - University of Caen 14032 Caen - France hanna@info.unicaen.fr Abstract In several applications,
Monte-Carlo Planning Look Ahead Trees. Alan Fern
Monte-Carlo Planning Look Ahead Trees Alan Fern 1 Monte-Carlo Planning Outline Single State Case (multi-armed bandits) A basic tool for other algorithms Monte-Carlo Policy Improvement Policy rollout Policy
Assembly systems with non-exponential machines: Throughput and bottlenecks
Nonlinear Analysis 69 (2008) 911 917 www.elsevier.com/locate/na Assembly systems with non-exponential machines: Throughput and bottlenecks ShiNung Ching, Semyon M. Meerkov, Liang Zhang Department of Electrical
To earn the extra credit, one of the following has to hold true. Please circle and sign.
CS 188 Fall 2018 Introduction to Artificial Intelligence Practice Midterm 1 To earn the extra credit, one of the following has to hold true. Please circle and sign. A I spent 2 or more hours on the practice
Stochastic Dual Dynamic Programming
1 / 43 Stochastic Dual Dynamic Programming Operations Research Anthony Papavasiliou 2 / 43 Contents [ 10.4 of BL], [Pereira, 1991] 1 Recalling the Nested L-Shaped Decomposition 2 Drawbacks of Nested Decomposition
POMDPs: Partially Observable Markov Decision Processes Advanced AI
POMDPs: Partially Observable Markov Decision Processes Advanced AI Wolfram Burgard Types of Planning Problems Classical Planning State observable Action Model Deterministic, accurate MDPs observable stochastic
Reinforcement Learning
Reinforcement Learning MDP March May, 2013 MDP MDP: S, A, P, R, γ, µ State can be partially observable: Partially Observable MDPs () Actions can be temporally extended: Semi MDPs (SMDPs) and Hierarchical
MDP Algorithms. Thomas Keller. June 20, University of Basel
MDP Algorithms Thomas Keller University of Basel June 20, 208 Outline of this lecture Markov decision processes Planning via determinization Monte-Carlo methods Monte-Carlo Tree Search Heuristic Search
Monte-Carlo Planning: Introduction and Bandit Basics. Alan Fern
Monte-Carlo Planning: Introduction and Bandit Basics Alan Fern 1 Large Worlds We have considered basic model-based planning algorithms Model-based planning: assumes MDP model is available Methods we learned
CHAPTER 6 CRASHING STOCHASTIC PERT NETWORKS WITH RESOURCE CONSTRAINED PROJECT SCHEDULING PROBLEM
CHAPTER 6 CRASHING STOCHASTIC PERT NETWORKS WITH RESOURCE CONSTRAINED PROJECT SCHEDULING PROBLEM 6.1 Introduction Project Management is the process of planning, controlling and monitoring the activities
Introducing Communication in Dis-POMDPs with Locality of Interaction
Web Intelligence and Agent Systems: An International Journal 0 (2009) 1 0 1 IOS Press Introducing Communication in Dis-POMDPs with Locality of Interaction Makoto Tasaki a,, Yuichi Yabu a, Yuki Iwanari
Subject : Computer Science. Paper: Machine Learning. Module: Decision Theory and Bayesian Decision Theory. Module No: CS/ML/10.
e-pg Pathshala Subject : Computer Science Paper: Machine Learning Module: Decision Theory and Bayesian Decision Theory Module No: CS/ML/0 Quadrant I e-text Welcome to the e-pg Pathshala Lecture Series
Square Grid Benchmarks for Source-Terminal Network Reliability Estimation
Square Grid Benchmarks for Source-Terminal Network Reliability Estimation Roger Paredes Leonardo Duenas-Osorio Rice University, Houston TX, USA. 03/2018 This document describes a synthetic benchmark data
Optimal Satisficing Tree Searches
Optimal Satisficing Tree Searches Dan Geiger and Jeffrey A. Barnett Northrop Research and Technology Center One Research Park Palos Verdes, CA 90274 Abstract We provide an algorithm that finds optimal
Topics in Computational Sustainability CS 325 Spring 2016
Topics in Computational Sustainability CS 325 Spring 2016 Note to other teachers and users of these slides. Andrew would be delighted if you found this source material useful in giving your own lectures.
Multiagent Systems. Multiagent Systems General setting Division of Resources Task Allocation Resource Allocation. 13.
Multiagent Systems July 16, 2014 13. Bargaining Multiagent Systems 13. Bargaining B. Nebel, C. Becker-Asano, S. Wölfl Albert-Ludwigs-Universität Freiburg July 16, 2014 13.1 General setting 13.2 13.3 13.4
CS221 / Spring 2018 / Sadigh. Lecture 7: MDPs I
CS221 / Spring 2018 / Sadigh Lecture 7: MDPs I cs221.stanford.edu/q Question How would you get to Mountain View on Friday night in the least amount of time? bike drive Caltrain Uber/Lyft fly CS221 / Spring
Lecture 7: MDPs I. Question. Course plan. So far: search problems. Uncertainty in the real world
Lecture 7: MDPs I cs221.stanford.edu/q Question How would you get to Mountain View on Friday night in the least amount of time? bike drive Caltrain Uber/Lyft fly CS221 / Spring 2018 / Sadigh CS221 / Spring
Markov Decision Processes
Markov Decision Processes Robert Platt Northeastern University Some images and slides are used from: 1. CS188 UC Berkeley 2. AIMA 3. Chris Amato Stochastic domains So far, we have studied search Can use
CS221 / Spring 2018 / Sadigh. Lecture 9: Games I
CS221 / Spring 2018 / Sadigh Lecture 9: Games I Course plan Search problems Markov decision processes Adversarial games Constraint satisfaction problems Bayesian networks Reflex States Variables Logic
DECISION ANALYSIS. Decision often must be made in uncertain environments. Examples:
DECISION ANALYSIS Introduction Decision often must be made in uncertain environments. Examples: Manufacturer introducing a new product in the marketplace. Government contractor bidding on a new contract.
The exam is closed book, closed calculator, and closed notes except your three crib sheets.
CS 188 Spring 2016 Introduction to Artificial Intelligence Final V2 You have approximately 2 hours and 50 minutes. The exam is closed book, closed calculator, and closed notes except your three crib sheets.
Lecture 9: Games I. Course plan. A simple game. Roadmap. Machine learning. Example: game 1
Lecture 9: Games I Course plan Search problems Markov decision processes Adversarial games Constraint satisfaction problems Bayesian networks Reflex States Variables Logic Low-level intelligence Machine
A selection of MAS learning techniques based on RL
A selection of MAS learning techniques based on RL Ann Nowé 14/11/12 Herhaling titel van presentatie 1 Content Single stage setting Common interest (Claus & Boutilier, Kapetanakis&Kudenko) Conflicting
Multi-armed bandit problems
Multi-armed bandit problems Stochastic Decision Theory (2WB12) Arnoud den Boer 13 March 2013 Set-up 13 and 14 March: Lectures. 20 and 21 March: Paper presentations (Four groups, 45 min per group). Before
All Investors are Risk-averse Expected Utility Maximizers. Carole Bernard (UW), Jit Seng Chen (GGY) and Steven Vanduffel (Vrije Universiteit Brussel)
All Investors are Risk-averse Expected Utility Maximizers Carole Bernard (UW), Jit Seng Chen (GGY) and Steven Vanduffel (Vrije Universiteit Brussel) First Name: Waterloo, April 2013. Last Name: UW ID #:
Monte-Carlo tree search for multi-player, no-limit Texas hold'em poker. Guy Van den Broeck
Monte-Carlo tree search for multi-player, no-limit Texas hold'em poker Guy Van den Broeck Should I bluff? Deceptive play Should I bluff? Is he bluffing? Opponent modeling Should I bluff? Is he bluffing?
Dynamic Risk Management in Electricity Portfolio Optimization via Polyhedral Risk Functionals
Dynamic Risk Management in Electricity Portfolio Optimization via Polyhedral Risk Functionals A. Eichhorn and W. Römisch Humboldt-University Berlin, Department of Mathematics, Germany http://www.math.hu-berlin.de/~romisch
CS188 Spring 2012 Section 4: Games
CS188 Spring 2012 Section 4: Games 1 Minimax Search In this problem, we will explore adversarial search. Consider the zero-sum game tree shown below. Trapezoids that point up, such as at the root, represent
DECISION ANALYSIS. (Hillier & Lieberman Introduction to Operations Research, 8 th edition)
DECISION ANALYSIS (Hillier & Lieberman Introduction to Operations Research, 8 th edition) Introduction Decision often must be made in uncertain environments Examples: Manufacturer introducing a new product
AMERICAN AND EUROPEAN MEAN-REVERSION OPTION USING TRINOMIAL LATTICES
ch10(1)_4559.qxd 9/9/05 3:47 PM Page 434 434 SOFTWARE APPLICATIONS FIGURE 10.35 Ten-Step Binomial Lattice Comparison Result AMERICAN AND EUROPEAN MEAN-REVERSION OPTION USING TRINOMIAL LATTICES The Mean-Reversion
6 -AL- ONE MACHINE SEQUENCING TO MINIMIZE MEAN FLOW TIME WITH MINIMUM NUMBER TARDY. Hamilton Emmons \,«* Technical Memorandum No. 2.
li. 1. 6 -AL- ONE MACHINE SEQUENCING TO MINIMIZE MEAN FLOW TIME WITH MINIMUM NUMBER TARDY f \,«* Hamilton Emmons Technical Memorandum No. 2 May, 1973 1 il 1 Abstract The problem of sequencing n jobs on
Unparalleled Performance, Agility and Security for NSE
white paper Intel Xeon and Intel Xeon Scalable Processor Family Financial Services Unparalleled Performance, Agility and Security for NSE The latest Intel Xeon processor platform provides new levels of
High Performance Risk Aggregation: Addressing the Data Processing Challenge the Hadoop MapReduce Way
High Performance Risk Aggregation: Addressing the Data Processing Challenge the Hadoop MapReduce Way A. Rau-Chaplin, B. Varghese 1, Z. Yao Faculty of Computer Science, Dalhousie University Halifax, Nova
Casino gambling problem under probability weighting
Casino gambling problem under probability weighting Sang Hu National University of Singapore Mathematical Finance Colloquium University of Southern California Jan 25, 2016 Based on joint work with Xue
All Investors are Risk-averse Expected Utility Maximizers
All Investors are Risk-averse Expected Utility Maximizers Carole Bernard (UW), Jit Seng Chen (GGY) and Steven Vanduffel (Vrije Universiteit Brussel) AFFI, Lyon, May 2013. Carole Bernard All Investors are
Logic and Artificial Intelligence Lecture 24
Logic and Artificial Intelligence Lecture 24 Eric Pacuit Currently Visiting the Center for Formal Epistemology, CMU Center for Logic and Philosophy of Science Tilburg University ai.stanford.edu/ epacuit
An Algorithm for Distributing Coalitional Value Calculations among Cooperating Agents
An Algorithm for Distributing Coalitional Value Calculations among Cooperating Agents Talal Rahwan and Nicholas R. Jennings School of Electronics and Computer Science, University of Southampton, Southampton
An Empirical Study of Optimization for Maximizing Diffusion in Networks
An Empirical Study of Optimization for Maximizing Diffusion in Networks Kiyan Ahmadizadeh Bistra Dilkina, Carla P. Gomes, Ashish Sabharwal Cornell University Institute for Computational Sustainability
Multistage Stochastic Programming
IE 495 Lecture 21 Multistage Stochastic Programming Prof. Jeff Linderoth April 16, 2003 April 16, 2002 Stochastic Programming Lecture 21 Slide 1 Outline HW Fixes Multistage Stochastic Programming Modeling
Foundations of Artificial Intelligence
Foundations of Artificial Intelligence 44. Monte-Carlo Tree Search: Introduction Thomas Keller Universität Basel May 27, 2016 Board Games: Overview chapter overview: 41. Introduction and State of the Art
Overview. ICE: Iterative Combinatorial Exchanges. Combinatorial Auctions. Motivating Domains. Exchange Example 1. Benjamin Lubin
Overview ICE: Iterative Combinatorial Exchanges Benjamin Lubin In Collaboration with David Parkes and Adam Juda Early work Giro Cavallo, Jeff Shneidman, Hassan Sultan, CS286r Spring 2004 Introduction ICE
Monte-Carlo Planning: Introduction and Bandit Basics. Alan Fern
Monte-Carlo Planning: Introduction and Bandit Basics Alan Fern 1 Large Worlds We have considered basic model-based planning algorithms Model-based planning: assumes MDP model is available Methods we learned
Behavioral Equilibrium and Evolutionary Dynamics
Financial Markets: Behavioral Equilibrium and Evolutionary Dynamics Thorsten Hens 1, 5 joint work with Rabah Amir 2 Igor Evstigneev 3 Klaus R. Schenk-Hoppé 4, 5 1 University of Zurich, 2 University of
Asset-Liability Management
Asset-Liability Management John Birge University of Chicago Booth School of Business JRBirge INFORMS San Francisco, Nov. 2014 1 Overview Portfolio optimization involves: Modeling Optimization Estimation
Decision making in the presence of uncertainty
CS 2750 Foundations of AI Lecture 20 Decision making in the presence of uncertainty Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square Decision-making in the presence of uncertainty Computing the probability
PARELLIZATION OF DIJKSTRA S ALGORITHM: COMPARISON OF VARIOUS PRIORITY QUEUES
PARELLIZATION OF DIJKSTRA S ALGORITHM: COMPARISON OF VARIOUS PRIORITY QUEUES WIKTOR JAKUBIUK, KESHAV PURANMALKA 1. Introduction Dijkstra s algorithm solves the single-sourced shorest path problem on a
Sum-Product: Message Passing Belief Propagation
Sum-Product: Message Passing Belief Propagation 40-956 Advanced Topics in AI: Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2015 All single-node marginals If we need the
Progressive Hedging for Multi-stage Stochastic Optimization Problems
Progressive Hedging for Multi-stage Stochastic Optimization Problems David L. Woodruff Jean-Paul Watson Graduate School of Management University of California, Davis Davis, CA 95616, USA dlwoodruff@ucdavis.edu
Roll No. :... Invigilator s Signature :.. CS/B.TECH(IT)/SEM-5/M(CS)-511/ OPERATIONS RESEARCH AND OPTIMIZATION TECHNIQUES
Name : Roll No. :.... Invigilator s Signature :.. CS/B.TECH(IT)/SEM-5/M(CS)-511/2011-12 2011 OPERATIONS RESEARCH AND OPTIMIZATION TECHNIQUES Time Allotted : 3 Hours Full Marks : 70 The figures in the margin
Dynamic Programming: An overview. 1 Preliminaries: The basic principle underlying dynamic programming
Dynamic Programming: An overview These notes summarize some key properties of the Dynamic Programming principle to optimize a function or cost that depends on an interval or stages. This plays a key role
Risk aversion in multi-stage stochastic programming: a modeling and algorithmic perspective
Risk aversion in multi-stage stochastic programming: a modeling and algorithmic perspective Tito Homem-de-Mello School of Business Universidad Adolfo Ibañez, Santiago, Chile Joint work with Bernardo Pagnoncelli
Decision Networks (Influence Diagrams) CS 486/686: Introduction to Artificial Intelligence
Decision Networks (Influence Diagrams) CS 486/686: Introduction to Artificial Intelligence 1 Outline Decision Networks Computing Policies Value of Information 2 Introduction Decision networks (aka influence
IEOR E4004: Introduction to OR: Deterministic Models
IEOR E4004: Introduction to OR: Deterministic Models 1 Dynamic Programming Following is a summary of the problems we discussed in class. (We do not include the discussion on the container problem or the
Fibonacci Heaps Y Y o o u u c c an an s s u u b b m miitt P P ro ro b blle e m m S S et et 3 3 iin n t t h h e e b b o o x x u u p p fro fro n n tt..
Fibonacci Heaps You You can can submit submit Problem Problem Set Set 3 in in the the box box up up front. front. Outline for Today Review from Last Time Quick refresher on binomial heaps and lazy binomial
2017 Annual General Meeting 2 May 2017
2017 Annual General Meeting 2 May 2017 Forward Looking Statements This presentation contains forward-looking statements that reflect GrandVision s current views with respect to future events and financial
ECSEL CALLS 2017 YVES GIGASE
ECSEL CALLS 2017 YVES GIGASE RESULTS OF CALLS Calls Beneficiaries H2020 costs EU Funding National Funding ESIF 2017-1-IA 183 469,322,749 110,067,691 105,233,389 3,687,541 2017-2-RIA 268 215,680,946 63,124,661
Handout 4: Deterministic Systems and the Shortest Path Problem
SEEM 3470: Dynamic Optimization and Applications 2013 14 Second Term Handout 4: Deterministic Systems and the Shortest Path Problem Instructor: Shiqian Ma January 27, 2014 Suggested Reading: Bertsekas
Project Risk Analysis and Management Exercises (Part II, Chapters 6, 7)
Project Risk Analysis and Management Exercises (Part II, Chapters 6, 7) Chapter II.6 Exercise 1 For the decision tree in Figure 1, assume Chance Events E and F are independent. a) Draw the appropriate
Valuation of Options: Theory
Valuation of Options: Theory Valuation of Options:Theory Slide 1 of 49 Outline Payoffs from options Influences on value of options Value and volatility of asset ; time available Basic issues in valuation:
Complex Decisions. Sequential Decision Making
Sequential Decision Making Outline Sequential decision problems Value iteration Policy iteration POMDPs (basic concepts) Slides partially based on the Book "Reinforcement Learning: an introduction" by
Essays on Some Combinatorial Optimization Problems with Interval Data
Essays on Some Combinatorial Optimization Problems with Interval Data a thesis submitted to the department of industrial engineering and the institute of engineering and sciences of bilkent university
Intelligent Systems (AI-2)
Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 9 Sep, 28, 2016 Slide 1 CPSC 422, Lecture 9 An MDP Approach to Multi-Category Patient Scheduling in a Diagnostic Facility Adapted from: Matthew
MARKET DYNAMICS IN EDGEWORTH EXCHANGE STEVEN GJERSTAD. Department of Economics McClelland Hall 401 University of Arizona Tucson, AZ 85721
MARKET DYNAMICS IN EDGEWORTH EXCHANGE STEVEN GJERSTAD Department of Economics McClelland Hall 1 University of Arizona Tucson, AZ 571 January 19, Abstract Edgeworth exchange is the fundamental general equilibrium
Risk Management for Chemical Supply Chain Planning under Uncertainty
for Chemical Supply Chain Planning under Uncertainty Fengqi You and Ignacio E. Grossmann Dept. of Chemical Engineering, Carnegie Mellon University John M. Wassick The Dow Chemical Company Introduction
Monte-Carlo Planning: Basic Principles and Recent Progress
Monte-Carlo Planning: Basic Principles and Recent Progress Alan Fern School of EECS Oregon State University Outline Preliminaries: Markov Decision Processes What is Monte-Carlo Planning? Uniform Monte-Carlo
w w w. I C A o r g
w w w. I C A 2 0 1 4. o r g Multi-State Microeconomic Model for Pricing and Reserving a disability insurance policy over an arbitrary period Benjamin Schannes April 4, 2014 Some key disability statistics:
A Combination of Different Resource Management Policies in a Multi-Project Environment
International Conference on Industrial Engineering and Systems Management IESM 2013 October 28 - October 30 RABAT - MOROCCO A Combination of Different Resource Management Policies in a Multi-Project Environment
Valuation of performance-dependent options in a Black- Scholes framework
Valuation of performance-dependent options in a Black- Scholes framework Thomas Gerstner, Markus Holtz Institut für Numerische Simulation, Universität Bonn, Germany Ralf Korn Fachbereich Mathematik, TU
INTERPRETING OPTION VOLATILITY
VOLUME NO. 5 INTERPRETING OPTION VOLATILITY This issue of Derivations will unravel some of the complexities of volatility and the role volatility plays in determining the price of an option. We will show
Introduction to Decision Making. CS 486/686: Introduction to Artificial Intelligence
Introduction to Decision Making CS 486/686: Introduction to Artificial Intelligence 1 Outline Utility Theory Decision Trees 2 Decision Making Under Uncertainty I give a robot a planning problem: I want
XWIN CryptoBet SUMMARY
XWIN CryptoBet SUMMARY XWIN is a transparent betting platform for sporting and other events with revenue sharing using the decentralized smart contracts on the Ethereum blockchain. The unprecedented development
CS 5522: Artificial Intelligence II
CS 5522: Artificial Intelligence II Uncertainty and Utilities Instructor: Alan Ritter Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley. All materials available at
Revenue optimization in AdExchange against strategic advertisers
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
G5212: Game Theory. Mark Dean. Spring 2017
G5212: Game Theory Mark Dean Spring 2017 Modelling Dynamics Up until now, our games have lacked any sort of dynamic aspect We have assumed that all players make decisions at the same time Or at least no
Binary Decision Diagrams
Binary Decision Diagrams Hao Zheng Department of Computer Science and Engineering University of South Florida Tampa, FL 33620 Email: zheng@cse.usf.edu Phone: (813)974-4757 Fax: (813)974-5456 Hao Zheng
Advanced Engineering Project Management Dr. Nabil I. El Sawalhi Assistant professor of Construction Management
Advanced Engineering Project Management Dr. Nabil I. El Sawalhi Assistant professor of Construction Management 1 Decision trees Decision trees are tools for classification and prediction. 2 Decision Trees
Martingale Pricing Theory in Discrete-Time and Discrete-Space Models
IEOR E4707: Foundations of Financial Engineering c 206 by Martin Haugh Martingale Pricing Theory in Discrete-Time and Discrete-Space Models These notes develop the theory of martingale pricing in a discrete-time,
Impact of Imperfect Information on the Optimal Exercise Strategy for Warrants
Impact of Imperfect Information on the Optimal Exercise Strategy for Warrants April 2008 Abstract In this paper, we determine the optimal exercise strategy for corporate warrants if investors suffer from
Designing efficient market pricing mechanisms
Designing efficient market pricing mechanisms Volodymyr Kuleshov Gordon Wilfong Department of Mathematics and School of Computer Science, McGill Universty Algorithms Research, Bell Laboratories August
Implementation of a Perfectly Secure Distributed Computing System
Implementation of a Perfectly Secure Distributed Computing System Rishi Kacker and Matt Pauker Stanford University {rkacker,mpauker}@cs.stanford.edu Abstract. The increased interest in financially-driven
CMSC 441: Design & Analysis of Algorithms
CMSC 441: Design & Analysis of Algorithms Hillol Kargupta http://www.cs.umbc.edu/~hillol/ hillol@cs.umbc.edu Today s Topics Amortized analysis April 19, 2011 CMSC 641 2 Amortized Analysis Aggregate Method
Global Joint Distribution Factorizes into Local Marginal Distributions on Tree-Structured Graphs
Teaching Note October 26, 2007 Global Joint Distribution Factorizes into Local Marginal Distributions on Tree-Structured Graphs Xinhua Zhang Xinhua.Zhang@anu.edu.au Research School of Information Sciences
Reasoning About Others: Representing and Processing Infinite Belief Hierarchies
Reasoning About Others: Representing and Processing Infinite Belief Hierarchies Sviatoslav Brainov and Tuomas Sandholm Department of Computer Science Washington University St Louis, MO 63130 {brainov,
APPLICATION DETERMINATION OF CREDIT FEASIBILITY IN SHARIA COOPERATIVE WITH C4.5 ALGORITHM
APPLICATION DETERMINATION OF CREDIT FEASIBILITY IN SHARIA COOPERATIVE WITH C4.5 ALGORITHM Siti Masripah AMIK BSI Jakarta Jl. RS. Fatmawati No. 24 Pondok Labu in South Jakarta email: siti.stm@bsi.ac.id
A theoretical examination of tax evasion among the self-employed
Theoretical and Applied Economics FFet al Volume XXIII (2016), No. 1(606), Spring, pp. 119-128 A theoretical examination of tax evasion among the self-employed Dennis BARBER III Armstrong State University,
March 5, 2010 To the Workshop Organizers or Whom it May Concern, We are looking forward to attending your workshop to discuss our recent work on probabilistic pursuit of adversarial evaders. Attached we
Binary Decision Diagrams
Binary Decision Diagrams Hao Zheng Department of Computer Science and Engineering University of South Florida Tampa, FL 33620 Email: zheng@cse.usf.edu Phone: (813)974-4757 Fax: (813)974-5456 Hao Zheng
FIGURE A1.1. Differences for First Mover Cutoffs (Round one to two) as a Function of Beliefs on Others Cutoffs. Second Mover Round 1 Cutoff.
APPENDIX A. SUPPLEMENTARY TABLES AND FIGURES A.1. Invariance to quantitative beliefs. Figure A1.1 shows the effect of the cutoffs in round one for the second and third mover on the best-response cutoffs
Economic decision analysis: Concepts and applications
Economic decision analysis: Concepts and applications Jeffrey M. Keisler Stockholm, 23 May 2016 My background and this work Education in DA and Economics Government and industry consulting Portfolio DA
Cooperative Games with Monte Carlo Tree Search
Int'l Conf. Artificial Intelligence ICAI'5 99 Cooperative Games with Monte Carlo Tree Search CheeChian Cheng and Norman Carver Department of Computer Science, Southern Illinois University, Carbondale,
Predicting Agent Strategy Mix of Evolving Populations
Predicting Agent Strategy Mix of Evolving Populations Sabyasachi Saha Department of Math & Computer Science The University of Tulsa sabyasachi-saha@utulsa.edu Sandip Sen Department of Math & Computer Science