Foundations of Machine Learning II TP1: Entropy
|
|
- Robert Johns
- 5 years ago
- Views:
Transcription
1 Foundatons of Machne Learnng II TP1: Entropy Gullaume Charpat (Teacher) & Gaétan Marceau Caron (Scrbe) Problem 1 (Gbbs nequalty). Let p and q two probablty measures over a fnte alphabet X. Prove that KL(p q) 0 Hnt: for a concave functon f and a random varable X, we have the Jensen s nequalty E f(x)] f (EX]). ln s a strctly concave functon. : We start by revewng some useful results from Boyd and Vandenberghe, 004; Cover and Thomas, 006. Defnton 1. A functon f : R n R s convex f dom f s a convex set and f for all x, y dom f and θ 0, 1], we have f(θx + (1 θ)y) θf(x) + (1 θ)f(y) (1) A functon s strctly convex when the equalty holds f θ = 0 or θ = 1. Also, a functon f s concave s f s convex. Provng that a gven functon f s convex s usually hard wth the prevous defnton. When f s twce-dfferentable, t s easer to use the second-order condton: Proposton 1. Let f be a twce-dfferentable functon, that s, ts Hessan or second-dervatve f exsts at each pont n dom f, whch s open. Then, f s convex f dom f s a convex set and f s postve semdefnte,.e., for all x dom f, we have x f x 0. Moreover, the functon s strctly convex f f s postve defnte. Proof. Use Taylor expanson (cf. Cover and Thomas, 006, p.6 for the unvarate case) By applyng ths result, we easly fnd that ln s a strctly concave functon. Now, we state the Jensen s nequalty requred n the proof of the Gbb s nequalty. Theorem. If f s a convex functon and X s a random varable, then gcharpa/machnelearnngcourse/ E f(x)] f(e X]) () 1
2 Proof. Inducton (cf. Cover and Thomas, 006, p.7) Fnally, we recall the defnton of the Kullback-Lebler dvergence: Defnton. Let p and q be two dscrete probablty dstrbutons over an alphabet X. The Kullback-Lebler dvergence s defned as: KL(p q) = x X p(x) ln p(x) q(x) (3) wth the conventon that 0 ln 0 0 = 0, 0 ln 0 b = 0 and 0 ln a 0 = We fnsh wth the proof gven by Cover and Thomas, 006, p.8. Theorem 3 (Gbb s Inequalty). Let p and q be two dscrete probablty dstrbutons over an alphabet X. The Kullback-Lebler dvergence has the followng property: KL(p q) 0 (4) wth equalty f p(x) = q(x) for all x X. Proof. Let A = {x : p(x) > 0}, then we have KL(p q) = x A = x A ln x A p(x) ln p(x) q(x) p(x) ln q(x) p(x) p(x) q(x) p(x) (5) (6) (7) = ln q(x) (8) x A ln q(x) (9) x X = ln 1 (10) = 0 (11) Snce ln s strctly concave, we have equalty n eq. (7) f q(x) p(x) s constant,.e., p(x) = cq(x). Then, we have equalty n eq. (9) f x A q(x) = x X q(x). Fnally, wth both equaltes, we have that c = 1. Problem (Evdence Lower bound (ELBO)). Prove the followng nequalty 1 : ln p(d) E θ β ln p(d θ)] + KL(β α) (1) 1 Further nformaton can be found at
3 where D s a dataset, p(d) s the probablty of the dataset, p(d θ) s the lkelhood probablty of the dataset gven the model parameters θ, β s a dstrbuton over the model parameters approxmatng the posteror dstrbuton π(θ) := p(θ D) and α s the pror dstrbuton over the model parameters. (a) Wrte down the natural logarthm of the Bayes rule n an expanded form: π(θ) = p(d θ)α(θ) p(d) (13) By applyng the propertes of the logarthm, we obtan: 0 = ln p(d θ) + ln α(θ) ln p(d) ln π(θ) (14) (b) Introduce a new densty functon β and rewrte the expresson n terms of expectaton w.r.t. β 0 = β(θ) (ln p(d θ) + ln α(θ) ln p(d) ln π(θ)) (15) = β(θ) ln p(d θ) + β(θ) ln α(θ) β(θ) ln p(d) (16) + β(θ) ln β(θ) β(θ) ln β(θ) β(θ) ln π(θ) (17) = β(θ) ln p(d θ) ln p(d) + β(θ) ln α(θ) β(θ) + β(θ) ln β(θ) (18) π(θ) = E β ln p(d θ)] ln p(d) KL(β α) + KL(β π) (19) whch mples ln p(d) = E β ln p(d θ)] + KL(β α) KL(β π) (0) (c) Use the Gbbs nequalty and wrte down the ELBO ln p(d) E β ln p(d θ)] + KL(β α) (1) (d) Interpret the ELBO n a machne learnng framework cf. Varatonal nference Bshop, 006, p.46 Problem 3 (Entropy). Compute the dfferental entropy of the followng dstrbutons: (a) unvarate Normal dstrbuton N (x µ, σ ) = 1 πσ exp ] (x µ) σ () 3
4 we obtan By takng the natural logarthm of the normal dstrbuton, ln N (x µ, σ ) = ln πσ and wth the defnton of the dfferental entropy, we have (x µ) σ (3) Ent N ( µ, σ ) ] = E ln N (x µ, σ ) ] (4) = E ln ] πσ + 1 σ E (x µ) ] (5) = ln πσ + 1 (6) = ln πeσ (7) (b) multvarate Normal dstrbuton 1 N (x µ, C) = (π)d C exp 1 ] (x µ) C 1 (x µ) (8) where x, µ R d and C s a covarance matrx (assumed to be symmetrc postve-defnte). By takng the natural logarthm, we obtan ln N (x µ, C) = d ln(π) 1 ln C 1 (x µ)t C 1 (x µ) (9) and wth the defnton of the dfferental entropy, we have Ent N ( µ, σ ) ] = E ln N (x µ, σ ) ] (30) d = E ln(π) + 1 ln C + 1 ] (x µ)t C 1 (x µ) (31) = d ln(π) + 1 ln C + 1 E (x µ) T C 1 (x µ) ] (3) = d ln(π) + 1 ln C + d (33) = d (ln(π) + 1) + 1 ln C (34) = ln (πe) d C (35) Note that we have used to followng dentty for eq. (3): E (x µ) T C 1 (x µ) ] = E tr((x µ) T C 1 (x µ)) ] (36) = E tr((c 1 (x µ)(x µ) T )) ] (37) = tr(c 1 E (x µ)(x µ) T ] ) (38) = tr(c 1 C) (39) = d (40) 4
5 where tr s the trace operator. In ths dentty, we use the fact that the trace of a scalar s equal to the scalar. Then, we use a well-known property of the trace for cyclng the varables. Fnally, snce tr and E are lnear operators, we can swtch them. Problem 4 (Mutual nformaton). We are nterested n computng the mutual nformaton between a multvarate Normal dstrbuton β = N (x µ, C) where x, µ R d and a product of dentcal unvarate Normal dstrbutons α = d =1 N (x µ, σ). (a) Express the KL dvergence n terms of entropy and expectaton w.r.t. β ] β(x) KL(β α) = β(x) ln (41) α(x) = β(x) ln α(x) ln β(x)] (4) where Ent(β(x)) = β(x) ln β(x). x (b) Compute the exact expresson of E x β ln α(x). We have = E x β ln α(x) Ent(β(x)) (43) E x β ln α(x) = E x β ln N (x µ, σ) (44) ] = d (x µ) ln(πσ ) E x β σ (45) = d ln(πσ ) 1 σ E x β (x µ) ] (46) = d ln(πσ ) 1 ( σ C + (µ µ) ) (47) where we have margnalzed β for each term of the sum n eq. (45) and where we have used: E x β (x µ) ] = E x β (x µ + µ µ) ] (48) = E x β (x µ ) + (x µ )(µ µ) + (µ µ) ] (c) Compute KL(β α) (49) = E x β (x µ ) ] + (µ µ) (50) = C + (µ µ) (51) 5
6 KL(β α) = E x β ln α(x)] Ent(β) (5) = d ln(πσ ) + 1 ( σ C + (µ µ) ) d (ln(π) + 1) 1 ln C (53) = 1 d ln(σ ) ln C d + 1 ( σ C + (µ µ) )] (54) (d) Suppose that µ = µ and C = σ for all. Smplfy the prevous expresson. KL(β α) = 1 d ln(σ ) ln C ] (55) (e) How the mutual nformaton could appear n the ELBO? References Bshop, Chrstopher M. (006). Pattern Recognton and Machne Learnng (Informaton Scence and Statstcs). Secaucus, NJ, USA: Sprnger-Verlag New York, Inc. sbn: Boyd, Stephen and Leven Vandenberghe (004). Convex Optmzaton. New York, NY, USA: Cambrdge Unversty Press. sbn: Cover, Thomas M. and Joy A. Thomas (006). Elements of Informaton Theory (Wley Seres n Telecommuncatons and Sgnal Processng). Wley- Interscence. sbn: Programmng exercses Problem 5 (Text entropy). In the followng, we are nterested n estmatng the entropy of dfferent texts. We wll work wth the novel Crme and Punshment by Fyodor Dostoyevsky. Other books n dfferent languages are also avalable.. To do so, we compute the entropy of dfferent models: The chosen books are avalable at texts.zp, thanks to the Gutenberg project. 6
7 1. Compute the entropy of a model based on the frequency of each sngle symbol n the chosen book (..d. model).. Use ths model to compute the cross-entropy of the dstrbuton from another book. Compare ths value wth the prevous entropy by computng the KL-dvergence. 3. Compute the entropy of a model based on the frequency of pars of symbols, and compare t wth the prevous model. Explan the dfference. 4. Compute the entropy rate of a Markov chan where each state s a symbol, and transton probabltes are estmated from the chosen book. 7
COS 511: Theoretical Machine Learning. Lecturer: Rob Schapire Lecture #21 Scribe: Lawrence Diao April 23, 2013
COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture #21 Scrbe: Lawrence Dao Aprl 23, 2013 1 On-Lne Log Loss To recap the end of the last lecture, we have the followng on-lne problem wth N
More informationSupplementary material for Non-conjugate Variational Message Passing for Multinomial and Binary Regression
Supplementary materal for Non-conjugate Varatonal Message Passng for Multnomal and Bnary Regresson October 9, 011 1 Alternatve dervaton We wll focus on a partcular factor f a and varable x, wth the am
More informationAppendix for Solving Asset Pricing Models when the Price-Dividend Function is Analytic
Appendx for Solvng Asset Prcng Models when the Prce-Dvdend Functon s Analytc Ovdu L. Caln Yu Chen Thomas F. Cosmano and Alex A. Hmonas January 3, 5 Ths appendx provdes proofs of some results stated n our
More informationIntroduction to PGMs: Discrete Variables. Sargur Srihari
Introducton to : Dscrete Varables Sargur srhar@cedar.buffalo.edu Topcs. What are graphcal models (or ) 2. Use of Engneerng and AI 3. Drectonalty n graphs 4. Bayesan Networks 5. Generatve Models and Samplng
More information3: Central Limit Theorem, Systematic Errors
3: Central Lmt Theorem, Systematc Errors 1 Errors 1.1 Central Lmt Theorem Ths theorem s of prme mportance when measurng physcal quanttes because usually the mperfectons n the measurements are due to several
More informationProblem Set 6 Finance 1,
Carnege Mellon Unversty Graduate School of Industral Admnstraton Chrs Telmer Wnter 2006 Problem Set 6 Fnance, 47-720. (representatve agent constructon) Consder the followng two-perod, two-agent economy.
More informationECE 586GT: Problem Set 2: Problems and Solutions Uniqueness of Nash equilibria, zero sum games, evolutionary dynamics
Unversty of Illnos Fall 08 ECE 586GT: Problem Set : Problems and Solutons Unqueness of Nash equlbra, zero sum games, evolutonary dynamcs Due: Tuesday, Sept. 5, at begnnng of class Readng: Course notes,
More informationAppendix - Normally Distributed Admissible Choices are Optimal
Appendx - Normally Dstrbuted Admssble Choces are Optmal James N. Bodurtha, Jr. McDonough School of Busness Georgetown Unversty and Q Shen Stafford Partners Aprl 994 latest revson September 00 Abstract
More informationRandom Variables. b 2.
Random Varables Generally the object of an nvestgators nterest s not necessarly the acton n the sample space but rather some functon of t. Techncally a real valued functon or mappng whose doman s the sample
More informationII. Random Variables. Variable Types. Variables Map Outcomes to Numbers
II. Random Varables Random varables operate n much the same way as the outcomes or events n some arbtrary sample space the dstncton s that random varables are smply outcomes that are represented numercally.
More informationIntroduction to game theory
Introducton to game theory Lectures n game theory ECON5210, Sprng 2009, Part 1 17.12.2008 G.B. Ashem, ECON5210-1 1 Overvew over lectures 1. Introducton to game theory 2. Modelng nteractve knowledge; equlbrum
More informationParallel Prefix addition
Marcelo Kryger Sudent ID 015629850 Parallel Prefx addton The parallel prefx adder presented next, performs the addton of two bnary numbers n tme of complexty O(log n) and lnear cost O(n). Lets notce the
More information2.1 Rademacher Calculus... 3
COS 598E: Unsupervsed Learnng Week 2 Lecturer: Elad Hazan Scrbe: Kran Vodrahall Contents 1 Introducton 1 2 Non-generatve pproach 1 2.1 Rademacher Calculus............................... 3 3 Spectral utoencoders
More informationMgtOp 215 Chapter 13 Dr. Ahn
MgtOp 5 Chapter 3 Dr Ahn Consder two random varables X and Y wth,,, In order to study the relatonshp between the two random varables, we need a numercal measure that descrbes the relatonshp The covarance
More informationLikelihood Fits. Craig Blocker Brandeis August 23, 2004
Lkelhood Fts Crag Blocker Brandes August 23, 2004 Outlne I. What s the queston? II. Lkelhood Bascs III. Mathematcal Propertes IV. Uncertantes on Parameters V. Mscellaneous VI. Goodness of Ft VII. Comparson
More informationChapter 5 Student Lecture Notes 5-1
Chapter 5 Student Lecture Notes 5-1 Basc Busness Statstcs (9 th Edton) Chapter 5 Some Important Dscrete Probablty Dstrbutons 004 Prentce-Hall, Inc. Chap 5-1 Chapter Topcs The Probablty Dstrbuton of a Dscrete
More informationLecture 7. We now use Brouwer s fixed point theorem to prove Nash s theorem.
Topcs on the Border of Economcs and Computaton December 11, 2005 Lecturer: Noam Nsan Lecture 7 Scrbe: Yoram Bachrach 1 Nash s Theorem We begn by provng Nash s Theorem about the exstance of a mxed strategy
More informationMultifactor Term Structure Models
1 Multfactor Term Structure Models A. Lmtatons of One-Factor Models 1. Returns on bonds of all maturtes are perfectly correlated. 2. Term structure (and prces of every other dervatves) are unquely determned
More information/ Computational Genomics. Normalization
0-80 /02-70 Computatonal Genomcs Normalzaton Gene Expresson Analyss Model Computatonal nformaton fuson Bologcal regulatory networks Pattern Recognton Data Analyss clusterng, classfcaton normalzaton, mss.
More informationLinear Combinations of Random Variables and Sampling (100 points)
Economcs 30330: Statstcs for Economcs Problem Set 6 Unversty of Notre Dame Instructor: Julo Garín Sprng 2012 Lnear Combnatons of Random Varables and Samplng 100 ponts 1. Four-part problem. Go get some
More informationSUPPLEMENT TO BOOTSTRAPPING REALIZED VOLATILITY (Econometrica, Vol. 77, No. 1, January, 2009, )
Econometrca Supplementary Materal SUPPLEMENT TO BOOTSTRAPPING REALIZED VOLATILITY Econometrca, Vol. 77, No. 1, January, 009, 83 306 BY SÍLVIA GONÇALVES AND NOUR MEDDAHI THIS SUPPLEMENT IS ORGANIZED asfollows.frst,wentroducesomenotaton.
More informationScribe: Chris Berlind Date: Feb 1, 2010
CS/CNS/EE 253: Advanced Topcs n Machne Learnng Topc: Dealng wth Partal Feedback #2 Lecturer: Danel Golovn Scrbe: Chrs Berlnd Date: Feb 1, 2010 8.1 Revew In the prevous lecture we began lookng at algorthms
More informationTests for Two Ordered Categorical Variables
Chapter 253 Tests for Two Ordered Categorcal Varables Introducton Ths module computes power and sample sze for tests of ordered categorcal data such as Lkert scale data. Assumng proportonal odds, such
More informationOption pricing and numéraires
Opton prcng and numérares Daro Trevsan Unverstà degl Stud d Psa San Mnato - 15 September 2016 Overvew 1 What s a numerare? 2 Arrow-Debreu model Change of numerare change of measure 3 Contnuous tme Self-fnancng
More informationoccurrence of a larger storm than our culvert or bridge is barely capable of handling? (what is The main question is: What is the possibility of
Module 8: Probablty and Statstcal Methods n Water Resources Engneerng Bob Ptt Unversty of Alabama Tuscaloosa, AL Flow data are avalable from numerous USGS operated flow recordng statons. Data s usually
More informationDynamic Analysis of Knowledge Sharing of Agents with. Heterogeneous Knowledge
Dynamc Analyss of Sharng of Agents wth Heterogeneous Kazuyo Sato Akra Namatame Dept. of Computer Scence Natonal Defense Academy Yokosuka 39-8686 JAPAN E-mal {g40045 nama} @nda.ac.jp Abstract In ths paper
More informationECONOMETRICS - FINAL EXAM, 3rd YEAR (GECO & GADE)
ECONOMETRICS - FINAL EXAM, 3rd YEAR (GECO & GADE) May 17, 2016 15:30 Frst famly name: Name: DNI/ID: Moble: Second famly Name: GECO/GADE: Instructor: E-mal: Queston 1 A B C Blank Queston 2 A B C Blank Queston
More informationTests for Two Correlations
PASS Sample Sze Software Chapter 805 Tests for Two Correlatons Introducton The correlaton coeffcent (or correlaton), ρ, s a popular parameter for descrbng the strength of the assocaton between two varables.
More informationCumulative Step-size Adaptation on Linear Functions
Cumulatve Step-sze Adaptaton on Lnear Functons Alexandre Chotard, Anne Auger, Nkolaus Hansen To cte ths verson: Alexandre Chotard, Anne Auger, Nkolaus Hansen. Cumulatve Step-sze Adaptaton on Lnear Functons.
More informationElements of Economic Analysis II Lecture VI: Industry Supply
Elements of Economc Analyss II Lecture VI: Industry Supply Ka Hao Yang 10/12/2017 In the prevous lecture, we analyzed the frm s supply decson usng a set of smple graphcal analyses. In fact, the dscusson
More informationBayesian belief networks
CS 2750 achne Learnng Lecture 12 ayesan belef networks los Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square CS 2750 achne Learnng Densty estmaton Data: D { D1 D2.. Dn} D x a vector of attrbute values ttrbutes:
More informationCorrelations and Copulas
Correlatons and Copulas Chapter 9 Rsk Management and Fnancal Insttutons, Chapter 6, Copyrght John C. Hull 2006 6. Coeffcent of Correlaton The coeffcent of correlaton between two varables V and V 2 s defned
More informationA Comparison of Statistical Methods in Interrupted Time Series Analysis to Estimate an Intervention Effect
Transport and Road Safety (TARS) Research Joanna Wang A Comparson of Statstcal Methods n Interrupted Tme Seres Analyss to Estmate an Interventon Effect Research Fellow at Transport & Road Safety (TARS)
More informationPhysicsAndMathsTutor.com
PhscsAndMathsTutor.com phscsandmathstutor.com June 2005 6. A scentst found that the tme taken, M mnutes, to carr out an eperment can be modelled b a normal random varable wth mean 155 mnutes and standard
More informationData Mining Linear and Logistic Regression
07/02/207 Data Mnng Lnear and Logstc Regresson Mchael L of 26 Regresson In statstcal modellng, regresson analyss s a statstcal process for estmatng the relatonshps among varables. Regresson models are
More informationEquilibrium in Prediction Markets with Buyers and Sellers
Equlbrum n Predcton Markets wth Buyers and Sellers Shpra Agrawal Nmrod Megddo Benamn Armbruster Abstract Predcton markets wth buyers and sellers of contracts on multple outcomes are shown to have unque
More information4.4 Doob s inequalities
34 CHAPTER 4. MARTINGALES 4.4 Doob s nequaltes The frst nterestng consequences of the optonal stoppng theorems are Doob s nequaltes. If M n s a martngale, denote M n =max applen M. Theorem 4.8 If M n s
More informationTCOM501 Networking: Theory & Fundamentals Final Examination Professor Yannis A. Korilis April 26, 2002
TO5 Networng: Theory & undamentals nal xamnaton Professor Yanns. orls prl, Problem [ ponts]: onsder a rng networ wth nodes,,,. In ths networ, a customer that completes servce at node exts the networ wth
More informationCHAPTER 3: BAYESIAN DECISION THEORY
CHATER 3: BAYESIAN DECISION THEORY Decson makng under uncertanty 3 rogrammng computers to make nference from data requres nterdscplnary knowledge from statstcs and computer scence Knowledge of statstcs
More information332 Mathematical Induction Solutions for Chapter 14. for every positive integer n. Proof. We will prove this with mathematical induction.
33 Mathematcal Inducton. Solutons for Chapter. Prove that 3 n n n for every postve nteger n. Proof. We wll prove ths wth mathematcal nducton. Observe that f n, ths statement s, whch s obvously true. Consder
More informationreferences Chapters on game theory in Mas-Colell, Whinston and Green
Syllabus. Prelmnares. Role of game theory n economcs. Normal and extensve form of a game. Game-tree. Informaton partton. Perfect recall. Perfect and mperfect nformaton. Strategy.. Statc games of complete
More informationOn the Moments of the Traces of Unitary and Orthogonal Random Matrices
Proceedngs of Insttute of Mathematcs of NAS of Ukrane 2004 Vol. 50 Part 3 1207 1213 On the Moments of the Traces of Untary and Orthogonal Random Matrces Vladmr VASILCHU B. Verkn Insttute for Low Temperature
More informationMidterm Exam. Use the end of month price data for the S&P 500 index in the table below to answer the following questions.
Unversty of Washngton Summer 2001 Department of Economcs Erc Zvot Economcs 483 Mdterm Exam Ths s a closed book and closed note exam. However, you are allowed one page of handwrtten notes. Answer all questons
More informationChapter 3 Descriptive Statistics: Numerical Measures Part B
Sldes Prepared by JOHN S. LOUCKS St. Edward s Unversty Slde 1 Chapter 3 Descrptve Statstcs: Numercal Measures Part B Measures of Dstrbuton Shape, Relatve Locaton, and Detectng Outlers Eploratory Data Analyss
More informationCHAPTER 9 FUNCTIONAL FORMS OF REGRESSION MODELS
CHAPTER 9 FUNCTIONAL FORMS OF REGRESSION MODELS QUESTIONS 9.1. (a) In a log-log model the dependent and all explanatory varables are n the logarthmc form. (b) In the log-ln model the dependent varable
More informationCS340 Machine learning Bayesian model selection
CS340 Machine learning Bayesian model selection Bayesian model selection Suppose we have several models, each with potentially different numbers of parameters. Example: M0 = constant, M1 = straight line,
More informationAn Approximate E-Bayesian Estimation of Step-stress Accelerated Life Testing with Exponential Distribution
Send Orders for Reprnts to reprnts@benthamscenceae The Open Cybernetcs & Systemcs Journal, 25, 9, 729-733 729 Open Access An Approxmate E-Bayesan Estmaton of Step-stress Accelerated Lfe Testng wth Exponental
More informationLecture Note 1: Foundations 1
Economcs 703 Advanced Mcroeconomcs Prof. Peter Cramton ecture Note : Foundatons Outlne A. Introducton and Examples B. Formal Treatment. Exstence of Nash Equlbrum. Exstence wthout uas-concavty 3. Perfect
More information3/3/2014. CDS M Phil Econometrics. Vijayamohanan Pillai N. Truncated standard normal distribution for a = 0.5, 0, and 0.5. CDS Mphil Econometrics
Lmted Dependent Varable Models: Tobt an Plla N 1 CDS Mphl Econometrcs Introducton Lmted Dependent Varable Models: Truncaton and Censorng Maddala, G. 1983. Lmted Dependent and Qualtatve Varables n Econometrcs.
More informationOPERATIONS RESEARCH. Game Theory
OPERATIONS RESEARCH Chapter 2 Game Theory Prof. Bbhas C. Gr Department of Mathematcs Jadavpur Unversty Kolkata, Inda Emal: bcgr.umath@gmal.com 1.0 Introducton Game theory was developed for decson makng
More informationProduction and Supply Chain Management Logistics. Paolo Detti Department of Information Engeneering and Mathematical Sciences University of Siena
Producton and Supply Chan Management Logstcs Paolo Dett Department of Informaton Engeneerng and Mathematcal Scences Unversty of Sena Convergence and complexty of the algorthm Convergence of the algorthm
More information15-451/651: Design & Analysis of Algorithms January 22, 2019 Lecture #3: Amortized Analysis last changed: January 18, 2019
5-45/65: Desgn & Analyss of Algorthms January, 09 Lecture #3: Amortzed Analyss last changed: January 8, 09 Introducton In ths lecture we dscuss a useful form of analyss, called amortzed analyss, for problems
More informationMaximum Likelihood Estimation of Isotonic Normal Means with Unknown Variances*
Journal of Multvarate Analyss 64, 183195 (1998) Artcle No. MV971717 Maxmum Lelhood Estmaton of Isotonc Normal Means wth Unnown Varances* Nng-Zhong Sh and Hua Jang Northeast Normal Unversty, Changchun,Chna
More informationNotes on experimental uncertainties and their propagation
Ed Eyler 003 otes on epermental uncertantes and ther propagaton These notes are not ntended as a complete set of lecture notes, but nstead as an enumeraton of some of the key statstcal deas needed to obtan
More informationConsumption Based Asset Pricing
Consumpton Based Asset Prcng Mchael Bar Aprl 25, 208 Contents Introducton 2 Model 2. Prcng rsk-free asset............................... 3 2.2 Prcng rsky assets................................ 4 2.3 Bubbles......................................
More informationEconomic Design of Short-Run CSP-1 Plan Under Linear Inspection Cost
Tamkang Journal of Scence and Engneerng, Vol. 9, No 1, pp. 19 23 (2006) 19 Economc Desgn of Short-Run CSP-1 Plan Under Lnear Inspecton Cost Chung-Ho Chen 1 * and Chao-Yu Chou 2 1 Department of Industral
More informationFast Laplacian Solvers by Sparsification
Spectral Graph Theory Lecture 19 Fast Laplacan Solvers by Sparsfcaton Danel A. Spelman November 9, 2015 Dsclamer These notes are not necessarly an accurate representaton of what happened n class. The notes
More informationMeasures of Spread IQR and Deviation. For exam X, calculate the mean, median and mode. For exam Y, calculate the mean, median and mode.
Part 4 Measures of Spread IQR and Devaton In Part we learned how the three measures of center offer dfferent ways of provdng us wth a sngle representatve value for a data set. However, consder the followng
More informationMode is the value which occurs most frequency. The mode may not exist, and even if it does, it may not be unique.
1.7.4 Mode Mode s the value whch occurs most frequency. The mode may not exst, and even f t does, t may not be unque. For ungrouped data, we smply count the largest frequency of the gven value. If all
More informationNotes on Debye-Hückel Theory
GEOS 0300/30300 Autumn 004 Thermodynamcs and Phase Change MS Ghorso Notes on Debye-Hückel Theory We seek: µ µ o + RT ln X + µ ex where µ ex RT ln λ or the devaton from deal behavor due to long range nteracton
More informationOn estimating the location parameter of the selected exponential population under the LINEX loss function
On estmatng the locaton parameter of the selected exponental populaton under the LINEX loss functon Mohd. Arshad 1 and Omer Abdalghan Department of Statstcs and Operatons Research Algarh Muslm Unversty,
More informationA Study on the Series Expansion of Gaussian Quadratic Forms
c WISRL, KAIST, APRIL, 202 A Study on the Seres Expanson of Gaussan Quadratc Forms Techncal Report WISRL-202-APR- KAIST Juho Park, Youngchul Sung, Donggun Km, and H. Vncent Poor All rghts reserved c WISRL,
More information4. Greek Letters, Value-at-Risk
4 Greek Letters, Value-at-Rsk 4 Value-at-Rsk (Hull s, Chapter 8) Math443 W08, HM Zhu Outlne (Hull, Chap 8) What s Value at Rsk (VaR)? Hstorcal smulatons Monte Carlo smulatons Model based approach Varance-covarance
More informationInterval Estimation for a Linear Function of. Variances of Nonnormal Distributions. that Utilize the Kurtosis
Appled Mathematcal Scences, Vol. 7, 013, no. 99, 4909-4918 HIKARI Ltd, www.m-hkar.com http://dx.do.org/10.1988/ams.013.37366 Interval Estmaton for a Lnear Functon of Varances of Nonnormal Dstrbutons that
More information1 A Primer on Linear Models. 2 Chapter 1 corrections. 3 Chapter 2 corrections. 4 Chapter 3 corrections. 1.1 Corrections 23 May 2015
A Prmer on Lnear Models. Correctons 23 May 205 2 Chapter correctons Fx page: 9 lne -7 denomnator ( ρ 2 ) s mssng, should read Cov(e t, e s ) = V ar(a t )/( ρ 2 ) Fx page: lne -5 sgn needs changng: cos(a
More informationElton, Gruber, Brown, and Goetzmann. Modern Portfolio Theory and Investment Analysis, 7th Edition. Solutions to Text Problems: Chapter 9
Elton, Gruber, Brown, and Goetzmann Modern Portfolo Theory and Investment Analyss, 7th Edton Solutons to Text Problems: Chapter 9 Chapter 9: Problem In the table below, gven that the rskless rate equals
More informationEvaluating Performance
5 Chapter Evaluatng Performance In Ths Chapter Dollar-Weghted Rate of Return Tme-Weghted Rate of Return Income Rate of Return Prncpal Rate of Return Daly Returns MPT Statstcs 5- Measurng Rates of Return
More informationGeneral Examination in Microeconomic Theory. Fall You have FOUR hours. 2. Answer all questions
HARVARD UNIVERSITY DEPARTMENT OF ECONOMICS General Examnaton n Mcroeconomc Theory Fall 2010 1. You have FOUR hours. 2. Answer all questons PLEASE USE A SEPARATE BLUE BOOK FOR EACH QUESTION AND WRITE THE
More informationA Utilitarian Approach of the Rawls s Difference Principle
1 A Utltaran Approach of the Rawls s Dfference Prncple Hyeok Yong Kwon a,1, Hang Keun Ryu b,2 a Department of Poltcal Scence, Korea Unversty, Seoul, Korea, 136-701 b Department of Economcs, Chung Ang Unversty,
More informationExplaining Movements of the Labor Share in the Korean Economy: Factor Substitution, Markups and Bargaining Power
Explanng Movements of the abor Share n the Korean Economy: Factor Substtuton, Markups and Barganng ower Bae-Geun, Km January 2, 26 Appendx A. Dervaton of the dervatve of et us start from eq. (). For notatonal
More information>1 indicates country i has a comparative advantage in production of j; the greater the index, the stronger the advantage. RCA 1 ij
69 APPENDIX 1 RCA Indces In the followng we present some maor RCA ndces reported n the lterature. For addtonal varants and other RCA ndces, Memedovc (1994) and Vollrath (1991) provde more thorough revews.
More informationQuadratic Games. First version: February 24, 2017 This version: December 12, Abstract
Quadratc Games Ncolas S. Lambert Gorgo Martn Mchael Ostrovsky Frst verson: February 24, 2017 Ths verson: December 12, 2017 Abstract We study general quadratc games wth mult-dmensonal actons, stochastc
More informationInference on Reliability in the Gamma and Inverted Gamma Distributions
Statstcs n the Twenty-Frst Century: Specal Volue In Honour of Dstngushed Professor Dr. Mr Masoo Al On the Occason of hs 75th Brthday Annversary PJSOR, Vol. 8, No. 3, pages 635-643, July Jungsoo Woo Departent
More informationThe Market Selection Hypothesis
The Market Selecton Hypothess Lawrence Blume and Davd Easley Department of Economcs Cornell Unversty 1 June 1999 The authors thank Alvaro Sandron for stmulatng conversaton, and The Natonal Scence Foundaton
More informationUnderstanding Annuities. Some Algebraic Terminology.
Understandng Annutes Ma 162 Sprng 2010 Ma 162 Sprng 2010 March 22, 2010 Some Algebrac Termnology We recall some terms and calculatons from elementary algebra A fnte sequence of numbers s a functon of natural
More informationQuadratic Games. First version: February 24, 2017 This version: August 3, Abstract
Quadratc Games Ncolas S. Lambert Gorgo Martn Mchael Ostrovsky Frst verson: February 24, 2017 Ths verson: August 3, 2018 Abstract We study general quadratc games wth multdmensonal actons, stochastc payoff
More informationSequential equilibria of asymmetric ascending auctions: the case of log-normal distributions 3
Sequental equlbra of asymmetrc ascendng auctons: the case of log-normal dstrbutons 3 Robert Wlson Busness School, Stanford Unversty, Stanford, CA 94305-505, USA Receved: ; revsed verson. Summary: The sequental
More informationCLOSED-FORM LIKELIHOOD EXPANSIONS FOR MULTIVARIATE DIFFUSIONS. BY YACINE AÏT-SAHALIA 1 Princeton University
The Annals of Statstcs 2008, Vol. 36, No. 2, 906 937 DOI: 10.1214/009053607000000622 Insttute of Mathematcal Statstcs, 2008 CLOSED-FORM LIKELIHOOD EPANSIONS FOR MULTIVARIATE DIFFUSIONS B ACINE AÏT-SAHALIA
More information0.1 Gradient descent for convex functions: univariate case
prnceton unv. F 16 cos 51: Advanced Algorthm Desgn Lecture 14: Gong wth the slope: offlne, onlne, and randomly Lecturer: Sanjeev Arora Scrbe:Sanjeev Arora hs lecture s about gradent descent, a popular
More informationGlobal sensitivity analysis of credit risk portfolios
Global senstvty analyss of credt rsk portfolos D. Baur, J. Carbon & F. Campolongo European Commsson, Jont Research Centre, Italy Abstract Ths paper proposes the use of global senstvty analyss to evaluate
More informationNew Distance Measures on Dual Hesitant Fuzzy Sets and Their Application in Pattern Recognition
Journal of Artfcal Intellgence Practce (206) : 8-3 Clausus Scentfc Press, Canada New Dstance Measures on Dual Hestant Fuzzy Sets and Ther Applcaton n Pattern Recognton L Xn a, Zhang Xaohong* b College
More informationTest Problems for Large Scale Nonsmooth Minimization
Reports of the Department of Mathematcal Informaton Technology Seres B. Scentfc Computng No. B. 4/007 Test Problems for Large Scale Nonsmooth Mnmzaton Napsu Karmtsa Unversty of Jyväskylä Department of
More information2) In the medium-run/long-run, a decrease in the budget deficit will produce:
4.02 Quz 2 Solutons Fall 2004 Multple-Choce Questons ) Consder the wage-settng and prce-settng equatons we studed n class. Suppose the markup, µ, equals 0.25, and F(u,z) = -u. What s the natural rate of
More informationVariational Inference for Graphical Models of Multivariate Piecewise-Stationary Time Series
18th Internatonal Conference on Informaton Fuson Washngton, DC - July 6-9, 2015 Varatonal Inference for Graphcal Models of Multvarate Pecewse-Statonary Tme Seres Hang Yu and Justn Dauwels School of Electrcal
More informationCS54701: Information Retrieval
CS54701: Informaton Retreval Federated Search 22 March 2016 Prof. Chrs Clfton Federated Search Outlne Introducton to federated search Man research problems Resource Representaton Resource Selecton Results
More informationMonte Carlo Rendering
Last Tme? Monte Carlo Renderng Monte-Carlo Integraton Probabltes and Varance Analyss of Monte-Carlo Integraton Monte-Carlo n Graphcs Stratfed Samplng Importance Samplng Advanced Monte-Carlo Renderng Monte-Carlo
More informationDoubly Random Parallel Stochastic Algorithms for Large Scale Learning
000 00 00 003 004 005 006 007 008 009 00 0 0 03 04 05 06 07 08 09 00 0 0 03 04 05 06 07 08 09 030 03 03 033 034 035 036 037 038 039 040 04 04 043 044 045 046 047 048 049 050 05 05 053 Doubly Random Parallel
More informationNumerical Analysis ECIV 3306 Chapter 6
The Islamc Unversty o Gaza Faculty o Engneerng Cvl Engneerng Department Numercal Analyss ECIV 3306 Chapter 6 Open Methods & System o Non-lnear Eqs Assocate Pro. Mazen Abualtaye Cvl Engneerng Department,
More informationStatic (or Simultaneous- Move) Games of Complete Information
Statc (or Smultaneous- Move) Games of Complete Informaton Nash Equlbrum Best Response Functon F. Valognes - Game Theory - Chp 3 Outlne of Statc Games of Complete Informaton Introducton to games Normal-form
More informationTesting for Omitted Variables
Testng for Omtted Varables Jeroen Weese Department of Socology Unversty of Utrecht The Netherlands emal J.weese@fss.uu.nl tel +31 30 2531922 fax+31 30 2534405 Prepared for North Amercan Stata users meetng
More informationProspect Theory and Asset Prices
Fnance 400 A. Penat - G. Pennacch Prospect Theory and Asset Prces These notes consder the asset prcng mplcatons of nvestor behavor that ncorporates Prospect Theory. It summarzes an artcle by N. Barbers,
More informationApplications of Myerson s Lemma
Applcatons of Myerson s Lemma Professor Greenwald 28-2-7 We apply Myerson s lemma to solve the sngle-good aucton, and the generalzaton n whch there are k dentcal copes of the good. Our objectve s welfare
More informationGames and Decisions. Part I: Basic Theorems. Contents. 1 Introduction. Jane Yuxin Wang. 1 Introduction 1. 2 Two-player Games 2
Games and Decsons Part I: Basc Theorems Jane Yuxn Wang Contents 1 Introducton 1 2 Two-player Games 2 2.1 Zero-sum Games................................ 3 2.1.1 Pure Strateges.............................
More informationOCR Statistics 1 Working with data. Section 2: Measures of location
OCR Statstcs 1 Workng wth data Secton 2: Measures of locaton Notes and Examples These notes have sub-sectons on: The medan Estmatng the medan from grouped data The mean Estmatng the mean from grouped data
More informationEconomics 1410 Fall Section 7 Notes 1. Define the tax in a flexible way using T (z), where z is the income reported by the agent.
Economcs 1410 Fall 2017 Harvard Unversty Yaan Al-Karableh Secton 7 Notes 1 I. The ncome taxaton problem Defne the tax n a flexble way usng T (), where s the ncome reported by the agent. Retenton functon:
More informationσ may be counterbalanced by a larger
Questons CHAPTER 5: TWO-VARIABLE REGRESSION: INTERVAL ESTIMATION AND HYPOTHESIS TESTING 5.1 (a) True. The t test s based on varables wth a normal dstrbuton. Snce the estmators of β 1 and β are lnear combnatons
More informationA Bayesian Classifier for Uncertain Data
A Bayesan Classfer for Uncertan Data Bao Qn, Yun Xa Department of Computer Scence Indana Unversty - Purdue Unversty Indanapols, USA {baoqn, yxa}@cs.upu.edu Fang L Department of Mathematcal Scences Indana
More informationQuiz on Deterministic part of course October 22, 2002
Engneerng ystems Analyss for Desgn Quz on Determnstc part of course October 22, 2002 Ths s a closed book exercse. You may use calculators Grade Tables There are 90 ponts possble for the regular test, or
More informationStill Simpler Way of Introducing Interior-Point method for Linear Programming
Stll Smpler Way of Introducng Interor-Pont method for Lnear Programmng Sanjeev Saxena Dept. of Computer Scence and Engneerng, Indan Insttute of Technology, Kanpur, INDIA-08 06 October 9, 05 Abstract Lnear
More informationPhysics 4A. Error Analysis or Experimental Uncertainty. Error
Physcs 4A Error Analyss or Expermental Uncertanty Slde Slde 2 Slde 3 Slde 4 Slde 5 Slde 6 Slde 7 Slde 8 Slde 9 Slde 0 Slde Slde 2 Slde 3 Slde 4 Slde 5 Slde 6 Slde 7 Slde 8 Slde 9 Slde 20 Slde 2 Error n
More information