Statistical Tools for Program Evaluation

Transcription

1 Statistical Tools for Program Evaluation

2 Jean-Michel Josselin Benoît Le Maux Statistical Tools for Program Evaluation Methods and Applications to Economic Policy, Public Health, and Education

3 Jean-Michel Josselin Faculty of Economics University of Rennes 1 Rennes, France Benoît Le Maux Faculty of Economics University of Rennes 1 Rennes, France ISBN ISBN (ebook) DOI / Library of Congress Control Number: # Springer International Publishing AG 2017 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Printed on acid-free paper This Springer imprint is published by Springer Nature The registered company is Springer International Publishing AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

4 Acknowledgments We would like to express our gratitude to those who helped us and made the completion of this book possible. First of all, we are deeply indebted to the Springer editorial team and particularly Martina BIHN whose support and encouragement allowed us to finalize this project. Furthermore, we have benefited from helpful comments by colleagues and we would like to acknowledge the help of Maurice BASLÉ, Arthur CHARPENTIER, Pauline CHAUVIN, Salah GHABRI, and Christophe TAVÉRA. Of course, any mistake that may remain is our entire responsibility. In addition, we are grateful to our students who have been testing and experimenting our lectures for so many years. Parts of the material provided here have been taught at the Bachelor and Master levels, in France and abroad. Several students and former students have been helping us improve the book. We really appreciated their efforts and are very grateful to them: Erwan AUTIN, Benoît CARRÉ, Aude DAILLÈRE, Kristýna DOSTÁLOVÁ, and Adrien VEZIE. Finally, we would like to express our sincere gratefulness to our families for their continuous support and encouragement. v

5 Contents 1 Statistical Tools for Program Evaluation: Introduction and Overview The Challenge of Program Evaluation Identifying the Context of the Program Ex ante Evaluation Methods Ex post Evaluation HowtoUsetheBook? Bibliography Part I Identifying the Context of the Program 2 Sampling and Construction of Variables A Step Not to Be Taken Lightly Choice of Sample Conception of the Questionnaire Data Collection Coding of Variables Bibliography Descriptive Statistics and Interval Estimation Types of Variables and Methods Tabular Displays Graphical Representations Measures of Central Tendency and Variability Describing the Shape of Distributions Computing Confidence Intervals References Measuring and Visualizing Associations Identifying Relationships Between Variables Testing for Correlation Chi-Square Test of Independence vii

6 viii Contents 4.4 Tests of Difference Between Means Principal Component Analysis Multiple Correspondence Analysis References Econometric Analysis Understanding the Basic Regression Model Multiple Regression Analysis Assumptions Underlying the Method of OLS Choice of Relevant Variables Functional Forms of Regression Models Detection and Correction of Estimation Biases Model Selection and Analysis of Regression Results Models for Binary Outcomes References Estimation of Welfare Changes Valuing the Consequences of a Project Contingent Valuation Discrete Choice Experiment Hedonic Pricing Travel Cost Method Health-Related Quality of Life References Part II Ex ante Evaluation 7 Financial Appraisal Methodology of Financial Appraisal Time Value of Money Cash Flows and Sustainability Profitability Analysis Real Versus Nominal Values Ranking Investment Strategies Sensitivity Analysis References Budget Impact Analysis Introducing a New Intervention Amongst Existing Ones Analytical Framework Budget Impact in a Multiple-Supply Setting Example Sensitivity Analysis with Visual Basic References

7 Contents ix 9 Cost Benefit Analysis Rationale for Cost Benefit Analysis Conceptual Foundations Discount of Benefits and Costs Accounting for Market Distortions Deterministic Sensitivity Analysis Probabilistic Sensitivity Analysis Mean-Variance Analysis Bibliography Cost Effectiveness Analysis Appraisal of Projects with Non-monetary Outcomes Cost Effectiveness Indicators The Efficiency Frontier Approach Decision Analytic Modeling Numerical Implementation in R-CRAN Extension to QALYs Uncertainty and Probabilistic Sensitivity Analysis Analyzing Simulation Outputs References Multi-criteria Decision Analysis Key Concepts and Steps Problem Structuring Assessing Performance Levels with Scoring Criteria Weighting Construction of a Composite Indicator Non-Compensatory Analysis Examination of Results References Part III Ex post Evaluation 12 Project Follow-Up by Benchmarking Cost Comparisons to a Reference Cost Accounting Framework Effects of Demand Structure and Production Structure on Cost Production Structure Effect: Service-Oriented Approach Production Structure Effect: Input-Oriented Approach Ranking Through Benchmarking References

8 x Contents 13 Randomized Controlled Experiments From Clinical Trials to Field Experiments Random Allocation of Subjects Statistical Significance of a Treatment Effect Clinical Significance and Statistical Power Sample Size Calculations Indicators of Policy Effects Survival Analysis with Censoring: The Kaplan-Meier Approach Mantel-Haenszel Test for Conditional Independence References Quasi-experiments The Rationale for Counterfactual Analysis Difference-in-Differences Propensity Score Matching Regression Discontinuity Design Instrumental Variable Estimation References