Not your average regression: A practical introduction to quantile regression. James Ellens

Transcription

1 11 th Mass Appraisal Valuation Symposium Innovation, Transformation, Knowledge Enhancement and Improved Efficiencies in Mass Appraisal Niagara Falls, Canada May 17-18, 2016 Not your average regression: A practical introduction to quantile regression James Ellens Manager, Assessment Standards & Mass Appraisal Municipal Property Assessment Corporation

2 OLS Regression Traditional regression analysis utilizes the ordinary least squares (OLS) method to determine the best fit OLS depends on the conditional mean to summarize the relationship of the dependent and independent variables OLS works well given symmetrically distributed data

3 Sample Data Distribution

4 Quantile Regression Created by Roger Koenker and Gilbert Bassett, 1978 Uses the conditional median, or other percentile as specified Semiparametric More robust to errors and outliers Capable of providing a more complete characterization of the conditional distribution

5 Software R quantreg package Python Statsmodel package Stata Matlab SPSS (with R integration) SAS Excel with XLSTAT software and many more

6 Model Specification: R OLS: lm(tadj_amt ~ qcareal +..., data = sales) Quantile: rq(tadj_amt ~ qcareal +..., data = sales, tau =.5, method = "fn") Where: tau is the quanitle(s) to be estimated (0.5 being the median) method: there are several different algorithm in the backend that can be used depending on the size of the regression dataset

7 Model Specification: SPSS

8 Model Output: Coefficients Variable Value Std. Error t value Pr(> t ) (Intercept) 38, , sqrt_ed quadj_ef 1, sqrefgsf qcareal vacant -28, , pool fireplcs 4, qut_fba aircon 3, spl_bfyes 6, , spl_sideyes 9, , spl_uncvyes -9, , des_rb_s 14, , pc const_d

9 Distribution Visualized

10 Result Comparison: Market Areas

11 Result Comparison 1 Urban: Niagara Falls OLS Quantile Rural: Simcoe North OLS Quantile Waterfront: Northern Ontario OLS Quantile Highrise Condo: Toronto OLS Quantile

12 Result Comparison 2 Waterfront: Kawartha Lakes OLS Quantile Highrise Condo: Waterloo, Guelph OLS Quantile Townhouse Condos: London OLS Quantile Urban: North Mississauga OLS Quantile

13 Result Comparison: Holdout Urban: Niagara Falls OLS Quantile Rural: Simcoe North OLS Quantile

14 Outlier Resilience Niagara Falls Urban dataset: 3,991 sales Randomly selected one sale and tripled it s time adjusted sale amount (the dependent variable) sample(1:nrow(sales),1) [1] 865 > sales[865,]$tadj_amt #sale amount before [1] > sales[865,]$tadj_amt <- sales[865,]$tadj_amt * 3 > sales[865,]$tadj_amt #sale amount after [1] Compared the resultant coefficient changes

15 Outlier Resilience: Quantile 4 Coefficients Changed Coefficients Variable Before After Change h2725a % h2725a % h2725a % h2725a %

16 Outlier Resilience: OLS 42 Coefficients Changed Coefficients Coefficients Variable Before After Change Variable Before After Change (Intercept) % h2725a % sqrt_ed % h2725a % quadj_ef % h2725a % vacant % h2725a % fireplcs % h2725a % qut_fba % h2725a % aircon % h2725a % spl_bfyes % h2725a % spl_sideyes % h2725a % spl_uncvyes % h2725a % des_rb_s % h2725a % pc % h2725a % const_d % h2725a % detwrkaq % h2725a % h2725a % h2725a % h2725a % h2725a % h2725a % h2725a % h2725a % h2725a % h2725a % h2725i % h2725a % h2725q % h2725a % h2725q %

17 Questions? James Ellens Manager, Assessment Standards and Mass Appraisal Office of the Chief Assessor Municipal Property Assessment Corporation