Expert Systems with Applications

Transcription

1 Expert Systems with Applications 40 (2013) Contents lists available at SciVerse ScienceDirect Expert Systems with Applications journal homepage: Measuring firm performance using financial ratios: A decision tree approach Dursun Delen a,, Cemil Kuzey b, Ali Uyar b a Department of Management Science and Information Systems, Spears School of Business, Oklahoma State University, United States b Department of Management, Fatih University, Buyukcekmece, Istanbul 34500, Turkey article info abstract Keywords: Firm performance Financial ratios Exploratory factor analysis Decision trees Sensitivity analysis Determining the firm performance using a set of financial measures/ratios has been an interesting and challenging problem for many researchers and practitioners. Identification of factors (i.e., financial measures/ratios) that can accurately predict the firm performance is of great interest to any decision maker. In this study, we employed a two-step analysis methodology: first, using exploratory factor analysis (EFA) we identified (and validated) underlying dimensions of the financial ratios, followed by using predictive modeling methods to discover the potential relationships between the firm performance and financial ratios. Four popular decision tree algorithms (CHAID, C5.0, QUEST and C&RT) were used to investigate the impact of financial ratios on firm performance. After developing prediction models, information fusion-based sensitivity analyses were performed to measure the relative importance of independent variables. The results showed the CHAID and C5.0 decision tree algorithms produced the best prediction accuracy. Sensitivity analysis results indicated that Earnings Before Tax-to-Equity Ratio and Net Profit Margin are the two most important variables. Ó 2013 Elsevier Ltd. All rights reserved. 1. Introduction Evaluating firm performance using financial ratios has been a traditional yet powerful tool for decision-makers, including business analysts, creditors, investors, and financial managers. Rather than employing the total amounts observed on financial statements, these analyses were conducted using a number of financial ratios to obtain meaningful results. Ratio analysis can help stakeholders analyze the financial health of a company. Using these financial ratios, comparisons can be made across companies within an industry, between industries, or within a firm itself. Such a tool can also be used to compare the relative performance of different size companies. Accounting and finance text books generally organize financial ratios into classes including liquidity, profitability, long-term solvency, and asset utilization or turnover ratios. Liquidity ratios evaluate the ability of a company to pay a short-term debt, whereas long-term solvency ratios investigate how risky an investment in the firm could be for creditors. Profitability ratios examine the profit-generating ability of a firm based on sales, equity, and assets. Asset utilization or turnover ratios measure how successfully the company generates revenues through utilizing assets, collecting receivables, and selling its inventories. As part of an empirical research, Matsumoto, Shivaswamy, and Hoban (1995) conducted a survey of security analysts to ascertain their perceptions regarding financial ratios. They discovered that growth rates were considered to be the most important, followed by valuation, and then profitability ratios. The analysts ranked earnings per share and leverage ratio slightly lower than the above three. They also found that the ranking orders of ratio groups were quite different for retailers and manufacturers. Previously, various methodologies had been implemented in order to evaluate the financial performance of companies in association with financial ratios. While the earlier studies primarily used traditional statistical techniques (e.g., Factor analysis, ANOVA, linear regression, etc.) more recent studies employed advanced decision-making approaches. One of the most popular approaches has been the decision tree analysis, which is often preferred because of its simplicity, transparency, descriptive and predictive power. In this study, using decision tree analyses along with several financial ratios, we evaluated the financial performance of Turkish companies listed on the Istanbul Stock Exchange. The remainder of this paper is organized as follows: the next section (Section 2) provides a literature review; Section 3 presents the methodology developed and followed in this study, and documenting its findings. The Section 4 summarizes and concludes the paper. 2. Literature review Corresponding author. Tel.: +1 (918) ; fax: +1 (918) addresses: dursun.delen@okstate.edu (D. Delen), ckuzey@fatih.edu.tr (C. Kuzey), auyar@fatih.edu.tr (A. Uyar). Use of financial ratios to assess the firm performance is not new. A simple literature search can find literally thousands of publica /$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved.

2 D. Delen et al. / Expert Systems with Applications 40 (2013) tions on this topic. The underlying studies often differentiate themselves from the rest by developing and using different independent variables (financial ratios) and/or employing different statistical or machine learning based analysis techniques. For instance, Horrigan (1965) claimed that the development of financial ratios ought to be a unique product of the evolution of accounting procedures and practices in the U.S.; further stating that the origin of financial ratios and their initial use goes back to the late 19th century. Financial ratios, which are calculated by using variables commonly found on financial statements, can provide the following benefits (Ross, Westerfield, & Jordan, 2003): Measuring the performance of managers for the purpose of rewards; Measuring the performance of departments within multi-level companies; Projecting the future by supplying historical information to existing or potential investors; Providing information to creditors and suppliers; Evaluating competitive positions of rivals; Evaluating the financial performance of acquisitions. Other than the benefits provided above, financial ratios are also used for the purpose of predicting future performance. For example, they are used as inputs for empirical studies or are used to develop models to predict financial distress or failures (Altman, 1968; Beaver, 1966). In fact, a vast majority of the recent studies focused on analyzing and potentially predicting bankruptcy as a means to identify characteristics (in term of financial ratios) of good or bad-performing firms and their potential values (Kumar & Ravi, 2007). Thousands of studies conducted in bankruptcy prediction distinguished themselves from those of the others by using a somewhat unique set of financial characteristics or employing a different set of prediction models (statistical or machine learning based) (Alfaro, García, Gámez, & Elizondo, 2008; Holsapple & Wu, 2011; Lee, Han, & Kwon, 1996; Martín-Oliver & Salas-Fumás, 2012; Olson, Delen, & Meng, 2012; Wilson & Sharda, 1994). Though many of these studies are successful in predicting bankruptcy outcomes, they often fall short on identifying and explaining the characteristics that can be used as determinants of the firm performance. There is no universally agreed-upon list regarding the type, calculation methods and number of financial ratios used in earlier studies. For instance, Gombola and Ketz (1983) used 58 ratios to detect financial ratio patterns of within retail and manufacturing organizations, while Ho and Wu (2006) used 59 ratios, Cinca, Molinero, and Larraz (2005) used 16 ratios, Uyar and Okumusß (2010) used 15 ratios, and Karaca and Çiğdem (2012) used 24 ratios. However, most text books and research studies published in reputable journals provided somewhere in between 20 to 30 of the more commonly used ratios, which are often found to be sufficient to evaluate the performance of a firm. Earlier studies have provided empirical evidence that the structure of financial ratio patterns differs between retail and manufacturing firms (Gombola & Ketz, 1983). Cinca et al. (2005) proved that the size of the company and the country where the company is located impact the financial ratio structure. Uyar and Okumusß (2010) investigated the impact of the recent global financial crisis on publicly traded Turkish industrial enterprises using financial ratios, finding that firms had been weakened financially during the crisis period. In earlier studies, researchers utilized statistical methods which are prone to unrealistic normality and linearity assumptions. For example, Altman (1968) applied multiple discriminant analysis, which requires data to meet normality, equal covariance and independency of variables conditions. The superiority of decision tree methods (arguably the most popular data mining techniques) is that they are free from these limiting assumptions. Furthermore, decision trees can be represented as easily understandable graphical displays, making them transparent and easily understandable to managers. Therefore, in this study we chose to use the most popular decision tree methods as our analysis tools. Previous studies have also focused primarily on financial performance, stock return, and bankruptcy or financial distress prediction by using various statistical and data mining techniques such as decision trees and neural network (Chen & Du, 2009; Lam, 2004; Sun & Hui, 2006; Wang, Jiang, & Wang, 2009; Yu & Wenjuan, 2010). For instance, Zibanezhad, Foroghi, and Monadjemi (2011) employed classification and regression trees (C&RT) to predict financial bankruptcy using financial ratios as well as to determine the most important variables. Wang et al. (2009) implemented the bagging-decision tree model to predict stock returns by using fifty financial ratios. Sun and Hui (2006) focused on financial distress prediction of Chinese listed firms applying decision tree and genetic algorithms. Yu and Wenjuan (2010) used the decision tree to examine which financial ratios have strong influence on the profit growth of listed logistics companies; they have employed C5.0, which is one of the decision tree techniques. In this study, we used four popular the decision tree algorithms to develop prediction models and by the way of conducting information fusion based sensitivity analysis on these prediction models, we discovered which financial ratios have the strongest impact on financial performance. In our analysis, we used a large and feature rich financial database of Turkish public companies listed on Istanbul Stock Exchange. 3. Methodology 3.1. Data and sample Our goal was to identify and use a large and feature rich dataset. After an exhaustive search we identified FINNET, which is a company providing variety of financial data, software, and Web-based analysis tools to their members. FINNET has the largest financial database on Turkish firms. Even though the FINNET data is large and feature-rich in content, it had variety of data problems; demanding a through process of data cleaning and pre-processing. The initial sample of the study consisted of all Turkish listed public companies from 2005 to In total, 2722 data records/ cases were available for analysis. Out of this, 371 cases had significant missing-date problems on financial ratio values; therefore they were eliminated. Also, 6 cases were identified as extreme outliers, and therefore they were also eliminated from the dataset. At the end of the cleaning and pre-processing procedures, there were 2345 usable cases for model building and testing purposes. The final dataset of financial ratios covered the time period of 2005 to For this study, 31 financial ratios were calculated and used. Table 1 lists and briefly defines these financial ratios. The main tasks/steps employed in this study are presented in a graphical form in Fig Exploratory factor analysis (EFA) The Exploratory factor analysis (EFA) was adopted in order to identify and validate the underlying dimensions of the financial ratios. To locate the underlying dimensions, the principal component factor analysis was used. Principal component analysis (PCA) decomposes given data into a set of linear components within the data. It indicates how a variable contributes to that component, while factor analysis establishes a mathematical model from which factors are estimated (Dunteman, 1989). PCA is a mathematical

3 3972 D. Delen et al. / Expert Systems with Applications 40 (2013) Table 1 List of Financial Ratios. Liquidity Ratios Quick Ratio Liquidity Ratio Cash Ratio Asset Utilization or Turnover Ratios Receivable Turnover Rate Inventory Turnover Rate Net Working Capital Turnover Rate Asset Turnover Rate Equity Turnover Rate Fixed Asset Turnover Rate Long-term Assets Turnover Rate Current Assets Turnover Rate Profitability Ratios Gross Profit Margin EBITDA Margin Net Profit Margin Earnings Before Tax-to-Equity Ratio Return on Equity Return on Assets Operating Expense-to-Net Sales Ratio Growth Ratios Assets Growth Rate Net Profit Growth Rate Sales Growth Rate Asset Structure Ratios Current Assets-to-Total Assets Ratio Inventory-to-Current Assets Ratio Cash and Cash Equivalents-to-Current Assets Ratio Long-term Assets-to-Total Assets Ratio Solvency Ratios Short Term Financial Debt-to-Total Debt Short Term Debt-to-Total Debt Interest Coverage Ratio Debt Ratio Leverage Ratio Total Financial Debt-to-Total Debt (Current Assets Inventory) Current Liabilities Current Assets Current Liabilities Cash and Cash Equivalents Current Liabilities Sales Accounts Receivable Cost of Goods Sold Inventory Sales (Current Assets Current Liabilities) Sales Total Assets Sales Owners Equity Sales Fixed Assets Sales Long-term Assets Sales Current Assets Gross Profit Sales Earnings Before Interest, Tax, Depreciation, and Amortization Sales Net Income Sales Earnings Before Tax Owners Equity Net Income Owners Equity Net Income Total Assets Operating Expense Net Sales (Total Assets t Total Assets t 1 ) Total Assets t 1 (Net Income t Net Income t 1 ) Net Income t 1 (Sales t Sales t 1 ) Sales t 1 Current Assets Total Assets Inventory Current Assets Cash and Cash Equivalents Current Assets Long-term Assets Total Assets Short Term Financial Debt Total Liabilities Current Liabilities Total Liabilities Earnings Before Interest and Tax Interest Total Liabilities Owners Equity Total Liabilities Total Assets Total Financial Debt Total Liabilities procedure which is similar to discriminant function analysis and MANOVA. To begin, a matrix representing the relationships between variables is employed. Following this, the factors (linear components) of the given matrix are calculated by determining the eigenvalues of the matrix. The eigenvectors are calculated by using the determined eigenvalues. Eigenvectors prove the loading of a particular variable on a particular factor (Field, 2005) Outline of PCA Assuming that X mxn is a data matrix, it is a dimensional vector sample in terms of its degree of variance (a higher degree of variance indicates greater significance). PCA determines which vector is significant in the data set. Singular value decomposition (SVD) is employed to transform the data set X mxn into an ordered series of eigenvectors and eigenvalues. The covariance matrix S is obtained for the given data set to produce eigenvectors. The covariance matrix is defined as: S nxn ¼ 1 X T X n where, X mxn = U mxn S mxn V T nxn, U T U = I mxm and V T V = I nxn (I: Identity matrix, U and V: Orthogonal). k 1,k 2,...,k n are the eigenvalues of the covariance matrix and S, k 1 P k 2 P k n P 0 are sorted in order. The proportion of variance between the eigenvectors and the data set is obtained by dividing the eigenvalues to the total sum of the eigenvalues. Eigenvectors are mutually orthogonal to the exiting set of axes. It reduces the sum of squared error distance between the data points and their projections on the component axis. Different degrees of variance are attributed to each eigenvector. The m eigenvectors correspond to the largest m eigenvalues of S, which represent the greatest degree of variance. The first principal component has the highest degree of variance; the second principal component has the second highest degree of variance, and so forth (Kantardzic, 2003) EFA results The EFA procedure started with 29 financial ratio items. The Fixed Asset Turnover Rate, Long-term Assets Turnover Rate and Receivable Turnover Rate were eliminated from the analysis due to their low factor loadings. The remaining 26 items were analyzed again, and the results obtained are presented below (Tables 2 4). PCA with varimax orthogonal rotation was carried out to assess the underlying dimensions of the provided items for financial ratios. As a result of EFA, eleven manageable and meaningful factors were identified. It is crucial to determine the suitability of the data size before factor analysis. Both the KMO (Kaiser Meyer Olkin Measure of Sampling Adequacy) Index and Bartlett s Test of Sphericity were used to check the adequacy of sample size. The KMO index represents the ratio of the squared correlation between variables to the squared partial correlation between variables. The values of KMO range between 0 and 1. Any value close to 1 indicates that the patterns of correlation are compact, and therefore the analysis should

4 D. Delen et al. / Expert Systems with Applications 40 (2013) Fig. 1. Steps followed in the research methodology. result in distinct and reliable factors (Field, 2005). It is considered to be an adequate sample size if the obtained KMO value lies between 0.5 and 1. According to Kaiser (1974), KMO values between.7 and.8 are good; values between.8 and.9 are great; and values above.9 are superb. The sample size of the data set in this study is adequate for use in factor analysis according to KMO test results, since the KMO Index value is 60.3% (Table 2), which is sufficient. In addition, Bartlett s Test of Sphericity signifies whether the R-matrix is an identity matrix. It should be significant at p < 0.05, and it determines whether the population correlation matrix resembles an identity matrix. If there is an identity matrix, every variable correlates poorly with all the other variables, which means correlation coefficients are close to zero, leaving them perfectly independent from each other. In factor analysis, clusters of variables that measure similar things are identified. To determine clusters, the variables should correlate. Therefore, the test provided statistical analysis to prove that the matrix has significant correlations among the variables (Field, 2005). Bartlett s Test of Sphericity demonstrated that it is highly significant, p <.001 (Table 2). This result indicated that the correlation coefficient matrix was not an identity matrix. In addition, it showed that meaningful factors will be obtained as a result of the exploratory factor analysis. Accordingly, the data used in this study is quite sufficient for exploratory factor analysis procedures. The beginning of the factor extraction process is designed to determine the linear components (eigenvectors) within the data sets by calculating the eigenvalues of the correlation coefficient matrix. The largest eigenvalue associated with each of the eigenvectors provided a single indicator of the substantive importance of each component. Factors with relatively large eigenvalues were retained, while those factors with relatively small eigenvalues were omitted. SPSS uses Kaiser s criterion of retaining factors with eigenvalues greater than 1. Table 3 lists the eigenvalues associated with each component (factor). There are twenty-six components, which correlate with twenty-six eigenvectors. It is obvious that the first eleven factors explain relatively large amounts of variance, whereas the rest of the factors explain only small amounts of variance. SPSS by default extracted all factors with eigenvalues greater than 1, which left us with eleven main factors. Table 4 shows the factor loadings together with the communalities. Communality is the proportion of a common variance within a variable. After factors are extracted, the amount of variance in common is revealed under the communalities. In other words, the amount of variance in each variable that could be explained by the retained factors is represented by the communalities after extraction (Field, 2005). Table 4 also demonstrates the factor loadings of each variable and its respective factor, as well as necessary quality indicators such as eigenvalues, the proportion of explained variance, and the cumulative explained variance. The varimax orthogonal rotation of the factor structure clarified the matrix considerably. The suppression of loadings was set to 0.4, to make it easier to interpret the factors. Factors with more than 0.40 loadings were retained while anything less than this value was ignored. Based on the items loading on each factor, they were labeled as Liquidity; Asset Structure; Asset & Equity Turnover Rate; Gross Profit Margin; Financial Debt Ratio; Current Assets; Leverage; Net Profit Margin; Net Working Capital Turnover Rate; Sales & Profit Growth Rate; and Asset Growth Rate. The twenty-six variables of exploratory factor analysis results indicated that these factors explained 70.4% of the total variance. The results demonstrated that 11.5% of that variance was accounted for by the first factor (Liquidity) while the second (Asset Structure) and the third factors (Asset & Equity Turnover Rate) accounted for 9.6% and 9.1% respectively. The results also revealed that more than 30% of the variance was accounted for by the first three factors. Factor 1: Liquidity: The liquidity factor was the most significant, explaining 11.48% of the total variance. Three ratios: Liquidity Ratio, Cash Ratio, and Quick Ratio were loaded under this factor. The loaded variables were all positive, having high factor loadings values of 0.996, and respectively. These ratios ranked

5 3974 D. Delen et al. / Expert Systems with Applications 40 (2013) equally at the high end of the loadings, and predicted the ability of a firm to pay a short-term debt. Factor 2: Asset Structure: Three ratios: Long-Term Assets-to-Total Assets Ratio, Current Assets-to-Total Assets Ratio, and Short Term Debt-to-Total Debt Ratio were loaded under this factor. These ratios were named Asset Structure, comprising the second most significant factor, and explained 9.59% of the total variance. While the Current Assets-to-Total Assets (0.903) and Short Term Debt-to- Total Debt (0.771) ratios were loaded positively, the Long-Term Assets-to-Total Assets ( 0.910) ratio had a negative high loading value. This is due to the fact that the size of the Long-Term Assets and Current Assets inversely affected each other: when the percentage of one was increased, the percentage of the other decreased. Factor 3: Asset & Equity Turnover Rate: The third factor was named Asset & Equity Turnover Rate, and explained 9.1% of the total variance. It is an equally important Asset Structure factor. Three ratios: Asset Turnover Rate, Equity Turnover Rate, and Current Assets Turnover Rate were loaded under this factor. The loaded values were all positive and had relatively strong values, with 0.914, 0.895, and respectively. The Asset Turnover Rate had the highest load. These three ratios indicate how efficiently a company uses its assets and equity to generate sales revenues. Factor 4: Gross Profit Margin: The fourth factor had very high loadings on the EBITDA (Earnings before interest, taxes, depreciation and amortization) Margin (0.934) and the Gross Profit Margin (0.930). Both ratios were very strong and loaded positively. This factor was named the Gross Profit Margin, explaining 6.95% of the variations, and contributing adequately to the overall explained variation. While the Gross Profit Margin measures how much a company controls its costs, the EBITDA measures how it controls operating expenses, along with costs. Factor 5: Financial Debt Ratio: The Short Term Financial Debt-to- Total Debt Ratio (0.890) and the Total Financial Debt-to-Total Debt Ratio (0.882) were loaded positively and strongly under this factor. It was named Financial Debt Ratio and explained 6.58% of the variations. This was equally as important as the Gross Profit Margin, in terms of explained variations. These two ratios indicated the amount of financial debt accruing interest within a total debt. An increase in this ratio indicates an increase in the interest burden of the company, and, eventually, financial distress. Factor 6: Current Assets: The sixth factor was named Current Assets, and explained 5.29% of the variance. It had a strong negative loading of Inventory-to-Current Assets ( 0.726) and a strong positive loading on the Cash and Cash Equivalents-to-Current Assets (0.713) Ratios, while it had a weak loading on the Inventory Turnover Rate (0.456) Financial Ratio. These ratios provided information regarding the structure of current assets. The negative loading of Inventory-to-Current Assets indicated the inverse relationship with the Inventory Turnover Rate. As inventories within Current Assets increased, the Inventory Turnover Rate decreased. Table 2 KMO and Bartlett s test results. Kaiser Meyer Olkin Measure of Sampling Adequacy Bartlett s Test of Sphericity v Degree of Freedom 325 Significance Table 3 Total variance explained by initial eigen values (Factors with eigenvalue greater than 1 are in bold). Component Total % of Variance Cumulative % Factor 7: Leverage: This factor explained only 4.83% of the variations, and was named Leverage. The main ratios loaded under this factor were the Earnings Before Tax-to-Equity Ratio (0.696) and the Debt Ratio (0.685). The Leverage Ratio (0.439) had a moderate factor loading. All of these ratios were positive. Factor 8: Net Profit Margin: The eighth factor was named Net Profit Margin, and explains 4.81% of the variance. It was effective as leverage in terms of variance explanations. The Operating Expense-to-Net Sales Ratio (0.791) had the highest positive loading on the Net Profit Margin. The Net Profit Margin (0.770) also had a strong positive loading value on this factor. These ratios demonstrate the profitability of a company relative to its sales revenues. Factor 9: Net Working Capital Turnover Rate: There were two ratios under this factor, named Net Working Capital Turnover Rate. These explained 3.99% of the variance. The Net Working Capital Turnover Rate had the highest positive factor, loading with a value, while the Interest Coverage Ratio had a moderate negative factor, loading with a value. Opposite loading signs might be attributable to the fact that as current liabilities increased, the net working capital turnover rate increased, and this might have decreased the Interest Coverage Ratio. Factor 10: Sales & Profit Growth Rate: The tenth factor was named the Sales & Profit Growth Rate. It explained only 3.92% of the variance, which was weak. The Sales Growth Rate Ratio had a high positive loading, with a factor loading of The Net Profit Growth Rate Ratio also had a medium positive factor loading with a value of The Sales & Profit Growth Rates measure the difference in sales and profits in the current year relative to the previous year. Factor 11: Asset Growth Rate: This is the last factor. It explains 3.89% of the variance. There was a single variable loaded under this factor, which was named Assets Growth Rate. It had a very strong

6 Table 4 Varimax rotated component matrix and communalities. Items (Financial Ratios) Components Communalities Liquidity (F1) Asset Structure (F2) Asset & Equity turnover rate (F3) Gross Profit margin (F4) Financial debt ratio (F5) Current assets (F6) Leverage (F7) Net profit margin (F8) Net working capital turnover rate (F9) Sales & Profit growth rate (F10) Asset growth rate (F11) Liquidity Ratio Cash Ratio Quick Ratio Long-term Assets-to Total Assets Ratio Current Assets-to-Total Assets Ratio Short Term Debt-to- Total Debt Asset Turnover Rate Equity Turnover Rate Current Assets Turnover Rate EBITDA Margin Gross Profit Margin Short Term Financial Debt-to-Total Debt Total Financial Debt-to- Total Debt Inventory-to-Current Assets Ratio Cash and Cash Equivalents-to- Current Assets Inventory Turnover Rate Earnings Before Tax-to Equity Ratio Debt Ratio Leverage Ratio Operating Expense-to Net Sales Ratio Net Profit Margin Net Working Capital Turnover Rate Interest Coverage Ratio Sales Growth Rate Net Profit Growth Rate Assets Growth Rate Eigenvalue a Variance explained (%) Cumulative variance explained (%) Notes: Extraction method: principal component analysis; rotation method: Varimax with Kaiser normalization. a Values obtained after rotation. D. Delen et al. / Expert Systems with Applications 40 (2013)

7 3976 D. Delen et al. / Expert Systems with Applications 40 (2013) positive factor loading value of This ratio provides information regarding the increase in assets in the current year relative to the previous year Decision tree algorithms Decision trees are commonly used methods in data mining. There are two main types of tasks for decision trees: classification tree analysis and regression tree analysis. Decision trees are becoming increasingly more popular for data mining because they are easy to understand and interpret, require little data preparation, handle numerical and categorical data, and they perform very well with a large data set in a short time. Decision trees produce excellent visualizations of results and their relationships. Although there are many specific decision tree algorithms, the ID3, C4.5, C5.0, C&RT, and CHAID and QUEST algorithms are the most commonly used ones. CHAID: Chi-squared Automatic Interaction Detector (CHAID) is an extremely effective statistical technique developed by Kass (1980). Its main use is for segmentation, or tree growing. CHAID is a decision tree technique based on adjusted significance testing. It can be used for predictions in the same way for regression analysis and classification as well as detecting interaction between variables. Differing from other decision tree techniques, CHAID can produce more than two categories at any level in the tree; therefore it is not a binary tree method. Its output is highly visual and easy to interpret since it uses multi-way splits by default. It creates a wider tree than the binary growing methods. This algorithm works for any type of variable since it accepts both case weights and frequency variables. CHAID handles missing values by treating them all as a valid single category. C5.0: This was developed by Quinlan (1993). It offers a number of improvements on C4.5: it is significantly faster than C4.5; it is more memory efficient than C4.5; it creates a considerably smaller decision tree while producing similar results; it boosts the trees, improving them and creating more accuracy; it makes it possible to weight different attributes and misclassification types; as well, it automatically winnows the data to help reduce noise. As a result, it improves the objectivity and precision of the decision tree classification algorithm. Boosting is part of the C5.0 decision tree algorithm as an integration technology which improving the accuracy of classification. It also uses pre-pruning and post-pruning methods to establish the decision tree, starting from the top level of the details. The set of training examples is partitioned into two or more subsets, based on the outcome of a test of the value of a single attribute. The particular test is chosen by an information theoretic heuristic that generally gives close to optimal partitioning. This is repeated on each new subset until a subset contains only examples of a single class, or the partitioning tree has reached a predetermined maximum depth. C&RT: Classification and Regression Trees were established by Breiman, Friedman, Olshen, and Stone (1984). C&RT is a binary decision tree algorithm capable of processing continuous or categorical predictor or target variables. It works recursively: data is partitioned into two subsets to make the records in each subset more homogeneous than in the previous subset; the two subsets are then split again until the homogeneity criterion or some other stopping criteria is satisfied. The same predictor field may be used many times in the tree. The ultimate aim of splitting is to determine the right variable associated with the right threshold to maximize the homogeneity of the sample subgroups. In addition, C&RT handles missing values by using surrogate splitting to make the best use of the data. This algorithm produces a sequence of nested pruned trees, each of which can be optimal. The right size is determined by evaluation of the predictive performance of each tree in the pruning sequence on the independent test data or via cross-validation, rather than using internal data (training-data-based). Selection of the optimal tree proceeds after test-data-based evaluation. This mechanism provides optional automatic class balancing as well as missing value handling and allows cost-sensitive learning. QUEST: The Quick, Unbiased, Efficient Statistical Tree (QUEST) algorithm is a relatively new binary-split decision tree algorithm for classification and data mining (Loh & Shih, 1997). It is similar to the C&RT algorithm (Breiman et al., 1984). However, there are some minor differences. For instance, QUEST employs an unbiased variable selection method, uses imputation for dealing with missing values instead of surrogate splits, and handles categorical variables with many categories. It deals with split selection and split-point selection separately. The univariate split performs unbiased field selection, which means that if all the predictor fields are equally informative with respect to the target field, it chooses any of the predictor fields with equal probability. It produces unmanageable trees, but it allows for applying automatic cost-complexity pruning to minimize their size (SPSS, 2007) Decision tree comparative analysis A total of twenty-six inputs (independent variables) and two outputs (dependent variables) were implemented. The independent input variables are: Liquidity Ratio; Cash Ratio; Quick Ratio; Long-Term Assets-to-Total Assets Ratio; Current Assets-to-Total Assets Ratio; Short Term Debt-to-Total Debt; Asset Turnover Rate; Equity Turnover Rate; Current Assets Turnover Rate; EBITDA Margin; Gross Profit Margin; Short term Financial Debt-to-Total Debt; Total Financial Debt-to-Total Debt; Inventory-to-Current Assets Ratio; Cash and Cash Equivalents-to-Current Assets; Inventory Turnover Rate; Earnings Before Tax-to-Equity Ratio; Debt Ratio; Leverage Ratio; Operating Expense-to-Net Sales Ratio; Net Profit Margin; Net Working Capital Turnover Rate; Interest Coverage Ratio; Sales Growth Rate; Net Profit Growth Rate; and Assets Growth Rate. The dependent variables as outputs are Return On Equity (ROE) and Return On Assets (ROA); they were entered into the models as binary variables. These output variables represent the financial performance of companies. Central tendency measure (median) values for return on equity and return on assets were employed as a split criterion: the class with a performance score above the median values was rated as successful and the class with a performance score below the median values was rated as unsuccessful. Therefore, the binary variables as a performance measure of each company were identified as either successful or unsuccessful. The performance of models used in binary (two-groups) is often measured by using a confusion matrix (Table 5). A confusion matrix contains valuable information about the actual and predicted classifications created by the classification model (Kohavi & Provost, 1998). For purposes of this study, we used well-known performance measures such as overall accuracy, AUC (Area Under ROC Curve), Recall and F-measure. All of these measures were used to evaluate each model in the study, after which the models were compared on the basis of the proposed performance measurements List of performance measures Overall Accuracy (AC): Accuracy is defined as the percentage of records that are correctly predicted by the model. It is also defined as being the ratio of correctly predicted cases to the total number of cases.

8 D. Delen et al. / Expert Systems with Applications 40 (2013) Table 5 Confusion matrix for financial performance of firms. Predicted Actual Unsuccessful Successful Unsuccessful True Negative False Positive Successful False Negative True Positive TP þ TN Accuracy ¼ TP þ TN þ FP þ FN Precision: Precision is defined as the ratio of the number of True Positive (correctly predicted cases) to the sum of the True Positive and the False Positive. Recall: Recall is also known as the Sensitivity or True Positive rate. It is defined as the ratio of the True Positive (the number of correctly predicted cases) to the sum of the True Positive and the False Negative. F-Measure: F-measures take the harmonic mean of the Preci- Table 6 Prediction results for return on equity. Model type a Accuracy (AC) Sensitivity/True Positive Rate/Recall (TP) Specificity/True Negative Rate (TN) False Positive rate (FP) False Negative Rate (FN) Precision (P) F- Measure CHAID C C&RT QUEST a Area Under Curve (AUC) Acronyms for model types: CHAID: Chi-squared Automatic Interaction Detector; C&RT: Classification and Regression Trees; QUEST: Quick, Unbiased, Efficient Statistical Tree; C5.0: Extension of C4.5 and ID3 decision tree algorithms. Table 7 Confusion (coincidence) matrices of each decision tree model based on test data set (output variable: Return on Equity). Model type Unsuccessful (0) Successful (1) Overall accuracy (%) Per-class accuracy (%) C5.0 Unsuccessful (0) Correct Successful (1) Wrong Sum C&RT Unsuccessful (0) Correct Successful (1) Wrong Sum QUEST Unsuccessful (0) Correct Successful (1) Wrong Sum CHAID Unsuccessful (0) Correct Successful (1) Wrong Sum Table 8 Prediction results for Return on Assets. Model type Accuracy (AC) Sensitivity/True Positive Rate/Recall (TP) Specificity/True Negative Rate (TN) False Positive Rate (FP) False Negative Rate (FN) Precision (P) F- Measure CHAID C C&RT QUEST Area Under Curve (AUC) Table 9 Confusion (coincidence) matrices of each decision tree model based on test data set (output variable: Return on Assets). Model type Unsuccessful (0) Successful (1) Overall Accuracy (%) Per Class Accuracy (%) C5.0 Unsuccessful (0) Correct Successful (1) Wrong Sum C&RT Unsuccessful (0) Correct Successful (1) Wrong Sum QUEST Unsuccessful (0) Correct Successful (1) Wrong Sum CHAID Unsuccessful (0) Correct Successful (1) Wrong Sum

9 3978 D. Delen et al. / Expert Systems with Applications 40 (2013) Fig. 2. Evaluation of Testing Data Set (Gain Chart) for Return on Equity. Fig. 3. Evaluation of Testing Data Set (Gain Chart) for Return on Assets. sion and Recall Performance measures. Therefore, it takes into consideration both the Precision and the Recall Performance as being important measurement tools for these calculations (Witten & Frank, 2005). Precision Recall F measure ¼ 2 Precision þ Recall Specificity: This is also known as the True Negative Rate (TN). It is defined as the ratio of the number of the True Negative to the sum of the True Negative and the False Positive Decision Tree Analysis Results In this study, decision tree algorithms were used to identify the best performing classification models. Four types of decision tree algorithms were employed: CHAID; C&RT; C5.0; and QUEST. These algorithms were tested for return on equity and for return on assets using holdout samples. To determine how well our models worked with data in the real world, we held back a subset of records for testing and validation purposes. Therefore, the data set was split for training and testing. 70% of the data was used for training to generate the model, and 30% was used to test it. For performance analysis, the test data sets were used for assessment. Analysis results were examined in two sections. In the first part of the analysis, the Return on Equity (ROE) Ratio was a dependent variable (Tables 6 and 7) while the Return on Assets (ROA) was a dependent variable in the second section of the analysis (Tables 8 and 9) Examining the Results for the Dependent Variable, Return on Equity (ROE). According to the overall accuracy rate, the CHAID model demonstrated the highest performance level (93.2%) and the C5.0 model had the second highest performance measurement

10 D. Delen et al. / Expert Systems with Applications 40 (2013) Table 10 Aggregated variable importance values of financial ratios for Return on Equity. Financial ratios Decision tree model types CHAID C5.0 C&RT QUEST V (Fused) Asset Turnover Rate (F3) Assets Growth Rate (F11) Cash & Cash Equivalents-to-Current Assets (F6) Cash Ratio (F1) Current Assets Turnover Rate (F3) Current Assets-to-Total Assets Ratio (F2) Debt Ratio (F7) Earnings Before Tax-to-Equity Ratio (F7) EBITDA Margin (F4) Equity Turnover Rate (F3) Gross Profit Margin (F4) Interest Coverage Ratio (F9) Inventory Turnover Rate (F6) Inventory-to-Current Assets Ratio (F6) Leverage Ratio (F7) Liquidity Ratio (F1) Long-term Assets-to-Total Assets Ratio (F2) Net Profit Growth Rate (F10) Net Profit Margin (F8) Net Working Capital Turnover Rate (F9) Operating Expense-to-Net Sales Ratio (F8) Quick Ratio (F1) Sales Growth Rate (F10) Short Term Debt-to-Total Debt (F2) Short term Financial Debt-to-Total Debt (F5) Total Financial Debt-to-Total Debt (F5) Table 11 Aggregates variable importance values of financial ratios for Return on Assets. Financial ratios Decision tree model types CHAID C5.0 C&RT QUEST V (Fused) Asset Turnover Rate (F3) Assets Growth Rate (F11) Cash and Cash Equivalents-to-Current Assets (F6) Cash Ratio (F1) Current Assets Turnover Rate (F3) Current Assets-to-Total Assets Ratio (F2) Debt Ratio (F7) Earnings Before Tax-to-Equity Ratio (F7) EBITDA Margin (F4) Equity Turnover Rate (F3) Gross Profit Margin (F4) Interest Coverage Ratio (F9) Inventory Turnover Rate (F6) Inventory-to-Current Assets Ratio (F6) Leverage Ratio (F7) Liquidity Ratio (F1) Long-term Assets-to-Total Assets Ratio (F2) Net Profit Growth Rate (F10) Net Profit Margin (F8) Net Working Capital Turnover Rate (F9) Operating Expense-to-Net Sales Ratio (F8) Quick Ratio (F1) Sales Growth Rate (F10) Short Term Debt-to-Total Debt (F2) Short Term Financial Debt-to-Total Debt (F5) Total Financial Debt-to-Total Debt (F5) (92.6%). Even though the C&RT and the QUEST models did not perform as well as the CHAID and the C5.0, they still produced a considerably high overall prediction rate of 88.2% and 83.5% respectively. The CHAID decision tree model significantly outperformed in terms of AUC, F-measure, and sensitivity performance measurements. However, the C5.0 decision tree model also revealed high performance in terms of specificity and precision performance measurements; at the same time, these measures are not significantly higher than the other models performances (Table 6). Prediction accuracy for the Successful class was significantly higher than the prediction accuracy of the Unsuccessful class in all four decision tree models (Table 7). The coincidence matrix showed that all the decision tree models predicted the Successful companies in terms of ROE with better than 90% accuracy while the CHAID, C5.0 and C&RT DT models also revealed successful pre-

11 3980 D. Delen et al. / Expert Systems with Applications 40 (2013) Fig. 4. Representation of sensitivity analysis result for Return on Equity (ROE). diction results on predicting Unsuccessful companies in terms of ROE, with almost 92%, 89%, and 81% respectively (Table 7) Examining the Results for the Dependent Variable, Return on Equity (ROE). The results obtained by investigating the Return on Equity (ROE) dependent variable revealed that the CHAID model performed significantly better than the other decision tree models with a 92.1% overall accuracy rate, a 95.6% sensitivity rate, an 86% specificity rate, a 92.2% precision rate, a 93.9% F-measure, and a 97% AUC rate. The C5.0 model s performance measure was second best, with a 90.1% accuracy rate. The C&RT and the QUEST models demonstrated lower accuracy rates with 85.6% and 73.2% overall accuracy rates respectively. The CHAID and the C5.0 powerful decision tree models consistently revealed significant performance measures in terms of sensitivity specificity, precision F-measure and AUC (Table 8). As well, the CHAID and the C5.0 models demonstrated strong performance measures when investigating the ROE output variable as well. For ROA, the coincidence matrix results revealed valuable information. Prediction accuracy for the Successful case was higher in the CHAID, C&RT and QUEST models than for the Unsuccessful case. The CHAID model predicted Successful companies in terms of ROA with better than 92% accuracy, while predicting Unsuccessful companies with almost 92% accuracy. In comparison, the C5.0 model predicted Unsuccessful companies with almost 91% accuracy, while predicting Successful firms in terms of ROA with almost 90% accuracy rate. The C&RT and QUEST models predicted Successful and Unsuccessful companies with almost 86% accuracy and over 70% accuracy respectively (Table 9) Graphical representation of performance measures by using gain charts. Gains are defined as the proportion of total hits that occur in each quantile; they are computed as the (number of hits in quantile/ total number of hits) 100%. The Gain Charts rise steeply towards 100% and then level off in a good model (SPSS, 2007). The graphical representation of performance measures for each