Assessing the Accuracy of Small Firms Internal Control Disclosures

Transcription

1 Assessing the Accuracy of Small Firms Internal Control Disclosures Weili Ge Associate Professor Foster School of Business University of Washington Allison Koester Assistant Professor McDonough School of Business Georgetown University Sarah McVay Associate Professor Foster School of Business University of Washington Current Draft: February 2014 First Draft: April 2013 Acknowledgements: We thank Joe Carcello, Asher Curtis, Linda Myers, Zoe-Vonna Palmrose, Hollis Skaife, workshop participants at the University of Tennessee, conference participants at the 2014 FARS Mid-Year Meeting, and brownbag participants at Georgetown University for their valuable feedback. Koester acknowledges financial support from Georgetown University s Center for Financial Markets and Policy. Electronic copy available at:

2 Assessing the Accuracy of Small Firms Internal Control Disclosures ABSTRACT We investigate the accuracy of small firms internal control disclosures. Our prediction model suggests that 22.2% of non-accelerated filers maintain ineffective internal controls, whereas only 15.7% of these firms disclose ineffective controls in their Section 404(a) reports, suggesting an additional 42% of firm-years should have disclosed ineffective internal controls. We provide evidence that the financial reporting quality of suspected misreporters is lower than that of firms that credibly disclose effective internal controls. In fact, we find that suspected misreporters generally have financial reporting quality that is indistinguishable from firms disclosing ineffective internal controls. Although these findings suggest that a material number of small firms are misreporting the effectiveness of their internal controls under Section 404(a), we find only modest evidence that requiring these firms to comply with Section 404(b) would curb this misreporting. Overall, our evidence supports the conclusions of Kinney and Shepardson (2011) who favor limiting costly internal control disclosure regulation. Key words: internal controls over financial reporting; non-accelerated filers; financial reporting quality Electronic copy available at:

3 Assessing the Accuracy of Small Firms Internal Control Disclosures 1. Introduction This paper examines the accuracy of non-accelerated filers internal control disclosures. Although both management and a firm s external auditor assess the effectiveness of internal controls of accelerated filers, non-accelerated filers are permitted to forgo the external auditor assessment. 1 Despite non-accelerated filers being subject to less oversight, Kinney and Shepardson (2011) document that these firms disclose ineffective internal controls at a similar frequency as small accelerated filers and incur a fraction of the audit fees. These findings lead the authors to conclude that for small firms, management internal control reports and traditional financial audits may be a cost effective disclosure alternative to full application of SOX 404(b) (p. 413). Smaller firms, however, generally have fewer resources available to devote to implementing and maintaining a sophisticated financial reporting system and fewer employees available to ensure duties are fully segregated. Although they also generally have fewer complicated accounting issues, prior research illustrates that smaller firms tend to more often maintain ineffective controls (e.g., Ge and McVay, 2005). Bedard and Graham (2011) find that auditors detect about three-fourths of internal control deficiencies for smaller accelerated filers, indicating the importance of auditor oversight; therefore, it is likely there are a large number of smaller firms that fail to discover or disclose ineffective internal controls absent auditor 1 Throughout this paper we use the terms non-accelerated filers and small firms interchangeably and we use the term internal controls to refer to internal controls over financial reporting. Section 404(a) of the Sarbanes-Oxley Act of 2002 requires management s assessment of a firm s internal control effectiveness while Section 404(b) requires external auditor assessment. The term accelerated filer (non-accelerated filer) refers to public firms with greater (less) than $75 million in public float. Public float is defined under SEC Rule 12b-2 as aggregate worldwide market value of [an issuer s] voting and non-voting common equity held by its non-affiliates...as of the last business day of the issuer s most recently completed second fiscal quarter ( 2.html). Public float is meant to serve as a reasonable measure of company size and investor interest. We discuss the internal control disclosure rules in greater detail in the next section. 1

4 oversight. This failure would weaken the desired investor protection of Section 404. Consistent with this argument, Krishnan and Yu (2012) and Holder et al. (2013) conclude that exempting small firms from Section 404(b) has compromised financial reporting quality. In an October 2011 letter to the U.S. House of Representatives Financial Services Committee opposing exemptions from 404(b), the Charted Financial Analysts Institute, Center for Audit Quality, and Council of Institutional Investors argued that compliance with Section 404(b) has contributed to an increase in overall audit quality in the years since the passage of SOX and that investors should have the same level of protection regarding the effectiveness of firms internal controls regardless of firm size (CFA, 2011). Therefore, whether some firms should be exempted from auditor oversight of internal control disclosures remains unresolved. Non-accelerated filers were permanently exempted from Section 404(b) compliance in July 2010, and legislators continue to explore expanding the exemption, increasing the importance of the cost/benefit debate (Gupta et al., 2013). As of 2011, more than 6,000 publicly traded U.S. firms were exempted from 404(b) compliance (GAO, 2013). The deliberations and implications of this debate are important as a 404(b) exemption expansion would significantly increase the already large number of firms currently exempt. Determining whether larger firms should be exempt from 404(b) is difficult, as the counterfactual (i.e., the costs and benefits of these firms not complying with 404(b)) is unobservable. We shed light on the above debate by investigating the accuracy of internal control disclosures by small firms subject to only 404(a). Specifically, we estimate the extent to which small firms misreport internal control effectiveness when they are not subject to an independent auditor s assessment of their internal controls. Evidence suggesting these firms internal control disclosures are accurate provides evidence that 404(b) could be unnecessary. In 2

5 contrast, evidence suggesting these firms internal control disclosures are inaccurate suggests these firms could benefit from auditor oversight. To infer the degree of internal control effectiveness misreporting under Section 404(a), we first develop a model to predict small firms internal control effectiveness using data from the smallest accelerated filers (e.g., firms subject to 404(b) with end-of-year market capitalizations of $300 million or less in the three years surrounding the November 15, 2004 Section 404(b) effective date). 2 We use this model to generate out-of-sample estimates of internal control quality for small firms not subject to 404(b). We validate the out-of-sample estimates using several approaches, all of which consistently suggest our model is able to identify both accurate and inaccurate ineffective internal controls disclosures. 3 Based on this model, we infer that 22.2% of non-accelerated filers disclose effective internal controls but should be disclosing ineffective internal controls, whereas only 15.7% of these firms acknowledge ineffective controls in their Section 404(a) reports. This suggests that the number of non-accelerated filers disclosing ineffective internal controls should be approximately 42% higher than the frequency observed. We next explore how financial reporting quality varies among our suspected misreporters relative to other firms. Using our prediction model, we document that potential misreporters (all of whom disclose effective internal controls but are predicted to maintain ineffective internal controls) exhibit higher incidences of future restatements, lower working capital accruals quality, and larger absolute values of discretionary accruals relative to small firms that credibly disclose effective internal controls. Interestingly, the financial reporting quality of potential misreporters is generally indistinguishable from the financial reporting quality of small firms that disclose ineffective internal controls. This finding helps to validate our model and is consistent with our 2 Alternate market capitalization thresholds are discussed in Section Despite our validity tests, we acknowledge that our model cannot perfectly predict internal control effectiveness and our inferences are limited by the power of our prediction model. 3

6 assessment that potential misreporters inaccurately conclude their internal controls are effective, on average. Our financial reporting quality results indicate that auditors of small firms are not able to audit around the deficient internal controls, and thus the accurate reporting of these firms low internal control quality would be indicative of their low financial reporting quality. Thus, accurate internal control effectiveness disclosures would provide investors with information about firms low financial reporting quality. It does not immediately follow, however, that our suspected misreporters would have accurately disclosed ineffective internal controls had they been subject to 404(b), as the PCAOB notes that one in every three audits do not obtain the supporting documentation necessary to provide an audit opinion (Chasan, 2014) and there is known misreporting under 404(b) (Rice and Weber, 2012). Thus, we benchmark the magnitude of misreporting among non-accelerated filers against the analogous estimate of misreporting among small accelerated filers to assess the incremental accuracy of firms subject to 404(b). 4 We find that the magnitude of estimated misreporting among small accelerated filers is 36%, which is only slightly lower than the 42% misreporting by non-accelerated filers. This finding suggests that internal control misreporting applies to both small accelerated filers and non-accelerated filers, and auditor oversight under Section 404(b) provides only a modest incremental reduction of 6% in misreporting over 404(a). Even if we find that a material amount of non-accelerated filers falsely claim their internal controls are effective, this misreporting is only costly if investors benefit from more accurate disclosures; i.e., if the internal control report contains a material amount of new information. Our final analysis attempts to shed light on the economic importance of misreporting. In particular, curbing misreporting by 6% could be economically meaningful if the 4 In particular, we estimate the prediction model on firms with a market capitalization between $300 and $600 million and apply this prediction model to small accelerated filers (i.e., accelerated filers with an end-of-year market capitalization of $300 million or less in the year prior to, during, or after the first year Section 404(b) was effective). 4

7 disclosure of ineffective internal controls provides new information to investors. Although prior research finds that internal control disclosures made by accelerated filers are largely uninformative to investors (Beneish et al., 2008), non-accelerated filers operate in much weaker information environments, suggesting their disclosures may be more informative. We find little evidence of stock price reactions to the disclosure of ineffective internal controls by nonaccelerated filers under 404(a). This is consistent with investors being able to infer this information prior to the disclosures, suggesting formal disclosure of internal control ineffectiveness provides little new information to investors. Our study contributes to the debate over the costs and benefits of internal control disclosure regulation. We provide evidence that a larger proportion of non-accelerated filers subject to only 404(a) fail to disclose internal control weaknesses relative to small accelerated filers subject to 404(b), but that the difference is modest (42% versus 36%), indicating the rather limited benefit of Section 404(b) in curtailing misreporting. In addition, disclosures of internal controls ineffectiveness do not appear to be informative to non-accelerated filer investors. Given the high audit costs of 404(b), our results support the conclusions of Kinney and Shepardson (2011) who favor limiting costly internal control disclosure regulation. The remainder of the paper is organized as follows. Section 2 discusses the background and our predictions. Section 3 describes our sample and descriptive statistics. Section 4 presents our empirical results, and Section 5 concludes. 2. Background, Related Literature, and Hypothesis Development 2.1 Background The Sarbanes-Oxley Act of 2002 (SOX) contains three sections (302, 404a, and 404b) that relate to the disclosure of internal control effectiveness. Under Section 302 (effective for all 5

8 publicly traded firms for fiscal periods ending on or after August 29, 2002), management is required to have evaluated the effectiveness of their firm s disclosure controls and procedures and present its conclusion on the effectiveness of the firm s disclosure controls and procedures within each Form 10-Q/10-K (SEC, 2002). Section 404(a) requires companies to include management s assessment of the effectiveness of the firm s internal control structure and procedures in the firm s annual report, while Section 404(b) requires an auditor-provided opinion regarding the assessment of the same internal control structures and procedures to be included in the annual report (U.S. Congress, 2002). We focus on disclosures under Section 404 and not Section 302 because Section 404: (1) has stricter documentation and more detailed disclosure requirements; (2) offers the opportunity to study differences in manager versus auditor attestation requirements and the value of auditor attestation (404(b)) in particular is still a subject of debate; and (3) has compliance costs that are considered to be more onerous (Palmrose, 2010). 5 Amid the debates regarding the costs and benefits of Section 404, the costs Section 404 imposes on small firms is of particular concern. In response to this concern, and to partially alleviate the pressure of Section 404 compliance on small firms, the SEC granted nonaccelerated filers multiple Section 404 compliance deadline extensions. Section 404(a) became 5 Section 404 imposes stricter internal control disclosure requirements on publicly traded firms than Section 302. While Section 404(a) guidance states that management is responsible for maintaining evidential matter, including documentation, to provide reasonable support for its assessment [of] whether internal controls over financial reporting are effective (SEC, 2007), Section 302 does not require management to retain evidence supporting its internal controls evaluation process. In addition, Section 404(a) guidance states that because the goal underlying all disclosure in this area is to provide an investor with disclosure and analysis that goes beyond describing the mere existence of a material weakness.companies should also consider providing disclosure that allows investors to understand the cause of the control deficiency and to assess the potential impact of each particular material weakness (SEC, 2007). The SEC defines a material weakness as a deficiency, or a combination of deficiencies, in [internal controls over financial reporting] such that there is a reasonable possibility that a material misstatement of the registrant s annual or interim financial statements will not be prevented or detected on a timely basis (SEC, 2007, as defined in Exchange Act Rule 12b-2 [17 CFR b-2] and Rule 1-02 of Regulation S-X [17 CFR ]). As Section 302 guidance does not encourage managers to disclose anything other than their evaluation on the effectiveness of the firm s internal controls, Section 404 disclosures are generally more detailed than Section 302 disclosures. Section 404 is also considered to be the most controversial of the SOX requirements because of its compliance costs (Palmrose, 2010). 6

9 effective for non-accelerated filers for fiscal years ending on or after December 15, 2007, more than three years after the effective date for accelerated filers to comply with both Sections 404(a) and 404(b). Although Section 404(b) compliance for non-accelerated filers was expected to begin for fiscal years ending on or after June 15, 2010, lawmakers permanently exempted nonaccelerated filers from complying with Section 404(b) in July 2010 in Section 989G of the 2010 Dodd-Frank Wall Street Reform and Consumer Protection Act (U.S. Congress, 2010). Legislators continue to explore expanding the exemption, increasing the importance of the cost/benefit debate (Gupta et al., 2013). For example, in 2011 the Securities and Exchange Commission (SEC) completed a Congressionally mandated study regarding expanding the Section 404(b) exemption to include firms with up to $250 million in public float (SEC, 2011) and the Small Company Job Growth and Regulatory Relief Act of 2011 called for exempting firms with up to $500 million in public float (U.S. Congress, 2011). Some of these proposals are gaining traction. For example, the Jumpstart our Business Startups (JOBS) Act signed into law in April 2012 exempts new public companies classified as emerging growth firms from Section 404(b) compliance for five years (U.S. Congress, 2012). 6 As of 2011 more than 6,000 publicly traded U.S. firms were exempt from 404(b) compliance (GAO, 2013). 2.2 Related Literature Initial research findings examining non-accelerated filers 404(a) reports are mixed. In support of 404(b) opponents, Kinney and Shepardson (2011) find that a similar proportion of non-accelerated filers disclose ineffective internal controls under Section 404(a) relative to a benchmark group of small accelerated filers that complied with Sections 404(a) and 404(b) and incurred only a fraction of the audit fees paid by small accelerated filers. The authors conclude 6 These firms are no longer exempt from 404(b) if their revenues exceed $1 billion, their public float exceeds $700 million, or they issue more than $1 billion in nonconvertible debt during a three year period (U.S. Congress, 2012). 7

10 by stating that for small firms, management internal control reports and traditional financial audits may be a cost effective disclosure alternative to full application of SOX 404(b) (p. 413). Further, Kinney et al. (2013) question the value of 404(b) by arguing that it is difficult for auditors to identify material weaknesses in internal control absent a financial misstatement. They go on to note that, even under existing auditing standards (AU 550), auditors would apply additional procedures if management states that its internal controls are effective when the auditor disagrees. In contrast, Krishnan and Yu (2012) and Holder et al. (2013) conclude that exemption from 404(b) has compromised the financial reporting quality of non-accelerated filers. These studies attempt to infer the benefits of 404(b) compliance by comparing financial reporting quality between accelerated and non-accelerated filers. Specifically, Krishnan and Yu (2012) document that discretionary revenues are higher for non-accelerated filers than for small accelerated filers, and Holder et al. (2013) show that discretionary accruals and working capital accruals quality has improved post-sox for accelerated filers but has deteriorated for nonaccelerated filers. These studies offer support for the proponents of 404(b); however, they do not examine whether small firms accurately disclose internal control effectiveness, which is the key contribution of our paper. 2.3 Hypothesis Development We investigate how accurately non-accelerated filers identify ineffective internal controls under Section 404(a). The disclosure of ineffective internal controls is a joint function of the (1) existence of internal control weaknesses and (2) incentives to detect and disclose internal control weaknesses (e.g., Ashbaugh-Skaife et al., 2007). Our interest is in how well Section 404(a) disclosures accurately describe firms underlying internal control effectiveness. Kinney and 8

11 Shepardson (2011) document that a similar proportion of non-accelerated filers subject only to Section 404(a) and small accelerated filers subject to both Sections 404(a) and 404(b) disclose ineffective internal controls, which they interpret as a lack of evidence of misreporting among non-accelerated filers. The authors implicit assumption is that the underlying distribution of internal control issues is the same for both non-accelerated and small accelerated filers. The authors are unable to conclude that a similar proportion of all non-accelerated filers with ineffective internal controls accurately identify and disclose their internal control problems under Section 404(a) relative to small accelerated filers under Section 404(b). Therefore, we form an expectation model of the underlying existence of ineffective internal controls among nonaccelerated filers based on firm characteristics. Conditional on the existence of ineffective internal controls, managers and auditors are expected to discover and disclose these weaknesses. Because 404(b) requires auditor involvement (as they must opine on the firm s internal control effectiveness), the key question is whether the absence of auditor oversight under 404(a) affects the accuracy of the disclosures of internal control effectiveness. On the one hand, it is possible that the absence of 404(b) allows managers to exert minimal effort to discover internal control weaknesses, or might even result in managers failing to disclose the existence of known internal control weaknesses. Section 404(b) proponents believe that managers of non-accelerated filers will only seriously evaluate and disclose the effectiveness of their firm s internal control environment when auditors are involved. 7 For example, Sonia Luna, CEO of the consulting firm SOX Solutions, stated in a 2009 CFO.com article that: in practice, when 404(b) kicks in, companies begin to take their controls very seriously. They create new forms, new tasks, and new checklists so that someone 7 Consistent with this, Bedard et al. (2009) provide evidence that auditor intervention (measured using the fourth quarter relative to the first three fiscal quarters) increases the disclosure of material weaknesses in internal control. 9

12 externally can follow the trail. Without it, there s no sanity check [to decide whether or not a material weakness exists]. My fear is that [without requiring Section 404(b)], companies will fail to acknowledge the number of material weaknesses they actually have, because they don t have the competency in-house [to determine whether a weakness exists] (Stuart, 2009). This is especially salient because Bedard and Graham (2011) find that auditors detect about three-fourths of unremediated internal control deficiencies, suggesting auditor oversight may be necessary to identify ineffective internal controls. On the other hand, it is possible that auditors provide a significant monitoring role even without 404(b). Auditors can become aware of ineffective internal controls during their normal audit procedures, as auditors are required to understand their clients internal controls to determine the extent of their substantive testing and source the underlying causes of identified adjustments from their substantive testing. Under AU550, auditors are required to read the other information in documents containing audited financial statements to identify material inconsistencies, and management s Section 404(a) opinion is considered one type of other information. Thus, if auditors have knowledge suggesting the firm s internal controls are not effective, AU550 requires that auditors ask management to correct the inaccurate other information included in documents containing audited financial statements. 8 Thus, even though the auditor does not have to provide a specific opinion regarding the effectiveness of the firm s internal controls, they may become aware of a firm s internal control effectiveness without the required testing under 404(b) (Kinney et al., 2013). Moreover, Kinney et al. (2013) argue that generally auditors first identify financial misstatements and then trace the issue back to the internal control failure, suggesting that full internal control audits might have contributed little to identified material weaknesses in internal controls. Thus, whether firms not subject to the 8 If management refuses to alter the other information, auditors should notify those charged with the firm s governance and alter their auditor s report, withhold the audit report, or withdraw from the engagement (see AU , pages ). 10

13 auditor attestation requirements of Section 404(b) misreport their internal control effectiveness under 404(a) is an empirical question, leading to the following hypothesis (stated in the null form): H1: Non-accelerated filers accurately disclose ineffective internal controls under Section 404(a). In particular, we investigate whether the proportion of non-accelerated filers disclosing ineffective internal controls is as high as the proportion we predict maintain ineffective internal controls. Failure to reject H1 suggests auditor oversight under 404(b) is unnecessary. In contrast, rejecting H1 suggests 404(a) is insufficient. It does not immediately follow, however, that misreporting among non-accelerated filers under Section 404(a) would be ameliorated if they were subject to Section 404(b). To provide evidence on whether 404(b) would curb any misreporting, we benchmark our estimate of misreporting among non-accelerated filers against an analogous estimate of misreporting among small accelerated filers that are also subject to 404(b), thereby allowing us to infer the incremental misreporting by non-accelerated filers who are exempt from 404(b). 3. Sample Selection and Descriptive Statistics We require our sample to have Section 404(a) internal control disclosure data from Audit Analytics and non-missing total assets, market values (defined as PRCC_F CSHO), CIKs, and fiscal year ends (DATADATE) in Compustat. We remove firm-years with market values less than $5 million because it is likely the prediction model developed from the small accelerated filers would not apply to these extremely small firms. Because our goal is to examine the accuracy of internal control disclosures absent 404(b), we also exclude non-accelerated filers that voluntarily comply with 404(b) from our sample. 11

14 Table 1 lists the numbers and percentages of non-accelerated filer firm-years with internal control disclosures from 2007 through For comparison purposes, Table 1 also lists analogous numbers and percentages of accelerated filers. Because we base our prediction models on smaller accelerated filers (those with market capitalization of less than $300 million), we present smaller and larger accelerated filers separately. 9 This also allows us to illustrate that the proportion of firms disclosing ineffective internal control is higher for smaller accelerated filers. For these two groups, we present the proportion of firms that disclosed they maintained ineffective internal controls in their first five years of Section 404 compliance (i.e., 2004 through 2008), as well as more recent years (i.e., 2009 through 2011). 10 We benchmark the first five years of accelerated filers disclosures under Section 404(b) ( ) to the first five years of non-accelerated filers disclosures under Section 404(a) ( ). For completeness we include accelerated filer data for years beyond 2008 in Table 1 (italicized), although these data are not used in our analysis. As reported in Table 1, 10.3% of larger accelerated filers initially disclose they maintain ineffective internal controls in 2004, compared to 18.8% of smaller accelerated filers. The proportion of non-accelerated filers disclosing they maintain ineffective internal controls is similar to smaller accelerated filers, at 17.8%. It is interesting to note that there is a marked decline in the percentage of firms disclosing ineffective internal controls in the subsequent years 9 Kinney and Shepardson (2011) also define small accelerated filers for purposes of their study as firms subject to Section 404(b) with a market capitalization of $300 million or less at the end of their fiscal year during the year prior to November 15, 2004 (the Section 404(b) effective date). 10 To make the time series comparison meaningful, we define the years based on the effective date of Section 404. For example, the first Section 404(a) and 404(b) compliance year for accelerated filers includes fiscal years ending between November 15, 2004 and November 14, 2005; the second compliance year includes fiscal years ending between November 15, 2005 and November 14, 2006; etc. For ease of exposition, we refer to these years as 2004, 2005, etc. Similarly, the first (second) Section 404(a) compliance year for non-accelerated filers includes fiscal years ending between December 15, 2007 and December 14, 2008 (December 15, 2008 and December 14, 2009), etc., and we refer to this year as 2008 ( 2009 ), etc. 12

15 for both larger accelerated filers and smaller accelerated filers, but there is not a similar trend among non-accelerated filers. 11 Table 2 reports the types of issues identified in firms internal control disclosures. 12 We report these frequencies separately for larger accelerated filers, smaller accelerated filers, and non-accelerated filers. The descriptive evidence presented in these panels yields four interesting insights. First, as shown in Panel A, non-accelerated filers appear more likely to disclose issues with their accounting personnel (74.6%) than larger accelerated filers (55.3%) and smaller accelerated filers (56.2%). Second, non-accelerated filers are also more likely to disclose weaknesses related to segregation of duties (47.6%) relative to larger accelerated filers (14.1%) or small accelerated filers (19.5%). Both findings are consistent with the explanation that small firms are constrained by the resources they can allocate to internal controls (Ge and McVay 2005; Doyle et al. 2007a; Ashbaugh-Skaife et al. 2007), and this limitation is more likely to be reflected in the types of controls that require additional accounting personnel. Third, it appears that non-accelerated filers are less likely to disclose information that identifies the underlying cause of the internal control weaknesses (52.8%) than larger and smaller accelerated filers (less than 10%). While this could potentially influence the frequencies of the types of issues discussed above, it also speaks to the disclosure quality of the internal control disclosure among non-accelerated filers, as disclosure of the type(s) of weakness(es) identified is mandated under Section 404(a). Fourth, as reported in Panel B, non-accelerated 11 Kinney and Shepardson (2011) provide two possible explanations for this decline. Accelerated filers could have improved their internal controls and, as a result, more firms have remediated their known internal control weaknesses than firms disclosing new material weaknesses. Alternatively, Auditing Standard No. 5 (the auditing standard that governs 404(b) audits for fiscal years ending on or after November 15, 2007) requires less testing than Auditing Standard No. 2 (the auditing standard that governed 404(b) audits for fiscal periods ending prior to November 15, 2007), leading to auditors failing to detect some material weaknesses post-november 15, Audit Analytics primarily uses the variables NOTEFF_ACC_REAS_KEYS and NOTEFF_OTHER_REAS_KEYS to capture the qualitative information firms provide in firms internal controls reports. 13

16 filers disclose fewer material weaknesses related to revenue recognition and cost of sales issues (approximately 11%) than larger and smaller accelerated filers (approximately 20% to 30%). This pattern is consistent with non-accelerated filers having less complex financial reporting and therefore being less likely to be affected by internal control weaknesses resulting from complex transactions. 4. Research Design and Results 4.1 Accurately identifying and disclosing ineffective internal controls We test our hypothesis using a multi-step process. To explore whether non-accelerated filers accurately disclose ineffective internal controls, we require an assessment of the underlying internal control effectiveness of these firms. We present an out-of-sample prediction model of internal control effectiveness using data from small accelerated filers in Section 4.1.1, apply this model to non-accelerated filers under Section 404(a) in Section 4.1.2, and test the validity of this model in Section We then explore cross-sectional variation in financial reporting quality in Section 4.2 and if 404(b) would curb any misreporting in Section 4.3. Finally, in Section 4.4 we assess the economic importance of misreporting by examining the stock price reactions to the disclosure of ineffective internal controls Out-of-sample prediction model of internal control effectiveness. To estimate nonaccelerated filers propensity to maintain ineffective internal controls in year t, we compile 17 determinants of internal controls effectiveness from prior research (Ge and McVay, 2005; Doyle et al., 2007a; Ashbaugh-Skaife et al., 2009) and generate the following model: [1a] Maintain_Ineffective j,t = β 1 Bus_Seg j,t-3,t-1 + β 2 Foreign j,t-3,t-1 + β 3 M&A j,t-3,t-1 + β 4 Restructure j,t-3,t-1 + β 5 Sales_Growth j,t-1 + β 6 Inventory j,t-3,t-1 + β 7 Size j,t-3,t-1 + β 8 Pct_Loss j,t-3,t-1 + β 9 Auditor_Change j,t-3,t-1 + β 10 Large_Auditor j,t-1 + β 11 Restatements j,t-3,t-1 + β 12 Inst_Own j,t + β 13 Litigious_Ind j,t + β 14 CFO j,t-3,t-1 + β 15 Age j,t + β 16 Computer_Ind j,t + β 17 Bank_Ind j,t + ε j,t 14

17 where Maintain_Ineffective j,t is equal to one if a firm maintains ineffective internal controls in year t, and equal to zero otherwise. As we want to identify the firms that have ineffective internal controls, not firms that disclose ineffective internal controls, we set this indicator variable equal to one if a firm either (1) discloses ineffective internal controls (Disclosed_Ineffective = 1; N=707) or (2) discloses effective internal controls but subsequently restates its financial statements due to internal control issues or amends its Section 404(a) report to conclude that internal controls were actually ineffective (N=184), and equal to zero otherwise. 13 We omit the model intercept because we wish to avoid setting a base-line proportion of firms maintaining ineffective internal controls. 14 Instead, we wish to capture as many economic determinants of ineffective internal controls as possible and form our estimation based solely on economic determinants. Independent variables are defined in detail in the Appendix. We face several complications in forming our prediction model test. The first complication relates to which accelerated filers and years to use in estimating the internal control effectiveness prediction model. Accelerated filers are significantly larger than non-accelerated filers, however, and prior research finds that determinants of internal control quality vary by firm size (Ge and McVay, 2005). Our data supports this finding, as we see a much higher proportion of non-accelerated filers disclosing material weaknesses over segregation of duties (see Panel A 13 We thank Sarah Rice, Dave Weber, and Biyu Wu for making available their data on firms that report effective controls in their Form 10-K and subsequently restate their financial statements due to ineffective internal control issues (N=167 firm-years in our sample) from Rice et al. (2013). We identify another 17 firm-years that filed an amended 404(b) report. These 184 firm-years suggest that small accelerated filers should have had 26.0% more firm-years reporting ineffective internal controls (i.e., , where 707 is the number of firm-years that reported ineffective controls in Table 3, Panel A). We discuss this point in greater detail when we specifically model the proportion of misreporting among small accelerated filers (see Section 4.1.3). 14 The tenor of the paper remains unchanged if we allow an intercept (not tabulated). However, Kinney and Shepardson (2011) already document that a similar proportion of non-accelerated filers and small accelerated filers disclose ineffective internal controls. The purpose of our study is to generate, based on economic determinants, an estimate of which firms are expected to maintain ineffective controls, but not restrict the proportion to be similar to smaller accelerated filers (ergo the removal of the intercept base-line ). 15

18 of Table 2), which is an issue more likely to affect smaller firms. For this reason, we estimate our logistic model using small accelerated filers, defined as firms required to comply with Section 404(b) with an end-of-year market capitalization of $300 million or less in the year prior to, during, or after the first year Section 404(b) was effective (e.g., fiscal years ending between November 15, 2003 and November 14, 2006). Because non-accelerated filers are required to comply with Section 404(a) for fiscal years ending on or after December 15, 2007, we assess the accuracy of the first five years of Section 404(a) disclosures (fiscal years ending December 15, 2007 through December 14, 2011). Thus, we estimate our prediction model using data from small accelerated filers in their first five years of Section 404(b) compliance (fiscal years ending November 15, 2004 through November 14, 2009). We then apply the estimated coefficients generated using small accelerated filer data to non-accelerated filer data from their first five years of compliance with Section 404(a). This matching of compliance years is important as we see that the frequency of ineffective internal controls for accelerated filers declines over time (see Table 1). The second complication we face is which determinants to include in our model. Prior research modeling internal control effectiveness determinants included accelerated filers of all sizes (e.g., Doyle et al., 2007a), so it is possible some variables will not have predictive power among smaller accelerated filers. Following Dechow et al. (2011), we use a backward elimination technique when estimating Equation [1a] to determine which independent variables are significantly related to ineffective internal controls in year t. We rely on the computational algorithm of Lawless and Singhal (1978) to compute a first-order approximation of the remaining slope estimates for subsequent variable eliminations. We use a stepwise logistic regression and set the significance level for elimination at This selection procedure 16

19 identifies 11 of the 17 independent variables as significantly associated with ineffective internal controls in year t. 15 The final prediction model follows. [1b] Maintain_Ineffective j,t = β 1 M&A j,t-3,t-1 + β 2 Restructure j,t-3,t-1 + β 3 Inventory j,t-3,t-1 + β 4 Size j,t-3,t-1 + β 5 Large_Auditor j,t-1 + β 6 Restatements j,t-3,t-1 + β 7 Inst_Own j,t + β 8 Litigious_Ind j,t + β 9 Age j,t + β 10 Computer_Ind j,t + β 11 Bank_Ind j,t + ε j,t As previously noted, the coefficients from this model are generated by estimating the model on Section 404(b) data from the first five years of small accelerated filer compliance. We then apply these coefficients to our sample of non-accelerated filers to form out-of-sample predictions of the likelihood that non-accelerated filers maintain ineffective internal controls. Panel A of Table 3 provides the number of small accelerated filer observations available for the backward induction described above (5,770 of the 6,253 firm-year observations related to Smaller Accelerated Filers in their first five years of compliance in Table 1). Panel B provides descriptive statistics for the Equation [1b] variables, and the estimated coefficients are provided in Panel C. 16 Although not directly comparable to prior research because of the time period examined and our focus on small accelerated filers, the coefficients on the determinant variables are in the predicted direction (Ge and McVay, 2005; Doyle et al., 2007a; Ashbaugh-Skaife et al., 2009). As shown in Panel C, our multivariate analysis provides evidence that merger and acquisition activity (M&A) and corporate restructuring (Restructure) increase the likelihood of ineffective internal controls, while larger firms (Size) are less likely to have ineffective internal controls (p < 0.01). Industry membership also plays a significant role, with an increased propensity in the computer software industry (Computer_Ind) and a lower propensity in both the 15 In untabulated analyses we also consider two alternate cut-off values. Using a 0.20 significance level generates a model identical to model tabulated. Using a 0.10 level omits one variable (Inventory) and generates a model with a ROC of (slightly less than the ROC of in the model tabulated). 16 Coefficient values are insensitive to clustering standard errors at the firm level (untabulated). 17

20 banking industry (Bank_Ind) and in litigious industries (Litigious_Ind) (p < 0.01). 17 Firms using large auditors, with high institutional ownership, and without historical restatements (Large_Auditor, Inst_Own, and Restatements, respectively) also have a lower propensity of maintaining ineffective internal controls (p < 0.05). The model s ROC curve of 0.76 suggests the model has reasonable explanatory power Forming predictions of internal control effectiveness among non-accelerated filers. Panel A of Table 4 provides the number of observations available to form an assessment of the effectiveness of non-accelerated filers internal controls (6,105 of the 6,946 firm-year observations related to Non-Accelerated Filers in Table 1). In Panel B of Table 4, we present descriptive statistics of the predictor variables from Equation [1b] for these 6,105 nonaccelerated filer firm-years. To determine the expected likelihood of ineffective internal controls for non-accelerated filers in year t, we apply the coefficients in Panel C of Table 3 to nonaccelerated filer firm-year observations to calculate the dependent variable RawProbability as follows: [2] RawProbability j,t = M&A j,t-3,t Restructure j,t-3,t Inventory j,t-3,t Size j,t-3,t Large_Auditor j,t Restatements j,t-3,t Inst_Own j,t Litigious_Ind j,t Age j,t Computer_Ind j,t Bank_Ind j,t For ease of interpretation we transform RawProbability into Predicted_Ineffective, a variable bound between zero and one, using the equation Predicted_Ineffective = exp(rawprobability) 17 Note that our prediction for litigious industries is non-directional. While firms in more litigious industries are expected to be more likely to establish and maintained effective internal controls, they also have stronger incentives to disclose ineffective internal controls if they exist. 18 In untabulated analyses we estimate Equation [1a] using two alternate small accelerated filer samples: firms required to comply with Section 404(b) with an end-of-year market capitalization of $250 ($150) million or less in the year prior to, during, or after the first year Section 404(b) was effective. The backwards selection procedure identifies 10 (11) of the 17 Equation [1a] variables from the $250 ($150) million sub-sample as significant, and the model s ROC curve is 0.76 (0.74). We use the $300 million subsample because (1) this model s ROC curve is slightly higher than either of these two alternate specifications and (2) the market capitalization cutoff parallels the cutoff used in Kinney and Shepardson (2011). 18

21 (1+exp(RawProbability)). Predicted_Ineffective values closer to one (zero) indicate a greater (lower) expected likelihood a firm maintains ineffective internal controls in year t. The mean Predicted_Ineffective value for non-accelerated filers is (as reported in Panel A of Table 5). This predicted likelihood is higher than the 15.4% (= 891 5,770) of small accelerated filers that maintain ineffective internal controls in the sample used to generate the model coefficients (Panel B of Table 3), consistent with non-accelerated filers having a higher probability of maintaining ineffective internal controls relative to small accelerated filers (Kinney and Shepardson, 2011; McVay, 2011). This 22.2% is also higher than the 15.7% of non-accelerated filers disclosing ineffective internal controls (Panel A of Table 4), consistent with some non-accelerated filers failing to disclose ineffective internal controls under 404(a). 19 Our model estimates that 1,355 firm-years (i.e., 22.2% of the 6,105 non-accelerated filer firmyears sample) should disclose ineffective internal controls, yet we observe only 957 firm-years disclosing ineffective internal controls. This suggests that the number of non-accelerated filers maintaining ineffective internal controls is 42% higher than the number disclosed ((1, ) 957) Validity tests of the prediction model. We validate our prediction model in several ways. First, we expect firms disclosing ineffective internal controls to have higher Predicted_Ineffective values relative to firms disclosing effective internal controls. Panel A of Table 5 shows that Predicted_Ineffective has a mean of 32.2% among firms disclosing ineffective internal controls and a mean value of 20.3% among firms disclosing effective internal controls; this difference is statistically significant (p < 0.01). We further explore the relation between internal control effectiveness disclosures and Predicted_Ineffective by ranking 19 We expect the disclosure of ineffective internal controls to be credible (Hutton et al., 2003) and thus only consider firms that report effective internal controls when our model suggests their internal controls are actually ineffective. 19

22 observations into quintiles based on Predicted_Ineffective values and determining the proportion of firms disclosing ineffective or effective internal controls within each quintile (Dechow et al., 2011). If our model has no predictive ability, we would expect effective and ineffective firmyear observations to be evenly distributed across quintiles. In contrast, if our model is able to predict internal control effectiveness, we expect ineffective internal control firm-year observations to cluster in the fourth and fifth Predicted_Ineffective quintiles. Panel B of Table 5 illustrates that firm-years disclosing ineffective internal controls (Disclosed_Ineffective = 1) are monotonically increasing in frequency across Predicted_Ineffective quintiles and clustered in quintiles 4 and 5, as expected. To illustrate, 39.6% of the Disclosed_Ineffective = 1 observations fall in the fifth Predicted_Ineffective quintile whereas only 6.4% are in the first quintile. 20 In contrast, the predicted probabilities for Disclosed_Ineffective = 0 observations are monotonically decreasing in frequency across Predicted_Ineffective quintiles (as expected). Finally, we explore how our Predicted_Ineffective measure is related to 404(a) report amendments. We expect that Predicted_Ineffective values will be systematically higher among observations that originally disclose effective internal controls and amend their 404(a) report to reflect their internal controls were actually ineffective (Amended_404(a) jt = 1). 21 Panel C of Table 5 reports that 26 of our 5,148 non-accelerated filer firm-year observations that initially disclosed effective internal controls subsequently amended their 404(a) report to reflect that their internal controls were actually ineffective. Mean and median values of Predicted_Ineffective are higher among the 26 amending 404(a) firm-years (0.295 and 0.231, respectively) relative to the 5,122 non-amending firm-years (0.203 and 0.161, respectively); these differences are statistically 20 Panel B of Table 5 also illustrates the capacity for our model to fail to identify firms that maintain ineffective internal controls. For example, there are 61 firms that our model estimates to have the lowest risk of maintaining ineffective internal controls but disclose that they maintain ineffective internal controls. 21 We exclude the firm-years that initially disclose ineffective internal controls from Panel C of Table 5. 20

23 significant (p < 0.01). Taken together, the results reported in Table 5 suggest that our model generates predicted values that identify misreported instances of ineffective internal controls. 4.2 Financial reporting quality of non-accelerated filers In this section we investigate financial reporting quality at firms suspected of misreporting their internal controls to provide evidence on the economic significance of this misreporting. In particular, we investigate whether firms financial reporting quality varies between firm-years we suspect of misreporting and firm-years we believe to have credibly disclosed their internal controls. For this analysis, we categorize our sample firm-years into three groups: (1) those that disclosed ineffective internal controls (Disclosed_Ineffective = 1), (2) those that disclosed effective internal controls and our prediction model suggests maintained ineffective internal controls (Potential_Misreporter = 1), and (3) those that disclosed effective internal controls and our prediction model suggests maintained effective internal controls (Credibly_Effective = 1). The latter two categories require us to identify a threshold over which we categorize each firm-year with Disclosed_Ineffective = 0 as a potential misreporter or a credibly effective observation. We classify the 824 firm-years in the fifth Predicted_Ineffective quintile (as reported in Panel B of Table 5) as potential misreporters and the 4,306 firm-years in the first through fourth quintiles as credibly effective. 22 This investigation is a joint test of the validity of our model and the economic significance of misreporting (i.e., why misreporting matters). In particular, if our model identifies the subset of firms disclosing effective internal controls but maintaining ineffective internal controls, it follows that a difference in financial reporting quality would provide 22 This classification yields a Potential_Misreporter sample of observations with a Predicted_Ineffective value of at least (untabulated) with a mean value of While the top quintile is an arbitrary cutoff, this cutoff does not affect our prior inferences regarding the proportion of firms that are misreporting where we conclude there should be 42% more firm-years reporting ineffective internal controls. 21