The Usefulness of Accounting Estimates for Predicting Cash Flows

The Usefulness of Accounting Estimates for Predicting Cash Flows and Earnings Baruch Lev* New York University Siyi Li University of Illinois Theodore Sougiannis University of Illinois and ALBA January, 2009 * Contact information: Baruch Lev ([email  protected] nyu. edu), Stern School of Business, New York University, New York, NY 10012.The authors are indebted to the editor and reviewers of the Review of Accounting Studies for suggestions and guidance, and to Louis Chan, Ilia Dichev, John Hand, James Ohlson, Shiva Rajgopal, and Stephen Ryan for helpful comments, as well as to participants of seminars at Athens University of Economics and Business, London Business School, Penn State University, Purdue University, University of Illinois at Urbana-Champaign, University of Texas at Dallas, Washington University in St.Louis, the joint Columbia–NYU Seminar, the 16th Financial Economics and Accounting Conference, the 2006 AAA FARS Midyear Meeting, and the 2008 AAA Annual Meeting. 1 ABSTRACT Estimates and projections are embedded in most financial statement items. These estimates potentially improve the relevance of financial information by providing managers the means to convey to investors forward-looking, inside information (e. g. , on future collections from customers via the bad debt provision).On the other hand, the quality of financial information is compromised by: (i) the increasing difficulty of making reliable forecasts in a fastchanging, often turbulent economy, and (ii) the frequent managerial misuse of estimates to manipulate financial data. Given the ever-increasing prevalence of estimates in accounting data, whether these opposing forces result in an improvement in the quality of financial information or not is among the most fundamental issues in accounting. We examine in this study he contribution of accounting estimates embedded in accruals to the quality of financial information, as reflected by their usefulness in the prediction of enterpr ise cash flows and earnings. Our extensive out-of-sample tests, reflecting both the statistical and economic significance of estimates, indicate that accounting estimates beyond those in working capital items do not improve the prediction of cash flows. Estimates do, however, improve the prediction of next year’s earnings, though not of subsequent years’ earnings. Our economic significance tests corroborate that accounting estimates do not improve cash flow or earnings prediction.We conclude that the usefulness of accounting estimates to investors is limited, and provide suggestions for improving their usefulness. 2 The Usefulness of Accounting Estimates For Predicting Cash Flows and Earnings 1. Introduction Financial statement information, be it balance sheet items such as net property, plant and equipment, goodwill and other intangibles, accounts receivable and inventories, or key income statement figures, such as revenues, pension expense, in-process R&D, or the rec ently expensed employee stock options, is largely based on managerial estimates and projections.The economic condition of the enterprise and the consequences of its operations as portrayed by quarterly and annual financial reports are therefore an intricate and ever changing web of facts and conjectures, where the dividing line between the two is largely unknown to information users. With the current move of accounting standard-setters in the U. S. and abroad toward increased fair-value measurement of assets and liabilities, the role of estimates and projections in financial reports will further increase.We ask in this study: what is the effect of the multitude of managerial estimates embedded in accounting data on the usefulness of financial information? straightforward. The answer is far from On the one hand, estimates/projections are potentially useful to investors because they are the primary means for managers to convey credibly forward-looking proprietary information to invest ors1. Thus, for example, the bad debt provision, if estimated properly, informs investors on expected future cash flows from customers, restructuring charges predict future employee severance payments and plant closing costs, and the capitalized portion of We say â€Å"credibly† primarily because post Sarbanes-Oxley the firm’s CEO and CFO have to certify that â€Å"†¦information contained in the periodic report fairly represents, in all material respects, the financial condition and results of operations of the issuer†¦Ã¢â‚¬  3 software development costs (SFAS 86) informs investors about development projects that passed successfully technological feasibility tests and are accordingly expected to enhance future revenues and earnings. 2 This potential contribution of managerial estimates to investors’ ssessment of future enterprise cash flows underlies the oft-quoted statement by the Financial Accounting Standard Board (FASB) in its Conceptual Framewor k about the superiority of accruals earnings—mostly based on estimates—over the largely fact-based cash flows in predicting future enterprise cash flows: Information about enterprise earnings based on accruals accounting generally provides a better indication of an enterprise’s present and continuing ability to generate favorable cash flows than information limited to the financial aspects of cash receipts and payments (FASB, 1978, p. IX).On the other hand, the contribution of estimates to the usefulness of financial information is counteracted by two major factors: (i) Objective difficulties. In the current volatile and largely unpredictable business environment, due to fast-changing market conditions (deregulation, privatization, emerging economies) and rapid technological changes, it is increasingly difficult for managers to make reliable projections of business events. Consider, for example, the estimated future return on pension assets—a key componen t of the pension expense: This estimate is essentially a prediction of the long-term performance of capital markets.Are managers better predictors of market performance than investors? 3 Or, reflect on the generally large impairment charges of fixed assets and acquired intangibles (including goodwill) mandated by SFAS 121 and SFAS 142: The determination of these 2 Indeed, Aboody and Lev (1998) document a positive association between capitalized software development costs and future earnings. 3 Consider, for example, the 2001 pension footnotes of three financial institutions, Merrill Lynch, Bank of NewYork, and Charles Schwab, which report the following estimates of the expected returns on pension assets: 6. 60%, 10. 50%, and 9. 00%, respectively (Zion, 2002). The wide range of estimates (6. 6%-10. 5%) of the long term performance of capital markets reflects the inherently large uncertainty (unreliability) of the pension expense estimate. 4 charges requires managers to estimate futur e cash flows from tangible and intangible assets. In today’s highly competitive and contested markets the reliability of asset cash flows forecasted over several years is obviously questionable.Accordingly, the accounting estimates and projections underlying financial information introduce a considerable and unknown degree of noise, and perhaps bias to financial information, clearly detracting from their usefulness. 4 (ii) Manipulation. Add to the above objective difficulties in generating reliable estimates the expected and frequently documented susceptibility of accounting estimates to managerial manipulation, and the consequent adverse impact of estimates on the usefulness of financial information becomes apparent.Given that it is very difficult to â€Å"settle up† with manipulators of estimates—even if an estimate turns out ex post to be far off the mark, it is virtually impossible to prove that ex ante the estimate was intentionally manipulated—there are no effective disincentives for managers to manipulate accounting estimates. Indeed, many of the Securities and Exchange Commission (SEC) enforcement cases alleging financial reporting manipulation concern misuse of estimates underlying accruals (e. g. Dechow et al. , 1996). Thus, the impact of estimates underlying accounting measurement and reporting procedures on the usefulness of financial information is an open question, to be examined in this study. The relevance of this examination cannot be overstated. Accounting estimates and projections underlie much of Generally Accepted Accounting Principles (GAAP) and consume 4 A case in point (Wall Street Journal, August 4, 2004, p. c1): â€Å"Investors in Travelers have needed more than that ed umbrella protection from what has been raining on them since the company was spun out from Citigroup in early 2002. Late last month, St. Paul Travelers Cos. , †¦ announced what Morgan Stanley termed a ‘blockbuster reserve chargeà ¢â‚¬â„¢ of $1. 625 billion. The charge was about twice as large as analysts have been expecting. The insurer contends that the charge stems largely from the need to reconcile differing accounting treatments at the two companies [Travelers and its acquisition—St. Paul Cos. ]. It was just a â€Å"reserve valuation adjustment,† the company said†¦.Sadly there seems to be little reason why Travelers’ executives didn’t anticipate problems with St. Paul’s insurance methodologies†¦ Mr. Benet [Travelers’ CFO] said:†¦we recognized early on that there was a difference in some of the methodologies [to estimate reserves] that would have to be addressed. † (emphasis ours). Thus, different accounting methodologies used to estimate the same reserves, all approved by auditors, yield a difference of $1. 625 billion. 5 most of standard-setters’ time and efforts.Just consider the major issues addressed by the FASB in recent yearsâ₠¬â€financial instruments, employee stock options, fixed assets and goodwill impairment, and the valuation of acquired intangibles, to name a few—all require major estimates and forecasts in the process of accounting measurement and reporting. If these and other accounting estimates do not contribute significantly to the usefulness of financial information, the efforts of accounting regulators, and even more importantly, the resources society devotes to the generation of estimates in the process of financial statement preparation and their auditing, are misdirected.Worse yet, if financial information users are led by the estimates-based accounting information to misallocate resources, an additional dead-weight cost is imposed on society. We define and test the usefulness of estimates embedded in accrual earnings in terms of their ability to predict enterprise performance. 5 This predictive use of financial information is central to security analysis and valuation and is also a fundamental premise of the FASB’s Conceptual Framework as indicated by the quote above. Future enterprise performance is mainly reflected by cash flows and earnings.Future cash flows are at the core of asset and liabilities accounting valuation rules. Thus, for example, asset impairment (SFAS 144) is determined by expected cash flows, and the useful lives of acquired intangibles (SFAS 142) are a function of future cash flows. More fundamentally, asset or enterprise cash flows are postulated by economic theory as the major determinants of their value. Given a certain ambiguity about the specific definition of cash flows used by investors, we perform our tests with two widelyused and frequently prescribed cash flow constructs: cash from operations (CFO) and free cash flows (FCF).Much of prior related research focused on CFO. Free cash flows are central to 5 There are, of course, other uses of financial data, such as in contracting arrangements, which are not aimed at predicti ng future enterprise performance. 6 many practitioners’ valuation models (e. g. Brealey and Myers, 2003), and play an important role in research too (e. g. , FCF is a primary variable in the valuation constructs of Feltham and Ohlson, 1995). Cash flow prediction is thus a predominant element of accounting measurements and practitioners’ valuation processes.Despite the prominence of cash flows in economic asset valuation models, there is no denying that many investors and analysts are using financial data to predict earnings. The underlying heuristics are somewhat obscured; perhaps investors predict earnings first, and derive future cash flow estimates from the predicted earnings. In any case, earnings prediction is prevalent in practice, and we therefore also examine the usefulness of accounting estimates for the prediction of earnings, both operating and net income.The focus of this study is on accounting estimates, but many of the estimates underlying financial infor mation are not disclosed in the financial reports. 6 We, therefore, focus in this study on accruals, most of which are based on estimates. In particular, we distinguish between accruals which are largely unaffected by estimates (changes in working capital items, excluding inventory), and accruals which are primarily based on estimates (most non-working capital accruals). This enables us to draw sharper inferences on the effect of estimates on the usefulness of financial information.We also analyze a smaller sample of firms with data on specific estimates which we split into recurring and non-recurring to separate noise (the non-recurring estimates) from information (the recurring estimates). Our empirical analysis is based on a sample of all non-financial Compustat firms with the required data—ranging from roughly 1,500 to 3,200 companies per year—and spanning the 6 For example, General Electric reports in its revenue recognition footnote that various components of rev enues derived from long-term projects are based on the estimated profitability of these projects.GE, however, does not break down total revenues into estimates and â€Å"facts. † 7 period 1988-2005. Our tests are conducted in three stages: (1) In-sample, industry-by-industry, predictions of future enterprise cash flows and earnings, based on: (a) current cash flows only (the benchmark), (b) earnings, and (c) the set of cash flows, the change in working capital (excluding inventory), and various components of accruals based on estimates. Here we follow the regression procedures of Barth, Cram, and Nelson (2001) and find, on more recent data, results which are generally consistent with Barth et al.This is our departure point. (2) Out-of sample firm specific predictions of future cash flows and earnings using the industry specific parameter estimates of the in-sample regressions. The focus of this analysis is on the improvement in the quality of predictions brought about by the addition of estimates (accruals) to the predictors. We thus predict cash flow from operations, free cash flows, net income before extraordinary items, and operating income over various horizons: one year ahead, second year ahead, aggregate two years ahead, and aggregate three years ahead.Our results show that accounting estimates do not improve the prediction of future cash flows (both operating and free cash flows), compared with predictions based on current CFO and the change in working capital excluding inventory. However, accruals do improve next year’s prediction of net and operating income. Notably, cash flow predictions based on current earnings only are significantly inferior to those generated by current CFO, contrary to Kim and Kross (2005). In our small sample analysis, neither recurring nor nonrecurring estimates improved significantly the predictions of either cash flows or earnings.The bottom line—accounting estimates beyond those in working capital items (except inventory) do not improve the prediction of cash flows. 8 (3) Finally, we examine the economic significance of estimates. These tests complement stage two, which is based on the statistical significance of differences in the quality of alternative predictors. Since it is difficult to gauge economic significance from statistical significance, we perform various portfolio tests, where portfolios are constructed from predicted cash flows and earnings based on various predictors, some of which are based on estimates.The abnormal returns on these portfolios, generated by alternative predictors, are our gauge of economic significance. The focus here is on comparing the returns on portfolios constructed from predictions based on current cash flows only (the benchmark), with returns on portfolios constructed from predictions based on current earnings or current cash flows plus changes in working capital and estimates. The results from these tests generally corroborate the out-of-sa mple prediction tests.In practically all our portfolio tests the model that uses current operating cash flows only to predict firm performance generates higher abnormal returns than models which add estimates to the prediction process used for the portfolio formation, though most of these returns are insignificant. Furthermore, the portfolios constructed from predictions based on current cash flows only yield abnormal returns with generally lower standard deviation than the alternative portfolios which include earnings or estimates among the predictors. We caution against sweeping conclusions.We examine the usefulness of accounting estimates in terms of predictive ability with respect to future firm performance. Accounting information is used for other purposes too (contracting, national accounting), for which estimates may be useful. Furthermore, our prediction tests are based on fairly simple models. Users may be using different, more sophisticated models where estimates could pro ve to be useful. 9 Nevertheless, we believe that our findings draw attention to the significant vulnerability of financial information from the multitude of underlying estimates and projections, and to the urgent need for improving the eliability of estimates, on which we comment in the concluding section. The order of discussion is as follows: Section 2 relates our findings to available research, and Section 3 outlines our research design. Section 4 describes our sample, and Section 5 reports our prediction tests. Section 6 informs on a battery of robustness checks, and Section 7 focuses on a subsample with an extended set of accounting estimates. Section 8 reports our portfolio (economic significance) tests, while Section 9 concludes the study. 2.Relation to Available Research Our study interfaces with several active research areas, and below we comment on the relation between our work and various representative studies. We are not familiar with empirical studies which assess the impact of accounting estimates on the informativeness of financial information, but there is a substantial number of studies that examine the contribution of accruals to the prediction of future cash flows and other variables. These studies can be roughly classified into regression-based (in-sample) analyses, and out-of-sample prediction tests.An example of the former is the comprehensive work by Barth, Cram and Nelson (2001), who regress CFO on lagged values of CFO and components of accruals (primarily the changes in accounts receivable, inventories, and accounts payable, as well as depreciation & amortization and other accruals). The authors report (p. 27) that â€Å"each accrual component reflects different information relating to future cash flows†¦[and] is significant with the predicted sign in predicting future cash flows, incremental to current cash flows. Note that 10 predictive ability is assessed in this and similar studies by the significance of the estimated accrua ls’ coefficients and by the improvement inR 2. 7 An interesting extension of the regression strand is provided by Subramanyam and Venkatachalam (2007) who examine the relative explanatory power of earnings and cash flows with respect to an ex post measure of the intrinsic value of equity which uses Ohlson’s (1995) equity valuation framework, based on realized values of earnings and book values.The authors argue that such measurement of equity values avoids the necessity to assume capital market efficiency, as in Dechow’s (1994) study relating accruals to contemporaneous stock returns. Dechow documents a significant association between accruals and stock returns, but the implications of such association for market efficiency are challenged by Sloan’s (1996) findings of strong return reversals (market inefficiency) following extreme accruals.Subramanyam and Venkatachalam (2007) conclude that operating cash flows are more strongly associated with future cash flows than earnings, and that current earnings are more strongly associated with future earnings than cash flows. Regressing the ex-post equity measure on earnings and cash flows indicates that earnings exhibit a higher explanatory power than cash flows. By and large, the in-sample regression studies suggest that accruals are associated with subsequent cash flows and contemporaneous equity values, a finding we largely update and corroborate in the initial stage of our analysis (Section 5. ). However, as is argued in Section 5. 1, in-sample regressions are not prediction tests, and may even provide misleading inferences concerning prediction power. We move, therefore, to out-of-sample tests. An early and innovative out-of-sample prediction test is Finger (1994), who concludes from a sample of 50 companies with long historical data that cash flow is marginally superior to 7 Bowen et al. (1986) and Greenberg et al. (1986) perform similar regression-based, in-sample predictions. 11 ear nings for short-term predictions and performs similar to earnings in long-term cash flow predictions.However, time-series and cross-sectional out-of-sample short-term prediction tests by Lorek and Willinger (1996) and Kim and Kross (2005), respectively, show that current earnings predict more accurately future cash flows than current cash flows do. Thus, a mixed picture emerges from the out-of-sample tests, calling for further research. Note also that most previous studies, in- and out-of-sample, focus on the prediction of cash from operations, despite the fact that free cash flows (a measure included in our tests) is frequently used by analysts and investors.Barth, Beaver, Hand and Landsman (2005) provide an interesting perspective on the usefulness of accruals. Using the valuation framework of Feltham and Ohlson (1995, 1996), they examine the ability to predict equity value of various disaggregations of earnings: aggregate earnings, cash flows and total accruals, as well as cash f lows and four major components of accruals. The prediction methodology is out-of-sample in a particular sense: cross-sectional valuation models are run for each year (equity values regressed on contemporaneous earnings disaggregations), excluding each time a particular sample firm.The equity value of that firm is then predicted from the estimated coefficients of the models. Barth et al. (2005, p. 5) â€Å"†¦find evidence of some reduction in mean prediction errors from disaggregating earnings into cash flows and total accruals, and some additional reduction from disaggregating total accruals into its four major components†¦median prediction errors generally support disaggregation of earnings only into cash flows and total accruals. Overall, these findings vary considerably by industry, and appear to indicate a more consistent success for the cash flows and total accruals model than for the cash flows and disaggregated accruals model. 8 8 Studies such as Bathke et al. (198 9) and Lorek et al. (1993) also perform out-of-sample prediction tests. 12 The substantial body of research on the accruals anomaly initiated by Sloan (1996) is tangentially related to our study.This research establishes that accruals are often misinterpreted by investors: large (small) accruals firms are contemporaneously overvalued (undervalued) in capital markets, and these misvaluations are largely reversed within a couple of years. Notably, much of the accruals anomaly resides in small, thinly traded firms, which are unattractive to most institutional investors (Lev and Nissim, 2006), a fact that contributes significantly to the persistence of this anomaly. It is important to note that our focus in this study is different from the ccruals anomaly research: we do not examine investors’ perceptions of accruals, and the consequences of such perceptions. Rather, we focus on the contribution of accruals and by implication of the embedded estimates to the primary role of finan cial information—assisting users in predicting future enterprise performance. The short-term market inefficiencies highlighted by the accruals anomaly are, of course, worth noting, but they do not inform much on the presumed role of accruals—to improve the prediction of enterprise performance.Stated differently, while extreme accruals are often mispriced contemporaneously by investors, a misperception corrected fairly shortly thereafter, accounting accruals in general, prevalent in every financial report, may still enhance the multi-year prediction of firm performance. It is this fundamental role of accruals and their underlying estimates that is the main theme of our study. The lack of convergence of the extant accruals’ usefulness research makes it very difficult to draw firm conclusions.Some studies are in-sample, while others are out-of-sample; some researchers relate accruals to contemporaneous returns or equity values whereas others to future values. Some predict cash flows while others predict equity values based on models using forecasted or realized residual earnings. Our main contribution to extant research is the focus on the estimates embedded in accruals and the provision of certain closure to the usefulness of 13 accruals issue. We distinguish between accruals which are largely based on facts and those primarily reflecting estimates, to focus on the usefulness of accounting estimates.Our main tests are out-of-sample predictions, replicating what most investors actually do—predict, with no ex post information (as implicitly assumed by in-sample studies), various versions of future earnings and cash flows. The comprehensiveness of our predicted performance measures (two versions of earnings and two of cash flows), and the number of future periods examined (years t+1, t+2, and aggregate next two years and next three years) enables us, we believe, to draw general conclusions about the contribution of estimates to firm perf ormance rediction. Furthermore, our study is the first, we believe, to examine both the statistical and economic performance of accruals-based prediction models. Inferences from statistical significance are sometimes difficult to draw and generalize. Consider, for example, the Barth, Beaver, Hand and Landsman (2005, p. 5) conclusion: â€Å"we find evidence of some reduction in mean prediction errors from disaggregating earnings†¦Ã¢â‚¬  (emphasis ours). While definitely interesting, this conclusion leaves open the important question of: how material is â€Å"some reduction†?Is it, for example, sufficiently large to support the current move of the FASB and IASB toward increased reliance on estimates in financial reports (fair value, stock option expensing, etc. )? Statistical significance coupled with economic significance, as provided below, allows for a more comprehensive evaluation of the evidence. 9 The focus on accounting estimates, the out-ofsample methodology, and the examination of both statistical and economic significance, all bringing certain closure to the research question, is our main contribution. 3. Research Design Examples of studies including economic significance tests are Ou and Penman (1989), Stober (1992), Abarbanell and Bushee (1998), and Piotroski (2000). 14 Our research design consists of three stages: (a) in-sample association tests of cash flows (earnings) regressed on lagged values of these variables and accruals, (b) out-of-sample forecasts of cash flows (earnings) based on these variables and accruals and (c) calculation of hedge future excess returns on portfolios constructed from the out-of-sample predicted cash flows (earnings) in stage (b).We conduct the first stage as a link to and departure from previous research by estimating cross-sectional in-sample regressions as in the Barth, Cram and Nelson (2001) study (BCN hereafter). We use several prediction constructs, primarily to distinguish between accruals largely based on facts and those based on estimates. At one extreme of the accruals disaggregation we classify all the accruals in the â€Å"operations† section of the cash flow statement into working capital changes excluding inventory (? WC*) and the remaining accruals, termed â€Å"estimates† (EST): EARNINGSCash from Working Capital Operations Change excluding (CFO) inventory (? WC*) Estimates (EST) ACCRUALS Working capital items with the exception of inventory, such as accounts payable and short-term marketable securities, are generally not materially impacted by managerial estimates,10 whereas 10 The accounts receivable change, net of the provision, is an exception, since it is subject to an estimate. But this estimate is included in our second accruals component, EST. 15 most of the remaining accruals are in fact pure estimates (e. g. , depreciation and amortization, bad debt provision, in-process R&D).At the other end of the accruals disaggregation we separate out the c hange in inventory (? INV) from the aggregate estimates (EST), given the evidence (e. g. , Thomas and Zhang, 2002) that much of the accruals anomaly resides in inventory, probably due to intentional and unintentional misestimations of this item. We further break out depreciation and amortization (D&A) and deferred taxes (DT) from other estimates because the identification of these items is possible from Compustat data over the entire sample period. This disaggregation is depicted thus: EARNINGS CFO WC* (minus inventory) ?Inventory (? INV) Dep. & Amortization (D&A) ACCRUALS Def. Taxes (DT) Other estimates (EST*) The various components of accruals along with cash from operations (CFO),11 depicted in the two exhibits above are the independent variables in the estimation models underlying our in-sample predictions. We add to these variables the cash flow statement figure of capital expenditures (CAPEX), since the dependent variables in our models are future cash flows or earnings, which are generally affected by current investment (capital expenditures). We believe 11We measure CFO as in Barth et al. (2001), namely net cash flow from operating activities, adjusted for the accrual portion of extraordinary items and discontinued operations. 16 that the addition of capital expenditures to the regressors improves the specification of the insample prediction models, and sharpens our focus on the relative performance of the accruals components, our focus of study. Indeed, the capital expenditures variable is statistically significant in most of our annual in-sample predictions models. 12 3. 1 Prediction tests Our prediction tests take the following general form.We predict two versions of cash flows (cash from operations and free cash flows) and two constructs of earnings (net income before extraordinary items and operating income) in years t+1 and t+2, as well as in aggregate years t+1 & t+2, and t+1 through t+3. To gain insight into the usefulness of estimates in predi cting firm performance, we use five prediction models with increasing disaggregation of accruals (regressors): Model 1: current CFO only—the benchmark model; Model 2: current net income (NI) only; Model 3: current CFO and the change in working capital items excluding inventory (?WC*)—namely, largely fact-based regressors; Model 4: current CFO, the change in working capital items excluding inventory ? WC*, and total remaining accruals, largely based on estimates (EST); and Model 5: current CFO, the change in working capital items excluding inventory ? WC*, the change in inventories (? INV), depreciation & amortization (D&A), the change in deferred taxes (DT), and all other estimates (EST*)—the most disaggregated model. The purpose is to examine whether the gradual addition of components of accruals 12 For robustness, we reran our predictions (reported in Table 3) without capital expenditures, and conclude that one of our inferences changes in the absence of capit al expenditures. 17 estimates to current cash flows (the benchmark) improves the prediction of future cash flows or earnings. Increasing the disaggregation of accruals should, in general, enhance the quality of prediction (from model 1 to 5), since the individual accrual components are allowed to have different effects (multiples) on the predicted values. We examine model 2 because the predictor, earnings, is a summary accounting variable that has been extensively investigated for its information content and has been used in most prior studies (e. . BCN and Kim and Kross 2005). It is important to note that the cross-sectional estimates of the five in-sample prediction models are obtained for 2-digit SIC industry groups. These industry specific estimates make the implicit assumption of constancy of coefficients across firms reasonably tenable. We implement the second stage of our research design by using the industry specific estimated coefficients from each of the above five predict ion models to calculate firm specific predicted values for cash from operations (CFO), free cash flows (FCF), net income (NI) and operating income (OI).We then calculate firm specific prediction errors as the difference between the actual and predicted values of each variable examined. The following examples of the prediction of free cash flows (FCF) will clarify our prediction procedures. A. Prediction of next year’s free cash flows, FCF (t+1) (a) Benchmark Model using CFO only (example for 1990): 1. Estimate cross-sectionally for each 2-digit industry the following regression: FCF (89) = ? + ? CFO(88) + ? . , 2. Predict for each firm in a given 2-digit industry: EFCF (90) = ? + ? CFO(89) using the previously determined industry specific estimated coefficients. . Determine prediction error for each firm in a given 2-digit industry: EFCF (90) . FCF (90) – 18 Here we predict 1990 free cash flows (EFCF(90) from current cash from operations, CFO (89) (and capital expendit ures). First, for each 2-digit industry we regress cross-sectionally free cash flows of 1989 on CFO in 1988, and obtain the estimated coefficients ? and . ? Those coefficients are then used to predict firm specific free cash flows (EFCF) in 1990, using the firm’s actual CFO of 1989. Then, a firm specific prediction error is determined by comparing the firm’s actual 1990 FCF with the predicted one.The same procedure is repeated for every firm and sample year. (b) Restricted Estimates, Model 4 (example for 1990): Estimate cross-sectionally for each 2-digit industry: FCF (89) = ? + ? 1CFO(88) + ? 2? WC * (88) + ? 3EST (88) + ? . The subsequent prediction and error determinations are done as in (a) above. Here we predict 1990 free cash flows from CFO, ? WC* (change in working capital items excluding inventory), EST (estimates), and capital expenditures (not shown in the equation). First, a cross-sectional regression of 1989 free cash flows is run on the 1988 values of CFO, ? WC*, and EST, yielding coefficients ? ? 1, ? 2, and ? 3. Then, firm specific 1990 free cash flows are predicted, using the four industry specific estimated coefficients and the 1989 actual values of CFO, ? WC*, and EST. Finally, these 1990 FCF predictions are compared with the 1990 actual free cash flows to determine the prediction error. The same procedure is repeated for each firm and sample year. (c) Expanded Estimates, Model 5 (example for 1990): Estimate cross-sectionally for each 2-digit industry: FCF (89) = ? + ? 1 CFO(88) + ? 2 ? WC * (88) + ? 3? INV (88) + ? 4 D & A(88) + ? 5 DT (88) + ? 6 EST * (88) + ? . 19The prediction and error determinations are done as in (a) above. Here we predict 1990 free cash flows from 1989 CFO, capital expenditures, and the disaggregated set of estimates (see second diagram at the beginning of this Section). Once more, we run by industry a cross-sectional regression of 1989 FCF on the 1988 values of the independent variables, estimating the ? and ? 1†¦ ? 6 coefficients (and a ? 7 coefficient for 1988 capital expenditures). The firm-specific 1990 free cash flows are predicted using these industry specific coefficients and the actual values of the independent variables in 1989.Computation of the 1990 FCF prediction error follows. B. Prediction of year 2 free cash flows, FCF (t+2) Benchmark Model (example for 1992): 1. Estimate cross-sectionally by 2-digit industry: FCF (90) = ? + ? 1CFO(88) + ? 2. Predict for each firm in a given 2-digit industry: EFCF (92) = ? + ? 1CFO(90) 3. Prediction Error for each firm in a given 2-digit industry: FCF (92) – EFCF (92) This is the prediction of free cash flows in t+2. It follows the earlier procedure with one difference: Here the cross-sectional estimate (first equation) and the forecast (second equation) involve a two-year lag (e. . , FCF in 1990 regressed on CFO of 1988). Same procedure is performed for each firm and sample year. The expanded prediction models incorpora ting disaggregated accruals follow steps (b) and (c), above. We also predict free cash flows for aggregate years t+1 plus t+2, and t+1 through t+3. These predictions are based on the procedures described above, except that aggregated future free cash flows are substituted for single year free cash flows as left-hand variables in the various models. The procedure demonstrated above for FCF is also used to predict cash from operations 20 CFO) in t+1, t+2, and aggregated future years, and to predict earnings in t+1, t+2 and aggregated future years. Two versions of earnings—net income before extraordinary items (NI) and operating income (OI)—are predicted. The various prediction models for earnings are identical to those of free cash flows described above, except that earnings in t+1 and t+2 are substituted for FCF in those models. To summarize, we perform out-of-sample predictions of two versions of cash flows and two versions of earnings from current values of CFO, curre nt values of NI, and CFO plus changes in working capital and various combinations of accruals.To evaluate the quality of the out-of-sample predictions, we compute summary measures of prediction errors derived from the firm- and year-specific estimated errors: the mean and median signed prediction errors indicating the bias in the forecasts, and the mean and median absolute prediction errors which abstract from the sign of the error and indicate forecast accuracy. The firm-specific prediction error in a given year is computed as the realized value of cash flow or earnings minus the predicted cash flow or earnings, divided by average total assets in year t. . 2 Portfolio analysis The third stage of our research design is motivated by Poon and Granger (2003, p. 491) who note: â€Å"Instead of striving to make some statistical inference, [prediction] model performance could be judged on some measures of economic significance. † We interpret their statement as saying that we shoul d not rely solely on the statistical significance of our prediction errors calculated in stage two but should also examine and perhaps even rely more on measures of economic significance.To gauge the economic significance of the contribution of estimates to the usefulness of financial information we perform a series of portfolio tests focusing on the incremental stock returns generated by the estimates-based prediction models. 21 Essentially, we use the out-of-sample predicted values of cash flows (CFO and FCF) and alternatively of earnings (NI and OI), obtained in the second stage of our analysis, to form portfolios.Specifically, for each sample year we rank all firms (across all industries) on predicted firm-specific cash flows or earnings (four rankings, two for cash flows and two for earnings), scaled by average total assets in the end of year t. We then form ten portfolios from each annual ranking and compute risk-adjusted (size & book-to-market adjusted) returns from holding t hese portfolios over several future periods. In assessing the performance of the various predictors (CFO, NI, ? WC*, accruals of estimates), we primarily focus on a zero-investment (hedge) strategy: going long (investing) in the top ortfolio (the 10% of firms with the largest (scaled) predicted cash flows or earnings), and shorting (selling) the bottom portfolio (10% of firms with the lowest predicted cash flows or earnings). The abnormal returns on these zeroinvestment portfolios indicate the economic contribution to investors of using accounting estimates as predictors. Thus, if estimates are useful to investors then portfolios constructed from predictions based on current cash flows and estimates-based accruals should consistently outperform portfolios formed from predictions based on current cash flows only.It should be noted that if markets are efficient concerning the information in accruals—a big if, in light of Sloan (1996)—and if investors select securities us ing procedures similar to our industry-based prediction models specified above, then our subsequent portfolio abnormal returns should be roughly zero. Our purpose in these portfolio tests, however, is not to examine market efficiency, rather to compare the performance of portfolio selection procedures with the estimates-based accruals against similar procedures without accruals (based on past cash flows only).We are thus focusing on the with- and without-accruals comparisons, being agnostic about market efficiency. Stated differently, the comparative abnormal hedge returns across the 22 five prediction models, rather than the statistical significance of those returns, is our focus of analysis. 4. Sample Selection and Descriptive Statistic We obtain accounting data from the 2006 Compustat annual industrial, full coverage, and research files, and use data from the statement of cash flows because Collins and Hribar (2002) suggest that such data are preferable to accruals derived from t he balance sheet.Since reporting a statement of cash flows was mandated by SFAS 95 in 1987, our accounting data span the period 1988 to 2005. 13 In the in-sample regression analysis, each year from 1988 to 2004 is a predictor year (generating the independent variables) while each year from 1989 to 2005 is a predicted year (providing the dependent variables). Thus, 17 in-sample annual regressions are estimated for each industry. Our sample selection procedure is as follows. We start with 75,571 observations with values for NI, CFO, ? WC*, INV, D&A, DT, EST, EST* and CAPEX for the current year, year t, and for NI over a three-year horizon, t-1 to t+1. Firms with all fiscal year ends are included. We control for outliers by following the procedures in Barth et al. (2001). Thus, after eliminating the top and bottom one percentile of current NI and CFO we are left with 73,324 firm-year observations. By excluding observations with market value of equity or sales of less than $10 million, or with share prices below $1, to eliminate economically marginal firms, the number of observations decreases to 51,301.By deleting observations with studentized residuals greater than 3 or less than -3, we are left with 50,288 observations. Since we conduct industry-byindustry in-sample regression analysis we require each industry to have a minimum of 600 observations over the period 1988 to 2004. This criterion reduces the sample to its final size of 13 Valid statement of cash flows data for the year 1987 are available for a relatively small number of firms not enough to do a meaningful industry-by-industry analysis. Thus, we do not use 1987 data. 23 41,124 observations.We obtain stock returns data for the portfolio analysis from the 2006 CRSP files. 14 Table 1 provides summary statistics (variables are scaled by average total assets) and a correlation matrix for out test variables. Panel A shows that depreciation and amortization (D&A) constitutes the bulk of the estimates underl ying accruals (EST): The mean (median) of D&A is 0. 054 (0. 047), close to the mean (median) of EST, 0. 059 (0. 052). The mean of net estimates (EST*), excluding D&A and deferred taxes, is quite large, 0. 019, and is driven mainly by large positive values, as the median value of 0. 04, Q1 of 0. 000 and Q3 of 0. 019 imply. CFO has the lowest while NI has the highest variability (standard deviations of 0. 129 versus 0. 149) among the various earnings and cash flow variables. In panel B all correlations are significant at the 5% level or better. We note the high negative correlations of our estimates variables, EST and EST*, with the income variables, NI and OI. However, the correlations of EST and EST* with both the cash flow variables, CFO and FCF, are much lower; positive for EST and negative for EST*. 4 We repeated all of our analyses with a sample without any outlier removal, namely where we only require non- missing values for the key variables, and at least 600 observations in e ach 2-digit SIC over the sample period 19882004. This sample consists of 65,178 observations and is substantially larger than the sample of 41,124 observations used in the analysis reported below. We find that for many industries the R-squares in the in-sample regressions are higher for the un-truncated data than for the truncated data.The forecast error results are essentially identical to the results from the truncated sample in terms of inferences but the errors are larger. The portfolio abnormal returns results exhibit similar patterns to the results from truncated data. Overall, the un-truncated data yield very similar results to those of the truncated data reported below. 24 5. Empirical Findings: Prediction Tests 5. 1 Stage one: In-sample Regressions Table 2 reports cross-sectional annual regressions, by industry, of CFO (cash from operations) on lagged values of CFO and earnings components (Model 5 in Section 3).The reported coefficient estimates for each industry are the me ans of the yearly coefficients over the 17 year period, 1988 to 2004. The significance of these mean coefficients is based on (nonreported) t-statistics calculated using the mean and standard errors of the 17 yearly coefficients, as in Fama and MacBeth (1973). We report the results for the CFO regressions so that they can be compared to the CFO results reported by BCN. The, in-sample regression results for FCF, NI and OI are very similar to those reported in Table 2. It is evident that in each of the twenty-three ndustries in Table 2 the lagged CFO and ? WC* (change in working capital minus inventory) are highly significant. In the majority of the industries, ? INV (inventory change) is also significant, as is D&A. However, DT (deferred taxes) and EST* (other accruals estimates) are significant for about half of the industries only. These results are quite consistent with BCN’s results reported in their Table 6, Panel B (note that the sum of our DT and EST* variables is the O TH variable in BCN). The fairly large R2s, ranging across industries from 0. 29 to 0. 71, are also consistent with the R2s reported by BCN.Thus, the BCN regression results over the period 1987 to 1996 hold well over our longer period, 1988-2004. Overall, the estimates indicate a strong association between CFO and lagged earnings components, raising expectations about strong out-of-sample performance as well. However, it is important to note that a regression analysis of a given variable on lagged values of that variable along with other data, as frequently conducted in accounting and finance research, is not a conclusive test of predictive ability. As noted in Poon and Granger’s (2003, p. 25 92) survey: â€Å"In all forecast evaluations, it is important to distinguish in-sample and out-ofsample forecasts. In-sample forecast, which is based on parameters estimated using all data in the sample, implicitly assumes parameter estimates are stable through time. In practice, time v ariation of parameter estimates is a critical issue in forecasting. A good forecasting model should be one that can withstand the robustness of an out-of-sample test, a test design that is closer to reality. In our analyses of empirical findings†¦ we focus our attention on studies that implement out-of-sample forecasts. A dramatic example of misplaced inferences drawn on the basis of regression analysis has been recently provided by Goyal and Welch (2007). Their focus is on the prediction of stock market returns based on a variety of variables suggested by prior studies (e. g. dividend yield, earnings-price ratio, book-to-market ratio), using in-sample regression models. After a comprehensive analysis, Goyal and Welch conclude that â€Å"these models have predicted poorly both in-sample and out-of-sample for thirty years now; these models seem unstable, as diagnosed by their out-of-sample predictions nd other statistics; and these models would not have helped an investor with access only to available information to profitably time the market† (Abstract). This important insight motivates our primary analysis which focuses on out-of-sample prediction tests. In the case of predicting stock returns, Goyal and Welch’s concern, in-sample regression results are generally weak and it is therefore not surprising that the out-of-sample predictions of Goyal and Welch perform poorly too.In contrast, in our case of predicting cash flows and earnings, the in-sample regressions (Table 2) perform well, so, whether the more realistic out-of-sample predictions of cash flows and earnings perform equally well is an important empirical issue which we examine next. 26 5. 2 Stage two: Out-of-sample Prediction Tests Table 3 summarizes our main out-of-sample prediction findings. Recall that we predict four key performance indicators: cash from operations (CFO); free cash flows, defined as CFO minus capital expenditures (FCF); net income before extraordinary items (N I); and operating income (OI).There are four prediction horizons: next year, second year ahead, aggregate next two years, and aggregate next three years. Five prediction models are examined (they were discussed and demonstrated in Section 3), where the predictive (independent) variables are: (1) CFO only—the benchmark model, (2) NI only, (3) CFO and the annual change in working capital items excluding inventory (? WC*), (4) CFO plus the change in working capital items excluding inventory (? WC*), as well as the total remaining accruals (EST) which are largely estimates based, including the change in inventory, and (5) our most disaggregated model: CFO, ?WC*, the change in inventories, depreciation and amortization, deferred taxes, and all remaining estimates. Current capital expenditure is included as an additional variable in each of the five models. We report in Table 3 four summary statistics for the prediction errors of our five models: the pooled firm-specific mean absol ute error (MAER) of each of the five models, the pooled mean signed error, or bias (MER), the mean R2s from annual regressions of firm-specific actual values of future cash flows or earnings on the corresponding predicted values, and the average over the years of Theil’s U-statistics. 5 We indicate with an ampersand (&), asterisk (*) or a hash (#) the pooled mean absolute prediction errors (MAER) which are significantly different 15 The reported Theil's U-statistic is the average of the yearly U-statistics. Theil’s U is defined as the square root of ?(actual-forecast)2/? (actual)2. The U statistic can range from zero to one, with zero implying a perfect forecast. Thus, models generating better predictions should have lower U statistics. 27 between Models 1 and 2, Models 1 and 3, and Models 3 and 4, and Models 3 and 5, respectively. 6 We have also computed the sample median signed errors, median absolute errors, and root mean square errors. Results from these indicators are very similar to those reported in Table 3 (we comment in the text on the occasional differences). Below are the main inferences we draw from Table 3, and additional analyses: 1. Prediction of cash flows. Considering the prediction of cash from operations (CFO) and free cash flows (FCF)—left two quadruples of columns in Table 3—we note that the predictions derived from net income only (Model 2) are always significantly inferior to the predictions based on cash from operations only (Model 1).This is true across the four forecast horizons and the four error summary statistics. For example, in predicting one-year-ahead cash from operations (top left panel), the MAER, MER and Theil’s U are lower for Model 1 than for Model 2 (0. 056 vs. 0. 062, 0. 001 vs. 0. 003, and 0. 58 vs. 0. 64, respectively), while the R2 of Model 1 is higher than that of Model 2 (0. 46 vs. 0. 37). The difference in the MAERs is statistically significant, as indicted by the & sign. This pat tern is evident across all eight panels reporting predictions of cash from operations and free cash flows for various horizons.Thus, for one- to three-year forecast horizons, current cash from operations is a better predictor of future cash from operations and free cash flows than current net income. This result is inconsistent with Kim and Kross (2005) findings that in one-year-ahead predictions of cash flows current earnings performs better than current cash flows. 17 16 All the absolute forecast errors (MAER) in Table 3 are statistically significant, with p-values of 0. 01 or better. The majority of the signed errors (MER) are also significant at p-values of 0. 1 or better, and many are statistically significant at least at p-values of 0. 05. The following signed errors are insignificant: Model 1 in forecasting Years 12 CFO, Models 1 and 3 in forecasting Years 1-3 CFO, and Models 2, 4 and 5 in forecasting Years 1-3 OI. 17 It is important to note that Kim and Kross (2005) use bala nce sheet items to calculate cash from operations while we use statement of cash flows data. We were able to replicate the out-of-sample prediction results of Kim and 28Moving on to Model 3, (predictors: CFO and the change in working capital items minus inventory), we note that the CFO and FCF predictions derived from current CFO only (Model 1) under-perform predictions based on current CFO and the change in working capital items excluding inventory, ? WC*. Thus, the mean absolute errors of Model 3 are significantly lower than those of Model 1 in all CFO and FCF panels, except in the FCF panel for the aggregate next three years horizon (bottom FCF panel). 18 The reported R2s and Theil’s U statistics also indicate the under-performance of Model 1 relative to Model 3.For example, in predicting one-yearahead cash from operations (top left panel), the MAER and Theil’s U are lower for Model 3 than for Model 1 (0. 054 vs. 0. 056, and 0. 56 vs. 0. 58, respectively), while the R2 of Model 3 is higher than that of Model 1 (0. 50 vs. 0. 46). Thus, for one- to three-year forecast horizons, the total change in working capital items excluding inventory is incrementally informative over current cash flows. This is relevant for our focus on the usefulness of accounting estimates, because the working capital items, excluding inventory, and with the exception of accounts receivable, are largely free of estimates.We now move to examine the contribution of accounting estimates to cash flow prediction. We do this by comparing the performance of Models 4 and 5 to that of Model 3, where Model 3 becomes now our benchmark given its superior performance up to this point. We note that CFO and FCF predictions derived from Model 4 (based on CFO, the change in working capital items excluding inventory (? WC*), as well as all other accruals including the change in inventory) and Model 5 (based on CFO, ? WC*, the change in inventories, depreciation and amortization, Kross usin g balance sheet items for our sample period.Accordingly, the difference in the results between the two studies is due to the data used. As shown by Collins and Hribar (2002), the cash from operations, and accruals derivation from the statement of cash flows is preferable. 18 Note that despite the very small difference between the MAERs of Models 1 and 3, the mean differences are statistically significant at the 0. 05 level or better (see asterisks). 29 deferred taxes, and all remaining accruals) equally perform or under-perform the predictions from Model 3 (based on CFO and ?WC*). Specifically, the mean absolute errors of Model 3 are significantly lower than or equal to the mean absolute errors of Models 4 and 5 in all the CFO and FCF panels. Furthermore, the reported MERs, R2s and Theil’s U statistics are also consistent with the under-performance of Models 4 and 5 relative to Model 3. For example, in predicting one-year-ahead cash from operations (top left panel), the MAER, MER and Theil’s U for Model 3 are either equal to or lower than for Models 4 and 5 (0. 054 vs. 0. 054 and 0. 055; 0. 001 vs. 0. 02 and 0. 002; and 0. 56 vs. 0. 57 and 0. 57, respectively), while the R2 of Model 3 is equal to or higher than the R2s of Models 4 and 5 (0. 50 vs. 0. 50 and 0. 49). Accordingly, we conclude that for one- to three-year forecast horizons the accounting estimates embedded in accruals, either as a lump sum or disaggregated, do not improve cash flow predictions over current cash from operations and the change in working capital (excluding inventory). 19 Conclusions: Neither total earnings, nor disaggregated estimates-based accruals ystematically improve the prediction of cash flows (CFO or FCF) over the predictions based on current CFO and the change in working capital (excluding inventory). This finding is inconsistent with the FASB’s conceptual stipulation that â€Å"Information about enterprise earnings†¦generally provides a better indi cation of an enterprise’s present and continuing ability to generate favorable cash flows than information limited to the financial aspects of cash receipts and payments† (FASB, 1978, p. IX), though our data start ten years after this statement was issued 2. Prediction of earnings.The two quadruples of columns to the right of Table 3 report prediction performance statistics for net income (NI) and operating income (OI). Here, the 19 These inferences do not change when we examine median signed and absolute prediction errors (available on request). 30 predictions derived from net income (Model 2) significantly outperform those based on cash from operations only (Model 1), for the one-year-ahead forecasts. For example, in predicting next year’s operating income (top right panel), the MAER of Model 2 is significantly lower than that of Model 1 (0. 057 vs. 0. 061).The R2s and Theil’s Us confirm the stronger performance of Model 2, for one-year predictions. Inte restingly, Model 2’s predictions are significantly inferior to Model 1’s in the two-years-ahead and aggregate next three years predictions (second and bottom NI and OI panels). For example, in predicting aggregate three-years-ahead operating income (bottom right panel), the MAER of Model 2 is significantly higher than that of Model 1 (0. 257 vs. 0. 253). Thus, for a one-year-ahead forecast horizon, current net income is a better predictor of future net income and operating income than current cash from operations. 0 Of the five models examined for earnings predictions, the best performer is Model 4— with three variables: CFO, ? WC* (change in working capital excluding inventory), and EST (all other accruals)—for all forecast horizons. Intriguingly, Model 5, where EST is disaggregated to several estimates-based accruals, is somewhat inferior to Model 4. Apparently, predicting from disaggregated accruals results in noisy forecasts. Conclusions: Earnings is a better predictor of near-term earnings than cash flow.Accounting accruals, when disaggregated to working capital items and other accruals, improve further the prediction of operating and net income. No further improvement is achieved from a finer disaggregation of accruals. 6. Robustness Checks 1. How good are our prediction models? 20 Our prediction models are admittedly The median absolute errors are lower for Model 2 than for Model 1 in all NI and OI panels except in the bottom two panels (for the aggregate next two and three years horizons). 31 simple—they obviously abstract from many of the complexities of real life security analysis.Nevertheless, the R2s in Table 3—derived from annual regressions of actual values (future cash flows or earnings) on predicted values—are quite large. Thus, for example, for next year’s predictions (top panels of Table 3), the R2 range is 0. 33-0. 58. As expected, the R2s drop for second year predictions, yet they are still in the reasonable range of 0. 21-0. 37. Thus, despite their simplicity, our prediction models perform reasonably well. 2. Trimming extreme prediction errors. The results of Table 3 are after trimming the top 2% of the absolute forecast errors.We also computed prediction errors after trimming the top and bottom 1% of the forecast errors and without any trimming. The resulting patterns of prediction errors (not reported) are in both cases very similar to those of Table 3. As expected, Table 3 trimmed errors are substantially smaller than the non-trimmed errors, the R2s are larger, and the Theil’s U statistics are lower, yet our conclusions regarding the relative performance of the five models equally apply to the non-trimmed errors. substantially our inferences. 3. Classification by size of accruals.Since the estimates we examine are components of total accruals, we classified the sample firms into three groups, by the size of accruals, to check whether accruals size affe cts our findings. Specifically, for each sample year we ranked the firms by the size of total accruals (scaled by total assets), and then formed three groups: the top 25% of firms (high accruals), the middle 50% (medium accruals), and the bottom 25% (low accruals). We then generated cash flow and earnings predictions for each of the three accruals g