An Evaluation of CBO’s Past Outlay Projections

Since its inception, CBO has regularly published baseline projections of federal revenues and outlays. Those projections, which reflect the assumption that current laws will generally remain unchanged, typically underlie the budget resolutions prepared by the House and Senate Budget Committees as well as CBO’s cost estimates for proposed legislation. The baseline projections can also be useful to policymakers seeking to identify and address budgetary trends that are likely to play out over the coming years if current laws remain in place.

As part of the process of preparing its projections, the agency regularly assesses the quality of its past estimates of federal spending and revenues to refine its methods and improve the accuracy of its projections in the future. For this analysis, CBO reviewed the baseline projections of total outlays as well as those of broad categories of spending that it has issued each spring, focusing on two fiscal years in the period spanned by each projection: the second year (often called the budget year), which usually begins about six months after a spring baseline is released, and the sixth year. To assess the relative quality of its estimates, CBO compared its budget-year projections with those of the Administration.

The quality of CBO’s projections can be measured in various ways, but in this assessment, CBO focuses primarily on two characteristics: statistical bias (the tendency of a set of projections to err in the same direction) and accuracy (how close projected values are to actual amounts).

Any comparison of actual outcomes with projections is complicated by changes in law made after the projections are prepared. Although CBO does not attempt to predict future legislative changes or their effects on outlays when preparing its baseline budget projections, actual outlays are affected by those changes. CBO therefore adjusted its projections to incorporate the estimated effects of legislation that was enacted after the projections were produced. Thus, the analysis presented in this report concentrates on the remaining differences between projected and actual outlays, which are related to CBO’s economic forecast or other factors (referred to as technical factors). For the purposes of this analysis, the agency also removed outlays for the housing entities Fannie Mae and Freddie Mac from its projections and the actual amounts reported by the Treasury Department because CBO and the Administration account for those entities’ transactions differently.

All told, CBO’s projections of outlays for both the budget year and the sixth year have generally been close to actual amounts, although they have been too high, on average. The Administration has also tended to overestimate baseline outlays for the budget year. (The Administration has not published detailed information on differences between its projections and actual outlays over the six-year horizon, so CBO could not compare those longer-term projections.) In general, both CBO’s and the Administration’s projection errors followed similar patterns and were larger for years in which unanticipated events that had large budgetary effects occurred. The Administration’s projections for budget years 1993 through 2005 were about as accurate as CBO’s projections. Since 2005, however, the Administration has overestimated spending in the budget year by more than CBO has in all but one year.

How Close Were CBO’s Budget-Year Projections to Actual Outlays?

Since 1984, CBO has tended to overestimate total outlays (after adjustments for legislative changes) for the budget year; the average error for total outlays is 1.7 percent. Of the 32 budget-year projections produced from 1984 to 2015, 25 exceeded actual outlays.

Although often too high, CBO’s projections of total outlays for the budget year have generally been close to actual amounts. Half of the agency’s projections made since 1984 have differed from actual outlays by less than 2 percent of the actual amounts. A few projections, however, had significantly larger errors. For example, in 1992, CBO’s estimate of total outlays for 1993 was too high by about 8 percent.

The mean absolute error of the budget-year projections—that is, the average of all errors without regard for whether they were positive or negative—equals 2.3 percent. If CBO’s current budget-year projection of $4.1 trillion in total outlays had an error the size of that mean absolute error, actual outlays for 2018 would be higher or lower than the agency projected by $0.1 trillion—or about 0.5 percent of gross domestic product (GDP).

How Close Were CBO’s Sixth-Year Projections to Actual Outlays?

CBO’s sixth-year projections also tended to be too high, exceeding actual outlays in 20 of 28 years. Both the average error (3.0 percent) and the mean absolute error (5.9 percent) of the sixth-year projections made between 1984 and 2011 were larger than those of the budget-year projections covered by this analysis. Of the 28 sixth-year projections, 9 were within 4 percent of actual outlays, and 4 were off by 10 percent or more. If CBO’s June 2017 projection of $5.2 trillion in total outlays for 2022 had an error equal to that mean absolute error, actual outlays would be higher or lower than the agency projected by $0.3 trillion (or 1.3 percent of GDP).

What Factors Have Contributed to Differences Between CBO’s Projections and Actual Outlays?

Many factors account for the differences between CBO’s projections and actual outlays. Some of those factors relate to the agency’s economic forecast. Forecasting interest rates has been particularly challenging, even during relatively stable periods, and errors in interest rate forecasts have significantly affected CBO’s projections of interest payments. In addition, turning points in the economy can have significant effects on the budget. During recessions, outlays for countercyclical income security programs—to pay unemployment benefits, for example—can increase rapidly. Such turning points, and the cumulative budgetary effects that result from them, are very difficult to anticipate.

Other factors unrelated to the economic forecast have also contributed to misestimates in CBO’s projections. For example, quickly identifying new trends—particularly in federal spending on health care—and incorporating them into the agency’s baseline projections has proved difficult. Anticipating large and sudden increases or decreases in spending for government programs related to the financial sector has also been challenging. Although such developments related to the financial sector have been rare, when they have occurred, they have resulted in CBO’s misestimating outlays in its budget-year projections.

After observing how those and other factors affected the accuracy of its past outlay projections, CBO noted the following:

• Projections of net interest outlays were not as accurate as those of most other categories of spending, primarily because CBO significantly overestimated interest rates during the most recent recession and subsequent recovery (as did other economic forecasters). The sixth-year projections of such outlays were substantially less accurate than the budget-year projections. When net interest outlays are excluded from the evaluation of the projections for 1997 to 2016, the mean error of CBO’s sixth-year projections of overall outlays over that period is reduced by more than half.
• Sixth-year projections of Medicare and Medicaid spending were significantly less accurate than those of all other categories of spending except for net interest outlays, in part because it took CBO several years to fully incorporate into its projections the slowing growth in spending for those programs that occurred between 1996 and 2002 and between 2008 and 2014. (Budget-year projections for those programs were considerably more accurate.)
• Occasionally, large errors in projections of spending for other programs—mostly related to financial activities—significantly influenced overall measures of bias and accuracy. For example, unanticipated changes in spending for deposit insurance that stemmed from the savings and loan crisis accounted for roughly 90 percent of the large difference between CBO’s budget-year projection of total outlays for 1993 and the actual total. In 2010, the discrepancies between the projections of outlays for deposit insurance and for the Troubled Asset Relief Program (TARP) prepared the previous year and the actual amounts together exceeded the difference between the projected and actual amounts of total outlays. Excluding those two programs would lower the average error of CBO’s budget-year projections of total outlays for 1993 to 2016 by about two-thirds of a percentage point.
• Differences between CBO’s estimates of total outlays in the budget year and actual outlays increased notably for the years immediately following the 2007–2009 recession and have narrowed somewhat since then. The differences between CBO’s sixth-year projections and actual outlays also increased during those years, but those larger differences—stemming from projections made before 2012—have persisted in recent years.