At a Glance
In this report, the Congressional Budget Office assesses its two-year and five-year economic forecasts and compares them with forecasts of the Administration and the Blue Chip consensus, an average of about 50 private-sector forecasts.
Variables Examined. CBO examines its forecasts of output growth, the unemployment rate, inflation, interest rates, and wages and salaries.
Measures of Quality. CBO focuses on four measures of forecast quality—average error, mean absolute error, root mean square error, and two-thirds spread of errors—that help the agency identify the centeredness (that is, the opposite of statistical bias), accuracy, and dispersion of its forecast errors.
The Quality of CBO’s Forecasts. CBO’s forecasts of selected economic variables are, on average, too high by small amounts. As measured by the root mean square error and the mean absolute error, the two-year forecasts are not, on the whole, more accurate than the five-year ones.
Comparison With Other Forecasts. The degree of centeredness varies by forecaster and variable. For example, CBO and the Blue Chip consensus tend to produce more-centered forecasts of output growth but less-centered forecasts of interest rates than the Administration. CBO’s forecasts are slightly more accurate and, for most variables, have the same or smaller two-thirds spreads. For all four quality measures, CBO’s forecasts are roughly comparable to those of the Blue Chip consensus.
Sources of Forecast Errors. All forecasters failed to anticipate certain key economic developments, resulting in significant forecast errors. The main sources of those errors are turning points in the business cycle, changes in labor productivity trends and crude oil prices, the persistent decline in interest rates, the decline in labor income as a share of gross domestic product, and data revisions.
Forecast Uncertainty. CBO uses its past forecast errors to gauge the uncertainty of its current forecasts. For example, using the root mean square error, CBO estimated in July 2021 that there is approximately a two-thirds chance that economic growth will average between 1.9 percent and 4.3 percent over the next five years. Its central estimate is 3.1 percent.
The Congressional Budget Office has corrected this report since its original publication. Corrections are listed at the end of the report.
Each year, the Congressional Budget Office prepares economic forecasts that underlie its projections of the federal budget. CBO forecasts hundreds of economic variables, but some—including output growth, the unemployment rate, inflation, interest rates, and wages and salaries—play a particularly significant role in the agency’s budget projections. To evaluate the quality of its economic projections, estimate uncertainty ranges, and isolate the effect of economic errors on budgetary projections, CBO regularly analyzes its historical forecast errors. That analysis serves as a tool for assessing the usefulness of the agency’s projections.
In this report, CBO evaluates its two-year and five-year economic forecasts from as early as 1976 and compares them with analogous forecasts from the Administration and the Blue Chip consensus—an average of about 50 private-sector forecasts published in Blue Chip Economic Indicators.1 External comparisons help identify areas in which the agency has tended to make larger errors than other analysts. They also indicate the extent to which imperfect information may have caused all forecasters to miss patterns or turning points in the economy.
How CBO Measures Forecast Quality
CBO’s analysis focuses on four metrics of forecast quality—average error, mean absolute error, root mean square error, and two-thirds spread of errors:
- The average error is CBO’s primary measure of centeredness, which indicates how close the average forecast value is to the average actual value over time. Centeredness is the opposite of statistical bias, which quantifies the degree to which a forecaster’s projections are too high or too low over a period of time.
- The mean absolute error is calculated by taking the average of the absolute value of the forecast errors to show the size of the error rather than its direction.
- The root mean square error, which is calculated by squaring the forecast errors, averaging those squares, and taking the square root of that average, is CBO’s primary measure of accuracy, or the degree to which forecast values are dispersed around actual outcomes.
- The two-thirds spread, computed as the range between the minimum and maximum errors after removing the one-sixth largest errors and the one-sixth smallest errors, illustrates a forecaster’s typical range of errors and provides information about the extent of the dispersion of those errors.
The Quality of CBO’s Forecasts
Forecasts that have an average error of zero are, on average, neither too high nor too low. CBO’s forecasts of economic variables considered in this report tend to exhibit small positive average errors—that is, on average, they are too high by small amounts.2
In general, it is difficult to compare quality measures across variables because the magnitudes of variables can differ substantially, and some variables are easier or harder to forecast than others. It is possible, however, to compare those measures across forecast horizons for a given variable. For example, as measured by the mean absolute error or the root mean square error, five-year forecasts of interest rates and the unemployment rate are less accurate than two-year forecasts of those variables. For the other variables, CBO’s forecasts are as accurate or more accurate at the five-year horizon on the basis of those two measures of forecast quality. For most variables, the agency’s two- and five-year forecasts exhibit a similar two-thirds spread of errors.
Comparing CBO’s Forecasts With Those of the Administration and the Blue Chip Consensus
In general, forecasts produced by CBO, the Administration, and the Blue Chip consensus display similar error patterns over time. Because all forecasters faced the same challenges, periods in which CBO made large overestimates typically coincide with periods in which other forecasters made similarly large overestimates. Over time, however, even small differences in the magnitude of errors can result in appreciable differences in measures of forecast quality.
CBO and the Blue Chip consensus tend to produce more-centered forecasts of output growth but less-centered forecasts of interest rates than the Administration (see Table 1). As measured by the root mean square error, half of CBO’s forecasts are slightly more accurate than the Administration’s forecasts, and the others are equally accurate; by that measure, CBO’s forecasts are roughly comparable to the Blue Chip consensus forecasts. Finally, with a few exceptions, CBO’s forecasts tend to exhibit smaller two-thirds spreads than the Administration’s forecasts do; the agency’s forecasts tend to have two-thirds spreads that are similar to those of the Blue Chip consensus forecasts.
Sources of Forecast Errors
Forecasters made large errors in their economic forecasts as a result of several economic developments:
- Turning points in the business cycle;
- Changes in labor productivity trends;
- Changes in crude oil prices;
- The persistent decline in interest rates;
- The decline in the labor share—that is, labor income as a share of gross domestic product (GDP);
- Data revisions; and
- The coronavirus pandemic.
Some of those developments resulted in errors in forecasting specific variables. Changes in crude oil prices, for example, resulted in misestimates of inflation in the consumer price index (CPI). Other developments, such as turning points in the business cycle, affect the entirety of an economic forecast, and their effects are observable in the error patterns of several variables.
How CBO Uses Its Forecast Errors to Estimate Uncertainty
CBO most frequently uses the root mean square error of its historical forecasts to quantify the uncertainty of its current economic projections. For example, CBO’s baseline forecast for the growth of real GDP (that is, of GDP adjusted to remove the effects of inflation) over the next five years is 3.1 percent. Using its historical root mean square error for that variable (1.2 percentage points), CBO then estimates that there is approximately a two-thirds chance that the rate of real GDP growth over the next five years will be between 1.9 percent and 4.3 percent. Although the agency typically uses the historical root mean square error to estimate uncertainty, it uses the two-thirds spread of errors in instances where the statistical bias of its historical forecasts is high relative to its root mean square error.
CBO’s Methods for Evaluating Forecasts
To evaluate the quality of its forecasts, CBO examines its historical forecast errors and compares them with errors made by the Administration and the Blue Chip consensus. Comparison with the Blue Chip consensus is particularly useful because that forecast incorporates a wide variety of viewpoints and methods, and some research has suggested that composite forecasts often provide better estimates than projections made by a single forecaster.3
For this analysis, CBO reviewed the economic projections that it published each winter, usually in January, beginning in 1976, as the basis for its baseline budget projections. Each of those economic projections spanned the current year (that is, the calendar year already under way) and either 5 or 10 subsequent years.
This report evaluates CBO’s economic forecasts over the first two years and first five years of its baseline projection period. CBO evaluates forecasts over the two-year time horizon because that interval is most relevant when the agency is preparing its baseline budget projections for the upcoming fiscal year. (That second year is often called the budget year.) CBO evaluates forecasts over the five-year time horizon to better understand the quality of its longer-term projections.4 The agency calculates errors by subtracting the average actual value over the time period being analyzed from the average projected value over that period.
The span of years evaluated for this analysis varies by economic indicator and depends on two factors: the availability of historical forecast data and the availability of actual economic data. To ensure that differences in the availability of forecast data do not affect the comparisons of forecast errors, those comparisons begin in the earliest year for which forecast data were available for all three sets of forecasts. So, although CBO has two-year forecasts of real output growth dating back to 1976, its forecast errors for comparison purposes are computed starting in 1980—the first year the Blue Chip consensus data were available.5 Likewise, the final year of forecast analysis depends on the availability of actual economic data. This report incorporates data through the end of 2020, which allows CBO to analyze two-year forecasts that were made through the beginning of 2019 and five-year forecasts that were made through the beginning of 2016. (See the appendix for details.)
Selecting Forecast Variables
CBO examines 10 forecast variables on the basis of their relative importance in the economic outlook and their relevance to projections of revenues, outlays, and deficits. Those variables include the following:
- Output growth,
- The unemployment rate,
- Interest rates, and
- Wages and salaries.
Projections of real and nominal output growth are fundamental to CBO’s budget projections. Faster output growth is typically accompanied by faster growth in real income and hence faster growth of revenues from individual income taxes. Similarly, periods of faster output growth are typically associated with smaller transfer payments and smaller expenditures on unemployment insurance, resulting in lower outlays.
This analysis also examines CBO’s errors in forecasting the unemployment rate, a key component of the agency’s economic forecast. CBO’s forecast of the unemployment rate affects its projections of inflation, interest rates, and other labor market variables and also informs the agency’s projections of certain outlays, including those for unemployment compensation.
CBO’s evaluation of inflation forecasts focuses on two measures: the percentage change in the CPI and the inflation differential, which is computed as the difference between growth in the CPI and growth in the output price index.6 All else being equal, higher CPI inflation implies faster growth in federal outlays (because the index is used to adjust payments to Social Security beneficiaries as well as payments made for some other programs) and slower growth in federal revenues (because elements of the individual income tax, including the tax brackets, are indexed to the CPI).7 Growth in the output price index is closely linked to growth in nominal income subject to federal taxes, which implies faster growth in revenues. Consequently, if CPI inflation was higher than anticipated and the output price index grew more slowly than anticipated, the projected deficit would generally be larger than expected.
The interest rates on 3-month Treasury bills and 10-year Treasury notes summarize CBO’s projections of short- and long-term interest rates, respectively. Interest rates primarily affect the budget through their effect on net interest outlays—the difference between income earned on interest-bearing assets and the cost of servicing the debt. As a result, overestimates of interest rates result in overestimates of debt and deficits. This report also analyzes the real 3-month Treasury bill rate, which is computed by removing the effects of CPI inflation from forecasts of the nominal 3-month Treasury bill rate. Considering errors in projecting the real 3-month Treasury bill rate isolates interest rate errors from errors in inflation projections.
Finally, CBO examines growth in wage and salary disbursements and the change in those disbursements as a share of output. Wages and salaries are the largest component of national income, and their growth informs CBO’s revenue projections and its analysis of the distribution of income. Analyzing wages and salaries as a share of output offers an approximation of forecasters’ views about the labor share of national income and helps isolate errors in projecting wages and salaries from errors in projecting nominal output.
Calculating Forecast Errors
CBO computes each forecast error as the difference between the average forecast value and the average actual value. (See Box 1 for an example of how CBO calculates its forecast errors.) The actual values are reported as calendar year averages and are based on the latest available data from various agencies. A positive error indicates that the forecast value exceeded the actual value, whereas a negative error indicates that the forecast value was below the actual value.
How CBO Calculates Economic Forecast Errors
The Congressional Budget Office calculates forecast errors by subtracting the average actual value of an economic indicator over a two-year (or five-year) period from the average projected value of that indicator over the same period. For example, to calculate the error for the two-year forecast of the growth of real gross domestic product (that is, GDP adjusted to remove the effects of inflation) that was published in the January 2000 Budget and Economic Outlook, CBO first calculated the geometric average of the projected growth rates of real GDP for calendar years 2000 and 2001, which was 3.2 percent.1 The agency then calculated the average actual growth rate of real GDP for those two years, which was 2.6 percent. Finally, it subtracted the average actual rate of 2.6 percent from the average projected rate of 3.2 percent, resulting in an error of 0.6 percentage points. To determine the error for the five-year forecast made that same year, CBO took the averages of projected and actual output growth rates for calendar years 2000 through 2004.
1. The geometric average is calculated by first multiplying the growth rates for each year and then taking the nth root of that product. It is the appropriate measure for averaging growth rates and interest rates and is used to calculate the average for all indicators except the change in wages and salaries as a percentage of output. Because that indicator is a ratio rather than a growth rate, the appropriate measure for averaging is the arithmetic average.
Example: Calculating the Error in the Two-Year Forecast of Real GDP That CBO Published in January 2000
The method used to calculate forecast errors for this report differs from the method used in CBO’s other forecast evaluations.8 In those reports, errors were calculated for a single fiscal year. For example, the error in CBO’s two-year revenue projection for 2007 is the percentage difference between the actual amount of revenues received in fiscal year 2007 and the revenues projected for that year in January 2006.9 In this report, errors are calculated over a span of either two or five calendar years. For example, the errors in the two-year forecasts of economic variables made in January 2006 are the average of the errors for 2006 and 2007.
Measuring Forecast Quality
This evaluation focuses on four metrics of forecast quality: average error, mean absolute error, root mean square error, and two-thirds spread of errors. Together, those measures help CBO identify the centeredness, accuracy, and dispersion of its forecast errors. Other measures of forecast quality, such as whether forecasters optimally incorporate all relevant information when making their projections, are harder to assess.10
Average Error. CBO primarily uses the average error—the average of forecast errors—to measure the centeredness of each variable. The agency uses centeredness to determine whether its forecasts are systematically too high or too low relative to actual economic outcomes. CBO’s goal is to provide forecasts of economic indicators that lie in the middle of the distribution of possible outcomes.
The average error does not, however, provide a complete characterization of the quality of a forecast. Because positive and negative errors are added together to calculate the average, forecast underestimates and overestimates offset one another. A small average error might indicate that all forecasts had small errors, but a small average error can also result from large overestimates and large underestimates that mostly offset one another. CBO uses the average error as its primary measure of statistical bias because it is widely used and easily interpretable.11
Mean Absolute Error. CBO uses mean absolute error as another measure for calculating forecast errors. It is calculated by taking the average of the absolute value of the forecast errors, and it is useful to determine the magnitude of the error regardless of the direction.
Root Mean Square Error. CBO’s primary measure of forecast accuracy, the root mean square error, is calculated by squaring the forecast errors, averaging those squares, and taking the square root of that average.12 That calculation places greater weight on instances in which the forecast values deviate significantly from actual values. Unlike the computation of mean error, forecast underestimates and overestimates do not offset one another when computing the root mean square error.
CBO also uses the root mean square error of its historical forecasts to quantify the uncertainty of its current economic projections. When CBO’s historical forecasts are well centered and have an approximately normal distribution of errors (that is, the values are distributed symmetrically around the average), about two-thirds of actual values will be within a range of plus or minus one root mean square error of the forecasted values. Since 2017, CBO has used that method to quantify the uncertainty of its projections of real GDP.
Two-Thirds Spread. CBO uses the two-thirds spread of errors—defined as the difference between the minimum and maximum error after removing the one-sixth largest and one-sixth smallest errors—to measure the dispersion of its forecast errors. Larger two-thirds spreads imply greater variability in forecast errors, whereas smaller two-thirds spreads imply a narrower range of forecast misestimates.
In certain cases, CBO uses the two-thirds spread instead of the root mean square error to quantify the uncertainty of its current economic projections. The agency relies on the two-thirds spread when the ratio of the average error to the root mean square error is greater than or equal to 0.3. Above that threshold, the root mean square error reflects a substantial amount of bias in the estimates in addition to their variance. When that criterion is met, CBO estimates that about two-thirds of actual values will be within a range of plus or minus one-half of the two-thirds spread of the forecasted values.
Limitations of the Forecast Evaluations
CBO’s interpretation of forecast errors is somewhat limited for two reasons. First, forecast methodology continues to evolve. Over time, CBO and other forecasters have adjusted the procedures they use to develop economic forecasts in response to changes in the economy and advances in forecasting methods. Even when such adjustments improve the quality of forecasts, they make it difficult to draw inferences about the size and direction of future errors.
The second challenge is understanding the effects of different assumptions about future fiscal policy. CBO is required by statute to assume that future fiscal policy will generally reflect the provisions in current law, an approach that derives from the agency’s responsibility to provide a benchmark for lawmakers as they consider proposed legislative changes.13 When the Administration prepares its forecasts, however, it assumes that the fiscal policy in the President’s proposed budget will be adopted. The private forecasters included in the Blue Chip survey all make their own assumptions about fiscal policy, but the survey does not report them.
Forecast errors may be affected by those different fiscal policy assumptions, especially when forecasts are made while policymakers are considering major legislative changes. In early 2009, for example, contributors to the Blue Chip consensus reported that they expected additional fiscal stimulus, which implied stronger output growth than would be expected under current law. By contrast, CBO’s growth projections were tempered by the requirement that its forecasts reflect current law. In February 2009, shortly after CBO’s forecast was published, lawmakers enacted the American Recovery and Reinvestment Act (Public Law 111-5). Similarly, in early 2017, the Blue Chip consensus forecast for 2017 and 2018 probably incorporated some anticipation of a tax cut, as well as other changes in fiscal policy that would boost output in those years. Those anticipated changes probably led the Blue Chip consensus economic forecast to be stronger than CBO’s in early 2017.14 Finally, during the pandemic, some Blue Chip forecasters assumed that legislative action would occur to ease the crisis, which CBO’s current-law forecast did not take into account; however, the forecast error data for that period are not yet available.
The Quality of CBO’s Forecasts
Although CBO’s forecasts of the examined economic variables exhibit small positive average errors, there are notable differences between variables. In particular, CBO’s forecasts of interest rates, particularly for 3-month Treasury bills, exhibit larger average errors than its forecasts of most other economic indicators in the two-year horizon. In the five-year time frame, interest rates, growth of nominal output, and growth of wages and salaries exhibit higher average errors than in other forecasts.
Similarly, as measured by the root mean square error and the mean absolute error, CBO’s two-year forecasts are not, on the whole, more accurate than its five-year forecasts, because anticipating short-term fluctuations is often more difficult than identifying long-term trends. For example, the agency’s two-year forecasts of inflation as measured by the CPI are less accurate than its five-year forecasts of that variable. That pattern does not hold for all variables, however. For example, CBO’s two-year forecasts of interest rates (for both the 3-month Treasury bill and the 10-year Treasury note) are more accurate than its five-year forecasts of that variable, as is the two-year forecast for the unemployment rate.
CBO’s forecasts also tend to exhibit similar two-thirds spreads for most variables across the two- and five-year forecast horizons, but exceptions do occur. For example, CBO’s five-year forecasts of the unemployment rate and the real interest rate on 3-month Treasury bills have larger error spreads than its comparable two-year forecasts. Conversely, the agency’s forecasts of CPI-U inflation and growth of wages and salaries have smaller two-thirds spreads over the five-year horizon than over the two-year horizon.
A Comparison of Forecast Quality
CBO compares its economic forecasts with analogous forecasts produced by the Administration and the Blue Chip consensus. (For a comparison of CBO’s two-year forecasts with forecasts made by the Federal Reserve, see Box 2.) The agency examines projections of output growth, the unemployment rate, inflation, interest rates, and growth in wages and salaries. Each set of forecasts displays similar error patterns over time, but small differences in the magnitude of those errors lead to some appreciable differences in measures of forecast quality.
Comparing CBO’s and the Federal Reserve’s Two-Year Forecasts
Like the Administration and the Blue Chip consensus, the Federal Reserve regularly produces economic forecasts that serve as useful comparisons when evaluating the quality of forecasts by the Congressional Budget Office. The scope of the Federal Reserve’s forecasts is limited: It does not publish forecasts of interest rates for Treasury securities or growth in wages and salaries, nor does it publish any five-year forecasts. As a result, CBO did not include the Federal Reserve’s forecasts in the principal analysis for this report. However, the Federal Reserve does publish comparable forecasts of real output growth and consumer price inflation for a two-year time horizon.
CBO’s forecasts of real output growth are generally similar to the Federal Reserve’s forecasts (see the figure). For all measures of forecast quality, both CBO and the Federal Reserve produce forecasts of real output growth that are relatively unbiased and that display a similar degree of accuracy. CBO’s forecasts, however, have a smaller two-thirds spread of errors.
CBO’s and the Federal Reserve’s forecasts of consumer price inflation are also quite similar. Each set of forecasts has a similar degree of accuracy and comparable two-thirds spreads. However, CBO’s forecast of consumer price inflation tends to display an upward bias, whereas the Federal Reserve tends to produce unbiased forecasts.
The Federal Reserve’s forecasts differ from CBO’s forecasts in two ways. First, the Federal Reserve’s forecasts include the effects of anticipated changes in fiscal policy, whereas CBO’s forecasts reflect the assumption that current laws governing fiscal policy will remain generally unchanged. Second, the Federal Reserve’s recent forecasts are modal—that is, they represent the single most likely outcome for the economy. By contrast, CBO’s forecasts represent the middle of the range of possible outcomes. In periods when the range of possible outcomes is highly skewed, the Federal Reserve’s forecasts will differ from CBO’s.
Comparison of Two-Year Forecasts by CBO and the Federal Reserve
For average errors, CBO’s two- and five-year economic forecasts are broadly similar to forecasts produced by the Administration and the Blue Chip consensus. However, CBO and the Blue Chip consensus have smaller average errors when forecasting output growth, whereas the Administration produces the most-centered forecasts of interest rates. In terms of root mean square errors, CBO’s forecasts are similar to the Blue Chip consensus forecasts and are equal to or slightly more accurate than the Administration’s forecasts. Finally, CBO’s forecasts tend to display two-thirds spreads that are smaller than those of the Administration’s forecasts and similar to those of the Blue Chip consensus forecasts.15 There are exceptions, however, for some variables and forecast horizons (see Table 1).
Real and Nominal Output Growth
CBO’s forecasts of real and nominal output growth are more centered and display the same accuracy as or greater accuracy than the Administration’s forecasts. CBO also has equal or smaller two-thirds spreads than the Administration, except for nominal output growth at the five-year time horizon. Compared with the Blue Chip consensus forecasts, CBO’s forecasts are roughly comparable for the various quality measures. Although CBO’s nominal output forecasts are the same as or slightly more centered than the Blue Chip consensus forecasts, its real output forecasts are the same as or slightly less centered than those of the Blue Chip consensus. The Blue Chip consensus produces five-year forecasts of real output growth that have smaller two-thirds spreads than the corresponding CBO forecasts.
CBO, the Administration, and the Blue Chip consensus tend to produce similar forecasts of the unemployment rate, which results in similar measures of forecast quality. Over both the two- and five-year time horizons, all three sets of forecasts show little variation in their average errors and root mean square errors. Although CBO has slightly overestimated the unemployment rate over the two-year time horizon, all three forecasters produced underestimates over the five-year horizon. The forecasts also show little variation in the two-thirds spread over the two-year horizon but slight differences in the five-year spread, in which the Administration’s spread is larger than those of CBO and the Blue Chip consensus.
CBO’s forecasts of CPI inflation are similar to those of the Administration and the Blue Chip consensus over the two-year time horizon. At the five-year time horizon, CBO’s inflation forecasts are more centered and more accurate than forecasts from the Blue Chip consensus. Likewise, CBO’s five-year inflation forecasts display a smaller two-thirds spread than the forecasts of both the Administration and the Blue Chip consensus.
CBO’s forecasts of the inflation differential are more centered and more accurate than those of the Administration and the Blue Chip consensus. CBO’s forecasts also have a smaller dispersion of errors than the Administration’s forecasts but are comparable to the Blue Chip consensus forecasts. Over the five-year time horizon, each set of forecasts has underestimated the inflation differential, on average.
All forecasters have struggled to produce high-quality forecasts of real and nominal interest rates. At the two-year horizon, CBO, the Administration, and the Blue Chip consensus produce interest rate forecasts that tend to overpredict the actual value. The Administration’s interest rate forecasts display more centeredness but also slightly less accuracy than the other forecasters because its forecasts tend to include large and partially offsetting overpredictions and underpredictions. The two-thirds spread measures do not follow a clear pattern; CBO and the Administration have similar two-thirds spreads for all variables except the interest rate on 10-year Treasury notes. In addition, the spread for CBO’s forecasts of the nominal interest rate on 3-month Treasury bills is notably larger than that of the Blue Chip consensus.
Over the five-year time horizon, a similar pattern holds. CBO produces interest rate forecasts that are considerably less centered than, but roughly as accurate as, the Administration’s forecasts. CBO’s forecasts are comparable to the Blue Chip consensus forecasts. Results over the five-year horizon differ from those over the two-year horizon in one respect: The Administration produces forecasts with a much larger two-thirds spread than the forecasts of both CBO and the Blue Chip consensus.
Wages and Salaries
Over the two-year horizon, CBO’s forecasts of the growth in wages and salaries are more centered and more accurate than the Administration’s forecasts. (The Blue Chip consensus does not provide forecasts of the growth in wages and salaries.) Over the five-year time horizon, CBO’s forecasts are similar in centeredness and accuracy but have a smaller two-thirds spread than the Administration’s forecasts. Over both time horizons, CBO’s and the Administration’s forecasts exhibit significant upward bias.16 The forecasts of wages and salaries as a share of output are nearly indistinguishable from one another.
Some Sources of Forecast Errors
Forecast errors often occur in response to difficulties in anticipating significant economic developments. Such developments include turning points in the business cycle, changes in labor productivity trends, changes in crude oil prices, the persistent decline in interest rates, the declining labor share, and data revisions. Some of those developments are closely tied to errors in forecasting specific variables—for example, the changes in crude oil prices that resulted in misestimates of CPI inflation. Other developments, such as turning points in the business cycle, have wide-ranging effects on economic forecasts and affect the projections of many variables.
Turning Points in the Business Cycle
Business cycle peaks and troughs mark the beginning and end of recessions, or periods of significant economic contraction. This analysis covers five complete recessions—those in 1980, 1981 to 1982, 1990 to 1991, 2001, and 2007 to 2009—as well as the most recent recession brought on by the pandemic in 2020 (see Box 3). Although the depth and duration of the recessions differed, all contributed to forecast misestimates that were substantially larger than those made in nonrecession years. The root mean square errors for all the series considered in this analysis and for all three forecasters are larger in periods that overlap with recessions (see Figure 1). For all series and forecasters, the frequency of large errors (defined as the top three largest errors in absolute value) is greater during periods of recessions.
Forecast Errors and the 2020 Recession
The 2020 recession brought on by the coronavirus pandemic dramatically affected many of the economic variables that the Congressional Budget Office forecasts. It will take several more years before the effect of that recession on forecast errors can be fully assessed (see the table below).1 However, the most recent two- and five-year forecasts completed in 2016 and 2019, respectively, include CBO’s forecast for 2020, so it is possible to analyze how the 2020 recession affected those forecast errors.
Forecasts in purple are analyzed in this report. Forecasts in blue will be analyzed in future reports.
Years Spanned by Forecasts
Two-Year Forecasts Completed in 2019
CBO, the Administration, and the Blue Chip consensus did not anticipate the 2020 recession when they completed their two-year forecasts in January 2019. The decline in gross domestic product (GDP) in the second quarter of 2020 (31 percent at an annual rate) and the increase in real GDP (that is, GDP adjusted to remove the effects of inflation) in the third quarter of 2020 (33 percent at an annual rate) were unprecedented (see the next figure). The errors associated with the two-year forecasts completed in early 2019 are therefore quite large.
Two-Year Forecasts of Real GDP Growth
Number of Quarters When Real GDP Growth Was Within a Given Range, 1948 to 2020
Although the forecast errors for the two-year forecasts of real GDP in 2019 are large, because of the sharp rebound in early 2020, they are not the largest that CBO and the other forecasters have experienced over the past three decades. The largest two-year real GDP forecast errors for all three forecasters were made in 2008 (see the table below). The particularly large underestimates in the late 1990s were probably due to an unanticipated increase in labor productivity.
Three Largest Negative and Positive Two-Year Forecast Errors for Real GDP
Five-Year Forecasts Completed in 2016
The 2020 recession brought on by the pandemic was not anticipated when CBO and other forecasters completed their five-year forecasts in early 2016 (see the next figure). However, because the 2020 recession affected only one of the five years covered in those forecasts, the overall effect on the forecast errors was more muted than that for the two-year forecasts. The five-year forecasts for real GDP completed in early 2016 resulted in five-year forecast errors of 1.1 percentage points for CBO, 1.3 percentage points for the Administration, and 1.1 percentage points for the Blue Chip consensus. Those forecast errors are in the upper quartile of the five-year real GDP forecast errors experienced by all three forecasters. However, they are not among the top three five-year real GDP forecast errors for any of the forecasters (see the next table). The time periods during which the largest five-year forecast errors occurred are generally the same as the time periods during which the largest two-year forecast errors occurred. Five-year forecasts completed in 2017 through 2020 are more likely to be significantly affected by the 2020 recession than the 2016 forecasts.
Five-Year Forecasts of Real GDP Growth
Three Largest Negative and Positive Five-Year Forecast Errors for Real GDP
1. The forecast errors associated with the two-year forecasts completed in 2020 and the five-year forecasts completed in 2017, 2018, 2019, and 2020 will also be affected by the 2020 recession.
Forecasters struggle to produce accurate forecasts around business cycle downturns for three main reasons. Because business cycle downturns are hard to predict, the first challenge is anticipating when one will occur. During periods of growth, it is often difficult to identify which economic imbalances will ultimately result in a recession. Moreover, it is often difficult to know whether the economy is in a recession until well after it has begun. Thus, forecasts made just before a recession tend to be overly optimistic about economic outcomes.
The second challenge is identifying the length and severity of a recession. Recessions often coincide with periods of great uncertainty, both in economic outcomes and in fiscal and monetary policy. Under those conditions, a wide range of outcomes can appear equally probable, making it difficult to produce a forecast that is in the middle of the distribution of possible outcomes.
The final challenge is predicting the speed with which the economy will recover from a recession.17 Until the early 1990s, the U.S. economy typically grew rapidly for several quarters after a recession ended. Since then, however, recoveries have been much slower.18 Failing to predict slower economic recoveries has caused forecasters to overestimate economic growth in the aftermath of business cycle downturns.
Changes in Labor Productivity Trends
Growth of labor productivity in the nonfarm business sector—the ratio of real output to labor hours worked—is a key input into CBO’s forecast of real output growth. Although growth in labor productivity fluctuates widely from quarter to quarter, its average growth rate tends to remain stable over long periods of time. The stability of that average typically helps forecasters estimate real output growth over longer time horizons. However, three shifts in labor productivity trends have contributed to errors in projecting real output growth (see Figure 2).
The first shift occurred in 1974, stemming in part from the 1973 recession. Whereas productivity had grown at an average rate of 2.6 percent per year over the previous 25 years, it grew by an average of only about 1.5 percent per year through the mid-1990s. Partly because most forecasters in the 1970s expected that the productivity trend of the previous decades would prevail, their forecasts of real output growth in the latter half of the 1970s turned out to be too optimistic (see Figure 3).
The second shift occurred in 1997, when growth in labor productivity in the nonfarm business sector accelerated, averaging more than 3 percent per year for nearly a decade. For the first several years of that period, forecasters underestimated the trend of productivity growth, which partly explains why their projections of the economy’s growth rate were too low and their projections of inflation in the output price index were too high.19 The acceleration in labor productivity stemmed from a pickup in technological progress (especially in information technology) and an increase in the amount of capital per worker as firms invested heavily in new technology.
The third shift occurred in 2006, when the average growth of labor productivity slowed to 1.4 percent per year through 2020 for reasons that are not fully understood. The slowdown partly reflects cyclical factors related to the severe recession that occurred from 2007 to 2009 and the ensuing weak recovery. In addition, the growth of the labor force decelerated, which in turn slowed the growth of investment and of capital services. That slowdown in investment may have also reduced the rate at which businesses could introduce new technologies into the production process. Some research suggests that other long-term structural problems might be impeding the rate at which new technologies diffuse through industries.20
In general, shifts in labor productivity are difficult to forecast. Such shifts are typically the result of changes in capital accumulation, changes in educational attainment, and technological innovation—factors that are difficult to forecast and that are only easily identified several years after the fact. Consequently, if CBO and other forecasters make incorrect inferences about the percentage of the labor force receiving higher education degrees, for example, that error affects their projections of productivity growth and, by extension, real output growth.
Changes in Crude Oil Prices
Crude oil is an important energy source in the United States, and petroleum accounts for more than one-third of total energy consumption.21 As a result, crude oil prices are a major component of consumer price inflation. Relative to overall prices, crude oil prices are volatile, and they fluctuate widely in response to economic and geopolitical developments (see Figure 4). Some of the largest errors in forecasting CPI inflation can be attributed to forecasters’ inability to predict major changes in crude oil prices. For example, the rise in the price of oil in the late 1970s and early 1980s probably contributed to the sizable underprediction of inflation by CBO and the Administration; those forecasters underpredicted the rise in inflation over that time period when they completed their forecasts at the end of the 1970s (see Figure 5).22
Large changes in crude oil prices reflect producers’ and consumers’ limited capacity to quickly adjust supply and demand in response to changing market conditions.23 Fluctuations in oil prices are often difficult to forecast because markets for petroleum products can be sensitive to developments that forecasters cannot reasonably be expected to predict. For example, during the 1973–1981 period, oil prices spiked in response to the oil embargo imposed by the Organization of Arab Petroleum Exporting Countries (1973 to 1974), the Iranian Revolution (1979), and the start of the Iran–Iraq War (1980). Those developments affected forecasters’ ability to correctly forecast CPI inflation.
Political factors remain a source of uncertainty, but they appear to have become less important in explaining volatility. Recently, oil prices rose steeply in the lead-up to the 2007–2009 recession and fell sharply in 2015 and 2016 because of shifts in global supply and demand as well as technological changes, such as horizontal drilling and hydraulic fracturing.
The Persistent Decline in Interest Rates
Interest rates have trended downward since the early 1980s (see Figure 6). Although that decline is partly attributable to a lower average rate of inflation over the past two decades, the effect persists even after taking into account the change in prices. Recent research has identified several factors that may have contributed to the decline in real interest rates: the aging of the population, increased income inequality, a trend toward slower output growth, and increased saving among emerging market economies.24
Over the past two decades, forecasters have underestimated the effect of those factors on interest rates, and they did not anticipate the extent or persistence of the eventual decline. Thus, all forecasters have tended to make sizable overpredictions of both short- and long-term interest rates since the early 2000s.
The Decline in the Labor Share
Labor share refers to the total compensation paid to workers as a percentage of GDP. The labor share is composed of several components, the largest of which is total wages and salaries paid to employees, which accounts for about 80 percent of all labor income. Therefore, misestimates of the labor share typically result in misestimates of wages and salaries.
In the early 2000s, the labor share experienced a structural decline. That decline has partly reversed over the past decade, and the reasons for it are only partially understood (see Figure 7). One theory is that globalization may have increased incentives for businesses to move their production of labor-intensive goods abroad.25 Another is that technological innovation may have increased the returns on capital more than it has increased the returns on labor. Either way, forecasters at the start of the millennium projected the labor share to stabilize or return to its historical average. That expectation resulted in upwardly biased projections of wage and salary growth (see Figure 8).
Transient factors can also cause movements in the labor share and lead to forecast misestimates. One such factor is the downward shift in the number of employees enrolling in employment-based health insurance plans that occurred in the late 1980s and early 1990s.26 The decline in enrollment was largely the result of rising employee premiums and stagnant employer contributions, which reflected the rising cost of medical care during that period. Employees who declined employment-based health insurance implicitly reduced their total compensation, because employers’ payments toward health insurance premiums are counted in total employee compensation. That development led forecasters to overestimate labor’s share of income and contributed to overestimates of the growth in wages and salaries between 1989 and 1994.
Many of the data series analyzed in this report are periodically revised in response to new data, methods, and definitions. For example, the Bureau of Economic Analysis (BEA) periodically issues comprehensive revisions of its national income and product accounts, which contain historical data on real output growth and growth of wages and salaries. Those data revisions affect the computation of forecast errors and, by extension, measures of forecast quality.
One way that data revisions affect the computation of forecast errors is by creating a wedge between the currently available data and the data that were available when the forecast was completed. For example, the root mean square error of CBO’s two-year forecasts of real output growth is 1.3 percentage points if calculated from the most recently available data (see Table 1) but is 1.2 percentage points if calculated from the real-time data available immediately after the conclusion of the two-year horizon. Although some research suggests that real-time data may be more appropriate for that kind of analysis, CBO uses the most recently available data.27 That decision simplifies the analysis and helps account for definitional changes that affect the interpretation of certain data series over time.28
In addition to affecting the analysis of forecast errors after the fact, data revisions can affect a forecaster’s projections in real time. For example, BEA made several downward revisions to estimates of real GDP growth during the 2007–2009 recession (see Figure 9). When CBO prepared its baseline forecast in January 2009, real GDP had reportedly fallen at an annualized rate of 0.5 percent during the third quarter of 2008; however, revised data now show a 2.1 percent drop during that quarter. Similarly, the latest revisions show that the average annual growth of real GDP was nearly one-quarter of a percentage point lower during the recession than BEA initially reported in January 2010. Had CBO and other forecasters known the true state of the economy at the time of their forecast, their projections probably would have been different.
Quantifying the Uncertainty of CBO’s Economic Forecasts
To quantify the uncertainty of its economic projections, CBO most frequently relies on the root mean square error of its historical forecasts. For each variable, CBO computes root mean square errors at the one-, three-, and four-year time horizons, in addition to the two- and five-year time horizons presented in this report.29 Those root mean square errors are applied symmetrically to the agency’s baseline projections to produce the range of the middle two-thirds of outcomes.
Such analysis has informed the uncertainty ranges surrounding CBO’s projections of real GDP in recent editions of The Budget and Economic Outlook. Publishing those ranges helps illustrate the breadth of possible outcomes and the degree of certainty underlying CBO’s economic forecasts. For example, CBO estimated in July 2021 that there is approximately a two-thirds chance that the average annual growth rate of real GDP will be between 1.9 percent and 4.3 percent over the next five years (see Figure 10). Its central estimate is 3.1 percent.
Sometimes CBO uses the two-thirds spread instead of the root mean square error to quantify the uncertainty of its current economic projections. When the ratio of the average error to the root mean square error is greater than or equal to 0.3, CBO estimates that about two-thirds of actual values will fall within a range of plus or minus one-half of the two-thirds spread of forecasted values. For example, CBO’s two-year forecasts of 10-year Treasury notes have an average error of 0.5 percentage points and a root mean square error of 0.8 percentage points (see Table 1). Because the ratio of those numbers is greater than 0.3, CBO uses the two-thirds spread to estimate that there is a two-thirds chance that the average interest rate on 10-year Treasury notes will fall within a range of roughly 1.2 percentage points above or below the agency’s central estimate of 2.1 percent.
Forecast errors for economic variables have implications for the budget. For example, according to CBO’s rules of thumb, if interest rates were 0.1 percentage point higher than projected for five consecutive years (and all else remained constant), the budget deficit over that period would be an estimated $75 billion higher than the baseline budget projection. Similarly, if inflation was 0.1 percentage point higher than projected for five consecutive years (and all else remained constant), the budget deficit would be an estimated $66 billion higher than the baseline budget projection.30
1. For the purposes of this report, CBO examined the following economic variables: real gross domestic product (that is, GDP adjusted to remove the effects of inflation) and nominal GDP (GDP valued at prices in the current year), the unemployment rate, inflation as measured by the consumer price index for urban consumers (CPI-U), 3-month Treasury bills, 10-year Treasury notes, and wages and salaries.
2. Forecast errors throughout this report were calculated as projected values minus actual values; thus, a positive error is an overestimate.
3. See Allan Timmermann, “Forecast Combinations,” in Graham Elliott, Clive W. J. Granger, and Allan Timmermann, eds., Handbook of Economic Forecasting, vol. 1 (North Holland, 2006), pp. 135–196, https://doi.org/10.1016/S1574-0706(05)01004-9; Andy Bauer and others, “Forecast Evaluation With Cross-Sectional Data: The Blue Chip Surveys,” Economic Review, vol. 88, no. 2 (Federal Reserve Bank of Atlanta, 2003), pp. 17–31, https://tinyurl.com/y3qmyjzg (PDF, 219 KB); Henry Townsend, “A Comparison of Several Consensus Forecasts,” Business Economics, vol. 31, no. 1 (January 1996), pp. 53–55, www.jstor.org/stable/23487509; and Robert T. Clemen, “Combining Forecasts: A Review and Annotated Bibliography,” International Journal of Forecasting, vol. 5, no. 4 (1989), pp. 559–583, https://doi.org/10.1016/0169-2070(89)90012-5.
4. CBO cannot compare forecasts over longer time horizons because some data are not available. Although the agency has produced 11-year economic forecasts since 1992, it produced only 6-year economic forecasts before then. In addition, the Blue Chip consensus currently produces long-run economic forecasts spanning just 7 years.
5. The time periods used in the comparison of forecast quality vary by the availability of comparable data. See Table 1 for the sample periods used for each comparison.
6. For most years examined here, the inflation forecasts were for the CPI-U, which measures inflation in the prices of goods and services consumed by all urban consumers. Some forecasts, however, were for the CPI-W, which measures inflation in the prices of goods and services consumed by urban wage earners and clerical workers. CBO forecast the CPI-W from 1976 to 1978 and again from 1986 to 1989; the Administration forecast the CPI-W through 1991. For the purpose of this evaluation, the distinction between the two measures was most consequential in 1984, when inflation in the CPI-U and CPI-W diverged by 0.9 percentage points. For the output price index, CBO used the gross national product price index for forecasts made before 1992 and the GDP price index for forecasts made between 1992 and 2019.
7. Starting in 2018, tax brackets were indexed to the chained CPI.
8. See Congressional Budget Office, An Evaluation of CBO’s Past Revenue Projections (August 2020), www.cbo.gov/publication/56499, An Evaluation of CBO’s Past Deficit and Debt Projections (September 2019), www.cbo.gov/publication/55234, and An Evaluation of CBO’s Past Outlay Projections (November 2017), www.cbo.gov/publication/53328.
9. In evaluating its revenue projections, CBO calculated errors as the percentage difference (rather than the simple difference used in this report) between the projected and actual values because revenues are expressed as dollar amounts. If the errors in revenue projections were measured as simple differences in dollar amounts, they would be difficult to compare over time. (A $5 billion error in 1992, for example, would be significantly larger than a $5 billion error in 2014.) The simple difference is more appropriate in this report because it evaluates errors in forecasts of economic indicators that are expressed as rates or percentages—growth rates, interest rates, and changes in wages and salaries as a percentage of output. Forecast errors in this report are thus percentage-point differences between forecast and actual values.
10. Several studies that examined how well CBO’s economic forecasts incorporate relevant information—a characteristic referred to as forecast efficiency—found that the agency’s forecasts are relatively efficient. See, for example, Robert Krol, “Forecast Bias of Government Agencies,” Cato Journal, vol. 34, no. 1 (Winter 2014), pp. 99–112, https://tinyurl.com/y7cmapw3 (PDF, 88 KB); Stephen M. Miller, “Forecasting Federal Budget Deficits: How Reliable Are US Congressional Budget Office Projections?” Applied Economics, vol. 23, no. 12 (December 1991), pp. 1789–1799, http://doi.org/10.1080/00036849100000168; and Michael T. Belongia, “Are Economic Forecasts by Government Agencies Biased? Accurate?” Review, vol. 70, no. 6 (Federal Reserve Bank of St. Louis, November/December 1988), pp. 15–23, http://tinyurl.com/ychze7ah. Although statistical tests can identify sources of inefficiency in a forecast after the fact, they generally do not indicate how such information could be used to improve forecasts when they are being made.
11. Several analysts outside of CBO have used more elaborate techniques to test for bias in the agency’s forecasts. One such alternative approach to testing a forecast for bias is based on linear regression analysis of actual values compared with forecast values. For details of that method, see Jacob A. Mincer and Victor Zarnowitz, “The Evaluation of Economic Forecasts,” in Jacob A. Mincer, ed., Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance (National Bureau of Economic Research, 1969), pp. 3–46, www.nber.org/chapters/c1214.
Studies that have used that method to evaluate CBO’s and the Administration’s short-term forecasts have not found statistically significant evidence of bias over short forecast horizons. See, for example, Robert Krol, “Forecast Bias of Government Agencies,” Cato Journal, vol. 34, no. 1 (Winter 2014), pp. 99–112, https://tinyurl.com/y7cmapw3 (PDF, 88 KB); Graham Elliott and Allan Timmermann, “Economic Forecasting,” Journal of Economic Literature, vol. 46, no. 1 (March 2008), pp. 3–56, https://doi.org/10.1257/jel.46.1.3; George A. Krause and James W. Douglas, “Institutional Design Versus Reputational Effects on Bureaucratic Performance: Evidence From U.S. Government Macroeconomic and Fiscal Projections,” Journal of Public Administration Research and Theory, vol. 15, no. 2 (April 2005), pp. 281–306, https://doi.org/10.1093/jopart/mui038; and Michael T. Belongia, “Are Economic Forecasts by Government Agencies Biased? Accurate?” Review, vol. 70, no. 6 (Federal Reserve Bank of St. Louis, November/December 1988), pp. 15–23, http://tinyurl.com/ychze7ah.
For more elaborate studies of bias that included CBO’s forecasts among a sizable sample, see J. Kevin Corder, “Managing Uncertainty: The Bias and Efficiency of Federal Macroeconomic Forecasts,” Journal of Public Administration Research and Theory, vol. 15, no. 1 (January 2005), pp. 55–70, https://doi.org/10.1093/jopart/mui003; and David Laster, Paul Bennett, and In Sun Geoum, “Rational Bias in Macroeconomic Forecasts,” Quarterly Journal of Economics, vol. 114, no. 1 (February 1999), pp. 293–318, https://doi.org/10.1162/003355399555918.
12. The mean square forecast error is equal to the square of the bias in the errors plus the variance of the errors. The variance measures the average squared difference between the errors and the average error.
13. For details about some exceptions to that rule, see Congressional Budget Office, What Is a Current-Law Economic Baseline? (June 2005), www.cbo.gov/publication/16558.
14. Different assumptions about monetary policy can also make it difficult to compare CBO’s forecasts with other forecasts. CBO’s forecasts incorporate the assumption that monetary policy will reflect the economic conditions that the agency expects to prevail under the fiscal policy specified in current law.
15. This description of relative forecast quality is strictly qualitative. CBO also conducted a series of statistical tests to assess the differences in root mean square errors among forecasters. Compared with the Administration’s forecasts, CBO’s forecasts had statistically smaller root mean square errors for 5 of 20 variables examined in this report. The Administration’s forecasts were never more accurate than CBO’s forecasts by statistically significant amounts. Compared with the Blue Chip consensus forecasts, CBO’s forecasts had statistically smaller root mean square errors for 2 of 20 variables examined in this report. The Blue Chip consensus forecasts were more accurate than CBO’s forecasts for one of those variables by a statistically significant amount.
16. CBO also examined its forecasts of real growth in wages and salaries, which inform its analysis of the distribution of taxable income. Examining errors in real wage and salary growth helps isolate errors in forecasting nominal wage and salary growth from errors in forecasting growth in consumer price inflation. Although those forecasts exhibited slightly less statistical bias than their nominal counterparts, the accuracy of the forecasts was similar. For that reason, CBO elected not to include those results in its primary summary tables.
17. For more information about what caused recent recoveries to be slow, see Congressional Budget Office, The Slow Recovery of the Labor Market (February 2014), www.cbo.gov/publication/45011.
18. Another change that caught most forecasters by surprise was the reduction in the volatility of GDP growth beginning in the mid-1980s. In particular, from the mid-1980s until the 2007–2009 recession, quarterly movements in real GDP growth were more muted than in previous decades. See, for example, Jordi Galí and Luca Gambetti, “On the Sources of the Great Moderation,” American Economic Journal: Macroeconomics, vol. 1, no. 1 (January 2009), pp. 26–57, https://tinyurl.com/y65mv7fj.
19. See Spencer Krane, “An Evaluation of Real GDP Forecasts: 1996–2001,” Economic Perspectives, vol. 27, no. 1 (Federal Reserve Bank of Chicago, January 2003), pp. 2–21, http://tinyurl.com/y8wadllm; and Scott Schuh, “An Evaluation of Recent Macroeconomic Forecast Errors,” New England Economic Review (Federal Reserve Bank of Boston, January/February 2001), pp. 35–56, http://tinyurl.com/ych7zk8d.
20. See Ryan A. Decker and others, Declining Business Dynamism: Implications for Productivity? Hutchins Center Working Paper 23 (Brookings Institution, September 2016), http://tinyurl.com/lv9cs9h; and Dan Andrews, Chiara Criscuolo, and Peter N. Gal, The Global Productivity Slowdown, Technology Divergence, and Public Policy: A Firm Level Perspective, Hutchins Center Working Paper 24 (Brookings Institution, September 2016), http://tinyurl.com/km6942w.
21. See Energy Information Administration, Monthly Energy Review (November 2021), Table 1.3, https://go.usa.gov/xyVJg (PDF, 2.48 MB).
22. The Blue Chip consensus forecast of CPI inflation was not available until 1981.
23. In the near term, consumers are constrained by the existing energy efficiency of their homes, places of work, and modes of transportation; producers are constrained by their equipment, technology, and the availability and accessibility of natural resources. For further discussion, see Congressional Budget Office, Energy Security in the United States (May 2012), www.cbo.gov/publication/43012.
24. See Edward N. Gamber, The Historical Decline in Real Interest Rates and Its Implications for CBO’s Projections, Working Paper 2020-09 (Congressional Budget Office, December 2020), www.cbo.gov/publication/56891; Lukasz Rachel and Thomas D. Smith, “Are Low Real Interest Rates Here to Stay?” International Journal of Central Banking (September 2017), pp. 1–42, www.ijcb.org/journal/ijcb17q3a1.htm; and Council of Economic Advisers, Long-Term Interest Rates: A Survey (July 2015), https://go.usa.gov/xyVS3 (PDF, 1.4 MB).
25. See, for example, Michael W. L. Elsby, Bart Hobijn, and Aysegul Sahin, “The Decline of the U.S. Labor Share,” Brookings Papers on Economic Activity, vol. 44, no. 2 (Fall 2013), pp. 1–63, https://brook.gs/2VCVbyx.
26. For information about changes in employers’ contributions to health insurance during the late 1990s, see David M. Cutler, Employee Costs and the Decline in Health Insurance Coverage, Working Paper 9036 (National Bureau of Economic Research, July 2002), www.nber.org/papers/w9036.
27. See, for example, Tom Stark and Dean Croushore, “Forecasting With a Real-Time Data Set for Macroeconomists,” Journal of Macroeconomics, vol. 24, no. 4 (December 2002), pp. 507–531, https://doi.org/10.1016/S0164-0704(02)00062-9.
28. For example, business and government spending on computer software was once treated as spending for an intermediate good—that is, an input into the production process—and thus did not count as a component of GDP, which measures only final spending. But in 1999, BEA reclassified such spending as investment, which is a category of final spending. That same year, BEA adopted new methods for calculating the price indexes for various categories of consumption. Largely as a result of those changes, BEA increased its estimates of growth in real GDP for the 1980s and 1990s. In particular, BEA’s estimates of average annual growth in real GDP from 1992 to 1998 rose by 0.4 percentage points, and inflation in the GDP price index for those years was revised downward by 0.1 percentage point per year.
29. The root mean square errors for CBO’s forecasts of real output growth at the one-, three-, and four-year horizons are 1.1, 1.2, and 1.3 percentage points, respectively.
30. See Congressional Budget Office, “Workbook for How Changes in Economic Conditions Might Affect the Federal Budget, June 2021” (June 2021), www.cbo.gov/publication/57191.
AppendixThe Data CBO Uses to Evaluate Its Economic Forecasting Record
This appendix provides an overview of the forecast and historical data that the Congressional Budget Office uses to evaluate its forecasting record. In the report, CBO analyzes its errors in forecasting output growth, the unemployment rate, inflation, interest rates, and changes in wages and salaries. Forecast values are primarily compiled from CBO’s annual report, The Budget and Economic Outlook, typically published in the first quarter of each calendar year; the Administration’s annual budget documents; and Blue Chip Economic Indicators reports from the first quarter of each calendar year. Actual values for each variable are based on the latest available data from various agencies, including the Bureau of Economic Analysis (BEA) and the Bureau of Labor Statistics (BLS).
Forecasts Included in This Evaluation
This report evaluates forecasts published between 1976—the first year that CBO made economic projections—and 2019. From 1976 to 1984, however, those reports did not regularly include CBO’s forecasts of wages and salaries, so this analysis incorporates some unpublished forecasts of those amounts.
For comparison, this report also evaluates the Administration’s forecasts made between 1976 and 2019, all but two of which were taken from the Office of Management and Budget’s annual budget documents. In Presidential transition years, forecasts are taken from the budget documents of incoming Presidential administrations, with two exceptions: First, the Reagan Administration’s 1981 economic projections were based on revisions of the Carter Administration’s final budget and were released separately. Second, the Clinton Administration chose not to make its own economic projections before releasing its 1993 budget, adopting instead CBO’s economic projections as the basis of its budget. As a result, errors in the 1993 forecast are the same for both forecasters.
The Blue Chip Economic Indicators was published beginning in 1976, but because of data constraints this report analyzes only Blue Chip forecasts dating back to 1979. Although the Blue Chip consensus is published monthly, only the March and October forecasts extend beyond two years. As a result, this analysis uses the January publication to obtain two-year forecasts of each economic variable and the March publication to obtain five-year forecasts. One exception is the five-year forecasts published in 1980, which were released in May. Because the Blue Chip consensus forecasts are only available starting in 1979, this report’s comparisons of forecast quality do not include CBO’s forecasts produced from 1976 through 1978 (see Table 1).
The Federal Reserve’s forecasts are taken from two sources. The first is the Federal Reserve’s Greenbook, a publication prepared by the staff of its Board of Governors. The Greenbook contains two-year projections for a wide range of economic variables, but it is not released to the public until several years after the forecast is made. As a result, this analysis incorporates only Greenbook forecasts from 1979 to 2015. The second source is the economic projections prepared by individual members of the Federal Reserve’s Federal Open Market Committee (FOMC). The ranges and central tendencies of those forecasts encompass a smaller set of variables, but they have been published regularly since 2007. In this analysis, CBO uses the median of the FOMC forecasts in 2016 and 2017. All of the Federal Reserve forecasts used in this analysis were issued in January or February of the initial forecast year or in December of the preceding year.
Growth of real output (that is, output adjusted to remove the effects of inflation) and nominal output is computed from calendar year averages of the most recently available quarterly data published by BEA. To calculate the average annual growth rate over two- and five-year time horizons, CBO computes the geometric average of each series over the relevant sample period.1
One major methodological shift occurred in 1991, when BEA changed its featured measure of output from gross national product (GNP) to gross domestic product (GDP). In response, most economic forecasters switched to forecasting GDP starting in 1992. As a result, for this analysis CBO computes forecast errors for GNP growth before 1992 and forecast errors for GDP growth between 1992 and 2019.
The unemployment rate is computed from calendar year averages of monthly data published by BLS. To calculate the average unemployment rate over two- and five-year time horizons, CBO computes the arithmetic average of the annual unemployment rate series over the relevant sample period.
A major methodological shift occurred in 1994, when BLS updated and modernized its Current Population Survey, which is used to measure the unemployment rate. Because the methodology was changed in February of that year, some forecasters included projections of unemployment on the new basis as well as the old basis. For this report, CBO computes forecast errors relative to the old basis for forecasts between 1976 and 1994 and relative to the new basis for forecasts made from 1995 to 2019.
CBO computes inflation in the consumer price index (CPI) from calendar year averages of monthly data published by BLS. To calculate the average annual growth rate over two- and five-year time horizons, CBO computes the geometric average of each series over the relevant sample period. In January 1978, BLS began publishing the CPI-U (the price index for all urban consumers) in addition to the CPI-W (the price index for urban wage earners and clerical workers). Although annual fluctuations in the two price indexes are typically very similar, they occasionally diverge. To account for those divergences, this report compares CPI-U forecasts with CPI-U actuals and CPI-W forecasts with CPI-W actuals.
The introduction of the CPI-U also resulted in different approaches to forecasting consumer price inflation. The Administration published forecasts of the CPI-W from 1976 to 1991 because that measure was used to index most of the federal government’s spending for entitlement programs, but it switched to forecasting the CPI-U starting in 1992. By contrast, CBO based all but four of its inflation forecasts after 1978 on the CPI-U. The exceptions are CBO’s forecasts from 1986 to 1989, which were based on the CPI-W. The Blue Chip consensus and the Federal Reserve have always forecasted the CPI-U.
In this report, CBO also analyzes errors in the inflation differential—the difference between CPI inflation and inflation in the output price index. To compute growth in the output price index, CBO uses calendar year averages of the most recently available quarterly data published by BEA. Similar to its computations of real output growth, the agency’s computations of the inflation differential use the GNP price index for forecasts made before 1992 and the GDP price index for forecasts made between 1992 and 2019. CBO subtracts growth in the output price index from growth in the CPI to compute the inflation differential.
CBO computes the interest rate on 3-month Treasury bills from calendar year averages of monthly data published by the Federal Reserve. To calculate the average interest rate over two- and five-year time horizons, CBO computes the geometric average of the annual interest rate series over the relevant sample period. The agency’s analysis of forecasts of the interest rate on 3-month Treasury bills focuses on two main measures: the new-issue rate and the secondary-market rate.
The new-issue rate reflects the interest that would be earned by an investor who purchased a bill at auction and held it to maturity. The secondary-market rate corresponds to the price of 3-month bills traded outside of Treasury auctions. Such transactions occur continually in markets that have many more traders than there are bidders in Treasury auctions. The difference between the calendar year averages of the two series is small, which makes them easily comparable across time horizons.
The Administration published forecasts of the new-issue rate until 2001, when it switched to forecasting the secondary-market rate. Unlike the Administration, CBO has always forecasted the secondary-market rate. The Blue Chip consensus, by contrast, has alternated between the two rates and, in 1981, even projected a third—the six-month commercial paper rate. The Blue Chip consensus forecast the new-issue rate from 1982 to 1985, the secondary-market rate from 1986 to 1991, and the new-issue rate again from 1992 to 1997. Since March 1997, the Blue Chip consensus has forecast the secondary-market rate.
As part of this analysis, CBO also computes errors in forecasting the real interest rate on 3-month Treasury bills. To do that, the agency deflates each forecaster’s annualized projections of nominal interest rates by that forecaster’s annualized projections of CPI inflation. Although the forecasters used different measures of inflation and short-term interest rates, the resulting real interest rate forecasts are comparable across time horizons.
CBO’s analysis of long-term interest rates focuses on two measures: the 10-year Treasury note rate, published by the Federal Reserve, and Moody’s Aaa corporate bond rate. For each series, CBO uses calendar year averages of the latest available monthly data. To calculate the average interest rate over two- and five-year time horizons, CBO computes the geometric average of the annual interest rate series over the relevant sample period.
Both the 10-year Treasury note and the Aaa corporate bond are widely considered low-risk financial securities. Although the return on 10-year Treasury notes is guaranteed by the federal government, the return on Aaa corporate bonds depends on the solvency of the corporations deemed most likely to pay back their financial obligations. Because investors perceive the government to be slightly more creditworthy than the safest corporations, the 10-year Treasury note rate is consistently slightly lower than the Aaa corporate bond rate. Annual fluctuations in the two series are highly correlated, however, making them roughly comparable across time horizons.
CBO projected the Aaa corporate bond rate in 1984 and 1985 but since then has projected the 10-year Treasury note rate. The Administration has always published forecasts of the 10-year Treasury note rate. The Blue Chip consensus forecast the Aaa corporate bond rate until 1996, when it switched to the 10-year Treasury note rate. Because of data constraints, it is not possible to compare forecasts made before 1984.
Wages and Salaries
The growth of wages and salaries is computed from calendar year averages of the most recently available quarterly data published by BEA. To calculate the average annual growth rate over two- and five-year time horizons, CBO computes the geometric average of each series over the relevant sample period. Because the Blue Chip Economic Indicators report does not publish a consensus forecast of the growth in wages and salaries, it is omitted from the forecast analysis of that variable.
In this report, CBO also analyzes errors in projections of wages and salaries as a share of output. For the wages and salaries component, the agency uses calendar year averages of the most recently available quarterly data published by BEA. For the output component, the agency uses nominal GNP for forecasts made between 1976 and 1991 and nominal GDP for forecasts made between 1992 and 2019. The agency then divides the amount of wages and salaries by the amount of nominal output to compute wages and salaries as a share of output. CBO uses the arithmetic average to compute the average share of wages and salaries over two- and five-year time horizons.
1. A geometric average is calculated by first multiplying the growth rates for each year and then taking the nth root of that product. The geometric average is the appropriate measure for averaging growth rates.
About This Document
The Congressional Budget Office regularly evaluates the accuracy of its economic forecasts by comparing them with the economy’s actual performance and with others’ forecasts. Such evaluations help guide CBO’s efforts to improve the quality of its forecasts and, as background information, are also intended to assist Members of Congress in using the agency’s estimates. In keeping with CBO’s mandate to provide objective, impartial analysis, the report makes no recommendations.
Erin Deal and Edward Gamber prepared the report with guidance from Robert Arnold, John Kitchen, and Jeffrey Werling. Aaron Feinstein, Kerk Phillips, and James Williamson offered comments. Grace Berry fact-checked the report.
Jeffrey Kling and Robert Sunshine reviewed the report. Rebecca Lanning edited it, and R. L. Rebach created the graphics and prepared the text for publication. The report is available at www.cbo.gov/publication/57579.
CBO seeks feedback to make its work as useful as possible. Please send comments to email@example.com.
Phillip L. Swagel
The Congressional Budget Office has corrected this report since its original publication. Both the PDF and online versions were corrected, but for ease of reference, this list indicates the locations of the corrections in the PDF.
The following change was made on December 15, 2021:
Page 22, footnote 30: “How Changes in Economic Conditions Might Affect the Federal Budget (June 2018)” was changed to “‘Workbook for How Changes in Economic Conditions Might Affect the Federal Budget, June 2021’ (June 2021).”