A Look at the Many Ways to Evaluate Tax Incentives Part II

In a two-part essay written for The Pew Charitable Trusts, Jim Landers, associate professor of clinical public affairs and Enarson fellow, John Glenn College of Public Affairs, The Ohio State University, examines the key components of effective incentive evaluations, including examining design and administration, reviewing usage data to analyze program administration and operation, and using surveys to collect data in support of conducting a robust evaluation.

These pieces were originally published in a newsletter distributed to tax incentive evaluators and scholars in August and December of 2019.

Evaluation perspectives

How Do We Evaluate Tax Incentive Programs? There Are Multiple Approaches.

Jim Landers
Associate Professor of Clinical Public Affairs, Enarson Fellow
John Glenn College of Public Affairs
The Ohio State University

How should we evaluate tax incentives and what approaches should we use? I started my last column offering these and other questions that my colleagues and I considered when tax incentive evaluation was initiated in Indiana in 2014. Incentive evaluators throughout the states surely have posed similar questions. It’s also a good bet that when posing these questions, evaluators are considering both incentives’ effectiveness and economic impacts. This column focuses on the simple and complex approaches that evaluators are using to get at these questions of incentive effectiveness and economic impacts.

Using Econometric Modeling to Estimate the Impact of Tax Incentives on Economic Activity. Quasi-experimental designs employing econometric techniques such as difference-in-differences regression models, matched pair regression models, and panel regression models can generate rigorous and informative estimates of the effectiveness or impact of a tax incentive on employment, investment, or other economic activity of the incentive recipients. The upside of these statistical approaches is substantial.  They control for the impact of factors other than the tax incentive that affect the economic performance of recipients such as general economic conditions, other tax incentives, or factors resulting in selection bias. By eliminating the effects of these confounding factors, these statistical techniques estimate the causal impact of the tax incentive alone on the economic performance of the recipient.

However, these statistical techniques are not without their challenges. They require large datasets that are time intensive to assemble. They also require a significant level of technical expertise to construct and troubleshoot statistical models and properly interpret statistical outputs. As a result, econometric modeling may only be a worthwhile undertaking for major tax incentive programs.

Excellent examples of econometric work are available in incentive evaluations from:

• Florida, where evaluators use a matched pair regression approach to evaluate grant programs for worker training.
• Iowa, where evaluators use a difference-in-differences regression approach to evaluate a jobs tax credit.
• Indiana, where evaluators use a matched pair regression approach to evaluate the TIF program and a panel regression approach to evaluate a venture capital investment tax credit.

Employing Simulations to Assess Regional Economic Impacts. Simulating the regional economic impact of a tax incentive is different from the causal analysis discussed above. These simulations estimate the impact of new spending, investment, or employment attributable to a tax incentive on regional economic activity, employment, personal income, or taxes.

Typically, incentive evaluators use a commercial input-output model like REMI or IMPLAN to generate their regional economic impact estimates. The premise of these models is that there are linkages between sectors of an economy and, thus, outputs from one industry sector become inputs in other industry sectors. Consequently, a tax incentive for the purchase of heating, air conditioning, and insulation products will flow from the sectors producing these products to other industries that supply inputs to production. The estimated impacts include the direct impact of the incentive (increasing production by heating, air conditioning, and insulation companies), the indirect impact (from the production companies increasing purchases of inputs from suppliers), and the induced impacts (from increased purchasing by new employees of both the production companies and the input suppliers).

Much like econometric modeling, using REMI or IMPLAN to simulate the regional economic impact of tax incentives comes with steep requirements in terms of expertise, technology, and data. To begin with, these models are expensive to acquire. There is also a significant learning curve to understanding how the models work, the data comprising the models, and the assumptions about the relationships or linkages between different industry sectors. These models also do not estimate the causal relationship between a tax incentive and the economic activity of recipients. Rather, evaluators must estimate or assume the quantitative effect of the tax incentive on spending, investment, or employment by incentive recipients for the purposes of regional economic impact simulations.

Washington’s evaluations of tax incentives for spending on data center equipment and motion picture production expenses are examples of a good way to use a regional economic model. They use the REMI model and make assumptions about the percentage (ranging from zero to 100 percent) of spending “caused” by these incentives to construct scenarios of the net change in employment, personal income, overall economic activity, and state taxes resulting from the incentive. They must also estimate the break-even point or spending percentage required for the net change to be zero. While the scenarios do not indicate whether an incentive leads to the incentivized economic activity, the scenarios and break-even percentages can be very compelling for judging the overall benefit and usefulness of the incentive.

Using Simple Descriptive Measures and Hypotheticals as Indicators of Effectiveness and Impact. Estimates of the cause-and-effect relationship between a tax incentive and the economic response by recipients are often unachievable because of insufficient data. Still, evaluators can construct simple descriptive measures (e.g., the percentage of a recipient’s tax liability, investment, or wage cost) or hypothetical scenarios (e.g., the impact scenarios used by Washington evaluators) that are reasonable indicators of the potential effectiveness or net impact of a tax incentive. While these are not true cause-and-effect estimates, they provide useful information for policymakers to judge the effectiveness or benefit of a tax incentive.

Indiana’s evaluations of state tax incentives for home insulation products and solar-powered roof vent/fan installation, business property tax abatements, and property tax abatements for dwellings in distressed areas are instructive examples of this method. Various pieces of information, including incentive usage and taxpayer data, are used to construct measures of:

• the percentage reduction in tax liability of incentive recipients;
• the percentage reduction in cost of the incentivized economic activity (e.g., cost of insulation products, cost to install roof vent or fan, or wage cost of a business receiving a tax abatement); or
• the increase in ROI of residential property rehab completed by tax abatement recipients. 

Conclusion. In most cases, the central role of incentive evaluation is to get at questions about incentive effectiveness or economic impacts. Employing econometric techniques or economic impact models (e.g., REMI) for these purposes is, among other challenges, complex and time-consuming. Still, several evaluations demonstrate the benefits of these approaches and provide extensive guides to employing them. In contrast, a number of other evaluations employ simple techniques that can also provide important information about incentive effectiveness or economic impacts.