Tips for Setting Tax Incentive Metrics and Benchmarks

On Nov. 13, 2013, The Pew Charitable Trusts submitted a memo to the Vermont Legislative Joint Fiscal Office that outlined best practices for economic development tax incentive metrics and benchmarks, including that they should:

1. Reflect the goals of incentives.
2. Be coordinated to allow comparisons between programs with similar goals.
3. Place the benefits of tax incentives in the context of their costs.
4. Consider the effects on the state’s economy as a whole.
5. Define how each metric will be calculated.
6. Reflect an intention to isolate the impact of the incentive.

Pew experts also discussed how states can determine success or failure through a comparative approach that examines the results of incentives to determine whether they are improving over time and how they compare with those of other economic development strategies in the state.  

To: Ms. Sara Teachout, Senior Fiscal Analyst
From: Josh Goodman and Janelle Mummey
Date: November 13, 2013
Subject: Setting metrics and benchmarks for tax incentives

Concepts to help set metrics for economic development tax incentives

The following six concepts have helped other states develop meaningful metrics for evaluating tax incentives and may be useful to you in Vermont.

1: Metrics should reflect the goals of incentives

There is no one right metric for states to use when evaluating their economic development tax incentives. Instead, the right metrics are ones that reflect the goals of the program and help answer whether those goals have been met. If an incentive is supposed to produce high-quality jobs, for example, you might measure the wages of the jobs created or whether they offer health insurance—but it wouldn’t be adequate to only count the number of jobs, because that doesn’t say anything about quality. One implication of this point is that you can’t decide on a metric until you have goals. They need to be developed in a progression—from goals to metrics.

2: Metrics should be coordinated to allow for comparisons between programs with similar goals

As we discuss below (see “How states can determine success or failure”), one way states can draw valuable conclusions about their tax incentives is to compare different programs to one another. If two incentives share a goal, it may make sense for them to share a metric as well, to make those types of comparisons easier.

3: Metrics should place the benefits of tax incentives in the context of their costs

The decision to use tax incentives involves a choice about how to utilize scarce state resources, since a dollar of revenue that is forgone through tax preferences can’t be used to pay for state services or tax cuts. Because of the trade-offs involved, it’s ill-advised to draw conclusions about tax incentives based solely on the economic outcomes. A program that creates 1,000 jobs might seem more successful than one that creates 500 jobs, but that impression changes if the former program costs 10 times more than the latter. For this reason, metrics usually provide the most insight when they combine fiscal costs and economic benefits into one.

Cost-per-job is probably the most common metric states have used. Although cost-per-job is an appropriate metric for some programs, its suitability will depend on the goals of the specific incentive. One reason to consider alternative metrics is that states are often concerned with the quality of jobs, such as their salaries and benefits, in addition to their quantity. There are also factors that may be less obvious, such as whether existing state residents or people from out-of-state are filling the jobs. Usually, states place more value on hiring their own residents, although that is not necessarily the case where population loss is a major concern.

4: Metrics should take into account the effects of tax incentives on the state’s economy as a whole

Incentives do not only affect the businesses that directly receive them. Providing incentives for some companies may also help other businesses to grow by giving them new customers. On the other hand, an incentive might harm existing businesses by introducing competition. Sometimes, these indirect effects are substantial. For example, an evaluation of a Louisiana incentive conducted by the state’s economic development agency found that for certain economic sectors, 90 percent of the jobs created displaced other jobs by introducing new competition—rather than creating additional jobs in the state.

It is for this reason that Tim Bartik, Senior Economist at the Upjohn Institute, recommended using growth in earnings per capita for all local residents in his 2011 presentation titled “What Works in Job Creation and Economic Development.” Such a measure takes into account how the economy as a whole is affected, not just the businesses receiving incentives. To add consideration of the fiscal costs, the metric might be defined as, “cost-per-dollar-of-growth in earnings-per-capita for all local residents.”

5: Part of defining a metric is defining how it will be calculated

When states measure the economic impact of their tax incentives, no metric is more common than jobs. Yet even something so ubiquitous and seemingly straightforward actually can mean several different things.

In many cases, every job is counted as one job regardless of its duration. In other cases, states have converted all jobs—whether they’re temporary, part-time, or ongoing—into a single measure that takes into account the duration of the jobs. For example, when the Massachusetts Department of Revenue calculates cost-per-job in its film tax credit evaluation, it measures the number of jobs in terms of annual full-time equivalents. So, a full-time job that lasted six months or a job that lasted all year but was only half-time would both count the same.

Decisions about how metrics are calculated are often as important as the selection of the metrics themselves. If, for example, a state is hoping to boost employment in a lasting way and an evaluation doesn’t differentiate between temporary and permanent jobs, policymakers may not be able to determine whether the program achieved its purpose.

6: Metrics should reflect an intention to isolate the impact of the incentive

Tax credits and exemptions are intended to get businesses to do something they otherwise would not have done, but sometimes they at least partially reward businesses for doing what they would have done anyway. For that reason, a key question that any evaluation of a tax incentive must answer is ‘to what extent did the incentive change business behavior?’ Metrics should be calculated with this question in mind.

The Minnesota Legislative Auditor’s review of the Job Opportunity Building Zone (JOBZ) program is a good example of why this consideration is so important. For JOBZ, the Minnesota economic development department placed the cost-per-job at around $5,000. The auditor’s evaluation, however, pointed out that this number didn’t take into account a number of relevant factors, including that many of the jobs would have been created even without the incentives (see pages 91-95 of Evaluation Report: JOBZ Program). Based on a body of academic research examining the extent to which tax incentives influence business decisions, the auditor’s office estimated that only 21 percent of the new jobs were created because of the incentive. The other 79 percent would have been created anyway. If only about a fifth of the jobs are actually attributable to the incentive, the cost-per-job is five times higher. So, this was the primary reason the auditor’s office estimated that JOBZ’s cost-per-job was between $26,900 and $30,800—not $5,000.

How states can determine success or failure

Once states set metrics, the next logical question is, “What does the metric need to show to declare success?” Some states have been intrigued by the idea that the success of a program can be determined by a specific benchmark (e.g. the cost-per-job should be X.) But this approach may not be practical or desirable because:

1. There is not a widely accepted set of benchmarks to draw upon.
2. It is not clear that policymakers would (or necessarily should) be bound by the benchmark, since the results that are possible depend on broader economic circumstances—not just the effectiveness of the incentives.
3. A pass/fail approach does not provide valuable information about how the program could be improved.

As an illustration of these points, consider cost-per-job. Even though it is a common metric, there is no single established benchmark to indicate what cost-per-job a state should achieve to make its tax incentives worthwhile. One reason is that the cost-per-job depends to a great extent on the geographic area offering the incentives.

A presentation from 2010 by Tim Bartik presents the cost-per-job of various state and federal programs. Bartik’s calculations show that most federal endeavors—from tax cuts to unemployment benefits—cost somewhere close to $105,000 per job (see pg. 6). The reason, he says, is that this is roughly the amount that GDP needs to increase to create one additional job, so investments that aren’t specifically well-targeted to job creation will cost about that amount to create new jobs.

But, as you can see in the state examples in his presentation, Bartik makes the point that the baseline cost-per-job should be lower on the state level. That is because it’s easier to create a truly new job on the state level than on the national level: If an incentive moves a job from one state to another, that counts as a new job from the perspective of the state where it is created, but not from the perspective of the country as a whole.

Other scholars have made similar observations. Both David Neumark and Jennifer Weiner discuss specific cost-per-job benchmarks, but also point out the limitations of this approach. Neumark, for example, makes the intuitive argument that it’s easier for hiring credits to spur job creation in bad economic times, when more people are looking for work.

Moreover, because different evaluations have calculated cost-per-job differently, apples-to-apples comparisons are difficult. For those reasons, Weiner argues, “While standard thresholds can be useful, analysts should ideally compare the cost-effectiveness of any business tax credit with those of other policies designed to achieve similar goals. Even a credit with an ‘acceptable’ level of cost-effectiveness may not be the best deal if another policy would yield a bigger bang for the buck.”

We see this comparative approach to benchmarks as very valuable. We recommend thinking in terms of questions such as:

• Is the incentive getting better results than its own past performance? Several states have taken the approach that they should constantly try to make their incentives more effective.
• As Weiner suggests, is the incentive more effective than other economic development strategies the state is pursuing? The results of a tax incentive could be compared to other incentives. They could also be compared to other potential approaches to improving the economy, from cutting tax rates to boosting state spending.

Kentucky is one state that has used this comparative approach effectively. An evaluation commissioned by the state’s Legislative Research Commission compared the effectiveness of the Kentucky’s incentives to one another, as well as to an alternative use of the money, namely general business tax cuts (see pages 97-106). Likewise, when the Wisconsin Department of Commerce reviewed the state’s film tax credit, they showed that it got much worse results that other economic development programs (see pages 18-22), which helped them make a compelling case that the program was flawed. An evaluation in North Carolina took a similar approach.

Endnotes

1. Louisiana Economic Development, “Enterprise Zone Program 2009 Annual Report,” March 2010, http://www.louisianaeconomicdevelopment.com/documents/additional-resources/2009_Annual_Report_Enterprise_Zone.pdf.
2. Timothy J. Bartik, "What Works in Job Creation and Economic Development," Transforming Communities Conference of the National Employment Law Project, Flint, MI, June 1, 2011, http://research.upjohn.org/cgi/viewcontent.cgi?article=1023&context=presentations.
3. Massachusetts Department of Revenue, “A Report on the Massachusetts Film Industry Tax Incentives,” March 21, 2013, http://www.mass.gov/dor/docs/dor/news/2012filmincentivereport.pdf.
4. Minnesota Office of the Legislative Auditor, “Evaluation Report: JOBZ Program,” Feb. 2008, http://www.auditor.leg.state.mn.us/ped/pedrep/jobz.pdf.
5. Timothy J. Bartik, "Estimating the Costs per Job Created of Employer Subsidy Programs,"  Labor Markets in Recession and Recovery conference of the Upjohn Institute, Kalamazoo, MI, Oct. 22–23, 2010, http://research.upjohn.org/cgi/viewcontent.cgi?article=1021&context=confpapers.
6. David Neumark, “Spurring Job Creation in Response to Severe Recessions: Reconsidering Hiring Credits,” March 2011, http://www.nber.org/papers/w16866.
7. Jennifer Weiner, “State Business Tax Incentives: Examining Evidence of their Effectiveness,” Discussion Paper, Federal Reserve Bank of Boston, Dec. 2009, http://www.bostonfed.org/economic/neppc/dp/2009/neppcdp0903.pdf.
8. Anderson Economic Group, LLC (for the Kentucky Legislative Research Commission), “Review of Kentucky’s Economic Development Incentives,” June 11, 2012, http://www.lrc.ky.gov/Lrcpubs/AEG%20KY%20Incentive%20Report_jun112012.pdf.
9. Wisconsin Department of Commerce, “Cost Benefit Analysis of Wisconsin Film Tax Credit Program,” March 2009, http://www.ctvoices.org/sites/default/files/bud09wisconsinppt.pdf.
10. University of North Carolina Center for Competitive Economies, “An Evaluation of North Carolina’s Economic Development Incentive Programs: Final Report,” (for the North Carolina General Assembly Joint Select Committee on Economic Development Incentives), July 2009, http://www.ncleg.net/documentsites/committees/JSCEDI/UNC%20C3E%202009%20final%20report%20to%20NCGA%20Joint%20Select%20Committee%20on%20Economic%20Development%20Incentives.pdf.