Metrics Are The Difference Between Good And Bad Policy

By: Abigail Aiken
December 30, 2016

Publsihed in the Austin-American Statesman

In Travis County, we are witnessing a robust debate about how public funds are being spent to improve health care for those who need it most. And in that debate, two words stand out: accountability and metrics. Being accountable to the taxpayer and to those served means showing how money is being spent and assessing whether the spending is having a positive impact on health.

Community leaders who demand accountability from Travis Central Health and want to measure its progress with hard data are absolutely correct to do so. But the real challenge lies in deciding which metrics will actually provide a meaningful picture of health care for the most vulnerable in our community. In short, simply having metrics is not enough. The metrics have to be the right ones.

In my 14 years of studying and working in health care, I’ve learned that coming up with the right metrics is a whole lot harder than it sounds. There is the issue of how to weigh quality versus quantity of care. But even more importantly, we must bear in mind that metrics create incentives — and incentives shape behavior. It’s stunningly easy to encourage perverse behaviors by placing a groundless faith in metrics for metrics’ sake.

It’s especially easy to get this wrong in something as complex as health care. Disease is complicated and doctors aren’t robots. In fact, metrics with the wrong hidden incentives can end up hurting the very population that the public health-care system is charged to serve.

Here in the United States, we have seen renewed interest among public health professionals in promoting the use of long-acting contraceptive methods, such as an intrauterine contraceptive (IUC) or implant. These IUCs and implants are highly effective and have been shown repeatedly in studies to reduce unintended pregnancy. It’s therefore natural to consider the following metric: How many women in our reproductive-health clinics are receiving the most effective methods?

Yet there’s a very real worry here. Setting a target like this — and making doctors aim for it — may incentivize coercion, especially of low-income women and women of color, who have the largest share of unintended pregnancies. In fact, recent history shows that these fears are well-founded. In England, IUC and implant uptake metrics have indeed been mandated in the post-abortion setting. The result is that women with few resources are disproportionately pressured to receive IUCs and implants, and providers are strongly incentivized to pressure them regardless of these patients’ wishes. One government-funded program even went a step further: It made acceptance of an IUC or implant a requirement for women to receive help in recovering their children from temporary government custody.

At best, these kind of hasty and naïve decisions about how to measure program performance or effect improvements in health-care delivery are a waste of taxpayers’ money. At worst, they are an affront to reproductive justice and to the bodily autonomy of the most vulnerable. I hope we can all agree that this kind of coercion is unconscionable. But we must also realize that placing a blind trust in metrics without an appreciation for the lived reality of patients and providers can make it all but inevitable.

The future of health care, and of Central Health’s ability to meets its core mission to serve the indigent population of Travis County, is highly uncertain. As Inauguration Day approaches, there are so many unknowns. How will health care funding at the federal level change? Will Medicaid as we know it persist? How will the state government react? In these uncertain times, the people of Travis County deserve an affirmative vision of progress in line with Central’s Health’s core mission. They also deserve thoughtful reflection on what exactly needs to be achieved and what measurement and evaluation efforts are going to demonstrate the results without creating perverse incentives.

As a professor of public policy, I have a strong appreciation for the importance of metrics. But to get the metrics right requires genuine conversations with members of the community and with health-care providers, coupled with a deep engagement with measurement methodology and health-care policy implementation. For Central Health, anything less will be a disservice to the people of Travis County who are relying on us to get it right.


  •   Map
  • Population Research Center

    University of Texas at Austin
    305 E. 23rd Street / RLP 2.602
    Mail Stop G1800
    Austin, Texas 78712-1699
    512-471-5514