• By Dartington SRU
  • Posted on Thursday 29th May, 2014

Picking them, passing them, and doing them

There are many levels of success in prevention science. Finding an effective evidence-based programme is, to use a baseball analogy, like hitting a single. If the programme is supported in the State legislature, it’s akin to a double or a triple. The true home run is getting the programme fully integrated into mainstream systems. Such was the message delivered by Steve Aos ahead of the launch of Investing in Children.

Aos is the Associate Director of the Washington State Institute for Public Policy (WSIPP) in the US. The Institute’s success at bringing hard-nosed economic analysis to Washington State’s investment decisions is known and respected around the world. He has been working with the Social Research Unit at Dartington over three years to translate the WSIPP model for the UK.

This sophisticated approach has arguably led to the greatest density of evidence-based programmes in the US. Nowhere is this truer than in Washington State’s criminal justice system. New prisons typically cost about $250 million to build and $45 million a year to operate. So in 2005 the Washington legislature asked Aos and his team to examine the economics of a whole range of crime fighting options.

They found that although the boom in prison building had helped drive down the crime rate by two-fifths, the cost to the taxpayer had more than doubled. The challenge was to find alternatives that would fight crime but at a lower cost.

Aos and his team took a three-step approach to find these programmes. The first thing they did was look at the research. ”That’s the starting point,” Aos said. “If you don’t know whether or not it works there is no point doing an economic analysis.”

Having reviewed over 500 studies, the team next turned to the economics of implementing the programmes in Washington State. The WSIPP model is well respected for its consistent, conservative estimates of the economic benefits of evidence-based programmes both for government and individuals.

“Only when we know what works, and what the costs and benefits are do we go portfolio,” observed Aos. That’s the third step. “The portfolio is the list of policies and programmes that can be relied on to deliver an outcome that matters to policy makers, less crime for example.”

Aos and his team delivered a portfolio for crime fighting. The result of this analysis was that money slated to build a new prison was instead redirected into programmes. Millions of dollars were invested in programmes like Functional Family Therapy, Multisystemic Therapy and Nurse Family Partnership.

Pre-school education is now funded by Washington, not simply because of its potential impact on children’s intellectual development, but also because it is a cost-effictive way to reduce crime.

WSIPP’s methods have been celebrated as a major innovation in common-sense government. Less famous, however, are the efforts of the Institute to monitor the policy changes it has recommended. The question is simple: does the portfolio deliver what the economic model said it would?

The programmes are currently under evaluation from a couple different angles. For instance, is the programme reaching the people for whom it’s intended? According to Aos, “If we are going to realise some of the financial benefits we anticipate from our model we need to get the right people to the right programmes. The ‘right people’ means the high-risk populations, those who are candidates for expensive interventions like custody.”

Aos said it isn’t enough to put the programmes in place; you have to educate everyone about what they do. He talked about one finding “where judges referred non drug-offending youth to drug-offending programmes.” Clearly, there were growing pains.

An adjustment fixed the problem. Aos said, “Each child who gets convicted is now assessed using a psychometric tool to ensure that they get the right evidence-based programme.”

Many of the programmes required similar fine-tuning. Functional Family Therapy (FFT) is one of foundations of the WSIPP portfolio. Early evaluations showed that poor implementation of FFT had in some cases exacerbated problem behaviours rather than cured them. So WSIPP recommended improved assessment and supervision systems, and the problems began to resolve themselves.

The state legislature also forced the hand of some providers. According to Aos, “The State Legislature said ‘If you don’t put a fidelity system in place we will cut funding of the programme’.” He said it was easy to follow which providers took the threat seriously. “My marker is, do people who aren’t delivering with fidelity get fired or re-assigned? If they don’t, then programme providers probably aren’t taking fidelity seriously.”

Playing with funding formulas has also been a good way to keep quality up. If agencies have clear financial incentives, they will deliver programmes as intended. Local buy-in is also encouraged by giving communities some choice in what programmes work best for them. Communities can choose the programmes from an approved list.

A people business

It wasn’t easy, Aos said, but over time the culture of government services in Washington State changed. “In the 1990s I was focusing on juvenile justice and there was great resistance to doing evidence-based programmes.” His focus was eventually directed elsewhere and someone else took over his responsibilites at the Institute. When Aos returned to the issue years later, he said a seismic shift had occurred. “I attended a practitioner team meeting where they were discussing their focus and they listed assessment and evidence-based programmes as the number one thing they do. I thought they were kidding me. But it was down to the persistence of the State legislators over the years.”

Aos said one reason for WSIPP’s success is its non-partisan approach. “Republicans and Democrats are interested in crime reduction, and Democrats and Republicans are both interested in saving taxpayer dollars. Our Institute gives independent advice. There are no political bumper stickers on the cars of our staff.”

In conclusion, Aos struck a reflective note. “I have come to appreciate that the analytics are less than half the challenge. The public policy advice business is a ‘people’ business and we don’t expect perfect use of the information we provide. In fact I think of it as a baseball hitter might think of it. If I can be successful 35 per cent of the time I can be counted as a great hitter. That means 65 per cent of the time I will fail.”

Return to Features