• By Dartington SRU
  • Posted on Tuesday 25th November, 2014

Learning how to improvise with lightly tied hands

One tension in the thinking about programme "fidelity" is between the stipulation that a developer’s specifications are followed in order to guarantee effectiveness, and the need for practitioners and local communities to feel they “own” what they are being asked to deliver.

A balance between authority and autonomy has to be struck – and autonomy and ownership usually involve a degree of license to make changes.

On both sides of the equation, these difficulties are acknowledged, but so far there have been few attempts to establish exactly what changes to an intervention communities are most likely to advocate and whether they will tend to strengthen or weaken a programme’s effect.

Results from a study of violence and drug prevention programmes in schools in northern California by researchers from Berkeley suggest over three quarters of teachers’ and pupils’ adaptations would have been given the green light by the programmes’ developers.

The team spent two years monitoring implementation of Project Towards No Drug Abuse(TND) and Too Good for Drugs and Violence in schools. Both well regarded and and gradually advancing through the US accreditation system, the programmes were being introduced as part of a new health curriculum.

Staff were given extensive training. Once the programmes were running, staff were interviewed and their classes were observed on numerous occasions. Participating pupils were invited to take part in focus groups.

All of the adaptations and suggestions for change that emerged were later assessed by the programme developers. Were they likely to undermine the model and so jeopardize the effectiveness of the programme?

In the main, teachers tried to integrate details from everyday life to bring the curricula to life. For example, they shared the detail of how they dealt with ordinary crises, such as a car breaking down, or they brought in newspaper articles to illustrate points made in the programme. “Yeah, that’s what good teaching is. You find something that’s happened today and then put it in there,” explained one teacher.

Teachers also made a few changes to the language - to make it more intelligible and relevant to their pupils. More worryingly, they cut out sections they felt were less important to save teaching time, and they changed the order of the classes in response to the mood of the children.

During interviews teachers acknowledged the tension between implementing with high fidelity and providing effective teaching. They generally followed the script on the first run-through to get to know it, and they added their own touches as their self confidence increased.

Most of the curricular improvements they suggested were linked to teaching practices. For example, they were keen to introduce more group work and more interactive sessions than the programme specified.

Pupils, on the other hand, were primarily interested in “surface” adaptations, to make the programme more realistic for their age and cultural backgrounds. Their alterations were generally minor but they felt they were critical if the sessions were not to be considered boring.

The developers gave the green light to over three quarters of the adaptations and concluded that pupils’ suggestions were no more likely than teachers’ to threaten the logic model.

As to where the roads divided: pupils recommended encounters with speakers who had experiences of addiction and recovery; the developers said they would weaken the programme. They were also dubious about teachers' suggestions that there should be more information about other drugs, and more group work; the developers explained these had not been empirically tested.

Where to go from here? Ozer and her collaborators look to the medical practice of evidence “farming”, a process by which practitioners’ clinical experience is integrated into evidence based on randomized controlled trials.

No programme in the world can use language or and examples that are relevant to everyone, the authors say. More work is needed to analyze and specify the parameters that will help communities to adapt programmes without diluting their effect.

Ozer E J, Wanis M G and Bazell N (2009), “Diffusion of School-Based Prevention Programmes in Two Urban Districts: Adaptations, rationales, and suggestions for change’, Prevention Science, online preview

Return to Features