Interventions are more effective when they are carried out as intended. But how can programme implementers keep tabs on how faithfully a programme is delivered?
“Adherence” is the degree to which practitioners follow the protocols of the intervention – whether they go by the book. It’s one aspect of implementation fidelity, a broader concept that also measures whether the intervention has been delivered skillfully.
Adherence matters because, over and over, delivering a programme as intended has been shown to be key in achieving programme impact.
Indeed, developing reliable, inexpensive, and easy-to-use measures of adherence is especially important when an intervention is leaving the lab and moving into real-world use. When programme developers are no longer involved in implementation, there is a risk that core components may be left out or delivered differently than originally intended.
This lack of adherence more often than not results in poorer programme outcomes. If delivered badly, the programme may have little or no effect – or it may even be worse than doing nothing at all. A good adherence test not only monitors programme delivery, but also encourages practitioners to adhere to an intervention’s core components.
In response to this need to measure programme adherence, researchers at The Norwegian Centre for Child Behavioural Development based at the University of Oslo, Norway, developed an adherence tool for Brief Parent Training (BPT). BPT aims to improve parenting practices in families with children aged 3-12 who have either developed conduct disorder or are displaying its early stages.
Parents complete the adherence measure at the end of the intervention. The questionnaire includes 28 statements to assess whether the core topics and components of BPT are being delivered. These questions cover key components such as positive involvement, skills encouragement, discipline, monitoring, and problem solving, and are rated using a five-point scale. The measure includes statements such as “I got practical advice on how to make my child cooperate better” and “We talked about how children learn best through use of praise and encouragement.”
Parent reports of adherence are not perfect: parents may over-report the level of adherence when they have positive attitudes towards the practitioner. But unlike more reliable methods such as independent observation, parent reports are inexpensive and easy to use – important considerations in designing a tool that can be widely used.
To assess the impact of programme adherence on parenting practices and child conduct disorder, researchers used the adherence questionnaires completed by 92 Norwegian families upon completion of the BPT programme. They analysed the relationship between practitioner adherence and several post-intervention aspects of parenting.
Results showed that adherence to BPT delivery protocol by practitioners predicted greater reductions in the use of inconsistent discipline as well as greater increases in the use of appropriate discipline and positive parenting.
The relationship between adherence and child conduct disorder was less conclusive, although this may be partly due to the relatively small sample size.
So what does this mean?
Successfully testing an adherence measure such as this has a number of implications. First, it demonstrates a reliable method of measuring practitioner adherence to programme protocol in an evidence-based parenting programme – enabling meaningful and practical quality assurance of programme delivery to take place in the future.
Second, it provides even more evidence that greater programme adherence results in greater programme impact: parenting practices will improve more when practitioners of BPT ensure they are delivering the core components in the right way, according to tried and tested delivery protocol.
Although these are positive results for a programme adherence tool, further testing is required before its use can become embedded in BPT programme delivery. Including evidence from practitioner reports and direct observations of BPT sessions would provide greater levels of detail about the validity of the tool and would also avoid bias in parent-reported results.
In addition, it is important to remember that programme adherence is only one part of programme fidelity. Practitioner competence, not measured by this adherence tool, is another vital aspect of fidelity that has also been shown to impact on programme outcomes.
Kjobli, J., Bjorknes, R., & Askeland, E. (2012). Adherence to brief parent training as a predictor of parent and child outcomes in real-world settings. Journal of Children’s Services, 7(3), 165-177.Return to Features