SUPPORT Summaries
Evidence of the effects of health system interventions
for low- and middle-income countries
Evidence of the effects of health system interventions
for low- and middle-income countries
February, 2017 - SUPPORT Summary of a systematic review | print this article | download PDF
Audit and feedback is commonly used as a strategy to improve professional practice. It appears logical that healthcare professionals would be prompted to modify their practice if given feedback that their clinical practice was inconsistent with that of their peers or accepted guidelines.
Key messages
People making decisions concerning use of audit and feedback to improve the quality of health care.
Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2012; (6): CD000259.
A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.
SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.
Audit and feedback can be defined as any summary of clinical performance of healthcare over a specified period of time that is fed back to healthcare providers with the aim of improving practice or the organisation of care. Feedback can be given in a written, electronic or verbal format. The summary of clinical performance may also include recommendations for clinical action.
As audit and feedback is used widely within healthcare organisations, it is important to consider its likely effects on clinical performance.
| Review objectives: To assess the effects of audit and feedback on the practice of healthcare professionals and on patient outcomes. | ||
| Type of | What the review authors searched for | What the review authors found |
|---|---|---|
| Study designs & interventions |
Randomised trials assessing the effects of audit and feedback. Interventions were only included if audit and feedback was a core or essential element. |
140 randomised trials were included. The interventions used were highly heterogeneous with respect to their content, format, timing and source. Targeted behaviours were prescribing (39 trials), management of patients with diabetes or cardiovascular diseases (34), and test ordering (31). The remaining trials varied widely in terms of health conditions and targeted behaviours.
|
| Participants | Healthcare professionals responsible for patient care. |
In most of the trials the healthcare professionals were physicians. Other targeted providers included dentists (1 trial), nurses (3), pharmacists (2), and a mix of providers (14). |
| Settings | Healthcare settings |
USA (69 trials), Canada (11), UK or Ireland (21), Australia or New Zealand (10), and elsewhere (29). Only 5 studies were conducted in low and middle income countries: Sudan (2), Thailand (1), Laos (1), Argentina and Uruguay (1). 94 trials were in outpatient settings, 36 in inpatient settings, and the clinical setting was unclear in 10 trials.
|
| Outcomes | Objectively measured provider performance or healthcare outcomes |
There was large variation in outcome measures, and many trials reported multiple primary out comes. Most trials measured professional practice, with some also reporting patient outcomes. |
| Date of most recent search: December 2010 | ||
| Limitations: This is a well conducted systematic review with only minor limitations. | ||
Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2012; (6): CD000259.
High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.
Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.
Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.
Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.
* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.
† Substantially different = a large enough difference that it might affect a decision.
See last page for more information.
The review included 140 trials. Most trials were conducted in high income countries (136). Only five trials were conducted in low and middle income countries (two in Sudan and one each in Thailand, Laos, and Argentina and Uruguay).
The interventions varied in terms of content, format, timing, and source. In 121 trials, audit and feedback was targeted at physicians. In 91 trials one or more groups received a multifaceted intervention where audit and feedback was considered the core, essential component.
Many trials reported multiple primary outcomes. Most trials reported professional practice outcomes, such as prescribing or use of laboratory tests, while some trials also reported patient outcomes, such as smoking status or blood pressure.
There was important heterogeneity in the results across trials.
Audit and feedback compared to usual care |
|||
| People: Health care professionals. Settings: Primary and secondary care in high, middle and low income countries. Intervention: Audit and feedback with or without other interventions. Comparison: Usual care. |
|||
| Outcomes | Impact (weighted absolute improvement or decrease)1 | No of Participants (studies) |
Quality of the evidence (GRADE) |
| Adherence to desired practice (dichotomous outcomes) |
Median absolute increase in desired practice: 4.3% (IQR 0.5% to 16.0%) |
82 comparisons from 49 studies2 [2310 clusters/groups of health providers (from 32 cluster trials) and 2053 health professionals (from 17 trials allocating individual providers)] |
Moderate
|
| Adherence to desired practice (continuous outcomes) |
Median percent change in desired practice: 1.3% (IQR 1.3% to 28.9%) |
26 comparisons from 21 studies [661 clusters/groups of health providers (from 13 cluster trials) and 605 health professionals (from 8 trials allocating individual providers)] |
Moderate
|
| Patient outcomes (dichotomous) | Median absolute decrease in desired outcomes: 0.4% (IQR -1.3% to 1.6%) | 12 comparisons from 6 studies |
Low
|
| Patient outcomes (continuous) | Median percent change in desired outcomes: 17% (IQR 1.5 to 17%) | 8 comparisons from 5 studies |
Low
|
| IQR: Interquartile range GRADE: GRADE Working Group grades of evidence (see above and last page) |
|||
|
1.The post intervention risk differences are adjusted for pre intervention differences between the comparison groups to account for baseline differences. The effect was weighted across studies by the number of health professionals involved in the study to ensure that small trials did not contribute as much to the estimate of effect as large trials. 2.Many studies had more than two arms and therefore contributed multiple comparisons of audit and feedback versus usual care. |
|||
A total of 22 comparisons from 20 trials were included in this analysis.
| Findings | Interpretation* |
|---|---|
| APPLICABILITY | |
|
The 140 randomised trials reviewed covered an extensive range of interventions and settings, but only five of the studies were conducted in low and middle income countries.
|
Decisions about if and how to use audit and feedback to improve professional practice should be guided by pragmatic factors and local circumstances, including whether: -The known or anticipated baseline adherence to guidelines or recommended practice is low -Conducting regular audits is feasible and the resources needed to collect these data are low -The data available for audit purposes are reliable -Supervisors are available to provide feedback and to link this to an action plan
|
| EQUITY | |
| Overall, the included studies provided little data regarding differential effects of the interventions for disadvantaged populations. |
The resources needed to implement audit and feedback may be less easily available in disadvantaged settings. |
| ECONOMIC CONSIDERATIONS | |
| Few trials reported the cost or cost effectiveness of the interventions. |
The resources needed to implement audit and feedback are likely to vary across settings and need to be estimated based on specific local conditions, including the availability of reliable routinely collected data and personnel costs.
|
| MONITORING & EVALUATION | |
|
Few rigorous studies of the effects of audit and feedback have been conducted in low or middle income countries.
|
The implementation of audit and feedback in low income settings should be accompanied by rigorous evaluation. Future studies of audit and feedback should compare different ways of providing feedback and their resource implications. |
|
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low income countries. For additional details about how these judgements were made see:www.supportsummaries.org/methods
|
|
Ivers NM, Sales A, Colquhoun S, et al. No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci 9(1):14.
Ivers NM, Grimshaw JM, Jamtvedt G, et al. Growing Literature, Stagnant Science? Systematic Review, Meta Regression and Cumulative Analysis of Audit and Feedback Interventions in Health Care. J Gen Intern Med 2014; 29:1534-41.
Gardner B, Whittington C, McAteer J, et al. Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback. Soc Sci Med 2010; 70: 1618–25.
Hysong SJ. Meta-Analysis: audit and feedback features impact effectiveness on care quality. Med Care 2009; 47: 356–63.
NorthStar is a tool that provides a range of information, checklists, examples and tools on how to best design and evaluate quality improvement interventions. https://www.dropbox.com/sh/8mn8co0au0eupt1/AAC55wRS6YwLPRDsMWuTtueda?dl=0
Tomás Pantoja, Department of Family Medicine, Pontificia Universidad Católica de Chile, Chile and Signe Flottorp, Norwegian Institute of Public Health, Oslo, Norway
None declared. For details, see: www.supportsummaries.org/coi
This summary has been peer reviewed by: Airton Stein and Noah Ivers.
Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2012; (6): CD000259.
Pantoja T, Flottorp S. Does providing healthcare professionals with data about ther performance improve their practice? A SUPPORT Summary of a systematic review. February 2017. www.supportsummaries.org
evidenceinformed health policy, evidence based, systematic review, health systems research, health care, low and middleincome countries, developing countries, primary health care, audit, feedback, audit and feedback.
