SUPPORT Summaries
Evidence of the effects of health system interventions
for low- and middle-income countries
Evidence of the effects of health system interventions
for low- and middle-income countries
September, 2009 - SUPPORT Summary of a systematic review | print this article |
Audit and feedback of critical incidents has become an established part of obstetric practice in many settings. Longitudinal data in some settings has suggested that the practice may improve maternal and perinatal mortality. This review sought to establish whether there is evidence of an effect on maternal and perinatal mortality, or any potential harmful effects and costs. Critical incidents refer to mortality and severe morbidity. The authors searched for randomised controlled trials of “any summary of clinical performance over a specified period of time” and “method of feeding that information back to the clinicians”, that reported objectively measured professional practice in a healthcare setting or healthcare outcomes.
Pattinson RC, Say L, Makin JD, Bastos MH. Critical incident audit and feedback to improve perinatal and maternal mortality and morbidity. Cochrane Database of Systematic Reviews 2005, Issue 4. Art. No.: CD002961. DOI: 10.1002/14651858.CD002961.pub2. See in Cochrane Library
A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.
SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.
Audits of maternal and perinatal mortality are commonly undertaken to establish numbers of deaths, causes of deaths, and potential modifiable factors. Longitudinal data from the UK suggests that such maternal mortality audits have contributed to a decline in maternal mortality in the UK over several decades. Serial data from Mozambique have also shown audit and feedback to have a signficant positive impact on intrapartum foetal mortality and perinatal mortality (Bugalho 1993). However, other countries which routinely report maternal and perinatal mortality have not experienced similar improvements in outcomes.
A systematic review of the effects of audit and feedback on professional practice found that audit and feedback can be effective in improving professional practice (Jamvedt 2006). The effects were typically small to moderate (a 4% median increase in compliance with desired practice for audit and feedback alone).
This summary is based on a systematic review published in 2005 by Pattinson and colleagues, which sought to assess the evidence of the effectiveness of critical incident audit and feedback in improving maternal and perinatal mortality and morbidity.
| Review Objectives: To assess whether critical incident audit and feedback improves maternal and perinatal mortality and morbidity. | ||
| / | What the review authors searched for | What the review authors found |
|---|---|---|
| Interventions | RCTs of audit and feedback of maternal and perinatal morbidity and mortality |
No trials were identified which met the review’s inclusion criteria. |
| Participants | Maternity units | None |
| Settings | Not specified |
None |
| Outcomes |
Primary outcomes: |
None |
| Date of most recent search: January 2005 | ||
| Limitations: This is a good quality systematic review with only minor limitations | ||
Pattinson RC, Say L, Makin JD, Bastos MH. Critical incident audit and feedback to improve perinatal and maternal mortality and morbidity. Cochrane Database of Systematic Reviews 2005, Issue 4. Art. No.: CD002961. DOI: 10.1002/14651858.CD002961.pub2. See in Cochrane Library
High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.
Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.
Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.
Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.
* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.
† Substantially different = a large enough difference that it might affect a decision.
See last page for more information.
The review did not find randomised controlled trials that assessed the effect of critical incident audit and feedback on perinatal mortality rate, maternal mortality ratio, or severe neonatal and maternal morbidity.
The authors of the review argue that collecting data to monitor perinatal and maternal mortality is vital, and that evidence from serial data suggests more benefit than harm.
Randomised controlled trials are required to evaluate the effectiveness of feedback mechanism to clinicians and other people, and the inclusion of data on suboptimal care. Such studies should also include economic evaluations of critical incident audit and feedback systems.
| Findings | Interpretation* |
|---|---|
| APPLICABILITY | |
| No data were available for any country (low, middle or high-income). |
The collection and use of maternal and perinatal morbidity and mortality data is important in all settings. However, it is uncertain whether, and what type of, feedback to clinicians is effective, and whether including indicators of suboptimal care results in improvements in morbidity and mortality. |
| EQUITY | |
| No data were provided. Critical incident audits have been used in high-income countries such as the UK over a relatively long period. The review does not however indicate whether it has promoted equity in such settings. | Critical incident audit and feedback can be implemented in most health care settings with sufficient clinical and data management capacity. However, a lack of resources to address system constraints which may underly suboptimal care could limit its effectiveness in resource constrained settings. |
| ECONOMIC CONSIDERATIONS | |
| No data were available. | There will be financial and and human resource requirements associated with collecting additional data, and ensuring feedback to clinicians and other service providers. The effectiveness as well as the cost-effectiveness of critical incident audit and feedback should be assessed. |
| MONITORING & EVALUATION | |
| No data were available. |
A critical incident audit is a form of monitoring and evaluation. However, the use of critical incident audit and feedback should include indicators to assess its impact and costs. Ideally, this should be done in the context of randomised trials. |
| *Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low and middle-income countries. For additional details about how these judgements were made see: http://supportsummaries.org/support-summaries/how-support-summaries-are-prepared/ | |
Related literature
Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. The Cochrane Database of Systematic Reviews 2006, Issue 2.
Flottorp S. Does providing healthcare professionals with data about their performance improve their practice? A SUPPORT Summary of a systematic review. August 2008. htttp://www.support-collaboration.org/summaries.htm
Perinatal Education Program. Unit 49 maternal and perinatal mortality audits. Available at: http://pepcourse.co.za/index.php?option=com_content&task=view&id=42&Itemid=32
Mcfarlane A, Mugford M,Henderson J, Furtado A, Stevens J, Dunn A. Birth Counts: statistics of pregnancy and child-birth. Norwich, United Kingdom: The Stationery Office, 2000.
Bugalho A, Bergström S. Value of perinatal audit in obstetric care in the developing world: a ten-year experience of the Maputo model. Gynecol Obstet Invest 1993;36(4):239-43.
Saving Mothers – Report on Confidential Enquiries into Maternal Deaths in South Africa
http://www.doh.gov.za/docs/index.html
This summary was prepared by
Lilian D Dudley & Charles Shey Wiysonge, South African Cochrane Centre, South African Medical Research Council, Cape Town, South Africa
Conflict of interest
None declared. For details, see: Conflicts of Interest
Acknowledgements
This summary has been peer reviewed by: Robert Pattinson, South Africa; Metin Gumelzoglu, Switzerland; Pisake Lumbiganon, Thailand; Fernando Althabe, Uruguay; Esperança Sevene, Alda do Rosário Elias Mariano and Julie Cliff, Mozambique
This summary should be cited as
Dudley LD. Wiysonge CS. Does critical incident audit and feedback improve perinatal and maternal mortality and morbidity? A SUPPORT Summary of a systematic review. September 2009
