SUPPORT Summaries
Evidence of the effects of health system interventions
for low- and middle-income countries
Evidence of the effects of health system interventions
for low- and middle-income countries
January, 2017 - SUPPORT Summary of a systematic review | print this article | download PDF
Anumber of interventions aiming to increase the use of systematic review evidence in decision making are currently in use. These include summaries of systematic reviews designed to improve the accessibility of the findings (“information products”) and changes to organisational structures, such as employing specialist groups to synthesise evidence in order to inform local decision making.
Key messages
People wanting to improve the use of evidence by health system managers, policy makers, or clinicians.
Murthy L, Shepperd S, Clarke MJ, et al. Interventions to improve the use of systematic reviews in decision making by health system managers, policy makers and clinicians. Cochrane Database Syst Rev 2012; 9: CD009401.
A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.
SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.
Systematic reviews provide a transparent and robust summary of existing research. However, health system managers, national and local policymakers, and healthcare professionals can face several obstacles when attempting to utilise this evidence. These include dealing with a large volume of research evidence and difficulties making judgements about the applicability of evidence from systematic reviews. In an attempt to increase the use of systematic review evidence in decision making, a number of interventions have been developed.
| Review objectives:To identify and assess the effects of information products based on the findings of systematic review evidence, and organisational supports and processes designed to support the uptake of systematic review evidence by health system managers, policy makers, and healthcare professionals. | ||
| Type of | What the review authors searched for | What the review authors found |
|---|---|---|
| Study designs & interventions | Randomised trials, interrupted time series studies and controlled before after studies of interventions intended to improve the uptake of evidence from systematic reviews in decision making |
Eight studies:multifaceted interventions (2 cluster randomised trials); summary of findings tables (1 randomised trial); analgesic league table (1 randomised trial);organizational intervention (knowledge broker, access to systematic review repository, provision of tailored messages) (1 randomised trial); dissemination of printed bulletins (3 interrupted timeseries studies). |
| Participants | Health system managers, policymakers and clinicians |
Nurses (1 study); physicians and nurses (1); public health professionals (1); evidence based practice workshop participants (1); healthcare professionals and students (1); National Health Service (NHS) clinicians and decision makers (3) |
| Settings | Any setting |
UK (5 studies); Canada (1); Mexico and Thailand (1); setting not specified (1) |
| Outcomes | Utilisation of research, acceptability of the way information was presented, knowledge, utilisation of healthcare resources, patient related outcomes. | Utilisation of research (6 studies); utilisation of healthcare resources (costs) (3); knowledge (1 RCT); perceived understanding and ease of use (1); preferences and attitudes (1); patientrelated outcomes (pain management and use of analgesia) (2) |
| Date of most recent search: March 2012 | ||
| Limitations:This is a well-conducted systematic review with only minor limitations. | ||
Murthy L, Shepperd S, Clarke MJ, et al. Interventions to improve the use of systematic reviews in decision making by health system managers, policy makers and clinicians. Cochrane Database Syst Rev 2012; 9: CD009401.
High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.
Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.
Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.
Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.
* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.
† Substantially different = a large enough difference that it might affect a decision.
See last page for more information.
The review identified eight studies that evaluated the effectiveness of different interventions designed to support the use of systematic review evidence decision making.
Three studies evaluated the effects of multifaceted interventions on obstetric practice (1 each in Mexico, Thailand and the UK). One study evaluated the effects of a multifaceted intervention on patient outcomes (pain and use of analgesics) for patients on four orthopaedic wards in the UK. The interventions included access to a database (3 studies) or a league table of analgesic efficacy (1 study), training (interactive workshops or educational outreach), audit and feedback (2 studies), and a coordinator to assist use of the database (2 studies).
Multifaceted interventions to improve the use of evidence derived from systematic reviews in decision making by clinicians |
|||
| People: Clinicians Settings: Secondary care settings in the UK (2 studies), Mexico (1) and Thailand (1) Intervention: Multifaceted interventions to support the use of evidence derived from systematic reviews Comparison: No intervention/usual care |
|||
| Outcomes | Median effect (range) | Certainty of the evidence (GRADE) |
|
|
Professional practice for six obstetric practices (2 studies), four obstetric practices (1) |
Median effect 3.5% improvement in desired professional practice (adjusted for baseline differences) (2.0% to 3.5%)
|
Low |
|
|
Patient outcomes pain and use of analgesics (1 study) |
There was little or no difference in the average amount of pain or use of analgesics |
Low |
|
|
1 The size of the differences varied widely within each study and most of the effect estimates had wide confidence intervals. 2 There was a serious risk of bias and imprecision. GRADE: GRADE Working Group grades of evidence (see above and last page) |
|||
Three studies evaluated the effects of information products targeted at clinicians.
Summaries of systematic reviews targeted at clinicians:
Summaries of systematic reviews targeted at clinicians |
|||
| People: General practitioners (1 study), surgeons (1), and diverse clinicians (1) Settings: Primary care in the UK (1 study), secondary care in the UK (1), and a workshop in Norway (1) Intervention: A mailed summary of a systematic review (2 studies) and a summary of findings table (1) with a systematic review Comparison: No intervention/usual care (2 studies) and a systematic review without a summary of findings table (1) |
|||
| Outcomes | Effect | Certainty of the evidence (GRADE) |
|
|
Professional practice use of antidepressant drugs (1 study) and gromlet insertion (1 study) |
Inappropriate use decreased |
Low |
|
|
Perceived accessibility of the findings proportion that perceived the main findings to be very accessible - 6 or 7 on a scale from 1 (very inaccessible) to 7 (very accessible) |
Difference 24% more (41% versus 17%; p=0.037) |
Moderate |
|
|
1 Non randomized (interrupted time series) studies 2 Serious risk of bias GRADE: GRADE Working Group grades of evidence (see above and last page |
|||
One study in Canada evaluated the added effects of tailored messages (one group) and tailored messages together with knowledge brokers (one group) together with access to a database of systematic reviews compared to access to the database alone.
| Findings | Interpretation* |
|---|---|
| APPLICABILITY | |
| None of the included studies was conducted in a low income country. Evaluated interventions included passive dissemination of systematic review evidence (in the form of bulletins) and multifaceted interventions (which provide access to, and training in the use of systematic review evidence). |
The impact of information products derived from systematic reviews (printed bulletins) and multifaceted interventions evaluated by the included studies is uncertain in low income countries. |
| EQUITY | |
| The included studies did not report data regarding differential effects of systematic review information products and organisational processes (bulletins, multifaceted interventions) evaluated across different settings. |
The interventions evaluated (summary of findings tables, league tables, knowledge brokers, systematic review repositories) require information systems and technical skills that may be lacking in low income settings. This may limit access and utilisation in low income settings. |
| ECONOMIC CONSIDERATIONS | |
| No data were available from low income settings. Three studies in high income countries reported data on health resource utilisation (costs): implementation cost of a single educational visit, and costs of production / distribution and potential savings of bulletins summarising systematic review evidence. |
Costs of interventions intended to improve the use of systematic review evidence in decision making include costs related to training, production, and dissemination of information products. Such costs may limit utilisation, and consequently effectiveness of these interventions in low income countries. |
| MONITORING & EVALUATION | |
| There was no eligible study of interventions for improving use of research evidence in decision making processes in low income countries. |
Randomised trials are needed to evaluate the effects of different interventions designed to support uptake of systematic review evidence in low income countries. |
|
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low income countries. For additional details about how these judgements were made see: www.supportsummaries.org/methods. |
|
Tricco AC, Cardoso R, Thomas SM, et al. Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review. Implement Sci. 2016;11:4.
Wallace J, Byrne C, Clarke M. Making evidence more wanted: a systematic review of facilitators to enhance the uptake of evidence from systematic reviews and meta analyses. Int J Evid Based Healthc 2012;10(4):338-46.
Chambers D, Wilson PM, Thompson CA, et al. Maximizing the impact of systematic reviews in health care decision making: a systematic scoping review of knowledge translation resources. Millbank Quarterly 2011;89(1):131-56.
Perrier L, Mrklas K, Shepperd S, et al. Interventions encouraging the use of systematic reviews in clinical decision making: a systematic review. J Gen Intern Med 2011;26(4):419-26.
Perrier L, Mrklas K, Lavis J, Straus S. Interventions encouraging the use of systematic reviews by health policymakers and managers: a systematic review. Implementation Sci 2011;6:43.
Dobbins M, Cockerill R, Barnsley J. Factors affecting the utilization of systematic reviews. A study of public health decision makers. Int J Technol Assess Health Care 2001;17(2):203–14.
Newton Opiyo, KEMRI Wellcome Trust Research Programme, Kenya.
None declared. For details, see: www.supportsummaries.org/coi
This summary has been peer reviewed by Airton Stein. We did not receive any comments from the review authors.
Murthy L, Shepperd S, Clarke MJ, et al. Interventions to improve the use of systematic reviews in decision making by health system managers, policy makers and clinicians. Cochrane Database Syst Rev 2012; 9: CD009401.
Opiyo N. What are the effects of interventions to improve the use of systematic reviews in decision making by health system managers, policy makers and clinicians? A SUPPORT Summary of a systematic review. January 2017. www.supportcollaboration.org/summaries.htm
evidence informed health policy, evidence based, systematic review, health systems research, health care, low and middle income countries, developing countries, primary health care.
