August, 2008 - SUPPORT Summary of a systematic review | print this article |

Which clinical guideline dissemination strategies improve professional practice?

A clinical guideline is a systematically developed statement intended to assist practi-tioners make appropriate decisions about health care for specific clinical circum-stances. Potential dissemination strategies for clinical guidelines include use of:
• Educational materials, i.e. distribution of published or printed recommendations for clinical care, including clinical practice guidelines and audiovisual materials;
• Educational meetings, i.e. participation by healthcare providers in conferences, lectures, workshops or traineeships;
• Audit and feedback, i.e. any summary of clinical performance of health care over a specified period;
• Patient-mediated interventions, i.e. new clinical information (not previously available) collected directly from patients and given to the provider;
• Reminders, i.e. patient- or encounter-specific information, provided verbally, on paper or on a computer screen, which is intended to prompt a health profession-al to recall information;
• Educational outreach, i.e. use of trained persons who meet with providers in their practice settings to give information with the intent of changing providers’ practice.
Key messages
 None of the studies included in this summary were conducted in a low-income country, only two were conducted in middle-income countries, and the rest were from high-income countries.
 The studies yield moderate quality evidence that use of the dissemination strate-gies (either individually or in combination) leads to improvements in guideline implementation and patient outcomes.
 Resources available for maintaining and improving quality of care need to be con-sidered when assessing whether the intervention effects are likely to be transfer-able to other settings in high- and middle-income countries. Rigorous studies from low-income countries are needed to fully assess applicability in all health-care settings.

A clinical guideline is a systematically developed statement intended to assist practi-tioners make appropriate decisions about health care for specific clinical circum-stances. Potential dissemination strategies for clinical guidelines include use of:

- Educational materials, i.e. distribution of published or printed recommendations for clinical care, including clinical practice guidelines and audiovisual materials;
- Educational meetings, i.e. participation by healthcare providers in conferences, lectures, workshops or traineeships;
- Audit and feedback, i.e. any summary of clinical performance of health care over a specified period;
- Patient-mediated interventions, i.e. new clinical information (not previously available) collected directly from patients and given to the provider;
- Reminders, i.e. patient- or encounter-specific information, provided verbally, on paper or on a computer screen, which is intended to prompt a health profession-al to recall information;
- Educational outreach, i.e. use of trained persons who meet with providers in their practice settings to give information with the intent of changing providers’ practice.

 

Key messages

  • None of the studies included in this summary were conducted in a low-income country, only two were conducted in middle-income countries, and the rest were from high-income countries.
  • The studies yield moderate quality evidence that use of the dissemination strate-gies (either individually or in combination) leads to improvements in guideline implementation and patient outcomes.
  • Resources available for maintaining and improving quality of care need to be con-sidered when assessing whether the intervention effects are likely to be transfer-able to other settings in high- and middle-income countries. Rigorous studies from low-income countries are needed to fully assess applicability in all health-care settings.

Who is this summary for?

People making decisions concerning dissemination strategies for improving clinical guideline implementation.

This summary includes:

  • Key findings from research based on a systematic review
  • Considerations about the relevance of this research for low income countries

Not included:

  • Recommendations
  • Additional evidence not included in the systematic review
  • Detailed descriptions of interventions or their implementation

This summary is based on the following systematic review:

Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8(6). See in Publisher Site

What is a systematic review?

A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.

SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.

Glossary of terms used in this report

Background references on this topic

Background

Clinical practice guidelines are an increasingly common element of clinical care throughout the world. Such guidelines have the potential to improve the care received by patients by promoting interventions of proven benefit and discouraging ineffective interventions. However, a clinical guideline will only impact on practice if it is disseminated effectively to, and implemented by, the target audience. This summary is based on a health technology assessment published in 2004 by Grimshaw and colleagues, and focuses on the effects of various dissemination strategies in improving guideline implementation.

How this summary was prepared

After searching widely for systematic reviews that can help inform decisions about health systems, we have selected ones that provide information that is relevant to low-income countries. The methods used to assess the reliability of the review and to make judgements about its relevance are described here

Knowing what’s not known is important

A reliable review might not find any studies from low-income countries or might not find any well-designed studies. Although that is disappointing, it is important to know what is not known as well as what is known.

A lack of evidence does not mean a lack of effects. It means the effects are uncertain. When there is a lack of evidence, consideration should be given to monitoring and evaluating the effects of intervention, if it is used.



About the systematic review underlying this summary

Review Objectives: To assess the effects of guideline dissemination strategies in improving professional practice
/ What the review authors searched for What the review authors found
Interventions

Randomised controlled trials (RCT), con-trolled clinical trials (CCTs), controlled before and after (CBA) studies, and inter-rupted time series (ITS) designs.

235 studies: 139 RCTs, 17 CCTs, 40 CBAs, and 39 ITS.
Participants

Medically qualified healthcare professionals.

Studies of multi-professional groups were also in-cluded if more than 50% were medically qualified.
Settings

Any healthcare setting e.g. primary care, inpatient, and mixed settings

 Studies from healthcare settings in the USA (71%), UK (11%), Canada (6%), Australia and Netherlands (3%); One each from Denmark, France, Germany, Israel, Mexico, New Zealand, Norway, Sweden, and Thailand.
Outcomes

Objective measures of provider behaviour and /or patient outcome.

Provider behaviours targeted included general man-agement, prescribing, test ordering, prevention, pa-tient education and advice, diagnosis, discharge planning, referrals, record keeping, etc, either indi-vidually (47%) or in combination. Patient outcomes included proportion of patients who received appro-priate treatment or advice and those who achieved the desired outcome (e.g. stopped smoking)
Date of most recent search: February 2004
Limitations: This is a good quality systematic review, which found evidence of moderate quality

Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8(6). See in Publisher Site

About the certainty of the 
evidence (GRADE)


High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.


Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.


Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.


Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.

* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.

† Substantially different = a large enough difference that it might affect a decision.

See last page for more information.

Summary of findings

The review included 235 studies. The settings were mainly primary (58%) and inpatient (19%) care. Physicians alone were the target of the guideline dissemination intervention in 74% of studies; most of them (57%) involving only one medical speciality, most commonly general practice or family medicine (24%). Only the findings of trials are summarised below, given their superiority to other designs in assessing the effectiveness of healthcare interventions.

 

1) Educational materials

Nine trials were identified which assessed the effects of distributing published or printed recommendations for clinical care (including clinical practice guidelines, audiovisual materials and electronic publications) to physicians; through personal delivery or mass mailings.

  • The studies showed moderate quality evidence that the distri-bution of educational materials to health professionals im-proves the process of care and patient outcomes.

General management of a clinical problem, prescribing, prevention services, or test ordering
Patient or population: Physicians
Settings: Primary care in the USA, UK, Canada, Australia, Netherlands
Intervention: Dissemination of educational materials to physicians
Comparison: No intervention
Outcomes Impact

Number of participants (studies)

 Quality of the evidence
(GRADE)

Dichotomous process measures (e.g. proportion of patients receiving appropriate treatment)

Median effect: +8.1% (range +3.6 to +17%) absolute improvement

(5 studies)

Continuous process measures (e.g. number of prescriptions issued by providers)

Relative improvement: +34.7% (standardized mean difference [SMD]+0.25)

 (3 studies)

Continuous outcome measure (e.g. mean symptom score).

 Median effect: +17.1% (SMD +0.86) relative improvement (1 study)

GRADE: GRADE Working Group grades of evidence (see above and last page)

 

2) Educational meetings
Three trials assessed the effects on improving professional practice of participation by healthcare providers in conferences, lectures, workshops or traineeships. 
A synthesis of the trial results provided evidence of low quality that educational meet-ings improve patient care.

2) Educational meetings

Three trials assessed the effects on improving professional practice of participation by healthcare providers in conferences, lectures, workshops or traineeships.

  • A synthesis of the trial results provided evidence of low quality that educational meet-ings improve patient care.

General management of a clinical problem, prescribing, prevention services, or test ordering
Patient or population: Physicians
Settings: Primary or inpatient care in the USA (1), UK (1), Netherlands (1)
Intervention: Educational meetings
Comparison: No intervention
Outcomes Impact Number of participants
(studies)		 Quality of the evidence
(GRADE)
Dichotomous process measures
 Median effect: +1% absolute improvement

(1 study)
Continuous process measures
 Median effect: +27% relative improvement
 (1 study)
Continuous outcome measure Median effect: -3.6% relative deterioration (1 study)

GRADE: GRADE Working Group grades of evidence (see above and last page)

 

3) Audit and feedback

Eight trials evaluated the effects of audit and feedback on improving professional practice, including for the general management of a clinical problem, prevention services, test ordering, and/or discharge planning.

  • The trials showed evidence of moderate quality that audit and feedback leads to significant improvements in patient care.

General management of a clinical problem, prevention services, test ordering, and discharge planning.
Patient or population: Physicians
Settings: Primary or inpatient care in the USA (7), and UK (1)
Intervention: Audit and feedback
Comparison: No intervention
Outcomes Impact Number of participants
(studies)		 Quality of the evidence
(GRADE)
Dichotomous process measures
 Median effect: +7.0% (range +1.3 to +16.0%) absolute improvement in performance

(5 studies)
Continuous process measures
 Median effect: +15.4% (range 0 to +20.3%) relative improvement in performance
 (3 studies)
GRADE: GRADE Working Group grades of evidence (see above and last page)

 

2) Educational meetings
Three trials assessed the effects on improving professional practice of participation by healthcare providers in conferences, lectures, workshops or traineeships. 
A synthesis of the trial results provided evidence of low quality that educational meet-ings improve patient care.

4) Patient-mediated interventions

Seven trials evaluated the effects of patient-mediated interventions on improving professional practice for prevention services and the general management of a clinical problem.

  • The studies provided moderate quality evidence that patient-mediated interventions improve professional performance

Prevention services and general management of a clinical problem
Patient or population: Physicians
Settings: Primary or inpatient care in the USA (6) and Canada (1)
Intervention: Patient-mediated interventions (i.e. new clinical information collected directly from patients and given to the provider)
Comparison: No intervention
Outcomes Impact Number of participants
(studies)		 Quality of the evidence
(GRADE)
Dichotomous process measures
 Median effect: +20.8% (range +10.0 to +25.4%) absolute improvement in care

(6 studies)
Continuous process measures
 Median effect: -9.1% (SMD -0.67) relative deterioration in performance
 (1 study)
Continuous outcome measure Median effect: +5.0% (SMD +0.09) relative improvement in performance
 (2 studies)

GRADE: GRADE Working Group grades of evidence (see above and last page)

 

 

5) Reminders

Thirty five trials evaluated the effects of reminders in improving professional practice for prevention services, general management, prescribing, discharge planning, and financial procedures.

  • The studies yielded low to moderate quality evidence that the use of reminders leads to improvements in patient care

Prevention services, general management, prescribing, and discharge planning
Patient or population: Physicians
Settings: Primary or inpatient care in the USA (6), Israel (2), Canada (1) and Thailand (1)
Intervention: Reminders
Comparison: No intervention
Outcomes Impact Number of participants
(studies)		 Quality of the evidence
(GRADE)
Dichotomous process measures
 Median effect: +14.1% (range -1.0 to +34.0%) absolute improvement in care

(31 studies)
Continuous process measures
 Median effect: +5.7% (range -41.8 to +36.0%) relative improvement in care
 (6 studies)

GRADE: GRADE Working Group grades of evidence (see above and last page)

 

6) Multifaceted interventions

A total of 117 studies (including 47 trials) evaluated 68 different combinations of interventions, including 26 combinations of two interventions, 19 combinations of three interventions, 16 combinations of four interventions and seven combinations of five or more interventions. The maximum number of comparisons of the same combination of interventions was 11. These studies provided:

  • Low to moderate quality evidence that multifaceted interventions incorporating educational outreach improve performance.
  • Moderate quality evidence that combinations of reminders and patient-mediated interventions lead to improvement in performance.
  • Moderate quality evidence that combinations of educational materials, educational meetings, and audit and feedback lead to improvement in performance.

General management, prevention, prescribing, and test ordering
Patient or population: Physicians
Settings: Primary care, inpatient care, or mixed settings in the USA (82), UK (16), Canada (6) and Australia (5), 1 each in Mexico and Thailand
Intervention: Multifaceted interventions
Comparison: No intervention
Outcomes Impact Number of participants
(studies)		 Quality of the evidence
(GRADE)
Dichotomous process measures 

[multifaceted interventions incorporating educational outreach]

Median effect: +6.0% (range -4 to +17.4%) absolute improvement in care

(13 studies)
Continuous process measures [multifaceted interventions incorporating educational outreach]

Median effect: +15.0% (range +1.7 to +24.0%) relative improvement in performance

 (5 studies)
Dichotomous process measures [combinations of reminders and patient-mediated interventions]

Median effect: +11.5% (range +1.3 to +20.0%) absolute improvement in performance

 (4 studies)
Dichotomous process measures [combinations of educational materials, educational meetings, and audit and feedback] Median effect: +3.0% (range +2.6 to +9.0%) absolute improvement in performance (3 studies)

GRADE: GRADE Working Group grades of evidence (see above and last page)

 

7) General remark

The differences in effect sizes between various strategies should be interpreted with caution, as the effect sizes might be context-specific. For example, larger effects might be found for single intervention studies because they may target relatively simpler clinical behaviours more amenable to change. Similarly, the finding that higher numbers of interventions are not associated with larger effect sizes for multifaceted interventions may be because such multifaceted interventions were used when more difficult barriers to change were anticipated, or because these interventions did not sufficiently target key factors that influence change.

Relevance of the review for low-income countries

Findings Interpretation*
APPLICABILITY
  • The included trials were mainly from North America and Western Europe, with only two studies from middle-income countries (Mexico & Thailand) and none from low-income countries. The range of study settings and the consistent pattern of findings suggest that the measured effects may be transferable across settings in high- and middle-income countries.
  • Resources available for maintaining and improving quality of care need to be considered when assessing whether the intervention effects are likely to be transferable to other high- and middle-income countries.
  • We need rigorous studies from low-income countries to fully assess applicability to all healthcare settings.
EQUITY
  • The included trials did not provide data regarding differential effects of the interventions for disadvantaged populations.
  • Some dissemination strategies (e.g. reminders and audit and feedback) relied on technologies that may not always be appropriate when attempting to contact practitioners in low-income settings. Implementation of interventions in these settings utilizing such technologies may exacerbate health inequities or fail to address them adequately.
  • Where used appropriately, these strategies may have the potential to improve the delivery of effective care in under-resourced settings.
ECONOMIC CONSIDERATIONS

 

  • The results of the review are based on trials in which the levels of organization and support were potentially higher than those available outside of research settings. Respondents to the key informant survey which accompanied the review rarely identified existing budgets to support guideline dissemination strategies.
  • Decision makers need to use considerable judgement about how best to use the limited resources they have for maintaining and improving the quality of care in order to maximize population benefits. They need to consider the potential areas for clinical effectiveness activities, the likely benefits and costs required to introduce guidelines and the likely benefits and costs as a result of any changes in provider behaviour.
MONITORING & EVALUATION
  • Multiple guideline dissemination strategies were evaluated across a wide range of targeted behaviours and in a wide range of settings. Though the results suggest that various dissemination strategies may have a moderate effect on guideline implementation, information is limited on their effectiveness in low-income settings

 

  • When guideline disemination interventions are implemented in low and middle-income countries, implying different barriers and effect modifiers, robust monitoring and evaluation mechanisms should be put in place to inform better the choice of interventions.
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low and middle-income countries. For additional details about how these judgements were made see: http://www.support-collaboration.org/summaries/methods.htm

Additional information

Related literature

Fairall LR, Zwarenstein M, Bateman ED, Bachmann M, Lombard C, Majara BP, et al. Effect of educational outreach to nurses on tuberculosis case detection and primary care of respiratory illness: pragmatic cluster randomised controlled trial. BMJ 2005;331:750-4.

 

Althabe F, Buekens P, Bergel E, Belizán JM, Campbell MK, Moss N, et al. A be-havioral intervention to improve obstetrical care. N Engl J Med 2008;358:1929-40.

 

Althabe F, Bergel E, Cafferata ML, Gibbons L, Ciapponi A, Alemán A, et al. Strategies for improving the quality of health care in maternal and child health in low- and middle-income countries: an overview of systematic reviews. Paediatr Perinat Epidemiol 2008;22(Suppl 1):42-60.

 

Barosi G. Strategies for dissemination and implementation of guidelines. Neurol Sci 2006;27 (Suppl 3):S231-4.

 

Chaillet N, Dubé E, Dugas M, Audibert F, Tourigny C, Fraser WD, Dumont A. Evidence-based strategies for implementing guidelines in obstetrics: a systematic review. Obstet Gynecol 2006;108:1234-45.

 

Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, Vale L. Toward evidence-based quality improvement. Evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966-1998. J Gen Intern Med 2006;21 (Suppl 2):S14-20.

 

This summary was prepared by

Charles Shey Wiysonge, South African Cochrane Centre, South African Medical Research Council, Cape Town, South Africa

 

Conflict of interest

None declared. For details, see:Conflicts of Interest

 

Acknowledgements

This summary has been peer reviewed by: Martin Eccles, UK; Robbie Foy, UK; Suzanne Kiwanuka, Uganda; Simon Lewin, UK; Tracey Perez Koehlmoos, Bangla-desh

 

This summary should be cited as

Wiysonge CS. Which clinical guideline dissemination strategies improve professional practice? A SUPPORT Summary of a systematic review. August 2008.

 



About certainty of evidence (GRADE)

The “certainty of the evidence” is an assessment of how good an indication the research provides of the likely effect, i.e. the likelihood that the effect will be substantially different from what the research found. By “substantially different” we mean a large enough difference that it might affect a decision. These judgements are made using the GRADE system, and are provided for each outcome. The judgements are based on the study design (randomised trials versus observational studies), factors that reduce the certainty (risk of bias, inconsistency, indirectness, imprecision, and publication bias) and factors that increase the certainty (a large effect, a dose response relationship, and plausible confounding). For each outcome, the certainty of the evidence is rated as high, moderate, low or very low using the definitions on page 3.

For more information about GRADE

SUPPORT collaborators:

The Cochrane Effective Practice and Organisation of Care Group (EPOC) is part of the Cochrane Collaboration. The Norwegian EPOC satellite supports the production of Cochrane reviews relevant to health systems in low- and middle-income countries . www.epocoslo.cochrane.org

The Evidence-Informed Policy Network (EVIPNet) is an initiative to promote the use of health research in policymaking in low- and middle-income countries. www.evipnet.org

The Alliance for Health Policy and Systems Research (HPSR) is an international collaboration that promotes the generation and use of health policy and systems research in low- and middle-income countries. www.who.int/alliance-hpsr

Norad, the Norwegian Agency for Development Cooperation, supports the Norwegian EPOC satellite and the production of SUPPORT Summaries. www.norad.no

The Effective Health Care Research Consortium is an international partnership that prepares Cochrane reviews relevant to low-income countries. www.evidence4health.org

To receive e-mail notices of new SUPPORT summaries or provide feedback on this summary, go to: www.supportsummaries.org/contact

Comments