August, 2009 - SUPPORT Summary of a systematic review | print this article |

Do printed educational materials have any effect on professional practice and health care outcomes?

Printed educational materials are widely used passive-dissemination strategies to improve knowledge, awareness, attitudes, skills, professional practice, and patient outcomes. Traditionally they are presented in paper format such as monographs, publication in peer-reviewed journals, and clinical guidelines and appear to be the most frequently adopted method for disseminating information.

Key messages

Printed educational materials may lead to little or no difference in professional practice.
The effect of printed educational materials on patient out-comes is uncertain.
Printed educational materials may be optimised if implemented alongside other quality improvement strategies with proven effectiveness.
All the studies included in the review were conducted in high-income settings. Rigorous studies from low- and middle-income countries are needed to fully assess the impact of printed educational materials on professional practice and health outcomes.

Who is this summary for?

People making decisions concerning interventions for improving professional practice and health care outcomes.

This summary includes:

Key findings from research based on a systematic review
Considerations about the relevance of this research for low income countries

Not included:

Recommendations
Additional evidence not included in the systematic review
Detailed descriptions of interventions or their implementation

This summary is based on the following systematic review:

Farmer AP, Légaré F, Turcot L, Grimshaw J, Harvey E, McGowan JL, Wolf F. Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2008, Issue 3. Art. No.: CD004398.
See in Cochrane Library

What is a systematic review?

A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.

SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.

Glossary of terms used in this report

Background references on this topic

Background

Effectiveness of printed educational materials (PEMs) aimed at changing provider behaviour may be influenced by at least four factors: (1) characteristics of the intervention, (2) characteristics of the provider, (3) characteristics of the behaviour that the intervention is trying to change, and (4) characteristics of the organisation and context. Important characteristics of PEMs include: the source of the information, the content and the channel by which it is delivered. There is limited research on which characteristics of PEMs influence clinical practice. Post-dissemination compliance with clinical guidelines may be higher when recommendations are compatible with existing norms and values, are easy to follow, or are supported with evidence and do not require skills and knowledge

This summary is based on a systematic review published in 2008 by Farmer and colleagues on the effects of printed educational materials on professional practice and health care outcomes.

How this summary was prepared

After searching widely for systematic reviews that can help inform decisions about health systems, we have selected ones that provide information that is relevant to low-income countries. The methods used to assess the reliability of the review and to make judgements about its relevance are described here

Knowing what’s not known is important

A reliable review might not find any studies from low-income countries or might not find any well-designed studies. Although that is disappointing, it is important to know what is not known as well as what is known.

A lack of evidence does not mean a lack of effects. It means the effects are uncertain. When there is a lack of evidence, consideration should be given to monitoring and evaluating the effects of intervention, if it is used.

About the systematic review underlying this summary

/	What the review authors searched for	What the review authors found
Review Objectives: To determine the effectiveness of printed educational materials in improving process outcomes (including the behaviour of healthcare professionals) and patient outcomes.
Interventions	Randomised controlled trials (RCTs), con-trolled clinical trials (CCTs), controlled be-fore and after studies (CBAs), and inter-rupted time series analyses (ITS) assess-ing the effects of printed educational materials such as clinical practice guidelines, journals, and monographs.	23 studies: 12 randomised controlled trials, one con-trolled before and after study, and 10 interrupted time series analyses.
Participants	Any health care professionalsNewborn infants with suspected late onset sepsis, Any health care professionals.	Mostly physicians.
Settings	Studies originating from any setting.	United States (7), United Kingdom (7), Canada (7), Nether-land (1) and Denmark (1). General family or community-based practice (14), managed be-havioural healthcare organisation (1), obstetrics (2), and hospitals (4).
Outcomes	Any objective measure of performance (such as number of tests ordered) or patient health outcomes.	Prescribing behaviour (12 studies), prevention and general man-agement conditions (6 studies), test ordering (3 studies), surgical rates (2 studies).
Date of most recent search: March 2007
Limitations: This is a good quality systematic review with only minor limitations.

About the certainty of the  evidence (GRADE)

High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.

Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.

Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.

Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.

* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.

† Substantially different = a large enough difference that it might affect a decision.

See last page for more information.

Summary of findings

The review included 23 studies conducted in high-income countries: 12 randomised controlled trials, 1 controlled before and after study, and 10 interrupted time series, were included in the review. Only randomised controlled trials are summarised in the table below.

The randomised controlled trials show that:

Printed educational materials may lead to little or no difference in professional performance.
The effect of printed educational materials on patient out-comes is uncertain.

Printed educational materials
Patient or population: Health care professionals Settings: Mostly general practice settings in high-income countries Intervention: Printed educational materials Comparison: No intervention
Outcomes	Adjusted absolute risk difference Median (Interquartile range)*	Number of Participants (studies)	Quality of the evidence (GRADE)
Categorical process outcomes	Median increase of +4.3% (Range -8.0% to +9.6%)	6 studies
Continuous process outcomes	Median increase of +13.6% (Range -5.0% to +26.6%)	4 studies
Categorical patient outcomes	Median -4.3% (Range -4.6% to -0.4%)	3 studies
Continuous patient outcomes	Two studies reported deteriorations in continuous patient outcome data (e.g. depression score, smoking cessation attempts) of -20.5% and -10.0%	2 studies
GRADE: GRADE Working Group grades of evidence (see explanations)
*Across the board for both professional and patient outcomes, the studies are small and there is heterogeneity of both interventions and outcome measures

Relevance of the review for low-income countries

Findings	Interpretation*
APPLICABILITY
The studies included covered a variety of settings in high-income countries. There were no included studies from low-income settings.	The applicability of the findings to low-income countries is limited because of the marked differences in health systems between the two settings.
EQUITY
No information was provided regarding differential effects of the interventions for disadvantaged populations.	Resources needed for printed educational materials may be less available in disadvantaged populations, and use of these materials may exacerbate health inequities.
ECONOMIC CONSIDERATIONS
The studies did not provide information on the costs of printed educational materials used.	The cost of printed educational materials is likely to be highly variable and must be estimated based on specific local conditions outside research settings. Where such data are not available, further primary studies may be needed to inform decision-making. The studies should aim to collect data on costs of resources used as well as implementation costs.
MONITORING & EVALUATION
No information was provided on monitoring and evaluation of use of printed educational materials.	Although printed educational materials are widely used for dissemination purposes in several low- and middle-income countries, rigorous impact evaluations using objective measures of professional practice and patient outcomes should be planned and undertaken prior to continue using this strategy in new initiatives. The cost-effectiveness of printed educational materials in such settings should be evaluated after demonstrating their effectiveness on relevant outcomes.
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low- and middle-income countries. For additional details about how these judgements were made see: http://www.support-collaboration.org/summaries/methods.htm

Additional information

Related literature

Freemantle N,Harvey EL,Wolf F, Grimshaw JM, Grilli R, Bero LA. Printed educa-tional materials: effects on professional practice and health care outcomes. Coch-rane Database of Systematic Reviews 1997, Issue 2. [DOI: 10.1002/14651858. CD000172].

Grimshaw J, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior: an overview of systematic reviews of interventions. Medical Care 2001;39 (Suppl 8):II2–II45.

Ockene J, Zapka J. Provider education to promote implementation of clinical prac-tice guidelines. Chest 2000;118(Suppl 2):33S–39S.

de Vos M, Graafmans W, Kooistra M, Meijboom, van Der Voort P, Westert G. Us-ing quality indicators to improve hospital care: a review of the literature. Interna-tional Journal for Quality in Health Care 2009:21(2); 119-29.

This summary was prepared by

Charles I. Okwundu & Charles Shey Wiysonge, South African Medical Research Council, Cape Town, South Africa

Conflict of interest

None declared. For details, see: Conflicts of Interest

Acknowledgements

This summary has been peer reviewed by: Fernando Althabe, Argentina & Newton Opiyo, Kenya

This summary should be cited as

Okwundu CI, Wiysonge CS. Do printed educational materials have any effects on professional practice and health care outcomes? A SUPPORT Summary of a sys-tematic review. May 2009.

This summary was prepared with additional support from:

The South African Medical Research Council aims to improve health and quality of life in South Africa, through promoting and conducting relevant and responsive health research. www.mrc.ac.za/

The South African Cochrane Centre, the only centre of the inter-national Cochrane Collaboration in Africa, aims to ensure that health care decision making in Africa is informed by high quality, timely and relevant research evidence. www.mrc.ac.za/cochrane/cochrane.htm

About certainty of evidence (GRADE)

The “certainty of the evidence” is an assessment of how good an indication the research provides of the likely effect, i.e. the likelihood that the effect will be substantially different from what the research found. By “substantially different” we mean a large enough difference that it might affect a decision. These judgements are made using the GRADE system, and are provided for each outcome. The judgements are based on the study design (randomised trials versus observational studies), factors that reduce the certainty (risk of bias, inconsistency, indirectness, imprecision, and publication bias) and factors that increase the certainty (a large effect, a dose response relationship, and plausible confounding). For each outcome, the certainty of the evidence is rated as high, moderate, low or very low using the definitions on page 3.

For more information about GRADE

SUPPORT collaborators:

The Cochrane Effective Practice and Organisation of Care Group (EPOC) is part of the Cochrane Collaboration. The Norwegian EPOC satellite supports the production of Cochrane reviews relevant to health systems in low- and middle-income countries . www.epocoslo.cochrane.org

The Evidence-Informed Policy Network (EVIPNet) is an initiative to promote the use of health research in policymaking in low- and middle-income countries. www.evipnet.org

The Alliance for Health Policy and Systems Research (HPSR) is an international collaboration that promotes the generation and use of health policy and systems research in low- and middle-income countries. www.who.int/alliance-hpsr

Norad, the Norwegian Agency for Development Cooperation, supports the Norwegian EPOC satellite and the production of SUPPORT Summaries. www.norad.no

The Effective Health Care Research Consortium is an international partnership that prepares Cochrane reviews relevant to low-income countries. www.evidence4health.org

To receive e-mail notices of new SUPPORT summaries or provide feedback on this summary, go to: www.supportsummaries.org/contact

SUPPORT Summaries