August, 2016 - SUPPORT Summary of a systematic review | print this article | download PDF

What are the effects of printed educational materials on professional practice and healthcare outcomes?

Printed educational materials (PEMs), including clinical guidelines, monographs and publications in peer-reviewed journals, are common channels to distribute recommendations for clinical care and evidence to inform the practice of healthcare providers. PEMs are used across a range of settings as a strategy to improve professional practice and healthcare outcomes through promoting clinical practices that have been shown to be beneficial and discouraging the use of ineffective interventions. The wide use of PEMs in many settings, particularly in the form of clinical guidelines, is linked to the fact that they are seen as familiar, accessible, relatively inexpensive and convenient.

Key message

When used alone, printed educational materials may slightly improve practice outcomes among healthcare providers, compared to no intervention.

The effects of printed educational materials on patient outcomes are uncertain.

Of the 45 studies included in the review, 44 were from high-income countries. Rigorous studies from low-income countries are needed to assess the impacts of printed educational materials on professional practice in these settings.

Who is this summary for?

People making decisions concerning the use of printed education materials to improve professional practice and healthcare outcomes.

This summary includes:

Key findings from research based on a systematic review
Considerations about the relevance of this research for low income countries

Not included:

Recommendations
Additional evidence not included in the systematic review
Detailed descriptions of interventions or their implementation

This summary is based on the following systematic review:

Giguère A, Légaré F, Grimshaw J et al. Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2012, Issue 10

What is a systematic review?

A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.

SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.

Glossary of terms used in this report

Background references on this topic

Background

Printed educational materials (PEMs) are one of the most common approaches used to support the translation of research findings into clinical practice. PEMs have the potential to improve the care received by patients by promoting clinical practices and interventions of proven benefit and discouraging ineffective practices or interventions. Key questions regarding the use of PEMs to improve professional practice and patient outcomes include: (1) the effects of the use of PEMs compared to no intervention; (2) how the effects of PEMs are influenced by their characteristics, e.g. mode of delivery, source of information, and format; and (3) the role of health systems wide considerations, including, human resource challenges, lack of required inputs and inadequate funding.

How this summary was prepared

After searching widely for systematic reviews that can help inform decisions about health systems, we have selected ones that provide information that is relevant to low-income countries. The methods used to assess the reliability of the review and to make judgements about its relevance are described here

Knowing what’s not known is important

A reliable review might not find any studies from low-income countries or might not find any well-designed studies. Although that is disappointing, it is important to know what is not known as well as what is known.

A lack of evidence does not mean a lack of effects. It means the effects are uncertain. When there is a lack of evidence, consideration should be given to monitoring and evaluating the effects of intervention, if it is used.

About the systematic review underlying this summary

Type of	What the review authors searched for	What the review authors found
Review objectives:To determine the effects of printed educational materials in improving professional practice and patient outcomes.
Study designs & interventions	Randomized trials, non-randomized trials, controlled before-after studies, and interrupted time series studies assessing the effects of printed educational materials, such as clinical practice guidelines, journals, and monographs, delivered personally, through mass mailing or passively via wider channels such as the internet or mass media.	45 studies: 8 cluster-randomized trials, 6 randomized trials, and 31 interrupted time series studies. Most studies (36/45) evaluated a single PEM. Two studies evaluated simultaneously several PEMs (respectively 12 and 11 distinct PEMs) that presented similar characteristics; and three interrupted time series (ITS) studies assessed more than two or three PEMs with very similar characteristics. The 45 studies included the following PEMS: journal publications (n=23), evidence-based guidelines (n=16), newsletters (n=6), summaries of clinical guidelines (n=3) and clinical article reprints (n=1).
Participants	Any type of healthcare professionals.	Physicians, psychologists, psychiatrists, nurses, critical care fellows, Masters-level therapists, and allied health professionals in the field of community health.
Settings	Studies originating from any setting.	Country: Canada (12 studies), United States (11), United Kingdom (11), Spain (1), Belgium (1), The Netherlands (2), Finland (1), Ireland (1), Germany (1), Italy (1), Japan (2), Brazil (1), United States & Canada (1). Healthcare setting: general family or community-based practice (10 studies), outpatient (ambulatory) settings (9), hospitals (6), mixed settings (3), municipal health centre (1), managed behavioural healthcare organisation (1), clinical setting unclear (15).
Outcomes	Any objective measure either of professional practice (e.g. the number of tests ordered, prescriptions for a particular drug) or of patient health outcomes (e.g. blood pressure, complications after surgery).	Prescribing/treatment (39 studies); financial (resource use) (2); general management of a problem (8); diagnosis (4); procedures (7); referrals (4); test ordering (5); surgery (5); patient education/advice (4); clinical prevention service (3); screening (2 ); reporting (1); discharge planning (2); patient health outcome (4).
Date of most recent search: June 2011.
Limitations: This is a well-conducted systematic review with only minor limitations.

Giguère A, Légaré F, Grimshaw J et al. Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2012, Issue 10

About the certainty of the  evidence (GRADE)

High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.

Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.

Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.

Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.

* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.

† Substantially different = a large enough difference that it might affect a decision.

See last page for more information.

Summary of findings

The review included 45 studies, of which 44 were conducted in high-income countries, mostly in outpatient or community settings.

When used alone, printed educational materials may slightly improve practice outcomes among healthcare providers, compared to no intervention. The certainty of this evidence is low.

The effects of printed educational materials on patient outcomes are uncertain because the certainty of this evidence is very low.

Printed educational material compared to no intervention
People: Healthcare professionals (physicians in 9/10 studies) Settings: Multiple settings, mostly general practice settings in high-income countries Intervention: Printed educational materials Comparison: No intervention
Outcomes*	Standard median effect size / impact	Number of participants (studies)	Certainty of the evidence (GRADE)
**Categorical measures of professional practice Absolute risk difference across various outcomes Mean follow-up: 6 months	0.02 higher (range from -0.06 to +0.29)	294,937 (7 studies)	Low
***Continuous measures of professional practice Standardised mean difference across various outcomes Mean follow-up: 9 months	0.13 higher (range from -0.16 to +1.96)	297 (3 studies)	Very Low
Patient outcomes	Very few studies assessed these outcomes and the impact is uncertain	(4 studies)	Very Low
* Where studies reported more than one measure of each endpoint, the primary measure (as defined by the authors of the study) or the median measure was abstracted. For categorical measures, the odds ratio between the intervention of interest and the control intervention was calculated. For continuous endpoints*, standardised mean difference was calculated by dividing the mean score difference of the intervention and comparison groups in each study by the pooled estimate standard deviation for the two groups. GRADE: GRADE Working Group grades of evidence (see above and last page).

Relevance of the review for low-income countries

Findings	Interpretation*
APPLICABILITY
The studies reviewed were mostly from high-income countries. Only one of the 45 included studies was from a middle-income country.	In low-income countries (LICs), where health systems may be weaker than in the settings where these studies were done, it may be difficult to ensure that up-to-date PEMs reach the appropriate providers timeously. It is therefore unclear whether similar effects would be found in LICs. Differences across settings in the extent to which healthcare providers use different PEMs may also affect the impact of these interventions. In some LICs, guidelines may be more accessible and more used than other PEMs. The use of out-of-date PEMs has the potential to cause harm.
EQUITY
The studies included were conducted largely in urban settings No information was reported on the differential effects of the intervention on rural or disadvantaged populations.	In many LIC settings it may be difficult to obtain the resources needed to produce and distribute up-to-date PEMs. Poor access to up-to-date PEMs among healthcare providers in rural or remote areas, in other disadvantaged settings or in settings other than hospitals, may worsen equities in health in populations in these settings.
ECONOMIC CONSIDERATIONS
Information on the cost or cost effectiveness of printed educational materials was not reported.	The costs of producing and effectively disseminating PEMs needs to be estimated in the context in which these interventions are being considered . The costs and effects of PEMs should be compared to alternative strategies to improve professional performance and patient healthcare outcomes, such as audit and feedback interventions and educational meetings for healthcare providers.
MONITORING & EVALUATION
The review did not identify any rigorous evaluations of PEMs from LICs Very few studies have evalauted the effects of PEMs on patient outcomes	Rigorous evaluations of the use of printed educational materials for improvement of professional practice and patient outcomes are required, especially in low income settings. Such studies should evaluate implementation strategies, effects on professional practice and patient outcomes and resource use for different PEMS (guidelines, journal publications, newsletters etc.)
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see: www.supportsummaries.org/methods

Additional information

Related literature

Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2012, Issue 6. Art. No.: CD000259.

O'Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, Forsetlund L, Bainbridge D, Freemantle N, Davis D, Haynes RB, Harvey E. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2007, Issue 4. Art. No.: CD000409.

Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf FM, Davis D, Odgaard-Jensen J, Oxman AD. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2009, Issue 2. Art. No.: CD003030.

This summary was prepared by

Chigozie Jesse Uneke, Faculty of Clinical Medicine, College of Health Sciences, Ebonyi State University Abakaliki, Nigeria.

Conflict of interest

None declared. For details, see: www.supportsummaries.org/coi

Acknowledgements

This summary has been peer reviewed by: Juliet Nabyonga Orem and Anik Giguère

This review should be cited as

Giguère A, Légaré F, Grimshaw J, Turcotte S, Fiander M, Grudniewicz A,Makosso-Kallyth S, Wolf FM, Farmer AP, Gagnon MP. Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Re-views 2012, Issue 10. Art. No.: CD004398. DOI: 10.1002/14651858.CD004398.pub3.

The summary should be cited as

Uneke CJ. What are the effects of printed educational materials on professional practice and healthcare outcomes? A SUPPORT Summary of a systematic review. August 2016. www.supportsummaries.org

Keywords

All Summaries:

Evidence-informed health policy, evidence-based, systematic review, health systems research, healthcare, low and middle-income countries, developing countries, primary healthcare, care healthcare providers, professional practice, patient outcomes.

This summary was prepared with additional support from:

The South African Cochrane Centre, the only centre of the international Cochrane Collaboration in Africa, aims to ensure that health care decision making in Africa is informed by high quality, timely and relevant research evidence. www.mrc.ac.za/cochrane/cochrane.htm

About certainty of evidence (GRADE)

The “certainty of the evidence” is an assessment of how good an indication the research provides of the likely effect, i.e. the likelihood that the effect will be substantially different from what the research found. By “substantially different” we mean a large enough difference that it might affect a decision. These judgements are made using the GRADE system, and are provided for each outcome. The judgements are based on the study design (randomised trials versus observational studies), factors that reduce the certainty (risk of bias, inconsistency, indirectness, imprecision, and publication bias) and factors that increase the certainty (a large effect, a dose response relationship, and plausible confounding). For each outcome, the certainty of the evidence is rated as high, moderate, low or very low using the definitions on page 3.

For more information about GRADE

SUPPORT collaborators:

The Cochrane Effective Practice and Organisation of Care Group (EPOC) is part of the Cochrane Collaboration. The Norwegian EPOC satellite supports the production of Cochrane reviews relevant to health systems in low- and middle-income countries . www.epocoslo.cochrane.org

The Evidence-Informed Policy Network (EVIPNet) is an initiative to promote the use of health research in policymaking in low- and middle-income countries. www.evipnet.org

The Alliance for Health Policy and Systems Research (HPSR) is an international collaboration that promotes the generation and use of health policy and systems research in low- and middle-income countries. www.who.int/alliance-hpsr

Norad, the Norwegian Agency for Development Cooperation, supports the Norwegian EPOC satellite and the production of SUPPORT Summaries. www.norad.no

The Effective Health Care Research Consortium is an international partnership that prepares Cochrane reviews relevant to low-income countries. www.evidence4health.org

To receive e-mail notices of new SUPPORT summaries or provide feedback on this summary, go to: www.supportsummaries.org/contact

SUPPORT Summaries