May, 2015 - SUPPORT Summary of a systematic review | print this article |

Do on-screen, point of care computer reminders improve the processes of care and clinical outcomes?

Gaps between recommended practice and routine care are widely known. Interventions designed to close these gaps while providers make decisions, such as point of care computer reminders, offer a promising strategy.

Key messages

On-screen, point of care computer reminders may slightly improve process outcomes
On-screen, point of care computer reminders may slightly improve clinical outcomes
None of the included studies were conducted in a low-income country

Who is this summary for?

People deciding wether to introduce health information technology into practice.

This summary includes:

Key findings from research based on a systematic review
Considerations about the relevance of this research for low income countries

Not included:

Recommendations
Additional evidence not included in the systematic review
Detailed descriptions of interventions or their implementation

This summary is based on the following systematic review:

Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane database of systematic reviews. 2009 (3):CD001096. PubMed PMID: 19588323.

What is a systematic review?

A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.

SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.

Glossary of terms used in this report

Background references on this topic

Background

Reminders that are embedded into electronic medical records or order entry systems (e.g. for diagnostic tests), can automatically alert physicians and other health care providers about clinical information relevant to the specific clinical task he/she is about to perform. These “point of care” reminders are of great interest to those involved in quality improvement efforts because of their likely low marginal cost, and because they can address multiple topics. This review did not include other types of reminders, e.g. letters sent to physicians to remind them to follow up specific patients.

As for most other quality improvement interventions, reminders primarly focus on improving processes of care, such as prescribing of drugs in accordance with clinical practice guidelines or encouraging smokers to stop. The main goal is, of course, to improve clinical outcomes, i.e. improve the patients’ health.

How this summary was prepared

After searching widely for systematic reviews that can help inform decisions about health systems, we have selected ones that provide information that is relevant to low-income countries. The methods used to assess the reliability of the review and to make judgements about its relevance are described here

Knowing what’s not known is important

A reliable review might not find any studies from low-income countries or might not find any well-designed studies. Although that is disappointing, it is important to know what is not known as well as what is known.

A lack of evidence does not mean a lack of effects. It means the effects are uncertain. When there is a lack of evidence, consideration should be given to monitoring and evaluating the effects of intervention, if it is used.

About the systematic review underlying this summary

Review objectives: : To evaluate the effects on processes and outcomes of care attributable to on-screen computer reminders delivered to clinicians at the point of care.
Study designs & interventions	Randomised and quasi-randomised trials assessing on-screen, point of care computer reminders.	28 studies included. Four studies contained two compari-sons, resulting in 32 included comparisons (6 from quasi-randomised trials) Type of reminder: specific (n=18)/generic (n=9); active (n=28) / passive (n=4) mode of delivery; deliv-ered via CPOE (n=14)/ No CPOE (n=18).
Participants	Studies in which the majority of providers (> 50%) consisted of physicians or physician trainees	Outpatient (24 comparisons) and inpatient (8 comparisons) health care providers.
Settings	Points of care that could deliver computer reminder to clinicians at the time they are engaged in the target activity of interest.	19 comparisons came from the United States and 13 from United Kingdom, Italy, Norway, Australia, Canada, New Zea-land, the Netherlands
Outcomes	Process outcomes: percentage of patients receiving a target recommended process of care, duration of antibiotic therapy or time to respond to a lab value. Clinical outcomes: endpoints as death or de-velopment of a pulmonary embolism, and in-termediate endpoints, such as achievement of a target blood pressure or serum cholesterol level, or mean blood pressure or cholesterol level.	All process outcomes (N = 32) Prescription of medications (N = 21) Prescription of recommended vaccines (N = 6) Test ordering (N = 13) Elements of recommended documentation (N = 3) Other process outcomes (N = 7; i.e. composite compliance with a guideline). Clinical outcomes (N = 8; i.e. target and mean blood pres-sure, cholesterol targets, pulmonary embolism, and mortali-ty)
Date of most recent search: July 2008
Limitations: This is a well-conducted systematic review with minor limitations.

Review objectives: : To evaluate the effects on processes and outcomes of care attributable to on-screen computer reminders delivered to clinicians at the point of care.

Type of

What the review authors searched for

What the review authors found

Study designs & interventions

Randomised and quasi-randomised

trials assessing on-screen, point of care computer reminders.

28 studies included. Four studies contained two compari-sons, resulting in 32 included comparisons (6 from quasi-randomised trials) Type of reminder: specific (n=18)/generic (n=9); active (n=28) / passive (n=4) mode of delivery; deliv-ered via CPOE (n=14)/ No CPOE (n=18).

Participants

Studies in which the majority of providers (> 50%) consisted of physicians or physician trainees

Outpatient (24 comparisons) and inpatient (8 comparisons) health care providers.

Settings

Points of care that could deliver computer reminder to clinicians at the time they are engaged in the target activity of interest.

19 comparisons came from the United States and 13 from United Kingdom, Italy, Norway, Australia, Canada, New Zea-land, the Netherlands

Outcomes

Process outcomes: percentage of patients receiving a target recommended process of care, duration of antibiotic therapy or time to respond to a lab value.

Clinical outcomes: endpoints as death or de-velopment of a pulmonary embolism, and in-termediate endpoints, such as achievement of a target blood pressure or serum cholesterol level, or mean blood pressure or cholesterol level.

All process outcomes (N = 32)

Prescription of medications (N = 21)

Prescription of recommended vaccines (N = 6)

Test ordering (N = 13)

Elements of recommended documentation (N = 3)

Other process outcomes (N = 7; i.e. composite compliance with a guideline).

Clinical outcomes (N = 8; i.e. target and mean blood pres-sure, cholesterol targets, pulmonary embolism, and mortali-ty)

Date of most recent search: July 2008

Limitations: This is a well-conducted systematic review with minor limitations.

About the certainty of the  evidence (GRADE)

High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.

Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.

Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.

Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.

* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.

† Substantially different = a large enough difference that it might affect a decision.

See last page for more information.

Summary of findings

The findings from twenty-eight included studies showed that computer reminders achieved small to modest improvements in both process and clinical outcomes. Surprisingly, the effect was not higher in the studies where reminders were used in combination with other quality improvement interventions.

On-screen, point of care computer reminders may

Slightly improve process outcomes. The certainty of this evidence is low.
Slightly improve clinical outcomes. The certainty of this evidence is low.

Improvements in process adherence and clinical outcomes

Patient or population: Health care providers (> 50%) consisted of physicians

Settings: Ambulatory care or hospital centres

Intervention: On-screen, point of care computer reminders

Comparison: Usual care

Outcomes

Impact

Certainty of the evidence
(GRADE)

Adherence to process outcomes

All process outcomes: 4.2% (0.8% to 18.8%)

Prescription of medications: 3.3% (0.5% to 10.6%)
Prescription of recommended vaccines: 3.8% (0.5% to 6.6%)
Test ordering: 3.8% (0.4% to 16.30%)
Elements of recommended documentation: 0.0% (-1.0% to 1.3%)
Other process outcomes: 1.0% (0.8% to 8.5%)

Proportion of patients that improved clinical outcomes

2.5% (1.3% to 4.2%)

Systolic blood pressure: median reduction of 1.0 mmHg (Interquartile range from 2.3 mmHg reduction to 2.0 mmHg increase).

GRADE: GRADE Working Group grades of evidence (see above and last page).

Relevance of the review for low-income countries

Findings	Interpretation*
APPLICABILITY
None of the included studies were conducted in a low-income country 25% of the studies took place in academic centers.	In many low-income country settings computer reminders are irrelevant due to lack of computers and/or electricity etc. Important issues in adopting on-screen, point of care computer reminders when the required hardware is in place, include user acceptance, workflow integration, system maturity and upgrade availability.
EQUITY
The included studies provide no data about differential effects of the intervention in disadvantaged populations	It is uncertain what, if any, impacts computer reminders might have on inequities. Strategies targeting disadvantaged populations might reduce health inequity. However, barriers to establishing computer reminders, as the lack of computers or electronic health records, will likely limit their use in these settings , and thus widen inequities.
ECONOMIC CONSIDERATIONS
The included studies provide no data about cost of the interventions.	It is not possible to estimate the cost-effectiveness of practice facilitation based on the available information – especially not for low-income country settings.
MONITORING & EVALUATION
There were no trials of on screen reminders set in low-income countries. No study characteristic or reminder feature significantly predicted the magnitude of effect.	Further research should try to identify the design features and contextual factors that are associated with larger improvements in provider behaviour The effects of computer reminders, with or without additional interventions, should be rigorously evaluated before scale-up in low-income countries Economic evaluations should be included in future studies of computer reminders.
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see: www.supportsummaries.org/methods

Findings

Interpretation*

APPLICABILITY

None of the included studies were conducted in a low-income country
25% of the studies took place in academic centers.

In many low-income country settings computer reminders are irrelevant due to lack of computers and/or electricity etc. Important issues in adopting on-screen, point of care computer reminders when the required hardware is in place, include user acceptance, workflow integration, system maturity and upgrade availability.

EQUITY

The included studies provide no data about differential effects of the intervention in disadvantaged populations

It is uncertain what, if any, impacts computer reminders might have on inequities. Strategies targeting disadvantaged populations might reduce health inequity. However, barriers to establishing computer reminders, as the lack of computers or electronic health records, will likely limit their use in these settings , and thus widen inequities.

ECONOMIC CONSIDERATIONS

The included studies provide no data about cost of the interventions.

It is not possible to estimate the cost-effectiveness of practice facilitation based on the available information – especially not for low-income country settings.

MONITORING & EVALUATION

There were no trials of on screen reminders set in low-income countries.
No study characteristic or reminder feature significantly predicted the magnitude of effect.

Further research should try to identify the design features and contextual factors that are associated with larger improvements in provider behaviour
The effects of computer reminders, with or without additional interventions, should be rigorously evaluated before scale-up in low-income countries
Economic evaluations should be included in future studies of computer reminders.

*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see:
www.supportsummaries.org/methods

Additional information

Related literature

These systematic reviews provide evidence that could be relevant to understand computer reminders

Arditi C, Rege-Walther M, Wyatt JC, Durieux P, Burnand B. Computer-generated reminders delivered on paper to healthcare professionals; effects on professional practice and health care outcomes. Cochrane database of systematic reviews. 2012;12:CD001175. PubMed PMID: 23235578.

Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decision-support systems: a systematic review. Annals of internal medicine. 2012 Jul 3;157(1):29-43. PubMed PMID: 22751758.

This study describes practical lessons learned from implementing systems in a wide range of challenging environ-ments over the last decade.

Fraser HS, Blaya J. Implementing medical information systems in developing countries, what works and what doesn't. AMIA Annual Symposium proceedings / AMIA Symposium AMIA Symposium. 2010;2010:232-6. PubMed PMID: 21346975. Pubmed Central PMCID: 3041413.

This summary was prepared by

Agustín Ciapponi, Instituto de Efectividad Clínica y Sanitaria, Buenos Aires, Argentina

Conflict of interest

None declared. For details, see: www.supportsummaries.org/coi

Acknowledgements

This summary has been peer reviewed by: Ola Kdouh, Lebanon; Kaveh G. Shojania, Canada

This review should be cited as

The summary should be cited as

Ciapponi A, Do on-screen, point of care computer reminders improve the processes of care and clinical outcomes? A SUPPORT Summary of a systematic review. May 2015. www.supportsummaries.org

Keywords

All Summaries:

evidence-informed health policy, evidence-based, systematic review, health systems research, health care, low-income countries, developing countries, primary health care, decision making, computer-assisted, decision support systems, outcome and process assessment (health care), point-of-care systems, reminder systems.

About certainty of evidence (GRADE)

The “certainty of the evidence” is an assessment of how good an indication the research provides of the likely effect, i.e. the likelihood that the effect will be substantially different from what the research found. By “substantially different” we mean a large enough difference that it might affect a decision. These judgements are made using the GRADE system, and are provided for each outcome. The judgements are based on the study design (randomised trials versus observational studies), factors that reduce the certainty (risk of bias, inconsistency, indirectness, imprecision, and publication bias) and factors that increase the certainty (a large effect, a dose response relationship, and plausible confounding). For each outcome, the certainty of the evidence is rated as high, moderate, low or very low using the definitions on page 3.

For more information about GRADE

SUPPORT collaborators:

The Cochrane Effective Practice and Organisation of Care Group (EPOC) is part of the Cochrane Collaboration. The Norwegian EPOC satellite supports the production of Cochrane reviews relevant to health systems in low- and middle-income countries . www.epocoslo.cochrane.org

The Evidence-Informed Policy Network (EVIPNet) is an initiative to promote the use of health research in policymaking in low- and middle-income countries. www.evipnet.org

The Alliance for Health Policy and Systems Research (HPSR) is an international collaboration that promotes the generation and use of health policy and systems research in low- and middle-income countries. www.who.int/alliance-hpsr

Norad, the Norwegian Agency for Development Cooperation, supports the Norwegian EPOC satellite and the production of SUPPORT Summaries. www.norad.no

The Effective Health Care Research Consortium is an international partnership that prepares Cochrane reviews relevant to low-income countries. www.evidence4health.org

To receive e-mail notices of new SUPPORT summaries or provide feedback on this summary, go to: www.supportsummaries.org/contact

SUPPORT Summaries