September, 2009 - SUPPORT Summary of a systematic review | print this article |

Do computerized clinical decision support systems have effects on practitioner performance and patient outcomes?

Computerized clinical decision support systems (CDSSs) are information systems designed to improve clinical decision making. Characteristics of individual patients are matched to a computerized knowledge base, and software algorithms generate patient specific recommendations.

These systems provide several modes of decision support, including alerts of critical values, reminders of overdue preventive health tasks, advice for drug prescribing, critiques of existing health care orders, and suggestions for various active care issues.

Key messages

CDSS for diagnosis and reminders for prevention may improve the practitioner performance and may not lead to any difference in the patient outcomes.
Reminder systems probably improve practitioner perform-ance in ambulatory care.
CDSS for disease management may improve the practitioner performance.
CDSS for disease management lead to slightly more im-provements in patient outcome compared to conventional disease management.
CDSS for Drug Dosing and Drug Prescribing may improve the practitioner performance and may not lead to any difference in patient outcomes.
All studies were done in high-income countries, and to establish and use electronic information systems may present challenges in low-income countries where these systems are lacking.

Who is this summary for?

People deciding wether to introduce prompts to physicians in different clinician settings.

This summary includes:

Key findings from research based on a systematic review
Considerations about the relevance of this research for low income countries

Not included:

Recommendations
Additional evidence not included in the systematic review
Detailed descriptions of interventions or their implementation

This summary is based on the following systematic review:

Garg X, Neill A, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB. Effects of Computerized Clinical Decision Support Systems on Practitioner Performance and Patient Outcomes: A Systematic Review. JAMA. 2005;293:1223-1238.

What is a systematic review?

A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.

SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.

Glossary of terms used in this report

Background references on this topic

Background

Developers of health care software have attributed improvements in patient care to these applications. Compared with manual clinical decision support systems automatic systems may improve integration into practitioner workflow as well as provide better opportunities to correct inadvertent deficiencies in health care. Many evaluations have shown that many CDSSs improve practitioner performance. However, further research is needed to elucidate the effects of such systems on patient health. As with any health care innovation, CDSSs should be rigorously evaluated before widespread dissemination into clinical practice. This study evaluated the effectiveness of CDSSs on practitioner performance and patient outcomes and updated earlier reviews from the authors.

How this summary was prepared

After searching widely for systematic reviews that can help inform decisions about health systems, we have selected ones that provide information that is relevant to low-income countries. The methods used to assess the reliability of the review and to make judgements about its relevance are described here

Knowing what’s not known is important

A reliable review might not find any studies from low-income countries or might not find any well-designed studies. Although that is disappointing, it is important to know what is not known as well as what is known.

A lack of evidence does not mean a lack of effects. It means the effects are uncertain. When there is a lack of evidence, consideration should be given to monitoring and evaluating the effects of intervention, if it is used.

About the systematic review underlying this summary

Review Objectives: To review controlled trials assessing the effects of computerized clinical decision support systems (CDSSs) and to identify study characteristics predicting benefit patient outcomes./What the review authors searched forWhat the review authors foundInterventionsRandomized and nonrandomized controlled trials that evaluated the effect of a CDSS compared with care provided without a CDSS on practitioner performance or patient outcomes.

100 trials met the defined criteria. 88% were randomized. Of the randomized trials, 49% were cluster randomized and 40% used a cluster as the unit of analysis or adjusted for clustering in the analysis. Ninety seven trials described the effect of CDSS on at least 1 measure of health care practitioner performance. Fifty-two trials assessed at least 1 patient outcome.

ParticipantsThe population of interest was composed of Physicians and Practitioners in practice or training.
92% of trials enrolled physicians as primary users, 48% enrolled training health care practitioners (in-terns and residents) as users, 34% described pilot testing with users prior to implementation and 42% described user instructional training at the time of implementation.
SettingsAny clinical setting.
76% of trials took place in academic centers, and 33% were inpatient-based. Most of them were conducted in the United States (69%), followed by the United Kingdom (14%), Canada (5%), Australia (4%), Italy(2%), and Austria, France, Germany, Israel, Norway, and Switzerland (1% each).OutcomesThe outcomes assessed were the effect of CDSSs on clinician performance, a meas-ure of the process of care, or patient outcomes.A myriad of outcomes were measured using CDDS for diagnosis, reminder system for prevention, disease management and Drug Dosing and Drug Prescribing.
Date of most recent search: september 2009Limitations: This is a well conducted systematic review. However it is limited by restriction of the searches to English-language studies and the methods used for data analysis.

About the certainty of the  evidence (GRADE)

High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.

Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.

Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.

Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.

* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.

† Substantially different = a large enough difference that it might affect a decision.

See last page for more information.

Summary of findings

The review included 100 studies. All the studies were Randomized and non Randomized Controlled Trials and were done in high-income settings (mostly in the USA).

1) Computerized systems for diagnosis compared to conventional diagnosis

There were 10 trials evaluating diagnostic system in mental health, for acute cardiac ischemia and for a few other conditions. All the studies measured practitioner performance and 5 studies assesed patient outcomes

Computerized systems for diagnosis may improve the practi-tioner performance and may not lead to any difference in the patient outcomes.

Conventional diagnosis

Computerized systems for diagnosis compared to conventional diagnosis
Patient or population: Physicians and Practitioners in practice or training Settings: Ambulatory care, emergency rooms and paediatric and surgical hospital services Intervention: Computerized systems for diagnosis Comparison: Conventional diagnosis
Outcomes	Impact	Number of Participants (studies)	Quality of the evidence (GRADE)	Comments
Practitioner performance	4 out of 10 studies demonstrated the CDSSs was beneficial	10 studies		2 succesfull studies decreased the rate of unnecessary hospital or coronary admission by 15% The third increased mood disorder screening in a posttraumatic stress disorder clinic by 25% The fourth improved the time to diagnosis of acute bowel obstruction by 94%
Patient outcomes	No improvement reported (0%)	5 studies
GRADE: GRADE Working Group grades of evidence (see explanations)

2) Computerized reminder systems for prevention compared to conventional prevention

There were 21 trials evaluating reminder systems in cancer screening, vaccination and preventive care. Performance outcomes were usually rates of screening, counselling, vaccination, testing, medication use or the identification of risk behaviors. All trials measured performance and only one study evaluated patient outcomes.

Computerized reminder systems for prevention may improve practitioners’ performance
Reminder systems probably improve practitioners’ performance in ambulatory care
Computerized reminder systems may not lead to any difference in patient outcomes

Computerized reminder system for prevention compared to conventional prevention
Patient or population: Physicians and Practitioners in practice or training Settings: Ambulatory care and hospital services Intervention: Computerized reminder systems for prevention Comparison: Conventional prevention
Outcomes	Impact	Number of Participants (studies)	Quality of the evidence (GRADE)	Comments
Practitioner performance	16 out of 21 studies demonstrated CDSS was beneficial	21 studies		Post hoc subgroup analyses demonstrated a significant reduction in winter hospitalization and emergency department visits in patients eligible for pneumococcal or influenza vaccination.
Patient outcomes	No improvement reported (0%)	1 studies
GRADE: GRADE Working Group grades of evidence (see explanations)

3) Computarized system for disease management compared to conventional disease management

There were 40 studies of CDSSs for active health conditions including diabetes, cardiovascular prevention and a myriad of different non-classified conditions.

For diabetes care, practitioner performance was usually judged by rates of retinal, foot, urine protein, blood pressure, and cholesterol examinations. In studies of cardiovascular prevention, performance was judged by blood pressure and cholesterol assessment, identification of smoking, and use of cardio protective medications.

Computerized systems for disease management may improve practitioners’ performance
For diabetes management and cardiovascular prevention more precise data findings are provided
Computerized systems for disease management lead to slightly more improvements in patient outcomes

System for disease management compared to conventional disease management
Patient or population: Physicians and Practitioners in practice or training Settings: Ambulatory care, emergency rooms, hospital services and nursing homes Intervention: System for disease management Comparison: Conventional disease management
Outcomes	Impact	Number of Participants (studies)	Quality of the evidence (GRADE)	Comments
Practitioner performance	23 out of 37 studies measuring this outcome improved some measure of practitioner performance	37 studies		For diabetes care 71% of trials reported improvements in studies of cardiovascular prevention 38% of 13 trials reporting improvements
Patient outcomes	Of the 27 trials measuring patient outcomes, 5 (18%) demonstrated improvements.	27 studies		One CDSS improved blood pressure control. A second CDSS reduced urinary incontinence in nursing home residents over a 10-week period. A third CDSS improved scores of barotraumas and organ dysfunction in mechanically venti- lated patients with acute respiratory distress syndrome. A fourth CDSS reduced patient-reported asthma exacerbations, emergency nebulizer use, and the need for additional consultations for asthma management. A fifth CDSS reduced hospital length of stay patients with a variety of general medical diagnoses.
GRADE: GRADE Working Group grades of evidence (see explanations)

4) Computarized system for disease management compared to conventional disease management

Computerized system for Drug Dosing and Drug Prescribing compared to conventional Drug dosing and prescribing

Computerized systems for Drug Dosing and Drug Prescribing may improve practitioners’ performance, but may not lead to any difference in patient outcomes

System for Drug Dosing and Drug Prescribing
Patient or population: Physicians and Practitioners in practice or training Settings: Ambulatory care, emergency rooms and hospital services Intervention: System for Drug Dosing and Drug Prescribing Comparison: Conventional Drug dosing and prescribing
Outcomes	Impact	Number of Participants (studies)	Quality of the evidence (GRADE)	Comments
Practitioner performance	15 out of 24 single drug dosing and 4 out of 5 multiple drug prescribing studies improved some measure of practitioner performance	29 studies		The 24 single-drug dosing systems ranged from a simple calculator for parenteral nutrition to more complex algorithms that considered the pharmacokinetics of warfarin, amino glycosides, or theophylline.
Patient outcomes	2 out of 18 single drug dosing studies improve some measure of patient outcomes.	18 studies		The majority of patients outcomes measured were not improved in these trials.
GRADE: GRADE Working Group grades of evidence (see explanations)

Relevance of the review for low-income countries

Findings	Interpretation*
APPLICABILITY
All studies were conducted in high-income countries All the interventions were very different, as were the populations and settings. 76% of the studies took place in academic Centers. Better performance was identified in studies in which the trial authors also developed the CDSS software. Review included only English-language studies	Important issues in adopting CDSS include user acceptance, workflow integration, compatibility with legacy applications, system maturity and upgrade availability. Availability of computers or elec- tricity and back up generators in resource poor settings that are not on the power grid should be considered. Patient reminders or some mechanism to involve patients, might be used to achieve better preventive care performance. The strict definitions for inclusion and improvement of a CDSS may have underestimated the influence of some system and study methodological factors on CDSS success
EQUITY
The included studies provide no data about differential effects of the intervention in disadvantaged populations.	Strategies and resources explicitly pointed to disadvantaged populations might reduce health inequity in preventive care. However the challenges of establishing CDSS, eg lack of computers especially in LIC that do not routinely use electric records, will limit their use in these settings and may therefore limit access to disadvantaged populations.
ECONOMIC CONSIDERATIONS
The included studies provide no data about cost of the interventions. Most CDSSs used research funding to facilitate implementation. Up to 21% of trials used staff paid by research funds for data entry or CDSS recommendation delivery	Although some studies have assessed the costs when outcomes were improved, the cost effectiveness of these systems remains unknown The cost of CDSS could be higher for a practice without computers or a database, which could be the situation in LMICs. Resources available for implementing CDSS need to be considered when assessing whether the intervention effects are likely to be transferable to settings in low- and middle-income countries. Funding for support personnel and health perssonel training is an additional cost to be considered.
MONITORING & EVALUATION
In this study, effects of CDSS over performance and patient outcomes were assessed.	CDSS with improved technical performance and usability are proliferating in some high- and middle-income countries. Further rigourous evaluations particularly in relation to their effect on patient health and in LMIC are needed. . Economic evaluations should be included in future primary studies about CDSS.
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low and middle-income countries. For additional details about how these judgements were made see: http://www.support-collaboration.org/summaries/methods.htm

Additional information

Related literature

Robert A. Greenes. “Clinical Decision Support. The Road Ahead”. Elsevier, Inc, 2007.

Eta S. Berner. “Clinical Decision Support Systems, 2nd Edition”. Springer-Verlag, December 2006.

Randell R, Mitchell N, Dowding D, Cullum N, Thompson C. “Effects of computerized decision support systems on nursing performance and patient outcomes: a systematic review”. J Health Serv Res Policy. 2007 Oct;12(4):242-9.

Bryan C, Boren SA. “The use and effectiveness of electronic clinical decision support tools in the ambu- latory/primary care setting: a systematic review of the literature”. Inform Prim Care. 2008;16(2):79-91.

Tan K, Dear PR, Newell SJ. “Clinical decision support systems for neonatal care”. Cochrane Database Syst Rev. 2005 Apr 18;(2):CD004211.

Mollon B, Chong J Jr, Holbrook AM, Sung M, Thabane L, Foster G. “Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials”. BMC Med Inform Decis Mak. 2009 Feb 11;9:11.

This summary was prepared by

Gabriel Bastías and Cristian Herrera, Pontificia Universidad Católica de Chile, Chile

Conflict of interest

None declared. For details, see: www.support-collaboration.org/summaries/coi.htm

Acknowledgements

This summary has been peer reviewed by: Fernando Althabe, Argentina, Mike English, Kenya, Tracey Perez Koehlmoos, Bangladesh

This summary should be cited as

Gabriel Bastías and Cristian Herrera. Do the Computerized Clinical Decision Support Systems have effects on Practitioner Performance and Patient Outcomes? A SUPPORT Summary of a systematic review. March 2009. www.support-collaboration.org/summaries.htm

About certainty of evidence (GRADE)

The “certainty of the evidence” is an assessment of how good an indication the research provides of the likely effect, i.e. the likelihood that the effect will be substantially different from what the research found. By “substantially different” we mean a large enough difference that it might affect a decision. These judgements are made using the GRADE system, and are provided for each outcome. The judgements are based on the study design (randomised trials versus observational studies), factors that reduce the certainty (risk of bias, inconsistency, indirectness, imprecision, and publication bias) and factors that increase the certainty (a large effect, a dose response relationship, and plausible confounding). For each outcome, the certainty of the evidence is rated as high, moderate, low or very low using the definitions on page 3.

For more information about GRADE

SUPPORT collaborators:

The Cochrane Effective Practice and Organisation of Care Group (EPOC) is part of the Cochrane Collaboration. The Norwegian EPOC satellite supports the production of Cochrane reviews relevant to health systems in low- and middle-income countries . www.epocoslo.cochrane.org

The Evidence-Informed Policy Network (EVIPNet) is an initiative to promote the use of health research in policymaking in low- and middle-income countries. www.evipnet.org

The Alliance for Health Policy and Systems Research (HPSR) is an international collaboration that promotes the generation and use of health policy and systems research in low- and middle-income countries. www.who.int/alliance-hpsr

Norad, the Norwegian Agency for Development Cooperation, supports the Norwegian EPOC satellite and the production of SUPPORT Summaries. www.norad.no

The Effective Health Care Research Consortium is an international partnership that prepares Cochrane reviews relevant to low-income countries. www.evidence4health.org

To receive e-mail notices of new SUPPORT summaries or provide feedback on this summary, go to: www.supportsummaries.org/contact

SUPPORT Summaries