SUPPORT Summaries
Evidence of the effects of health system interventions
for low- and middle-income countries
Evidence of the effects of health system interventions
for low- and middle-income countries
December, 2016 - SUPPORT Summary of a systematic review | print this article | download PDF
Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity (closeness to the truth), results and relevance to an individual’s work. The goal of formal training in critical appraisal skills is to help healthcare workers in understanding results of research studies and their relevance to patient care. This review focused on training for qualified health professionals in practice, and not health professional students.
A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.
SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.
| Review objectives: To assess the effects of teaching critical appraisal skills to health professionals on the process of care, patient outcomes and knowledge of health professionals | ||
| Type of | What the review authors searched for | What the review authors found |
|---|---|---|
| Study designs & interventions | Randomised trials, non-randomised trials, controlled before-after studies, and interrupted time series studies that examined the effectiveness of educational interventions teaching critical appraisal to health professionals |
Three randomised trials of: journal club supported by a half-day workshop (1 study); critical appraisal materials ( a package including papers with methodological reviews), list-serve discussions and articles (1 study); and a half-day workshop based on a Critical Appraisal Skills Programme (CASP) (1 study) |
| Participants | Any qualified healthcare professionals (including managers and purchasers) with direct patient care. Studies involving students were excluded |
Interns (1 study) and physicians (2 studies) |
| Settings | Any clinical setting |
USA, UK and Canada |
| Outcomes | Process of care, patient mortality, morbidity, quality of life and satisfaction |
Knowledge (2 studies) and critical appraisal skills (3 studies) |
| Date of most recent search: June 2011 | ||
| Limitations:This was generally a well-conducted systematic review with only minor limitations. | ||
Horsley T, Hyde C, Santesso N, et al. Teaching critical appraisal skills in healthcare settings. Cochrane Database Syst Rev 2011; 9 (11): CD001270.
High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.
Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.
Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.
Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.
* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.
† Substantially different = a large enough difference that it might affect a decision.
See last page for more information.
The review identified three studies involving 272 participants. None of the included studies were conducted in a low-income country.
|
Teaching critical appraisal for improving process of care variables, patient outcomes and health professionals’ knowledge |
||||
|
Population Health professionals Settings Any setting Intervention Teaching critical appraisal Comparison Usual care |
||||
|
Outcomes |
Effects |
No of participants (studies) |
Certainty of the evidence (GRADE) |
|
|
Process of care |
Not reported |
- |
- |
|
|
Patient-related outcomes |
Not reported |
- |
- |
|
|
Critical appraisal skills Heterogenous measurement scales Follow-up: 0 to 6 months |
Effects on critical appraisal skills were uncertain |
160 (3 studies) |
Very low |
|
|
Knowledge Follow-up: 0 to 6 months |
Critical appraisal training resulted in improvements in knowledge in both studies |
146 (2 studies) |
||
| Findings | Interpretation* |
|---|---|
| APPLICABILITY | |
| None of the included studies were conducted in a low-income country. Interventions assessed included journal club supported by a half-day workshop, critical appraisal materials, listserve discussions and articles and a half-day Critical Appraisal Skills Programme (CASP) workshop |
It is uncertain whether some of the interventions that were evaluated would be feasible in low-income countries. Although impacts on knowledge might be similar, the certainty of this evidence was low, and the certainty of the evidence for other outcomes was very low. |
| EQUITY | |
| The included studies did not report data regarding differential effects of critical appraisal educational interventions (workshops, materials and programmes) across populations with varying socioeconomic status. |
The critical appraisal interventions assessed (workshops, materials, programmes) may require resources and technical skills that are often scarce in low-income settings; this may limit their utilisation and effectiveness in those settings. |
| ECONOMIC CONSIDERATIONS | |
| None of the included studies assessed costs associated with critical appraisal educational interventions. In one study one-off workshops was estimated to cost GBP 250 per participant. |
The potential benefits of critical appraisal educational interventions need to be weighed against the time, efforts and money associated with their implementation on a large scale (and gains from other uses of resources, such as direct investments in patient care). |
| MONITORING & EVALUATION | |
| There was no eligible study on the effects of educational interventions teaching critical appraisal skills in healthcare settings in low-income countries. |
Randomised trials of educational interventions teaching critical appraisal to health professionals in low-income countries are needed. Outcomes assessed should include professional practice, patient outcomes and costs.
|
|
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see: |
|
Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews. PLoS One 2014; 9(1): e86706.
Akl EA, Kairouz VF, Sackett KM, et al. Educational games for health professionals. Cochrane Database Syst Rev 2013; 3: CD006411.
Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ 2004; 329(7473):1017.
Newton Opiyo, KEMRI-Wellcome Trust Research Programme, Kenya.
None declared. For details, see: www.supportsummaries.org/coi
This summary has been peer reviewed by: Chris Hyde, Nancy Santesso and Frode Forland.
Horsley T, Hyde C, Santesso N, et al. Teaching critical appraisal skills in healthcare settings. Cochrane Database Syst Rev 2011; 9 (11): CD001270.
Opiyo N. What are the impacts of teaching critical appraisal skills in healthcare settings? A SUPPORT Summary of a systematic review. December 2016. www.support-collaboration.org/summaries.htm
Keywords
evidence-informed health policy, evidence-based, systematic review, health systems research, health care, information literacy, low and middle-income countries, developing countries, primary health care
