SUPPORT Summaries
Evidence of the effects of health system interventions
for low- and middle-income countries
Evidence of the effects of health system interventions
for low- and middle-income countries
November, 2016 - SUPPORT Summary of a systematic review | print this article | download PDF
External inspection systems are used in healthcare to improve adherence to quality standards. They are intended to promote changes in organizational structures or processes, in healthcare provider behavior and consequently in patient outcomes.
Key messages
A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.
SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.
An external inspection has been defined as “a system, process or arrangement in which some dimensions or characteristics of a healthcare provider organisation and its activities are assessed or analysed against a framework of ideas, knowledge, or measures derived or developed outside that organisation”. It is used within healthcare settings to promote improvements in the quality of care, changes in organizational structures or processes, in healthcare provider behavior and thereby in patient outcomes.
| Review objectives:To evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes. | ||
| Type of | What the review authors searched for | What the review authors found |
|---|---|---|
| Study designs & interventions | Randomised trials, non-randomised trials, interrupted time series studies and controlled before-after studies evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes |
1 cluster randomised trial conducted in South Africa and 1 before-after study reanalysed as an interrupted time series study, conducted in England. The study in South Africa assessed the effects of external inspection on compliance with hospital accreditation standards. The study conducted in England assessed the effects of the Healthcare Commissions Infection Inspection program on compliance with standards related to healthcare-acquired infections. |
| Participants | Hospitals, primary healthcare organisations and other community-based healthcare organisations containing health professionals |
20 public hospitals in Kwa Zulu province of South Africa, and all acute hospital trusts in England |
| Settings | Any health system |
1 study was conducted in South Africa and 1 in England |
| Outcomes | Measures of healthcare organisational change (e.g. organisational performance, waiting list times, inpatient hospital stay time); measures of healthcare professional behaviour (e.g. referral rate, prescribing rate); measure of patient outcomes (e.g. mortality and condition-specific measures) |
Outcomes assessed in one study were related with adherence to standards in: medical records, patient outcomes such as satisfaction and patient education, and outcomes related with health processes. The other study assessed the rate of hospital-acquired infections. |
| Date of most recent search: May 2011 | ||
| Limitations: This is a well-conducted systematic review with only minor limitations. | ||
Flodgren G, Pomey MP, Taber SA, Eccles MP. Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes. Cochrane Database of Systematic Reviews2011, Issue 11
High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.
Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.
Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.
Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.
* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.
† Substantially different = a large enough difference that it might affect a decision.
See last page for more information.
This review included two studies, one conducted in South Africa and the other in England. The South African study assessed the effects of external inspection on the compliance with national hospital accreditation standards defined by COHSASA (the Council for Health Services Accreditation for South Africa). The study conducted in England assessed the effects of the Healthcare Commissions Infection Inspection Program on compliance with standards for the prevention of healthcare-acquired infections.
1) External inspection of compliance with COHSASA hospital accreditation standards compared with no intervention
The COHSASA accreditation system has 6000 indicators for assessing hospital services. A subgroup of 421 indicators, which were considered as critical quality criteria, was analysed separately. An additional subgroup of 8 indicators of quality of care in hospitals was measured by the study that included: (i) nurses perception of clinical quality, participation and teamwork; (ii) patient satisfaction with care; (iii) patient medication education; (iv) medical records: accessibility and accuracy; (v) medical records: completeness; (vi) completeness of peri-operative notes; (vii) completeness of ward stock medicine labeling and (viii) hospital sanitation.
|
External inspection of compliance with COHSASA hospital accreditation standards |
|||
|
People Health professionals and patients in 20 hospitals Settings Kwa Zulu Region, South Africa secondary care Intervention External inspection of compliance with accreditation standards and performance related to indicators for hospital quality of care Comparison No intervention |
|||
| Outcomes | Impact | Certainty of the evidence GRADE) |
|
|
Compliance with COHSASA accreditation standards |
The total compliance score with the accreditation standards was greater in intervention hospitals compared with control hospitals |
Very low |
|
|
Compliance with COHSASA accreditation standards - subgroup of critical criteria analysed |
The compliance score for a sub-section of predefined critical criteria, deemed crucial for the function of the service elements, was greater in intervention hospitals compared with control hospitals |
Very low |
|
|
Indicators for hospital quality of care |
The performance as measured by indicators of hospital quality of care was similar in intervention hospitals compared with control hospitals |
Very low |
|
|
GRADE: GRADE Working Group grades of evidence (see above and last page) |
|||
2) External inspection of compliance with the ’Code of Practice’ and the law related to healthcare-acquired infections
The Code of Practice and the Healthcare Act, 2006 are used as standards in the Healthcare Commission’s Inspections Programme in England which aims to decrease the number of healthcare-acquired infections. In this study only data on rates of hospital-acquired methicillin-resistant Staphylococcus Aureus (MRSA) infections, one of the most frequent bacteria producing health-acquired infections in hospital setting, had a sufficient number of measurements before and after the intervention to allow re-analysis as an interrupted time series.
|
External inspection of compliance with the ’Code of Practice’ and the law related to healthcare-acquired infections for improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes |
|||
|
People Health professionals and patients in hospitals Settings Hospitals in England Intervention External inspection of compliance with the Code of Practice and the Health Care Act related to healthcare-acquired infections Comparison No external inspection (in the same hospitals prior to the intervention) |
|||
| Outcomes | Impact | Certainty of the evidence GRADE) |
|
|
MRSA infection rate |
Re-analysis of the quarterly reported rate of MRSA cases, as an interrupted time series, did not indicate an impact of the Healthcare Commission’s Infection Inspection Programme. |
Very low |
|
|
GRADE: GRADE Working Group grades of evidence (see above and last page) MRSA: methicillin-resistant Staphylococcus Aureus |
|||
| Findings | Interpretation* |
|---|---|
| APPLICABILITY | |
|
Neither of the two studies included in this review were conducted in a low-income country: one was done in South Africa and the other in England. Both studies assessed the effect of external inspection of compliance with different standards on quality of hospital services, and one on hospital-acquired infection rates. According to the findings in this review, it is uncertain whether external inspection contributes to improving quality of health services in hospital settings. |
External inspection of compliance with standards may have varying acceptability and impact across different healthcare and cultural settings; may involve different components from training to organisational restructuring; and may impact in different ways on consumer and provider satisfaction across different settings. Although quality of care is an objective of care in all health systems, it is not possible to be confident about the applicability of the reported interventions to low-income countries and to settings other than hospital care. |
| EQUITY | |
| The included trials did not provide data regarding differential effects of the interventions for disadvantaged populations. |
The resources needed to implement an external inspection of compliance with standards could affect provision of services in areas with fewer resources for healthcare. However if it resulted in improvements in patient health outcomes, it could lead to more benefits in disadvantaged populations. |
| ECONOMIC CONSIDERATIONS | |
| The studies did not include direct evidence of the costs or cost-effectiveness of external inspection of compliance with care standards in hospital settings. |
The cost of different types of external inspection of compliance with care standards is likely to be highly variable and costs must be estimated based on the specific intervention and local conditions outside of research settings. If external inspection results in improved healthcare behaviours and outcomes, then these programmes might result in cost savings |
| MONITORING & EVALUATION | |
| The certainty of the evidence is very low and no evidence from ambulatory care settings or low-income countries was identified. |
Cluster randomised trials or interrupted time series studies would be the best study designs to assess the effects of these interventions. These studies should assess the effects of external inspection in different settings and consider relevant patient outcomes and use of resources. |
|
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see: |
|
Walsche K, Freeman T, Latham L, Wallace L, Spurgeon P. Chapter 6. The development of external reviews of clinical governance. Clinical governance - from policy to practice. Birmingham, UK: University of Birmingham, Health Services Management Centre, 2000.
Blanca Penaloza, Unit for Health Policy and System Research, Program of Evidence Based Health Care, Pontificia Universidad Catolica de Chile.
None declared. For details, see: www.supportsummaries.org/coi
This summary has been peer reviewed by: Gerd Flodgren and an anonymous referee.
Flodgren G, Pomey MP, Taber SA, Eccles MP. Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes. Cochrane Database of Systematic Reviews 2011, Issue 11.
Penaloza B. Do external inspections of compliance with standards improve quality of care in healthcare organisations? A SUPPORT Summary of a systematic review. November 2016. www.supportsummaries.org
Keywords
evidence-informed health policy, evidence-based, systematic review, health systems research, health care, low and middle-income countries, developing countries, external supervision, quality of health care, compliance with quality standards, hospital quality of care
This summary was prepared with additional support from:
Pontificia Universidad Catolica de Chile.
