SUPPORT Summaries
Evidence of the effects of health system interventions
for low- and middle-income countries
Evidence of the effects of health system interventions
for low- and middle-income countries
August, 2016 - SUPPORT Summary of a systematic review | print this article | download PDF
Information about the performance of health plans (health insurance or subscription-based medical care), hospitals, and healthcare professionals is increasingly available in the public domain. However, the effects of such public disclosure of performance data are unclear.
Key messages
-May lead to patients selecting health plans that have better quality ratings .
-Has uncertain effects on quality improvement activities.
-May improve health outcomes.
-May lead to little or no difference in patient selection of hospitals.
-Probably stimulates quality improvement activities.
-May improve health outcomes.
-Probably leads to patients selecting providers that have better quality ratings.
-Has uncertain effects on quality improvement activities.
-May improve health outcomes.
-Public disclosure of performance may be difficult to implement in low-income countries because of limitations of the ability of health facilities and providers to produce accurate data, the capacity to disseminate the data, the ability of patients to interpret the data and, in some places, patients not having multiple facilities or providers from which to choose.
People making decisions concerning public disclosure of patient care performance data.
Fung CH, Lim YW, Mattke S, et al. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008; 148:111-23.
A summary of studies addressing a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise the relevant research, and to collect and analyse data from the included studies.
SUPPORT – was an international project to support the use of policy relevant reviews and trials to inform decisions about maternal and child health in low- and middle-income countries, funded by the European Commission (FP6) and the Canadian Institutes of Health Research.
Public disclosure of performance data has been proposed as a mechanism for improving the quality of care by providing more transparency and by making healthcare providers more accountable. It is assumed that such public disclosures would lead patients to seek care from betterperforming healthcare providers and help healthcare providers to identify areas in which they are deficient and thus improve their performance. Examples of publicly disclosed performance data include mortality rates, patient satisfaction, length of stay, and immunisation coverage.
|
Review objectives: To synthesise the evidence for using public disclosure of performance data to improve healthcare quality |
||
| Type of | What the review authors searched for | What the review authors found |
|---|---|---|
|
Study designs & interventions |
Peer-reviewed articles published between 1986 and 2006. Type of studies not pre-specified. |
2 randomised trials, 2 non-randomised trials, 1 controlled before-after study, 9 interrupted time series studies, and 31 other observational studies. |
|
Participants |
Not pre-specified. |
Hospitals, patients, and hospital staff (45 studies). |
|
Settings |
Not pre-specified. |
USA (43 studies), United Kingdom (1), Canada (1). |
|
Outcomes |
Selection of health plans, hospitals, and individual providers, quality improvement activity, clinical outcomes, unintended consequences. |
Selection of health plans (8 studies), selection of hospitals (9), selection of individual providers (7), quality improvement activity (11), clinical outcomes (11), unintended consequences (13). |
|
Date of most recent search: March 2006. |
||
|
Limitations: Only peer-reviewed, English-language articles were included.. |
||
Fung CH, Lim YW, Mattke S, et al. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008; 148:111-23.
High: This research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different† is low.
Moderate: This research provides a good indication of the likely effect. The likelihood that effect will be substantially different† is moderate.
Low: This research provides some indication of the likely effect. Howevver, the likelihood that it will be substantially different† is high.
Very low: This research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different† is very high.
* This is sometimes referred to as 'quality of evidence' or 'confidence in the estimate'.
† Substantially different = a large enough difference that it might affect a decision.
See last page for more information.
The review identified 45 studies conducted in high-income countries. The results are organised by reporting level (health plans, hospitals, individual providers).
1) Health plans
Ten studies assessed the effects of public disclosure of performance data on health plans and showed that:
|
Effects of public disclosure on health plan quality |
||||
|
People: Users of health insurance plans; health insurance plans |
||||
| Outcomes | Impact | Certainty of the evidence (GRADE) |
||
|
Selection of health plans |
2 studies found no effect of public disclosure on health plan choices, 6 studies found that public disclosure may lead users to select better-rated plans or to avoid lower-scoring ones. |
Low |
||
|
Quality improvement activity Quality
improvement activity
|
No studies of health plan performance data and quality improvement activity were identified. |
- | ||
|
Clinical outcomes |
1 study reported that technical performance measures and patient experience measures were higher for health plans with publicly disclosed data. |
Low |
||
| GRADE: GRADE Working Group grades of evidence (see above and last page) | ||||
2) Hospitals
Eighteen studies assessed the effects of the public disclosure of performance data on hospitals and showed that such public reporting:
|
Effects of public disclosure on quality of care provided in hospitals |
||||
|
People: Patients, hospitals, hospital staff |
||||
| Outcomes | Impact | Certainty of the evidence (GRADE) |
||
|
Selection of hospitals |
3 studies reported that hospitals with better outcomes experienced higher rates of growth in market share, but 6 studies found no association. |
Low |
||
|
Quality |
9 studies reported that public disclosure led to quality improvement activities; however, 2 studies showed only minimal impact on quality improvement activities. |
Moderate |
||
|
Clinical outcomes |
7 studies found decreases in mortality or improvement in other clinical outcomes; however, 4 studies did not find any impact on clinical outcomes. |
Low |
||
| GRADE: GRADE Working Group grades of evidence (see above and last page) | ||||
3) Individual healthcare providers
Eight studies assessed the effects of public disclosure of performance data on individual providers and showed that:
|
Effects of public disclosure on quality of care provided by individual healthcare providers |
||||
|
People: Patients, physicians, or physician groups |
||||
| Outcomes | Impact | Certainty of the evidence (GRADE) |
||
|
Selection of providers |
6 studies reported that public disclosure affected selection of health providers, while 1 study found no association. |
Moderate |
||
|
Quality |
No published studies of the effect of public disclosure of performance data on quality improvement activity among physicians or physician groups were identified by the authors. |
- | ||
|
Clinical outcomes |
1 study showed that risk-adjusted mortality rates for surgeons decreased after the data were released. |
Low |
||
| GRADE: GRADE Working Group grades of evidence (see above and last page) | ||||
| Findings | Interpretation* |
|---|---|
|
APPLICABILITY |
|
|
The studies, all conducted in high-income countries, provided limited evidence of the benefits of public disclosure of performance data. |
|
|
EQUITY |
|
|
In the post-report period of one study, it was noted that patients from more affluent neighborhoods were more likely to be treated by doctors with better quality ratings. Patients from poorer neighborhoods were more likely to be treated by doctors with worse ratings. |
|
|
ECONOMIC CONSIDERATIONS |
|
|
None of the studies examined the costs or cost‑effectiveness of publicly disclosing performance data. |
|
|
MONITORING & EVALUATION |
|
|
There is important uncertainty about the effects of public disclosure of performance data, especially in low-income countries. |
|
|
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low-income countries. For additional details about how these judgements were made see: www.supportsummaries.org/methods |
|
Related literature
Faber M, Bosch M, Wollersheim H, et al. Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care 2009; 47:1-8.
Henderson A, Henderson S. Provision of a surgeon's performance data for people considering elective surgery. Cochrane Database of Systematic Reviews 2010, Issue 11. Art. No.: CD006327.
Ketelaar NABM, Faber MJ, Flottorp S, et al. Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations. Cochrane Database of Systematic Reviews 2011, Issue 11. Art. No.: CD004538.
Kolstad JT, Chernew ME. Quality and consumer decision making in the market for health insurance and health care services. Medical Care Research and Review 2009 ;66(Suppl 1):28S-52S.
This summary was prepared by
Charles Shey Wiysonge, Centre for Evidence-based Health Care, Stellenbosch University, & Cochrane South Africa, South African Medical Research Council, Cape Town, South Africa.
Conflict of interest
None declared. For details, see: www.supportsummaries.org/coi
Acknowledgements
This summary has been peer reviewed by: Constance H. Fung, Hanna Bergman, and a reviewer who prefers to remain anonymous.
This review should be cited as
Fung CH, Lim YW, Mattke S, et al. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008; 148:111-23.
The summary should be cited as
Wiysonge CS. Does public disclosure of performance data improve quality of healthcare? A SUPPORT Summary of a systematic review. August 2016. www.supportsummaries.org
Keywords
evidence-informed health policy, evidence-based, systematic review, health systems research, health care, low and middle-income countries, developing countries, primary health care, public release of performance data.
This summary was prepared with additional support from:
The University of Cape Town, South Africa
The University of Cape Town aspires to become a premier academic meeting point between South Africa, the rest of Africa, and the world. Taking advantage of expanding global networks and our distinct vantage point in Africa, we are committed, through innovative research and scholarship, to grapple with the key issues of our natural and social worlds.
