November, 2010 - SUPPORT Summary of a systematic review | print this article |

What are the impacts of health sector accreditation?

Accreditation can provide an indication of the quality of organisational performance. Despite the substantial costs, many health care organisations participate in some form of accrediation process. However, the evidence base for accreditation is incomplete.

 

Key messages

  • The most consistent benefits of accreditation were the promotion of institutional change and professional devel-opment
  • Only very low-quality evidence was available, mostly from high-income countries
  • Inconsistent findings or incomplete evidence were identified in the attitudes of professionals to accreditation, the organ-isational impact of accreditation, its financial impact, quality measures as clinical performance measures , programme as-sessment, accreditation impacts on consumer views or pa-tient-satisfaction ratings, the effect of public accreditation disclosure, and surveyor-related issues in accreditation proc-esses
  • Decisions about implementing health sector accreditation must be guided by pragmatic factors such as institutional circumstances or needs, feasibility and the associated costs of accreditation programmes.

Background

Institutional accreditation is based on the application of nationally and internationally agreed standards for assessing and benchmarking the performance of organisations. This involves certification by an external body, often following formalised visits by peer assessors or surveyors. ‘Accreditation’ is not a single technology but a cluster of activities which interact to produce documented processes and organisational changes. The primary aim of certification is to identify poor, satisfactory or exemplary performance. It is also used to indicate which organisations deliver products or services that are acceptable to consumers, funders and stakeholders.

Accreditation is an accepted and important element of quality improvement activities very sensitive to consumers who want to improve the safety and quality of health care services. Nevertheless, the evidence base for the usefulness of accreditation is thought to be incomplete. The need to undertake research in this area is of considerable public interest given that accreditation is frequently used and the cost of accreditation processes.



About the systematic review underlying this summary

Review Objectives: To assess the impact or effectiveness of accreditation programmes/What the review authors searched forWhat the review authors foundInterventionsStudies centred on how accreditation works, what it does, the results achieved and accreditation surveyors and the processes they used
66 studies were included. 4 descriptive studies were based in LMICs: Zambia (1), India (1), Thailand (1), and South Africa (1). The remainder were located in high-income countries (HICs)
Participants

- Health care centres
- Health care staff
- Stakeholders

Doctors, health and laboratory managers, allied health deans, medical technologists, nurses, carers, quality co-ordinators, dentistry practitioners and programme directors, hospital administrators, governmental officials and insurance representatives
SettingsUnrestricted
Included both rural and urban locations of differ-ent sizes, mostly in HICs
OutcomesImpacts on institutional control, organisational development, professional regulation, financial allocation, and public accountability
The outcomes assessed in the 66 studies included in the review were: professional attitudes to ac-creditation (17 studies), promoting change (4), or-ganisational impact (4), financial impact (6), quality measures (18), programme assessment (13), con-sumer views on patient satisfaction (4), public dis-closure (1), professional development (4) and sur-veyor issues (3)
Date of most recent search: May 2007Limitations: Limitations: This is a moderate quality systematic review with serious limitations. The search strategy was ap-propriate but no mention was made of selection processes, data extraction or methodological quality assess-ment. No description of study designs was included because of the diversity of accreditation programme meth-ods used in the studies. Only studies written in English were included.

Greenfield D, Braithwaite J. Health sector accreditation research: a systematic review. Int J Qual Health Care 2008;20:172-83. 
See in Publisher Site

Summary of findings

No numerical data were reported, so we describe the key findings in each outcome category below.

 

1) Attitudes to accreditation in professions

A total of 17 studies assessed this outcome. Health professionals supported accreditation programmes (6 studies) or were in agreement about their respective accreditation standards (6 studies) because they consider accreditation as an effective strategy for assuring quality resulting in better organizational performance. Participants from all the professions examined viewed accrediation programmes positively except in 2 studies that assessed the views of doctors on hospital accreditation programmes.

Accreditation programmes were viewed positively for the following reasons: they were seen as an effective strategy for assuring quality, better organisational performance, facilitating collegial decision making, and as a way of providing a guide to external stakeholders in terms of how quality and safety are managed within an organisation. Every study – except one – did not examine the impact of the programmes.

Nine studies indicated that health professionals had concerns regarding accreditation programmes in terms of their bureaucracy and the amount of time required. These programmes were perceived as adding little value to patient care (2 studies), to be expensive (4 studies), to vary by assessor (1 study) and to have problems with accreditation standards (3 studies).

A study in India identified caution amongst stakeholders regarding the introduction of a proposed accreditation programme.

Professionals from rural health services failed to participate in an accreditation programme due to the costs of the programme (1 study) or the difficulty of meeting standards and collecting data (1 study).

 

2) Promoting change

Four studies showed favorable results of accreditation programmes.

Tewnty-three accredited hospitals in Australia, compared to not accredited ones, showed significant change in six areas, most notably in nursing organisation and safety. Another study showed that an university hospital center in France changed policy and decision-making behaviours and introduced a continuous quality improvement programme.

Similarly, participating in an accreditation programme and a randomised clinical trial led to significant improvements in both the dissemination and quality of clinical guidelines. A review of the development of several accreditation programmes produced changes on both individual organisations and at a system level. Accreditation diffused into the health care systems of different countries, being adapted to meet the wider policy needs of different national circumstances.

 

3) Organisational impact

Four studies assessed this outcome. Enhancements to patient care were reported in one study as a result of three organisational strategies introduced following participation in an accreditation programme. The strategies were: a patient communication strategy, an evaluation strategy, and a quality improvement strategy. A participative management style and organisational support for the accreditation process affected the outcome positively. One study found no differences in the organization and delivery of cognitive rehabilitation therapy. between accredited and non-accredited programmes, while another study found improved organisationl outcomes when a health service was accredited. A review of accredited hospitals in France showed a non-statistically significant trend of larger hospitals receiving more numerous recommendations with reservations or major reservations.

 

4) Financial impact

Six studies assessed this outcome. Accreditation preparation costs accounted for the majority of the total expenditure. Preparation for accreditation was seen as labour intensive, particularly in the final months prior to the survey. Three studies judged the costs to be high for individual organisations and questioned whether accreditation was an appropriate use of resources. One study examining an accreditation programme in a developing country (Zambia), found that the overall financial viability of the programme and costs for individual organisations were unsustainable. Another study noted that the costs incurred in participating in accreditation should be viewed as an essential investment.

 

5) Quality measures

Eighteen studies of quality measurement assessed items that had been defined as clinical indicators, quality indicators or as clinical performance measures. Conflicting findings hold in comparing accredited and non-accredited hospital quality indicator performance.

In general, no relationship was found between specified measures of quality and an accreditation outcomes (4 studies). Quality indicators did not differ between hospitals that voluntarily participated with quality improvement initiatives and those that did not (1 study). Another study found no difference between accredited hospitals, non-accredited hospitals and nursing homes in terms of medication’s administration errors. One study showed improved compliance with accreditation standards but no effect on clinical indicator performance. Another found that the effects of quality improvement activities atributted to the process of accreditation were often small and inconsistent. It has been argued that different quality measures, developed and implemented in different ways, should not be expected to result in similar outcomes.

One study revealed that accredited hospitals performed better on a range of quality indicators than non-accredited hospitals, albeit with considerable variations in performance within the accredited hospitals.

While quality measures are not always an essential part of accreditation programmes, some have been shown to improve care outcomes in health organisations (8 studies). Similarly, participation in accreditation programmes and at he same time in a randomised clinical trial promoted improvement in a quality measure, in this case a clinical guideline.

 

6) Programme assessment

Thirteen studies assessed this outcome and the findings were inconsistent. Accreditation programmes in 6 studies were deemed to be credible. In other instances, the validity of accreditation programmes was questioned and the authors argued for the need for the improvement and clarification of standards (4 studies). In one study, the use of an accreditation programme as a measure of patient safety was questioned, due to a failure to detect an error-prone medication usage system.

Two descriptive studies examined the development or implementation of accreditation programmes in developing countries. An accreditation programme for hospitals in South Africa was deemed to be beneficial but in Zambia was regarded as unsustainable.

In the United States of America (USA), some have argued in favour of the use of specialised organisations for setting standards and undertaking accreditation programmes for healthcare delivery (1 study). A study of the difficulties experienced by an accrediting organisation in the United Kingdom suggested that there was an imbalance between the setting of standards and their implementation.

 

7) Consumer views or patient satisfaction

Four studies found no relationship between accreditation and either consumer views or patient satisfaction. An examination of the relationship between not-for-profit hospital accreditation scores and patient satisfaction ratings found no association. Similarly, it was not possible to differentiate between patient-reported measures of quality and satisfaction in accredited and non-accredited health plans (1 study).

The views of patients and health professionals regarding compliance with accreditation standards have been compared. Satisfaction rank-order correlations between the two groups were similar despite differences between the groups in terms of specific details.

 

8) Public disclosure

A study in Japan showed a positive association between accreditation scores and the public disclosure of hospital accreditation reports. Larger, public, or rural hospitals were significantly more likely to disclose their accrediations publicly than were smaller, private, or urban hospitals.

Score measurements for patient-focused care and efforts to fulfil community needs were higher in those hospitals that disclosed their accreditations compared with those that did not. Public disclosure was regarded by the majority of respondents as good for both consumers and hospitals; however, concern was expressed by hospitals regarding [potentially negative?] public reaction to lower accreditation scores. A significant number of hospitals that disclosed their accreditation reports perceived that their public disclosure created incentives for improvement and increased the credibility of hospitals within the community.

 

9) Professional development

Four studies assessed professional development outcomes The association was shown to be positive in 3 studies and negative in 1 study.

One study revealed that health professionals who received training in an accredited education programme were more likely to pass a professional accreditation exam than their colleagues who had received training in a non-accredited programme. An accreditation programme had a small but beneficial impact on the ongoing professional education of medical professionals or had a positive influence on individuals applying for professional organisation membership. In contrast, graduates from accredited nursing education programmes did not perform better than those from non-accredited programmes.

 

10) Surveyor issues

Three studies assessed skills and qualities of surveyors or the challenges they faced when undertaking accreditation surveys. One study, set in Thailand, included an examination of the opinions of surveyors and health professionals and showed that both shared a similar prioritisation of concerns. The surveyors focused more attention on care-related items than health professionals who focused more on multidisciplinary process-related problems associated with the accreditation programme.

There is very low quality of evidence that accreditation programmes:

  • Promote institutional change and professional development
  • Show inconsistent findings in the attitudes of professions to accreditation, organisational impact, financial impact, quality measures and programme assessment.

There is insufficient evidence to draw conclusions regarding the impacts of accreditation programmes on consumer views or patient satisfaction, public disclosure and surveyor issues.

Relevance of the review for low-income countries

FindingsInterpretation*
APPLICABILITY

 

  • Only 4 descriptive studies were from LMICs. Another 3 assessed smaller or rural sites in other settings
  • A study set in a developing country found that the overall financial viability of a programme was unsustainable given the costs for individual organisations

 

  • There are important differences between the implementation of accreditation programmes and the consequences they have on institutional control, organisational development, professional regulation, and financial allocation
  • Decisions about health sector accreditation must be guided by pragmatic factors such as institutional circumstances, feasibility, the costs of programmes, management competence and authority, professional self-regulation, and other variables limiting the transfer of accreditation as a technology in LMICs
  • Levels of organisation and support in LMICs may often be lower than those available in the research settings of more developed countries [correct]
EQUITY
  • A study reported that costs were substantially larger for rural organisations than urban ones. Other significant barriers were the difficulty in meeting accreditation standards and the collection of data
  • Resources needed for accreditation programmes may be less available in disadvantaged and rural populations
ECONOMIC CONSIDERATIONS
  • Cost-effectiveness was not assessed
  • Providing adequate support for accreditation programmes is important to ensure effectiveneness when scaling up
MONITORING & EVALUATION
  • A study from a developing country found that the overall financial viability of the programme concerned was unsustainable for individual organisations
  • Accreditation programmes have some initial positive effects during their first year. Future research is needed to assess the long-term effects of using accreditation in health institutions
  • Research is particulary needed in LMICs. The impact of accreditation programmes should be monitored, including impacts on health and health care utilisation
  • Interrupted time series studies could be an option, if a randomised impact evaluation is not feasible for assessing the effects on health, overall expenditure, and cost effectiveness
*Judgements made by the authors of this summary, not necessarily those of the review authors, based on the findings of the review and consultation with researchers and policymakers in low and middle-income countries. For additional details about how these judgements were made see: http://www.support-collaboration.org/summaries/methods.htm

Additional information

Related literature

Braithwaite J, Greenfield D, Westbrook J, et al. Health service accreditation as a predictor of clinical and organisational performance: a blinded, random, stratified study. Quality and Safety in Health Care 2010;19:14-21.

 

Al Tehewy M, Salem B, Habil I, El Okda S. Evaluation of accreditation program in non-governmental organizations' health units in Egypt: short-term out-comes. Int J Qual Health Care 2009;21:183-9.

 

Groene O, Alonso J, Klazinga N. Development and validation of the WHO self-assessment tool for health promotion in hospitals: results of a study in 38 hospitals in eight countries. Health Promot Int. 2010;25:221–9.

 

Shaw C. Toolkit for Accreditation Programs. In: The International Society for Quality In Health Care CS, East , Melbourne V, Australia, eds. Melbourne, Australia; 2004. Available at: http://ps4h.org/docs3/Shaw%202005%20toolkit.pdf

 

This summary was prepared by

Agustín Ciapponi, and Sebastián García Martí, Argentine Cochrane Centre IECS -Institute for Clinical Effectiveness and Health Policy- Iberoamerican Cochrane Network, Argentina.

 

Conflict of interest

None declared. For details, see: Conflicts of Interest

 

Acknowledgements

This summary has been peer reviewed by: Anonymous, United Kingdom (UK); David Greenfield, Australia.

 

This summary should be cited as

Ciapponi A, García Martí S. What are the impacts of health sector accreditation? A SUPPORT Summary of a systematic review. March 2009.



Comments