How SUPPORT Summaries are prepared

We aim to summarise the best and most relevant evidence from systematic reviews of health systems interventions relevant to low-and middle-income countries (LMIC). These summaries extract information from systematic reviews, evaluate the information and present it in a user-friendly manner so that decision-makers can quickly review the evidence and decide whether a particular innovation is likely to be effective in their own context.

Information presented in our summaries is based on systematic reviews because they have several advantages:

• They reduce the risk of bias in selecting and interpreting the results of studies.
• They reduce the risk of being misled by the play of chance in identifying studies for inclusion, or the risk of focusing on a limited subset of relevant evidence.
• They provide a critical appraisal of the available research and place individual studies or subgroups of studies in the context of all of the relevant evidence.
• They allow others to appraise critically the judgements made in selecting studies and the collection, analysis and interpretation of the results.

We have screened thousands of references to identify relevant systematic reviews. In order to be selected for a SUPPORT Summary a review must

• Be relevant to LMIC
• Examine the effects of delivery, financial or governance arrangements, or implementation strategies
• Have a methods section with explicit selection criteria

We have searched the Cochrane Effective Practice and Organisation of Care Group’s database of systematic reviews, which contains reviews from the Cochrane Database of Systematic Reviews, DARE, PubMed, and Google Scholar.

After a review has been identified, two or more reviewers independently assess its relevance based on the abstracts and – when potentially relevant – based on the full text. Reviews that only include studies from high-income countries are included if the interventions examined and the results are considered relevant to LMIC. Next we assess the quality of all included reviews using a checklist and extract data about the focus and main results. The quality of evidence for the main comparisons in each review is graded using the GRADE system. Finally, we consider the applicability of the results to LMIC, impacts on equity, economic considerations, and the need for monitoring and evaluation. Completed summaries are peer-reviewed by policymakers and researchers in LMIC, other people with relevant expertise, and authors of the reviews being summarised.

Judgements about the relevance of a review for low- and middle-income countries

Decisions about how to apply the results of systematic reviews are always located in specific contexts. As such they need to take into account local conditions, needs, values, costs and the local availability of resources. We include a summary of key considerations about the relevance of each review for LMIC. This includes judgements about possible differences between where the research was done and its application in LMIC. For health systems interventions this includes considerations of the following questions: 

• Were the studies included in the systematic review conducted in the same setting or were the findings consistent across settings or time periods?
• Are there important differences in on-the-ground realities and constraints that might substantially alter the feasibility and acceptability of an intervention?
• Are there important differences in health system arrangements that may mean an intervention could not work in the same way?
• Are there important differences in the baseline conditions that might yield different absolute effects even if the relative effectiveness was the same?

We address potential impacts on equity by examining the findings of the review and considering possible differential effects of the interventions for disadvantaged populations. The potential for impacts on equity is considered in relationship to economic status, employment or occupation, education, place of residence, gender, and ethnicity. This includes consideration of the following questions:

• Are there plausible reasons for anticipating differences in the relative effectiveness of the intervention in disadvantaged groups or settings within the country?
• Are there likely to be different baseline conditions across groups or settings within the country, such that that the absolute effectiveness of the intervention would be different, and the problem more or less important, for disadvantaged groups or settings?
• Are there important considerations that should be made when implementing the option in order to ensure that inequities are reduced, if possible, and that they are not increased?

In settings with limited resources, policymakers are deeply concerned about the economic consequences of an intervention. The costs and cost-effectiveness of an intervention need to be assessed in a specific setting in order to take account of differences in patterns of resource use and prices. To help decision-makers to make these assessments, we consider what the most important economic consequences are that need to be considered prior to rolling out or scaling up an intervention or policy. This includes consideration of:

• What are the most important costs or savings, including the costs of implementing and sustaining the intervention?
• What information is there about those costs or savings, either from systematic reviews or other sources, for example economic analyses?
• Is there important uncertainty about the medium- to long-term economic consequences?
• Is there important uncertainty about the applicability of any reported economic consequences?

Finally, because there is often uncertainty about the effects and cost-effectiveness of interventions, we address the need for monitoring and evaluation, including consideration of:

• Are there important uncertainties that should be addressed prior to making a decision?
• What should potentially be monitored and how?
• What should potentially be evaluated and how?

For each SUPPORT Summary we consult people in LMIC regarding these judgements about the relevance of the review for LMIC.