846 Akobeng State objectives of the review and outline eligibility criteria /gShoctiabhateemomet abulate chara tal and co t is not signifi Apply eligibility criterig and iustify any exclusions ond rep esents Lhe mond sh Assemble the most co ensive dataset feasible of the reatment group of the no that less diarr OR up.If dia hes the lin ere th veen the groups being g con es not de I)an was statistically significant. the high he hierarchy ce for eta-analysis is a two stage proce The first stage use it is stated to be an ic revicw.Just a 95% (Cl) ch indi ual study ect include odds ratios (R),relative risks ( The validity of the trial methodology .The magnitude and precision of the treatment effect ability of the results to your patient or that.in meta ights a often th he VALIDITY OF TRIAL ly to sample size. Cdaopaeheehsofamcanas ocused research question at the The forest plot ude the ors (wh at int o the 2 shon the th Thi gure is d m 2eonehedkdnhereie alysis dep Description of the forest plot h studies ha the appropriate stud The nain fig the stud which ned the squares represent the odds ratios of the individual studiesrationale for a meta-analysis is that, by combining the samples of the individual studies, the overall sample size is increased, thereby improving the statistical power of the analysis as well as the precision of the estimates of treatment effects.11 Meta-analysis is a two stage process.12 The first stage involves the calculation of a measure of treatment effect with its 95% confidence intervals (CI) for each individual study. The summary statistics that are usually used to measure treatment effect include odds ratios (OR), relative risks (RR), and risk differences. In the second stage of meta-analysis, an overall treatment effect is calculated as a weighted average of the individual summary statistics. Readers should note that, in meta￾analysis, data from the individual studies are not simply combined as if they were from a single study. Greater weights are given to the results from studies that provide more information, because they are likely to be closer to the ‘‘true effect’’ we are trying to estimate. The weights are often the inverse of the variance (the square of the standard error) of the treatment effect, which relates closely to sample size.12 The typical graph for displaying the results of a meta-analysis is called a ‘‘forest plot’’.13 The forest plot The plot shows, at a glance, information from the individual studies that went into the meta-analysis, and an estimate of the overall results. It also allows a visual assessment of the amount of variation between the results of the studies (heterogeneity). Figure 2 shows a typical forest plot. This figure is adapted from a recent systematic review and meta￾analysis which examined the efficacy of probiotics compared with placebo in the prevention and treatment of diarrhoea associated with the use of antibiotics.14 Description of the forest plot In the forest plot shown in fig 2, the results of nine studies have been pooled. The names on the left of the plot are the first authors of the primary studies included. The black squares represent the odds ratios of the individual studies, and the horizontal lines their 95% confidence intervals. The area of the black squares reflects the weight each trial contributes in the meta-analysis. The 95% confidence intervals would contain the true underlying effect in 95% of the occasions if the study was repeated again and again. The solid vertical line corresponds to no effect of treatment (OR = 1.0). If the CI includes 1, then the difference in the effect of experimental and control treatment is not significant at conventional levels (p.0.05).15 The overall treatment effect (calculated as a weighted average of the individual ORs) from the meta-analysis and its CI is at the bottom and represented as a diamond. The centre of the diamond represents the combined treatment effect (0.37), and the horizontal tips represent the 95% CI (0.26 to 0.52). If the diamond shape is on the Left of the line of no effect, then Less (fewer episodes) of the outcome of interest is seen in the treatment group. If the diamond shape is on the Right of the line, then moRe episodes of the outcome of interest are seen in the treatment group. In fig 2, the diamond shape is found on the left of the line of no effect, meaning that less diarrhoea (fewer episodes) was seen in the probiotic group than in the placebo group. If the diamond touches the line of no effect (where the OR is 1) then there is no statistically significant difference between the groups being compared. In fig 2, the diamond shape does not touch the line of no effect (that is, the confidence interval for the odds ratio does not include 1) and this means that the difference found between the two groups was statistically significant. APPRAISING A SYSTEMATIC REVIEW WITH OR WITHOUT META-ANALYSIS Although systematic reviews occupy the highest position in the hierarchy of evidence for articles on effectiveness of interventions,8 it should not be assumed that a study is valid merely because it is stated to be an systematic review. Just as in RCTs, the main issues to consider when appraising a systematic review can be condensed into three important areas8 : N The validity of the trial methodology. N The magnitude and precision of the treatment effect. N The applicability of the results to your patient or population. Box 1 shows a list of 10 questions that may be used to appraise a systematic review in all three areas.16 ASSESSING THE VALIDITY OF TRIAL METHODOLOGY Focused research question Like all research reports, the authors should clearly state the research question at the outset. The research question should include the relevant population or patient groups being studied, the intervention of interest, any comparators (where relevant), and the outcomes of interest. Keywords from the research question and their synonyms are usually used to identify studies for inclusion in the review. Types of studies included in the review The validity of a systematic review or meta-analysis depends heavily on the validity of the studies included. The authors should explicitly state the type of studies they have included in their review, and readers of such reports should decide whether the included studies have the appropriate study design to answer the clinical question. In a recent systematic review which determined the effects of glutamine supple￾mentation on morbidity and weight gain in preterm babies the investigators based their review only on RCTs.17 State objectives of the review and outline eligibility criteria Comprehensively search for trials that seem to meet eligibility criteria Tabulate characteristics of each trial identified and assess its methodological quality Apply eligibility criteria and justify any exclusions Assemble the most comprehensive dataset feasible Analyse results of eligible RCT's using statistical synthesis of data (meta-analysis) if appropriate and possible) Compare alternative analyses if appropriate and possible Prepare a critical summary of the review, stating aims, describing materials, and reporting results Figure 1 Methodology for a systematic review of randomised controlled trials.9 846 Akobeng www.archdischild.com