Understanding systematic reviews and meta-analysis 847 Odds ratic og5 Weiht侧 0.370.161o0.81 0460.18o1.18 12 1.670.47o5.89 020.10o048 0.880.22o3.5 0.230.00 0.580.07o4. Overall 0.370.26o0.521 0.0 Fovo Figure2 probioticson the riskof i ociated diarthoea 4 lack en if relevan ould have vere pooled in a meta the naltbl bna are tially interventions d to al ove.which of glutamine supplementation on should bea of th the when appraising a in the re ults of s I t Did the review stion? 。 Did the reviewers ry oid Did the the of all the studies er of many me a-analyses If the results of the study have been combined,was it tatistical tests. tudy find what are the main Magnitude of the treatment effect to your corfopulaion e as a result of the www.archdishild.com Search strategy used to identify relevant articles There is evidence that single electronic database searches lack sensitivity and relevant articles may be missed if only one database is searched. Dickersin et al showed that only 30–80% of all known published RCTs were identifiable using MEDLINE.18 Even if relevant records are in a database, it can be difficult to retrieve them easily. A comprehensive search is therefore important, not only for ensuring that as many studies as possible are identified but also to minimise selection bias for those that are found. Relying exclusively on one database may retrieve a set of studies that are unrepresentative of all studies that would have been identified through a comprehensive search of multiple sources. Therefore, in order to retrieve all relevant studies on a topic, several different sources should be searched to identify relevant studies (published and unpublished), and the search strategy should not be limited to the English language. The aim of an extensive search is to avoid the problem of publication bias which occurs when trials with statistically significant results are more likely to be published and cited, and are preferentially published in English language journals and those indexed in Medline. In the systematic review referred to above, which examined the effects of glutamine supplementation on morbidity and weight gain in preterm babies, the authors searched the Cochrane controlled trials register, Medline, and Embase,17 and they also hand searched selected journals, cross referencing where necessary from other publications. Quality assessment of included trials The reviewers should state a predetermined method for assessing the eligibility and quality of the studies included. At least two reviewers should independently assess the quality of the included studies to minimise the risk of selection bias. There is evidence that using at least two reviewers has an important effect on reducing the possibility that relevant reports will be discarded.19 Pooling results and heterogeneity If the results of the individual studies were pooled in a meta￾analysis, it is important to determine whether it was reasonable to do so. A clinical judgement should be made about whether it was reasonable for the studies to be combined based on whether the individual trials differed considerably in populations studied, interventions and comparisons used, or outcomes measured. The statistical validity of combining the results of the various trials should be assessed by looking for homogeneity of the outcomes from the various trials. In other words, there should be some consistency in the results of the included trials. One way of doing this is to inspect the graphical display of results of the individual studies (forest plot, see above) looking for similarities in the direction of the results. When the results differ greatly in their direction—that is, if there is significant heterogeneity—then it may not be wise for the results to be pooled. Some articles may also report a statistical test for heterogeneity, but it should be noted that the statistical power of many meta-analyses is usually too low to allow the detection of heterogeneity based on statistical tests. If a study finds significant heterogeneity among reports, the authors should attempt to offer explana￾tions for potential sources of the heterogeneity. Magnitude of the treatment effect Common measures used to report the results of meta-analyses include the odds ratio, relative risk, and mean differences. If the outcome is binary (for example, disease v no disease, remission v no remission), odds ratios or relative risks are used. If the outcome is continuous (for example, blood pressure measure￾ment), mean differences may be used. 0.37 (0.16 to 0.88) 0.46 (0.18 to 1.18) 1.67 (0.47 to 5.89) 0.22 (0.10 to 0.48) 0.88 (0.22 to 3.52) 0.23 (0.09 to 0.56) 0.58 (0.07 to 4.56) 0.25 (0.05 to 1.43) 0.34 (0.09 to 1.38) 0.37 (0.26 to 0.52) Odds ratio (95% Cl) 15.1 12.1 3.5 29.9 3.9 21.2 2.2 5.2 7.0 Weight (%) Surawicz34 McFarland37 Lewis38 Adam31 Tankanow35 Vanderhoof39 Orrhage36 Wunderlich34 Gotz32 Overall Study 1 10 Favours treatment Favours control Odds ratio 0.01 Figure 2 Effect of probiotics on the risk of antibiotic associated diarrhoea.14 Box 1: Questions to consider when appraising a systematic review16 N Did the review address a clearly focused question? N Did the review include the right type of study? N Did the reviewers try to identify all relevant studies? N Did the reviewers assess the quality of all the studies included? N If the results of the study have been combined, was it reasonable to do so? N How are the results presented and what are the main results? N How precise are the results? N Can the results be applied to your local population? N Were all important outcomes considered? N Should practice or policy change as a result of the evidence contained in this review? Understanding systematic reviews and meta-analysis 847 www.archdischild.com