Quantifying and Adjusting the Effects of Publication Bias in Continuous Meta-Analysis

This study uses simulated meta-analysis to assess the effects of publication bias on meta-analysis estimates and to evaluate the efficacy of the trim and fill method in adjusting for these biases. The estimated effect sizes and the standard error were evaluated in terms of the statistical bias and the coverage probability. The results demonstrate that if publication bias is not adjusted it could lead to up to 40% bias in the treatment effect estimates. Utilization of the trim and fill method could reduce the bias in the overall estimate by more than half. The method is optimum in presence of moderate underlying bias but has minimal effects in presence of low and severe publication bias. Additionally, the trim and fill method improves the coverage probability by more than half when subjected to the same level of publication bias as those of the unadjusted data. The method however tends to produce false positive results and will incorrectly adjust the data for publication bias up to 45 % of the time. Nonetheless, the bias introduced into the estimates due to this adjustment is minimal

• N.R.N. Idris

• Publication bias
• Trim and Fill method
• percentage relative bias
• coverage probability

[1] C.B.Begg and J.A. Berlin, "Publication bias: a problem in interpreting
medical data" J. R. Stat. Soc. Series A, vol 151, pp. 419-463, 1988.
[2] L.J. Peters, A.J. Sutton , D.R. Jones, K.R. Abrams, L. Rushton,
"Comparison of two methods to detect publication bias in metaanalysis",
JAMA, vol 295, pp. 676-680, 2006.
[3] M. Egger., S.G. Davey, M. Schneider, C. Mider, "Bias in meta-analyses
detected by a simple graphical test", BMJ, vol 315, pp. 629-634, 1997.
[4] S.J. Duval and R.L. Tweedie. "A non-parametric "trim and fill" method
of accounting for publication bias in meta-analysis", J.Amer Statist
Assoc, vol 95 , pp.89-98, 2000a.
[5] S.J. Duval and R.L. Tweedie, "Trim and Fill : A simple funnel- plotbased
method of testing and adjusting for publication bias in metaanalysis",
Biometric, vol 56, pp. 455-463, 2000b.
[6] K.Dickersin, K., 2005. "Publication bias: Recognizing the problem,
understanding its origins and scope, and preventing harm". In H.
Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication Bias in
Meta Analysis: Prevention, Assessment and Adjustments, pp. 11-34,.
Chichester, UK: Wiley, 2005.
[7] A.J. Sutton, S.J. Duvall, R.L. Tweedie, K.R Abrams., D.R. Jones,
"Empirical assessment of effect of publication bias on meta analyses"
BMJ, vol 320, pp. 1574-1576, 2000.
[8] M.D. Jennions and A.P. Moller, "Publication bias in ecology and
evolution: an empirical assessment using the trim and fill method",
Biological Reviews, vol 77, pp. 211-222, 2002.
[9] M.Borenstein, L.V.Hedges, P.T.Higgins, H.R.Rothstein,"Introduction to
meta-analysi", John Wiley&sons, London ,2009.
[10] N.R. Idris, "Estimating meta analysis parameters in non standard data".
PhD Thesis. Glasgow: University of Strathclyde, 1996.
[11] P. Macaskill, S.D. Walter and L. Irwig, "A comparison of methods to
detect publication bias in meta analysis", Statist. Med., vol 20, pp. 641-
654, 2001.
[12] C.B.Begg and Mazumdar, M., 1994. "Operating characteristics of a rank
correlation test for publication bias", Biometrics, vol 50, pp.1088-1101,
1994.
[13] J.A.C. Sterne, D. Gavaghan, M. Egger, "Publication and related bias in
meta analysis", J.ournal of Clinical Epidemiology, vol 53, pp. 1119-
1129, 2000.
[14] N. Terrin, C. H.Schmid, J. Lau, "An empirical evaluation of the funnel
plot, researchers could not visually identify publication bias", J. Clin.
Epidemiol, vol 58, pp. 894-901, 2003.