A Sequential Approach to Random-Effects Meta-Analysis
The objective of meta-analysis is to combine results
from several independent studies in order to create generalization
and provide evidence base for decision making. But recent studies
show that the magnitude of effect size estimates reported in many
areas of research significantly changed over time and this can
impair the results and conclusions of meta-analysis. A number of
sequential methods have been proposed for monitoring the effect
size estimates in meta-analysis. However they are based on statistical
theory applicable only to fixed effect model (FEM) of meta-analysis.
For random-effects model (REM), the analysis incorporates the
heterogeneity variance, τ 2 and its estimation create complications.
In this paper we study the use of a truncated CUSUM-type test with
asymptotically valid critical values for sequential monitoring in REM.
Simulation results show that the test does not control the Type I error
well, and is not recommended. Further work required to derive an
appropriate test in this important area of applications.
[1] Casper W Bollen, Cuno SPM Uiterwaal, Adrianus J van Vught, and
Ingeborg van der Tweel. Sequential meta-analysis of past clinical trials
to determine the use of a new trial. Epidemiology, 17(6):644–649, 2006.
[2] S. Brugger, J.M. Davis, S. Leucht, and J.M. Stone. Proton magnetic
resonance spectroscopy and illness stage in schizophreniaa systematic
review and meta-analysis. Biological psychiatry, 69(5):495–503, 2011.
[3] Rebecca DerSimonian and Raghu Kacker. Random-effects model for
meta-analysis of clinical trials: an update. Contemporary clinical trials,
28(2):105–114, 2007.
[4] Rebecca. DerSimonian and Nan. Laird. Meta-analysis in clinical trials.
Controlled clinical trials, 7(3):177–188, 1986.
[5] Lynn Friedman. Estimators of random effects variance components
in meta-analysis. Journal of Educational and Behavioral Statistics,
25(1):1–12, 2000.
[6] B. Gehr, C. Weiss, and F. Porzsolt. The fading of reported effectiveness.
a meta-analysis of randomised controlled trials. BMC medical research
methodology, 6(1):25, 2006.
[7] E Gombay. Sequential change-point detection and estimation. Sequential
Analysis, 22(3):203–222, 2003.
[8] E. Gombay and D. Serbian. An adaptation of Pages CUSUM test for
change detection. Periodica Mathematica Hungarica, 50(1):135–147,
2005.
[9] Shelly Grabe, L Monique Ward, and Janet Shibley Hyde. The role
of the media in body image concerns among women: a meta-analysis
of experimental and correlational studies. Psychological bulletin,
134(3):460, 2008 [10] Larry V Hedges and Jack L Vevea. Fixed-and random-effects models in
meta-analysis. Psychological methods, 3(4):486, 1998.
[11] Julian Higgins, Anne Whitehead, and Mark Simmonds. Sequential
methods for random-effects meta-analysis. Statistics in medicine,
30(9):903–921, 2011.
[12] M. J. Hodgson, D. K. Parkinson, and M. Karpf. Chest x-ray
in hypersensitivity pneumonities: A meta analysis of secular trends.
American journal of industrial medicine, 16(1):45–53, 1989.
[13] Mingxiu Hu, Joseph C Cappelleri, and KK Gordon Lan. Applying
the law of iterated logarithm to control type I error in cumulative
meta-analysis of binary outcomes. Clinical Trials, 4(4):329–340, 2007.
[14] J.S. Hyde, E. Fennema, and S.J. Lamon. Gender differences in
mathematics performance: a meta-analysis. Psychological bulletin,
107(2):139, 1990.
[15] John Ioannidis and Thomas A Trikalinos. Early extreme contradictory
estimates may appear in published research: The proteus phenomenon
in molecular genetics research and randomized trials. Journal of clinical
epidemiology, 58(6):543–549, 2005.
[16] E. Kulinskaya and J. Koricheva. Use of quality control charts for
detection of outliers and temporal trends in cumulative meta-analysis.
Research Synthesis Methods, 2010.
[17] Elena Kulinskaya and John Wood. Trial sequential methods for
meta-analysis. Research Synthesis Methods, 2013.
[18] Sofie Kuppens and Patrick Onghena. Sequential meta-analysis to
determine the sufficiency of cumulative knowledge: The case of early
intensive behavioral intervention for children with autism spectrum
disorders. Research in Autism Spectrum Disorders, 6(1):168–176, 2012.
[19] KK Gordon Lan, Mingxiu Hu, and Joseph C Cappelleri. Applying the
law of iterated logarithm to cumulative meta-analysis of a continuous
endpoint. Statistica Sinica, 13(4):1135–1146, 2003.
[20] J. Lau, E.M. Antman, J. Jimenez-Silva, B. Kupelnick, F. Mosteller,
and T.C. Chalmers. Cumulative meta-analysis of therapeutic trials
for myocardial infarction. New England Journal of Medicine,
327(4):248–254, 1992.
[21] R. Leimu and J. Koricheva. Cumulative meta–analysis: a new tool
for detection of temporal trends and publication bias in ecology.
Proceedings of the Royal Society of London. Series B: Biological
Sciences, 271(1551):1961–1966, 2004.
[22] Dennis J Nieuwkamp, Larissa E Setz, Ale Algra, Francisca HH Linn,
Nicolien K de Rooij, and Gabri¨el JE Rinkel. Changes in case fatality
of aneurysmal subarachnoid haemorrhage over time, according to age,
sex, and region: a meta-analysis. The Lancet Neurology, 8(7):635–642,
2009.
[23] Robert C. Paule and John Mandel. Consensus values and weighting
factors. Journal of Research of the National Bureau of Standards,
87(5):377–385, 1982.
[24] J.M. Pogue and S. Yusuf. Cumulating evidence from randomized trials:
utilizing sequential monitoring boundaries for cumulative meta-analysis.
Controlled clinical trials, 18(6):580–593, 1997.
[25] Jean M. Twenge., Sara. Konrath, Joshua D. Foster., W Keith Campbell.,
and Brad J. Bushman. Egos inflating over time: A cross-temporal
meta-analysis of the narcissistic personality inventory. Journal of
personality, 76(4):875–902, 2008.
[26] Wolfgang Viechtbauer. Confidence intervals for the amount of
heterogeneity in meta-analysis. Statistics in medicine, 26(1):37–52, 2007.
[27] J. Wetterslev, K. Thorlund, J. Brok, and C. Gluud. Trial sequential
analysis may establish when firm evidence is reached in cumulative
meta-analysis. Journal of Clinical Epidemiology, 61(1):64–75, 2008.
[28] Anne Whitehead. A prospectively planned cumulative meta-analysis
applied to a series of concurrent clinical trials. Statistics in medicine,
16(24):2901–2913, 1997.
[29] John Whitehead. The design and analysis of sequential clinical trials.
John Wiley & Sons, 1997.
[1] Casper W Bollen, Cuno SPM Uiterwaal, Adrianus J van Vught, and
Ingeborg van der Tweel. Sequential meta-analysis of past clinical trials
to determine the use of a new trial. Epidemiology, 17(6):644–649, 2006.
[2] S. Brugger, J.M. Davis, S. Leucht, and J.M. Stone. Proton magnetic
resonance spectroscopy and illness stage in schizophreniaa systematic
review and meta-analysis. Biological psychiatry, 69(5):495–503, 2011.
[3] Rebecca DerSimonian and Raghu Kacker. Random-effects model for
meta-analysis of clinical trials: an update. Contemporary clinical trials,
28(2):105–114, 2007.
[4] Rebecca. DerSimonian and Nan. Laird. Meta-analysis in clinical trials.
Controlled clinical trials, 7(3):177–188, 1986.
[5] Lynn Friedman. Estimators of random effects variance components
in meta-analysis. Journal of Educational and Behavioral Statistics,
25(1):1–12, 2000.
[6] B. Gehr, C. Weiss, and F. Porzsolt. The fading of reported effectiveness.
a meta-analysis of randomised controlled trials. BMC medical research
methodology, 6(1):25, 2006.
[7] E Gombay. Sequential change-point detection and estimation. Sequential
Analysis, 22(3):203–222, 2003.
[8] E. Gombay and D. Serbian. An adaptation of Pages CUSUM test for
change detection. Periodica Mathematica Hungarica, 50(1):135–147,
2005.
[9] Shelly Grabe, L Monique Ward, and Janet Shibley Hyde. The role
of the media in body image concerns among women: a meta-analysis
of experimental and correlational studies. Psychological bulletin,
134(3):460, 2008 [10] Larry V Hedges and Jack L Vevea. Fixed-and random-effects models in
meta-analysis. Psychological methods, 3(4):486, 1998.
[11] Julian Higgins, Anne Whitehead, and Mark Simmonds. Sequential
methods for random-effects meta-analysis. Statistics in medicine,
30(9):903–921, 2011.
[12] M. J. Hodgson, D. K. Parkinson, and M. Karpf. Chest x-ray
in hypersensitivity pneumonities: A meta analysis of secular trends.
American journal of industrial medicine, 16(1):45–53, 1989.
[13] Mingxiu Hu, Joseph C Cappelleri, and KK Gordon Lan. Applying
the law of iterated logarithm to control type I error in cumulative
meta-analysis of binary outcomes. Clinical Trials, 4(4):329–340, 2007.
[14] J.S. Hyde, E. Fennema, and S.J. Lamon. Gender differences in
mathematics performance: a meta-analysis. Psychological bulletin,
107(2):139, 1990.
[15] John Ioannidis and Thomas A Trikalinos. Early extreme contradictory
estimates may appear in published research: The proteus phenomenon
in molecular genetics research and randomized trials. Journal of clinical
epidemiology, 58(6):543–549, 2005.
[16] E. Kulinskaya and J. Koricheva. Use of quality control charts for
detection of outliers and temporal trends in cumulative meta-analysis.
Research Synthesis Methods, 2010.
[17] Elena Kulinskaya and John Wood. Trial sequential methods for
meta-analysis. Research Synthesis Methods, 2013.
[18] Sofie Kuppens and Patrick Onghena. Sequential meta-analysis to
determine the sufficiency of cumulative knowledge: The case of early
intensive behavioral intervention for children with autism spectrum
disorders. Research in Autism Spectrum Disorders, 6(1):168–176, 2012.
[19] KK Gordon Lan, Mingxiu Hu, and Joseph C Cappelleri. Applying the
law of iterated logarithm to cumulative meta-analysis of a continuous
endpoint. Statistica Sinica, 13(4):1135–1146, 2003.
[20] J. Lau, E.M. Antman, J. Jimenez-Silva, B. Kupelnick, F. Mosteller,
and T.C. Chalmers. Cumulative meta-analysis of therapeutic trials
for myocardial infarction. New England Journal of Medicine,
327(4):248–254, 1992.
[21] R. Leimu and J. Koricheva. Cumulative meta–analysis: a new tool
for detection of temporal trends and publication bias in ecology.
Proceedings of the Royal Society of London. Series B: Biological
Sciences, 271(1551):1961–1966, 2004.
[22] Dennis J Nieuwkamp, Larissa E Setz, Ale Algra, Francisca HH Linn,
Nicolien K de Rooij, and Gabri¨el JE Rinkel. Changes in case fatality
of aneurysmal subarachnoid haemorrhage over time, according to age,
sex, and region: a meta-analysis. The Lancet Neurology, 8(7):635–642,
2009.
[23] Robert C. Paule and John Mandel. Consensus values and weighting
factors. Journal of Research of the National Bureau of Standards,
87(5):377–385, 1982.
[24] J.M. Pogue and S. Yusuf. Cumulating evidence from randomized trials:
utilizing sequential monitoring boundaries for cumulative meta-analysis.
Controlled clinical trials, 18(6):580–593, 1997.
[25] Jean M. Twenge., Sara. Konrath, Joshua D. Foster., W Keith Campbell.,
and Brad J. Bushman. Egos inflating over time: A cross-temporal
meta-analysis of the narcissistic personality inventory. Journal of
personality, 76(4):875–902, 2008.
[26] Wolfgang Viechtbauer. Confidence intervals for the amount of
heterogeneity in meta-analysis. Statistics in medicine, 26(1):37–52, 2007.
[27] J. Wetterslev, K. Thorlund, J. Brok, and C. Gluud. Trial sequential
analysis may establish when firm evidence is reached in cumulative
meta-analysis. Journal of Clinical Epidemiology, 61(1):64–75, 2008.
[28] Anne Whitehead. A prospectively planned cumulative meta-analysis
applied to a series of concurrent clinical trials. Statistics in medicine,
16(24):2901–2913, 1997.
[29] John Whitehead. The design and analysis of sequential clinical trials.
John Wiley & Sons, 1997.
@article{"International Journal of Engineering, Mathematical and Physical Sciences:68702", author = "Samson Henry Dogo and Allan Clark and Elena Kulinskaya", title = "A Sequential Approach to Random-Effects Meta-Analysis", abstract = "The objective of meta-analysis is to combine results
from several independent studies in order to create generalization
and provide evidence base for decision making. But recent studies
show that the magnitude of effect size estimates reported in many
areas of research significantly changed over time and this can
impair the results and conclusions of meta-analysis. A number of
sequential methods have been proposed for monitoring the effect
size estimates in meta-analysis. However they are based on statistical
theory applicable only to fixed effect model (FEM) of meta-analysis.
For random-effects model (REM), the analysis incorporates the
heterogeneity variance, τ 2 and its estimation create complications.
In this paper we study the use of a truncated CUSUM-type test with
asymptotically valid critical values for sequential monitoring in REM.
Simulation results show that the test does not control the Type I error
well, and is not recommended. Further work required to derive an
appropriate test in this important area of applications.
", keywords = "Meta-analysis, random-effects model, sequential testing, temporal changes in effect sizes.", volume = "9", number = "1", pages = "23-7", }
