TITLE: A Method to Evaluate the Validity of a Literature Claim
SPEAKERS: S. Stanley Young, PhD, CGSTAT
MODERATOR: Alfred H. Balch
Well over 80% of claims made in observational studies fail to replicate. The problems appear systemic. There is a need to be able to evaluate the reliability of literature claims; some are true, but most have no valid statistical support. Our idea is to evaluate the claim of interest by examining a meta-analysis study. A SAS JMP Add-In, MetaEval starts with table of risk ratios and their confidence limits and computes a p-value for each meta-analysis base paper. A p-value plot completes the analysis. About 400,000 meta-analysis studies have been published since 2015. Our evidence is that many/most of these studies exhibit heterogeneity, which strongly implies that positive base studies are the result of excessive model searching, p-hacking. The benefit of our evaluation strategy is the easy evaluation of the claim coming from a meta- analysis. Our results indicate that most meta-analysis studies based on observational studies and the base papers used therein have no valid statistical support.
Dr. S. Stanley Young is currently the CEO of CGStat and previously worked at Eli Lilly, GlaxoSmithKline and the National Institute of Statistical Sciences on questions of applied statistics. His current interest is studying methods used in the evaluation of observational studies. He also works on bioinformatics problems.
Dr. Young graduated from North Carolina State University, BS, MES and a PhD in Statistics and Genetics. He worked in the pharmaceutical industry on all phases of pre-clinical research. He has authored or co-authored over 70 papers including six “best paper” awards, and a highly cited book, Resampling-Based Multiple Testing. He has three issued patents. He is interested in all aspects of applied statistics. He conducts research in data mining.
Dr. Young is a Fellow of the American Statistical Association and the American Association for the Advancement of Science. He is an adjunct professor of statistics at North Carolina State University, the University of Waterloo, and the University of British Columbia where he has co- directed thesis work. He is also an adjunct professor of biostatistics in the Jiann-Ping Hsu College of Public Health at Georgia Southern University. Dr. Young is on the Scientific Advisory Board of the U.S. Environmental Protection Agency.