TITLE: Evaluating Predictive Accuracy of Survival Models Using the ROC Curve and Related Measures in Clinical Trial Applications
SPEAKER: Kassu Mehari Beyene and Ding-Geng (Din) Chen, ASU
Moderator: Alfred H. Balch
Abstract: In clinical trials, one of the common endpoints is time-to-event, which can be defined as the duration between randomization and the occurrence of a well-defined event of interest. The event, for example, can be death due to certain disease, disease progression,treatment failure or the recurrences of a disease. In such follow-up studies, the event of interest is not necessarily experienced by all study participants at the end of the study, so the actual event times for some subjects are unknown. This loss of information on time-to-event is known as censoring, which may occur when a subject withdraws from the study, lost to follow-up, or the study ends before the event has occurred. Survival analysis, or more generally, time-to-event analysis, is a standard tool to take this unique feature into account. Survival models, for example, Cox proportional hazards model, can be used to identify significant survival-related biomarkers or compare the efficacy between a new treatment and a placebo or an active treatment.
Like many other statistical models, however, ensuring their validity is crucial before using them in routine clinical practice. Assessing the validity of survival models often involves measuring the predictive accuracy, the ability of a model to predict the risk of a future event given a set of baseline factors. In literature, there are several ways for assessing the predictive accuracy of survival models which are often described by two components: calibration and discrimination. Discrimination characterizes how well the model can classify subjects into one of the two groups (e.g., diseased and non-diseased), whereas calibration measures the degree of closeness between the predicted probabilities and the observed outcome. There are a variety of methods to evaluate the discriminatory ability of a survival model. The most commonly used are the receiver operating characteristic (ROC) curve and the associated summary measures.
This tutorial is to discuss the ROC curve and its associated summary measures in the context of censored data with detailed step-by-step illustrations and implementations using the R software from R package cenROC. In this tutorial, we will use publicly available dataset from the Mayo Clinic trial in primary biliary cirrhosis (PBC) of the liver and other clinical trial datasets to illustrate the ROC curve and the associated summary measures analysis.
Dr. Kassu Mehari Beyene is a postdoctoral research scholar in Biostatistics at the College of Health Solutions, Arizona State University at Phoenix. After earning a PhD in Biostatistics from Catholic University of Louvain (Belgium), he worked an assistant professor in biostatistics at the Department of Statistics, Wollo University, Ethiopia. His research interest includes the ROC curve and other related approaches for evaluating predictive accuracy of biomarkers, survival data analysis, infectious disease modeling, and longitudinal data analysis. He has three research publications in ROC curve estimation for censored time-to-event data and also published an R package for ROC curve and related measures estimation in the context of survival data. He presented talks about ROC curve and related measures estimation and survival data analysis at various professional conferences and seminars.
Dr. Din Chen received his Ph.D. in Statistics from the University of Guelph (Canada) in 1995 and is now the executive director and professor in Biostatistics at the College of Health Solutions, Arizona State University, USA. He is also an extraordinary professor and the DST-NRF-SAMRC SARCHI research chair (Tier 1) in biostatistics, at the Department of Statistics, University of Pretoria, South Africa. Dr. Chen served as the Wallace H. Kuralt distinguished professor in biostatistics at the University of North Carolina-Chapel Hill from 2005 to 2021, a biostatistics professor at the University of Rochester Medical Center from 2020 to 2025, the Karl E. Peace endowed eminent scholar chair in biostatistics from the Jiann-Ping Hsu College of Public Health at the Georgia Southern University from 2019 to 2020. Dr. Chen is an elected fellow of the American Statistical Association and a senior expert consultant for biopharmaceuticals and government agencies with extensive expertise in clinical trial biostatistics. Dr. Chen has more than 200 scientific publications and co-authored/co-edited 38 books on clinical trials, survival data, meta-analysis, causal inference and structural equation modeling, Monte-Carlo simulation-based statistical modelling. His research has been funded as PI/Co-PI from NIH R01s and other governmental agencies.