TL;DR: It is demonstrated thatRNA-seq outperforms microarrays in determining the transcriptomic characteristics of cancer, while RNA-seq and microarray-based models perform similarly in clinical endpoint prediction.
Abstract: Gene expression profiling is being widely applied in cancer research to identify biomarkers for clinical endpoint prediction. Since RNA-seq provides a powerful tool for transcriptome-based applications beyond the limitations of microarrays, we sought to systematically evaluate the performance of RNA-seq-based and microarray-based classifiers in this MAQC-III/SEQC study for clinical endpoint prediction using neuroblastoma as a model. We generate gene expression profiles from 498 primary neuroblastomas using both RNA-seq and 44 k microarrays. Characterization of the neuroblastoma transcriptome by RNA-seq reveals that more than 48,000 genes and 200,000 transcripts are being expressed in this malignancy. We also find that RNA-seq provides much more detailed information on specific transcript expression patterns in clinico-genetic neuroblastoma subgroups than microarrays. To systematically compare the power of RNA-seq and microarray-based models in predicting clinical endpoints, we divide the cohort randomly into training and validation sets and develop 360 predictive models on six clinical endpoints of varying predictability. Evaluation of factors potentially affecting model performances reveals that prediction accuracies are most strongly influenced by the nature of the clinical endpoint, whereas technological platforms (RNA-seq vs. microarrays), RNA-seq data analysis pipelines, and feature levels (gene vs. transcript vs. exon-junction level) do not significantly affect performances of the models. We demonstrate that RNA-seq outperforms microarrays in determining the transcriptomic characteristics of cancer, while RNA-seq and microarray-based models perform similarly in clinical endpoint prediction. Our findings may be valuable to guide future studies on the development of gene expression-based predictive models and their implementation in clinical practice.
TL;DR: Compared with overall survival and time to progression, which must be evaluated in randomized trials, response rates can be accurately assessed using a single-arm trial and are optimally evaluated by assessing tumor progression in a randomized trial.
Abstract: Overall survival remains the gold standard for the demonstration of clinical benefit. An improvement in overall survival is a direct clinical benefit to patients. An analysis of overall survival requires larger patient numbers and longer follow-up than other endpoints. Survival analysis may be confounded by subsequent therapies. Time to progression usually requires smaller clinical trials and may be more rapidly assessed than trials using overall survival as an endpoint. This endpoint is not confounded by subsequent therapies. Time to progression must use the same evaluation techniques and schedules for all therapies being evaluated. Blinding of trials or the use of an external blinded radiographic review committee is recommended in assessing time to progression. Unlike overall survival and time to progression, which must be evaluated in randomized trials, response rates can be accurately assessed using a single-arm trial. Stable disease is not included in a response rate determination and is optimally evaluated by assessing tumor progression in a randomized trial. Improvement in disease-related symptoms is considered clinical benefit and may be an appropriate endpoint for drug approval.
TL;DR: An estimand for evaluating a principal surrogate, the causal effect predictiveness (CEP) surface, is introduced, which quantifies how well causal treatment effects on the biomarker predict causal treatment results on the clinical endpoint.
Abstract: Frangakis and Rubin (2002, Biometrics 58, 21-29) proposed a new definition of a surrogate endpoint (a "principal" surrogate) based on causal effects. We introduce an estimand for evaluating a principal surrogate, the causal effect predictiveness (CEP) surface, which quantifies how well causal treatment effects on the biomarker predict causal treatment effects on the clinical endpoint. Although the CEP surface is not identifiable due to missing potential outcomes, it can be identified by incorporating a baseline covariate(s) that predicts the biomarker. Given case-cohort sampling of such a baseline predictor and the biomarker in a large blinded randomized clinical trial, we develop an estimated likelihood method for estimating the CEP surface. This estimation assesses the "surrogate value" of the biomarker for reliably predicting clinical treatment effects for the same or similar setting as the trial. A CEP surface plot provides a way to compare the surrogate value of multiple biomarkers. The approach is illustrated by the problem of assessing an immune response to a vaccine as a surrogate endpoint for infection.
TL;DR: A counterpoint is presented in which data are presented that strongly and logically support the use of FVC as a valid and robust measure that fulfils the criteria for an ideal clinical endpoint and that is meaningful to patient and clinician alike.
Abstract: Idiopathic pulmonary fibrosis causes progressive morbidity and has a worldwide incidence that is increasing. There are a number of promising therapies, one of which has been approved in Europe, parts of Asia, and India, and others that are at various stages of development. Despite this, there continues to be debate about the most appropriate clinical endpoint that should be used in future randomized controlled clinical trials of novel therapies in idiopathic pulmonary fibrosis. In a recent Pulmonary Perspective in this journal, the case for the use of a variety of clinical endpoints was analyzed, and the article concluded that FVC, the endpoint most commonly used recently and in ongoing studies, was not an appropriate option. In this Pulmonary Perspective we present a counterpoint in which we explore the basis on which this conclusion is drawn and present data that strongly and logically support the use of FVC as a valid and robust measure that fulfils the criteria for an ideal clinical endpoint and that is meaningful to patient and clinician alike.