Using virtual clinical trials to determine the accuracy of AI-based quantitative imaging biomarkers in oncology trials using standard-of-care CT

TL;DR: The results indicate the feasibility of using VCTs to estimate the repeatability and reproducibility of AI-based biomarkers used in clinical trials with standard-of-care CT.

Abstract: There is a tremendous potential for AI-based quantitative imaging biomarkers to make clinical trials with standardof- care CT more efficient. There is, however, a well-recognized gap between discovery and the translation to practice for AI-based imaging biomarkers. Our goal is to enable more efficient and effective imaging clinical trials by characterizing the repeatability and reproducibility AI-based imaging biomarkers. We used virtual imaging clinical trials (VCTs) to simulate the data pathway by estimating the probability distributions functions for patient-, disease-, and imaging-related sources of variability. We evaluated the bias and variance in estimating the volume of liver lesions and the variance of an algorithm, that has shown success in predicting mortality risk for NSCLC patients. We used the volumetric XCAT anthropomorphic simulated phantom with inserted lesions with varied shape, size, and location. For CT acquisition and reconstruction we used the CatSim package and varied acquisition mAs and image reconstruction kernel. For each combination of parameters we generated 20 independent realizations with quantum and electronic noise. The resulting images were analyzed with the two AI-based imaging biomarkers described above, and from that we computed the mean and standard deviation of the results. Mean values and/or bias results were counter-intuitive in some cases, e.g. lower mean bias in scans with lower mAs. Addition of variations in lesion size, shape and location increased variance of the estimated parameters more than the mAs effects. These results indicate the feasibility of using VCTs to estimate the repeatability and reproducibility of AI-based biomarkers used in clinical trials with standard-of-care CT.