A performance evaluation study: Variant annotation tools - the enigma of clinical next generation sequencing (NGS) based genetic testing

TL;DR: In this article , the authors evaluated the performance of three variant annotation tools: Alamut® Batch, Ensembl Variant Effect Predictor (VEP) and ANNOVAR, benchmarked by a manually curated ground truth set of 298 variants from the medical exome database at the Molecular Diagnostics Laboratory at Lurie Children's Hospital.

Abstract: Dramatically expanding our ability for clinical genetic testing for inherited conditions and complex diseases such as cancer, next generation sequencing (NGS) technologies are allowing for rapid interrogation of thousands of genes and identification of millions of variants. Variant annotation, the process of assigning functional information to DNA variants based on the standardized Human Genome Variation Society (HGVS) nomenclature, is a fundamental challenge in the analysis of NGS data that has led to the development of many bioinformatic algorithms. In this study, we evaluated the performance of three variant annotation tools: Alamut® Batch, Ensembl Variant Effect Predictor (VEP) and ANNOVAR, benchmarked by a manually curated ground truth set of 298 variants from the medical exome database at the Molecular Diagnostics Laboratory at Lurie Children’s Hospital. Of the three tools, VEP produces the most accurate variant annotations (HGVS nomenclature for 297 of the 298 variants) due to usage of updated gene transcript versions within the algorithm. Alamut® Batch called 296 of the 298 variants correctly; strikingly, ANNOVAR exhibited the greatest number of discrepancies (20 of the 298 variants, 93.3% concordance with ground truth set). Adoption of validated methods of variant annotation is critical in post analytical phases of clinical testing.