New tool judges accuracy of computer predictions of tumor’s genetic diversity

Cancers are often made up of many cells which vary genetically to each other. These genetic differences mean the cancer may be particularly susceptible or resistant to a given treatment. As a result, identifying these variations can help clinicians decide which treatment is most likely to be successful for a specific patient.

Because simple clinical methods to test for genetic variation are vulnerable to missing a lot of cell-to-cell variability, recent computer tools have been developed to predict and characterize genetic diversity within clinical tumor samples. However, there is no existing common benchmarking approach to determine the most accurate computational methods.

The study, published in Nature Biotechnology, developed open-source software that can be used to judge the accuracy of computer predictions and establish this benchmark.

The team developed a simulation framework and scoring system to determine how accurately each algorithm predicted various measures of genetic diversity. These included: the proportion of cancerous cells in the tumor sample; the number of genetically different groups of cancerous cells in the tumor sample; the proportion of cells within each of these groups; which genetic mutations were in each group; and the genetic relationship between the groups.

Our new framework provides a foundation which, over time as it is run against more tumors, will hopefully become a much-needed, unbiased, gold-standard benchmarking tool for assessing models that aim to characterize a tumor's genetic diversity,"

Maxime Tarabichi, joint-lead author, postdoc in the Cancer Genomics Laboratory at the Crick.

The researchers built upon an existing computer software to generate and analyze the 580 predictions in this research, adding new features to the software to create more realistic tumors. This tumor-simulation software and the marking framework are publicly available for other researchers to use either directly or to help develop their own scoring framework.

Computer simulations in cancer genomics are helping us develop more accurate tools, as we understand where these tools perform well, and where they need improvement. Further developing these tools, so they more closely match real-life tumors, should ultimately help clinicians better match patients with personalized medicines."

Peter Van Loo, author, group leader in the Cancer Genomics Laboratory at the Crick