SIIM, ACR announce top 10 winners of pneumothorax detection machine learning challenge

Download PDF Copy

Reviewed

Reviewed by James Ives, M.Psych. (Editor)Sep 24 2019

The American College of Radiology (ACR) and the Society for Imaging Informatics in Medicine (SIIM) announced the official results of their first machine learning challenge today during the SIIM-ACR Pneumothorax Challenge ceremony at SIIM's 4th annual Conference on Machine Intelligence in Medical Imaging (C-MIMI).

The SIIM-ACR Pneumothorax Detection and Localization Challenge required teams to develop high quality pneumothorax detection algorithms to prioritize patients for expedited review and treatment. A total of 1,475 teams took part in the challenge, and 352 submitted results during the evaluation phase of the competition.

The challenge made use of a publicly available chest radiograph dataset from the National Institutes of Health (NIH). The augmented annotations were created by radiologists from SIIM and the Society of Thoracic Radiology (STR), under the leadership of Carol Wu, MD, using a commercial web-based tool from MD.ai. The augmented annotations also follow the ACR Data Science Institute's structured artificial intelligence (AI) use case for pneumothorax detection.

SIIM is very pleased to have cooperated with the ACR, Google, Kaggle and the Society of Thoracic Radiology in hosting this challenge. In addition to the medical and data science aspects, SIIM introduced the use of FHIR and DICOMweb in a medical imaging data challenge for the first time in Kaggle's history, as those API's are key in moving AI tools into clinical production."

Steve Langer, PhD, CIIP, FSIIM, Informatics Physicist and Radiology Imaging Architect, Mayo Clinic

Langer is also the Co-Chair of the SIIM Machine Learning Committee.

"Kaggle challenges like this one now incorporate some useful parameters that are more likely to result in the winners producing AI tools with potential for clinical production," said Bibb Allen Jr., MD, FACR, ACR Data Science Institute Chief Medical Officer. "Congratulations to the winners. They have developed new healthcare solutions that may one day improve patient care."

The challenge was run on a Kaggle, Inc. platform (owned by Google LLC), which provides access to datasets, a discussion forum for participants, the repository of submitted results and a leaderboard that runs throughout the challenge.

The Top 10 Winning teams are: