AI Platform Near Perfect for Predicting Thyroid Malignancy

An artificial intelligence (AI)-supported ultrasound system noninvasively distinguished between benign and malignant thyroid nodules with near-perfect accuracy, a large retrospective analysis showed.

In the training and validation sets combined, the system predicted malignancy with 98.7% overall accuracy. The performance translated into an area under the receiver operating characteristic curve (AUC) of 0.99 (1.00=100% accuracy).

The multimodal system demonstrated similar accuracy for predicting T stage, extracapsular extension, and presence of BRAF mutation, reported Annie W. Chan, MD, of the Mass General Cancer Center in Boston, at the Multidisciplinary Head and Neck Cancers Symposium in Phoenix.

“We have created a platform to predict malignancy as well as to predict surgical and genomic outcome,” she said. “Different AI models integrate with each other. It would be nice to perform multicenter studies with prospective validation.”

The current approach to evaluating thyroid nodules is based almost entirely on the radiologist’s perception. Descriptive terms are imprecise, such as “stable in appearance,” “suspicious,” “worrisome,”or “concerning,” said Chan. Shape, appearance, and size form the basis for diagnosis.

Beyond the initial evaluation, the current approach to managing thyroid nodules is time-consuming, subject to significant inter-observer variability, and invasive, as staging cannot be performed with CT or MRI but requires ultrasound-guided biopsy.

“We need to minimize subjectivity,” said Chan. “We need to be able to make a more precise and accurate, as well as quantitative, prediction. We need to distinguish real versus fake. We need to remove noises from the images. We need to remove noises from our data.”

The integrated platform combines multiple types of imaging and AI technology to achieve greater accuracy than any of the individual modalities. Key components of the imaging platform are:

• The Thyroid Imaging Reporting and Data System (TI-RADS)
• Radiomics: quantifies texture and differing gray level intensity in an image
• Topological data analysis (TDA): examines the kinetics of pixel formation
• Deep learning: repeated scanning of images to filter out noise and retain important data

The approaches are complementary, said Chan. Radiomics technology is established and has known sensitivity to image noise. TDA offers robust noise analysis. Deep learning requires no image segmentation or feature selection but requires a large data set, is sensitive to noise, and the accuracy depends on the quality of the training set.

Chan asked meeting participants if they would like to know the result of a biopsy before it is performed, and know the pathology result before surgery. With currently available technology, “this is actually very feasible nowadays.”

Investigators used the system to evaluate 1,346 ultrasound images of 784 patients with known thyroid nodule status, including internal training and validation studies and an external validation study. The internal studies involved 156 malignant nodules and 357 benign lesions, whereas the external validation study included 270 malignant nodules and 50 benign growths.

By internal validation, radiomics performed best for predicting malignancy (88.7%, AUC 0.87), followed by deep learning (87.4%, AUC 0.92), TDA (81.5%, AUC 0.81), and TI-RADS (80%, AUC 0.76). The integrated multimodal system significantly outperformed each of the individual components of the system (P=0.002 vs deep learning, P<0.001 vs the others).

The integrated-technology platform had 93% accuracy for predicting T stage of malignancies, 98% accuracy for predicting extracapsular extension, and 96% accuracy for predicting BRAF mutation. All of the values outperformed the individual components of the system.

For the external validation study, the integrated platform achieved 93% accuracy for malignancy prediction and an AUC of 0.94.

The results compared favorably with four other studies of AI-based prediction of thyroid malignancy published in 2020 and 2021, said Chan. All four relied exclusively on deep learning. Three studies from China had accuracy rates of 87%-96%, and one from the U.S. had 82% accuracy. The multimodal system’s AUC (0.99), true-positive rate (0.98), and true-negative rate (0.99) exceeded all of the other studies.

Correlational analysis showed that the different components of the integrated model are not correlated but provide information independent of each other, said Chan.