Integrating imaging and genomic data for the discovery of distinct glioblastoma subtypes: a joint learning approach

Scientific Reports · 2024 · 23 citations

• Computer Science
• Artificial Intelligence
• Computational biology

Glioblastoma is a highly heterogeneous disease, with variations observed at both phenotypical and molecular levels. Personalized therapies would be facilitated by non-invasive in vivo approaches for characterizing this heterogeneity. In this study, we developed unsupervised joint machine learning between radiomic and genomic data, thereby identifying distinct glioblastoma subtypes. A retrospective cohort of 571 IDH-wildtype glioblastoma patients were included in the study, and pre-operative multi-parametric MRI scans and targeted next-generation sequencing (NGS) data were collected. L21-norm minimization was used to select a subset of 12 radiomic features from the MRI scans, and 13 key driver genes from the five main signal pathways most affected in glioblastoma were selected from the genomic data. Subtypes were identified using a joint learning approach called Anchor-based Partial Multi-modal Clustering on both radiomic and genomic modalities. Kaplan-Meier analysis identified three distinct glioblastoma subtypes: high-risk, medium-risk, and low-risk, based on overall survival outcome (p < 0.05, log-rank test; Hazard Ratio = 1.64, 95% CI 1.17-2.31, Cox proportional hazard model on high-risk and low-risk subtypes). The three subtypes displayed different phenotypical and molecular characteristics in terms of imaging histogram, co-occurrence of genes, and correlation between the two modalities. Our findings demonstrate the synergistic value of integrated radiomic signatures and molecular characteristics for glioblastoma subtyping. Joint learning on both modalities can aid in better understanding the molecular basis of phenotypical signatures of glioblastoma, and provide insights into the biological underpinnings of tumor formation and progression.

From Revisions to Insights: Converting Radiology Report Revisions into Actionable Educational Feedback Using Generative AI Models

Deleted Journal · 2024 · 17 citations

• Computer Science
• Artificial Intelligence
• Computer Science

Expert feedback on trainees' preliminary reports is crucial for radiologic training, but real-time feedback can be challenging due to non-contemporaneous, remote reading and increasing imaging volumes. Trainee report revisions contain valuable educational feedback, but synthesizing data from raw revisions is challenging. Generative AI models can potentially analyze these revisions and provide structured, actionable feedback. This study used the OpenAI GPT-4 Turbo API to analyze paired synthesized and open-source analogs of preliminary and finalized reports, identify discrepancies, categorize their severity and type, and suggest review topics. Expert radiologists reviewed the output by grading discrepancies, evaluating the severity and category accuracy, and suggested review topic relevance. The reproducibility of discrepancy detection and maximal discrepancy severity was also examined. The model exhibited high sensitivity, detecting significantly more discrepancies than radiologists (W = 19.0, p < 0.001) with a strong positive correlation (r = 0.778, p < 0.001). Interrater reliability for severity and type were fair (Fleiss' kappa = 0.346 and 0.340, respectively; weighted kappa = 0.622 for severity). The LLM achieved a weighted F1 score of 0.66 for severity and 0.64 for type. Generated teaching points were considered relevant in ~ 85% of cases, and relevance correlated with the maximal discrepancy severity (Spearman ρ = 0.76, p < 0.001). The reproducibility was moderate to good (ICC (2,1) = 0.690) for the number of discrepancies and substantial for maximal discrepancy severity (Fleiss' kappa = 0.718; weighted kappa = 0.94). Generative AI models can effectively identify discrepancies in report revisions and generate relevant educational feedback, offering promise for enhancing radiology training.

Image segmentations produced by BAMF under the AIMI Annotations initiative

arXiv (Cornell University) · 2024 · 369 citations

• Computer Science
• Artificial Intelligence
• Medical physics

The Imaging Data Commons (IDC)(https://imaging.datacommons.cancer.gov/) [1] connects researchers with publicly available cancer imaging data, often linked with other types of cancer data. Many of the collections have limited annotations due to the expense and effort required to create these manually. The increased capabilities of AI analysis of radiology images provide an opportunity to augment existing IDC collections with new annotation data. To further this goal, we trained several nnUNet [2] based models for a variety of radiology segmentation tasks from public datasets and used them to generate segmentations for IDC collections. To validate the model's performance, roughly 10% of the AI predictions were assigned to a validation set. For this set, a board-certified radiologist graded the quality of AI predictions on a Likert scale. If they did not 'strongly agree' with the AI output, the reviewer corrected the segmentation. This record provides the AI segmentations, Manually corrected segmentations, and Manual scores for the inspected IDC Collection images. Only 10% of the AI-derived annotations provided in this dataset are verified by expert radiologists . More details, on model training and annotations are provided within the associated manuscript to ensure transparency and reproducibility. This work was done in two stages. Versions 1.x of this record were from the first stage. Versions 2.x added additional records. In the Version 1.x collections, a medical student (non-expert) reviewed all the AI predictions and rated them on a 5-point Likert Scale, for any AI predictions in the validation set that they did not 'strongly agree' with, the non-expert provided corrected segmentations. This non-expert was not utilized for the Version 2.x additional records. Likert Score Definition: Guidelines for reviewers to grade the quality of AI segmentations. 5 Strongly Agree - Use-as-is (i.e., clinically acceptable, and could be used for treatment without change) 4 Agree - Minor edits that are not necessary. Stylistic differences, but not clinically important. The current segmentation is acceptable 3 Neither agree nor disagree - Minor edits that are necessary. Minor edits are those that the review judges can be made in less time than starting from scratch or are expected to have minimal effect on treatment outcome 2 Disagree - Major edits. This category indicates that the necessary edit is required to ensure correctness, and sufficiently significant that user would prefer to start from the scratch 1 Strongly disagree - Unusable. This category indicates that the quality of the automatic annotations is so bad that they are unusable. Zip File Folder Structure Each zip file in the collection correlates to a specific segmentation task. The common folder structure is ai-segmentations-dcm This directory contains the AI model predictions in DICOM-SEG format for all analyzed IDC collection files qa-segmentations-dcm This directory contains manual corrected segmentation files, based on the AI prediction, in DICOM-SEG format. Only a fraction, ~10%, of the AI predictions were corrected. Corrections were performed by radiologist (rad*) and non-experts (ne*) qa-results.csv CSV file linking the study/series UIDs with the ai segmentation file, radiologist corrected segmentation file, radiologist ratings of AI performance. qa-results.csv Columns The qa-results.csv file contains metadata about the segmentations, their related IDC case image, as well as the Likert ratings and comments by the reviewers. Column Description Collection The name of the IDC collection for this case PatientID PatientID in DICOM metadata of scan. Also called Case ID in the IDC StudyInstanceUID StudyInstanceUID in the DICOM metadata of the scan SeriesInstanceUID SeriesInstanceUID in the DICOM metadata of the scan Validation true/false if this scan was manually reviewed Reviewer Coded ID of the reviewer. Radiologist IDs start with ‘rad’ non-expect IDs start with ‘ne’ AimiProjectYear 2023 or 2024, This work was split over two years. The main methodology difference between the two is that in 2023, a non-expert also reviewed the AI output, but a non-expert was not utilized in 2024. AISegmentation The filename of the AI prediction file in DICOM-seg format. This file is in the ai-segmentations-dcm folder. CorrectedSegmentation The filename of the reviewer-corrected prediction file in DICOM-seg format. This file is in the qa-segmentations-dcm folder. If the reviewer strongly agreed with the AI for all segments, they did not provide any correction file. Was the AI predicted ROIs accurate? This column appears one for each segment in the task for images from AimiProjectYear 2023. The reviewer rates segmentation quality on a Likert scale. In tasks that have multiple labels in the output, there is only one rating to cover them all. Was the AI predicted {SEGMENT_NAME} label accurate? This column appears one for each segment in the task for images from AimiProjectYear 2024. The reviewer rates each segment for its quality on a Likert scale. Do you have any comments about the AI predicted ROIs? Open ended question for the reviewer Do you have any comments about the findings from the study scans? Open ended question for the reviewer File Overview brain-mr.zip Segment Description: brain tumor regions: necrosis, edema, enhancing IDC Collection: UPENN-GBM Links: model weights, github breast-fdg-pet-ct.zip Segment Description: FDG-avid lesions in breast from FDG PET/CT scans QIN-Breast IDC Collection: QIN-Breast Links: model weights, github breast-mr.zip Segment Description: Breast, Fibroglandular tissue, structural tumor IDC Collection: duke-breast-cancer-mri Links: model weights, github kidney-ct.zip Segment Description: Kidney, Tumor, and Cysts from contrast enhanced CT scans IDS Collection: TCGA-KIRC, TCGA-KIRP, TCGA-KICH, CPTAC-CCRCC Links: model weights, github liver-ct.zip Segment Description: Liver from CT scans IDC Collection: TCGA-LIHC Links: model weights, github liver2-ct.zip Segment Description: Liver and Lesions from CT scans IDC Collection: HCC-TACE-SEG, COLORECTAL-LIVER-METASTASES Links: model weights, github liver-mr.zip Segment Description: Liver from T1 MRI scans IDC Collection: TCGA-LIHC Links: model weights, github lung-ct.zip Segment Description: Lung and Nodules (3mm-30mm) from CT scans IDC Collections: Anti-PD-1-Lung LUNG-PET-CT-Dx NSCLC Radiogenomics RIDER Lung PET-CT TCGA-LUAD TCGA-LUSC Links: model weights 1, model weights 2, github lung2-ct.zip Improved model version Segment Description: Lung and Nodules (3mm-30mm) from CT scans IDC Collections: QIN-LUNG-CT, SPIE-AAPM Lung CT Challenge Links: model weights, github lung-fdg-pet-ct.zip Segment Description: Lungs and FDG-avid lesions in the lung from FDG PET/CT scans IDC Collections: ACRIN-NSCLC-FDG-PET Anti-PD-1-Lung LUNG-PET-CT-Dx NSCLC Radiogenomics RIDER Lung PET-CT TCGA-LUAD TCGA-LUSC Links: model weights, github prostate-mr.zip Segment Description: Prostate from T2 MRI scans IDC Collection: ProstateX, Prostate-MRI-US-Biopsy Links: model weights, github Changelog 2.0.2 - Fix the brain-mr segmentations to be transformed correctly 2.0.1 - added AIMI 2024 radiologist comments to qa-results.csv 2.0.0 - added AIMI 2024 segmentations 1.X - AIMI 2023 segmentations and reviewer scores

Federated learning enables big data for rare cancer boundary detection

Nature Communications · 2022 · 326 citations

• Computer Science
• Computer Science
• Machine Learning

Although machine learning (ML) has shown promise across disciplines, out-of-sample generalizability is concerning. This is currently addressed by sharing multi-site data, but such centralization is challenging/infeasible to scale due to various limitations. Federated ML (FL) provides an alternative paradigm for accurate and generalizable ML, by only sharing numerical model updates. Here we present the largest FL study to-date, involving data from 71 sites across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, reporting the largest such dataset in the literature (n = 6, 314). We demonstrate a 33% delineation improvement for the surgically targetable tumor, and 23% for the complete tumor extent, over a publicly trained model. We anticipate our study to: 1) enable more healthcare studies informed by large diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further analyses for glioblastoma by releasing our consensus model, and 3) demonstrate the FL effectiveness at such scale and task-complexity as a paradigm shift for multi-site collaborations, alleviating the need for data-sharing.

Clinical measures, radiomics, and genomics offer synergistic value in AI-based prediction of overall survival in patients with glioblastoma

Scientific Reports · 2022 · 86 citations

• Computer Science
• Artificial Intelligence
• Machine Learning

Multi-omic data, i.e., clinical measures, radiomic, and genetic data, capture multi-faceted tumor characteristics, contributing to a comprehensive patient risk assessment. Here, we investigate the additive value and independent reproducibility of integrated diagnostics in prediction of overall survival (OS) in isocitrate dehydrogenase (IDH)-wildtype GBM patients, by combining conventional and deep learning methods. Conventional radiomics and deep learning features were extracted from pre-operative multi-parametric MRI of 516 GBM patients. Support vector machine (SVM) classifiers were trained on the radiomic features in the discovery cohort (n = 404) to categorize patient groups of high-risk (OS < 6 months) vs all, and low-risk (OS ≥ 18 months) vs all. The trained radiomic model was independently tested in the replication cohort (n = 112) and a patient-wise survival prediction index was produced. Multivariate Cox-PH models were generated for the replication cohort, first based on clinical measures solely, and then by layering on radiomics and molecular information. Evaluation of the high-risk and low-risk classifiers in the discovery/replication cohorts revealed area under the ROC curves (AUCs) of 0.78 (95% CI 0.70-0.85)/0.75 (95% CI 0.64-0.79) and 0.75 (95% CI 0.65-0.84)/0.63 (95% CI 0.52-0.71), respectively. Cox-PH modeling showed a concordance index of 0.65 (95% CI 0.6-0.7) for clinical data improving to 0.75 (95% CI 0.72-0.79) for the combination of all omics. This study signifies the value of integrated diagnostics for improved prediction of OS in GBM.

Applications of Radiomics and Radiogenomics in High-Grade Gliomas in the Era of Precision Medicine

Cancers · 2021 · 74 citations

• Computer Science
• Medicine
• Medical physics

Machine learning (ML) integrated with medical imaging has introduced new perspectives in precision diagnostics of high-grade gliomas, through radiomics and radiogenomics. This has raised hopes for characterizing noninvasive and in vivo biomarkers for prediction of patient survival, tumor recurrence, and genomics and therefore encouraging treatments tailored to individualized needs. Characterization of tumor infiltration based on pre-operative multi-parametric magnetic resonance imaging (MP-MRI) scans may allow prediction of the loci of future tumor recurrence and thereby aid in planning the course of treatment for the patients, such as optimizing the extent of resection and the dose and target area of radiation. Imaging signatures of tumor genomics can help in identifying the patients who benefit from certain targeted therapies. Specifying molecular properties of gliomas and prediction of their changes over time and with treatment would allow optimization of treatment. In this article, we provide neuro-oncology, neuropathology, and computational perspectives on the promise of radiomics and radiogenomics for allowing personalized treatments of patients with gliomas and discuss the challenges and limitations of these methods in multi-institutional clinical trials and suggestions to mitigate the issues and the future directions.

Coronavirus Disease (COVID-19)-Related Disseminated Leukoencephalopathy: A Retrospective Study of Findings on Brain MRI

American Journal of Roentgenology · 2020 · 35 citations

• Medicine
• Pathology
• Radiology

Among 2820 inpatients with coronavirus disease (COVID-19), 59 (2.1%) underwent brain MRI. Of them, six (10.2%) had MRI findings suspicious for COVID-19-related disseminated leukoencephalopathy (CRDL), which is characterized by extensive confluent or multifocal white matter lesions (with characteristics and locations atypical for other causes), microhemorrhages, diffusion restriction, and enhancement. CRDL is an uncommon but important differential consideration in patients with neurologic manifestations of COVID-19.

Pearls &amp; Oy-sters: Bilateral globus pallidus lesions in a patient with COVID-19

Neurology · 2020 · 23 citations

• Sociology
• Medicine
• Medical education

Neurologic complications are occurring in coronavirus disease 2019 (COVID-19), and these patients should be monitored for neurologic symptoms.c When evaluating abnormal imaging findings in patients with COVID-19, the presence and specific pattern of deep gray structure involvement can be an important clue to etiology. Oy-sters cBrain imaging should be considered in the context of patients with COVID-19 with neurologic symptoms, even in the absence of focal findings on neurologic examination.c Given the dissociation between degree of hypoxemia and clinical symptoms that can be seen in patients with COVID-19, it is possible that unusual presentations of hypoxicischemic brain injury may emerge.COVID-19, caused by the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was originally described as a viral infection primarily affecting the respiratory tract.Neurologic complications are emerging, and have been reported in 36% of patients hospitalized with COVID-19 and in 46% of those with severe respiratory involvement. 1 The most common neurologic manifestations reported are dizziness, headache, impaired consciousness, dysgeusia, and hyposmia.An increased risk of stroke has also been identified.We report the case of a 52-year-old woman with bilateral globus pallidus lesions in the setting of COVID-19.The patient had a history of hypertension and newly diagnosed, poorly controlled type II diabetes mellitus (hemoglobin A1c of 17.4).She developed bilateral hand paresthesias the week prior to presentation, followed by dyspnea, cough, headache, and confusion.She presented to the emergency department and was afebrile, but tachycardic (115 beats per minute), hypertensive (220/118 mm Hg), and hypoxemic (oxygen saturation 49% on room air).She was alert and conversant, with no focal neurologic deficits.She had refractory hypoxemia despite 20 L/min supplemental oxygen.She was intubated and placed on mechanical ventilation for hypoxemic respiratory failure within 1 hour of presentation.SARS-CoV-2 was detected by rapid, real-time reverse-transcriptase polymerase chain reaction on the Cepheid GeneXpert system from a nasopharyngeal swab sample.Chest CT scan showed extensive bilateral, patchy, peripheral-predominant ground glass opacities with consolidation.Head CT demonstrated symmetric hypoattenuation in the bilateral globi pallidi with surrounding small foci of hyperattenuation (figure, A).Carboxyhemoglobin was not elevated and urine toxicology screen was negative.

Single-Cell Analyses Identify Brain Mural Cells Expressing CD19 as Potential Off-Tumor Targets for CAR-T Immunotherapies

Cell · 2020 · 451 citations

• Biology
• Cancer research
• Immunology