Adaptive Estimation and Optimal Control in Offline Contextual MDPs without Stationarity

ArXiv.org · 2026-05-05

Contextual MDPs are powerful tools with wide applicability in areas from biostatistics to machine learning. However, specializing them to offline datasets has been challenging due to a lack of robust, theoretically backed methods. Our work tackles this problem by introducing a new approach towards adaptive estimation and cost optimization of contextual MDPs. This estimator, to the best of our knowledge, is the first of its kind, and is endowed with strong optimality guarantees. We achieve this by overcoming the key technical challenges evolving from the endogenous properties of contextual MDPs; such as non-stationarity, or model irregularity. Our guarantees are established under complete generality by utilizing the relatively recent and powerful statistical technique of $T$-estimation (Baraud, 2011). We first provide a procedure for selecting an estimator given a sample from a contextual MDP and use it to derive oracle risk bounds under two distinct, but nevertheless meaningful, loss functions. We then consider the problem of determining the optimal control with the aid of the aforementioned density estimate and provide finite sample guarantees for the cost function.

Adaptive Estimation and Optimal Control in Offline Contextual MDPs without Stationarity

arXiv (Cornell University) · 2026-05-05

Contextual MDPs are powerful tools with wide applicability in areas from biostatistics to machine learning. However, specializing them to offline datasets has been challenging due to a lack of robust, theoretically backed methods. Our work tackles this problem by introducing a new approach towards adaptive estimation and cost optimization of contextual MDPs. This estimator, to the best of our knowledge, is the first of its kind, and is endowed with strong optimality guarantees. We achieve this by overcoming the key technical challenges evolving from the endogenous properties of contextual MDPs; such as non-stationarity, or model irregularity. Our guarantees are established under complete generality by utilizing the relatively recent and powerful statistical technique of $T$-estimation (Baraud, 2011). We first provide a procedure for selecting an estimator given a sample from a contextual MDP and use it to derive oracle risk bounds under two distinct, but nevertheless meaningful, loss functions. We then consider the problem of determining the optimal control with the aid of the aforementioned density estimate and provide finite sample guarantees for the cost function.

26-A-17196-ACC OPPORTUNISTIC SCREENING FOR ADVERSE CARDIOVASCULAR EVENTS USING ROUTINE CLINICAL CHEST COMPUTED TOMOGRAPHY

Journal of the American College of Cardiology · 2026-03-27

Self‐supervised out‐of‐distribution detection—Metal implants and other anomaly

Medical Physics · 2026-02-01

BACKGROUND: Despite the high precision of deep learning models on internal tests on CT, their effectiveness often drops on external validation due to artifacts caused by patient motion and implants like metal or silicone that were not accounted for in the carefully curated training data. Potential wide categories of "unknown" anomalies within a CT exam makes training a supervised model for out-of-distribution (OOD) identification impractical, especially when considering unseen external data. PURPOSE: To develop an artificial intelligence (AI) model to detect and identify anomalies/OOD data in abdominal-pelvis CT exams for the purpose of improving the performance of downstream applications. METHODS: Our proposed 2D and 3D generative architecture receives the third lumbar vertebra (L3) slice (slice-level model) or all the slices from the series (series-level model), generates a reconstruction and the secondary part of our architecture-anomaly score computation block, computes the anomalies pixels/voxels (slice-level/series-level) to identify anomalous L3-slices/volumes (slice-level/series-level). We trained on data from over 2850 abdominal-pelvis CT volumes from adults over age 50 years collected throughout multiple Mayo Clinic campuses (60% female; mean age: 66.9, 92.4% non-Hispanic White) and tested on a prospective test set of 544 CTs from July 2024 (47.3% female; mean age: 70.9, 94% non-Hispanic White) as well as an external test set. RESULTS: We found that while traditional methods show moderate success, our generative models-Vector Quantized Variational Autoencoder (VQVAE) and Vision Transformer-Masked Autoencoder (VIT-MAE)-deliver excellent results with negligible false positives (FPs) and are also superior in identifying varied types of OOD samples. Prospective analysis showed the model was able to handle the under-documentation of anomaly in radiology reports with 86.11% true positive (TP) rate. We also performed external validation using the publicly available AbdominalCT-1k dataset, which contains 1062 CT scans compiled from several existing benchmark datasets. The model achieved a 75.26% TP rate, while the 24.7% FP rate was primarily triggered by anomalies located outside the body. CONCLUSIONS: The proposed method can be leveraged to detect both intra- and interclass OOD data from abdominal CT images and can assess the quality of CT datasets to provide actionable insights. This workflow is particularly valuable for nonshareable healthcare collaborations, where it can be deployed as a service within local firewalls for automated dataset curation without prior knowledge about the OOD types. The implementation of the algorithm is available in the GitHub: https://github.com/gokul-ramasamy/implant_detection.git.

Artificial intelligence–based quantification of breast arterial calcifications to predict cardiovascular morbidity and mortality

European Heart Journal · 2026-02-10 · 2 citations

BACKGROUND AND AIMS: Women are underdiagnosed and undertreated for cardiovascular disease (CVD). Automatic quantification of breast arterial calcification (BAC) on screening mammography can identify women at risk for CVD. This study aimed to determine whether artificial intelligence-based automatic quantification of BAC from screening mammograms predicts CVD and mortality beyond PREVENT scores in a large, racially diverse, multi-institutional population. METHODS: This retrospective cohort study included 123 762 women from two healthcare systems who had screening mammograms. Breast arterial calcification was quantified using a transformer-based neural network for segmentation. Breast arterial calcification severity was categorized as zero (0 mm2), mild (>0-10 mm2), moderate (>10-25 mm2), and severe (>25 mm2). Kaplan-Meier analysis, Cox proportional hazards, and Fine-Gray competing event models were used to examine the association between BAC and major adverse cardiovascular events (MACE). RESULTS: Breast arterial calcification was detected in 16.1% (internal cohort) and 20.6% (external cohort) of women and provided significant prognostic value incremental to the PREVENT score. In PREVENT adjusted models, a clear dose-response was observed. Compared with zero BAC, mild [internal: hazard ratio (HR) 1.32, 95% confidence interval (CI) 1.10-1.59; external: HR 1.28, 95% CI 1.17-1.39], moderate (internal: HR 1.75, 95% CI 1.23-2.50; external: HR 1.79, 95% CI 1.55-2.06), and severe BAC (internal: HR 3.29, 95% CI 2.15-5.05; external: HR 2.80, 95% CI 2.36-3.32) were all prognostic for any MACE. Each 1 mm2 increase in BAC conferred an additional 2%-3% risk for MACE (P < .001). CONCLUSIONS: Automatically quantified BAC is an independent predictor of MACE and mortality, adding prognostic value to the PREVENT score. This approach may provide an opportunistic cardiovascular risk assessment during routine mammography screening without additional radiation exposure to guide earlier and more effective preventive care for women.

26-A-16927-ACC ECHOYOLO: AUTOMATED FINE-GRAINED DETECTION OF CARDIAC STRUCTURES IN ECHOCARDIOGRAPHY

Journal of the American College of Cardiology · 2026-03-27

CLEAR-AI: confounder-aware learning for equitable and accurate reasoning in AI for diagnosis

Journal of Medical Imaging · 2026-05-22

PurposeA critical challenge impeding the deployment of artificial intelligence (AI) models in healthcare lies in implicit bias against multiple correlated sensitive attributes.ApproachWe developed a multibranch adversarial debiasing approach that can debias a multilabel diagnostic model simultaneously for correlating confounding factors using a dynamic weighted gradient reversal technique to reduce disparities. The proposed methodology was evaluated on chest X-ray imaging data—trained on CheXpert and evaluated on two external datasets [MIMIC-CXR (Beth Israel, Massachusetts, United States) and Emory Healthcare (Atlanta, Georgia, United States)].ResultsThe disease classification performance of the proposed debiased method significantly overlaps with the baselines, indicating no drop in the task performance. We found the debiased model to reduce TPR and FPR disparities across multifactor subgroups [age, race, and support device(s)] while maintaining overall task performance.ConclusionsAlthough we focused on three specific confounders, the proposed adversarial debiasing framework readily extends to account for an arbitrary number of sensitive variables. The findings highlight the promising potential of adversarial training techniques to enhance fairness and trustworthiness in the deployment of AI models in diverse healthcare settings.

CXR-LT 2024: A MICCAI challenge on long-tailed, multi-label, and zero-shot disease classification from chest X-ray

Medical Image Analysis · 2025-07-29 · 5 citations

Multi-Analyte, Swab-Based Automated Wound Monitor with AI

2025-08-10

Diabetic foot ulcers (DFUs), a class of chronic wounds, affect ~750,000 individuals every year in the US alone and identifying non-healing DFUs that develop to chronic wounds early can drastically reduce treatment costs and minimize risks of amputation. There is therefore a pressing need for diagnostic tools that can detect non-healing DFUs early. We develop a low cost, multi-analyte 3D printed assays seamlessly integrated on swabs that can identify non-healing DFUs and a Wound Sensor iOS App - an innovative mobile application developed for the controlled acquisition and automated analysis of wound sensor data. By comparing both the original base image (before exposure to the wound) and the wound-exposed image, we developed automated computer vision techniques to compare density changes between the two assay images, which allow us to automatically determine the severity of the wound. The iOS app ensures accurate data collection and presents actionable insights, despite challenges such as variations in camera configurations and ambient conditions. The proposed integrated sensor and iOS app will allow healthcare professionals to monitor wound conditions real-time, track healing progress, and assess critical parameters related to wound care.

Foundation versus domain-specific models for left ventricular segmentation on cardiac ultrasound

npj Digital Medicine · 2025-06-06 · 2 citations

The Segment Anything Model (SAM) was fine-tuned on the EchoNet-Dynamic dataset and evaluated on external transthoracic echocardiography (TTE) and Point-of-Care Ultrasound (POCUS) datasets from CAMUS (University Hospital of St Etienne) and Mayo Clinic (99 patients: 58 TTE, 41 POCUS). Fine-tuned SAM was superior or comparable to MedSAM. The fine-tuned SAM also outperformed EchoNet and U-Net models, demonstrating strong generalization, especially on apical 2-chamber (A2C) images (fine-tuned SAM vs. EchoNet: CAMUS-A2C: DSC 0.891 ± 0.040 vs. 0.752 ± 0.196, p < 0.0001) and POCUS (DSC 0.857 ± 0.047 vs. 0.667 ± 0.279, p < 0.0001). Additionally, SAM-enhanced workflow reduced annotation time by 50% (11.6 ± 4.5 sec vs. 5.7 ± 1.7 sec, p < 0.0001) while maintaining segmentation quality. We demonstrated an effective strategy for fine-tuning a vision foundation model for enhancing clinical workflow efficiency and supporting human-AI collaboration.