About

James C. Gee, Ph.D., is a Professor of Radiologic Science in Radiology at the Perelman School of Medicine, University of Pennsylvania. He serves as the Director of the Penn Image Computing and Science Laboratory within the Department of Radiology and is Co-Director of the Translational Bio-Imaging Center at the Institute of Translational Medicine and Therapeutics. Additionally, he is a Faculty Affiliate at the General Robotics, Automation, Sensing and Perception (GRASP) Laboratory and the Director of the MSE in Data Science Online Program at the School of Engineering and Applied Science. Dr. Gee's educational background includes a B.S. in Computer Science and Electrical Engineering from the University of Washington, obtained in 1987, followed by an M.S. in Electrical Engineering in 1989, and a Ph.D. in Computer and Information Science from the University of Pennsylvania in 1996. His research focuses on biomedical imaging, informatics, and the development of computational methods for medical image analysis. He is actively involved in academic leadership, including co-chairing the Committee on Appointments and Promotions in the Department of Radiology. His contributions extend to advancing the fields of neuroimaging, machine learning applications in medical imaging, and the development of automated segmentation techniques, as evidenced by his numerous publications.

Research topics

• Computer Science
• Biology
• Anatomy
• Computational biology
• Artificial Intelligence
• Neuroscience
• Medicine
• Genetics
• Data Mining
• Machine Learning
• Cartography
• Data science
• Environmental health
• Evolutionary biology
• Geography
• Software engineering
• Programming language
• Cell biology

Selected publications

• Factored Levenberg-Marquardt for Diffeomorphic Image Registration: An efficient optimizer for FireANTs

  ArXiv.org · 2026-03-19

  articleOpen accessSenior author

  FireANTs introduced a novel Eulerian descent method for plug-and-play behavior with arbitrary optimizers adapted for diffeomorphic image registration as a test-time optimization problem, with a GPU-accelerated implementation. FireANTs uses Adam as its default optimizer for fast and more robust optimization. However, Adam requires storing state variables (i.e. momentum and squared-momentum estimates), each of which can consume significant memory, prohibiting its use for significantly large images. In this work, we propose a modified Levenberg-Marquardt (LM) optimizer that requires only a single scalar damping parameter as optimizer state, that is adaptively tuned using a trust region approach. The resulting optimizer reduces memory by up to 24.6% for large volumes, and retaining performance across all four datasets. A single hyperparameter configuration tuned on brain MRI transfers without modification to lung CT and cross-modal abdominal registration, matching or outperforming Adam on three of four benchmarks. We also perform ablations on the effectiveness of using Metropolis-Hastings style rejection step to prevent updates that worsen the loss function.

  PublisherOA PDF

• Deep Computational Anatomy via Latent-Aligned Multiview Normalizing Flows

  bioRxiv (Cold Spring Harbor Laboratory) · 2026-05-10

  articleOpen access

  In modeling complex probability distributions, normalizing flows provide exact-likelihood, bijective mappings between empirical data and tractable latent spaces. Building on this foundation, latent-aligned multiview normalizing (LAMNr) flows leverage these salient properties to learn shared latent subspaces across heterogeneous, multimodal datasets while simultaneously topologically unfolding the sampled data manifold into a continuous vector space. Formal latent-alignment constraints are used to model shared structural features separate from view-specific variations, coordinating latent projections into a shared geometric subspace. By applying this transformation in the context of biological imaging, the framework establishes a potential basis for a deep learning interpretation of foundational computational anatomy concepts, such as the population template, latent distances, and geodesic pairwise image interpolation. Additionally, the proposed framework enables closed-form conditional modeling for exact cross-view imputation and other latent space manipulations. Evaluations and illustrations on both imaging-derived phenotypes (IDPs) and multimodal MRI demonstrate the proposed framework and potential applications. To further motivate our work, we provide a robust and comprehensive, 2D and 3D open-source implementation in PyTorch, natively integrated with the ANTsX ecosystem (i.e., ANTsTorch) for efficient training and subsequent data transformation, manipulation, and analysis.

  PublisherDOI

• Knowledge Transfer Scaling Laws for 3D Medical Imaging

  ArXiv.org · 2026-05-07

  articleOpen access

  Vision foundation models are increasingly moving beyond 2D to volumetric domains such as 3D medical imaging, where unified pretraining across different imaging modalities (i.e. CT, MRI, and PET) could provide foundational models for diverse clinical tasks. However, training such models requires mixing heterogeneous imaging domains, and current mixture strategies remain largely heuristic. In this work, we observe that different medical imaging domains scale at variable rates during pretraining, and knowledge transfer between domains is strongly asymmetric: training on one domain can substantially improve another, but the reverse may be much weaker. Interestingly, both MAE reconstruction loss and cross-domain transfer follow predictable power-law trends with domain-specific behaviors. Motivated by these findings, we formulate data allocation as a scaling-law optimization problem. The derived allocations reveal an interpretable hub-and-island structure: highly transferable domains emerge as hubs that benefit many others and deserve strategic allocation, while isolated domains act as islands requiring direct investment. Empirically, transfer-aware allocation outperforms data-proportional sampling by up to 58% and generalizes well to unseen budgets with r=0.989. Downstream validation on disease classification and organ/lesion segmentation further confirms that the derived transfer-aware mixtures provide stronger pretrained representations for clinical 3D medical imaging tasks.

  PublisherOA PDF

• Toward explainable AI approaches for breast imaging: adapting foundation models to diverse populations

  2026-04-02

  article
  PublisherDOI

• Contusions bias cortical thickness estimates after traumatic brain injury: A TRACK-TBI study

  NeuroImage Clinical · 2026-01-01

  articleOpen access

  BACKGROUND: Traumatic brain injury (TBI) is characterized by both focal and diffuse pathology. Automated cortical thickness estimation is widely used to quantify structural brain changes following TBI; however, the impact of focal pathology such as contusions on cortical thickness estimates in TBI remains unknown. METHODS: We evaluated lesion-induced bias in cortical thickness under three lesion-handling strategies in 86 TRACK-TBI participants with MRI at 2 weeks and 6 months post-injury. Cortical thickness was estimated using the ANTsNetCT longitudinal pipeline with the default pipeline (No Masking), masking lesion voxels from summarization (Atlas Masking), and masking lesion voxels from cortical thickness estimation (Full Masking). Cross-sectional and longitudinal cortical thickness in unilaterally lesioned regions were compared with their contralesional homologues using linear mixed-effects models. The effectiveness of each lesion handling strategy was then evaluated using nonparametric bootstrap analyses to test whether bias was systematically present across all regions. RESULTS: At 2 weeks post-injury, six cortical regions demonstrated significant lesion-associated bias. Collectively across all regions, bias was observed in the No Masking and Atlas-Masking approaches. This bias was significantly attenuated in the Fully Masked approach. Longitudinally, the unmasked data also showed significant lesion-related differences in cortical thickness change across multiple temporal and frontal regions, with persistent effects in the Atlas- and Full Masking approaches. CONCLUSIONS: Contusions appear to introduce cross-sectional and longitudinal bias in cortical thickness estimates, inflating cross-sectional values and potentially exaggerating atrophy longitudinally. Excluding lesion voxels from tissue probability maps attenuates cross-sectional bias, providing a baseline that improves accuracy and interpretation of neuroimaging biomarkers in TBI.

  PublisherDOI

• Factored Levenberg-Marquardt for Diffeomorphic Image Registration: An efficient optimizer for FireANTs

  arXiv (Cornell University) · 2026-03-19

  preprintOpen accessSenior author

  FireANTs introduced a novel Eulerian descent method for plug-and-play behavior with arbitrary optimizers adapted for diffeomorphic image registration as a test-time optimization problem, with a GPU-accelerated implementation. FireANTs uses Adam as its default optimizer for fast and more robust optimization. However, Adam requires storing state variables (i.e. momentum and squared-momentum estimates), each of which can consume significant memory, prohibiting its use for significantly large images. In this work, we propose a modified Levenberg-Marquardt (LM) optimizer that requires only a single scalar damping parameter as optimizer state, that is adaptively tuned using a trust region approach. The resulting optimizer reduces memory by up to 24.6% for large volumes, and retaining performance across all four datasets. A single hyperparameter configuration tuned on brain MRI transfers without modification to lung CT and cross-modal abdominal registration, matching or outperforming Adam on three of four benchmarks. We also perform ablations on the effectiveness of using Metropolis-Hastings style rejection step to prevent updates that worsen the loss function.

  PublisherDOI

• NEUSEG: Interpretable Unsupervised GM/WM Segmentation in Brain Histopathology Using Nuclei Morphometrics

  2026-04-08

  article

  We present NEUSEG, a fully automated, interpretable, and unsupervised pipeline for gray/white matter (GM/WM) segmentation in brain histopathology whole-slide images (WSIs). GM/WM segmentation in WSIs is challenging due to their gigapixel-scale resolution and substantial variability in staining, regional morphology, and underlying pathology. To address this, NEUSEG combines nuclei morphometrics with a two-component Gaussian Mixture Model and refines boundaries using a Conditional Random Field and morphological post-processing. In comparison with a supervised CNN baseline, NEUSEG achieved comparable accuracy on in-distribution slides and superior robustness under out-of-distribution conditions. Across 252 WSIs with expert-annotated regions of interest (ROIs), median annotation-to-contour distances were 32.89 μm for the GMBG boundary and 129.79 μm for the GM-WM boundary, with no significant difference across stains or pathology types and only minor regional variation. Percent area occupied between expertand segmentation-derived ROIs showed near-perfect agreement (r=0.987). NEUSEG is lightweight, CPU-operable, and processes each WSI within minutes, enabling scalable and robust GM/WM segmentation across heterogeneous histopathology datasets.

  PublisherDOI

• Knowledge Transfer Scaling Laws for 3D Medical Imaging

  arXiv (Cornell University) · 2026-05-07

  preprintOpen access

  Vision foundation models are increasingly moving beyond 2D to volumetric domains such as 3D medical imaging, where unified pretraining across different imaging modalities (i.e. CT, MRI, and PET) could provide foundational models for diverse clinical tasks. However, training such models requires mixing heterogeneous imaging domains, and current mixture strategies remain largely heuristic. In this work, we observe that different medical imaging domains scale at variable rates during pretraining, and knowledge transfer between domains is strongly asymmetric: training on one domain can substantially improve another, but the reverse may be much weaker. Interestingly, both MAE reconstruction loss and cross-domain transfer follow predictable power-law trends with domain-specific behaviors. Motivated by these findings, we formulate data allocation as a scaling-law optimization problem. The derived allocations reveal an interpretable hub-and-island structure: highly transferable domains emerge as hubs that benefit many others and deserve strategic allocation, while isolated domains act as islands requiring direct investment. Empirically, transfer-aware allocation outperforms data-proportional sampling by up to 58% and generalizes well to unseen budgets with r=0.989. Downstream validation on disease classification and organ/lesion segmentation further confirms that the derived transfer-aware mixtures provide stronger pretrained representations for clinical 3D medical imaging tasks.

  PublisherDOI

• Functional Thoracic MRI: Recent Advances in Pulmonary Assessment

  Radiology Cardiothoracic Imaging · 2025-09-04 · 1 citations

  review

  Functional thoracic MRI provides regional assessment of the three principal components of lung function: ventilation, perfusion, and gas exchange. It offers advantages over pulmonary function tests like spirometry, which yield only global measurements. MRI enables comprehensive evaluation of respiratory mechanics, including chest wall and diaphragm motion, dynamic large airway instability, and lung ventilation using various contrast mechanisms and gas agents. Perfusion imaging, with or without exogenous contrast material, further supports the assessment of mechanical lung properties in both healthy and diseased states. Advanced MRI techniques also allow for quantification of distal airspace dimensions and gas exchange or diffusion capacity using inert noble gases, at both global and regional levels. Dynamic contrast-enhanced perfusion MRI enables assessment of key pathophysiologic mechanisms, such as hypoxic pulmonary vasoconstriction, and provides direct visualization of ventilation-perfusion mismatch across various lung diseases. Emerging noninvasive, non-contrast-enhanced techniques, including combined ventilation-perfusion imaging based on signal oscillations from blood flow and respiration, hold substantial promise for clinical translation. This review provides an overview of recent advances in functional thoracic MRI for evaluating regional lung function and pathophysiology.

  PublisherDOI

• Toward a robust lesion detection model in breast DCE-MRI: adapting foundation models to high-risk women

  ArXiv.org · 2025-09-02

  preprintOpen access

  Accurate breast MRI lesion detection is critical for early cancer diagnosis, especially in high-risk populations. We present a classification pipeline that adapts a pretrained foundation model, the Medical Slice Transformer (MST), for breast lesion classification using dynamic contrast-enhanced MRI (DCE-MRI). Leveraging DINOv2-based self-supervised pretraining, MST generates robust per-slice feature embeddings, which are then used to train a Kolmogorov--Arnold Network (KAN) classifier. The KAN provides a flexible and interpretable alternative to conventional convolutional networks by enabling localized nonlinear transformations via adaptive B-spline activations. This enhances the model's ability to differentiate benign from malignant lesions in imbalanced and heterogeneous clinical datasets. Experimental results demonstrate that the MST+KAN pipeline outperforms the baseline MST classifier, achieving AUC = 0.80 \pm 0.02 while preserving interpretability through attention-based heatmaps. Our findings highlight the effectiveness of combining foundation model embeddings with advanced classification strategies for building robust and generalizable breast MRI analysis tools.

  PublisherOA PDFDOI

Recent grants

• Advanced Normalization Tools

  NIH · $2.7M · 2022–2026

• NIH Grant R03EB009321

  NIH · $158k · 2012

• Waxholm Space for Rodent Neuroinformatics

  NIH · $1.8M · 2016–2022

• ITK-Lung: A Software Framework for Lung Image Processing and Analysis

  NIH · $2.3M · 2017–2024

• NIH Grant R03MH062100

  NIH · $159k · 2004

Frequent coauthors

• Brian Avants

  144 shared

• Nicholas J. Tustison

  University of Virginia

  105 shared

• Paul A. Yushkevich

  University of Pennsylvania

  85 shared

• Murray Grossman

  73 shared

• Hui Zhang

  55 shared

• Corey T. McMillan

  University of Pennsylvania

  55 shared

• Jeffrey Duda

  University of Pennsylvania

  54 shared

• Philip A. Cook

  University of Pennsylvania

  53 shared