About

David Forsyth holds the Fulton Watson Copp Chair in Computer Science at the University of Illinois Urbana-Champaign. His research interests include computer vision, computer graphics, and machine learning. Forsyth is known for his work in these areas, contributing to the development of computational tools and techniques. He has taught courses related to computer vision, autonomous vehicle systems, and applied machine learning, and is recognized for his engagement in teaching, research, and advising within the field of computing and data science.

Research topics

• Artificial Intelligence
• Computer Science
• Machine Learning
• Theoretical computer science

Selected publications

• InvRGB+L: Inverse Rendering of Complex Scenes with Unified Color and LiDAR Reflectance Modeling

  2025-10-19

  articleOpen access

  We present InvRGB+L, a novel inverse rendering model that reconstructs large, relightable, and dynamic scenes from a single RGB+LiDAR sequence. Conventional inverse graphics methods rely primarily on RGB observations and use LiDAR mainly for geometric information, often resulting in suboptimal material estimates due to visible light interference. We find that LiDAR's intensity values-captured with active illumination in a different spectral range-offer complementary cues for robust material estimation under variable lighting. Inspired by this, InvRGB+L leverages LiDAR intensity cues to overcome challenges inherent in RGB-centric inverse graphics through two key innovations: (1) a novel physics-based LiDAR shading model and (2) RGB-LiDAR material consistency losses. The model produces novel-view RGB and LiDAR renderings of urban and indoor scenes and supports relighting, night simulations, and dynamic object insertions, achieving results that surpass current state-of-the-art methods in both scene-level urban inverse rendering and LiDAR simulation.

  PublisherOA PDFDOI

• A pseudo-3D computational approach for ultrasonic inspection of thin composite laminates with delamination emanating from a countersunk hole

  Nondestructive Testing And Evaluation · 2025-07-06 · 1 citations

  articleSenior author
  PublisherDOI

• Bimanual 3D Hand Motion and Articulation Forecasting in Everyday Images

  ArXiv.org · 2025-10-07

  preprintOpen access

  We tackle the problem of forecasting bimanual 3D hand motion & articulation from a single image in everyday settings. To address the lack of 3D hand annotations in diverse settings, we design an annotation pipeline consisting of a diffusion model to lift 2D hand keypoint sequences to 4D hand motion. For the forecasting model, we adopt a diffusion loss to account for the multimodality in hand motion distribution. Extensive experiments across 6 datasets show the benefits of training on diverse data with imputed labels (14% improvement) and effectiveness of our lifting (42% better) & forecasting (16.4% gain) models, over the best baselines, especially in zero-shot generalization to everyday images.

  PublisherOA PDFDOI

• Dequantization and Color Transfer with Diffusion Models

  2025-02-26 · 2 citations

  articleSenior author

  We demonstrate an image dequantizing diffusion model that enables novel edits on natural images. We propose operating on quantized images because they offer easy abstraction for patch-based edits and palette transfer. In particular, we show that color palettes can make the output of the diffusion model easier to control and interpret. We first establish that existing image restoration methods are not sufficient, such as JPEG noise reduction models. We then demonstrate that our model can generate natural images that respect the color palette the user asked for. For palette transfer, we propose a method based on weighted bipartite matching. We then show that our model generates plausible images even after extreme palette transfers, respecting user query. Our method can optionally condition on the source texture in part or all of the image. In doing so, we overcome a common problem in existing image colorization methods that are unable to produce colors with a different luminance than the input. We evaluate several possibilities for texture conditioning and their tradeoffs, including luminance, image gradients, and thresholded gradients, the latter of which performed best in maintaining texture and color control simultaneously. Our method can be usefully extended to another practical edit: recoloring patches of an image while respecting the source texture. Our procedure is supported by several qualitative and quantitative evaluations.

  PublisherDOI

• How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions

  2025-06-10

  article

  We tackle the novel problem of predicting 3D hand motion and contact maps (or Interaction Trajectories) given a single RGB view, action text, and a 3D contact point on the object as input. Our approach consists of (1) Interaction Codebook: a VQVAE model to learn a latent codebook of hand poses and contact points, effectively tokenizing interaction trajectories, (2) Interaction Predictor: a transformer-decoder module to predict the interaction trajectory from test time inputs by using an indexer module to retrieve a latent affordance from the learned codebook. To train our model, we develop a data engine that extracts 3D hand poses and contact trajectories from the diverse HoloAssist dataset. We evaluate our model on a benchmark that is 2.5-10× larger than existing works, in terms of diversity of objects and interactions observed, and test for generalization of the model across object categories, action categories, tasks, and scenes. Experimental results show the effectiveness of our approach over transformer & diffusion baselines across all settings.

  PublisherDOI

• Copy or Not? Reference-Based Face Image Restoration with Fine Details

  2025-02-26 · 1 citations

  article
  PublisherDOI

• Uncertainty Evaluation of Crack Sizing Capability Incorporating Model-based Inversion Applied to Bolt-hole Eddy Current Inspections

  Materials Evaluation · 2025-08-01 · 1 citations

  article

  There is a need for innovative methods to provide accurate sizing of discontinuities in structures, to guide maintenance actions and better inform engineering of the structure state. This paper highlights the development of the capability to size the length and depth of cracks in multilayer fastener sites using bolt-hole eddy current (BHEC) techniques. Technical efforts include improvements to model calibration, liftoff compensation, and the inversion process. In addition, an expanded set of surrogate models was developed that address crack sizing for titanium, aluminum, and steel structures across multiple frequencies and for varying hole diameters. A comprehensive crack sizing evaluation study was performed under a wide range of test conditions, demonstrating improved sizing capability over using peak amplitude. Crack length estimates were found to have less error than crack depth estimates, although crack depth is the more critical parameter for informing maintenance actions.

  PublisherDOI

• How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions

  ArXiv.org · 2025-04-16

  preprintOpen access

  We tackle the novel problem of predicting 3D hand motion and contact maps (or Interaction Trajectories) given a single RGB view, action text, and a 3D contact point on the object as input. Our approach consists of (1) Interaction Codebook: a VQVAE model to learn a latent codebook of hand poses and contact points, effectively tokenizing interaction trajectories, (2) Interaction Predictor: a transformer-decoder module to predict the interaction trajectory from test time inputs by using an indexer module to retrieve a latent affordance from the learned codebook. To train our model, we develop a data engine that extracts 3D hand poses and contact trajectories from the diverse HoloAssist dataset. We evaluate our model on a benchmark that is 2.5-10X larger than existing works, in terms of diversity of objects and interactions observed, and test for generalization of the model across object categories, action categories, tasks, and scenes. Experimental results show the effectiveness of our approach over transformer & diffusion baselines across all settings.

  PublisherOA PDFDOI

• UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video

  2025-03-25 · 1 citations

  article

  We present UrbanIR (Urban Scene Inverse Rendering), a new inverse graphics model that enables realistic, free-viewpoint renderings of scenes under various lighting conditions with a single video. It accurately infers shape, albedo, visibility, and sun and sky illumination from wide-baseline videos, such as those from car-mounted cameras, differing from NeRF's dense view settings. In this context, standard methods often yield subpar geometry and material estimates, such as inaccurate roof representations and numerous ‘floaters’. UrbanIR addresses these issues with novel losses that reduce errors in inverse graphics inference and rendering artifacts. Its techniques allow for precise shadow volume estimation in the original scene. The model's outputs support controllable editing, enabling photorealistic free-viewpoint renderings of night simulations, relit scenes, and inserted objects, marking a significant improvement over existing state-of-the-art methods. Our code and data will be made publicly available upon acceptance.

  PublisherDOI

• Denoising Monte Carlo Renders with Diffusion Models

  2025-03-25

  articleSenior author

  Physically-based renderings contain Monte Carlo noise, with variance that increases as the number of rays per pixel decreases. This noise, while zero-mean for good modern renderers, can have heavy tails (most notably, for scenes containing specular or refractive objects). Learned methods for restoring low fidelity renders are highly developed, because suppressing render noise means one can save compute and use fast renders with few rays per pixel. We demonstrate that a diffusion model can denoise low fidelity renders successfully. Furthermore, our method can be conditioned on a variety of natural render information, and this conditioning helps performance. Quantitative experiments show that our method is competitive with SOTA across a range of sampling rates. Qualitative examination of the reconstructions suggests that the image prior applied by a diffusion method strongly favors reconstructions that are “like” real images - so have straight shadow boundaries, curved specularities and no “fireflies.”

  PublisherDOI

Recent grants

• Interpreting Human Behaviour in Video using FSA's and Object Context

  NSF · $300k · 2006–2010

• RI: Medium: Creating Knowledge with All-Novel-Class Computer Vision

  NSF · $1.2M · 2021–2026

• RI: Small: Exploiting Geometric and Illumination Context in Indoor Scenes

  NSF · $450k · 2009–2013

• Collaborative Research: Computational Behavioral Science: Modeling, Analysis, and Visualization of Social and Communicative Behavior

  NSF · $1.5M · 2010–2017

Frequent coauthors

• Azrif Manut

  Universiti Teknologi MARA

  67 shared

• Ahmad Sabirin Zoolfakar

  Universiti Teknologi MARA

  66 shared

• Ahmad Alabqari

  Tun Hussein Onn University of Malaysia

  66 shared

• Badrul Hisham

  Technical University of Malaysia Malacca

  66 shared

• Ma Radzi

  Universiti Teknologi MARA

  66 shared

• Chutisant Kerdvibulvech

  National Institute of Development Administration

  65 shared

• Datta Chavan

  National Institute of Development Administration

  65 shared

• Azli Yahya

  65 shared

Education

• Ph.D., Computer Science

  University of Illinois at Urbana-Champaign

  1990

• M.S., Computer Science

  University of Illinois at Urbana-Champaign

  1986

• B.S., Computer Science

  University of Illinois at Urbana-Champaign

  1984

Awards & honors

• Mark Everingham Prize