About

Gopi Meenakshisundaram is a professor in the Department of Computer Science at UC Irvine's Donald Bren School of Information & Computer Sciences. His research areas include computer graphics, computer vision, and human-computer interaction, with a focus on geometry, topology, graph algorithms, user-interaction, and collaboration. His work involves generating, capturing, representing, rendering, and interacting with synthetic and real-world images and video, as well as understanding how people adopt, adapt, and respond to information systems.

Selected publications

• RegBoost: Enhancing mouse brain image registration using geometric priors and Laplacian interpolation

  NeuroImage · 2024-12-26

  articleOpen accessSenior authorCorresponding

  We show in this work that incorporating geometric features and geometry processing algorithms for mouse brain image registration broadens the applicability of registration algorithms and improves the registration accuracy of existing methods. We introduce the preprocessing and postprocessing steps in our proposed framework as RegBoost. We develop a method to align the axis of 3D image stacks by detecting the central planes that pass symmetrically through the image volumes. We then find geometric contours by defining external and internal structures to facilitate image correspondences. We establish Dirichlet boundary conditions at these correspondences and find the displacement map throughout the volume using Laplacian interpolation. We discuss the challenges in our standalone framework and demonstrate how our new approaches can improve the results of existing image registration methods. We expect our new approach and algorithms will have critical applications in brain mapping projects. • Geometric features for registration improve upon existing registration results. • Using 3D over 2D operators ensures better results and continuities across 3D data. • Fully automated registration framework aligns input data to an annotated atlas.

  PublisherDOI

• Real-Time Seamless Multi-Projector Displays on Deformable Surfaces

  IEEE Transactions on Visualization and Computer Graphics · 2024-03-04 · 8 citations

  article

  Prior works on multi-projector displays have focused primarily on static rigid objects, some focusing on dynamic rigid objects. However, works on projection based displays on deformable dynamic objects have focused only on small scale single projector displays. Tracking a deformable dynamic surface and updating projections precisely in real time on it is a significantly challenging task, even for a single projector system. In this paper, we present the first end-to-end solution for achieving a real-time, seamless display on deformable surfaces using mutliple unsychronized projectors without requiring any prior knowledge of the surface or device parameters. The system first accurately calibrates multiple RGB-D cameras and projectors using the deformable display surface itself, and then using those calibrated devices, tracks the continuous changes in the surface shape. Based on the deformation and projector calibration, the system warps and blends the image content in real-time to create a seamless display on a surface that continuously changes shape. Using multiple projectors and RGB-D cameras, we provide the much desired aspect of scale to the displays on deformable surfaces. Most prior dynamic multi-projector systems assume rigid objects and depend critically on the constancy of surface normals and non-existence of local shape deformations. These assumptions break in deformable surfaces making prior techniques inapplicable. Point-based correspondences become inadequate for calibration, exacerbated with no synchronization between the projectors. A few works address non-rigid objects with several restrictions like targeting semi-deformable surfaces (e.g. human face), or using single coaxial (optically aligned) projector-camera pairs, or temporally synchronized cameras. We break loose from such restrictions and handle multiple projector systems for dynamic deformable fabric-like objects using temporally unsynchronized devices. We devise novel methods using ray and plane-based constraints imposed by the pinhole camera model to address these issues and design new blending methods dependent on 3D distances suitable for deformable surfaces. Finally, unlike all prior work with rigid dynamic surfaces that use a single RGB-D camera, we devise a method that involve all RGB-D cameras for tracking since the surface is not seen completely by a single camera. These methods enable a seamless display at scale in the presence of continuous movements and deformations. This work has tremendous applications on mobile and expeditionary systems where environmentals (e.g. wind, vibrations, suction) cannot be avoided. One can create large displays on tent walls in remote, austere military or emergency operations in minutes to support large scale command and control, mission rehearsal or training operations. It can be used to create displays on mobile and inflatable objects for tradeshows/events and touring edutainment applications.

  PublisherDOI

• Automated Landmark Detection for AR-Based Craniofacial Surgical Assistance System

  Smart innovation, systems and technologies · 2024-01-01 · 1 citations

  book-chapter
  PublisherDOI

• Illuminating precise stencils on surgical sites using projection-based augmented reality

  Smart Health · 2024-04-05 · 5 citations

  articleOpen access

  In this paper we propose a system that connects surgeons to remote or local experts who provide real-time surgical guidance by illuminating salient markings or stencils (e.g. points, lines and curves) on the physical surgical site using a projector. The projection can be modified in real time by the expert using a GUI and can be seen by all in the operating room (OR) without the use of any wearables. This system overcomes the limitations of AR/VR headsets which can overlay information through a headset, but are obtrusive, not very accurate with movements, and visible only to the surgeon excluding others in the room. Overlaying information, at high precision, directly on the physical surgical site that can be seen by everyone in the OR can become an useful tool for skill transfer, expert consultation and training, especially in telemedicine. In addition to the projector, the system comprises of a RGB-D camera (e.g. Kinect) for feedback, together designated as the PDC (Projector Depth Camera) unit. The PDC is driven by a PC. The RGB-D camera provides depth information in addition to an image at video frame rates. A high resolution mesh of the surgical site is captured using the PDC unit initially. During the surgical planning, training or execution session, this digital model can be marked by appropriate incision markings on a tablet or monitor using touch based or mouse based interface, on the same local machine or after being transmitted to a remote machine. These markings are then communicated back to the PDC unit and illuminated at high precision via the projector on the surgical site in real time. If the surgical site moves during the process, the movement is tracked and updated quickly on the surgical site. Our method specifically overcomes the obtrusive, exclusive, and indirect attributes of headsets and displays while maintaining high accuracy of registration with movements.

  PublisherDOI

• Self-Calibrating Dynamic Projection Mapping System for Dynamic, Deformable Surfaces with Jitter Correction and Occlusion Handling

  2023-10-16 · 7 citations

  article

  Dynamic projection mapping (DPM) is becoming increasingly popular, enabling viewers to visualize information on moving and deformable surfaces. Examples include large data visualization on the moving walls of tents deployed in austere remote locations during emergency management or defense operations. A DPM system typically comprises a RGB-D camera and a projector. In this paper, we present the first fully functional DPM system that auto-calibrates (without any physical props like planar checkerboard or rigid 3D objects) and creates a comprehensible display in the presence of large and fast movements by managing jitter and occlusion by passing objects.Prior DPM systems need specific calibration props, manual inputs and in order to deliver sub-pixel calibration accuracy. Recalibration in the face of movement or change in system setup becomes a time consuming process where the calibration prop needs to be brought back. When rendering content using DPM, errors in calibration are exacerbated and the noise in the depth camera leads to jitter, making the projection unreadable or incomprehensible. Occlusion may disrupt operations completely by jumbling up even the unoccluded parts of the display.In this paper we propose key hardware-agnostic methods for DPM calibration and rendering to make DPM systems easily deployable, stable and legible. First, we present a novel projector-camera calibration that does not need synchronization of the devices and leverages the moving surface itself, a counter-intuitive proposition. We project ArUCo markers on the moving surface and use corresponding detected features of these markers in the RGB and depth camera over multiple frames to accurately estimate the intrinsics and extrinsics of both the projector and the RGB-D camera. Second, we present a DPM rendering method that uses Kalman filtering models to reduce jitter and predict the surface shape in the presence of short term occlusions by other static objects. This results in the first DPM system, to the best of our knowledge, that can auto-calibrate in minutes and can render high resolution content like high-resolution text or images comprehensible even in the presence of fast movements, deformations and occlusions. We compare and evaluate the accuracy with prior methods and analyze the effect of surface movement on the calibration accuracy.

  PublisherDOI

• Projector-Camera Calibration on Dynamic, Deformable Surfaces

  2023-03-01 · 8 citations

  article

  Dynamic projection mapping (DPM) enables viewers to visualize information on dynamic, deformable objects. Such systems often comprise of a RGB-D camera and projector pair that must be calibrated apriori. Most calibration techniques require specific static calibration objects of known geometry. In this paper, we propose the first projector-camera calibration technique for DPM that uses the dynamic, deformable surface itself to calibrate the devices, without needing to bring in a static, rigid calibration object. Our method is hardware agnostic, fast, and accurate and allows quick recalibration.

  PublisherDOI

• Projector Illuminated Precise Stencils on Surgical Sites

  2023-03-01 · 3 citations

  article

  We propose a system that provides realtime guidance to surgeons by illuminating salient markings on the physical surgical site using a projector. In addition to the projector, the system uses a RGB-D camera for feedback. This unit is called the projector-depth-camera or PDC unit. Using the PDC, we perform structured light scanning and generate a high resolution mesh of the surgical site. During planning or execution of the surgery, this digital model is marked by appropriate incision markings through a GUI. These markings are then illuminated at high precision via the PDC unit on the surgical site in realtime. If the surgical site moves during the process, the movement is tracked by the system and updated quickly on the moved surgical site.

  PublisherDOI

• 3D Gamut Morphing for Non-Rectangular Multi-Projector Displays

  IEEE Transactions on Visualization and Computer Graphics · 2023-05-18 · 3 citations

  articleSenior author

  In a spatially augmented reality system, multiple projectors are tiled on a complex shaped surface to create a seamless display on it. This has several applications in visualization, gaming, education and entertainment. The main challenges in creating seamless and undistorted imagery on such complex shaped surfaces are geometric registration and color correction. Prior methods that provide solutions for the spatial color variation in multi-projector displays assume rectangular overlap regions across the projectors that is possible only on flat surfaces with extremely constrained projector placement. In this article, we present a novel and fully automated method for removing color variations in a multi-projector display on arbitrary shaped smooth surfaces using a general color gamut morphing algorithm that can handle any arbitrarily shaped overlap between the projectors and assures imperceptible color variations across the display surface.

  PublisherDOI

• Dynamic projection mapping on deformable stretchable materials using boundary tracking

  Computers & Graphics · 2022-01-17 · 25 citations

  articleSenior author
  PublisherDOI

• Augmented Reality Patient-Specific Registration for Medical Visualization

  2022-11-22 · 4 citations

  articleSenior author

  In recent years, medical research has made extensive use of Augmented Reality (AR) for visualization. These visualizations provide improved 3D understanding and depth perception for surgeons and medical staff during surgical planning, medical training, and procedures. Often, AR in medicine involves impractical and extensive instrumentation in order to provide the precision needed for clinical use. We propose a mobile AR 3D model registration system for use in a practical, non-instrumented hospital setting. Our registration system takes as input a patient-specific model and overlays it on the patient using an accurate pose registration technique that requires a single marker as a point of reference to initialize a point cloud-based pose refinement technique. Our method is automatic, easy to use, and runs in real-time on a mobile phone. We conduct quantitative and qualitative analysis of the registration. The results confirm that our AR pose registration system produces an accurate and visually correct overlay of the medical data in real-time.

  PublisherDOI

Awards & honors

• $1.4 million awarded for Alzheimer’s disease research traini…
• UCI wins 5-year, $14M NIH grant to study brain circuits susc…