About

Dr. David Eckman is an Assistant Professor in the Department of Industrial & Systems Engineering at Texas A&M University. He holds a Ph.D. in Operations Research from Cornell University, obtained in 2019. His research focuses on the use of stochastic simulation for decision-making under uncertainty, encompassing the design and analysis of ranking-and-selection procedures, the comparison of simulation-optimization algorithms, and the development of new methods for simulation output analysis.

Research topics

• Computer Science
• Artificial Intelligence
• Machine Learning
• Mathematics
• Software engineering
• Programming language
• Algorithm
• Mathematical optimization
• Theoretical computer science
• World Wide Web

Selected publications

• Quantifying and Attributing Submodel Uncertainty in Stochastic Simulation Models and Digital Twins

  Open MIND · 2026-02-18

  preprint

  Stochastic simulation is widely used to study complex systems composed of various interconnected subprocesses, such as input processes, routing and control logic, optimization routines, and data-driven decision modules. In practice, these subprocesses may be inherently unknown or too computationally intensive to directly embed in the simulation model. Replacing these elements with estimated or learned approximations introduces a form of epistemic uncertainty that we refer to as submodel uncertainty. This paper investigates how submodel uncertainty affects the estimation of system performance metrics. We develop a framework for quantifying submodel uncertainty in stochastic simulation models and extend the framework to digital-twin settings, where simulation experiments are repeatedly conducted with the model initialized from observed system states. Building on approaches from input uncertainty analysis, we leverage bootstrapping and Bayesian model averaging to construct quantile-based confidence or credible intervals for key performance indicators. We propose a tree-based method that decomposes total output variability and attributes uncertainty to individual submodels in the form of importance scores. The proposed framework is model-agnostic and accommodates both parametric and nonparametric submodels under frequentist and Bayesian modeling paradigms. A synthetic numerical experiment and a more realistic digital-twin simulation of a contact center illustrate the importance of understanding how and how much individual submodels contribute to overall uncertainty.

  DOI

• Quantifying and Attributing Submodel Uncertainty in Stochastic Simulation Models and Digital Twins

  ArXiv.org · 2026-02-18

  articleOpen access

  Stochastic simulation is widely used to study complex systems composed of various interconnected subprocesses, such as input processes, routing and control logic, optimization routines, and data-driven decision modules. In practice, these subprocesses may be inherently unknown or too computationally intensive to directly embed in the simulation model. Replacing these elements with estimated or learned approximations introduces a form of epistemic uncertainty that we refer to as submodel uncertainty. This paper investigates how submodel uncertainty affects the estimation of system performance metrics. We develop a framework for quantifying submodel uncertainty in stochastic simulation models and extend the framework to digital-twin settings, where simulation experiments are repeatedly conducted with the model initialized from observed system states. Building on approaches from input uncertainty analysis, we leverage bootstrapping and Bayesian model averaging to construct quantile-based confidence or credible intervals for key performance indicators. We propose a tree-based method that decomposes total output variability and attributes uncertainty to individual submodels in the form of importance scores. The proposed framework is model-agnostic and accommodates both parametric and nonparametric submodels under frequentist and Bayesian modeling paradigms. A synthetic numerical experiment and a more realistic digital-twin simulation of a contact center illustrate the importance of understanding how and how much individual submodels contribute to overall uncertainty.

  PublisherOA PDF

• Quantifying Uncertainty From Machine Learning Surrogate Models Embedded in Simulation Models

  2025-12-07

  article
  PublisherDOI

• Plausible Intervals: Global Inference from Limited Simulation of Structured Problems

  ACM Transactions on Modeling and Computer Simulation · 2025-12-26 · 1 citations

  articleOpen access

  We introduce a framework for constructing confidence intervals for the performance of a system as a function of a parameter, decision variable or system state, even when the system is not simulated at the particular parameter, decision variable or state. The proposed methods leverage observations from some other simulated model instances and known functional properties of the performance function being evaluated. The intervals, termed p lausible intervals, deliver a desired coverage probability uniformly over all model instances as the minimum sample size at the simulated model instances increases, and they attain the strongest possible consistency from simulating a finite number of model instances. We illustrate the versatility and effectiveness of plausible intervals through two numerical experiments.

  PublisherDOI

• Generators for Large-Scale Stochastic Simulation-Optimization Experiments

  2025-03-20

  articleOpen accessSenior author

  Running stochastic simulation-optimization solvers on a testbed of diverse problem instances allows users to evaluate their empirical performance, but obtaining meaningful results requires executing many replications. When problem instances feature realistic simulators, the associated computational costs are often prohibitively expensive. Cheaper, synthetic problem instances generally fail to preserve essential aspects of simulators, and a solver's performance on them may not be representative of its performance in practice. We propose a novel class of problem instance designed to imitate important features of simulation test problems and generate representative solver performance data at relatively low computational cost. We augment existing models predominantly used for emulation, namely, Gaussian processes, generalized lambda models, and kernel regression models, with an approximation of a Gaussian copula process. This adaptation facilitates efficient coordinated sampling across solutions (via common random numbers) and across solvers (via the sharing of sample-path functions) while keeping the number of user-specified parameters manageable.

  PublisherDOI

• An Agglomerative Clustering Algorithm for Simulation Output Distributions Using Regularized Wasserstein Distance

  INFORMS Journal on Data Science · 2025-09-17 · 3 citations

  articleSenior author

  Using statistical learning methods to analyze stochastic simulation outputs can significantly enhance decision making by uncovering relationships among different simulated systems and between a system’s inputs and outputs. We present a novel agglomerative clustering algorithm that utilizes the regularized Wasserstein distance to cluster multivariate empirical distributions of simulation outputs to identify patterns and trade-offs among performance measures. This framework has several important use cases, including anomaly detection, preoptimization, and online monitoring. In numerical experiments involving a call center model, we demonstrate how this methodology can identify staffing plans that yield similar performance outcomes and inform policies for intervening when queue lengths signal potentially worsening system performance. History: Eunshin Byon served as the senior editor for this article. Funding: This work is supported by the National Science Foundation [Grant CMMI-2206972]. Data Ethics & Reproducibility Note: The code is available at https://github.com/mohammadmgh78/Agglomerative_Clustering_Distribution and the Python package at https://pypi.org/project/distclust and in the e-Companion to this article (available at https://doi.org/10.1287/ijds.2024.0056 ).

  PublisherDOI

• Methods of Plausible Inference: The Definitive Cookbook

  2025-12-07

  article
  PublisherDOI

• An Agglomerative Clustering of Simulation Output Distributions Using Regularized Wasserstein Distance

  arXiv (Cornell University) · 2024-07-16

  preprintOpen accessSenior author

  Using statistical learning methods to analyze stochastic simulation outputs can significantly enhance decision-making by uncovering relationships between different simulated systems and between a system's inputs and outputs. We focus on clustering multivariate empirical distributions of simulation outputs to identify patterns and trade-offs among performance measures. We present a novel agglomerative clustering algorithm that utilizes the regularized Wasserstein distance to cluster these multivariate empirical distributions. This framework has several important use cases, including anomaly detection, pre-optimization, and online monitoring. In numerical experiments involving a call-center model, we demonstrate how this methodology can identify staffing plans that yield similar performance outcomes and inform policies for intervening when queue lengths signal potentially worsening system performance.

  PublisherOA PDFDOI

• Rate-Optimal Budget Allocation for the Probability of Good Selection

  2024-12-15

  articleSenior author

  This paper studies the allocation of simulation effort in a ranking-and-selection (R&S) problem with the goal of selecting a system whose performance is within a given tolerance of the best. We apply large-deviations theory to derive an optimal allocation for maximizing the rate at which the so-called probability of good selection (PGS) asymptotically approaches one, assuming that systems' output distributions are known. An interesting property of the optimal allocation is that some good systems may receive a sampling ratio of zero. We demonstrate through numerical experiments that this property leads to serious practical consequences, specifically when designing adaptive R&S algorithms. In particular, we observe that the convergence and even consistency of a simple plug-in algorithm designed for the PGS goal can be negatively impacted. We offer empirical evidence of these challenges and a preliminary exploration of a potential correction.

  PublisherDOI

• Data Farming the Parameters of Simulation-Optimization Solvers

  ACM Transactions on Modeling and Computer Simulation · 2024-07-23 · 3 citations

  article

  The performance of a simulation-optimization algorithm, a.k.a. a solver, depends on its parameter settings. Much of the research to date has focused on how a solver’s parameters affect its convergence and other asymptotic behavior. While these results are important for providing a theoretical understanding of a solver, they can be of limited utility to a user who must set up and run the solver on a particular problem. When running a solver in practice, good finite-time performance is paramount. In this article, we explore the relationship between a solver’s parameter settings and its finite-time performance by adopting a data farming approach. The approach involves conducting and analyzing the outputs of a designed experiment wherein the factors are the solver’s parameters and the responses are assorted performance metrics measuring the solver’s speed and solution quality over time. We demonstrate this approach with a study of the ASTRO-DF solver when solving a stochastic activity network problem and an inventory control problem. Through these examples, we show that how some of the solver’s parameters are set greatly affects its ability to achieve rapid, reliable progress and gain insights into the solver’s inner workings. We discuss the implications of using this framework for tuning solver parameters, as well as for addressing related questions of interest to solver specialists and generalists.

  PublisherDOI

Frequent coauthors

• Shane G. Henderson

  Cornell University

  16 shared

• Matthias Poloczek

  4 shared

• Barry L. Nelson

  Northwestern University

  4 shared

• Sara Shashaani

  4 shared

• Xueqi Zhao

  3 shared

• Andrew J. Schaefer

  Rice University

  3 shared

• Matthew Plumlee

  3 shared

• Naijia Anna Dong

  3 shared

Education

• Ph.D. Operations Research, Operations Research and Information Engineering

  Cornell University

  2019

• B.S. Industrial Engineering, Industrial Engineering

  University of Pittsburgh

  2014