About

My research focuses on the creation of new algorithms for solving complex optimization problems. Over the years, this research has been motivated by applications as diverse as weather forecasting, engineering design and machine learning. There are always new challenges as scientist and engineers create models of increasing nonlinearity and dimensionality, amid uncertainty. To test the power of our algorithms and to make them widely available, my group has developed several software packages (some open source and some commercial) that are used in a wide range of applications. They include L-BFGS, KNITRO and L-BFGS-B. My view is that theory, algorithm design, and software are equally important in the creation of new algorithms. This is reflected in the textbook “Numerical Optimization”, which I co-authored with Steve Wright.

Research topics

• Computer Science
• Mathematical optimization
• Mathematics
• Applied mathematics
• Algorithm

Selected publications

• A feasible method for constrained derivative-free optimization

  Operations Research Letters · 2025-12-10

  articleSenior author
  PublisherDOI

• Design Guidelines for Noise-Tolerant Optimization with Applications in Robust Design

  SIAM Journal on Scientific Computing · 2025-05-02 · 1 citations

  articleSenior author
  PublisherDOI

• Small sample composite fault diagnosis of hydraulic bearings based on improved VME algorithm and mRVM

  Journal of applied artificial intelligence. · 2024-10-18

  article1st authorCorresponding

  The working environment of rolling bearings is complex. Once a fault occurs, various parts will affect each other and produce a compound fault. Traditional methods often use signal separation algorithms to separate different types of signals for fault diagnosis, but it is difficult to analyze specific faults efficiently and accurately. To solve this problem, this paper combines variational mode decomposition (VMD), Laplace energy index (LE) and variational mode extraction (VME) for signal extraction. Multi-class relevance vector machine (mRVM) and DS evidence theory are used for intelligent fault diagnosis, focusing on the context of small sample data. First, the VMD-LE-VME method is used to extract effective fault information from the fault signal and obtain multi-domain features. Then, the multi-domain features are input into mRVM for fault identification. Finally, the classification results are fused through DS evidence theory to obtain the final classification results. The effectiveness and superiority of this method in processing small sample data are verified by experiments.

  PublisherDOI

• A Feasible Method for Constrained Derivative-Free Optimization

  arXiv (Cornell University) · 2024-02-19

  preprintOpen accessSenior author

  This paper explores a method for solving constrained optimization problems when the derivatives of the objective function are unavailable, while the derivatives of the constraints are known. We allow the objective and constraint function to be nonconvex. The method constructs a quadratic model of the objective function via interpolation and computes a step by minimizing this model subject to the original constraints in the problem and a trust region constraint. The step computation requires the solution of a general nonlinear program, which is economically feasible when the constraints and their derivatives are very inexpensive to compute compared to the objective function. The paper includes a summary of numerical results that highlight the method's promising potential.

  PublisherOA PDFDOI

• Design Guidelines for Noise-Tolerant Optimization with Applications in Robust Design

  arXiv (Cornell University) · 2024-01-26

  preprintOpen accessSenior author

  The development of nonlinear optimization algorithms capable of performing reliably in the presence of noise has garnered considerable attention lately. This paper advocates for strategies to create noise-tolerant nonlinear optimization algorithms by adapting classical deterministic methods. These adaptations follow certain design guidelines described here, which make use of estimates of the noise level in the problem. The application of our methodology is illustrated by the development of a line search gradient projection method, which is tested on an engineering design problem. It is shown that a new self-calibrated line search and noise-aware finite-difference techniques are effective even in the high noise regime. Numerical experiments investigate the resiliency of key algorithmic components. A convergence analysis of the line search gradient projection method establishes convergence to a neighborhood of stationarity.

  PublisherOA PDFDOI

• An Iterative Approach for Solving the SCOPF Problem Applying LP, SOCP, and NLP Subproblems

  2024-07-19

  reportOpen access1st authorCorresponding

  We propose to develop efficient algorithms and software for the SCOPF problem. We will employ an iterative approach that will: a) use linear subproblems and other active set filtering techniques to identify the most important contingencies and drastically reduce the SCOPF model size; b) solve SOCP relaxations of the reduced SCOPF to converge to the neighborhood of the global optimal solution and establish a lower bound on the solution, and; c) use a non-convex, nonlinear interior-point solver, Artelys Knitro, to converge quickly to the optimal solution. To identify the most effective approach, we will experiment with several techniques to identify the tradeoffs between contingency subproblem complexity and fast solvability.

  PublisherOA PDFDOI

• A Trust-Region Algorithm for Noisy Equality Constrained Optimization

  arXiv (Cornell University) · 2024-11-04

  preprintOpen accessSenior author

  This paper introduces a modified Byrd-Omojokun (BO) trust region algorithm to address the challenges posed by noisy function and gradient evaluations. The original BO method was designed to solve equality constrained problems and it forms the backbone of some interior point methods for general large-scale constrained optimization. A key strength of the BO method is its robustness in handling problems with rank-deficient constraint Jacobians. The algorithm proposed in this paper introduces a new criterion for accepting a step and for updating the trust region that makes use of an estimate in the noise in the problem. The analysis presented here gives conditions under which the iterates converge to regions of stationary points of the problem, determined by the level of noise. This analysis is more complex than for line search methods because the trust region carries (noisy) information from previous iterates. Numerical tests illustrate the practical performance of the algorithm.

  PublisherOA PDFDOI

• Constrained and composite optimization via adaptive sampling methods

  IMA Journal of Numerical Analysis · 2023-05-12 · 13 citations

  articleOpen accessSenior author

  Abstract The motivation for this paper stems from the desire to develop an adaptive sampling method for solving constrained optimization problems, in which the objective function is stochastic and the constraints are deterministic. The method proposed in this paper is a proximal gradient method that can also be applied to the composite optimization problem min $f(x) + h(x)$, where $f$ is stochastic and $h$ is convex (but not necessarily differentiable). Adaptive sampling methods employ a mechanism for gradually improving the quality of the gradient approximation so as to keep computational cost to a minimum. The mechanism commonly employed in unconstrained optimization is no longer reliable in the constrained or composite optimization settings, because it is based on pointwise decisions that cannot correctly predict the quality of the proximal gradient step. The method proposed in this paper measures the result of a complete step to determine if the gradient approximation is accurate enough; otherwise, a more accurate gradient is generated and a new step is computed. Convergence results are established both for strongly convex and general convex $f$. Numerical experiments are presented to illustrate the practical behavior of the method.

  PublisherOA PDFDOI

• A trust region method for noisy unconstrained optimization

  Mathematical Programming · 2023-03-24 · 24 citations

  articleSenior authorCorresponding
  PublisherDOI

• Constrained Optimization in the Presence of Noise

  SIAM Journal on Optimization · 2023-08-10 · 8 citations

  articleSenior author

  .The problem of interest is the minimization of a nonlinear function subject to nonlinear equality constraints using a sequential quadratic programming (SQP) method. The minimization must be performed while observing only noisy evaluations of the objective and constraint functions. In order to obtain stability, the classical SQP method is modified by relaxing the standard Armijo line search based on the noise level in the functions, which is assumed to be known. Convergence theory is presented giving conditions under which the iterates converge to a neighborhood of the solution characterized by the noise level and the problem conditioning. The analysis assumes that the SQP algorithm does not require regularization or trust regions. Numerical experiments indicate that the relaxed line search improves the practical performance of the method on problems involving uniformly distributed noise, compared to a standard line search.Keywordsconstrained optimizationnoisy optimizationnonlinear optimizationMSC codes90C5690C5390C30

  PublisherDOI

Recent grants

• Collaborative Research: Market-Based Calibration of Pricing Models for Financial and Energy Option Contracts

  NSF · $160k · 2010–2013

• Nonlinear Optimization: Algorithms, Theory and Software

  NSF · $291k · 2008–2012

• Active-Set and Interior Algorithms for Non-Linear Optimization

  NSF · $250k · 2005–2009

• Collaborative Research: Algorithms for Large-Scale Stochastic and Nonlinear Optimization

  NSF · $270k · 2016–2020

• Zero-Order and Stochastic Methods for Large-Scale Optimization

  NSF · $200k · 2020–2024

Frequent coauthors

• Richard H. Byrd

  University of Colorado Boulder

  43 shared

• Richard Byrd

  University of Colorado Boulder

  18 shared

• José Luis Morales

  Instituto Politécnico Nacional

  15 shared

• Richard A. Waltz

  University of Southern California

  14 shared

• Figen Öztoprak

  Gebze Technical University

  14 shared

• Raghu Bollapragada

  13 shared

• Hao-Jun Michael Shi

  10 shared

• Frank E. Curtis

  Lehigh University

  9 shared

Labs

• Northwestern UniversityPI

Education

• B.S.

  UNAM, Mexico

• Ph.D.

  Rice University

Awards & honors

• 2012 George B. Dantzig Prize
• 2017 Von Neumann Theory Prize