Integral Quadratic Constraints for Repeated ReLU

arXiv (Cornell University) · 2026-03-17

This paper presents a new dynamic integral quadratic constraint (IQC) for the repeated Rectified Linear Unit (ReLU). These dynamic IQCs can be used to analyze stability and induced $\ell_2$-gain performance of discrete-time, recurrent neural networks (RNNs) with ReLU activation functions. These analysis conditions can be incorporated into learning-based controller synthesis methods, which currently rely on static IQCs. We show that our proposed dynamic IQCs for repeated ReLU form a superset of the dynamic IQCs for repeated, slope-restricted nonlinearities. We also prove that the $\ell_2$-gain bounds are nonincreasing with respect to the horizon used in the dynamic IQC filter. A numerical example using a simple (academic) RNN shows that our proposed IQCs lead to less conservative bounds than existing IQCs.

Integral Quadratic Constraints for Repeated ReLU

ArXiv.org · 2026-03-17

This paper presents a new dynamic integral quadratic constraint (IQC) for the repeated Rectified Linear Unit (ReLU). These dynamic IQCs can be used to analyze stability and induced $\ell_2$-gain performance of discrete-time, recurrent neural networks (RNNs) with ReLU activation functions. These analysis conditions can be incorporated into learning-based controller synthesis methods, which currently rely on static IQCs. We show that our proposed dynamic IQCs for repeated ReLU form a superset of the dynamic IQCs for repeated, slope-restricted nonlinearities. We also prove that the $\ell_2$-gain bounds are nonincreasing with respect to the horizon used in the dynamic IQC filter. A numerical example using a simple (academic) RNN shows that our proposed IQCs lead to less conservative bounds than existing IQCs.

Exp3.P-Based Autonomous Decision Algorithm Against Nonstationary Opponents With Partially Known Policies

IEEE Transactions on Games · 2025-06-16 · 1 citations

This paper takes into account multi-agent games during which the opponents can change policies and their policy sets are partially known. Our goal is to generate an effective policy such that our agent can obtain a higher reward and meanwhile guarantee bounded regret. Considering such games against non-stationary opponents with partially known policies, Exp3.P-based Autonomous Decision (EAD) algorithm is put forward which contains three steps. Firstly, we learn embedding of opponent policy via Conditional Encoder-Decoder and employ conditional RL to generate the targeted policy. Secondly, we estimate the opponent policy through online Bayesian belief updates. Finally, we select the adversarial and targeted policy via a multi-armed bandit algorithm. Theoretical analysis is performed for the EAD algorithm. We give the lower bound of the expected reward when using the targeted policy, and prove the EAD algorithm has a bounded regret. Experimental results on Kuhn poker, Grid-world Predator-Prey and Grid world show the effectiveness of the proposed EAD algorithm.

Hybrid Gradient-Based Policy Optimization for Sample-Efficient Policy Learning in Autonomous Systems

2025-07-08

This paper introduces HyGIPO, a novel gradient-based iterative policy optimization technique designed for efficient policy learning in autonomous systems, especially in the presence of modeling errors. Performance of control algorithms for autonomous systems is often limited by mismatches between a simplified nominal model and a complex real system. To address this degradation, HyGIPO leverages a hybrid gradient optimization approach, combining gradients of dynamics from a nominal model with real-world data to optimize control policies. We apply this method to the quadcopter waypoint tracking problem, with the controller parameterized by a neural network, demonstrating its effectiveness in both simulation and hardware experiments. In simulation, HyGIPO rapidly learns the policy within a hundred samples, showing orders of magnitude higher sample efficiency compared to reinforcement learning methods. The hardware experiments further validate the method, achieving successful tracking results in just tens of samples.

Decision‐Making Problem for Two‐Player Markov Game: Perspective of Feedback Control

International Journal of Robust and Nonlinear Control · 2025-09-20

ABSTRACT This paper investigates the decision‐making problem for two‐player Markov game from the perspective of feedback control, and we hope to find solutions which are explicitly given. For the noncooperative game, we firstly prove the existence and uniqueness of Nash equilibrium pair. Then based on the nonlinear dynamic equation of Markov chain and the quadratic performance metrics, we deduce the theoretical solution via dynamic programming. Further, taking into account restrictions on the transition probabilities, practical solution is then given by comparing the location of theoretical solution with the admissible domain. Finally, an iterative algorithm is proposed to search for the Nash equilibrium pair. Following the similar steps, a theoretical solution is deduced for a cooperative Markov game. By using the Lagrangian method, we obtain the practical solution with the corresponding algorithm given. Numerical simulations verify the effectiveness of our proposed method.

An Exact, Finite Dimensional Representation for Full-Block, Circle Criterion Multipliers

ArXiv.org · 2025-11-26

This paper provides the first finite-dimensional characterization for the complete set of full-block, circle criterion multipliers. We consider the interconnection of a discrete-time, linear time-invariant system in feedback with a non-repeated, sector-bounded nonlinearity. Sufficient conditions for stability and performance can be derived using: (i) dissipation inequalities, and (ii) Quadratic Constraints (QCs) that bound the input/output pairs of the nonlinearity. Larger classes of QCs (or multipliers) reduce the conservatism of the conditions. Full-block, circle criterion multipliers define the complete set of all possible QCs for non-repeated, sector-bounded nonlinearities. These provide the least conservative conditions. However, full-block multipliers are defined by an uncountably infinite number of constraints and hence do not lead to computationally tractable solutions if left in this raw form. This paper provides a new finite-dimensional characterization for the set of full-block, circle criterion multipliers. The key theoretical insight is: the set of all input/output pairs of non-repeated sector-bounded nonlinearities is equal to the set of all incremental pairs for an appropriately constructed piecewise linear function. Our new description for the complete set of multipliers only requires a finite number of matrix copositivity constraints. These conditions have an exact, computationally tractable implementation for problems where the nonlinearity has small input/output dimensions $(\le 4)$. We illustrate the use of our new characterization via a simple example.

Scalable Decomposition for Stability Analysis of Feedback Systems with High-Dimensional Neural Network Components

2025-12-09

Stability conditions of dynamical systems with neural network components can often be formulated as semidefinite programs (SDPs). However, such SDP-based conditions typically scale poorly with the depth and width of the neural networks, leading to severe computational challenges. In this paper, we address this scalability issue by developing a matrix decomposition approach for contraction-based stability analysis of feedback systems with high-dimensional neural network modules. By leveraging novel analytical derivations, we develop new SDP conditions whose computational complexity depends solely on the final layer width, achieving effective independence from network depth and overall architecture size. This depthagnostic framework substantially enhances the scalability of stability analysis for systems with large neural networks. We demonstrate the scalability and effectiveness of our methods through a series of numerical studies on systems with large neural network components, accompanied by comprehensive comparisons against existing stability conditions.

Discrete-Time Stability Analysis of ReLU Feedback Systems via Integral Quadratic Constraints

ArXiv.org · 2025-11-16

This paper analyzes internal stability of a discrete-time feedback system with a ReLU nonlinearity. This feedback system is motivated by recurrent neural networks. We first review existing static quadratic constraints (QCs) for slope-restricted nonlinearities. Next, we derive hard integral quadratic constraints (IQCs) for scalar ReLU by using finite impulse filters and structured matrices. These IQCs are combined with a dissipation inequality leading to an LMI condition that certifies internal stability. We show that our new dynamic IQCs for ReLU are a superset of the well-known Zames-Falb IQCs specified for slope-restricted nonlinearities. Numerical results show that the proposed hard IQCs give less conservative stability margins than Zames-Falb multipliers and prior static QC methods, sometimes dramatically so.

Toward Engineering AGI: Benchmarking the Engineering Design Capabilities of LLMs

ArXiv.org · 2025-07-01

Modern engineering, spanning electrical, mechanical, aerospace, civil, and computer disciplines, stands as a cornerstone of human civilization and the foundation of our society. However, engineering design poses a fundamentally different challenge for large language models (LLMs) compared with traditional textbook-style problem solving or factual question answering. Although existing benchmarks have driven progress in areas such as language understanding, code synthesis, and scientific problem solving, real-world engineering design demands the synthesis of domain knowledge, navigation of complex trade-offs, and management of the tedious processes that consume much of practicing engineers' time. Despite these shared challenges across engineering disciplines, no benchmark currently captures the unique demands of engineering design work. In this work, we introduce EngDesign, an Engineering Design benchmark that evaluates LLMs' abilities to perform practical design tasks across nine engineering domains. Unlike existing benchmarks that focus on factual recall or question answering, EngDesign uniquely emphasizes LLMs' ability to synthesize domain knowledge, reason under constraints, and generate functional, objective-oriented engineering designs. Each task in EngDesign represents a real-world engineering design problem, accompanied by a detailed task description specifying design goals, constraints, and performance requirements. EngDesign pioneers a simulation-based evaluation paradigm that moves beyond textbook knowledge to assess genuine engineering design capabilities and shifts evaluation from static answer checking to dynamic, simulation-driven functional verification, marking a crucial step toward realizing the vision of engineering Artificial General Intelligence (AGI).

Capabilities of Large Language Models in Control Engineering: A Benchmark Study on GPT-4, Claude 3 Opus, and Gemini 1.0 Ultra

arXiv (Cornell University) · 2024-04-04 · 19 citations

In this paper, we explore the capabilities of state-of-the-art large language models (LLMs) such as GPT-4, Claude 3 Opus, and Gemini 1.0 Ultra in solving undergraduate-level control problems. Controls provides an interesting case study for LLM reasoning due to its combination of mathematical theory and engineering design. We introduce ControlBench, a benchmark dataset tailored to reflect the breadth, depth, and complexity of classical control design. We use this dataset to study and evaluate the problem-solving abilities of these LLMs in the context of control engineering. We present evaluations conducted by a panel of human experts, providing insights into the accuracy, reasoning, and explanatory prowess of LLMs in control engineering. Our analysis reveals the strengths and limitations of each LLM in the context of classical control, and our results imply that Claude 3 Opus has become the state-of-the-art LLM for solving undergraduate control problems. Our study serves as an initial step towards the broader goal of employing artificial general intelligence in control engineering.