Physics-Informed Neural Networks for Solving Derivative-Constrained PDEs

Derivative-Constrained Physics-Informed Neural Networks: A Comprehensive Analysis

Introduction

This paper proposes Derivative-Constrained Physics-Informed Neural Networks (DC-PINNs), a unified framework for solving partial differential equations (PDEs) with additional physically or structurally motivated derivative constraints. While standard PINNs embed PDEs and boundary/initial data into the loss function and recast PDE solving as a function space optimization, many real-world applications require enforcing derivative-based equalities and inequalities (monotonicity, convexity, incompressibility, bounds) that are critical for physical fidelity. The DC-PINN framework extends PINNs to systematically incorporate and adaptively balance these constraints, thereby improving both feasibility and reliability of neural approximation for constrained PDEs.

Problem Formulation and Methodology

DC-PINNs generalize the conventional supervised learning with neural networks for PDEs into a multi-objective, constrained optimization setting. The network $u_\theta$ must satisfy the governing PDE, boundary data, and a set of general nonlinear derivative constraints:

• PDE residual: $f(x, D u_\theta(x)) = 0, \; x \in \Omega$
• Boundary/initial conditions: $b(x, D u_\theta(x)) = 0, \; x \in \partial\Omega$
• Derivative-based constraints: $h(x, D_h u_\theta(x)) \leq 0, \; x \in \Omega$

Here, $D$ is a set of differential operators (computed by automatic differentiation through the network). Derivative constraints are encoded via soft penalties using a carefully designed, numerically stable hinge loss to avoid optimizer instability. The overall objective aggregates supervised, PDE, boundary, and constraint violation losses, with both component-wise and sample-wise adaptive weights, dynamically adjusted according to gradients and violation magnitudes.

This approach is contrasted with hard-constraint baselines (architectural transformations, penalty “homotopy,” or augmented Lagrangian methods) commonly used in constrained optimization. The hard-constraint approaches either lack generality for derivative inequalities, incur optimization stiffness, or require manual hyperparameter tuning.

Adaptive Loss Balancing

Training PINNs with multiple objectives is sensitive to the relative magnitude of error terms and can require exhaustive hyperparameter tuning. DC-PINNs adopt a two-level adaptive balancing scheme:

1. Sample-wise scaling ($m$): Automatic per-sample reweighting of loss contributions, updated by gradient ascent concurrently with network training.
2. Category-wise scaling ($\lambda$): Category balancing by tracking average gradient magnitudes, ensuring no single objective dominates the training dynamics and that rare but severe constraint violations remain influential.

These mechanisms, updated at user-specified intervals, mitigate the ill-conditioning endemic to multi-objective neural PDE optimization and stabilize convergence across disparate magnitudes and frequencies of constraint violations.

Numerical Experiments and Benchmarks

The effectiveness of DC-PINNs is demonstrated on several benchmarks characterized by derivative constraints:

1. Heat Equation

DC-PINNs excel at enforcing non-positivity of the second spatial derivative and time monotonicity, resulting in near-exact RMSEs under initial, boundary, and PDE constraints and minimal violation of derivative inequalities. Standard PINNs exhibit spurious oscillations and boundary bias, while AL-PINNs and other hard-constraint baselines over-smooth or collapse higher-order dynamics, leading to suboptimal accuracy or excessive flattening of the learned solution.

2. Volatility Surface Calibration (Quantitative Finance)

Calibration of option price surfaces under the local volatility model is subject to strict monotonicity/convexity/arbitrage-free constraints on derivatives. DC-PINNs robustly enforce these inequalities, producing smooth, economically valid volatility surfaces. Fixed-penalty PINNs and traditional PINNs fail to generalize derivative constraints throughout the domain, particularly in low-data/maturity boundaries, while hard-constraint variants suffer from optimization instability. Adaptive weighting ensures constraint satisfaction without degrading overall fit.

3. Incompressible Navier-Stokes Flow

For two-dimensional unsteady flow past a cylinder, key constraints include incompressibility and local bounds on pressure gradient (following Bernoulli’s law). DC-PINNs consistently reduce violation rates on these constraints versus unconstrained or naïve PINNs, while the numerical error in velocity/pressure fields is comparable to high-fidelity finite element solvers. The adaptive scheme recovers physical model parameters accurately, demonstrating efficacy for system identification tasks.

Quantitative Results

Across all tasks, DC-PINNs:

• Achieve the lowest or near-lowest RMSE for constraint satisfaction (especially inequality residuals).
• Outperform rivals in stability (lowest normalized total variation of error trajectories) and overall violation reduction.
• Require 1.5–2$\times$ longer wall-clock training time relative to unconstrained PINNs, with this cost consistently offset by improved physical plausibility and stable optimization.
• Provide robust performance over a range of penalty/balancing hyperparameters, as confirmed by a comprehensive sensitivity analysis.

Notably, while DC-PINNs prioritize feasibility (constraint satisfaction), there are trade-offs. In highly overconstrained or multi-field PDEs (e.g., Navier-Stokes), hard-constraint methods can outperform in pure data fit or PDE residuals under fixed computational budgets, yet often at the expense of significant constraint violations.

Implications for Theory and Application

This work formally unifies PDE solving and inequality enforcement in a single, adaptive variational framework applicable to arbitrary physics- or finance-driven constraints expressible as differentiable operators. By leveraging automatic differentiation, DC-PINNs subsume both equality and inequality constraints of arbitrary differential order, with wide applicability in scientific computing, engineering, and financial modeling.

Theoretical implications include the explicit embedding of physical minimum principles and derivative-induced structural priors into neural approximation of PDEs, moving beyond the residual-centric perspective. This leads to solutions consistent with the underlying physics and physical admissibility, a critical step for deploying neural surrogates in safety-critical or rigorously regulated domains.

Practically, DC-PINNs offer stable, reliable convergence for derivative-constrained PDEs, reducing reliance on laborious hyperparameter search and decreasing oscillatory artifacts, especially in high-dimensional or severely underdetermined settings. The framework is readily extensible to higher dimensions (subject to standard sampling/capacity considerations) and is compatible with advanced PINN technologies (XPINNs, operator learning, adaptive sampling, and large-scale data parallelism).

Outlook and Future Directions

Future research directions include:

• Jointly learning or inferring constraint-penalty schedules and adaptive learning rates using meta-optimization.
• Developing alternative penalty forms near constraint boundaries for improved optimization landscape smoothness.
• Integrations with multifidelity and operator learning architectures to address scalability bottlenecks and push applicability to full-scale nonlinear, multi-physics PDEs.
• Application to coupled systems with competing or hierarchical constraints, and extension to stochastic or parametric PDEs.

The DC-PINN framework thus opens the path toward general-purpose, physics- and constraint-consistent neural PDE solvers, with impact on simulation, calibration, and inverse modeling in domains as diverse as climate science, turbulent flow, and computational finance.

Conclusion

The Derivative-Constrained PINN framework systematically incorporates general nonlinear derivative constraints into neural PDE solvers using soft penalty functionals and self-adaptive loss balancing. On benchmarks in heat diffusion, finance, and fluid dynamics, DC-PINNs consistently reduce constraint violations and stabilize training, with accuracy commensurate with or exceeding baseline PINN and hard-constraint methods. The approach generalizes readily, introducing new capabilities for solving constrained PDEs with neural networks, and highlights critical trade-offs between feasibility and data-fit that will guide future development in the field.

Reference: "Physics-Informed Neural Networks for Solving Derivative-Constrained PDEs" (2604.13723)