Optimal Control

Classical data-driven methods in machine learning tend to be data-inefficient and they discard expert modeling knowledge. However, during the last decade a different approach has emerged under the name physics-informed machine learning. First posted in 2017 and published in 2019, physics-informed neural networks (PINNs) [6] combined deep learning with prior knowledge of the governing equations by directly learning the solution using the PDE residual as a soft penalty in the loss. In 2018, Neural ODEs [1] took a complementary route of embedding the physics in the model itself: the right-hand side of an ordinary differential equation $\dot y=f(y,t)$ is replaced by a neural network $f\approx f_\theta$, trained to reproduce observed data. The same principle was soon extended to stochastic differential equations [7] and to partial differential equations [5, 11, 12] — the latter is what we call here a Neural PDE. ...

Machine learning feels recent, but one of its core mathematical ideas dates back to 1952, when Richard Bellman published a seminal paper titled “On the Theory of Dynamic Programming” [6, 7], laying the foundation for optimal control and what we now call reinforcement learning. Later in the 50s, Bellman extended his work to continuous-time systems, turning the optimal condition into a PDE. What he later found was that this was identical to a result in physics published a century before (1840s), known as the Hamilton-Jacobi equation. ...