[{"content":"There is a beautiful connection between probability theory and partial differential equations (PDEs), given by the famous Feynman–Kac theorem.\nThis result allows us to translate between finite-dimensional stochastic problems and infinite-dimensional deterministic problems. It appears throughout finance, physics, control theory, and machine learning (generative modeling and reinforcement learning).\nWhat makes this connection especially useful is that it works in both directions.\nThe solution to a high-dimensional PDE (which can be very expensive to compute numerically) can be evaluated at a single point simply by simulating a random process. Conversely, a difficult question about a stochastic process can be transformed into a deterministic PDE and tackled using standard techniques.\nExample. Take the Dirichlet problem $$ \\begin{cases} \\Delta u = 0 \u0026amp; \\text{in } \\Omega, \\\\ u = g \u0026amp; \\text{on } \\partial\\Omega. \\end{cases} $$ Then its solution satisfies $$ u(x) \\;=\\; \\mathbb{E}_x\\big[\\,g(B_\\tau)\\,\\big] $$ where $B$ is a Brownian motion started at $x$ and $\\tau$ is the first time it leaves $\\Omega$.\nThe proof is short and worth seeing once before the general statement. It needs one tool: the chain rule for Brownian paths.\nItô\u0026rsquo;s formula. For $\\varphi\\in C^2(\\mathbb{R}^d)$ and a Brownian motion $B$, $$ d\\varphi(B_t) \\;=\\; \\nabla\\varphi(B_t)^\\top dB_t \\;\u0026#43;\\; \\tfrac12\\Delta\\varphi(B_t)\\,dt . $$ The second term is what an ordinary chain rule misses. The path is nowhere differentiable, but it accumulates quadratic variation at a definite rate, $d\\langle B^i,B^j\\rangle_t=\\delta_{ij}\\,dt$, and the Laplacian is what that second-order term collects.\nProof of the example. Let $\\Omega$ be bounded, say $\\Omega\\subset B(0,R)$, let $u\\in C^2(\\Omega)\\cap C(\\overline\\Omega)$ solve the problem, let $B$ start at $x\\in\\Omega$, and let $\\tau=\\inf\\{s\\ge0:\\;B_s\\notin\\Omega\\}$. That such a $u$ exists is a separate question; here we take one as given and show it has to be the average of its own boundary data.\nCheck first that the path leaves at all. Itô\u0026rsquo;s formula on $h(y)=|y|^2$, for which $\\tfrac12\\Delta h=d$, gives $$ |B_{t\\wedge\\tau}|^2 =|x|^2\u0026#43;2\\int_0^{t\\wedge\\tau}B_s^\\top dB_s\u0026#43;d\\cdot(t\\wedge\\tau), $$ where $t\\wedge\\tau=\\min(t,\\tau)$ means that we stop the path when it exits. The stochastic integral has expectation zero: before $\\tau$, its integrand stays in the bounded domain, so it is square-integrable on every finite time interval. Since the stopped path stays in $\\overline\\Omega$, taking expectations yields $$ d\\cdot\\mathbb{E}_x[t\\wedge\\tau] =\\mathbb{E}_x\\big[|B_{t\\wedge\\tau}|^2\\big]-|x|^2\\;\\le\\;R^2 . $$ Letting $t\\to\\infty$ and using monotone convergence, we obtain $\\mathbb{E}_x[\\tau]\\le R^2/d$; in particular, $\\tau\u0026lt;\\infty$ almost surely, and since Brownian paths are continuous, the exit point $B_\\tau$ lies on $\\partial\\Omega$.\nNow apply Itô\u0026rsquo;s formula to $u$. Since $\\nabla u$ need not stay bounded near $\\partial\\Omega$, stop just short of it, at $$ \\tau_n:=\\inf\\{s\\ge0:\\;\\operatorname{dist}(B_s,\\partial\\Omega)\\le 1/n\\}, $$ with $n$ large enough that $\\operatorname{dist}(x,\\partial\\Omega)\u0026gt;1/n$; the stopped path then lives in a compact subset of $\\Omega$, and $\\tau_n\\uparrow\\tau$. Itô\u0026rsquo;s formula gives $$ u(B_{t\\wedge\\tau_n}) =u(x)\u0026#43;\\int_0^{t\\wedge\\tau_n}\\nabla u(B_s)^\\top dB_s \u0026#43;\\frac12\\int_0^{t\\wedge\\tau_n}\\Delta u(B_s)\\,ds. $$ The last integral vanishes because $u$ is harmonic, and the stochastic integral has expectation zero because $\\nabla u$ is bounded there, so $\\mathbb{E}_x[u(B_{t\\wedge\\tau_n})]=u(x)$. Since $u$ is continuous on the compact set $\\overline\\Omega$, it is bounded, so bounded convergence turns this into $$ u(x) \\;=\\; \\mathbb{E}_x\\big[\\,g(B_\\tau)\\,\\big]. \\qquad\\blacksquare $$ 1. The Feynman–Kac theorem With the previous example in mind, we now turn to the classical Feynman–Kac theorem, which connects stochastic differential equations (SDEs) with parabolic PDEs. The idea is the same: follow a random path and average the data it reaches. Here we run the process to a fixed terminal time, allowing for drift, a space-dependent diffusion, discounting, and a source accumulated along the path.\nLet $X$ solve the SDE $$ dX_t \\;=\\; b(X_t)\\,dt \u0026#43; \\sigma(X_t)\\,dW_t $$ in $\\mathbb{R}^d$, with $b,\\sigma$ globally Lipschitz. Its generator is the spatial differential operator $$ \\mathcal{L}\\varphi \\;:=\\; \\tfrac12\\operatorname{tr}\\big(\\sigma\\sigma^\\top D^2\\varphi\\big)\u0026#43;b\\cdot\\nabla\\varphi . $$ It is the drift term in Itô\u0026rsquo;s formula for $\\varphi(X_t)$: the first-order part comes from the drift of $X$, and the second-order part comes from its quadratic variation.\nTheorem (Feynman–Kac). Let $V,g:\\mathbb{R}^d\\to\\mathbb{R}$ and $f:[0,T]\\times\\mathbb{R}^d\\to\\mathbb{R}$ be continuous, with $V\\ge0$, and suppose $u\\in C^{1,2}\\big([0,T)\\times\\mathbb{R}^d\\big)\\cap C\\big([0,T]\\times\\mathbb{R}^d\\big)$ solves $$ \\partial_t u \u0026#43; \\mathcal{L}u - Vu \u0026#43; f \\;=\\; 0, \\qquad u(T,\\cdot)=g . $$ If $u$, $f$, and $\\sigma^\\top\\nabla u$ grow at most polynomially in $x$, uniformly in $t$, then $$ u(t,x) \\;=\\; \\mathbb{E}_{t,x}\\Big[\\, e^{-\\int_t^T V(X_r)dr}\\,g(X_T) \\;\u0026#43;\\; \\int_t^T e^{-\\int_t^s V(X_r)dr}\\, f(s,X_s)\\,ds \\,\\Big], $$ where $\\mathbb{E}_{t,x}$ is the expectation for the path started at $X_t=x$.\nThe proof in the appendix follows the same strategy as the Dirichlet example: Itô\u0026rsquo;s formula produces a martingale, and taking expectations gives the representation.\nThis statement gives uniqueness for free, but it presumes that a solution exists. The converse of the theorem (that the expectation on the right-hand side is itself a solution of the PDE) needs considerably more technical work. That construction is carried out in Friedman [5], Ch. 6, §§4–5.\nHistorical note. Feynman (1948) described the evolution of the Schrödinger equation $$ i\\hbar\\,\\partial_t\\psi \\;=\\; -\\tfrac{\\hbar^2}{2m}\\Delta\\psi \u0026#43; V\\psi $$ by summing over every path joining the two endpoints, each weighted by a complex number of modulus one.\nKac (1949) later developed a related idea for the heat equation (the Schrödinger equation in imaginary time): $$ \\partial_t u \\;=\\; \\tfrac12\\Delta u - Vu . $$ He observed that the solution can be written as an average over random Brownian paths: $$ u(t,x) \\;=\\; \\mathbb{E}_x\\Big[e^{-\\int_0^t V(B_s)ds}\\,g(B_t)\\Big]. $$ Black–Scholes (1973) is the same theorem read through the finance dictionary. Take the risk-neutral geometric Brownian motion $dX_t=rX_t\\,dt\u0026#43;\\sigma X_t\\,dW_t$ with $V\\equiv r$ constant, $f=0$, and $g$ the payoff at maturity: the PDE is $$ \\partial_t u \u0026#43; \\tfrac12\\sigma^2x^2\\,\\partial_{xx}u \u0026#43; rx\\,\\partial_x u - ru = 0, \\qquad u(T,\\cdot)=g, $$ and the representation is the pricing formula $u(t,x)=e^{-r(T-t)}\\,\\mathbb{E}\\big[g(X_T)\\mid X_t=x\\big]$. Every ingredient has a name on both sides: $V$ is the discount rate, a source $f$ is a dividend or running payoff, and $\\sigma^\\top\\nabla u$ is the hedging portfolio. Read in that dictionary, the proof in the appendix is the statement that a hedged position has no drift.\n2. Feynman–Kac family of equations Look back at the proof and notice how little of it was about the heat equation. Itô\u0026rsquo;s formula turned the operator into a drift, the equation cancelled that drift, and what was left was a martingale. Not a single step of that argument asks which operator it was handed. Change the operator and the same three lines still run.\nThe dictionary below collects the correspondences in that order, with $\\mathcal{L}$ the generator of $dX_t=b(X_t)\\,dt\u0026#43;\\sigma(X_t)\\,dW_t$ as above and, for a path started at time $t_0$, the exit time defined by $$ \\tau=\\inf\\{t\u0026gt;t_0:\\,X_t\\notin\\Omega\\}. $$\n2.1 Boundary conditions Name PDE ingredient Stochastic object Dirichlet condition $u=g$ on $\\partial\\Omega$ process killed on contact Neumann condition $\\partial_n u=0$ reflected diffusion Robin condition $\\partial_n u\u0026#43;\\alpha u=0$, $\\alpha\\ge0$ reflected diffusion killed at rate $\\alpha$ in boundary local time 2.2 Other families So far, we have only considered stochastic processes driven by Brownian motion, but the Feynman–Kac theorem is more general.\nA finite state space Let $\\xi$ be a continuous-time Markov chain on $\\{1,\\dots,n\\}$ with generator matrix $Q$, and let $V\\in\\mathbb{R}^n$ be a killing rate. The \u0026ldquo;PDE\u0026rdquo; is then a linear system of ODEs, $$ u\u0026#39;(t) \\;=\\; \\big(Q-\\operatorname{diag}V\\big)\\,u(t), \\qquad u(0)=g\\in\\mathbb{R}^n , $$ and the Feynman–Kac formula reads $$ u_i(t)\\;=\\;\\mathbb{E}_i\\Big[e^{-\\int_0^t V(\\xi_s)\\,ds}\\,g(\\xi_t)\\Big], \\qquad\\text{that is}\\qquad u(t)=e^{t(Q-\\operatorname{diag}V)}\\,g . $$ The generator has replaced the Laplacian and a matrix exponential has replaced the heat semigroup; not a single other word of the argument changes. It is a useful version to keep in mind because it makes plain that the theorem is a statement about generators and semigroups rather than Brownian motion specifically.\nLévy processes Replace $W$ by a Lévy process. The simplest case is a compound Poisson path that waits for an exponentially distributed time with rate $\\lambda$ and then jumps, $$ X_t \\;=\\; x\u0026#43;\\sum_{i=1}^{N_t}Z_i , $$ with $N_t$ a Poisson process of rate $\\lambda$ and the $Z_i$ independent draws from a distribution $\\mu$. Its generator is an integral operator, and the equation becomes $$ \\begin{cases} \\partial_t u(t,x) = \\lambda\\!\\int\\big[u(t,x\u0026#43;z)-u(t,x)\\big]\\,\\mu(dz)\\;-\\;V(x)\\,u(t,x),\\\\ u(0,\\cdot)=g, \\end{cases} $$ while the representation remains unchanged: $$ u(t,x)=\\mathbb{E}_x\\big[e^{-\\int_0^tV(X_s)ds}g(X_t)\\big]. $$ What is new is that the operator is nonlocal: the value at $x$ is tied to the value at every $x\u0026#43;z$ the path can reach in one jump, not merely to an infinitesimal neighborhood.\nLetting the jump measure be $\\nu(dz)=c_{d,\\alpha}|z|^{-d-\\alpha}dz$ makes $X$ an $\\alpha$-stable process and $\\mathcal{L}=-(-\\Delta)^{\\alpha/2}$, so the fractional heat equation is Feynman–Kac for a pure-jump path. Nonlocality does cost something at the boundary: a jump can leave $\\Omega$ without ever touching $\\partial\\Omega$, so a Dirichlet datum must be prescribed on the whole complement $\\Omega^c$.\nDeterministic flows Take the opposite extreme and switch the noise off, $\\sigma\\equiv0$. The process is the ODE flow $\\dot X_s=b(X_s)$ started at $X_t=x$; its law is a Dirac mass, and the expectation in the theorem has nothing left to average over: $$ u(t,x)\\;=\\;e^{-\\int_t^TV(X_r)dr}\\,g(X_T)\\;\u0026#43;\\;\\int_t^Te^{-\\int_t^sV(X_r)dr}\\,f(s,X_s)\\,ds . $$ This is the method of characteristics for the linear transport equation $\\partial_tu\u0026#43;b\\cdot\\nabla u-Vu\u0026#43;f=0$: the characteristic through $(t,x)$ is the trajectory itself, $V$ damps the datum carried along it, $f$ feeds it, and with $V=f=0$ the solution is the terminal value transported back, $u(t,x)=g(X_T)$.\nTwo things do change, and they are what make the degenerate case worth stating rather than dismissing.\nThe boundary. A Brownian path hits every point of $\\partial\\Omega$, which is why the Dirichlet problem of the opening example takes data on all of it; a trajectory arrives at exactly one point, so data may be prescribed only where characteristics enter, on the inflow boundary $\\{y\\in\\partial\\Omega:\\,b(y)\\cdot n(y)\u0026lt;0\\}$, and prescribing it anywhere else overdetermines the problem. That is the degenerate end of the dictionary of §2.1, and the mirror image of the nonlocal case, where a jump can leave $\\Omega$ without ever touching $\\partial\\Omega$ and the datum has to be given on the whole complement.\nRegularity. Diffusion smooths; transport does not. With $\\sigma\\sigma^\\top$ nondegenerate the representation is an average over infinitely many paths and $u$ is smooth however rough $g$ is, whereas here $u(t,\\cdot)$ is exactly $g$ transported, and a discontinuity in the datum travels along its characteristic forever.\nReferences [1] R. P. Feynman. Space-Time Approach to Non-Relativistic Quantum Mechanics. Reviews of Modern Physics 20 (1948) 367–387.\n[2] M. Kac. On Distributions of Certain Wiener Functionals. Transactions of the American Mathematical Society 65 (1949) 1–13.\n[3] F. Black and M. Scholes. The Pricing of Options and Corporate Liabilities. Journal of Political Economy 81 (1973) 637–654.\n[4] I. Karatzas and S. Shreve. Brownian Motion and Stochastic Calculus. 2nd ed., Springer, 1991. (§4.2–4.4 for the Dirichlet problem, §5.7 for Feynman–Kac.)\n[5] A. Friedman. Stochastic Differential Equations and Applications, Vol. 1. Academic Press, 1975. (Ch. 6, §§4–5: fundamental solutions for parabolic equations and the stochastic representation of solutions.)\nAppendix: Proof of the Feynman–Kac theorem Fix $t\u0026lt;T$ and start the SDE at $X_t=x$. The discount factor $$ D_s:=e^{-\\int_t^s V(X_r)\\,dr} $$ satisfies $D_t=1$ and $dD_s=-V(X_s)D_s\\,ds$. Since $V\\ge0$, we also have $0\u0026lt;D_s\\le1$.\nFirst apply Itô\u0026rsquo;s formula to the time-dependent function $u(s,X_s)$, for $s\u0026lt;T$: $$ du(s,X_s) =\\big(\\partial_s u\u0026#43;\\mathcal{L}u\\big)(s,X_s)\\,ds \u0026#43;\\nabla u(s,X_s)^\\top\\sigma(X_s)\\,dW_s. $$ The product rule gives $d(D_su)=D_s\\,du\u0026#43;u\\,dD_s$; there is no quadratic covariation term because $D$ has finite variation. Consequently, $$ d\\big(D_su(s,X_s)\\big) =D_s\\big(\\partial_s u\u0026#43;\\mathcal{L}u-Vu\\big)(s,X_s)\\,ds \u0026#43;D_s\\nabla u(s,X_s)^\\top\\sigma(X_s)\\,dW_s. $$ The PDE says that the drift in parentheses is $-f$. Adding the accumulated source therefore cancels it: if $$ Y_s:=D_su(s,X_s)\u0026#43;\\int_t^s D_r f(r,X_r)\\,dr, $$ then $$ Y_s=u(t,x)\u0026#43;\\int_t^s D_r\\nabla u(r,X_r)^\\top\\sigma(X_r)\\,dW_r. $$ To take expectations, we must check that this stochastic integral is a true martingale. Globally Lipschitz coefficients give finite moments on a finite time interval: $$ \\mathbb{E}_{t,x}\\!\\left[\\sup_{t\\le r\\le T}|X_r|^p\\right]\u0026lt;\\infty \\qquad\\text{for every }p\\ge2. $$ By the polynomial growth assumption, choose $C\u0026gt;0$ and $m\\ge1$ such that $|\\sigma^\\top\\nabla u(r,y)|\\le C(1\u0026#43;|y|^m)$. Together with $D_r\\le1$, this gives $$ \\mathbb{E}_{t,x}\\!\\left[\\int_t^T D_r^2\\,|\\sigma(X_r)^\\top\\nabla u(r,X_r)|^2\\,dr\\right] \\le 2C^2(T-t)\\,\\mathbb{E}_{t,x}\\!\\left[1\u0026#43;\\sup_{t\\le r\\le T}|X_r|^{2m}\\right] \u0026lt;\\infty. $$ Thus the stochastic integral is square-integrable and has expectation zero. For every $s\u0026lt;T$ we obtain $$ u(t,x)=\\mathbb{E}_{t,x}\\!\\left[ D_su(s,X_s)\u0026#43;\\int_t^s D_r f(r,X_r)\\,dr\\right]. $$ Finally, let $s\\uparrow T$. Continuity and the terminal condition give $u(s,X_s)\\to g(X_T)$ almost surely. The polynomial growth of $u$ and $f$ bounds the expression inside the expectation by $C\u0026#39;(1\u0026#43;\\sup_{t\\le r\\le T}|X_r|^q)$ for suitable $C\u0026#39;\u0026gt;0$ and $q\\ge2$. This bound is integrable by the same moment estimate, so dominated convergence yields $$ u(t,x)=\\mathbb{E}_{t,x}\\!\\left[ e^{-\\int_t^T V(X_r)\\,dr}\\,g(X_T) \u0026#43;\\int_t^T e^{-\\int_t^s V(X_r)\\,dr}\\,f(s,X_s)\\,ds \\right]. \\qquad\\blacksquare $$","permalink":"https://dani2442.github.io/posts/feynman-kac/","summary":"\u003cp\u003eThere is a beautiful connection between probability theory and partial differential equations (PDEs), given by the famous Feynman–Kac theorem.\u003c/p\u003e\n\u003cp\u003eThis result allows us to translate between finite-dimensional stochastic problems and infinite-dimensional deterministic problems. It appears throughout finance, physics, control theory, and machine learning (generative modeling and reinforcement learning).\u003c/p\u003e\n\u003cp\u003eWhat makes this connection especially useful is that it works in both directions.\u003c/p\u003e\n\u003cp\u003eThe solution to a high-dimensional PDE (which can be very expensive to compute numerically) can be evaluated at a single point simply by simulating a random process. Conversely, a difficult question about a stochastic process can be transformed into a deterministic PDE and tackled using standard techniques.\u003c/p\u003e","title":"Feynman–Kac formula"},{"content":"I spent two weeks at the Benasque Science Center for the XI edition of “Partial Differential Equations, Optimization, Numerics, and Machine Learning” (August 23 – September 4, 2026), where I gave three talks and co-chaired the thematic session “PDEs and Machine Learning” together with Aidan Chaumet.\nHere are the slides, in case you’re interested in these topics:\nHybrid Generative Modeling for Neuroscience (August 25) — whole-brain dynamics and the fitting problem, joint work with L. Liverani, E. Zuazua and X. Kobeleva. Link Data-Driven Control in Infinite-Dimensional Spaces (August 27) — informativity and persistency of excitation, with applications to data-driven controllability and LQR. Link Neural ODEs: A Representer Theorem (September 1) — joint work with Zhengping Ji. Link The full program is available on the conference website.\n","permalink":"https://dani2442.github.io/events/2026_09_01_benasque/","summary":"\u003cp\u003eI spent two weeks at the Benasque Science Center for the XI edition of “Partial Differential Equations, Optimization, Numerics, and Machine Learning” (August 23 – September 4, 2026), where I gave three talks and co-chaired the thematic session “PDEs and Machine Learning” together with Aidan Chaumet.\u003c/p\u003e\n\u003cp\u003eHere are the slides, in case you’re interested in these topics:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eHybrid Generative Modeling for Neuroscience\u003c/strong\u003e (August 25) — whole-brain dynamics and the fitting problem, joint work with L. Liverani, E. Zuazua and X. Kobeleva. \u003ca href=\"/files/benasque_hybrid_generative_modeling.pdf\"\u003eLink\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eData-Driven Control in Infinite-Dimensional Spaces\u003c/strong\u003e (August 27) — informativity and persistency of excitation, with applications to data-driven controllability and LQR. \u003ca href=\"/files/benasque_data_driven_control.pdf\"\u003eLink\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eNeural ODEs: A Representer Theorem\u003c/strong\u003e (September 1) — joint work with Zhengping Ji. \u003ca href=\"/files/benasque_neural_odes_representer_theorem.pdf\"\u003eLink\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe full program is available on the \u003ca href=\"https://benasque.org/2026pde/\"\u003econference website\u003c/a\u003e.\u003c/p\u003e","title":"Talks at Benasque (Spain)"},{"content":"\nClassical data-driven methods in machine learning tend to be data-inefficient and they discard expert modeling knowledge. However, during the last decade a different approach has emerged under the name physics-informed machine learning.\nFirst posted in 2017 and published in 2019, physics-informed neural networks (PINNs) [6] combined deep learning with prior knowledge of the governing equations by directly learning the solution using the PDE residual as a soft penalty in the loss. In 2018, Neural ODEs [1] took a complementary route of embedding the physics in the model itself: the right-hand side of an ordinary differential equation $\\dot y=f(y,t)$ is replaced by a neural network $f\\approx f_\\theta$, trained to reproduce observed data. The same principle was soon extended to stochastic differential equations [7] and to partial differential equations [5, 11, 12] — the latter is what we call here a Neural PDE.\nA Neural PDE applies the Neural-ODE construction to an evolution equation in space and time: the unknown coefficient — typically a reaction or source term — becomes a neural network embedded inside a partial differential equation [5], i.e.,\n$$\\partial_t y -\\Delta y = f_\\theta\\big(x,t,y\\big) $$ The two constructions are the same idea applied to a finite- and an infinite-dimensional state. The goal of this post is to make that parallel precise within the physics-informed framework. We\nstate both problem formulations in a single notation; define the learning problem (loss, data term, and the regularizer); derive the adjoint equations that give the gradient of the loss; and fit data using explicit Euler for the Neural ODE, and explicit Euler in time with finite elements in space for the Neural PDE. A brief note on geometric deep learning. Embedding knowledge into the model is one instance of a broader trend. Geometric deep learning [8] pursues the same goal with a different prior: it builds geometric structure — symmetries, invariances, and the shape of the data domain — directly into the architecture. Many successful models are special cases of this principle: convolutional networks encode translation equivariance (1998), graph neural networks respect permutation symmetry (2008), and transformers can be read as attention on a fully connected graph (2017). Where physics-informed learning constrains the model with a governing equation, geometric deep learning constrains it with a symmetry group.\n1. Problem formulation Throughout, $\\theta\\in\\mathbb{R}^p$ denotes the network parameters and $f_\\theta$ is a neural network (smooth, e.g. $\\tanh$ activations).\n1.1 Neural ODE The state $y(t)\\in\\mathbb{R}^n$ solves the initial-value problem\n$$ \\dot y = f_\\theta\\big(y,t\\big)\\quad\\text{in }(0,T], \\qquad y(0)=y_0, \\tag{ODE} $$ with $f_\\theta:\\mathbb{R}^n\\times[0,T]\\to\\mathbb{R}^n$. We write $y_\\theta$ for the solution map $\\theta\\mapsto y(\\,\\cdot\\,;\\theta)$ where we use a discretization scheme such as explicit Euler.\n1.2 Neural PDE (Parabolic) Let $\\Omega\\subset\\mathbb{R}^m$ be a bounded Lipschitz domain with boundary $\\Gamma=\\partial\\Omega$, and let $T\u0026gt;0$ be a finite time horizon. Over the space–time cylinder $\\Omega\\times(0,T)$ and its lateral boundary $\\Gamma\\times(0,T)$, the state $y(x,t)$ solves the semilinear parabolic problem\n$$ \\begin{aligned} \\partial_t y \u0026#43; \\mathcal{A}y \u0026#43; d\\big(x,t,y\\big) \u0026amp;= f_\\theta\\big(x,t,y\\big) \u0026amp;\u0026amp; \\text{in } \\Omega\\times(0,T),\\\\ \\partial_{\\nu_{\\mathcal A}} y \u0026#43; b\\big(x,t,y\\big) \u0026amp;= g \u0026amp;\u0026amp; \\text{on } \\Gamma\\times(0,T),\\\\ y(\\cdot,0) \u0026amp;= y_0 \u0026amp;\u0026amp; \\text{in } \\Omega, \\end{aligned}\\tag{PDE-P} $$ where $\\mathcal{A}$ is the second-order elliptic operator in divergence form and $\\partial_{\\nu_{\\mathcal A}}y$ is its associated conormal derivative — the boundary flux that goes with $\\mathcal A$, i.e. the outward normal component of the flux $a_{ij}\\,\\partial_{x_i}y$ that $\\mathcal A$ takes the divergence of,\n$$ \\mathcal{A}y = -\\sum_{i,j=1}^{m}\\partial_{x_j}\\!\\big(a_{ij}(x)\\,\\partial_{x_i} y\\big), \\qquad \\partial_{\\nu_{\\mathcal A}}y = \\sum_{i,j=1}^{m} a_{ij}\\,\\partial_{x_i} y\\,\\nu_j , $$ with $(a_{ij})$ symmetric, bounded, and uniformly elliptic, and $\\nu$ the outer normal. When $\\mathcal A=-\\Delta$ we have $a_{ij}=\\delta_{ij}$, so the conormal derivative reduces to the ordinary normal derivative $\\partial_{\\nu_{\\mathcal A}}y=\\sum_i\\partial_{x_i}y\\,\\nu_i=\\nabla y\\cdot\\nu=\\partial_\\nu y$ — the most common example. The learnable component is the source term $f_\\theta(x,t,y)$ or a boundary term $g_\\theta(x,t,y)$.\n1.3 Neural PDE (Elliptic) On the same domain $\\Omega$ with boundary $\\Gamma=\\partial\\Omega$, the state $y(x)$ solves the semilinear elliptic problem\n$$ \\begin{aligned} \\mathcal{A}y \u0026#43; c_0(x)\\,y \u0026amp;= f_\\theta\\big(x,y\\big) \u0026amp;\u0026amp; \\text{in } \\Omega,\\\\ \\partial_{\\nu_{\\mathcal A}} y \u0026#43; \\alpha(x)\\,y \u0026amp;= g \u0026amp;\u0026amp; \\text{on } \\Gamma, \\end{aligned}\\tag{PDE-E} $$ with $\\mathcal A$ the same divergence-form elliptic operator as before, a nonnegative potential $c_0\\in L^\\infty(\\Omega)$, and a boundary condition of the third kind (Robin) with nonnegative coefficient $\\alpha\\in L^\\infty(\\Gamma)$ and datum $g\\in L^2(\\Gamma)$. As before the learnable component is the source term $f_\\theta(x,y)$.\nThe three problems line up term by term: the finite-dimensional vector field $f_\\theta$ stays the learnable right-hand side, the derivative $\\dot y$ becomes $\\partial_t y$, and the new ingredients are the spatial operator $\\mathcal A$ (with a boundary condition) and a fixed reaction $d$. Setting $\\partial_t y=0$ then collapses the parabolic problem (PDE-P) onto the elliptic problem (PDE-E), with the reaction $d$ playing the role of the potential $c_0$.\n2. The learning problem We are given data: noisy observations of the true state. We phrase fitting as the minimization of a regularized cost functional $J$. Keeping the general Bolza form lets the same expressions specialize to tracking or terminal objectives.\n2.1 Cost functionals Neural ODE. With a running cost $\\varphi$ and terminal cost $\\phi$,\n$$ J(\\theta)=\\int_0^T \\varphi\\big(y_\\theta(t),t\\big)\\,dt \u0026#43;\\phi\\big(y_\\theta(T)\\big) \u0026#43;\\frac{\\gamma}{2}\\|\\theta\\|^2 \\qquad \\text{(J-ODE)}. $$ Neural PDE (Parabolic). With a distributed cost $\\varphi$ on $\\Omega\\times(0,T)$, a boundary cost $\\psi$ on $\\Gamma\\times(0,T)$, and a terminal cost $\\phi$ on $\\Omega$,\n$$ J(\\theta)=\\iint_{\\Omega\\times(0,T)} \\varphi\\big(x,t,y_\\theta\\big)\\,dx\\,dt \u0026#43;\\iint_{\\Gamma\\times(0,T)} \\psi\\big(x,t,y_\\theta\\big)\\,ds\\,dt \u0026#43;\\int_\\Omega \\phi\\big(x,y_\\theta(\\cdot,T)\\big)\\,dx \u0026#43;\\frac{\\gamma}{2}\\|\\theta\\|^2 \\qquad \\text{(J-PDE)}. $$ Neural PDE (Elliptic). With a distributed cost $\\varphi$ on $\\Omega$ and a boundary cost $\\psi$ on $\\Gamma$ (no terminal term, since there is no time),\n$$ J(\\theta)=\\int_\\Omega \\varphi\\big(x,y_\\theta\\big)\\,dx \u0026#43;\\int_\\Gamma \\psi\\big(x,y_\\theta\\big)\\,ds \u0026#43;\\frac{\\gamma}{2}\\|\\theta\\|^2 \\qquad \\text{(J-ELL)}. $$ 2.2 Data term Let the data be samples of the true state on a finite set of points,\n$$ \\mathcal D_{\\mathrm{ODE}}=\\big\\{(t_k,\\,y^{\\mathrm{obs}}_k)\\big\\}_{k=1}^{N}, \\qquad \\mathcal D_{\\mathrm{PDE}}=\\big\\{(x_j,t_k,\\,y^{\\mathrm{obs}}_{jk})\\big\\}_{j,k}. $$ A least-squares data term is the choice\n$$ \\varphi(y,t)=\\tfrac12\\sum_{k}\\big|y-y^{\\mathrm{obs}}_k\\big|^2\\,\\delta(t-t_k), \\qquad \\varphi(x,t,y)=\\tfrac12\\,\\omega(x,t)\\,\\big|y-y^{\\mathrm{obs}}(x,t)\\big|^2, $$ where $\\omega=\\sum_{j,k}\\delta(x-x_j)\\delta(t-t_k)$ is the empirical observation measure (and $\\psi=\\phi=0$). Concretely the data parts of $J$ read\n$$ J_{\\mathrm{data}}^{\\mathrm{ODE}}(\\theta)=\\tfrac12\\sum_{k=1}^{N}\\big|y_\\theta(t_k)-y^{\\mathrm{obs}}_k\\big|^2, \\qquad J_{\\mathrm{data}}^{\\mathrm{PDE}}(\\theta)=\\tfrac12\\sum_{j,k}\\big|y_\\theta(x_j,t_k)-y^{\\mathrm{obs}}_{jk}\\big|^2 . $$ 2.3 Regularization and coercivity The Tikhonov term $\\frac{\\gamma}{2}\\|\\theta\\|^2$ is not merely cosmetic — the data term alone need not be coercive in $\\theta$ — changes of the parameters can leave the fit almost unchanged, so a minimizing sequence may run off to infinity — and indeed network training problems need not attain an optimum at all [9]. Adding $\\frac{\\gamma}{2}\\|\\theta\\|^2$ makes the reduced functional coercive,\n$$ J(\\theta)\\ \\ge\\ \\frac{\\gamma}{2}\\|\\theta\\|^2\\ \\xrightarrow[\\ \\|\\theta\\|\\to\\infty\\ ]{}\\ \\infty , $$ so sublevel sets are bounded; together with weak lower semicontinuity of $\\theta\\mapsto J(\\theta)$ this yields existence of a minimizer.\n3. Adjoint equations To minimize $J$ we need $\\nabla_\\theta J$. The adjoint (costate) method, classical in optimal control [2] and PDE-constrained control [4], computes it at the cost of one backward solve, independent of $p=\\dim\\theta$. We give the continuous adjoint in both settings; the discrete adjoint is what reverse-mode autodiff computes when we differentiate through the integrator, which is why the code can simply call loss.backward().\n3.1 Neural ODE adjoint Introduce the costate $p:[0,T]\\to\\mathbb{R}^n$ and the Lagrangian\n$$ \\mathcal L=\\int_0^T\\varphi(y,t)\\,dt\u0026#43;\\phi(y(T))\u0026#43;\\frac{\\gamma}{2}\\|\\theta\\|^2 -\\int_0^T p^\\top\\!\\big(\\dot y-f_\\theta(y,t)\\big)\\,dt . $$ Stationarity with respect to $y$ gives the backward adjoint IVP\n$$ -\\dot p(t)=\\Big(\\partial_y f_\\theta\\big(y_\\theta,t\\big)\\Big)^{\\!\\top} p(t)\u0026#43;\\partial_y\\varphi\\big(y_\\theta,t\\big), \\qquad p(T)=\\partial_y\\phi\\big(y_\\theta(T)\\big), \\tag{A-ODE} $$ and the gradient is\n$$ \\boxed{\\;\\nabla_\\theta J=\\int_0^T\\Big(\\partial_\\theta f_\\theta\\big(y_\\theta,t\\big)\\Big)^{\\!\\top} p(t)\\,dt\u0026#43;\\gamma\\,\\theta\\;} $$ For the data term, $\\partial_y\\varphi(y_\\theta,t)=\\sum_k\\big(y_\\theta(t_k)-y^{\\mathrm{obs}}_k\\big)\\delta(t-t_k)$, i.e. the adjoint receives a jump of size $y_\\theta(t_k)-y^{\\mathrm{obs}}_k$ at each observation time.\n3.2 Neural PDE (Parabolic) adjoint Let $\\mathcal A^{*}$ be the adjoint of $\\mathcal A$. Repeating the Lagrangian computation with the weak form of (PDE) and integrating by parts in time (terminal data, since $\\delta y(0)=0$) and in space (Green\u0026rsquo;s identity), the costate $p(x,t)$ solves the backward parabolic problem\n$$ \\begin{aligned} -\\partial_t p \u0026#43; \\mathcal A^{*}p \u0026#43; \\big(d_y-\\partial_y f_\\theta\\big)\\big(x,t,y_\\theta\\big)\\,p \u0026amp;= \\varphi_y\\big(x,t,y_\\theta\\big) \u0026amp;\u0026amp; \\text{in } \\Omega\\times(0,T),\\\\ \\partial_{\\nu_{\\mathcal A^{*}}}p \u0026#43; b_y\\big(x,t,y_\\theta\\big)\\,p \u0026amp;= \\psi_y\\big(x,t,y_\\theta\\big) \u0026amp;\u0026amp; \\text{on } \\Gamma\\times(0,T),\\\\ p(\\cdot,T) \u0026amp;= \\phi_y\\big(x,y_\\theta(\\cdot,T)\\big) \u0026amp;\u0026amp; \\text{in } \\Omega, \\end{aligned}\\tag{A-PDE} $$ where $d_y=\\partial_y d$ is the derivative of the known reaction and $\\partial_y f_\\theta$ that of the learned source — they enter the linearized dynamics as the combined reaction $d-f_\\theta$. The gradient is\n$$ \\boxed{\\;\\nabla_\\theta J=\\iint_{\\Omega\\times(0,T)} p(x,t)\\,\\partial_\\theta f_\\theta\\big(x,t,y_\\theta\\big)\\,dx\\,dt\u0026#43;\\gamma\\,\\theta\\;} $$ The structure mirrors (A-ODE) exactly — including the $\u0026#43;$ sign, since $f_\\theta$ now sits on the right-hand side just as in (ODE): a backward evolution driven by the cost sensitivity $\\varphi_y$, with the adjoint of the linearized dynamics, and a gradient pairing the costate against $\\partial_\\theta f_\\theta$. For the data term, $\\varphi_y=\\omega\\,(y_\\theta-y^{\\mathrm{obs}})$.\n3.3 Neural PDE (Elliptic) adjoint With no time direction the adjoint is itself a steady elliptic problem. Repeating the Lagrangian computation for the weak form of (PDE-E), the costate $p(x)$ solves\n$$ \\begin{aligned} \\mathcal A^{*}p \u0026#43; \\big(c_0-\\partial_y f_\\theta\\big)\\big(x,y_\\theta\\big)\\,p \u0026amp;= \\varphi_y\\big(x,y_\\theta\\big) \u0026amp;\u0026amp; \\text{in } \\Omega,\\\\ \\partial_{\\nu_{\\mathcal A^{*}}}p \u0026#43; \\alpha\\,p \u0026amp;= \\psi_y\\big(x,y_\\theta\\big) \u0026amp;\u0026amp; \\text{on } \\Gamma, \\end{aligned}\\tag{A-ELL} $$ and the gradient is\n$$ \\boxed{\\;\\nabla_\\theta J=\\int_\\Omega p(x)\\,\\partial_\\theta f_\\theta\\big(x,y_\\theta\\big)\\,dx\u0026#43;\\gamma\\,\\theta\\;} $$ This is (A-PDE) with the time derivative removed: the same linearized reaction $c_0-\\partial_y f_\\theta$ and the same conormal-plus-Robin boundary operator, but solved as a single boundary-value problem rather than marched backward in time.\n4. Examples with code The code is in neural-pde/, pure PyTorch. We differentiate through the unrolled integrators with autograd (the discrete adjoint), so the gradients of §3 are obtained without hand-coding the backward solves.\n4.1 Neural ODE Ground truth follows the nonlinear (cubic) spiral example in torchdiffeq [10], $\\dot y = A^{\\top}y$ with $A=\\big[\\begin{smallmatrix}-0.1 \u0026amp; 2\\\\ -2 \u0026amp; -0.1\\end{smallmatrix}\\big]$ (the cube acting componentwise). We integrate it with explicit Euler, add Gaussian noise to obtain the data $\\mathcal D_{\\mathrm{ODE}}$, and fit a two–hidden–layer $\\tanh$ network $f_\\theta$ by minimizing $J_{\\mathrm{ODE}}$ with the regularizer of §2.3. The Euler step is\n$$ y^{(k\u0026#43;1)}=y^{(k)}\u0026#43;\\Delta t\\,f_\\theta\\big(y^{(k)},t_k\\big). $$ Optimizing one long, full step rollout directly is ill-conditioned, so we use the standard Neural-ODE demo trick [10]: each gradient step is a stochastic estimate of the data term computed from a mini-batch of short sub-trajectories started at points along the data; we keep the checkpoint with the lowest full-trajectory error.\nIn schematic form, the whole method is the vector field $f_\\theta$, the Euler integrator, and a training loop in which loss.backward() is the discrete adjoint of (A-ODE):\nclass ODEFunc(nn.Module): # f_theta: a tanh MLP on R^2 def __init__(self, dim=2, hidden=64): super().__init__() self.net = nn.Sequential( nn.Linear(dim, hidden), nn.Tanh(), nn.Linear(hidden, hidden), nn.Tanh(), nn.Linear(hidden, dim)) def forward(self, y, t): return self.net(y) def euler_rollout(f, y0, t): # explicit Euler: y += dt * f(y, t) ys, y = [y0], y0 for k in range(len(t) - 1): y = y + (t[k+1] - t[k]) * f(y, t[k]) ys.append(y) return torch.stack(ys) for it in range(ITERS): # minimize J(theta) y0_b, t_b, y_target = get_batch() # short sub-trajectories from data y_pred = euler_rollout(func, y0_b, t_b) data = 0.5 * ((y_pred - y_target) ** 2).sum(-1).mean() reg = 0.5 * GAMMA * sum((p ** 2).sum() for p in func.parameters()) opt.zero_grad(); (data + reg).backward(); opt.step() # discrete adjoint 4.2 Neural PDE (Parabolic) Ground truth is a 1D reaction–diffusion equation on $\\Omega=(0,1)$ with homogeneous Neumann (conormal) data, $\\mathcal A y=-\\nu\\,\\partial_{xx}y$, known reaction $d\\equiv 0$, and source $f_{\\mathrm{true}}(y)=5\\,y(1-y)$. The state $y(x,t)$ solves\n$$ \\begin{aligned} \\partial_t y - \\nu\\,\\partial_{xx} y \u0026amp;= f_\\theta(y) \u0026amp;\u0026amp; \\text{in } (0,1)\\times(0,T),\\\\ \\partial_x y \u0026amp;= 0 \u0026amp;\u0026amp; \\text{on } \\{0,1\\}\\times(0,T),\\\\ y(\\cdot,0) \u0026amp;= y_0 \u0026amp;\u0026amp; \\text{in } \\Omega. \\end{aligned} $$ We learn the source $f_\\theta(y)$ using a $\\tanh$ network acting on nodal values.\nWeak formulation. Multiplying by a test function $v\\in H^1(\\Omega)$, integrating over $\\Omega$, and integrating the diffusion term by parts, the boundary term $\\nu\\,\\partial_x y\\,v\\big|_0^1$ vanishes by the homogeneous Neumann condition. We seek $y(t)\\in H^1(\\Omega)$ with\n$$ \\int_\\Omega \\partial_t y\\,v\\,dx \u0026#43; \\int_\\Omega \\nu\\,\\partial_x y\\,\\partial_x v\\,dx = \\int_\\Omega f_\\theta(y)\\,v\\,dx \\qquad \\forall\\,v\\in H^1(\\Omega), $$ which is exactly the form the $P_1$ finite-element discretization below discretizes.\nA reaction can only be identified where the data lives, so we simulate several initial conditions; without this coverage the recovered $f_\\theta$ is accurate only on the narrow range a single trajectory visits.\nDiscretization. With $P_1$ finite elements on a uniform mesh we assemble the mass matrix $M$ and stiffness matrix $K$,\n$$ M_{ij}=\\int_\\Omega \\varphi_i\\varphi_j\\,dx,\\qquad K_{ij}=\\int_\\Omega \\nu\\,\\varphi_i\u0026#39;\\varphi_j\u0026#39;\\,dx, $$ and the Neumann condition is natural (no constraint on $K$). Lumping the mass to the diagonal $M_L$, the semidiscrete nodal vector $Y(t)$ obeys\n$$ M_L\\dot Y \u0026#43; \\nu K Y = M_L\\,f_\\theta(Y) \\quad\\Longleftrightarrow\\quad \\dot Y = -L\\,Y \u0026#43; f_\\theta(Y),\\quad L=\\nu\\,M_L^{-1}K , $$ so $L$ is the discrete $\\mathcal A$. Marching with explicit Euler (subject to the diffusion stability limit $\\Delta t\u0026lt;h^2/2\\nu$),\n$$ Y^{(k\u0026#43;1)}=Y^{(k)}\u0026#43;\\Delta t\\,\\big(-L\\,Y^{(k)}\u0026#43;f_\\theta\\big(Y^{(k)}\\big)\\big). $$ We generate data by simulating $f_{\\mathrm{true}}$ from each initial condition, add noise, subsample in time to obtain $\\mathcal D_{\\mathrm{PDE}}$, and fit $f_\\theta$ by minimizing $J_{\\mathrm{PDE}}$. The rightmost panel shows that the learned source matches the true nonlinearity across the shaded range of states visited by the data.\nThe schematic mirrors the Neural-ODE one: the only changes are that the integrator carries the assembled discrete operator $L=\\nu M_L^{-1}K$, the state $Y$ is a batch of nodal vectors (one per initial condition), and $f_\\theta$ acts pointwise on nodal values:\nA_DISC = NU * (1.0 / Mlump)[:, None] * K # L = nu * M_L^{-1} K (discrete A) def fem_euler(source, Y0, t): # M_L Y\u0026#39; + nu K Y = M_L f(Y) Ys, Y = [Y0], Y0 # Y0: (n_ic, N_nodes) for k in range(len(t) - 1): Y = Y + (t[k+1] - t[k]) * (-(Y @ A_DISC.t()) + source(Y)) Ys.append(Y) return torch.stack(Ys) class Source(nn.Module): # f_theta(y), applied at each node def __init__(self, hidden=64): super().__init__() self.net = nn.Sequential( nn.Linear(1, hidden), nn.Tanh(), nn.Linear(hidden, hidden), nn.Tanh(), nn.Linear(hidden, 1)) def forward(self, Y): return self.net(Y.unsqueeze(-1)).squeeze(-1) for epoch in range(EPOCHS): # fit all initial conditions jointly Y_pred = fem_euler(model, Y0, t_grid) data = 0.5 * ((Y_pred[obs_idx] - Y_obs[obs_idx]) ** 2).sum(-1).mean() reg = 0.5 * GAMMA * sum((p ** 2).sum() for p in model.parameters()) opt.zero_grad(); (data + reg).backward(); opt.step() References [1] Chen, Ricky T. Q., Yulia Rubanova, Jesse Bettencourt, and David K. Duvenaud. \u0026ldquo;Neural Ordinary Differential Equations.\u0026rdquo; Advances in Neural Information Processing Systems 31 (2018).\n[2] Pontryagin, L. S., V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko. The Mathematical Theory of Optimal Processes. Translated by K. N. Trirogoff, edited by L. W. Neustadt, Interscience Publishers (John Wiley \u0026amp; Sons), 1962.\n[3] Tröltzsch, Fredi. Optimal Control of Partial Differential Equations: Theory, Methods and Applications. Graduate Studies in Mathematics 112, American Mathematical Society, 2010. Translated by Jürgen Sprekels. (Semilinear parabolic optimal control: §5.5, problem (5.7)–(5.9), Theorems 5.5 and 5.7, Assumption 5.6. Semilinear elliptic theory: §2.3, §2.5, and §4.2, Theorems 2.7 and 4.4, Assumptions 4.2 and 4.3.)\n[4] Lions, Jacques-Louis. Optimal Control of Systems Governed by Partial Differential Equations. Translated by S. K. Mitter. Grundlehren der mathematischen Wissenschaften 170, Springer-Verlag, 1971.\n[5] Sirignano, Justin, Jonathan F. MacArt, and Jonathan B. Freund. \u0026ldquo;DPM: A deep learning PDE augmentation method with application to large-eddy simulation.\u0026rdquo; Journal of Computational Physics 423 (2020): 109811. https://doi.org/10.1016/j.jcp.2020.109811. (First posted as arXiv:1911.09145 in 2019 under the author order Freund, MacArt, Sirignano.)\n[6] Raissi, Maziar, Paris Perdikaris, and George E. Karniadakis. \u0026ldquo;Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.\u0026rdquo; Journal of Computational Physics 378 (2019): 686–707. https://doi.org/10.1016/j.jcp.2018.10.045. (First posted as arXiv:1711.10561, 2017.)\n[7] Li, Xuechen, Ting-Kam Leonard Wong, Ricky T. Q. Chen, and David Duvenaud. \u0026ldquo;Scalable Gradients for Stochastic Differential Equations.\u0026rdquo; Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 108:3870–3882, 2020.\n[8] Bronstein, Michael M., Joan Bruna, Taco Cohen, and Petar Veličković. \u0026ldquo;Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges.\u0026rdquo; arXiv preprint arXiv:2104.13478, 2021.\n[9] Le, Quoc-Tung, Elisa Riccietti, and Rémi Gribonval. \u0026ldquo;Does a sparse ReLU network training problem always admit an optimum?\u0026rdquo; Advances in Neural Information Processing Systems 36 (NeurIPS 2023).\n[10] Chen, Ricky T. Q. torchdiffeq: differentiable ODE solvers with full GPU support and O(1)-memory backpropagation. GitHub repository, https://github.com/rtqichen/torchdiffeq, example examples/ode_demo.py.\n[11] Sun, Yifan, Linan Zhang, and Hayden Schaeffer. \u0026ldquo;NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data.\u0026rdquo; Proceedings of the First Mathematical and Scientific Machine Learning Conference (MSML), PMLR 107:352–372, 2020. (First posted as arXiv:1908.03190, 2019.)\n[12] Long, Zichao, Yiping Lu, Xianzhong Ma, and Bin Dong. \u0026ldquo;PDE-Net: Learning PDEs from Data.\u0026rdquo; Proceedings of the 35th International Conference on Machine Learning (ICML), PMLR 80:3208–3216, 2018. (First posted as arXiv:1710.09668, 2017.)\nAppendix: well-posedness For the learning problem of §2 to make sense, the solution map $\\theta\\mapsto y_\\theta$ must be well-defined (and the cost finite) for the parameters explored during training.\nA.1 Neural ODE — Picard–Lindelöf Proposition (existence, uniqueness). Suppose $f_\\theta(\\cdot,t)$ is measurable in $t$, and there is $L\u0026gt;0$ with $$|f_\\theta(y_1,t)-f_\\theta(y_2,t)|\\le L\\,|y_1-y_2|\\quad\\forall\\,y_1,y_2,\\ \\text{a.e. }t,$$ together with the linear growth bound $|f_\\theta(y,t)|\\le C(1\u0026#43;|y|)$. Then for every $y_0\\in\\mathbb{R}^n$ the problem (ODE) has a unique absolutely continuous solution $y_\\theta\\in C([0,T];\\mathbb{R}^n)$, and $y_\\theta$ depends continuously (indeed differentiably) on $\\theta$ and $y_0$.\nA feed-forward network with globally Lipschitz activations ($\\tanh$, ReLU) and bounded weights is globally Lipschitz in $y$ with a constant controlled by the product of the weight-matrix norms, so the hypotheses hold; the growth bound gives existence on the whole of $[0,T]$ (no finite-time blow-up). Differentiable dependence on $\\theta$ is what makes $\\nabla_\\theta J$ in §3.1 meaningful.\nA.2 Neural PDE — Tröltzsch\u0026rsquo;s parabolic theory Tröltzsch [3], §5.5, studies the semilinear parabolic optimal control problem with a distributed control $v$ and a boundary control $u$,\n$$ \\min_{v,u}\\ J(y,v,u):=\\int_\\Omega \\phi\\big(x,y(x,T)\\big)\\,dx \u0026#43;\\iint_{\\Omega\\times(0,T)} \\varphi\\big(x,t,y,v\\big)\\,dx\\,dt \u0026#43;\\iint_{\\Gamma\\times(0,T)} \\psi\\big(x,t,y,u\\big)\\,ds\\,dt, \\tag{5.7} $$ subject to the state equation\n$$ \\begin{aligned} y_t-\\Delta y \u0026#43; d(x,t,y) \u0026amp;= v \u0026amp;\u0026amp; \\text{in } \\Omega\\times(0,T),\\\\ \\partial_\\nu y \u0026#43; b(x,t,y) \u0026amp;= u \u0026amp;\u0026amp; \\text{on } \\Gamma\\times(0,T),\\\\ y(0) \u0026amp;= y_0 \u0026amp;\u0026amp; \\text{in }\\Omega, \\end{aligned}\\tag{5.8} $$ and the control constraints\n$$ v_a\\le v\\le v_b\\ \\text{ a.e. in } \\Omega\\times(0,T),\\qquad u_a\\le u\\le u_b\\ \\text{ a.e. in } \\Gamma\\times(0,T).\\tag{5.9} $$ Our (PDE-P) is exactly the state equation (5.8) with $\\mathcal A=-\\Delta$ and the right-hand sides taken to be the network $f_\\theta$ in the interior and $g$ on the boundary (here $b$ is given). State well-posedness — the solution map being well-defined, which is what §2 needs — is Tröltzsch\u0026rsquo;s Theorem 5.5.\nTheorem 5.5 (Tröltzsch [3]). Suppose that Assumptions 5.1 and 5.4 hold. Then the semilinear parabolic initial–boundary value problem $$y_t\u0026#43;\\mathcal A y\u0026#43;d(x,t,y)=f \\ \\text{ in } \\Omega\\times(0,T),\\qquad \u0026gt; \\partial_{\\nu_{\\mathcal A}}y\u0026#43;b(x,t,y)=g \\ \\text{ on } \\Gamma\\times(0,T),\\qquad \u0026gt; y(0)=y_0 \\ \\text{ in }\\Omega$$ has a unique weak solution $y\\in W(0,T)\\cap C(\\overline{\\Omega\\times(0,T)})$ for any triple $f\\in L^r(\\Omega\\times(0,T))$, $g\\in L^s(\\Gamma\\times(0,T))$, and $y_0\\in C(\\overline\\Omega)$ with $r\u0026gt;N/2\u0026#43;1$ and $s\u0026gt;N\u0026#43;1$. Moreover, there is a constant $c_\\infty\u0026gt;0$, which is independent of $d$, $b$, $f$, $g$, and $y_0$, such that $$\\|y\\|_{W(0,T)}\u0026#43;\\|y\\|_{C(\\overline{\\Omega\\times(0,T)})}\\le \u0026gt; c_\\infty\\big(\\|f-d(\\cdot,0)\\|_{L^r(\\Omega\\times(0,T))}\u0026#43;\\|g-b(\\cdot,0)\\|_{L^s(\\Gamma\\times(0,T))}\u0026#43;\\|y_0\\|_{C(\\overline\\Omega)}\\big).\\tag{5.6}$$ Assumptions 5.1 and 5.4 are the structural hypotheses on $\\mathcal A$, $d$, and $b$ (measurability, boundedness/Lipschitz, and the monotonicity $d_y\\ge 0$, $b_y\\ge 0$); the integrability thresholds $r\u0026gt;N/2\u0026#43;1$ and $s\u0026gt;N\u0026#43;1$ buy the continuous representative $y\\in C(\\overline{\\Omega\\times(0,T)})$. For existence of an optimal solution the problem (5.7)–(5.9) strengthens these to the twice-differentiable Assumption 5.6, which we state together with the existence result verbatim.\nAssumption 5.6 (Tröltzsch [3]). (i) $\\Omega\\subset\\mathbb{R}^N$ is a bounded Lipschitz domain.\n(ii) The functions $$d=d(x,t,y):(\\Omega\\times(0,T))\\times\\mathbb{R}\\to\\mathbb{R},\\qquad \\phi=\\phi(x,y):\\Omega\\times\\mathbb{R}\\to\\mathbb{R},$$ $$\\varphi=\\varphi(x,t,y,v):(\\Omega\\times(0,T))\\times\\mathbb{R}^2\\to\\mathbb{R},\\qquad b=b(x,t,y):(\\Gamma\\times(0,T))\\times\\mathbb{R}\\to\\mathbb{R},$$ $$\\psi=\\psi(x,t,y,u):(\\Gamma\\times(0,T))\\times\\mathbb{R}^2\\to\\mathbb{R}$$ are measurable with respect to $(x,t)$ for all $y,v,u\\in\\mathbb{R}$ and, for almost every $(x,t)$ in $\\Omega\\times(0,T)$ or $\\Gamma\\times(0,T)$, twice differentiable with respect to $y$, $v$, and $u$. Moreover, they satisfy the boundedness and local Lipschitz conditions (4.24)–(4.25) of order $k=2$; this means that for $\\varphi$, for example, there exist some $K\u0026gt;0$ and a constant $L(M)\u0026gt;0$ for any $M\u0026gt;0$ such that, with the objects $\\nabla\\varphi$ and $\\varphi\u0026#39;\u0026#39;$ explained in the book, $$|\\varphi(x,t,0,0)|\u0026#43;|\\nabla\\varphi(x,t,0,0)|\u0026#43;|\\varphi\u0026#39;\u0026#39;(x,t,0,0)|\\le K,$$ $$|\\varphi\u0026#39;\u0026#39;(x,t,y_1,v_1)-\\varphi\u0026#39;\u0026#39;(x,t,y_2,v_2)|\\le L(M)\\big\\{|y_1-y_2|\u0026#43;|v_1-v_2|\\big\\},$$ for almost every $(x,t)\\in \\Omega\\times(0,T)$ and any $y_i,v_i\\in[-M,M]$, $i=1,2$.\n(iii) We have $d_y(x,t,y)\\ge 0$ for almost every $(x,t)\\in \\Omega\\times(0,T)$ and $b_y(x,t,y)\\ge 0$ for almost every $(x,t)\\in\\Gamma\\times(0,T)$. Moreover, $y_0\\in C(\\overline\\Omega)$.\n(iv) The bounds $u_a,u_b$ and $v_a,v_b:E\\to\\mathbb{R}$ belong to $L^\\infty(E)$ for $E=\\Gamma\\times(0,T)$ and $E=\\Omega\\times(0,T)$, respectively, and $u_a(x,t)\\le u_b(x,t)$ and $v_a(x,t)\\le v_b(x,t)$ for almost every $(x,t)\\in E$.\nTheorem 5.7 (Tröltzsch [3]). Suppose that Assumption 5.6 holds, and let $\\varphi$ and $\\psi$ be convex with respect to $v$ and $u$, respectively. Then the optimal control problem (5.7)–(5.9) has at least one optimal pair $(\\bar v,\\bar u)$ with associated optimal state $\\bar y=y(\\bar v,\\bar u)$.\nThe monotonicity $d_y\\ge 0$, $b_y\\ge 0$ in (iii) is the structural condition behind well-posedness of the state equation (5.8). In our learning problem the control is the network $v=f_\\theta$, so the relevant nonlinearity is the effective reaction $d-f_\\theta$: to stay inside this theory we need $\\partial_y(d-f_\\theta)\\ge 0$ together with the order-$2$ boundedness/Lipschitz bounds — automatic on bounded sets for $\\tanh$ networks except for the monotonicity, which one keeps by (i) letting $f_\\theta$ depend only on $(x,t)$ (genuine control, $v\\in L^\\infty(\\Omega\\times(0,T))$), or (ii) penalizing $\\partial_y f_\\theta$. Existence of a minimizer in $\\theta$ comes instead from the Tikhonov coercivity of §2.3 — the finite-dimensional analogue of the convexity hypothesis on $\\varphi,\\psi$ in Theorem 5.7. In the example the fixed coercive diffusion $\\mathcal A$ dominates on the resolved mesh.\nA.3 Neural PDE (Elliptic) — Tröltzsch\u0026rsquo;s elliptic theory For the linear problem ($f_\\theta=f_\\theta(x)$ independent of $y$) Tröltzsch [3], §2.3 and §2.5, gives existence, uniqueness, and an a priori bound for the boundary condition of the third kind.\nTheorem 2.7 (Tröltzsch [3]). Let $\\Omega\\subset\\mathbb{R}^N$ be a bounded Lipschitz domain and suppose $c_0\\in L^\\infty(\\Omega)$ and $\\alpha\\in L^\\infty(\\Gamma_1)$ satisfy $c_0(x)\\ge 0$ and $\\alpha(x)\\ge 0$ almost everywhere. If one of (i) $|\\Gamma_0|\u0026gt;0$ (a Dirichlet part of positive measure), or (ii) $\\Gamma_1=\\Gamma$ and $\\int_\\Omega c_0(x)^2\\,dx\u0026#43;\\int_\\Gamma \\alpha(x)^2\\,ds(x)\u0026gt;0$, holds, then for all $f\\in L^2(\\Omega)$ and $g\\in L^2(\\Gamma_1)$ the problem (PDE-E) has a unique weak solution $y\\in V$, and there is a constant $c_{\\mathcal A}\u0026gt;0$, depending on neither $f$ nor $g$, with $$\\|y\\|_{H^1(\\Omega)}\\le c_{\\mathcal A}\\big(\\|f\\|_{L^2(\\Omega)}\u0026#43;\\|g\\|_{L^2(\\Gamma_1)}\\big).$$ Condition (i) or (ii) excludes the pure-Neumann constant null space (otherwise $y$ is determined only up to a constant): some zeroth-order term — a Dirichlet boundary, the potential $c_0$, or the Robin coefficient $\\alpha$ — must pin the solution.\nFor the semilinear problem the source depends on $y$, and Tröltzsch [3], §4.2, gives well-posedness under the following hypotheses. The boundary value problem is $$\\mathcal A y \u0026#43; c_0\\,y \u0026#43; d(x,y) = f \\ \\text{ in } \\Omega,\\qquad \\partial_{\\nu_{\\mathcal A}}y \u0026#43; \\alpha\\,y \u0026#43; b(x,y) = g \\ \\text{ on } \\Gamma,\\tag{4.5}$$ associated with the bilinear form $$a[y,v]:=\\int_\\Omega\\sum_{i,j=1}^N a_{ij}(x)\\,D_i y\\,D_j v\\,dx \u0026#43;\\int_\\Omega c_0\\,y\\,v\\,dx\u0026#43;\\int_\\Gamma\\alpha\\,y\\,v\\,ds.\\tag{4.7}$$ Assumption 4.2 (Tröltzsch [3]). (i) $\\Omega\\subset\\mathbb{R}^N$, $N\\ge 2$, is a bounded Lipschitz domain with boundary $\\Gamma$, and $\\mathcal A$ is an elliptic differential operator of the form (2.19) with bounded and measurable coefficient functions $a_{ij}$ that satisfy the symmetry condition and the condition (2.20) of uniform ellipticity.\n(ii) The functions $c_0:\\Omega\\to\\mathbb{R}$ and $\\alpha:\\Gamma\\to\\mathbb{R}$ are bounded, measurable and almost everywhere nonnegative. At least one of these functions does not vanish almost everywhere, that is, $\\|c_0\\|_{L^\\infty(\\Omega)}\u0026#43;\\|\\alpha\\|_{L^\\infty(\\Gamma)}\u0026gt;0$.\n(iii) The functions $d=d(x,y):\\Omega\\times\\mathbb{R}\\to\\mathbb{R}$ and $b=b(x,y):\\Gamma\\times\\mathbb{R}\\to\\mathbb{R}$ are bounded and measurable with respect to $x\\in\\Omega$ and $x\\in\\Gamma$, respectively, for every fixed $y\\in\\mathbb{R}$. Moreover, they are continuous and monotone increasing in $y$ for almost every $x\\in\\Omega$ and $x\\in\\Gamma$, respectively.\nIn particular $d(x,0)$ and $b(x,0)$ are then bounded and measurable in $\\Omega$ and $\\Gamma$.\nAssumption 4.3 (Tröltzsch [3]). For almost every $x\\in\\Omega$ (respectively, $x\\in\\Gamma$) we have $d(x,0)=0$ (respectively, $b(x,0)=0$). Moreover, $b$ and $d$ are globally bounded, that is, there is a constant $M\u0026gt;0$ such that for any $y\\in\\mathbb{R}$ we have $$|b(x,y)|\\le M\\ \\text{ for a.e. } x\\in\\Gamma,\\qquad \u0026gt; |d(x,y)|\\le M\\ \\text{ for a.e. } x\\in\\Omega.\\tag{4.6}$$ Theorem 4.4 (Tröltzsch [3]). Suppose that Assumptions 4.2 and 4.3 hold. Then the elliptic boundary value problem (4.5) has for any pair $f\\in L^2(\\Omega)$ and $g\\in L^2(\\Gamma)$ of right-hand sides a unique weak solution $y\\in H^1(\\Omega)$. Moreover, there is some constant $c_M\u0026gt;0$, which is independent of $d$, $b$, $f$, and $g$, such that $$\\|y\\|_{H^1(\\Omega)}\\le c_M\\big(\\|f\\|_{L^2(\\Omega)}\u0026#43;\\|g\\|_{L^2(\\Gamma)}\\big).\\tag{4.9}$$ The monotonicity of $d$ and $b$ in (iii) is the structural condition behind well-posedness, exactly as in the parabolic case. In our learning problem the source is the network, so the relevant nonlinearity is the effective reaction $d-f_\\theta$ (with $d\\equiv 0$ in the example): to stay inside this theory we need $\\partial_y\\big(d-f_\\theta\\big)\\ge 0$, which one keeps by letting $f_\\theta$ depend only on $x$ or by penalizing $\\partial_y f_\\theta$. Existence of a minimizer in $\\theta$ again comes from the Tikhonov coercivity of §2.3.\n","permalink":"https://dani2442.github.io/posts/neural-pde/","summary":"\u003cp\u003e\u003cimg alt=\"Neural ODEs and Neural PDEs\" loading=\"lazy\" src=\"/posts/neural-pde/continuous_rl-neural-pde.drawio.svg\"\u003e\u003c/p\u003e\n\u003cp\u003eClassical data-driven methods in machine learning tend to be data-inefficient and  they discard expert modeling knowledge. However, during the last decade a different approach has emerged under the name \u003cem\u003ephysics-informed machine learning\u003c/em\u003e.\u003c/p\u003e\n\u003cp\u003eFirst posted in 2017 and published in 2019, \u003cem\u003ephysics-informed\nneural networks\u003c/em\u003e (PINNs) \u003ca href=\"#references\"\u003e[6]\u003c/a\u003e combined deep learning with prior\nknowledge of the governing equations by directly learning the solution using the PDE residual as a soft penalty\nin the loss. In 2018, \u003cem\u003eNeural ODEs\u003c/em\u003e \u003ca href=\"#references\"\u003e[1]\u003c/a\u003e took a complementary\nroute of embedding the physics in the \u003cem\u003emodel\u003c/em\u003e itself: the right-hand side of an\nordinary differential equation $\\dot y=f(y,t)$ is replaced by a neural network\n$f\\approx f_\\theta$, trained to reproduce observed data. The same principle was\nsoon extended to stochastic differential equations \u003ca href=\"#references\"\u003e[7]\u003c/a\u003e and to\npartial differential equations \u003ca href=\"#references\"\u003e[5, 11, 12]\u003c/a\u003e — the latter is what we call\nhere a \u003cem\u003eNeural PDE\u003c/em\u003e.\u003c/p\u003e","title":"Neural ODEs and Neural PDEs"},{"content":"\nMachine learning feels recent, but one of its core mathematical ideas dates back to 1952, when Richard Bellman published a seminal paper titled \u0026ldquo;On the Theory of Dynamic Programming\u0026rdquo; [6, 7], laying the foundation for optimal control and what we now call reinforcement learning.\nLater in the 50s, Bellman extended his work to continuous-time systems, turning the optimal condition into a PDE. What he later found was that this was identical to a result in physics published a century before (1840s), known as the Hamilton-Jacobi equation.\nOnce that structure is visible, several topics line up naturally:\ncontinuous-time reinforcement learning stochastic control diffusion models optimal transport In this post I want to turn our attention to two applications of Bellman\u0026rsquo;s work: continuous-time reinforcement learning, and how the training of generative models (diffusion models) can be interpreted through stochastic optimal control\n1. Introduction Bellman originally formulated dynamic programming in discrete time in the early 1950s [6, 7]. Consider a Markov decision process with state space $\\mathcal X$, action space $\\mathcal A$, transition kernel $P(\\cdot\\mid x,a)$, reward function $r(x,a)$, and discount factor $\\gamma\\in(0,1)$. A policy $\\pi$ maps each state to a distribution over actions. If the state evolves as a controlled Markov chain $$ X_{n\u0026#43;1}\\sim P(\\cdot\\mid X_n,a_n), $$ with one-step reward $r(x,a)$ and discount factor $\\gamma\\in(0,1)$, then the objective is $$ J(\\pi):=\\mathbb E\\left[\\sum_{n=0}^\\infty \\gamma^n r(X_n,a_n)\\right],\\qquad a_n\\sim \\pi(\\cdot\\mid X_n), $$ and the value function is defined as: $$ V(x):=\\sup_\\pi \\mathbb E\\left[\\sum_{n=0}^\\infty \\gamma^n r(X_n,a_n)\\,\\middle|\\,X_0=x\\right]. $$ Under mild conditions, $V$ satisfies the Bellman equation $$ V(x)=\\max_{a\\in\\mathcal A}\\left\\{r(x,a)\u0026#43;\\gamma\\,\\mathbb E \\left[V(X_{n\u0026#43;1})\\mid X_n=x,a_n=a\\right]\\right\\}.\\tag{Bellman equation} $$ This says: choose the action that maximizes immediate reward plus continuation value. Continuous time keeps the same local logic, but now the time step has length $h$ and we send $h\\downarrow 0$.\nTo isolate the main idea, first ignore noise and consider the non-autonomous deterministic control system $$ \\dot X_s = f(s,X_s,a_s),\\qquad X_t=x, $$ with finite-horizon value function $$ V(t,x):=\\sup_{a_\\cdot}\\left[\\int_t^T r(s,X_s,a_s)\\,ds\u0026#43;g(X_T)\\,\\middle|\\,X_t=x\\right]. $$ Theorem (HJB, deterministic non-autonomous). For $V\\in C^1$, the value function satisfies $$ \u0026gt; -\\partial_t V(t,x) = H\\bigl(t,x,\\nabla_x V(t,x)\\bigr), \\tag{HJB} \u0026gt; $$ where the Hamiltonian is $H(t,x,p):=\\sup_{a\\in\\mathcal A}\\left\\{r(t,x,a)\u0026#43;p^\\top f(t,x,a)\\right\\}$.\nProof. Fix $(t,x)$ and $h\u0026gt;0$. The dynamic programming principle gives $$ V(t,x)=\\sup_{a_\\cdot}\\left[\\int_t^{t\u0026#43;h} r(s,X_s,a_s)\\,ds \u0026#43; V(t\u0026#43;h,X_{t\u0026#43;h})\\right]. $$ To first order in $h$, it is enough to optimize over constant actions $a$ on $[t,t\u0026#43;h]$. For smooth $V$ and deterministic dynamics: $$ V(t\u0026#43;h,X_{t\u0026#43;h})=V(t,x)\u0026#43;h\\,\\partial_t V(t,x)\u0026#43;h\\,\\nabla_x V(t,x)^\\top f(t,x,a)\u0026#43;o(h), $$ $$ \\int_t^{t\u0026#43;h} r(s,X_s,a)\\,ds = h\\,r(t,x,a)\u0026#43;o(h). $$ Substituting into the DPP, cancelling $V(t,x)$, dividing by $h$, and letting $h\\downarrow 0$: $$ 0=\\sup_{a\\in\\mathcal A}\\left\\{r(t,x,a)\u0026#43;\\nabla_x V(t,x)^\\top f(t,x,a)\u0026#43;\\partial_t V(t,x)\\right\\}, $$ which rearranges to $-\\partial_t V(t,x) = H(t,x,\\nabla_x V(t,x))$. $\\quad\\blacksquare$\nConnection to Hamilton–Jacobi. What Bellman realized in the 1950s is that the partial differential equation produced by dynamic programming has exactly the same structure as the 19th-century Hamilton–Jacobi equation from classical mechanics. Writing the running reward as minus a Lagrangian, $r(t,x,a)=-L(t,x,a)$, define $$ H(t,x,p):=\\sup_{a\\in\\mathcal A}\\{p^\\top f(t,x,a)-L(t,x,a)\\}. $$ The HJB equation then becomes identical to Hamilton’s equation for the action $S(t,q)$, $$ \\frac{\\partial S}{\\partial t}\u0026#43;H\\!\\left(q,\\frac{\\partial S}{\\partial q}\\right)=0. $$ Under the correspondence $S\\leftrightarrow V$ and $q\\leftrightarrow x$, the two equations are the same at the level of PDE structure.\nControlled diffusions (Itô processes) We now move to the stochastic setting: continuous time, continuous state and action spaces, and Itô dynamics. Assume the system evolves according to the SDE $$ dX_t = f(X_t,a_t)\\,dt \u0026#43; \\Sigma(X_t,a_t)\\,dW_t $$ where $X_t$ is the state, $a_t$ is the control, $W_t$ is a standard Wiener process, and $f$ and $\\Sigma$ describe drift and diffusion. The reward is given by $r(x,a)$, and the objective is to maximize expected discounted reward over an infinite horizon: $$ J(\\pi):=\\mathbb{E}\\Big[\\int_0^\\infty e^{-\\rho t}r(X_t,a_t)\\,dt\\Big],\\qquad a_t\\sim \\pi(\\cdot\\mid X_t) $$ where $\\rho\u0026gt;0$ is the discount rate. The associated value function is $$ V(x):=\\sup_\\pi \\mathbb{E}\\Big[\\int_0^\\infty e^{-\\rho t}r(X_t,a_t)\\,dt \\Big| X_0=x\\Big] $$ Theorem (Hamilton-Jacobi-Bellman equation for a controlled diffusion). Under suitable regularity conditions:\n$f(\\cdot,a)$, $\\Sigma(\\cdot,a)$, $r(\\cdot,a)$ are continuous in $(x,a)$; Lipschitz in $x$ uniformly in $a$. $\\Sigma\\Sigma^\\top(x,a)$ is bounded and uniformly nondegenerate (for classical $C^2$ theory; if you drop this you typically work in viscosity form). $r$ is bounded (or has at most linear growth with enough integrability). $V\\in C^2(\\mathbb R^d)$ and bounded (or polynomial growth, with the usual technical modifications). Then the value function satisfies the Hamilton-Jacobi-Bellman (HJB) PDE $$ \\rho V(x)=\\max_{a\\in \\mathcal{A}}\\Big\\{ r(x,a)\u0026#43;\\mathcal{L}^a V(x)\\Big\\}.\\tag{1} $$ where $\\mathcal{L}^a$ is the infinitesimal generator under action $a$: $$ \\mathcal{L}^a \\varphi(x):=\\nabla \\varphi(x)^\\top f(x,a) \u0026#43;\\tfrac12 \\mathrm{Tr}\\big(\\Sigma\\Sigma^\\top \\nabla^2 \\varphi(x)\\big) $$ Proof. The structure is the same as in the deterministic case. The only new ingredient is the short-time expansion of $\\mathbb E_x[V(X_h)]$: by Itô\u0026rsquo;s formula, $$ \\mathbb E_x[V(X_h)] = V(x) \u0026#43; h\\,\\mathcal L^a V(x) \u0026#43; o(h), $$ where the generator $\\mathcal{L}^a$ replaces the directional derivative $\\nabla V^\\top f$, adding the curvature term $\\tfrac{1}{2}\\mathrm{Tr}(\\Sigma\\Sigma^\\top\\nabla^2 V)$ coming from the quadratic variation of $W$. The rest is unchanged: substitute into the DPP, cancel $V(x)$, divide by $h$, and let $h\\downarrow 0$ to obtain (1). $\\quad\\blacksquare$\nThe same argument also yields the non-autonomous HJB when $f$, $\\Sigma$, and $r$ depend explicitly on time; see Appendix C.\nHistorical Note: In 1960, Rudolf E. Kalman published his seminal paper on the linear-quadratic regulator (LQR) problem [8], which is a continuous-time optimal control problem with linear dynamics and quadratic cost. The solution to the LQR problem is given by the algebraic Riccati equation, which can be derived from the Hamilton-Jacobi-Bellman (HJB) equation for continuous-time control problems.\n2. Continuous-time Reinforcement Learning Define the continuous-time analogue of the Q-function by $$ Q(x,a):=\\frac{1}{\\rho}\\Big(r(x,a)\u0026#43;\\mathcal{L}^a V(x)\\Big).\\tag{2} $$ From the HJB (1), it follows immediately that $V(x)=\\max_{a} Q(x,a)$. This identity is the basis of policy improvement: once we have an estimate of $V$, the greedy action is $a^*(x) = \\arg\\max_a Q(x,a)$.\nThis stationary, discounted form is the RL convention used in the next two sections.\n2.1 Policy Iteration We solve the HJB numerically with policy iteration (PI), alternating between evaluating the current policy and improving it through the Q-function. Both the value $V_\\theta$ and the policy $\\alpha_\\phi$ are represented by MLPs.\nThis algorithm is model-based: it assumes known dynamics through $f(x,a)$ and $\\Sigma(x,a)$ (equivalently, access to the generator $\\mathcal L^a$). The model is used both to simulate closed-loop trajectories in policy evaluation and to compute $\\mathcal L^aV$ in policy improvement.\nWe iterate the following steps until convergence:\nPolicy evaluation (value under current policy $\\alpha_k$): $$ \\rho V_k(x)=r\\big(x,\\alpha_k(x)\\big)\u0026#43;\\mathcal{L}^{\\alpha_k(x)}V_k(x). $$ In practice, we estimate $V_k\\approx V^{\\alpha_k}$ by Monte Carlo rollouts of the closed-loop SDE and fit $V_\\theta$ by regression.\nPolicy improvement (greedy with respect to $V_k$): $$ \\alpha_{k\u0026#43;1}(x)\\in\\arg\\max_{a\\in\\mathcal A}\\{r(x,a)\u0026#43;\\mathcal L^aV_k(x)\\} =\\arg\\max_{a\\in\\mathcal A} Q_k(x,a), $$ where $$ Q_k(x,a):=\\frac{1}{\\rho}\\big(r(x,a)\u0026#43;\\mathcal L^aV_k(x)\\big). $$ With a differentiable actor $\\alpha_\\phi$, this becomes gradient ascent on $$ \\max_\\phi\\;\\mathbb E_x\\big[Q_k\\big(x,\\alpha_\\phi(x)\\big)\\big]. $$ Diagnosis / stopping: $$ \\mathcal R_{\\mathrm{HJB}}(x)=\\rho V(x)-\\max_a\\{r(x,a)\u0026#43;\\mathcal L^aV(x)\\}, $$ and stop when sampled norms of $\\mathcal R_{\\mathrm{HJB}}$ and parameter changes plateau.\nIntuition: evaluation estimates the value landscape induced by the current policy, and improvement moves the policy uphill on that landscape. Repeating the two steps drives $(V,\\alpha)$ toward a fixed point of the HJB.\nComputing the generator $\\mathcal{L}^a V$ The generator requires $\\nabla V$ and $\\nabla^2 V$, obtained from $V_\\theta$ via autograd. The diffusion $\\Sigma(x,a)$ is problem-given (model-based setting).\ndef compute_generator(V_net, x, f_xa, Sigma_xa): \u0026#34;\u0026#34;\u0026#34;L^a V(x) = ∇V · f + ½ Tr(ΣΣᵀ ∇²V).\u0026#34;\u0026#34;\u0026#34; V = V_net(x) # (batch, 1) grad_V = autograd.grad(V.sum(), x, create_graph=True)[0] # (batch, d) drift = (grad_V * f_xa).sum(-1, keepdim=True) # ∇V · f d = x.shape[1] H = torch.stack([autograd.grad(grad_V[:,i].sum(), x, create_graph=True)[0] for i in range(d)], dim=1) A = Sigma_xa @ Sigma_xa.transpose(-1,-2) # ΣΣᵀ diff = 0.5 * (A * H).sum(dim=(-2,-1)).unsqueeze(-1) # ½Tr(AH) return drift + diff During policy improvement, $\\nabla V$ and $\\nabla^2 V$ are detached from $\\theta$ so gradients flow only through $\\phi$.\nPolicy evaluation (Feynman–Kac MC) For a fixed policy $\\alpha$, $V^\\alpha$ solves the linear PDE $$ \\rho V^\\alpha(x)=r\\big(x,\\alpha(x)\\big)\u0026#43;\\mathcal L^{\\alpha(x)}V^\\alpha(x). $$ By the Feynman-Kac representation, for any truncation horizon $T\u0026gt;0$, $$ V^\\alpha(x)= \\mathbb E_x\\!\\left[\\int_0^\\infty e^{-\\rho s}\\,r\\big(X_s,\\alpha(X_s)\\big)\\,ds\\right]=\\mathbb E_x\\!\\left[\\int_0^T e^{-\\rho s}\\,r\\big(X_s,\\alpha(X_s)\\big)\\,ds \u0026#43; e^{-\\rho T}V^\\alpha(X_T)\\right] $$ In Monte Carlo policy evaluation, we approximate this expectation with simulated trajectories and use the critic to bootstrap the terminal value at time $T$.\nPolicy improvement At collocation points, compute $Q_k(x,\\alpha_\\phi(x))$ using the detached $\\nabla V_k$, $\\nabla^2 V_k$, and maximise $\\mathbb{E}[Q_k]$ w.r.t. $\\phi$:\ngrad_V, H = compute_generator_detached(V_net, x) # frozen V a = policy_net(x) # differentiable in φ f_xa, S_xa = problem.drift(x, a), problem.diffusion(x, a) L_V = generator_from_precomputed(grad_V, H, f_xa, S_xa) Q = (problem.reward(x, a) + L_V) / rho loss = -Q.mean() # gradient ascent on Q loss.backward() opt_pi.step() 2.2 Model-Free: Continuous-Time Q-learning Policy iteration is model-based. A complementary route is Q-learning, which can be implemented model-free from sampled transitions.\nIn continuous time, the Q-function satisfies the PDE $$ \\rho Q(x,a)=r(x,a)\u0026#43;\\mathcal L^a\\big(\\max_{a\u0026#39;\\in\\mathcal A}Q(x,a\u0026#39;)\\big).\\tag{4} $$ With neural networks, set $$ Q_\\psi(x,a)\\approx Q(x,a),\\qquad a_\\omega(x)\\approx \\arg\\max_{a}Q_\\psi(x,a), $$ where $Q_\\psi$ (critic) and $a_\\omega$ (actor) are MLPs.\nUsing short transitions $(X_t,a_t,r_t,X_{t\u0026#43;\\Delta t})$ and a small step size $\\Delta t$, a practical TD target is $$ y_t = r_t\\,\\Delta t \u0026#43; e^{-\\rho\\Delta t}\\,\\bar V(X_{t\u0026#43;\\Delta t}), \\qquad \\bar V(x):=Q_{\\bar\\psi}(x,a_\\omega(x))\\ \\text{(or }\\max_a Q_{\\bar\\psi}(x,a)\\text{)}. $$ Then train the critic with $$ \\mathcal L_Q(\\psi)=\\mathbb E\\big[(Q_\\psi(X_t,a_t)-y_t)^2\\big]. $$ The actor is updated by ascent on $$ \\max_\\omega\\;\\mathbb E\\big[Q_\\psi(X_t,a_\\omega(X_t))\\big]. $$ This mirrors the usual actor-critic split: one network fits state-action values, while the other outputs actions that maximize them.\nExample 1 — Stochastic LQR The linear-quadratic regulator is the canonical continuous-time control benchmark: linear dynamics, quadratic cost, and a closed-form solution. That makes it ideal for validating a numerical solver.\nProblem setup Dynamics (additive noise, 1-D scalar): $$dX_t = (\\alpha\\,X_t \u0026#43; \\beta\\,a_t)\\,dt \u0026#43; \\sigma\\,dW_t$$ Reward (negative quadratic cost): $$r(x,a) = -\\tfrac{1}{2}(q\\,x^2 \u0026#43; r_a\\,a^2)$$ Symbol Meaning Value $\\alpha$ open-loop drift (stable if $\u0026lt;0$) $-0.5$ $\\beta$ control effectiveness $1.0$ $q$ state cost weight $1.0$ $r_a$ action cost weight $0.1$ $\\sigma$ diffusion (noise intensity) $0.3$ $\\rho$ discount rate $0.1$ Analytical solution Substituting a quadratic ansatz $V(x) = -\\tfrac{1}{2}Px^2 - c$ into the HJB and optimising over $a$ yields (see Appendix A for the full derivation):\nThe closed-form objects are: $$ a^*(x)=-\\frac{\\beta P}{r_a}x=: -Kx, $$ $$ \\rho P = q \u0026#43; 2\\alpha P - \\frac{\\beta^2}{r_a}P^2, $$ $$ c = \\frac{\\sigma^2 P}{2\\rho}. $$ Code class StochasticLQR(ControlProblem): def drift(self, x, a): return x @ A.T + a @ B.T def diffusion(self, x, a): return D.unsqueeze(0).expand(x.shape[0], -1, -1) def reward(self, x, a): return -0.5*((x@Q*x).sum(-1,keepdim=True) + (a@R*a).sum(-1,keepdim=True)) P, c, K = solve_are(A, B, Q, R, D, rho=0.1) # exact Riccati solution solver = PolicyIteration(problem, cfg) history = solver.solve() # neural PI Results Learned $V_\\theta$ and $\\alpha_\\phi$ closely match $V^*(x) = -\\tfrac{1}{2}Px^2 - c$ and $a^*(x) = -Kx$:\nSample trajectories under the learned policy ($x_0=1.5$) and cumulative discounted reward:\nConvergence diagnostics (value-fit MSE, policy objective, HJB residual):\nExample 2 — Merton Portfolio Merton\u0026rsquo;s (1969) portfolio problem asks how an investor should allocate wealth between a risk-free bond and a risky asset while also choosing a consumption rate. The objective is to maximize expected lifetime CRRA (constant relative risk aversion) utility of consumption. It also admits a closed-form solution, making it a useful second benchmark with multiplicative noise rather than the additive noise of LQR.\nProblem setup State: wealth $X_t \u0026gt; 0$. Controls: $a_t = (\\pi_t, k_t)$ — risky-asset fraction and consumption-to-wealth ratio $k = c/X$.\nDynamics (geometric / multiplicative noise): $$dX_t = \\big[r_f \u0026#43; \\pi_t(\\mu - r_f) - k_t\\big] X_t dt \u0026#43; \\pi_t \\sigma X_t dW_t$$ Reward (CRRA utility of consumption flow, $\\gamma \\neq 1$): $$r(x,a) = \\frac{(k\\,x)^{1-\\gamma}}{1-\\gamma}$$ Symbol Meaning Value $r_f$ risk-free rate $0.03$ $\\mu$ risky asset expected return $0.08$ $\\sigma$ risky asset volatility $0.20$ $\\gamma$ relative risk aversion (CRRA) $2.0$ $\\rho$ subjective discount rate $0.05$ Analytical solution Substituting a power-law ansatz $V(x) = \\frac{A}{1-\\gamma}x^{1-\\gamma}$ into the HJB and optimising over $(\\pi, k)$ yields (see Appendix B for the full derivation):\nThe closed-form controls and value are: $$ \\pi^*=\\frac{\\mu-r_f}{\\gamma\\sigma^2}=0.625, $$ $$ k^*=\\frac{\\rho-(1-\\gamma)M}{\\gamma},\\qquad M:=r_f\u0026#43;\\frac{(\\mu-r_f)^2}{2\\gamma\\sigma^2},\\qquad k^*\\approx 0.0478, $$ $$ V^*(x)=\\frac{A}{1-\\gamma}x^{1-\\gamma},\\qquad A=\\left(\\frac{\\gamma}{\\rho-(1-\\gamma)M}\\right)^\\gamma. $$ Both optimal controls are constant — independent of wealth and time.\nCode class MertonProblem(ControlProblem): def drift(self, x, a): # a = (π, c_rate) pi, cr = a[:, 0:1], a[:, 1:2] return (self.r_f + pi*(self.mu - self.r_f) - cr) * x def diffusion(self, x, a): return (a[:, 0:1] * self.sigma * x).unsqueeze(-1) def reward(self, x, a): c = (a[:, 1:2] * x).clamp(min=1e-8) return c.pow(1 - self.gamma) / (1 - self.gamma) Results Learned value function matches the exact power-law $V^*\\propto x^{1-\\gamma}$; both controls converge to the analytical constants:\nSample wealth trajectories under the learned policy ($X_0 = 1$) and cumulative discounted reward:\nConvergence diagnostics:\n3. Diffusion Models The same HJB machinery also appears in diffusion models once reverse-time sampling is written as a control problem. Let $p_{\\text{data}}(x)$ be the target data distribution. For simplicity, consider a forward diffusion whose noise coefficient depends only on time, as in standard score-based SDE formulations: $$ dY_t = f(Y_t, t)\\,dt \u0026#43; \\sigma(t)\\,dB_t, \\qquad Y_0 \\sim p_{\\text{data}}. $$ Let $p_t(x)$ denote the marginal density of $Y_t$. Under standard regularity assumptions, the time reversal of this process is again a diffusion. Instead of writing time backward from $T$ down to $0$, define $$ X_t := Y_{T-t}, \\qquad t\\in[0,T]. $$ Then $X_0\\sim p_T$, $X_T\\sim p_{\\text{data}}$, and $X_t$ has marginal $p_{T-t}$.\nTo expose the control structure [9], define the reverse-time drift and diffusion coefficients $$ \\mu(x, t) := -f(x, T-t), \\qquad \\Sigma(t) := \\sigma(T-t). $$ Now consider a family of controlled diffusions $X_t^u$ driven by an arbitrary control field $u(x, t)$: $$ dX_t^u = \\big(\\mu(X_t^u, t) \u0026#43; \\Sigma(t) u(X_t^u, t)\\big)\\,dt \u0026#43; \\Sigma(t)\\,dW_t, \\qquad X_0^u \\sim p_T. $$ The goal is to choose $u$ so that the terminal law of $X_T^u$ matches $p_{\\text{data}}$.\nNow define the candidate value function $$ V(x, t) := -\\log p_{T-t}(x). $$ This is the negative log-density of the reverse-time marginals, so its terminal value is $$ V(x, T) = -\\log p_{\\text{data}}(x). $$ To identify the PDE satisfied by $V$, recall that the forward marginals $p_t$ solve the Fokker-Planck equation for $Y_t$. Consequently, $\\rho_t(x) := p_{T-t}(x) = e^{-V(x, t)}$ satisfies the reverse-time Fokker-Planck PDE $$ \\partial_t \\rho_t = -\\operatorname{div}(\\mu\\,\\rho_t) - \\tfrac{1}{2}\\operatorname{Tr}\\big(\\Sigma\\Sigma^\\top \\nabla_x^2 \\rho_t\\big). $$ Substituting $\\rho_t = e^{-V}$, $\\nabla_x \\rho_t = -e^{-V}\\nabla_x V$, and $\\nabla_x^2 \\rho_t = e^{-V}(\\nabla_x V\\nabla_x V^\\top - \\nabla_x^2 V)$ into that PDE and dividing by $-e^{-V}$ yields $$ \\partial_t V = \\operatorname{div}\\mu - \\mu \\cdot \\nabla_x V \u0026#43; \\tfrac{1}{2}\\|\\Sigma^\\top \\nabla_x V\\|^2 - \\tfrac{1}{2}\\operatorname{Tr}\\big(\\Sigma\\Sigma^\\top \\nabla_x^2 V\\big). $$ To rewrite this as a control problem, introduce a control variable $u$ through the convex-conjugate identity for the quadratic function $g(y) = \\frac{1}{2}\\|y\\|^2$: $$ \\tfrac{1}{2}\\|y\\|^2 = \\sup_{u\\in\\mathbb{R}^d} \\left\\{ u \\cdot y - \\tfrac{1}{2}\\|u\\|^2 \\right\\}. $$ Setting $y = -\\Sigma^\\top \\nabla_x V$ lets us rewrite the quadratic gradient term as $$ \\tfrac{1}{2}\\|-\\Sigma^\\top \\nabla_x V\\|^2 = \\sup_{u} \\left\\{ u^\\top (-\\Sigma^\\top \\nabla_x V) - \\tfrac{1}{2}\\|u\\|^2 \\right\\} = -\\inf_{u} \\left\\{ \\tfrac{1}{2}\\|u\\|^2 \u0026#43; (\\Sigma u) \\cdot \\nabla_x V \\right\\}. $$ This is the key step: it replaces a quadratic gradient term with a linear one that can be interpreted as controlled drift.\nPlugging this back into the PDE for $V$, multiplying by $-1$, and collecting the $u$-independent terms inside the infimum gives the finite-horizon HJB equation $$ -\\partial_t V = \\inf_u \\left\\{ \\tfrac{1}{2}\\|u\\|^2 - \\operatorname{div}\\mu \u0026#43; (\\mu \u0026#43; \\Sigma u)\\cdot \\nabla_x V \u0026#43; \\tfrac{1}{2}\\operatorname{Tr}\\big(\\Sigma\\Sigma^\\top \\nabla_x^2 V\\big) \\right\\}. \\tag{3} $$ This PDE shows that $V(x,t) = -\\log p_{T-t}(x)$ is exactly the value function of a stochastic control problem with dynamics $X_t^u$ and cost $$ J(u; x, t) = \\mathbb{E}\\left[ \\int_t^T \\left( \\tfrac{1}{2}\\|u(X_s^u,s)\\|^2 - \\operatorname{div}\\mu(X_s^u,s) \\right) ds - \\log p_{\\text{data}}(X_T^u) \\;\\middle|\\; X_t^u=x \\right]. $$ The optimal control law $u^*(x, t)$ is the minimizer in the HJB equation. From the quadratic optimization above, the minimizer of $\\frac{1}{2}\\|u\\|^2 \u0026#43; u^\\top (\\Sigma^\\top \\nabla_x V)$ is obtained by setting its derivative with respect to $u$ to zero: $$ u^* \u0026#43; \\Sigma^\\top \\nabla_x V = 0 \\implies u^*(x, t) = -\\Sigma^\\top(t)\\nabla_x V(x, t). $$ Since $V(x, t) = -\\log p_{T-t}(x)$, we have $\\nabla_x V = -\\nabla_x \\log p_{T-t}(x)$. Substituting this identity gives the exact optimal control law $$ u^*(x, t) = \\Sigma^\\top(t)\\nabla_x \\log p_{T-t}(x) = \\sigma(T-t)^\\top \\nabla_x \\log p_{T-t}(x). $$ Therefore the controlled drift becomes $$ \\mu(x,t)\u0026#43;\\Sigma(t)u^*(x,t) =-f(x,T-t)\u0026#43;\\Sigma(t)\\Sigma(t)^\\top \\nabla_x \\log p_{T-t}(x), $$ which is exactly the reverse-time score correction.\nApplying Itô\u0026rsquo;s formula to $V(X_s^u, s)$ along an arbitrary controlled trajectory and then using the HJB gives the verification identity $$ J(u; x, t) = V(x, t) \u0026#43; \\frac{1}{2} \\mathbb{E}\\left[ \\int_t^T \\| u(X_s^u,s) - u^*(X_s^u,s) \\|^2 ds \\;\\middle|\\; X_t^u = x \\right]. $$ This identity is the control-theoretic backbone of diffusion models:\nDepending on the parameterization, the network can predict either the score $s(x,t)=\\nabla_x\\log p_{T-t}(x)$ or the scaled control $u(x,t)=\\Sigma^\\top(t)s(x,t)$. The verification gap is quadratic in the control error. Under the standard diffusion-model reparameterization used in practice, the resulting ELBO reduces to a weighted denoising score-matching objective [9]. When $u=u^*$, the gap vanishes and the terminal law of the reverse process matches the data distribution exactly. So diffusion-based generative modeling can be viewed as a finite-horizon stochastic optimal control problem whose optimal policy is precisely the score-induced reverse-time drift correction.\nReferences [1] Jia, Yanwei, and Xun Yu Zhou. \u0026ldquo;q-Learning in continuous time.\u0026rdquo; Journal of Machine Learning Research 24, no. 161 (2023): 1-61.\n[2] Jia, Yanwei, and Xun Yu Zhou. \u0026ldquo;Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms.\u0026rdquo; Journal of Machine Learning Research 23, no. 275 (2022): 1-50.\n[3] Hamilton-Jacobi-Bellman Equations, Stochastic Differential Equations by Benjamin Moll https://benjaminmoll.com/wp-content/uploads/2019/07/Lecture4_ECO521_web.pdf\n[4] Fleming, Wendell H., and H. Mete Soner. Controlled Markov processes and viscosity solutions. New York, NY: Springer New York, 2006.\n[5] Yong, Jiongmin, and Xun Yu Zhou. Stochastic controls: Hamiltonian systems and HJB equations. Vol. 43. Springer Science \u0026amp; Business Media, 1999.\n[6] Bellman, Richard. \u0026ldquo;On the Theory of Dynamic Programming.\u0026rdquo; Proceedings of the National Academy of Sciences 38, no. 8 (1952): 716-719. https://doi.org/10.1073/pnas.38.8.716.\n[7] Bellman, Richard Ernest. An Introduction to the Theory of Dynamic Programming. Santa Monica, CA: RAND Corporation, 1953. https://www.rand.org/pubs/reports/R245.html.\n[8] Kalman, Rudolf E. \u0026ldquo;Contributions to the Theory of Optimal Control.\u0026rdquo; Boletin de la Sociedad Matematica Mexicana 5 (1960): 102-119. https://boletin.math.org.mx/pdf/2/5/BSMM%282%29.5.102-119.pdf.\n[9] Berner, Julius, Lorenz Richter, and Karen Ullrich. \u0026ldquo;An optimal control perspective on diffusion-based generative modeling.\u0026rdquo; Transactions on Machine Learning Research, 2024. https://arxiv.org/abs/2211.01364.\nAppendix A: LQR Derivation Ansatz. Guess $V(x) = -\\tfrac{1}{2}Px^2 - c$ with $P \u0026gt; 0$. Then $V\u0026#39;=-Px$, $V\u0026#39;\u0026#39;=-P$.\nGenerator. With drift $f = \\alpha x \u0026#43; \\beta a$ and constant diffusion $\\sigma$: $$ \\mathcal{L}^a V =V\u0026#39;(\\alpha x\u0026#43;\\beta a)\u0026#43;\\tfrac{1}{2}\\sigma^2V\u0026#39;\u0026#39; =-Px(\\alpha x\u0026#43;\\beta a)-\\tfrac{1}{2}\\sigma^2P =-\\alpha P x^2-\\beta P a x-\\tfrac{1}{2}\\sigma^2P. $$ HJB. Substituting into $\\rho V = \\max_a \\{r \u0026#43; \\mathcal{L}^a V\\}$: $$ -\\tfrac{1}{2}\\rho P x^2 - \\rho c =\\max_{a}\\Big\\{-\\tfrac{1}{2}qx^2-\\tfrac{1}{2}r_a a^2-\\alpha P x^2-\\beta P a x-\\tfrac{1}{2}\\sigma^2P\\Big\\}. $$ Optimality in $a$. The RHS is concave in $a$; set $\\partial_a(\\cdot) = 0$: $$ -r_a\\,a-\\beta P x=0 \\implies a^*(x)=-\\frac{\\beta P}{r_a}\\,x=: -Kx. $$ Riccati equation. Substituting $a^* = -Kx$ back and matching the $x^2$ coefficient and the constant:\n$$x^2:\\quad \\rho P = q \u0026#43; 2\\alpha P - \\frac{\\beta^2}{r_a}P^2, \\qquad \\text{const:}\\quad c = \\frac{\\sigma^2 P}{2\\rho}$$ The first is the discounted algebraic Riccati equation. In the scalar case it is a quadratic in $P$. $\\quad\\blacksquare$\nAppendix B: Merton Derivation Ansatz. Power-law value: $V(x) = \\frac{A}{1-\\gamma}\\,x^{1-\\gamma}$, $A \u0026gt; 0$. Then: $$V\u0026#39;(x) = A\\,x^{-\\gamma},\\qquad V\u0026#39;\u0026#39;(x) = -\\gamma A\\,x^{-\\gamma-1}$$ Generator. Using drift $\\mu_X x = [r_f \u0026#43; \\pi(\\mu-r_f) - k]x$ and diffusion $\\sigma_X x = \\pi\\sigma x$: $$\\mathcal{L}^a V = A\\,x^{-\\gamma}\\cdot\\mu_X x - \\tfrac{1}{2}\\gamma A\\,x^{-\\gamma-1}\\cdot\\sigma_X^2 x^2 = A\\,x^{1-\\gamma}\\big[r_f \u0026#43; \\pi(\\mu-r_f) - k - \\tfrac{1}{2}\\gamma\\pi^2\\sigma^2\\big]$$ HJB. Substituting and dividing through by $x^{1-\\gamma} \u0026gt; 0$: $$\\frac{\\rho A}{1-\\gamma} = \\max_{\\pi,\\,k}\\left\\{\\frac{k^{1-\\gamma}}{1-\\gamma} \u0026#43; A\\big[r_f \u0026#43; \\pi(\\mu-r_f) - k - \\tfrac{1}{2}\\gamma\\pi^2\\sigma^2\\big]\\right\\}$$ Optimality in $\\pi$. FOC $\\partial_\\pi(\\cdot) = 0$: $\\;A[(\\mu-r_f) - \\gamma\\pi\\sigma^2] = 0$: $$\\pi^* = \\frac{\\mu - r_f}{\\gamma\\,\\sigma^2}$$ This is the myopic portfolio rule — independent of wealth and time. Higher risk aversion $\\gamma$ or volatility $\\sigma$ reduces exposure.\nOptimality in $k$. FOC $\\partial_k(\\cdot) = 0$: $\\;k^{-\\gamma} - A = 0$, so $k^* = A^{-1/\\gamma}$.\nSolving for $A$. Define the certainty-equivalent growth rate $M := r_f \u0026#43; \\frac{(\\mu-r_f)^2}{2\\gamma\\sigma^2}$. Substituting the optimisers back: $$ \\frac{\\rho A}{1-\\gamma} =\\frac{(A^{-1/\\gamma})^{1-\\gamma}}{1-\\gamma}\u0026#43;A\\big[M-A^{-1/\\gamma}\\big] =\\frac{A^{(\\gamma-1)/\\gamma}}{1-\\gamma}\u0026#43;AM-A^{(\\gamma-1)/\\gamma}. $$ Multiplying by $1-\\gamma$ and collecting terms gives $$ \\rho A=(1-\\gamma)AM\u0026#43;\\gamma A^{(\\gamma-1)/\\gamma}. $$ Since $A\u0026gt;0$, divide by $A^{(\\gamma-1)/\\gamma}$ to obtain $$ A^{1/\\gamma}=\\frac{\\gamma}{\\rho-(1-\\gamma)M}. $$ Therefore $$ A = \\left(\\frac{\\gamma}{\\rho - (1-\\gamma)M}\\right)^\\gamma, \\qquad k^* = \\frac{\\rho - (1-\\gamma)M}{\\gamma} $$ The denominator $\\rho - (1-\\gamma)M$ must be positive — this is the feasibility condition ensuring lifetime utility is finite. With our parameters: $M \\approx 0.04563$, so $k^* \\approx 0.0478$. $\\quad\\blacksquare$\nAppendix C: Non-autonomous and Finite-Horizon Cases Let the dynamics and reward depend on time: $$ dX_t=f(t,X_t,a_t)\\,dt\u0026#43;\\Sigma(t,X_t,a_t)\\,dW_t,\\qquad r=r(t,x,a). $$ For the discounted infinite-horizon problem, define the time-dependent value $$ V(t,x):=\\sup_\\pi \\mathbb E\\Big[\\int_t^\\infty e^{-\\rho(s-t)} r(s,X_s,a_s)\\,ds\\ \\Big|\\ X_t=x\\Big]. $$ Then the time-dependent generator is $$ \\mathcal L_t^a \\varphi(x)=\\nabla \\varphi(x)^\\top f(t,x,a)\u0026#43;\\tfrac12\\mathrm{Tr}\\big(\\Sigma\\Sigma^\\top(t,x,a)\\nabla^2\\varphi(x)\\big), $$ and the HJB becomes $$ \\rho V(t,x)=\\max_{a\\in\\mathcal A}\\Big\\{r(t,x,a)\u0026#43;\\partial_t V(t,x)\u0026#43;\\mathcal L_t^a V(t,x)\\Big\\}. $$ Equivalently, $$ -\\partial_t V(t,x)=\\max_{a\\in\\mathcal A}\\Big\\{r(t,x,a)\u0026#43;\\mathcal L_t^a V(t,x)\\Big\\}-\\rho V(t,x). $$ In the autonomous case, $V(t,x)$ is time-independent, so $\\partial_t V=0$ and you recover (1).\nFor the finite-horizon deterministic case, set $\\Sigma\\equiv 0$ and define $$ V(t,x):=\\sup_{a_\\cdot}\\left[\\int_t^T r(s,X_s,a_s)\\,ds\u0026#43;g(X_T)\\,\\middle|\\,X_t=x\\right]. $$ Then $$ -\\partial_t V(t,x)=\\sup_{a\\in\\mathcal A}\\left\\{r(t,x,a)\u0026#43;\\nabla_x V(t,x)^\\top f(t,x,a)\\right\\}, \\qquad V(T,x)=g(x). $$ Writing $r=-L$ and $$ H(t,x,p):=\\sup_{a\\in\\mathcal A}\\{p^\\top f(t,x,a)-L(t,x,a)\\}, $$ this becomes $$ \\partial_t V(t,x)\u0026#43;H\\bigl(t,x,\\nabla_x V(t,x)\\bigr)=0, \\qquad V(T,x)=g(x), $$ which is the classical Hamilton-Jacobi form.\n","permalink":"https://dani2442.github.io/posts/continuous-rl/","summary":"\u003cp\u003e\u003cimg loading=\"lazy\" src=\"/posts/continuous-rl/continuous_rl.drawio.svg\"\u003e\u003c/p\u003e\n\u003cp\u003eMachine learning feels recent, but one of its core mathematical ideas dates back to 1952, when Richard Bellman published a seminal paper titled \u0026ldquo;On the Theory of Dynamic Programming\u0026rdquo; \u003ca href=\"#references\"\u003e[6, 7]\u003c/a\u003e, laying the foundation for optimal control and what we now call reinforcement learning.\u003c/p\u003e\n\u003cp\u003eLater in the 50s, Bellman extended his work to continuous-time systems, turning the optimal condition into a PDE. What he later found was that this was identical to a result in physics published a century before (1840s), known as the Hamilton-Jacobi equation.\u003c/p\u003e","title":"Hamilton-Jacobi-Bellman Equation: Reinforcement Learning and Diffusion Models"},{"content":"I gave a talk at our internal machine learning seminar, together with Max Golightly, on the road that leads from optimal control to reinforcement learning.\nWe started from the deterministic picture — variational calculus, Hamilton–Jacobi, the Pontryagin maximum principle — and then moved to the probabilistic one, going through Markov decision processes, dynamic programming and policy gradient methods.\nI’m sharing the slides here in case you’re interested in these topics Link.\n","permalink":"https://dani2442.github.io/events/2026_03_01_ml_seminar/","summary":"\u003cp\u003eI gave a talk at our internal machine learning seminar, together with Max Golightly, on the road that leads from optimal control to reinforcement learning.\u003c/p\u003e\n\u003cp\u003eWe started from the deterministic picture — variational calculus, Hamilton–Jacobi, the Pontryagin maximum principle — and then moved to the probabilistic one, going through Markov decision processes, dynamic programming and policy gradient methods.\u003c/p\u003e\n\u003cp\u003eI’m sharing the slides here in case you’re interested in these topics \u003ca href=\"/files/ml_seminar_optimal_control.pdf\"\u003eLink\u003c/a\u003e.\u003c/p\u003e","title":"Talk at the Internal Machine Learning Seminar"},{"content":"\nThe Peter-Weyl theorem is one of the most beautiful results in Harmonic Analysis, and it has found surprisingly wide applications in machine learning (drug discovery, molecular dynamics, robotics, particle physics, graphs, etc).\nIn many real-world problems, data is scarce, complexity is high, or we simply want our models to generalize better and produce more robust solutions. In these cases, it makes sense to leverage any prior knowledge we have about the problem structure. For example, in meteorology the data lives on a sphere (the Earth), or we might know that the underlying dynamics roughly satisfy the Navier-Stokes equations. The desire to incorporate this kind of domain-specific knowledge into machine learning models gave rise to what we now call Physics-Informed ML and Geometry-Informed ML. The former led to Neural ODEs and Physics-Informed Neural Networks (PINNs), but in this post we focus on the latter.\nOne of the reasons Convolutional Neural Networks work so well is a mathematical property that allows the model to recognize patterns even when they are shifted along the x- or y-axis. This is known as translation invariance (or equivariance), and we owe it to the convolution operation.\nWhat\u0026rsquo;s interesting is that this same idea generalizes far beyond translation. It extends to arbitrary symmetries (rotations, reflections, permutations, etc) and to spaces beyond Euclidean geometry, such as graphs (giving us Graph Convolutional Networks). In full generality, it applies to locally compact groups.\nIn this post, we explore the theorem that makes all of this possible, and walk through how to compute the convolution on two concrete examples: SO(2) and SO(3).\n1. Preliminaries We start by introducing some basic notions from representation theory and harmonic analysis on compact groups.\nLet $G$ be a locally compact group with a fixed (left) Haar measure $\\mu$.\nDefinition (Unitary representation). A (unitary) representation of $G$ is a continuous homomorphism $\\pi:G\\to \\mathfrak{U}(\\mathcal{H}_\\pi)$, where $\\mathfrak{U}(\\mathcal{H}_\\pi)$ is the unitary group of a complex Hilbert space $\\mathcal{H}_\\pi$.\nEquivalently, $\\pi(gh)=\\pi(g)\\pi(h)$ for all $g,h\\in G$ and $g\\mapsto \\pi(g)\\xi$ is continuous for every $\\xi\\in \\mathcal{H}_\\pi$.\nWe denote $d_\\pi:=\\dim(\\mathcal{H}_\\pi)\\in \\mathbb{Z}_{\\ge 0}\\cup\\{\\infty\\}$.\nFor compact $G$, one often starts with (not necessarily unitary) finite-dimensional representations $\\pi:G\\to \\mathrm{GL}(V)$. In that case, averaging an inner product over Haar measure produces a $G$-invariant inner product, so every continuous finite-dimensional representation is equivalent to a unitary one.\nExample (Left regular representation). On $\\mathcal{H}:=L^2(G)$, the left regular representation $\\lambda:G\\to \\mathfrak{U}(\\mathcal{H})$ is defined by $$ (\\lambda(g)f)(x):=f(g^{-1}x),\\qquad f\\in L^2(G). $$ Definition (Equivalence). Two unitary representations $\\pi_1:G\\to\\mathfrak{U}(\\mathcal{H}_1)$ and $\\pi_2:G\\to\\mathfrak{U}(\\mathcal{H}_2)$ are equivalent if there exists a unitary $U:\\mathcal{H}_1\\to\\mathcal{H}_2$ such that $$ U\\pi_1(g)=\\pi_2(g)U,\\qquad \\forall g\\in G. $$ We write $\\pi_1\\cong \\pi_2$ and denote by $[\\pi]$ the equivalence class of $\\pi$.\nDefinition (Irreducible). A unitary representation $\\pi:G\\to\\mathfrak{U}(\\mathcal{H}_\\pi)$ is irreducible if the only closed subspaces of $\\mathcal{H}_\\pi$ invariant under every $\\pi(g)$ are $\\{0\\}$ and $\\mathcal{H}_\\pi$.\nWe denote by $\\widehat{G}$ the set of equivalence classes of irreducible unitary representations of $G$.\nIf $G$ is abelian, then every irreducible unitary representation is one-dimensional; in particular, $\\widehat{G}$ coincides with the character group (Pontryagin dual) of $G$.\nGiven representations $\\pi$ and $\\eta$, the tensor product representation $\\pi\\otimes \\eta$ acts on $\\mathcal{H}_\\pi\\otimes \\mathcal{H}_\\eta$ via $$ (\\pi\\otimes \\eta)(g)(\\xi\\otimes \\zeta)=\\pi(g)\\xi\\otimes \\eta(g)\\zeta,\\qquad g\\in G. $$ If $n\\in \\mathbb{Z}^\u0026#43;\\cup\\{\\infty\\}$, then $n\\cdot \\pi:=\\bigoplus_{i=1}^n \\pi$ denotes the direct sum of $n$ copies of $\\pi$.\nDefinition (Commutant). If $\\pi$ is a unitary representation of $G$, then $$ \\pi(G)\u0026#39; := \\{A\\in \\mathcal{B}(\\mathcal{H}_\\pi) : A\\pi(g)=\\pi(g)A,\\ \\forall g\\in G\\} $$ is a *-subalgebra of $\\mathcal{B}(\\mathcal{H}_\\pi)$ called the commutant of $\\pi(G)$.\nTheorem (Schur\u0026rsquo;s lemma / commutant characterization). For a unitary representation $\\pi$, the following are equivalent:\n$\\pi$ is irreducible. $\\pi(G)\u0026#39;=\\mathbb{C}I$. Every nonzero $\\xi\\in \\mathcal{H}_\\pi$ is cyclic, i.e. $$ \\overline{\\mathrm{span}}\\{\\pi(g)\\xi:g\\in G\\}=\\mathcal{H}_\\pi. $$ Proof. (1)$\\Rightarrow$(2): Let $A\\in \\pi(G)\u0026#39;$. The real and imaginary parts of $A$ are self-adjoint and still commute with $\\pi(G)$, so it suffices to consider self-adjoint $A$. If $A$ is not a scalar multiple of the identity, the spectral theorem produces a nontrivial spectral projection $P$ of $A$ (e.g. $P=\\mathbf{1}_{(-\\infty,t]}(A)$ for a suitable $t$). Since $P$ is a Borel functional calculus of $A$, it also commutes with $\\pi(G)$; hence $\\mathrm{ran}(P)$ is a nontrivial closed $\\pi$-invariant subspace, contradicting irreducibility.\n(2)$\\Rightarrow$(1): If $W\\subseteq \\mathcal{H}_\\pi$ is a nontrivial closed $\\pi$-invariant subspace, the orthogonal projection $P_W$ belongs to $\\pi(G)\u0026#39;$. By (2), $P_W$ must be $0$ or $I$, contradiction.\n(1)$\\Leftrightarrow$(3): For $\\xi\\neq 0$, the cyclic subspace $\\overline{\\mathrm{span}}\\{\\pi(g)\\xi:g\\in G\\}$ is a nonzero closed invariant subspace, so it equals $\\mathcal{H}_\\pi$ iff $\\pi$ is irreducible. $\\square$\n2. Peter-Weyl Theorem In this section we specialize to compact groups. A locally compact group $G$ is compact iff $\\mu(G)\u0026lt;\\infty$; in that case Haar measure can be normalized uniquely by requiring $\\mu(G)=1$. With this normalization we write $$ \\int_G f(g)\\,dg $$ for integration against Haar measure.\n2.1. Matrix coefficients and the coefficient space Let $\\pi$ be a finite-dimensional unitary representation of $G$ on $\\mathcal{H}_\\pi$ and let $\\{e_1,\\dots,e_{d_\\pi}\\}$ be an orthonormal basis. The functions $$ \\varphi_{ij}^{\\pi}(g):=\\langle e_i,\\pi(g)e_j\\rangle,\\qquad 1\\le i,j\\le d_\\pi, $$ are called the matrix coefficients (or coordinate functions) of $\\pi$.\nMore generally, for any $\\xi,\\eta\\in \\mathcal{H}_\\pi$ the function $g\\mapsto \\langle \\xi,\\pi(g)\\eta\\rangle$ is continuous.\nDefinition (Coefficient space). Let $E_G$ be the linear span of all functions of the form $$ g\\mapsto \\langle \\xi,\\pi(g)\\eta\\rangle, $$ where $\\pi$ ranges over all irreducible unitary representations of $G$ and $\\xi,\\eta\\in\\mathcal{H}_\\pi$.\nWe will write $E:=E_G$ when $G$ is clear.\nSince every finite-dimensional unitary representation of a compact group decomposes as a direct sum of irreducibles, every matrix coefficient coming from an arbitrary finite-dimensional representation belongs to $E$.\nRemark (Abelian case and trigonometric polynomials).\nWhen $G$ is abelian, the functions in $E$ are often called trigonometric polynomials. For example, if $$ G=\\mathbb{T}=\\mathbb{R}/2\\pi\\mathbb{Z}, $$ then every $f\\in E$ has the form $$ f(\\theta)=\\sum_{n=-N}^N c_n e^{in\\theta} = \\sum_{n=0}^N \\bigl(a_n\\cos(n\\theta)\u0026#43;b_n\\sin(n\\theta)\\bigr). $$ Remark (Conjugation and inversion).\nIf $f\\in E$, then $\\overline{f}\\in E$. In terms of coordinate functions, $$ \\overline{\\varphi_{ij}^{\\pi}(g)} =\\overline{\\langle e_i,\\pi(g)e_j\\rangle} =\\langle \\pi(g)e_j,e_i\\rangle =\\langle e_j,\\pi(g^{-1})e_i\\rangle =\\varphi_{ji}^\\pi(g^{-1}). $$ In particular, if $f\\in E$, then $\\widetilde f(g):=f(g^{-1})$ also lies in $E$ (e.g. by passing to the contragredient representation). The same argument works inside $E_{\\mathrm{fin}}$, since the contragredient of a finite-dimensional representation is again finite-dimensional.\n2.2. Hilbert-Schmidt norms Definition (Hilbert-Schmidt norm). Let $M_n$ be the complex $n\\times n$ matrices. If $A=(a_{ij})\\in M_n$, define the Hilbert-Schmidt norm by $$ \\|A\\|_{\\mathrm{HS}}^2=\\sum_{i,j}|a_{ij}|^2. $$ Remark. If $A\\in M_n$, then $$ \\|A\\|_{\\mathrm{HS}}^2=\\mathrm{tr}(AA^*). $$ In particular, if $B=UAU^*$ for a unitary matrix $U$, then $\\|A\\|_{\\mathrm{HS}}=\\|B\\|_{\\mathrm{HS}}$. Hence, if $\\pi$ is finite-dimensional, the value $\\|\\pi(g)\\|_{\\mathrm{HS}}$ depends only on the equivalence class $[\\pi]$.\n2.3. Statement of the Peter-Weyl theorem Our goal is to prove the following result.\nTheorem (Peter-Weyl). Let $G$ be a compact group.\nEvery irreducible unitary representation of $G$ is finite-dimensional. If $\\lambda$ is the left regular representation of $G$ on $L^2(G)$, then $$ \\lambda \\;\\cong\\; \\bigoplus_{[\\pi]\\in \\widehat{G}} d_\\pi\\cdot \\pi. $$ (Separation of points) Given $g\\in G$ with $g\\ne e$, there exists $[\\pi]\\in\\widehat G$ such that $\\pi(g)\\ne I$. The space $E$ is dense in $C(G)$ (hence also dense in $L^p(G)$ for all $1\\le p\u0026lt;\\infty$). (Plancherel / Parseval identity) If $f\\in L^2(G)$, then $$ \\|f\\|_2^2 = \\sum_{[\\pi]\\in\\widehat G} d_\\pi\\,\\mathrm{tr}\\bigl(\\widehat f(\\pi)\\,\\widehat f(\\pi)^*\\bigr) = \\sum_{[\\pi]\\in\\widehat G} d_\\pi \\|\\widehat f(\\pi)\\|_{\\mathrm{HS}}^2, $$ where the operator-valued Fourier transform $\\widehat f(\\pi)$ is defined by $$ \\widehat f(\\pi)\\;:=\\;\\int_G f(g)\\,\\pi(g^{-1})\\,dg \\quad\\in\\mathcal{B}(\\mathcal{H}_\\pi). $$ We will prove this theorem using a sequence of preliminary results.\n2.4. Intertwiners and invariant subspaces Lemma 1 (Adjoints of intertwiners). Let $\\pi$ and $\\eta$ be unitary representations of a locally compact group $G$. If $A:\\mathcal{H}_\\pi\\to\\mathcal{H}_\\eta$ is bounded and satisfies $$ A\\pi(g)=\\eta(g)A,\\qquad \\forall g\\in G, $$ then $A^*$ also intertwines: $$ A^*\\eta(g)=\\pi(g)A^*,\\qquad \\forall g\\in G. $$ In particular, $A^*A\\in \\pi(G)\u0026#39;$ and $AA^*\\in \\eta(G)\u0026#39;$.\nProof. Taking adjoints in $A\\pi(g)=\\eta(g)A$ gives $\\pi(g^{-1})A^*=A^*\\eta(g^{-1})$, and since $\\pi$ and $\\eta$ are unitary this is equivalent to $A^*\\eta(g)=\\pi(g)A^*$. The commutant statements follow by multiplying by $A$ and $A^*$. $\\square$\nLemma 2 (Nonzero intertwiners from irreducibles). Suppose $\\pi$ and $\\eta$ are unitary representations of a locally compact group $G$, with $\\pi$ irreducible. If $A:\\mathcal{H}_\\pi\\to\\mathcal{H}_\\eta$ is a nonzero bounded operator satisfying $$ A\\pi(g)=\\eta(g)A,\\qquad \\forall g\\in G, $$ then:\n$A\\mathcal{H}_\\pi$ is a closed $\\eta$-invariant subspace of $\\mathcal{H}_\\eta$; $\\pi\\cong \\eta|_{A\\mathcal{H}_\\pi}$. Proof. By Lemma 1, $A^*A\\in \\pi(G)\u0026#39;$, so Schur\u0026rsquo;s lemma gives $A^*A=\\lambda I$ for some $\\lambda\\ge 0$. Since $A\\neq 0$, we have $\\lambda\u0026gt;0$. Then $B:=\\lambda^{-1/2}A$ satisfies $B^*B=I$, so $B$ is an isometry and $A\\mathcal{H}_\\pi=B\\mathcal{H}_\\pi$ is closed.\nThe intertwining identity implies $B\\pi(g)=\\eta(g)B$, so $B\\mathcal{H}_\\pi$ is $\\eta$-invariant. Finally, $B$ is a unitary equivalence between $\\pi$ and $\\eta$ restricted to $B\\mathcal{H}_\\pi$. $\\square$\n2.5. Orthogonality relations The next proposition is the key orthogonality statement for matrix coefficients, and it depends crucially on compactness of $G$.\nProposition (Schur orthogonality). Let $\\pi$ and $\\eta$ be irreducible finite-dimensional unitary representations of a compact group $G$. Fix orthonormal bases $\\{e_i^\\pi\\}_{i=1}^{d_\\pi}$ of $\\mathcal{H}_\\pi$ and $\\{e_k^\\eta\\}_{k=1}^{d_\\eta}$ of $\\mathcal{H}_\\eta$, and define $$ \\varphi_{ij}^\\pi(g)=\\langle e_i^\\pi,\\pi(g)e_j^\\pi\\rangle, \\qquad \\varphi_{kl}^\\eta(g)=\\langle e_k^\\eta,\\eta(g)e_l^\\eta\\rangle. $$ If $\\pi\\not\\cong \\eta$, then $$ \\int_G \\varphi_{ij}^\\pi(g)\\,\\overline{\\varphi_{kl}^\\eta(g)}\\,dg=0 \\qquad\\text{for all }i,j,k,l. $$ If $\\pi\\cong \\eta$, then $$ \\int_G \\varphi_{ij}^\\pi(g)\\,\\overline{\\varphi_{kl}^\\pi(g)}\\,dg = \\frac{\\delta_{ik}\\delta_{jl}}{d_\\pi}, \\qquad\\text{for all }i,j,k,l. $$ Proof. Fix a bounded operator $B:\\mathcal{H}_\\pi\\to\\mathcal{H}_\\eta$ and define $$ A:=\\int_G \\eta(g)\\,B\\,\\pi(g^{-1})\\,dg. $$ Since $\\pi$ and $\\eta$ are finite-dimensional, $g\\mapsto \\eta(g)B\\pi(g^{-1})$ is continuous in operator norm, so the integral is well-defined (as a Bochner integral).\nFor $r\\in G$, left invariance of Haar measure implies $$ A\\pi(r)= \\int_G \\eta(g)\\,B\\,\\pi(g^{-1}r)\\,dg= \\int_G \\eta(rg)\\,B\\,\\pi(g^{-1})\\,dg= \\eta(r)\\,A, $$ so $A$ intertwines $\\pi$ and $\\eta$.\n(1) If $\\pi\\not\\cong\\eta$, then Lemma 2 forces $A=0$. Fix indices $j,k$ and take the rank-one operator $B_{kj}(\\xi)=\\langle \\xi,e_j^\\pi\\rangle e_k^\\eta$. Then for any $i,l$, $$ \\begin{aligned} 0\u0026amp;=\\langle A e_i^\\pi, e_l^\\eta\\rangle \\\\ \u0026amp;= \\int_G \\left\\langle \\eta(g)\\,B_{kj}\\,\\pi(g^{-1})e_i^\\pi,\\ e_l^\\eta\\right\\rangle dg \\\\\u0026amp;= \\int_G \\left\\langle B_{kj}\\,\\pi(g^{-1})e_i^\\pi,\\ \\eta(g^{-1})e_l^\\eta\\right\\rangle dg \\\\\u0026amp;= \\int_G \\langle \\pi(g^{-1})e_i^\\pi,e_j^\\pi\\rangle\\ \\langle e_k^\\eta,\\eta(g^{-1})e_l^\\eta\\rangle\\ dg \\\\\u0026amp;= \\int_G \\varphi_{ij}^\\pi(g)\\ \\overline{\\varphi_{lk}^\\eta(g)}\\ dg, \\end{aligned} $$ using unitarity in the last step. Relabeling indices yields the desired orthogonality.\n(2) Now assume $\\pi=\\eta$. Then $A\\in \\pi(G)\u0026#39;$, so by Schur\u0026rsquo;s lemma $A=\\lambda I$ for some $\\lambda\\in \\mathbb{C}$. Taking traces gives $$ \\lambda d_\\pi=\\mathrm{tr}(A)=\\int_G \\mathrm{tr}\\!\\left(\\pi(g)B\\pi(g^{-1})\\right)dg= \\int_G \\mathrm{tr}(B)\\,dg=\\mathrm{tr}(B), $$ so $\\lambda=\\mathrm{tr}(B)/d_\\pi$. Taking $B=B_{jl}$ on $\\mathcal{H}_\\pi$ (i.e. $B_{jl}(\\xi)=\\langle \\xi,e_l^\\pi\\rangle e_j^\\pi$) gives $\\lambda=\\delta_{jl}/d_\\pi$, and $$ \\int_G \\varphi_{ij}^\\pi(g)\\,\\overline{\\varphi_{kl}^\\pi(g)}\\,dg= \\langle A e_i^\\pi,e_k^\\pi\\rangle= \\lambda\\,\\delta_{ik}= \\frac{\\delta_{ik}\\delta_{jl}}{d_\\pi}. $$ $\\square$\n2.6. Proof of the Peter\u0026ndash;Weyl theorem Let $E_{\\mathrm{fin}}\\subseteq E$ denote the subspace spanned by matrix coefficients coming from finite-dimensional irreducible representations of $G$.\nWe first show that it suffices to prove that $E_{\\mathrm{fin}}$ is dense in $C(G)$.\nAssume $E_{\\mathrm{fin}}$ is dense in $C(G)$. Since $C(G)$ is dense in $L^2(G)$, it follows that $E_{\\mathrm{fin}}$ is also dense in $L^2(G)$.\nFor each irreducible $\\pi$, Proposition (Schur orthogonality) shows that the $d_\\pi^2$ functions $\\{\\sqrt{d_\\pi}\\,\\varphi_{ij}^\\pi\\}_{i,j}$ form an orthonormal set in $L^2(G)$, and their linear span contains all matrix coefficients of $\\pi$. Hence the closed span of all such normalized coefficients contains $E_{\\mathrm{fin}}$, and therefore equals $L^2(G)$. In other words, these normalized matrix coefficients form an orthonormal basis of $L^2(G)$.\nThis already implies item (1): if there were an irreducible unitary representation $\\sigma$ with $\\dim(\\mathcal{H}_\\sigma)=\\infty$, then the same averaging operator as in the proof of Schur orthogonality would produce an intertwiner $A:\\mathcal{H}_\\sigma\\to \\mathcal{H}_\\pi$ for each finite-dimensional irreducible $\\pi$. Any such nonzero $A$ would be injective (its kernel is a closed $\\sigma$-invariant subspace), which is impossible into a finite-dimensional space. Hence $A=0$, so every matrix coefficient of $\\sigma$ is orthogonal to every element of $E_{\\mathrm{fin}}$. By density, all coefficients of $\\sigma$ would have to be $0$, contradiction.\nNow fix a finite-dimensional irreducible $\\pi$ and let $H_\\pi\\subseteq L^2(G)$ be the closed span of its matrix coefficients (equivalently, the span of $\\{\\varphi_{ij}^\\pi\\}_{i,j}$).\nObserve the standard coefficient identity: for $g,t\\in G$, $$ \\varphi_{ij}^\\pi(g^{-1}t) = \\langle e_i^\\pi,\\pi(g^{-1}t)e_j^\\pi\\rangle = \\sum_{k=1}^{d_\\pi} \\varphi_{ik}^\\pi(g^{-1})\\,\\varphi_{kj}^\\pi(t). $$ For each fixed $j$, define a linear map $A_j:\\mathcal{H}_\\pi\\to H_\\pi$ by $$ A_j(e_i^\\pi)=\\varphi_{ij}^\\pi. $$ Using the identity above, one checks that $A_j$ intertwines $\\pi$ with the left regular representation $\\lambda$ restricted to $H_\\pi$: $$ \\lambda(g)\\,A_j = A_j\\,\\pi(g). $$ By Schur orthogonality we have, for fixed $j$, $$ \\left\\langle \\sqrt{d_\\pi}\\,\\varphi_{ij}^\\pi,\\ \\sqrt{d_\\pi}\\,\\varphi_{kj}^\\pi\\right\\rangle_{L^2} = \\delta_{ik}, $$ so $\\widetilde A_j:=\\sqrt{d_\\pi}\\,A_j$ is an isometry $\\mathcal{H}_\\pi\\to L^2(G)$ and its range $$ H_{\\pi,j}:=\\mathrm{span}\\{\\varphi_{ij}^\\pi:1\\le i\\le d_\\pi\\} $$ is closed. Moreover, for $j\\neq l$ the subspaces $H_{\\pi,j}$ and $H_{\\pi,l}$ are orthogonal. Hence $$ H_\\pi=\\bigoplus_{j=1}^{d_\\pi} H_{\\pi,j} $$ as an orthogonal direct sum of $\\lambda$-invariant subspaces.\nSince each $\\widetilde A_j$ is a nonzero intertwiner from the irreducible representation $\\pi$ into $\\lambda|_{H_{\\pi,j}}$, Lemma 2 yields $\\lambda|_{H_{\\pi,j}}\\cong \\pi$. Therefore $$ \\lambda|_{H_\\pi}\\cong \\bigoplus_{j=1}^{d_\\pi} \\pi \\;\\cong\\; d_\\pi\\cdot \\pi. $$ Summing over $[\\pi]\\in\\widehat G$ yields item (2): $$ \\lambda \\cong \\bigoplus_{[\\pi]\\in\\widehat G} d_\\pi\\cdot\\pi. $$ At this point we know $\\{\\sqrt{d_\\pi}\\,\\varphi_{ij}^\\pi\\}$ is an orthonormal basis of $L^2(G)$, so every $f\\in L^2(G)$ has an expansion $$ f=\\sum_{[\\pi]\\in\\widehat G}\\ \\sum_{i,j=1}^{d_\\pi} c_{ij}^\\pi\\,\\sqrt{d_\\pi}\\,\\varphi_{ij}^\\pi $$ with $$ \\|f\\|_2^2=\\sum_{[\\pi]\\in\\widehat G}\\ \\sum_{i,j=1}^{d_\\pi} |c_{ij}^\\pi|^2. $$ Moreover, by definition of Fourier coefficients in an orthonormal basis, $$ \\begin{aligned} c_{ij}^\\pi \u0026amp;= \\left\\langle f,\\ \\sqrt{d_\\pi}\\,\\varphi_{ij}^\\pi\\right\\rangle_{L^2} = \\int_G f(g)\\,\\overline{\\sqrt{d_\\pi}\\,\\varphi_{ij}^\\pi(g)}\\,dg \\\\\u0026amp;= \\sqrt{d_\\pi}\\int_G f(g)\\,\\langle e_j^\\pi,\\pi(g^{-1})e_i^\\pi\\rangle\\,dg = \\sqrt{d_\\pi}\\,\\langle e_j^\\pi,\\widehat f(\\pi)e_i^\\pi\\rangle. \\end{aligned} $$ Therefore $$ \\begin{aligned} \\|f\\|_2^2 \u0026amp;= \\sum_{[\\pi]\\in\\widehat G} d_\\pi\\sum_{i,j=1}^{d_\\pi} |\\langle e_j^\\pi,\\widehat f(\\pi)e_i^\\pi\\rangle|^2 \\\\ \u0026amp;= \\sum_{[\\pi]\\in\\widehat G} d_\\pi\\,\\|\\widehat f(\\pi)\\|_{\\mathrm{HS}}^2, \\end{aligned} $$ which is item (5).\nItem (3) follows from item (4): if $g\\neq e$ and $\\pi(g)=I$ for every irreducible $\\pi$, then every matrix coefficient (hence every $f\\in E$) would satisfy $f(g)=f(e)$, so $E$ could not be dense in $C(G)$.\nThus it only remains to prove:\nClaim. $E_{\\mathrm{fin}}$ is dense in $C(G)$.\nWe prove this using Hilbert\u0026ndash;Schmidt operators and an approximate identity.\n2.7. Hilbert\u0026ndash;Schmidt operators and convolution Let $(X,\\nu)$ be a measure space and let $K\\in L^2(X\\times X,\\nu\\otimes\\nu)$. Define an operator $T:L^2(X)\\to L^2(X)$ by $$ Tf(x)=\\int_X K(x,y)f(y)\\,d\\nu(y). $$ Such operators are called Hilbert\u0026ndash;Schmidt operators; they are compact, and if $K(x,y)=\\overline{K(y,x)}$, then $T$ is self-adjoint. In particular, each eigenspace $$ H_\\alpha=\\{f\\in L^2(X): Tf=\\alpha f\\} $$ is finite-dimensional.\nIn our case we take $X=G$ with normalized Haar measure and $K$ continuous. Then $x\\mapsto K(x,\\cdot)$ is continuous as a map into $L^2(G)$, and therefore $Tf(x)=\\langle f, K(x,\\cdot)\\rangle_{L^2}$ defines a continuous function for every $f\\in L^2(G)$.\nNow fix $k\\in C(G)$ satisfying $$ k(g)=\\overline{k(g^{-1})},\\qquad \\forall g\\in G. $$ Define the right-convolution operator $T:L^2(G)\\to L^2(G)$ by $$ Tf := f*k, \\qquad Tf(g)=\\int_G f(r)\\,k(r^{-1}g)\\,dr. $$ Equivalently, $T$ is Hilbert\u0026ndash;Schmidt with kernel $K(g,r)=k(r^{-1}g)$, and the symmetry assumption implies $K(g,r)=\\overline{K(r,g)}$, so $T$ is self-adjoint and compact.\nLemma. Let $k$ and $T$ be as above. For each $\\alpha\\in\\mathbb{R}\\setminus\\{0\\}$, the eigenspace $$ H_\\alpha=\\{f\\in L^2(G): Tf=\\alpha f\\} $$ is contained in $E_{\\mathrm{fin}}$.\nProof. Since $T$ is compact and self-adjoint, $H_\\alpha$ is finite-dimensional. Moreover, $H_\\alpha\\subseteq C(G)$: if $Tf=\\alpha f$ with $\\alpha\\neq 0$, then $f=\\alpha^{-1}Tf$ and $Tf$ is continuous by the discussion above.\nThe space $H_\\alpha$ is invariant under the left regular representation. Indeed, for $g\\in G$, $$ T(\\lambda(g)f) = (\\lambda(g)f)*k = \\lambda(g)(f*k) = \\lambda(g)(Tf), $$ so if $Tf=\\alpha f$, then $T(\\lambda(g)f)=\\alpha\\,\\lambda(g)f$ and $\\lambda(g)f\\in H_\\alpha$.\nLet $\\{f_1,\\dots,f_r\\}$ be an orthonormal basis of $H_\\alpha$. Define continuous functions $$ \\theta_{ij}(g):=\\langle \\lambda(g)f_i,f_j\\rangle,\\qquad 1\\le i,j\\le r. $$ Since $\\lambda(g)f_i\\in H_\\alpha$, we can expand $$ (\\lambda(g)f_i)=\\sum_{k=1}^r \\theta_{ki}(g)\\,f_k. $$ Thus $g\\mapsto (\\theta_{ij}(g))_{i,j}$ is the matrix of the unitary operator $\\lambda(g)|_{H_\\alpha}$ in the basis $\\{f_i\\}$, so it defines a finite-dimensional unitary representation $\\rho$ of $G$ on $\\mathbb{C}^r$.\nFinally, evaluating at the identity element $e\\in G$ gives $$ f_i(g^{-1})=(\\lambda(g)f_i)(e)=\\sum_{k=1}^r \\theta_{ki}(g)\\,f_k(e), $$ so each $f_i$ is a linear combination of the coefficient functions $\\theta_{ki}(g^{-1})$. Since $E_{\\mathrm{fin}}$ is closed under inversion, we conclude $f_i\\in E_{\\mathrm{fin}}$. $\\square$\nLemma. Let $k$ and $T$ be as above. Then for every $f\\in L^2(G)$ we have $$ Tf\\in \\overline{E_{\\mathrm{fin}}}^{\\|\\cdot\\|_2}. $$ In particular, $Tf$ can be approximated in $L^2$ by elements of $E_{\\mathrm{fin}}$.\nProof. Let $\\{ \\phi_n \\}$ be a complete orthonormal set of eigenvectors for the nonzero eigenspaces of $T$, with eigenvalues $\\alpha_n\\ne 0$: $$ T\\phi_n=\\alpha_n\\phi_n.$$ Then by the spectral theorem for compact self-adjoint operators, any $f\\in L^2(G)$ can be written as $$ f=\\sum_{n} c_n\\phi_n \u0026#43; f_0, \\qquad Tf_0=0, $$ with $\\sum_n |c_n|^2\u0026lt;\\infty$. Hence $$ Tf=\\sum_n \\alpha_n c_n \\phi_n. $$ Approximating this series by partial sums yields $L^2$-convergence, and each $\\phi_n\\in E_{\\mathrm{fin}}$ by the previous lemma. Thus $Tf$ is in the $L^2$-closure of $E_{\\mathrm{fin}}$. $\\square$\nLemma (Smoothing upgrades to uniform approximation). Let $k\\in C(G)$ satisfy $k(g)=\\overline{k(g^{-1})}$ and let $T:L^2(G)\\to L^2(G)$ be right-convolution by $k$: $$ Tf=f*k. $$ Then for every $f\\in L^2(G)$ we have $$ Tf\\in \\overline{E_{\\mathrm{fin}}}^{\\|\\cdot\\|_\\infty}\\subseteq C(G). $$ Proof. Since $T$ is compact and self-adjoint, it has nonzero eigenvalues $\\alpha$ with finite-dimensional eigenspaces $$ H_\\alpha=\\{f\\in L^2(G): Tf=\\alpha f\\}, $$ and $L^2(G)=\\ker T \\oplus \\bigoplus_{\\alpha\\ne 0} H_\\alpha$ orthogonally. For $\\varepsilon\u0026gt;0$, define the finite-rank truncation by eigenspaces $$ T_\\varepsilon := \\sum_{|\\alpha|\\ge \\varepsilon} \\alpha\\,P_\\alpha, $$ where the sum ranges over nonzero eigenvalues $\\alpha$ of $T$ and $P_\\alpha$ is the orthogonal projection onto the eigenspace $H_\\alpha=\\{f: Tf=\\alpha f\\}$. Since $\\alpha\\to 0$, the sum is finite, so $T_\\varepsilon$ has finite rank and $T_\\varepsilon f\\in E_{\\mathrm{fin}}$.\nMoreover, each $H_\\alpha$ is $\\lambda$-invariant, hence $P_\\alpha$ (and therefore $T_\\varepsilon$) commutes with $\\lambda$. Since $T_\\varepsilon$ has finite rank and maps into $C(G)$, it has a continuous kernel $K_\\varepsilon(g,r)$, and the commutation relation implies $$ K_\\varepsilon(tg,tr)=K_\\varepsilon(g,r),\\qquad \\forall t,g,r\\in G. $$ Taking $t=r^{-1}$ gives $K_\\varepsilon(g,r)=K_\\varepsilon(r^{-1}g,e)$. Defining $$ k_\\varepsilon(x):=K_\\varepsilon(x,e)\\in C(G) $$ we obtain $K_\\varepsilon(g,r)=k_\\varepsilon(r^{-1}g)$, i.e. $$ T_\\varepsilon f = f*k_\\varepsilon. $$ Now Cauchy\u0026ndash;Schwarz gives, for every $g\\in G$, $$ |(Tf-T_\\varepsilon f)(g)| =|(f*(k-k_\\varepsilon))(g)| \\le \\|f\\|_2\\,\\|k-k_\\varepsilon\\|_2, $$ so $$ \\|Tf-T_\\varepsilon f\\|_\\infty\\le \\|f\\|_2\\,\\|k-k_\\varepsilon\\|_2. $$ Finally, for the right-convolution operator $S_u(f)=f*u$ on a compact group one has $\\|S_u\\|_{\\mathrm{HS}}=\\|u\\|_2$ (compute from the kernel $u(r^{-1}g)$), so $$ \\|k-k_\\varepsilon\\|_2=\\|T-T_\\varepsilon\\|_{\\mathrm{HS}}. $$ By the spectral theorem for compact self-adjoint operators, $$ \\|T-T_\\varepsilon\\|_{\\mathrm{HS}}^2 =\\sum_{|\\alpha|\u0026lt;\\varepsilon} (\\dim H_\\alpha)\\,|\\alpha|^2 \\xrightarrow[\\varepsilon\\downarrow 0]{} 0, $$ so $\\|k-k_\\varepsilon\\|_2\\to 0$ and therefore $T_\\varepsilon f\\to Tf$ uniformly, with $T_\\varepsilon f\\in E_{\\mathrm{fin}}$. $\\square$\n2.8. Approximate identities and density We now use the existence of an approximate identity in $C(G)$.\nProposition (Approximate identity). If $G$ is compact, there exists a net $\\{k_U\\}\\subset C(G)$ indexed by neighborhoods $U$ of $e\\in G$ such that:\n$k_U\\ge 0$, $\\mathrm{supp}(k_U)\\subseteq U$, $\\displaystyle \\int_G k_U(g)\\,dg = 1$, $k_U(g)=k_U(g^{-1})$, for every $f\\in C(G)$, both $k_U*f\\to f$ and $f*k_U\\to f$ uniformly on $G$. Proof. (Existence.) Fix a neighborhood $U$ of $e$. Choose a symmetric neighborhood $V$ of $e$ with $\\overline V\\subset U$ (possible in a topological group). By Urysohn\u0026rsquo;s lemma, there exists $\\psi\\in C(G)$ with $0\\le \\psi\\le 1$, $\\psi\\equiv 1$ on $\\overline V$, and $\\mathrm{supp}(\\psi)\\subset U$. Then $h(g):=\\psi(g)\u0026#43;\\psi(g^{-1})$ is continuous, nonnegative, symmetric, supported in $U$, and $\\int_G h\u0026gt;0$. Setting $k_U:=h/\\int_G h$ gives (1)\u0026ndash;(3).\n(Approximation.) Fix $f\\in C(G)$ and $\\varepsilon\u0026gt;0$. By uniform continuity of $f$ (compactness of $G$), there exists a neighborhood $W$ of $e$ such that $$ |f(t^{-1}g)-f(g)|\u0026lt;\\varepsilon,\\qquad \\forall g\\in G,\\ \\forall t\\in W. $$ If $U\\subseteq W$, then for all $g\\in G$, $$ \\begin{aligned} |(k_U*f)(g)-f(g)| \u0026amp;= \\left|\\int_G k_U(t)\\bigl(f(t^{-1}g)-f(g)\\bigr)\\,dt\\right| \\\\ \u0026amp;\\le \\int_G k_U(t)\\,\\varepsilon\\,dt =\\varepsilon. \\end{aligned} $$ Thus $k_U*f\\to f$ uniformly. The proof for $f*k_U\\to f$ is the same, using uniform continuity of right translations. $\\square$\nNow let $T_U$ be the convolution operator $$ T_Uf := f*k_U. $$ Each $k_U$ satisfies the hypotheses of the Hilbert\u0026ndash;Schmidt lemmas above, and $T_Uf\\to f$ uniformly for every $f\\in C(G)$.\nBy the previous lemma, $T_Uf\\in \\overline{E_{\\mathrm{fin}}}^{\\|\\cdot\\|_\\infty}$ for every $f\\in L^2(G)$, hence in particular for $f\\in C(G)$. Since $T_Uf\\to f$ uniformly on $G$, we conclude $$ \\overline{E_{\\mathrm{fin}}}^{\\|\\cdot\\|_\\infty}=C(G). $$ This proves the claim, and therefore completes the proof of the Peter\u0026ndash;Weyl theorem. $\\blacksquare$\n3. Examples 3.1. The circle group SO(2) We now spell out the Peter\u0026ndash;Weyl theorem concretely for $G=\\mathrm{SO}(2)$, showing that it reduces to classical Fourier series on the circle.\nIdentify $\\mathrm{SO}(2)$ and its Haar measure Every element of $\\mathrm{SO}(2)$ is a rotation $$ R_\\theta= \\begin{pmatrix} \\cos\\theta \u0026amp; -\\sin\\theta \\\\ \\sin\\theta \u0026amp; \\cos\\theta \\end{pmatrix}, \\qquad \\theta\\in\\mathbb{R}. $$ Matrix multiplication corresponds to adding angles: $$ R_\\theta R_\\varphi = R_{\\theta\u0026#43;\\varphi}. $$ Hence $\\mathrm{SO}(2)\\cong \\mathbb{T}\\cong \\mathbb{R}/2\\pi\\mathbb{Z}$ (a compact abelian group).\nThe normalized Haar measure is $$ dg = \\frac{d\\theta}{2\\pi},\\qquad \\theta\\in[0,2\\pi), $$ so for $f\\in L^1(\\mathrm{SO}(2))$, $$ \\int_{\\mathrm{SO}(2)} f(g)\\,dg = \\frac{1}{2\\pi}\\int_0^{2\\pi} f(R_\\theta)\\,d\\theta. $$ Irreducible unitary representations are characters Since $G=\\mathrm{SO}(2)$ is compact abelian, every irreducible unitary representation is $1$-dimensional (a character). Concretely, they are indexed by $n\\in\\mathbb{Z}$: $$ \\pi_n:\\mathrm{SO}(2)\\to \\mathbb{C}^\\times, \\qquad \\pi_n(R_\\theta)=e^{in\\theta}. $$ Each $\\pi_n$ is unitary since $|e^{in\\theta}|=1$, and irreducibility is automatic in dimension $1$. Thus $$ \\widehat{\\mathrm{SO}(2)}\\cong \\mathbb{Z}, \\qquad [\\pi_n]\\longleftrightarrow n. $$ Matrix coefficients and the Peter\u0026ndash;Weyl space $E_{\\mathrm{fin}}$ For a representation $\\pi$, a matrix coefficient has the form $g\\mapsto \\langle \\pi(g)v,w\\rangle$.\nHere $\\pi_n$ is $1$-dimensional, so its coefficient functions are exactly the characters themselves: $$ R_\\theta \\longmapsto \\pi_n(R_\\theta)=e^{in\\theta}. $$ Therefore the Peter\u0026ndash;Weyl subspace generated by finite-dimensional irreps is the space of trigonometric polynomials: $$ E_{\\mathrm{fin}} = \\mathrm{span}\\{e^{in\\theta}:n\\in\\mathbb{Z}\\} = \\left\\{\\sum_{|n|\\le N} c_n e^{in\\theta}: N\\in\\mathbb{N},\\ c_n\\in\\mathbb{C}\\right\\}. $$ The left regular representation decomposes into Fourier modes Identifying $\\mathrm{SO}(2)\\cong [0,2\\pi)$ via $R_\\varphi$, the left regular representation acts by translation: $$ (\\lambda(R_\\theta)f)(\\varphi)=f(\\varphi-\\theta). $$ For the Fourier modes $e_n(\\varphi):=e^{in\\varphi}$, $$ (\\lambda(R_\\theta)e_n)(\\varphi) = e^{in(\\varphi-\\theta)} = e^{-in\\theta}\\,e^{in\\varphi}. $$ Thus each $\\mathbb{C}e^{in\\varphi}$ is $\\lambda$-invariant and carries the character $\\pi_{-n}$. Since $\\{e^{in\\varphi}\\}_{n\\in\\mathbb{Z}}$ is an orthonormal basis of $L^2([0,2\\pi),\\frac{d\\varphi}{2\\pi})$, we get $$ L^2(\\mathrm{SO}(2)) \\cong \\bigoplus_{n\\in\\mathbb{Z}} \\mathbb{C}e^{in\\varphi}, \\qquad \\lambda \\cong \\bigoplus_{n\\in\\mathbb{Z}} \\pi_n. $$ (Here the last display just reindexes $n\\mapsto -n$.)\nOrthogonality relations With the normalized Haar measure, the characters are orthonormal: $$ \\left\\langle e^{in\\theta},e^{im\\theta}\\right\\rangle_{L^2(\\mathrm{SO}(2))}= \\frac{1}{2\\pi}\\int_0^{2\\pi} e^{i(n-m)\\theta}\\,d\\theta = \\delta_{nm}. $$ This is Schur orthogonality in the abelian case.\nSeparation of points and density in $C(\\mathrm{SO}(2))$ If $R_\\theta\\neq R_0$, then $\\theta\\not\\equiv 0\\pmod{2\\pi}$ and $$ \\pi_1(R_\\theta)=e^{i\\theta}\\neq 1, $$ so $\\widehat G$ separates points (Peter\u0026ndash;Weyl item (3)).\nMoreover, Peter\u0026ndash;Weyl item (4) says that the span of matrix coefficients is dense in $C(G)$ in the sup norm. Here this becomes: $$ \\overline{E_{\\mathrm{fin}}}^{\\|\\cdot\\|_\\infty}=C(\\mathrm{SO}(2)), $$ i.e. trigonometric polynomials are uniformly dense in $C(\\mathbb{T})$ (e.g. via Fejér kernels / Cesàro means of Fourier series).\nFourier transform and Plancherel (Parseval) For $f\\in L^1(\\mathrm{SO}(2))$, the Peter\u0026ndash;Weyl Fourier transform at $\\pi_n$ is scalar-valued: $$ \\widehat f(\\pi_n) = \\int_{\\mathrm{SO}(2)} f(g)\\,\\pi_n(g^{-1})\\,dg. $$ Writing $g=R_\\theta$ gives $g^{-1}=R_{-\\theta}$ and $\\pi_n(g^{-1})=e^{-in\\theta}$, hence $$ \\widehat f(\\pi_n) = \\frac{1}{2\\pi}\\int_0^{2\\pi} f(\\theta)\\,e^{-in\\theta}\\,d\\theta, $$ which is the usual Fourier coefficient $\\widehat f(n)$.\nSince $d_{\\pi_n}=1$, Plancherel becomes the classical Parseval identity: $$ \\|f\\|_2^2 = \\sum_{n\\in\\mathbb{Z}} |\\widehat f(\\pi_n)|^2 = \\sum_{n\\in\\mathbb{Z}} |\\widehat f(n)|^2. $$ 3.2. The rotation group SO(3) We now spell out the Peter\u0026ndash;Weyl theorem concretely for $G=\\mathrm{SO}(3)$, where the Fourier basis is given by Wigner $D$-matrix coefficients.\nIdentify $\\mathrm{SO}(3)$ and its Haar measure Recall $$ \\mathrm{SO}(3)=\\{R\\in M_{3\\times 3}(\\mathbb{R}) : R^\\top R=I,\\ \\det R=1\\}, $$ a compact (non-abelian) Lie group.\nUsing Euler angles, every $g\\in \\mathrm{SO}(3)$ can be written (non-uniquely on a null set) as $$ g = R_z(\\alpha)\\,R_y(\\beta)\\,R_z(\\gamma), \\qquad (\\alpha,\\gamma)\\in[0,2\\pi),\\ \\beta\\in[0,\\pi]. $$ With Haar measure normalized by $\\mu(\\mathrm{SO}(3))=1$, one has $$ dg=\\frac{1}{8\\pi^2}\\,\\sin\\beta\\,d\\alpha\\,d\\beta\\,d\\gamma, $$ so for $f\\in L^1(\\mathrm{SO}(3))$, $$ \\int_{\\mathrm{SO}(3)} f(g)\\,dg = \\frac{1}{8\\pi^2}\\int_0^{2\\pi}\\int_0^\\pi\\int_0^{2\\pi} f\\bigl(R_z(\\alpha)R_y(\\beta)R_z(\\gamma)\\bigr)\\,\\sin\\beta\\,d\\alpha\\,d\\beta\\,d\\gamma. $$ Irreducible unitary representations The irreducible unitary representations of $\\mathrm{SO}(3)$ are indexed by $\\ell\\in\\mathbb{Z}_{\\ge 0}$. Denote a choice of representative by $$ \\pi_\\ell:\\mathrm{SO}(3)\\to \\mathfrak{U}(\\mathcal{H}_\\ell), \\qquad d_\\ell:=\\dim \\mathcal{H}_\\ell = 2\\ell\u0026#43;1. $$ Equivalently, $\\pi_\\ell$ may be realized as the natural action on spherical harmonics of degree $\\ell$. Thus $$ \\widehat{\\mathrm{SO}(3)}\\cong \\mathbb{Z}_{\\ge 0}, \\qquad [\\pi_\\ell]\\longleftrightarrow \\ell. $$ Matrix coefficients and the Peter\u0026ndash;Weyl space $E_{\\mathrm{fin}}$ Fix an orthonormal basis $\\{e_m^\\ell\\}_{m=-\\ell}^{\\ell}$ of $\\mathcal{H}_\\ell$. The matrix coefficients of $\\pi_\\ell$ are $$ D^{\\ell}_{mn}(g):=\\langle \\pi_\\ell(g)e_n^\\ell,e_m^\\ell\\rangle, \\qquad -\\ell\\le m,n\\le \\ell, $$ the (unitary) Wigner $D$-matrix elements. Therefore $$ E_{\\mathrm{fin}} = \\mathrm{span}\\bigl\\{D^{\\ell}_{mn}:\\ell\\in\\mathbb{Z}_{\\ge 0},\\ -\\ell\\le m,n\\le \\ell\\bigr\\}. $$ The left regular representation decomposes into $D$-matrix blocks For each $\\ell$, the span of $\\{D^\\ell_{mn}\\}_{m,n=-\\ell}^\\ell$ is a $\\lambda$-invariant subspace of $L^2(\\mathrm{SO}(3))$ isomorphic to $\\mathcal{H}_\\ell\\otimes \\mathcal{H}_\\ell^*$. In particular, each $\\pi_\\ell$ occurs in the left regular representation with multiplicity $d_\\ell$: $$ L^2(\\mathrm{SO}(3)) \\cong \\bigoplus_{\\ell=0}^\\infty \\mathcal{H}_\\ell\\otimes \\mathcal{H}_\\ell^*, \\qquad \\lambda \\cong \\bigoplus_{\\ell=0}^\\infty d_\\ell\\cdot \\pi_\\ell. $$ Orthogonality relations Schur orthogonality specializes to $$ \\int_{\\mathrm{SO}(3)} D^{\\ell}_{mn}(g)\\,\\overline{D^{\\ell\u0026#39;}_{m\u0026#39;n\u0026#39;}(g)}\\,dg = \\frac{1}{2\\ell\u0026#43;1}\\,\\delta_{\\ell\\ell\u0026#39;}\\,\\delta_{mm\u0026#39;}\\,\\delta_{nn\u0026#39;}. $$ Equivalently, $\\{\\sqrt{2\\ell\u0026#43;1}\\,D^\\ell_{mn}\\}$ is an orthonormal basis of $L^2(\\mathrm{SO}(3))$.\nSeparation of points and density in $C(\\mathrm{SO}(3))$ The standard representation $\\pi_1(g)=g$ on $\\mathbb{R}^3$ (and hence on $\\mathbb{C}^3$ by complexification) is the $\\ell=1$ irreducible representation. Its matrix coefficients are the coordinate functions $g\\mapsto g_{ij}$, which clearly separate points: if $g\\neq h$ then $g_{ij}\\neq h_{ij}$ for some $i,j$. Moreover, $$ \\overline{E_{\\mathrm{fin}}}^{\\|\\cdot\\|_\\infty}=C(\\mathrm{SO}(3)). $$ Fourier transform and Plancherel (Parseval) For $f\\in L^1(\\mathrm{SO}(3))$, the Peter\u0026ndash;Weyl Fourier transform at $\\pi_\\ell$ is matrix-valued: $$ \\widehat f(\\pi_\\ell) := \\int_{\\mathrm{SO}(3)} f(g)\\,\\pi_\\ell(g^{-1})\\,dg \\in \\mathbb{C}^{(2\\ell\u0026#43;1)\\times (2\\ell\u0026#43;1)}. $$ In the basis above, $$ \\bigl(\\widehat f(\\pi_\\ell)\\bigr)_{mn} = \\int_{\\mathrm{SO}(3)} f(g)\\,D^{\\ell}_{mn}(g^{-1})\\,dg = \\int_{\\mathrm{SO}(3)} f(g)\\,\\overline{D^{\\ell}_{nm}(g)}\\,dg. $$ Plancherel becomes $$ \\|f\\|_2^2 = \\sum_{\\ell=0}^\\infty (2\\ell\u0026#43;1)\\,\\|\\widehat f(\\pi_\\ell)\\|_{\\mathrm{HS}}^2. $$ 4. Group Convolutional Neural Networks We are interested in studying learnable functions that satisfy certain symmetries. In particular, we want to study functions that are equivariant with respect to the action of a compact group $G$.\nDefinition (Equivariant function). Let $G$ be a compact group acting on functions $f\\in L^2(G)$ by left translation: $$(\\lambda(g)f)(h) = f(g^{-1}h), \\quad g,h \\in G.$$ A linear operator $F$ is $G$-equivariant if $$ F(\\lambda(g) f) = \\lambda(g) (F f) \\qquad \\forall g \\in G, f \\in L^2(G). $$ Definition (Group convolution). Define the (right) convolution with a kernel $L$ by $$ (Ff)(h) = (f \\ast L)(h) = \\int_G f(hg^{-1}) L(g) \\, d\\mu(g). $$ Equivariant operators are exactly right-convolutions with suitable kernels.\n4.1 Example SO(2) In the example $G = \\mathrm{SO}(2)$, using a change of variables $d\\mu = \\frac{d\\theta}{2\\pi}$. The formula becomes $$(Ff)(\\theta) = \\frac{1}{2\\pi}\\int_0^{2\\pi} f(\\theta - \\phi) L(\\phi) \\, d\\phi,$$ which is the classical convolution on the circle.\nFourier transform of group convolutions However, the Peter-Weyl theorem allows us to change the viewpoint and improve the efficiency of computations. Indeed, $$ \\widehat{Ff}(\\pi)=\\widehat{f*L}(\\pi)=\\widehat L(\\pi)\\,\\widehat f(\\pi), $$ where $\\widehat{L}(\\pi)$ is a $d_\\pi \\times d_\\pi$ matrix. In the abelian case $d_\\pi=1$ so this is scalar multiplication, but for non-abelian groups the matrix product order matters.\nExample using $G = \\mathrm{SO}(2)$, we have $d_{\\pi_n} = 1$ for all $n \\in \\mathbb{Z}$:\nimport numpy as np N = 64 theta = np.linspace(0, 2*np.pi, N, endpoint=False) # Discretize SO(2) ~ [0, 2π) dtheta = 2 * np.pi / N f = np.sin(3*theta) + 0.5*np.cos(5*theta) # Example signal f(θ) on the circle L = np.exp(-0.5 * (((theta + np.pi) % (2*np.pi) - np.pi) / 0.5)**2) # Method A: Direct discretized integral (O(N^2)) # (Ff)(θ_k) ≈ (1/(2π)) Σ_{j=0}^{N-1} f(θ_k - θ_j) L(θ_j) dθ conv_direct = np.empty(N) j = np.arange(N) for k in range(N): idx = (k - j) % N # f(θ_k - θ_j) corresponds to f[(k-j) mod N] on the grid conv_direct[k] = (1/(2*np.pi)) * np.sum(f[idx] * L) * dtheta # Method B: Fourier / Peter–Weyl (FFT) (O(N log N)) # ifft(fft(f) * fft(L))[k] = Σ f[k-j] L[j] # We still multiply by 1/(2π) to match the integral\u0026#39;s factor. f_hat = np.fft.fft(f) L_hat = np.fft.fft(L) conv_fft = np.fft.ifft(f_hat * L_hat).real * dtheta / (2*np.pi) # Check agreement max_err = np.max(np.abs(conv_direct - conv_fft)) print(\u0026#34;Max |direct - fft| =\u0026#34;, max_err) 4.2. Example SO(3) Let $G=\\mathrm{SO}(3)$ with normalized Haar measure $$ dg=\\frac{1}{8\\pi^2}\\,\\sin\\beta\\,d\\alpha\\,d\\beta\\,d\\gamma $$ in $ZYZ$ Euler angles $g=R_z(\\alpha)R_y(\\beta)R_z(\\gamma)$.\nFor $\\ell\\in\\mathbb{Z}_{\\ge 0}$ let $D^\\ell(g)\\in\\mathbb{C}^{(2\\ell\u0026#43;1)\\times(2\\ell\u0026#43;1)}$ denote the Wigner $D$-matrix of the irreducible representation $\\pi_\\ell$, with entries $$ D^\\ell_{mn}(\\alpha,\\beta,\\gamma)=e^{-im\\alpha}\\,d^\\ell_{mn}(\\beta)\\,e^{-in\\gamma}, \\qquad -\\ell\\le m,n\\le \\ell, $$ where $d^\\ell(\\beta)$ is the (real) small Wigner $d$-matrix.\nWith the Peter\u0026ndash;Weyl Fourier transform convention $$ \\widehat f(\\pi_\\ell)=\\int_{\\mathrm{SO}(3)} f(g)\\,D^\\ell(g)^*\\,dg\\in\\mathbb{C}^{(2\\ell\u0026#43;1)\\times(2\\ell\u0026#43;1)}, $$ the (right) group convolution $$ (f*L)(g)=\\int_{\\mathrm{SO}(3)} f(gh^{-1})\\,L(h)\\,dh $$ satisfies the convolution theorem $$ \\widehat{f*L}(\\pi_\\ell)=\\widehat L(\\pi_\\ell)\\,\\widehat f(\\pi_\\ell), $$ which is a matrix product (and is not commutative in general).\nNumerically, one can discretize $(\\alpha,\\beta,\\gamma)$ on a tensor grid, approximate integrals using quadrature weights proportional to $\\sin\\beta$, and truncate the inverse transform to $\\ell\\le L_{\\max}$: $$ f(g)\\approx \\sum_{\\ell=0}^{L_{\\max}} (2\\ell\u0026#43;1)\\,\\mathrm{tr}\\!\\left(\\widehat f(\\pi_\\ell)\\,D^\\ell(g)\\right). $$ In code, the spectral convolution step is literally matrix multiplication at each frequency $\\ell$:\nalpha, beta, gamma, weights, rotations = make_euler_zyz_grid(N_ALPHA, N_BETA, N_GAMMA) # Helper functions. f = create_signal(rotations) L = normalize_kernel(create_kernel(rotations), weights) D_grid = precompute_wigner_D_grid(alpha, beta, gamma, L_MAX) f_hat = fourier_coefficients_from_samples(f, D_grid, weights) L_hat = fourier_coefficients_from_samples(L, D_grid, weights) conv_hat = {ell: L_hat[ell] @ f_hat[ell] for ell in f_hat} conv = inverse_fourier_on_grid(conv_hat, D_grid, L_MAX) References [-1] The code is available at: https://github.com/dani2442/dani2442_code/tree/main/group-cnn\n[0] Dana P. Williams, The Peter\u0026ndash;Weyl Theorem for Compact Groups, notes. https://math.dartmouth.edu/~dana/bookspapers/pw.pdf\n[1] Elias M. Stein, Topics in Harmonic Analysis Related to the Littlewood\u0026ndash;Paley Theory, Annals of Mathematics Studies, Princeton University Press.\n[2] Walter Rudin, Fourier Analysis on Groups, Wiley Classics Library, John Wiley \u0026amp; Sons.\n[3] Gerald B. Folland, A Course in Abstract Harmonic Analysis, Studies in Advanced Mathematics, CRC Press.\n","permalink":"https://dani2442.github.io/posts/peter-weyl/","summary":"\u003cp\u003e\u003cimg loading=\"lazy\" src=\"/posts/peter-weyl/so3_equivariance.gif\"\u003e\u003c/p\u003e\n\u003cp\u003eThe Peter-Weyl theorem is one of the most beautiful results in Harmonic Analysis, and it has found surprisingly wide applications in machine learning (drug discovery, molecular dynamics, robotics, particle physics, graphs, etc).\u003c/p\u003e\n\u003cp\u003eIn many real-world problems, data is scarce, complexity is high, or we simply want our models to generalize better and produce more robust solutions. In these cases, it makes sense to leverage any prior knowledge we have about the problem structure. For example, in meteorology the data lives on a sphere (the Earth), or we might know that the underlying dynamics roughly satisfy the Navier-Stokes equations. The desire to incorporate this kind of domain-specific knowledge into machine learning models gave rise to what we now call Physics-Informed ML and Geometry-Informed ML.\nThe former led to Neural ODEs and Physics-Informed Neural Networks (PINNs), but in this post we focus on the latter.\u003c/p\u003e","title":"Harmonic Analysis: Peter-Weyl Theorem and Machine Learning"},{"content":" What do books, art, music, video games, and mathematics have in common?\nI remember the last time I went to the cinema and was genuinely impressed by a movie’s visual effects: Avatar (2009). Since then, my impression is that we’ve been living through diminishing returns: more polish, higher resolution, better lighting, but fewer moments that feel like a new era. The same seems true in video games: Red Dead Redemption 2 (2018) felt like a leap, and many people are hoping GTA VI will be the next one. But few expect another order-of-magnitude jump anytime soon.\nThe answer is that none of these things can be used up. A painting doesn’t stop being a painting because a new painting is made. A theorem doesn’t lose its truth because a new theorem is proved. In many cases, the opposite happens: works become more valuable with time (classic literature and art are the obvious examples).\nThis has a bleak implication for the industries built around them: the effective supply of things worth consuming grows monotonically, but demand is capped by attention, time, and (ultimately) population. The library keeps expanding; a day remains 24 hours.\nNew creators have the Herculean task of competing with the entire history of human creativity. AI may postpone this problem for a while by increasing diversity and lowering costs, but it also accelerates the rate at which the library expands. Either way, the underlying dynamic remains: more of what we can consume, not more time to consume it.\nYou may argue that I am being too pessimistic, so let\u0026rsquo;s consider some counterarguments:\nDiscovery is limited: Most people can’t access the “entire history” meaningfully. But recommendation systems, search, and social graphs keep getting better at surfacing relevant work (sometimes favoring the already-popular, but often making the long tail reachable). Novelty is local: Even if “true novelty” is rare, novelty relative to a person’s taste, culture, and moment is abundant. A work doesn’t need to be historically unprecedented to feel new. Social coordination matters: Releases are events. Part of the value of a new album, game, or paper is that other people are experiencing it now, so there’s something to share, discuss, and build on together. Experience is more than the artifact: Concerts, live streams, mods, communities, and “how it was made” content add layers that don’t exist in a static archive. Curation becomes the product: In an oversupplied world, selection, sequencing, and trust (editors, critics, curators, friends) become as economically important as production. If this diagnosis is right, the scarce resources aren’t ideas or pixels, they’re attention and trust. The “industry” doesn’t die so much as it shifts: away from pure production and toward distribution, community, and curation.\n","permalink":"https://dani2442.github.io/essays/oversupply/death_by_oversupply/","summary":"\u003cblockquote\u003e\n\u003cp\u003eWhat do \u003cem\u003ebooks, art, music, video games,\u003c/em\u003e and \u003cem\u003emathematics\u003c/em\u003e have in common?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI remember the last time I went to the cinema and was genuinely impressed by a movie’s visual effects: \u003cem\u003eAvatar\u003c/em\u003e (2009). Since then, my impression is that we’ve been living through diminishing returns: more polish, higher resolution, better lighting, but fewer moments that feel like a new era. The same seems true in video games: \u003cem\u003eRed Dead Redemption 2\u003c/em\u003e (2018) felt like a leap, and many people are hoping \u003cem\u003eGTA VI\u003c/em\u003e will be the next one. But few expect another order-of-magnitude jump anytime soon.\u003c/p\u003e","title":"Industries destined to collapse"},{"content":"How do we know if a machine learning model will perform well on unseen data? What happens if you continue to add samples to the dataset? Is it better to have a more complex model or a simpler one?\nThese questions have been around for many years and are central to the field of statistical learning theory. Generalization theory provides mathematical guarantees and bounds on the generalization capability of families of functions. I have prepared a few notes on the basics of generalization theory. The only prerequisite is probability theory, and it is intended to be self-contained. It includes the most important results, such as Dudley\u0026rsquo;s Theorem and McDiarmid\u0026rsquo;s Theorem.\nLink to pdf\n","permalink":"https://dani2442.github.io/posts/generalization/","summary":"\u003cp\u003eHow do we know if a machine learning model will perform well on unseen data? What happens if you continue to add samples to the dataset? Is it better to have a more complex model or a simpler one?\u003c/p\u003e\n\u003cp\u003eThese questions have been around for many years and are central to the field of statistical learning theory. Generalization theory provides mathematical guarantees and bounds on the generalization capability of families of functions. I have prepared a few notes on the basics of generalization theory. The only prerequisite is probability theory, and it is intended to be self-contained. It includes the most important results, such as Dudley\u0026rsquo;s Theorem and McDiarmid\u0026rsquo;s Theorem.\u003c/p\u003e","title":"Notes on Generalization Theory"},{"content":"Following the trend of neobanks, I think it is only a matter of time before \u0026ldquo;neo hospitals\u0026rdquo; start spreading. The way things are run right now feels so wrong: slow, inefficient and expensive. Especially, for the ones that can\u0026rsquo;t afford quality private healthcare.\nThere\u0026rsquo;s evident demand for better (or different) healthcare services. For instance, almost 40% of Americans trust chatbots for medical advice [2], and ChatGPT\u0026rsquo;s report [1] even mention the category \u0026ldquo;Healthcare\u0026rdquo;. And recently, GPT5 outperformed licensed human expersts by 25-30% on the US medical licensing exam [3]. And this is without mentioning the future problem of doctor shortages due to aging populations.\nAll this progress has happened despite patient data being almost impossible to use. The field of \u0026ldquo;medicine+ai\u0026rdquo; feels capped and constraint by the current state of infrastructure, regulations, data privacy and governance, etc. The room for improvement is massive.\nSimilarly, I expect the legal system to be disrupted as well [4], but that\u0026rsquo;s a topic for another day.\nReferences [1] https://cdn.openai.com/pdf/a253471f-8260-40c6-a2cc-aa93fe9f142e/economic-research-chatgpt-usage-paper.pdf\n[2] https://www.rollingstone.com/culture/culture-features/ai-chatbot-medical-advice-study-1235399973/\n[3] Wang, Shansong, Mingzhe Hu, Qiang Li, Mojtaba Safari, and Xiaofeng Yang. \u0026ldquo;Capabilities of gpt-5 on multimodal medical reasoning.\u0026rdquo; arXiv preprint arXiv:2508.08224 (2025).\n[4] https://researchprofiles.ku.dk/en/publications/artificial-intelligence-and-legal-disruption-a-new-model-for-anal/\n","permalink":"https://dani2442.github.io/essays/neo-hospital/","summary":"\u003cp\u003eFollowing the trend of neobanks, I think it is only a matter of time before \u0026ldquo;neo hospitals\u0026rdquo; start spreading. The way things are run right now feels so wrong: slow, inefficient and expensive. Especially, for the ones that can\u0026rsquo;t afford quality private healthcare.\u003c/p\u003e\n\u003cp\u003eThere\u0026rsquo;s evident demand for better (or different) healthcare services. For instance, almost 40% of Americans trust chatbots for medical advice \u003ca href=\"#references\"\u003e[2]\u003c/a\u003e, and ChatGPT\u0026rsquo;s report \u003ca href=\"#references\"\u003e[1]\u003c/a\u003e even mention the category \u0026ldquo;Healthcare\u0026rdquo;. And recently, GPT5 outperformed licensed human expersts by 25-30% on the US medical licensing exam \u003ca href=\"#references\"\u003e[3]\u003c/a\u003e. And this is without mentioning the future problem of doctor shortages due to aging populations.\u003c/p\u003e","title":"Neo Hospitals"},{"content":"Piracy is as old as trade itself. When Spain and Portugal began sending treasure fleets carrying gold, silver, and goods across the Atlantic in the 16th century, it didn\u0026rsquo;t take long for pirates to appear. You might recognize some of the most famous names: Francis Drake, Henry Morgan, Blackbeard, John Hawkins, and others.\nSome pirates resembled more like companies than the anarchic individuals you might imagine. English piracy often took a special form: privateering, where privately owned ships were authorized by the Crown (via letters of marque) to raid enemy shipping during wartime.\nMarek Rużyk (1965, Polish painter) - Unrest at Sea\nA new era of piracy How countries, companies, and individuals attack and disrupt each other has changed dramatically: it\u0026rsquo;s more digital and complex. But the underlying incentives remain the same: steal value from others to enrich themselves.\nIf the space industry continues to develop as expected, it\u0026rsquo;s only a matter of time before space piracy becomes a real concept. All these expensive satellites and data centers orbiting look very juicy and largely unprotected!\nWhat counts as space piracy?\nI\u0026rsquo;ll use space piracy as a broad umbrella: extracting value from a space asset without authorization by taking control of it, degrading it, or coercing its operator. Some specific examples include:\nService theft: using someone else\u0026rsquo;s satellite capacity (bandwidth, compute, sensing) without paying. Data theft: exfiltrating high-value data products (imagery, signals intelligence, proprietary processing outputs) by compromising the ground segment or the downlink pipeline. Ransom/denial: threatening to disable, deorbit, or simply make an asset useless unless paid. Signal hijacking: interfering with user links, spoofing signals, or impersonating a satellite to users on the ground. Physical capture or tampering: rendezvous and proximity operations, docking, attaching a device, or relocating the satellite. Orbit changes the constraints, not the incentives.\n","permalink":"https://dani2442.github.io/essays/space-piracy/","summary":"\u003cp\u003ePiracy is as old as trade itself. When Spain and Portugal began sending treasure fleets carrying gold, silver, and goods across the Atlantic in the 16th century, it didn\u0026rsquo;t take long for pirates to appear. You might recognize some of the most famous names: Francis Drake, Henry Morgan, Blackbeard, John Hawkins, and others.\u003c/p\u003e\n\u003cp\u003eSome pirates resembled more like companies than the anarchic individuals you might imagine.\nEnglish piracy often took a special form: \u003ccode\u003eprivateering\u003c/code\u003e, where privately owned ships were authorized by the Crown (via letters of marque) to raid enemy shipping during wartime.\u003c/p\u003e","title":"Space Piracy"},{"content":"We all know the slogan \u0026ldquo;knowledge is power\u0026rdquo;. In this essay, I want to argue for the opposite:\ntoo much knowledge can hinder creativity\nNot because learning is bad, but because what you already know quietly narrows what you consider plausible. In the age of AI, where solutions are increasingly cheap, this matters: the bottleneck shifts from \u0026ldquo;finding an answer\u0026rdquo; to \u0026ldquo;finding an original direction\u0026rdquo;.\nSo\u0026hellip; should we stay ignorant? Creativity needs ingredients: skills, taste, vocabulary, technique, references. But there\u0026rsquo;s a trade-off between exploration (searching broadly) and exploitation (reusing what already works) [2]. Expertise pulls you toward exploitation and AI makes exploitation nearly frictionless.\nThe failure mode looks like this:\nYou reach for the first \u0026ldquo;known-good\u0026rdquo; template. The template works well enough, so you stop searching. Over time, your outputs converge toward a local optimum: polished, competent, and predictable. This is related to several well-studied phenomena:\nCurse of knowledge: once you know something, it becomes hard to imagine not knowing it - changing how you explain, teach, and even think [3]. Einstellung effect: familiar methods block better ones [4]. Functional fixedness / design fixation: example solutions \u0026ldquo;anchor\u0026rdquo; your imagination and reduce novelty [5, 6]. A simple picture of the trap Assume the circles represent the \u0026ldquo;nearby\u0026rdquo; ideas you can easily access. Person B has more knowledge, but that also means more well-worn paths, so exploring outside the circles feels less cost-effective.\nEvidence (and what it actually implies) Richard Hamming was one of the first to point this out in \u0026ldquo;You and Your Research\u0026rdquo; [1]:\n- Question: How much effort should go into library work?\n- Hamming: It depends upon the field. I will say this about it. There was a fellow at Bell Labs, a very, very, smart guy. He was always in the library; he read everything. If you wanted references, you went to him and he gave you all kinds of references. But in the middle of forming these theories, I formed a proposition: there would be no effect named after him in the long run. He is now retired from Bell Labs and is an Adjunct Professor. He was very valuable; I\u0026rsquo;m not questioning that. He wrote some very good Physical Review articles; but there\u0026rsquo;s no effect named after him because he read too much. If you read all the time what other people have done you will think the way they thought. If you want to think new thoughts that are different, then do what a lot of creative people do - get the problem reasonably clear and then refuse to look at any answers until you\u0026rsquo;ve thought the problem through carefully how you would do it, how you could slightly change the problem to be the correct one. So yes, you need to keep up. You need to keep up more to find out what the problems are than to read to find the solutions. The reading is necessary to know what is going on and what is possible. But reading to get the solutions does not seem to be the way to do great research. So I\u0026rsquo;ll give you two answers. You read; but it is not the amount, it is the way you read that counts.\nAnother interesting extension of the well-known Arthur C. Clarke quote might be:\n\u0026ldquo;If an elderly but distinguished scientist LLM says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong.\u0026rdquo;\n— Arthur C. Clarke.\nCreators in other fields describe the same intuition in plain language:\n\u0026ldquo;Film is not the art of scholars but of illiterates.\u0026rdquo;\n—Werner Herzog\n\u0026ldquo;I want them [models or actors] to be intact, virgin. What I want from them is the unknown.\u0026rdquo;\n—Robert Bresson\nThese quotes can be read as an argument for protecting the unknown: leaving room for accidents, misreadings, and naive attempts that experts would reject too early.\nHow to use AI without losing originality AI is not the enemy; premature convergence is.\nResearch on human-AI ideation is still young, but it is already clear that how you use these tools matters [8, 9]. One practical workflow I have seen recommended is:\nWrite your \u0026ldquo;pre-AI\u0026rdquo; draft: your best explanation, outline, or sketch from memory. Force divergence: list 5-10 alternatives (inversions, constraints, different audiences, different mediums). Then query AI, but ask for breadth: counterarguments, edge cases, weird analogies, uncommon constraints. Incubate: step away, come back, and rewrite from your own perspective [7]. Use AI last for polish: clarity, structure, proofreading. References [1] Richard Hamming, \u0026ldquo;You and Your Research\u0026rdquo;. https://www.cs.virginia.edu/~robins/YouAndYourResearch.html\n[2] March, J. G. (1991). Exploration and exploitation in organizational learning. Organization Science, 2(1), 71-87. https://doi.org/10.1287/orsc.2.1.71\n[3] Camerer, C., Loewenstein, G., \u0026amp; Weber, M. (1989). The curse of knowledge in economic settings: An experimental analysis. Journal of Political Economy, 97(5), 1232-1254. https://doi.org/10.1086/261651\n[4] Luchins, A. S. (1942). Mechanization in problem solving: The effect of Einstellung. Psychological Monographs, 54(6), i-95. https://doi.org/10.1037/h0093502\n[5] Duncker, K. (1945). On problem-solving. Psychological Monographs, 58(5), i-113. https://doi.org/10.1037/h0093599\n[6] Jansson, D. G., \u0026amp; Smith, S. M. (1991). Design fixation. Design Studies, 12(1), 3-11. https://doi.org/10.1016/0142-694X(91)90003-F\n[7] Sio, U. N., \u0026amp; Ormerod, T. C. (2009). Does incubation enhance problem solving? A meta-analytic review. Psychological Bulletin, 135(1), 94-120. https://doi.org/10.1037/a0014212\n[8] Kim, H. K., Roknaldin, A., Nayak, S., Zhang, X., Yang, M., Twyman, M., Hwang, A. H.-C., \u0026amp; Lu, S. C.-Y. (2024). ChatGPT and Me: Collaborative Creativity in a Group Brainstorming with Generative AI. https://doi.org/10.18260/1-2--48457\n[9] \u0026ldquo;Augmented Brainstorming with AI\u0026rdquo; - Research Approach for Identifying Design Criteria for Improved Collaborative Idea Generation Between Humans and AI. (2023). https://doi.org/10.3233/FAIA230113\n","permalink":"https://dani2442.github.io/essays/creativity/","summary":"\u003cp\u003eWe all know the slogan \u0026ldquo;knowledge is power\u0026rdquo;. In this essay, I want to argue for the opposite:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003etoo much knowledge can hinder creativity\u003c/strong\u003e\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eNot because learning is bad, but because what you already know quietly narrows what you consider plausible. In the age of AI, where solutions are increasingly cheap, this matters: the bottleneck shifts from \u0026ldquo;finding an answer\u0026rdquo; to \u0026ldquo;finding an original direction\u0026rdquo;.\u003c/p\u003e\n\u003ch2 id=\"so-should-we-stay-ignorant\"\u003eSo\u0026hellip; should we stay ignorant?\u003c/h2\u003e\n\u003cp\u003eCreativity needs ingredients: skills, taste, vocabulary, technique, references. But there\u0026rsquo;s a trade-off between \u003cstrong\u003eexploration\u003c/strong\u003e (searching broadly) and \u003cstrong\u003eexploitation\u003c/strong\u003e (reusing what already works) \u003ca href=\"#references\"\u003e[2]\u003c/a\u003e. Expertise pulls you toward exploitation and AI makes exploitation nearly frictionless.\u003c/p\u003e","title":"Creativity in the Age of AI: The curse of knowledge"},{"content":" I think there are 3 major milestones remaining for humanity, and one of them is clean and abundant energy. Fusion energy has the potential to provide a nearly limitless source of clean energy by replicating the processes that power the sun. However, achieving controlled fusion reactions on Earth has proven to be a formidable and very challenging task.\nThe most well-known fusion prototype is the tokamak, which uses a magnetic field to confine the plasma within a toroidal chamber (doughnut-shaped).\nRecently, I saw that DeepMind has made significant strides in this area by using RL to optimize the control of tokamak reactors [3]. I read their paper, and I wanted to summarize what I learned: physical laws, how to derive the equations, and the numerical methods used to solve them.\nThe equations used to model the plasma are Diffusion/Heat PDEs of this type:\n$$ \\frac{\\partial u}{\\partial t} = \\nabla \\cdot (D \\nabla u) \u0026#43; \\text{Sources} $$ The interesting thing is that by using toroidal flux coordinates, the complex 3D geometry of the tokamak can be simplified into a 1D radial coordinate system.\nSource: [2]\nIntroduction We have 4 PDE equations and 5 variables in normalized toroidal flux coordinates, $\\hat \\rho$, with $0 \\leq \\hat \\rho \\leq 1$. The important quantities being modeled are:\n${\\color{red}{T_i}}, {\\color{orange}{T_e}}$: Ion and Electron Temperatures (Energy per particle). High temperature is required for fusion. ${\\color{brown}{n_i}}, {\\color{blue}{n_e}}$: Ion and Electron Densities (Particles per cubic meter). However, we do not model ion density explicitly, it is computed from electron density assuming quasi-neutrality. ${\\color{teal}{\\psi}}$: Poloidal Magnetic Flux. This describes the magnetic field structure and stability. Here we summarize the variables and parameters used in the equations:\nSymbol Description Units / Notes $ {\\color{red}{T_i}}, {\\color{orange}{T_e}} $ Ion and electron temperatures keV or eV $ {\\color{brown}{n_i}}, {\\color{blue}{n_e}} $ Ion and electron densities m⁻³ $ {\\color{teal}{\\psi}} $ Poloidal magnetic flux Wb (Weber) $ \\Phi(\\psi) $ Toroidal magnetic flux enclosed by the magnetic poloidal flux surface Wb $ \\Phi_b $ Toroidal flux enclosed by the plasma boundary (last-closed-flux surface) Wb $ \\hat{\\rho}= \\sqrt{\\tfrac{\\Phi({\\color{teal}{\\psi}})}{\\Phi_b}} $ Normalized toroidal flux coordinate Dimensionless $ V\u0026#39;\\equiv\\tfrac{dV}{d\\hat \\rho} $ Derivative of plasma volume w.r.t. $ \\hat{\\rho} $, i.e. flux surface volume derivative m³ $ g_0 = \\langle \\nabla V \\rangle $ Flux-surface-averaged gradient of the plasma volume m² $ g_1 = \\langle (\\nabla V)^2 \\rangle $ Flux-surface-averaged square of the volume gradient m⁴ $ g_2 = \\left\\langle \\frac{(\\nabla V)^2}{R^2} \\right\\rangle $ Flux-surface-averaged geometric factor involving major radius m² $ g_3 = \\left\\langle \\frac{1}{R^2} \\right\\rangle $ Flux-surface-averaged reciprocal of squared major radius 1/m² $ R $ Major radius along the flux surface m $ B_\\varphi $ Toroidal magnetic field T (Tesla) $ \\mathbf{B} $ Total magnetic field T (Tesla) $ \\sigma_\\parallel $ Plasma neoclassical conductivity S/m $ \\mu_0 $ Magnetic permeability of free space $4\\pi \\times 10^{-7}$ H/m $ F = R B_\\varphi $ Toroidal field flux function T·m $ \\chi_i, \\chi_e $ Ion and electron heat conductivities m²/s $ q_{0i}^{conv}, q_{0e}^{conv} $ Ion/electron heat convective losses W/m³ $ D_e $ Electron particle diffusivity m²/s $ V_e $ Electron particle convection velocity m/s $ Q_i, Q_e $ Ion and electron heat sources W/m³ $ S_n $ Total electron particle source 1/s or m⁻³·s⁻¹ $ \\mathbf{j}_{ni} $ Non-inductive current density (bootstrap + external drive) A/m² $ I_p $ Total plasma current A $ J $ Jacobian determinant or related geometric factor (see text) — $ R_0 $ Major radius at magnetic axis m $ \\langle \\cdot \\rangle $ Flux-surface average — Boundary Conditions: The boundary conditions are as follows. All equations have a zero-derivative boundary condition at $\\hat{\\rho} = 0$ (Von-Neumann). The $T_i$, $T_e$, $n_e$ equations have fixed boundary conditions at $\\hat{\\rho} = 1$. The $\\psi$ equation has a Neumann (derivative) boundary condition at $\\hat{\\rho} = 1$, which sets the total plasma current through the relation:\n$$ I_p = \\left[ \\frac{\\partial \\psi}{\\partial \\rho} \\frac{g_2 g_3}{\\rho} \\frac{R_0 J}{16 \\pi^4 \\mu_0} \\right]_{LCFS} $$ where $LCFS$ stands for Last Closed Flux Surface (i.e. $\\hat{\\rho} = 1$). The equations are derived from the transport equations in toroidal geometry under the assumption of static background flux surfaces [2].\n1.1 Electron Particle Transport Equation For an arbitrary plasma species $\\alpha$, which may refer to electrons, main or impurity ion species, or fusion-born $\\alpha$ particles, let $n_\\alpha$ be the local particle density of the species and $\\mathbf{u}_\\alpha$ be the local velocity of the particles. The continuity equation for this species is stated as $$ \\frac{\\partial n_\\alpha}{\\partial t} \u0026#43; \\nabla \\cdot (n_\\alpha \\mathbf{u}_\\alpha) = s_\\alpha $$ Here $s_\\alpha$ is a localized particle source. By writing the above equation inside a toroidal flux surface we obtain $$ \\left(\\frac{\\partial}{\\partial t} \u0026#43; \\frac{\\dot{B}_0}{2B_0}\\rho\\frac{\\partial}{\\partial\\rho}\\right) (\\langle n_\\alpha\\rangle V\u0026#39;) = -\\frac{\\partial\\Gamma_\\alpha}{\\partial\\rho} \u0026#43; V\u0026#39;S_\\alpha.\\tag{1} $$ A detailed derivation of the transport equations is provided in the Appendix.\nWe denote $\\alpha=e$ (by the electron species) and denoting $\\langle n_e \\rangle \\equiv n_e$. Using the specific form of the flux-surface averaged radial particle flux $$ \\Gamma_e=- D_e n_e \\frac{g_1}{V\u0026#39;}\\frac{\\partial n_e}{\\partial \\rho}\u0026#43; g_0 V_e n_e. $$ Then $$ -\\frac{\\partial \\Gamma_e}{\\partial \\rho}= \\frac{\\partial}{\\partial \\rho}\\Big[ D_e n_e\\frac{g_1}{V\u0026#39;}\\frac{\\partial n_e}{\\partial \\rho}- g_0 V_e n_e\\Big]. $$ Plugging (1), we get: $$ \\Big(\\frac{\\partial}{\\partial t}\u0026#43;\\frac{\\dot B_0}{2B_0}\\rho\\frac{\\partial}{\\partial \\rho}\\Big)(n_eV\u0026#39;) = \\frac{\\partial}{\\partial \\rho}\\Big[ D_e n_e\\frac{g_1}{V\u0026#39;}\\frac{\\partial n_e}{\\partial \\rho}- g_0 V_e n_e \\Big] \u0026#43;V\u0026#39;S_n . $$ Rewriting the grid-motion term using the toroidal flux $\\Phi_b$ and a normalized radius $\\hat\\rho$. Using $\\hat\\rho^2 = \\frac{\\Phi}{\\Phi_b(t)}$, so if $\\Phi_b$ changes in time, by a chain-rule term:\n$$ \\left.\\frac{\\partial}{\\partial t}\\right|_{\\hat\\rho}= \\left.\\frac{\\partial}{\\partial t}\\right|_{\\Phi} \u0026#43;\\frac{\\dot\\Phi_b}{2\\Phi_b}\\hat\\rho\\frac{\\partial}{\\partial \\hat\\rho}. $$ That is exactly the operator appearing in the final equation: $$ \\Big(\\frac{\\partial}{\\partial t}-\\frac{\\dot\\Phi_b}{2\\Phi_b}\\hat\\rho\\frac{\\partial}{\\partial \\hat\\rho}\\Big)(n_eV\u0026#39;). $$ Putting the pieces together gives: $$ \\boxed{\\; \\Big(\\frac{\\partial}{\\partial t}-\\frac{\\dot\\Phi_b}{2\\Phi_b}\\hat\\rho\\frac{\\partial}{\\partial \\hat\\rho}\\Big)(n_eV\u0026#39;) = \\frac{\\partial}{\\partial \\hat\\rho}\\Big[ D_e n_e \\frac{g_1}{V\u0026#39;} \\frac{\\partial n_e}{\\partial \\hat\\rho}-g_0 V_e n_e\\Big]\u0026#43;V\u0026#39;S_n.\\;} $$ 1.2 Current Diffusion Equation. In Appendix, we derived the 1D equation for the evolution of the poloidal flux $\\psi$ using Ohm\u0026rsquo;s law and Faraday\u0026rsquo;s law:\n$$ \\sigma_\\parallel \\left( \\frac{\\partial\\psi}{\\partial t} \u0026#43; \\rho\\frac{\\dot{B}_0}{2B_0}\\frac{\\partial\\psi}{\\partial\\rho} \\right)= \\frac{R_0 J^2}{\\mu_0 \\rho}\\frac{\\partial}{\\partial\\rho} \\left(\\frac{G^2}{J}\\frac{\\partial\\psi}{\\partial\\rho}\\right)- \\frac{V\u0026#39;}{2\\pi\\rho}(j_{bs} \u0026#43; j_{cd}) $$ Using the change of variables to $\\hat\\rho$ and $\\Phi_b$, we get, as before, that: $$ \\left(\\frac{\\partial \\psi}{\\partial t}\\right)_{\\Phi}= \\left(\\frac{\\partial \\psi}{\\partial t}\\right)_{\\hat\\rho}-\\frac{\\dot{\\hat\\Phi}_b}{2\\hat\\Phi_b}\\hat\\rho \\frac{\\partial \\psi}{\\partial \\hat\\rho}. $$ The diffusion term is $$ \\frac{R_0J^2}{\\mu_0\\rho}\\frac{\\partial}{\\partial\\rho} \\left(\\frac{G_2}{J}\\frac{\\partial\\psi}{\\partial\\rho}\\right), \\qquad J=\\frac{F}{R_0B_0},\\quad G_2=\\frac{V\u0026#39;}{4\\pi^2}\\Big\\langle\\frac{(\\nabla\\rho)^2}{R^2}\\Big\\rangle. $$ Transport-code forms typically package all those geometric combinations into something like $$ \\frac{\\partial}{\\partial\\hat\\rho}\\left(\\frac{g_2g_3}{\\hat\\rho}\\frac{\\partial\\psi}{\\partial\\hat\\rho}\\right), $$ where $g_2=\\langle (\\nabla V)^2/R^2\\rangle$ and $g_3=\\langle 1/R^2\\rangle$. Therefore we obtain the final form of the current diffusion equation $$ \\boxed{\\; \\frac{16 \\pi^2 \\sigma_{\\parallel}}{\\mu_0} \\frac{\\hat\\rho^2 \\dot{\\Phi}_b^2}{F^2} \\left( \\frac{\\partial \\psi}{\\partial t} - \\frac{\\hat\\rho \\dot{\\Phi}_b}{2 \\Phi_b} \\frac{\\partial \\psi}{\\partial \\hat\\rho} \\right) = \\frac{\\partial}{\\partial \\hat\\rho} \\left( \\frac{g_2 g_3}{\\hat\\rho} \\frac{\\partial \\psi}{\\partial \\hat\\rho} \\right) - \\frac{8 \\pi^2 V\u0026#39; \\mu_0 \\Phi_b}{F^2} \\langle \\mathbf{B} \\cdot \\mathbf{j}_{ni} \\rangle\\;} $$ 1.3 Ion and Electron Heat Transport Equations In a similar manner as the particle transport equation, an equation for energy transport can be derived [4, Eq. 7.46].\n$$ \\frac{3}{2}(V\u0026#39;)^{5/3} \\left(\\frac{\\partial}{\\partial t} - \\frac{\\dot B}{2B_0}\\frac{\\partial}{\\partial \\rho}\\rho\\right)\\left[(V\u0026#39;)^{-5/3} n_\\alpha T_\\alpha\\right] \u0026#43; \\frac{1}{V\u0026#39;}\\frac{\\partial}{\\partial \\rho} \\left( q_\\alpha \u0026#43; \\frac{5}{3}T_\\alpha \\Gamma_\\alpha \\right) = P_\\alpha $$ where $P_\\alpha$ is the heat source for species $\\alpha$ (ions or electrons), $q_\\alpha$ is the heat flux, and $\\Gamma_\\alpha$ is the particle flux\n$$ \\Gamma_\\alpha = -V\u0026#39; G_1 n_\\alpha \\left[\\sum_{\\beta\\in \\text{species}} \\left(D^\\alpha_{n_\\beta} \\frac{1}{n_\\beta}\\frac{\\partial n_\\beta}{\\partial \\rho} \u0026#43; D^\\alpha_{T_\\beta} \\frac{1}{T_\\beta}\\frac{\\partial T_\\beta}{\\partial \\rho}\\right) \u0026#43; D_E \\frac{E_\\|}{B_p}\\right] $$$$ q_\\alpha = -V\u0026#39;G_1 T_\\alpha n_\\alpha \\left[\\sum_{\\beta\\in \\text{species}} \\left(\\chi^\\alpha_{n_\\beta} \\frac{1}{n_\\beta}\\frac{\\partial n_\\beta}{\\partial \\rho} \u0026#43; \\chi^\\alpha_{T_\\beta} \\frac{1}{T_\\beta}\\frac{\\partial T_\\beta}{\\partial \\rho}\\right) \u0026#43; \\chi_E \\frac{E_\\|}{B_p}\\right] $$ where $E_\\|$ is the parallel electric field and $G_1 := \\langle (\\nabla \\rho)^2\\rangle$. The various transport coefficients $D^\\alpha_{x_\\beta}, \\chi^\\alpha_{x_\\beta}$ quantify, respectively the particle and heat flux of species $\\alpha$ due to a gradient in the profile of $x_\\beta$, while $D_E$ and $\\chi_E$ represent (typically inward) fluxes due to parallel electric fields.\nWe use the same change of variables as before to express the equations in terms of $\\hat\\rho$ and $\\Phi_b$. Using\n$$ q_\\alpha\u0026#43;\\frac{5}{2}T_\\alpha\\Gamma_\\alpha \\quad\\equiv\\quad -\\Big(\\chi_\\alpha n_\\alpha \\frac{g_1}{V\u0026#39;}\\frac{\\partial T_\\alpha}{\\partial\\hat\\rho}\\ -\\ g_0 q_\\alpha^{\\mathrm{conv}} T_\\alpha\\Big), $$ we get to the final form of the ion and electron heat transport equations.\n$$ \\boxed{\\; \\begin{aligned} \\frac{3}{2} V\u0026#39;^{-5/3} \\left( \\frac{\\partial}{\\partial t} - \\frac{\\dot{\\Phi}_b}{2\\Phi_b} \\frac{\\partial}{\\partial \\hat{\\rho}} \\right) \\left[ V\u0026#39;^{5/3} n_i {{T_i}} \\right] \u0026amp;= \\frac{1}{V\u0026#39;} \\frac{\\partial}{\\partial \\hat{\\rho}} \\left[ \\chi_i n_i \\frac{g_1}{V\u0026#39;} \\frac{\\partial {T_i}}{\\partial \\hat{\\rho}} - g_0 q_i^{\\text{conv}} {T_i} \\right] \u0026#43; Q_i \\\\ \\frac{3}{2} V\u0026#39;^{-5/3} \\left( \\frac{\\partial}{\\partial t} - \\frac{\\dot{\\Phi}_b}{2\\Phi_b} \\frac{\\partial}{\\partial \\hat{\\rho}} \\right) \\left[ V\u0026#39;^{5/3} {n_e} {T_e} \\right] \u0026amp;= \\frac{1}{V\u0026#39;} \\frac{\\partial}{\\partial \\hat{\\rho}} \\left[ \\chi_e {n_e} \\frac{g_1}{V\u0026#39;} \\frac{\\partial {T_e}}{\\partial \\hat{\\rho}} - g_0 q_e^{\\text{conv}} {T_e} \\right] \u0026#43; Q_e \\end{aligned}\\;} $$ 2. Discretization and Numerical Methods In [1], the authors use a finite-volume method to discretize the spatial derivatives in the transport equations.\n2.1 Spatial discretization: Finite Volume Method (FVM) The 1D spatial domain, $0 \\leq \\hat{\\rho} \\leq 1$, is divided into a uniform grid of $N$ cells, each with a width of $d \\hat{\\rho} = 1/N$. The cell centers are denoted by $\\hat{\\rho}_i$, where $i = 1, 2,..., N-1$, and the $N\u0026#43;1$ cell faces are located at $\\hat{\\rho}_{i\\pm1/2}$. Both $\\hat{\\rho}=0$ and $\\hat{\\rho}=1$ are on the face grid.\nFor a generic conservation law of the form: $$ \\frac{\\partial x}{\\partial t} \u0026#43; \\nabla \\cdot \\mathbf{\\Gamma} = S $$ where $x$ is a conserved quantity, $\\mathbf{\\Gamma}$ is the flux, and $S$ is a source term, the FVM approach involves integrating the PDE over a control volume (named a \u0026ldquo;cell\u0026rdquo;) and applying the divergence theorem to convert volume integrals to surface integrals. For 1D systems, following dividing by the cell volume, the finite volume method reduces to a special case of finite differences:\n$$ \\frac{\\partial }{\\partial t}(x_i) \u0026#43; \\frac{1}{d \\hat{\\rho}}({\\Gamma}_{i\u0026#43;1/2} - {\\Gamma}_{i-1/2}) = S_i $$ where $x_i$ is the cell-averaged value of $x$ in cell $i$, $\\Gamma_{i\u0026#43;1/2}$ is the flux at face $i\u0026#43;1/2$, and $S_i$ is the cell-averaged source term in cell $i$.\nIn general, the fluxes are decomposed as\n$$ \\Gamma = -D\\frac{\\partial x}{\\partial \\hat{\\rho}} \u0026#43; Vx $$ where $D$ is a diffusion coefficient and $V$ now denotes a convection coefficient, leading to:\n$$ \\begin{aligned} \\Gamma_{i\u0026#43;1/2} \u0026amp;= -D_{i\u0026#43;1/2}\\frac{x_{i\u0026#43;1} - x_{i}}{d\\hat{\\rho}} \u0026#43; V_{i\u0026#43;1/2}x_{i\u0026#43;1/2} \\\\ \\Gamma_{i-1/2} \u0026amp;= -D_{i-1/2}\\frac{x_{i} - x_{i-1}}{d\\hat{\\rho}} \u0026#43; V_{i-1/2}x_{i-1/2} \\end{aligned} $$ The diffusion and convection coefficients are thus calculated on the face grid. The value of $x$ on the face grid is approximated by implementing a power-law scheme for Péclet-number weighting, which smoothly transitions between central differencing and upwinding, as follows:\n$$ \\begin{aligned} x_{i\u0026#43;1/2} \u0026amp;= \\alpha_{i\u0026#43;1/2}x_i \u0026#43; (1 - \\alpha_{i\u0026#43;1/2}) x_{i\u0026#43;1} \\\\ x_{i-1/2} \u0026amp;= \\alpha_{i-1/2}x_i \u0026#43; (1 - \\alpha_{i-1/2}) x_{i-1} \\\\ \\end{aligned} $$ where the $\\alpha$ weighting factor depends on the Péclet number, defined as:\n$$ Pe = \\frac{V d \\hat{\\rho}}{D} $$ where $V$ is convection and $D$ is diffusion. The power-law scheme is as follows:\n$$ \\alpha = \\begin{cases} \\frac{Pe - 1}{Pe} \u0026amp; \\text{if } Pe \u0026gt; 10, \\\\ \\frac{(Pe - 1) \u0026#43; (1 - Pe/10)^5}{Pe} \u0026amp; \\text{if } 0 \u0026lt; Pe \u0026lt; 10, \\\\ \\frac{(1 \u0026#43; Pe/10)^5 - 1}{Pe} \u0026amp; \\text{if } -10 \u0026lt; Pe \u0026lt; 0, \\\\ -\\frac{1}{Pe} \u0026amp; \\text{if } Pe \u0026lt; -10. \\end{cases} $$ The Péclet number quantifies the relative strength of convection and diffusion. If the Péclet number is small and diffusion dominates, then the weighting scheme converges to central differencing. If the absolute value of the Péclet number is large, and convection dominates, then the scheme converges to upwinding.\nBoundary conditions are taken into account by introducing ghost cells $x_{N}$ and $x_{-1}$ whose values are determined by assuming linear extrapolation through the edge cells and the face boundary conditions (for Dirichlet boundary conditions), or by directly satisfying the derivative (Neumann) boundary conditions.\nThe above equations, when combined, define the elements of the discretization matrix and boundary condition vectors for the PDE diffusion term.\n2.2 Time discretization and solver options TORAX uses the theta method for time discretization, and employs several options for solving the discretized PDE system. These are described below.\nThe θ-method is a weighted average between the explicit and implicit Euler methods. For a generic ODE of the form:\n$$ \\frac{dx}{dt} = F(x, t) $$ where $x$ is the state vector, the theta method approximates the solution at time $t \u0026#43; \\Delta t$ as:\n$$ x_{t \u0026#43; \\Delta t} - x_t = \\Delta t \\big[ \\theta F(x_{t \u0026#43; \\Delta t}, t \u0026#43; \\Delta t) \u0026#43; (1 - \\theta) F(x_t, t)\\big] \\tag{1} $$ where $\\theta$ is a user-selected weighting parameter in the range $[0, 1]$. Different values of $\\theta$ correspond to well-known solution methods: explicit Euler ($\\theta = 0$), Crank-Nicolson ($\\theta = 0.5$), and implicit Euler ($\\theta = 1$), which is unconditionally stable.\nUpon inspection of the equations, we generalize equation (1) and write the state evolution equation as:\n$$\\tag{2} \\begin{aligned} \u0026amp; \\mathbf{\\tilde{T}}(x_{t \u0026#43; \\Delta t}, u_{t \u0026#43; \\Delta t})\\odot \\mathbf{x}_{t \u0026#43; \\Delta t} - \\mathbf{\\tilde{T}}(x_t, u_t)\\odot\\mathbf{x}_t = \\\\ \u0026amp; \\Delta t \\big[ \\theta \\big( \\mathbf{\\bar{C}}(x_{t\u0026#43;\\Delta t}, u_{t\u0026#43;\\Delta t}) \\mathbf{x}_{t\u0026#43;\\Delta t} \u0026#43; \\mathbf{c}(x_{t\u0026#43;\\Delta t}, u_{t\u0026#43;\\Delta t}) \\big) \u0026#43; (1-\\theta) \\big( \\mathbf{\\bar{C}}(x_t, u_t)\\mathbf{x}_t \u0026#43; \\mathbf{c}(x_{t}, u_{t}) \\big) \\big] \\end{aligned} $$ Starting from an initial condition $\\mathbf{x}_0$, equation (2) solves for $\\mathbf{x}_{t\u0026#43;\\Delta t}$ at each timestep. $\\mathbf{x}_t$ is the evolving state vector at time $t$, including all variables being solved by the system, and is of length $\\#N$, where $\\#$ is the number of solved variables.\nSource: [1]\n$\\mathbf{u}_t$ corresponds to all known input parameters at time $t$. This includes boundary conditions, prescribed profiles (e.g. $n_e$ in the example above), and input parameters such as heating powers or locations.\n$\\mathbf{\\tilde{T}}$ is the transient term, where $\\odot$ signifies element-wise multiplication. For example, for the $T_e$ equation, $\\mathbf{\\tilde{T}}=\\mathbf{n_e}$, which makes the system nonlinear if $\\mathbf{n_e}$ itself is an evolving variable.\n$\\mathbf{\\bar{C}}(x_t, u_t)$ and $\\mathbf{\\bar{C}}(x_{t\u0026#43;\\Delta t}, u_{t\u0026#43;\\Delta t})$ are the discretization matrices, of size $\\#N\\times \\#N$. In general, depending on the physics models used, $\\mathbf{\\bar{C}}$ depends on state variables $\\mathbf{x}$, for example through state-variable dependencies of transport coefficients $\\chi$, $D$, $V$, plasma conductivity, and ion-electron heat exchange, making the system nonlinear due to the $x_{t\u0026#43;\\Delta t}$ dependence. $\\mathbf{c}$ is a vector, containing source terms and boundary condition terms.\nThere are two solver options for solving the nonlinear evolution system of equations.\nLinear solver This solver addresses the nonlinearity of the PDE system with fixed-point iteration, also known as the predictor-corrector method. For $K$ iterations (user-configurable), an approximation for $\\mathbf{x}_{t\u0026#43;\\Delta t}$ is obtained by solving the following equation iteratively with $k=1,2,..,K$:\n$$ \\begin{aligned} \u0026amp; \\mathbf{\\tilde{T}}(x_{t \u0026#43; \\Delta t}^{k-1}, u_{t \u0026#43; \\Delta t})\\odot \\mathbf{x}_{t \u0026#43; \\Delta t}^k - \\mathbf{\\tilde{T}}(x_t, u_t)\\odot\\mathbf{x}_t = \\\\ \u0026amp; \\Delta t \\big[ \\theta \\big( \\mathbf{\\bar{C}}(x_{t\u0026#43;\\Delta t}^{k-1}, u_{t\u0026#43;\\Delta t})\\mathbf{x}_{t\u0026#43;\\Delta t}^k \u0026#43; \\mathbf{c}(x_{t\u0026#43;\\Delta t}^{k-1}, u_{t\u0026#43;\\Delta t}) \\big) \u0026#43; (1-\\theta) \\big( \\mathbf{\\bar{C}}(x_t, u_t)\\mathbf{x}_t \u0026#43; \\mathbf{c}(x_{t}, u_{t}) \\big) \\big] \\end{aligned} $$ and where $\\mathbf{x}_{t\u0026#43;\\Delta t}^{0} = \\mathbf{x}_t$.\nBy replacing $\\mathbf{x}_{t\u0026#43;\\Delta t}$ with $\\mathbf{x}_{t\u0026#43;\\Delta t}^{k-1}$ within the coefficients $\\mathbf{\\tilde{T}}$, $\\mathbf{\\bar{C}}$ and $\\mathbf{c}$, these coefficients become known at every iteration step, describing a linear system of equations. $\\mathbf{x}_{t\u0026#43;\\Delta t}^k$ can then be solved using standard linear algebra methods implemented in JAX.\nNewton-Raphson Solver This solver solves the nonlinear PDE system, using a gradient-based iterative Newton-Raphson root-finding method for finding the value of $\\mathbf{x}_{t\u0026#43;\\Delta t}$ that renders the residual vector zero: $$\\tag{3} \\mathbf{R}(\\mathbf{x}_{t\u0026#43;\\Delta t},\\mathbf{x}_t,\\mathbf{u}_{t\u0026#43;\\Delta t}, \\mathbf{u}_t, \\theta, \\Delta t) = 0 $$ where $\\mathbf{R}$ is the LHS-RHS of equation (2).\nStarting from an initial guess $\\mathbf{x}_{t\u0026#43;\\Delta t}=\\mathbf{x}_{t\u0026#43;\\Delta t}^0$, the Newton-Raphson method linearizes equation (3) about iteration $\\mathbf{x}_{t\u0026#43;\\Delta t}^k$ and solves the linear system for a step $\\delta\\mathbf{x}$:\n$$ \\mathbf{\\bar{J}}(\\mathbf{x}_{t\u0026#43;\\Delta t}^k) \\delta\\mathbf{x} = -\\mathbf{R}(\\mathbf{x}_{t\u0026#43;\\Delta t}^k) $$ where $\\mathbf{\\bar{J}}$ is the Jacobian of $\\mathbf{R}$ with respect to $\\mathbf{x}_{t\u0026#43;\\Delta t}$. Crucially, JAX automatically calculates $\\mathbf{\\bar{J}}$ using auto-differentiation.\nWith $\\delta\\mathbf{x} = \\mathbf{x}_{t\u0026#43;\\Delta t}^{k\u0026#43;1} - \\mathbf{x}_{t\u0026#43;\\Delta t}^{k}$, $\\mathbf{x}_{t\u0026#43;\\Delta t}^{k\u0026#43;1}$ is solved using standard linear algebra methods implemented in JAX such as LU decomposition. This process iterates until the residual falls below a user-configurable tolerance $\\varepsilon$, i.e: $\\| \\mathbf{R}(\\mathbf{x}_{t\u0026#43;\\Delta t}^{k\u0026#43;1}) \\|_2 \u0026lt; \\varepsilon$, where $\\|\\cdot\\|_2$ is the vector two-norm.\n3. Code Implementation (Torax) You can see more examples and documentation at the Torax GitHub repository.\nimport torax torax_config = torax.build_torax_config_from_file(\u0026#39;examples/iterhybrid_rampup.py\u0026#39;) # load Iter hybrid config torax_config.update_fields({ \u0026#34;numerics.fixed_dt\u0026#34;: 1.0, \u0026#34;numerics.t_final\u0026#34;: 100.0, }) dt, state_history = torax.run_simulation(torax_config) And after some post-processing, you can generate plots like the following:\nFigure: Axial view of plasma inside a tokamak during a simulation.\nThe most important quantities are the ones related to fusion power density, which can be plotted as follows:\nfig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5)) # Plot p_alpha_i on first subplot dt.profiles.p_alpha_i.plot(ax=ax1) ax1.set_xlabel(\u0026#39;$\\\\hat{\\\\rho}$\u0026#39;) ax1.set_title(\u0026#39;Ion heating power density (W/m$^3$)\u0026#39;) # Plot p_alpha_e on second subplot dt.profiles.p_alpha_e.plot(ax=ax2) ax2.set_xlabel(\u0026#39;$\\\\hat{\\\\rho}$\u0026#39;) ax2.set_title(\u0026#39;Electron heating power density (W/m$^3$)\u0026#39;) plt.savefig(\u0026#39;fusion_power_density.png\u0026#39;, dpi=300, bbox_inches=\u0026#39;tight\u0026#39;) plt.show() Figure: Ion and electron heating power density from fusion reactions during a simulation of ITER hybrid scenario ramp-up phase.\nReferences [0] The code used for the figures is available at: https://github.com/dani2442/dani2442_code/tree/main/torax\n[1] Citrin, Jonathan, Ian Goodfellow, Akhil Raju, Jeremy Chen, Jonas Degrave, Craig Donner, Federico Felici et al. \u0026ldquo;TORAX: A fast and differentiable tokamak transport simulator in JAX.\u0026rdquo; arXiv preprint arXiv:2406.06718 (2024).\n[2] Felici, Federico. \u0026ldquo;Real-time control of tokamak plasmas: from control of physics to physics-based control.\u0026rdquo; (2011).\n[3] https://deepmind.google/discover/blog/bringing-ai-to-the-next-generation-of-fusion-energy/\n[4] Hinton, F. L. and R. D. Hazeltine (1976). “Theory of plasma transport in toroidal confinement systems.” In: Rev. Mod. Phys. 48.2, pp. 239–308. doi: 10.1103/RevModPhys. 48.239.\nAppendix: Derivation of the 1D Transport Equations Derivation of the Poloidal Flux Diffusion Equation\nWe derive the poloidal flux diffusion equation, describing the temporal evolution of the poloidal flux under the assumption of static background flux surfaces. We follow [2] Preliminaries. We use the useful relation $$ \\begin{align*} \\langle \\nabla \\cdot \\mathbf{F} \\rangle \u0026amp;= \\frac{\\partial}{\\partial V}\\int (\\nabla \\cdot \\mathbf{F}) dV = \\frac{\\partial}{\\partial V} \\int (\\nabla \\cdot \\mathbf{F}) Rd\\phi d\\ell_p \\frac{d\\psi}{|\\nabla\\psi|} \\\\ \u0026amp;= \\frac{\\partial }{\\partial V} \\oint \\mathbf{F} \\cdot \\frac{\\nabla V}{|\\nabla V|} Rd\\phi d\\ell_p \\frac{d\\psi}{|\\nabla\\psi|} \\\\ \u0026amp;= \\frac{\\partial }{\\partial V}2\\pi \\oint \\mathbf{F} \\cdot \\nabla V \\frac{\\partial \\psi}{\\partial V} \\frac{Rd\\ell_p}{|\\nabla\\psi|} \\\\ \u0026amp;= \\frac{\\partial}{\\partial V} \\langle \\mathbf{F} \\cdot \\nabla V\\rangle \\tag{A.0} \\end{align*} $$ Here, we used that the average of $\\nabla \\cdot \\mathbf{F}$ over a flux surface can be represented by the derivative with respect to the enclosed volume $V$ of the volume integral of $\\nabla \\cdot \\mathbf{F}$. Then we change to flux coordinates $$ dV = Rd\\phi d\\ell_p \\frac{d\\psi}{|\\nabla\\psi|} $$ where $\\phi$ is the toroidal angle and $\\ell_p$ the poloidal length along the flux surface. Then we use Gauss divergence theorem to convert volume to surface integral, $$ \\int_V (\\nabla \\cdot \\mathbf{F}) dV = \\oint_{\\partial V} \\mathbf{F} \\cdot \\mathbf{n} dS = \\oint_{\\partial V} \\mathbf{F} \\cdot \\frac{\\nabla V}{|\\nabla V|}dS $$ Because $\\phi$ is symmetric (axis) then the integral over $\\phi$ gives a factor $2\\pi$. Finally, the differential surface area of the flux surface is $$dS = 2\\pi R d\\ell_p$$ Next, by the chain rule, we have $$ |\\nabla V| = \\frac{\\partial V}{\\partial \\psi} |\\nabla \\psi| $$ Thus, $$ \\langle\\mathbf{F} \\cdot \\nabla V\\rangle = \\int_V (\\mathbf{F}\\cdot \\nabla V) dV = \\oint \\frac{(\\mathbf{F}\\cdot V) dS}{|\\nabla V|} = \\oint (\\mathbf{F}\\cdot \\nabla \\psi) \\frac{\\partial \\psi}{\\partial V} \\frac{dS}{|\\nabla \\psi|} $$ And this finishes the proof of the relation.\nConsider a surface of constant poloidal flux whose boundary moves with velocity $\\mathbf{u}_\\psi$. For this surface: $$ \\frac{\\partial \\psi}{\\partial t} \u0026#43; \\mathbf{u}_\\psi \\cdot \\nabla \\psi = 0 \\tag{A.1} $$ For a scalar field $F(t, \\mathbf{x})$, define $H(t) = \\int_V F\\,dV$, where $V$ is the volume enclosed by $\\psi = \\text{const}$ moving with $\\mathbf{u}_\\psi$. By the Reynolds Transport Theorem, we have $$ \\begin{align*} \\frac{\\partial H}{\\partial t}\\bigg|_{\\psi=\\text{const}} \u0026amp;= \\overbrace{\\int_V \\frac{\\partial F}{\\partial t} dV}^{\\text{change inside volume}}\u0026#43;\\overbrace{\\oint_S F \\mathbf{u}_\\psi \\cdot \\mathbf{dS}_\\psi}^{\\text{change due to moving boundary}}\\\\ \u0026amp;= \\int_V \\frac{\\partial F}{\\partial t} dV \u0026#43; \\oint_S F \\mathbf{u}_\\psi \\cdot \\frac{\\nabla \\psi}{|\\nabla \\psi|} dS \\end{align*}\\tag{A.1b} $$ Using this equality, the time rate of change of toroidal flux $\\Phi$ enclosed by $\\psi = \\text{const}$, i.e., $F=\\mathbf{B}\\cdot\\nabla\\phi$ and $\\Phi(t)=\\int_V \\mathbf{B}\\cdot\\nabla\\phi\\, dV$, is: $$ \\begin{aligned} \\frac{\\partial \\Phi}{\\partial t}\\bigg|_{\\psi=\\text{const}} \u0026amp;= \\frac{1}{2\\pi} \\frac{\\partial}{\\partial t} \\int_V \\mathbf{B}\\cdot\\nabla\\phi\\, dV \\nonumber \\\\ \u0026amp;= \\frac{1}{2\\pi} \\int_V \\frac{\\partial \\mathbf{B}}{\\partial t}\\cdot\\nabla\\phi\\, dV \u0026#43; \\frac{1}{2\\pi} \\oint_S (\\mathbf{B}\\cdot\\nabla\\phi)(\\mathbf{u}_\\psi\\cdot\\nabla\\psi) \\frac{dS}{|\\nabla\\psi|} \\end{aligned}\\tag{A.2} $$ Poloidal electric field. Using Faraday’s law $\\partial_t \\mathbf{B} = -\\nabla\\times\\mathbf{E}$, we can rewrite the first volume integral (A.2) as: $$ \\begin{aligned} \\int_V \\frac{\\partial \\mathbf{B}}{\\partial t}\\cdot\\nabla\\phi\\, dV \u0026amp;= -\\int_V (\\nabla\\times\\mathbf{E})\\cdot\\nabla\\phi\\, dV = -\\int_V \\nabla\\cdot(\\mathbf{E}\\times\\nabla\\phi)\\, dV \\nonumber\\\\ \u0026amp;= -\\oint_S (\\mathbf{E}\\times \\nabla \\phi)\\frac{\\nabla\\psi}{|\\nabla \\psi|}dS \\tag{Gauss}\\\\ \u0026amp;= -\\oint_S \\mathbf{E}\\cdot(\\nabla\\phi\\times\\nabla\\psi)\\, \\frac{dS}{|\\nabla\\psi|} = -2\\pi \\oint_S \\mathbf{E}\\cdot\\mathbf{B}_p \\frac{dS}{|\\nabla\\psi|} \\end{aligned} $$ Toroidal electric field. We can rewrite the second integral of (A.2) using Ampère’s law: $$ \\begin{aligned} \\nabla\\psi\\cdot\\frac{\\partial\\mathbf{B}}{\\partial t} \u0026amp;= -\\nabla\\psi\\cdot(\\nabla\\times\\mathbf{E}) = \\nabla\\cdot(\\nabla\\psi\\times\\nabla \\mathbf{E})- \\underbrace{ \\mathbf{E}\\cdot(\\nabla\\times \\nabla\\psi)}_{=0}\\\\ \u0026amp;= \\nabla \\cdot (\\nabla \\psi \\times \\nabla \\phi R E_\\phi) =-\\nabla\\cdot(2\\pi \\mathbf{B}_p R E_\\phi) \\end{aligned} $$ Using that $\\mathbf{B} \\cdot \\nabla \\psi = 0$, we have $$ 0 = \\partial_t (\\mathbf{B} \\cdot \\nabla \\psi) = \\partial_t \\mathbf{B} \\cdot \\nabla \\psi \u0026#43; \\mathbf{B} \\cdot \\nabla (\\partial_t \\psi) $$ Then, $$ \\begin{aligned} \\mathbf{B} \\cdot \\nabla (\\partial_t \\psi) \u0026amp;= -\\partial_t \\mathbf{B} \\cdot \\nabla \\psi = \\nabla\\cdot(2\\pi \\mathbf{B}_p R E_\\phi) \\\\ \u0026amp;= \\mathbf{B}_b \\cdot \\nabla (2\\pi R E_\\phi) \u0026#43; 2\\pi R E_\\phi \\underbrace{\\nabla \\cdot \\mathbf{B}_p}_{=0}\\\\ \u0026amp;= \\mathbf{B} \\cdot \\nabla (2\\pi R E_\\phi) \\end{aligned} $$ Thus, using previous equation and (A.1): $$ \\mathbf{B} \\cdot \\nabla (\\partial_t \\psi) = 2\\pi R E_\\phi = -\\mathbf{u}_\\psi \\cdot \\nabla\\psi $$ Combining previous results into (A.2): $$ \\begin{aligned} \\frac{\\partial \\Phi}{\\partial t}\\bigg|_{\\psi=\\text{const}} \u0026amp;= -\\oint_S (\\mathbf{E}\\cdot\\mathbf{B}_p \u0026#43; B_\\phi E_\\phi) \\frac{dS}{|\\nabla\\psi|} = -\\oint_S \\mathbf{E}\\cdot\\mathbf{B} \\frac{dS}{|\\nabla\\psi|} = -\\frac{\\partial V}{\\partial\\psi}\\langle \\mathbf{E}\\cdot\\mathbf{B}\\rangle \\end{aligned} $$ Rate of change of poloidal flux. $$ \\begin{aligned} \u0026amp;\\frac{\\partial\\psi}{\\partial t}\\bigg|_{\\Phi=\\text{const}} = \\frac{\\partial\\psi}{\\partial V}\\frac{\\partial V}{\\partial\\Phi}\\frac{\\partial\\Phi}{\\partial t}\\bigg|_{\\psi=\\text{const}} \\\\ \u0026amp;\\frac{\\partial\\psi}{\\partial t}\\bigg|_{\\rho} \u0026#43; \\frac{\\partial\\psi}{\\partial\\rho}\\frac{\\partial\\rho}{\\partial t}\\bigg|_{\\Phi}= -\\frac{\\partial V}{\\partial\\Phi}\\langle \\mathbf{E}\\cdot\\mathbf{B}\\rangle \\tag{A.3}\\\\ \u0026amp;\\frac{\\partial\\psi}{\\partial t}\\bigg|_{\\rho} - \\rho\\frac{\\dot{B}_0}{2B_0}\\frac{\\partial\\psi}{\\partial\\rho} = -2\\pi R_0^2 \\frac{\\langle \\mathbf{E}\\cdot\\mathbf{B}\\rangle}{F\\langle R_0^2/R^2\\rangle} \\end{aligned} $$ Define the equivalent cylindrical fields: $$ \\begin{aligned} B_{po} \u0026amp;= \\frac{1}{2\\pi R_0} \\frac{\\partial\\psi}{\\partial\\rho} \\\\ E_o \u0026amp;= R_0 \\frac{\\langle \\mathbf{E}\\cdot\\mathbf{B}\\rangle}{F\\langle R_0^2/R^2\\rangle} \\end{aligned} $$ Then, for $\\dot{B}_0=0$: $$ \\frac{\\partial B_{p0}}{\\partial t} = -\\frac{\\partial E_0}{\\partial\\rho} $$ Ohm’s law. We can write the flux-surface-averaged Ohm’s law as: $$ \\langle \\mathbf{j}\\cdot\\mathbf{B}\\rangle = \\sigma_\\parallel \\langle \\mathbf{E}\\cdot\\mathbf{B}\\rangle \u0026#43; \\langle \\mathbf{j}_{ni}\\cdot\\mathbf{B}\\rangle $$ where $\\mathbf{j}_{ni} = \\mathbf{j}_{bs} \u0026#43; \\mathbf{j}_{cd}$ is the non-inductive current density, which includes the bootstrap current density $\\mathbf{j}_{bs}$ and the external current drive density $\\mathbf{j}_{cd}$. Equivalently $$ \\sigma_\\parallel \\frac{\\langle \\mathbf{E}\\cdot\\mathbf{B}\\rangle}{B_0} = \\frac{\\langle \\mathbf{j}\\cdot\\mathbf{B}\\rangle}{B_0} - \\frac{\\langle \\mathbf{j}_{bs}\\cdot\\mathbf{B}\\rangle}{B_0} - \\frac{\\langle \\mathbf{j}_{cd}\\cdot\\mathbf{B}\\rangle}{B_0} \\quad \\Leftrightarrow \\quad \\sigma_\\parallel E_\\parallel = j_\\parallel - j_{bs} - j_{cd}\\tag{A.4} $$ Parallel current. Finally, we rewrite the parallel current $j_\\parallel = \\frac{\\langle \\mathbf{j}\\cdot\\mathbf{B}\\rangle}{B_0}$ in terms of the poloidal flux $\\psi$: $$ \\begin{aligned} \\frac{\\langle \\mathbf{j}\\cdot\\mathbf{B}\\rangle}{B_0} \u0026amp;= \\frac{T}{2\\pi \\mu_0 B_0}\\langle \\nabla\\cdot (\\nabla \\psi/R^2)\\rangle \u0026#43; \\frac{1}{2\\pi \\mu_0 B_0}\\langle \\nabla T \\cdot \\nabla \\psi /R^2\\rangle\\\\ \u0026amp;=\\frac{1}{2\\pi \\mu_0 B_0}\\left(\\frac{4\\pi^2 T}{V\u0026#39;} \\frac{\\partial}{\\partial \\rho}\\left(G^2\\frac{\\partial\\psi}{\\partial \\rho}\\right) \u0026#43; \\frac{4\\pi^2}{V\u0026#39;}\\frac{V\u0026#39;}{4\\pi^2} \\left\\langle \\frac{(\\nabla \\rho)^2}{R^2}\\right\\rangle \\frac{\\partial T}{\\partial \\rho}\\frac{\\partial \\psi}{\\partial \\rho}\\right)\\\\ \u0026amp;=\\frac{2\\pi R_0 J^2}{\\mu_0 V\u0026#39;}\\left(\\frac{1}{J} \\frac{\\partial}{\\partial \\rho}\\left(G^2 \\frac{\\partial \\psi}{\\partial \\rho}\\right) \u0026#43; \\frac{G^2}{J^2}\\frac{\\partial J}{\\partial \\rho}\\frac{\\partial \\psi}{\\partial \\rho}\\right) \\\\ \u0026amp;=\\frac{2\\pi R_0 J^2}{\\mu_0 V\u0026#39;} \\frac{\\partial}{\\partial\\rho} \\left(\\frac{G^2}{J}\\frac{\\partial\\psi}{\\partial\\rho}\\right)\\tag{A.5} \\end{aligned} $$ The key step is converting flux-surface averages of divergences into 1D (radial) derivatives: $$ \\left\\langle \\nabla\\cdot \\mathbf{F}\\right\\rangle = \\frac{1}{V\u0026#39;}\\frac{\\partial}{\\partial\\rho}\\left(V\u0026#39;\\left\\langle \\mathbf{F}\\cdot\\nabla\\rho\\right\\rangle\\right),\\qquad V\u0026#39;\\equiv\\frac{\\partial V}{\\partial\\rho}. $$ Taking $\\mathbf{F}=\\nabla\\psi/R^2$ and using $\\psi=\\psi(\\rho)$ gives $\\nabla\\psi=(\\partial\\psi/\\partial\\rho)\\nabla\\rho$ and hence $\\mathbf{F}\\cdot\\nabla\\rho=((\\nabla\\rho)^2/R^2)(\\partial\\psi/\\partial\\rho)$. Also, since $T=T(\\rho)$, $\\langle \\nabla T\\cdot\\nabla\\psi/R^2\\rangle = \\langle (\\nabla\\rho)^2/R^2\\rangle(\\partial T/\\partial\\rho)(\\partial\\psi/\\partial\\rho)$. Defining $G^2\\equiv \\frac{V\u0026#39;}{4\\pi^2}\\left\\langle \\frac{(\\nabla\\rho)^2}{R^2} \\right\\rangle$ packages the terms and the product rule yields the final compact form. with $$ J = \\frac{F}{R_0 B_0}, \\quad G^2 = \\frac{V\u0026#39;}{4\\pi^2}\\left\\langle \\frac{(\\nabla\\rho)^2}{R^2} \\right\\rangle, \\quad V\u0026#39; = \\frac{\\partial V}{\\partial\\rho} $$ The poloidal flux diffusion equation. Combining (A.3), (A.4) and (A.5): $$ \\sigma_\\parallel \\left( \\frac{\\partial\\psi}{\\partial t} \u0026#43; \\rho\\frac{\\dot{B}_0}{2B_0}\\frac{\\partial\\psi}{\\partial\\rho} \\right) = \\frac{R_0 J^2}{\\mu_0 \\rho}\\frac{\\partial}{\\partial\\rho} \\left(\\frac{G^2}{J}\\frac{\\partial\\psi}{\\partial\\rho}\\right)- \\frac{V\u0026#39;}{2\\pi\\rho}(j_{bs} \u0026#43; j_{cd}) $$ Derivation of the Particle Transport Equation\nFor an arbitrary plasma species $\\alpha$, let $n_\\alpha$ be the density, $\\mathbf{u}_\\alpha$ the velocity, $s_\\alpha$ the source term. The continuity equation is: $$ \\frac{\\partial n_\\alpha}{\\partial t} \u0026#43; \\nabla\\cdot(n_\\alpha \\mathbf{u}_\\alpha) = s_\\alpha $$ Integrating over the volume enclosed by a flux surface (and using Gauss theorem for the second term): $$ \\int \\frac{\\partial n_\\alpha}{\\partial t} dV \u0026#43; \\oint n_\\alpha \\mathbf{u}_\\alpha\\cdot\\frac{\\nabla\\Phi}{|\\nabla\\Phi|} dS = \\int s_\\alpha dV $$ Using Eq. (A.1b) with $F=n_\\alpha$ and $\\Phi=\\text{const}$ and the fact that $\\nabla \\rho \\| \\nabla \\Phi$: $$ \\frac{\\partial}{\\partial t}\\bigg|_\\Phi \\int n_\\alpha dV \u0026#43; \\oint n_\\alpha (\\mathbf{u}_\\alpha - \\mathbf{u}_\\Phi)\\cdot\\frac{\\nabla\\rho}{|\\nabla\\rho|} dS = \\int s_\\alpha dV $$ Using flux surface averaging: $$ \\frac{\\partial}{\\partial t}\\bigg|_\\Phi \\langle n_\\alpha\\rangle V\u0026#39; d\\rho \u0026#43; V\u0026#39; \\langle n_\\alpha(\\mathbf{u}_\\alpha - \\mathbf{u}_\\Phi)\\cdot\\nabla\\rho\\rangle = \\int\\langle s_\\alpha\\rangle V\u0026#39;d\\rho $$ Then taking the derivative with respect to $\\rho$: $$ \\frac{\\partial}{\\partial t}\\bigg|_\\Phi [\\langle n_\\alpha\\rangle V\u0026#39;] \u0026#43; \\underbrace{\\frac{\\partial}{\\partial \\rho} \\left( V\u0026#39; \\langle n_\\alpha(\\mathbf{u}_\\alpha - \\mathbf{u}_\\Phi)\\cdot\\nabla\\rho\\rangle \\right)}_{\\Gamma_\\alpha} = \\underbrace{\\langle s_\\alpha\\rangle}_{S_\\alpha} V\u0026#39; $$ where $\\Gamma_\\alpha$ is the particle flux and $S_\\alpha$ the source term. Finally, at constant $\\rho$: $$ \\frac{1}{V\u0026#39;}\\left(\\frac{\\partial}{\\partial t} \u0026#43; \\frac{\\dot{B}_0}{2B_0}\\rho\\frac{\\partial}{\\partial\\rho}\\right) (\\langle n_\\alpha\\rangle V\u0026#39;) \u0026#43; \\frac{1}{V\u0026#39;}\\frac{\\partial\\Gamma_\\alpha}{\\partial\\rho} = S_\\alpha. $$","permalink":"https://dani2442.github.io/posts/torax/","summary":"\u003cfigure\u003e\n  \u003cvideo src=\"/posts/torax/particles.mp4\" autoplay loop muted playsinline aria-label=\"Animation of simulated particles in a tokamak cross-section\" role=\"img\"\u003e\u003c/video\u003e\n\u003c/figure\u003e\n\n\u003cp\u003eI think there are 3 major milestones remaining for humanity, and one of them is clean and abundant energy. Fusion energy has the potential to provide a nearly limitless source of clean energy by replicating the processes that power the sun. However, achieving controlled fusion reactions on Earth has proven to be a formidable and very challenging task.\u003c/p\u003e","title":"Fusion Energy Simulation: Tokamak"},{"content":"This week I had the chance to present at the workshop “The Mathematics of Scientific Machine Learning and Digital Twins” in Palermo, Italy.\nMy talk focused on the intersection between kernel methods and machine learning.\nI’m sharing the slides here in case you’re interested in these topics Link.\n","permalink":"https://dani2442.github.io/events/2025_11_20_erice/","summary":"\u003cp\u003eThis week I had the chance to present at the workshop “The Mathematics of Scientific Machine Learning and Digital Twins” in Palermo, Italy.\u003c/p\u003e\n\u003cp\u003eMy talk focused on the intersection between kernel methods and machine learning.\u003c/p\u003e\n\u003cp\u003eI’m sharing the slides here in case you’re interested in these topics \u003ca href=\"/files/erice_ppt.pdf\"\u003eLink\u003c/a\u003e.\u003c/p\u003e","title":"Talk at Workshop in Palermo (Italy)"},{"content":"\nFor some time, I’ve been thinking about how to generalize self-attention mechanisms. Most existing attention mechanisms rely on pairwise similarities (dot products) between query and key vectors. However, higher-order relationships (involving triples or tuples of elements) could capture richer interactions. I then found that several people are already exploring this idea under the name “higher-order attention” [5].\nHowever, this approach comes with a performance cost. Traditional self-attention has a complexity of O(n^2), while higher-order attention is even more computationally expensive. In this post, I’d like to share my perspective on this topic, connecting it with kernel methods and generalized metric spaces.\n1. Motivation: Self-Attention We are all familiar with the self-attention mechanism used in transformers: Given input vectors $x_i \\in \\mathbb{R}^{d_m}$, define: $$ Q_i = W_Q x_i, \\quad K_j = W_K x_j, \\quad V_j = W_V x_j $$ where $W_Q, W_K \\in \\mathbb{R}^{d_m \\times d_a}, W_V \\in \\mathbb{R}^{d_m \\times d_v}$ are learned projection matrices.\nThen the attention from token $i$ to token $j$ is $$ \\text{Attn}(x_i, x_j) = \\text{softmax}_j\\left(Q_i K_j^\\top\\right) V_j = \\frac{\\exp(\\langle Q_i, K_j \\rangle)}{\\sum_{k} \\exp(\\langle Q_i, K_k \\rangle)} V_j $$ where we have omitted scaling factors for simplicity. and the aggregated output for token $i$ is $$ \\text{Attn}(x_i, X) = \\sum_j \\alpha_{ij} V_j $$ We can see that attention is fundamentally based on pairwise interactions between elements (queries and keys): $$ \\kappa_{i,j}\\equiv \\kappa(x_i, x_j) = \\exp(\\langle Q_i, K_j \\rangle) = \\exp(Q_{i,\\ell} K_{j,\\ell}) $$ with summation over $\\ell$ implied. This is why attention is $\\mathcal{O}(n^2)$ in the number of tokens $n$. Moreover, attention can be interpreted as a kernel method, where the kernel is given by the dot product between the projected vectors.\nThis can be implemented as follows:\nn = 15 # number of tokens d_model, d_attn, d_value = 5, 4, 16 # dm, da, dv X = torch.randn(n, d_model) Wq = torch.randn(d_model, d_attn) Wk = torch.randn(d_model, d_attn) Wv = torch.randn(d_model, d_value) def self_attention(X, Wq, Wk, Wv, scale=1.0): Q = X @ Wq * scale # [n, da] K = X @ Wk * scale # [n, da] V = X @ Wv # [n, dv] scores = Q @ K.T # or torch.einsum(\u0026#39;il,jl-\u0026gt;ij\u0026#39;, Q, K); [n,n] attn = torch.softmax(scores, axis=1) # [n,n] Y = attn @ V # or torch.einsum(\u0026#39;ij,jd-\u0026gt;id\u0026#39;, attn, V); [n, dv] return Y Y = self_attention(X, Wq, Wk, Wv, scale=1.0/np.sqrt(d_attn)) The following picture summarizes several variations of self-attention: Figure: Variations of attention mechanisms that have been proposed recently [6]\n1.1 Tensor Product Attention In higher-order attention [5], the idea is to consider interactions among triples or tuples of elements. For example, a triadic attention mechanism could be defined as: $$ \\kappa_{i,j_1,j_2} = \\exp(Q_{i,\\ell} K_{j_1, \\ell} K_{j_2, \\ell}) $$ And the generalization to $m$-ary attention would involve products over $m$ keys: $$ \\kappa_{i,j_1,\\ldots,j_m} = \\exp(Q_{i,\\ell} \\prod_{k=1}^m K_{j_k, \\ell}) $$ However, most of these approaches make use of tensor products and therefore lose the nice properties of kernels (positive-definiteness, representer theorem, etc.). The following code summarizes the idea:\nWk1 = torch.randn(d_model, d_attn) Wk2 = torch.randn(d_model, d_attn) def self_attention_3d(X, Wq, Wk1, Wk2, Wv): Q = X @ Wq # [n, da] K1 = X @ Wk1 # [n, da] K2 = X @ Wk2 # [n, da] V = X @ Wv # [n, dv] scores = torch.einsum(\u0026#39;il,jl,kl-\u0026gt;ijk\u0026#39;, Q, K1, K2) # [n, n, n] attn = torch.softmax(scores.reshape(n, -1), dim=1).reshape(n, n, n) # [n, n, n] Vjk = torch.cat([V.unsqueeze(1)+V.unsqueeze(0)], dim=0) # [n, n, dv] Y = torch.einsum(\u0026#39;ijk,jkd-\u0026gt;id\u0026#39;, attn, Vjk) # [n, dv] return Y Y_3d = self_attention_3d(X, Wq, Wk1, Wk2, Wv) 2. Self-Attention and Kernel Methods As mentioned, self-attention can be interpreted as a kernel method, where the kernel is given by the dot product between the projected vectors. In this section, we will explore the synergy between self-attention and kernel methods, and how to generalize to higher-order attention using computational efficient approximations.\n2.1 Random Fourier Features for Attention Computing all $\\kappa(Q_i, K_j)$ requires $\\mathcal O(n^2)$ time for sequence length $n$, since we need to evaluate the kernel for all pairs $(i,j)$. How can we reduce this cost?\nRandom Fourier Attention (RFA) [7] provides a way to approximate shift-invariant kernels using random features, enabling linear-time computation.\nBochner’s theorem characterizes continuous, shift-invariant, positive-definite kernels on $\\mathbb{R}^d$ (the more general case covers locally compact abelian groups, but we focus on $\\mathbb{R}^d$ for simplicity):\nTheorem (Bochner). A continuous, shift-invariant kernel $\\kappa(x,y)=\\kappa(x-y)$ on $\\mathbb{R}^d$ is positive definite iff it is the Fourier transform of a non-negative measure $\\mu.$ $$ \\kappa(z) = \\int_{\\Omega} e^{-i\\langle z, \\omega \\rangle} d\\mu(\\omega) $$ That is, there exists a non-negative measure $\\mu$ such that $$ \\begin{aligned} \\kappa(x-y) \u0026amp;= \\int_{\\Omega} e^{-i\\langle x-y, \\omega \\rangle} d\\mu(\\omega) = \\mathbb{E}_{\\omega \\sim \\mu} \\left[ e^{-i\\langle x-y, \\omega \\rangle}\\right] \\\\ \u0026amp;= \\mathbb{E}_{\\omega \\sim \\mu} \\left[ \\cos(\\langle x-y, \\omega \\rangle) \\right] \\\\ \u0026amp;= \\mathbb{E}_{\\omega \\sim \\mu} \\left[ \\begin{pmatrix}\\cos(\\langle x, \\omega \\rangle) \\\\ \\sin(\\langle x, \\omega \\rangle) \\end{pmatrix}^\\top \\begin{pmatrix}\\cos(\\langle y, \\omega \\rangle) \\\\ \\sin(\\langle y, \\omega \\rangle) \\end{pmatrix} \\right] \\end{aligned}\\tag{1} $$ where we can obtain $\\mu$ as the inverse Fourier transform of $\\kappa$. Indeed, if we assume that $\\kappa$ is the Gaussian kernel: $$ \\kappa(z) = \\exp\\left(-\\frac{\\|z\\|^2}{2\\sigma^2}\\right) $$ then we have: $$ \\mu(\\omega) = \\frac{1}{(2\\pi)^d} \\int_{\\mathbb{R}^d} \\kappa(z) e^{-i\\langle z, \\omega \\rangle} dz = (2\\pi)^{-d/2}\\sigma^d \\exp\\left(-\\frac{\\sigma^2 \\|\\omega\\|^2}{2} \\right) = \\mathcal{N}(\\omega; 0, \\sigma^{-2} I) $$ Therefore, we can approximate (1) with a feature map: $$ \\phi(x) = \\frac{1}{\\sqrt{N}} [\\sin(\\omega_1^\\top x),\\dots, \\sin(\\omega_N^\\top x), \\cos(\\omega_1^\\top x),\\dots, \\cos(\\omega_N^\\top x)]^\\top $$ where $\\omega_1, \\dots, \\omega_N \\stackrel{i.i.d.}{\\sim} \\mu$. By the law of large numbers, the error goes to zero at rate $\\mathcal O(1/\\sqrt{N})$ as $N\\to\\infty$.\nThis trick makes the problem linear in $n$: we obtain a feature map $\\phi: \\mathbb{R}^{d_k}\\to\\mathbb{R}^r$ such that $$ \\kappa(Q_i, K_j) \\approx \\phi(Q_i)^\\top \\phi(K_j), $$ with $r \\ll n$. We can then rewrite $$ \\sum_j \\kappa(Q_i, K_j)V_j \\approx \\sum_j \\phi(Q_i)^\\top \\phi(K_j)V_j = \\phi(Q_i)^\\top \\Big( \\sum_j \\phi(K_j)V_j \\Big), $$ which can be computed in $\\mathcal O(nr)$ time instead of $\\mathcal O(n^2)$.\ndef rff_attention(X, Wq, Wk, Wv, sigma=1.0, N=1024, eps=1e-8): Q = X @ Wq # [n, da] K = X @ Wk # [n, da] V = X @ Wv # [n, dv] omega = torch.randn(d_attn, N) / sigma # [da, N] def phi(T): TOm = T @ omega # [n, N] return torch.cat([torch.cos(TOm), torch.sin(TOm)], -1) / N**0.5 # [n, 2N] phiQ = phi(Q) # [n, 2N] phiK = phi(K) # [n, 2N] CK = torch.exp((K**2).sum(axis=1, keepdims=True) / (2 * sigma**2)) # [n, 1] S = (phiK*CK).T @ V # [2N, dv] D = torch.sum(phiK * CK, axis=0, keepdim=True) # [1, 2N] denom = phiQ @ D.T # [n, 1] numer = phiQ @ S # [n, dv] return numer / (denom + eps) Y_rfa = rff_attention(X, Wq, Wk, Wv, sigma=np.sqrt(d_attn), N=64) Interestingly, the error between exact attention and RFA attention for $N=64$ is:\n2.2 Towards Higher-Order Attention To extend this to higher-order attention, we need to consider kernels that take multiple arguments, e.g., $\\kappa(x,y,z)$ for triadic attention. A natural question arises:\nAre there generalizations of Bochner’s theorem for multi-argument kernels?\nThe answer is yes, but first let us introduce some concepts:\nDefinition. A function $f : (\\mathbb{R}^d)^n \\to \\mathbb{C}$ is positive semidefinite of order $n$ if, for all integers $m \\ge 1$, for all choices of points $x^{(1)}_i, \\dots, x^{(m)}_i \\subseteq \\mathbb{R}^d$ for $i=1,\\dots, n$, and for all complex coefficients $c_1, \\dots, c_m$, $$ \\sum_{i,j=1}^m c_{i} \\overline{c_{j}} \\, f\\big(x^{(i)}_{1}-x^{(j)}_{1}, \\dots, x^{(i)}_n-x^{(j)}_n\\big) \\ge 0.$$ And the generalization of Bochner’s theorem for $n$-ary functions [8]:\nTheorem (Generalized Bochner Theorem). If $f: (\\mathbb{R}^d)^n \\to \\mathbb{C}$ is a continuous, stationary, $n$-positive definite function, then there exists a unique finite positive Borel measure $\\mu$ on $(\\mathbb{R}^d)^n$ such that, for all $x_1,\\dots,x_n\\in \\mathbb{R}^d$, $$ \u0026gt; f(x_1,\\dots,x_n) = \\int e^{i\\sum_j (\\omega_j\\cdot x_j)} \\, d\\mu(\\omega_1,\\dots,\\omega_n), \u0026gt; $$ where $\\mu$ is a positive measure.\nAs in the 2D case, we can rewrite the kernel as an $(n-1)$-dimensional Fourier transform of its lag form: $$ K(x_1, \\dots, x_n) = f(x_2 - x_1, \\dots, x_n - x_1) $$ Let $T(\\omega_1,\\dots,\\omega_{n-1}) = (-\\sum_{r=1}^{n-1} \\omega_r, \\omega_1, \\dots, \\omega_{n-1})$. Then $\\mu=T_{\\#}\\nu$ is the pushforward of a unique finite positive Borel measure $\\nu$ on $(\\mathbb{R}^d)^{n-1}$ such that $$\\begin{aligned}K(x_1,\\dots,x_n) \u0026amp;= \\int e^{i\\sum_{j=1}^n x_j\\cdot \\omega_j} \\, d\\mu(\\omega_1,\\dots,\\omega_n)\\\\\u0026amp;= \\int e^{i\\sum_{j=2}^n (x_j - x_1)\\cdot \\omega_{j-1}} \\, d\\nu(\\omega_1,\\dots,\\omega_{n-1})\\\\ \u0026amp;= k(x_2 - x_1, \\dots, x_n - x_1).\\end{aligned}$$ Example 1: i.i.d. Gaussian frequencies $$ K(x_1,\\ldots,x_n)=\\exp\\Big(-\\frac{1}{2\\sigma^2}\\sum_{j=1}^n \\|x_j\\|^2\\Big). $$ Moreover, in this case, $\\omega_j \\stackrel{\\text{iid}}{\\sim}\\mathcal N(0,\\sigma^{-2} I_d)$.\nWe are interested in approximating the lagged kernel: $$ k(\\Delta_2,\\dots,\\Delta_n) =\\exp\\left(-\\frac{1}{2\\sigma^2}\\sum_{j=2}^n |\\Delta_j|^2\\right) =\\prod_{j=2}^n \\exp\\left(-\\frac{|\\Delta_j|^2}{2\\sigma^2}\\right). $$ Its spectral measure $\\nu$ factorizes: $$ \\eta_{j-1}\\ \\stackrel{\\text{i.i.d.}}{\\sim}\\ \\mathcal N(0,\\sigma^{-2}I_d),\\qquad j=2,\\dots,n. $$ So we can use either construction above with $\\eta_{j-1}\\sim \\mathcal N(0,\\sigma^{-2}I_d)$. And the feature map is $$\\phi_j(x)=\\frac{1}{\\sqrt{N}} [\\sin(\\eta_1^\\top x),\\dots, \\sin(\\eta_N^\\top x), \\cos(\\eta_1^\\top x),\\dots, \\cos(\\eta_N^\\top x)]^\\top$$ n, d = 5, 50 x = torch.randn(n, d) def k_mc(x, sigma=1.0, N=128): n, d = x.shape diffs = x[1:] - x[0] # [n-1, d] omegas = torch.randn(n-1, d, N) / sigma m = torch.einsum(\u0026#39;ij,ijk-\u0026gt;ik\u0026#39;, diffs, omegas) # [n-1, N] phi = torch.cat([torch.cos(m), torch.sin(m)], -1) / N**0.5 # [n-1, 2N] return torch.mean(torch.prod(phi, dim=0)) def k_exact(x, sigma=1.0): diffs = x[1:] - x[0] # [n-1, d] return torch.exp(-0.5 * sigma**2 * torch.sum(diffs**2)) f = lambda : torch.norm(k_mc(x, sigma=1.0, N=16) - k_exact(x, sigma=1.0)) errors = np.array([f().item() for _ in range(1000)]) print(f\u0026#34;L2 Error mean: {errors.mean():.9f}\u0026#34;, f\u0026#34;std: {errors.std():.9f}\u0026#34;) L2 Error mean: 0.000141528 std: 0.000105931 The error is surprisingly small because the kernel factorizes into a product of simpler kernels.\nExample 2: Correlated Gaussian frequencies. Let $R\\in\\mathbb{R}^{n\\times n}$ be any correlation matrix (symmetric PSD with ones on the diagonal). Then $$ K(x_1,\\ldots,x_n) =\\exp\\Big(-\\frac{1}{2\\sigma^2}\\sum_{j,k=1}^n \\mathbf R_{jk}\\, x_j\\cdot x_k\\Big), $$ which couples different points via $\\mathbf R$.\n2.3 Nyström Method This method is based on the spectral decomposition of positive-definite kernels, known as Mercer’s theorem:\nTheorem (Mercer). For every continuous symmetric and positive-definite kernel $K$ on a compact $X$, there exists an orthonormal basis of $L^2$ functions $\\{\\phi_i\\}_{i=1}^\\infty$ and non-negative eigenvalues $\\{\\lambda_i\\}_{i=1}^\\infty$ such that $$ K(x,y) = \\sum_{i=1}^\\infty \\lambda_i \\phi_i(x) \\phi_i(y). $$ We use the truncated expansion to $m$ and absorb eigenvalues: $$ \\psi_i(x)=\\sqrt{\\lambda_i}\\,\\phi_i(x). $$ In practice we don\u0026rsquo;t usually have a closed-form $(\\lambda_i,\\phi_i)$, so we obtain $\\psi$ data-adaptively via Nyström with $m$ landmarks $Z=\\{z_1,\\dots,z_m\\}$:\n$W\\in\\mathbb{R}^{m\\times m}$, $W_{ab}=\\kappa(z_a,z_b)$ $k_Z(x)=[\\kappa(x,z_1),\\dots,\\kappa(x,z_m)]^\\top$ Feature map: $\\displaystyle \\Psi(x)=W^{-1/2} k_Z(x)\\in\\mathbb{R}^m$ This yields $\\kappa(x,x\u0026#39;)\\approx \\Psi(x)^\\top\\Psi(x\u0026#39;)$ and is equivalent to a truncated Mercer form on the landmark subspace.\nOne-off (per head) cost: compute $W^{-1/2}$ in $\\mathcal O(m^3)$ offline; reuse it at train/inference.\nn, d = 100, 5 # Total data points and data dimension m = 20 # number of inducing points X = torch.randn(n, d) rbf = lambda X, Y: torch.exp(-0.5 * torch.cdist(X, Y)**2) # Nyström approximation Y = torch.randn(m, d) K_mm = rbf(Y, Y) eigvals, eigvecs = torch.linalg.eigh(K_mm) K_mm_inv_sqrt = eigvecs @ torch.diag(1/torch.sqrt(torch.clamp(eigvals, 1e-5))) @ eigvecs.T phi = K_mm_inv_sqrt @ rbf(X, Y).T # [N, m] K_approx = phi.T @ phi K_exact = rbf(X, X) error = torch.linalg.norm(K_exact - K_approx, \u0026#39;fro\u0026#39;)/torch.linalg.norm(K_exact, \u0026#39;fro\u0026#39;) print(f\u0026#34;Relative error: {error:.4f}\u0026#34;) Relative error: 0.4351 2.4 Nyström Method for Higher-order kernels The Nyström method extends to n-ary kernels via a generalized Mercer theorem:\nTheorem (Generalized Mercer). Let $f: (\\mathbb{R}^d)^n \\to \\mathbb{C}$. And view $f$ as a two–argument kernel $K(x_1,z)=f(x_1,z)$. Then the associated Hilbert–Schmidt operator $$ S:L^2(\\mu^{\\otimes(n-1)})\\to L^2(\\mu),\\qquad (Sg)(x_1)=\\int K(x_1,z)g(z)\\,d\\mu^{\\otimes(n-1)}(z), $$ admits a singular system $(\\sigma_j,u_j,v_j)$ with $(\\sigma_j\\ge 0)$, $(u_j)$ orthonormal in $(L^2(\\mu))$ and $(v_j)$ orthonormal in $(L^2(\\mu^{\\otimes(n-1)}))$, yielding the one-vs-rest expansion $$ f(x_1,\\dots,x_n)=\\sum_{j=1}^\\infty \\sigma_j u_j(x_1) v_j(x_2,\\dots,x_n) \\quad\\text{in }L^2(\\mu^{\\otimes n}). $$ If $f$ is continuous on a compact domain, the series converges uniformly.\nThe stronger result $$ f(x_1,\\dots,x_n)=\\sum_{i=1}^\\infty \\lambda_i \\phi_i(x_1)\\cdots\\phi_i(x_n),\\qquad \\lambda_i\\ge 0, $$ is not true in general, this property is usually reffered as $f$ being orthogonally decomposable.\nNyström approximation for tri-argument kernels.\nFix inducing points $C=\\{c_1,\\dots,c_m\\}\\subset\\mathcal{X}$ and define the mode-1 unfolding of the inducing Gram tensor $$ G \\;\\coloneqq\\; K(C,C,C)_{(1)} \\in \\mathbb{R}^{m\\times m^2},\\qquad G_{i,\\,(jk)} \\;=\\; K(c_i,c_j,c_k). $$ For any $x\\in\\mathcal{X}$ define likewise the (row) vector $$ k_x \\;\\coloneqq\\; K(x,C,C)_{(1)} \\in \\mathbb{R}^{1\\times m^2},\\qquad (k_x)_{(jk)} \\;=\\; K(x,c_j,c_k). $$ Now, compute a compact SVD $G=U\\Sigma V^\\top$ with $U\\in\\mathbb{R}^{m\\times r}$, $\\Sigma\\in\\mathbb{R}^{r\\times r}$, $V\\in\\mathbb{R}^{m^2\\times r}$ ($r\\le m$). Define $$ W \\;\\coloneqq\\; U\\,\\Sigma^{-1/2}V^\\top \\in \\mathbb{R}^{m\\times m^2}, $$ so that $GWW^\\top G^\\top = UU^\\top$ is the orthogonal projector onto $\\operatorname{col}(G)$. The Nyström feature map for the first mode is then $$ \\phi(x) \\;\\coloneqq\\; k_x\\, W^\\top \\;\\in\\; \\mathbb{R}^{m}. $$ By construction, $\\phi(c_i)$ are orthonormal coordinates of the mode-1 fibers $G_{i,:}$, and for any $x$ the vector $k_x$ is projected onto the subspace $\\operatorname{span}\\{G_{1,:},\\dots,G_{m,:}\\}$.\nDefine the tri-linear approximation $$ \\widehat{K}(x,y,z) \\;\\coloneqq\\; \\langle \\phi(x),\\phi(y),\\phi(z)\\rangle \\;=\\; \\sum_{r=1}^{m} \\phi_r(x)\\,\\phi_r(y)\\,\\phi_r(z). $$ Expanding with $W$ shows this is precisely the orthogonal projection (in the unfolding space) of the mode-1 fibers associated with $x,y,z$ onto $\\operatorname{col}(G)$ for each mode, yielding the best approximation in the induced least-squares sense among all tri-linear forms supported on that span. In particular, if $x\\in C$ then $k_x\\in\\operatorname{col}(G)$ and the projection is exact for that mode.\nn, d = 100, 5 # Total data points and data dimension m = 10 # number of inducing points X = torch.randn(n, d) def K3(X, Y, Z): dXY = torch.cdist(X, Y).square().unsqueeze(2) # [x, y, 1] dYZ = torch.cdist(Y, Z).square().unsqueeze(0) # [1, y, z] dZX = torch.cdist(X, Z).square().unsqueeze(1) # [x, 1, z] return torch.exp(-0.5 * (dXY + dYZ + dZX)) # Nyström approximation C = torch.randn(m, d) K_mm = K3(C, C, C) # [m, m, m] U, S, Vh = torch.linalg.svd(K_mm.view(m, m*m)) K_mm_inv_sqrt = U @ F.pad(torch.diag_embed(1/torch.sqrt(S.clamp_min(1e-5))), (0, m*m - m)) @ Vh phi = K3(X, C, C).view(n, m*m) @ K_mm_inv_sqrt.T # [N, m] K_approx = torch.einsum(\u0026#39;ir, jr, kr-\u0026gt;ijk\u0026#39;, phi, phi, phi) K_exact = K3(X, X, X) error = torch.linalg.norm(K_exact - K_approx) print(f\u0026#34;Relative error: {error:.4f}\u0026#34;) 3. Generalized Metric Spaces (G-Metric Spaces) An alternative approach I came across is the concept of G-metric spaces, which generalize the notion of distance to triples of points.\nThe concept of a metric space is fundamental in analysis and topology.\nDefinition (Metric Space). A metric space is a pair $(X,d)$ where $X$ is a set and $d: X \\times X \\to \\mathbb{R}_{\\ge 0}$ satisfies:\n$d(x,x)=0$. $d(x,y) \u0026gt; 0$ if $x\\neq y$ (positivity/identity of indiscernibles). $d(x,y)=d(y,x)$ (symmetry). $d(x,z)\\le d(x,y)\u0026#43;d(y,z)$ (triangle inequality). This structure underlies the standard notions of convergence, continuity, compactness, and completeness.\nIt is often useful to consider distances on triples of points rather than only on pairs. This motivates the notion of G-metric spaces, which extend the usual (pairwise) metric to a symmetric, three-argument distance.\nDefinition (G-Metric Space). Following Mustafa–Sims, a G-metric on a set $X$ is a map $G: X \\times X \\times X\\to \\mathbb{R}_{\\ge 0}$ such that, for all $x,y,z,a\\in X$:\n$G(x,y,z)=0$ if and only if $x=y=z$. $G(x,x,y)\u0026gt;0$ whenever $x\\neq y$. $G(x,x,y)\\le G(x,y,z)$ whenever $y\\neq z$. $G$ is symmetric in its three arguments. $G(x,y,z)\\le G(x,a,a)\u0026#43;G(a,y,z)$. When these axioms hold, $(X,G)$ is called a G-metric space. Two simple examples (built from an ordinary metric $d$) are $$ G(x,y,z)=\\max\\{d(x,y),\\,d(y,z),\\,d(z,x)\\},\\qquad G(x,y,z)=d(x,y)\u0026#43;d(y,z)\u0026#43;d(z,x). $$ Both satisfy the axioms above whenever $d$ is a metric. The concept naturally extends to $n$-ary (higher-order) distances, but we focus on the triadic case for clarity.\n3.1 Topology, Convergence and Completeness The G-metric generates a natural topology on $X$ via G-balls. For $x_0\\in X$ and $r\u0026gt;0$, define the $G$-ball $$ B_G(x_0, r) := \\{ y\\in X : G(x_0, y, y)\\lt r \\}. $$ One checks that it is “symmetric” in the sense that if $G(x_0,y,z)\\lt r$ then both $y$ and $z$ lie in $B_G(x_0,r)$. Proposition 4 of Mustafa–Sims shows that these G-balls form a basis for a topology $\\tau(G)$ on $X$, and moreover $\\tau(G)$ coincides with the metric topology arising from the associated metric $d_G(x,y) = G(x,y,y) \u0026#43; G(x,x,y)$ [2]. In particular, every G-metric space is topologically equivalent to an ordinary metric space.\nFrom this one deduces that convergence and continuity in G-metric spaces behave just as in metric spaces. A sequence $(x_n)\\subset X$ is G-convergent to $x\\in X$ if $x_n\\to x$ in the $\\tau(G)$-topology. Equivalently, as shown by Mustafa–Sims, $(x_n)$ converges in the G-sense to $x$ if and only if $$ d_G(x_n, x) \\xrightarrow{n\\rightarrow\\infty} 0, $$ that is, it converges in the usual metric sense. In fact, the following are equivalent [2]:\n$(x_n)$ is G-convergent to $x$. $d_G(x_n,x)\\to0$ as $n\\to\\infty$. $G(x_n,x_n,x)\\to0$ as $n\\to\\infty$. $G(x_n,x_m,x)\\to0$ as $n,m\\to\\infty$. Thus a map $f:(X,G)\\to (X\u0026#39;,G\u0026#39;)$ is G-continuous (continuous in $\\tau(G)$) exactly when it is continuous as a map between the corresponding metric spaces. In particular, $G(x,y,z)$ itself is jointly continuous in all three variables.\nOne likewise defines G-Cauchy sequences: $(x_n)$ is G-Cauchy if for every $\\varepsilon\u0026gt;0$ there exists $N$ such that $$ G(x_n, x_m, x_\\ell) \u0026lt;\\varepsilon \\qquad \\forall n,m,\\ell \\geq N. $$ Mustafa–Sims show this is equivalent to $(x_n)$ being Cauchy in $(X,d_G)$. Hence $(X,G)$ is G-complete (every G-Cauchy sequence converges in $G$) if and only if the metric space $(X,d_G)$ is complete. Equivalently, G-completeness coincides with ordinary completeness. As a result, the usual consequences hold: closed subsets of a G-complete space are G-complete, and one obtains a Baire Category theorem in G-metric spaces (every complete G-metric space is non-meager).\nCompactness is similarly characterized: a G-metric space is compact if it is G-complete and totally bounded (equivalently, $(X,d_G)$ is compact). In fact, the following are equivalent:\n$(X,G)$ is compact G-metric. $(X,\\tau(G))$ is compact. $(X,d_G)$ is compact. Every sequence has a G-convergent subsequence. G-metric spaces have been extensively used in fixed-point theory, often yielding generalizations or new forms of contraction-type theorems.\nTheorem (G-Banach fixed point) [4]. Suppose $(X,G)$ is G-complete and $T:X\\to X$ satisfies a contractive condition in the G-metric, e.g., $$G(Tx,Ty,Tz)\\leq k\\,G(x,y,z), \\qquad 0\\le k\u0026lt;1,$$ for all $x,y,z\\in X$. Then $T$ has a unique fixed point in $X$.\n3.2 Example: G-metric not induced by a pairwise metric Let $X$ be the vertex set of a connected, edge-weighted, undirected graph. For three vertices $x,y,z\\in X$, define $$ G(x,y,z)=\\{\\text{length of a minimum Steiner tree that connects } \\{x,y,z\\}\\}. $$ Equivalently, the minimum total weight of a connected subgraph whose vertex set contains $\\{x,y,z\\}$.\nThis $G$ satisfies the axioms:\n$G(x,x,x)=0$. If $x\\neq y$, then $G(x,x,y)$ is the shortest-path distance between $x$ and $y$, hence $\u0026gt;0$. Symmetry is immediate (the definition depends only on the set $\\{x,y,z\\}$). A “rectangle-type” inequality holds: for any $a\\in X$, the union of Steiner trees for $\\{x,y,a\\},\\{x,a,z\\},\\{a,y,z\\}$ is a connected subgraph spanning $\\{x,y,z\\}$ whose total length is at most the sum of the three lengths. Since $G(x,y,z)$ is the minimum such length, $$ G(x,y,z)\\le G(x,y,a)\u0026#43;G(x,a,z)\u0026#43;G(a,y,z). $$ If a G-metric were composed of ordinary metrics, it would be determined by the three pairwise distances. Steiner length is not determined solely by $d(x,y),d(y,z),d(z,x)$—graph topology matters.\n3.3 Applications to operator-valued kernels Let $Z$ be a measurable space with measure $\\mu$ and define\n$K_z(x,y):=K(x,y,z)$ for $z\\in Z$. Fix a Hilbert space of functions of $z$ (e.g. $\\mathcal H=L^2(\\mu)$). Define an operator-valued kernel\n$$ \\mathbb{K}(x,y): \\mathcal{H}\\to\\mathcal{H}, \\qquad \\bigl[\\mathbb{K}(x,y)f\\bigr](z)=K(x,y,z)\\,f(z). $$ $\\mathbb{K}$ is a valid positive–definite (PD) operator-valued kernel iff $K_z$ is PD for $\\mu$-a.e. $z$. Then all the vector-valued RKHS machinery applies (representer theorem; kernel ridge/SVM for multi-output functions), with predictors of the form\n$$ F:X\\to\\mathcal H,\\qquad F(\\cdot)=\\sum_{i}\\mathbb K(\\cdot,x_i)\\,c_i,\\quad c_i\\in\\mathcal H . $$ In this multiplicative case,\n$$ [F(x)](z)=\\sum_i K(x,x_i,z)\\,c_i(z). $$ Positive-definiteness of $\\mathbb K$ means that for all $x_i\\in X$ and $c_i\\in\\mathcal H$,\n$$ \\sum_{i,j}\\big\\langle c_i,\\ \\mathbb K(x_i,x_j)c_j\\big\\rangle_{\\mathcal H} =\\int_Z \\sum_{i,j} c_i(z)\\,c_j(z)\\,K(x_i,x_j,z)\\,d\\mu(z)\\ \\ge 0 . $$ Before giving an example, recall the notion of a conditionally negative-definite (CND) kernel.\nDefinition. A symmetric kernel $K(x,y)$ is conditionally negative-definite (CND) if, for every finite set of points $x_1,\\dots,x_n$ and scalars $c_1,\\dots,c_n$ satisfying $\\sum_i c_i=0$, we have $$ \\sum_{i,j} c_i c_j K(x_i,x_j)\\le 0. $$ Consider a Hilbert-type metric $d(x,y)=\\|\\Phi(x)-\\Phi(y)\\|$ induced by a map $\\Phi:X\\to\\mathcal H$. Then define the G-metric\n$$ G(x,y,z):=\\tfrac12\\big(\\|\\Phi(x)-\\Phi(y)\\|^2\u0026#43;\\|\\Phi(y)-\\Phi(z)\\|^2\u0026#43;\\|\\Phi(z)-\\Phi(x)\\|^2\\big). $$ This satisfies the axioms, and for each fixed $z$ one can write\n$$ K_z(x,y)=G(x,y,z) = \\tfrac12\\|\\Phi(x)-\\Phi(y)\\|^2\u0026#43;u_z(x)\u0026#43;u_z(y), \\qquad u_z(x):=\\tfrac12\\|\\Phi(x)-\\Phi(z)\\|^2. $$ Since $\\|\\Phi(x)-\\Phi(y)\\|^2$ is CND (Schoenberg), and additive terms of the form $u_z(x)\u0026#43;u_z(y)$ vanish in the CND test (because $\\sum_i c_i=0$), it follows that $K_z$ is CND. Therefore, for any $t\u0026gt;0$,\n$$ \\bigl[\\overline{\\mathbb K}(x,y) f\\bigr](z) := e^{-t\\, G(x,y,z)}\\, f(z) $$ defines an operator-valued PD kernel.\n4. Possible future applications 4.1 Triplet-based representation learning In deep metric learning, we often use triplet loss: $$ L = \\max(0, d(f(x_a), f(x_p)) - d(f(x_a), f(x_n)) \u0026#43; \\alpha) $$ which explicitly considers triples (anchor, positive, negative).\nA G-metric could encode this triplet geometry directly, rather than defining it indirectly through pairwise distances. One could design a model $G_\\theta(x,y,z)$ that is symmetric and satisfies the G-metric axioms while learning consistent triplet relationships.\n4.2 Higher-order relational learning In graph representation learning or knowledge graphs, relationships are often ternary or higher, e.g., subject–predicate–object. A G-metric provides a natural way to model triadic similarity or hypergraph distances.\n4.3 Higher-order attention In higher-order attention, the focus is on interactions among triples or tuples of elements: $$ \\operatorname{Attn}(x_i, x_j, x_k) = f(G(x_i, x_j, x_k)) $$ Here, the G-metric measures triadic coherence among the three embeddings, not just pairwise similarity. This contrasts with common approaches to higher-order attention that rely on tensor products [5]. This captures contextual or relational dependencies (e.g., how three tokens jointly influence meaning). It’s useful for relational reasoning, scene understanding, or multi-agent interactions, where relationships are inherently non-pairwise.\nReferences [1] Agarwal, R.P., Karapınar, E., O’Regan, D., Roldán-López-de-Hierro, A.F. (2015). G-Metric Spaces. In: Fixed Point Theory in Metric Type Spaces. Springer, Cham. https://doi.org/10.1007/978-3-319-24082-4_3\n[2] Mustafa, Zead, and Brailey Sims. \u0026ldquo;A new approach to generalized metric spaces.\u0026rdquo; Journal of Nonlinear and Convex Analysis 7, no. 2 (2006): 289.\n[3] Jleli, Mohamed, and Bessem Samet. \u0026ldquo;Remarks on G-metric spaces and fixed point theorems.\u0026rdquo; Fixed Point Theory and Applications 2012, no. 1 (2012): 210.\n[4] Alghamdi, Maryam A., and Erdal Karapınar. \u0026ldquo;G-β-ψ-contractive type mappings in G-metric spaces.\u0026rdquo; Fixed Point Theory and Applications 2013, no. 1 (2013): 123.\n[5] Omranpour, Soroush, Guillaume Rabusseau, and Reihaneh Rabbany. \u0026ldquo;Higher Order Transformers: Efficient Attention Mechanism for Tensor Structured Data.\u0026rdquo; arXiv preprint arXiv:2412.02919 (2024).\n[6] https://github.com/MoonshotAI/Kimi-Linear/blob/master/tech_report.pdf\n[7] Peng, Hao, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah A. Smith, and Lingpeng Kong. \u0026ldquo;Random feature attention.\u0026rdquo; arXiv preprint arXiv:2103.02143 (2021).\n[8] Berg, Christian, Jens Peter Reus Christensen, and Paul Ressel. Harmonic analysis on semigroups: theory of positive definite and related functions. Vol. 100. New York: Springer, 1984.\n","permalink":"https://dani2442.github.io/posts/higher-order-attention/","summary":"\u003cp\u003e\u003cimg alt=\"alt text\" loading=\"lazy\" src=\"/posts/higher-order-attention/image.png\"\u003e\u003c/p\u003e\n\u003cp\u003eFor some time, I’ve been thinking about how to generalize self-attention mechanisms. Most existing attention mechanisms rely on pairwise similarities (dot products) between query and key vectors. However, higher-order relationships (involving triples or tuples of elements) could capture richer interactions. I then found that several people are already exploring this idea under the name “higher-order attention” \u003ca href=\"#references\"\u003e[5]\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eHowever, this approach comes with a performance cost. Traditional self-attention has a complexity of O(n^2), while higher-order attention is even more computationally expensive.\nIn this post, I’d like to share my perspective on this topic, connecting it with kernel methods and generalized metric spaces.\u003c/p\u003e","title":"Self-Attention, Kernel Methods and G-Metric Spaces"},{"content":"Initially, I wanted this post to focus solely on metrics in machine learning. However, the concept of metrics is far more universal, and it doesn’t make sense to treat it as an isolated problem. This is more of a philosophical post, the ultimate goal is to make you think and reflect.\nWe live surrounded by metrics: grades from teachers, performance reviews from employers, publication counts in academia, FLOPs in computing, ELO in chess, salary, IQ for intelligence, movie ratings on IMDB, book ratings on Goodreads, stars/reviews on Amazon, election results in democracies, F1 score in machine learning, GDP for countries, EBITDA in finance, likes/followers on Instagram, time spent on TikTok for content recommendation algorithms, etc.\nIn most cases, these metrics try to summarize a complex system with a single number, which is often inaccurate. For example, a well-behaved metric should be comparable, meaning:\nif A \u0026gt; B and B \u0026gt; C, then A \u0026gt; C\nIn mathematics, this property is called transitivity. However, in real life, this almost never holds; there are usually trade-offs.\nIn some cases, choosing the wrong metrics can lead to catastrophic failure (as we’ll see with some examples). I want to argue that choosing the right metrics is as important as solving the problem itself (just as asking the right questions is as important as answering them).\nEgregious examples of bad metrics People coined the term enshitification [8] to describe when a system becomes corrupted and degraded due to poor incentives; a phenomenon we’ve seen in many companies. The most famous example is Airbnb: it began as an alternative to hotels, where a host might share a room and both host and guest benefited for a modest price. Today, Airbnb prices are often similar to hotels, and the platform has contributed to gentrification (and toxic tourism) in many European cities [9].\nHere are several other examples of bad metrics and perverse incentives:\np-hacking in statistics: In statistics, it’s common to use p-values to describe confidence in the results of a test. The standard threshold for statistical significance is 0.05. This has led many researchers to aim for this arbitrary threshold at all costs. We’ve seen numerous retractions over the years, and countless others likely go unnoticed [2].\noverfitting in machine learning: Over the years, there have been many examples of data leaking—training the model using the test set split [3]. Also, when your only goal is to optimize a single metric, you may overfit to that task, even without using the test set, resulting in a model that is virtually useless.\nCooking the books in planned economies: In the 20th century, during the USSR under Khrushchev, company and factory leaders often manipulated production data or falsified results to appear more successful than they actually were. Because the Soviet economic system rewarded meeting targets rather than genuine efficiency or quality, managers frequently inflated numbers, hid waste, or produced useless goods just to report success. This, among other causes, contributed to the fall of the USSR.\nIncentives tied to management: CEOs were often paid bonuses based on quarterly earnings. As a result, many prioritized short-term wins over long-term health, using financial tricks that ultimately led to poor company performance.\nIn-class assessment: Although the main goal of school is learning, it’s usually measured through exams or projects. Many people practice the art of cheating. Another form of “cheating” is memorizing instead of understanding, which leads to poor learning and forgetting concepts after the exam.\nAcademia: The most important metric for researchers is the number of publications and citations. This has created a major misalignment of goals, and today, there’s a broken culture where publishing many and fast is the norm [[4]]. The following graphs show the evolution of the number of publications and average citations per paper [6, 7]. It seems we are producing so much information that no one reads\u0026hellip;\nPrivate healthcare: When the goal of a hospital is not to cure the patient but to make money, it can create perverse incentives: overcharging, prolonging treatment, and offering expensive solutions to simple or non-existent problems. I highly recommend this video [1]. The same happens in education systems.\nDemocracy: Political parties only care about the results of the next elections and not the long-term prosperity or planning of the city. Unaffordability of housing, short-term thinking, etc.\nThese are a few examples I could come up with, but I think anyone can see the pattern and come up with others.\nIf we look at the root problem, we see it’s a recurring issue throughout history—a byproduct of misaligned incentives. Sometimes, taking your eyes off the spreadsheet and reflecting can help solve these issues. One of the most famous quotes in investing highlights the importance of this concept:\n\u0026ldquo;Show me the incentive and I will show you the outcome\u0026rdquo;\n— Charlie Munger\nWe may think this is something new, but it’s far from the truth. For example, in the 15th century, pirates and marines were not taught how to swim to prevent them from running away in battle. There are cases of positive incentives as well: during ancient Rome, soldiers were paid directly from war spoils, and the same happened with Hannibal in the Second Punic War. In the Iberian Peninsula during the Reconquista, peasants were encouraged to settle these dangerous frontier lands by offering tax exemptions and other benefits. More recently, people thought the invention of the atomic bomb would bring about the end of the world; however, we saw the opposite—countries are disincentivized to attack others holding nuclear arsenals.\nThis last example also relates to the well-known Prisoner\u0026rsquo;s Dilemma in game theory, where we study optimal behaviour given a set of rules and incentives.\nWe’ve seen similar questions arise in AI safety and alignment, where people try to set the right incentives for AI to safeguard the common good. However, it remains an open question whether intelligence can, in principle, be combined with any kind of goal, allowing scientists to shape AI incentives. This is known as the Orthogonality Thesis [17]:\n\u0026ldquo;Are Intelligence and final goals (values, objectives) orthogonal?\u0026rdquo;\nThe most difficult part of Reinforcement Learning is usually setting the reward (incentive) correctly. This has been one of the field’s most important challenges, and people still struggle with how to set the reward during training. Setting a difficult environment hinders training, and choosing a single reward can lead to reward hacking [16].\nI think of an incentive like a second derivative: it’s a force that moves people’s interests in one direction. At first, people barely notice it, but the consequences become clear over time. Or, in mathematical terms, using the Taylor expansion:\n$$f(t) = f(0) \u0026#43; tf\u0026#39;(0) \u0026#43; \\frac{t^2}{2}f\u0026#39;\u0026#39;(0) \u0026#43; \\mathcal{O}(t^3)$$ When $t$ is small, higher-order terms are negligible; as $t$ grows, the influence of the second derivative and beyond can dominate.\nDesigning systems that remain robust over time is often difficult. In the long run, benchmarks and incentives can be gamed in unanticipated ways. Goodhart’s law captures this:\n\u0026ldquo;When a measure becomes a target, it ceases to be a good measure.\u0026rdquo;\nCorrosion is often slow and unnoticed until it’s significant. That said, I don’t think corruption is inevitable: well-designed metrics and adaptive governance can lead to stable equilibria. We need to be deliberate about which metrics we choose, not leave results to chance, and stay ready to adjust them when incentives produce unintended consequences.\nHow can we apply this to our daily life? Metrics influence everyone, including you. Ask yourself which artificial metrics shape your decisions, and whether they reflect your values.\nThe first thing that comes to mind is how each of us measures success: Are you guided by money and status, or by internal goals? It’s worth aligning external measures with personal values rather than blindly chasing others’ expectations.\nAnd what about machine learning? Even simple loss functions such as Mean Squared Error (MSE) and Mean Absolute Error (MAE) have different properties: MAE promotes sparsity and is more robust to outliers, while MSE penalizes large errors more heavily and is differentiable everywhere. Choosing a metric affects model behaviour.\nMore importantly, the dataset and benchmarks you use matter. For example, in computer vision, the ImageNet dataset helped spark the AI revolution; similarly, progress in NLP has been accelerated by high-quality, diverse benchmarks [10]. Good benchmarks encourage generalization rather than overfitting to a single metric. If one metric is insufficient, use multiple: confusion matrices, precision/recall, calibration, and task-specific measures provide a richer picture than a single scalar.\nChoosing the right metric in ML is very important, but having access to the right dataset and benchmark is crucial and often undervalued. The long-term impact of benchmarking is almost always unnoticed. The competition that sparked the modern AI revolution began with the ImageNet dataset. Recent advances in NLP are partly due to high-quality, well-curated, and diverse benchmarks. The abundance and variety of benchmarks in NLP completely overshadow other fields [10]. This has allowed teams to focus on generalization rather than optimizing a single benchmark. It almost seems trivial to say, but if a single metric is insufficient, use more than one. For example, use the confusion matrix rather than a single-number score.\nOne historical reason robotics underperformed compared to other fields was its poor benchmarks. However, we’ve seen a lot of progress lately with new environments and the push for RL [11, 14, 15]. In AI for materials and drug discovery, we are seeing a golden age as well [12, 13].\nReferences [1] \u0026ldquo;I Was An MIT Educated Neurosurgeon Now I\u0026rsquo;m Unemployed And Alone In The Mountains How Did I Get Here?\u0026rdquo; https://www.youtube.com/watch?v=25LUF8GmbFU\u0026amp;t\n[2] p-hacking: https://en.wikipedia.org/wiki/Data_dredging\n[3] data leaking: https://en.wikipedia.org/wiki/Leakage_(machine_learning)\n[4] \u0026ldquo;Scientific publishing is broken\u0026rdquo; https://www.theguardian.com/science/2025/jul/13/quality-of-scientific-papers-questioned-as-academics-overwhelmed-by-the-millions-published\n[5] \u0026ldquo;The misalignment of incentives in academic publishing and implications for journal reform\u0026rdquo; https://www.pnas.org/doi/10.1073/pnas.2401231121\n[6] Monteiro, Maria \u0026amp; Séneca, Joana \u0026amp; Magalhães, Catarina. (2014). The History of Aerobic Ammonia Oxidizers: from the First Discoveries to Today. Journal of microbiology (Seoul, Korea). 52. 537-47. 10.1007/s12275-014-4114-0.\n[7] Mike Thelwall, Pardeep Sud; Scopus 1900–2020: Growth in articles, abstracts, countries, fields, and journals. Quantitative Science Studies 2022; 3 (1): 37–50. doi: https://doi.org/10.1162/qss_a_00177\n[8] https://en.wikipedia.org/wiki/Enshittification\n[9] Rabiei-Dastjerdi, Hamidreza, Gavin McArdle, and William Hynes. \u0026ldquo;Which came first, the gentrification or the Airbnb? Identifying spatial patterns of neighbourhood change using Airbnb data.\u0026rdquo; Habitat International 125 (2022): 102582.\n[10] https://main--dasarpai.netlify.app/dsblog/nlp-benchmarks1/\n[11] https://github.com/newton-physics/newton\n[12] https://ai.meta.com/research/publications/open-molecular-crystals-2025-omc25-dataset-and-models/\n[13] https://moleculenet.org/datasets-1\n[14] https://gymnasium.farama.org/\n[15] Kaup, Michael, Cornelius Wolff, Hyerim Hwang, Julius Mayer, and Elia Bruni. \u0026ldquo;A review of nine physics engines for reinforcement learning research.\u0026rdquo; arXiv preprint arXiv:2407.08590 (2024).\n[16] https://en.wikipedia.org/wiki/Reward_hacking\n[17] https://www.lesswrong.com/w/orthogonality-thesis\n","permalink":"https://dani2442.github.io/essays/metrics/","summary":"\u003cp\u003eInitially, I wanted this post to focus solely on metrics in machine learning. However, the concept of metrics is far more universal, and it doesn’t make sense to treat it as an isolated problem. This is more of a philosophical post, the ultimate goal is to make you think and reflect.\u003c/p\u003e\n\u003cp\u003eWe live surrounded by metrics: grades from teachers, performance reviews from employers, publication counts in academia, FLOPs in computing, ELO in chess, salary, IQ for intelligence, movie ratings on IMDB, book ratings on Goodreads, stars/reviews on Amazon, election results in democracies, F1 score in machine learning, GDP for countries, EBITDA in finance, likes/followers on Instagram, time spent on TikTok for content recommendation algorithms, etc.\u003c/p\u003e","title":"The Importance of (Good) Metrics"},{"content":"Gaussian Process Regression is one of the most elegant and theoretically rich algorithms in machine learning. With this post, I want to celebrate the mathematical beauty underlying Gaussian Processes. I will divide this post into two sections: theory and practice, accompanied by code examples.\nOne of the key advantages of Gaussian Processes compared to Deep Learning methods is that they inherently provide interpretability (through confidence intervals and uncertainty estimation). They also offer excellent extrapolation properties, as we will see, and a way to incorporate knowledge about the structure of the data into the model. However, these benefits come at a cost. The algorithm has a wide variety of hyperparameters that are difficult to configure; for instance, kernel selection alone is challenging. Understanding and having a good intuition for the inner workings of this algorithm (and the data) is key to making the most of it.\n1. Theory For context, a Gaussian process is a stochastic process that generalizes the multivariate normal distribution to (potentially) infinite index sets. Wiener studied related objects in 1923 [3], and Krige and others later popularized Gaussian process for regression.\nFor completeness, here is a standard definition of a Gaussian process:\nA process $X(t)$, $t\\in I$, is Gaussian if every finite collection $(X(t_1), \\dots, X(t_n))$ has a multivariate normal distribution for any $t_1,\\dots,t_n\\in I$, $n\\in\\mathbb{N}$.\nConsequently, a Gaussian process is fully specified by its mean function $\\mu(t)=\\mathbb{E}[X(t)]$ and covariance function $\\operatorname{Cov}(X(t),X(s)) = \\mathbb{E}[(X(t)-\\mu(t))(X(s)-\\mu(s))]$.\nGaussian processes belong to the family of kernel methods and are studied using the mathematical space: Reproducing Kernel Hilbert Spaces (RKHS). Kernels let us inject prior assumptions about function smoothness, periodicity, linear trends, etc., and often enable powerful nonparametric models with manageable computational cost for modest dataset sizes. Other kernel methods you may know include Kernel PCA (a nonlinear variant of PCA) and Support Vector Machines (SVMs). Even the self-attention can be explained in terms of kernels [4].\nGaussian process regression aims to infer a function $f$ mapping inputs $x$ to outputs $y$, i.e. $y = f(x)$ and we assume that this $f$ is a Gaussian Process with a given covariance. We set up the problem as follows:\nTraining inputs: $X=\\{x_i\\}_{i=1}^n$. Latent function values at training inputs: $f = [f(x_1),\\dots,f(x_n)]^\\top$. Likelihood, $p(y|f)$, is Gaussian: $$y_i = f(x_i) \u0026#43; \\varepsilon_i \\text{, with noise }\\varepsilon_i\\stackrel{\\text{iid}}{\\sim}\\mathcal N(0,\\sigma^2)$$ Prior on $f(\\cdot)\\sim \\mathcal{GP}$: a Gaussian process with mean $m(\\cdot)$ and covariance kernel $k(\\cdot,\\cdot)$. For now assume $m(\\cdot)\\equiv 0$; we’ll generalize later. Test input: $x_*$ with latent value $f_* = f(x_*)$. Define covariance matrices/vectors:\n$K \\in\\mathbb R^{n\\times n}$ with $K_{ij}=k(x_i,x_j)$. $k_* \\in\\mathbb R^n$ with $(k_*)_i = k(x_i,x_*) = \\operatorname{Cov}(y_i,f_*)$. Note noise on $y$ is independent of $f_*$. $k_{**} = k(x_*,x_*) = \\operatorname{Var}(f_*)$. Covariance of the observed vector $y$: $\\mathrm{Cov}(y,y)=K\u0026#43;\\sigma^2 I \\equiv C_{yy}$. Some commonly used kernels (covariance functions) in GPR are: $$ \\begin{aligned} k(x,x\u0026#39;) \u0026amp;= \\exp\\left\\{-\\frac{1}{2\\ell^2}\\|x-x\u0026#39;\\|^2\\right\\} \\quad\\text{(Radial Basis Function / RBF)}\\\\ k(x,x\u0026#39;) \u0026amp;= (\\gamma x^\\top x\u0026#39; \u0026#43; r)^d \\quad\\text{(Polynomial kernel)}\\\\ k(x,x\u0026#39;) \u0026amp;= \\frac{x^\\top x\u0026#39;}{\\|x\\|\\cdot \\|x\u0026#39;\\|} \\quad\\text{(Cosine similarity kernel)}\\\\ k(x,x\u0026#39;) \u0026amp;= \\exp\\left\\{-\\frac{\\sin^2\\big(\\pi|x-x\u0026#39;|/p\\big)}{\\ell^2}\\right\\} \\quad\\text{(Periodic kernel)} \\end{aligned} $$ In the following sections we will see three different ways to understand the mathematical formulation of GPR.\n(Option 1) Best Linear Unbiased Estimator (BLUE) Our goal is to find the linear estimator $\\hat f_* := w^\\top y$ that minimizes mean squared error (MSE): $$ \\mathrm{MSE}(w)=\\mathbb E\\big[(f_* - w^\\top y)^2\\big]. $$ Expand the MSE (expectations are over the joint prior of $f$ and the noise):\n$$ \\begin{aligned} \\mathrm{MSE}(w) \u0026amp;= \\mathbb E[f_*^2] - 2\\mathbb E[f_* w^\\top y] \u0026#43; \\mathbb E[w^\\top y y^\\top w] \\\\ \u0026amp;= k_{**} - 2 w^\\top \\mathbb E[y f_*] \u0026#43; w^\\top \\mathbb E[y y^\\top] w. \\end{aligned} $$ But $\\mathbb E[y f_*] = k_*$ and $\\mathbb E[yy^\\top] = C_{yy} = K \u0026#43; \\sigma^2 I$. So\n$$ \\mathrm{MSE}(w) = k_{**} - 2 w^\\top k_* \u0026#43; w^\\top C_{yy} w. $$ This is a quadratic function of $w$. Differentiate w.r.t. $w$ and set gradient to zero:\n$$ \\frac{\\partial}{\\partial w}\\mathrm{MSE}(w) = -2 k_* \u0026#43; 2 C_{yy} w = 0 \\quad\\Rightarrow\\quad C_{yy} w^* = k_*. $$ Assuming $C_{yy}$ is invertible (typical if $\\sigma^2\u0026gt;0$ or $K$ is full rank),\n$$ \\boxed{w^* = C_{yy}^{-1} k_* = (K\u0026#43;\\sigma^2 I)^{-1} k_*.} $$ Thus the best linear estimator is\n$$ \\boxed{\\hat f_* = {w^*}^\\top y = k_*^\\top (K\u0026#43;\\sigma^2 I)^{-1} y.} $$ This is exactly the usual GP posterior mean (for a zero prior mean).\nThe minimal MSE (plug $w^*$ back in) is\n$$ \\begin{aligned} \\mathrm{MSE}_{\\min} \u0026amp;= k_{**} - {k_*}^\\top C_{yy}^{-1} k_*. \\end{aligned} $$ This equals the GP posterior variance at $x_*$. Thus the minimal achievable MSE by any linear estimator equals the posterior variance.\n(Option 2) Orthogonality interpretation An equivalent derivation uses the orthogonality principle (linear projection): the error $e = f_* - w^\\top y$ of the best linear estimator must be uncorrelated with the data used in the estimator:\n$$ \\mathbb E[e y] = 0 \\quad\\Rightarrow\\quad \\mathbb E[(f_* - w^\\top y) y] = 0. $$ Thus $\\mathbb E[f_* y] - \\mathbb E[y y^\\top] w = 0$, i.e. $k_* - C_{yy} w = 0$, giving the same solution $w = C_{yy}^{-1} k_*$. So the GP predictor is the linear projection of $f_*$ onto the subspace spanned by the observed $y$.\n(Option 3) Joint Gaussian conditioning The standard GP route uses the fact that the joint distribution of $y$ and $f_*$ is Gaussian:\n$$\\begin{bmatrix} y \\\\ f_* \\end{bmatrix} \\sim \\mathcal N\\left(0, \\begin{bmatrix} C_{yy} \u0026amp; k_* \\\\ k_*^\\top \u0026amp; k_{**} \\end{bmatrix} \\right).$$ This follows because any finite collection of evaluations of a Gaussian process is jointly Gaussian. The conditional distribution of a joint Gaussian is $$ f_* \\mid y \\sim \\mathcal N\\big(k_*^\\top C_{yy}^{-1} y , k_{**} - k_*^\\top C_{yy}^{-1} k_*\\big) \\tag{Posterior} $$ Therefore, the conditional expectation and variance for a multivariate Gaussian yields $$\\mathbb E[f_* \\mid y] = k_*^\\top C_{yy}^{-1} y, \\qquad \\operatorname{Var}(f_* \\mid y) = k_{**} - k_*^\\top C_{yy}^{-1} k_*.$$ So the posterior mean equals the linear estimator we derived by minimizing MSE. For Gaussian processes, the posterior mean is both the Bayes (minimum mean squared error over all measurable estimators) and, as we\u0026rsquo;ve shown, the best linear estimator.\nImportant conceptual note: for a Gaussian prior the optimal estimator under squared loss is the conditional mean, and because of Gaussianity that conditional mean is linear in $y$. For non-Gaussian priors the best linear estimator (projection) is still $k_*^\\top C_{yy}^{-1} y$ but it will not, in general, equal the conditional mean.\n1.2 Nonzero mean function If the GP has mean function $m(\\cdot)$, write $m_X = [m(x_1),\\dots,m(x_n)]^\\top$ and $m_* = m(x_*)$. The model is $y = m_X \u0026#43; (f - m_X) \u0026#43; \\varepsilon$. Working with centered quantities $y - m_X$ and $f_* - m_*$ (which have zero mean) gives the estimator\n$$ \\mathbb E[f_* \\mid y] = m_* \u0026#43; k_*^\\top (K\u0026#43;\\sigma^2 I)^{-1} (y - m_X). $$ In practice you subtract the prior mean from the observations, apply the same weight matrix, and add the prior mean back at the test point.\n1.3 Complexity analysis Training complexity is $\\mathcal O(n^3)$ due to the inversion of the the $n\\times n$ matrix $C_{yy}$. Prediction: Mean: $\\mathcal O(n)$ per test point after precomputing $C_{yy}^{-1}y$ . Covariance: $\\mathcal O(n^2)$ per test point if you compute full predictive covariances; computing only marginal variances is cheaper. For larger datasets, approximate or structured methods (sparse GPs, inducing points, kernel approximations, or GPU-accelerated routines) can improve the speed drastically.\n2. Practice and Code For this implementation we will use the RBF Kernel: $$ k(x,x\u0026#39;) = \\exp\\left(-\\frac{1}{2\\ell^2}\\|x-x\u0026#39;\\|^2\\right). $$ This kernel is widely used because it is local and universal (see Corollary 4.58 [1]). \u0026ldquo;Universality\u0026rdquo; here means the associated RKHS is rich enough to approximate a wide class of continuous functions on compact domains under suitable conditions.\nThe implementation is straightforward from the previous section.\ndef rbf_kernel(x1, x2, sigma=1.0): y = x1.reshape(-1, 1)-x2.reshape(1,-1) return np.exp(-0.5 / sigma**2 * np.square(y)) class GaussianProcess: def __init__(self, kernel=rbf_kernel, noise=1e-3): self.kernel = kernel self.noise = noise def fit(self, X_train, y_train): self.X_train = X_train self.y_train = y_train self.K = self.kernel(X_train, X_train) + self.noise * np.eye(len(X_train)) self.K_inv = np.linalg.inv(self.K) def predict(self, X_s): K_s = self.kernel(self.X_train, X_s) K_ss = self.kernel(X_s, X_s) mu_s = K_s.T.dot(self.K_inv).dot(self.y_train) cov_s = K_ss - K_s.T.dot(self.K_inv).dot(K_s) return mu_s, cov_s Now we use a very simple dataset, fit the Gaussian Process and plot the results.\n# Training data (noisy observations) X_train = np.array([[-4], [-3], [-2], [-1], [1]]).astype(float) y_train = np.sin(X_train).ravel() + 0.1 * np.random.randn(X_train.shape[0]) # Test points X_test = np.linspace(-5, 5, 100).reshape(-1, 1) # Fit GP and predict gp = GaussianProcess() gp.fit(X_train, y_train) mu_s, cov_s = gp.predict(X_test) std_s = np.sqrt(np.diag(cov_s)) # Plot plt.figure(figsize=(10, 6)) plt.plot(X_train, y_train, \u0026#39;ro\u0026#39;, label=\u0026#39;Train data\u0026#39;) plt.plot(X_test, mu_s, \u0026#39;b\u0026#39;, label=\u0026#39;Prediction mean\u0026#39;) plt.fill_between(X_test.ravel(), mu_s.ravel() - 2 * std_s, mu_s.ravel() + 2 * std_s, alpha=0.2, label=\u0026#39;Confidence interval\u0026#39;) plt.legend() plt.title(\u0026#34;Gaussian Process Regression\u0026#34;) plt.show() Kernel selection and combination If the underlying signal is periodic, a periodic kernel is a natural choice. Encoding useful structure in the kernel typically improves extrapolation and predictive performance. For example, the periodic kernel:\ndef periodic_kernel(x1, x2, period=2*np.pi, l=1.5, sigma=2.0): y = x1.reshape(-1, 1) - x2.reshape(1, -1) return sigma**2 * np.exp(-2 * (np.sin(np.pi * y / period)**2) / l**2) Here we are interested in extrapolation outside the data points. Compared to the RBF kernel, a periodic kernel often generalizes better when the signal is truly periodic.\nNow, we will explore a more complex example. Assume that our data has a stationary component and also a trend. This problem has been extensively studied in time series and there are already models that can perform this task (see the ARMA family [8]). However, using the Bayesian approach given by Gaussian Processes can lead to similar performance with the added benefit of interpretability.\nFor this case we will use some of the most interesting properties of kernels:\nKernels can be composed using addition and multiplication.\nIn this case we are going to combine the periodic kernel with the linear using the sum.\ndef linear_kernel(x1, x2, c=0.0, sigma=1.0): return sigma**2 * (x1.reshape(-1, 1) - c) * (x2.reshape(1, -1) - c) X_train = (-10 + 20 * np.random.rand(20, 1)).astype(float) y_train = np.sin(X_train).ravel() + X_train.ravel() / 2 + 0.2 * np.random.randn(20) # Test points X_test = np.linspace(-12, 10, 100).reshape(-1, 1) # Fit GP and predict addition_kernel = lambda x1, x2: periodic_kernel(x1, x2) + linear_kernel(x1, x2) gp = GaussianProcess(addition_kernel, noise=0.1) gp.fit(X_train, y_train) mu_s, cov_s = gp.predict(X_test) std_s = np.sqrt(np.diag(cov_s)) And we obtain the following result:\nReferences \u0026amp; Supplementary Material [1] Steinwart, I. and Christmann, A., 2008. Support vector machines. Springer Science \u0026amp; Business Media. [2] Williams, Christopher KI, and Carl Edward Rasmussen. Gaussian processes for machine learning. Vol. 2, no. 3. Cambridge, MA: MIT press, 2006. [3] https://djalil.chafai.net/docs/M2/history-brownian-motion/Wiener%20-%201923.pdf [4] https://x.com/docmilanfar/status/1974328880564752525\n[5] Visual explanation of Gaussian Processes: https://distill.pub/2019/visual-exploration-gaussian-processes/ [6] Scikit code examples (optimized): https://scikit-learn.org/stable/modules/gaussian_process.html [7] GPytorch (Efficient GP library using torch as backend): https://github.com/cornellius-gp/gpytorch [8] Python package for time-series: https://www.statsmodels.org/stable/index.html\nAppendix (A) Gaussian Process Classification As we saw in section 1, in the case of classification there is no closed form formula to calculate it. In GP classification, the outputs $y_i \\in \\{0,1\\}$, so a Gaussian likelihood doesn’t make sense. Instead, we assume the likelihood is given by a sigmoid function (or logistic): $$ p(y_i=1 \\mid f_i) \\equiv \\sigma(f_i)= \\frac{1}{1 \u0026#43; e^{-f_i}} \\tag{Likelihood} $$ The joint model is: $$ p(y, f) = p(y \\mid f) p(f) $$ where the prior is given by the Gaussian process, as in previous section, $$ p(f) = \\mathcal{N}(f \\mid 0, K) \\tag{Prior} $$ and $$ p(y \\mid f) = \\sigma(f)^{y} [1 - \\sigma(f)]^{1 - y} $$ The problem is that the posterior is intractable. We want: $$ p(f \\mid y) = \\frac{p(y \\mid f) p(f)}{p(y)} \\tag{Posterior} $$ but since $p(y \\mid f) $ is non-Gaussian, the posterior is not Gaussian and $p(y) = \\int p(y \\mid f) p(f) df $ is intractable. The solution is to approximate the posterior by a Gaussian distribution. The following table compares both methods.\nAspect GP Regression GP Classification Likelihood Gaussian Bernoulli via logistic/probit Posterior Analytic Approximate (Laplace/EP) Output Continuous mean, variance Class probabilities Training Linear algebra only Iterative (Newton-Raphson) We will use the Laplace approximation to approximate the intractable distribution with a Gaussian centered at its mode.\nThe idea: $$p(f \\mid y) \\approx \\mathcal{N}(f \\mid \\hat{f}, \\Sigma)$$ where:\n$ \\hat{f} = \\arg\\max_{f} \\log p(f \\mid \\mathbf{y}) $ is the mode, $ \\Sigma^{-1} = - \\nabla^2_{f} \\log p(f \\mid \\mathbf{y}) \\big|_{f = \\hat{f}} $ is given by the curvature / Hessian at the mode. Why this makes sense?\nThe Laplace method relies on a second-order Taylor expansion of the log-posterior around its mode.\n(Step 1) Approximate the posterior.\nLet: $$ \\psi(f) = \\log p(f \\mid \\mathbf{y}) $$ Then we can approximate $\\psi(f)$ near its maximum $\\hat{f}$ by a quadratic function (the first derivative is zero):\n$$ \\psi(f) \\approx \\psi(\\hat{f}) - \\frac{1}{2} (f - \\hat{f})^T A (f - \\hat{f}) $$ where: $A = - \\nabla^2 \\psi(f) \\big|_{\\hat{f}}$. Exponentiating both sides gives:\n$$ p(f \\mid \\mathbf{y}) \\propto e^{\\psi(f)} \\approx e^{\\psi(\\hat{f})} \\exp\\left( -\\frac{1}{2} (f - \\hat{f})^T A (f - \\hat{f}) \\right) $$ That’s just a Gaussian density: $$ p(f \\mid \\mathbf{y}) \\approx \\mathcal{N}(f \\mid \\hat{f}, A^{-1}) $$ Let us apply this to GP classification. We start from the log posterior:\n$$ \\log p(f \\mid \\mathbf{y}) = \\log p(\\mathbf{y} \\mid f) \u0026#43; \\log p(f) \u0026#43; \\text{const} $$ Applying log to the (Prior), we obtain $$ \\log p(f) = -\\frac{1}{2} f^T K^{-1} f - \\frac{1}{2} \\log |K| \u0026#43; \\text{const} $$ And likewise, to the likelihood: $$ \\log p(\\mathbf{y} \\mid f) = \\sum_i \\big[ y_i \\log \\sigma(f_i) \u0026#43; (1 - y_i)\\log (1 - \\sigma(f_i)) \\big] $$ (Step 2) Now we will find the mode, denoted by $\\hat{f}$.\nWe maximize $\\log p(f \\mid \\mathbf{y})$ using Newton-Raphson:\n$$ \\nabla \\log p(f \\mid \\mathbf{y}) = \\nabla \\log p(\\mathbf{y} \\mid f) - K^{-1} f $$ $$ \\nabla^2 \\log p(f \\mid \\mathbf{y}) = -W - K^{-1} $$ where $$ W = -\\nabla^2 \\log p(\\mathbf{y} \\mid f) = \\text{diag}(\\pi_i (1 - \\pi_i)), \\quad \\pi_i = \\sigma(f_i) $$ We iteratively update: $$ f_{\\text{new}} = f - (K^{-1} \u0026#43; W)^{-1} (\\nabla \\log p(\\mathbf{y} \\mid f) - K^{-1} f) $$ until convergence to $\\hat{f}$.\nFor a new input $x_*$, the joint prior is: $$\\begin{bmatrix} \\mathbf{f} \\\\ f_* \\end{bmatrix} \\sim \\mathcal{N}\\left( 0, \\begin{bmatrix} K \u0026amp; k_* \\\\ k_*^T \u0026amp; k_{**} \\end{bmatrix} \\right) $$ Then, using the joint-conditional argument, the probability of $\\mathbf{f}$ under the approximate Gaussian posterior gives: $$ p(f_* \\mid \\mathbf{y}) \\approx \\mathcal{N}(f_* \\mid m_*, v_*) $$ with: $$ m_* = k_*^T K^{-1} \\hat{\\mathbf{f}}, \\quad v_* = k_{**} - k_*^T (K^{-1} - K^{-1}\\Sigma K^{-1}) k_* $$ Then, the predictive class probability is: $$ p(y_* = 1 \\mid \\mathbf{y}) = \\int \\sigma(f_*) p(f_* \\mid \\mathbf{y}) , df_* \\approx \\Phi\\left( \\frac{m_*}{\\sqrt{1 \u0026#43; v_*}} \\right) $$ Now, let us see the results in the code.\nclass GaussianProcessClassifier: def __init__(self, kernel, noise=1e-6): self.kernel = kernel self.noise = noise def fit(self, X_train, y_train, max_iter=100): self.X_train = X_train self.y_train = y_train.reshape(-1) self.K = self.kernel(X_train, X_train) + self.noise * np.eye(len(X_train)) f = np.zeros_like(self.y_train) # Newton-Raphson iteration for Laplace approximation for _ in range(max_iter): pi = 1 / (1 + np.exp(-f)) # logistic sigmoid W = np.diag(pi * (1 - pi)) B = np.eye(len(X_train)) + np.sqrt(W) @ self.K @ np.sqrt(W) B_inv = np.linalg.inv(B) sqrt_W = np.sqrt(W) b = sqrt_W @ B_inv @ sqrt_W @ self.K @ (self.y_train - pi) f = self.K @ b # update latent f self.f = f self.W = W self.pi = pi self.B_inv = B_inv # store for prediction def predict_proba(self, X_s): K_s = self.kernel(self.X_train, X_s) K_ss = self.kernel(X_s, X_s) # Predictive mean and variance for latent function sqrt_W = np.sqrt(self.W) v = sqrt_W @ K_s f_mean = K_s.T.dot(self.y_train - self.pi) f_var = np.diag(K_ss - v.T @ self.B_inv @ v) # Approximate class probability via probit approximation probs = norm.cdf(f_mean / np.sqrt(1 + f_var)) return probs And now let us test the result with a dataset that classifies 1 if the data is positive, otherwise it classifies negative.\nnp.random.seed(42) X_train = np.linspace(-5, 5, 15).reshape(-1, 1) y_train = (np.sin(X_train) \u0026gt; 0).astype(int) # binary labels (0/1) X_test = np.linspace(-6, 10, 200).reshape(-1, 1) gp_clf = GaussianProcessClassifier(rbf_kernel) gp_clf.fit(X_train, y_train) probs = gp_clf.predict_proba(X_test) preds = gp_clf.predict(X_test) # Plot plt.figure(figsize=(10, 6)) plt.scatter(X_train, y_train, c=y_train, cmap=\u0026#39;coolwarm\u0026#39;, s=100, edgecolors=\u0026#39;k\u0026#39;, label=\u0026#39;Training data\u0026#39;, zorder=3) plt.plot(X_test, probs[:], \u0026#39;b-\u0026#39;, label=\u0026#39;P(y=1)\u0026#39;, linewidth=2) plt.plot(X_test, preds, \u0026#39;r--\u0026#39;, label=\u0026#39;Predictions\u0026#39;, linewidth=2, alpha=0.7) plt.xlabel(\u0026#39;X\u0026#39;, fontsize=12) plt.ylabel(\u0026#39;Value\u0026#39;, fontsize=12) plt.legend() plt.grid(alpha=0.3) plt.title(\u0026#39;Gaussian Process Classification\u0026#39;) plt.tight_layout() plt.show() Note: in this example a periodic kernel would be more appropriate than a plain RBF kernel. I leave trying that variant as an exercise to the reader heheheh.\n(B) Functional Gaussian Processes So far, inputs have been finite-dimensional vectors. The extension to the Euclidean space ($x\\in\\mathbb{R}^n$, $y\\in\\mathbb{R}^m$) is straightforward. But we can go further:\nWhat if each input is itself a function?\nThis question lies at the heart of Functional Data Analysis (FDA). We can define kernels on function spaces and apply GP machinery to map functions to scalars or to other functions. For simplicity we focus on scalar outputs $y\\in\\mathbb{R}$.\nFor example, suppose inputs are Brownian-motion trajectories and the goal is to predict the endpoint $x(T)$ given the trajectory on [0,T]. One can use a functional RBF kernel based on the L^2 distance between functions: $$ k(x,x\u0026#39;) = \\exp\\left(-\\frac{1}{2\\sigma^2}\\|x-x\u0026#39;\\|_{L^2}^2\\right) \\tag{Functional RBF} $$ and proceed with the standard GP workflow. Example code follows.\ndef rbf_kernel(x1, x2, dt, sigma=1.0): y = np.square(x1[:, None, :] - x2[None, :, :]) integral = np.sum(dt * (y[...,:-1] + y[...,1:]) * 0.5, axis=-1) return np.exp(-1 / (2*sigma**2) * integral) Generate the dataset and use the Gaussian Process to fit and predict. Notice that we use the same GaussianProcess class as before.\nN = 100 num = 10 dt = 1/(N-1) t = np.linspace(0,1,N) def generate_brownian(num, N): incs = np.sqrt(dt) * np.random.randn(num, N-1) return np.concatenate((np.zeros((num,1)), np.cumsum(incs, axis=1)), axis=1) X_train = generate_brownian(num, N) X_test = generate_brownian(3, N) # Training data (noisy observations) y_train = X_train[:,-1] + 0.1 * np.random.randn(X_train.shape[0]) y_test = X_test[:,-1] # Fit GP and predict gp = GaussianProcess(rbf_kernel, dt) gp.fit(X_train, y_train) mu_s, cov_s = gp.predict(X_test) std_s = np.sqrt(np.diag(cov_s)) We obtain the following results for the the 3 test samples.\nThe results are quite good despite only using 10 training samples.\n(C) Stable Gaussian Process with Cholesky decomposition One practical aspect we have skipped is numerical stability when inverting $C_{yy}$. Direct inversion is not recommended. A better approach is to use the Cholesky decomposition. Note that $C_{yy} = K \u0026#43; \\sigma^2 I$ is positive definite when $\\sigma^2\u0026gt;0$ (since $K$ is positive semi-definite). Hence we can write $$ C_{yy} = L L^\\top $$ (1) Predictive mean: $$ \\mathbb{E}[f_* \\mid y] = k_*^\\top C_{yy}^{-1} y $$ Instead of computing $C_{yy}^{-1}$, we solve the equivalent linear system using the Cholesky factors:\nSolve $L v = y$ Solve $L^\\top \\alpha = v$ Now $\\alpha = C_{yy}^{-1} y$. So the mean is: $$ \\mathbb E[f_* \\mid y] = k_*^\\top \\alpha $$ (2) Predictive covariance\nThe predictive covariance is:\n$$ \\operatorname{Var}(f_* \\mid y) = k_{**} - k_*^\\top C_{yy}^{-1} k_* $$ Again, avoid $C_{yy}^{-1}$. Instead compute:\nSolve $L v = k_*$. Then $k_*^\\top C_{yy}^{-1} k_* = v^\\top v$. So: $$ \\operatorname{Var}(f_* \\mid y) = k_{**} - v^\\top v $$ The result looks as follows\nclass GaussianProcess: def __init__(self, kernel, noise=1e-3): self.kernel = kernel self.noise = noise def fit(self, X_train, y_train): self.X_train = X_train self.y_train = y_train K = self.kernel(X_train, X_train) + self.noise * np.eye(len(X_train)) self.L = np.linalg.cholesky(K) def predict(self, X_s): K_s = self.kernel(self.X_train, X_s) K_ss = self.kernel(X_s, X_s) v = np.linalg.solve(self.L, self.y_train) # First solve L v = y alpha = np.linalg.solve(self.L.T, v) # Then solve L^T alpha = v mu_s = K_s.T.dot(alpha) v = np.linalg.solve(self.L, K_s) # Solve for L\\K_s cov_s = K_ss - v.T.dot(v) return mu_s, cov_s (D) Relation between Gaussian Process Regression and Maximum Mean Discrepancy (MMD) Let $\\mathcal H$ be the RKHS with reproducing kernel $k(\\cdot,\\cdot)$. For a location $x$ denote the representer $k_x := k(\\cdot,x)\\in\\mathcal H$. For two (signed) measures $\\mu,\\nu$ the squared MMD (kernel mean embedding distance) is $$ \\mathrm{MMD}^2(\\mu,\\nu) = \\|\\mu-\\nu\\|_{\\mathcal H}^2 = \\| m_\\mu - m_\\nu\\|_{\\mathcal H}^2, $$ where $m_\\mu=\\int k(\\cdot,x),d\\mu(x)$ is the kernel mean map. For discrete/signed measures this reduces to kernel sums.\nTake the target distribution $\\mu=\\delta_{x_*}$ (a unit mass at the test point) so $m_\\mu = k_{x_*}$. Take a weighted empirical signed measure $\\nu_w=\\sum_{i=1}^n w_i \\delta_{x_i}$ so $m_{\\nu_w}=\\sum_i w_i k_{x_i}$. Then $$ \\mathrm{MMD}^2\\big(\\delta_{x_*},\\nu_w\\big) = \\|k_{x_*} - \\sum_{i} w_i k_{x_i}\\|_{\\mathcal H}^2. $$ Use reproducing-kernel identities to expand that squared RKHS norm: $$ \\begin{aligned} \\|k_{x_*} - \\sum_i w_i k_{x_i}\\|_{\\mathcal H}^2 \u0026amp;= \\langle k_{x_*},k_{x_*}\\rangle - 2\\sum_i w_i\\langle k_{x_*},k_{x_i}\\rangle \\sum_{i,j} w_i w_j \\langle k_{x_i},k_{x_j}\\rangle \\\\ \u0026amp;= k_{**} - 2 w^\\top k_* \u0026#43; w^\\top K w. \\end{aligned} $$ This is the MMD between $\\delta_{x_*}$ and the weighted empirical measure $\\nu_w$.\nRelation to the MSE quadratic form\nRecall from the linear-estimation derivation (zero-mean case) that the MSE of the linear estimator $\\hat f_* = w^\\top y$ equals $$ \\mathrm{MSE}(w) = \\mathbb E\\big[(f_* - w^\\top y)^2\\big] = k_{**} - 2 w^\\top k_* \u0026#43; w^\\top (K \u0026#43; \\sigma^2 I) w. $$ Compare this with the MMD expansion above: the difference is exactly the noise term $\\sigma^2 w^\\top w = \\sigma^2 \\|w\\|_2^2$. Thus we can write $$ \\boxed{\\mathrm{MSE}(w) = \\underbrace{\\mathrm{MMD}^2 (\\delta_{x_*},\\nu_w )}_{\\text{deterministic RKHS discrepancy}}\u0026#43; \\sigma^2 \\|w\\|_2^2.} $$ So minimizing mean-squared error over linear estimators is equivalent to choosing weights $w$ that minimize the RKHS discrepancy (MMD) between the target at $x_*$ and the weighted training points, regularized by the squared $\\ell_2$-norm of the weights weighted by the noise variance.\n","permalink":"https://dani2442.github.io/posts/gaussian-processes/","summary":"\u003cp\u003eGaussian Process Regression is one of the most elegant and theoretically rich algorithms in machine learning. With this post, I want to celebrate the mathematical beauty underlying Gaussian Processes. I will divide this post into two sections: theory and practice, accompanied by code examples.\u003c/p\u003e\n\u003cp\u003eOne of the key advantages of Gaussian Processes compared to Deep Learning methods is that they inherently provide interpretability (through confidence intervals and uncertainty estimation). They also offer excellent extrapolation properties, as we will see, and a way to incorporate knowledge about the structure of the data into the model. However, these benefits come at a cost. The algorithm has a wide variety of hyperparameters that are difficult to configure; for instance, kernel selection alone is challenging. Understanding and having a good intuition for the inner workings of this algorithm (and the data) is key to making the most of it.\u003c/p\u003e","title":"Gaussian Processes"},{"content":"Last week I came across this very high-quality image, at first I thought that it must be one of those expensive satellites that cost millions of dollars. Then I started digging and found out how ignorant i am about the recent advances in satellites.\nApparently, there are three main orbits used by current satellites [1]:\nLEO (Low Earth Orbit): ~160–2,000 km; short orbital period (90–120 min); low latency; used for Earth observation, Starlink, ISS. MEO (Medium Earth Orbit) ~2,000–35,786 km; orbital period ~2–12 hrs; GPS/GNSS satellites. GEO (Geostationary Earth Orbit): exactly 35,786 km, equatorial; 24-hr period, appears fixed above Earth; ideal for telecom, TV, weather satellites. Before talking about the satellites, we need to understand the main driver of these advancements. There\u0026rsquo;s been a huge decrease on rocket failure rates and reduction in the costs of putting them into orbit. To put things in perspective, between 1970s and the 2000s the average launch was about 18.5k USD/kg, while recently the Falcon Heavy only costs around 1.4k USD/kg (more than 10x reduction!!!) [2]. Also, success rate over 99.46% [5].\nThis is perfect example of the Rebound Effect phenomenon [4]:\n\u0026ldquo;as costs fall, previously unviable uses become attractive, creating new markets or applications.\u0026rdquo;\nIn another chapter, these reductions in costs actually open the door to transplanetary flights. There is still significant room for improvement. For example, SpaceX recently opened its new factory, enabling the production of up to 1,000 rockets per year. Much like Henry Ford, who achieved major cost reductions through mass production, I expect a similar outcome here.\nThe current market cap is approx 1.3 trillion$ (almost GDP of Spain) and it is estimated that more than 10,000 satellites are currently orbiting Earth. They can be divided into two main categories:\n1. Communication and internet The best-known are SpaceX (Starlink) and Amazon (Project Kuiper) with approximately 8000 and 3000 in orbit, respectively. It has limitations in speed limit: average 20-100Mbps for download and 10-20Mbps for upload and ~50ms latency, for a modest price of 29€/month [7, 8].\nYou may not be able to play csgo or ranked matches, but certainly covers 99% of daily tasks. Also, more than 31% of the global population does not have access to mobile network [6, 9]. And many studies suggest that lack of access to technology exacerbates exclusion and poverty [10, 11].\n2. Earth Observation This category includes Governmental devices (mainly reconnaissance and spy) and imagery satellites. Apparently, they provide real-time and high-quality images with resolutions down to 30cm.\nSatellites can carry a wide variety of sensors. The most used are Optical Imagery (regular camera; only day), Synthetic Aperture Radar or SAR (microwaves and reflections; day and night), Hyperspectral Imagery (infrared spectrum), Thermal Infrared, atmospheric/radio measurements (weather prediction and meteorology). These cutting-edge sensors can cost up to 40M USD each one and they can be very difficult to replicate.\nThere are a few players in the game: Planet Labs, Maxar, Satellotic, Capella Space, etc. For context, Planet Labs has more than 200 satellites in orbit.\nThe interesting part is that each company generate an overwhelming ~100 TB of data per day! You need a very strong infrastructure to manage and deliver that amount of information. More importantly, you need substantial compute resources to process this data to extract the relevant information. A few months ago, deepmind offered a glimpse of how to process such amounts of data with AlphaEarth [12].\nCosts and business model The mass of these satellites is typically 200-800kg for communication and between 50-200kg for imagery satellites. Putting in orbit a single satellite can cost approx 4k-400k USD [13]. So the vast majority of the cost is spent down on mainland. For example, Planet Labs subscription costs 20k USD/year. The median life span of these Low-Earth Orbit satellites is roughly 5-7 years (friction is greater), so profitability is not trivial (assuming the materials and labour do not go into orbit hehehe).\nBut who uses these services and information? I guess we all can figure most of the use cases for communication services [14]:\nUse Case Examples Maritime Cruises, Ships, yachts, vessels, containers needing Aviation In-flight internet, both for commercial airlines and smaller aircraft. Remote places Agriculture, mining, oil \u0026amp; gas, construction, refuges, rural areas, poor countries without internet infrastructure, etc. Nomads and tourists Users who travel frequently and want connectivity on the go. Government and military emergency services, defense, reliability, redundancy. Ukranian war for example. In contrast, for me, it is not entirely obvious why would anyone want to use the imagery services. I was very curious about some use cases from their users. Here are some examples that become possible thanks to recent to recent advancements.\nDisaster response \u0026amp; rapid damage assessment: During the huge flood that isolated Valencia last year [15], the most affected areas where quickly identifed thanks to these new resources Agriculture: Developed countries are seeing a decline in working-age populations. The most arduous and hazardous jobs are often left vacant. Automation will be necessary in the coming years, including monitoring of crops, water content, health status, pest control, etc. [16]. Tracking ships and suspicious activity in the coast: The ocean is vast and lacks any posible infraestructure for monitoring. Despite being the 21st century, problems with piracy and smuggling. Also, there is an onoing humanitarian criese with refugees attempting to cross Mediterranina and costing lives on a daily basis. SAR imagery makes it almost imposible to miss these scenarios and the maritine authorities are already making use of these tools [17] Industrial monitoring, prevention and planning: Many suppliers, customers can benefit from a better information flow provided by the satellites. It can be use for road planning, traffic congestion measurement, port and parking monitoring, and industrial site surveillance. This information can also be valuable in financial markets to improve revenue estimates or to verify and compare less reliable data sources. References [1] https://ntrs.nasa.gov/api/citations/20180007067/downloads/20180007067.pdf\n[2] https://ourworldindata.org/grapher/cost-space-launches-low-earth-orbit\n[3] https://www.researchgate.net/publication/361415873_How_to_Solve_Big_Problems_Bespoke_Versus_Platform_Strategies\n[4] https://en.wikipedia.org/wiki/Rebound_effect_(conservation)\n[5] https://en.wikipedia.org/wiki/List_of_Falcon_9_and_Falcon_Heavy_launches\n[6] https://www.itu.int/itu-d/reports/statistics/2024/11/10/ff24-mobile-network-coverage\n[7] https://www.ookla.com/articles/starlink-us-performance-2025\n[8] https://www.starlink.com/updates/network-update?srsltid=AfmBOoosuD4fnN8rKGG4OcG3ewEZTshE07hDrUNSLN6ytRHLRmJz0yOG\n[9] https://www.itu.int/en/mediacentre/Pages/PR-2023-09-12-universal-and-meaningful-connectivity-by-2030.aspx\n[10] https://journals.sagepub.com/doi/full/10.1177/21582440221116104\n[11] https://www.sciencedirect.com/science/article/pii/S2666378322000162\n[12] https://deepmind.google/discover/blog/alphaearth-foundations-helps-map-our-planet-in-unprecedented-detail/\n[13] https://www.esa.int/Applications/Observing_the_Earth/Copernicus/Sentinel-2\n[14] https://en.wikipedia.org/wiki/Starlink#Applications\n[15] https://www.bbc.com/news/articles/cz7wvpyewxlo\n[16] https://eos.com/products/crop-monitoring/satellite-images-for-agriculture/\n[17] https://marine.copernicus.eu/services/use-cases/coastguard-tracking-shorelines-satellite-imagery-understanding-coastal-changes\nSupplementary Material A useful website that shows real-time satellite locations is https://satellitemap.space/. I highly recommend checking it out:\n","permalink":"https://dani2442.github.io/posts/low-earth-orbit/","summary":"\u003cp\u003eLast week I came across this very high-quality image, at first I thought that it must be one of those expensive satellites that cost millions of dollars. Then I started digging and found out how ignorant i am about the recent advances in satellites.\u003c/p\u003e\n\u003cp\u003eApparently, there are three main orbits used by current satellites \u003ca href=\"#References\"\u003e[1]\u003c/a\u003e:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eLEO (Low Earth Orbit): ~160–2,000 km; short orbital period (90–120 min); low latency; used for Earth observation, Starlink, ISS.\u003c/li\u003e\n\u003cli\u003eMEO (Medium Earth Orbit) ~2,000–35,786 km; orbital period ~2–12 hrs; GPS/GNSS satellites.\u003c/li\u003e\n\u003cli\u003eGEO (Geostationary Earth Orbit): exactly 35,786 km, equatorial; 24-hr period, appears fixed above Earth; ideal for telecom, TV, weather satellites.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eBefore talking about the satellites, we need to understand the main driver of these advancements. There\u0026rsquo;s been a huge decrease on rocket failure rates and reduction in the costs of putting them into orbit. To put things in perspective, between 1970s and the 2000s the average launch was about 18.5k USD/kg, while recently the Falcon Heavy only costs around 1.4k USD/kg (more than 10x reduction!!!) \u003ca href=\"#references\"\u003e[2]\u003c/a\u003e. Also, success rate over 99.46% \u003ca href=\"#references\"\u003e[5]\u003c/a\u003e.\u003c/p\u003e","title":"Low-Earth Orbit"},{"content":"I always thought that actions matter more than words. The ultimate expression of this idea is the stock market and, more recently, prediction markets (such as Polymarket). For example, you can bet on a wide variety of topics that stretch from who might win an election to how many tweets Elon Musk will post in a single day.\nOver the past few years, public trust in institutions and journalism has eroded [1], and this distrust has fueled the rise of new financial instruments like prediction markets. This data manipulation became clear during the last US election, where a big gap emerged between conventional polling data and the probabilities implied by market activity [2].\nIn contrast, traditional financial markets often resemble a \u0026ldquo;big fish game\u0026rdquo;, where there is an asymmetry of information and influence. The information is aggregated, resulting in a single number, the price. The effectiveness of this \u0026ldquo;wisdom of the crowds\u0026rdquo; approach is supported by mathematical theories of information aggregation [3], and has been largely validated by recent results. However, we must not glorify them, as there is a risk of insider trading and a perverse incentive for market manipulation.\nNow, how does it work behind the scenes?\nFirst, let\u0026rsquo;s see how this compares to traditional markets and what the differences are. Traditional market theory tells us that the current price of an asset falls between the lowest price a seller is asking and the highest price a bidder is offering (the difference between these two is called the spread and represents the market\u0026rsquo;s liquidity). In a prediction market, the \u0026ldquo;price\u0026rdquo; represents the odds of a binary outcome (win or lose). Once the bet is settled, you receive the payout. For example, if you bet €20 on an outcome with a 20% probability, you will receive €100 if you\u0026rsquo;re right or €0 if you\u0026rsquo;re wrong. The interesting part is that you can sell your bet at any point in time. For instance, if the market\u0026rsquo;s perceived probability rises to 40%, you could sell your position for €40.\nMathematical Modeling: The application of advanced mathematics in finance is a relatively recent development. For example, Modern Portfolio Theory, introduced by Harry Markowitz, dates to the \u0026rsquo;50s; the Black-Scholes equation for options pricing appeared in the \u0026rsquo;70s; and the first major interest rate models, such as the Vasicek and CIR models, emerged in the late \u0026rsquo;70s and \u0026rsquo;80s [4, 5, 6, 7]. I expect to see further developments in predictive modeling in the coming years.\nNow, we can derive a simple mathematical model for this task. There are several models that can be used:\nMathematical model Stochastic Differential Equation Two-point information-filtering model $dP_t = \\sigma P_t(1-P_t) dW_t$ Log-diffusion model (see Appendix A) $dP_t = P_t(1-P_t)(\\mu \u0026#43; \\frac{1}{2}\\sigma^2(1-2P_t))dt \u0026#43; P_t(1-P_t)\\sigma dW_t $ Wright-Fisher model $dP_t =\\sqrt{\\gamma P_t(1-P_t)} dW_t$ We will focus on the two-point information-filtering model as it is Martingale process, i.e., $\\mathbb{E}[P_{t\u0026#43;1} | P_{t}] = P_t$. This property is very natural to assume.\nLet us denote the unknown event by $X\\in\\{0,1\\}$ with prior probabilities\n$$ P(X=1)=p,\\qquad P(X=0)=1-p. $$ We assume the market (or an information stream) observes the continuous process\n$$ \\xi_t=\\sigma X t \u0026#43; B_t,\\tag{1} $$ where $B_t$ is a standard Brownian motion (noise) and $\\sigma\u0026gt;0$ is a signal-to-noise parameter (how strong the signal is). So conditional on $X$,\n$$ \\xi_t\\mid\\{X=1\\}\\sim N(\\sigma t,\\;t),\\qquad \\xi_t\\mid\\{X=0\\}\\sim N(0,\\;t). $$ Compute the likelihood ratio $L_t:=\\frac{f(\\xi_t\\mid X=1)}{f(\\xi_t\\mid X=0)}$. Gaussian densities give\n$$ \\frac{f(\\xi_t\\mid X=1)}{f(\\xi_t\\mid X=0)} =\\exp\\!\\Big(-\\frac{(\\xi_t-\\sigma t)^2-\\xi_t^2}{2t}\\Big) =\\exp(\\sigma\\xi_t-\\tfrac12\\sigma^2 t). $$ Then using the natural filtration $\\mathcal{F}_t=\\sigma(\\xi_s : s\\leq t)$ and Bayes yields\n$$ P_t:=\\Pr(X=1\\mid\\mathcal F_t) =\\frac{pf(\\xi_t | X = 1)}{pf(\\xi_t|X=1) \u0026#43; (1-p)f(\\xi_t|X=0)} =\\frac{p\\exp(\\sigma\\xi_t-\\tfrac12\\sigma^2 t)}{p\\exp(\\sigma\\xi_t-\\tfrac12\\sigma^2 t)\u0026#43;(1-p)}. $$ This is an explicit closed form for the posterior in terms of $\\xi_t$. It’s a logistic-type formula. Now, Differentiate $\\xi_t$ in (1):\n$$ d\\xi_t=\\sigma X\\,dt \u0026#43; dB_t. $$ Given the market filtration $\\mathcal F_t$ (generated by $\\xi$), the conditional expectation of the instantaneous drift is $\\sigma P_t$. So define the innovation\n$$ dW_t := d\\xi_t - \\sigma P_t\\,dt. $$ Properties:\n$E[dW_t\\mid\\mathcal F_t]=E[d\\xi_t\\mid\\mathcal F_t]-\\sigma P_t dt=\\sigma\\mathbb{E}[X |\\mathcal{F}_t]dt - \\sigma P_t dt =0$. So $W_t$ is a continuous martingale. $(dW_t)^2=(d\\xi_t)^2=(dB_t)^2=dt$ (cross terms with $dt$ vanish), so $\\langle W\\rangle_t = t$. By Lévy’s characterization, $W_t$ is a Brownian motion relative to $\\mathcal F_t$. This is the standard innovation Brownian motion.\nLet us define $\\varphi_t := \\sigma\\xi_t-\\tfrac12\\sigma^2 t$, so the closed-form posterior is $P_t = \\dfrac{p e^{\\varphi_t}}{p e^{\\varphi_t}\u0026#43;1-p}$. Let $f(\\varphi)=P(\\varphi)$. Two useful derivatives (logistic identity):\n$$ f\u0026#39;(\\varphi)=P_t(1-P_t),\\qquad f\u0026#39;\u0026#39;(\\varphi)=P_t(1-P_t)(1-2P_t). $$ Compute $d\\varphi_t$:\n$$ d\\varphi_t=\\sigma\\,d\\xi_t-\\tfrac12\\sigma^2 dt = \\sigma(dW_t\u0026#43;\\sigma P_t dt)-\\tfrac12\\sigma^2 dt = \\sigma\\,dW_t \u0026#43; \\sigma^2\\big(P_t-\\tfrac12\\big)dt. $$ Apply Itô’s formula $dP_t=f\u0026#39;(\\varphi_t)d\\varphi_t\u0026#43;\\tfrac12 f\u0026#39;\u0026#39;(\\varphi_t)(d\\varphi_t)^2$. Since $(d\\varphi_t)^2=\\sigma^2(dW_t)^2=\\sigma^2 dt$, substitute:\n$$ dP_t = P_t(1-P_t)\\big(\\sigma\\,dW_t \u0026#43; \\sigma^2(P_t-\\tfrac12)dt\\big) \u0026#43;\\tfrac12 P_t(1-P_t)(1-2P_t)\\sigma^2 dt $$ $$ = \\sigma P_t(1-P_t)\\,dW_t \u0026#43; \\sigma^2 P_t(1-P_t)\\big(P_t-\\tfrac12 \u0026#43; \\tfrac12(1-2P_t)\\big)dt. $$ But the bracket in the drift vanishes:\n$$ P_t-\\tfrac12 \u0026#43; \\tfrac12(1-2P_t)=0. $$ So the drift term cancels exactly, leaving the tidy filtering SDE\n$$ \\boxed{\\,dP_t=\\sigma P_t(1-P_t)\\,dW_t\\,.} $$ Notice that the model satisfies $P_t \\in [0,1] $ and becomes constant when $P_t$ is 0 or 1. However, this model might not be able to arrive at a conclusion when the event is supposed to end, i.e., $P_T = 0 \\text{ or }1$ (see the Experiments). See Appendix B for a follow up on this issue.\nExperiment We can simulate the two-point information-filtering model that obtained in the previous section using a discretization scheme like Euler-Maruyama. We obtain the following figure\nI also attach the code for completeness\nT, N, n_sim = 4, 2000, 7 # time horizon and steps dt = T/N # time step sigma = 2.0 # information flow parameter p0 = 0.6 # prior t = np.linspace(0, T, N+1) # Simulate SDE P = np.zeros((n_sim, N+1)) P[:, 0] = np.full(n_sim, p0) for i in range(N): dW = np.sqrt(dt) * np.random.randn(n_sim) P[:, i+1] = P[:, i] + sigma * P[:, i] * (1 - P[:, i]) * dW P[:, i+1] = np.clip(P[:, i+1], 0, 1) # keep in [0,1] fig, ax = plt.subplots(figsize=(10, 6)) for i in range(n_sim): ax.plot(t, P[i], alpha=0.6, c=plt.cm.Greens(i / n_sim) if P[i, -1] \u0026gt; 0.5 else plt.cm.Reds(i / n_sim)) ax.set_xlabel(\u0026#34;time\u0026#34;) ax.set_ylabel(\u0026#34;Probability\u0026#34;) ax.set_title(\u0026#34;Prediction market price dynamics\u0026#34;) ax.set_yticks([0, 1]) ax.set_yticklabels([\u0026#39;0\u0026#39;, \u0026#39;1\u0026#39;]) ax.set_xticks([]) plt.savefig(\u0026#34;prediction_market_simulations.png\u0026#34;, dpi=300, bbox_inches=\u0026#39;tight\u0026#39;) plt.show() Main takeaways (opinion):\nI believe this market will continue to grow rapidly, as the information it provides is very useful to consumers and the market participants themselves. A drawback of platforms like Polymarket is that capital tied up in long-term bets doesn\u0026rsquo;t accrue interest. This means users are exposed to inflation risk, as the underlying currency is a stablecoin pegged to the dollar, i.e., 1$ = 1 USDT I expect institutions to become more involved, shifting the user base from its current majority of retail investors.. I think we will see the introduction of more complex financial instruments such as derivatives on prediction markets outcomes. Increase in regulation (avoid insider trading, market manipulation, etc). Future work in mathematical models, introduction of Modern Portfolio Theory, hedging, risk, etc. References [1] https://news.gallup.com/poll/1597/confidence-institutions.aspx\n[2] https://edition.cnn.com/2024/11/08/business/polymarket-election-trump-nightcap\n[3] https://projects.iq.harvard.edu/files/yiling/files/icecr04.pdf\n[4] Baxter, M. y Rennie, A., Financial calculus an introduction to derivative pricing, Cambridge University Press\n[5] Hull, J., Options, futures, and other derivative securities, Prentice-Hall.\n[6] Neftci, S.N., An Introduction to the Mathematics of Financial Derivatives, Academic Press\n[7] Wilmott, P., Howison, S. y Dewynne, J, Option pricing: mathematical models and computation, Oxford Finantial.\nAppendix (A) Derivation of the Log-odd diffusion model This model is an analogue to Black–Scholes idea. Assume log-odds $Y_t=\\log\\frac{P_t}{1-P_t}$ follows an arithmetic Brownian motion\n$$ dY_t = \\mu\\,dt \u0026#43; \\sigma\\,dW_t. $$ Set $P_t = f(Y_t)$ with $f(y) = \\frac{1}{1\u0026#43;e^{-y}}$ (the sigmoid). By Itô:\n$$ f\u0026#39;(y)=P(1-P),\\qquad f\u0026#39;\u0026#39;(y)=P(1-P)(1-2P). $$ Thus\n$$ dP_t = f\u0026#39;(Y_t)\\,dY_t \u0026#43; \\frac{1}{2} f\u0026#39;\u0026#39;(Y_t)\\,(dY_t)^2 = P_t(1-P_t)\\big(\\mu\\,dt \u0026#43; \\sigma\\,dW_t\\big)\u0026#43; \\frac{1}{2}\\sigma^2 P_t(1-P_t)(1-2P_t)\\,dt, $$ or collecting terms\n$$ \\boxed{\\,dP_t = P_t(1-P_t)\\,\\sigma\\,dW_t \\;\u0026#43;\\;P_t(1-P_t)\\Big(\\mu \u0026#43; \\frac{1}{2}\\sigma^2(1-2P_t)\\Big)\\,dt. \\,} $$ (B) Event triggering at $T$ We have studied the two-point information-filtering model. However, this model does not handle a very important aspect. Imagine an event is resolver at time $T$. The model may not converge in time and in most applications the last moments are the most important.\nWe can make the same filtering model but force the information to become infinitely informative as $t \\rightarrow T^-$, which drives the posterior to 0 or 1 by time $T$. We can modify the equation with a time-dependent signal strength that guarantees resolution at $T$.\n$$ dP_t = \\sigma P_t(1-P_t)dW_t $$ Replace the constant signal coefficient by a deterministic time-dependent one. Let\n$$ d\\xi_t = \\sigma_t X\\,dt \u0026#43; dB_t,\\qquad t\\in[0,T), $$ with $X\\in\\{0,1\\}$ and prior $\\Pr(X=1)=p$. Here $\\sigma_t\u0026gt;0$ is deterministic and continuous on $[0,T)$. (When needed we think of $\\xi_0=0$.)\nWhen $\\sigma_t\\equiv\\sigma$ this is the previous model; now $\\sigma_t$ may blow up as $t\\to T$ to create arbitrarily large signal-to-noise.\nThe likelihood ratio up to time $t$ is\n$$ L_t =\\frac{d\\mathbb P(\\xi_{[0,t]}\\mid X=1)}{d\\mathbb P(\\xi_{[0,t]}\\mid X=0)} =\\exp\\Big(\\int_0^t \\sigma_s\\,d\\xi_s - \\tfrac12\\int_0^t\\sigma_s^2\\,ds\\Big). $$ This is the Girsanov / Gaussian likelihood formula (check by completing the square). By Bayes,\n$$ P_t=\\frac{p\\exp\\big(\\int_0^t\\sigma_s\\,d\\xi_s-\\tfrac12\\int_0^t\\sigma_s^2\\,ds\\big)} {p\\exp\\big(\\int_0^t\\sigma_s\\,d\\xi_s-\\tfrac12\\int_0^t\\sigma_s^2\\,ds\\big)\u0026#43;(1-p)}. $$ This reduces to the earlier closed form when $\\sigma_s\\equiv\\sigma$. As before define the innovation\n$$ dW_t := d\\xi_t - \\sigma_t P_t\\,dt. $$ One checks $W_t$ is an $\\mathcal F_t$-Brownian motion. Repeating the Itô calculation (same logistic derivatives as before) yields the SDE\n$$ \\boxed{\\,dP_t=\\sigma_t\\,P_t(1-P_t)\\,dW_t\\,.} $$ So $P_t$ remains a martingale (no drift) and its instantaneous volatility is $\\sigma_t P_t(1-P_t)$.\nCondition that forces resolution at $T$\nIntuition: the posterior is driven by the log-odds\n$$ \\Lambda_t:=\\log\\frac{P_t}{1-P_t} =\\log\\frac{p}{1-p}\u0026#43;\\int_0^t\\sigma_s\\,d\\xi_s-\\tfrac12\\int_0^t\\sigma_s^2\\,ds. $$ Under the true value $X$, write $d\\xi_s=\\sigma_s X\\,ds\u0026#43;dB_s$. Then\n$$ \\Lambda_t =\\log\\frac{p}{1-p}\u0026#43;\\int_0^t\\sigma_s\\,dB_s \u0026#43;\\int_0^t\\sigma_s^2(X-\\tfrac12)\\,ds. $$ The deterministic term $\\int_0^t\\sigma_s^2(X-\\tfrac12)\\,ds$ has sign depending on $X$. If the cumulative information $\\int_0^T\\sigma_s^2\\,ds$ diverges to $\u0026#43;\\infty$, this deterministic term dominates the stochastic martingale $\\int_0^t\\sigma_s\\,dB_s$ as $t\\to T$, and $\\Lambda_t$ tends to $\u0026#43;\\infty$ when $X=1$ and to $-\\infty$ when $X=0$. Therefore\n$$ P_T:=\\lim_{t\\to T^-}P_t=\\begin{cases}1,\u0026amp; X=1,\\\\ 0,\u0026amp; X=0,\\end{cases} \\quad\\text{a.s.} $$ So a sufficient condition for almost-sure resolution by time $T$ is\n$$ \\boxed{\\ \\int_0^T \\sigma_s^2\\,ds = \u0026#43;\\infty.\\ } $$ If $\\int_0^T\\sigma_s^2 ds\u0026lt;\\infty$ then the posterior need not hit $\\{0,1\\}$ at $T$; the information gathered is finite and the posterior may remain in $(0,1)$ with positive probability.\nTake for example\n$$ \\sigma_s = \\frac{\\kappa}{\\sqrt{T-s}},\\quad\\quad 0\\leq s\\leq T $$ for some $\\kappa\u0026gt;0$. Thus this $\\sigma_s$ forces resolution at (or before) $T$. The SDE for $P_t$ becomes\n$$ dP_t=\\frac{\\kappa}{\\sqrt{T-t}}\\,P_t(1-P_t)\\,dW_t, $$ Here, I show the result of a simulation using this new model:\n","permalink":"https://dani2442.github.io/posts/prediction-markets/","summary":"\u003cp\u003eI always thought that actions matter more than words. The ultimate expression of this idea is the stock market and, more recently, prediction markets (such as Polymarket). For example, you can bet on a wide variety of topics that stretch from who might win an election to how many tweets Elon Musk will post in a single day.\u003c/p\u003e\n\u003cp\u003eOver the past few years, public trust in institutions and journalism has eroded \u003ca href=\"#references\"\u003e[1]\u003c/a\u003e, and this distrust has fueled the rise of new financial instruments like prediction markets. This data manipulation became clear during the last US election, where a big gap emerged between conventional polling data and the probabilities implied by market activity \u003ca href=\"#references\"\u003e[2]\u003c/a\u003e.\u003c/p\u003e","title":"Prediction Markets"},{"content":"A few days ago, this paper was published in Nature [1] claiming a huge improvement in LLM inference using analog computing:\n\u0026ldquo;Our architecture reduces attention latency and energy consumption by up to two and four orders of magnitude\u0026rdquo;\nThis could mean we may soon run LLMs on devices no bigger than smartphones while consuming less power than a light bulb. After taking a deep dive, I believe this might turn out to be one of the most influential results of 2025.\nIt is no surprise that current digital computing is very inefficient, there is a lot of overhead on communication (between cache and SRAM) and synchronization (clock cycle), all to maintain the general-purpose and deterministic paradigm that we are used to.\nFor specific workloads like inference, there exist some alternatives (specialized hardware that focuses on inference [2], but they remain digital; and neuromorphic computing [5], although it has never quite taken off ). And, unlike current digital hardware, that uses all this big and complex framework to perform these rather simplistic operations (self-attention and feed-forward), using built-in math operations on the chip using the analog operations, we can speed things near the speed of light!\nBack to the paper, what they have done up to now is modified the self-attention mechanism (flash attention) so that it can be implemented using analog circuits, while still remaining compatible with differentiable programming frameworks like pytorch:\nThey trained this analog-inspired simulation within a GPT-2 architecture and compared it with standard digital models (ensure accuracy wasn’t sacrificed):\nCaveats: The paper only showcases the simulated version of the model, they haven\u0026rsquo;t benchmarked it on real hardware, and I expect a big decay in performance from noise in the analog version (see for example Sim2Real problems arising in reinforcement learning). They share a possible result between the digital model and the analog implementation:\nVery recently, I have seen someone on twitter who is trying to implement it on hardware of this work [4]\nFor so long, we have adapted the hardware to fit our software, it\u0026rsquo;s time to turn things around! Richard Sutton has long argued this point (see Bitter Lesson [3]). Just as transformers mapped neatly onto the strengths of GPUs, we may now see a new wave of research into models designed to be analog-friendly, i.e., architectures that can be efficiently ported to analog computing.\nInference, in particular, seems ripe for disruption. Training will likely remain dominated by gpus with Nvidia still leading, i.e., train on gpu -\u0026gt; inference on custom-device. It might be interesting exploring the implications of this result (if this hold), we will probably have inference at no cost and a surge of custom-built analog components for different use cases.\nAdditional info:\nThe code is available [6] They used PyTorch and Triton (see the requirements) SPICE software for circuit simulation References [1] Leroux, N., Manea, PP., Sudarshan, C. et al. Analog in-memory computing attention mechanism for fast and energy-efficient large language models. Nat Comput Sci (2025). https://doi.org/10.1038/s43588-025-00854-1\n[2] https://www.chipstrat.com/p/the-future-of-edge-inference-hardware\n[3] http://www.incompleteideas.net/IncIdeas/BitterLesson.html\n[4] https://x.com/BrianRoemmele/status/1966473607795929527\n[5] https://en.wikipedia.org/wiki/Neuromorphic_computing\n[6] https://github.com/NathanLeroux-git/GainCellAttention/\n","permalink":"https://dani2442.github.io/posts/analog-computing/","summary":"\u003cp\u003eA few days ago, this paper was published in Nature \u003ca href=\"#references\"\u003e[1]\u003c/a\u003e claiming a huge improvement in LLM inference using analog computing:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026ldquo;Our architecture reduces attention latency and energy consumption by up to two and four orders of magnitude\u0026rdquo;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003e\u003cimg alt=\"alt text\" loading=\"lazy\" src=\"/posts/analog-computing/image-3.png\"\u003e\u003c/p\u003e\n\u003cp\u003eThis could mean we may soon run LLMs on devices no bigger than smartphones while consuming less power than a light bulb. After taking a deep dive, I believe this might turn out to be one of the most influential results of 2025.\u003c/p\u003e","title":"Analog computing in LLMs"},{"content":"This post is an extract of my Master\u0026rsquo;s Thesis in Mathematics, it can be found in this GitHub repository for a complete and autocomplete document: repo.\nIn our continuous pursuit of enlightenment, knowledge, and understanding of the world around us, we turn to the tool of mathematics. For thousands of years, mathematicians have sought to translate our sensory experiences into a language that captures their essence in the most abstract, universal, and general form.\n“Mathematics is the language in which God has written the universe.”\n— Galileo Galilei\nMathematics has provided a unique framework to express many foundational ideas about the natural world. For example, in 250 BCE, Archimedes proposed a model to describe the upward force exerted by a fluid on a submerged object, i.e., $F = \\rho · V · g$ where $F$ is the force, $\\rho$ the density, $V$ the volume and $g$ the force of gravity. More than a thousand years later, in the 17th century, Kepler formulated an equation to describe the motion of the planets around the Sun, i.e., $dA/dt$ is constant, where $A$ is the area of the line connecting a planet and the Sun. Shortly afterward, Newton introduced the three Laws of Motion, which became the foundation of classical mechanics and later led to more complex models such as the Naview-Stokes equation.\nMathematics has provided a unique framework to express many foundational ideas about the natural world. For example, in 250 BCE, Archimedes proposed a model to describe the upward force exerted by a fluid on a submerged object, i.e., $F = \\rho \\cdot V \\cdot g$ where $F$ is the force, $\\rho$ the density, $V$ the volume and $g$ the force of gravity. More than a thousand years later, in the 17th century, Kepler formulated an equation to describe the motion of the planets around the Sun, i.e., $\\frac{dA}{dt}$ is constant, where $A$ is the area of the line connecting a planet and the Sun. Shortly afterward, Newton introduced the three Laws of Motion, which became the foundation of classical mechanics and later led to more complex models such as the Naview-Stokes equation.\nMost of the equations governing the Universe—though not all—are expressed in terms of differentials. These problems are studied in the field of differential equations. The main objective in this field is to describe the solutions of these equations. Unfortunately, some of them do not have closed-form solutions, requiring us to rely on approximations provided by numerical methods. A key challenge with numerical methods is ensuring that the approximations are reliable. This involves analyzing the consistency, stability, and convergence of the methods. Another significant issue is that numerical computations can be very resource-intensive.\nIn recent years, alternative approaches to traditional numerical methods have emerged, particularly for handling complex geometries or solving ill-posed problems—such as those with unknown boundary conditions. One such alternative is machine learning, which has gained significant traction in this context. The leitmotiv of machine learning is the learn a function from a dataset composed of samples. In our case, our goal may be to learn a physics phenomenon from a dataset, which is why this approach is often referred to as data-driven. A key drawback of data-driven methods is their heavy reliance on large quantities of data, which may not always be available. Moreover, these methods often disregard prior knowledge about the physical system. This limitation has led to the development of physics-informed machine learning. The motivation behind physics-informed machine learning is twofold: to harness the strengths of machine learning while incorporating prior physical knowledge of the system. Combining these two domains increases the complexity of the problem, and this is the central focus of our study in this manuscript.\nPhysics-Informed Machine Learning Data-driven methods often require large quantities of data, which may be scarce or expensive to obtain. Moreover, they typically ignore valuable prior knowledge about the underlying physical system.Incorporating physical knowledge into the learning process:\nPhysics-Informed Neural Networks [Raissi, 2019] NeuralODEs [Chen, 2019] Neural Networks (with tanh) A Neural Network $u_\\theta: \\mathbb{R}^{d_1}\\rightarrow \\mathbb{R}^{d_2}$ is defined by $$ u_\\theta = \\mathcal{A}_{H\u0026#43;1}\\circ (\\phi \\circ \\mathcal{A}_H) \\circ \\dots \\circ (\\phi \\circ \\mathcal{A}_1), $$ where $\\mathcal{A}_k$ is an affine function and $\\phi$ is the activation function, e.g., tanh. And, $\\theta\\in\\Theta$ is the parameter of the neural network.\nWe denote $\\text{NN}_H^D$ the set of neural networks where $H$ is the number of layers and $D$ is the depth.\nKey results:\n$\\cup_{D\\in \\mathbb{N}} \\text{NN}_H^D$ is dense in $(C^\\infty(\\bar{\\Omega}, \\mathbb{R}^{d_2}), \\|\\cdot\\|_{C^K(\\Omega)})$. $\\|u_\\theta\\|_{C^K(\\Omega)} \\leq C\\|\\theta\\|_2$. $\\|u_\\theta - u_{\\theta\u0026#39;}\\|_{C^K(\\Omega)} \\leq L\\|\\theta-\\theta\u0026#39;\\|_2$. $\\|u_\\theta(x) - u_\\theta(y)\\|_2 \\leq \\tilde{L} \\|x-y\\|_2$. Physics-Informed Neural Networks (PINNs) — Empirical Risk Function $$\\widehat{R}_{n_1, n_2, n_3}(\\textcolor{lightblue}{u_\\theta}) := \\frac{1}{n_1}\\sum_{i=1}^{n_1} \\|\\textcolor{lightblue}{u_\\theta}(\\textcolor{red}{X_i})-\\textcolor{red}{Y_i}\\|_2^2 \u0026#43;\\frac{1}{n_2}\\sum_{i=1}^{n_2}\\|\\textcolor{lightblue}{u_\\theta}(\\textcolor{green}{X^{(2)}_j}) - \\textcolor{purple}{h}(\\textcolor{green}{X_j^{(2)}})\\|_2^2 \u0026#43;\\frac{1}{n_3}\\sum_{k=1}^M \\sum_{\\ell=1}^{n_3} \\textcolor{orange}{\\mathfrak{F}_k}(\\textcolor{lightblue}{u_\\theta}, \\textcolor{brown}{X_\\ell^{(3)}})^2$$ where $\\textcolor{lightblue}{u_\\theta} \\in \\text{NN}_H^D$, $\\{(\\textcolor{red}{X_i}, \\textcolor{red}{Y_i})\\}_{i=1}^{n_1}$ i.i.d distributed as $(X,Y)\\in \\Omega\\times \\mathbb{R}^{d_2}$, $\\textcolor{purple}{h} \\in C(\\partial\\Omega)$ condition, $\\{\\textcolor{green}{X_j^{(2)}}\\}_{j=1}^{n_2}$ i.i.d. $\\partial\\Omega$, $\\{\\textcolor{brown}{X_\\ell^{(3)}}\\}_{\\ell=1}^{n_3}$ i.i.d. in $\\Omega$, and $\\textcolor{orange}{\\mathfrak{F}_k}: C^\\infty(\\Omega, \\mathbb{R}^{d_2}) \\times \\Omega\\rightarrow \\mathbb{R}^{d_2}$.\n— Theoretical Risk Function\n$$\\widehat{R}_{n_1}(\\textcolor{lightblue}{u_\\theta}) := \\frac{1}{n_1}\\sum_{i=1}^{n_1} \\|\\textcolor{lightblue}{u_\\theta}(\\textcolor{red}{X_i})-\\textcolor{red}{Y_i}\\|_2^2 \u0026#43;\\mathbb{E}_{X\\in \\mu_X}\\|\\textcolor{lightblue}{u_\\theta}(X) - \\textcolor{purple}{h}(X)\\|_2^2 \u0026#43;\\frac{1}{|\\Omega|}\\sum_{k=1}^M \\int_\\Omega \\textcolor{orange}{\\mathfrak{F}_k}(\\textcolor{lightblue}{u_\\theta}, x)^2dx $$ where $\\textcolor{lightblue}{u_\\theta} \\in \\text{NN}_H^D$, $\\{(\\textcolor{red}{X_i}, \\textcolor{red}{Y_i})\\}_{i=1}^{n_1}$ i.i.d, $\\textcolor{purple}{h} \\in C(\\partial\\Omega)$ condition, and $\\textcolor{orange}{\\mathfrak{F}_k}: C^\\infty(\\Omega, \\mathbb{R}^{d_2})\\times \\Omega\\rightarrow \\mathbb{R}^{d_2}$.\nExample: Dynamic Friction Model Let $\\Omega:=(0,T)$ and $\\mathfrak{F}(u,x) := mu\u0026#39;\u0026#39;(x) \u0026#43; \\gamma u\u0026#39;(x)$ where $u\\in C^2(\\overline{\\Omega}, \\mathbb{R})$ and $x\\in \\Omega$.\nFix $n_1\\in \\mathbb{N}$. Let $\\{\\hat{\\theta}_n(n_2, n_3)\\}_{n\\in \\mathbb{N}}$ be a minimizing sequence for the empirical risk, i.e., $$ \\lim_{n\\rightarrow\\infty} \\widehat{R}_{n_1, n_2, n_3}(u_{\\hat{\\theta}_n(n_2, n_3)}) = \\inf_{\\theta\\in \\Theta_{H,D}} \\widehat{R}_{n_1, n_2, n_3}(u_\\theta)$$ We say that $\\{\\hat{\\theta}_n(n_2, n_3)\\}_{n\\in \\mathbb{N}}$ satisfies the risk-consistency with respect to the theoretical risk $R_{n_1}$ if $$\\lim_{n_2, n_3\\rightarrow\\infty} \\lim_{n\\rightarrow \\infty} R_{n_1}(u_{\\hat{\\theta}_n(n_2, n_3)}) = \\inf_{\\theta\\in \\Theta_{H,D}} R_{n_1}(u_\\theta).$$ The previous example can be used as a counterexample. We can define a sequence $\\{\\hat{\\theta}_n(n_2, n_3)\\}_{n\\in\\mathbb{N}}$ that verifies: $$ \\lim_{n\\rightarrow\\infty} \\widehat{R}_{n_1, n_2, n_3}(u_{\\hat{\\theta}_n(n_2, n_3)}) = 0 \\quad\\quad \\text{and} \\lim_{n\\rightarrow\\infty} R_{n_1}(u_{\\hat{\\theta}_n(n_2, n_3)}) = \\infty$$ First, we consider the case where $m=1$ and $\\gamma=1$. We use the conditions $\\{(0, 0), (1, 1)\\}$, which yield the exact solution $$u^\\star(x) = \\frac{e}{e-1}(1-e^{-x}).$$\nRidge PINNs The ridge empirical risk function is defined by $$ \\widehat{R}^{(\\text{ridge})}_{n_1, n_2, n_3}(u_\\theta) := \\widehat{R}_{n_1, n_2, n_3}(u_\\theta) \u0026#43; \\lambda_{(\\text{ridge})}\\|\\theta\\|_2^2.$$ Theorem 3.8: Risk-consistency of ridge PINNs \\ Let $\\{\\hat{\\theta}^{(\\text{ridge})}_n(n_2, n_3)\\}_{n\\in \\mathbb{N}}$ a minimizing sequence of the ridge empirical risk. Then, for some $\\lambda_{(\\text{ridge})}(n_2, n_3)$ depending only on $n_2, n_3$, then $$ \\lim_{n_2,n_3\\rightarrow\\infty} \\lim_{n\\rightarrow\\infty} R_{n_1}(u_{\\hat{\\theta}^{(\\text{ridge})}_n(n_2, n_3)}) = \\inf_{\\theta\\in \\Theta_{H,D}}R_{n_1}(u_\\theta) $$ In other words, it is risk-consistent.\nExperiment: Lid-Driven Cavity Flow Let $\\Omega := (0,2)^2 \\times [0,5]$, where $(x,y)\\in (0,2)^2$ is the spatial domain and $t\\in [0,5]$ is the time interval.\nThe objective is to find the state variables—velocity components $(u,v)$ and pressure $p$—as a continuous function $\\mathbf{v}=(u, v, p) \\in C(\\Omega, \\mathbb{R}^3)$ that satisfies the following initial and boundary conditions:\n$$ \\begin{cases} u(x,y,0) = v(x,y,0)= p(x,y,0) = 0 \u0026amp; \\forall x,y \\in [0,2]\\\\ v(x,0,t) = v(x,2,t) = 0 \u0026amp; \\forall x \\in [0,2] \\quad\\forall t \\in [0,5]\\\\ u(0,y,t) = 0 \u0026amp; \\forall y \\in [0,2] \\quad\\forall t \\in [0,5]\\\\ u(2,y,t) = 1 \u0026amp; \\forall y \\in [0,2] \\quad\\forall t \\in [0,5]\\\\ \\frac{\\partial p}{\\partial \\mathbf{n}}(x,0,t)=\\frac{\\partial p}{\\partial \\mathbf{n}}(x,2,t)=0 \u0026amp; \\forall x \\in [0,2] \\quad\\forall t \\in [0,5]\\\\ \\frac{\\partial p}{\\partial \\mathbf{n}}(0,y,t)=\\frac{\\partial p}{\\partial \\mathbf{n}}(2,y,t)=0 \u0026amp; \\forall y \\in [0,2] \\quad\\forall t \\in [0,5] \\end{cases} $$\nReferences [Raissi, 2019] Raissi, M., Perdikaris, P., and Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707.\n[Chen, 2019] Chen, R. T. Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D. (2019). Neural Ordinary Differential Equations. arXiv:1806.07366 [cs].\n","permalink":"https://dani2442.github.io/posts/pinns/","summary":"\u003cp\u003eThis post is an extract of my Master\u0026rsquo;s Thesis in Mathematics, it can be found in this GitHub repository for a complete and autocomplete document: \u003ca href=\"https://github.com/dani2442/tfm\"\u003erepo\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eIn our continuous pursuit of enlightenment, knowledge, and understanding of the world around us, we turn to the tool of mathematics. For thousands of years, mathematicians have sought to translate our sensory experiences into a language that captures their essence in the most abstract, universal, and general form.\u003c/p\u003e","title":"Physics-Informed Neural Networks"},{"content":"This work is part of project done for a class in the MSc Applied Mathematics in the Autonomous University of Madrid. You can find the complete work here. In this manuscript, we explore the application of dimensionality reduction algorithms to real-world datasets within the context of functional data analysis. We establish several theoretical results related to Principal Component Analysis (PCA) for functional data and introduce a novel variation, Fourier PCA, inspired by Fourier theory. Additionally, we extend Kernel PCA to the functional data setting by proposing new kernels, adapted from well-known finite-dimensional counterparts, and provide theoretical foundations for their use. Finally, we evaluate and compare the performance of these methods. All code associated with this study is available in a GitHub repository.\nFor many years, principal component analysis has been a key dimension reduction tool for multivariate data in classical statistics and machine learning. Lately, it has been extended to functional data and termed functional principal component analysis (FPCA). FPCA has taken off to become the most prevalent tool in functional data analysis. This is partly because FPCA facilitates the conversion of inherently infinite-dimensional functional data to a finite-dimensional vector. Under mild assumptions, the underlying stochastic process can be expressed as a countable sequence of uncorrelated random variables, which are then truncated to a finite vector. Then the tools of multivariate data analysis can be readily applied to infinite-dimensional data without the hassle of handling lower-dimensional representations of the data.\nSpecifically, the dimension reduction is achieved through the Karhunen-Loève expansion, which is a well-known result in stochastic processes. It decomposes observed random trajectories $X_i(t)$ in a functional basis that consists of the eigenvectors of the covariance operator of the process $X$, i.e., $K(s,t) := \\text{Cov}(X_s,X_t)$.\nThe extension of FPCA to multivariate functional data is hence of high practical relevance. Existing approaches for multivariate functional principal component analysis (mFPCA) are based on the multivariate functional Karhunen-Loève expansion to compress the data into a finite-dimensional space (Berrendero, 2011; Chiou, 2014; Jacques, 2014; Ramsay, 2005). However, (Berrendero, 2014) proposed a PCA-like method to compress a multivariate function space $L^2(\\mathbb{R}, \\mathbb{R}^p)$ to a lower-dimensional multivariate function space $L^2(\\mathbb{R}, \\mathbb{R}^k)$ with $k\\ll p$, using the principles of standard finite-dimensional PCA.\nWe will denote by $\\mathcal{X}$ the space of our input space; in our case, it will be $L^2(\\mathbb{R},\\mathbb{R}^d)$. We will denote by $\\mathcal{Z}$ the space of the principal component space or latent space; it can be a finite or infinite-dimensional space, e.g., $\\mathbb{R}^n$ or $L^2(\\mathbb{R},\\mathbb{R}^k)$, see the next Table. The compression method has two functions, the projection that transforms the input into the lower dimensional space, and the functions that reverse or $$ \\begin{align} \\Phi: \\;\u0026amp;\\mathcal{X} \\rightarrow \\mathcal{Z} \\tag{Transform}\\\\ \\Phi^*: \\;\u0026amp;\\mathcal{Z} \\rightarrow \\mathcal{X} \\tag{Inverse Transform} \\end{align} $$ Let us denote by $P$ the projection onto the subspace spanned by the principal components, i.e., $$ \\begin{align*} P: \\mathcal{X}\\rightarrow \\mathcal{X}\\quad\\quad\\quad P=\\Phi^* \\circ \\Phi \\end{align*} $$ Let $X$ be a random variable taking values in the space $\\mathcal{X}$. The principal component analysis problem can be interpreted from two equivalent perspectives: maximizing the variance of the projections or minimizing the mean squared reconstruction error:\n$$ \\max_\\Phi \\;\\text{Var}\\left( \\Phi(X)\\right) \\quad\\quad \\Leftrightarrow \\quad\\quad \\min_\\Phi \\mathbb{E} \\|X-P_\\Phi X\\|^2_{\\mathcal{X}} $$\nExperiments 1. Correlated $n$-dimensional Brownian Motion We simulate a multidimensional Brownian motion with covariance given by the product of $\\Sigma = CC^T$ (it needs to be symmetric) where $C$ is a random matrix. Then, the simulated Brownian motion is given by $$ X_{t\u0026#43;1} = X_{t} \u0026#43; \\sqrt{\\Delta t}\\mathcal{N}(0, \\mathbf\\Sigma) $$ For this experiment, we use 5 dimensions, i.e., $p=5$, and our goal is to compress each observation into one ($k=1$). The signals are correlated, so it should be possible with great error despite the noisy signal. The results for 100 experiments can be found in Figure. Observe that both algorithms perform identically in this regime. This is an interesting result and may require further consideration.\n2. Anomaly Detection for satellite telemetry ESA released a large-scale, real-life satellite telemetry anomaly dataset. Solar arrays power regulators switch off, video processing unit reset, attitude disturbances…Throughout its life after launch, a spacecraft is subject to several unexpected behaviours that can lead to loss of scientific data, inadequate performance, and sometimes triggering of spacecraft safe mode, which typically entails a challenging recovery. The dataset, derived from three missions and totaling 31 GB, is curated and annotated to aid the development of AI models for anomaly detection. It consists of over 50 channels and 15 telecommands with over 50M data instances. Figure shows a sample of the data within the spacecraft dataset. We have discarded those channels that spark and contain discontinuities and do not have a functional structure. The way to process the signal is to create a sliding window to process the data. This allows multiple samples of the data and makes it computationally feasible. However, we need to configure the sliding windows length parameter.\n3. Kernel Functional PCA vs Functional PCA We create a set of functions that we are not able to separate using any linear projection; hence, Functional PCA is not able to separate both classes. We create three different Gaussian process distributions. We use a periodic covariance, specifically, we use the Exponential Sine Square kernel that is given by $$ k(x, x\u0026#39;) = \\sigma^2 \\exp\\left(-\\frac{2}{\\ell^2} \\sin^2\\left( \\pi \\frac{|x - x\u0026#39;|}{p} \\right) \\right) $$ where $\\sigma^2$ is the overall variance ($\\sigma$ is also known as amplitude), $\\ell$ is the lengthscale, and $p$ the period, which is the distance between repetitions. We choose $p=1$ and we subtract the resulting sampling to the first value so that they start at the same value. We create the first class with mean $\\mu(t)=\\pm\\sin(t\\pi)$ and the second class with zero mean. Moreover, we use the Kernel of the RBF for the PCA kernel, i.e., $$ K(x,y) := \\exp\\{-\\|x-y\\|_{\\mathcal{H}}^2/(2\\sigma^2)\\} $$ where we use $\\sigma^2=1$ as the kernel.\nFinally, we see that using only the 2nd principal component we are able to separate both classes. We can see that, alternatively, the standard Functional PCA is not able to separate both categories of functions. In fact, it can be proven that using a single principal component, we are not able to separate the classes.\n","permalink":"https://dani2442.github.io/posts/functional-pca/","summary":"\u003cp\u003eThis work is part of project done for a class in the MSc Applied Mathematics in the Autonomous University of Madrid. You can find the complete work \u003ca href=\"https://github.com/dani2442/Functional-Data-Analysis/blob/main/FPCA%20manuscript.pdf\"\u003ehere\u003c/a\u003e. In this manuscript, we explore the application of dimensionality reduction algorithms to real-world datasets within the context of functional data analysis. We establish several theoretical results related to Principal Component Analysis (PCA) for functional data and introduce a novel variation, \u003cem\u003eFourier PCA\u003c/em\u003e, inspired by Fourier theory. Additionally, we extend Kernel PCA to the functional data setting by proposing new kernels, adapted from well-known finite-dimensional counterparts, and provide theoretical foundations for their use. Finally, we evaluate and compare the performance of these methods. All code associated with this study is available in a \u003ca href=\"https://github.com/dani2442/Functional-Data-Analysis\"\u003eGitHub repository\u003c/a\u003e.\u003c/p\u003e","title":"Functional Principal Component Analysis"},{"content":"We have recently published a paper with GMV at Elsevier and I wanted to share it here also.\nLink to Article: https://www.sciencedirect.com/science/article/pii/S2352467724002248 Abstract This paper presents an exploration of the application of control theory, particularly utilizing a gradient-based algorithm, to automate and optimize the operation of photovoltaic panels and refrigeration systems in warehouse environments. The study emphasizes achieving coordination between energy generation and consumption, specifically harnessing surplus solar energy for efficient refrigeration. The complex interplay between fluctuating solar irradiance, thermal dynamics of the warehouse, and refrigeration needs underscores the significance of control theory in designing algorithms to dynamically adjust PV panel output and refrigeration system operation. The paper discusses foundational control theory principles, proposes a tailored framework for warehouse operations, and highlights the potential for sustainable energy practices. This paper explores the use of data-driven approaches based on NeuralODEs vs classical ones using physics equations.\n","permalink":"https://dani2442.github.io/posts/agraria-paper/","summary":"\u003cp\u003eWe have recently published a paper with GMV at Elsevier and I wanted to share it here also.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eLink to Article: \u003ca href=\"https://www.sciencedirect.com/science/article/pii/S2352467724002248\"\u003ehttps://www.sciencedirect.com/science/article/pii/S2352467724002248\u003c/a\u003e\u003c/strong\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"abstract\"\u003eAbstract\u003c/h2\u003e\n\u003cp\u003eThis paper presents an exploration of the application of control theory, particularly utilizing a gradient-based algorithm, to automate and optimize the operation of photovoltaic panels and refrigeration systems in warehouse environments. The study emphasizes achieving coordination between energy generation and consumption, specifically harnessing surplus solar energy for efficient refrigeration. The complex interplay between fluctuating solar irradiance, thermal dynamics of the warehouse, and refrigeration needs underscores the significance of control theory in designing algorithms to dynamically adjust PV panel output and refrigeration system operation. The paper discusses foundational control theory principles, proposes a tailored framework for warehouse operations, and highlights the potential for sustainable energy practices. This paper explores the use of data-driven approaches based on NeuralODEs vs classical ones using physics equations.\u003c/p\u003e","title":"Differentiable programming for gradient-based control and optimization in physical systems"},{"content":"What do traffic congestion, supermarket lines and fluid dynamics have in common? While we are driving, we are used to think of cars as single individuals/entities. Although, every individual has its own driving tendencies and peculiarities, at a higher-scale, we behave within certain constraints and collective behavior. This quantities can be interpreted in many cases as a homogeneous dense fluid of cars. In this post I will focus on modelling traffic flow using fluid dynamics principles.\nExtracted from Traffic Fluid Dynamics by Treiber and Kesting.\nContinuity Equation The continuity equation describes the conservation of mass (vehicles) in terms of density (traffic density) and the velocity\n$$\\tag{Continuity Equation} \\frac{\\partial \\rho}{\\partial t} \u0026#43; \\nabla \\cdot [\\rho \\pmb{v}] = 0 $$ This equation will be used along 1-dimensional axis (x), which result in the following equation $$ \\frac{\\partial \\rho}{\\partial t} \u0026#43; \\frac{\\partial (\\rho v)}{\\partial x} = 0 $$ One important quantity that will be use in this model is the flow $Q$, that can be interpreted as $Q = \\rho V$. This equation can be solved using different numerical methods (there is no analytical solution) based on finite difference or finite volume method. However, this equation is not sufficient to solve for $\\rho$ and $v$. In the following section, Lighthill, Whitham, and Richards assumed that $v$ could be expressed in terms of $\\rho$.\nIt is interesting to consider some edge cases, for example when there is a decrease(increase) in the number of traffic lanes, an on-/off-ramps intersection, etc. Lighthill-Whitham-Richards Model This model is based on the continuity equation and assumes that $Q(x,t)$ depends only on the density $\\rho$ and therefore the right term in the continuity equation can be expressed as follows \\begin{equation} \\frac{\\partial (\\rho v)}{\\partial x} = \\frac{\\partial Q}{\\partial x} = \\frac{dQ(\\rho)}{d\\rho}\\frac{\\partial \\rho}{\\partial x} \\end{equation} Therefore, the model can be rewritten \\begin{equation}\\tag{LWR Model} \\frac{\\partial \\rho}{\\partial t} + \\frac{dQ(\\rho)}{d\\rho}\\frac{\\partial \\rho}{\\partial x} = 0 \\end{equation}\nHere is an example of solution of the equation using finite differnces where the $x$-axis represent the position, $y$-axis the time, and the colors are the density of the cars (maximum 120 vehicles per km). We observe that there is a traffic jam at the 60-th km, that is becuase we reduce the number of lanes to 0.01, simulating a crash that force the flow of cars to be reduced significantly.\nWe experimented with both numerical methods:\nfirst-order approximation using finite-differences $$ \\frac{\\rho_j^{n\u0026#43;1} - \\rho_j^n}{\\Delta t} \u0026#43; \\frac{dQ_e(\\rho_j^n)}{d\\rho}\\frac{\\rho_{j}^n - \\rho_{j-1}^n}{\\Delta x} = 0 $$ Godunov\u0026rsquo;s method (Finite Volume Method): $$ \\rho^{n\u0026#43;1}_j = \\rho_j^n - \\lambda [g^G(\\rho_j^n,\\rho_{j\u0026#43;1}^n) - g^G(\\rho_{j-1}^n,\\rho_j^n)] $$ where we define $g^G(u,v)$ as follows $$ g^G(u_l, u_r):= \\begin{cases} \\min_{u\\in[u_l, u_r]} f(u) \u0026amp; \\text{if }u_l \\leq u_r\\\\ \\max_{u\\in[u_l, u_r]} f(u) \u0026amp; \\text{if }u_l \u0026gt; u_r \\end{cases} $$ Higher-Order Approximations There are high-order methods that take into account more complex behaviour. This section needs to be completed.\nExperiment: Real Scenario Using real data from Open Street Maps 1, we can simulate the traffic of a small city or neighborhood. The data is given as a graph network, where each edge is a treet segment. and many edges can intersect in nodes. As well, there are different road sizes and speeds in the network. We wanted to keep it simple at first. We use data from a small neighborhood in California, Piedmont. We code it using a high-performance mathematical framework called Jax 2. We use the graph representation form, where each operation can be expressed using the adjacency matrix. We have different features in the edges of the adjacency matrix: traffic density, boolean adjacency, road lanes, segment length. There are around 3000 nodes in the graph, therefore, the matrix is $\\mathbb{R}^{3000} \\times \\mathbb{R}^{3000} \\times \\mathbb{R}^4$ and total dimension of 3.6e10. However, in future versions we can use the sparse notation to speed up the executions.\nWe can see in the following picture, the simulation of 6 hours of traffic in this graph\nThe code is publicly available at GitHub: https://github.com/dani2442/TrafficFlow\nFuture Research Travel time estimation, Fuel consumption and emissions, traffic flow optimization\n","permalink":"https://dani2442.github.io/posts/traffic-flow/","summary":"\u003cp\u003eWhat do traffic congestion, supermarket lines and fluid dynamics have in common? While we are driving, we are used to think of cars as single individuals/entities. Although, every individual has its own driving tendencies and peculiarities, at a higher-scale, we behave within certain constraints and collective behavior. This quantities can be interpreted in many cases as a homogeneous dense fluid of cars. In this post I will focus on modelling traffic flow using fluid dynamics principles.\u003c/p\u003e","title":"Traffic Flow Dynamics"},{"content":"Several non-linear problems relevant in practical applications can be expressed as a fixed point equation. In many cases, it is crucial to investigate how the model’s behavior changes with variations in a parameter, denoted as $\\lambda$. In practical applications, $\\lambda$ represents a physical or empirical magnitude of interest. Bifurcation Theory is a subfield in Nonlinear Functional Analysis that tries to study the general behavior of the equations that can be written as $\\mathfrak{F}(\\lambda, u)=0$ where $\\lambda$ is the bifurcation parameter.\nExamples of Bifurcations One-Dimensional Example Considering the one-dimensional problem where $U=\\mathbb{R}$ $$ \\mathfrak{F}(\\lambda, u) = \\lambda u \u0026#43; \\frac{1}{2}\\cos(2\\lambda)u^2 $$ Here, $\\mathfrak{F}$ exhibits a bifurcation behavior at $(\\lambda, u)=(0,0)$ as shown in the following figure. Here, we have a trivial branch of solutions $(\\lambda, u)=(\\lambda, 0)$ for every $\\lambda\\in\\mathbb{R}$.\nLogistic Map Another example of bifurcation emerges from the logistic map, $$ x_{n\u0026#43;1} = rx_n(1-x_n) $$ The chaotic behavior of arises from a very simple nonlinear dynamical equation. And it describes the many physical effects such as the population reproduction and starvation of a population, i.e., given a population $x\\in [0,1]$ where 1 represent the population maximum capacity, the population of the next year can be given by $x_{n\u0026#43;1} = rx_n$. However, we need to further constraint to the maximum capacity, and that is why the factor $(x_n-1)$ appears. This equation exhibits a bifurcation behavior in the stable points (the limit of the sequences) where we take the bifurcation parameter $r$. You can see in the next figure that multiple bifurcation points emerge (the most notorious is when $r=2$) Bifurcation from Simple Eigenvalues One of the most important theorems in Bifurcation Theory is the Crandall-Rabinowitz Theorem [Crandall \u0026amp; Rabinowitz, 1971] which state the sufficient conditions to find a simple bifurcation point, it is build upon the Implicit Function Theorem and therefore the convergence of the Newton method. Firstly, it defines a function $\\mathfrak{G}$ as follows\n$$ \\tag{1} \\mathfrak{G}(s, \\lambda, y):= \\begin{cases} s^{-1}\\mathfrak{F}(\\lambda, s(\\varphi_0 \u0026#43; y)) \u0026amp; \\text{if }s\\not=0\\\\ D_u\\mathfrak{F}(\\lambda, 0)(\\varphi_0 \u0026#43; y)\u0026amp; \\text{if }s=0 \\end{cases} $$ where $\\varphi_0$ is the simple eigenvector of $D_u \\mathfrak{F}_u(\\lambda_0)$ whose corresponding eigenvalue is 0. Using the Implicit Function Theorem, we can prove that there exists two unique mappings $\\lambda(s)$ and $y(s)$ such that $\\mathfrak{G}(s, \\lambda(s), y(s))=0$.\nApplication to equation: $-u\u0026#39;\u0026#39; = \\lambda u - a(x) u^3$ We will study the general case of the equation $$\\tag{2} \\begin{cases} -u\u0026#39;\u0026#39; = \\lambda u - a(x)u^3\\quad\\quad \\text{in}\\quad (0,L),\\\\ u(0) = u(L) = 0 \\end{cases} $$ where $a\\in\\mathcal{C}[0,L]$. Using the Crandall-Rabinowitz Theorem, we can prove that there are bifurcation points at $(\\lambda, u) = (\\frac{n^2\\pi^2}{L^2}, 0)$ for every $n\\geq 1$. And the bifurcation diagram will look like the following figure. [López-Gomez, 2020]\nWe will introduce in the following section some numerical methods for bifurcation theory.\nNumerical Methods in Bifurcation Theory The numerical methods in bifurcation theory can be categorized into three main groups:\nContinuation methods: These methods extend a branch fo solutions from a given solution point, progressing in both directions along the branch. Bifurcation point detection: These methods identify bifurcation points within a branch of solutions, where qualitative changes in the system behavior occur. Branch Switching: Starting from a detected bifurcation point, these methods facilitate the exploration of other solution branches that intersect at that bifurcation point. Continuation Methods This method are most two-steps based algorithms, the first part give a plausible guess for the extension of the branch and the second one corrects the guess point to satisfy the equation. That is the reason why they are called prediction-correction methods.\nPredicion Method Correction Method Yuri A. Kuznetsov. Elements of Applied Bifurcation Theory, 2023 Prediction: The main core of the algorithm is to use $v^j$ to calculate the next point $\\tilde{y}^{j\u0026#43;1}$. $$ \\tilde{y}^{j\u0026#43;1} := y^j \u0026#43; \\eta \\frac{v^j}{|v^j|} $$ The most simple methods are the secant and the tangent method. The secant method uses previous points to calculate the extrapolate the next possible value $$v^j := y^j - y^{j-1}.$$ A more sophisticated method is to use the tangent at the given point to calculate the next point. The method can be derived from the initial condition: $$ \\mathfrak{F}(y^j) = 0 \\Leftrightarrow \\mathfrak{F}_y (y^j)v^j = 0 $$ This equation does not have a unique solution because $\\mathfrak{F}: \\mathbb{R}\\times U \\rightarrow U$, therefore, we need another equation to provide a unique solution. A common trick is to use restraint with the following condition: $\\langle v^{j-1}, v^j \\rangle = 1$.\nCorrection: The correction uses a Newton-based method in most cases. It uses two equations $$ \\begin{cases} \\mathfrak{F}(y) = 0 \\\\ g^j (y) = 0 \\end{cases} $$ The simplest method, also called Natural, defines $g^j$ as follows: $g^j(y) = y_{i_0} - \\tilde{y}_{i_0}^{j\u0026#43;1}$. Geometrically, it restraints to solution to the prediction $i_0$-plane. However, it is noteworthy that this method does not work when the curve is turning in that axis.\nAn alternative method is the so called Pseudo-arclength that defines $g^j(y) = \\langle y-\\tilde{y}^{j\u0026#43;1} , v^j \\rangle$. Intuitively, it restraints the solutions to the perpendicular plane to $v^j$ at the point $\\tilde{y}^{j\u0026#43;1}$.\nBifurcation Point Detection Defining the following test function \\begin{equation} \\psi(y^k) = \\lambda_1\\lambda_2\\dots \\lambda_n \\end{equation} where $\\lambda_n$ are the eigenvalues of the $D_u \\mathfrak{F}(y^k)$. We can test whether two consecutive points are a fold bifurcation point when \\begin{equation} \\psi(y^k)\\psi(y^{k+1}) \u0026lt; 0 \\end{equation}\nOnce, detected a bifurcation point, we can improve the point precision using an iterative algorithm based on the Bisection method:\n$\\tilde{y}^{k\u0026#43;1/2} := \\frac{1}{2}(y^{k} \u0026#43; y^{k\u0026#43;1})$ Use a correction method on $\\tilde{y}^{k\u0026#43;1/2}$ to obtain $y^{k\u0026#43;1/2}$. Calculate $\\psi(y^{j\u0026#43;1/2})$ and test it again: $$ \\begin{cases} \\text{Bisection}(y^{k}, y^{k\u0026#43;1/2}) \u0026amp; \\text{If }\\psi(y^{k})\\psi(y^{n})\u0026lt;0\\\\ \\text{Bisection}(y^{k\u0026#43;1/2}, y^{k\u0026#43;1}) \u0026amp; \\text{If }\\psi(y^{n})\\psi(y^{k\u0026#43;1})\u0026lt;0 \\end{cases} $$ Branch Switching We will use a method based on the proof of the Crandall-Rabinowitz Theorem [], where we will apply a Newton-like algorithm. Let us use the following function $$ \\mathfrak{G}(s, \\lambda, y) := \\begin{cases} s^{-1}\\mathfrak{F}(\\lambda, s(\\varphi_0 \u0026#43; y)) \u0026amp; \\text{if } s\\not= 0\\\\ D_u\\mathfrak{F}(\\lambda, 0)(\\varphi_0 \u0026#43; y) \u0026amp; \\text{if } s=0 \\end{cases} $$ Furthermore, we restraint the solution $y$ to the closed subspace of $U$ such that $Y \\oplus N[D_u \\mathfrak{F}(y)] = U$. Hence, $y\\in N[\\mathfrak{L}_0]^\\perp = N[\\mathfrak{F}_u(\\lambda_0, u_0)]^\\perp$. This condition can be expressed as follows $$ \\langle y, \\phi \\rangle=0 $$ where $\\text{Span}[\\phi] = N[\\mathfrak{F}^T_u (\\lambda_0, u_0)]$ due to the Fundamental Theorem of Linear Algebra. Therefore, Crandall-Rabinowitz assures us that the Newton method converges to a solution in the non-trivial branch.\nResults Using the above-mentioned algorithms to the problem described in (1), we obtain the following bifurcation diagram, that coincides with the results obtained using the Crandall-Rabinowitz Theorem The full code has been uploaded to the public repository: https://github.com/dani2442/bifurcationjax\n","permalink":"https://dani2442.github.io/posts/bifurcation-theory/","summary":"\u003cp\u003eSeveral non-linear problems relevant in practical applications can be expressed as a fixed point equation. In many cases, it is crucial to investigate how the model’s behavior changes with variations in a parameter, denoted as $\\lambda$. In practical applications, $\\lambda$ represents a physical or empirical magnitude of interest. Bifurcation Theory is a subfield in Nonlinear Functional Analysis that tries to study the general behavior of the equations that can be written as $\\mathfrak{F}(\\lambda, u)=0$ where $\\lambda$ is the \u003cem\u003ebifurcation parameter\u003c/em\u003e.\u003c/p\u003e","title":"Nonlinear Functional Analysis: Bifurcation Theory"},{"content":"Hi, I\u0026rsquo;m Daniel López Montero. I\u0026rsquo;m a PhD student in mathematics at Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), in the Chair for Dynamics, Control, Machine Learning and Numerics (Alexander von Humboldt Professorship), supervised by Prof. Enrique Zuazua.\nMy research sits between machine learning and control theory: kernel methods and Gaussian processes, neural ODEs, physics-informed learning, data-driven control, and hybrid models of whole-brain dynamics. Before FAU I did an MSc in Mathematics and Applications at the Universidad Autónoma de Madrid (UAM) and worked at GMV on applied projects in energy systems, space operations and cybersecurity.\nIn winter semester 2026/27 I\u0026rsquo;m the teaching assistant for Prof. Zuazua\u0026rsquo;s course Mathematical Foundations of Control and Machine Learning.\nOn this site I post notes, projects and essays on topics I\u0026rsquo;m interested in. Talks and slides are in events. Outside research I like to program, read, and play football, basketball and padel.\nPublications Gaussian mixture models in Hilbert spaces via kernel methods. D. López-Montero, A. Álvarez-López, M. Matabuena. NeurIPS 2026. arXiv:2605.05996 Data-driven control in infinite-dimensional spaces: fundamental lemma and applications. D. López-Montero. Preprint, 2026. arXiv:2609.25431 Hybrid modeling of whole-brain dynamics balances data fidelity and interpretability. D. López-Montero, L. Liverani, E. Zuazua, X. Kobeleva. Preprint, 2026. Reinforcement learning for automated cybersecurity penetration testing. D. López-Montero, J. L. Álvarez-Aldana, A. Morales-Martínez, M. Gil-López, J. M. Auñón García. Preprint, 2025. arXiv:2507.02969 Differentiable programming for gradient-based control and optimization in physical systems. D. López-Montero, P. Hernando-Sánchez, M. Limones-Andrade, et al. Sustainable Energy, Grids and Networks 39, 101495, 2024. Article · Blog post Towards INtelligent automated Functional and Security Testing (INFAST). D. López-Montero, A. Benítez-Buenache, N. Carvalho-dos-Santos, et al. 17th International Conference on Space Operations (SpaceOps), 2023. The up-to-date list is on Google Scholar and FAU CRIS.\nContact Email: dlopez31415@gmail.com · GitHub · LinkedIn · ORCID\n","permalink":"https://dani2442.github.io/about/","summary":"\u003cp\u003eHi, I\u0026rsquo;m Daniel López Montero. I\u0026rsquo;m a PhD student in mathematics at \u003ca href=\"https://www.fau.eu/\"\u003eFriedrich-Alexander-Universität Erlangen-Nürnberg (FAU)\u003c/a\u003e, in the \u003ca href=\"https://dcn.nat.fau.eu/\"\u003eChair for Dynamics, Control, Machine Learning and Numerics\u003c/a\u003e (Alexander von Humboldt Professorship), supervised by \u003ca href=\"https://dcn.nat.fau.eu/enrique-zuazua/\"\u003eProf. Enrique Zuazua\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eMy research sits between machine learning and control theory: kernel methods and Gaussian processes, neural ODEs, physics-informed learning, data-driven control, and hybrid models of whole-brain dynamics. Before FAU I did an MSc in Mathematics and Applications at the Universidad Autónoma de Madrid (UAM) and worked at GMV on applied projects in energy systems, space operations and cybersecurity.\u003c/p\u003e","title":"About"}]