Antares Chen University of Chicago
We introduce the Sherrington-Kirkpatrick model and Parisi’s variational principle. We motivate key mathematical objects that emerge from proving the principle: (1) replica symmetry breaking, (2) Guerra-Toninelli interpolation, (3) ultrametricity, (4) pure state decompositions, (5) the Ruelle Probability Cascades, and (6) the cavity method. We conclude by outlining the high-level organization of subsequent presentations and a motivation for why study the Parisi variational principle from an algorithmic standpoint.
We introduced the Sherrington-Kirkpatrick model, translating spin glass vocabulary such as overlap, the partition function, and free energy, to concepts in optimization.
We stated the Parisi Variational Principle and attempted to build a geometric picture of how the dynamics in Parisi’s equations evolve.
We stated an anology between Parisi’s equations and Hamilton-Jacobi equations where the “Hamiltonian” is given by the L2-norm squared.
We outlined various conjectures regarding how certain first-order optimization methods such as (stochastic) gradient descent might arise from a more algorithmic proof of Parisi’s Variational Principle, using how we currently understand them to arise from the studying Hamilton-Jacobi equations as an analogy.
We stated the over-arching theme of this tutorial sequence, and outlined plans for subsequent meetings.
Next time we focus on the mathematical objects that arise in proving Parisi’s variational principle, and begin discussing the replica symmetry (breaking) ansatzen.
Historical origins of Parisi’s variational principle
Proving the Parisi variational principle
Pure state geometry of the Gibbs distribution
Modern algorithmic applications of the variational principle