Overview: Making Time Flow

ODE Methods & Conservation | Numerical Methods Module 3 | COMP 536: Modeling the Universe

Author

Anna Rosen

The Big Picture: Learning Dynamics Through Failure

A Story That Changes Everything

In 1889, King Oscar II of Sweden offered a prize for solving the n-body problem — predicting the motion of multiple gravitationally interacting bodies forever into the future. Henri Poincaré won by proving something shocking: the problem was impossible to solve analytically for n \(\geq\) 3. Even worse, he showed that tiny differences in initial conditions led to completely different outcomes. Chaos was born.

But here’s what happened next: Instead of giving up, astronomers turned to numerical integration. By 1960, they were using computers to track planetary positions centuries into the future. By 1990, they could simulate the solar system for millions of years. Today, we simulate galaxy collisions over billions of years and track spacecraft to Jupiter with meter precision.

The key insight that made this possible wasn’t faster computers or better mathematics — it was discovering that not all integration methods are created equal. Some methods that seem accurate destroy the physics they’re trying to simulate. Others that seem crude preserve what matters. The difference between a planet spiraling into the sun and orbiting stably for a billion years isn’t computational power — it’s choosing the right algorithm.

This is the heart of what you’re about to learn: how to make time flow numerically while preserving the fundamental laws of physics.

Your Mission: Preserve Physics Through Time

You’re about to discover that simulating the universe’s evolution is a battle against accumulating errors:

Why does Earth’s orbit spiral outward with Euler’s method? Energy systematically increases
Why do pulsars lose synchronization in simulations? Phase errors accumulate relentlessly
How can we simulate galaxies for cosmic time? Symplectic integrators preserve geometry
When do numerical methods explode? Stability analysis predicts catastrophic failure

But here’s the revelation: geometric structure matters more than local accuracy. A second-order method that preserves phase space beats a tenth-order method that violates conservation laws. Master this principle, and you can simulate anything from binary pulsars to galaxy mergers.

Why This Matters Now More Than Ever

Modern astrophysics runs on temporal evolution:

LIGO detections require integrating Einstein’s equations to generate waveform templates
Gaia’s billion-star catalog needs orbit integration to detect perturbations from dark matter
JWST observations of early galaxies require N-body simulations for comparison
Exoplanet discoveries depend on integrating orbital dynamics with extreme precision
Dark energy constraints come from simulating structure formation over cosmic time

You NEED these methods to do modern astrophysics. This module ensures you understand not just how to call scipy.integrate.odeint, but why symplectic integrators keep planets stable, when implicit methods become essential, and how to diagnose numerical disasters before they happen.

Quick Navigation Guide

🎯 Choose Your Learning Path

🏃 Fast Track

Essential concepts only

🚶 Standard Path

Full conceptual understanding

Everything in Fast Track, plus:

Energy drift analysis
RK derivations
Symplectic proof
Stiff equations
All “Check Your Understanding” boxes

🧗 Complete Path

Deep dive with all details

Complete module including:

All mathematical derivations
Modified Hamiltonian
Adaptive timesteps
Implicit methods
Higher-order symplectic
All worked examples

🎯 Navigation by Project Needs

Quick Jump to What You Need by Project

For Project 2 (N-body Dynamics): - Leapfrog implementation - Your main integrator - Energy monitoring - Conservation checks - Timestep selection - Setting h appropriately - Close encounters - Handling singularities - Phase space structure - Understanding orbits

For Project 3 (Monte Carlo Radiative Transfer): - Ray integration - Path through varying media - Adaptive methods - Efficiency in varying opacity - Stability for stiff cooling - Radiative processes

For Project 4 (MCMC): - Hamiltonian Monte Carlo - Symplectic integration - Detailed balance - Preserving equilibrium - Trajectory length - Choosing integration time

For Project 5 (Gaussian Processes): - SDE integration - Stochastic differential equations - Kernel evolution - Time-dependent covariance - Stability analysis - Numerical convergence

For Final Project (Neural Networks): - Gradient descent as ODE - Training dynamics - Learning rate as timestep - Adaptive optimization - Momentum methods - Accelerated convergence - Stability of SGD - When training diverges

💭 Why This Module Exists: A Personal Note from Your Instructor

This module has a secret mission: teaching you that numerical integration isn’t about memorizing Runge-Kutta coefficients — it’s about understanding the deep conflict between discrete computation and continuous physics.

I’ve watched too many students implement RK4 perfectly, then wonder why their solar system flies apart after a million years. They increase the order to RK8, use smaller timesteps, switch to adaptive methods — nothing works. The planets still spiral into the sun or escape to infinity.

What makes this different: We start with failure. You’ll watch Euler’s method destroy a simple harmonic oscillator, see energy grow exponentially when it should be constant. This isn’t a bug — it’s a fundamental property of the discretization. Then you’ll discover that symplectic methods, which look almost identical to Euler, preserve energy for billions of years.

The revelation students have: - Wait, higher accuracy makes things WORSE for long-term integration? Sometimes! - A second-order method beats fourth-order for orbits? If it’s symplectic! - The same equations that govern planets also train neural networks? Exactly!

This module makes those connections explicit from the start.

Here’s what ODE integration actually is: creating alternate realities with slightly different physics. Euler creates a universe where energy spontaneously appears. RK4 creates one where energy slowly leaks away. Leapfrog creates one where energy oscillates but stays bounded. Your job is choosing which alternate reality best preserves the physics you care about.

By the end, you’ll understand not just the formulas but the deep principles: why geometric structure beats local accuracy, when stability matters more than precision, and how the same mathematical framework spans from planetary dynamics to machine learning. These aren’t separate topics — they’re all facets of making time flow while preserving what matters.

Mathematical Foundations

📖 Core Notation and Concepts

Before diving in, let’s establish the mathematical language for temporal evolution:

ODE Notation

Symbol	Meaning	Physical Example
\(\frac{dx}{dt} = f(x,t)\)	General ODE	Equation of motion
\(x(t_0) = x_0\)	Initial condition	Starting position/velocity
\(h\) or \(\Delta t\)	Timestep	Integration step size
\(x_{n+1} = \Phi_h(x_n)\)	Discrete map	Numerical integrator
\(\tau\)	Local truncation error	Per-step error
\(E\)	Global error	Total accumulated error

Hamiltonian Notation

Symbol	Meaning	Physical Example
\(H(q,p)\)	Hamiltonian	Total energy
\(q\)	Generalized coordinates	Positions
\(p\)	Conjugate momenta	Momenta
\(\{f,g\}\)	Poisson bracket	Structure preservation
\(\omega\)	Symplectic form	Phase space geometry

Key Concepts Preview

Order of accuracy: How error scales with timestep - Local: \(\tau = O(h^{p+1})\) per step - Global: \(E = O(h^p)\) over interval

Stability: When methods remain bounded - Absolute stability: \(|R(h\lambda)| \leq 1\) - A-stability: Stable for entire left half-plane

Symplecticity: Preservation of phase space structure - Volume preservation (Liouville) - Bounded energy error - Time reversibility

Why This Matters: Every simulation creates a discrete approximation to continuous physics. Understanding these approximations — their errors, stability, and structure preservation — determines whether your simulations reveal truth or nonsense.

Module Contents

Part 1: The Failure of Naive Integration

Witness Euler’s method catastrophically violate energy conservation. Understand why local accuracy doesn’t guarantee global stability. See how systematic geometric errors compound until physics is destroyed. Learn to recognize failure modes before they corrupt your simulations.