This module builds your understanding from first principles, following the progression of Projects 1-3. We start with scalars and vectors describing individual stellar properties and positions (Project 1), build to matrices transforming entire systems (Project 2), and explore how eigenvalues reveal stability and oscillation modes (Project 3). Module 0b will extend these foundations to statistical methods and machine learning.
Part 1: The Opening - A Globular Cluster as a Mathematical Universe¶
Priority: 🟡 Important - Provides physical context for abstract concepts
📅 When You’ll Use This
First appears in: Project 2 (N-body dynamics)
Critical for: Understanding phase space throughout the course
Returns in: Every project dealing with multi-particle systems
1.1 The Physical System That Motivates Everything¶
Consider Omega Centauri, the most massive globular cluster orbiting our galaxy. It contains approximately 10 million stars, all gravitationally bound, orbiting their common center of mass for the past 12 billion years. At its core, stellar densities reach thousands of stars per cubic parsec – if Earth orbited a star there, our night sky would blaze with thousands of stars brighter than Venus.
To simulate this cluster, we need to track for each star:
Position:ri=(xi,yi,zi) measured from the cluster center
Velocity:vi=(vx,i,vy,i,vz,i) relative to the cluster’s motion
That’s 6 million numbers evolving according to Newton’s laws. The gravitational force on star i is:
Each star feels forces from all others. For N=107 stars (Omega Centauri’s estimated count), the number of unique pairwise interactions is N(N−1)/2≈5×1013. That’s approximately 50 trillion unique force pair calculations per timestep! Without linear algebra, this would be impossible. With it, we can organize these calculations efficiently, identify conserved quantities, and understand the cluster’s long-term evolution.
where mi is the mass of star i, vi is its velocity vector, G is the gravitational constant, and ∣ri−rj∣ is the distance between stars i and j. The first term is the total kinetic energy and the second term is the total gravitational potential energy (summed over all unique pairs).
where ri is the position vector of star i from the cluster center and × denotes the cross product. This measures the total rotational momentum of the system.
These conservation laws aren’t accidents – they arise from fundamental symmetries of space and time. Emmy Noether proved that every continuous symmetry implies a conservation law:
Symmetry
Conservation Law
Mathematical Structure
Time translation invariance
Energy
Quadratic forms
Space translation invariance
Momentum
Vector addition
Rotational invariance
Angular momentum
Cross products, orthogonal matrices
These symmetries manifest mathematically as properties of vectors and matrices. Translation invariance means physics doesn’t change when we add the same vector to all positions. Rotational invariance means physics is preserved under orthogonal transformations.
🌟 The More You Know: Emmy Noether’s Revolutionary Theorem
Priority: 🟢 Enrichment
Emmy Noether (1882-1935) fundamentally changed physics with her 1915 theorem connecting symmetries to conservation laws. Despite facing discrimination that prevented her from holding a paid position for years, she persisted in her work at the University of Göttingen.
When Einstein’s general relativity seemed to violate energy conservation, it was Noether who resolved the paradox. She proved that in general relativity, energy conservation becomes local rather than global—a subtle but profound insight. Einstein wrote to Hilbert: “Yesterday I received from Miss Noether a very interesting paper on invariant forms. I’m impressed that such things can be understood in such a general way” (Letter from Einstein to Hilbert, May 24, 1918, as documented in The Collected Papers of Albert Einstein, Volume 8).
Her theorem now underlies all of modern physics. Every conservation law you use — energy, momentum, angular momentum, electric charge — exists because of an underlying symmetry. When you implement conservation checks in your N-body code, you’re using Noether’s profound insight that geometry determines physics.
Source: Osen, L. M. (1974). Women in Mathematics. MIT Press. pp. 141–152.
Priority: 🟡 Important - Conceptual foundation for Projects 2-6
Here’s the profound insight that changes everything: the million-star cluster isn’t really moving through 3D space. It’s tracing a trajectory through a 6-million-dimensional phase space where each axis represents one position or velocity component. The system’s entire state is a single point in this vast space, and its time evolution is a trajectory through it.
This perspective reveals hidden simplicities:
Conservation laws constrain the trajectory to a lower-dimensional surface - Just as a ball rolling in a bowl is confined to the bowl’s 2D surface despite existing in 3D space, the cluster’s evolution is restricted to a much smaller subspace where energy, momentum, and angular momentum remain constant. The 6-million dimensions collapse to far fewer effective dimensions.
Near equilibrium, motion decomposes into independent oscillation modes (eigenvectors) - Like a drum that vibrates in distinct patterns (fundamental, overtones), the cluster has natural oscillation modes. Small disturbances trigger these modes independently, each oscillating at its own characteristic frequency - these are the eigenvectors and eigenvalues of the system.
Statistical properties emerge from the geometry of this high-dimensional space - Velocity dispersion, dynamical stability, and evolution rates aren’t separate properties but geometric features of how the system explores phase space. The volume it occupies relates to entropy, the shape of its trajectory relates to energy distribution, and the curvature of energy surfaces determines bound vs. unbound orbits. (See Module 1a, Section 4 for the deep connection between stellar dynamics and statistical mechanics.)
Part 2: Vectors - The Atoms of Physical Description¶
Priority: 🔴 Essential - Foundation for everything that follows
2.1 From Scalars to Vectors: Building Deep Understanding {#part-2-vectors}¶
The simplest quantities in physics are scalars – single numbers that have magnitude but no direction. Mass (2×1033 g), temperature (5800 K), luminosity (3.8×1033 erg/s), and density (1.4 g/cm³) are all scalars. They tell us “how much” but not “which way.” For many physical quantities, however, magnitude alone is insufficient – we need direction too. This is where vectors enter.
A vector is simultaneously three complementary things, and masterful computational scientists fluidly shift between these perspectives:
Physical Perspective: A vector represents any quantity with both magnitude and direction. The velocity of Earth orbiting the Sun is a vector — it has a speed (30 km/s) and a direction (tangent to the orbit). Forces, electric fields, angular momenta — all are vectors because they have this magnitude-direction character that scalars cannot express.
Geometric Perspective: A vector is an arrow in space. Crucially, this arrow is free — it doesn’t have a fixed starting point. The displacement “3 km north” is the same vector whether you start from your house or from campus. This freedom is what allows us to add forces acting at different points on a rigid body.
Algebraic Perspective: A vector is an ordered list of numbers — its components in some coordinate system. Earth’s velocity might be written as:
v=⎝⎛−15.225.80.0⎠⎞ km/s (in ecliptic coordinates)
But here’s the crucial insight: these numbers are not the vector itself – they’re just one representation. The vector exists independently of any coordinate system.
A vector space is a set equipped with two operations (vector addition and scalar multiplication) that satisfy eight axioms. These axioms aren’t arbitrary mathematical rules – each captures an essential physical property:
Axiom
Mathematical Statement
Physical Meaning
Closure under addition
u+v∈V
Adding velocities gives velocity
Commutativity
u+v=v+u
Order of displacements doesn’t matter
Associativity
(u+v)+w=u+(v+w)
Grouping of forces doesn’t affect total
Zero vector
∃0:v+0=v
State of no motion or displacement
Additive inverse
∀v,∃−v:v+(−v)=0
Every motion has an opposite
Scalar closure
cv∈V
Scaling a force gives another force
Distributivity
c(u+v)=cu+cv
Scaling preserves addition
Identity
1⋅v=v
Multiplying by 1 changes nothing
Linear independence means vectors cannot be written as combinations of each other - they represent truly independent directions in space.
2.3 The Dot Product: Projection, Angle, and Energy¶
Priority: 🔴 Essential - Used constantly throughout all projects
The dot product is perhaps the most important operation in physics because it answers a fundamental question: “How much does one vector contribute in the direction of another?”
Only the component of force along the displacement does work. A force perpendicular to motion (like the normal force on a sliding block) does zero work.
i-component: Take the y and z components: (aybz−azby)
j-component: Take the x and z components with a minus sign: −(axbz−azbx)
k-component: Take the x and y components: (axby−aybx)
Notice the pattern: each component uses the two coordinates that aren’t its own, and the j-component has a negative sign (this comes from the alternating signs in determinant expansion).
Geometric Intuition: To find the direction of a×b, use the right-hand rule: curl your right hand’s fingers from the first vector a toward the second vector b through the smaller angle. Your thumb points in the direction of the cross product. This isn’t arbitrary - it ensures consistency with our choice of right-handed coordinate systems.
Key Properties:
Magnitude: ∣a×b∣=∣a∣∣b∣sinθ (area of parallelogram)
Priority: 🟡 Important - Essential for understanding basis vectors
Two vectors are orthogonal (perpendicular) when their dot product is zero: a⋅b=0. This isn’t just a geometric curiosity - orthogonality is why we can decompose complex systems into independent components.
Why Orthogonal Bases Are Special:
When basis vectors are orthonormal (orthogonal AND unit length), calculations become dramatically simpler:
Projections are independent: The component along one axis doesn’t affect others
Pythagoras works: ∣v∣2=vx2+vy2+vz2
Rotations preserve dot products: Orthogonal transformations preserve angles and lengths
Physical Example: In quantum mechanics, energy eigenstates are orthogonal. This means measuring one energy level doesn’t affect the probability of finding the system in another level. The orthogonality of spherical harmonics is why we can decompose the cosmic microwave background into independent multipole moments!
Astrophysical Application: When tracking stellar proper motions, observations often come in non-orthogonal coordinate systems (right ascension, declination, radial velocity). Orthogonalization creates independent velocity components where each represents truly independent motion - essential for understanding 3D stellar kinematics and discovering moving groups in the solar neighborhood.
Priority: 🟢 Enrichment - Helps understand QR decomposition
Physical Motivation: You’re observing a galaxy and measure velocities in a skewed coordinate system. Gram-Schmidt creates an orthonormal basis where velocity components are independent - essential for understanding the true motion patterns!
The Gram-Schmidt process creates an orthonormal basis from any linearly independent set. This is essential for understanding QR decomposition.
Given vectors: v1=(1,1,0), v2=(1,0,1), v3=(0,1,1)
You’ve mastered vectors - quantities with magnitude and direction. But what happens when you need to transform many vectors simultaneously? This is where matrices emerge naturally. A matrix isn’t just a grid of numbers - it’s a machine that transforms entire vector spaces. When you multiply a matrix by a vector, you’re asking: “Where does this vector go under this transformation?” This perspective transforms matrices from abstract number arrays into concrete geometric operations. When we multiply matrix A by vector v, we get a new vector Av. The crucial property is linearity:
Note: 23≈0.866 and 21=0.5. The z-component is unchanged (last row is [0, 0, 1]) because we rotate around z. Imagine looking down the z-axis: this matrix spins everything counterclockwise by 30° like a record player, preserving all lengths.
Doubles x, triples y, leaves z unchanged. Only diagonal elements are non-zero! This stretches space non-uniformly - imagine stretching a rubber sheet more in one direction than another.
3.4 The Determinant: Volume, Orientation, and Information¶
Priority: 🟡 Important - Helps understand singularity and numerical stability
The determinant tells us three crucial things:
det(A)
Meaning
Physical Interpretation
∣det(A)∣
Volume scaling factor
How much the transformation stretches/shrinks space
det(A)>0
Preserves orientation
Right-handed stays right-handed
det(A)<0
Flips orientation
Right-handed becomes left-handed
det(A)=0
Singular (non-invertible)
Information is lost, dimension collapses
When det(A)=0, the matrix is singular and has rank less than its dimension. Let’s understand what this really means:
Intuitive Understanding of Rank:
Full rank (rank = n for n×n matrix): All rows/columns point in independent directions. The matrix preserves dimensionality—3D space stays 3D.
Rank deficient (rank < n): Some rows/columns are linear combinations of others. The matrix collapses space—maybe 3D becomes a 2D plane or even a 1D line.
This matrix has rank 2, not 3! The third column (log luminosity) is completely determined by the second column. We’re really only measuring 2 independent quantities, not 3. The covariance matrix of this data would be singular—we can’t invert it because one dimension is redundant.
Geometric Interpretation:
Rank 3 matrix: Maps 3D space to full 3D space
Rank 2 matrix: Squashes 3D space onto a 2D plane
Rank 1 matrix: Collapses 3D space onto a 1D line
Rank 0 matrix: Maps everything to a single point (zero matrix)
Specifically, an n×n matrix with rank r<n maps n-dimensional space onto an r-dimensional subspace, losing (n−r) dimensions of information.
Geometric interpretation: For 2×2 matrices, the determinant equals the (signed) area of the parallelogram formed by the column vectors. For 3×3 matrices, it’s the (signed) volume of the parallelepiped.
For a 3×3 matrix, the determinant can be computed by cofactor expansion along any row or column. The pattern extends to larger matrices, though computation becomes expensive (O(n!) operations).
Notice you divide by the determinant – this is why singular matrices (det = 0) have no inverse!
Bridge to Module 0b: These matrix operations become statistical in Module 0b, where ATA creates covariance matrices encoding correlations. The transpose operation you learned here will be essential for understanding how data matrices generate covariance structures.
These eigenvectorsv and their eigenvaluesλ reveal fundamental structure.
Geometric Intuition: Imagine stretching a rubber sheet. Most directions get rotated as the sheet deforms, but along eigenvector directions, points only move closer to or farther from the origin without changing direction. These are the “natural axes” of the transformation.
Physical Example - Spinning Objects:
Consider a football. It has three principal axes:
Long axis (through the points)
Two short axes (perpendicular to long axis)
Spin it around the long axis → stable rotation
Spin it around a short axis → stable rotation
Spin it at any other angle → it wobbles!
The principal axes are eigenvectors of the moment of inertia tensor.
4.2 Finding Eigenvalues: The Characteristic Equation¶
To find eigenvalues:
Rearrange: (A−λI)v=0
For non-trivial solutions: det(A−λI)=0
This gives the characteristic equation
For an n×n matrix, the characteristic polynomial has degree n, potentially yielding n eigenvalues (counting multiplicity) in the complex numbers. The trace (sum of diagonal elements) equals the sum of eigenvalues.
If λ<0: perturbation decays → stable (like a ball in a valley)
If λ>0: perturbation grows → unstable (like a ball on a hill)
If λ is complex: oscillation with growth/decay (spiral behavior)
The largest eigenvalue determines the fate: even one positive eigenvalue means the system will eventually fly apart!
For instance, a 2D rotation matrix has eigenvalues e±iθ, where the imaginary unit explicitly encodes the rotation angle θ. This shows how complex eigenvalues naturally describe oscillatory behavior in physical systems.
Eigenvalues appear in most projects in this course:
Project
Where Eigenvalues Appear
What They Tell You
Project 2
Linearized dynamics near equilibrium
Orbital stability (stable if all λ<0)
Project 3
Scattering matrix
Preferred scattering directions
Project 4
MCMC transition matrix
Convergence rate: ∼1/∣1−λ2∣ where λ2 is the second-largest eigenvalue by absolute value (with ∣λ2∣<1)
Project 5
GP kernel matrix
Effective degrees of freedom
Final Project
Neural network Hessian
Optimization landscape curvature
🌟 The More You Know: Jacobi’s Method Born from Astronomy
Priority: 🟢 Enrichment
Carl Gustav Jacob Jacobi (1804-1851) developed his famous eigenvalue algorithm while studying the rotation of celestial bodies. In 1846, he published “Über ein leichtes Verfahren die in der Theorie der Säcularstörungen vorkommenden Gleichungen numerisch aufzulösen” (On an easy method to numerically solve equations occurring in the theory of secular perturbations).
The problem arose from calculating planetary perturbations — small gravitational influences planets exert on each other. These perturbations accumulate over centuries (hence “secular”) and are described by symmetric matrices whose eigenvalues determine the long-term stability of orbits.
Jacobi’s insight was to diagonalize the matrix through a sequence of rotations, each zeroing out one off-diagonal element. Though computers didn’t exist, his method was designed for hand calculation — each step is simple enough to do with pencil and paper. Today, variants of Jacobi’s method run on every supercomputer simulating galaxy collisions or climate models.
The next time your code calls numpy.linalg.eigh(), remember: you’re using an algorithm invented to predict whether the solar system is stable over millions of years!
Source: Jacobi, C.G.J. (1846). Journal für die reine und angewandte Mathematik (Crelle’s Journal), 30, pp. 51-94.
📚 Mathematical Deep Dive: Proof that Symmetric Matrices Have Real Eigenvalues
Priority: 🟢 Enrichment
This proof reveals why physics “prefers” symmetric matrices.
Theorem: Every real symmetric matrix has only real eigenvalues.
Proof:
Let A be a real symmetric matrix and suppose λ is an eigenvalue with eigenvector v. We’ll show λ must be real.
Consider the complex conjugate of the eigenvalue equation:
Physical Interpretation: Symmetric matrices represent measurements where the order doesn’t matter (distance from A to B equals distance from B to A). Complex eigenvalues would imply complex measurements, violating physical reality.
where m is the mantissa (fractional part) and e is the exponent. Think of it as scientific notation in binary.
What This Means for 64-bit Doubles:
15-17 significant decimal digits (53 bits of mantissa precision, guaranteeing at least 15 decimal digits)
Largest number: ~10^308 (exponent can go up to 1023)
Smallest positive normal: ~10^-308 (exponent down to -1022)
Machine epsilon: ~2.22×10^-16 (smallest distinguishable relative difference near 1.0)
The Shocking Truth About 0.1:
0.1 + 0.2 == 0.3 # False!
print(f"{0.1 + 0.2:.17f}") # 0.30000000000000004
# Why? 0.1 in binary is actually:
# 0.00011001100110011001100110011... (repeating forever!)
# The computer truncates this at 53 bits, creating rounding error
Just like 1/3 = 0.333... repeats forever in decimal, 1/10 repeats forever in binary. The computer must truncate, introducing tiny errors that compound.
Catastrophic Cancellation - The Silent Killer:
When you subtract nearly equal numbers, you lose precision catastrophically. Here’s why:
# Example: Computing small angles in astronomy
star_position_1 = 89.99999999 # degrees (16 digits total)
star_position_2 = 90.00000001 # degrees (16 digits total)
angle_difference = star_position_2 - star_position_1
# Result: 0.00000002 (only 1 significant digit!)
# We went from 16 digits of precision to 1!
# Better approach: Reformulate to avoid subtraction
# Instead of (A + small) - A, compute small directly
Real Project 2 Example - Gravitational Forces:
# BAD: Force between very close stars
def gravitational_force_bad(pos1, pos2, m1, m2):
dx = pos2[0] - pos1[0] # Catastrophic if positions nearly equal!
dy = pos2[1] - pos1[1]
dz = pos2[2] - pos1[2]
r_squared = dx**2 + dy**2 + dz**2 # Can become 0 or negative!
return G * m1 * m2 / r_squared # Division by ~0!
# GOOD: With softening parameter (regularization)
def gravitational_force_good(pos1, pos2, m1, m2, epsilon=1e-4):
dx = pos2[0] - pos1[0]
dy = pos2[1] - pos1[1]
dz = pos2[2] - pos1[2]
r_squared = dx**2 + dy**2 + dz**2 + epsilon**2 # Never zero!
return G * m1 * m2 / r_squared
basis: A set of linearly independent vectors that span the entire vector space
characteristic equation: det(A−λI)=0, whose roots are eigenvalues
components: The scalar coefficients representing a vector in a chosen basis
condition number: κ(A) = σ_max/σ_min, ratio of largest to smallest singular value; measures how much errors amplify
conservation law: A physical quantity that remains constant over time due to an underlying symmetry
cross product: Vector operation producing a perpendicular vector: a×b
determinant: Scalar value measuring how a linear transformation scales volumes
dot product: Scalar operation on vectors: a⋅b=∣a∣∣b∣cosθ
eigenvalue: The scaling factor for an eigenvector
eigenvector: A vector that is only scaled (not rotated) by a transformation
floating-point: Computer representation of real numbers with finite precision using scientific notation in binary
globular cluster: A spherical collection of 10⁴ to 10⁶ stars bound by gravity, orbiting as satellites of galaxies
linear algebra: The branch of mathematics concerning vector spaces and linear transformations between them
linear independence: Vectors that cannot be written as linear combinations of each other
linearity: Property where f(αx+βy)=αf(x)+βf(y) for all scalars α, β and vectors x, y
machine epsilon: Smallest distinguishable floating-point increment (~2.2×10⁻¹⁶ for float64); the gap between 1 and the next representable number
matrix: A linear transformation represented as a rectangular array of numbers
matrix inverse: The transformation that undoes another: A−1A=I
matrix multiplication: Operation combining two transformations into one: (AB)v=A(Bv)
orthogonal: Perpendicular; vectors with zero dot product
orthonormal: Basis vectors that are both orthogonal (perpendicular) and normalized (unit length)
parameter: Most fundamentally: a number that controls the behavior of a mathematical function or system. In physics: quantities like mass, temperature, or coupling constants that define system properties but are not the variables being solved for
parameter space: The space of all possible parameter values; each point represents a different configuration or model of the system
phase space: The space of all possible states of a system; for N particles in 3D, has 6N dimensions (3 position + 3 velocity per particle)
rank: The number of linearly independent rows (or columns) in a matrix; the dimension of the space the matrix actually maps to
scalar: A quantity with magnitude only, represented by a single number
symplectic: Transformation preserving phase space volume (determinant = 1)
trace: Sum of diagonal elements, equals sum of eigenvalues
trajectory: The path traced by a system’s state through phase space as it evolves in time
transformation: A function mapping vectors to vectors that preserves vector space structure
transpose: Operation swapping rows and columns: (AT)ij=Aji
vector: A mathematical object with both magnitude and direction that transforms according to specific rules
vector space: A set equipped with addition and scalar multiplication operations satisfying eight specific axioms
With this mathematical foundation firmly in place, you’re ready to tackle the computational challenges of modern astrophysics. In Project 1, you’ll immediately apply vector operations and matrix manipulations to handle stellar populations efficiently. In Project 2, you’ll see how eigenvalues determine orbital stability.
Module 0b continues with positive definite matrices, advanced topics, and the bridge to machine learning. Together, these modules provide the complete linear algebra foundation for computational astrophysics.
Remember: every algorithm you implement ultimately reduces to linear algebra. When numerical issues arise – and they will – return to these foundations. Check condition numbers, verify matrix properties, use appropriate decompositions. The mathematics you’ve learned here isn’t separate from computation – it IS computation in its purest form.
Welcome to computational astrophysics. You now speak its language.