Lecture 13 · Measurements, Probability, and Geometry

COMP 521 · DATA AND INFERENCE

Measurements, probability, and geometry

How do imperfect measurements constrain a physical quantity?

Anna Rosen · San Diego State University

THE ROAD THROUGH THE FOUNDATIONS

Today: from light to inference

Light and models: What produces a transit and what does a detector measure?
Probability: How do repeated measurements vary, and how can measurements be related?
Geometry: How do vectors and matrices organize measurements and transformations?

Use right for the main sequence and down for additional worked details.

THE SCIENTIFIC QUESTION

A transit: from geometry to a light curve

A transit occurs when a planet passes in front of its star, blocking some of its light. The curve shows brightness relative to the star’s usual level.

Uniform star; central transit; no limb darkening.

LIGHT FROM STAR TO DETECTOR

What is flux?

Flux is the energy arriving per unit time per unit collecting area, in a specified wavelength band.

F=\frac{dE}{dA\,dt}.

A detector records counts from the light collected during an exposure.

Light travels from a star to a telescope, which collects it; a detector records counts.

MEASURE BRIGHTNESS OVER TIME

From detector counts to a light curve

After calibration, compare brightness at each time with the out-of-transit brightness:

f(t)=\frac{F(t)}{F_{\rm out}}.

This normalized flux is dimensionless. Outside transit it is near 1; a transit makes a dip.

The measured counts depend on the telescope, detector, exposure time, and background as well as the incoming light.

AN IDEALIZED FORWARD MODEL

What does the depth tell us?

For the fully overlapping planet in this model, the blocked fraction of a uniformly bright stellar disk is

\text{depth}=\left(\frac{R_p}{R_\star}\right)^2=(0.1)^2=0.01.

The normalized flux falls from 1 to 0.99. Real stars have limb darkening, and real light curves include additional variation and measurement error.

SIGNAL AND VARIATION

Why are measurements uncertain?

Photon counting: the number of detected photons fluctuates between exposures.
Background and detector noise: sky light and electronic readout add variation.
Instrument and star: changing sensitivity and stellar variability can add patterns to the light curve.

Some variation is random; other patterns can be correlated in time or reflect real stellar behavior. Their size matters when deciding which transit models fit the data.

FROM A PHYSICAL EVENT TO DATA

What do we actually measure?

Synthetic normalized flux measurements over time, with an illustrative box-shaped transit model.

Each point is a flux measurement at a time.

A transit produces a dip; a forward model predicts its shape. The scatter and remaining patterns need explanation too.

QUESTION → PREDICTION → COMPARISON

Forward model and inference

A forward model predicts measurements from physical parameters.

y_i=m(t_i;\boldsymbol\theta)+\epsilon_i.

$y_i$ is observed flux; $m$ is the predicted flux; $\epsilon_i$ is their discrepancy.

Inference compares predictions with observations.

Physical parameters feed a forward model and predicted light curve; inference compares the prediction with observations to constrain parameters.

PHYSICAL PARAMETERS → LIGHT CURVE

Forward modeling: predict the observations

Choose the star, planet, orbit, and observation times. The model predicts a flux at each time.

For the uniform-star transit,

\delta=\left(\frac{R_p}{R_\star}\right)^2.

The radius ratio determines the full-transit depth.

THE UNIFORM-STAR MODEL

Project 1: geometry predicts the dip

$p=R_p/R_\star$ is the radius ratio; $z=d/R_\star$ is the projected separation in stellar radii.

As the projected separation changes, the overlap between the planet and stellar disk changes.

A planet overlaps a stellar disk. The arrow from the center of the star to the center of the planet shows projected separation.

THE UNIFORM-STAR MODEL

Transit model: three geometric cases

For a smaller planet in front of a uniform star:

f(p,z)=\begin{cases} 1 & z\geq 1+p,\\ 1-p^2 & z\leq 1-p,\\ 1-\dfrac{\text{overlap area}}{\pi R_\star^2} & \text{partial overlap}. \end{cases}

The normalized flux is the fraction of the stellar disk that remains visible.

CIRCULAR, EDGE-ON ORBIT

Project 1: position becomes a light curve

For orbital radius $a$ and period $P$ , near a primary transit with mid-transit at $t=0$ :

z(t)=\frac{a}{R_\star}\left|\sin\!\left(\frac{2\pi t}{P}\right)\right|,\qquad f_{\rm model}(t)=f\bigl(p,z(t)\bigr).

Time gives the planet’s projected position. The overlapping area gives the predicted brightness.

The projected separation alone does not tell us whether the planet is in front of or behind the star.

PROJECT 2 · LIMB DARKENING

Why does position on the star matter?

A stellar disk is brightest at its center and dimmer toward its edge, with a small planet covering part of the disk.

The center of a star’s disk can appear brighter than its edge. A planet of the same size blocks more light where the surface is brighter.

The transit shape therefore depends on both the planet’s path and the star’s brightness pattern.

LIMB DARKENING

Project 2: stars are not uniformly bright

In the linear limb-darkening model, $r$ is distance from the disk center in units of $R_\star$ :

\begin{aligned} \mu(r)&=\sqrt{1-r^2},\\ I(r)&=I_0\left[1-u(1-\mu)\right]. \end{aligned}

$I_0$ is the central intensity; $u$ sets the strength of limb darkening. For $u>0$ , the edge is dimmer.

Model light curves for a uniform star and a limb-darkened star. The limb-darkened model is deeper at the center of transit. — Same orbit and planet; two stellar brightness models.

INTEGRATE ACROSS THE STELLAR DISK

Project 2: add up the blocked light

For each annulus, $c(r,z)$ is the fraction covered by the planet. Add the blocked contribution from all annuli:

\Delta F(z)=\frac{\int_0^1 I(r)\,2\pi r\,c(r,z)\,dr}{F_{\rm tot}},\qquad F_{\rm tot}=\pi I_0(1-u/3).

The normalized model is $f(z)=1-\Delta F(z)$ . When $u=0$ , it recovers the uniform-disk model.

TIME → ORBIT → LIGHT CURVE

Project 3: an eccentric orbit

For eccentricity $e$ and period $P$ , take $t=0$ at periastron. The mean anomaly $M$ advances uniformly; solve Kepler’s equation for eccentric anomaly $E$ :

M=\frac{2\pi t}{P},\qquad M=E-e\sin E.

Then compute the orbital position and projected separation $z(t)$ , check that the planet is in front, and evaluate the transit model.

Changing the orbit changes when and how the planet crosses the star. Changing the stellar brightness model changes how much light it blocks.

LIGHT CURVE → PARAMETER ESTIMATES

Inverse modeling: learn from measurements

Which parameter values reproduce the observed light curve?

Try parameters, calculate predictions, and compare them with the data.

Different combinations can produce similar curves. Uncertainty determines how precisely the data distinguish them.

PREDICTION MEETS KEPLER DATA

From a forward model to Project 5

The projects built a path from physical parameters to predicted brightness:

\text{orbit}\;\longrightarrow\; z(t)\;\longrightarrow\; f_{\rm model}(t;\boldsymbol\theta).

With observations, compare the prediction at each observation time to the measured flux:

r_i=y_i-f_{\rm model}(t_i;\boldsymbol\theta).

The residuals tell us where the model and the light curve disagree. Project 5 brings this modeling idea to Kepler photometry.

A FIRST INVERSE PROBLEM

A simple model for depth

Before fitting a full transit curve, use a fixed-shape model to see how fitting works. Take a baseline of 1 and a fixed transit template:

m_i(\delta)=1-\delta T_i,\qquad \mathbf T=(0,1,1,0).

$T_i=0$ outside transit; $T_i=1$ inside. The unknown parameter $\delta$ is the depth.

For $\delta=0.01$ , the model predicts $(1.00,0.99,0.99,1.00)$ . Press down for the worked fit.

FOUR OBSERVATIONS

Compare candidate light curves

Four flux measurements compared with three box-shaped model curves of different transit depths.

The data are $(1.00,0.98,0.99,1.00)$ .

Each curve uses the same baseline and timing. Only the depth changes.

Which depth best matches the two in-transit measurements?

DATA MINUS MODEL

Residuals make the comparison quantitative

For each observation, subtract the model:

\begin{aligned} r_i(\delta)&=y_i-m_i(\delta)\\ &=y_i-1+\delta T_i. \end{aligned}

With equal weights, compare candidate depths:

S(\delta)=\sum_i r_i(\delta)^2.

A fit minimizes this sum.

MINIMIZE THE RESIDUALS

Derive the best-fitting depth

S(\delta)=\sum_i(y_i-1+\delta T_i)^2.

\frac{dS}{d\delta}=2\sum_iT_i(y_i-1+\delta T_i)=0.

\hat\delta=\frac{\sum_iT_i(1-y_i)}{\sum_iT_i^2}=\frac{0.02+0.01}{2}=0.015.

A DEPTH OF 1.5%

Inspect the fit and residuals

Four flux measurements and a fitted transit model, with a lower panel showing residuals of negative and positive 0.005 inside transit.

The in-transit prediction is 0.985.

Residuals: -0.005 and +0.005.

The fitted depth summarizes the dip. The residuals show what remains around the model.

FROM A FIT TO A PHYSICAL QUANTITY

What did we infer?

For a uniform star and a fully overlapping planet:

\hat\delta=0.015\quad\Rightarrow\quad \frac{R_p}{R_\star}=\sqrt{0.015}\approx0.122.

The light curve constrains a radius ratio through the forward model. To find $R_p$ itself, we also need $R_\star$ .

Here we held the transit timing, shape, and baseline fixed. Estimating them changes the inverse problem.

PART I

Probability describes variation

If we repeated a measurement under the same conditions, how would its outcomes vary?

A probability distribution describes that variation.

RANDOM VARIABLES

From a measurement to a distribution

A random variable $Y$ represents possible measurement outcomes.

For a continuous quantity:

P(a<Y<b)=\int_a^b p(y)\,dy.

Probability is area over an interval.

A probability-density curve with the region between a and b shaded to show the probability of that interval.

The density at a single value is not the probability of that exact value.

AVERAGE OVER POSSIBLE OUTCOMES

What is an expectation value?

The expectation of a random quantity is its probability-weighted average over possible outcomes.

For discrete outcomes $y_k$ with probabilities $P(Y=y_k)$ :

E[Y]=\sum_k y_k\,P(Y=y_k).

It is a property of the probability model. With many independent repetitions from the same distribution, the sample average approaches it when the mean is finite.

WEIGHT EACH OUTCOME BY ITS PROBABILITY

Expectation: a worked example

Suppose a repeat measurement of normalized brightness $Y$ is 0.98 with probability $1/4$ and 1.00 with probability $3/4$ :

E[Y]=0.98\left(\frac14\right)+1.00\left(\frac34\right)=0.995.

Across repeated measurements, the average approaches 0.995. In this example, neither individual outcome equals the expectation.

THE SAME IDEA FOR ANY QUANTITY

Expectation of a function

To average a function $g(Y)$ , weight each value of the function by the same outcome probabilities:

E[g(Y)]=\sum_k g(y_k)P(Y=y_k).

For a continuous variable with density $p(y)$ , replace the sum with an integral:

E[g(Y)]=\int_{-\infty}^{\infty}g(y)p(y)\,dy.

The expectation exists when the relevant weighted sum or integral is finite.

SUMMARIZE A DISTRIBUTION

Center and spread

\mu=E[Y],\qquad \operatorname{Var}(Y)=E[(Y-\mu)^2],\qquad \sigma=\sqrt{\operatorname{Var}(Y)}.

Mean $\mu$ : center of the distribution.

Standard deviation $\sigma$ : scale of the spread, in the same units as $Y$ .

EXPECTATION FOR A CONTINUOUS VARIABLE

From a density to a mean

\mu=E[Y]=\int_{-\infty}^{\infty}y\,p(y)\,dy.

Each possible value contributes according to its probability density. The density integrates to one; the mean is a property of the distribution.

The same expectation rule applies to other functions, including squared deviations from the mean.

SPREAD ABOUT THE MEAN

From a density to a variance

\operatorname{Var}(Y)=\int_{-\infty}^{\infty}(y-\mu)^2p(y)\,dy,\qquad \sigma=\sqrt{\operatorname{Var}(Y)}.

The variance is the density-weighted mean squared deviation. The standard deviation is in the original units.

CALCULATING MEAN AND SPREAD

Worked example: a uniform distribution

Let $p(y)=1/2$ for $0\leq y\leq2$ , and zero elsewhere. Then

\mu=\int_0^2 y\frac12\,dy=1.

\operatorname{Var}(Y)=\int_0^2(y-1)^2\frac12\,dy=\frac13,\qquad \sigma=\frac1{\sqrt3}.

DISTRIBUTION AND DATA

One dataset, one estimated mean

Given measurements $y_1,\ldots,y_n$ , their sample mean is

\bar y=\frac{1}{n}\sum_{i=1}^{n}y_i.

$\bar y$ is computed from the observed sample. $\mu$ is the mean of the underlying distribution we are trying to learn about.

For a changing star or changing transit signal, separate measurements may have different expected values. A common mean is a model assumption.

DISTRIBUTION AND DATA

Sample variance describes observed spread

For $n>1$ observed measurements with sample mean $\bar y$ :

s^2=\frac{1}{n-1}\sum_{i=1}^n(y_i-\bar y)^2.

The sample mean is estimated from the same data. The $n-1$ correction makes $s^2$ unbiased for independent, identically distributed measurements with finite variance.

WHY REPEATED MEASUREMENTS HELP

Spread of data vs. spread of an estimate

If independent measurements have a common mean and variance $\sigma^2$ :

\operatorname{Var}(\bar Y)=\frac{\sigma^2}{n},\qquad \operatorname{SD}(\bar Y)=\frac{\sigma}{\sqrt n}.

$\sigma$ : spread of individual outcomes.

$\sigma/\sqrt n$ : standard error of the mean across repeated samples.

A shared calibration bias need not average away.

DEPENDENCE

Independence is a model assumption

Events $A$ and $B$ are independent if $P(A,B)=P(A)P(B)$ .

A shared instrumental drift can make nearby errors dependent.

Two measurement deviations that rise and fall together because they share an instrumental drift.

For random variables, independence means the joint distribution factors into the individual distributions.

DEPENDENCE AND LINEAR ASSOCIATION

Covariance: how deviations combine

\operatorname{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])].

Positive covariance: the average product of deviations is positive.

Negative covariance: the average product is negative.

Four paired deviations showing an upward pattern.

Covariance carries the product of the variables’ units.

NORMALIZE BY THE SPREADS

Correlation puts covariance on a common scale

\rho_{XY}=\frac{\operatorname{Cov}(X,Y)}{\sigma_X\sigma_Y}.

Correlation has no units and ranges from -1 to 1 when both standard deviations are nonzero.

The sign and magnitude describe the direction and strength of the linear association.

DIRECTION AND STRENGTH

What do correlation values mean?

Near +1

Strong positive linear association.

Near 0

Little linear association.

Near -1

Strong negative linear association.

Correlation measures association, not causation. A nonlinear dependence can have zero correlation.

SAMPLE COVARIANCE AND CORRELATION

Calculate correlation from paired data

s_{XY}=\frac{1}{n-1}\sum_i(x_i-\bar x)(y_i-\bar y),\qquad r_{XY}=\frac{s_{XY}}{s_Xs_Y}.

Each product uses the $X$ and $Y$ deviations from the same observation.

Divide by the two sample standard deviations to remove units and compare linear associations on the same scale.

FROM PAIRED DEVIATIONS TO A NUMBER

Four pairs: work out the correlation

Four labeled paired observations, increasing from lower left to upper right, shown relative to the two means.

$\bar x=\bar y=0$ .

$\sum x_iy_i=4$ , $\sum x_i^2=5$ , $\sum y_i^2=4$ .

r=\frac{4}{\sqrt{5\cdot4}}\approx0.894.

The positive value matches the upward pattern.

KEEP THE SCALE AND THE ASSOCIATION DISTINCT

From covariance to correlation: a second example

If $\sigma_X=2$ , $\sigma_Y=3$ , and $\operatorname{Cov}(X,Y)=3$ , then

\rho_{XY}=\frac{3}{2\cdot3}=0.5.

Covariance carries the product of the variables’ units. Correlation divides out their spreads and has no units.

COVARIANCE IS NOT ALL DEPENDENCE

Three patterns

Three synthetic point patterns: increasing, decreasing, and a symmetric curved relation with zero covariance. — Centered example points. The curved relation has zero covariance.

Press down for the worked counterexample.

OPTIONAL DETAIL · A COMPLETE COUNTEREXAMPLE

Uncorrelated can still be dependent

Let $X$ be -1, 0, or 1, each with probability $1/3$ . Let $Z=X^2$ .

E[X]=0,\quad E[XZ]=E[X^3]=0,\quad \operatorname{Cov}(X,Z)=0.

Yet knowing $X=0$ tells us $Z=0$ with certainty, while $P(Z=0)=1/3$ without that information.

Independence implies zero covariance when the relevant moments exist; the converse fails.

UNCERTAINTY IN A SUM

Why do cross terms appear?

Let $Z=X+Y$ , and write $\Delta X=X-\mu_X$ and $\Delta Y=Y-\mu_Y$ .

Then $Z-E[Z]=\Delta X+\Delta Y$ . Its variance starts with the square

\operatorname{Var}(Z)=E[(\Delta X+\Delta Y)^2].

The two deviations appear together because the result depends on both quantities.

THE MIXING TERM

Expand the square

\operatorname{Var}(X+Y)=E[(\color{#087e81}{\Delta X}+\color{#486d9f}{\Delta Y})^2].

(\Delta X+\Delta Y)^2=\color{#087e81}{\Delta X^2}+\color{#486d9f}{\Delta Y^2}+\color{#9b6617}{2\Delta X\Delta Y}.

\operatorname{Var}(X+Y)=\color{#087e81}{\operatorname{Var}(X)}+\color{#486d9f}{\operatorname{Var}(Y)}+\color{#9b6617}{2\operatorname{Cov}(X,Y)}.

The cross term is the average product of the deviations.

THE CROSS TERM CHANGES SIGN

Sums and differences

\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y),

\operatorname{Var}(X-Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)-2\operatorname{Cov}(X,Y).

Positive shared variation reinforces the sum and partly cancels in the difference.

TWO CORRELATED QUANTITIES

Work the cross terms out

With $\sigma_X=2$ , $\sigma_Y=3$ , and $\rho=0.5$ :

\operatorname{Cov}(X,Y)=\rho\sigma_X\sigma_Y=0.5(2)(3)=3.

\operatorname{Var}(X+Y)=4+9+2(3)=19.

\operatorname{Var}(X-Y)=4+9-2(3)=7.

At zero covariance, both variances would be $4+9=13$ .

SUM VS. DIFFERENCE

Shared variation changes the spread

Horizontal standard-deviation bars comparing the sum and difference of two variables at correlation 0.5 and correlation zero. — Bars show one standard deviation of each combination.

APPLY THE SUM FORMULA

Derive the uncertainty of an average

\operatorname{Var}\!\left(\frac{Y_1+Y_2}{2}\right) =\frac{\operatorname{Var}(Y_1)+\operatorname{Var}(Y_2)+2\operatorname{Cov}(Y_1,Y_2)}{4}.

The factor $1/4$ comes from squaring the coefficient $1/2$ .

The covariance term tells us how the two measurements move together.

TWO EQUALLY PRECISE MEASUREMENTS

Dependence changes the mean’s uncertainty

For common variance $\sigma^2$ and correlation $\rho$ :

\operatorname{Var}\!\left(\frac{Y_1+Y_2}{2}\right)=\frac{\sigma^2}{2}(1+\rho).

$\rho=0$ : variance is $\sigma^2/2$ .

$\rho>0$ : shared variation reduces the gain in precision.

A shared bias shifts the mean; correlation describes shared variation.

A PROBABILITY MODEL

Why Gaussian distributions?

p(y)=\frac{1}{\sqrt{2\pi}\sigma}\exp\left[-\frac{(y-\mu)^2}{2\sigma^2}\right].

The center is $\mu$ ; $\sigma$ sets the spread.

Many small contributions can produce an approximately Gaussian error under suitable conditions.

BUILDING TOWARD THE CLT

One outcome is not Gaussian

Imagine an outcome that is 1 with probability $1/2$ and 0 with probability $1/2$ .

One outcome has only two possible values. Its mean is $1/2$ , and its variance is $(1/2)(1/2)=1/4$ .

Take 100 independent outcomes. Their sample mean is the fraction of 1s.

DISTRIBUTIONS OF SAMPLE MEANS

What happens as the sample grows?

Each experiment supplies one mean. Larger samples concentrate those means around 0.5.

THE DISTRIBUTION OF A SAMPLE MEAN

Repeat the entire experiment

Each experiment produces one fraction.

Across repeated experiments, those fractions have mean 0.5 and standard deviation

\sqrt{\frac{1/4}{100}}=0.05.

Exact binomial probabilities for the fraction of ones in 100 outcomes compared with a Gaussian approximation. — Exact binomial probabilities; curve is Gaussian density multiplied by bin width.

WHAT APPROACHES A GAUSSIAN?

The central limit theorem

For independent, identically distributed variables with finite mean $\mu$ and finite nonzero variance $\sigma^2$ :

\frac{\bar Y-\mu}{\sigma/\sqrt n}\xrightarrow{\ d\ }\mathcal N(0,1)\quad\text{as }n\to\infty.

The distribution of the standardized mean approaches a Gaussian. Individual outcomes do not become Gaussian.

WHY A GAUSSIAN CAN BE USEFUL

From the CLT to measurement errors

A recorded measurement can combine many small contributions to its error. When those contributions satisfy appropriate independence and balance conditions, their sum can be approximately Gaussian.

The resulting distribution describes possible errors in a measurement. A shared instrumental drift or a few dominant effects can change that distribution.

CONNECT THE MODEL TO THE MEASUREMENTS

When is the Gaussian approximation useful?

Strong dependence changes how observations combine.
Heavy tails or a few dominant effects can make the approximation poor.
A systematic bias can remain even when random errors average down.
Selection changes which measurements we see.

The accuracy of the approximation depends on the measurement process and sample size.

WHY SHOULD UNCERTAINTY AFFECT A FIT?

Two brightness measurements

Two normalized brightness measurements with one-standard-deviation error bars and vertical lines for the weighted and unweighted means.

$0.980\pm0.010$ and $1.000\pm0.020$ .

Assume independent, unbiased Gaussian errors, known $\sigma_i$ , and one constant brightness.

The ordinary mean is 0.990. Press down for the likelihood and weighted fit.

A MODEL FOR MEASUREMENT VARIATION

A likelihood for two measurements

With independent Gaussian errors and known $\sigma_i$ :

L(\mu)=\prod_i\frac{1}{\sqrt{2\pi}\sigma_i} \exp\!\left[-\frac{(y_i-\mu)^2}{2\sigma_i^2}\right].

Each measurement contributes one factor.

Relative likelihood curves for the two measurements and their joint likelihood, peaking at brightness 0.984. — Each curve is scaled to its own maximum.

TAKE THE LOGARITHM

From likelihood to a fitting objective

\log L(\mu)=\text{constant}-\frac12\sum_i\frac{(y_i-\mu)^2}{\sigma_i^2}.

The constant does not depend on $\mu$ because the uncertainties $\sigma_i$ are fixed.

Maximizing likelihood minimizes the weighted squared residuals.

FROM A NOISE MODEL TO AN OBJECTIVE

Derive the weighted mean

With independent Gaussian errors, maximizing the likelihood is equivalent to minimizing

Q(\mu)=\sum_i\frac{(y_i-\mu)^2}{\sigma_i^2}.

Differentiating and setting $dQ/d\mu=0$ gives

\hat\mu=\frac{\sum_i y_i/\sigma_i^2}{\sum_i1/\sigma_i^2}.

Each residual is divided by its uncertainty; the objective sums the squared ratios.

INVERSE-VARIANCE WEIGHTING

The weighted result

The first measurement has one-quarter the variance and four times the weight:

\hat\mu=\frac{4(0.980)+1.000}{5}=0.984.

\operatorname{SD}(\hat\mu)=\frac{1}{\sqrt{\sum_i1/\sigma_i^2}}\approx0.0089.

PART II

Data live in representations

Vectors and matrices organize the quantities we measure and the operations we perform on them.

ONE STAR, MANY TIMES

Why vectors?

A light curve contains measurements at ordered times:

\mathbf y=\begin{bmatrix}y_1\\y_2\\\vdots\\y_n\end{bmatrix},\qquad \mathbf r=\mathbf y-\mathbf m(\boldsymbol\theta).

A prediction and a residual have one entry for each observation.

The order, units, and correspondence between time and flux matter.

A DIFFERENT DATASET

Rows are objects; columns are features

Two illustrative stars, each measured in two normalized flux bands:

X=\begin{bmatrix}0.9&1.1\\1.0&0.8\end{bmatrix}.

Rows: stars. First star’s feature vector: $(0.9,1.1)$ .

Columns: band A and band B. Each is a feature.

Here there are two feature dimensions, not two spatial dimensions.

HOW A MATRIX COMBINES FEATURES

Matrix multiplication, one row at a time

\begin{bmatrix}a&b\\c&d\end{bmatrix} \begin{bmatrix}x_1\\x_2\end{bmatrix} =\begin{bmatrix}ax_1+bx_2\\cx_1+dx_2\end{bmatrix}.

Each row supplies the weights for one output.

Two inputs each contribute to two outputs; each output combines the inputs using one row of a matrix.

A RULE THAT MAPS VECTORS TO VECTORS

What is a matrix transformation?

\mathbf z=A\mathbf x.

The columns of $A$ tell us where the input basis directions go. Matrix multiplication combines those directions to produce the output.

Linear transformations preserve addition and scaling:

A(a\mathbf x+b\mathbf y)=aA\mathbf x+bA\mathbf y.

A matrix can scale, rotate, combine, or project coordinates. The physical model itself need not be linear.

VARIANCES AND COVARIANCES

Put the terms in a covariance matrix

For the same pair, $\operatorname{Var}(X)=4$ , $\operatorname{Var}(Y)=9$ , and $\operatorname{Cov}(X,Y)=3$ :

\Sigma=\begin{bmatrix}\operatorname{Var}(X)&\operatorname{Cov}(X,Y)\\ \operatorname{Cov}(X,Y)&\operatorname{Var}(Y)\end{bmatrix} =\begin{bmatrix}4&3\\3&9\end{bmatrix}.

The diagonal entries are variances; off-diagonal entries are the mixing terms.

UNCERTAINTY OF A LINEAR COMBINATION

A matrix computes every mixing term

For $Z=aX+bY$ and $\mathbf a=(a,b)^T$ :

\operatorname{Var}(Z)=a^2\operatorname{Var}(X)+b^2\operatorname{Var}(Y)+2ab\operatorname{Cov}(X,Y).

The same expression is compactly written

\operatorname{Var}(Z)=\mathbf a^T\Sigma\mathbf a.

Multiplying $\mathbf a^T\Sigma\mathbf a$ produces both variance terms and the covariance cross term.

TRANSFORMING A RANDOM VECTOR

How does a transformation change covariance?

If $\mathbf z=A\mathbf x$ and $\mathbf x$ has covariance $\Sigma$ , the covariance of $\mathbf z$ is

\Sigma_{\mathbf z}=A\Sigma A^T.

Each output is a combination of inputs. The multiplication carries their variances and covariances into the output.

THE CROSS TERMS REAPPEAR

Sum and difference as a transformation

Choose $\mathbf z=(X+Y,X-Y)^T$ . Then

A=\begin{bmatrix}1&1\\1&-1\end{bmatrix},\qquad \Sigma=\begin{bmatrix}4&3\\3&9\end{bmatrix}.

A\Sigma A^T=\begin{bmatrix}19&-5\\-5&7\end{bmatrix}.

The output variances, 19 and 7, agree with the earlier sum-and-difference calculation.

AVERAGE TWO BANDS

A transformation we can compute

\mathbf x=\begin{bmatrix}0.9\\1.1\end{bmatrix},\qquad A=\frac12\begin{bmatrix}1&1\\1&1\end{bmatrix}.

A\mathbf x=\begin{bmatrix}1.0\\1.0\end{bmatrix}.

Both output coordinates equal the average. The result lies on the equal-band line.

EIGENVECTORS AND EIGENVALUES

Directions preserved by a matrix

A transformation usually changes a vector’s direction as well as its length. An eigenvector stays on the same line: the matrix can stretch, shrink, or reverse it.

A\mathbf v=\lambda\mathbf v.

The eigenvalue $\lambda$ gives the scale factor. For a covariance matrix, eigenvectors give the principal directions and eigenvalues give the variances along them.

AN UNCERTAINTY ELLIPSE

The geometry of covariance

An upward-tilted ellipse representing a two-dimensional covariance matrix with positive covariance. — A contour of the covariance matrix, centered on the mean.

For deviations $\Delta\mathbf x$ from the mean, the ellipse shows $\Delta\mathbf x^T\Sigma^{-1}\Delta\mathbf x=1$ .

Its tilt shows the positive covariance: deviations along the longer direction change together.

The principal axes follow the eigenvectors. Their lengths are the square roots of the corresponding eigenvalues.

THE GEOMETRY OF A REPRESENTATION

Projection retains one component

A two-band vector, its orthogonal projection on the equal-band line, and the discarded component connecting projection to original.

Original = projection + difference.

\begin{aligned} (0.9,1.1)&=(1.0,1.0)\\ &\quad+(-0.1,0.1). \end{aligned}

The discarded difference might contain useful color information.

A CHOICE ABOUT INFORMATION

Why reduce dimensionality?

A spectrum may have thousands of wavelength features. A smaller representation can make patterns easier to visualize, store, and model.

But compression discards information. A low-variance feature can carry a rare and scientifically important signal.

Later: PCA finds directions of large variance in centered data; SVD provides a way to compute them. Scaling and the scientific question matter.

COMPARING VECTORS

Dot products connect geometry and fitting

\mathbf x^{\mathsf T}\mathbf y=\sum_i x_i y_i.

The dot product combines corresponding entries and describes alignment together with vector lengths.

Suppose a model predicts $a\mathbf x$ : a known direction $\mathbf x\ne\mathbf0$ , with amplitude $a$ to estimate.

A SIMPLE LEAST-SQUARES MODEL

Fit an amplitude by projection

Choose $a$ to minimize the squared residual:

S(a)=\|\mathbf y-a\mathbf x\|^2.

S'(a)=-2\mathbf x^{\mathsf T}\mathbf y+2a\mathbf x^{\mathsf T}\mathbf x=0.

\hat a=\frac{\mathbf x^{\mathsf T}\mathbf y}{\mathbf x^{\mathsf T}\mathbf x}.

This unweighted model has no separate baseline.

DATA = FIT + RESIDUAL

Work through the projection

Let $\mathbf x=(1,2)$ and $\mathbf y=(2,3)$ :

\hat a=\frac{2+6}{1+4}=1.6,\qquad \hat a\mathbf x=(1.6,3.2).

The residual is $(0.4,-0.2)$ and its dot product with $\mathbf x$ is zero. The fitted vector is the closest point on the model line.

THURSDAY

From foundations to a first fit

Inspect measurements, units, flags, gaps, and uncertainties.
Choose a forward model and calculate residuals.
Use a noise model to weight those residuals.
Examine what the fit leaves unexplained.

Kepler photometry will provide the application. Today also includes Project 4 questions and completion time.