Lecture notes. Written from the material of Berkeley Stat 153, licensed CC BY 4.0. These are notes, not a transcript: the material has been reorganised and rewritten. This adaptation carries the same licence, and the original is linked above.

111. The Sinusoidal Model¶

What this covers¶

This chapter defines a sinusoid as a regression function for periodic data, shows why sampling it only at the integer times $t=1,\dots,n$ lets the frequency parameter be confined to $[0,1/2]$ without loss of generality, and covers the two ways the model's parameters — in particular the frequency $f$ — get estimated once that reduction is made: a least-squares grid search, and a Bayesian posterior for $f$. It assumes comfort with ordinary linear regression, the residual sum of squares (RSS), and picks up the sinusoidal model mid-course, after it has already been introduced and fit to data in an earlier lecture.

The sinusoidal model¶

A sinusoid is the function of time

\[s(t) := \beta_0 + R\cos(2\pi f t + \phi).\]

The four constants each have a name:

$R$, the amplitude — the height of the oscillation above and below its centre line $\beta_0$.
$f$, the frequency — the number of oscillations per unit time. If $t$ is in seconds, $f$ is in Hertz.
$1/f$, the period — the time to complete one full oscillation.
$\phi$, the phase — with $\phi=0$ the wave is at its maximum at $t=0$; a nonzero $\phi$ shifts it in time, capturing that two oscillations of the same frequency need not be "in step."
$2\pi f$, the angular frequency, often written $\omega = 2\pi f$ — the rate of change of the angle inside the cosine.

For fitting purposes this parametrisation is awkward, because $R$ and $\phi$ enter $s(t)$ nonlinearly. The angle-addition identity $\cos(\alpha+\beta) = \cos\alpha\cos\beta - \sin\alpha\sin\beta$ turns the sinusoid into an equivalent form that is linear in its unknown coefficients:

\[s(t) = \beta_0 + \beta_1 \cos 2\pi f t + \beta_2 \sin 2\pi f t,\]

where $\beta_1 = R\cos\phi$ and $\beta_2 = R\sin\phi$ recover the amplitude and phase (amplitude and phase are what you'd read off a picture of the wave; $\beta_1,\beta_2$ are what a regression can estimate). The point of working in this second form is exactly that: once $f$ is treated as fixed, $s(t)$ is a linear function of $(\beta_0,\beta_1,\beta_2)$, so ordinary least squares applies directly to the coefficients — all the nonlinearity in the model is confined to $f$ alone.

Fitting the model to an observed series $y_1,\dots,y_n$ means adding noise,

\[y_t = \beta_0 + \beta_1\cos(2\pi f t) + \beta_2 \sin(2\pi f t) + \epsilon_t, \qquad \epsilon_t \overset{\text{i.i.d.}}{\sim} N(0,\sigma^2),\]

and the quantity that ends up doing the work of inferring $f$ is $RSS(f)$, the residual sum of squares left over after fitting $\beta_0,\beta_1,\beta_2$ by least squares for a fixed value of $f$:

\[RSS(f) := \min_{\beta_0,\beta_1,\beta_2} \sum_{t=1}^n \big(y_t - \beta_0 - \beta_1\cos(2\pi f t) - \beta_2 \sin(2\pi f t)\big)^2.\]

Everything below is about how $f$ is chosen using $RSS(f)$ — first by cutting down the range of $f$ that needs to be searched, then by two different estimation procedures over that range.

Discrete sampling folds every frequency into $[0,1/2]$¶

When the sinusoid is only ever observed at the integers $t=1,\dots,n$, frequencies outside $[0,1/2]$ turn out to be redundant: for any $f$ whatsoever there is another frequency $f_0 \in [0,1/2]$ whose sinusoid takes exactly the same values at every integer time, for a suitably adjusted phase. This is worth stating carefully, because it is the reason the search for $f$ later in the chapter only ever has to scan $[0,1/2]$.

Fact. For every $f \in (-\infty,\infty)$ and $\phi \in (-\infty,\infty)$ there exist $f_0 \in [0,1/2]$ and $\phi_0$ such that

\[\beta_0 + R\cos(2\pi f t + \phi) = \beta_0 + R\cos(2\pi f_0 t + \phi_0) \quad \text{for all integers } t = 1,\dots,n.\]

Proof. Three cases cover every $f$.

$f<0$. Cosine is even, so $\cos(2\pi f t + \phi) = \cos(-(2\pi f t + \phi)) = \cos(2\pi(-f)t
\phi)$, and $-f \ge 0$.
$f \ge 1$. Write $[f]$ for the integer part of $f$. Since $t$ is an integer, $2\pi[f]t$ is a multiple of $2\pi$, and cosine has period $2\pi$: $$\cos(2\pi f t + \phi) = \cos\big(2\pi[f]t + 2\pi(f-[f])t + \phi\big) = \cos\big(2\pi(f-[f])t + \phi\big),$$ and $0 \le f - [f] < 1$.
$f \in [1/2,1)$. Using $\cos(2\pi t - x) = \cos x$ for integer $t$ (again periodicity), $$\cos(2\pi f t + \phi) = \cos\big(2\pi t - 2\pi(1-f)t + \phi\big) = \cos\big(2\pi(1-f)t - \phi\big),$$ and $0 < 1-f \le 1/2$.

Case 2 (possibly followed by case 3) brings any $f$ down into $[0,1)$, and case 3 brings anything in $[1/2,1)$ the rest of the way into $[0,1/2]$; case 1 handles negative $f$ first. In every case the amplitude $R$ and the sample values at integer $t$ are unchanged — only the frequency and phase are relabelled. $\blacksquare$

This is why, from here on, $f$ is always taken to range over $[0,1/2]$ when the model is fit to data at integer times. The figure below shows the phenomenon directly: a frequency $f=1.2$, sampled only at integers, is indistinguishable from its reduced frequency $f_0 = f - [f] = 0.2$, even though the two curves disagree almost everywhere in between.

Solid: $\cos(2\pi \cdot 0.2\, t)$. Dashed: $\cos(2\pi \cdot 1.2\, t)$. The two curves disagree almost everywhere, but the marked points — the only ones a sample at integer $t$ ever sees — coincide exactly, because $1.2 - [1.2] = 0.2$.

The two boundary frequencies behave in recognisably different ways. At $f=0$ the sinusoid is simply the constant $\beta_0 + R\cos\phi$ — no oscillation at all. At $f=1/2$,

\[s(t) = \beta_0 + R\cos(\pi t + \phi) = \beta_0 + R(\cos\phi)\cos(\pi t) = \beta_0 + R(-1)^t \cos\phi,\]

which alternates between $\beta_0 + R\cos\phi$ and $\beta_0 - R\cos\phi$ at every step — the fastest oscillation a sample at integer times can register at all. So $[0,1/2]$ is exactly the range from "no oscillation" to "oscillation as fast as discrete sampling can resolve," and no frequency outside it adds anything a sample of the series could not already show.

Least squares estimation of $\beta, f, \sigma$¶

With $f$ confined to $[0,1/2]$, the same strategy used earlier in the course for the change-of-slope model applies here: since $s(t)$ is linear in $\beta$ once $f$ is fixed, sweep over candidate values of $f$, and for each one solve an ordinary linear regression.

Take a grid of candidate values of $f$ in $[0,1/2]$.
For each grid value $f$, form the design matrix $X_f$ (columns $1$, $\cos(2\pi f t)$, $\sin(2\pi f t)$, evaluated at $t=1,\dots,n$), regress $y$ on $X_f$, and record the residual sum of squares $RSS(f)$.
Let $\hat f$ be the grid value minimising $RSS(f)$.
Take $\hat\beta$ and $\hat\sigma$ to be the usual least-squares regression estimates from regressing $y$ on $X_{\hat f}$.

The only new ingredient relative to an ordinary linear regression is that $f$ itself is not linear in the model, so it cannot be estimated by the same normal equations that give $\hat\beta$ — it has to be searched over, with $RSS(f)$ as the criterion to minimise at each candidate value.

The Bayesian posterior for $f$¶

A Bayesian treatment needs priors on $\beta,\sigma,f$. As in the earlier change-of-slope model, flat priors are put on $\beta_0,\beta_1,\beta_2$ and on $\log\sigma$:

\[\beta_0,\beta_1,\beta_2,\log\sigma \overset{\text{i.i.d.}}{\sim} \text{Unif}(-C,C),\]

and, because Fact 3.1 already justifies restricting $f$ there without loss of generality,

\[f \sim \text{Unif}[0,1/2].\]

The same calculation that produced the posterior for the breakpoint $c$ in the change-of-slope model goes through unchanged here, with $c$ replaced by $f$ and the indicator $I\{1<c<n\}$ replaced by $I\{0 \le f \le 1/2\}$: writing the joint posterior of $\beta,\sigma,f$ and integrating out $\beta$ and $\sigma$ leaves a posterior for $f$ alone,

\[\text{posterior}(f) \;\propto\; I\{0 \le f \le 1/2\}\, \big|X_f^\top X_f\big|^{-1/2} \left(\frac{1}{RSS(f)}\right)^{(n-p)/2},\]

to be evaluated numerically over a grid of $f \in [0,1/2]$, exactly as $RSS(f)$ was in the least-squares approach. Here $p=3$ is the number of columns of $X_f$ (one each for $\beta_0, \beta_1,\beta_2$).

One subtlety: the factor $|X_f^\top X_f|^{-1/2}$ blows up whenever $|X_f^\top X_f| = 0$, i.e. whenever $X_f$ fails to have full column rank — and this happens exactly at the two endpoints $f=0$ and $f=1/2$, where $\sin(2\pi f t)$ vanishes identically (at $f=0$) or the column is a constant multiple of $\cos(\pi t)$, making the three columns of $X_f$ collinear. Those two edge cases are therefore excluded when the posterior is actually computed:

\[\text{posterior}(f) \;\propto\; I\{0 < f < 1/2\}\, \big|X_f^\top X_f\big|^{-1/2} \left(\frac{1}{RSS(f)}\right)^{(n-p)/2}.\]

Sources¶

Definition of the sinusoid, its four parameters, and the linear reparametrisation $\beta_1=R\cos\phi,\ \beta_2=R\sin\phi$: 02-2-the-sinusoid.md (Fall 2025, Lecture Seven, §2 "The Sinusoid").
The noise model $y_t = \beta_0+\beta_1\cos(2\pi ft)+\beta_2\sin(2\pi ft)+\epsilon_t$ and the definition of $RSS(f)$ as the minimised sum of squares over $\beta$ at fixed $f$: the "Recap from last lecture" section of LectureSeven153248Spring2025.md (Spring 2025, Lecture Seven, §1). This source file is truncated mid-sentence in the conversion and contains nothing beyond that recap; the rest of the Spring 2025 lecture is not available here.
Fact 3.1 (restricting $f$ to $[0,1/2]$ under integer-time sampling) and its three-case proof, and the behaviour of $s(t)$ at $f=0$ and $f=1/2$: 03-3-discrete-sampling-and-restricting-to.md (Fall 2025, Lecture Seven, §3).
The least-squares grid-search algorithm for $\hat f, \hat\beta, \hat\sigma$: same file, §4 ("Least Squares Estimation of $\beta,f,\sigma$"). The source calls this "exactly the same method" used for a change-of-slope model in an earlier lecture; that earlier lecture is referenced but not supplied here.
The priors on $\beta,\sigma,f$ and the marginal posterior for $f$, including the edge-case exclusion at $f=0,1/2$: 04-5-bayesian-posterior.md (Fall 2025, Lecture Seven, §5 "Bayesian Posterior"). This section explicitly derives the posterior for $f$ by analogy with the posterior for the breakpoint $c$ in the change-of-slope model, again from an earlier lecture not supplied here.
Not supplied, and referenced only in passing by the source material: §1 "The Sinusoidal Model" (the section immediately preceding this one) and §6 "Efficient Computation of $RSS(f)$" (the section immediately following it).
All four source files are machine reconstructions of PDFs with no extractable text layer; each carries the note that "every equation is unverified." Equations are reproduced here as given in those files.

← 110. Introduction to Time Series Analysis · Contents · 112. Parameter Estimation in AR(1) →

111. The Sinusoidal Model¶

What this covers¶

The sinusoidal model¶

Discrete sampling folds every frequency into \([0,1/2]\)¶

Least squares estimation of \(\beta, f, \sigma\)¶

The Bayesian posterior for \(f\)¶

Sources¶