Spectral density provides a frequency-domain perspective on stochastic processes, complementing time-domain analysis. It represents stationary processes as sums of sinusoidal components with random amplitudes and phases, enabling spectral decomposition and power distribution analysis across frequencies.

The spectral density function, S(f), is the Fourier transform of the autocorrelation function. It's non-negative, symmetric, and integrates to the total process variance. This relationship allows interchangeable analysis in time and frequency domains, offering insights into process characteristics and behavior.

Spectral representation of stationary processes

• Spectral analysis provides a frequency-domain perspective on stochastic processes, complementing the time-domain view provided by the autocorrelation function
• Stationary processes can be represented as a sum of sinusoidal components with random amplitudes and phases, allowing for a spectral decomposition
• The spectral representation enables the analysis of the frequency content and the distribution of power across different frequencies in a stochastic process

Spectral density functions

Definition of spectral density

• The spectral density function, denoted as $S(f)$, represents the distribution of power or variance across different frequencies for a stationary stochastic process
• It is defined as the Fourier transform of the autocorrelation function $R(τ)$: $S(f) = \int_{-\infty}^{\infty} R(τ) e^{-j2πfτ} dτ$
• The spectral density provides information about the relative importance of different frequency components in the process

Properties of spectral density functions

• Spectral density functions are real-valued and non-negative for all frequencies
• The area under the spectral density curve represents the total power or variance of the process: $\int_{-\infty}^{\infty} S(f) df = R(0)$
• Spectral densities are symmetric about the origin, i.e., $S(f) = S(-f)$, due to the real-valued nature of the autocorrelation function

Relationship between spectral density and autocorrelation

• The spectral density and autocorrelation function form a Fourier transform pair
• The autocorrelation can be obtained from the spectral density through the inverse Fourier transform: $R(τ) = \int_{-\infty}^{\infty} S(f) e^{j2πfτ} df$
• This relationship allows for the interchangeable analysis of a process in both time and frequency domains

Calculating spectral densities

Spectral densities of common processes

• White noise process: The spectral density of white noise is constant across all frequencies, indicating equal power at all frequencies
• Autoregressive (AR) processes: The spectral density of an AR process exhibits peaks at certain frequencies, reflecting the resonant behavior of the process
• Moving average (MA) processes: The spectral density of an MA process has zeros at specific frequencies, indicating the absence of power at those frequencies

Spectral densities from linear filters

• Linear time-invariant (LTI) systems can be characterized by their transfer function $H(f)$ in the frequency domain
• The spectral density of the output process $S_y(f)$ is related to the input spectral density $S_x(f)$ and the transfer function: $S_y(f) = |H(f)|^2 S_x(f)$
• This relationship allows for the analysis of the spectral properties of a process after passing through an LTI system

Spectral densities of modulated processes

• Modulation techniques, such as amplitude modulation (AM) and frequency modulation (FM), can be used to shift the spectral content of a process
• The spectral density of a modulated process is shifted in frequency by the amount of the modulating frequency
• Modulation enables the transmission and analysis of signals in different frequency bands

Periodogram analysis

Definition of periodogram

• The periodogram is an estimate of the spectral density based on a finite set of observations from a stochastic process
• It is calculated by taking the squared magnitude of the Discrete Fourier Transform (DFT) of the observed data: $I(f) = \frac{1}{N} |X(f)|^2$, where $X(f)$ is the DFT of the observations
• The periodogram provides an empirical measure of the power distribution across frequencies

Periodogram vs spectral density

• The periodogram is an estimate of the true spectral density, which is an asymptotic property of the process
• The periodogram is subject to statistical fluctuations and has a high variance, especially for short data records
• Averaging or smoothing techniques are often applied to the periodogram to obtain a more reliable estimate of the spectral density

Smoothing periodograms for spectral estimation

• Smoothing techniques, such as Bartlett's method or Welch's method, are used to reduce the variance of the periodogram estimate
• Bartlett's method involves dividing the data into segments, computing the periodogram for each segment, and averaging the periodograms
• Welch's method extends Bartlett's method by allowing overlap between the segments and applying a window function to each segment before computing the periodogram
• Smoothing trades off frequency resolution for reduced variance in the spectral estimate

Spectral analysis applications

Signal detection in noise

• Spectral analysis can be used to detect the presence of a signal embedded in noise by examining the spectral content
• The signal-to-noise ratio (SNR) can be improved by focusing on the frequency band where the signal is expected to be present
• Techniques such as matched filtering or energy detection can be applied in the frequency domain for signal detection

System identification using spectral methods

• Spectral analysis can be employed to identify the characteristics of an unknown system based on its input-output behavior
• By comparing the spectral densities of the input and output signals, the transfer function of the system can be estimated
• Coherence analysis can be used to assess the linear relationship between the input and output signals in the frequency domain

Optimal linear filtering in frequency domain

• Linear filtering can be performed in the frequency domain by multiplying the spectral density of the input signal with the desired frequency response of the filter
• Optimal filters, such as the Wiener filter, can be designed in the frequency domain to minimize the mean squared error between the filtered output and a desired signal
• Frequency-domain filtering allows for efficient implementation and provides insights into the filtering process

Sampling and aliasing considerations

Nyquist frequency and aliasing

• The Nyquist frequency, denoted as $f_N$, is half the sampling frequency and represents the highest frequency that can be unambiguously represented in a sampled signal
• Aliasing occurs when the sampling frequency is insufficient to capture the highest frequency components present in the original continuous-time signal
• Frequencies above the Nyquist frequency are aliased and appear as lower-frequency components in the sampled signal

Effects of sampling on spectral density

• Sampling a continuous-time signal results in a periodic replication of its spectral density in the frequency domain
• The replicated spectra are centered at multiples of the sampling frequency
• If the original signal has frequency components above the Nyquist frequency, aliasing occurs, and the replicated spectra overlap, causing distortion

Anti-aliasing filters and downsampling

• Anti-aliasing filters are low-pass filters applied to the continuous-time signal before sampling to limit its bandwidth and prevent aliasing
• The cutoff frequency of the anti-aliasing filter is typically set below the Nyquist frequency to ensure proper sampling without aliasing
• Downsampling, or decimation, involves reducing the sampling rate of a signal while preserving its spectral content within the new Nyquist frequency
• Downsampling requires prior bandlimiting of the signal to avoid aliasing during the rate reduction process

Multivariate spectral analysis

Cross-spectral density functions

• Cross-spectral density functions extend the concept of spectral density to multiple processes, capturing the frequency-domain relationships between them
• The cross-spectral density between two processes, $S_{xy}(f)$, is the Fourier transform of their cross-correlation function $R_{xy}(τ)$
• The cross-spectral density provides information about the common frequency components and the phase relationship between the processes

Coherence and partial coherence

• Coherence measures the linear dependence between two processes as a function of frequency
• It is defined as the normalized cross-spectral density: $C_{xy}(f) = \frac{|S_{xy}(f)|^2}{S_x(f) S_y(f)}$, where $S_x(f)$ and $S_y(f)$ are the individual spectral densities
• Partial coherence extends the concept of coherence to multiple processes, measuring the linear dependence between two processes while controlling for the effects of other processes

Principal component analysis in frequency domain

• Principal component analysis (PCA) can be applied in the frequency domain to identify the dominant modes of variability in multivariate processes
• Frequency-domain PCA decomposes the cross-spectral density matrix into orthogonal components, each representing a different mode of variability
• The principal components correspond to the eigenvectors of the cross-spectral density matrix, and their associated eigenvalues indicate the amount of variance explained by each component
• Frequency-domain PCA allows for the identification of common frequency patterns and the extraction of meaningful features from multivariate processes