Expectation, variance, and moments are key concepts in probability theory, helping us understand and describe random variables. These tools allow us to measure central tendency, spread, and shape of probability distributions, giving us insights into data behavior.

These concepts are crucial for analyzing random phenomena in various fields. From finance to physics, they provide a foundation for making predictions, assessing risk, and understanding complex systems. Let's dive into these fundamental ideas and their practical applications.

Expected Value of a Random Variable

Concept and Calculation

• Expected value measures central tendency for probability distributions representing long-run average outcome of random variable
• Calculation methods differ for discrete and continuous random variables:
  • Discrete: Sum each possible outcome multiplied by its probability
  • Continuous: Integrate over probability density function
• Expected value of constant equals the constant itself
• Sum of expected values of random variables equals expected value of their sum

Properties and Applications

• Linearity of expectation: $E[aX + b] = aE[X] + b$ (a and b are constants, X is random variable)
• Practical applications include decision theory, risk assessment, and financial modeling
• Expected value may not correspond to possible outcome, especially in discrete distributions
• Examples:
  • Fair coin toss (Heads = 1, Tails = 0): $E[X] = 1 * 0.5 + 0 * 0.5 = 0.5$
  • Dice roll: $E[X] = 1 * (1/6) + 2 * (1/6) + 3 * (1/6) + 4 * (1/6) + 5 * (1/6) + 6 * (1/6) = 3.5$

Variance and Standard Deviation

Variance Calculation and Properties

• Variance measures dispersion or spread in probability distribution
• Quantifies deviation of random variable values from expected value
• Defined as $Var(X) = E[(X - μ)²]$ (μ is expected value of X)
• Alternative formula: $Var(X) = E[X²] - (E[X])²$
• Calculation methods:
  • Discrete: Sum $(x - μ)²$ multiplied by probability of each outcome x
  • Continuous: Integrate using probability density function
• Variance of constant equals zero
• Variance of linear transformation: $Var(aX + b) = a²Var(X)$ (a and b are constants)

Standard Deviation and Applications

• Standard deviation equals square root of variance
• Provides measure of spread in same units as original random variable
• Used in various fields (finance, quality control, scientific research)
• Examples:
  • Fair coin toss: $Var(X) = (1 - 0.5)² * 0.5 + (0 - 0.5)² * 0.5 = 0.25$, $SD(X) = \sqrt{0.25} = 0.5$
  • Normal distribution: 68% of data falls within one standard deviation of mean

Moments and Their Properties

Moment Definitions and Types

• Moments numerically describe shape and properties of probability distributions
• kth moment of random variable X defined as $E[X^k]$ (k is non-negative integer)
• First moment (k=1) corresponds to expected value
• Second moment (k=2) relates to variance
• Central moments defined as $E[(X - μ)^k]$ (μ is expected value of X)
• Variance is second central moment
• Third and fourth central moments relate to skewness and kurtosis respectively

Applications and Advanced Concepts

• Moment-generating functions compactly represent all moments of distribution
• Method of moments technique estimates parameters in statistical inference
• Equates sample moments to theoretical moments
• Examples:
  • Skewness (third standardized moment) measures asymmetry of distribution
  • Kurtosis (fourth standardized moment) measures tailedness of distribution
• Applications in finance (asset return distributions), physics (particle distributions), and social sciences (income distributions)

Law of the Unconscious Statistician

LOTUS Theorem and Applications

• LOTUS calculates $E[g(X)]$ without knowing distribution of g(X)
• Application methods:
  • Discrete: Sum g(x) multiplied by probability of each outcome x
  • Continuous: Integrate g(x) multiplied by probability density function of X
• Simplifies calculations for transformed random variables
• Avoids deriving new probability distributions
• Useful for non-linear transformations of random variables
• Extends to multivariate functions of several random variables

Examples and Advanced Uses

• Calculate moments of transformed random variables
• Derive properties of specific probability distributions
• Examples:
  • Expected value of squared normal random variable: $E[X²] = σ² + μ²$ (σ is standard deviation, μ is mean)
  • Variance of exponential function of uniform random variable on [0,1]: $Var(e^X) = e^2 - (e-1)^2$
• Applications in option pricing (Black-Scholes model), signal processing (power spectral density estimation), and reliability engineering (failure rate calculations)