Random variables are the building blocks of probability theory. They assign numerical values to random events, allowing us to analyze and predict outcomes. Understanding discrete and continuous variables is key to grasping how probability distributions work in real-world scenarios.

Expectation, variance, and standard deviation are crucial measures for random variables. These tools help us predict average outcomes, assess risk, and compare variability across different scenarios. The Law of Large Numbers and Central Limit Theorem connect individual events to overall patterns, forming the foundation of statistical inference.

Discrete vs Continuous Random Variables

Types of Random Variables

• Random variables assign numerical values to outcomes of random experiments
• Discrete random variables take on countable distinct values (counts or integers)
• Continuous random variables take on any value within a specified range (measurements on a continuous scale)
• Probability distributions describe likelihood of different outcomes for random variables
• Probability mass function (PMF) assigns probabilities to each possible value for discrete variables
• Probability density function (PDF) describes relative likelihood of specific values for continuous variables
• Cumulative distribution function (CDF) represents probability of random variable taking value less than or equal to given value
  • Applies to both discrete and continuous random variables

Examples of Random Variables

• Discrete random variable examples
  • Number of heads in 10 coin flips
  • Number of customers arriving at a store in an hour
  • Number of defective items in a batch of 100 products
• Continuous random variable examples
  • Height of a randomly selected person
  • Time required to complete a task
  • Temperature at a specific location

Probability Distribution Functions

• PMF for discrete random variables
  • Example: Rolling a fair six-sided die
    • P(X = 1) = P(X = 2) = P(X = 3) = P(X = 4) = P(X = 5) = P(X = 6) = 1/6
• PDF for continuous random variables
  • Example: Uniform distribution on interval [0, 1]
    • f(x) = 1 for 0 ≤ x ≤ 1, f(x) = 0 otherwise
• CDF for both discrete and continuous random variables
  • Example: CDF for rolling a fair six-sided die
    • F(x) = 0 for x < 1
    • F(x) = 1/6 for 1 ≤ x < 2
    • F(x) = 2/6 for 2 ≤ x < 3
    • F(x) = 3/6 for 3 ≤ x < 4
    • F(x) = 4/6 for 4 ≤ x < 5
    • F(x) = 5/6 for 5 ≤ x < 6
    • F(x) = 1 for x ≥ 6

Expectation, Variance, and Standard Deviation

Expectation Calculation

• Expectation represents long-term average of random variable
• For discrete random variables, calculated by summing product of each value and its probability
  • Formula: $E[X] = \sum_{x} x \cdot P(X = x)$
  • Example: Expected value of rolling a fair six-sided die
    • E[X] = 1(1/6) + 2(1/6) + 3(1/6) + 4(1/6) + 5(1/6) + 6(1/6) = 3.5
• For continuous random variables, determined by integrating product of variable and its PDF over entire range
  • Formula: $E[X] = \int_{-\infty}^{\infty} x \cdot f(x) dx$
  • Example: Expected value of uniform distribution on interval [0, 1]
    • E[X] = ∫[0 to 1] x · 1 dx = 1/2

Variance and Standard Deviation

• Variance measures spread or dispersion of random variable around its expected value
• Calculated as expected value of squared difference between random variable and its expectation
  • Formula: $Var(X) = E[(X - E[X])^2]$
• Standard deviation provides measure of dispersion in same units as original random variable
  • Formula: $\sigma = \sqrt{Var(X)}$
• Coefficient of variation allows comparison of variability between different random variables
  • Formula: $CV = \frac{\sigma}{E[X]}$
• Example: Variance and standard deviation of rolling a fair six-sided die
  • Var(X) = E[(X - 3.5)^2] = (1-3.5)^2(1/6) + (2-3.5)^2(1/6) + ... + (6-3.5)^2(1/6) ≈ 2.92
  • σ = √2.92 ≈ 1.71

Applications and Interpretations

• Expectation used for predicting average outcomes in repeated experiments
• Variance and standard deviation used for assessing risk and uncertainty
• Coefficient of variation allows comparison of relative variability across different scales
• Example: Investment returns
  • Stock A: E[X] = 10%, σ = 5%
  • Stock B: E[X] = 8%, σ = 3%
  • Coefficient of variation comparison
    • CV_A = 5% / 10% = 0.5
    • CV_B = 3% / 8% = 0.375
    • Stock B has lower relative variability despite lower expected return

Properties of Expectation

Basic Properties

• Expectation of constant equals constant itself
  • E[c] = c, where c is a constant
• Expectation of sum of random variables equals sum of their individual expectations
  • E[X + Y] = E[X] + E[Y]
• Linearity of expectation states E[aX + b] = aE[X] + b, where a and b are constants and X is random variable
• Expectation of product of independent random variables equals product of their individual expectations
  • E[XY] = E[X]E[Y], if X and Y are independent
• Law of total expectation allows calculation of expectation using conditional expectations
  • E[X] = E[E[X|Y]], where E[X|Y] is conditional expectation of X given Y

Advanced Properties and Applications

• Covariance measures joint variability of two random variables
  • Formula: $Cov(X, Y) = E[(X - E[X])(Y - E[Y])]$
• Correlation standardized version of covariance
  • Formula: $\rho_{X,Y} = \frac{Cov(X, Y)}{\sigma_X \sigma_Y}$
• Variance of sum of independent random variables equals sum of their individual variances
  • Var(X + Y) = Var(X) + Var(Y), if X and Y are independent
• Example: Portfolio risk assessment
  • Two stocks: X and Y
  • Var(aX + bY) = a^2Var(X) + b^2Var(Y) + 2abCov(X, Y)
  • Demonstrates impact of correlation on overall portfolio risk

Examples and Applications

• Investment portfolio optimization
  • Balancing expected returns and risk through diversification
• Quality control in manufacturing
  • Predicting defect rates and optimizing production processes
• Actuarial science
  • Calculating expected payouts and setting insurance premiums
• Example: Expected value of a linear function
  • Given: X is a random variable with E[X] = 5 and Var(X) = 4
  • Find E[2X + 3] and Var(2X + 3)
    • E[2X + 3] = 2E[X] + 3 = 2(5) + 3 = 13
    • Var(2X + 3) = 4Var(X) = 4(4) = 16

Law of Large Numbers and Central Limit Theorem

Law of Large Numbers

• States sample mean of large number of independent, identically distributed random variables converges to expected value
• Two forms: weak law and strong law, differing in type of convergence they describe
• Weak Law of Large Numbers (WLLN)
  • Convergence in probability
  • For any ε > 0, P(|X̄_n - μ| > ε) → 0 as n → ∞
• Strong Law of Large Numbers (SLLN)
  • Almost sure convergence
  • P(lim_{n→∞} X̄_n = μ) = 1
• Example: Coin flipping experiment
  • As number of flips increases, proportion of heads approaches 0.5

Central Limit Theorem

• States distribution of sum (or average) of large number of independent, identically distributed random variables approaches normal distribution
• Applies regardless of underlying distribution of random variables, given certain conditions met
• Formula: $\frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} \sim N(0,1)$ as n → ∞
• Standard error of mean quantifies variability of sample means
  • Formula: $SE = \frac{\sigma}{\sqrt{n}}$
• Enables use of normal distribution approximations for various probability calculations involving sums or averages of random variables

Applications and Examples

• Statistical inference
  • Estimation of population parameters from sample statistics
• Hypothesis testing
  • Constructing confidence intervals and performing significance tests
• Quality control
  • Monitoring manufacturing processes and detecting deviations
• Example: Sampling distribution of sample means
  • Population with μ = 50 and σ = 10
  • Sample size n = 25
  • Distribution of sample means: N(50, 10/√25) = N(50, 2)
• Example: Normal approximation to binomial distribution
  • Binomial(n, p) can be approximated by N(np, np(1-p)) for large n and p not too close to 0 or 1