Probability distributions are essential tools in econometrics, describing the likelihood of different outcomes in random processes. They come in discrete and continuous forms, each with unique properties and applications in statistical modeling and inference.

Understanding various distributions like Bernoulli, binomial, Poisson, uniform, normal, and exponential is crucial for econometric analysis. These distributions help model real-world phenomena, from binary outcomes to continuous variables, enabling researchers to make informed predictions and draw meaningful conclusions from data.

Types of probability distributions

• Probability distributions describe the likelihood of different outcomes in a random experiment or process
• Can be classified into discrete and continuous distributions based on the nature of the random variable
• Understanding probability distributions is crucial for statistical inference and modeling in econometrics

Properties of probability distributions

• All probability distributions must satisfy certain fundamental properties
• The probability of any event must be between 0 and 1 (inclusive)
• The sum of probabilities for all possible outcomes in a discrete distribution must equal 1
• The integral of the probability density function over the entire range of a continuous distribution must equal 1
• Probability distributions can be characterized by their moments, such as mean, variance, skewness, and kurtosis

Discrete probability distributions

Bernoulli distribution

• Models a single trial with two possible outcomes (success or failure)
• Characterized by a single parameter $p$, representing the probability of success
• The probability mass function is given by $P(X=x) = p^x(1-p)^{1-x}$ for $x \in {0,1}$
• Examples include coin flips (heads or tails) and binary survey responses (yes or no)

Binomial distribution

• Models the number of successes in a fixed number of independent Bernoulli trials
• Characterized by two parameters: $n$ (number of trials) and $p$ (probability of success in each trial)
• The probability mass function is given by $P(X=k) = \binom{n}{k}p^k(1-p)^{n-k}$ for $k=0,1,\dots,n$
• Examples include the number of defective items in a batch of products and the number of successful sales calls in a day

Poisson distribution

• Models the number of events occurring in a fixed interval of time or space
• Characterized by a single parameter $\lambda$, representing the average number of events per interval
• The probability mass function is given by $P(X=k) = \frac{e^{-\lambda}\lambda^k}{k!}$ for $k=0,1,2,\dots$
• Examples include the number of customer arrivals in a store per hour and the number of defects in a length of wire

Continuous probability distributions

Uniform distribution

• Models a random variable with equal probability over a fixed interval
• Characterized by two parameters: $a$ (lower bound) and $b$ (upper bound)
• The probability density function is given by $f(x) = \frac{1}{b-a}$ for $x \in [a,b]$
• Examples include the waiting time for a bus that arrives at regular intervals and the position of a randomly thrown dart on a target

Normal distribution

• Models a symmetric, bell-shaped distribution that is common in many natural phenomena
• Characterized by two parameters: $\mu$ (mean) and $\sigma$ (standard deviation)
• The probability density function is given by $f(x) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$ for $x \in \mathbb{R}$
• Examples include the distribution of heights in a population and the errors in a linear regression model

Exponential distribution

• Models the time between events in a Poisson process
• Characterized by a single parameter $\lambda$, representing the average rate of events
• The probability density function is given by $f(x) = \lambda e^{-\lambda x}$ for $x \geq 0$
• Examples include the time between customer arrivals in a store and the duration of phone calls in a call center

Joint probability distributions

Joint probability density function

• Describes the probability of two or more random variables taking on specific values simultaneously
• For continuous random variables $X$ and $Y$, the joint probability density function is denoted by $f(x,y)$
• The probability of $(X,Y)$ falling in a region $A$ is given by $P((X,Y) \in A) = \iint_A f(x,y) dxdy$

Joint cumulative distribution function

• Gives the probability that both $X$ and $Y$ are less than or equal to specific values
• For continuous random variables $X$ and $Y$, the joint cumulative distribution function is denoted by $F(x,y) = P(X \leq x, Y \leq y)$
• Can be obtained by integrating the joint probability density function: $F(x,y) = \int_{-\infty}^x \int_{-\infty}^y f(u,v) dvdu$

Marginal distributions

• Describe the probability distribution of a single random variable in a joint distribution
• Can be obtained by integrating the joint probability density function over the other variable(s)
• For continuous random variables $X$ and $Y$, the marginal probability density functions are given by $f_X(x) = \int_{-\infty}^{\infty} f(x,y) dy$ and $f_Y(y) = \int_{-\infty}^{\infty} f(x,y) dx$

Conditional probability distributions

Conditional probability density function

• Describes the probability distribution of one random variable given the value of another
• For continuous random variables $X$ and $Y$, the conditional probability density function of $Y$ given $X=x$ is denoted by $f_{Y|X}(y|x) = \frac{f(x,y)}{f_X(x)}$
• Represents the probability density of $Y$ at a specific value $y$ when $X$ is known to be $x$

Conditional expectation

• The expected value of one random variable given the value of another
• For continuous random variables $X$ and $Y$, the conditional expectation of $Y$ given $X=x$ is denoted by $E[Y|X=x] = \int_{-\infty}^{\infty} yf_{Y|X}(y|x) dy$
• Provides a measure of the average value of $Y$ for a given value of $X$
• Can be used to make predictions in regression analysis and other econometric models

Moments of probability distributions

Expected value

• The average value of a random variable over its entire range
• For a discrete random variable $X$ with probability mass function $p(x)$, the expected value is given by $E[X] = \sum_x xp(x)$
• For a continuous random variable $X$ with probability density function $f(x)$, the expected value is given by $E[X] = \int_{-\infty}^{\infty} xf(x) dx$

Variance and standard deviation

• Measure the dispersion of a random variable around its expected value
• The variance of a random variable $X$ is given by $Var(X) = E[(X-E[X])^2]$
• The standard deviation is the square root of the variance: $\sigma_X = \sqrt{Var(X)}$
• Provide information about the spread and variability of a probability distribution

Skewness and kurtosis

• Skewness measures the asymmetry of a probability distribution
• A positive skewness indicates a longer right tail, while a negative skewness indicates a longer left tail
• Kurtosis measures the heaviness of the tails of a probability distribution relative to a normal distribution
• A higher kurtosis indicates heavier tails and more extreme values
• Both skewness and kurtosis can be used to characterize the shape of a probability distribution

Transformations of random variables

Functions of random variables

• A new random variable can be created by applying a function to an existing random variable
• For a continuous random variable $X$ with probability density function $f_X(x)$ and a function $g$, the probability density function of $Y=g(X)$ is given by $f_Y(y) = f_X(g^{-1}(y)) \left| \frac{d}{dy}g^{-1}(y) \right|$
• Examples include taking the logarithm of a log-normally distributed random variable or squaring a normally distributed random variable

Moment-generating functions

• A function that generates the moments of a probability distribution
• For a random variable $X$, the moment-generating function is defined as $M_X(t) = E[e^{tX}]$
• The $n$-th moment of $X$ can be obtained by evaluating the $n$-th derivative of $M_X(t)$ at $t=0$
• Moment-generating functions can be used to derive properties of probability distributions and establish relationships between different distributions

Characteristic functions

• Another function that uniquely characterizes a probability distribution
• For a random variable $X$, the characteristic function is defined as $\phi_X(t) = E[e^{itX}]$, where $i$ is the imaginary unit
• Characteristic functions have properties similar to moment-generating functions and can be used to prove results in probability theory
• They are particularly useful when moment-generating functions do not exist, such as for the Cauchy distribution

Sampling distributions

Central Limit Theorem

• States that the sum (or average) of a large number of independent and identically distributed random variables will be approximately normally distributed, regardless of the underlying distribution
• More formally, if $X_1, X_2, \dots, X_n$ are i.i.d. random variables with mean $\mu$ and variance $\sigma^2$, then $\frac{\sum_{i=1}^n X_i - n\mu}{\sigma\sqrt{n}}$ converges in distribution to a standard normal random variable as $n \to \infty$
• The Central Limit Theorem is a fundamental result in probability theory and is the basis for many statistical inference procedures

Sample mean distribution

• The distribution of the sample mean $\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i$ for a random sample of size $n$ from a population with mean $\mu$ and variance $\sigma^2$
• By the Central Limit Theorem, the sample mean is approximately normally distributed with mean $\mu$ and variance $\frac{\sigma^2}{n}$ for large $n$
• The distribution of the sample mean is used in hypothesis testing and confidence interval construction for population means

Sample variance distribution

• The distribution of the sample variance $S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \bar{X})^2$ for a random sample of size $n$ from a population with variance $\sigma^2$
• The sample variance follows a chi-square distribution with $n-1$ degrees of freedom when the population is normally distributed
• The distribution of the sample variance is used in hypothesis testing and confidence interval construction for population variances

Applications in econometrics

Hypothesis testing with probability distributions

• Probability distributions are used to construct test statistics and critical regions for hypothesis testing
• Examples include using the standard normal distribution (z-test) for testing population means with known variance, the Student's t-distribution (t-test) for testing population means with unknown variance, and the chi-square distribution for testing population variances
• Hypothesis testing allows researchers to make statistical inferences about population parameters based on sample data

Confidence intervals using probability distributions

• Probability distributions are used to construct confidence intervals for population parameters
• For example, a confidence interval for a population mean can be constructed using the standard normal distribution (z-interval) when the population variance is known or the Student's t-distribution (t-interval) when the population variance is unknown
• Confidence intervals provide a range of plausible values for the population parameter with a specified level of confidence

Regression analysis assumptions

• Many econometric models, such as linear regression, rely on assumptions about the probability distributions of the variables involved
• Common assumptions include normality of the error terms, homoscedasticity (constant variance of the error terms), and independence of the error terms
• Violations of these assumptions can lead to biased or inefficient estimates and invalid inference
• Probability distributions are used to test for violations of these assumptions (e.g., using the Jarque-Bera test for normality or the Breusch-Pagan test for heteroscedasticity) and to develop robust estimation methods when assumptions are violated (e.g., using heteroscedasticity-consistent standard errors)