Discrete random variables are a fundamental concept in probability theory, describing outcomes that can only take on specific, countable values. They're essential for modeling real-world scenarios like coin flips, dice rolls, or the number of customers in a store.

This topic covers the key aspects of discrete random variables, including probability mass functions, expected values, and variance. It also explores common distributions like binomial and Poisson, as well as techniques for working with multiple variables and generating random samples.

Definition of discrete random variables

• Discrete random variables are variables that can only take on a countable number of distinct values, often integers or specific values from a finite set
• The probability of each possible value is defined by a probability mass function (PMF), which assigns a probability to each possible outcome
• Discrete random variables are commonly used to model situations where outcomes are distinct and countable, such as the number of defective items in a batch or the number of customers arriving at a store within a given time period

Probability mass functions

Requirements for valid PMFs

• A PMF must assign a probability between 0 and 1 (inclusive) to each possible value of the discrete random variable
• The sum of the probabilities for all possible values must equal 1, ensuring that the total probability is accounted for
• The probabilities assigned by the PMF must be non-negative, as negative probabilities are not meaningful in the context of probability theory

Common types of PMFs

• Uniform discrete distribution assigns equal probabilities to each possible value within a specified range
• Binomial distribution models the number of successes in a fixed number of independent trials with a constant probability of success
• Poisson distribution models the number of events occurring in a fixed interval of time or space, given a known average rate of occurrence

Expected value

Definition and calculation

• The expected value (or mean) of a discrete random variable is a measure of the central tendency of the distribution
• It is calculated by summing the product of each possible value and its corresponding probability, as defined by the PMF
• The expected value represents the average value of the random variable over a large number of trials or observations

Properties of expected values

• Linearity of expectation states that the expected value of the sum of random variables is equal to the sum of their individual expected values, even if the variables are not independent
• The expected value of a constant multiplied by a random variable is equal to the constant multiplied by the expected value of the random variable
• The expected value of a linear combination of random variables is equal to the linear combination of their individual expected values

Variance and standard deviation

Definition and calculation

• Variance is a measure of the dispersion or spread of a discrete random variable around its expected value
• It is calculated by taking the expected value of the squared difference between each possible value and the expected value of the random variable
• The standard deviation is the square root of the variance and provides a measure of dispersion in the same units as the random variable

Properties of variance

• The variance of a constant is zero, as there is no dispersion around a constant value
• The variance of a constant multiplied by a random variable is equal to the square of the constant multiplied by the variance of the random variable
• The variance of the sum of independent random variables is equal to the sum of their individual variances

Moment-generating functions

Definition and properties

• The moment-generating function (MGF) of a discrete random variable is a function that generates the moments of the distribution when evaluated at different values
• The MGF is defined as the expected value of $e^{tX}$, where $t$ is a real number and $X$ is the random variable
• The MGF uniquely determines the probability distribution of the random variable, and two random variables with the same MGF have the same distribution

Applications in probability calculations

• MGFs can be used to calculate the moments of a distribution, such as the expected value (first moment) and variance (second central moment)
• MGFs can be used to find the distribution of the sum of independent random variables by multiplying their individual MGFs
• MGFs can be used to derive the probability distributions of functions of random variables, such as the square or exponential of a random variable

Common discrete probability distributions

Bernoulli and binomial distributions

• The Bernoulli distribution models a single trial with two possible outcomes (success or failure), with a probability $p$ of success
• The binomial distribution models the number of successes in a fixed number of independent Bernoulli trials, with parameters $n$ (number of trials) and $p$ (probability of success)
• The PMF of the binomial distribution is given by $P(X=k) = \binom{n}{k}p^k(1-p)^{n-k}$, where $k$ is the number of successes

Geometric and negative binomial distributions

• The geometric distribution models the number of trials needed to achieve the first success in a series of independent Bernoulli trials, with parameter $p$ (probability of success)
• The negative binomial distribution models the number of failures before the $r$-th success in a series of independent Bernoulli trials, with parameters $r$ (number of successes) and $p$ (probability of success)
• The PMF of the geometric distribution is given by $P(X=k) = (1-p)^{k-1}p$, where $k$ is the number of trials needed for the first success

Poisson distribution

• The Poisson distribution models the number of events occurring in a fixed interval of time or space, given a known average rate of occurrence $\lambda$
• The PMF of the Poisson distribution is given by $P(X=k) = \frac{e^{-\lambda}\lambda^k}{k!}$, where $k$ is the number of events
• The Poisson distribution is often used to model rare events, such as the number of defects in a large batch of products or the number of customers arriving at a store within a given time period

Hypergeometric distribution

• The hypergeometric distribution models the number of successes in a fixed number of draws from a population without replacement, with parameters $N$ (population size), $K$ (number of successes in the population), and $n$ (number of draws)
• The PMF of the hypergeometric distribution is given by $P(X=k) = \frac{\binom{K}{k}\binom{N-K}{n-k}}{\binom{N}{n}}$, where $k$ is the number of successes in the sample
• The hypergeometric distribution is used when sampling from a finite population without replacement, such as in quality control or survey sampling

Joint probability mass functions

Definition and properties

• A joint probability mass function (joint PMF) is a function that assigns probabilities to each possible combination of values for two or more discrete random variables
• The joint PMF must satisfy the same requirements as a single-variable PMF, with probabilities between 0 and 1 and a total sum of 1
• The joint PMF fully describes the probability distribution of the random variables and can be used to calculate probabilities, expected values, and other properties of the joint distribution

Marginal and conditional PMFs

• The marginal PMF of a single random variable can be obtained from the joint PMF by summing the probabilities over all possible values of the other random variables
• The conditional PMF of one random variable given the value of another can be calculated using the definition of conditional probability, dividing the joint probability by the marginal probability of the given value
• Marginal and conditional PMFs provide information about the individual behavior of random variables and the relationships between them

Independence of discrete random variables

• Two discrete random variables are independent if their joint PMF is equal to the product of their individual (marginal) PMFs for all possible values
• Independence implies that the occurrence of one event does not affect the probability of the other event
• If random variables are independent, the expected value of their product is equal to the product of their individual expected values, and the variance of their sum is equal to the sum of their individual variances

Functions of discrete random variables

Probability distributions of functions

• The probability distribution of a function of a discrete random variable can be derived by determining the possible values of the function and their corresponding probabilities
• The PMF of the function can be obtained by summing the probabilities of the original random variable for all values that result in the same function value
• Common functions of discrete random variables include sums, products, and indicators (functions that take on the value 1 if a condition is met and 0 otherwise)

Expected value and variance of functions

• The expected value of a function of a discrete random variable can be calculated by summing the product of each possible function value and its corresponding probability
• The linearity of expectation property allows for the calculation of the expected value of a sum or linear combination of functions without requiring independence
• The variance of a function can be calculated using the definition of variance, taking the expected value of the squared difference between the function values and the expected value of the function

Sums of independent random variables

Convolution formula for PMFs

• The convolution formula is used to find the PMF of the sum of two independent discrete random variables
• The PMF of the sum is given by $P(Z=k) = \sum_{i} P(X=i)P(Y=k-i)$, where $X$ and $Y$ are the independent random variables and $Z=X+Y$
• The convolution formula can be extended to find the PMF of the sum of more than two independent random variables by repeatedly applying the formula

Distribution of sums

• The distribution of the sum of independent and identically distributed (i.i.d.) Bernoulli random variables is a binomial distribution, with parameters $n$ (number of variables) and $p$ (probability of success)
• The distribution of the sum of i.i.d. geometric random variables is a negative binomial distribution, with parameters $r$ (number of successes) and $p$ (probability of success)
• The distribution of the sum of i.i.d. Poisson random variables is also a Poisson distribution, with a parameter equal to the sum of the individual parameters

Generating random samples

Inverse transform method

• The inverse transform method is a technique for generating random samples from a given probability distribution using a uniform random number generator
• The method involves finding the cumulative distribution function (CDF) of the desired distribution, inverting the CDF, and evaluating the inverted CDF at a uniform random number between 0 and 1
• The resulting value is a random sample from the desired distribution

Acceptance-rejection method

• The acceptance-rejection method is another technique for generating random samples from a given probability distribution, particularly useful when the inverse CDF is difficult to compute or not available in closed form
• The method involves selecting a proposal distribution that is easy to sample from and has a known constant $c$ such that the ratio of the desired PDF to the proposal PDF is always less than or equal to $c$
• Random samples are generated from the proposal distribution and accepted with a probability equal to the ratio of the desired PDF to the proposal PDF multiplied by $c$, with rejected samples discarded and the process repeated until the desired number of samples is obtained