Expected value is a crucial concept in probability theory, representing the average outcome of a random variable over many trials. It's calculated differently for discrete and continuous variables, using sums or integrals of values multiplied by their probabilities.

This concept has wide-ranging applications, from gambling and insurance to investment analysis. It's closely related to other statistical measures like variance and standard deviation, and forms the basis for more advanced topics like conditional expectation and multivariate analysis.

Definition of expected value

• Expected value represents the average outcome of a random variable over a large number of trials or repetitions
• Denoted by the symbol $E(X)$ where $X$ is a random variable
• Provides a measure of the central tendency or long-term average behavior of a random variable

Calculation of expected value

Discrete random variables

• For a discrete random variable $X$ with probability mass function $p(x)$, the expected value is calculated as $E(X) = \sum_{x} x \cdot p(x)$
• Involves summing the product of each possible value of $X$ and its corresponding probability
• Example: For a fair six-sided die, the expected value is $E(X) = \frac{1+2+3+4+5+6}{6} = 3.5$

Continuous random variables

• For a continuous random variable $X$ with probability density function $f(x)$, the expected value is calculated as $E(X) = \int_{-\infty}^{\infty} x \cdot f(x) , dx$
• Involves integrating the product of each possible value of $X$ and its corresponding probability density over the entire range of $X$
• Example: For a standard normal distribution, the expected value is $E(X) = 0$

Properties of expected value

Linearity of expectation

• The expected value of the sum of random variables is equal to the sum of their individual expected values: $E(X+Y) = E(X) + E(Y)$
• Holds true even if the random variables are dependent or independent
• Allows for simplifying calculations by breaking down complex random variables into simpler components

Expectation of a constant

• The expected value of a constant $c$ is the constant itself: $E(c) = c$
• Follows from the definition of expected value, as the constant has a probability of 1

Expectation of a sum

• The expected value of a sum of random variables is equal to the sum of their individual expected values: $E(\sum_{i=1}^{n} X_i) = \sum_{i=1}^{n} E(X_i)$
• Extends the linearity of expectation property to any finite number of random variables

Expected value of functions

Linear functions

• For a linear function $g(X) = aX + b$, the expected value is $E(g(X)) = aE(X) + b$
• Follows from the linearity of expectation property
• Example: If $X$ is the number of heads in 10 coin flips and $g(X) = 2X + 1$, then $E(g(X)) = 2E(X) + 1 = 11$

Quadratic functions

• For a quadratic function $h(X) = aX^2 + bX + c$, the expected value is $E(h(X)) = aE(X^2) + bE(X) + c$
• Requires calculating the expected value of $X^2$, which can be done using the variance: $E(X^2) = Var(X) + [E(X)]^2$

Other functions

• The expected value of a function $f(X)$ can be calculated as $E(f(X)) = \sum_{x} f(x) \cdot p(x)$ for discrete random variables and $E(f(X)) = \int_{-\infty}^{\infty} f(x) \cdot f_X(x) , dx$ for continuous random variables
• Requires knowing the probability distribution of $X$ and the function $f(X)$

Applications of expected value

Gambling and games of chance

• Expected value helps determine the long-term average payout or loss in gambling scenarios (roulette, blackjack, lottery)
• Allows for comparing different betting strategies and assessing the fairness of games

Insurance and risk management

• Expected value is used to calculate the average payout for insurance policies (life insurance, car insurance)
• Helps insurers determine appropriate premiums based on the expected losses and claims

Investment and portfolio analysis

• Expected value is used to estimate the average return of an investment or portfolio over time (stocks, bonds, mutual funds)
• Assists in making informed investment decisions by comparing the expected returns of different assets

Relationship to other concepts

Variance and standard deviation

• Variance measures the average squared deviation from the expected value: $Var(X) = E[(X - E(X))^2]$
• Standard deviation is the square root of the variance and provides a measure of the dispersion of a random variable around its expected value

Moment generating functions

• The moment generating function of a random variable $X$ is defined as $M_X(t) = E(e^{tX})$
• Allows for calculating moments of a random variable, including the expected value (first moment) and variance (second central moment)

Law of large numbers

• States that the sample mean of a large number of independent and identically distributed random variables converges to their expected value as the sample size increases
• Provides a theoretical justification for using the sample mean as an estimator of the population mean

Conditional expectation

Definition and properties

• The conditional expectation of a random variable $X$ given an event $A$ is defined as $E(X|A) = \sum_{x} x \cdot P(X=x|A)$ for discrete random variables and $E(X|A) = \int_{-\infty}^{\infty} x \cdot f_{X|A}(x) , dx$ for continuous random variables
• Represents the average value of $X$ when considering only the outcomes where event $A$ occurs
• Satisfies properties similar to the regular expected value, such as linearity and the expectation of a constant

Calculation methods

• Can be calculated using the definition and the conditional probability distribution of $X$ given $A$
• For discrete random variables, the conditional probability mass function $P(X=x|A)$ is used
• For continuous random variables, the conditional probability density function $f_{X|A}(x)$ is used

Applications in decision making

• Conditional expectation is used in decision theory to determine the optimal choice based on available information (medical diagnosis, investment decisions)
• Allows for updating beliefs and making decisions based on new evidence or information

Expectation of random vectors

Joint expected values

• For a random vector $(X, Y)$, the joint expected value is defined as $E(X, Y) = \sum_{x} \sum_{y} (x, y) \cdot p(x, y)$ for discrete random variables and $E(X, Y) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x, y) \cdot f(x, y) , dx , dy$ for continuous random variables
• Represents the average value of the pair $(X, Y)$ over the joint probability distribution

Covariance and correlation

• Covariance measures the linear relationship between two random variables: $Cov(X, Y) = E[(X - E(X))(Y - E(Y))]$
• Correlation is a standardized version of covariance that ranges from -1 to 1: $Corr(X, Y) = \frac{Cov(X, Y)}{\sqrt{Var(X)Var(Y)}}$
• Positive correlation indicates a direct relationship, while negative correlation indicates an inverse relationship

Multivariate normal distribution

• A multivariate normal distribution is characterized by a mean vector $\mu$ and a covariance matrix $\Sigma$
• The expected value of a multivariate normal random vector is equal to its mean vector: $E(X) = \mu$
• The covariance matrix determines the shape and orientation of the distribution in the multivariate space

Estimating expected values

Sample mean as an estimator

• The sample mean $\bar{X} = \frac{1}{n} \sum_{i=1}^{n} X_i$ is an unbiased estimator of the population mean $\mu = E(X)$
• As the sample size increases, the sample mean converges to the true expected value (law of large numbers)

Confidence intervals for expected values

• A confidence interval provides a range of plausible values for the expected value based on the sample mean and the sample size
• For a normal distribution with known variance, a 95% confidence interval for the mean is $\bar{X} \pm 1.96 \frac{\sigma}{\sqrt{n}}$
• Indicates the uncertainty associated with the point estimate of the expected value

Hypothesis testing for expected values

• Hypothesis testing allows for determining whether the observed sample mean is consistent with a hypothesized value for the expected value
• Involves setting up null and alternative hypotheses, calculating a test statistic, and comparing it to a critical value or p-value
• Example: A t-test can be used to test whether the mean weight of a product is equal to a specified value