Probability fundamentals are the building blocks of understanding uncertainty in various scenarios. These concepts help us quantify and analyze the likelihood of different outcomes, from simple coin tosses to complex business decisions.

Sample spaces, events, and probability axioms provide a structured framework for calculating probabilities. By mastering these basics, we can tackle more advanced probability problems and make informed choices in uncertain situations.

Probability Fundamentals

Definition and role of probability

• Probability numerically measures the likelihood an event will occur
  • Assigns a value between 0 and 1 to each possible outcome
    • 0 indicates impossibility (event will never happen)
    • 1 indicates certainty (event will always happen)
  • Quantifies uncertainty by assigning probabilities to different outcomes (weather forecasting, stock market predictions)
• Enables decision-making under uncertainty
  • Helps identify the most likely outcomes to inform decisions
  • Allows calculation of expected values to compare different choices (investment options, insurance policies)

Sample space and events

• Sample space ($S$) contains all possible outcomes of a random experiment
  • Each outcome is a unique element of the sample space
  • Mutually exclusive outcomes cannot occur simultaneously (rolling a 1 and 2 on a die)
  • Collectively exhaustive includes all possible outcomes (all 6 faces of a die)
• An event ($E$) is a subset of the sample space
  • Can be a single outcome (drawing an ace from a deck) or a collection of outcomes (drawing a red card)
  • The probability of an event is the sum of the probabilities of its constituent outcomes
• Examples:
  • Tossing a coin: $S = {H, T}$
  • Rolling a die: $S = {1, 2, 3, 4, 5, 6}$
  • Drawing a card: $S = {2\heartsuit, 3\heartsuit, \ldots, K\heartsuit, A\heartsuit, 2\diamondsuit, \ldots}$

Axioms of probability

• Axiom 1: Non-negativity
  • The probability of any event $E$ is non-negative: $P(E) \geq 0$
  • Probabilities cannot be negative (a -10% chance of rain is nonsensical)
• Axiom 2: Unit measure
  • The probability of the entire sample space $S$ is 1: $P(S) = 1$
  • The sum of probabilities for all outcomes in $S$ must equal 1 (100%)
• Axiom 3: Additivity
  • For any two mutually exclusive events $A$ and $B$, the probability of their union is the sum of their individual probabilities: $P(A \cup B) = P(A) + P(B)$
  • Mutually exclusive events cannot occur together, so their probabilities are added (probability of rolling a 1 or 2 on a die is the sum of rolling a 1 and rolling a 2)
• These axioms form the foundation for calculating probabilities in various scenarios
  • Ensure probability values are consistent and coherent (no negative probabilities, all probabilities sum to 1)

Classical vs empirical probability

• Classical approach (a priori probability)
  • Assumes all outcomes in the sample space are equally likely
  • Probability of an event $A$ is the number of favorable outcomes divided by the total number of possible outcomes: $P(A) = \frac{|A|}{|S|}$
    • $|A|$ denotes the number of elements in event $A$
    • $|S|$ denotes the number of elements in the sample space $S$
  • Example: In a fair coin toss, $P(H) = \frac{1}{2}$ and $P(T) = \frac{1}{2}$
• Empirical approach (a posteriori probability)
  • Based on observed data or experimental results
  • Probability of an event $A$ is the relative frequency of its occurrence in a large number of trials: $P(A) = \frac{n(A)}{n}$
    • $n(A)$ is the number of times event $A$ occurs
    • $n$ is the total number of trials
  • As the number of trials increases, the empirical probability converges to the true probability
  • Example: If a die is rolled 1000 times and "4" is observed 150 times, the empirical probability of rolling a "4" is $\frac{150}{1000} = 0.15$