Probability axioms and properties are the building blocks of probability theory. They define the rules for calculating and interpreting probabilities, helping us understand how likely events are to occur and how different events relate to each other.

These fundamental concepts are crucial for grasping more advanced topics like conditional probability and Bayes' theorem. By mastering these basics, you'll be better equipped to tackle complex probability problems and apply statistical reasoning in data science.

Probability Fundamentals

Core Concepts of Probability Theory

• Sample space represents all possible outcomes of an experiment or random process
  • Denoted by Ω (omega)
  • Contains every possible result (coin toss: {heads, tails})
• Event constitutes a subset of the sample space
  • Can be a single outcome or a collection of outcomes
  • Represented by capital letters (A, B, C)
• Probability function assigns a numerical value to each event
  • Maps events to real numbers between 0 and 1
  • P(A) denotes the probability of event A occurring
• Kolmogorov's axioms form the foundation of probability theory
  • Axiom 1: Probability of any event is non-negative (P(A) ≥ 0)
  • Axiom 2: Probability of the entire sample space equals 1 (P(Ω) = 1)
  • Axiom 3: For mutually exclusive events, P(A ∪ B) = P(A) + P(B)

Probability Calculations and Interpretations

• Probability values range from 0 to 1
  • 0 indicates impossibility
  • 1 represents certainty
• Frequentist interpretation views probability as long-run relative frequency
  • Based on repeated trials or observations
• Bayesian interpretation treats probability as a degree of belief
  • Updates prior beliefs with new evidence
• Probability can be expressed as fractions, decimals, or percentages
  • $P(A) = \frac{\text{favorable outcomes}}{\text{total outcomes}}$ (assuming equally likely outcomes)
• Law of large numbers states that as sample size increases, observed probability approaches theoretical probability
  • Explains why casino games remain profitable over time

Event Relationships

Types of Event Relationships

• Mutually exclusive events cannot occur simultaneously
  • P(A ∩ B) = 0 for mutually exclusive events A and B
  • Drawing a red card and a black card from a single draw (cannot happen together)
• Exhaustive events cover all possible outcomes in the sample space
  • Union of exhaustive events equals the entire sample space
  • Rolling an even number or an odd number on a die (covers all possibilities)
• Complement of an event includes all outcomes not in the original event
  • Denoted as A' or A^c
  • P(A) + P(A') = 1 (complementary events sum to 1)
• Independent events do not affect each other's probabilities
  • Occurrence of one event does not change the probability of the other
  • P(A ∩ B) = P(A) P(B) for independent events

Practical Applications of Event Relationships

• Venn diagrams visually represent relationships between events
  • Overlapping circles show intersections and unions
  • Useful for understanding complex event interactions
• Set theory operations apply to events
  • Union (A ∪ B): outcomes in either A or B or both
  • Intersection (A ∩ B): outcomes common to both A and B
• De Morgan's laws relate complements of unions and intersections
  • (A ∪ B)' = A' ∩ B'
  • (A ∩ B)' = A' ∪ B'
• Pairwise independence does not guarantee mutual independence
  • Three events can be pairwise independent but not mutually independent

Probability Rules

Fundamental Probability Rules and Their Applications

• Addition rule calculates the probability of either event A or B occurring
  • For mutually exclusive events: P(A ∪ B) = P(A) + P(B)
  • For non-mutually exclusive events: P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
• Multiplication rule determines the probability of both events A and B occurring
  • For independent events: P(A ∩ B) = P(A) P(B)
  • For dependent events: P(A ∩ B) = P(A) P(B|A)
• Law of total probability breaks down complex events into simpler components
  • $P(A) = \sum_{i=1}^n P(A|B_i) P(B_i)$
  • Useful when direct calculation of P(A) is difficult

Advanced Probability Concepts and Techniques

• Conditional probability measures the likelihood of an event given another has occurred
  • $P(A|B) = \frac{P(A \cap B)}{P(B)}$
  • Adjusts probabilities based on new information
• Bayes' theorem relates conditional and marginal probabilities
  • $P(A|B) = \frac{P(B|A) P(A)}{P(B)}$
  • Allows updating probabilities with new evidence
• Inclusion-exclusion principle extends the addition rule to multiple events
  • $P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(A \cap C) - P(B \cap C) + P(A \cap B \cap C)$
• Chain rule of probability applies multiplication rule sequentially
  • $P(A \cap B \cap C) = P(A) * P(B|A) * P(C|A \cap B)$
  • Useful for calculating joint probabilities of multiple events