ABSTRACT
This directory establishes the fundamental rules and axioms of probability theory. It covers how to quantify uncertainty, update beliefs based on new evidence, and analyze the relationships between different events.
Knowledge Map
Basic Counting & Axioms
- Counting and Probability: The starting point for discrete probability. It defines the probability of an event as the ratio of successful outcomes to the total sample space for uniform distributions.
Relational Logic
- Independent Events: Explores scenarios where the occurrence of one event has no impact on the likelihood of another. This is the foundation for the product rule: .
- Conditional Probabilities: Analyzes how the probability of an event “shifts” when we are given additional information or a restricted sample space.
Inverse Probability
- Bayes’ Theorem: A mathematical formula used to determine “inverse” conditional probabilities. It is the primary tool for updating the probability of a hypothesis as more evidence becomes available.
Core Formulas
| Theorem | Context | Formula |
|---|---|---|
| Uniform Probability | Used when all outcomes in the sample space are equally likely. | |
| Product Rule | The general rule for the probability of two events occurring together. | |
| Independence | A simplified product rule valid only if and do not influence each other. | |
| Bayes’ Theorem | Used to calculate the probability of a cause () given an observed effect (). |
Key Notation
- : The number of outcomes in event (cardinality).
- : The total number of outcomes in the sample space.
- : The probability of occurring given that has already occurred.
- : The intersection (both events happening).
Summary of Event Relationships
Understanding how events overlap is critical for choosing the right law:
- Disjoint (Mutually Exclusive): The events cannot happen at the same time. .
- Independent: The events can happen together, but one does not “inform” the other.
- Dependent: Knowing that one event occurred changes your estimate for the other.
Related Toolkits
- Random Variables: Once the laws are established, we apply them to variables that map outcomes to real numbers.
- Expected Values and Analysis: Uses these laws to calculate long-run averages (Expected Value).
- Counting: Provides the permutations and combinations necessary to calculate and in complex scenarios.