What Is Probability? Definition & Core Theory

Probability is the mathematical branch dedicated to measuring uncertainty and quantifying the likelihood that a particular event will happen. Represented numerically between 0 (an event that is physically impossible) and 1 (an event that is completely guaranteed), probability underpins modern data analytics, statistical hypothesis testing, machine learning models, financial risk assessment, and scientific research.

When you flip an unbiased coin, roll a balanced die, or forecast whether a customer will convert on an e-commerce page, probability allows you to replace guesswork with concrete quantitative analysis.

The Three Primary Types of Probability

1. Classical (Theoretical)

Assumes all outcomes in the sample space are equally likely. Calculated purely by logic without running physical trials (e.g. rolling a 4 on a fair six-sided die has exactly a 1/6 probability).

2. Empirical (Experimental)

Calculated from historical observations or real-world experimental trials: divide the observed successes by the total sample size recorded in your dataset.

3. Subjective (Bayesian)

Based on personal judgment, domain expertise, or prior belief, updated dynamically when new empirical evidence emerges using Bayes' theorem.

Fundamental Probability Formulas & Rules

1. The Basic Probability Formula

P(A) = n(A) / n(S)

Where n(A) is the count of favorable outcomes meeting your condition, and n(S) is the total number of equally likely outcomes in the entire sample space.

2. The Addition Rule (Probability of A OR B)

P(A ∪ B) = P(A) + P(B) - P(A ∩ B)

Used when you want to know if at least one event will occur. The joint overlap P(A ∩ B) is subtracted to avoid double-counting. If the events are mutually exclusive (disjoint), P(A ∩ B) = 0, so the formula simplifies to P(A) + P(B).

3. The Multiplication Rule (Probability of A AND B)

P(A ∩ B) = P(A) × P(B | A)

Used to calculate the likelihood of both events occurring. If events A and B are statistically independent, the occurrence of A has no influence on B, meaning P(B | A) = P(B), giving P(A ∩ B) = P(A) × P(B).

4. Bayes' Theorem (Conditional Probability Revision)

P(A | B) = [P(B | A) × P(A)] / P(B)

Calculates the updated probability of hypothesis A given that evidence B has occurred. Widely utilized in medical diagnosis, spam filtering, and machine learning classification algorithms.

Step-by-Step Practical Examples

Example 1: Drawing an Ace or a Red Card from a Standard 52-Card Deck

Let Event A be drawing an Ace (4 cards in deck), and Event B be drawing a Red card (26 hearts and diamonds).

P(A) = 4 / 52
P(B) = 26 / 52
The overlap P(A ∩ B) is the 2 Red Aces (Ace of Hearts, Ace of Diamonds) = 2 / 52
P(A ∪ B) = 4/52 + 26/52 - 2/52 = 28/52 = 7/13 ≈ 0.5385 (53.85%)

Example 2: The Medical Screening Paradox (Bayes' Theorem in Action)

Suppose a rare disease affects 1% of the population (P(A) = 0.01). A diagnostic test is 95% accurate (sensitivity P(B|A) = 0.95) with a 5% false positive rate (P(B|A') = 0.05). If you test positive, what is the actual probability that you have the disease?

Numerator: 0.95 × 0.01 = 0.0095
Denominator: (0.95 × 0.01) + (0.05 × 0.99) = 0.0095 + 0.0495 = 0.0590
Posterior: 0.0095 / 0.0590 ≈ 0.1610 (16.1%)

Even with a 95% accurate test, your chance of having the disease after a single positive result is only about 16.1% due to the low base rate.

Frequently Asked Questions

What is the difference between probability and odds?

Probability expresses the chance of an event happening as a proportion of all possible outcomes (e.g. 1 in 6 = 16.7%). Odds, commonly used in betting and game theory, compare the count of favorable outcomes directly against unfavorable outcomes (e.g. 1 to 5).

Can a probability ever be negative or greater than 1?

No. By the fundamental Kolmogorov probability axioms, all valid probabilities must fall strictly within the inclusive range [0, 1] (or 0% to 100%). A result outside this interval indicates a calculation error or invalid initial assumptions.

What is the difference between mutually exclusive and independent events?

Mutually exclusive events cannot occur at the same time (e.g. rolling a 1 and rolling a 6 on a single die roll; P(A ∩ B) = 0). Independent events can happen at the same time, but the outcome of one does not alter the likelihood of the other (e.g. flipping heads on Coin 1 and tails on Coin 2).

How does probability connect to P-values in hypothesis testing?

A P-value is itself a conditional probability: P(Data as extreme as observed | Null Hypothesis is True). If this probability is very small (typically < 0.05), researchers reject the null hypothesis in favor of the alternative hypothesis. You can verify significance levels using our dedicated P-value calculator.

What is the Law of Large Numbers?

The Law of Large Numbers states that as the number of experimental trials increases, the empirical (observed) relative frequency converges closer and closer to the true theoretical probability.

How do I convert odds against to a probability?

If the odds against an event are b : a, the probability of the event occurring is given by P = a / (a + b). For instance, odds against of 4 : 1 correspond to a probability of 1 / (1 + 4) = 0.20 (20%).