Probability theory is the mathematical framework for quantifying uncertainty. It underpins much of modern science, statistics, economics, data science, artificial intelligence, and even philosophy of decision-making. Whether you’re tossing a coin, analyzing customer churn, or detecting disease using a diagnostic test, probability helps us make sense of randomness and draw conclusions from incomplete information.

This post provides a structured walkthrough of 10 fundamental concepts in probability. From simple random experiments and mutually exclusive events to advanced principles like Bayes’ Theorem and the Law of Large Numbers, each topic is presented with definitions, mathematical formulations, and real-world examples. Together, these principles form the bedrock of probabilistic reasoning used in statistical modeling and inference.

1. Random Experiment:

The word ‘experiment’ describes an act that can be repeated under same conditions. Their outcomes are completely dependent on chance.

Key Properties:

• Randomness: The exact outcome cannot be predicted in advance.
• Reproducibility: The experiment can be repeated multiple times.
• Well-defined outcomes: Every possible result is known (e.g., heads or tails in a coin toss).

Examples of Random Experiments

1. Coin Toss → Outcomes: {Heads, Tails}
2. Rolling a Die → Outcomes: {1, 2, 3, 4, 5, 6}
3. Drawing a Card from a Deck → Outcomes: {52 unique cards}
4. Measuring Rainfall in a Day → Outcomes: Any non-negative real number (continuous).

2. Equally likely events:

The outcomes of a random experiment are said to be equally likely if after taking into consideration all relevant evidence, none of them can be expected in preference to another. ‘Equally likely’ is synonymous to ‘equally probable’ which means we are trying to define probability in terms of equal probabilities. This definition can be little confusing because we are defining a probability even before measuring it. However, this is done more from a theoretical perspective.

3. Mutually Exclusive:

Events are said to be ‘mutually exclusive’ when two or more of them cannot occur simultaneously. This means that mutually exclusive events can occur only one at a time, & the occurrence of any event signifies impossibility of the remaining events in any particular performance of the random experiment. Example – while tossing a coin, the outcomes ‘head’ or ‘tail’ are mutually exclusive events. The occurrence of any event implies that the other event definitely hasn’t taken place.

Mathematically:

Events A and B are mutually exclusive if A∩B=∅(they have no common outcomes)

For multiple events E1,E2,…,En they are mutually exclusive if Ei∩Ej=∅for all i≠j

Examples of Mutually Exclusive Events

1. Coin Toss:
   • {Heads}, {Tails} → Mutually exclusive (both cannot happen at once).
2. Die Roll:
   • {1}, {2}, {3}, {4}, {5}, {6} → All are mutually exclusive.
   • {Even number}, {Odd number} → Mutually exclusive and exhaustive.
3. Card Draw:
   • {Drawing a King}, {Drawing a Queen} → Mutually exclusive (a single card can’t be both).

4. Exhaustive :

The complete set of all possible events of any random experiment creates an exhaustive set of events. The same example of tossing a coin is also relevant here. The two possible outcomes are exhaustive, as one of these two must necessarily occur.

Mathematically:

Events E1,E2,…,En are exhaustive if E1 U E2 U⋯U En=S(where S is the sample space)E1​ U E2​ U⋯U En = S (where S is the sample space)

This means no outcome lies outside all of these events.

Examples of Exhaustive Events

1. Coin Toss:
   • Events: {Heads}, {Tails} → Exhaustive (no other possible outcome).
2. Die Roll:
   • Events: {1, 2, 3}, {4, 5, 6} → Exhaustive (cover all outcomes).
   • Events: {1}, {2}, {3}, {4}, {5}, {6} → Exhaustive & Mutually Exclusive.

5. Classical Definition of Probability :

If a random experiment has ‘n’ possible outcomes which are mutually exclusive, exhaustive and equally likely and ‘m’ of these are favorable to an event A, then the probability of the event is defined as the ratio m/n.

Probability always lies between 0 and 1. ‘Zero’ probability is attained when none of the outcomes is favorable to the event or the event is rather impossible to occur. On the other hand, the maximum value of p is attained when all the possible outcomes are favorable or the event is said to be certain.

6. Conditional Probability:

Conditional probability is the probability of an event occurring given that another event has already occurred. It helps us update probabilities based on new information.

Mathematical Notation:

The conditional probability of event A given event B is written as: P(A|B)=P(A∩B)P(B) if P(B)>0)

• P(A∩B) = Probability of both A and B occurring (joint probability).
• P(B) = Probability of event B

7. Theorem of Total Probability :

If two events A & B are mutually exclusive, then the probability of occurrence of either A or B is given by the sum of their probabilities, i.e.,

P (A + B) = P(A) + P(B)

This is also known as Addition theorem. theorem of Total Probability can be extended to any number of mutually exclusive events. If there are 3 mutually exclusive events A, B, C, then by total probability theorem, P (A+B+C) = P(A) + P(B) + P(C)

8. Theorem of Compound Probability :

The probability of occurrence of the event A as well as B is given by the product of unconditioned probability of A and conditional probability of B, assuming that A has actually occurred i.e., probability of A and B = Probability of A * Conditional probability of B, assuming A.

In symbols, P(AB) =P(A) * P(B/A)

This is also known as Multiplication Theorem.

9. Bayes’ Theorem:

Bayes’ Theorem is a fundamental rule in probability that updates beliefs based on new evidence. It connects conditional probabilities and is widely used in statistics, machine learning, medicine, and more.

Bayes’ Theorem calculates the probability of an event A occurring given that B has occurred

P(A|B)=P(B|A)⋅P(A) / P(B)

Where:

• P(A|B) = Posterior probability (updated belief after observing B).
• P(B|A) = Likelihood (probability of observing B if A is true).
• P(A)= Prior probability (initial belief about A).
• P(B) = Marginal probability (total probability of B, considering all scenarios).

An example can be, “Given that I observed B, how likely is A now?”

• Prior (P(A) : Initial guess (e.g., “1% of people have the disease”).
• Likelihood (P(B|A) : How likely is the evidence if A is true? (e.g., “Test is 99% accurate if you have the disease”).
• Posterior (P(A|B): Updated probability after seeing the evidence.

10. Law of Large Numbers (LLN)

• As the number of trials increases, the sample mean converges to the expected value.
• Example:
  • Flipping a coin many times → Proportion of heads approaches 0.5.

A continuation of the law of large numbers would be the Central Limit Theorem (CLT)

• The sampling distribution of the mean becomes approximately normal as n→∞ regardless of the population distribution.

Summary Table

Concept	Key Idea	Formula
Sample Space	All possible outcomes	S={outcomes}S={outcomes}
Probability	Likelihood of an event	P(A)=Favorable/Total​
Mutually Exclusive	Events cannot occur together	P(A∪B)=P(A)+P(B)
Independent Events	One event doesn’t affect the other	P(A∩B)=P(A)P(B)
Conditional Probability	Probability given another event	P(A|B)=P(A∩B)P(B)
Bayes’ Theorem	Updates probability with evidence	P(A|B)=P(B|A)P(A)P(B)
Random Variables	Numerical outcomes of randomness	Discrete: X∈{x1,x2,… }X∈{x1​,x2​,…}
Expected Value	Long-run average	E[X]=∑xiP(xi)
Variance	Measure of spread	Var(X)=E[X2]−(E[X])^2
Law of Large Numbers	Sample mean → Expected value	Xˉn→μ as n→∞

Conclusion

Probability theory provides a powerful language for describing uncertainty and making rational decisions under incomplete information. From defining simple random experiments to applying Bayes’ Theorem for belief updating, the core concepts outlined above are essential tools for any student, analyst, or researcher working in data-driven fields. By mastering these foundational ideas, you build the groundwork for understanding more advanced topics such as statistical inference, stochastic processes, and probabilistic machine learning.

Whether you’re designing a fair experiment, modeling a medical diagnosis, or evaluating financial risk, the axioms and theorems of probability ensure your reasoning is mathematically sound. As you go deeper into the subject, you’ll find that probability is not just a branch of mathematics—it’s a fundamental lens through which we understand the world.