Unit: Probability, Random Variables & Probability Distributions
Chapter: Random Event Probability Calculating
Reference: – Sample spaces & Events, Probability definitions & Rules, Complementary & Mutually exclusive events, Conditional probability, Independence & Dependence, Probability distributions, Law of large number & Simulations, counting principles, Expected value & Variance, Central Limit theorem.
After studying this chapter, you should be able to:
- Sample spaces & Events & Conditional Probability.
- Probability definition & Rules.
- Mutually exclusive events & Probability Distributions.
- Law of Large Number & Simulations.
- Central Limit theorem
Sample Spaces & Events & Conditional Probability
Sample Spaces and Events:
Sample Space (S): The sample space is the set of all possible outcomes of a random experiment or situation. It is denoted by "S" and contains individual outcomes that are mutually exclusive.
Event (E): An event is a subset of the sample space. It consists of one or more outcomes that are of interest in a probability experiment.
Simple Event: A simple event is an individual outcome in the sample space, often represented by a single point.
Compound Event: A compound event is an event that consists of two or more simple events.
Empty Set (∅): The empty set represents an event that cannot occur. Its probability is zero.
Conditional Probability:
Conditional Probability (P(A|B)): The probability of event A occurring given that event B has already occurred. It is calculated as the ratio of the probability of the intersection of events A and B to the probability of event B.
Multiplication Rule for Independent Events: For independent events A and B, the probability of both events occurring is the product of their individual probabilities: P(A and B) = P(A) * P(B).
Multiplication Rule for Dependent Events: For dependent events A and B, the probability of both events occurring is the product of the conditional probability of A given B times the probability of event B: P(A and B) = P(A|B) * P(B).
Addition Rule for Mutually Exclusive Events: For mutually exclusive events A and B, the probability of either event occurring is the sum of their individual probabilities: P(A or B) = P(A) + P(B).
Addition Rule for Non-Mutually Exclusive Events: For non-mutually exclusive events A and B, the probability of either event occurring is the sum of their individual probabilities minus the probability of their intersection: P(A or B) = P(A) + P(B) – P(A and B).
Complementary Events: The probability of the complement of event A (not A) is 1 minus the probability of event A: P(not A) = 1 – P(A).
Independent Events: Two events A and B are independent if the occurrence of one does not affect the probability of the other occurring.
Dependent Events: Two events A and B are dependent if the occurrence of one affects the probability of the other occurring.
Bayes' Theorem: Bayes' theorem allows us to update the probability of an event based on new evidence or information.
Law of Total Probability: This law states that the probability of an event A is the sum of the probabilities of A occurring given each possible value of another event B, weighted by the probability of each value of B.
Probability Definition & Rules
Probability Definitions:
Probability: Probability is a numerical measure of the likelihood that an event will occur. It ranges from 0 (impossible) to 1 (certain).
Classical Probability: Classical probability is based on equally likely outcomes in a sample space. The probability of an event A is given by P(A) = (number of favorable outcomes for A) / (total number of outcomes).
Empirical Probability: Empirical probability is based on observed data. The probability of an event A is estimated as the relative frequency of A occurring in a large number of trials.
Subjective Probability: Subjective probability is based on personal judgment or belief about the likelihood of an event occurring. It is often used when objective data is lacking.
Probability Rules:
Addition Rule: The addition rule states that the probability of the union of two events A and B (A or B) is the sum of their individual probabilities minus the probability of their intersection: P(A or B) = P(A) + P(B) – P(A and B).
Multiplication Rule for Independent Events: For independent events A and B, the probability of both events occurring is the product of their individual probabilities: P(A and B) = P(A) * P(B).
Multiplication Rule for Dependent Events: For dependent events A and B, the probability of both events occurring is the product of the conditional probability of A given B times the probability of event B: P(A and B) = P(A|B) * P(B).
Complementary Rule: The probability of the complement of event A (not A) is 1 minus the probability of event A: P(not A) = 1 – P(A).
Conditional Probability: Conditional probability is the probability of event A occurring given that event B has already occurred: P(A|B) = P(A and B) / P(B).
Total Probability Theorem: The total probability theorem states that if events B1, B2, …, Bn form a partition of the sample space, then the probability of event A is the sum of the probabilities of A occurring given each Bi, weighted by the probability of each Bi: P(A) = Σ [P(A|Bi) * P(Bi)] for i = 1 to n.
Bayes' Theorem: Bayes' theorem relates the conditional probability of event A given event B to the conditional probability of event B given event A: P(A|B) = [P(B|A) * P(A)] / P(B).
Multiplication Rule for Counting: The multiplication rule for counting is used to calculate the total number of outcomes in a sequence of events. If event A can occur in m ways and event B can occur in n ways after A has occurred, then the total number of outcomes is m * n.
Addition Rule for Mutually Exclusive Events: For mutually exclusive events A and B, the probability of either event occurring is the sum of their individual probabilities: P(A or B) = P(A) + P(B).
Law of Large Numbers: The law of large numbers states that as the number of trials in a probability experiment increases, the relative frequency of an event approaches its true probability.
Probability Distributions: Probability distributions describe how probabilities are distributed among different possible values of a random variable. They provide a comprehensive view of all possible outcomes and their associated probabilities.
Mutually Exclusive Events & Probability Distributions
Mutually Exclusive Events:
Mutually Exclusive Events: Mutually exclusive events are events that cannot occur simultaneously. If one event happens, the other event cannot happen at the same time.
Intersection of Mutually Exclusive Events: The intersection (common outcome) of mutually exclusive events is an empty set (∅), meaning that they share no common outcomes.
Addition Rule for Mutually Exclusive Events: The probability of either of two mutually exclusive events A or B occurring is the sum of their individual probabilities: P(A or B) = P(A) + P(B).
Venn Diagrams: Venn diagrams are often used to illustrate mutually exclusive events, with separate circles representing each event, and the intersection being empty.
Collectively Exhaustive Events: A set of events is collectively exhaustive if their union covers the entire sample space. In other words, every possible outcome is covered by at least one of the events.
Partition of the Sample Space: A partition of the sample space consists of mutually exclusive and collectively exhaustive events that together cover all possible outcomes.
Non-Mutually Exclusive Events: Non-mutually exclusive events are events that can occur simultaneously, and their intersection is not empty.
Exclusive vs. Exhaustive: Mutually exclusive events deal with the overlap of events, while collectively exhaustive events ensure that all possibilities are accounted for.
Probability Distributions:
Probability Distribution: A probability distribution provides the probabilities of all possible outcomes of a random variable. It describes how the probabilities are distributed among different values.
Discrete Probability Distribution: A discrete probability distribution is applicable to a discrete random variable, where each possible value has a corresponding probability.
Continuous Probability Distribution: A continuous probability distribution applies to continuous random variables, such as measurements, and is described using probability density functions.
Probability Mass Function (PMF): The probability mass function gives the probability of each individual value of a discrete random variable. It satisfies the properties of being non-negative and summing to 1.
Probability Density Function (PDF): The probability density function gives the probability density at each point of a continuous random variable. The area under the curve over an interval represents the probability.
Expected Value (Mean): The expected value of a random variable is the weighted average of its possible values, where the weights are the corresponding probabilities.
Variance and Standard Deviation: Variance measures the spread of a probability distribution, while the standard deviation is its square root. They provide insights into the dispersion of values around the mean.
Binomial Distribution: The binomial distribution describes the number of successes in a fixed number of independent Bernoulli trials, where each trial has the same probability of success.
Poisson Distribution: The Poisson distribution models the number of events that occur in a fixed interval of time or space, given a known average rate of occurrence.
Normal Distribution: The normal distribution, also known as the Gaussian distribution, is a symmetric and bell-shaped continuous distribution that is widely used in statistics due to its prevalence in natural phenomena.
Sampling Distributions: Sampling distributions describe the distribution of sample statistics (e.g., sample mean) based on repeated random sampling from a population.
Central Limit Theorem: The Central Limit Theorem states that the distribution of the sample mean of a large enough sample from any population approaches a normal distribution, regardless of the shape of the original population distribution.
Law of large Number & Simulations
Law of Large Numbers:
Definition: The Law of Large Numbers (LLN) is a fundamental principle in probability and statistics that states that as the number of trials or observations in a random experiment increase, the sample mean (average) of the observed values approaches the true population mean.
Strong LLN and Weak LLN: There are two main versions of the Law of Large Numbers: The Strong Law of Large Numbers (SLLN) and the Weak Law of Large Numbers (WLLN). The SLLN guarantees almost sure convergence, while the WLLN guarantees convergence in probability.
Convergence: The LLN illustrates the concept of convergence, where the outcomes of a random experiment become more predictable and consistent as the sample size grows larger.
Sampling Variability: The LLN helps us understand that while individual observations may be subject to variability, the average of a large number of observations tends to stabilize around the expected value.
Practical Implications: The LLN has practical implications for decision-making, risk assessment, and statistical inference. It provides a foundation for making predictions based on observed data.
Simulations and the LLN:
Role of Simulations: Simulations involve creating artificial scenarios or models to mimic real-world situations. Simulations can help illustrate and validate the principles of the LLN.
Monte Carlo Simulations: Monte Carlo simulations use random sampling to estimate probabilities or simulate outcomes. These simulations often rely on the LLN to produce accurate approximations.
Accuracy with Large Samples: Simulations based on the LLN tend to produce more accurate results when using larger sample sizes, as the LLN indicates that the sample mean becomes closer to the true mean as sample size increases.
Visualization of Convergence: Simulations can visually demonstrate the convergence described by the LLN. As the number of simulation runs increases, the observed values tend to cluster around the expected value.
Limitations and Requirements: While simulations can provide powerful insights into the LLN, they require careful design and understanding of the underlying probability distributions. Additionally, the LLN assumes certain conditions, such as independence and identically distributed random variables.
Example: Rolling Two Dice
Suppose you have two fair six-sided dice (labeled 1 through 6), and you're interested in calculating the probability of different outcomes when rolling both dice.
Question: What is the probability of rolling a sum of 7 when two dice are rolled?
Solution: – Step 1: Determine the Sample Space The sample space consists of all possible outcomes when rolling two dice. Each die has 6 sides, so the total number of possible outcomes is 6 * 6 = 36.
Sample Space (S): {(1, 1), (1, 2), (1, 3), …, (6, 6)}
Step 2: Identify the Event We're interested in the event of rolling a sum of 7. Let's list the outcomes that result in a sum of 7:
Event (E): Rolling a sum of 7 {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)}
Step 3: Calculate Probability To calculate the probability of the event E, we'll use the formula:
Number of favorable outcomesTotal number of possible outcomesP(E)=Total number of possible outcomesNumber of favorable outcomes
Number of favorable outcomes = 6 (since there are 6 pairs that result in a sum of 7) Total number of possible outcomes = 36
So, the probability of rolling a sum of 7 when two dice are rolled is 1661 or approximately 0.1667.
Key Points
Sample Space (S): The sample space is the set of all possible outcomes of a random experiment. It includes all distinct individual outcomes.
Event (E): An event is a subset of the sample space. It consists of one or more outcomes of interest.
Probability (P): Probability is a measure of the likelihood of an event occurring. It ranges from 0 (impossible) to 1 (certain).
.
Empirical Probability: Empirical probability is based on observed data. The probability of an event A is estimated as the relative frequency of A occurring in a large number of trials.
Subjective Probability: Subjective probability is based on personal judgment or belief about the likelihood of an event occurring.
Mutually Exclusive Events: Mutually exclusive events are events that cannot occur simultaneously. The probability of the intersection of mutually exclusive events is zero.
.
Law of Large Numbers: The Law of Large Numbers states that as the number of trials or observations increases, the sample mean approaches the true mean or expected value.
Simulation: Simulation involves using random sampling to estimate probabilities or simulate outcomes. It is a useful tool for understanding and illustrating random event probabilities.
Probability Distributions: Probability distributions describe how probabilities are distributed among different values of a random variable. They provide insights into the likelihood of different outcomes.