What is Inference in Probability?
Inference in probability refers to the process of drawing conclusions about a population or a random phenomenon based on limited data or observed outcomes. In simple terms, it’s about using what we know (data) to make educated guesses about what we don’t know (true probabilities or distributions).
Key Goal
Make reliable statements or predictions about a random process or population using sample data.
Key Concepts
1. Descriptive vs. Inferential Probability
- Descriptive probability just summarizes the known: for example, rolling a fair die and saying the probability of getting a 6 is 1/6.
- Inferential probability comes in when we don’t know the die is fair and try to estimate the probability of getting a 6 based on repeated rolls.
2. Prior and Posterior Probabilities
- Prior probability is what we believe about an event before seeing any data.
- Posterior probability is the updated belief after incorporating the observed data.
3. Bayes’ Theorem
Bayes’ theorem provides a mathematical way to update our beliefs based on new evidence. It is central to inferential probability.
P(A|B) = [P(B|A) * P(A)] / P(B)
- P(A) = prior probability of A
- P(B|A) = likelihood of observing B if A is true
- P(B) = total probability of B
- P(A|B) = posterior probability (updated probability of A given B)
Examples
1. Medical Testing
Suppose 1% of people have a rare disease. A test for it is 99% accurate. If someone tests positive, what is the probability they actually have the disease?
Let D = has disease, ¬D = no disease Let T+ = tests positive P(D) = 0.01 P(¬D) = 0.99 P(T+|D) = 0.99 P(T+|¬D) = 0.01P(T+) = P(T+|D)P(D) + P(T+|¬D)P(¬D) = (0.99)(0.01) + (0.01)(0.99) = 0.0198 P(D|T+) = [0.99 * 0.01] / 0.0198 ≈ 0.5
Conclusion: Even with a positive result, there’s only a 50% chance the person actually has the disease.
2. Polling
If you survey 100 people and 60 say they support candidate A, what’s the probability that more than half the total population supports A?
We can model this using a binomial distribution and infer a confidence interval around the estimated proportion (60%).
3. Coin Tosses
Imagine you’re given a coin and asked whether it’s fair. You toss it 10 times and get 8 heads. You can use inference to estimate the likelihood it’s biased toward heads.
Likelihood
Likelihood is the probability of observing the data given a parameter. It’s used in Maximum Likelihood Estimation (MLE) to find the best estimate of the parameter.
Example: If 8 out of 10 tosses are heads: L(p) = p⁸(1-p)² (likelihood function) You find the p that maximizes L(p).
Frequentist vs. Bayesian Inference
| Aspect | Frequentist | Bayesian |
|---|---|---|
| Focus | Long-run frequency of outcomes | Belief updating using prior and evidence |
| Parameters | Fixed but unknown | Treated as random variables |
| Confidence | Confidence intervals | Credible intervals |
| Interpretation | Probability = limit of frequency | Probability = degree of belief |
Conclusion
Inference in probability is foundational in data science, medicine, research, and AI. It allows us to make educated decisions even when full information isn’t available. Understanding how to update beliefs based on new evidence—especially via Bayes’ theorem—makes you powerful in applying statistical thinking to real-life situations.