All posts by : bonaventurekulula207@gmail.com

Introduction

A joint distribution describes the probability behavior of two or more random variables simultaneously. It’s foundational in multivariate probability, helping us understand how variables interact and depend on one another.

Definition

For two random variables X and Y, their joint probability distribution gives the probability that X = x and Y = y simultaneously.

Discrete Case

P(X = x, Y = y) = p(x, y)

The joint probability mass function (pmf) must satisfy:

• p(x, y) ≥ 0 for all x, y
• ∑∑ p(x, y) = 1

Continuous Case

f(x, y) = joint probability density function

It must satisfy:

• f(x, y) ≥ 0
• ∬ f(x, y) dx dy = 1

Marginal Distributions

The marginal distributions give the individual probabilities for X or Y by summing or integrating over the other variable.

Discrete:

P(X = x) = ∑ p(x, y) P(Y = y) = ∑ p(x, y)

Continuous: f_X(x) = ∫ f(x, y) dy f_Y(y) = ∫ f(x, y) dx

Conditional Distributions

Conditional distributions tell us the probability of one variable given that the other has occurred.

Discrete:

P(X = x | Y = y) = P(X = x, Y = y) / P(Y = y)

Continuous: f(x | y) = f(x, y) / f_Y(y)

Independence

Two variables X and Y are independent if:

Discrete:  P(X = x, Y = y) = P(X = x) * P(Y = y)

Continuous: f(x, y) = f_X(x) * f_Y(y)

Practical Example: Discrete Joint Distribution Table

Suppose a factory produces two types of items, A and B, and records whether each item passes (1) or fails (0) quality inspection. Let:

• X: type of item (A=0, B=1)
• Y: result of inspection (Pass=1, Fail=0)

The joint distribution is given by the table below:

– P(X = 0, Y = 1) = 0.30
– P(X = 1 | Y = 1) = 0.40 / 0.70 ≈ 0.571
– Check for independence: P(X=0) * P(Y=1) = 0.40 × 0.70 = 0.28 ≠ 0.30 → Not independent

Example: Continuous Joint Distribution

Let f(x, y) = 2 for 0 ≤ x ≤ 1 and 0 ≤ y ≤ 1. Then:

• Total probability = ∬ f(x, y) dx dy = 2 × 1 × 1 = 2 → Not valid!
• Must normalize: use f(x, y) = 1 instead so total probability = 1

Python Example (Continuous)

import numpy as np

from scipy.stats import multivariate_normal

Define mean vector and covariance matrix

mu = [0, 0] cov = [[1, 0.5], [0.5, 1]]

Generate joint PDF values on grid

x, y = np.mgrid[-3:3:.1, -3:3:.1] pos = np.dstack((x, y)) z = multivariate_normal(mu, cov).pdf(pos)

Plot with matplotlib

import matplotlib.pyplot as plt plt.contourf(x, y, z) plt.title('Joint Normal Distribution') plt.colorbar() plt.show()

Applications

• Modeling correlations between variables
• Bayesian statistics and joint likelihoods
• Multivariate regression and classification
• Econometrics, finance, and machine learning

Conclusion

A joint distribution is crucial for analyzing relationships between multiple random variables. By understanding joint, marginal, and conditional distributions, we can uncover dependencies and structure in data, forming the backbone of multivariate statistics and data science.

Introduction

The normal distribution, also known as the Gaussian distribution, is one of the most important probability distributions in statistics. It models many natural phenomena such as heights, weights, test scores, and errors in measurements. Its distinctive bell-shaped curve makes it easily recognizable.

Definition

A random variable X is normally distributed with mean μ and standard deviation σ if its probability density function (PDF) is:

f(x) = (1 / (σ√(2π))) * e-(x - μ)² / (2σ²)

This is denoted as:

X ~ N(μ, σ²)

Characteristics

• Symmetric about the mean μ
• Mean, median, and mode are all equal
• Bell-shaped curve
• Total area under the curve = 1
• Follows the empirical rule (68-95-99.7 rule)

Empirical Rule (68-95-99.7)

For a normal distribution:

• ~68% of values fall within 1 standard deviation of the mean
• ~95% fall within 2 standard deviations
• ~99.7% fall within 3 standard deviations

Standard Normal Distribution

When μ = 0 and σ = 1, the distribution is called the standard normal distribution and is denoted as:

Z ~ N(0, 1)

You can convert any normal variable to a standard normal using the Z-score:

Z = (X - μ) / σ

Applications

• Modeling measurement errors
• Statistical inference (e.g., confidence intervals, hypothesis testing)
• Standardized test scoring
• Quality control in manufacturing

Example

Suppose the heights of adult men are normally distributed with mean μ = 175 cm and standard deviation σ = 10 cm. What is the probability that a randomly chosen man is taller than 190 cm?

Z = (190 - 175) / 10 = 1.5

P(X > 190) = P(Z > 1.5) ≈ 0.0668

So there’s about a 6.68% chance a randomly selected man is taller than 190 cm.

Python Code Example

from scipy.stats import norm

Mean and standard deviation

mu = 175 sigma = 10

Probability of height greater than 190 cm

p = 1 - norm.cdf(190, loc=mu, scale=sigma) print("P(X > 190):", p)

Conclusion

The normal distribution is central to statistics due to the Central Limit Theorem, which states that the sum of many independent random variables tends to be normally distributed. Its properties make it a cornerstone in many statistical procedures, from hypothesis testing to regression analysis.

Introduction

The exponential random variable is a continuous probability distribution that describes the time between independent events occurring at a constant average rate. It is widely used in reliability engineering, queuing theory, survival analysis, and various stochastic processes.

Definition

If a random variable X follows an exponential distribution with parameter λ > 0, we write:

X ~ Exp(λ)

Here, λ is the rate parameter, which represents the average number of events per unit time. The mean time between events is 1/λ.

Probability Density Function (PDF)

f(x) = λ * e-λx   for x ≥ 0

f(x) = 0              for x < 0

The PDF shows that the probability decreases exponentially as x increases. The distribution is heavily right-skewed.

Cumulative Distribution Function (CDF)

F(x) = 1 - e-λx  for x ≥ 0

F(x) = 0            for x < 0

The CDF gives the probability that the time until the next event is less than or equal to x.

Mean and Variance

• Mean: E[X] = 1/λ
• Variance: Var(X) = 1/λ²

Key Property: Memorylessness

The exponential distribution is the only continuous distribution that is memoryless. That is:

P(X > s + t | X > s) = P(X > t)

This means the probability that the process lasts at least another t units of time does not depend on how much time has already passed.

Example

Suppose the average number of phone calls received by a call center is 4 per hour. Then the time between two consecutive calls is exponentially distributed with λ = 4.

- Probability that you wait less than 15 minutes (0.25 hours) for the next call:

P(X < 0.25) = 1 - e-λx = 1 - e-4 * 0.25 = 1 - e-1 ≈ 0.632

So there's about a 63.2% chance a call comes within the first 15 minutes.

Applications

• Modeling time between arrivals in a Poisson process
• Reliability of systems (e.g. lifespan of electronic components)
• Survival analysis in medical statistics
• Queuing systems (e.g., wait time until next customer)

Python Code Example

from scipy.stats import expon

Set lambda (rate) = 4 => scale = 1/lambda

scale = 1 / 4

Mean and variance

mean, var = expon.mean(scale=scale), expon.var(scale=scale) print("Mean:", mean) print("Variance:", var)

Probability of waiting less than 15 minutes (0.25 hours)

p = expon.cdf(0.25, scale=scale) print("P(X < 0.25):", p)

Conclusion

The exponential random variable is essential for modeling the timing of random events. Its simplicity and powerful properties—particularly memorylessness—make it foundational in probability theory, stochastic modeling, and real-world applications involving waiting times and failure rates.

Introduction

The uniform random variable is one of the simplest and most fundamental probability distributions. It models a situation in which all outcomes in a given interval are equally likely. It’s often used as a building block for other distributions and is essential in simulations and Monte Carlo methods.

Definition

A uniform random variable can be either discrete or continuous. The most common and widely used is the continuous uniform distribution.

A continuous uniform random variable X is said to be uniformly distributed over the interval [a, b], written as:

X ~ U(a, b)

This means that the probability of X falling anywhere in the interval [a, b] is equally likely.

Probability Density Function (PDF)

The PDF of a uniform distribution on [a, b] is defined as:

f(x) = 1 / (b - a),  for a ≤ x ≤ b

f(x) = 0,           otherwise

This flat or constant density function indicates that all values in the interval [a, b] are equally probable.

Cumulative Distribution Function (CDF)

The CDF, which gives the probability that X is less than or equal to a certain value x, is:

F(x) = 0               if x < a

F(x) = (x - a) / (b - a) if a ≤ x ≤ b F(x) = 1                  if x > b

Mean and Variance

For X ~ U(a, b):

• Mean: E[X] = (a + b) / 2
• Variance: Var(X) = (b - a)2 / 12

Example

Suppose a bus arrives at a stop every 20 minutes. If you arrive at a random time, your waiting time X is uniformly distributed between 0 and 20 minutes: X ~ U(0, 20).

– The probability that you wait less than 5 minutes:

P(X ≤ 5) = (5 - 0) / (20 - 0) = 5 / 20 = 0.25

– The average wait time:

E[X] = (0 + 20) / 2 = 10 minutes

Discrete Uniform Distribution

A discrete uniform distribution assigns equal probability to a finite set of n outcomes. For example, rolling a fair 6-sided die produces outcomes {1, 2, 3, 4, 5, 6}, each with probability 1/6.

If X is uniformly distributed over {x1, x2, ..., xn}, then:

P(X = xi) = 1 / n  for all i

Applications

• Random number generation
• Simulation and modeling in Monte Carlo methods
• Estimating probabilities in uniformly distributed data
• Game theory and lotteries

Python Code Example

from scipy.stats import uniform

Continuous uniform from a = 0 to b = 20

mean, var = uniform.mean(loc=0, scale=20), uniform.var(loc=0, scale=20) print("Mean:", mean) print("Variance:", var)

Probability of waiting less than 5 minutes

p = uniform.cdf(5, loc=0, scale=20) print("P(X ≤ 5):", p)

Conclusion

The uniform random variable is a foundational concept in probability and statistics. It models situations with equal likelihood and serves as a simple but powerful tool in data analysis and simulation. Whether dealing with continuous ranges or finite discrete sets, understanding uniform distributions helps establish deeper intuition for randomness and fair systems.

Introduction

The negative binomial distribution is a fundamental probability distribution used in statistics to model the number of independent Bernoulli trials needed to achieve a fixed number of successes. It generalizes the geometric distribution, which models the number of trials until the first success.

This distribution is especially useful in situations where you are interested in counting the number of attempts needed to observe a specific number of successful outcomes, with each trial having the same probability of success.

Definition

Let X be a random variable representing the number of trials needed to get r successes in a sequence of independent Bernoulli trials, each with probability of success p. Then X follows a Negative Binomial Distribution.

The number of failures before the r-th success is what is often modeled. The probability mass function (PMF) is:

P(X = x) = C(x + r - 1, r - 1) * p^r * (1 - p)^x

for x = 0, 1, 2, ...

Here, X counts the number of failures before the r-th success, and C(n, k) denotes the binomial coefficient:

C(n, k) = n! / (k! * (n - k)!)

Parameters

• r: Number of desired successes (a positive integer)
• p: Probability of success on each trial (0 < p < 1)
• X: Number of failures before achieving r successes

Mean and Variance

If X follows a Negative Binomial Distribution with parameters r and p, then:

• Mean (Expected value): E[X] = r * (1 - p) / p
• Variance: Var(X) = r * (1 - p) / p²

Special Case: Geometric Distribution

The geometric distribution is a special case of the negative binomial distribution when r = 1. In that case, the negative binomial distribution simplifies to counting the number of failures before the first success.

Example

Suppose you are rolling a die, and you define success as rolling a 6 (p = 1/6). What is the probability that you roll the die 10 times and get the 3rd success on the 10th roll?

First, you must have had x = 7 failures before the 3rd success (since 10 – 3 = 7), and r = 3. Plug into the formula:

P(X = 7) = C(7 + 3 - 1, 3 - 1) * (1/6)^3 * (5/6)^7
     = C(9, 2) * (1/216) * (78125 / 279936)

Calculate and simplify for the numerical result.

Applications

• Modeling the number of accidents before a fixed number of safe days
• Predicting the number of failed transactions before reaching a success quota
• Call center analytics (e.g., number of calls before getting 5 successful sales)
• Quality assurance and manufacturing defects tracking

Python Code Example

from scipy.stats import nbinom

r = 3        # number of successes p = 1/6      # probability of success x = 7        # number of failures

Probability of exactly 7 failures before 3rd success

prob = nbinom.pmf(x, r, p) print(f"P(X = {x}) = {prob:.6f}")

Mean and variance

mean = nbinom.mean(r, p) var = nbinom.var(r, p) print(f"Mean: {mean}, Variance: {var}")

Conclusion

The negative binomial distribution is a versatile and powerful tool in probability, especially useful for modeling events where multiple successes are required over a sequence of trials. It generalizes the geometric distribution and has wide applications in quality control, economics, public health, and more.

Understanding its structure, formula, and behavior allows analysts and statisticians to model uncertainty in a wide range of real-world processes where success isn’t guaranteed on the first few tries.