Statistics vs Probability

Mean, Median, and Mode

The three primary measures of central tendency. The mean is the arithmetic average, the median is the middle value when data are ordered, and the mode is the most frequently occurring value. Each measure captures a different aspect of a dataset's center.

Standard Deviation

A measure of the spread or dispersion of a dataset relative to its mean. It is calculated as the square root of the variance, which is the average of squared deviations from the mean. A low standard deviation indicates data points cluster near the mean, while a high value indicates greater spread.

Normal Distribution

A symmetric, bell-shaped probability distribution defined by its mean $\mu$ and standard deviation $\sigma$. It is fundamental to statistics because of the Central Limit Theorem, which states that sample means tend toward a normal distribution regardless of the population's shape. Approximately 68% of data fall within one standard deviation of the mean, 95% within two, and 99.7% within three.

Hypothesis Testing

A formal procedure for using sample data to evaluate claims about a population. The process involves stating a null hypothesis (no effect or no difference) and an alternative hypothesis, calculating a test statistic, and determining whether the evidence is strong enough to reject the null hypothesis at a chosen significance level.

P-Value

The probability of obtaining results at least as extreme as the observed results, assuming the null hypothesis is true. A small p-value suggests that the observed data are unlikely under the null hypothesis, providing evidence against it. It does not measure the probability that the null hypothesis is true.

Sample Space and Events

The sample space is the set of all possible outcomes of a random experiment, while an event is any subset of the sample space. Defining these precisely is the first step in any probability analysis, since probabilities are assigned to events rather than to individual outcomes in many frameworks.

Conditional Probability

Conditional probability measures the likelihood of an event occurring given that another event has already occurred. It is defined as $P(A|B) = \frac{P(A \cap B)}{P(B)}$, provided $P(B) > 0$. This concept is essential for updating beliefs when new information becomes available.

Bayes' Theorem

Bayes' theorem provides a formula for reversing conditional probabilities: $P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}$. It allows us to update a prior belief about event $A$ after observing evidence $B$, forming the foundation of Bayesian inference and decision-making under uncertainty.

Law of Large Numbers

The Law of Large Numbers states that as the number of independent, identically distributed trials increases, the sample average converges to the expected value. This theorem explains why casinos are profitable in the long run and why polling works despite individual unpredictability.

Central Limit Theorem

The Central Limit Theorem states that the sum or average of a large number of independent random variables, regardless of their original distribution, tends toward a normal (Gaussian) distribution. This result justifies the widespread use of normal-distribution-based methods in statistics.