Hypothesis testing is a crucial statistical tool for making informed decisions based on data. It involves formulating null and alternative hypotheses, analyzing data, and drawing conclusions about population parameters.

Understanding p-values, significance levels, and statistical power helps researchers balance the risks of Type I and Type II errors. These concepts guide study design, data interpretation, and decision-making across various fields, from medicine to business.

Understanding Hypothesis Testing

Null vs alternative hypotheses

• Null hypothesis (H₀) assumes no effect or difference typically includes equality (=, ≤, or ≥) (Earth is flat)
• Alternative hypothesis (H₁ or Hₐ) opposes null hypothesis, aims to support researcher's claim typically includes inequality (≠, >, <) (Earth is round)
• Frame research question, guide statistical analysis, determine test direction (one-tailed or two-tailed)
• Examples: Drug effectiveness (H₀: new drug = placebo, H₁: new drug > placebo), Gender pay gap (H₀: male salary ≤ female salary, H₁: male salary > female salary)

Type I and Type II errors

• Type I error (false positive) rejects true null hypothesis probability α (alpha) leads to unnecessary changes, wasted resources (convicting innocent person)
• Type II error (false negative) fails to reject false null hypothesis probability β (beta) results in missed opportunities, undetected effects (acquitting guilty person)
• Trade-off between errors decreasing one increases the other
• Decision-making impact balances risks, considers costs and consequences of incorrect decisions (medical diagnosis, quality control)

P-values and significance levels

• P-value probability of extreme results assuming H₀ true calculated using test statistic and distribution
• Significance level (α) predetermined threshold for rejecting H₀ common values: 0.05, 0.01, 0.1
• Interpretation: reject H₀ if p-value < α, fail to reject if p-value ≥ α
• Smaller p-values indicate stronger evidence against H₀
• Limitations: doesn't measure effect size or practical significance
• Examples: Clinical trials (p = 0.03 < α = 0.05, reject H₀), Market research (p = 0.08 > α = 0.05, fail to reject H₀)

Statistical power in testing

• Probability of correctly rejecting false null hypothesis represented as 1 - β
• Affected by sample size, effect size, significance level (α), data variability
• Determines ability to detect true effects, aids study design and sample size calculation
• Power analysis used to determine required sample size for desired power balances Type I and II error risks
• Low power increases Type II error risk, reduces research reproducibility
• Examples: Drug trials (80% power to detect 20% improvement), Psychology experiments (90% power for medium effect size)