Hypothesis testing is a key statistical tool for drawing conclusions about populations based on sample data. It involves formulating null and alternative hypotheses, choosing an appropriate test statistic, and interpreting p-values to make decisions.

This process allows researchers to quantify uncertainty and control error rates when making inferences. Understanding the fundamentals of hypothesis testing is crucial for conducting and interpreting statistical analyses across various fields of study.

Hypothesis Testing in Statistics

Concept and Purpose

• Hypothesis testing is a formal procedure used to test a claim or hypothesis about a population parameter based on sample data
  • Follows a multi-step process to assess the strength of evidence and make decisions
• The purpose of hypothesis testing is to use sample data to draw inferences about the population
  • Determines whether there is enough statistical evidence to support or refute a claim about the population parameter
• Hypothesis tests have two mutually exclusive and exhaustive hypotheses
  • Null hypothesis (H₀) assumes no effect or relationship exists
  • Alternative hypothesis (Hₐ) proposes a specific effect or relationship

Controlling Uncertainty and Logic

• Hypothesis testing allows researchers to control and quantify uncertainty by setting an acceptable level of risk for making errors
  • Type I error: rejecting a true null hypothesis
  • Type II error: failing to reject a false null hypothesis
  • Significance level (α) represents the probability of a Type I error
• The logic of hypothesis testing involves assuming the null hypothesis is true
  • Assesses whether the observed data is sufficiently unlikely under the null to warrant rejecting it in favor of the alternative hypothesis
• The conclusions of a hypothesis test are either:
  • Reject the null hypothesis if the sample evidence strongly contradicts it
  • Fail to reject the null hypothesis if the sample evidence is not sufficiently convincing

Formulating Hypotheses

Null and Alternative Hypotheses

• The null hypothesis (H₀) is a statement of no effect, relationship, or difference
  • Assumed to be true unless there is strong evidence against it
  • Contains an equality sign (=, ≤, or ≥)
• The alternative hypothesis (Hₐ) represents the proposed effect, relationship, or difference that the researcher suspects or wants to prove
  • Contradicts the null hypothesis and is accepted if the null is rejected
• Alternative hypotheses can be:
  • One-sided (directional): specifies a direction (<, >)
  • Two-sided (non-directional): claims the parameter is not equal (≠) to the null value

Choosing Hypotheses

• The choice between a one-sided or two-sided alternative depends on:
  • Research question
  • Prior knowledge
  • Consequences of the conclusion
• One-sided tests are justified when:
  • The researcher has a directional theory
  • Only one direction is meaningful or actionable
• Hypotheses are formulated in terms of population parameters, not sample statistics
  • The parameter of interest depends on the research question and type of data (mean μ, proportion p, correlation ρ)
• Hypotheses should be stated in clear, specific terms before collecting data
  • Guides the choice of an appropriate test
  • Avoids bias
  • Hypotheses based on sample data are not valid

Choosing the Right Test

Test Statistics

• The test statistic is a standardized value calculated from the sample data
  • Measures the difference between the observed data and what is expected under the null hypothesis
  • Used to make a decision about the null hypothesis
• The type of test statistic depends on:
  • Research question
  • Data type
  • Assumptions
• Common test statistics include:
  • z (standard normal distribution)
  • t (t-distribution)
  • χ² (chi-square distribution)
  • F (F-distribution)
• The choice of test statistic determines the sampling distribution used to find critical values and p-values

Critical Values

• The critical value is the cutoff value on the sampling distribution that defines the rejection region for a hypothesis test at a given significance level (α)
  • Separates sample values that are likely vs. unlikely to occur if the null hypothesis is true
• For a two-sided test with significance level α, the critical values are:
  • The positive and negative z-scores or t-scores that encompass a middle area of 1-α
• For a one-sided test, there is a single critical value at one tail encompassing an area of 1-α
• Critical values can be found using statistical tables or software and depend on:
  • Significance level (α)
  • Directionality of the test (one-sided or two-sided)
  • Degrees of freedom (for t-tests)

Interpreting P-Values

Calculating and Interpreting P-Values

• The p-value is the probability of obtaining a sample statistic as extreme or more extreme than the observed value, assuming the null hypothesis is true
  • Measures the strength of evidence against the null hypothesis
• The p-value is found by:
  • Calculating the test statistic from the sample data
  • Determining the area under the sampling distribution more extreme than that value, in the direction of the alternative hypothesis
• A small p-value (typically < .05) indicates strong evidence against the null hypothesis
• A large p-value (> .05) suggests weak evidence against the null hypothesis
  • The smaller the p-value, the stronger the evidence against the null hypothesis

Making Decisions and Conclusions

• If the p-value is less than the pre-determined significance level (α), the null hypothesis is rejected in favor of the alternative
• If the p-value is greater than α, we fail to reject the null hypothesis due to insufficient evidence
• When the null hypothesis is rejected, the results are deemed "statistically significant" at the α level
  • Statistical significance does not necessarily imply practical importance (depends on context and consequences)
• The significance level (α) should be chosen before conducting the hypothesis test based on the acceptable Type I error rate
  • Common choices are α = .05 or .01
  • The α level is a maximum threshold and does not need to be attained exactly by the p-value
• Hypothesis test conclusions are always stated in terms of the null hypothesis (reject H₀ or fail to reject H₀)
  • Should include the p-value and significance level
  • Accepting or proving the alternative is never concluded, only support for it