Ch6. Hypothesis Testing — Using Data to Evaluate Claims

What Is Hypothesis Testing?

Hypothesis Testing: A statistical procedure for deciding whether sample data supports or contradicts a claim about a population.

Types of Hypotheses

Null Hypothesis H₀: The status quo; assumes no difference, no effect, no change
Alternative Hypothesis H₁: The new claim; asserts a difference, effect, or change

Example: Testing the effectiveness of a new drug
H₀: The drug has no effect (mean improvement = 0)
H₁: The drug has an effect (mean improvement ≠ 0)

One-tailed vs Two-tailed Tests:

Two-tailed: H₁: μ ≠ μ₀ (detects either direction)
Right-tailed: H₁: μ > μ₀
Left-tailed: H₁: μ < μ₀

Hypothesis Testing Procedure

State the null and alternative hypotheses
Choose significance level α (commonly 0.05 or 0.01)
Calculate the test statistic
Determine the rejection region or p-value
Decide whether to reject H₀
State the conclusion in context

Significance Level and Rejection Region

Significance Level (α): The maximum probability of rejecting H₀ when it is actually true

Rejection Region: If the test statistic falls in this region, reject H₀

Two-tailed test, α = 0.05:
Rejection region: Z < −1.96  or  Z > 1.96
Acceptance region: −1.96 ≤ Z ≤ 1.96

p-value

p-value: The probability of observing results at least as extreme as the observed data, assuming H₀ is true

p-value < α → Reject H₀ ("statistically significant")
p-value ≥ α → Fail to reject H₀

Important: The p-value is NOT the probability that H₀ is true.
→ p = 0.03 means “if H₀ were true, there is only a 3% chance of observing data this extreme”

Type I and Type II Errors

	Reject H₀	Fail to Reject H₀
H₀ is True	Type I Error (α)	Correct Decision
H₀ is False	Correct Decision	Type II Error (β)

Type I Error: Convicting an innocent person (= significance level α)
Type II Error: Acquitting a guilty person (= β)
Power = 1 − β: The probability of correctly rejecting H₀ when it is false

Reducing α increases β — there is an inherent trade-off.

Key Testing Methods

Z-Test

Used when population SD σ is known, or n is large.

Z = (x̄ − μ₀) / (σ/√n)

t-Test (Student’s t-test)

Used when σ is unknown and the sample is small.

t = (x̄ − μ₀) / (s/√n)   (degrees of freedom: n−1)

Two-Sample Independent t-Test

Comparing means of two groups (e.g., clinical trial: treatment vs control).

t = (x̄₁ − x̄₂) / √[s₁²/n₁ + s₂²/n₂]

Key Concept Cards

p-value ★★★★★ : Probability of observing data this extreme assuming H₀ is true. If p < 0.05, reject H₀. Memory tip: p-value < α → reject the null hypothesis

Type I vs Type II Error ★★★★★ : Type I = rejecting a true H₀ (α); Type II = failing to reject a false H₀ (β). Stricter α increases β. Memory tip: Type I = false positive; Type II = false negative

Statistical Power (1−β) ★★★★☆ : The probability of detecting a real effect when one exists. Increases with larger sample sizes. Memory tip: power = 1 − Type II error probability

Practice Questions

Q. You are testing a new teaching method. Which type of error is more serious?

Context-dependent. A Type I error (concluding the method works when it doesn’t) wastes resources on an ineffective program. A Type II error (failing to detect a truly effective method) means missing a better educational approach. Set α based on the costs and consequences of each type of error.

Q. In an experiment, H₀ is rejected at α = 0.05 but not at α = 0.01. What is the range of the p-value?

0.01 ≤ p-value < 0.05. It is small enough to reject H₀ at the 0.05 threshold but not small enough to reject at 0.01.

OIYO Editorial