15 Real-World Examples of Hypothesis Testing

An e-commerce retailer wants to increase checkout completions. The design team builds a single-page checkout flow (Version B) to replace the current three-step checkout flow (Version A).

• Null Hypothesis ($H_0$): The conversion rate for Version B is equal to or less than Version A.
• Alternative Hypothesis ($H_A$): The conversion rate for Version B is greater than Version A.

The Test: A two-proportion z-test compares the conversion rates of two random groups of shoppers over two weeks.

Real-World Outcome: Version B hits a conversion rate of 4.2%, while Version A stays at 3.5%. The resulting p-value is 0.012. Since 0.012 is less than 0.05, the retailer rejects the null hypothesis and launches the single-page checkout.

A software company sends a weekly newsletter to 100,000 subscribers. The marketing team wants to know if adding an emoji to the subject line boosts open rates.

• Null Hypothesis ($H_0$): Subject lines with emojis have the same or lower open rates than subject lines without emojis.
• Alternative Hypothesis ($H_A$): Subject lines with emojis have higher open rates than subject lines without emojis.

The Test: A chi-square test for independence checks the raw counts of opened versus unopened emails across two equal sample groups.

Real-World Outcome: The emoji subject line scores a 22% open rate, while the plain text subject line gets 19%. The p-value comes out to 0.003, indicating a statistically significant difference.

A streaming platform wants to raise its monthly price by two dollars. Management worries the hike will spike subscriber cancellations (churn). They test the new price on a small, random sample of users before rolling it out globally.

• Null Hypothesis ($H_0$): The churn rate under the higher price remains equal to or lower than the current churn rate.
• Alternative Hypothesis ($H_A$): The churn rate under the higher price is higher than the current churn rate.

The Test: A two-sample z-test for proportions evaluates the churn metrics between the test group and a control group.

Real-World Outcome: The test group shows a 4.1% churn rate, while the control group shows 4.0%. The p-value drops in at 0.65. Because 0.65 is much larger than 0.05, the company fails to reject the null hypothesis, concluding that the price hike does not cause a meaningful rise in cancellations.

A pharmaceutical firm creates a new drug designed to lower LDL cholesterol. They set up a clinical trial with 500 patients. Half receive the new drug, while the other half get the standard current medication.

• Null Hypothesis ($H_0$): The average drop in cholesterol from the new drug is equal to or less than the drop from the current medication.
• Alternative Hypothesis ($H_A$): The average drop in cholesterol from the new drug is greater than the drop from the current medication.

The Test: An independent two-sample t-test compares the mean cholesterol reductions of the two groups.

Real-World Outcome: Patients on the new drug see an average drop of 45 mg/dL, while the standard group experiences a 30 mg/dL drop. The p-value is less than 0.0001, allowing researchers to reject the null hypothesis and pursue regulatory approval.

A sports medicine clinic introduces a new resistance band routine for knee rehabilitation. They track the recovery time (in days) it takes for athletes to regain full range of motion.

• Null Hypothesis ($H_0$): The mean recovery time with the new routine is identical to the recovery time under the old stretching routine.
• Alternative Hypothesis ($H_A$): The mean recovery time with the new routine is different from the recovery time under the old stretching routine.

The Test: A two-tailed, two-sample t-test evaluates the recovery durations of both athlete groups.

Real-World Outcome: The new routine averages 18 days, while the old method averages 21 days. The calculated p-value is 0.04. Since 0.04 sits below the 0.05 threshold, the clinic rejects the null hypothesis and adopts the new resistance band protocol.

A regional hospital updates its digital patient-triage software to cut down emergency room wait times. The historic average wait time is 45 minutes.

• Null Hypothesis ($H_0$): The mean wait time with the new software is 45 minutes or longer.
• Alternative Hypothesis ($H_A$): The mean wait time with the new software is less than 45 minutes.

The Test: A one-sample t-test evaluates the mean wait times of 200 patients treated after the software update against the historic 45-minute baseline.

Real-World Outcome: The sample mean drops to 38 minutes with a standard deviation of 12 minutes, giving a t statistic of about -8.3. The p-value is far below 0.0001. The hospital rejects the null hypothesis, providing strong evidence that the software shortened wait times.

A national retail bank trains its call center staff using a new communication script designed to resolve customer complaints faster. The management team measures the average call duration in seconds.

• Null Hypothesis ($H_0$): Call durations after the training are equal to or longer than call durations before the training.
• Alternative Hypothesis ($H_A$): Call durations after the training are shorter than call durations before the training.

The Test: A paired t-test compares the same group of call center agents before and after completing the training program.

Real-World Outcome: Average call times drop from 240 seconds to 215 seconds. The p-value hits 0.02, leading the bank to reject the null hypothesis and expand the training program nationwide.

An investment advisor claims that a new algorithmic stock-picking strategy delivers higher monthly returns than a standard S&P 500 index fund.

• Null Hypothesis ($H_0$): The mean monthly return of the algorithmic strategy is equal to or less than the index fund return.
• Alternative Hypothesis ($H_A$): The mean monthly return of the algorithmic strategy is greater than the index fund return.

The Test: A one-tailed, two-sample t-test compares the monthly returns of both investment vehicles over a three-year window.

Real-World Outcome: The algorithmic strategy averages a 1.2% monthly return, while the index fund averages 0.9%. However, due to high market volatility, the p-value lands at 0.18. Because 0.18 is greater than 0.05, the advisor fails to reject the null hypothesis, meaning the algorithm cannot definitively claim superior performance.

An insurance firm allows its data entry department to work from home full-time. Management wants to ensure that this policy does not lower the daily volume of processed claims per employee.

• Null Hypothesis ($H_0$): The mean number of processed claims per day for remote workers is equal to or greater than the in-office baseline (50 claims).
• Alternative Hypothesis ($H_A$): The mean number of processed claims per day for remote workers is less than the in-office baseline.

The Test: A one-sample t-test evaluates the daily output of 80 remote employees against the historic average of 50.

Real-World Outcome: Remote employees process an average of 52 claims per day. The p-value is 0.88. The firm fails to reject the null hypothesis and finds no evidence that remote work hurt productivity levels.

A beverage factory fills bottles with an advertised volume of 500 milliliters. If the machines underfill the bottles, customers get cheated and regulatory fines follow. If they overfill, the company loses money.

• Null Hypothesis ($H_0$): The mean volume of liquid in the bottles is exactly 500 mL ($\mu = 500$).
• Alternative Hypothesis ($H_A$): The mean volume of liquid in the bottles is not 500 mL ($\mu \neq 500$).

The Test: A two-tailed, one-sample z-test checks a random batch of 100 bottles off the assembly line.

Real-World Outcome: The sample mean is 501.2 mL, with a known machine standard deviation of 4 mL. The p-value is 0.0027. Since this is well below 0.05, the plant manager rejects the null hypothesis and shuts down the assembly line to recalibrate the filling nozzles.

An electronics brand sources lithium-ion batteries from a new supplier who promises an average lifespan of 1,200 charge cycles. The brand's engineering team tests a sample of 40 batteries to verify this claim.

• Null Hypothesis ($H_0$): The average lifespan of the new batteries is 1,200 cycles or more.
• Alternative Hypothesis ($H_A$): The average lifespan of the new batteries is less than 1,200 cycles.

The Test: A one-sample t-test checks the experimental lifespans against the supplier's promise.

Real-World Outcome: The sample batteries average 1,150 cycles. The resulting p-value is 0.035. The engineering team rejects the null hypothesis and rejects the contract, citing that the batteries fail to meet the promised durability standard.

An aerospace firm tests a new carbon-fiber composite material from two manufacturing plants. They need to ensure that the material coming from Plant X has the same structural strength as the material from Plant Y.

• Null Hypothesis ($H_0$): The mean tensile strength of the material from Plant X is equal to the mean tensile strength from Plant Y.
• Alternative Hypothesis ($H_A$): The mean tensile strength of the material from Plant X is different from the mean tensile strength from Plant Y.

The Test: An independent two-sample t-test compares the breaking points of 30 samples from each location.

Real-World Outcome: Plant X samples break at an average of 720 MPa, while Plant Y samples break at 725 MPa. The p-value is 0.42. The firm fails to reject the null hypothesis, concluding that the materials from both plants are functionally identical in strength.

A school district buys an interactive math app for third-grade classrooms. To see if it helps, they introduce the app to five schools while keeping five other schools on the traditional textbook curriculum.

• Null Hypothesis ($H_0$): The mean standardized test scores of students using the app are equal to or lower than the scores of students using textbooks.
• Alternative Hypothesis ($H_A$): The mean standardized test scores of students using the app are higher than the scores of students using textbooks.

The Test: A two-sample t-test evaluates final exam scores across both classroom groups.

Real-World Outcome: App-using classrooms average a score of 84%, while textbook classrooms average 79%. The p-value lands at 0.008, meaning the district can reject the null hypothesis and fund the app for all schools.

A city council installs speed bumps along a busy residential avenue to slow down local traffic. The original average speed on the street was 34 miles per hour.

• Null Hypothesis ($H_0$): The mean vehicle speed after installing the speed bumps is 34 mph or higher.
• Alternative Hypothesis ($H_A$): The mean vehicle speed after installing the speed bumps is less than 34 mph.

The Test: A one-sample t-test checks the speeds of 500 cars clocked via radar after the installation.

Real-World Outcome: The average speed drops to 28 mph. The p-value is less than 0.0001. The council rejects the null hypothesis, supporting the conclusion that the speed bumps slowed traffic.

A city runs a major public health campaign promoting adult flu shots. They want to know if vaccination rates changed across four distinct age brackets (18–30, 31–45, 46–60, and 61+).

• Null Hypothesis ($H_0$): The proportion of residents who got a flu shot is independent of their age bracket.
• Alternative Hypothesis ($H_A$): The proportion of residents who got a flu shot is dependent on their age bracket.

The Test: A chi-square test of independence evaluates the survey responses of 1,200 residents.

Real-World Outcome: The data shows significantly higher vaccination uptakes in the 61+ group compared to younger demographics. The p-value is 0.0002, leading health officials to reject the null hypothesis and re-target future campaigns toward younger demographics.