In Six Sigma, hypothesis testing helps identify differences between machines, formulas, raw materials, etc. and are the differences statistically significant or not. Without such testing, teams can run around changing machine settings, formulas and so on causing more variation. These knee-jerk responses can amplify variation and cause more problems than doing nothing at all.
In manufacturing, you might want to compare two or more types of raw materials and determine if they produce the same quality. In other words, do the products have the same or different means and variances? If they are the same, which one is less expensive to produce? If they are different, which one best meets the customer's requirements?
Hypothesis testing helps identify ways to reduce costs and improve quality.
There are Three Types of Hypothesis Tests
Classical Method - comparing a test statistic to a critical value
p value Method - the probability of a test statistic being contrary to the null hypothesis
Confidence Interval Method - is the test statistic between or outside of the confidence interval
Setting Up a Hypothesis Test
First, you will need to define a null (H0) and an alternate (Ha) hypothesis.
By default, the null hypothesis assumes that the means, averages or variation are statistically the same. The goal is to prove that they are not statistically the same at some level of confidence (usually 95%, 99%).
Tests can be either two sided or one sided depending on how the null hypothesis is stated.
Null Hypothesis- two sided test Alternate Hypothesis - two sided test
Average(Sample 1) = Average(Sample 2)
Mean(Sample 1) = Mean(Sample 2)
Mean1 - Mean2 = 0 (no difference) Average(Sample 1) not = Average(Sample 2)
Mean(Sample 1) not = Mean(Sample 2)
Mean1 - Mean2 not = 0
Variance(Sample 1) = Variance(Sample 2) Variance(Sample 1) not = Variance(Sample 2)
Null Hypothesis - one sided test Alternate Hypothesis - one sided test
Average(Sample 1) <= Average(Sample 2)
Average(Sample 1) >= Average(Sample 2)
Mean(Sample 1) <= Mean(Sample 2)
Mean(Sample 1) >= Mean(Sample 2)
Mean1 - Mean2 <= 0
Mean1 - Mean2 >= 0 Average(Sample 1) > Average(Sample 2)
Average(Sample 1) < Average(Sample 2)
Mean(Sample 1) > Mean(Sample 2)
Mean(Sample 1) < Mean(Sample 2)
Mean1 - Mean2 > 0
Mean1 - Mean2 < 0
Variance(Sample 1) <= Variance(Sample 2)
Variance(Sample 1) > = Variance(Sample 2) Variance(Sample 1) > Variance(Sample 2)
Variance(Sample 1) < Variance(Sample 2)
Then, using data from the test:
Calculate the test statistic (t test, f test, z test, ANOVA, etc.).
The test statistic is often converted to a p value (probability), but not always.
Compare the test statistic to:
a significance level (a) or confidence level (1-a)
a critical value (e.g., Fcrit)
to determine if you can accept or reject the null hypothesis.
Hypothesis Test
Compare
Result
Classical Method test statistic > critical value
(i.e. F > F crit) Reject the null hypothesis
Classical Method test statistic < critical value
(i.e. F < F crit) Accept the null hypothesis
p value Method p value < a Reject the null hypothesis
p value Method p value > a Accept the null hypothesis
Determining the Correct Test Statistic
Consider your data.
Is it variable (i.e., measured - 3.4 lbs) or attribute (i.e., counted - 3 defects)?
Are there one, two or more samples?
Variable Data 1 Sample 2 Samples 2 or More Samples
t-test one sample F-Test
One Factor t-test ANOVA
One Way ANOVA
Two Factor Two Way ANOVA
Two Way ANOVA with Replication
Attribute Data 1 Sample 2 Samples 2 or More Samples
One Factor One Sample Proportion Test Two Sample Proportion Test Chi Square Test
Two Factor Chi Square Test
Are you comparing the means or variance?
Number of levels or treatments 1 2 3+
Mean (Average) t-test
one sample t test ANOVA
One Way ANOVA
Two Way ANOVA
Two Way ANOVA with Replication
Variance (Standard Deviation2) F test ANOVA
One Way ANOVA
Two Way ANOVA
Two Way ANOVA with Replication
Z Test
Type I and II Errors
Hypothesis testing seeks to determine if the means or variances are the same or different at some level of confidence. Since we can never be totally confident, it is possible to encounter two types of errors:
Type I error - Reject a null hypothesis that is true (Producer's Risk)
Type II error - Not reject a null hypothesis that is false (Consumer's Risk)
Choose a confidence (or significance) level that will minimize the risk associated with these errors.
All of these hypothesis tests are included in the QI Macros Statistical Process Control software.
