Hypothesis

A hypothesis is a provisional statement that has not yet been proven: what if what you think isn’t true? In an improvement project you test such a statement with data, to discover which causes really drive a problem.

Hypothesis tests to find causes

In the Analyse phase of a Lean Six Sigma project we use hypothesis tests to show which Xs are critical. Less cryptically: we look for the root causes of how the process performs, and a hypothesis test is a means of tracking down the most important influencing factors.

There are of course other ways. Brainstorming, a fishbone diagram or the map of the current process can just as well lead to a eureka moment for Green and Black Belts. By the end of Define, Measure and Analyse you find the main causes, and then you can work on Improve and Control in a focused way.

So why bother with statistics? Not all change agents are as keen on it as the professor of industrial statistics who shaped me into a Black Belt. The honest answer: you don’t have to, but you can. There are several roads to Rome. A hypothesis test does give you something brainstorming doesn’t: not only whether a cause has influence, but also how certain you are of it, and sometimes how big that influence is. Smart Belts first explore with brainstorming and the process map, and then test in a targeted way.

Guilty or innocent

A hypothesis test always consists of two statements, as shown in the image at the top of this article:

• The null hypothesis (H0): the cause has no influence. Nothing is going on; differences are noise.
• The alternative hypothesis (Ha): the cause does have influence.

As a Belt you secretly hope to reject the null hypothesis, because then you may have found a cause. It resembles a courtroom. You are innocent until proven guilty. The prosecutor gathers evidence to convince the judge that the null hypothesis (the defendant is innocent) can no longer be maintained.

In court, too, things can go wrong in two ways. An innocent person is convicted: you see a cause that isn’t there (a Type I error). Or a guilty person is acquitted: you miss a cause that is there (a Type II error). A hypothesis test deliberately keeps the chance of the first error small, usually at 5%.

Five steps

You test a hypothesis in five steps. Below they are shown with an example: does handling one type of complaint take longer than the other?

1. Formulate the hypothesis. Complaint handling takes too long, and there are two types of complaint. H0: complaint type makes no difference to lead time. Ha: one type of complaint is handled faster on average than the other.
2. Collect data. Collect the lead time per complaint, with the complaint type. If you want to know whether the day of the week matters, record the day too.
3. Determine the data type and number of groups. Lead time is continuous data; complaint type is discrete, with two groups. Read more about continuous and discrete data.
4. Choose the test. Which test fits depends on the data type and the number of groups. With a continuous Y and two groups it is the two-sample t-test; with three or more groups an ANOVA; with counts a chi-square test. Also check the assumptions, such as whether the data is roughly normally distributed. In the Green Belt and Black Belt courses we use a decision tree that helps you pick the right test.
5. Interpret the outcome. Look at the p-value.

The p-value

The p-value is the probability of finding a difference this large (or larger) if the null hypothesis is true, that is, if there is in reality no difference. So it is not the probability that the null hypothesis is true, a common mistake.

• p below 0.05: such a difference is unlikely to be chance. You reject the null hypothesis: a difference has been shown. The well-known mnemonic among Belts: if p is low, H0 must go.
• p of 0.05 or more: you don’t reject the null hypothesis. That doesn’t mean there is no difference, but that it hasn’t been shown. Perhaps the difference is small, or you had too little data.

Unsurprisingly, change agents sometimes jump for joy at a p-value below 0.05. But a significant difference is not necessarily a large one. So also look at the size: is it two minutes or two days?

In summary

• A hypothesis test establishes whether a suspected cause really has influence, or whether it could be chance.
• You test following a fixed plan, and the choice of test depends on the type of data.
• A small p-value is evidence of a difference; a large p-value is no proof that there is none.

The Green Belt and Black Belt courses pay ample attention to hypotheses and testing them. We don’t just explain what it is, we teach you to do it. Interested? Email [email protected].