P – Value
In this blog we will discuss the important functionality of p – value in statistical experiments. Why p – value is the deciding factor for accepting or rejecting a hypothesis we develop before any experiment.
Problem Statement:
You have launched a product (e.g. a phone) in the market. And you get customer feedback that the phone has over heating problem. As the phone is already launched in the market you can’t recall all of them to test if the majority of the phones have overheating problem due to some manufacturing problem.
Hence to address the issue you have decided to take surveys regarding the phone and do a statistical test to overrule your apprehension regarding the manufacturing issue. You ask all the employees of your company to share their feedback regarding the phone overheating problem. You also took an online survey from the customer regarding the same. Now you have a random sample of 500 feedbacks against the total number of 250000phones you have sold.
Population size = 25000, Sample size = 500
However, before the release of the product to the market you already have conducted a test and found that at maximum 3% of the phone may have the overheating problem which is due to some random event and it does not relate to any manufacturing issue. It may have occurred due to overcharging or overusing. This is acceptable to your company. Otherwise you have to recall all the phones from market to do a re-evaluation.
Now you have to take a decision whether you will recall the phone from the market or not.
Hypothesis:
The important step of your statistical experiment journey is to set up null hypothesis and alternate hypothesis first.
Null hypothesis (H0):= Overheating of phones are as expected and due the some random events which was observed during the production process.
Alternate hypothesis (H1):= Overheating of the phones are not due to some random events. There must be some strong reason behind the overheating.
If p value is large you accept null hypothesis.
If p value is small you fail to accept null hypothesis. You believe that the alternate hypothesis is somewhat acceptable. Your test is statistically significant.
Data:
We will demonstrate the test with two data scenarios.
Scenario 1:
Scenario 2:
The sample size n = 500.
m = 2. (Number of categorical values (Here they are Overheating & Non-overheating OR Yes & No))
Experiments and Results:
Before we proceed with the experiment, let’s set the confidence interval first and know about the types of error.
H1 Error: – We reject null hypothesis even though it is true. (In our example, even after observing that the overheating of phones happen due to random events we still reject null hypothesis and assume that the overheating happens due to some manufacturing issue.)
H2 Error: – We retain null hypothesis even though it is false. (In our example, even after observing that the overheating of phones happen due to some manufacturing issue we still accept null hypothesis and assume that the overheating happens due to random events.)
CI: Confidence interval for our test will be 95%. This means we are 95% confident that the test results of our sample will fall 95% close to the population.
α (Significance level) is the probability of H1 error. Here α = 0.05.
We will use Chi Square (X2) test to perform the experiments for below two scenarios.
Scenario 1:-
From the experiment we saw the Chi Square (X2) value is 0.3436 and p – value is 0.56 (calculated).
p – value is greater than the α (=0.05).
We can also search the critical value of X2 from the table provided below and find that the next critical value after X2=0.3436 is 2.706 and its corresponding p-value is 0.1. And our X2 value lies between X20.10 and X20.90. This means our p-value (though we have already calculated) calculated above is between 0.1 to 0.9 and it is not smaller than 0.05.
(Figure represents the critical value of Chi Square (X2).)
We fail to prove any evidence against null hypothesis. We can’t reject null hypothesis. This means the number of overheating phones we found from the survey is not significantly different than what we observed during our production process.
Scenario 2:
From the experiment we saw the Chi Square (X2) value is 16.8384 and p –value is 4.10E-05 (calculated). P value is smaller than the α.
We can also search the critical value of X2 from the table provided below and find that the next critical value after X2=16.83 fall beyond (right) X2.01 and its corresponding p-value is less than 0.01. This means our p-value (though we have already calculated) should be less than 0.01 and it is obviously less than 0.05 s well.
(Figure represents the critical value of Chi Square (X2).)
We have to reject null hypothesis. This means the number of overheating phones we found from the survey is significantly different than what we observed during our production process. And it is not due to some random events.
Outcome of the Problem Statement:
For the 1st scenario we will accept the null hypothesis and our phones don’t have any manufacturing problem.
For the 2nd scenario we have to check the phones for their manufacturing problem as we have strong evidence against the null hypothesis.
Conclusion:
Alternate hypothesis can’t be true. You can only fail to accept null hypothesis. That means you have weak evidence that null hypothesis is true. Hence your experiment (Scenario 2) suggests there might be some reason that your phones are overheating and needs to be addressed and they are not overheating due to some random events. P-value along with the confidence level plays a major role in hypothesis testing apart from the critical values of the test.
Dear readers, I will be pleased to receive your comments/suggestions on this post. Please feel free to post.
Thank you.
