“Missile approaching, Every one run for cover”. People sending final goodbye messages to their loved ones. Family members huddling and praying together. Mother clinging to children, while the father hurriedly makes for a safe getaway for the family to the basement. Imagine the state of confusion and chaos every Hawaiian would have faced on Jan 13, 2018 when they received an approaching ballistic missile alert.
Talk about trouble in paradise !!
Well given the current twitter threats by ‘Rocket Man’ as President Trump calls the North Korean dictator, many Hawaiian’s had all the logical reasons to believe the alert. Thankfully ‘Rocket Man’ had not pressed the tiny nuclear button probably because Trump had a bigger button !!
It all thankfully turned out to be a ‘False Alarm’. Apart from minor car accidents ,unfortunate heart attack suffered by a person and not to mention the drastic drop in pornhub views and sudden surge in pornhub views (“Stronger than ever before, EVER” in Trump speak), there was not much of a damage.
What if the missile alert was real ? The mere thought of it is scary enough.
During the world war 2, the British and allies were going through the same confusion and fear everyday. Something needed to be done.
British developed the ROC curve as part of the chain home project. The goal was to discern which was an incoming enemy plane and which was a seagull or Geese.
The Receiver Operating Curve (ROC) is a graphical plot that depicts the diagnostic ability of a binary classification system with varying threshold. It is a plot of True Positive Rate against False Positive Rate.
I know the terms False Positive Rate and True Positive rate might be confusing right now for some people, but the reason I bought the ROC into picture is because it will help set the narrative later on in the article for some interesting debunking about Missile Alert UI design.
What Causes the Confusion in Confusion Matrix ?
In my opinion the confusion gets created the moment we place words like predicted class on the left and actual class on top as illustrated below.
Our human brain gets into ‘match the following’ mode and gets confused due to the sheer cognitive burden.
How To Remove The Confusion ?
I must confess that I too used to get confused between False Positive and False Negative while building and interpreting various classification models, until i developed my own technique to remember it for life time. It works for me and perhaps it might for you too.
It is paramount to get the TP, TN, FP, FN right as they prove crucial in calculating TPR (a.k.a sensitivity), TNR (a.k.a Specificity) and many more. I will probably write an article with real examples on how to calculate these ratios without getting confused.
But first things first, steps to remove the confusion.
Step 1 : Keep the word “Predicted” in mind always
Step 2 : Read from right to left (Arabs and Persians might find this easier)
Step 3: If the word is FP, you should read it as Predicted Positive but False
Similarly, say the word is FN, you would read it as Predicted Negative but False
I don’t want to delve into TP (True Positives) and TN (True Negatives) as they are self explanatory.
Was the Hawaii Missile Alert System Designed Wrongly ?
Many articles have been written on how bad the UI was and how easily anyone could have made the mistake. The options were in the form of a drop down
- “Test Missile Alert”
- “Missile Alert”
Was the system badly designed ?
My answer with a Data Scientist Hat on, would be NO.
Here’s why :
The UI was designed to be a battle of True Positives (TP) vs False Positives (FP)
Remember the ROC Curve I had talked about earlier ? In a life or death situations, you would want to be alert all the time. In a battle between True Positive and False Positive, the winner is always YOU. Think about it. What is the worst that can happen.
Only two possibilities:
- True Positive : A Real missile is approaching and you take shelter
- False Positive : No missile approaching, yet the system says it is, and you take shelter
Either case you take shelter. You are safe!!
What if the battle was between False Negative (FN) vs True Negative (TN)
Well this is one battle were you want to root for True Negative always. why ?
Again two possibilities:
- False Negative : A Real Missile is approaching, yet system says not a speck in the sky. You don’t take shelter and………..(eternal silence)
- True Negative : No missile is approaching and No alerts. You are safe!!
So whosoever designed the UI did a good job. It will save more lives than otherwise.
Bottom line : Life or Death situation alert systems should always be designed with True Positives (TP)and False Positives (FP) options alone.
