Confidence Behind Every Yes
Confidence Behind Every Yes
When the AI Says "Yes", Can You Trust It? Accuracy tells us how often a model is correct overall. But sometimes, that's not the question we really care about. Imagine an AI that detects spam emails. Every time it labels an email as Spam, a user might never see that email unless they manually check their spam folder. So an important question arises. Whenever the AI says an email is spam, how often is it actually spam? This question is answered by Precision. Precision measures the reliability of the model's positive predictions. In other words, it tells us how much we can trust the AI when it predicts a particular class. To understand Precision, we need two terms from the Confusion Matrix. A True Positive is a case where the AI predicts Yes, and the prediction is correct. A False Positive is a case where the AI predicts Yes, but the prediction is actually wrong. Precision considers only these two outcomes. It ignores every prediction where the model said No. If an AI predicts that 100 emails are spam, and 95 of them are actually spam, while 5 are genuine emails, the model has a Precision of 95%. This means that whenever the AI predicted spam, it was correct 95% of the time. High Precision means the model raises very few false alarms. This is especially important in applications where a false positive can have serious consequences. Consider an AI system that approves large financial transactions for fraud investigation. If it incorrectly flags thousands of genuine customers every day, investigators waste valuable time reviewing transactions that were never suspicious. Similarly, if an email service incorrectly labels important business emails as spam, users may miss critical communication. In both cases, we want the AI to make positive predictions only when it has strong evidence. That is exactly what Precision measures. So when should you use Precision? Precision is the preferred metric when the cost of a false positive is high—when incorrectly saying "Yes" creates unnecessary problems. Examples include spam detection, fraud investigations, legal document classification, and recommendation systems where users expect highly relevant results. Precision answers a simple but important question: When the AI says "Yes," can you trust its decision? In the next module, we'll study Recall, which asks a completely different question. Out of all the cases that actually deserved a "Yes," how many did the AI manage to find?