Accuracy : When is it enough
Accuracy : When is it enough
Accuracy is the simplest and most commonly used evaluation metric in Machine Learning. It answers one fundamental question: Out of all the predictions made by the model, how many were correct? Suppose an AI classifies 1,000 emails as either spam or not spam. If it correctly classifies 950 of them, its accuracy is 95%. In other words, the model was correct 95 times out of every 100 predictions. Accuracy is calculated by dividing the total number of correct predictions by the total number of predictions made. Because of its simplicity, accuracy is usually the first metric people look at when evaluating a model. But is accuracy always enough? The answer depends on the data. Accuracy works best when the dataset is balanced. A balanced dataset is one where each class has a similar number of examples. For example, imagine an AI that classifies images of cats and dogs. If the dataset contains 5,000 cat images and 5,000 dog images, the classes are evenly represented. This is called a balanced dataset. In situations like this, accuracy gives a good indication of how well the model is performing because every class contributes almost equally to the final score. Now consider a different problem. A bank is building an AI to detect fraudulent transactions. It has a dataset of 1,000,000 transactions. Out of these, 999,500 are genuine, while only 500 are fraudulent. One class is much larger than the other. This is called an imbalanced dataset. Now imagine the AI predicts that every transaction is genuine. It correctly identifies 999,500 transactions. Its accuracy is 99. 95%. That sounds exceptional. But the AI failed to detect every single fraudulent transaction. The model achieved an outstanding accuracy while completely failing at the very task it was designed to perform. This is the most important limitation of accuracy. Accuracy tells us how often the model was correct overall, but it does not tell us whether it performed well on every class. Whenever one class is much more common than another, or when different mistakes have different consequences, accuracy alone can be misleading. So when should you use accuracy? Use it when: The dataset is reasonably balanced. Every type of mistake has a similar cost. When these conditions are not met, we need additional evaluation metrics that look beyond overall correctness. In the next modules, we'll learn those metrics, beginning with Precision, which answers a different question: When the AI predicts "Yes," how often is it actually correct?