30% Label Noise Costs 8.5 Points of Accuracy

30% label noise = 8.5 percentage points of accuracy gone. Not from a bad model, the wrong architecture, or insufficient compute. From dirty training data.

Everyone obsesses over the backbone: ResNet vs. EfficientNet, CNNs vs. Vision Transformers, fine-tuning vs. zero-shot with CLIP. These are real decisions with real trade-offs. But your backbone does not set the performance ceiling. The quality of the labels your model learned from does.

The Three Problems That Do the Most Damage

Class ambiguity. The boundary between categories is not crisp enough in your guidelines, so different annotators draw the line in different places. Your model learns the disagreement, not the distinction.

Class imbalance. Rare but important categories are so underrepresented that your model never learns them properly. It looks fine in aggregate metrics and falls apart on the cases that matter in production.

Annotator inconsistency. The same image gets different labels from different reviewers, with no reconciliation process in place. For video data, this compounds across frames in ways that are painful to diagnose.

Why You Don’t See It Coming

None of these problems announces itself. Training runs fine. Offline metrics look reasonable. Then production happens: confidence scores are miscalibrated, the model is brittle on edge cases, and nobody can explain why it worked in testing but not on real data.

What Actually Moves the Needle

• Annotation guidelines that define class boundaries with concrete examples, not just category names
• Inter-annotator agreement tracking throughout the project, not just at final delivery
• Gold-standard QA checks that catch systematic errors before they contaminate the training set
• Human-in-the-loop review on model-assisted pre-labeling