The instinct when a deep-learning model underperforms is to collect more data. More data means better coverage. More data means better performance. Right?
Not necessarily. Teams that ship reliable models don't just collect more — they collect smarter. Here's the difference.
The problem with indiscriminate data collection:
When you collect data without targeting specific gaps, you typically: - Replicate the distribution you already have (adding redundant samples) - Miss the rare but critical scenarios that are causing failures - Increase labelling costs without proportional improvement - Make your model harder to debug (more data, same blindspots)
Hexagon reduced their dataset size by 40% while maintaining model performance. Not by randomly removing data — by identifying and removing redundant samples while ensuring critical edge cases were covered.
The targeted approach to dataset curation:
1. Map your current coverage
Before collecting anything, understand what your current dataset contains. Where are the dense clusters (redundant samples)? Where are the sparse regions (underrepresented scenarios)?
2. Identify model-relevant gaps
Not every sparse region matters equally. The gaps that matter are the ones where your model is actually failing. Cross-reference your dataset coverage with your model's failure patterns.
3. Prioritise for maximum impact
With limited labelling budget, focus on the samples that will close your most impactful performance gaps. A few hundred targeted samples can outperform thousands of randomly collected ones.
4. Prune redundancy
Removing redundant samples speeds up training, reduces costs, and often improves generalisation. Not every sample in your dataset is earning its keep.
The result is a smaller, higher-quality dataset that trains faster and performs better.