How exactly are datasets larger than 250 MB reduced during benchmarking? What sampling strategy is used?
Hello ST Community,
I noticed that when using the benchmarking tool, datasets larger than 250 MB are
automatically reduced using a "representative sampling strategy" that preserves
the diversity and distribution of the original data.
I would like to understand more details about this process:
1. What specific sampling algorithm/strategy is used? (e.g., random sampling,
stratified sampling, systematic sampling, etc.)
2. How is the target size of 250 MB determined? Is this configurable?
3. How does the tool ensure that the sampled dataset preserves:
- The original data distribution?
- The diversity of data samples?
- Edge cases and boundary values?
4. After optimization, when we validate the model on the full dataset, what
performance differences should we typically expect between the sampled
dataset and the full dataset?
5. Is there any documentation that describes this sampling mechanism in detail?
Any information or pointers to relevant documentation would be greatly appreciated.
Thank you!
