How to significantly reduce benchmark time in NanoEdge AI Studio for large datasets? (Looking for practical solutions)

[chengchang]

Hi everyone,

I'm deploying a model on the NUCLEO‑H563ZI using NanoEdge AI Studio (v5.2.0). My full training dataset is about 2 GB, and I cannot downsample or reduce features – I must use the complete dataset.

My PC specs:

- CPU: 12th Gen Intel i7‑12700 @ 2.10 GHz (16 cores total)

- RAM: 32 GB (31.7 GB usable)

- Storage: 1.82 TB HDD (mechanical drive, WDC WD20EZBX‑00AYRA0)

- OS: Windows 64‑bit

- Cores allocated to benchmark: 16 (all available)

In a previous test with only 40 MB of data, the benchmark took 40 hours on this same machine. If time scaled linearly, 2 GB would take ~2000 hours – impossible for our schedule.

Since I cannot change the dataset itself, I'm looking for practical ways to reduce benchmark time through external measures. Specifically, I would like to know:

1. Storage bottleneck – My dataset resides on an HDD. Would moving the entire NEAI workspace and dataset to a fast NVMe SSD (or even a RAM disk) dramatically cut the time? Does NEAI repeatedly read from disk during benchmarking, or does it load data into memory once and cache it? If it re‑reads often, HDD could be the main culprit.

2. NEAI internal settings – Are there any configurable parameters (e.g., number of iterations, early stopping, or limiting the model search space to faster families) that can speed up benchmarking, perhaps with a slight trade‑off in accuracy? I'm not asking to manually select models, but to restrict the search scope if possible.

3. GPU acceleration – Does NanoEdge AI Studio support GPU for benchmarking, or is it strictly CPU‑bound? If GPU is supported, which configurations are recommended?

4. Cloud or server migration – Would running the benchmark on a cloud instance with SSD storage and higher‑end CPU (e.g., AWS EC2 or Azure) be a feasible workaround? If so, are there any known best practices or recommended instance types for NEAI?

5. Other system optimizations – For example, disabling Windows power saving, setting high process priority, or increasing virtual memory – have these helped anyone?

I'd greatly appreciate any concrete advice from those who have dealt with large‑scale benchmarking in NEAI. Thank you!

[Julian E.]

Hi ​@chengchang,

 

You don’t need to perform a complete benchmark everytime.

The idea of the benchmark is to look for the most optimized library, but I would suggest proceeding as follow:

• Load your dataset
• Launch a benchmark for 10-30 minutes
• Look at the results

If you get less than 70%, you will probably not get much better results even if you do a complete benchmark => work on your data (change the datarate, the length, the position of the sensor, the sensor type etc)

If you get satisfying performances, you can pause the benchmark and do some validation on test datasets to make sure it works. If you are satisfied, you can deploy these libraries or proceed with a full benchmark.
At the end of the benchmark, make sure to also validate the new libraries, to avoid overfitting.

 

If you cannot reach good metrics, you may be interested in bigger model's solutions, such as deep learning.

For that, I would recommend to take a look at the ST Edge AI Core and the ST Model Zoo services.

 

In any case, the new version of Nanoedge now filter your complete dataset to keep the most valuable signals, so it should be better. But what I said earlier is still applicable.

 

Have a good day,

Julian