Integrating Image Classification Model with X-CUBE-AI on STM32 (Input from SD Card)

[EmreTuncer]

Hi everyone,

I am working on implementing an image classification model on an STM32 board using the X-CUBE-AI extension. I am relatively new to embedded software development; my background is primarily in AI and deep learning. While I have managed to generate the necessary .c and .h files for my model using the X-CUBE-AI extension, I am having trouble providing input to the model from an image stored on an SD card. As far as understand from created cods I have modify acquire_and_process_data and post_process functions that have write down below. Also I have provide output of my code. In main function just  MX_X_CUBE_AI_Process function works in while for AI proces.

Here’s a brief summary of what I’ve done so far:

1. Model Conversion: Successfully converted my image classification model using the X-CUBE-AI tool and generated the necessary files.
2. SD Card Integration: Implemented functions to read BMP image files from the SD card and store them in a memory buffer. These functions are working correctly, and I can read the image data into an array.
3. Issue: When I feed the image data to the model, the output is always predicting the first class, regardless of the input image.

Questions

1. Input Preparation: How should I preprocess the BMP image data to ensure it is compatible with the model input?
2. Data Feeding: What is the correct way to load the image data into the input tensor for X-CUBE-AI models?
3. Debugging Tips: Are there any specific debugging steps or tools I can use to ensure the input is being fed correctly to the model?
4. Example Code: If anyone has successfully implemented a similar use case (loading images from SD card for inference), could you share a sample implementation?

The Codes:

int acquire_and_process_data(ai_i8* data[])
{
/* fill the inputs of the c-model
for (int idx=0; idx < AI_NETWORK_IN_NUM; idx++ )
{
data[idx] = ....
}
*/

uint32_t width, height;
uint8_t *pixelData = NULL;
uint8_t imageBuffer[64 * 64 * 3]; // Example with 64x64 resolution in RGB format

// Read the BMP file
FRESULT result = ReadBMPFile("image.bmp", imageBuffer, sizeof(imageBuffer));

if (result != FR_OK) {
HAL_UART_Transmit(&huart1, (uint8_t*)"Error reading BMP file.\n", 24, HAL_MAX_DELAY);
return -1;
}

// Allocate memory for data[0] with size 64x64x3
data[0] = (ai_i8*)malloc(64 * 64 * 3 * sizeof(ai_i8));

if (data[0] == NULL) {
HAL_UART_Transmit(&huart1, (uint8_t*)"Memory allocation failed.\n", 25, HAL_MAX_DELAY);
return -1;
}

uint16_t x = 0;
for (int i = 0; i < 64 * 64; i++) {
uint8_t *bfr = &imageBuffer[i * 3]; // Each pixel has 3 bytes (R, G, B)

// Copy the R, G, B components sequentially to the `data[0]` array
for (int j = 0; j < 3; j++) {
data[0][x++] = bfr[j]; // Transfer the data correctly
}
}

return 0;
}

int post_process(ai_i8* data[])
{
/* process the predictions
for (int idx=0; idx < AI_NETWORK_OUT_NUM; idx++ )
{
data[idx] = ....
}
*/

if (data[0] == NULL) {
HAL_UART_Transmit(&huart1, (uint8_t*)"Data[0] is NULL\r\n", 18, HAL_MAX_DELAY);
return -1; // Error condition
}

// Define class names
const char* class_names[] = {"Pizza", "Steak", "Sushi"};
const int num_classes = sizeof(class_names) / sizeof(class_names[0]);

// Process the results from the AI network
float* predictions = (float*)data[0]; // Cast the output to float

// 1. Find the maximum value and its index
float max_value = predictions[0];
int max_index = 0;
for (int i = 1; i < num_classes; i++) {
if (predictions[i] > max_value) {
max_value = predictions[i];
max_index = i;
}
}

// 2. Send the predicted class via UART
char uart_message[50]; // Buffer to hold the message
snprintf(uart_message, sizeof(uart_message), "Predicted class: %s\r\n", class_names[max_index]);
HAL_UART_Transmit(&huart1, (uint8_t*)uart_message, strlen(uart_message), HAL_MAX_DELAY);

// 3. Send all elements of the prediction array via UART
HAL_UART_Transmit(&huart1, (uint8_t*)"Predictions:\r\n", 13, HAL_MAX_DELAY);
for (int i = 0; i < num_classes; i++) {
char prediction_message[50];
snprintf(prediction_message, sizeof(prediction_message), "Prediction[%d]: %.2f\r\n", i, predictions[i]);
HAL_UART_Transmit(&huart1, (uint8_t*)prediction_message, strlen(prediction_message), HAL_MAX_DELAY);
}

return 0; // Operation complete
}

OUTPUT

FATFS mounted successfully. // from sd card test function
Opened BMP file: image.bmp // from sd card image read function
Read 12288 bytes from BMP file: image.bmp // from sd card image read function
File closed successfully.
Predicted class: Pizza //which is wrong
Prediction[0]: 4.32
Prediction[1]: -3.78
Prediction[2]: -0.45