Reading .dat file

[TomRett]

I am using the STEVAL-STWINKT1B evaluation board to gather microphone data with a 192kHz sampling rate. The data is saved in a .dat file which i want to read and process. However, when reading the IMP23ABSU_MIC.dat file i get completely different results.

Using a freeware hex editor i opened the .dat file and this is how it looks:

As the samples are saved in a 16-bit integer (according to the mic's datasheet), which corresponds to 2 bytes, im expecting to read 0x31FF. This is valid since the HSDatalog software also presents the same value in int16 (e.g. -207). This is what i have tried so far in STM32CubeIDE:

DIR dir;
	FILINFO fno;
	FATFS uSDFatFs;
	FIL MyFile;
	UINT br;
	FRESULT res;
	char uSDPath[3] = "/";
	static int16_t data_storage[48000] = { };

	/* Working with ...\raw_data\192kHz\1bar (for evaluation)\phi_1_l_20*/
	if (BSP_SD_Init() == MSD_OK) {
		if (f_mount(&uSDFatFs, (TCHAR const*) uSDPath, 0) == FR_OK) {
			res = f_opendir(&dir, uSDPath);
			if (res == FR_OK) {
				for (;;) {
					res = f_readdir(&dir, &fno); /* Read a directory item */
					if (res != FR_OK || fno.fname[0] == 0)
						break; /* Error or end of dir */
					if (fno.fattrib & AM_DIR) { /* Directory */
						printf("   <DIR>   %s\n", fno.fname);
					} else { /* File */
						printf("%10u %s\n", fno.fsize, fno.fname);
					}
				}
				res = f_open(&MyFile, "STWIN_00001/IMP23ABSU_MIC.DAT", FA_READ);
				if (res == FR_OK) {
					res = f_read(&MyFile, data_storage, sizeof(data_storage),
							&br);

					static int16_t read_buff[100];
//					f_gets(&read_buff, 100, &MyFile);
//					unsigned char c = fgetc(&MyFile);
//					int16_t c = fgetc(&MyFile);
					char_t c = fgetc(&MyFile);
					if (res == FR_OK) {
						// Successfully read into data_storage
						printf("Read %u bytes from the file.\n", br);
					} else {
						// Handle read error
					}
					f_close(&MyFile);
				}
			}
		}
	}

Both methods (using fgetc or f_read) for reading the file content provide different and wrong results
as

How can i read the content of the .dat file using Little Endian on this evaluation board with C?

[TomRett]

As the DeviceConfig.json points out, the data file format used is the HSD_1.0.0. My question therefore is, how is the data saved and how can it be read using C and not HSDatalog?

[SimonePradolini · Accepted solution]

Hello @TomRett 

I guess you used FP-SNS-DATALOG1 to log data. If so, be aware that .dat interleaves data and timestamps as described in the User Manual (DATALOG1 chapter 2.6.5).

By exploiting the DeviceConfig.json you can interpret the .dat file and export data in the format you need.

Moreover, you can use the provided Python SDK that already contains ready-to-use scripts for process and export .dat files.

Best regards

Simone