As zalesky.arie, I am also wondering if it is "NOST_DATA - ST_DATA" or "ST_DATA - NOST_DATA". I tested the procedure on a sensor that seems ok, and the results for "MAG_NOST_DATA - MAG_ST_DATA" are positive for each axis (i.e. in contradiction to the mentioned formula if the sensor I tested is working).
Shouldnt it be the ST_DATA FIRST , minus the NOST_DATA??
for LSM9DS1 magnetometer selftest you better use AN4602 including the limits since LSM9DS1 contains LIS3MDL magnetometer.
Retrieving data ...