This article describes embedded programmable features built-in in some ST MEMS motion sensors, specifically Finite State Machine (FSM), Machine Learning Core (MLC), and Intelligent Sensor Processing Unit (ISPU).
1. Introduction
ST MEMS motion sensors have two types of embedded features. Both features mainly focus on running data processing in the sensor instead of the MCU. This results in a lower power consumption of the system. Embedded configurable features (for example, wake-up, free-fall, 4D/6D orientation) are present in the majority of our motion sensors. It is possible to adjust them mainly using threshold and duration. However, these features are only configurable, as the name suggests, and it is not possible to program a custom data processing. Recent ST MEMS motion sensors embed programmable features like Finite State Machine (FSM), machine learning core (MLC) or intelligent sensor processing unit (ISPU) to incorporate custom data processing into the sensor. Knowing this, we have 3 main approaches to run a custom embedded data processing based on sensor data described in the list and picture below:
- Standard approach - MCU reads raw data provided by the sensor and desired processing is done inside the MCU,
- Programmable logic approach - targets the lowest power consumption on a specific processing model. MCU reads the model results and possibly raw data when an interrupt is triggered,
- Fully programmable DSP approach - any custom code (considering the computational and compiler limits) can be run inside the sensor and MCU reads processed data out of the sensor.
Generally, using programmable logic or fully programmable DSP approach brings the following advantages for the application:
- The power consumption of the embedded feature is distinctly lower than running the same feature inside the MCU
- Data reads from the sensor can be limited to events when interrupt is triggered -> MCU can be in sleep mode the rest of the time
- Traffic on I²C/SPI bus between sensor and MCU is reduced
- Offloading the MCU - either processing time can be used for different purpose or a less expensive MCU can be used
In this article, we briefly introduce embedded programmable features: FSM, MLC (both programmable logic approach), and ISPU (fully programmable DSP approach) to help you choose the most suitable feature for your application.
2. Finite State Machine (FSM)
A Finite State Machine is a behavioral model composed of finite number of states with specific transitions between states, which are similar to a flow chart able to process both internal and external data (if sensor hub is available). There are two types of states - Reset/Next Condition (RNC) and command (CMD). The RNC type of state is composed of two conditions, one to reset the state machine and one to continue the program flow. The detailed description of the RNC state is shown in the picture below. The first condition evaluated is reset. If the result is true, the state machine is reset, otherwise next condition is evaluated. The flow continues to next state if the next condition was satisfied, otherwise the flow is paused until next sample arrives and the evaluation (both reset and next condition) is performed with a new sample. The examples of conditions for Reset/Next state types are: no operation, timers, comparison to thresholds, etc. The second type of state (CMD) executes a command to modify the program behavior in terms of flow control and output. The examples of commands are: stop execution, reset pointer set/reset, etc. An interrupt of FSM is generated when the end state is reached or when some specific command is performed. It is possible to run multiple state machines in parallel. In the picture below a simple wake-up example based on accelerometer data is shown.
The Finite State Machine approach is most suitable for applications, where a user-defined gesture pattern needs to be recognized.
The latest ST sensors with FSM include an adaptive self-configuration (ASC). This means that the FSM interrupt can be used to trigger device setting changes (incl. ODR, FS, BW, power mode and FIFO), thus the MCU can be kept in sleep mode.
Find more information about Finite State Machine feature in ST's GitHub repository. This repository includes application and configuration examples. Application examples are available for various sensors and are ready to be uploaded into sensor and evaluated. The configuration examples are focused on providing information to configure and evaluate an own Finite State Machine using different tools provided by ST. The main software is Unico-GUI providing a dedicated tool to develop an FSM. For evaluation of created FSM configuration, it is possible to use either Unico-GUI or AlgoBuilder with Unicleo-GUI depending on evaluation hardware used.
Supported sensors
- Consumer: LSM6DSO, LSM6DSOX, LSM6DSO32, LSM6DSO32X, LSM6DSR, LSM6DSRX, LSM6DSV, LSM6DSV16X, LSM6DSV16BX, LIS2DUX12, LIS2DUXS12
- Industrial: ISM330DHCX, IIS2ICLX
- Automotive: ASM330LHHX, ASM330LHHXG1
3. Machine learning core (MLC)
The machine learning core is composed of a set of configurable parameters (also called features) and decision trees. A decision tree is represented as a binary tree composed from two types of nodes - inner nodes and leaves. Inner node of a decision tree is a node, which has child nodes. Since the decision tree inside the MLC is a binary tree, the number of children is two. Inner nodes represent a part of decision tree, where a statistical parameter automatically calculated from raw sensor data is compared to a threshold (if-then-else) in order to choose next path, which can be either a true or false path. A leaf node of decision tree is a node, which does not have any children nodes and contains one of user defined classes - result. A brief description of a decision tree with an Activity/Inactivity example using only accelerometer data can be seen in picture below.
The statistical parameters, which are compared against a threshold within inner nodes of a decision tree, are parameters selected by the user (for example, mean, variance, etc.) in defined time window from available sensor data - both internal and external data (if sensor hub is available).
The parameters selected by user are evaluated during decision tree training. After the training, only statistically relevant parameters are included in the final decision tree.
To use machine learning core capability, it is necessary to use supervised learning approach. This includes following steps described below:
- Define the classes to be recognized, at the same time covering all possible options
- Log sufficient amount of training data for each class
- Choose different parameters for each class (this can be an iterative process as it is not guaranteed to find the most suitable parameters during the first iteration)
- Create the decision tree using relevant software tool
- Transform trained decision tree into configuration to be loaded into sensor
ST's MEMS evaluation software package Unico-GUI includes an MLC tool, which guides the user through the whole process of creating a decision tree - from labeling the data logs up to creating a configuration file for specific sensor.
Suitable applications are the ones, which can be implemented by following an inductive approach, which involves searching patterns from observations. This includes application as: activity recognition, fitness activity recognition, motion intensity detection, vibration intensity detection, carrying position recognition, context awareness, etc.
More information about MLC enabled sensors can be found at st.com/MLC, or ST's MLC GitHub repository. The MLC repository includes application examples ready to be evaluated for various sensors and configuration examples with different ST's hardware and software tools to guide you through a process of making a decision tree. The main software is Unico-GUI providing a dedicated tool for MLC development. For evaluation of created MLC configuration it is possible to use either Unico-GUI or AlgoBuilder with Unicleo-GUI depending on evaluation hardware used.
Supported sensors
- Consumer: LSM6DSOX, LSM6DSO32X, LSM6DSRX, LSM6DSV16X, LSM6DSV16BX, LIS2DUX12, LIS2DUXS12
- Industrial: ISM330DHCX, IIS2ICLX
- Automotive: ASM330LHHX, ASM330LHHXG1
4. Intelligent sensor processing unit (ISPU)
The ISPU is a compact, ultra low-power and high-performance programmable core based on the STRED architecture, a proprietary architecture developed by STMicroelectronics. This core allows processing both internal (accelerometer, gyroscope and temperature) and external (connected through sensor hub to the sensor) data in up to 30 separate, user defined algorithms. The algorithms are executed each time a new set of data is sampled starting from algorithm 29 down to algorithm 0. The ISPU is supported by two separate RAMs - 32 KB for code and 8 KB for variable data and the clock can be set to run either at 5 or 10 MHz.
The picture below contains a scheme describing the processing of data and connection of sensor to microcontroller and external sensors.
The ISPU can run C language algorithms that can be compiled by ISPU toolchain, or anomaly detection library can be generated using NanoEdge AI Studio (offered for ISM330ISN). The X-CUBE-ISPU software package includes examples (including calibration algorithms, sensor fusion, wrist solutions, etc.) ready to be evaluated, a blank template project to be used for a new custom embedded feature, integration examples of anomaly detection projects and data log firmware to be used with Unicleo. X-CUBE-ISPU package is available at st.com or ST's GitHub page. For evaluation of ISPU configuration compiled by ISPU toolchain it is possible to use AlgoBuilder with Unicleo-GUI for selected evaluation hardware.
More information about ISPU sensor family can be found at st.com/ISPU.
Supported sensors
- Consumer: LSM6DSO16IS
- Industrial: ISM330IS, ISM330ISN
- Automotive: -
In this article we described embedded programmable features built-in some of ST MEMS motion sensors and its benefits for final application. STMicroelectronics is the only supplier on the market offering all discussed features allowing the customer to embed custom data processing inside the sensor. Thanks to the pinout compatibility it is very easy for customers currently using standard approach to upgrade the solution and take advantage of the benefits, that the embedded programmable features are able to offer. If you would like to know more and discover our full MEMS sensors portfolio, please visit st.com/mems.