Skip to main content
Franco Spediacci
Associate
Jun 18, 2020 1:25 PM UTC
Solved

How can I to store variables into a csv log file?

I see file storage saves a json. I tried to put a csv conversion node in the middle but nothing is saved.

Thanks

This topic has been closed for replies.
stephane.legargeant
ST Employee
Aug 12, 2020 1:12 PM UTC

Hello

The .stcm may not be processed properly by the node-red-contrib-json2csv because it is a list of json blocks. So each block should be parsed separately.

Here is a proposal to convert the stcm files in csv. It will output on each line of the csv file the variable name, x and y.

0693W000003PAbFQAW.png

Here is the flow to import :

[{"id":"cc0c8ae8.c69f98","type":"tab","label":"Convert stcm","disabled":false,"info":"This flow is used to convert stcm file in csv file\n\nThe file name must be entered in the \"set file name\" node\n\nThe csv output format is variable Name, x, y\nNote : the data may be processed by group, so the x value may not be in time order. The csv file may be sorted on the x value to ensure order coherency. "},{"id":"a9caceb2.eb94d","type":"file in","z":"cc0c8ae8.c69f98","name":"Read file","filename":"","format":"lines","chunk":false,"sendError":false,"encoding":"none","x":280,"y":360,"wires":[["99d027a1.b92d98"]]},{"id":"9350e2b8.9aa3e","type":"inject","z":"cc0c8ae8.c69f98","name":"Start","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":160,"wires":[["3371f5e6.b3b50a"]]},{"id":"99d027a1.b92d98","type":"json","z":"cc0c8ae8.c69f98","name":"Parse each line","property":"payload","action":"","pretty":false,"x":480,"y":360,"wires":[["badf8efa.bacee"]]},{"id":"badf8efa.bacee","type":"function","z":"cc0c8ae8.c69f98","name":"Format data","func":"let output=\"\";\nmsg.payload.variabledata.forEach (data=>output=output+ msg.payload.variablename+ \",\"+ data.x + \",\"+ data.y+\"\\n\");\nmsg.payload= output;\nreturn msg;","outputs":1,"noerr":0,"x":690,"y":360,"wires":[["38fb106e.c00f"]]},{"id":"d2859af6.e93368","type":"file","z":"cc0c8ae8.c69f98","name":"Write csv","filename":"","appendNewline":false,"createDir":false,"overwriteFile":"false","encoding":"none","x":920,"y":580,"wires":[[]]},{"id":"3371f5e6.b3b50a","type":"change","z":"cc0c8ae8.c69f98","name":"Set file name","rules":[{"t":"set","p":"filename","pt":"msg","to":"C:\\GIT\\test\\test3.stcm","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":330,"y":160,"wires":[["a9caceb2.eb94d","593d0b11.8d3654"]]},{"id":"f3d3a57a.c56e48","type":"comment","z":"cc0c8ae8.c69f98","name":"Enter your filename here","info":"","x":330,"y":120,"wires":[]},{"id":"38fb106e.c00f","type":"change","z":"cc0c8ae8.c69f98","name":"Set csv extension in filename","rules":[{"t":"change","p":"filename","pt":"msg","from":"stcm","fromt":"str","to":"csv","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":700,"y":580,"wires":[["d2859af6.e93368"]]},{"id":"b1ba9fe.177b06","type":"file","z":"cc0c8ae8.c69f98","name":"Eraze csv","filename":"","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":780,"y":160,"wires":[[]]},{"id":"593d0b11.8d3654","type":"change","z":"cc0c8ae8.c69f98","name":"Format csv header ","rules":[{"t":"change","p":"filename","pt":"msg","from":"stcm","fromt":"str","to":"csv","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"Variable, x, y","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":160,"wires":[["b1ba9fe.177b06"]]},{"id":"796c6b18.2aab34","type":"comment","z":"cc0c8ae8.c69f98","name":"Process the stcm file","info":"","x":490,"y":320,"wires":[]},{"id":"c2ec88a.7368678","type":"comment","z":"cc0c8ae8.c69f98","name":"Write the csv file","info":"","x":820,"y":540,"wires":[]}]

To use it, go to menu import , copy paste the text, and select "new flow".

Then for each conversion :

1) in the box "Set file name", enter the name of file to convert (absolute path + extension : "C:\log\test1.stcm")

2) press start. The file will be processed and stored with the csv extension.

I hope it will answer your need.

Stephane

12 replies

Landry
Visitor II
Jun 22, 2020 9:37 AM UTC

Dear @Franco Spediacci​ ,

Can you share your flow to try to understand your issue ?

Landry

Franco Spediacci
Associate
Jun 23, 2020 7:14 AM UTC

Hi Landry,

thanks for reply.

You can find two flows attached and related logs.

In one I do not use conversion. In the log I see json.

In the other I use CSV conversion. In the log I see only the variables names.

Franco

david.rojas
Associate
Aug 5, 2020 1:41 PM UTC

I have the exact same issue, I can only see the variable names.

Richard.Chvr
ST Technical Moderator
Aug 6, 2020 8:20 AM UTC

Hello @Franco Spediacci​  and @david.rojas​ 

You can try json-2-csv node to convert json to csv

https://flows.nodered.org/node/node-red-contrib-json2csv

Hope it helps

Richard.

To give better visibility on the answered topics, please click on Accept as Solution on the reply which solved your issue or answered your question.
FDi L.228
Associate II
Aug 10, 2020 10:06 PM UTC

Same issue here, did anyone find a way to export data to csv?

stephane.legargeant
ST Employee
Aug 12, 2020 1:12 PM UTC

Hello

The .stcm may not be processed properly by the node-red-contrib-json2csv because it is a list of json blocks. So each block should be parsed separately.

Here is a proposal to convert the stcm files in csv. It will output on each line of the csv file the variable name, x and y.

0693W000003PAbFQAW.png

Here is the flow to import :

[{"id":"cc0c8ae8.c69f98","type":"tab","label":"Convert stcm","disabled":false,"info":"This flow is used to convert stcm file in csv file\n\nThe file name must be entered in the \"set file name\" node\n\nThe csv output format is variable Name, x, y\nNote : the data may be processed by group, so the x value may not be in time order. The csv file may be sorted on the x value to ensure order coherency. "},{"id":"a9caceb2.eb94d","type":"file in","z":"cc0c8ae8.c69f98","name":"Read file","filename":"","format":"lines","chunk":false,"sendError":false,"encoding":"none","x":280,"y":360,"wires":[["99d027a1.b92d98"]]},{"id":"9350e2b8.9aa3e","type":"inject","z":"cc0c8ae8.c69f98","name":"Start","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":160,"wires":[["3371f5e6.b3b50a"]]},{"id":"99d027a1.b92d98","type":"json","z":"cc0c8ae8.c69f98","name":"Parse each line","property":"payload","action":"","pretty":false,"x":480,"y":360,"wires":[["badf8efa.bacee"]]},{"id":"badf8efa.bacee","type":"function","z":"cc0c8ae8.c69f98","name":"Format data","func":"let output=\"\";\nmsg.payload.variabledata.forEach (data=>output=output+ msg.payload.variablename+ \",\"+ data.x + \",\"+ data.y+\"\\n\");\nmsg.payload= output;\nreturn msg;","outputs":1,"noerr":0,"x":690,"y":360,"wires":[["38fb106e.c00f"]]},{"id":"d2859af6.e93368","type":"file","z":"cc0c8ae8.c69f98","name":"Write csv","filename":"","appendNewline":false,"createDir":false,"overwriteFile":"false","encoding":"none","x":920,"y":580,"wires":[[]]},{"id":"3371f5e6.b3b50a","type":"change","z":"cc0c8ae8.c69f98","name":"Set file name","rules":[{"t":"set","p":"filename","pt":"msg","to":"C:\\GIT\\test\\test3.stcm","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":330,"y":160,"wires":[["a9caceb2.eb94d","593d0b11.8d3654"]]},{"id":"f3d3a57a.c56e48","type":"comment","z":"cc0c8ae8.c69f98","name":"Enter your filename here","info":"","x":330,"y":120,"wires":[]},{"id":"38fb106e.c00f","type":"change","z":"cc0c8ae8.c69f98","name":"Set csv extension in filename","rules":[{"t":"change","p":"filename","pt":"msg","from":"stcm","fromt":"str","to":"csv","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":700,"y":580,"wires":[["d2859af6.e93368"]]},{"id":"b1ba9fe.177b06","type":"file","z":"cc0c8ae8.c69f98","name":"Eraze csv","filename":"","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":780,"y":160,"wires":[[]]},{"id":"593d0b11.8d3654","type":"change","z":"cc0c8ae8.c69f98","name":"Format csv header ","rules":[{"t":"change","p":"filename","pt":"msg","from":"stcm","fromt":"str","to":"csv","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"Variable, x, y","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":160,"wires":[["b1ba9fe.177b06"]]},{"id":"796c6b18.2aab34","type":"comment","z":"cc0c8ae8.c69f98","name":"Process the stcm file","info":"","x":490,"y":320,"wires":[]},{"id":"c2ec88a.7368678","type":"comment","z":"cc0c8ae8.c69f98","name":"Write the csv file","info":"","x":820,"y":540,"wires":[]}]

To use it, go to menu import , copy paste the text, and select "new flow".

Then for each conversion :

1) in the box "Set file name", enter the name of file to convert (absolute path + extension : "C:\log\test1.stcm")

2) press start. The file will be processed and stored with the csv extension.

I hope it will answer your need.

Stephane

FDi L.228
Associate II
Aug 12, 2020 1:23 PM UTC

It works, thank you!

It still would be nice to have this function directly inside STM32CubeMonitor.

stephane.legargeant
ST Employee
Mar 25, 2021 12:56 PM UTC