How can I to store variables into a csv log file?

[Franco Spediacci]

I see file storage saves a json. I tried to put a csv conversion node in the middle but nothing is saved.

Thanks

[Landry]

Dear @Franco Spediacci​ ,

Can you share your flow to try to understand your issue ?

Landry

[Franco Spediacci]

Hi Landry,

thanks for reply.

You can find two flows attached and related logs.

In one I do not use conversion. In the log I see json.

In the other I use CSV conversion. In the log I see only the variables names.

Franco

[david.rojas]

I have the exact same issue, I can only see the variable names.

[Richard.Chvr]

Hello @Franco Spediacci​  and @david.rojas​ 

You can try json-2-csv node to convert json to csv

https://flows.nodered.org/node/node-red-contrib-json2csv

Hope it helps

Richard.

[FDi L.228]

Same issue here, did anyone find a way to export data to csv?

[stephane.legargeant · Accepted solution]

Hello

The .stcm may not be processed properly by the node-red-contrib-json2csv because it is a list of json blocks. So each block should be parsed separately.

Here is a proposal to convert the stcm files in csv. It will output on each line of the csv file the variable name, x and y.

Here is the flow to import :

[{"id":"cc0c8ae8.c69f98","type":"tab","label":"Convert stcm","disabled":false,"info":"This flow is used to convert stcm file in csv file\n\nThe file name must be entered in the \"set file name\" node\n\nThe csv output format is variable Name, x, y\nNote : the data may be processed by group, so the x value may not be in time order. The csv file may be sorted on the x value to ensure order coherency. "},{"id":"a9caceb2.eb94d","type":"file in","z":"cc0c8ae8.c69f98","name":"Read file","filename":"","format":"lines","chunk":false,"sendError":false,"encoding":"none","x":280,"y":360,"wires":[["99d027a1.b92d98"]]},{"id":"9350e2b8.9aa3e","type":"inject","z":"cc0c8ae8.c69f98","name":"Start","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":160,"wires":[["3371f5e6.b3b50a"]]},{"id":"99d027a1.b92d98","type":"json","z":"cc0c8ae8.c69f98","name":"Parse each line","property":"payload","action":"","pretty":false,"x":480,"y":360,"wires":[["badf8efa.bacee"]]},{"id":"badf8efa.bacee","type":"function","z":"cc0c8ae8.c69f98","name":"Format data","func":"let output=\"\";\nmsg.payload.variabledata.forEach (data=>output=output+ msg.payload.variablename+ \",\"+ data.x + \",\"+ data.y+\"\\n\");\nmsg.payload= output;\nreturn msg;","outputs":1,"noerr":0,"x":690,"y":360,"wires":[["38fb106e.c00f"]]},{"id":"d2859af6.e93368","type":"file","z":"cc0c8ae8.c69f98","name":"Write csv","filename":"","appendNewline":false,"createDir":false,"overwriteFile":"false","encoding":"none","x":920,"y":580,"wires":[[]]},{"id":"3371f5e6.b3b50a","type":"change","z":"cc0c8ae8.c69f98","name":"Set file name","rules":[{"t":"set","p":"filename","pt":"msg","to":"C:\\GIT\\test\\test3.stcm","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":330,"y":160,"wires":[["a9caceb2.eb94d","593d0b11.8d3654"]]},{"id":"f3d3a57a.c56e48","type":"comment","z":"cc0c8ae8.c69f98","name":"Enter your filename here","info":"","x":330,"y":120,"wires":[]},{"id":"38fb106e.c00f","type":"change","z":"cc0c8ae8.c69f98","name":"Set csv extension in filename","rules":[{"t":"change","p":"filename","pt":"msg","from":"stcm","fromt":"str","to":"csv","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":700,"y":580,"wires":[["d2859af6.e93368"]]},{"id":"b1ba9fe.177b06","type":"file","z":"cc0c8ae8.c69f98","name":"Eraze csv","filename":"","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":780,"y":160,"wires":[[]]},{"id":"593d0b11.8d3654","type":"change","z":"cc0c8ae8.c69f98","name":"Format csv header ","rules":[{"t":"change","p":"filename","pt":"msg","from":"stcm","fromt":"str","to":"csv","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"Variable, x, y","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":160,"wires":[["b1ba9fe.177b06"]]},{"id":"796c6b18.2aab34","type":"comment","z":"cc0c8ae8.c69f98","name":"Process the stcm file","info":"","x":490,"y":320,"wires":[]},{"id":"c2ec88a.7368678","type":"comment","z":"cc0c8ae8.c69f98","name":"Write the csv file","info":"","x":820,"y":540,"wires":[]}]

To use it, go to menu import , copy paste the text, and select "new flow".

Then for each conversion :

1) in the box "Set file name", enter the name of file to convert (absolute path + extension : "C:\log\test1.stcm")

2) press start. The file will be processed and stored with the csv extension.

I hope it will answer your need.

Stephane

[FDi L.228]

It works, thank you!

It still would be nice to have this function directly inside STM32CubeMonitor.

[SAB]

I want to avoid making a new topic for my question, since it pertains to this subject.

I am trying to use this flow to parse multiple variables and have been unsuccessful. Using the flow you posted will append all variables to the same columns. Could you please help with this?

I am using the basic flow of having a variables node, and a process variables node to send the info to a graph.

Thank you!

[stephane.legargeant]

Hello SAB

The csv file put all the variables to the same columns because the time stamp of the acquisition may be different, so this is why we have proposed to generate one line for each data, with variable_name, acquisition time, and variable value.

An option could be to process the csv file in excel and sort and reorganize the data.

Nevertheless, it could be more efficient to process directly the data in STM32CubeMonitor and display it directly in graph without going through csv.

What processing do you want to perform with the csv file ?

Best regards

Stephane