Version 1.6.0 CSV Format Improvement suggestion.

[PDail.1]

Hi team,

I downloaded 1.6.0 today from the website and confirmed that it fixed my previous issues with brackets in variable names.

I see that you also changed the delimiter to commas, which I appreciate since it will now open in excel.

What is sort of a bummer is that variables in a sequential loop sampling frequency may start at different times for some reason. It makes it difficult to align data to a timestamp.

I tried to do a sampling frequency of 0.1Hz to see if that would align them better, but it failed to make a CSV at all.

I'd prefer sequential loop, but each entry for rows that don't yet have data could be a #NAN or a 0, I don't really care which, but the 1.5.0 put them in as a 0.

I then can sort the column by timestamps and filter to just one variable. Copy all of the Y axis values for that variable filtered and put it into a new sheet.

I then change the filter to a different variable and copy the Y axis values of that variable.

Then plot those two columns of data as a scatterplot. This only works if they are time aligned. I might end up with 1.5 seconds of ADC data before the other value starts to report data in the current form, I'm not exactly sure why.

To further describe my use case. I have a potentiometer adc value and I want to plot that as the x axis to a motor speed Y axis.

Cube monitor currently only supports x axis as time. To get this graph I need to export the csv and edit in excel.

It would almost be better to offer an option where there was a first row column header of

Time, Variable Name 1, Variable Name 2, Variable Name 3

and then each time through the sequential loop it appended a timestamp row of information. I can see that you'd lose some resolution on the timestamp as right now I get data split across milliseconds, but I don't know.

My current workflow is faster to go back to 1.5.0 and make temporary variables that copy the value of things that have brackets and do all of the above just because the 0's are recorded.

I feel like I might have been the one to say remove the zeros when nothing is changing, if I was, my bad, but maybe that should have been clarified.

I'd be more than happy to show someone how to replicate what I'm doing so that the product can be improved. You guys are doing great.

[stephane.legargeant · Accepted solution]

Hello

The export format has changed in v1.6.0. The time is now in seconds since the start of the acquisition, and give the exact sampling time of the variable. With multiple variables, it happens often that there is a small time shift between the variable.

In fact, the same acquisition principle was used in 1.5.0, so I dont understand why you have an issue with 1.6.0. and not 1.5.0.

Are you sure you have not used the option "Log only change" ?

I will do some tests to understand your point. Could you send me in private message an extract of your csv file ?

For the formatting "Time, Variable Name 1, Variable Name 2, Variable Name 3", you may have a look to the wiki. How to log data in a .csv file

There are 2 flows that can be used to sort the data. It can be a solution for your need.

Best regards

Stephane

[PDail.1]

Thank you very much for your response!

I had previously used that article linked, but did not have luck given limitations with my variable names.

I did end up getting the flow to work, but only after using the debugger.

There was some errors with the variables being boolean inputs, but treated as strings in the environment variable so a comparison to false would always be false.

I modified the "Delete Duplicate timestamps if user wants" subflow to be the following.

It isn't the cleanest code, and doesn't cover all cases of inputs, but it does work with strings and doesn't error out.

The example also delimits with semicolons, after an explanation previously was made about France needing semicolons as delimiters I understood, but pointed out the irony of the delimiter not being a comma in a csv file.

It may be worth looking into changing the template now. Or how the variables are being passed in and converted to strings instead of staying as a boolean.

let i;
let delDup = env.get("delete_time");
let nbVar = env.get("nbVar");
let nbVarInt;
 
if (nbVar == '1'){
    nbVarInt =1;
    }else{
    if (nbVar == '3'){
        nbVarInt=3;
    }
    }
 
let msgArray = new Array();
 
if(delDup === "false"){
    return msg;
}else{
    
    msgArray.push(msg.payload[0]);
    
    for(i=1; i < nbVarInt; i++){
        
        msgArray.push(msg.payload[i].y);
        
    }
    
    msg.payload = msgArray;
    return msg;
    
}

If you do anything, or do nothing, I'm satisfied at this point. This is a fantastic software and the latest fix has already helped me find a small bug in my code after I plotted things in excel.

Keep up the great work!