Writting text with Unicode::fromUTF8 bug

[AAlis.23]

Hi,

I haver two text areas AAA and BBB both with wildcard buffer to write to. When use the funcion Unicode::fromUTF8 to write some text and the text is larger than the buffer the values of the two text areas are concatenated and writted on the first area.

If title lenght is larger than text area buffer the two buffers are concatenated and pinted in the fist text area:

Unicode::fromUTF8((const uint8_t*)title, AAABuffer, AAA_SIZE);

Unicode::fromUTF8((const uint8_t*)title2, BBBBuffer, BBB_SIZE);

The solucion is to limit the size to one less:

Unicode::fromUTF8((const uint8_t*)title, AAABuffer, AAA_SIZE - 1);

Unicode::fromUTF8((const uint8_t*)title2, BBBBuffer, BBB_SIZE - 1);

I think this is a small bug and the compiler must limit it.

Thanks

[Yoann KLEIN]

Hello @AAlis.23​,

I don't really get it, what happens if you don't add this - 1 to your buffer size ?

Could you share screenshot of the bug / behavior you are experiencing ?

Thanks,

/Yoann

[AAlis.23]

Hi,

Simulator does not run when added too many custom code.. no screenshot. This is what happends:

Unicode::fromUTF8((const uint8_t*)title, AAABuffer, AAA_SIZE);

Unicode::fromUTF8((const uint8_t*)title2, BBBBuffer, BBB_SIZE);

If title is larger than AAA_SIZE the result is the two text concatenated on the AAA text area and the BBB text on it's place:

AAA -> "text_1text2"

BBB -> "text2"

Its like the buffer end character its overwritten and the function that shows the text is not stopping and getting the next text in memory...

[t.decker]

Hi @Yoann KLEIN​, I have a similar issue here. I think it's the same cause as the behaviour described by @AAlis.23​ in the fromUTF8()-function:

static uint16_t fromUTF8(const uint8_t* utf8, UnicodeChar* dst, uint16_t maxchars);

When the given utf8 string is longer than maxchars, dst won't get a terminating '\0'. Doesn't matter if you set maxchars to XYZ_SIZE, XYZ_SIZE-1 or a ver short number like 5. maxchars only limits the number of copied characters, but does not add the terminating char.

Unfortunately some standard c-lib function have the same behaviour, so this might be intentended? Then it should be noted in the documentation of the function. Then every call to fromUTF8 must be followed by a

dst[maxchars-1] = 0;

[AAlis.23]

hi,

My strings are well terminated with \0 character, the problem is with the fromUTF8 function when the string is larger than size, the next string in memory is concatenated (very rare because I think the next string is not at the next memory address), but, with the -1 I corrected this problem

[t.decker]

Hi @AAlis.23​, maybe you see the \0 character, because the memory had this value before? Also you say the contiguous strings get concatenated. That's only because the first one doesn't have the termination char.

I have seen random other texts in dst after the copied data when setting maxchars to half of dst length.

I hope I'm not wrong here with my assumptions :grimacing_face:

[AAlis.23]

Hi,

If I try to write a text longer than the "maxchars" value the function should add the \0 at the end to respect the max length of the text area, if the string is concatenated to the next string in memory and the number of chars printed are higher than the maxchars value... this is a bug, as simple as this and should be corrected.

Thanks

[t.decker]

@Yoann KLEIN​ Any news here from ST's side?

[AAlis.23]

no :(

[Yoann KLEIN]

Hello,

Sorry for the big delay.

I just discussed this with one of my colleague that knows more than me on the subject.

What @t.decker​ said previously is perfectly correct, the size you select in Designer has to take consideration of the \0 character, and this character has to be placed at the last position to avoid any issue with buffer management. The string you wanna put in the wildcard buffer should always be less than at least MAXCHAR-1 long. We strongly advice you to always select a buffer size at least slightly bigger than the longest string you want to put inside :)

Hope that helps,

/Yoann

[AAlis.23]

Hi,

For me this is a bug, this funcion should crop the text to avoid concatenation of strings, not responsibility of the user to ba safe that the text is MAXCHARS-1.

Thanks

[Piranha]

@Yoann KLEIN​ , as @t.decker​ explained, not setting the terminating null character is a bug of the fromUTF8() function/method.

> Unfortunately some standard c-lib function have the same behaviour, so this might be intentended?

The strncpy() is not really a function for copying strings. It is meant for filling a character arrays with a text from a string. Despite a confusing documentation, it actually does not null terminate strings at all, but instead fills the rest of the array with null characters. Obviously, if there is no space left, it does nothing.