How to write/draw something in my screen by touchGFX?

[Junde]

Hello there,

I want to recognize the digital and character written by hand with STM32F429BI.

I think there must be split into the following steps:

1. user input(write something in the screen by finger); (what widget I can use?)

2. transfer the screen pixel data to MCU from touchGFX; (How to get the partial pixel data in certain areas?)

3. MCU recognizes the content with some lib; (Can some Lib be used from ST?)

4. MCU sends the result to the screen for display; (I know how to do this only 😂)

Any advice and links are expected.

Thanks for your help.

[MM..1]

I mean nothink prepared for this freely exist, you can create custom widget and use canvas...Custom Widgets | TouchGFX Documentation

[GaetanGodart · Accepted solution]

Hello @Junde ,

1) user input(write something in the screen by finger); (what widget I can use?)
The best widget to use seems to be the canvas widget .
It is better to use this than to add a 1x1 box because you are limited in memory (and therefore limited in the amount of elements you can add).
You could get the position of the finder with the drag event or the handledragevent directly from the widget itself.
You could save the last position of the finger and draw a line between that last position and the current position. This would give you a boxy but simple implementation.
If you want to do things better, you can use 4 points instead of 2 to get a smoother curve using the quadraticbezierto() method.

2) transfer the screen pixel data to MCU from touchGFX; (How to get the partial pixel data in certain areas?)
You can get the framebuffer data by checking where you store the framebuffer.
Open your STM32CubeMX file (in your project folder, open the .ioc file), check "middleware", then "TouchGFX" and here you can see where your framebuffer is stored :

Your framebuffer size id width*height*RGBprecision. Here is another example of that : memory-consumption .

Now that you have the framebuffer location and size, you need to only fetch the data of the canvas (or alternatively you could isolate the drawing by saving the minX, minY, maxX and maxY and adding a bit of padding) and store it into whatever format you want (an array, a jpeg, a png, a bmp).

3) MCU recognizes the content with some lib; (Can some Lib be used from ST?)
It is up to you to do that research.
The first link on google mentions tesseract. I have personally used tesseract in Python and it worked fine (also exist in c++).

I hope this guidance is enough.

If this comment answers your problem, I invite you to select it as "best answer".
Otherwise, don't hesitate to specify where you are stuck.

Regards,

[Junde]

@GaetanGodart 

So many thanks to you for your detailed answer.

I will verify the "canvas widget" later, seems it can work well for my requirements.

I am trying to do it by LTDC Layer2, and I can draw something now.

As you said, it needs more RAM for the layer2 area.

[Junde]

@MM..1 

Thanks for your help, I will try it later!

[Junde]

.

[Junde]

.

[Junde]

Hi @GaetanGodart @MM..1 

I am trying the canvas widget, it seems to work, but still some issues:

  

The code above is trying to draw an "X", but the second line overwrites some area of the first line, as below:

And the delete button can NOT clear the canvas area, how to do it?

Thank you!

(The project is uploaded as Attachment.)

[MM..1]

You require more line objects one for each. Too init line is better place into setup func after base call.

For your idea you declare array lines ...

[Junde]

Hi @MM..1 

According to your help, I can add and delete the line dynamically.

The code and the phenomenon as below:

As the result in above picture, it is work well, but need to much heap.

In my last version(picture below):

I set the color of line painter to RGB565, is any possible to change the painter color to ARGB2222 to resolve the overwrite problem(may the alpha value can control the overwrite area)?

I want to try, but don't know how to change the color format, it should be similar with following code:

touchgfx::PainterARGB2222 linePainter;
linePainter.setColor(touchgfx::Color::getColorFromARGB(0, 0, 0, 0));  // compiler err

Thanks very much!