Increase VRING_SIZE from 16 to 32 in both processors

[KChar.1]

Hi there,

I am writing an application that requires large streams of IPCC. At the moment I have allocated the RPMSG vdev0 and vrings to SRAM3+4 and I have increased the buffer size of each ring to 1024.

The device tree now looks like this:

 vdev0vring0: vdev0vring0@10040000 {
			compatible = "shared-dma-pool";
			reg = <0x10040000 0x2000>;
			no-map;
		};
 
		vdev0vring1: vdev0vring1@10042000 {
			compatible = "shared-dma-pool";
			reg = <0x10042000 0x2000>;
			no-map;
		};
 
		vdev0buffer: vdev0buffer@10044000 {
			compatible = "shared-dma-pool";
			reg = <0x10044000 0x8000>;
			no-map;
		};

On the M4 Linker Script:

RAM2_ipc_shm		(xrw)	: ORIGIN = 0x10040000,	LENGTH = 0x00012000  

I am trying to increase the VRING_SIZE to more than 16 ( Ideally I would like to increase it to maximum possible value in my configuration). In Openapm_conf:

 #VRING_NUM_BUFFS         16

I have increased the memory in my device tree but I can not find the remoteproc config declaration for VRING size on linux side. Based on this article I assume that I have to increase the VRING_SIZE in both the rproc linux config and the openamp_conf in the M4. If there are any ides on how to configure VRING_SIZE on the linux side would be highly appreciated.

[Kevin HUBER]

Hello @KChar.1​ ,

Sorry for the late reply.

Did you succeed to increase the Vring size or find another way to make your application works?

Regards,

Kévin

[KChar.1]

Hi @Kevin HUBER​ ,

Thanks for getting back to me. Unfortunately I have not find a way to increase the VRING size yet. With my current linux image and device tree I can modify the VRING size on the M4 side (openamp_conf.h) to any value up to 16. When I attempt values above 16 I can not send or receive through IPCC. My assumption at the moment is that I have to allocate more memory in the linux kernel and specify my desired VRING size there, but I can not find exactly in which kernel object I have to do this adjustment.

Any advice would be really appreciated.

Regards,

Kyr

[Kevin HUBER · Accepted solution]

Hi @KChar.1​ ,

I would like to know, why you want to increase the number of vring buffer and why you do not just increase the size of the buffer, like in your previous post? https://community.st.com/s/question/0D53W00000qwjyiSAA/increased-1024-rpmsg-buffer-size-will-fail-on-mailbox-vdev-declaration

Anyway, I was able to realize the modification on my stm32mp157f-dk2 board with success, by increasing the VRING_NUM_BUFFS from 16 to 32.

You only need to modify the define in openamp_conf.h

#define VRING_NUM_BUFFS         32		  /* number of rpmsg buffer */

Then the modifications will be propagated to the linux automatically, thanks to the shared resource table. Defined in rsc_table.c in M4.

CONST struct shared_resource_table __resource __attribute__((used)) resource_table = {

	/* Vring rsc entry - part of vdev rsc entry */
	.vring0 = {VRING_TX_ADDRESS, VRING_ALIGNMENT, VRING_NUM_BUFFS, VRING0_ID, 0},
	.vring1 = {VRING_RX_ADDRESS, VRING_ALIGNMENT, VRING_NUM_BUFFS, VRING1_ID, 0},

On linux side, when the driver virtio is probed, it will get its information from the shared resource table, previously filled by the M4 core.

This is done inside the file drivers/virtio/virtio_ring.c

On M4 side, remember to increase the area in the dts. This is my configuration with 32 buffers of 512 bytes :

		vdev0vring0:vdev0vring0@10040000{
			compatible = "shared-dma-pool";
			reg = <0x10040000 0x2000>;
			no-map;
		};
 
		vdev0vring1:vdev0vring1@10042000{
			compatible = "shared-dma-pool";
			reg = <0x10042000 0x2000>;
			no-map;
		};
 
		vdev0buffer:vdev0buffer@10044000{
			compatible = "shared-dma-pool";
			reg = <0x10044000 0x8000>;
			no-map;
		};

In your case, since you increased the size of the buffers from 512 to 1024, please update the dts accordingly.

Then the final modifications is the linker file of M4. Where the ipc area length must be equal to the three areas defined in the dts.

In my case 0x2000 + 0x2000 + 0x8000 = 0xA000, but modify it regarding your dts.

  RAM2_ipc_shm		(xrw)	: ORIGIN = 0x10040000,	LENGTH = 0x0000A000

 And it should works!

Now you can verify your modifications thanks to /sys/kernel/debug.

Once your board boots, you can:

cat /sys/kernel/debug/remoteproc/remoteproc0/resource_table

To verify that you allocated 32 buffers for each side RX/TX

   Number of buffers 32

You can also checks the area that you defined inside the DTS by using:

cat /sys/kernel/debug/remoteproc/remoteproc0/carveout_memories

In my case:

Carveout memory entry:
        Name: vdev0vring0
        Virtual address: f8b4a303
        DMA address: 0x10040000
        Device address: 0x10040000
        Length: 0x2000 Bytes
 
Carveout memory entry:
        Name: vdev0vring1
        Virtual address: 135e15b0
        DMA address: 0x10042000
        Device address: 0x10042000
        Length: 0x2000 Bytes
 
Carveout memory entry:
        Name: vdev0buffer
        Virtual address: 00000000
        DMA address: 0x00000000
        Device address: 0x10044000
        Length: 0x8000 Bytes

These two files are very helpful to debug and verify that everything is correct on your board.

Regards,

Kevin

In order to give better visibility on the answered topics, please click on 'Select as Best' on the reply which solved your issue or answered your question. See also 'Best Answers'

[KChar.1]

Hi @Kevin HUBER​,

Thanks a lot for the detailed explanation! I tested successfully your example with 1024 buffer size and 32 VRINGS. I was missing the right alignment on the M4 linker script in previous attempts.

I am using a user-space application that has to send a big amount of data. Each string that leaves from A7 could be up to 1024 bytes (but most strings are aprox 128 bytes) This was the reason I already increased the buffer size in 1024 as in the previous posts. The A7 application is based on a clock that depending on user action can send multiple strings (up to 128) to the remote_proc[M4]. If the amount of strings is bigger than the VRING size then the application will hang until the message is processed by the M4 as expected. This is why I would like to increase the VRING size in the first place and try to find a workaround for these large IPCCs.

The debugging files are a huge help too! Thanks again for this!

Regards,

Kyr