STM32MPU Indirect Buffer Data Exchange from CA7 Core to CM4 Core Errors.

[MCarl.3]

I am working with a custom board, and are using the CM4 core to configure and mange FPGAs via the FMC.  In my system the FPGA configuration requires the transfer of a large data buffer (3MByte-10MByte) between the CA7 core and the CM4 core.  I understand this is sub-optimal due to the asynchronous connection between the AHB and AXI busses, but as the FPGAs are not configured very often that is okay.

I’ve set up shared data buffers using the indirect method, similar to the example described in How to exchange large data buffers with the coprocessor - example - stm32mpu.  The difference in my current implementation is that I am transferring data from CA7 core to CM4 core, and for now I am not using DMA, just memcpy.  Plan to add DMA later after getting a functional system for customer demo. Also I am using the standard rpmsg/OpenAMP framework for control messaging.

There is no problem on the CA7 side, the data buffer device is created by the CM4, and I can open a buffer, mmap and write/read from that buffer.  The interesting issue I am seeing is on the CM4 side.  The CM7 sends rpsmg to the CM4 with the physical address and length of the buffer.  The CM4 can read the buffer, but some of the data read is in error.  I send a CRC32 in the rpsmg from the CA7 so that I can check the data from the CM4.   It’s gets weird now.  On the CM4 I repeated read from the buffer and calculate the CRC32.  The CRC32 can change from iteration to iteration.  Yet at some point the CRC32 is correct, and after that is always correct.

Also I tried reading the buffer through CM4 JTAG, this always gives me the correct data.  Finally if I do a simple write/read compare function on the CM4 (writing and reading to the buffer) I do get some error.

I am thinking this is a bus issue between the CM4 and DDR, but not sure where to go from here.  The AHB 64 bit bus to AXI 32 bit bus may be the problem here, but I thought that would work ok.

Anyone have any suggestions where to look?  Currently reviewing the AXI endpoints setup.  I’m not all that familiar with AXI/AHB bus architecture so any insights would be greatly appreciated.

Thanks All!

-Mark

[PatrickF]

Maybe check that the buffers are declared as non-cacheable. Otherwise, it is probable that at the time the M4 read the data from it, DDR content is not yet updated. It will get updated later in the DDR when the data in the cache are replaced by other Linux activities (so, the time when DDR content is updated is not known).

Otherwise, maybe need some cache flush on CA7 side after the data has been written. I'm not Linux SW expert.

I'm not sure about the way the example has been done, but having buffers exchange from M4 to CA7 does not suffer from this problem because there is no cache when M4 side write a DDR buffer.

Regards.

[MCarl.3]

Thanks for the response! I believe you are correct, and there is a cache coherency issue. As you say, you are not a Linux SW expert, but I am allocating buffer in kernel using dma_alloc_coherent() which I thought would assign the proper attributes to the buffer to insure coherency for all devices and CM7 cpu.  Thinking I will try via ioctl to force a cache clean after the user space app writes to it. Will post a solution here if I find it.

[MCarl.3]

The problem was coherency, but I'm not sure why my use of dma_alloc_coherent()  didn't work. I modified the drive to add an ioctl that syncs the buffer using dma_sync_single_for_device(), and in my user space application I call the ioctl after writing to the buffer. Works fine now.