cancel
Showing results for 
Search instead for 
Did you mean: 

Slow MJPEG stream with LwIP, ERR_MEM when sending large data

Dangcaominh
Associate II

Hello!
I am using STM32F427VIT6 and OV5640 camera to stream MJPEG to a webserver. I used DCMI DMA to get a jpeg_frame, size around 10KB to 25KB per image, with almost 30FPS and send it to jpeg buffer in CCMRAM like this.

__attribute__((section(".ccmram"))) __attribute__((aligned(32))) uint8_t jpeg[32768];
__attribute__((section(".ccmram"))) __attribute__((aligned(32))) uint32_t jpegLength = 0;

 Then I used LwIP with raw TCP, tcp_write() function to send it to webserver but it's very laggy, sometimes return ERR_MEM in this function

LWIP_DEBUGF(TCP_OUTPUT_DEBUG | LWIP_DBG_LEVEL_SERIOUS, ("tcp_write : could not allocate memory for pbuf copy size %"U16_F"\n", seglen));
        goto memerr;


Although I spent nearly 100% of my RAM allocating for LwIP heap memory and SND buffer. Here is my LwIP mem config

/*----- Default Value for MEMP_NUM_TCP_PCB: 5 ---*/
#define MEMP_NUM_TCP_PCB 1
/*----- Value in opt.h for NO_SYS: 0 -----*/
#define NO_SYS 1
/*----- Value in opt.h for SYS_LIGHTWEIGHT_PROT: 1 -----*/
#define SYS_LIGHTWEIGHT_PROT 0
/*----- Value in opt.h for MEM_ALIGNMENT: 1 -----*/
#define MEM_ALIGNMENT 4
/*----- Default Value for MEM_SIZE: 1600 ---*/
#define MEM_SIZE 32768
/*----- Default Value for MEMP_NUM_TCP_SEG: 16 ---*/
#define MEMP_NUM_TCP_SEG 2048
/*----- Default Value for PBUF_POOL_SIZE: 16 ---*/
#define PBUF_POOL_SIZE 20
/*----- Default Value for PBUF_POOL_BUFSIZE: 592 ---*/
#define PBUF_POOL_BUFSIZE 1560
/*----- Value in opt.h for LWIP_ETHERNET: LWIP_ARP || PPPOE_SUPPORT -*/
#define LWIP_ETHERNET 1
/*----- Value in opt.h for LWIP_DNS_SECURE: (LWIP_DNS_SECURE_RAND_XID | LWIP_DNS_SECURE_NO_MULTIPLE_OUTSTANDING | LWIP_DNS_SECURE_RAND_SRC_PORT) -*/
#define LWIP_DNS_SECURE 7
/*----- Default Value for TCP_WND: 5840 ---*/
#define TCP_WND 16384
/*----- Default Value for TCP_MSS: 536 ---*/
#define TCP_MSS 1460
/*----- Default Value for TCP_SND_BUF: 2920 ---*/
#define TCP_SND_BUF 16384
/*----- Default Value for TCP_SND_QUEUELEN: 46 ---*/
#define TCP_SND_QUEUELEN 2048


Here is my send_jpeg function

void http_send_jpeg(struct tcp_pcb *pcb)
{
	err_t err;

	// Nếu chưa đang gửi JPEG, bắt đầu gửi frame mới
	if (!jpeg_state.sending)
	{
		jpeg_state.sent = 0;
		jpeg_state.sending = true;
		jpeg_state.current_length = jpegLength;

		// Gửi header MJPEG
		char header[128];
		sprintf(header, "--frame\r\nContent-Type: image/jpeg\r\nContent-Length: %lu\r\n\r\n", jpegLength);

		// Kiểm tra buffer trước khi gửi header
		if (tcp_sndbuf(pcb) < strlen(header))
		{
			printf("Not enough buffer for header\n");
			return;
		}

		err = tcp_write(pcb, header, strlen(header), TCP_WRITE_FLAG_COPY);
		if (err != ERR_OK)
		{
			printf("tcp_write header failed: %d\n", err);
			jpeg_state.sending = false;
			return;
		}
	}
	err = ERR_OK;
	// Gửi dữ liệu cho đến khi hết buffer hoặc hết dữ liệu
	while (jpeg_state.sent < jpeg_state.current_length && err == ERR_OK)
	{
		// Tính số byte thực sự sẽ gửi
		uint16_t to_send = min(16384, jpeg_state.current_length - jpeg_state.sent);

		// Gửi dữ liệu
		err = tcp_write(pcb, &jpeg[jpeg_state.sent], to_send, TCP_WRITE_FLAG_COPY);
		if (err != ERR_OK)
		{
			printf("tcp_write JPEG failed at: %lu, error: %d\n", jpeg_state.sent, err);
			tcp_output(pcb);
			return; // Thoát và thử lại sau
		}

		// Chỉ cập nhật số byte đã gửi khi tcp_write thành công
		jpeg_state.sent += to_send;
		if (jpeg_state.sent == jpeg_state.current_length)
		{
			tcp_output(pcb); // Đẩy dữ liệu ngay lập tức
		}
	}
	if (jpeg_state.sent < jpeg_state.current_length)
	{
	}
	else if (tcp_sndbuf(pcb) < 2)
	{
		// Không đủ buffer cho footer, đợi callback tiếp theo
		tcp_output(pcb);
		return;
	}
	else
	{
		err = tcp_write(pcb, "\r\n", 2, TCP_WRITE_FLAG_COPY);
		if (err != ERR_OK)
		{
			printf("tcp_write footer failed: %d\n", err);
			jpeg_state.sending = false;
			return;
		}

		tcp_output(pcb);
		jpeg_state.sending = false; // Đánh dấu đã gửi xong
		jpeg_state.sent = 0;		// Reset vị trí gửi
	}
}

 

sent_callback:

err_t http_sent_callback(void *arg, struct tcp_pcb *tpcb, u16_t len)
{
	// Sau khi mỗi phần được gửi thành công
	if (tpcb != NULL)
	{
		// Tiếp tục gửi
		http_send_jpeg(tpcb);
	}
	return ERR_OK;
}

And here is my build log

[build] Memory region         Used Size  Region Size  %age Used
[build]              RAM:      190256 B       192 KB     96.77%
[build]           CCMRAM:       32776 B        64 KB     50.01%
[build]            FLASH:      172004 B         2 MB      8.20%
[driver] Build completed: 00:00:09.461
[build] Build finished with exit code 0

Help me please. I spent too much time on this MJPEG server and still cannot figure it out.

6 REPLIES 6
Dangcaominh
Associate II

Help me pls :)))))))))))))))

hemant_ast
Associate II

Hey, we are on same stage, i am also trying to stream video over webserver with same camera ov5640, & controller STM32H743, how much frame rate you are getting, 

Like 5 FPS :))))

At what resolution.  You can try with RTOS ,netconn API, where you may get 50fps at 640 x480 resolution. 

How you getting video over the webserver, if possible please share webserver side code

hemant_ast
Associate II

If you don't mine we can discuss over the mail. 

Dangcaominh
Associate II

That's great man, my email is dangcaominhheo@gmail.com. I tried using netconn API and it works, somewhere around 30FPS woth 320x240 img but when running for a while, the stream is stuck, disconnected and I don't know why.