Buffer exchange between thread and ISR with message queue

[mdesousa]

Hello,

I'd like to set up a log management system that would allow me to write to both the UART and a file (NOR flash).
However, I'm encountering data architecture difficulties that I can't resolve.

I'll typical want to write in my file only after buffering few lines to write a the same times to limit the number of write operation. I based my design around message queue.

The idea, the call function:

mytrace(CRITICAL, “my trace message that can be longer than 64 bytes”)

The function definition. Since my trace buffer can be bigger that 64 bytes which is the limit for messages queue in Azure RTOS, I've to : allocates byte memory pool, and queues the pointer :

void mytrace(const char* buffer)
{
    char* queue_buffer;

    if (tx_byte_allocate(&m_trace_byte_pool,
                         (VOID**)&queue_buffer,
                          MAX_TRACE_SIZE, TX_NO_WAIT) == TX_SUCCESS)
    {
        // buffer successfully allocated
        snprintf(queue_buffer, MAX_TRACE_SIZE, "%s\n\r", buffer);

        if (tx_queue_send(&m_qTrace, &queue_buffer, TX_NO_WAIT) != TX_SUCCESS)
        {
            // failed to enqueue, release the buffer
            tx_byte_release(queue_buffer);
        }
    }
}

 The thread processing the queue : 

VOID th_trace_requests(ULONG initial_input)
{
    while (1)
    {
        uint8_t* received_buffer;
        if (tx_queue_receive(&m_qTrace, &received_buffer, TX_WAIT_FOREVER) == TX_SUCCESS)
        {
            // process data towards uart/file
            tx_byte_release(received_buffer);
        }
    }
}

This works like a charm EXCEPT, when my trace function is called from an ISR. In this scenerio, tx_byte_allocate cannot by used. 

Problems are : 

- Passing pointer to my message queue seems mandatory since it can't handle message bigger than 64 bytes

- I cannot replace tx_byte_allocate by malloc/free because I've read that it can result in potential heap corruption if I mixup threadx context with ISR context

- I cannot fragment my trace request for the queue because if a preemption occurs, and another thread try to trace too, it can result in trace buffer blend in my queue.

What can you recommend me to solve my need ? Is Message queue the good choice for it ? 

In a multithread (with threadx) environment, how can I distribute multiple buffer from different source to a thread without risking conflict in access/ownership?

Is there good exemple or good practice for this that can be found online ? 

[Andrew Neil]

---

@mdesousa wrote:

I'm encountering data architecture difficulties that I can't resolve. ? 

---

Perhaps you are making/adding unnecessary difficulties for yourself?

eg,

mytrace(CRITICAL, “my trace message that can be longer than 64 bytes”)

rather than embed the human-readable, variable-length text, why not just log a fixed-size code?

This is what I've done in the past: have a fixed-size even structure, with unique event codes and some space for a "parameter" or two.

typedef enum {
  fire,
  flood,
  famine
} event_id_t

struct {
  event_id_t event_id;
  uint32_t   data_1;
  uint32_t   data_2;
}

[mdesousa]

I understand the concept, but this solution will not fit my need.

Human readable is a need for multiple reason : possible AT command response, technician that want a brief idea of the state of the board, or even some protocole extraction ...