Reading a file backwards

[sb_st]

Hi there!

I'm working on a project wherein I'm trying to play an audio file in reverse. I'm using freeRTOS, fatFS, and a 4-wire SDIO setup, and am doing this little file read dance where I:

1. f_lseek() to the end of a file
2. calculate how many bytes I want to read
3. f_lseek() backwards
4. do the read()
5. goto 2 

Then, later, I reverse my audio samples. 

I'm experiencing some odd behavior that I don't understand, however. To try to wrap my head around it, I toggle a GPIO each time I start the above cycle. I would expect - when probing this GPIO with a scope - to see a reasonably constant square wave period that would help me understand how long the whole operation takes. 

What I see, however, is something different: I see a few high/low cycles, then a period of what looks like inactivity, then another section of high/low cycles:

I'm still coming up to speed with my understanding of fatFS, but I'm curious what might cause these 'pauses'?

I understand that what I'm doing - constantly seeking backwards in a rather large (47MB) file is ill-advised: I think what I'm doing is constantly traversing a tree from the beginning each time I do this seek (?). But it's not clear to me why this might take dramatically longer for some cycles and not others. 

I've also got some browser tabs open here trying to understand what other options I might consider for this (e.g. I wonder if a different filesystem might be more suited to this task? LittleFS?). But rather than just chalking this up to "this doesn't work", I'd like to understand more deeply what I'm experiencing. 

Thanks for any insight anyone may offer!

[Tesla DeLorean]

Need to be reading substantively large blocks, whose physical alignment (start) is aligned to cluster or sector boundaries.

Can't you use f_size() to determine length of files, just from the directory structure without traversing the FAT?

http://elm-chan.org/fsw/ff/doc/size.html

Caching of file system structures, could also help.

[sb_st]

---

@Tesla DeLorean wrote:

Need to be reading substantively large blocks, whose physical alignment (start) is aligned to cluster or sector boundaries.

Oh interesting; are you saying the non-uniformity of these reads is because I might not be landing exactly on a read spot that's aligned with a cluster or sector boundary? I'm doing no work to enforce things to do that; that's a helpful clue. 

---

Can't you use f_size() to determine length of files, just from the directory structure without traversing the FAT?

Oh yes, absolutely, good point, Though, I don't seek to the end of the file iteratively, so I don't think that's impacting my overall performance here. But good optimization note, thank you!

Caching of file system structures, could also help.

---

Can I trouble you to elaborate on this a bit? I'm not fluent enough yet to understand this, sorry. 

I'm curious if it's also the case that you'd advise exploring other filesystems; might there be some that are more suited to this task? 

Thanks!

[Tesla DeLorean]

The FAT Tables can be quite large, depends on the capacity, and the cluster size, and whether the file was written in a linear way. Most file systems are not optimized toward reading files backward.

Optimization here might have more to do with how much memory you can commit to the task, whether you can manage the backwardness via memory access instead of file access, or swapping in-situ.

There's probably ways of optimizing the representation of FAT cluster chain, within the file system's specific implementation, I'm not sure FATFS does that optimal, or do any caching beyond a sector or two. 

[sb_st]

---

Optimization here might have more to do with how much memory you can commit to the task, whether you can manage the backwardness via memory access instead of file access, or swapping in-situ.

Thank you, yes I'm investigating what's involved in extending the RAM I can offer here (I'm using a homemade development board, with only an stm32f405 onboard - no extra PSRAM or anything - but am considering adding something along those lines to the next hardware iteration).

Before I go too far down this path, I feel compelled to observe that I'm using the version of fatFS that gets installed when this option is selected in cubeMX, which is R0.12c, and to ask whether trying to update to a newer version of this middleware is an experiment worth trying. I see other posts on the forums indicating that there's some 'extra stuff' layered onto the bare fatFS files, and I'm unsure what kind of surgery is involved in bypassing the cubeMX installation and trying to just use a fresher download of the software?

Thanks!