cancel
Showing results for 
Search instead for 
Did you mean: 

Slight math error in "STM32H7- CORDIC co-processor" presentation

jacobq
Associate II

I was looking at some slides that I found called STM32H7- CORDIC co-processor (CORDIC, Revision 1.0) and noticed the following on slide 6: 

jacobq_0-1716499149144.png

According to this, the fractional number being represented by the fixed-point integer bits is (-1)^s multiplied by the sum of the scaled bits. For fractional numbers > 0 this is correct, but it incorrectly interprets 0x80000000 (Q1.31 representation of -1) as 0 since (-1)^1 * (0) = 0. Similarly, it gives the wrong value for other negative numbers like -0.25 (represented in Q1.31 as 0xE0000000) since 0xE --> 0b1110 so s=1, c30=1, c29=1, and all other ck = 0 --> (-1)^1 * (1/(2^1) + 1/(2^2)) = (-1) * (1/2 + 1/4) = -3/4.

I think the formula should instead be written more like the formula given here ("Converting from two's complement representation" section of Wikipedia article), e.g. fractional number = <ordinary 2's complement signed integer value> / 2^N (where N = 31 for Q1.31, N=15 for Q1.15, etc.)

Am I misunderstanding something here?

See also https://chummersone.github.io/qformat.html.

 

0 REPLIES 0