funcMatMul () require number range? about falcon-public HOT 1 OPEN

Apologies for the delay, have been on time off. This is somewhat expected as you might be reaching the precision limit. With 64-bit data type, you have about 32 bits of "space" before things overflow in a multiplication. If you use say a precision of 26 bits, that leaves about 6 bits of room for the integer part of any value. Thus, as you're approaching numbers whose integer part is about 2^6 = 64, you might have problems. 7.7338 * 6.7338 = 52.07786244 is in the ball park.

Try reducing the fixed-point precision of your computation. For example, you might be able to do 7.7 * 6.7 in about 11 bits of fixed-point precision or so (off the top of my head, you'll have to test this out).

from falcon-public.