WHen calculating the maximum output of the filter, the SHR calcualtion can overflow:
|
# (the 0x7FFFFF00 is the largest value a 1-block Stage1 filter can output) |
|
max_samp_out = np.dot( np.int64( 0x7FFFFF00) * np.sign(self.coefs), |
|
np.round(self.coefs ).astype(np.int64) ) |
|
|
|
# 32-bit VPU multiplications have a built-in 30-bit right-shift |
|
max_samp_out >>= 30 |
|
# We can do floor(log2()-30) or ceil(log2()-31) and the result is basically |
|
# the same. |
|
frac_shr = np.log2(np.abs(max_samp_out)) - 30 |
|
|
|
self.shr = np.int32(np.floor(frac_shr)) |
This is probably to do with the dot product occuring before the >>30 and overflowing int64 range. Reproduce this and update the calulations so they don't overflow.
Note this is quite pessamistic in it's assumption of the worst case PDM input, we can probably get away with a smaller shift
WHen calculating the maximum output of the filter, the SHR calcualtion can overflow:
lib_mic_array/python/mic_array/filters.py
Lines 140 to 150 in 2cdd502
This is probably to do with the dot product occuring before the >>30 and overflowing int64 range. Reproduce this and update the calulations so they don't overflow.
Note this is quite pessamistic in it's assumption of the worst case PDM input, we can probably get away with a smaller shift