numeric stack that can handle numbers across different OOMs and also negative numbers

[thedavidmeister]

Problem

This is broadly the issue:

https://hackmd.io/I7Z20eObQhu6dT9F2Z4Wgg

High level is that we run into the limits of 18 decimal fixed point math as we currently use it in two key areas:

• Precision loss when operating across multiple OOMs, e.g. a halflife as 0.5^x loses 1 bit of precision for every increment of x
• Don't have negative numbers

We can get signed math from PRB but all operations like pow top out quite quickly in terms of what they can represent with overflows/precision loss.

A real world example is a dutch auction that is scanning the market value of N tokens where we don't know the real market value of any pair of them. We want to scan a very large range, which could go into the 18 digits of precision sub 1 but could also be OOMs above 1 as well, e.g. we might want the auction to cover a full 20-50 OOMs. The precision starts to drop a lot if we do this with a halflife, resulting in hacks #245

That doc was written before doing any POCing, the latest state of things is at https://github.com/rainlanguage/rain.math.float

Solution

Create something like a 128 decimal float but with some key differences/notes:

• No "special values" like infinity/nan, error if we can't do something in a clear mathematically sensible way (e.g. no dividing by zero)
• Internal details of the algorithms are going to have to be whatever can be done gas efficiently onchain
• There might be some compromises for precision etc. such as when calculating logs/powers, in order to achieve good gas
• 2's complement logic internally, so no space for "negative zero" which is a float-ism but doesn't exist in real math