Refactoring dtypes (and constants)

[nicola-bastianello]

dtypes

I currently implemented a hacky Array.dtype in #7 , which matches dtypes by string name. this is not robust. also, the dtype handling doesn't follow the same abstract contract approach as the backend, since the dtype conversion is just a dict, which is not mandatory

I'm thinking of something like this:

• we add abstract properties to the _Backend object, one for each of the dtypes supported by the codebase. something like the code below. each property needs to return the framework-native object representing float64 (np.dtype(np.float64) for np and jnp, torch.float64, tf.float64), or None if it is not supported (it might also be possible to have the return be contextual on what device is selected, e.g. if float64 is not supported on GPU)

@property
@abstractmethod
def float64(self) -> Any:

• we then define a dtype object which, like Array, holds a framework-native dtype object in a slot/attribute named e.g. _dtype (an instance of numpy.dtype, torch.dtype, tf.DType), and exposes other useful methods, like __str__ returning "float64", and __eq__ to check if it is equal to another iop.dtype (by checking their _dtype attributes), and also allowing checks against framework-native objects
• the dtype should allow for init by passing a framework-native dtype (which is then used as the _dtype attribute) or a string (in which case the matching backend attribute is used)
• the dtype objects should be instantiated either at import of iop or registration of the backend, calling the corresponding backend property and erroring out if the backend is not defined. I'm not sure if creating them at import is doable

notes:

• numpy (and hence jax) defines np.float64 which is a type and np.dtype(np.float64) which is a dtype object. the latter is what we should use
• the naming convention seems to favor dtype for objects, only tf uses DType; so we can go for dtype I think
• see array API also

Array.dtype

as I mentioned above, the current dtype implementation is quite hacky. adding the features above, we would then replace this by a property that returns a iop.dtype object created by 1) taking the Array.value dtype and 2) using it to instantiate a iop.dtype() object

a related discussion here is: do we want to make Array.dtype a slot, or keep the current approach of instantiating the dtype only on demand? I currently prefer the latter option as I don't see a huge benefit and adding another slot might decrease performance slightly

constants

a similar approach should be taken for constants inf, e, pi, nan (see here): we define an abstract property in _Backend that should return the framework-native corresponding constant. then we have 4 custom objects (iop.inf etc) that hold the framework-native constant as an attribute, and expose methods that allow for all operations one can do on these constants

[Simpag]

I'm not sure I see the benefits of using a custom dtype class over a simple enum? Could you provide further examples?

I think the constants would be a good addition.

[nicola-bastianello]

there are several reasons: 1) this aligns to how all frameworks handle dtypes as custom objects; 2) it allows us to embed functionality on the objects directly; 3) at least in my opinion, an object named float32 is more intuitive than an Enum item, which is a bit less immediate.

I'll start working on a proposal based on these ideas and we can discuss further afterwards if we want to go for that

[nicola-bastianello]

@Simpag here is a WIP on this https://github.com/nicola-bastianello/decent-array/tree/dtypes

I went for a slightly different approach than my previous comment

currently there is a circular import and the dtypes are not actively used in the backend functions. I'll wait for #7 to be merged and then fix all this (I left DTypes for now, but it will be removed once I integrate dtype objects). I've currently already implemented a conditional check on float64 for jax; we can do the same based on the device and other framework settings, so I think this approach is quite nice

I also realized that _Backend instances don't have a name that can be inspected and used in error messages, so as part of this I'll add the name also

in the meantime, please let me know what you think about this approach

[Simpag]

I am still not sure I see the full picture but lets wait and see how it looks when its more complete!

[Simpag]

I dont think its inherently a bad approach, just might be a bit over-engineered but the use-case might be clearer once its implemented more