Float are being converted to rows.fields.IntegerField

[rhenanbartels]

I am trying to create a rows.Table object from a list of dicts, but when the value 0.0 is found it automatically converts the float object to rows.field.IntegerField

In [59]: from rows.fields import detect_types
In [58]: detect_types(['field1'], [[0.0]])
Out[58]: OrderedDict([('field1', rows.fields.IntegerField)])

[turicas]

I'm not sure if it should not be integer. See:

>>> detect_types(['field1'], [[0.0], [1.0]])
OrderedDict([('field1', rows.fields.IntegerField)])

>>> detect_types(['field1'], [[0.0], [1.2]])
OrderedDict([('field1', rows.fields.FloatField)])

But we can discuss. :-)
Do you have any use case of the library itself where this behaviour created a problem? Can you please share your code?

[turicas]

The following test will verify what we need to fix this bug and can be added to tests/tests_table.py:

    def test_detect_types_float_int(self):
        '''detect_types must preserve object type

        From issue: https://github.com/turicas/rows/issues/198
        '''

        field_names = ['field1', 'field2']
        expected = {'field1': fields.IntegerField,
                    'field2': fields.FloatField, }

        # TODO: test with locale
        # if not locale: 1.0 = error, 1 = ok, 1,0 = error
        # if locale: 1.0 = 10, 1 = ok, 1,0 = error
        #In [11]: locale.setlocale(locale.LC_ALL, 'pt_BR.UTF-8')
        #Out[11]: 'pt_BR.UTF-8'
        #
        #In [12]: locale.atoi('1')
        #Out[12]: 1
        #
        #In [13]: locale.atoi('1.0')
        #Out[13]: 10
        #
        #In [14]: locale.atoi('1,0')
        #---------------------------------------------------------------------------
        #ValueError                                Traceback (most recent call last)
        #<ipython-input-14-ab3c7e00fd50> in <module>()
        #----> 1 locale.atoi('1,0')
        #
        #~/software/pyenv/versions/3.6.2/lib/python3.6/locale.py in atoi(string)
        #    323 def atoi(string):
        #    324     "Converts a string to an integer according to the locale settings."
        #--> 325     return int(delocalize(string))
        #    326
        #    327 def _test():
        #
        #ValueError: invalid literal for int() with base 10: '1.0'

        assert expected == fields.detect_types(field_names, [[42, 1.2]])
        assert expected == fields.detect_types(field_names, [[42, 1.0]])

[cuducos]

@turicas I see you self-assigned that issue, but I decided to give it a try in #258 — I'm sorry if I step on you ; )

[gmonnerat]

Not sure if is the right place, but in some cases we should consider as TextField instead of FloatField.

codigo,descriçao
"1.1","Transparência pulmonar normal."
"1.10","Aumento da área cardíaca."

In this case, 1.10 will overwrite 1.1 probably because:

>>> Decimal("1.1") == Decimal("1.10")
True
>>> 1.1 == 1.10
True