KeyError: 'status' when trying to schedule spider

[kalessin]

I am getting this error when trying to schedule a spider. This is happening with version 2.3.1

Traceback (most recent call last):
  File "/home/molveyra/.local/share/virtualenvs/mollie-AtuAN_AE/lib/python3.8/site-packages/scrapinghub/legacy.py", line 157, in _decode_response
    if data['status'] == 'ok':
KeyError: 'status'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/molveyra/.local/share/virtualenvs/mollie-AtuAN_AE/lib/python3.8/site-packages/scrapinghub/client/exceptions.py", line 69, in wrapped
    return method(*args, **kwargs)
  File "/home/molveyra/.local/share/virtualenvs/mollie-AtuAN_AE/lib/python3.8/site-packages/scrapinghub/client/__init__.py", line 19, in _request
    return super(Connection, self)._request(*args, **kwargs)
  File "/home/molveyra/.local/share/virtualenvs/mollie-AtuAN_AE/lib/python3.8/site-packages/scrapinghub/legacy.py", line 143, in _request
    return self._decode_response(response, format, raw)
  File "/home/molveyra/.local/share/virtualenvs/mollie-AtuAN_AE/lib/python3.8/site-packages/scrapinghub/legacy.py", line 169, in _decode_response
    raise APIError("JSON response does not contain status")
scrapinghub.legacy.APIError: JSON response does not contain status

[vshlapakov]

@kalessin Is it reproducible or transient? Could you share payload you're sending (w/o project id, etc)?

[kalessin]

It is reproducible. It happens every time I try to schedule a spider. However, I just debugged it and the issue is that the response is

{'detail': 'Authentication credentials were not provided.'}

Still the issue is in the library, because it must handle this condition and print the correct failure.

[vshlapakov]

it must handle this condition and print the correct failure.

Agree, it should be addressed by the library

[kalessin]

An update on this. Regardless the response is 'Authentication credentials were not provided.', this really happens when the apikey is wrong, so there is a server side issue too.

If I don't pass any apikey, I get correctly a runtime error:

RuntimeError: No API key provided and neither SH_APIKEY nor SHUB_JOBAUTH environment variables is set

[vshlapakov]

Regardless the response is 'Authentication credentials were not provided.', this really happens when the apikey is wrong, so there is a server side issue too.

I believe that's intentional - our server would react the same if credentials aren't provided or it's wrong, was done for security purpose. But I think we could handle it on the client side anyway to improve experience.

[kalessin]

Right. Still the reaction is not the same. When you provide wrong apikey, then you should also get

RuntimeError: No API key provided and neither SH_APIKEY nor SHUB_JOBAUTH environment variables is set

instead of a meaningless KeyError exception.