curlify to support httpx:Response objects

[fenchu]

curlify==2.2.1
httpx==0.23.3

We are using curlify to print out curl requests when requests module requests fails, this has always worked great, but now I've moved my requests event hooks to httpx for http2 and asyncio support in our fastapi rest apis.

I believe httpx:Response is a bit different from requests.Response.

if I remove the curlify setting it works all fine

#!/usr/bin/env python3
import asyncio, curlify, httpx, json, textwrap

# we store each connection
connection = None

async def hook(response: httpx.Response) -> None:
    """ requests hook to see full rest requests+curl+response into connection_trace global, must not return anything """
    format_headers = lambda d: '\n'.join(f'{k}: {v}' for k, v in d.items())
    global connection
    out = ''
    await response.aread()
    print(f"response.{response}")
    request = response.request
    if not hasattr(request, 'body'):
        setattr(request, 'body', None)
    if not hasattr(response, 'reason'):
        setattr(response, 'reason', None)
    mycurl = curlify.to_curl(request)
    try:
        if response.text:
            out = json.dumps(json.loads(response.text), indent=2, sort_keys=True)
    except json.JSONDecodeError:
        out = response.text
    connection = textwrap.dedent('''
        ---------------- request ----------------
        {req.method} {req.url}
        {reqhdrs}

        {req.body}
        ------------------ curl ------------------
        {mycurl}
        ---------------- response ----------------
        {res.status_code} {res.reason} {res.url}
        {reshdrs}

        {out}
        -----------------------------------------
    ''').format(
        req=request,
        res=response,
        mycurl=mycurl,
        reqhdrs=format_headers(request.headers),
        reshdrs=format_headers(response.headers),
        out=out
    )

c = httpx.AsyncClient(event_hooks={'response': [hook]})
#c = httpx.AsyncClient()
r = asyncio.run(c.get("https://cat-fact.herokuapp.com/facts"))
print(connection)

I get this error:

python.exe .\bin\httpx-hook.py
response.<Response [200 OK]>
Traceback (most recent call last):
  File "C:\dist\work\trk-fullstack-test\bin\httpx-hook.py", line 50, in <module>
    r = asyncio.run(c.get("https://cat-fact.herokuapp.com/facts"))
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dist\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\dist\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dist\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\dist\venvs\trk-fullstack-test\Lib\site-packages\httpx\_client.py", line 1757, in get
    return await self.request(
           ^^^^^^^^^^^^^^^^^^^
  File "C:\dist\venvs\trk-fullstack-test\Lib\site-packages\httpx\_client.py", line 1533, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dist\venvs\trk-fullstack-test\Lib\site-packages\httpx\_client.py", line 1620, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dist\venvs\trk-fullstack-test\Lib\site-packages\httpx\_client.py", line 1648, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dist\venvs\trk-fullstack-test\Lib\site-packages\httpx\_client.py", line 1706, in _send_handling_redirects
    raise exc
  File "C:\dist\venvs\trk-fullstack-test\Lib\site-packages\httpx\_client.py", line 1688, in _send_handling_redirects
    await hook(response)
  File "C:\dist\work\trk-fullstack-test\bin\httpx-hook.py", line 19, in hook
    mycurl = curlify.to_curl(request)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dist\venvs\trk-fullstack-test\Lib\site-packages\curlify.py", line 44, in to_curl
    flat_parts.append(quote(v))
                      ^^^^^^^^
  File "C:\dist\Python311\Lib\shlex.py", line 329, in quote
    if _find_unsafe(s) is None:
       ^^^^^^^^^^^^^^^

removing the curlify it all works excellent: (it is a pubøic open api):

python.exe .\bin\httpx-hook.py
response.<Response [200 OK]>

---------------- request ----------------
GET https://cat-fact.herokuapp.com/facts
host: cat-fact.herokuapp.com
accept: */*
accept-encoding: gzip, deflate, br
connection: keep-alive
user-agent: python-httpx/0.23.3

None
---------------- response ----------------
200 None https://cat-fact.herokuapp.com/facts
server: Cowboy
connection: keep-alive
x-powered-by: Express
access-control-allow-origin: *
content-type: application/json; charset=utf-8
content-length: 1859
etag: W/"743-wddLBn/iA1pX11XPOeCdXfKM6GE"
set-cookie: connect.sid=s%3A1P6uo-hNorZRug0dYBtA0rUHUlf4DHeP.VyCKOnuAXShtRgYeKlO68tFuEmeXwfrr27QuqRtbuBE; Path=/; HttpOnly
date: Tue, 28 Mar 2023 14:53:52 GMT
via: 1.1 vegur

[
  {
    "__v": 0,
    "_id": "58e008800aac31001185ed07",
    "createdAt": "2018-03-06T21:20:03.505Z",
    "deleted": false,
    "source": "user",
    "status": {
      "sentCount": 1,
      "verified": true
    },
    "text": "Wikipedia has a recording of a cat meowing, because why not?",
    "type": "cat",
    "updatedAt": "2020-08-23T20:20:01.611Z",
    "used": false,
    "user": "58e007480aac31001185ecef"
  },
  {
    "__v": 0,
    "_id": "58e008630aac31001185ed01",
    "createdAt": "2018-02-07T21:20:02.903Z",
    "deleted": false,
    "source": "user",
    "status": {
      "sentCount": 1,
      "verified": true
    },
    "text": "When cats grimace, they are usually \"taste-scenting.\" They have an extra organ that, with some breathing control, allows the cats to taste-sense the air.",
    "type": "cat",
    "updatedAt": "2020-08-23T20:20:01.611Z",
    "used": false,
    "user": "58e007480aac31001185ecef"
  },
  {
    "__v": 0,
    "_id": "58e00a090aac31001185ed16",
    "createdAt": "2018-02-11T21:20:03.745Z",
    "deleted": false,
    "source": "user",
    "status": {
      "sentCount": 1,
      "verified": true
    },
    "text": "Cats make more than 100 different sounds whereas dogs make around 10.",
    "type": "cat",
    "updatedAt": "2020-08-23T20:20:01.611Z",
    "used": false,
    "user": "58e007480aac31001185ecef"
  },
  {
    "__v": 0,
    "_id": "58e009390aac31001185ed10",
    "createdAt": "2018-03-04T21:20:02.979Z",
    "deleted": false,
    "source": "user",
    "status": {
      "sentCount": 1,
      "verified": true
    },
    "text": "Most cats are lactose intolerant, and milk can cause painful stomach cramps and diarrhea. It's best to forego the milk and just give your cat the standard: clean, cool drinking water.",
    "type": "cat",
    "updatedAt": "2020-08-23T20:20:01.611Z",
    "used": false,
    "user": "58e007480aac31001185ecef"
  },
  {
    "__v": 0,
    "_id": "58e008780aac31001185ed05",
    "createdAt": "2018-03-29T20:20:03.844Z",
    "deleted": false,
    "source": "user",
    "status": {
      "sentCount": 1,
      "verified": true
    },
    "text": "Owning a cat can reduce the risk of stroke and heart attack by a third.",
    "type": "cat",
    "updatedAt": "2020-08-23T20:20:01.611Z",
    "used": false,
    "user": "58e007480aac31001185ecef"
  }
]
-----------------------------------------