fix(docker): bind uvicorn on every address family, not IPv4 only - #320
fix(docker): bind uvicorn on every address family, not IPv4 only#320HAAHIT wants to merge 7 commits into
Conversation
The compose network runs with enable_ipv6, so DNS for `backend` answers with
both an A and an AAAA record and nginx uses both. uvicorn was bound to
0.0.0.0, which is IPv4 only, so nothing was ever listening on the AAAA
address. Those requests hit ECONNREFUSED first:
[error] connect() failed (111: Connection refused) while connecting to
upstream, upstream: "http://[fd00:b010::2]:4321/api/workspaces"
[warn] upstream server temporarily disabled while connecting to upstream
nginx then marks the upstream down for fail_timeout and retries over IPv4, so
requests do complete -- the cost is a wasted connection attempt, recurring
error-log noise, and an upstream that keeps being marked unhealthy for a
reason that has nothing to do with the backend's health.
Confirmed from inside the container before the change:
IPv4 -> LISTENING
IPv6 -> REFUSED: [Errno 111] Connection refused
The empty host is deliberate, and "::" is not a substitute. asyncio sets
IPV6_V6ONLY on any AF_INET6 server socket, so binding "::" listens on IPv6
*only* and refuses IPv4 -- swapping one broken family for the other. That was
measured, not assumed:
host=0.0.0.0 sockets=[AF_INET] IPv4=OK IPv6=REFUSED
host=:: sockets=[AF_INET6] IPv4=REFUSED IPv6=OK
host="" sockets=[AF_INET, AF_INET6] IPv4=OK IPv6=OK
An empty host makes asyncio open one socket per family, which is real
dual-stack.
After the change both families listen, nginx reaches the backend over each of
them, and twelve proxied requests produced zero connection-refused errors and
zero "upstream server temporarily disabled" warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
🤖 CodeAnt AI — Review Status
|
|
Warning Review limit reached
Next review available in: 30 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (9)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Git for Windows defaults to core.autocrlf=true, and with no .gitattributes
the repo had nothing to stop that from rewriting backend/docker-entrypoint.sh
to CRLF on checkout. Docker copies the file into the Linux image verbatim,
where the kernel reads the shebang as "/bin/sh\r" and looks for an
interpreter by that name. The result is:
exec /app/backend/docker-entrypoint.sh: no such file or directory
on a file that is present and executable, so the backend container never
starts and compose fails with "dependency failed to start: container is
unhealthy". Nothing in that message points at line endings, and the build is
fine on Linux, macOS, and CI, so it only ever breaks Windows clones.
Pin the file types that Linux reads verbatim to LF regardless of platform.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…#317) bcrypt is intentionally slow -- measured at ~750ms per call at the default cost factor on our backend container. It is also CPU-bound and never yields, so calling it directly from an async handler parks the event loop for that whole time. The cost is not paid by the request doing the hashing; it is paid by every request that worker is serving. One signup froze the process for three quarters of a second. All five call sites were synchronous: login (checkpw), signup (gensalt + hashpw), change_password (checkpw + gensalt + hashpw), and reset_password (gensalt + hashpw). checkpw is as expensive as hashpw, so logins blocked just as hard as signups. Route all of them through two helpers that wrap bcrypt in run_in_threadpool, which this module already uses for the JWKS lookup. Hashes are unchanged -- same algorithm, same cost factor -- so existing stored passwords keep verifying. Measured with a 10ms heartbeat task watching for event-loop stalls: before work 341 ms | worst loop stall 357 ms after work 328 ms | worst loop stall 47 ms The work takes just as long; it just no longer blocks everything else. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Entropy-rgb <som3shkamad@gmail.com>
* perf(db): stop pinging the database before every pooled checkout
pool_pre_ping issues a liveness round trip before handing out every pooled
connection. It is not amortised -- it repeats on every request, including for
a connection that is demonstrably healthy and was used moments earlier. On the
app's own engine that doubled the cost of a checkout:
pre_ping=True (before) median 1089 ms
pre_ping=False (after) median 481 ms
Same query, same connection, 608ms saved per database-touching request. The
post-signup flow makes several of these back to back, so it compounds.
Worth being precise about what this is not: instrumenting the pool showed
zero new DBAPI connections across repeated checkouts in every configuration,
so connections were already being reused correctly. The cost was purely the
extra round trip, not reconnection.
What pre-ping buys is protection against a connection that died while idle in
the pool. pool_recycle already closes that window from the other side by
retiring connections on a timer, well inside any sane server or pooler idle
timeout, so pre-ping largely re-checks what recycling has handled. Default it
off and keep recycling at 300s.
Because that is a trade rather than a free win, every pool value is now
env-tunable. A deployment that genuinely sees stale connections can set
DB_POOL_PRE_PING=true without a code change.
statement_cache_size=0 was measured too and left alone -- it costs nothing
here (166ms vs 174ms per query) and is load-bearing for transaction pooling.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(build): pin shell scripts to LF so Windows clones can build (#316)
Git for Windows defaults to core.autocrlf=true, and with no .gitattributes
the repo had nothing to stop that from rewriting backend/docker-entrypoint.sh
to CRLF on checkout. Docker copies the file into the Linux image verbatim,
where the kernel reads the shebang as "/bin/sh\r" and looks for an
interpreter by that name. The result is:
exec /app/backend/docker-entrypoint.sh: no such file or directory
on a file that is present and executable, so the backend container never
starts and compose fails with "dependency failed to start: container is
unhealthy". Nothing in that message points at line endings, and the build is
fine on Linux, macOS, and CI, so it only ever breaks Windows clones.
Pin the file types that Linux reads verbatim to LF regardless of platform.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* perf(auth): run bcrypt in the threadpool instead of on the event loop (#317)
bcrypt is intentionally slow -- measured at ~750ms per call at the default
cost factor on our backend container. It is also CPU-bound and never yields,
so calling it directly from an async handler parks the event loop for that
whole time. The cost is not paid by the request doing the hashing; it is paid
by every request that worker is serving. One signup froze the process for
three quarters of a second.
All five call sites were synchronous: login (checkpw), signup (gensalt +
hashpw), change_password (checkpw + gensalt + hashpw), and reset_password
(gensalt + hashpw). checkpw is as expensive as hashpw, so logins blocked just
as hard as signups.
Route all of them through two helpers that wrap bcrypt in run_in_threadpool,
which this module already uses for the JWKS lookup. Hashes are unchanged --
same algorithm, same cost factor -- so existing stored passwords keep
verifying.
Measured with a 10ms heartbeat task watching for event-loop stalls:
before work 341 ms | worst loop stall 357 ms
after work 328 ms | worst loop stall 47 ms
The work takes just as long; it just no longer blocks everything else.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Entropy-rgb <som3shkamad@gmail.com>
* perf(db): stop pinging the database before every pooled checkout
pool_pre_ping issues a liveness round trip before handing out every pooled
connection. It is not amortised -- it repeats on every request, including for
a connection that is demonstrably healthy and was used moments earlier. On the
app's own engine that doubled the cost of a checkout:
pre_ping=True (before) median 1089 ms
pre_ping=False (after) median 481 ms
Same query, same connection, 608ms saved per database-touching request. The
post-signup flow makes several of these back to back, so it compounds.
Worth being precise about what this is not: instrumenting the pool showed
zero new DBAPI connections across repeated checkouts in every configuration,
so connections were already being reused correctly. The cost was purely the
extra round trip, not reconnection.
What pre-ping buys is protection against a connection that died while idle in
the pool. pool_recycle already closes that window from the other side by
retiring connections on a timer, well inside any sane server or pooler idle
timeout, so pre-ping largely re-checks what recycling has handled. Default it
off and keep recycling at 300s.
Because that is a trade rather than a free win, every pool value is now
env-tunable. A deployment that genuinely sees stale connections can set
DB_POOL_PRE_PING=true without a code change.
statement_cache_size=0 was measured too and left alone -- it costs nothing
here (166ms vs 174ms per query) and is load-bearing for transaction pooling.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(env): clarify when DB_POOL_PRE_PING should stay on
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Entropy-rgb <som3shkamad@gmail.com>
) get_health built an httpx.AsyncClient per call and never closed it: resp = await httpx.AsyncClient(timeout=5).get(jwks_url) The client owns a connection pool, so each probe abandoned one. /api/health is polled by the container healthcheck every 10-30s, forever, which makes this a slow resource leak on a fixed timer rather than a one-off. The same line also put a third-party network round trip on the healthcheck path. Backend logs showed a steady stream of outbound JWKS requests every ~13s with nobody using the app, and it left /api/health taking 1.8-3.2s. Share one client and cache the reachability answer for 60s. Whether Supabase is reachable does not change between two polls seconds apart, so the diagnostic keeps its value while the probe stops being a hot path. The client is closed on shutdown through the existing lifespan teardown. Behaviour is otherwise unchanged: the supabase_jwks field is still always populated, non-200 still reports unexpected_status:<code>, and a network failure still degrades to unreachable:<ExceptionName> rather than surfacing as a 500 -- health must not go down because a third party did. Verified against the running app: six consecutive /api/health calls plus the container's own polling produced exactly one outbound JWKS request. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The compose network runs with enable_ipv6, so DNS for `backend` answers with
both an A and an AAAA record and nginx uses both. uvicorn was bound to
0.0.0.0, which is IPv4 only, so nothing was ever listening on the AAAA
address. Those requests hit ECONNREFUSED first:
[error] connect() failed (111: Connection refused) while connecting to
upstream, upstream: "http://[fd00:b010::2]:4321/api/workspaces"
[warn] upstream server temporarily disabled while connecting to upstream
nginx then marks the upstream down for fail_timeout and retries over IPv4, so
requests do complete -- the cost is a wasted connection attempt, recurring
error-log noise, and an upstream that keeps being marked unhealthy for a
reason that has nothing to do with the backend's health.
Confirmed from inside the container before the change:
IPv4 -> LISTENING
IPv6 -> REFUSED: [Errno 111] Connection refused
The empty host is deliberate, and "::" is not a substitute. asyncio sets
IPV6_V6ONLY on any AF_INET6 server socket, so binding "::" listens on IPv6
*only* and refuses IPv4 -- swapping one broken family for the other. That was
measured, not assumed:
host=0.0.0.0 sockets=[AF_INET] IPv4=OK IPv6=REFUSED
host=:: sockets=[AF_INET6] IPv4=REFUSED IPv6=OK
host="" sockets=[AF_INET, AF_INET6] IPv4=OK IPv6=OK
An empty host makes asyncio open one socket per family, which is real
dual-stack.
After the change both families listen, nginx reaches the backend over each of
them, and twelve proxied requests produced zero connection-refused errors and
zero "upstream server temporarily disabled" warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…DB/bolodb into fix/uvicorn-dual-stack-bind
User description
Found while investigating slow account creation. One-line runtime change, but the reasoning behind which one line matters — see the trap below.
Problem
The compose network runs with
enable_ipv6: true, so DNS forbackendanswers with both an A and an AAAA record, and nginx uses both. uvicorn was bound to0.0.0.0— IPv4 only — so nothing was ever listening on the AAAA address:Confirmed from inside the container:
nginx marks the upstream down for
fail_timeoutand retries over IPv4, so requests do complete. The cost is a wasted connection attempt, recurring error-log noise, and an upstream repeatedly marked unhealthy for a reason that has nothing to do with the backend's health.The trap:
::is not the fixThe obvious change is
--host ::. That makes it worse in the other direction. asyncio setsIPV6_V6ONLYon anyAF_INET6server socket, so::listens on IPv6 only and refuses IPv4 — swapping one broken family for the other. I shipped that first and the verification caught it.Measured in-container rather than assumed:
--host0.0.0.0[AF_INET]::[AF_INET6]""(this PR)[AF_INET, AF_INET6]An empty host makes asyncio open one socket per family, which is real dual-stack. The kernel's
bindv6only=0is not enough on its own, because asyncio overrides it per socket.Verification
After the change:
Twelve proxied requests produced:
Container comes up healthy, migrations run, and the app serves end-to-end —
/200,/api/health200 through the nginx proxy, Vite dev server 200.🤖 Generated with Claude Code
CodeAnt-AI Description
Improve backend connectivity, health checks, and request responsiveness
What Changed
Impact
✅ Fewer IPv6 connection failures✅ Faster concurrent account requests✅ Lower health-check traffic to Supabase💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.