feat(http): add jitter to retry backoff

[k4its1t]

Summary

• apply full jitter after the exponential retry delay is capped, spreading concurrent retries across 0..delay
• expose ScraperConfig.retry_jitter=True with an opt-out that preserves the previous deterministic schedule and positional config compatibility
• cover jitter spread and bounds, cap ordering, opt-out behavior, and the shared sync/async retry paths
• document the new setting and keep explicit server Retry-After values unchanged

Closes #163

Validation

• pytest tests/ -q on Python 3.9: 514 passed, 6 skipped, 19 deselected
• full test suite on Python 3.13: 535 passed, 2 skipped, 19 deselected
• focused config/sync/async tests: 75 passed
• ruff check src/ tests/
• ruff format --check src/ tests/
• Python 3.9 compileall for src and tests
• python -m build (sdist and wheel)

mypy src/pyscrappy/ is not currently green on the unchanged project baseline: current mypy no longer accepts the repository's Python 3.9 target, and the suite also reports existing errors in unrelated code. The changed config file introduces no reported error; the reports in http.py are on pre-existing stealth-response branches.

AI assistance

OpenAI Codex assisted with issue screening, implementation, tests, and PR preparation. I reviewed the final diff and validation results.

[Copilot]

🟢 Approval recommended

No unresolved review comments were supplied.

Pull request overview

Adds configurable full jitter to capped exponential HTTP retry backoff, with deterministic opt-out behavior.

Changes:

• Adds retry_jitter, enabled by default.
• Applies jitter across sync and async retry paths.
• Adds tests and documentation.

File summaries

File	Summary
tests/test_core/test_http.py	Tests jitter bounds and sync retries.
tests/test_core/test_config.py	Tests jitter configuration.
tests/test_core/test_async_http.py	Tests async jittered retries.
src/pyscrappy/core/http.py	Implements capped full-jitter delays.
src/pyscrappy/core/config.py	Adds the retry jitter setting.
README.md	Documents the configuration option.
CHANGELOG.md	Records the retry behavior change.

Review details

• Files reviewed: 7/7 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[vedaant00]

Nice work, and welcome @k4its1t. I checked this out and it's exactly right: full jitter applied after the cap (correct order), in the shared backoff_delay so sync and async both get it, with a clean deterministic opt-out. Verified the spread (0..capped-delay over 1000 samples) and that retry_jitter=False keeps the old exact schedule. Full suite green (540 passed), lint clean, and it sits cleanly on top of 1.5.7. Server-set Retry-After values are correctly left untouched. Merging. Thanks for the thorough PR and validation notes.