feat: add scrapfly-webhooks skill#55
Merged
Merged
Conversation
4 tasks
Adds a complete provider skill covering Scrapfly's HMAC-SHA256 webhook signature scheme (uppercase hex over raw body, X-Scrapfly-Webhook-Signature header), routing by X-Scrapfly-Webhook-Resource-Type (scrape, extraction, screenshot), and Crawler API lifecycle events. Includes Express, Next.js, and FastAPI examples with tests that generate real Scrapfly signatures.
…viders.yaml - Correct X-Scrapfly-Webhook-Env header values (test/live, not production) - Document actual payload envelope (context.webhook, context.job) - Warn that payload echoes the signing secret at context.webhook.secret - Distinguish default vs opt-in crawler events - Add retry schedule and 100-failure auto-disable behavior - Note paid-plan requirement (FREE plan has webhook queue size 0) - Read scrape URL from payload.result.url (not the webhook context overlay) - Add Scrapfly row to README Provider Skills table - Add scrapfly entry to providers.yaml (docs URLs, notes, testScenario) https://claude.ai/code/session_01NNTgQRJss1V7gyzzJ9rjnB
a337f50 to
96f2c4b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a complete
scrapfly-webhooksprovider skill. Scrapfly is a web-scraping API platform with three products — Scrape API, Extraction API, and Screenshot API — that share a single webhook system. One registered webhook URL receives deliveries from all three; the product is distinguished by anX-Scrapfly-Webhook-Resource-Typeheader.What's included
skills/scrapfly-webhooks/SKILL.md— entry point with frontmatterskills/scrapfly-webhooks/references/— overview, setup, verification (raw-body HMAC-SHA256, uppercase + lowercase header support, idempotency-by-job-id recommendation)skills/scrapfly-webhooks/examples/— Express, Next.js App Router, FastAPI handlers that dispatch onX-Scrapfly-Webhook-Resource-Type(scrape / extraction / screenshot)providers.yamlandREADME.mdintegration entriesNotes
webhook_name=…query param; the destination URL cannot be passed per-call and there is no API for managing webhooks programmatically (dashboard only).X-Scrapfly-Webhook-Signature(uppercase hex) andX-Scrapfly-Webhook-Signature-Lowercase(lowercase hex). The skill's verifier accepts either using constant-time comparison.X-Scrapfly-Webhook-Job-Idinstead of inventing at=…window.context.webhook.secretfield echoes the signing secret. The skill warns handlers to never log or echo it.constructEvent-style helper. The skill uses stdlib HMAC (crypto.createHmacin Node,hmac/hashlibin Python) — no third-party HMAC library introduced.Test plan
cd skills/scrapfly-webhooks/examples/express && npm testcd skills/scrapfly-webhooks/examples/nextjs && npm testcd skills/scrapfly-webhooks/examples/fastapi && pytest test_webhook.py -vX-Scrapfly-Webhook-Resource-Typeacross the three productsnpx hookdeck-cli listen 3000 scrapfly --path /webhooks/scrapflyGeneration details
./scripts/generate-skills.sh generate scrapfly --config providers.yaml --model claude-opus-4-7X-Scrapfly-Webhook-Envenum values (test/live, notproduction)context.webhook,context.job) and the secret-leak warningpayload.result.url, not the webhook context overlay)https://claude.ai/code/session_01NNTgQRJss1V7gyzzJ9rjnB
Generated by Claude Code