ECO-251: Improve developers.uniswap.org agent readiness#1127
ECO-251: Improve developers.uniswap.org agent readiness#1127wkoutre wants to merge 2 commits intoUniswap:mainfrom
Conversation
Cloudflare's agent-readiness check rated developers.uniswap.org as Level 0 / Not Ready. This PR raises the score by aligning the canonical site URL, expanding the AI-bot allowlist, rewriting the top-level llms.txt to follow the llmstxt.org spec, and generating a site-wide /llms-full.txt at build time. - docusaurus.config.ts: set siteConfig.url to the actual served domain (developers.uniswap.org). Sitemap entries, canonical <link> tags, and OpenGraph URLs now match the deployed origin instead of the legacy docs.uniswap.org redirect source. - docusaurus.config.ts: add a customLLMFiles entry to the existing docusaurus-plugin-llms instance that emits /llms-full.txt with full markdown content for current docs (v4 + SDKs + APIs + concepts). Build verified — file lands at /llms-full.txt (~1.3MB) alongside the existing /v4-llms.txt and /v4-llms-full.txt. - static/llms.txt: rewrite to follow the llmstxt.org spec — H1 title, blockquote summary, and H2 sections of markdown link lists. URLs updated to developers.uniswap.org. Cross-links to /llms-full.txt and /v4-llms.txt for agents that want the full content. - static/robots.txt: update sitemap URL to https://developers.uniswap.org/sitemap.xml. Expand the AI-crawler allowlist with the major modern bots (OAI-SearchBot, ChatGPT-User, Claude-Web, anthropic-ai, Google-Extended, GoogleOther, Applebot, Applebot-Extended, PerplexityBot, Perplexity-User, CCBot, cohere-ai, Meta-ExternalAgent, Meta-ExternalFetcher, Amazonbot, DuckAssistBot, MistralAI-User, etc.). Block list extended with Bytespider. No theme or content changes outside the static/* files and docusaurus.config.ts. Build passes.
|
@wkoutre is attempting to deploy a commit to the Uniswap Team on Vercel. A member of the Team first needs to authorize it. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e7d423769f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ## Builder support | ||
|
|
||
| - [Get funded](https://developers.uniswap.org/builder-support/get-funded): Grants and funding programs from the Uniswap Foundation. | ||
| - [Hooks ecosystem](https://developers.uniswap.org/builder-support/hooks-ecosystem): Resources for hook developers, including audits and reusable templates. |
There was a problem hiding this comment.
The new Hooks ecosystem entry links to /builder-support/hooks-ecosystem, but this route does not exist in the repo (the docs/builder-support section only contains get-funded and security-resources). As a result, agents following llms.txt will hit a 404 for one of the curated core links, which degrades the file’s reliability as a discovery index.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed. Replaced the broken /builder-support/hooks-ecosystem link with /builder-support/security-resources, which is the existing page covering audit subsidies and safe harbor protections for builders.
…urces The /builder-support/hooks-ecosystem route does not exist in the repo. Replaced with the existing /builder-support/security-resources page, which covers audit subsidies and safe harbor protections for builders. Resolves Codex inline comment on PR Uniswap#1127.
|
The Vercel CI failure is a team authorization issue, not a code problem. The check message is: "@wkoutre is attempting to deploy a commit to the Uniswap Team on Vercel. A member of the Team first needs to authorize it." A Uniswap Vercel team admin needs to authorize the deployment — no code change can unblock this. |
1 participant
Summary
Cloudflare's agent-readiness check rated
developers.uniswap.orgas Level 0 / Not Ready. This PR raises the score with a focused set of static-config changes:siteConfig.urlupdated fromhttps://docs.uniswap.org(legacy redirect source) tohttps://developers.uniswap.org(actual served origin). Sitemap absolute URLs,<link rel="canonical">tags, and OpenGraph URLs now all match the deployed domain./llms.txtrewritten to spec — follows the llmstxt.org format strictly: H1 title, blockquote summary, and H2 sections of markdown link lists. URLs updated todevelopers.uniswap.org. Cross-links to/llms-full.txtand/v4-llms.txt./llms-full.txtgenerated at build time — added acustomLLMFilesentry to the existingdocusaurus-plugin-llmsplugin instance. Emits a ~1.3 MB plain-text concatenation of current docs (v4 + SDKs + APIs + concepts) at the site root. Legacy/v4-llms.txtand/v4-llms-full.txtcontinue to be generated.robots.txtmodernized — sitemap reference points todevelopers.uniswap.org/sitemap.xml. Allowlist expanded to cover the modern AI-bot set (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-Web, anthropic-ai, Google-Extended, GoogleOther, Applebot, Applebot-Extended, PerplexityBot, Perplexity-User, CCBot, cohere-ai, Meta-ExternalAgent, Meta-ExternalFetcher, Amazonbot, DuckAssistBot, MistralAI-User, xAI, Diffbot, etc.). Block list extended with Bytespider.Why these specifically
Cloudflare's agent-readiness rubric weighs the following highest: a permissive and well-formed
robots.txt, a present and parseable/llms.txt, an/llms-full.txtcompanion, a discoverablesitemap.xml, and consistent canonical URLs. This PR addresses each. JSON-LD structured data is the next-cheapest follow-up if Cloudflare's checker still flags gaps after this lands — out of scope here because it requires theme swizzling and per-page MDX edits (broader surface area, longer review).Verification
NODE_OPTIONS='--openssl-legacy-provider' yarn buildsucceeds. Pre-existing broken-anchor warnings in/api/subgraph/subgraphs-devs/*,/sdk/v2/*,/sdk/v3/*are unrelated and present onmain.build/:build/sitemap.xml→<loc>https://developers.uniswap.org/...</loc>for all entriesbuild/index.html→<link rel="canonical" href="https://developers.uniswap.org/">build/contracts/v4/overview/index.html→<link rel="canonical" href="https://developers.uniswap.org/contracts/v4/overview">build/llms.txt(7 KB, spec format) andbuild/llms-full.txt(1.3 MB, plugin-generated) both presentbuild/robots.txtadvertises the correct sitemap URLOut of scope
editUrlin the docs preset still points athttps://github.com/uniswap/uniswap-docs/tree/main/(a repo that doesn't exist; should beUniswap/docs). Pre-existing, unrelated to agent-readiness — recommend a separate PR.Linear
ECO-251
Test plan
https://developers.uniswap.organd confirm score moves up from Level 0https://developers.uniswap.org/llms.txtserves the new spec-format indexhttps://developers.uniswap.org/llms-full.txtserves the full content file (~1.3 MB)https://developers.uniswap.org/robots.txtadvertises the correct sitemap URL and modern AI-bot allowlisthttps://developers.uniswap.org/sitemap.xmlURLs all use thedevelopers.uniswap.orghost<link rel="canonical">tags in production HTML🤖 Autonomously generated by the Linear Task Worker skill via Claude Code. Review carefully before merging.