feat(api-lite): explorer API from the block recorder or RPC with a sqlite index, no Postgres - #699
Conversation
…l-core RPC with a sqlite index, no Postgres
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryMedium Risk Overview api-lite gets a multi-stage Alpine image: monorepo build, minimal runtime
CI: PR workflow job Reviewed by Cursor Bugbot for commit 8d535ae. Bugbot is set up for automated code reviews on this repo. Configure here. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
… events in sqlite
… formatting - rate-limit key from X-Forwarded-For behind the Railway edge - Cosmos poller keeps its cursor on non-2xx responses - staking event pagination orders on (block_height, _id) - withdraw proof cache no longer stores failed responses - DataCache bounded to 1000 entries with LRU eviction - hot-key decay timer catches sqlite errors - per-minute TPS buckets, GET /api/charts with Cache-Control and nginx proxy_cache, home charts fetch it first - fee chart USD values use convertToUsd formatting and full-hour buckets - biome format on loadtest scripts
- empty environment variables are treated as unset by loadConfig - nginx listens on IPv6 as well - rate limit keys on the TCP peer by default; RATE_LIMIT_KEY=$xff_last opts into the last X-Forwarded-For hop behind a proxy
… hop, which its edge rewrites
- assets, contracts and predicates are exempt from index retention; asset lookups fall back to fuel-core assetDetails and the verified-assets registry; registry assets are seeded on boot - suspicious flag derived from subId collisions with the registry - USD values for base-asset fees, inputs, outputs, mints and balances from the ETH price - exact 1-based counts on account, contract and block transaction lists and on minted assets - transactionsByOwner no longer falls back to fuel-core on forward paging past the index; fallback block rendering is parallel and bounded - search returns null and logs on malformed input or lookup failures - dashboard and chart series serve numbers, timestamps in milliseconds, per-bucket max tps and gas - utxos cap 2500 like production - scheduled uptime probe workflow for the explorer
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| env: | ||
| EXPLORER_URL: ${{ vars.API_LITE_EXPLORER_URL || 'https://explorer-production-f355.up.railway.app' }} | ||
| MAX_LAG_BLOCKS: "120" | ||
| steps: | ||
| - name: Health and index lag | ||
| run: | | ||
| body=$(curl -fsS --max-time 20 "$EXPLORER_URL/api/health") | ||
| echo "$body" | ||
| ok=$(echo "$body" | jq -r '.ok') | ||
| lag=$(echo "$body" | jq -r '.lag') | ||
| test "$ok" = "true" | ||
| test "$lag" -le "$MAX_LAG_BLOCKS" | ||
| - name: Home page | ||
| run: curl -fsS --max-time 20 -o /dev/null "$EXPLORER_URL/" | ||
| - name: Charts endpoint | ||
| run: | | ||
| curl -fsS --max-time 20 "$EXPLORER_URL/api/charts" | jq -e '.statistics and .tps' > /dev/null |
- Playwright suite under tests/api-lite: home, block, transaction, account, search and staking pages render real data from api-lite in RPC mode against the local node - playwright-api-lite.config.ts boots api-lite on 3004 and the vite preview on 3000 - tests-e2e-api-lite job in pr-tests.yml
| name: E2E Tests api-lite | ||
| runs-on: warp-ubuntu-latest-x64-4x | ||
| timeout-minutes: 14 | ||
| env: | ||
| TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }} | ||
| TURBO_TEAM: ${{ secrets.TURBO_TEAM }} | ||
| TURBO_REMOTE_ONLY: true | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v3 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Cache turbo build setup | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: .turbo | ||
| # Distinguish between a normal build and a E2E build, which disables code splitting for faster builds | ||
| key: ${{ runner.os }}-e2e-turbo-${{ hashFiles('pnpm-lock.yaml') }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-e2e-turbo- | ||
|
|
||
| - name: Setup Node | ||
| uses: FuelLabs/github-actions/setups/node@master | ||
| with: | ||
| node-version: 20.15.1 | ||
| pnpm-version: 9.10.0 | ||
|
|
||
| - name: Create .env file explorer | ||
| run: cp packages/app-explorer/.env.example packages/app-explorer/.env | ||
|
|
||
| - name: Build production version of explorer | ||
| env: | ||
| NODE_ENV: test | ||
| run: pnpm build:prod | ||
|
|
||
| - name: Setup Docker | ||
| uses: FuelLabs/github-actions/setups/docker@master | ||
| with: | ||
| username: ${{ github.repository_owner }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Start Test Node | ||
| timeout-minutes: 7 | ||
| run: pnpm node:start | ||
|
|
||
| - name: Install Playwright Browsers | ||
| run: pnpm exec playwright install --with-deps chromium | ||
|
|
||
| - name: Run E2E Tests | ||
| run: xvfb-run --auto-servernum -- pnpm test:e2e-api-lite | ||
| env: | ||
| PLAYWRIGHT_JSON_OUTPUT_NAME: playwright-report/api-lite/results.json | ||
| PLAYWRIGHT_FORCE_TTY: true | ||
| PLAYWRIGHT_FORCE_COLOR: true | ||
|
|
||
| - uses: actions/upload-artifact@v4 | ||
| if: always() | ||
| with: | ||
| name: playwright-report-api-lite | ||
| path: packages/e2e-tests/playwright-report/api-lite/ | ||
| retention-days: 30 | ||
|
|
||
| - name: Stop Test Node | ||
| run: pnpm node:stop |
… field is non-nullable
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
There are 5 total unresolved issues (including 3 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a9d2751. Configure here.
- fuel-core fallback pages number from the oldest end instead of reusing the newest ranks - a failed fallback render is backfilled from the remaining items instead of shrinking the page - list counts capped at 1001 and bounded to the contiguous index window - asset registry matches compare lowercase ids
- REST errors distinguish client from upstream: ValidationError maps to 400, a missing event to 404, everything else to 502 with a server-side log - index writes happen before the disk-cache write, so a crash between them re-fetches instead of leaving a pinned block unindexed - backfill clamps indexed_from to the retention floor when retention advanced it mid-fetch (real race found by the new regression test) - backfill gaps surfaced in /health - a 200 cosmos response without a tx_responses array no longer advances the cursor - long-lived viem client, single index.db path constant, decoder policies trimmed to the schema-backed four
Bugbot is paused — on-demand spend limit reachedBugbot uses usage-based billing for this team and has hit its on-demand spend limit. A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue. |
- block cache accounting is heap-aware: a measured 2.25x object-vs-JSON multiplier (applied as 2.5x) counts against the budget - 768 MB heap in the image with the memory math documented; the 960 MB compose tier overrides the cache to 48 MB - backfill batch size wired to config instead of a silent 20 - eviction under pinning terminates and logs once per minute - Railway restart policy retries 100 times instead of stranding the service after a crash loop
|
👍 |
2 participants
Summary
Adds api-lite, a small service that serves the existing explorer GraphQL schema without Postgres so the explorer can run on a single cheap host. Blocks come from the fuel-core S3 block recorder (decoded from protobuf and checked field by field against fuel-core on real mainnet blocks) or straight from fuel-core RPC, selected by config. A sqlite index covers a retention window for account history, search, charts, predicates and minted assets; older account history falls back to fuel-core. The frontend is unchanged apart from the homepage hero, whose tiles now load independently.
Changes
packages/api-lite: graphql-yoga server on the production schema files, resolvers backed by an S3 or RPC block source, a memory and disk block cache, and a sqlite index with retentiontransactionsByOwnerfallback and cursor continuation beyond ituseFuelExplorerStatussplit into three hooks so blocks, rolling stats and charts render as each arrives; newrollingStatssdk querydocker/vps: two Dockerfiles, compose for a 1 CPU / 1 GB budget, nginx envsubst template, production compose override, Railway configs,deploy.mdwith droplet and Railway steps, testnet envdocs/superpowers