Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,8 +161,8 @@ The ONLY viable path forward for a basic read-only frontend is using Reddit's pu
2. **Initial load.** When the app mounts or the active feed changes (different subreddit, or different sort: Hot / New / Top), fetch the first page of posts immediately.
3. **No pagination.** Posts are fetched once per feed selection — one request returning up to 100 posts (the RSS hard cap). No infinite scroll or "load more" mechanism exists. Do not re-introduce pagination; the single-request approach is intentional for rate limiting.
4. **Comments on demand.** Comments are fetched *only* when the user selects a specific post. Never fetch comments speculatively or in the background. No post is auto-selected on load to avoid fetching posts and comments simultaneously, preventing rate limit bursts. Any rate limits encountered are handled by `fetchWithRetry` which implements an exponential backoff strategy on 429s (max 5 attempts) and respects the `Retry-After` header if present before surfacing an error.
5. **Server-side cache.** The API proxy (`app/api/reddit/route.ts`) uses Next.js native Data Cache (`fetchOptions.next = { revalidate: false }`) with an **indefinite TTL** (FIFO eviction managed by the Next.js cache). This ensures the cache securely persists across Vercel serverless invocations — Reddit is only hit once per unique URL, ever, until the entry is evicted or manually invalidated. Do not add a fixed TTL or switch to a custom in-memory or `fs` disk cache. Do not set `Cache-Control` headers on successful responses.
6. **No background polling.** Do not auto-refresh feeds on a timer. The user can manually refresh using the refresh button (which sends `forceRefresh=true`, causing the proxy to fetch with `cache: "no-store"` — this bypasses the Next.js Data Cache and **replaces** the old cache entry with fresh content), or by re-selecting the feed / changing the sort tab (which serves from cache).
5. **Server-side cache.** The API proxy (`app/api/reddit/route.ts`) uses a standalone `fetchRedditRSS()` function annotated with `"use cache: remote"` (Next.js 16 Cache Components). This stores fetched RSS data in Vercel's **persistent remote Data Cache**, shared across all users and serverless invocations. The `cacheComponents: true` flag is enabled in `next.config.ts`. Each unique URL gets its own cache entry tagged via `cacheTag()`, with `cacheLife("max")` for near-indefinite TTL. **Do not** revert to `unstable_cache` (deprecated in Next.js 16, uses ephemeral in-memory storage that doesn't survive serverless cold starts), `fetch()` with `force-cache` (unreliable in Route Handlers since they are dynamic routes), or any custom in-memory / `fs` disk cache. Do not set `Cache-Control` headers on successful responses.
6. **No background polling.** Do not auto-refresh feeds on a timer. The user can manually refresh using the refresh button (which sends `forceRefresh=true`, causing the proxy to call `revalidateTag(tag, { expire: 0 })` to immediately expire the cache entry — the subsequent `fetchRedditRSS()` call then misses cache and fetches fresh data from Reddit), or by re-selecting the feed / changing the sort tab (which serves from cache).
7. **Backend Hardening.** The API proxy enforces an IP-based rate limit to prevent abuse and a strict 30-second timeout on upstream Reddit fetches (using `AbortSignal.timeout` or `AbortController`) to prevent hanging serverless invocations.

### RSS Limits — What Actually Matters
Expand Down
94 changes: 65 additions & 29 deletions app/api/reddit/route.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { NextRequest, NextResponse } from "next/server";
import { revalidateTag } from "next/cache";

/* eslint-disable @typescript-eslint/no-explicit-any */

Expand All @@ -20,6 +21,60 @@ const RATE_LIMIT_WINDOW_MS = 10 * 60 * 1000; // 10 minutes
const lastRefreshTimes = new Map<string, number>();
const REFRESH_COOLDOWN_MS = 30 * 1000; // 30 seconds

// ---------------------------------------------------------------------------
// Shared fetch headers for all upstream Reddit requests
// ---------------------------------------------------------------------------
const REDDIT_HEADERS = {
"User-Agent":
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36",
Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "none",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": "1",
};

// ---------------------------------------------------------------------------
// Cached fetcher — uses "use cache: remote" so the result is stored in
// Vercel's persistent remote Data Cache and shared across all users and
// serverless invocations. This replaces the deprecated unstable_cache
// approach which only stored in ephemeral in-memory cache that was destroyed
// between serverless invocations.
//
// cacheLife("max") = indefinite TTL (revalidate: 30 days, expire: 1 year).
// cacheTag() per URL enables targeted invalidation via revalidateTag().
// ---------------------------------------------------------------------------
async function fetchRedditRSS(url: string): Promise<string> {
"use cache: remote";

const { cacheLife, cacheTag } = await import("next/cache");
cacheLife("max");
cacheTag(cacheTagForUrl(url));

const res = await fetch(url, { headers: REDDIT_HEADERS, cache: "no-store" });
const text = await res.text();
if (!res.ok) {
throw new Error(`Reddit returned ${res.status}: ${text.slice(0, 200)}`);
}
return text;
}

// ---------------------------------------------------------------------------
// Deterministic cache tag from a Reddit RSS URL.
// Tags are limited to 256 chars, so we hash long URLs.
// ---------------------------------------------------------------------------
function cacheTagForUrl(url: string): string {
// Simple deterministic hash: prefix + condensed URL
const condensed = url
.replace(/^https?:\/\//, "")
.replace(/[^a-zA-Z0-9]/g, "_")
.slice(0, 200);
return `rss_${condensed}`;
}

export async function GET(request: NextRequest) {
// 1. Rate Limiting Check
const ip = request.headers.get("x-forwarded-for") || "unknown-ip";
Expand Down Expand Up @@ -79,7 +134,7 @@ export async function GET(request: NextRequest) {

if (lastRefresh && (now - lastRefresh < REFRESH_COOLDOWN_MS)) {
console.warn(`[Proxy] Cooldown active for ${urlString}. Ignoring forceRefresh.`);
// Strip the forceRefresh flag to serve the Next.js cache instead
// Strip the forceRefresh flag to serve the cache instead
forceRefresh = false;
} else {
// Update the last refresh time
Expand All @@ -88,37 +143,18 @@ export async function GET(request: NextRequest) {
}

try {
const fetchOptions: RequestInit = {
headers: {
"User-Agent":
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36",
Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "none",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": "1",
},
};

if (forceRefresh) {
fetchOptions.cache = "no-store";
} else {
// In Next.js 15+, fetch requests are uncached by default.
// We must explicitly use cache: "force-cache" to use the Data Cache.
fetchOptions.cache = "force-cache";
// Invalidate the remote cache entry for this URL, then re-fetch.
// revalidateTag with { expire: 0 } immediately expires the entry so
// the next call to fetchRedditRSS will miss cache and fetch fresh.
const tag = cacheTagForUrl(targetUrl.toString());
revalidateTag(tag, { expire: 0 });
}

const res = await fetch(targetUrl.toString(), fetchOptions);

const text = await res.text();

if (!res.ok) {
console.error("[Proxy] Reddit returned:", res.status, text.slice(0, 200));
return new NextResponse(text, { status: res.status });
}
// Serve from / populate the persistent remote Data Cache.
// If forceRefresh just expired the tag above, this call will miss
// the cache and fetch fresh data from Reddit, then cache it.
const text = await fetchRedditRSS(targetUrl.toString());

return new NextResponse(text, {
status: 200,
Expand Down
6 changes: 5 additions & 1 deletion app/layout.tsx
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import type { Metadata } from "next";
import { SpeedInsights } from "@vercel/speed-insights/next";
import "./outlook.css";

export const metadata: Metadata = {
Expand All @@ -13,7 +14,10 @@ export default function RootLayout({
}>) {
return (
<html lang="en">
<body>{children}</body>
<body>
{children}
<SpeedInsights />
</body>
</html>
);
}
4 changes: 3 additions & 1 deletion next.config.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import type { NextConfig } from "next";

const nextConfig: NextConfig = {};
const nextConfig: NextConfig = {
cacheComponents: true,
};

export default nextConfig;
39 changes: 39 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2",
"@radix-ui/react-dialog": "^1.1.17",
"@vercel/speed-insights": "^2.0.0",
"lucide-react": "^1.17.0",
"next": "16.2.7",
"react": "19.2.4",
Expand Down