Hubs StoryScraper — WebPage to Plain Text Converter

Fetches any webpage server-side (bypassing CORS), strips boilerplate, and returns clean readable plain text — ideal for feeding content into LLMs or running content audits.

Stack

• Backend: Node.js, Express, Axios, Cheerio
• Frontend: Single-page HTML with Tailwind CSS CDN + Vanilla JS

Run Locally

npm install
npm start

Open http://localhost:3000.

For development with auto-reload:

npm run dev

Deploy to Render

1. Push this repo to GitHub.
2. Create a new Web Service on Render.
3. Set Build Command: npm install
4. Set Start Command: npm start
5. Set Environment Variable: PORT=3000 (Render sets this automatically).

Deploy to Heroku

heroku create your-app-name
git push heroku main

Deploy to Vercel

Vercel works best for serverless. Convert server.js to api/convert.js as a Vercel serverless function, then deploy with vercel --prod. The public/ directory is served as static files automatically.

Features

• Server-side fetching with a real browser User-Agent (avoids basic bot blocks)
• Strips <script>, <style>, <noscript>, <iframe>, <svg>, forms, buttons, etc.
• Optional boilerplate removal: strips <header>, <footer>, <nav>, <aside>, and common nav/sidebar divs by class/ID pattern
• Prefers <main> / <article> content when boilerplate exclusion is enabled
• Whitespace normalization with preserved paragraph breaks
• Word count display
• Copy ALL text or Copy Selected text with visual feedback
• Clean error handling for invalid URLs, timeouts, and server errors