Reddit Post Scraper

Reddit Post Scraper for extracting post titles, scores, upvote ratios, selftext and nested comment threads from Reddit.com. This repo has a free Reddit post scraping script you can run right now, and a Reddit post data API that returns one post as 13 fields of structured JSON with its comment tree nested underneath.

This repo reads public posts in large public subreddits. Every capture, example and screenshot on this page targets such a post. Comment bodies and the pseudonyms attached to them are part of what a post page publishes, so they are in the JSON; they are not rendered into any image here, and nothing in this repo is built around a particular person's account.

Last updated: 2026-07-20. Working against Reddit.com as of July 2026, and re-verified whenever Reddit changes their markup.

This repo covers one endpoint in depth. For subreddit listings, search and public user feeds, see the Reddit Scraper repo.

Every JSON block on this page was captured from the live API on 2026-07-20. Arrays are trimmed where marked and each block says exactly what was cut; every field shown is verbatim, nulls included. Full uncut samples are committed in reddit_post_scraper_api_data/, so you can diff this page against them. Every code example calls the actual API and is runnable from reddit_post_scraper_api_codes/.

pip install requests
export CHOCODATA_API_KEY="your_key"     # free: 1,000 requests, one-time, no card
python reddit_post_scraper_api_codes/post.py "https://www.reddit.com/r/askscience/comments/627akk/do_giraffes_get_struck_by_lightning_more_often/"

Those three lines return this, live from Reddit.com (7 of the 13 post fields, verbatim; the full object is below):

{
  "id": "t3_627akk",
  "title": "Do giraffes get struck by lightning more often than other animals?",
  "score": 32564,
  "upvote_ratio": 0.9223354752935963,
  "num_comments": 924,
  "created": "2017-03-29T15:25:25.729000+0000",
  "domain": "self.askscience"
}

...with the comment thread nested underneath it, scores and all:

That is the whole point of this repo. The rest of this page is the free path, what Reddit does to it, and the endpoint reference.

---

Contents

• Free Reddit Post Scraper
• Avoid getting blocked when scraping Reddit posts
  • Using the Chocodata Reddit Post Scraper API
• Reddit Post Scraper API reference
  • Quickstart · Authentication · Parameters · Errors · Rate limits and concurrency
  • Post: title, score, upvote ratio, selftext and the nested comment tree
• Export a Reddit thread's comments to CSV
• Measured latency
• License

---

Free Reddit Post Scraper

Reddit renders a JSON view of any post if you append .json to the permalink. No key, no OAuth app, no cost. It is the first thing everybody tries, so it is the first thing in this repo:

python free_scraper/reddit_post_free_scraper.py "https://www.reddit.com/r/askscience/comments/627akk/do_giraffes_get_struck_by_lightning_more_often/"

Source: free_scraper/reddit_post_free_scraper.py. It requests the post's .json view, walks the two listings Reddit answers with (listing 0 holds the post, listing 1 holds the comment forest) and emits the post plus its top-level comments.

It parses first and only reports a refusal when the post is genuinely absent. A page can carry the word "blocked" in its own JavaScript and still hand you a perfectly good post, so string-matching the body before parsing it would manufacture a failure that never happened. If a post comes back, the script prints it and exits 0.

After running the command, your terminal should look something like this:

No post comes back. The next section is what we measured.

Avoid getting blocked when scraping Reddit posts

We ran that script from a residential connection while writing this page. Across 7 requests to reddit.com/r/{sub}/comments/{id}/.json, covering two different posts, three user agents and both the www and old hosts, every one came back the same:

What we measured	Value
HTTP status	403
Response size	189,908 bytes
Content-Type	text/html (from a URL ending in .json)
Content-Encoding	none
<title>	none: the body opens straight into <body class=theme-beta>
JSON parse	failed
posts returned	0, on 7 of 7

Every request in that table is recorded in free_scraper_runs.json. Note what is not happening: Reddit is not pretending. There is no 200-that-is-really-a-block and no fake post page. You get a straight 403. The traps are elsewhere.

What bites you	Why	What it costs you
HTML from a .json URL	The 403 body arrives as text/html, 190 KB of it, with no <title> and no Content-Encoding. A naive r.json() raises rather than returning an empty dict.	Your handler catches a JSONDecodeError and logs "bad JSON", so you go debugging your parser instead of your access.
Changing the User-Agent changed nothing	We sent the python-requests default, a desktop Chrome string, and a descriptive script UA. All three got the same 403 and the same 189,908 bytes, on both hosts.	You cannot header your way in. Reddit's own terms say to use the OAuth token and not to "misrepresent or mask either the user agent or OAuth identity" (redditinc.com/policies/data-api-terms, section 2.8, retrieved 2026-07-20).
The open surface has no vote data	.rss on the same post, same connection, same day, answered 200 with 145,157 bytes. It carries the post and its comments, and no score, no upvote_ratio and no comment counts anywhere in it.	You ship a "working" scraper that silently cannot answer the only question you had: did this thread land or not?
A post id that does not exist still returns a body	Reddit answers an unknown post id with a page rather than a 404, so a parser that tests for parse failure cannot tell "this post is gone" from "my parser broke". We measured the same thing downstream: three bogus ids each came back 200 with an all-null post object and num_comments: 0.	Your retry loop hammers something that will never resolve. Test the payload, not the status.
Removed and deleted comments are strings, not nulls	Reddit fills the gap with literal text. On our sample post, 4 of the 14 comments returned read "Comment removed by moderator" or "Comment deleted by user", with author.username set to "[deleted]".	Anything doing sentiment or topic analysis happily scores "Comment removed by moderator" as user text. thread_to_csv.py drops both strings and reports how many it dropped.
Limits are Reddit's call, not a published constant	"Reddit may set and enforce limits on your use of the Data APIs ... in our sole discretion" (redditinc.com/policies/data-api-terms, section 2.9, retrieved 2026-07-20).	Capacity you cannot plan against, and terms that can move under you.

One correction worth making, because it gets repeated: the popular free Reddit tools on GitHub do not all meet that 403. JosephLai241/URS (1,008 stars, last pushed 2026-07-11, actively maintained) is built on PRAW and talks to Reddit's official API through an OAuth app you register yourself, so it never goes through the door above. What meets the 403 is the no-key route the script in this repo takes, which is what we measured.

---

Using the Chocodata Reddit Post Scraper API

The managed option, and the one this repo is built around. The Chocodata Reddit Post Scraper API returns one post as parsed JSON with real vote data, nests the comment thread into a tree with parent_id and depth on every node, takes either a post URL or a bare post id, and runs at your plan's concurrency with no OAuth app to register. It reads the same surface Reddit's own web client reads, which is the one that carries scores. Free for the first 1,000 requests.

---

Reddit Post Scraper API reference

Below is the Reddit Post Scraper API reference to get you started: authentication, the parameters, error handling, concurrency, and the endpoint itself.

Quickstart

curl "https://api.chocodata.com/api/v1/reddit/post?api_key=YOUR_KEY&url=https://www.reddit.com/r/askscience/comments/627akk/do_giraffes_get_struck_by_lightning_more_often/"

import requests

r = requests.get(
    "https://api.chocodata.com/api/v1/reddit/post",
    params={"api_key": "YOUR_KEY",
            "url": "https://www.reddit.com/r/askscience/comments/627akk/"
                   "do_giraffes_get_struck_by_lightning_more_often/"},
    timeout=90,
)
d = r.json()
print(d["post"]["title"], d["post"]["score"], d["comments_returned"])
# Do giraffes get struck by lightning more often than other animals? 32564 14

After running the command, your terminal should look something like this:

Get a key at chocodata.com (1,000 requests, one-time, no card).

Authentication

Pass api_key as a query parameter on every request. That is the whole auth model. There is no OAuth handshake, no app registration, no token refresh, and no Reddit account of yours involved anywhere.

Parameters

Param	Type	Required	Default	Description
api_key	string	yes	-	Your Chocodata API key.
url	string (URL)	one of	-	Any reddit.com/.../comments/{id}/... post URL. Short and share links resolve too.
post_id	string	one of	-	The base-36 post id, with or without the t3_ prefix. Needs subreddit alongside it.
subreddit	string	with post_id	-	The subreddit the post lives in, case sensitive. A leading r/ is accepted and stripped. Non-canonical casing still answers 200, but with title, score, author and created all null: on 7 of 7 such calls across two subreddits we got that null post object back, while canonical casing returned the full object on 9 of 10. Take the casing from a permalink rather than typing it, and test post.title rather than the status code.
sort	enum	no	top	Comment sort: top, new or controversial. Changes which slice of the thread comes back, not just its order (see the field notes).

Each request costs 5 credits (= 1 request). Responses are billed only on success (2xx).

Errors

Status	error code	Meaning	Billed	What to do
400	invalid_params	Neither post_id nor url was sent. The body names the failing rule.	no	Fix the query string.
401	INVALID_API_KEY	Key missing, unrecognised, or revoked.	no	Check api_key. Get one at chocodata.com.
402	INSUFFICIENT_CREDITS	Balance exhausted.	no	Top up or upgrade at chocodata.com.
404	item_not_found	No post resolved from the input: a URL that is not a Reddit post, or a post_id with no subreddit. retryable: false.	no	Fix the input. Retrying will not help.
429	RATE_LIMITED	Over your plan's concurrency.	no	Back off and retry; see Rate limits.
502	-	Reddit did not serve this post on this request. retryable: true.	no	Retry shortly.

Two response shapes exist. Auth and billing errors nest under error.code in uppercase; scrape-layer errors are flat with a lowercase error string, a plain-English message and a retryable boolean. That boolean is the one to wire up, because it is the difference between "stop" and "try again": item_not_found carries retryable: false, so it should end your loop.

A bad key, verbatim:

{"error":{"code":"INVALID_API_KEY","message":"Api key not recognised."}}

Neither identifier sent, verbatim:

{
  "error": "invalid_params",
  "issues": [
    {
      "code": "custom",
      "message": "reddit.post requires either post_id or url",
      "path": []
    }
  ]
}

A URL that is not a Reddit post, verbatim. We sent url=https://example.com/ as a control, and the endpoint declined to fetch it rather than returning example.com under a Reddit-shaped schema:

{
  "error": "item_not_found",
  "message": "reddit.post: could not resolve a post id from the input. Pass a full reddit post URL, or post_id (+ subreddit).",
  "retryable": false
}

All four captured bodies, with their requests and statuses, are in errors.json. The scripts in this repo turn each of the statuses above into an actionable line, so a typo'd key does not hand you a stack trace:

Rate limits and concurrency

Two limits apply, and they are enforced independently.

Limit	Value
Requests per key	120 per 60 seconds (sliding window)
Concurrent requests, Free	10
Concurrent requests, Vibe	30
Concurrent requests, Pro	50
Concurrent requests, Custom	100 to 500+

Exceed either and you get 429, not a queue. The endpoint is a synchronous GET: there is no webhook, callback or async job to poll. The examples use timeout=90 because a request that has to be re-attempted upstream takes longer than the median (see Measured latency).

Sizing: the 120 requests/60s window is the binding constraint, so a single key sustains roughly 2 requests per second regardless of plan; concurrency decides how many can be in flight inside that window. Fan out with a thread pool sized to stay under both:

from concurrent.futures import ThreadPoolExecutor
import requests

def one(url):
    r = requests.get("https://api.chocodata.com/api/v1/reddit/post",
                     params={"api_key": KEY, "url": url}, timeout=90)
    return url, (r.json()["post"]["score"] if r.ok else None)

with ThreadPoolExecutor(max_workers=10) as pool:   # <= your plan's concurrency
    for url, score in pool.map(one, post_urls):
        print(score, url)

---

Post: title, score, upvote ratio, selftext and the nested comment tree

One Reddit post by URL or id: the post object with real vote data, and its comment thread nested underneath as a tree.

Param	Type	Required	Default	Description
url	string (URL)	one of	-	A reddit.com/.../comments/{id}/... post URL.
post_id + subreddit	string	one of	-	The base-36 id plus its subreddit, casing as Reddit spells it.
sort	enum	no	top	top, new or controversial.

curl "https://api.chocodata.com/api/v1/reddit/post?api_key=YOUR_KEY&url=https://www.reddit.com/r/askscience/comments/627akk/do_giraffes_get_struck_by_lightning_more_often/"

Real response, the post object in full. All 13 fields verbatim, nulls included (full sample with the comment tree):

{
  "id": "t3_627akk",
  "title": "Do giraffes get struck by lightning more often than other animals?",
  "author": {
    "username": "[deleted]",
    "id": null
  },
  "score": 32564,
  "upvote_ratio": 0.9223354752935963,
  "num_comments": 924,
  "created": "2017-03-29T15:25:25.729000+0000",
  "body": null,
  "permalink": "https://www.reddit.com/r/askscience/comments/627akk/do_giraffes_get_struck_by_lightning_more_often/",
  "external_url": null,
  "domain": "self.askscience",
  "is_locked": null,
  "is_removed": null
}

upvote_ratio is the field people come for and the one the post page never gives you a number for. A score of 32,564 at 0.922 is a thread the room agreed on; the same score at 0.55 is a fight. You cannot tell those apart from the score alone, and Reddit renders only a rounded percentage on the page.

And the comment tree, cut to 1 of the 8 top-level comments, with 1 of that comment's 3 replies; every object shown is complete and every field is verbatim:

{
  "comments": [
    {
      "id": "t1_dfkbj5r",
      "parent_id": null,
      "depth": 0,
      "score": 8868,
      "author": {
        "username": "electric_ionland",
        "id": null
      },
      "body": "So this is not my area of expertise and I hope other more qualified people chime in but I have found some informations (some of it even peer reviewed!). ...",
      "created": "2017-03-29T16:17:47.338000+0000",
      "permalink": "https://www.reddit.com/r/askscience/comments/627akk/comment/dfkbj5r/",
      "replies": [
        {
          "id": "t1_dfkevh3",
          "parent_id": "t1_dfkbj5r",
          "depth": 1,
          "score": 472,
          "author": {
            "username": "PHealthy",
            "id": null
          },
          "body": "I'm really sad for Brachiosaurus now.",
          "created": "2017-03-29T17:15:20.542000+0000",
          "permalink": "https://www.reddit.com/r/askscience/comments/627akk/comment/dfkevh3/",
          "replies": []
        }
      ]
    }
  ],
  "comments_returned": 14,
  "_meta": {
    "source": "svc-comments+post-page",
    "sort": "top",
    "pages_fetched": 2,
    "truncated": true
  }
}

Three things were trimmed there and nothing else was: the 8 top-level comments are shown as 1, that comment's 3 replies are shown as 1 (the one shown here is short and complete, so you can see a reply verbatim), and the long body string is truncated where the ... is. Compare against post.json.

Field notes:

1. comments is a real tree, not a flat list with a depth column. Every node carries parent_id, depth and its own replies array, so you can walk it without reassembling anything. comments_returned counts the whole tree, top level and replies together: 14 across 8 top-level comments on this post.

2. comments_returned is the visible slice, and post.num_comments is the thread. This post has 924 comments and returns 14. _meta.truncated is true whenever Reddit left subtrees collapsed behind a "load more", which is most threads of any size. Read num_comments for the size of the conversation and the tree for the top of it.

3. sort changes which comments you get, not just their order. Same post, same minute: top returned 14 comments, new returned 11, controversial returned 14. Of the new set, 1 comment was not in the top set; of the controversial set, 4 were not. If you want coverage rather than a headline, fetch more than one sort and merge on id. Every id from all three runs is in sort_runs.json.

4. _meta.source tells you whether the post object is real. It reads svc-comments+post-page when the post itself was parsed, and svc-comments when only the comment service answered. In the second case post.title, score, author and created come back null while the comment tree still fills in, so the response looks healthy at a glance and the status is 200 either way. Test post.title is not None, or read _meta.source, before you trust a null.

   

5. author.id is null on every comment. 29 of 29 across the two posts committed here. author.username is the only join key on a comment, and it is "[deleted]" when the account is gone. On the post object author.id is populated when the account still exists (t2_3z331 on the link post below) and null when it does not.

6. body is the post's selftext, and domain tells you what kind of post you have. self.{subreddit} marks a text post, anything else is the outbound host. This post is a text post whose question is the whole thing, so body is null and external_url is null. A link post fills both, verbatim from post_link_post.json:

   {
     "id": "t3_luq9oz",
     "title": "How I cut GTA Online loading times by 70%",
     "score": 19020,
     "upvote_ratio": 0.9727205092171612,
     "num_comments": 992,
     "body": null,
     "external_url": "https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/",
     "domain": "nee.lv"
   }

   (8 of 13 fields shown, verbatim.)

7. is_locked and is_removed are null on every post we sampled, so do not branch on them. A removed post surfaces as a 404 instead.

8. A post id that does not exist comes back 200, not 404. Three bogus ids each returned an all-null post object with num_comments: 0 and comments: []. That pair is what separates "no such post" from a stub: a stub for a real post keeps the real num_comments and a populated comment tree, as in the _meta.source note above. The whole response for a dead id is committed as post_dead_id.json:

   {
     "post": {"id": "t3_zzzzzz9", "title": null, "score": null, "num_comments": 0},
     "comments": [],
     "comments_returned": 0,
     "_meta": {"source": "svc-comments", "sort": "top", "pages_fetched": 1, "truncated": false}
   }

   (4 of the post object's 13 fields shown; the rest are null except permalink, which echoes the id you asked for. Full object in the file.)

9. Scores drift, so two captures of one post will not match. The numbers on this page come from 2026-07-20. Both example posts are years old and have settled, which is why they are the ones used here, but any live thread moves between calls and that is the signal you are collecting.

Runnable: reddit_post_scraper_api_codes/post.py

---

Export a Reddit thread's comments to CSV

Reading the thread is the reason to scrape a post, so that job is in the repo end to end rather than as a snippet. thread_to_csv.py reads a file of post URLs, fetches each thread, flattens the nested tree into one row per comment, and writes a CSV:

python reddit_post_scraper_api_codes/thread_to_csv.py reddit_post_scraper_api_codes/urls.txt thread.csv

That is a real run over the two posts in urls.txt. The nesting survives the flattening because every row keeps parent_id and depth, so you can rebuild the tree or just group by parent_id. The columns are post_id, post_title, comment_id, parent_id, depth, score, author, created, permalink, body; the first three rows of the giraffe thread, with post_title, created and permalink dropped here for width:

post_id	comment_id	parent_id	depth	score	author	body
t3_627akk	t1_dfkbj5r		0	8868	electric_ionland	So this is not my area of expertis...
t3_627akk	t1_dfkc49u	t1_dfkbj5r	1	2660	pwrwisdomcourage	It's also important to remember ma...
t3_627akk	t1_dfkevh3	t1_dfkbj5r	1	472	PHealthy	I'm really sad for Brachiosaurus n...

The 4 removed/deleted placeholder comments dropped line in that output is the reason the script exists rather than a one-liner. Reddit hands you "Comment removed by moderator" as a comment body, not as a null, so anything counting words or scoring sentiment needs those rows gone before it starts. A post that fails is printed with its status and the run carries on; a stub post object is skipped rather than written as a row of nulls.

One API request per post per sort, so the arithmetic is simple: 10 threads a day is 10 requests a day, and the 1,000 free requests cover about 100 days of that, one-time.

---

Measured latency

Real end-to-end wall-clock, measured from a laptop against the live API on 2026-07-20 across 6 public posts in 3 subreddits. This includes the upstream fetch, the anti-bot handling, the comment-tree assembly and the parse:

Endpoint	Median	Range	n
/reddit/post	4.6s	2.5 to 7.7s	22

Read the range, not just the median. This endpoint makes more than one upstream request per call (_meta.pages_fetched was 2 on every successful post fetch here), which is why its median sits where it does. Every measurement is committed in latency.json. Size your timeouts off the tail rather than the middle, and reproduce it with the scripts in this repo.

---

License

MIT. See LICENSE.