Reddit User Scraper for extracting public submissions, comments, subreddits, permalinks and timestamps from Reddit profiles. This repo has a free Reddit user scraping script you can run right now, and a Reddit user data API that returns one account's recent public activity as parsed JSON, up to 25 items per call with 10 fields each.
Last updated: 2026-07-20. Working against Reddit.com as of July 2026.
pip install requests
export CHOCODATA_API_KEY="your_key" # free: 1,000 requests, one-time, no card
python reddit_user_scraper_api_codes/user.py spezThose three lines return the account's profile block and its 25 most recent public items. Here is one of those items, complete, every field verbatim (the full response is below):
{
"type": "comment",
"id": "t1_oj7p61a",
"short_id": "oj7p61a",
"title": "/u/spez on Reddit Announces Q1’26 Earnings (plus AMA!)",
"subreddit": "RDDT",
"body": "We’ve got a couple: Google and OpenAI .",
"external_url": null,
"permalink": "https://www.reddit.com/r/RDDT/comments/1t07fra/reddit_announces_q126_earnings_plus_ama/oj7p61a/",
"created": "2026-04-30T22:13:25+00:00",
"score": null
}...multiplied by 25 items in a single request. This is u/ModCodeofConduct, the account Reddit uses to post moderation notices, in one call:
That is the whole point of this repo. The rest of this page is the free script, the measured evidence behind it, and the full API reference.
What this repo scrapes. Every capture committed here was requested for an account that is public in an official capacity: u/spez is Reddit's CEO, and u/ModCodeofConduct is the account Reddit uses to post moderator notices, whose own posts link to redditinc.com policy pages. The samples, tables and images in this repo are limited to those two. What a feed returns is whatever that account wrote in public, so a captured body can name another handle the way any public comment can; nothing in this repo renders an individual's handle into an image, and no account was sampled that is not one of the two above. The endpoint reads whatever public account you point it at, so that choice is yours to make.
This repo covers the profile feed on its own. The other Reddit surfaces, subreddit listings with real vote data, posts with their comment trees, and search, are documented in the Reddit Scraper.
- Free Reddit User Scraper
- Avoid getting blocked when scraping Reddit profiles
- Reddit User Scraper API reference
- Watch an official Reddit account for new posts
- Measured latency
- License
Reddit publishes every public profile as an Atom feed at /user/<name>/.rss. No key, no OAuth app, no browser, no JavaScript rendering. It is the same surface the API below reads, so when the feed comes back you are looking at the same source data, parsed by your code instead of ours:
python free_scraper/reddit_user_free_scraper.py spez ModCodeofConductSource: free_scraper/reddit_user_free_scraper.py. It fetches the feed, parses the Atom entries with the standard library, and emits type, id, title, subreddit, body, permalink, created. It parses before it judges: it only reports a refusal when no entries came back, so a live feed is never mislabelled because some marker string turned up in the markup.
Run it and see what your own IP gets. Here is what ours got, and it is the reason the rest of this page exists.
Reddit did not serve a CAPTCHA and did not serve a partial feed. It refused the request, in two different shapes, from the first call of the session. Every row below is committed in benchmarks/measurements.json, and benchmarks/measure.py reproduces the whole table on your own IP.
| What we measured (2026-07-20, one residential IP) | Value |
|---|---|
Requests to /user/<name>/.rss, recorded across four phases |
21 |
| Feeds parsed out of those | 0 |
| Statuses returned | 403 and 429, nothing else |
| The 403 body | 189,908 characters of HTML, 0 <entry> elements, no <feed> |
| What that page says | "You've been blocked by network security." |
| The 429 body | 0 bytes |
| User-Agent strings tried on the same URL | 4 (script, python-requests default, Chrome 126, curl), all refused |
| Polled once a minute after that | 10 minutes, still refused |
| Same feed through the API, same session | 200, 19,661 bytes, 25 items |
Note what that measurement does and does not say. It is 21 requests from one residential IP on one evening, and Reddit's refusals are known to vary by network and over time, so treat it as a reading rather than a rule and take your own. What it does establish is that a plain client asking for a public profile feed can get nothing at all, with no warning and no partial result, and that the four obvious header tweaks did not change the answer.
| What bites you | Why | What it costs you |
|---|---|---|
| A refusal arrives as HTML, not as an error | The 403 hands back 189,908 characters of a styled page. It is not XML, so an Atom parser raises on line 1 rather than returning an empty feed. | A crawler that catches ParseError and moves on records the account as having no activity. Check the status code and the content before you parse, and separate "no items" from "no answer". |
| Two refusal shapes, one meaning | The same URL answered 403 with 189,908 characters and 429 with 0 bytes in the same session, minutes apart. | Length is not a health check. A zero-byte body and a 190 KB body were both refusals, so any rule keyed on size gets one of the two wrong. |
| The User-Agent is not the lever | The same URL, four clients: a descriptive script string, python-requests default, a current Chrome string, and curl. All four came back HTTP 429 with an empty body. |
Time spent tuning headers is time not spent on the actual problem. It is also the one route Reddit has written against: its Data API Terms say you "will not misrepresent or mask either the user agent or OAuth identity when using the Data APIs" (redditinc.com/policies/data-api-terms, section 2.8, retrieved 2026-07-20). That clause governs the Data APIs rather than the public feed, but it tells you how the company reads a spoofed client. |
| Slowing down did not clear it | Requests were spaced 4 seconds apart in the cold phase, and the recovery phase polled once a minute for 10 minutes. Neither produced a feed. | The reflex fix, adding a sleep, did not release it inside the window we watched, so a backfill that stalls does not resume by waiting politely. |
| The feed carries no vote data, ever | This is a property of the format rather than of any block. The response names the ten fields it cannot carry: score, ups, downs, upvote_ratio, num_comments, awards, total_awards_received, author_karma, author_cake_day, flair. |
You ship a working profile scraper that cannot answer whether any of it landed. Budget a second call against a surface that does carry scores; the subreddit and post endpoints read the HTML surface that has them. |
| The window is 25 items, not a date range | On u/ModCodeofConduct, kind=submitted returned 25 items reaching back to 2022-09-13, while kind=comments returned 25 items reaching back only to 2026-06-25. Same account, same day, same limit: nearly four years of depth against three weeks. |
If you assume a fixed lookback you will silently under-collect on busy accounts and over-collect on quiet ones. There is no pagination cursor here, so history is built by polling and keeping your own state. |
Worth saying plainly, because it is the one thing that changes the decision: Reddit runs an official API, and tooling built on it is not fighting any of the above. JosephLai241/URS (1,009 stars, last pushed 2026-07-11, both read from the GitHub API on 2026-07-20) is built on PRAW and talks to that official API with an OAuth app you register yourself. It reaches profile data by the front door. What it costs is the registration, the OAuth credentials, the per-app rate limits, and agreement to Reddit's Data API terms. This repo is about the other route: the public surfaces, no account, no app.
The managed option, and the one this repo is built around. The Chocodata Reddit User Scraper API returns one public account's profile block and up to 25 recent items as parsed JSON, with a ~99% success rate and no proxy management. Free for the first 1,000 requests.
Below is the Reddit User Scraper API reference to get you started: authentication, the global parameter, error handling, concurrency, and the endpoint.
curl "https://api.chocodata.com/api/v1/reddit/user?api_key=YOUR_KEY&username=spez"import requests
r = requests.get(
"https://api.chocodata.com/api/v1/reddit/user",
params={"api_key": "YOUR_KEY", "username": "spez"},
timeout=90,
)
data = r.json()
print(data["profile"]["bio"], "|", data["total_results"], "items")
# Reddit CEO | 25 itemsAfter running the command, your terminal should look something like this:
Get a key at chocodata.com (1,000 requests, one-time, no card).
Pass api_key as a query parameter on every request. That is the whole auth model. No OAuth app to register, no Reddit login, no client secret: you never hand us or Reddit an account.
| Param | Type | Required | Default | Description |
|---|---|---|---|---|
api_key |
string | yes | - | Your Chocodata API key. The only parameter every endpoint shares. |
Each request costs 5 credits (= 1 request). Responses are billed only on success (2xx).
Real captured error responses, committed in reddit_user_scraper_api_data/errors.json. Nothing below is billed: you are only charged on a 2xx.
| Status | error code |
Meaning | Billed | What to do |
|---|---|---|---|---|
400 |
invalid_params |
username missing, limit outside 1 to 25, or kind not one of the three values. |
no | Fix the query string. The body names the offending parameter. |
401 |
INVALID_API_KEY |
Key missing, unrecognised, or revoked. | no | Check api_key. Get one at chocodata.com. |
402 |
INSUFFICIENT_CREDITS |
Balance exhausted. | no | Top up or upgrade at chocodata.com. |
404 |
item_not_found |
Reddit returned a 404 for this account: no such name, or it is suspended or deleted. retryable: false. |
no | Fix the username. Retrying will not help. |
429 |
RATE_LIMITED |
Over your plan's concurrency. | no | Back off and retry; see Rate limits. |
502 |
target_unreachable |
Reddit refused this request. retryable: true. |
no | Retry after ~8 seconds. If it repeats on one name, check the spelling. |
Two response shapes exist: auth and billing errors nest under error.code (uppercase), while scrape-layer errors are flat with a lowercase error string.
The scripts in this repo turn each of those statuses into an actionable message, so a typo'd key does not hand you a stack trace:
A bad key, verbatim:
curl "https://api.chocodata.com/api/v1/reddit/user?api_key=totally_invalid_key_123&username=spez"{"error":{"code":"INVALID_API_KEY","message":"Api key not recognised."}}A missing required parameter, verbatim. It names the exact field:
{"error":"invalid_params","issues":[{"code":"invalid_type","expected":"string","received":"undefined","path":["username"],"message":"Required"}]}A kind outside the enum, verbatim. It lists the values that work:
{"error":"invalid_params","issues":[{"received":"banana","code":"invalid_enum_value","options":["overview","submitted","comments"],"path":["kind"],"message":"Invalid enum value. Expected 'overview' | 'submitted' | 'comments', received 'banana'"}]}There is no per-minute request cap. The limit is concurrency: how many requests you may have in flight at once.
| Plan | Concurrent requests |
|---|---|
| Free | 10 |
| Vibe | 30 |
| Pro | 50 |
| Custom | 100 to 500+ |
Exceed it and you get 429, not a queue. The endpoint is a synchronous GET: there is no webhook, callback, or async job to poll. A request can take several seconds when Reddit forces a re-attempt (see Measured latency), which is why the examples use timeout=90.
Sizing: at Pro (50 concurrent) and the measured 1.94s median, one worker pool sustains roughly 50 / 1.94 = 26 requests/second. Fan out with a thread pool:
from concurrent.futures import ThreadPoolExecutor
import requests
def one(username):
r = requests.get("https://api.chocodata.com/api/v1/reddit/user",
params={"api_key": KEY, "username": username}, timeout=90)
return username, (r.json() if r.ok else None)
accounts = ["spez", "ModCodeofConduct"]
with ThreadPoolExecutor(max_workers=10) as pool: # <= your plan's concurrency
for username, data in pool.map(one, accounts):
print(username, data["total_results"] if data else "unavailable")One public account's recent submissions and comments: the item type, its subreddit, the title, the body text, the permalink and the timestamp, plus a small profile block.
| Param | Type | Required | Default | Description |
|---|---|---|---|---|
username |
string | yes | - | The account name. A leading u/ or user/ is stripped, so spez and u/spez both work. |
kind |
enum | no | overview |
Which slice of the profile feed: overview (both), submitted (posts only), comments (comments only). Anything else returns 400. |
limit |
integer | no | 25 |
Items to return, 1 to 25. limit=5 returned exactly 5. Above 25 returns 400, it is not clamped. |
curl "https://api.chocodata.com/api/v1/reddit/user?api_key=YOUR_KEY&username=spez"Real response. items is cut to 2 of 25, one of each type; both item objects are complete with all 10 fields verbatim, and the profile and _rss_limitations blocks are shown in full (full sample):
{
"profile": {
"username": "spez",
"profile_url": "https://www.reddit.com/user/spez",
"bio": "Reddit CEO",
"icon": "https://www.redditstatic.com/icon.png/",
"title": "overview for spez",
"total_karma": null,
"created": null
},
"total_results": 25,
"items": [
{
"type": "comment",
"id": "t1_oj7p61a",
"short_id": "oj7p61a",
"title": "/u/spez on Reddit Announces Q1’26 Earnings (plus AMA!)",
"subreddit": "RDDT",
"body": "We’ve got a couple: Google and OpenAI .",
"external_url": null,
"permalink": "https://www.reddit.com/r/RDDT/comments/1t07fra/reddit_announces_q126_earnings_plus_ama/oj7p61a/",
"created": "2026-04-30T22:13:25+00:00",
"score": null
},
{
"type": "submission",
"id": "t3_1tvsa59",
"short_id": "1tvsa59",
"title": "Hi redditstock, leave your questions, and we’ll be back later today to answer them.",
"subreddit": "redditstock",
"body": null,
"external_url": null,
"permalink": "https://www.reddit.com/r/redditstock/comments/1tvsa59/hi_redditstock_leave_your_questions_and_well_be/",
"created": "2026-06-03T15:11:07+00:00",
"score": null
}
],
"_source": "rss",
"_rss_limitations": {
"surface": "rss-atom",
"unavailable_fields": [
"score",
"ups",
"downs",
"upvote_ratio",
"num_comments",
"awards",
"total_awards_received",
"author_karma",
"author_cake_day",
"flair"
],
"note": "This response used the .rss/Atom fallback (the shreddit HTML surface that carries vote data was unreachable on this request). RSS does not expose any vote/award/comment counts or author karma - those fields are null (not fabricated). Identity, author, timestamps, permalinks, post/comment bodies ARE provided. Profile karma and cake-day are not in the user RSS feed (null). The feed returns the user's most recent public items only."
}
}permalink paired with created and subreddit is what people actually come for: it is a timestamped map of where an account is active, which is enough to answer "is this account posting in our space, and when did it start". type splits comments from submissions without a second call, and short_id is the id you hand to a post-level endpoint if you then want the vote numbers.
Read _source first. It says "rss" on this endpoint, and that is the whole explanation for the nulls. This endpoint reads Reddit's Atom profile feed. That format has no vote or award data in it, so score is null on every item, and profile.total_karma and profile.created are null on every account. Those fields are not missing because a request went badly; they are not in the surface. The response tells you so itself, and lists the ten field names in _rss_limitations.unavailable_fields. The subreddit and post endpoints read the HTML surface that does carry scores and comment counts, and are documented in the pillar repo linked at the top of this page.
Five more behaviours to code against. The counts come from the five samples in accounts_sampled.json:
kindchanges the depth, not just the filter. On u/ModCodeofConduct,submittedreached back to 2022-09-13 andcommentsreached back only to 2026-06-25, both returning 25 items on the same day. If you want an account's oldest reachable posts, ask forsubmitted; a slice that includes comments fills the window with recent activity and the reach closes accordingly.limitis a hard ceiling, not a hint.limit=5returned exactly 5 items.limit=99returns400naming the maximum rather than quietly clamping to 25, so a wrong value fails loudly, which is what you want.bodyisnullon link posts and on some submissions. Null on 2 of 25 items in each overview sample, and on 0 of 25 in both comments samples. A submission with no self-text has nothing to put there; a comment always has a body.external_urlis populated only when the item links off Reddit. Null on all 25 items for u/spez, whose posts are self-posts and image links; populated on 2 of 25 for u/ModCodeofConduct, both pointing at redditinc.com policy pages. Treat it as optional, not as a field that is broken.subredditreadsu/<name>for profile posts. An account's own profile posts come back withsubreddit: "u/spez"rather than a community name, so aGROUP BY subredditmixes the two unless you split on theu/prefix.
Runnable: reddit_user_scraper_api_codes/user.py
The feed is a rolling window with no pagination, so the way to build a history out of it is to poll it and keep your own state. That is the main reason people pull a single account repeatedly, so it is in the repo end to end rather than as a snippet. watch_accounts.py resolves every account concurrently, diffs the item ids against a small local state file, prints only what is new, and can append each new item to a CSV:
python reddit_user_scraper_api_codes/watch_accounts.py spez ModCodeofConductThe first run records what is there and reports nothing as new, which is the correct behaviour for a diff with no baseline. Every run after that prints the items that appeared since. Point it at a CSV when you want the history to accumulate on disk:
python reddit_user_scraper_api_codes/watch_accounts.py spez --csv activity.csv --workers 4The CSV carries username, type, id, short_id, subreddit, created, title, permalink, external_url and body, so a row stays useful without a second lookup. Keep --workers at or below your plan's concurrency.
Do this arithmetic before you schedule it: one request per account per poll. Watching 20 accounts every 15 minutes is 1,920 requests a day, which is more than the entire one-time free allowance in a single day of polling. Decide the interval from how fast the accounts you care about actually move; the two accounts sampled in this repo moved on a scale of days, not minutes.
Real end-to-end wall clock, measured from a laptop against the live API on 2026-07-20 by benchmarks/measure.py. This includes the upstream fetch and the parse:
| Endpoint | Median | Range | n |
|---|---|---|---|
/reddit/user |
1.94s | 1.48 to 5.49s | 8 |
Read the range, not just the median. The 5.49s is the interesting number: that is a request that ran into a refusal upstream and was re-attempted until real data came back, on the same evening the free script in this repo got 21 refusals in a row. Small sample, one machine, one day: run benchmarks/measure.py and use your own numbers.
MIT. See LICENSE.