release: v0.4.0 — non_exhaustive, place_id/cid, hl=en, proxy arg fix - #55
Merged
Merged
Conversation
Implements the three issues from today's improvement-research run (#52, #53, #54) plus live-validation of the DOM selectors added in #15 (bakeries, hotels via a real proxied Chrome session): - Place::place_id / Place::cid — stable Google identifiers parsed from the maps URL's data= blob (no new navigation, no new DOM selectors). Live-confirmed against real listings. - ScraperConfig::language (default Some("en")) — pins the Maps UI language via hl= on every navigation, so extraction no longer silently degrades behind non-EN/DE exit geos. Live-confirmed: addresses now read "Germany"/category "Bakery" instead of the proxy-geo language. - Place, ScraperConfig, and Error are now #[non_exhaustive], so future Option<T> fields / error variants land in minor releases instead of forcing a breaking bump every time. ScraperConfig is built via default() + field mutation; README updated accordingly. Bug found and fixed during live validation: chromiumoxide's arg() builder prepends "--" itself, but MapsScraper::launch was passing already-dashed strings ("--proxy-server=...", "--user-agent=...", "--disable-blink-features=...", "--lang=...", "--window-size=..."), double-prefixing them to "----proxy-server=..." etc. Chrome silently ignores unrecognized flags, so every one of these was a no-op — including the proxy, which is a real leak (the documented protection against a malformed proxy value did not cover this: the whole flag was malformed, not just the value). Fixed by passing bare (key, value) tuples per chromiumoxide's actual API. Also, live validation surfaced a genuine Google Maps DOM change: reviews_count is no longer shown inline next to the rating for most listing types (bakeries) as of 2026-07-30, though it's still present for others (hotels) via a body-text fallback added here. Documented honestly on the field instead of pretending it reliably works. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HHcB7Dhren7PSwn4va8BKv
📝 WalkthroughWalkthroughChangesThe 0.4.0 release adds non-exhaustive public types, 0.4.0 release and public API
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/lib.rs`:
- Around line 700-710: The review-label lookup in the review-count extraction
flow must reject non-count CTAs such as “Write a review.” Update the
`revEl`/`reviewsText` handling so its aria-label is retained only when it
contains a review count; otherwise leave it unset and allow the existing
`head.textContent` and `document.body.innerText` fallbacks to run.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 7308730b-e117-43fa-87ef-6c8a3b2d9ee9
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (4)
CHANGELOG.mdCargo.tomlREADME.mdsrc/lib.rs
Comment on lines
700
to
+710
| let reviewsText = null; | ||
| const revEl = document.querySelector( | ||
| 'button[aria-label*="review" i], button[aria-label*="Rezension" i], button[aria-label*="Bewertung" i], [aria-label*="reviews" i]' | ||
| ); | ||
| if (revEl) reviewsText = revEl.getAttribute('aria-label'); | ||
| if (!reviewsText && head) { | ||
| const m = head.textContent.match(/\(([\d., ]+)\)/); | ||
| const m = head.textContent.match(/\(([\d., ]+)\)/); | ||
| if (m) reviewsText = m[1]; | ||
| } | ||
| if (!reviewsText) { | ||
| const m = document.body.innerText.match(/([\d., ]+)\s+reviews?/i); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Files:"
git ls-files | rg '(^|/)src/lib\.rs$|Cargo\.lock$|package\.json$|README' || true
echo
echo "Relevant src/lib.rs around reported lines:"
sed -n '660,735p' src/lib.rs
echo
echo "Search for reviewsText and related fallbacks:"
rg -n "reviewsText|reviews.?|reviewsText = revEl|reviewsText = m|^#[derive|struct .*Review|parse" src/lib.rsRepository: Liohtml/google-maps-scraper-rs
Length of output: 4353
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Search for review handling and type definitions:"
rg -n "reviewsText|reviewsCount|reviews_count|PlaceDetailRaw|impl.*Place|parse|reviews" src/lib.rs
echo
echo "Outline around PlaceDetailRaw:"
ast-grep outline src/lib.rs --match PlaceDetailRaw --view expanded || true
echo
echo "Lines containing parse-like handling:"
rg -n -A 8 -B 8 "from_str|parse|reviews_count|reviews" src/lib.rsRepository: Liohtml/google-maps-scraper-rs
Length of output: 24807
🏁 Script executed:
#!/bin/bash
set -euo pipefail
node - <<'JS'
function parse_reviews_count(text) {
const matchBefore = (text.match(/(?i)(\d[\d.,\u{00A0}\u{202F} ]*)\s*(?:reviews?|rezensionen|bewertungen?)/) || [])[1];
if (matchBefore) {
const digits = matchBefore.replace(/\D/g, '');
return digits ? digits : null;
}
const matchParen = (text.match(/\((\d[\d.,\u{00A0}\u{202F} ]*)\)/) || [])[1];
if (matchParen) {
const digits = matchParen.replace(/\D/g, '');
return digits ? digits : null;
}
const matchAny = text.match(/(\d[\d.,\u{00A0}\u{202F} ]*)/);
if (matchAny) {
const digits = matchAny[1].replace(/\D/g, '');
return digits ? digits : null;
}
return null;
}
const inputs = [
'Write a review',
'Write a review (1,234)',
'(1,234)',
'1,234 reviews',
];
for (const input of inputs) {
const hasNumber = /\d/.test(input);
console.log(JSON.stringify(input), 'hasDigit=', hasNumber, 'parsed=', parse_reviews_count(input));
}
JS
echo
python3 - <<'PY'
import re
def parse_reviews_count(text):
match_before = re.search(r'(?i)(\d[\d.,\u00A0\u202F ]*)\s*(?:reviews?|rezensionen|bewertungen?)', text)
if match_before:
return int(''.join(c for c in match_before.group(1) if c.isdigit()))
match_paren = re.search(r'\((\d[\d.,\u00A0\u202F ]*)\)', text)
if match_paren:
return int(''.join(c for c in match_paren.group(1) if c.isdigit()))
match_any = re.search(r'\d[\d.,\u00A0\u202F ]*', text)
if match_any:
return int(''.join(c for c in match_any.group(0) if c.isdigit()))
return None
for input_text in ['Write a review', 'Write a review (1,234)', '(1,234)', '1,234 reviews']:
print(repr(input_text), 'has_digit=', bool(re.search(r'\d', input_text)), 'parsed=', parse_reviews_count(input_text))
PYRepository: Liohtml/google-maps-scraper-rs
Length of output: 1030
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from typing import Optional
import re
ANY_COUNT_RE = re.compile(r'\d[\d.,\u00A0\u202F ]*')
def parse_reviews_count(text: str) -> Optional[int]:
for p in [
r'(?i)(\d[\d.,\u00A0\u202F ]*)\s*(?:reviews?|rezensionen|bewertungen?)',
r'\((\d[\d.,\u00A0\u202F ]*)\)',
r'\d[\d.,\u00A0\u202F ]*',
]:
m = re.search(p, text)
if m:
digits = ''.join(c for c in m.group(1) if c.isdigit())
if digits:
return int(digits)
return None
rev_aria_with_fallbacks = 'Write a review\n(1,234)'
rev_aria_no_fallbacks = '(1,234)\nWrite a review'
for text in [rev_aria_with_fallbacks, rev_aria_no_fallbacks]:
print(repr(text), 'has_digit=', bool(re.search(r'\d', text)), 'parsed=', parse_reviews_count(text))
PYRepository: Liohtml/google-maps-scraper-rs
Length of output: 278
🏁 Script executed:
#!/bin/bash
set -euo pipefail
node - <<'JS'
function parseCountFallback(text) {
let out = null;
const head = '(1,234)\nWrite a review';
if (!out && head) {
const m = head.match(/\(([\d., ]+)\)/);
if (m) out = m[1];
}
if (!out) {
const body = 'Write a review\n(1,234)';
const m = body.match(/([\d., ]+)\s+reviews?/i);
if (m) out = m[1];
}
return out;
}
for (const input of ['Write a review', 'Write a review (1,234)']) {
const hasNumber = /\d/.test(input);
const fallback = parseCountFallback(input);
console.log(JSON.stringify(input), 'hasDigit=', hasNumber, 'fallback=', JSON.stringify(fallback));
}
JS
node - <<'JS'
function parseCountFallback2(revAria) {
let reviewsText = null;
if (revAria && /\d/.test(revAria)) reviewsText = revAria;
if (!reviewsText) {
const head = '(1,234)\nWrite a review';
const m = head.match(/\(([\d., ]+)\)/);
if (m) reviewsText = m[1];
}
if (!reviewsText) {
const body = 'Write a review\n(1,234)';
const m = body.match(/([\d., ]+)\s+reviews?/i);
if (m) reviewsText = m[1];
}
return reviewsText;
}
for (const input of ['Write a review', 'Write a review (1,234)']) {
console.log(JSON.stringify(input), 'fallback=', JSON.stringify(parseCountFallback2(input)));
}
JSRepository: Liohtml/google-maps-scraper-rs
Length of output: 372
Do not use non-count review CTAs before the fallbacks.
querySelector can match “Write a review”, and assigning that label makes parse_reviews_count() fall back to extracting any number in the label. Only keep the aria-label when it looks like a review count, so the parenthesized/body-text fallbacks can still run for labels without a count.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/lib.rs` around lines 700 - 710, The review-label lookup in the
review-count extraction flow must reject non-count CTAs such as “Write a
review.” Update the `revEl`/`reviewsText` handling so its aria-label is retained
only when it contains a review count; otherwise leave it unset and allow the
existing `head.textContent` and `document.body.innerText` fallbacks to run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the three proposals from today's improvement-research run (#52, #53, #54) and live-validates the DOM selectors added in #15, using a real proxied Chrome session against live Google Maps (bakeries in Berlin, hotels in Hamburg). Closes #52, #53, #54.
Added
Place::place_id/Place::cid([research] Add Place::place_id (and CID) parsed from the maps URL — land in the 0.4.0 window #52) — stable Google identifiers parsed from thedata=blob of the maps URL. No new navigation, no new DOM selectors. Live-confirmed against real listings (e.g.place_id=ChIJyTPda-JRqEcRFHRtJJAZIwE,cid=81937325099938836).ScraperConfig::language([research] Pin the Maps UI language with ?hl=en on all navigations (config-overridable) #53, defaultSome("en")) — pins the Maps UI language viahl=on every navigation. Previously Google picked the UI language from the exit IP's geo, silently breaking label-based extraction behind non-EN/DE proxies. Live-confirmed: addresses now read"...Germany"/ category"Bakery"instead of following the proxy's geo language.Changed (breaking)
Place,ScraperConfig, andErrorare now#[non_exhaustive]([research] Ship #[non_exhaustive] on Place/Error (+ ScraperConfig strategy) in 0.4.0, not "before 1.0" #54). FutureOption<T>fields / error variants land in minor releases instead of forcing a breaking bump every time — done now while 0.4.0 is already breaking and the crate has ~14 downloads, i.e. as cheap as this will ever be.ScraperConfigis built viadefault()+ field mutation (README updated);Placeis output-only;Errormatches need a_arm.Fixed — real bug found during live validation
chromiumoxide'sarg()builder prepends--itself, butMapsScraper::launchwas passing already-dashed strings ("--proxy-server=...","--user-agent=...","--disable-blink-features=...","--lang=...","--window-size=..."), double-prefixing them to"----proxy-server=..."etc. Chrome silently ignores unrecognized flags, so every one of these was a no-op — including the proxy. This is a real leak:check_proxy's existing whitespace guard only covers a malformed value, not a malformed whole flag, so a correctly-configuredproxy/PROXY_URLwas never actually applied and Chrome connected directly. Fixed by passing bare(key, value)tuples per chromiumoxide's actual API — verified by tracing the real Chrome invocation through a proxy and confirming the flag renders with a single--.Also from live validation
reviews_countis honestly documented now: as of 2026-07-30 Google's detail panel no longer shows a review count inline next to the rating for most listing types (confirmed empirically — only "Write a review" remains for e.g. bakeries), though it's still present for others (hotels showed2,568 reviews/2,081 reviewsvia a body-text fallback added here). Kept the multi-selector best-effort chain (harmless, degrades toNone) but updated the doc comment instead of pretending the field reliably populates.Verification
cargo test: 15 tests pass (13 unit + 2 doc)cargo fmt --all -- --check,cargo clippy --all-targets -- -D warnings,cargo doc --no-deps: cleancargo audit: 0 advisories across 150 dependenciesAfter merging
Per
CLAUDE.md's release process: bump the CHANGELOG's[Unreleased]→ already done here as[0.4.0]. Once merged, tagv0.4.0on the merge commit (git tag v0.4.0 && git push origin v0.4.0, or create the GitHub release in the UI) —release.ymlverifies, publishes to crates.io, and creates the GitHub release.🤖 Generated with Claude Code
https://claude.ai/code/session_01HHcB7Dhren7PSwn4va8BKv
Generated by Claude Code
Summary by CodeRabbit
place_idandcid.