Skip to content

feat: extract rating, review count and category (completes #11) - #15

Merged
Liohtml merged 3 commits into
mainfrom
claude/repo-issues-features-z8Q0f
Jul 17, 2026
Merged

feat: extract rating, review count and category (completes #11)#15
Liohtml merged 3 commits into
mainfrom
claude/repo-issues-features-z8Q0f

Conversation

@Liohtml

@Liohtml Liohtml commented May 29, 2026

Copy link
Copy Markdown
Owner

Overview

Completes the remaining scope of #11 (coordinates already shipped in #13). Place now also carries the rating, review count, and business category from the Google Maps detail panel.

What changed

  • Place gains three fields (all Option, backwards-compatible):
    • rating: Option<f32> — average star rating (0.0–5.0)
    • reviews_count: Option<u32> — number of reviews
    • category: Option<String> — primary business category
  • extract_place_details reads them from the panel with defensive, multi-selector fallbacks (EN + DE). Review count is targeted via review-keyword aria-label / a parenthesised count so it stays separate from the rating.
  • Pure, unit-tested parsers:
    • parse_rating — accepts , or . decimals ("4,5", "4.5 stars", "4,5 Sterne"), enforces the 0–5 range.
    • parse_reviews_count — strips thousands separators (., ,, spaces, non-breaking spaces): "1,234 reviews", "1.234 Rezensionen", "(1 234)"1234.
  • Drive-by: fixed two doc comments that had drifted onto the wrong function.
  • README ("What you get back" + roadmap) and CHANGELOG updated.

⚠️ Important caveat — selectors not live-validated

This environment has no local Chrome, so the DOM selectors could not be verified against live Google Maps. The risk is contained:

  • The brittle parsing logic is fully unit-tested (8 tests total, all green).
  • The selectors are best-effort with EN/DE fallbacks and degrade gracefully (each field is Option, so a missed selector yields None, never a crash).

They may need real-world tuning. Now that Browserless support exists (#12), the selectors can be validated against real Google Maps via a remote Chrome (BROWSERLESS_URL) — happy to do a follow-up tuning pass once an endpoint is available.

Test plan

  • cargo build
  • cargo test (8 passed)
  • cargo fmt --all -- --check
  • cargo clippy --all-targets -- -D warnings
  • Live scrape to confirm the rating/reviews/category selectors (needs Chrome or a BROWSERLESS_URL)

Addresses #11.

https://claude.ai/code/session_01TPpTHPokxsZ3dQpRzg4NkD


Generated by Claude Code

Summary by CodeRabbit

  • New Features

    • Locations now include average rating, review count, and business category extracted from place details.
  • Documentation

    • README and changelog updated to describe the new location fields and roadmap adjustments.
  • Tests

    • Added unit tests for parsing and validating rating and review count values.

Review Change Stack

Completes the remaining scope of #11 (coordinates already shipped in #13):
- Place gains rating: Option<f32>, reviews_count: Option<u32>, category:
  Option<String>.
- extract_place_details reads them from the detail panel with defensive,
  multi-selector fallbacks (EN/DE), keeping review count separate from the
  rating via keyword/parenthesis targeting.
- Pure, unit-tested parse_rating (locale-tolerant decimal, 0–5 range check)
  and parse_reviews_count (strips thousands separators) helpers.
- Drive-by: fix two doc comments that had drifted onto the wrong fn.

Note: the DOM selectors could not be validated against a live Chrome in this
environment; the parsing logic is fully unit-tested and the selectors are
best-effort with fallbacks — they may need real-world tuning. README/CHANGELOG
updated.

https://claude.ai/code/session_01TPpTHPokxsZ3dQpRzg4NkD
@coderabbitai

coderabbitai Bot commented May 29, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@Liohtml, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 12 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6ecf32d4-42af-4adf-83f2-ea0c504bfa83

📥 Commits

Reviewing files that changed from the base of the PR and between 321c147 and d1685f2.

📒 Files selected for processing (3)
  • CHANGELOG.md
  • README.md
  • src/lib.rs
📝 Walkthrough

Walkthrough

This PR extends Place with optional fields rating, reviews_count, and category; expands the JS extractor to read these values; adds PlaceDetailRaw fields; introduces parse_rating and parse_reviews_count helpers with unit tests; and populates the new fields during search result enrichment.

Changes

Richer Place Data Extraction

Layer / File(s) Summary
Public Place struct and documentation
src/lib.rs, README.md, CHANGELOG.md
Place gains optional rating (f32), reviews_count (u32), and category (String). README and CHANGELOG document the new fields and roadmap status.
Extraction JS and intermediate struct
src/lib.rs
extract_place_details JS payload is extended to extract rating text, review-count labels, and the primary category. PlaceDetailRaw adds matching optional fields and Rust captures the raw strings.
Parsing and validation helpers
src/lib.rs
New parse_rating normalizes decimal separators and validates 0.0..=5.0; parse_reviews_count strips thousands/space-like separators and parses to u32.
Search result enrichment
src/lib.rs
search_many populates the three new Place fields from parsed PlaceDetailRaw when constructing enriched results.
Comment relocation
src/lib.rs
“Navigate with timeout” comment moved to be above goto_with_timeout.
Unit tests
src/lib.rs
Tests added for parse_rating and parse_reviews_count covering multiple formats and locale variations.

Sequence Diagram

sequenceDiagram
  participant search_many
  participant extract_place_details
  participant DOM
  participant parse_helpers
  participant PlaceDetailRaw
  participant Place

  search_many->>extract_place_details: request place detail extraction
  extract_place_details->>DOM: inject JS to read rating/reviews/category
  DOM-->>extract_place_details: raw extracted strings (rating, reviews, category)
  extract_place_details->>parse_helpers: pass rating string
  parse_helpers-->>extract_place_details: Option<f32> parsed rating
  extract_place_details->>parse_helpers: pass reviews string
  parse_helpers-->>extract_place_details: Option<u32> parsed reviews_count
  extract_place_details->>PlaceDetailRaw: store parsed values
  search_many->>Place: enrich Place with PlaceDetailRaw fields
  Place-->>search_many: enriched Place returned
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

  • Issue #11: Directly related; both concern extending the Place struct and extract_place_details to capture additional place fields (this PR implements rating, reviews_count, and category).

Poem

🐰 I sniffed the detail pane at night,
Found stars and counts and labels bright,
I parsed each comma, dot, and space,
And tucked the fields in Place with grace.
Hoppity hops — the map's more right!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and accurately summarizes the main change: adding extraction of rating, review count, and category fields to the Place model, with explicit reference to completing issue #11.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/repo-issues-features-z8Q0f

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib.rs`:
- Around line 636-642: parse_reviews_count currently takes the first numeric
token (via REVIEWS_RE.find) which mis-parses strings like "4.7 stars from 1,234
reviews"; update it to pick the numeric token nearest a review keyword instead.
Change parse_reviews_count to: use REVIEWS_RE.find_iter(text) to gather all
numeric tokens (with their byte positions), normalize each token to digits only,
locate a review-keyword match (e.g., regex or simple case-insensitive search for
"review", "reviews", "rating", "ratings", "votes") and choose the numeric token
whose position is closest to that keyword; if no keyword is found, fall back to
the current first-match behavior and return digits.parse().ok(); keep REVIEWS_RE
and function signature the same and ensure no panics on parse failures.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 28a92d14-b5e6-47a2-9181-b36dfab4cc5b

📥 Commits

Reviewing files that changed from the base of the PR and between 7056cae and 25e748b.

📒 Files selected for processing (3)
  • CHANGELOG.md
  • README.md
  • src/lib.rs

Comment thread src/lib.rs
Address CodeRabbit review on #15: parse_reviews_count took the first numeric
token, so a combined label like "4.7 stars from 1,234 reviews" yielded 47
instead of 1234. Now prefer the number adjacent to a review keyword
(reviews/rezensionen/bewertungen), then a parenthesised count, then any number.
Adds combined-label regression tests (EN + DE).

https://claude.ai/code/session_01TPpTHPokxsZ3dQpRzg4NkD
…flicts)

- lib.rs: keep check_proxy from main alongside the new rating/reviews/
  category extraction
- CHANGELOG: move the rating/reviews/category entry from the (now shipped)
  0.2.0 section to Unreleased

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HHcB7Dhren7PSwn4va8BKv
@Liohtml
Liohtml merged commit bfa67dc into main Jul 17, 2026
4 checks passed
Liohtml pushed a commit that referenced this pull request Jul 30, 2026
Implements the three issues from today's improvement-research run (#52,
#53, #54) plus live-validation of the DOM selectors added in #15
(bakeries, hotels via a real proxied Chrome session):

- Place::place_id / Place::cid — stable Google identifiers parsed from
  the maps URL's data= blob (no new navigation, no new DOM selectors).
  Live-confirmed against real listings.
- ScraperConfig::language (default Some("en")) — pins the Maps UI
  language via hl= on every navigation, so extraction no longer silently
  degrades behind non-EN/DE exit geos. Live-confirmed: addresses now
  read "Germany"/category "Bakery" instead of the proxy-geo language.
- Place, ScraperConfig, and Error are now #[non_exhaustive], so future
  Option<T> fields / error variants land in minor releases instead of
  forcing a breaking bump every time. ScraperConfig is built via
  default() + field mutation; README updated accordingly.

Bug found and fixed during live validation: chromiumoxide's arg()
builder prepends "--" itself, but MapsScraper::launch was passing
already-dashed strings ("--proxy-server=...", "--user-agent=...",
"--disable-blink-features=...", "--lang=...", "--window-size=..."),
double-prefixing them to "----proxy-server=..." etc. Chrome silently
ignores unrecognized flags, so every one of these was a no-op —
including the proxy, which is a real leak (the documented protection
against a malformed proxy value did not cover this: the whole flag was
malformed, not just the value). Fixed by passing bare (key, value)
tuples per chromiumoxide's actual API.

Also, live validation surfaced a genuine Google Maps DOM change:
reviews_count is no longer shown inline next to the rating for most
listing types (bakeries) as of 2026-07-30, though it's still present
for others (hotels) via a body-text fallback added here. Documented
honestly on the field instead of pretending it reliably works.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HHcB7Dhren7PSwn4va8BKv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants