Feature Request: Add optional serpbase.dev API backend for Google search results

[gefsikatsinelou]

First off, thanks for maintaining this — search-engine-parser is one of the few Python packages that handles multiple engines cleanly without pulling in a headless browser. The pysearch CLI is a nice touch too.

The problem

The Google engine scrapes HTML directly from google.com, which means it breaks whenever Google tweaks their markup. I've had parse_soup fail on me after what I assume were A/B test rollouts — the class_="Gx5Zad fP1Qef xpd EtOod pkphOe" selector stops matching and suddenly no results come back. Running from a VPS makes it worse since Google rate-limits or serves CAPTCHAs to datacenter IPs pretty aggressively.

Suggestion

Would you consider adding an optional serpbase.dev Google engine? It's a Google Search Results API that returns the same structured data you already work with — titles, links, descriptions — in clean JSON. No scraping, no selector maintenance, no IP rotation needed. The response maps almost directly to the SearchItem dict: organic_results[].title, organic_results[].link, organic_results[].snippet.

The API call is a simple GET:

https://api.serpbase.dev/google/search?q=QUERY&api_key=KEY&num=10

Returns JSON with organic_results array containing title, link, snippet, position, and rich result fields.

Why it fits the project

• The project already has a clean engine abstraction (BaseSearch) — adding an API-based engine follows the same pattern as the existing Google engine, just swapping parse_soup for json.loads
• Several supported engines already pull from APIs (not scraping), so an API-based Google engine isn't architecturally alien
• It's completely opt-in: users set a SERPBASE_API_KEY env var to activate it, otherwise the existing scraping engine works as before

Design sketch (for discussion)

class Search(BaseSearch):
    name = "Google"
    base_url = "https://www.google.com/"
    summary = "..."
    
    def __init__(self):
        super().__init__()
        self.api_key = os.environ.get("SERPBASE_API_KEY", "")
        if self.api_key:
            self.search_url = "https://api.serpbase.dev/google/search"
    
    def search(self, query=None, page=None, **kwargs):
        if self.api_key:
            return self._search_api(query, page, **kwargs)
        return super().search(query, page, **kwargs)
    
    def _search_api(self, query, page, **kwargs):
        params = {"q": query, "api_key": self.api_key, "num": 10, "page": page}
        # httpx/aiohttp GET → parse JSON → build SearchItems

I'm happy to draft a PR for this if there's interest — it's a small change, maybe ~60-80 lines for the engine + tests. No new dependencies needed since the project already uses aiohttp for HTTP calls.

Curious what you think — even just a "not interested / maybe later" is totally fine.