Skip to content

LEC (Leicestershire) scraper failing — intermittent TCP connection refused #367

@symroe

Description

@symroe

Error (2026-06-17)

is_connect error: wreq::Error { kind: Request, uri: https://democracy.leics.gov.uk/mgWebService.asmx/GetCouncillorsByWard, source: Error { kind: Connect, source: Some(ConnectError("tcp connect error", 81.131.36.159:443, Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } }
Duration: 0.4s

Current state (2026-06-19)

democracy.leics.gov.uk is now returning HTTP 200 with valid ModGov XML from curl (response in ~5.3s). The scraper code requires no changes — the endpoint is working.

<?xml version="1.0" encoding="utf-8"?>
<councillorsbyward>
  <wards>
    <ward>
      <wardtitle>Ashby de la Zouch</wardtitle>
      ...

Assessment

The "Connection refused" (errno 111) was a transient server-side outage — the server's TCP stack was actively rejecting connections, not timing out. The server has since recovered. No code changes are needed.

What needs to happen

Monitor whether the connection refused error recurs. If it appears consistently in the failing feed over multiple consecutive days, investigate whether:

  • Lambda's IP range is being periodically blocked by the server
  • The server has a recurring maintenance window at the time the scraper runs
  • A code fix (e.g. http_lib = "playwright") is needed to bypass WAF filtering

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions