Skip to content

Fix stale cache data surviving a GEDCOM file reload - #8

Open
dmarshall-rgb wants to merge 5 commits into
airy10:mainfrom
dmarshall-rgb:fix-load-gedcom-stale-cache
Open

Fix stale cache data surviving a GEDCOM file reload#8
dmarshall-rgb wants to merge 5 commits into
airy10:mainfrom
dmarshall-rgb:fix-load-gedcom-stale-cache

Conversation

@dmarshall-rgb

Copy link
Copy Markdown
Contributor

Problem

load_gedcom_file creates a new Parser and calls _rebuild_lookups to refresh the individual/family/source/note lookup dictionaries when a new file is loaded, but never cleared GedcomContext's three application-level LRU caches (person_details_cache, person_relationships_cache, neighbor_cache).

Root cause

Since these caches are keyed by GEDCOM pointer strings (e.g. @i54@), which are only unique within a single file, loading a second file into the same running context could return cached results from the previously-loaded file for any ID that happens to collide, even though the underlying data now belongs to a different person or family entirely.

Fix

Call gedcom_ctx.clear_caches() as part of every load, immediately after parsing and before rebuilding lookups, so a reload always starts from a clean state.

Testing

Verified via repro script: loading File A and querying @i54@ populates person_details_cache (size 1); loading File B into the same context resets it to size 0 before any new query, confirming the stale entry is cleared rather than surviving the reload.

dmarshall-rgb and others added 5 commits July 20, 2026 12:53
Three related gaps in citation resolution, all sharing the same root cause: GEDCOM SOUR citations nested under a specific event (BIRT, DEAT, MARR, etc.) or attribute were either invisible entirely or only captured as a bare pointer string.

1. _get_sources_internal only scanned an entity's direct top-level children for SOUR, missing any citation nested inside an event or (for a person) inside a FAMS-linked family's events (e.g. MARR).

2. decode_event_details captured event-level SOUR citations but only stored the raw pointer string, never resolving title/author/publication/repository/page/quality, nor the previously-uncaptured DATA/TEXT, DATA/DATE, and citation-level NOTE.

3. The ATTRIBUTE_TYPES branch in _get_events_internal had an independent, separately-shallow copy of the same SOUR-as-bare-string bug.

Fix: added a shared _resolve_source_citation(sour_element, gedcom_ctx) helper that fully resolves a single SOUR element, and applied it at all three call sites. _get_sources_internal now also scans EVENT_TYPES-matching children (and FAMS-linked family events, for persons) the same way _get_events_internal already does, tagging each result with which event (and family_id, where applicable) it supports.

Verified against Thomas Marshall (@i54@) in a real FTM export: get_sources now returns all 8 event-level citations (2 BIRT, 1 RESI, 2 DEAT, 3 MARR across two marriages) with full bibliographic and citation detail, matching get_events output exactly. Tested via direct repro scripts and live MCP tool calls.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
load_gedcom_file creates a new Parser and calls _rebuild_lookups to refresh the individual/family/source/note lookup dictionaries when a new file is loaded, but never cleared GedcomContext's three application-level LRU caches (person_details_cache, person_relationships_cache, neighbor_cache).

Since these caches are keyed by GEDCOM pointer strings (e.g. @i54@), which are only unique within a single file, loading a second file into the same running context could return cached results from the previously-loaded file for any ID that happens to collide, even though the underlying data now belongs to a different person or family entirely.

Fix: call gedcom_ctx.clear_caches() as part of every load, immediately after parsing and before rebuilding lookups, so a reload always starts from a clean state.

Verified via repro script: loading File A and querying @i54@ populates person_details_cache (size 1); loading File B into the same context resets it to size 0 before any new query, confirming the stale entry is cleared rather than surviving the reload.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances GEDCOM source and citation handling by introducing a helper function _resolve_source_citation to extract detailed bibliographic and citation-specific fields, and refactoring _get_sources_internal to include event-level and family-level citations. It also fixes a bug in _get_notes_internal that caused subsequent notes to be silently dropped. The review feedback highlights that making gedcom_ctx a required parameter in _resolve_source_citation and decode_event_details introduces breaking changes for existing callers and tests, and recommends making it optional. Additionally, the feedback suggests guarding against potential AttributeErrors and ensuring type consistency for citation fields.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +390 to +445
def _resolve_source_citation(sour_element, gedcom_ctx) -> Dict[str, Any]:
"""Given a single SOUR child element (from a record, event, or attribute),
resolve it into full citation detail: bibliographic fields from the
referenced source record, plus page/quality/text/date/note from the
citation itself."""
source_ref = sour_element.get_value()
citation = {
"reference": source_ref,
"title": "",
"author": "",
"publication": "",
"repository": "",
"page": "",
"quality": "",
"text": "",
"citation_date": "",
"note": "",
}

def decode_event_details(element, event_tag: str) -> Dict[str, Any]:
# Resolve the referenced source record's bibliographic fields
source_elem = gedcom_ctx.source_lookup.get(source_ref)
if source_elem and hasattr(source_elem, "get_child_elements"):
for source_child in source_elem.get_child_elements():
tag = source_child.get_tag()
value = source_child.get_value()
if tag == "TITL":
citation["title"] = value or ""
elif tag == "AUTH":
citation["author"] = value or ""
elif tag == "PUBL":
citation["publication"] = value or ""
elif tag == "REPO":
citation["repository"] = value or ""

# Extract citation-specific detail from the SOUR element itself
if hasattr(sour_element, "get_child_elements"):
for citation_child in sour_element.get_child_elements():
tag = citation_child.get_tag()
value = citation_child.get_value()
if tag == "PAGE":
citation["page"] = value
elif tag == "QUAY":
citation["quality"] = value
elif tag == "NOTE":
citation["note"] = value
elif tag == "DATA":
if hasattr(citation_child, "get_child_elements"):
for data_child in citation_child.get_child_elements():
data_tag = data_child.get_tag()
data_value = data_child.get_value()
if data_tag == "TEXT":
citation["text"] = data_value or ""
elif data_tag == "DATE":
citation["citation_date"] = data_value or ""

return citation

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The new helper function _resolve_source_citation has a few issues:

  1. Breaking Change / Missing Default: gedcom_ctx is a required parameter, but if this function or decode_event_details is called without it (e.g., in existing tests or external integrations), it will raise a TypeError. Making it Optional[GedcomContext] = None preserves backward compatibility.
  2. Potential AttributeError: If gedcom_ctx is None, calling gedcom_ctx.source_lookup will raise an AttributeError. We should guard this lookup.
  3. Type Inconsistency: The citation dictionary is initialized with empty strings for all fields. However, page, quality, and note are assigned value directly, which can be None. This introduces type inconsistency (some fields are strings, others are None), which can cause downstream TypeErrors. Using value or "" ensures they remain strings.
def _resolve_source_citation(sour_element, gedcom_ctx: Optional[GedcomContext] = None) -> Dict[str, Any]:
    """Given a single SOUR child element (from a record, event, or attribute),
    resolve it into full citation detail: bibliographic fields from the
    referenced source record, plus page/quality/text/date/note from the
    citation itself."""
    source_ref = sour_element.get_value()
    citation = {
        "reference": source_ref,
        "title": "",
        "author": "",
        "publication": "",
        "repository": "",
        "page": "",
        "quality": "",
        "text": "",
        "citation_date": "",
        "note": "",
    }

    # Resolve the referenced source record's bibliographic fields
    source_elem = gedcom_ctx.source_lookup.get(source_ref) if gedcom_ctx else None
    if source_elem and hasattr(source_elem, "get_child_elements"):
        for source_child in source_elem.get_child_elements():
            tag = source_child.get_tag()
            value = source_child.get_value()
            if tag == "TITL":
                citation["title"] = value or ""
            elif tag == "AUTH":
                citation["author"] = value or ""
            elif tag == "PUBL":
                citation["publication"] = value or ""
            elif tag == "REPO":
                citation["repository"] = value or ""

    # Extract citation-specific detail from the SOUR element itself
    if hasattr(sour_element, "get_child_elements"):
        for citation_child in sour_element.get_child_elements():
            tag = citation_child.get_tag()
            value = citation_child.get_value()
            if tag == "PAGE":
                citation["page"] = value or ""
            elif tag == "QUAY":
                citation["quality"] = value or ""
            elif tag == "NOTE":
                citation["note"] = value or ""
            elif tag == "DATA":
                if hasattr(citation_child, "get_child_elements"):
                    for data_child in citation_child.get_child_elements():
                        data_tag = data_child.get_tag()
                        data_value = data_child.get_value()
                        if data_tag == "TEXT":
                            citation["text"] = data_value or ""
                        elif data_tag == "DATE":
                            citation["citation_date"] = data_value or ""

    return citation


return citation

def decode_event_details(element, event_tag: str, gedcom_ctx) -> Dict[str, Any]:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Adding gedcom_ctx as a required parameter to decode_event_details is a breaking change for any external callers or unit tests that call this function with only two arguments. Making it optional with a default value of None maintains backward compatibility.

Suggested change
def decode_event_details(element, event_tag: str, gedcom_ctx) -> Dict[str, Any]:
def decode_event_details(element, event_tag: str, gedcom_ctx: Optional[GedcomContext] = None) -> Dict[str, Any]:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant