Extract people and organizations from memory content, not just metadata - #13
Open
ali-ulu wants to merge 1 commit into
Open
Extract people and organizations from memory content, not just metadata#13ali-ulu wants to merge 1 commit into
ali-ulu wants to merge 1 commit into
Conversation
Entity extraction only ever read connector metadata (attendees, from/to, speakers). That covers calendar/email/transcript imports and leaves the most common case empty: a note typed by hand, a commit message, a captured decision. Those memories produced zero entities, so the graph, People/Organizations views and the trust corroboration signal all stayed blank for anyone not importing a mailbox — the demo corpus was carrying the whole demonstration. Adds server/core/text_entities.py: deterministic, high-precision text signals only (no NER model, no LLM, no I/O), consistent with the rest of the entity layer: - e-mail addresses in the body -> person (+ organization via domain) - @handle mentions -> person - "Acme Inc" / "Zephyr Labs" -> organization - "met with X" / "X ile görüştüm" -> person Precision is favoured over recall: a noisy graph is worse than a sparse one. The prose heuristics are skipped entirely for source code, detected by extension, by connector path, or by the shape of the content itself, so a repo import can't mint an entity per capitalised identifier. A company matched by prose folds into its e-mail domain when both appear in one memory, so "Zephyr Labs" doesn't fork a node next to zephyrlabs.io. Wired in at the single shared point (people_in_memory), so the graph, People and Organizations views all pick it up at once. Known limitation, unchanged by this commit: entity resolution is still per-key, so a company named in prose in one memory and by e-mail domain in another remains two nodes. Cross-memory alias merging needs its own design.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Entity extraction only ever read connector metadata (
attendees,from/to,speakers). That covers calendar/email/transcript imports but left the most common case empty: a note typed by hand, a commit message, a captured decision. Those memories produced zero entities, so the knowledge graph, People/Organizations views, and the trust corroboration signal all stayed blank for anyone not importing a mailbox — the demo corpus was carrying the whole demonstration, and removing it emptied the product.Adds
server/core/text_entities.pywith deterministic, high-precision text signals only (no NER model, no LLM, no I/O — consistent with the rest of the entity layer):@handlementionAcme Inc/Zephyr Labsmet with X/X ile görüştümWired in at a single shared point (
people_in_memory), so the graph, People, and Organizations views all pick it up at once.Precision over recall. A noisy graph is worse than a sparse one, so:
Zephyr Labsdoesn't fork a node next tozephyrlabs.io.Known limitation (unchanged by this PR): entity resolution is still per-key, so a company named in prose in one memory and by e-mail domain in another remains two nodes. Cross-memory alias merging needs its own design and isn't attempted here.
The demo-corpus assertion in
test_seed_demo.pymoves from 21→22 entities: the one addition is the prose-namedZephyr Labsorganization, which has no e-mail domain to key off.Test plan
python -m pytest tests/ -q— 620 passed (17 new)Generated by Claude Code