Tighten basic PII redaction boundaries

CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## Unreleased
+
+### Changed
+
+- Expanded the basic PII redaction profile to `basic-contact-v2`, adding common dashed Korean resident/foreigner registration number patterns and dotted Korean mobile-number formats.
+- PII-redacted document-summary and rendered-output cache entries cold-start after this redaction policy id change.
+- Documented current limitations and alternative boundaries, including provider prompt caching, embedding caches, LlamaIndex ingestion, and custom Redis/SHA256 cache tradeoffs.
+
 ## 0.5.0 - 2026-07-06
 
 ### Changed

README.ko.md

@@ -174,6 +174,27 @@ python -m document_briefing_cache.cli benchmark \
 - 섹션 순서나 ID가 계속 바뀌는 문서
 - 최신 뉴스, 가격, 정책, 법률처럼 외부 최신성이 중요한 요청
 
+## 한계와 대안
+
+Briefprint는 넓은 DLP 스캐너, URL crawler, RAG framework, provider 과금 profiler가 아닙니다.
+
+현재 한계:
+
+- PII redaction은 기본형입니다. `basic-contact-v2`는 이메일, `010-1234-5678`, `010 1234 5678`, `01012345678`, `010.1234.5678` 같은 한국 휴대폰 형식, 미국 전화번호, 하이픈이 있는 흔한 주민등록번호/외국인등록번호 모양을 다룹니다. 한국 이름, 카카오나 네이버 계정 ID, 주소, 계좌번호는 범위 밖입니다.
+- 기본 `rules` 요약기는 결정적 얕은 추출기입니다. 캐시 데모와 smoke check에는 유용하지만 의미론적 요약 품질을 보장하지 않습니다.
+- 캐시 파일은 plaintext JSON입니다. HMAC은 위변조를 탐지하지만 내용을 암호화하지 않습니다.
+- CLI 입력은 로컬 파일입니다. 원격 URL은 호출자가 먼저 가져와 로컬 파일이나 정규화된 payload로 넘겨야 합니다.
+- JSON 캐시는 write 때 atomic replacement를 쓰지만 프로세스 간 파일 lock은 없습니다. 여러 writer가 같은 cache dir을 공유해야 한다면 외부 lock을 추가하세요.
+- CLI 내장 LLM adapter는 OpenAI 전용입니다. Python 호출자는 Anthropic, Gemini, 내부 모델용 `BaseSummarizer` 구현체를 따로 주입할 수 있습니다.
+- 벤치마크 토큰 절약 수치는 로컬 harness 추정치이며 provider 과금 telemetry가 아닙니다.
+
+대안과 경계:
+
+- LangChain CacheBackedEmbeddings는 텍스트 해시로 embedding 계산 결과를 캐시합니다. embedding 재계산이 비싼 경우에는 유용하지만, 구조화 문서 요약 캐시는 아닙니다. Briefprint의 재렌더링 가능한 `DocumentSummaryState`도 제공하지 않습니다.
+- LlamaIndex `IngestionPipeline`과 docstore는 transformation cache와 document hash 추적을 갖춘 더 넓은 ingestion/RAG pipeline입니다. indexing과 retrieval이 필요하면 적합하지만, 반복 브리핑 렌더링만 필요하면 Briefprint가 더 작습니다.
+- OpenAI나 Anthropic provider prompt caching은 보완 관계입니다. provider prompt caching은 반복 prompt-prefix 처리 비용을 줄일 수 있고, Briefprint는 cache hit 문서의 summarizer 호출 자체를 피할 수 있습니다.
+- 직접 Redis와 SHA256으로 캐시를 만들 수도 있지만 schema versioning, redaction-policy keying, evidence validation, TTL, render key를 직접 설계해야 합니다.
+
 ## 입력 범위
 
 현재 CLI의 `--input`은 로컬 파일 경로를 받습니다. `http://` 또는 `https://` URL을 직접 가져오지는 않습니다.
@@ -232,7 +253,7 @@ python -m document_briefing_cache.cli run \
 
 민감 문서의 안전한 기본값은 영구 캐시를 남기지 않는 것입니다. `--sensitive`는 `--cache-policy ephemeral --no-output-cache --redact-pii --delete-on-exit created`의 편의 옵션입니다.
 
-`--redact-pii`는 `basic-contact-v1` 프로필을 적용합니다. 이메일, 한국 휴대폰 번호, 미국 전화번호를 다루지만 완전한 PII 탐지기는 아닙니다.
+`--redact-pii`는 `basic-contact-v2` 프로필을 적용합니다. 이메일, 한국 휴대폰 번호, 하이픈이 있는 흔한 주민등록번호/외국인등록번호 모양, 미국 전화번호를 다루지만 완전한 PII 탐지기는 아닙니다.
 
 `--redact-secrets`는 `basic-secrets-v1` 프로필을 적용합니다. bearer tokens, API keys, webhook URLs, card-like values, secret-shaped JSON keys를 best-effort로 가립니다. 비밀값 마스킹은 `--sensitive`에 포함되지 않으므로, 민감 문서에 비밀값이 들어갈 수 있으면 별도로 켜야 합니다.
 

README.md

@@ -174,6 +174,27 @@ Honest number warning: these are deterministic local estimates from the benchmar
 - Inputs with unstable section IDs or reordered sections, unless you pass structured records with stable IDs.
 - Requests that require fresh external facts, policy changes, current prices, or live news.
 
+## Limits And Alternatives
+
+Briefprint is intentionally not a broad DLP scanner, URL crawler, RAG framework, or provider billing profiler.
+
+Current limits:
+
+- PII redaction is basic. `basic-contact-v2` covers common email addresses, Korean mobile formats such as `010-1234-5678`, `010 1234 5678`, `01012345678`, and `010.1234.5678`, US phone numbers, and common dashed Korean resident or foreigner registration number patterns. Korean names, Kakao or Naver account IDs, addresses, and bank account numbers remain out of scope.
+- The default `rules` summarizer is deterministic shallow extraction. It is useful for cache demos and smoke checks, not semantic summary quality.
+- Cache files are plaintext JSON. HMAC detects tampering; it does not encrypt contents.
+- CLI inputs are local files. Remote URLs should be fetched by the caller and passed as local files or normalized payloads.
+- The JSON cache uses atomic replacement for writes but has no cross-process file lock. Avoid sharing one cache directory across concurrent writers unless you add external locking.
+- The built-in CLI LLM adapter is OpenAI-only. Python callers can provide another `BaseSummarizer` implementation for Anthropic, Gemini, or an internal model.
+- Benchmark token savings are local estimates from the harness, not provider billing telemetry.
+
+Alternative boundaries:
+
+- LangChain CacheBackedEmbeddings caches embedding calculations by text hash. It is useful when embedding recomputation is the expensive step, but it is not a structured document-summary cache and does not provide Briefprint's rerenderable `DocumentSummaryState`.
+- LlamaIndex `IngestionPipeline` with a docstore is a broader ingestion/RAG pipeline with transformation caches and document hash tracking. Use it when you need indexing and retrieval; Briefprint stays smaller for repeated briefing renders.
+- OpenAI or Anthropic provider prompt caching is complementary. Provider prompt caching can reduce repeated prompt-prefix processing cost; Briefprint can avoid cache-hit document summarizer calls entirely.
+- A custom Redis plus SHA256 cache can be flexible, but you need to design schema versioning, redaction-policy keying, evidence validation, TTLs, and render keys yourself.
+
 ## Input Scope
 
 The CLI `--input` option currently accepts local file path values. It does not fetch URLs such as `http://` or `https://`.
@@ -232,7 +253,7 @@ python -m document_briefing_cache.cli run \
 
 For sensitive documents, the safe default is no persistent cache. `--sensitive` is a convenience alias for `--cache-policy ephemeral --no-output-cache --redact-pii --delete-on-exit created`.
 
-`--redact-pii` applies the built-in `basic-contact-v1` profile before cache misses are summarized. It covers common email addresses, Korean mobile numbers, and US phone numbers. It is not a complete PII detector.
+`--redact-pii` applies the built-in `basic-contact-v2` profile before cache misses are summarized. It covers common email addresses, Korean mobile numbers, common dashed Korean resident or foreigner registration number patterns, and US phone numbers. It is not a complete PII detector.
 
 `--redact-secrets` applies the built-in `basic-secrets-v1` profile. It is best-effort and targets bearer tokens, API keys, webhook URLs, card-like values, and string values under secret-shaped JSON keys. Secret redaction is not included in --sensitive; enable it explicitly when secret-shaped values may appear.
 

references/best-practices.md

@@ -71,7 +71,7 @@ python -m document_briefing_cache.cli run \
 
 `--sensitive` expands to `--cache-policy ephemeral --no-output-cache --redact-pii --delete-on-exit created`.
 
-Use `--redact-pii` when basic contact information should not reach LLM cache-miss calls or local cache files. Redaction is a profile, so include its policy id in document and output cache keys. The built-in `basic-contact-v1` redaction profile covers common email addresses, Korean mobile numbers, and US phone numbers. It is not a complete PII detector for names, addresses, national IDs, account numbers, cards, API keys, or access tokens.
+Use `--redact-pii` when basic contact information should not reach LLM cache-miss calls or local cache files. Redaction is a profile, so include its policy id in document and output cache keys. The built-in `basic-contact-v2` redaction profile covers common email addresses, Korean mobile numbers, common dashed Korean resident or foreigner registration number patterns, and US phone numbers. It is not a complete PII detector for names, addresses, account numbers, cards, API keys, access tokens, or general account IDs.
 
 Use `--redact-secrets` when bearer tokens, API keys, webhook URLs, card-like values, or string values under secret-shaped JSON keys may appear. It applies the built-in `basic-secrets-v1` profile. Secret redaction is best-effort and is not included in `--sensitive`; enable it explicitly when secrecy matters more than preserving exact operational identifiers.
 

src/document_briefing_cache/privacy.py

@@ -7,13 +7,14 @@
 
 
 NO_REDACTION_POLICY_ID = "none"
-REDACTION_POLICY_ID = "basic-contact-v1"
+REDACTION_POLICY_ID = "basic-contact-v2"
 SECRET_REDACTION_POLICY_ID = "basic-secrets-v1"
 
 
 PII_PATTERNS: tuple[tuple[str, re.Pattern[str], str], ...] = (
     ("email", re.compile(r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b"), "[REDACTED:email]"),
-    ("phone", re.compile(r"\b01[016789][-\s]?\d{3,4}[-\s]?\d{4}\b"), "[REDACTED:phone]"),
+    ("kr-registration-id", re.compile(r"(?<!\d)\d{6}-[1-8]\d{6}(?!\d)"), "[REDACTED:kr-registration-id]"),
+    ("phone", re.compile(r"\b01[016789][-.\s]?\d{3,4}[-.\s]?\d{4}\b"), "[REDACTED:phone]"),
     ("phone", re.compile(r"\b(?:\+?1[-.\s]?)?\(?\d{3}\)?[-.\s]\d{3}[-.\s]\d{4}\b"), "[REDACTED:phone]"),
 )
 SECRET_ASSIGNMENT_PATTERN = re.compile(

tests/test_docs.py

@@ -69,9 +69,10 @@ def test_readme_documents_redaction_scope_and_security_limits():
     best_practices = (ROOT / "references" / "best-practices.md").read_text(encoding="utf-8")
     combined = "\n".join([readme, skill, best_practices])
 
-    assert "basic-contact-v1" in combined
+    assert "basic-contact-v2" in combined
     assert "email" in combined
     assert "Korean mobile" in combined
+    assert "Korean resident or foreigner registration number" in combined
     assert "US phone" in combined
     assert "not a complete PII detector" in combined
     assert "--cache-policy ephemeral" in combined
@@ -82,6 +83,30 @@ def test_readme_documents_redaction_scope_and_security_limits():
     assert "not encryption" in combined
 
 
+def test_readmes_document_limits_and_alternative_boundaries():
+    readme = (ROOT / "README.md").read_text(encoding="utf-8")
+    korean = (ROOT / "README.ko.md").read_text(encoding="utf-8")
+
+    assert "## Limits And Alternatives" in readme
+    assert "Kakao or Naver account IDs" in readme
+    assert "bank account numbers remain out of scope" in readme
+    assert "no cross-process file lock" in readme
+    assert "The built-in CLI LLM adapter is OpenAI-only" in readme
+    assert "CacheBackedEmbeddings caches embedding calculations by text hash" in readme
+    assert "not a structured document-summary cache" in readme
+    assert "provider prompt caching is complementary" in readme
+
+    assert "## 한계와 대안" in korean
+    assert "카카오나 네이버 계정 ID" in korean
+    assert "주민등록번호/외국인등록번호" in korean
+    assert "계좌번호는 범위 밖" in korean
+    assert "프로세스 간 파일 lock은 없습니다" in korean
+    assert "CLI 내장 LLM adapter는 OpenAI 전용" in korean
+    assert "CacheBackedEmbeddings는 텍스트 해시로 embedding 계산 결과를 캐시" in korean
+    assert "구조화 문서 요약 캐시는 아닙니다" in korean
+    assert "provider prompt caching은 보완 관계" in korean
+
+
 def test_readme_documents_secret_redaction_scope_and_sensitive_boundary():
     readme = (ROOT / "README.md").read_text(encoding="utf-8")
 

tests/test_hashing.py

@@ -38,9 +38,9 @@ def test_cache_keys_change_by_redaction_policy():
     fingerprint = document_content_fingerprint(docs[0])
 
     raw_summary_key = document_summary_cache_key(docs[0], fingerprint, "rules", "0.2", redaction_policy_id="none")
-    redacted_summary_key = document_summary_cache_key(docs[0], fingerprint, "rules", "0.2", redaction_policy_id="basic-contact-v1")
+    redacted_summary_key = document_summary_cache_key(docs[0], fingerprint, "rules", "0.2", redaction_policy_id="basic-contact-v2")
     raw_output_key = output_cache_key(docs, "brief", "general", "ko-KR", "0.2", "t1", "rules", redaction_policy_id="none")
-    redacted_output_key = output_cache_key(docs, "brief", "general", "ko-KR", "0.2", "t1", "rules", redaction_policy_id="basic-contact-v1")
+    redacted_output_key = output_cache_key(docs, "brief", "general", "ko-KR", "0.2", "t1", "rules", redaction_policy_id="basic-contact-v2")
 
     assert raw_summary_key != redacted_summary_key
     assert raw_output_key != redacted_output_key

tests/test_privacy.py

@@ -4,7 +4,7 @@
 
 from document_briefing_cache.models import CacheConfig, DocumentInput
 from document_briefing_cache.pipeline import BriefingPipeline
-from document_briefing_cache.privacy import redact_document_input, redact_pii_text, redact_secret_text
+from document_briefing_cache.privacy import redaction_policy_id, redact_document_input, redact_pii_text, redact_secret_text
 from document_briefing_cache.summarizers import RuleBasedExtractiveSummarizer
 
 
@@ -193,6 +193,25 @@ def test_pii_redaction_preserves_non_pii_protected_values():
     assert "183 ms" in redacted
 
 
+def test_pii_redaction_covers_kr_registration_ids_and_dotted_mobile_without_names_or_accounts():
+    text = "홍길동 RRN 900101-1234567, ARN 900101-5123456, mobile 010.1234.5678, account 110-123-456789."
+
+    redacted, count = redact_pii_text(text)
+
+    assert count == 3
+    assert "900101-1234567" not in redacted
+    assert "900101-5123456" not in redacted
+    assert "010.1234.5678" not in redacted
+    assert "[REDACTED:kr-registration-id]" in redacted
+    assert "[REDACTED:phone]" in redacted
+    assert "홍길동" in redacted
+    assert "110-123-456789" in redacted
+
+
+def test_pii_redaction_policy_id_changes_when_profile_scope_changes():
+    assert redaction_policy_id(redact_pii=True) == "basic-contact-v2"
+
+
 def test_secret_redaction_covers_common_tokens_and_preserves_operational_values():
     text = (
         "Action: Rotate api_key=sk_test_123456789abcdef and Authorization: Bearer abcdef1234567890. "