Skip to content

Misc improvements#3778

Merged
lferran merged 2 commits into
mainfrom
better-ingestion-logcs
Jul 8, 2026
Merged

Misc improvements#3778
lferran merged 2 commits into
mainfrom
better-ingestion-logcs

Conversation

@lferran

@lferran lferran commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Description

  • Add missing @logs_foreign_key_error decorator.
  • Add kbid and rid extra to logs where possible, to make it easier to query in Grafana later on.
  • Also, removed some mypy warnings about unchecked functions.

How was this PR tested?

Describe how you tested this PR.

@lferran lferran requested review from a team as code owners July 8, 2026 08:11
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.78788% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.42%. Comparing base (76e1ac1) to head (f0a2ade).

Files with missing lines Patch % Lines
nucliadb/src/nucliadb/purge/__init__.py 82.35% 3 Missing ⚠️
nucliadb/src/nucliadb/ingest/processing.py 33.33% 2 Missing ⚠️
...adb/src/nucliadb/ingest/orm/processor/processor.py 66.66% 1 Missing ⚠️
nucliadb/src/nucliadb/ingest/service/writer.py 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3778      +/-   ##
==========================================
+ Coverage   85.25%   85.42%   +0.16%     
==========================================
  Files         571      571              
  Lines       50435    50428       -7     
  Branches    15291    15291              
==========================================
+ Hits        43000    43078      +78     
+ Misses       6802     6718      -84     
+ Partials      633      632       -1     
Flag Coverage Δ
nidx 80.94% <ø> (-0.02%) ⬇️
nucliadb 73.94% <72.72%> (+0.28%) ⬆️
nucliadb-ingest 43.85% <33.33%> (-0.01%) ⬇️
nucliadb-reader 43.69% <21.21%> (+0.01%) ⬆️
nucliadb-search 54.84% <24.24%> (+0.01%) ⬆️
nucliadb-standalone 46.03% <24.24%> (+0.01%) ⬆️
nucliadb-train 44.68% <24.24%> (+0.01%) ⬆️
nucliadb-writer 47.25% <24.24%> (-0.01%) ⬇️
nucliadb_dataset 73.40% <ø> (ø)
nucliadb_models 69.72% <ø> (ø)
nucliadb_sdk 82.28% <ø> (ø)
nucliadb_telemetry 85.67% <ø> (ø)
nucliadb_utils 80.87% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@lferran lferran changed the title Better ingestion structured logs Misc improvements Jul 8, 2026
if shard_id is None:
logger.warning(f"Resource {uuid} does not exist")
logger.warning(
"Resource shard not found: Skipping delete", extra={"kbid": kbid, "rid": uuid}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we log the missing shard id too?

Comment on lines 313 to +314
logger.info(
"Resource does not exist: skipping txn", extra={"kbid": kbid, "slug": message.slug}
"Resource slug not found: skipping txn", extra={"kbid": kbid, "slug": message.slug}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't happen. Should it be a warning?

@lferran lferran merged commit b8f8914 into main Jul 8, 2026
45 checks passed
@lferran lferran deleted the better-ingestion-logcs branch July 8, 2026 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants