Skip to content

fix(delete): lifecycle handling for inherited hosts#125

Open
berlikm wants to merge 29 commits into
OpensourceICTSolutions:developmentfrom
berlikm:fix/inherited-host-lifecycle
Open

fix(delete): lifecycle handling for inherited hosts#125
berlikm wants to merge 29 commits into
OpensourceICTSolutions:developmentfrom
berlikm:fix/inherited-host-lifecycle

Conversation

@berlikm

@berlikm berlikm commented Jul 14, 2026

Copy link
Copy Markdown

Summary

Replaces inherited-host lifecycle lookup by hostname with a durable ZabbixHostBinding model keyed by (Zabbix server, NetBox content type, object ID). All lifecycle operations (rename, exclusion, decommission, assignment removal, hard deletion) act on the bound hostid instead of hostname.

Safety properties

  • First sync creates a binding; second sync reuses the same hostid
  • Rename updates the Zabbix host in-place (same hostid, no duplicate)
  • Exclusion, decommission, and assignment removal delete by bound hostid
  • Hard deletion captures binding IDs before the object is deleted, enqueues via transaction.on_commit(), and retains failed bindings for bounded RQ retries
  • Managed-tag backfill adopts existing Zabbix hosts by nb_type/nb_id tags
  • Unmanaged same-name hosts and duplicate identity bindings fail closed
  • Legacy direct-assignment hostid values migrate into bindings on first sync
  • Both binding and legacy hostid fields use PositiveBigIntegerField

Reconciliation integration

The periodic sync candidate set is the union of:

currently covered objects ∪ objects with durable bindings

Objects that lose all assignments are revisited and retired. Deterministic RQ job IDs suppress overlapping active jobs.

Migration

0014_zabbixhostbinding

Dependencies

Stacks on PRs #115, #123, #124, #113. Merge last, after all dependencies.

Testing

  • 80 focused zero-touch/lifecycle tests passed
  • Full CI-isolated nbxSync suite: 1,128 tests passed
  • makemigrations --check: no changes detected
  • ruff format --check: all files formatted
  • E2E verified against real Zabbix 7.0.28: create, second-sync reuse, rename, decommission, reactivate, hard delete, auto-sync via background scheduler, multi-level inheritance, assignment removal, exclusion

Part of #121.

berlikm added 8 commits July 14, 2026 14:15
Adds Site, SiteGroup, and Region as valid assignment targets for all
ZabbixServerAssignment, ZabbixHostgroupAssignment, ZabbixTemplateAssignment,
ZabbixTagAssignment, ZabbixMacroAssignment, and ZabbixHostInventory objects.

This allows assignments made at the Site/SiteGroup/Region level to be
inherited by all devices and VMs at that site or below, via the existing
inheritance_chain mechanism.

Changes:
- Constants: Site/SiteGroup/Region added to ASSIGNMENT_MODELS, ASSIGNMENT_TYPE_TO_FIELD, PATH_LABELS
- Forms: All 7 assignment forms get Site/SiteGroup/Region DynamicModelChoiceField + FieldSet
- Settings: inheritance_chain extended with Site/SiteGroup/Region resolution paths
- Jobs: SyncHostJob._prepare_assignment() creates detached copies for inherited assignments
- HostSync: _get_sync_target() resolves actual Device/VM for Jinja2 rendering
- HostSync: get_tag_attributes() deduplicates tags by (tag, value) to prevent
  Zabbix 7 rejecting duplicate pairs inherited from multiple sources
- HostInterfaceSync: get_create_params() falls back to the device's primary IP
  when the interface has no IP assigned (e.g. when inherited from SiteGroup)
- HostInterfaceSync: errors are caught per-interface so one failing interface
  does not prevent the remaining interfaces and templates from being synced
- Inheritance: ConfigGroup interface expansion removed; interfaces are now
  resolved naturally via the inheritance chain with primary IP fallback
- SyncBase: _is_inherited_copy guard prevents save()/update_sync_info() on inherited copies
- Views: ZabbixSiteTabView, ZabbixSiteGroupTabView, ZabbixRegionTabView added
- Tests: 9 new tests for inheritance resolution

Tested: all 1076 existing tests pass. Verified end-to-end on dev NetBox.
- Filter inherited/direct ZabbixServerAssignment and ZabbixHostInterface by zabbixserver
- Resolve ConfigGroup interface IP from primary_ip4/primary_ip6
- Add seen-set cycle guard to resolve_path
- Fixes silent dropping of Site/SiteGroup assignments
Fixes inherited Site/SiteGroup/Region assignments reading custom fields from
the intermediate object instead of the actual Device/VM.
Adds a new exclude_tag plugin setting that allows excluding hosts from
Zabbix sync entirely via a ZabbixTag assigned to any object in the
inheritance chain (Role, Platform, Site, SiteGroup, Manufacturer, or
directly on a Device/VM).

When a ZabbixTagAssignment with a tag matching exclude_tag is present
in the resolved assignments, SyncHostJob skips sync entirely. If the
host was previously synced, it is deleted from Zabbix.

The tag itself is filtered before Jinja2 rendering in
get_tag_attributes(), so it never reaches Zabbix as a host tag.

The setting defaults to '' (disabled), so existing deployments are
unaffected. This follows the same pattern as no_alerting_tag — a
signal tag that controls sync behavior without being pushed to Zabbix.

Tested: 7 new tests covering exclusion enabled/disabled, tag present
absent, and delete-on-exclude behavior. All 1092 existing tests pass.
… in minimal.html

- zabbixconfigurationgroupassignment.py: Add Site, SiteGroup, Region as
  DynamicModelChoiceField with FieldSet tabs and Meta.fields entries.
  Matches the pattern used by the other 6 assignment forms.
- minimal.html: Add '+Add' buttons for Host Inventory and Configuration Group
  in the Miscellaneous panel. When no assignment exists, users see a blue
  Add button instead of just '-'. Links pre-fill assigned_object_type/id.
…ants

- Add _get_eligible_instances() to turn SiteGroup/Site/Region/Role/Platform/
  Manufacturer/DeviceType/Cluster/ClusterType assignments into Device/VM qs.
- Rewrite SyncObjectsJob.run() to enqueue eligible instances, deduplicate by
  (app_label, model, pk), and check sync_enabled on both assignment + server.
- Fix duplicate path: return -> continue.
- Update tests and docs for production interval recommendation (360 min).
@berlikm
berlikm force-pushed the fix/inherited-host-lifecycle branch 9 times, most recently from e39d16c to 32f0888 Compare July 14, 2026 22:48
@berlikm
berlikm force-pushed the fix/inherited-host-lifecycle branch from 4f68894 to 0c6268f Compare July 14, 2026 23:38
berlikm added 9 commits July 14, 2026 23:46
- Add _walk_ancestors() helper that yields an object and each parent
  until None, cycle-safe.
- In resolve_inherited_zabbix_assignments(), when a path ends with
  'parent', check the resolved object plus all ancestors; nearest
  ancestor wins via existing seen_* sets.
- Add tests for 3-level SiteGroup hierarchy.
Replaces hostname-based deletion fallback with a first-class binding model:
- ZabbixHostBinding maps (server, device/vm/vdc) -> hostid + hostname
- HostSync resolves/persists hostid via binding, backfills from managed tags,
  raises conflicts for unmanaged/ambiguous hosts, and migrates legacy direct
  assignment hostids into bindings
- SyncHostJob retires bound hosts when their server assignment disappears
- DeleteHostJob deletes by binding first, then legacy fallback

Adds test coverage for inherited creation, second-sync reuse, rename,
decommission, missing-remote idempotency, legacy migration, managed-tag
backfill, delete-job hard deletion, and unassigned-binding retirement.
@berlikm
berlikm marked this pull request as ready for review July 15, 2026 09:27
@berlikm

berlikm commented Jul 15, 2026

Copy link
Copy Markdown
Author

Commit history note

This PR currently shows 24 commits because it includes stacked commits from dependencies (#115, #113, #123, #124). Once the dependency PRs are merged into development, this branch will be rebased and squash-merged.

The actual durable-binding delta (after dependencies land) consists of ~7 commits:

  1. feat(binding): durable ZabbixHostBinding for host identity — the core model + migration
  2. fix(binding): reconcile hosts after assignment removal — candidate set union
  3. fix(binding): preserve delete job compatibility — legacy instance-arg support
  4. fix(binding): preserve legacy host ids on delete — capture unmigrated hostids
  5. fix(binding): preserve legacy sync compatibility — getattr defaults
  6. fix(exclusion): keep tag rendering contract — don't pass object to render()
  7. fix(exclusion): retire delete-mapped hosts — excluded hosts still retire

Full verification on the integration branch (integration/all-prs): 1,128 tests pass, E2E confirmed against Zabbix 7.0.28.

berlikm added a commit to berlikm/nbxsync that referenced this pull request Jul 15, 2026
# Conflicts:
#	nbxsync/utils/sync/hostsync.py
Adds ZabbixSyncBase._should_persist() returning False for transient
inherited copies (those carrying _is_inherited_copy). Replaces the 4
inline getattr reads in syncbase.py and the 1 read in hostsync.py
(_clear_direct_hostid / verify_hostinterfaces) with the helper call.

Centralizes the persistence guard so a future change to the inherited-copy
mechanism touches one method, not nine call sites. The model-level guard
in sync_info.py:25 stays as the backstop.

Adds two test_syncbase cases: _is_inherited_copy=True skips save() and
update_sync_info(); absent/False runs them as before.
@berlikm
berlikm force-pushed the fix/inherited-host-lifecycle branch 2 times, most recently from b0ad05c to 8790c40 Compare July 16, 2026 06:15
berlikm added 2 commits July 16, 2026 06:17
…lifecycle

# Conflicts:
#	nbxsync/utils/sync/hostsync.py
#	nbxsync/utils/sync/syncbase.py
…ompat

Three fixes on fix/inherited-host-lifecycle (after merge with
feature/configgroup-site-form which introduced _should_persist):

- synchost.py: narrow interface-sync except Exception to except RuntimeError
  so programming errors (TypeError/AttributeError) propagate instead of
  being swallowed. Adds TypeError-propagates + RuntimeError-continues tests.

- hostsync.py: replace 2 remaining _is_inherited_copy reads
  (_clear_direct_hostid, _clear_deleted_host_state) with _should_persist().

- deletehost.py: add explanatory comment on the legacy binding_ids compat branch.
@berlikm
berlikm force-pushed the fix/inherited-host-lifecycle branch from 3d1a054 to cacfcf1 Compare July 16, 2026 07:34
@berlikm

berlikm commented Jul 16, 2026

Copy link
Copy Markdown
Author

Conflict resolution note for rebase

When rebasing this PR after #110 and #115 land, there will be two conflicts in nbxsync/utils/sync/hostsync.py:

Conflict 1: host_binding import (line ~14)

Correct resolution: Take this PR's version (the import is needed for durable bindings).

Conflict 2: tag loop (line ~417)

Correct resolution: Combine all three concerns:

exclude_tag = getattr(self.pluginsettings, 'exclude_tag', '')
for assigned_tag in (self.context.get('all_objects', {}).get('tags', []) or []):
    if exclude_tag and assigned_tag.zabbixtag.tag == exclude_tag:
        continue
    value, _ = assigned_tag.render(object=sync_target)

Important: The render(object=sync_target) call from #110 must be preserved. This PR's "keep tag rendering contract" commit reverted it to render(), but that was written before #110 was merged. With both PRs in the integration, render(object=sync_target) is the correct version — it renders Jinja2 tags against the actual device being synced, not the assignment's source object.

Migration

After rebasing, rename 0014_zabbixhostbinding.py to the next available number after the template-rule migrations (#1170014, #1190015). The #117 template-rule migration should depend on this binding migration to keep the chain linear: 00130014_zabbixhostbinding0015_zabbixtemplaterule0016_zabbixtemplaterule_hostgroup_tag.

Verified in the integration-test branch: 1135 tests pass, E2E confirmed (create, sync, verify hostgroups/tags, delete, verify Zabbix removal).

Three related fixes for dynamic (Jinja2) hostgroup templates:

1. synchost.py: Skip safe_sync(HostGroupSync) for template-based
   assignments. Dynamic groups like {{ object.role.name }} cannot be
   rendered against the assignment's assigned_object (e.g. DeviceRole).
   They are created on-demand during HostSync.get_groups() with the
   actual device context.

2. hostgroupsync.py: Add try_create() override that returns None for
   unrenderable template assignments. Also early-return from
   get_name_value, get_create_params, and set_id when the template
   can't render, preventing errors in the Zabbix API call.

3. synchost.py: Wrap the final safe_sync(HostSync) in try/except so
   template conflicts (e.g. 'Cannot inherit item with key
   snmptrap.fallback') don't abort the entire sync. The host,
   interfaces, tags, and hostgroups are already synced by this point.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant