fix(network/dns): default-config peer-name DNS resolves (CHAOS-1478)#19
Open
chrisgeo wants to merge 5 commits into
Open
fix(network/dns): default-config peer-name DNS resolves (CHAOS-1478)#19chrisgeo wants to merge 5 commits into
chrisgeo wants to merge 5 commits into
Conversation
apple/container's peer DNS returned NXDOMAIN by default because: - Utility.getAttachmentConfigurations gated the registered hostname on dns.domain and stored bare containerId form when unset. - ContainerDNSHandler passed canonical FQDN form (with trailing dot) to AttachmentAllocator.lookup, which did exact-string dictionary reads. - Mismatch: stored "probe-pg" vs query "probe-pg." → NXDOMAIN. Fix (belt-and-suspenders, no alias plumbing): 1. AttachmentAllocator normalizes trailing root-label dots in allocate/lookup/deallocate so registration is dot-insensitive. 2. Utility always registers the bare containerId form (drops the FQDN gate); the dnsDomain parameter on getAttachmentConfigurations is removed since registration no longer depends on it. 3. ContainerDNSHandler now receives dns.domain via init and strips both the trailing dot and the configured dns.domain suffix from question.name before lookup, so both nslookup foo and nslookup foo.<dns.domain> resolve to the same allocation. 4. APIServer+Start wires containerSystemConfig.dns.domain into the ContainerDNSHandler init. 5. NetworksService.lookup doc comment updated to reflect the new contract. No public schema changes. Multi-alias support (CHAOS-1476) intentionally out of scope.
…HAOS-1478) Polish on top of 23ef3d0: - Update the 'attach the first network using the fqdn' comment to describe the new behavior (always-bare registration, allocator-side normalization). - Use registrationHostname uniformly across all network attachments; this makes both arms of the previous offset == 0 guard identical, so collapse the guard. Behavior was already equivalent thanks to allocator normalization, but keeping two arms produced confusing dead code.
- AttachmentAllocator: normalization symmetry across allocate/lookup/ deallocate, single-dot scope. - ContainerDNSHandler.registrationKey: trailing-dot stripping, dns.domain suffix stripping with boundary-only matching, and dns.domain=nil/empty passthrough. - Utility.getAttachmentConfigurations: bare-form registration for plain, trailing-dot, and dot-internal containerIds; consistent hostname across multiple network attachments. Adds APIServerTests target (Package.swift) and widens ContainerDNSHandler.registrationKey to internal access for @testable visibility.
…Server lib to APIServerCore (CHAOS-1478) Follow-up to 19355b7. The previous test commit landed two parallel implementations of the registrationKey logic: - inline 'internal func registrationKey' on ContainerDNSHandler - 'DNSRegistrationKey.registrationKey' in the new APIServer library Tests covered the library copy, but production ran the inline copy — silent-drift risk if the two ever diverged. Changes: - Sources/APIServer/ContainerDNSHandler.swift now imports APIServerCore and delegates directly to DNSRegistrationKey.registrationKey from answerHost / answerHost6. Inline duplicate removed. - Library target renamed APIServer -> APIServerCore to avoid the module-name / type-name ambiguity with 'struct APIServer' (the @main executable's main type at Sources/APIServer/APIServer.swift). - Sources/APIServerLib -> Sources/APIServerCore (path matches target). - container-apiserver executable now depends on APIServerCore. - Test imports updated to '@testable import APIServerCore'. swift build clean. 32/32 tests pass across the three filtered suites.
…CHAOS-1478) Completes the CHAOS-1478 fix: ``container run --rm busybox nslookup <peer>`` now resolves a sibling container in the default configuration, with no ``sudo``, no ``container system dns create``, and no manual ``dns.domain`` configuration. The earlier source change (23ef3d0) made the embedded DNS handler register peer hostnames in their bare form. That fix was necessary but not sufficient: queries from inside containers still had no path to reach ``127.0.0.1:2053`` because ``vmnet``'s built-in DNS proxy on ``192.168.x.1:53`` forwards unmatched queries upstream rather than to the embedded handler. Three architectural options were evaluated. (A) Bind the handler on ``192.168.x.1:53`` and (B) run a UDP forwarder there from inside the ``container-network-vmnet`` plugin both turned out to be infeasible: ``com.apple.security.virtualization`` does not grant privileged-port binding, and every helper - apiserver and ``container-network-vmnet`` included - runs as the invoking uid (verified empirically via ``bind('192.168.x.1', 53)`` -> EACCES from uid 501). macOS 26 does expose ``vmnet_network_configuration_disable_dns_proxy()``, but disabling the proxy still leaves the handler unable to claim ``:53``. We therefore fall through to (C), the documented ``/etc/resolver`` mechanism, but automate it at install time so the user never sees a runtime sudo prompt. The chain at runtime: VM glibc/musl appends ``.test`` via ``search`` directive vmnet :53 proxy forwards ``probe-pg.test`` to macOS system resolver macOS resolver matches /etc/resolver/containerization.test, routes to 127.0.0.1 port 2053 apiserver :2053 strips ``.test`` (DNSRegistrationKey), looks up bare ``probe-pg``, returns 192.168.x.y Changes: - ``ContainerPersistence/ContainerSystemConfig.swift``: ``DNSConfig`` now defaults ``domain`` to ``\"test\"`` (centralised in ``defaultDomain``). Both the no-arg initialiser and the TOML decode path fall back to the default; an explicit nil/empty string from config is still honoured. The default propagates to the embedded handler's ``dnsDomain`` argument and to per-container DNS plumbing. - ``ContainerAPIService/Client/Utility.swift``: when constructing a container's ``DNSConfiguration`` and the user has not specified any search domains, inject ``[domain]`` so bare-name queries actually pick up the suffix and traverse the resolver chain above. Honours user overrides; explicit empty strings remain empty. - ``scripts/pkg-scripts/postinstall``: new ``.pkg`` postinstall script that writes ``/etc/resolver/containerization.test`` (filename and contents matching ``HostDNSResolver`` so existing add/remove tooling composes cleanly) and refreshes ``mDNSResponder``. The .pkg installer already runs as root, so this introduces no new sudo prompt. - ``Makefile``: pass ``--scripts scripts/pkg-scripts`` to ``pkgbuild`` so the postinstall is bundled into the installer. - ``scripts/uninstall-container.sh``: best-effort cleanup of the resolver entry on uninstall, keeping the host's ``/etc/resolver`` state symmetric with install. - ``Tests/ContainerPersistenceTests/ContainerSystemConfigDNSDefault\ Tests.swift``: 8 new unit tests covering the default value, explicit overrides, the empty-string opt-out, and the TOML decode paths (missing section, empty section, explicit override). The 32 tests from the earlier CHAOS-1478 commits continue to pass (40 / 40). Live repro on a clean container restart (debug build, macOS 26): $ container system stop && sudo make all install && container system start $ container run --rm -d --name probe-pg -e POSTGRES_PASSWORD=test postgres:alpine $ container run --rm busybox nslookup probe-pg Server: 192.168.65.1 Address: 192.168.65.1:53 Name: probe-pg.test Address: 192.168.65.8 $ dig @127.0.0.1 -p 2053 probe-pg. +short # key-form fix intact 192.168.65.8 $ dig @127.0.0.1 -p 2053 probe-pg.test. +short # suffix-strip works 192.168.65.8 $ dscacheutil -q host -a name probe-pg.test # /etc/resolver host path ip_address: 192.168.65.8 Constraints honoured: * No new public Codable schema changes; ``DNSConfig.domain`` remains ``String?`` and existing TOMLs decode unchanged. * No runtime sudo prompts. Sudo is required only at ``.pkg`` install time, which the user already accepts via ``installer -pkg``. * The 32 pre-existing CHAOS-1478 unit tests continue to pass. * CHAOS-1476 (multi-alias) remains out of scope.
50f4a43 to
359d70d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
nslookup probe-pgfrom a sibling container now resolves in the default configuration (nodns.domainset, nosudo, nocontainer system dns create) — previously returned NXDOMAIN.23ef3d0→11d7948): register peer hostnames in their bare form and strip the configureddns.domainsuffix at the embedded handler so bare and FQDN queries map to the same allocator key.50f4a43): automate the/etc/resolver/install at.pkgpostinstall time and inject the matchingsearchdirective into containers, so VM queries actually traverse the chain to127.0.0.1:2053.AttachmentAllocatorgains anormalizehelper that strips trailing root-label dots, making bare and FQDN-form keys equivalent at the storage boundary.Utility.getAttachmentConfigurationsdrops thedns.domain-gated FQDN path and always registers the barecontainerId; thednsDomainparameter is removed.ContainerDNSHandlerreceivesdns.domainviainitand strips both the trailing dot and the configured domain suffix fromquestion.namebefore callingnetworkService.lookup, so bothnslookup fooandnslookup foo.<dns.domain>resolve correctly.DNSConfig.domaindefaults to"test"(centralised indefaultDomain); containersearchDomainsauto-injects[domain]when empty; the.pkgpostinstall writes/etc/resolver/containerization.testso the macOS resolver chain forwards*.testqueries fromvmnet's DNS proxy to the embedded handler.Why
Today's behavior.
Utility.getAttachmentConfigurationsgates hostname registration on whetherdns.domainis configured. Whendns.domainis absent (the default), it stores the barecontainerId(e.g.,"probe-pg") inAttachmentOptions.hostname. Whendns.domainis set, it stores the FQDN form ("probe-pg.containers.local"). Meanwhile,ContainerDNSHandlerpassesquestion.name— which the DNS wire format always delivers in canonical FQDN form with a trailing root-label dot ("probe-pg.") — directly tonetworkService.lookup, which in turn callsAttachmentAllocator.lookupas an exact-string dictionary read. The stored key"probe-pg"never matches the queried key"probe-pg.".The user-facing failure. Running
nslookup probe-pgfrom a sibling container on the same network returnsNXDOMAIN. The container is reachable by IP; the allocation exists in the table. The failure is purely a key-form mismatch. This is the repro documented in CHAOS-1478: two containers on the default network, nodns.domainin~/.config/container/config.json, bare peer-name lookup fails every time.Why default-config should just work. Docker's libnetwork registers bare names by default on user-defined networks;
nslookup peerresolves without any domain configuration.apple/containerhas all the architectural pieces —AttachmentAllocator,ContainerDNSHandler,NetworksService.lookup— to do the same. The FQDN gate made the default flow inert: it registered the right key but the handler queried the wrong one. Removing the gate and normalizing at the allocator boundary closes the gap without touching multi-alias plumbing.Why the source fix alone wasn't enough. Live testing on macOS 26 after the source fix landed showed bare-name resolution still failed. Diagnosis:
dig @127.0.0.1 -p 2053 probe-pg.succeeded (the embedded handler did register and answer correctly), butnslookup probe-pgfrom inside a container still returned NXDOMAIN. Containers send their queries to192.168.x.1:53(the vmnet bridge gateway), andvmnet's built-in DNS proxy on:53forwards unmatched queries to upstream DNS rather than to the embedded handler — so peer names never reached127.0.0.1:2053to begin with.Three architectural options for routing. All evaluated, two ruled out empirically.
192.168.x.1:53directlybind('192.168.x.1', 53)from uid 501 returnsEACCES. Privileged port, andcom.apple.security.virtualizationdoes not grant:53binding.container-network-vmnetplugin/etc/resolver/<domain>at install time.pkgpostinstall already runs as root; no new runtime sudo prompt; reuses the existingHostDNSResolver-style file format sosystem dns create / deletetooling composes cleanly.(macOS 26 does expose
vmnet_network_configuration_disable_dns_proxy()(vmnet.h L1244), which would let us silence vmnet's proxy — but that still leaves the handler unable to claim:53, so it does not unblock A or B.)The routing chain at runtime, after this PR:
Changes
Source fix (already on this branch)
Sources/Services/ContainerNetworkService/Server/AttachmentAllocator.swift— adds privatestatic func normalize(_ hostname: String) -> Stringthat strips a single trailing root-label dot. All three mutating/reading entry points —allocate,deallocate,lookup— route through it.Sources/Services/ContainerAPIService/Client/Utility.swift— removes thednsDomainparameter fromgetAttachmentConfigurations. Replaces the two-armoffset == 0guard (identical output in both arms after allocator normalization) with a singleregistrationHostnamebinding that strips any accidental trailing dot fromcontainerId.Sources/APIServer/ContainerDNSHandler.swift— addsdnsDomain: String?stored property and correspondinginitparameter.answerHostandanswerHost6compute the lookup key by delegating toDNSRegistrationKey.registrationKey(for:dnsDomain:).Sources/APIServerCore/RegistrationKey.swift— new library target.container-apiserveris@mainand cannot be@testable-imported, so the pure-logic translation from a DNS question name to the bare allocator key lives here. NamedAPIServerCore(notAPIServer) to avoid colliding withstruct APIServer: AsyncParsableCommand.Sources/APIServer/APIServer+Start.swift— threadscontainerSystemConfig.dns.domainintoContainerDNSHandler.init. One line.Sources/Services/ContainerAPIService/Server/Networks/NetworksService.swift— doc comment update onlookup(hostname:)to document the new contract.Routing fix (
50f4a43)Sources/ContainerPersistence/ContainerSystemConfig.swift—DNSConfig.defaultDomain = "test"constant. Both the no-arg initialiser and the TOML decode path fall back to it whendomainis absent. Explicitniland explicit""from user TOML are still preserved (opt-out path).Sources/Services/ContainerAPIService/Client/Utility.swift— when constructing a container'sDNSConfigurationand the user has not specified any search domains, inject[domain]so bare-name queries pick up the suffix and traverse the resolver chain. Honours user overrides; explicit empty arrays remain empty.scripts/pkg-scripts/postinstall— new.pkgpostinstall script. Writes/etc/resolver/containerization.test(filename / contents matchingHostDNSResolverso existingsystem dns create | deletetooling composes cleanly) and refreshesmDNSResponder. The installer already runs as root; no new sudo prompt.Makefile— pass--scripts scripts/pkg-scriptstopkgbuildso the postinstall is bundled into the installer.scripts/uninstall-container.sh— best-effort removal of the resolver entry on uninstall, keeping/etc/resolver/state symmetric with install.Verification
Build
No new warnings introduced. Clean build on Apple silicon, macOS 26, Xcode 26.
Unit tests — 40 / 40 across 4 suites, all passing in 0.002s wall
AttachmentAllocatorTest(19 tests) — normalization symmetry acrossallocate/lookup/deallocate, idempotent re-allocation in canonical form, single-dot scope.ContainerDNSHandlerRegistrationKeyTest(8 tests) — trailing-dot stripping, configured-domain suffix stripping, label-boundary matching, anddnsDomain == nil/ empty-string passthrough.UtilityRegistrationTests(5 tests) — bare-form registration for plain, trailing-dot, and dot-internalcontainerIds; consistent hostname across multiple network attachments.ContainerSystemConfigDNSDefaultTests(8 tests, new in this commit) —DNSConfigdefault value, explicit overrides, the empty-string opt-out, and the TOML decode paths (missing section, empty section, explicit override).Live repro on macOS 26 (debug build, this branch)
Wire compatibility
No
Codableschema change toAttachmentConfiguration,AttachmentOptions,DNSConfig, or any persisted type.DNSConfig.domainremainsString?and existing TOMLs decode unchanged — the change is the default fallback when the key is absent. The droppeddnsDomainparameter onUtility.getAttachmentConfigurationsis internal to the API server process; it is not part of any public API, XPC message type, or on-disk format. The newAPIServerCorelibrary target is purely additive. Existing containers and networks running mid-upgrade are unaffected:AttachmentAllocator.normalizeaccepts both"foo"and"foo."as equivalent keys, so any entry stored in either form before the upgrade remains findable after it.Out of scope
LocalhostDNSHandler— that handler serves the macOS realhost resolver and follows a different code path.container-network-vmnet's reserved-mode variant).References
3b724ae):Utility.swift#L286-L323— the FQDN gateAttachmentAllocator.swift#L60-L62— the exact-match lookupContainerDNSHandler.swift#L78-L88— the canonical-form callervmnet.hL1009-L1027 —vmnet_network_configuration_create, default DNS proxy enabledvmnet.hL1244 —vmnet_network_configuration_disable_dns_proxy(added macOS 26)