M5-2d1: close descriptor ownership at the request boundary - #97
Open
mgrossmann wants to merge 1 commit into
Open
M5-2d1: close descriptor ownership at the request boundary#97mgrossmann wants to merge 1 commit into
mgrossmann wants to merge 1 commit into
Conversation
An unauthorised client in another address space could drive another client's sockets through the documented SVC, open since M5-2a. req_socket was sock_lookup(r->sockdesc) and nothing else, and sock_lookup validates the table index and the slot generation and nothing about who is asking. The descriptor is not merely forgeable but GUESSABLE: one socket table, sock_alloc returns the lowest free index regardless of who asks, and the descriptor is (gen<<16)|idx with idx in 0..63 -- so a client calibrates against its own and walks the neighbours in tens of attempts. There were two resolution points and only one was obvious. sel_scan resolves one descriptor per SELECT mask item and never goes near req_socket, so it probes many descriptors per request; it is the wider door and it is the one nobody had looked at. Resolution now runs through ONE function, nsfreq_sock_owned, which resolves AND checks -- a function handing back a SOCKCB for the caller to validate is how this hole came to exist -- with exactly two callers, req_socket and sel_scan. soc_complete's lookup is declared internal: it resolves the request's own socket during completion, so a check there would be the stack checking itself. THE PRESCRIBED COMPARISON WAS WRONG AND A TEST FOUND IT. Comparing s->apptok against r->apptok deadlocked test_roundtrip: the process sat at 0.0% CPU with sample(1) showing wait_parked -> pthread_cond_wait, because a blocking RECVFROM was refused EBADF before it could park and nothing woke the waiter. The cause is that r->apptok is not populated on most requests -- nsfeza.c sets it on three verbs of about twenty (INITAPI reads it back, SOCKET and TERMAPI set it) and leaves it zero elsewhere, so the check refused BIND, CONNECT, SEND and RECV for every honest client. It would not have been a check even where the field is set, since the client supplies it. The direction that works reverses the arrow: the socket names its owning app slot, the slot names the address space that opened it, and that is compared with the (ascb, asid) the SVC routine captured from the FLIH. No client-supplied value takes part, and the one-to-many reverse lookup disappears because it is never performed. Pinned by an assertion in which a client drives its own socket while presenting junk in apptok and succeeds. TWO SCOPES, DELIBERATELY DIFFERENT. Teardown stays scoped by TOKEN, because two INITAPIs from one address space share an ASCB and app 1's TERMAPI must not destroy app 2's sockets. Access control is scoped by ADDRESS SPACE, because that is where the protection boundary is -- one AS is one key and one storage image, so a finer split buys nothing an attacker cannot bypass by reading memory. Both are right and they do not conflict, but someone finding only one will align it to the other and either direction of that is a defect. ADR-0046 2.3 carries it, and the permissive case is pinned by an assertion that ALLOWS it, so tightening it later costs one named line. A foreign descriptor returns NULL through the same return statement as an unknown one, and all nine req_socket callers map NULL to EBADF identically, so no caller can distinguish them. SELECT's foreign entry takes the existing s == NULL branch: silently not ready, count unaffected, rest of the mask served, no error on the call. A distinguishable refusal would turn SELECT into an existence oracle for descriptor numbers. No new errno anywhere. The token is still authenticated at the boundary, not for ownership but to protect do_socket's s->apptok = r->apptok stamp: without it a client could not USE a foreign socket but could still CREATE one labelled as another app's. RQ_INITAPI is exempt explicitly and not by ordering, since it arrives with no token and writes apptok as its output. tools/check-sock-lookup.sh (CI job sock-lookup-callers) fails the build on an unclassified sock_lookup caller. The rule it encodes is not "add a check" but "a new resolution point must be classified by a person" -- req_socket never had one and nobody saw sel_scan. It carries its own positive control: finding zero call sites exits 2, because a broken pattern and a clean tree are otherwise identical. Scope is production code; tests carry no marker at all, so none can be mistaken for coverage. The parked SELECT re-scan needed its own test. Every other SELECT test uses the poll form, which never parks, so nsfsel_on_notify -- a different path reading the identity from a different source -- would have been covered by nothing. The identity is captured in sel_alloc, so busy implies an identity and there is no window in which a SELCB is parked with a zero identity, which would have defaulted fail-open. g_cur_ascb/g_cur_asid are set at dispatch entry and deliberately NOT cleared: a stale read denies (fail-closed, visible) where a cleared read would skip the check (fail-open, silent). sock_lookup keeps its signature and nsfsoc.c and below learn nothing. NSFRQE stays frozen at 64 bytes, the anchor layout does not move, ANCVERNO stays 3. Host 3414 -> 3469 PASS / 0 FAIL. Revert test in three states with exactly the ownership assertions moving (0 -> 12 FAIL -> 0), the middle state rendering the hole positively rather than as an absence. Guard verified to discriminate. Alias scan 246 unique <= 8 chars; cross-build clean, 6 modules + 53 test modules. NOT LIVE-VERIFIED. Everything here is in the Phase-1 NSF module, where the identity is zero and the check is inert, so every host assertion concerns the zero-identity path except the new ones that pass an identity explicitly. The inherited-child case is met as no-regression only -- the real gate is a cross-AS accept, and it is live.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the sharpest thing in the tree, open since M5-2a: an unauthorised client in
another address space could drive another client's sockets through the documented SVC —
and the descriptor is not merely forgeable but guessable in tens of attempts (one
socket table,
sock_allocreturns the lowest free index regardless of who asks,descriptor
(gen<<16)|idxwithidxin 0..63, so a client calibrates against its ownand walks the neighbours).
Design: ADR-0046. Spec §17.3 now carries the status of all three surfaces it named.
Depends on the survey in #96 (docs-only) for its evidence base, but does not require it
to merge first.
1. The kickoff's §1 was wrong, and a deadlock found it — not a review
The instruction prescribed identity → app slot → token → compare against the socket's
apptok, without saying how a request reaches its app slot. The only existing routeis
r->apptok— the field the same section forbade two paragraphs earlier.I implemented it as prescribed and
test_roundtripdeadlocked: 0.0 % CPU,sample(1)showing
wait_parked → pthread_cond_waitwhile the executive idled. A blockingRECVFROMwas refused
EBADFbefore it could park, so nothing ever woke the waiter.Root cause:
r->apptokis not populated on most requests.src/nsfeza.csets it onexactly three verbs of about twenty — INITAPI reads it back (
:177), SOCKET (:214) andTERMAPI (
:617) set it — and leaves it zero everywhere else. The check refused BIND,CONNECT, SEND and RECV for every honest client. It would not have been a check even
where the field is set, since the client supplies it.
The direction that works reverses the arrow: from the socket. The socket names its
owning app slot, the slot names the address space that opened it, and that is compared
with the FLIH-captured
(ascb, asid). No client-supplied value takes part, and theone-to-many reverse lookup disappears because it is never performed.
Pinned in both directions: a client drives its own socket while presenting junk in
apptokand succeeds; and a client presenting another's stolen token together withtheir descriptor — the most favourable input a token-comparing check could get — is
still refused.
2. Two scopes, deliberately different — the paragraph that prevents a later "fix"
RQ_TERMAPI→term_one,src/nsfreq.c:483)INITAPIs from one AS share an ASCB, so app 1's TERMAPI must not destroy app 2's sockets — the reason the M5-2c memo kept the tokennsfreq_sock_owned)Both are right and they do not conflict, but someone finding only one will align it to the
other, and either direction of that is a defect. In ADR-0046 §2.3 as well as here,
and the permissive case is pinned by an assertion that allows it (
"same address space, second instance: ALLOWED"), so tightening it later costs one named line and thetest says so out loud.
3. Shape
One check function,
nsfreq_sock_owned, which resolves AND checks — a functionhanding back a
SOCKCBfor the caller to validate is how this hole came to exist — withexactly two callers:
req_socket, andsel_scan(the wider door: one descriptor perSELECT mask item, and the one nobody had looked at).
soc_complete's lookup is declaredinternal.
return NULL, and all ninereq_socketcallers map it to
NSF_EBADFidentically. SELECT's foreign entry takes the existings == NULLbranch — silently not-ready, rest of the mask served, no error. Adistinguishable refusal would make SELECT an existence oracle. No new errno anywhere.
do_socket'ss->apptok = r->apptokstamp.RQ_INITAPIexempt explicitly, not byordering.
tools/check-sock-lookup.sh(CI jobsock-lookup-callers) failsthe build on an unclassified
sock_lookupcaller. The rule is not "add a check" but"a new resolution point must be classified by a person". It carries its own positive
control — finding zero call sites exits 2, since a broken pattern and a clean tree are
otherwise identical. Tests carry no marker at all, so none can be mistaken for coverage.
which never parks, so
nsfsel_on_notify— a different path reading the identity from adifferent source — would have been covered by nothing. The identity is captured in
sel_alloc, sobusyimplies an identity and there is no window defaulting fail-open.g_cur_ascb/g_cur_asidare set atdispatch entry and deliberately not cleared — a stale read denies (visible) where a
cleared read would skip the check (silent).
sock_lookupkeeps its signature;nsfsoc.cand below learn nothing. NSFRQE frozen at64 bytes, anchor unmoved,
ANCVERNO3.4. Verification — host-verified and live-verified, separated
Host / offline — verified
make test-host3414 → 3469 PASS / 0 FAIL12 FAIL (TSTREQ 4, TSTSEL 8) → 0 FAIL. The middle state renders the hole
positively, not as an absence:
got 0is success — B drove A's socket. Every control assertion passed in all threestates.
src/nsfudp.c→ rc=1naming file and line; removed → rc=0.
Live — nothing. The column is empty.
No part of this ran on MVS. Everything changed is in the Phase-1
NSFmodule, wherethe identity is zero and the check is inert, so every host assertion here concerns the
zero-identity path except the new ones that pass an identity explicitly.
5. OPEN LIVE-GATE ITEM — the inherited child, and it is not met
"§5.1 first" is not satisfied, and cannot be host-side. What ran is (a) a socket
stamped by hand to model
tcp_child_create— that models the stamp, it is not anaccepted TCP connection — and (b) TSTTCP's real accept path, green at 841, which runs at
zero identity so the check is inert there too. Both are no-regression evidence only.
The real gate is a cross-AS accept, and it is live. A check that is too strict breaks
every accepted connection, which is every server this stack exists for, and nothing has
tested that yet. It is the first case of d1's live round.
6. Scope held
#67 stays c3's, d2 stays decoupled, and Q1's robustness half stays reclassified and open.