Skip to content

M5-2d1: close descriptor ownership at the request boundary - #97

Open
mgrossmann wants to merge 1 commit into
mainfrom
m5-2d1-ownership
Open

M5-2d1: close descriptor ownership at the request boundary#97
mgrossmann wants to merge 1 commit into
mainfrom
m5-2d1-ownership

Conversation

@mgrossmann

Copy link
Copy Markdown
Contributor

Closes the sharpest thing in the tree, open since M5-2a: an unauthorised client in
another address space could drive another client's sockets through the documented SVC —
and the descriptor is not merely forgeable but guessable in tens of attempts (one
socket table, sock_alloc returns the lowest free index regardless of who asks,
descriptor (gen<<16)|idx with idx in 0..63, so a client calibrates against its own
and walks the neighbours).

Design: ADR-0046. Spec §17.3 now carries the status of all three surfaces it named.
Depends on the survey in #96 (docs-only) for its evidence base, but does not require it
to merge first.


1. The kickoff's §1 was wrong, and a deadlock found it — not a review

The instruction prescribed identity → app slot → token → compare against the socket's
apptok
, without saying how a request reaches its app slot. The only existing route
is r->apptok — the field the same section forbade two paragraphs earlier.

I implemented it as prescribed and test_roundtrip deadlocked: 0.0 % CPU, sample(1)
showing wait_parked → pthread_cond_wait while the executive idled. A blocking RECVFROM
was refused EBADF before it could park, so nothing ever woke the waiter.

Root cause: r->apptok is not populated on most requests. src/nsfeza.c sets it on
exactly three verbs of about twenty — INITAPI reads it back (:177), SOCKET (:214) and
TERMAPI (:617) set it — and leaves it zero everywhere else. The check refused BIND,
CONNECT, SEND and RECV for every honest client. It would not have been a check even
where the field is set, since the client supplies it.

The direction that works reverses the arrow: from the socket. The socket names its
owning app slot, the slot names the address space that opened it, and that is compared
with the FLIH-captured (ascb, asid). No client-supplied value takes part, and the
one-to-many reverse lookup disappears because it is never performed.

Pinned in both directions: a client drives its own socket while presenting junk in
apptok and succeeds; and a client presenting another's stolen token together with
their descriptor
— the most favourable input a token-comparing check could get — is
still refused.

2. Two scopes, deliberately different — the paragraph that prevents a later "fix"

what scoped by why
teardown (RQ_TERMAPIterm_one, src/nsfreq.c:483) the token two INITAPIs from one AS share an ASCB, so app 1's TERMAPI must not destroy app 2's sockets — the reason the M5-2c memo kept the token
access control (nsfreq_sock_owned) the address space that is where the protection boundary is: one AS is one key and one storage image, so a finer split buys nothing an attacker cannot bypass by reading memory

Both are right and they do not conflict, but someone finding only one will align it to the
other, and either direction of that is a defect. In ADR-0046 §2.3 as well as here,
and the permissive case is pinned by an assertion that allows it ("same address space, second instance: ALLOWED"), so tightening it later costs one named line and the
test says so out loud.

3. Shape

One check function, nsfreq_sock_owned, which resolves AND checks — a function
handing back a SOCKCB for the caller to validate is how this hole came to exist — with
exactly two callers: req_socket, and sel_scan (the wider door: one descriptor per
SELECT mask item, and the one nobody had looked at). soc_complete's lookup is declared
internal.

  • Foreign ≡ unknown by construction: same return NULL, and all nine req_socket
    callers map it to NSF_EBADF identically. SELECT's foreign entry takes the existing
    s == NULL branch — silently not-ready, rest of the mask served, no error. A
    distinguishable refusal would make SELECT an existence oracle. No new errno anywhere.
  • The token is still authenticated at the boundary — not for ownership, but to protect
    do_socket's s->apptok = r->apptok stamp. RQ_INITAPI exempt explicitly, not by
    ordering.
  • Enumeration guard tools/check-sock-lookup.sh (CI job sock-lookup-callers) fails
    the build on an unclassified sock_lookup caller. The rule is not "add a check" but
    "a new resolution point must be classified by a person". It carries its own positive
    control — finding zero call sites exits 2, since a broken pattern and a clean tree are
    otherwise identical. Tests carry no marker at all, so none can be mistaken for coverage.
  • The parked re-scan needed its own test: every other SELECT test uses the poll form,
    which never parks, so nsfsel_on_notify — a different path reading the identity from a
    different source — would have been covered by nothing. The identity is captured in
    sel_alloc, so busy implies an identity and there is no window defaulting fail-open.
  • The one piece of hidden state fails closed: g_cur_ascb/g_cur_asid are set at
    dispatch entry and deliberately not cleared — a stale read denies (visible) where a
    cleared read would skip the check (silent).

sock_lookup keeps its signature; nsfsoc.c and below learn nothing. NSFRQE frozen at
64 bytes
, anchor unmoved, ANCVERNO 3.

4. Verification — host-verified and live-verified, separated

Host / offline — verified

  • make test-host 3414 → 3469 PASS / 0 FAIL
  • Revert test, three states, exactly the ownership assertions moving: 0 FAIL →
    12 FAIL (TSTREQ 4, TSTSEL 8) → 0 FAIL. The middle state renders the hole
    positively, not as an absence:
    FAIL: B on A's descriptor -> refused (got 0, want 9)
    FAIL: B presenting A's STOLEN TOKEN and A's descriptor -> still refused (got 0, want 9)
    FAIL: SELECT counted exactly ONE ready socket (got 2, want 1)
    FAIL: A's readiness does NOT complete B's parked SELECT (got 1)
    
    got 0 is success — B drove A's socket. Every control assertion passed in all three
    states.
  • Guard verified to discriminate: unclassified caller added to src/nsfudp.c → rc=1
    naming file and line; removed → rc=0.
  • Alias scan 246 unique, all ≤ 8 chars. Cross-build clean: 6 modules + 53 test modules.

Live — nothing. The column is empty.

No part of this ran on MVS. Everything changed is in the Phase-1 NSF module, where
the identity is zero and the check is inert, so every host assertion here concerns the
zero-identity path except the new ones that pass an identity explicitly.

5. OPEN LIVE-GATE ITEM — the inherited child, and it is not met

"§5.1 first" is not satisfied, and cannot be host-side. What ran is (a) a socket
stamped by hand to model tcp_child_create — that models the stamp, it is not an
accepted TCP connection — and (b) TSTTCP's real accept path, green at 841, which runs at
zero identity so the check is inert there too. Both are no-regression evidence only.

The real gate is a cross-AS accept, and it is live. A check that is too strict breaks
every accepted connection, which is every server this stack exists for, and nothing has
tested that yet
. It is the first case of d1's live round.

6. Scope held

#67 stays c3's, d2 stays decoupled, and Q1's robustness half stays reclassified and open.

An unauthorised client in another address space could drive another client's
sockets through the documented SVC, open since M5-2a. req_socket was
sock_lookup(r->sockdesc) and nothing else, and sock_lookup validates the table
index and the slot generation and nothing about who is asking. The descriptor is
not merely forgeable but GUESSABLE: one socket table, sock_alloc returns the
lowest free index regardless of who asks, and the descriptor is (gen<<16)|idx
with idx in 0..63 -- so a client calibrates against its own and walks the
neighbours in tens of attempts.

There were two resolution points and only one was obvious. sel_scan resolves one
descriptor per SELECT mask item and never goes near req_socket, so it probes
many descriptors per request; it is the wider door and it is the one nobody had
looked at.

Resolution now runs through ONE function, nsfreq_sock_owned, which resolves AND
checks -- a function handing back a SOCKCB for the caller to validate is how
this hole came to exist -- with exactly two callers, req_socket and sel_scan.
soc_complete's lookup is declared internal: it resolves the request's own socket
during completion, so a check there would be the stack checking itself.

THE PRESCRIBED COMPARISON WAS WRONG AND A TEST FOUND IT. Comparing s->apptok
against r->apptok deadlocked test_roundtrip: the process sat at 0.0% CPU with
sample(1) showing wait_parked -> pthread_cond_wait, because a blocking RECVFROM
was refused EBADF before it could park and nothing woke the waiter. The cause is
that r->apptok is not populated on most requests -- nsfeza.c sets it on three
verbs of about twenty (INITAPI reads it back, SOCKET and TERMAPI set it) and
leaves it zero elsewhere, so the check refused BIND, CONNECT, SEND and RECV for
every honest client. It would not have been a check even where the field is set,
since the client supplies it.

The direction that works reverses the arrow: the socket names its owning app
slot, the slot names the address space that opened it, and that is compared with
the (ascb, asid) the SVC routine captured from the FLIH. No client-supplied
value takes part, and the one-to-many reverse lookup disappears because it is
never performed. Pinned by an assertion in which a client drives its own socket
while presenting junk in apptok and succeeds.

TWO SCOPES, DELIBERATELY DIFFERENT. Teardown stays scoped by TOKEN, because two
INITAPIs from one address space share an ASCB and app 1's TERMAPI must not
destroy app 2's sockets. Access control is scoped by ADDRESS SPACE, because that
is where the protection boundary is -- one AS is one key and one storage image,
so a finer split buys nothing an attacker cannot bypass by reading memory. Both
are right and they do not conflict, but someone finding only one will align it
to the other and either direction of that is a defect. ADR-0046 2.3 carries it,
and the permissive case is pinned by an assertion that ALLOWS it, so tightening
it later costs one named line.

A foreign descriptor returns NULL through the same return statement as an
unknown one, and all nine req_socket callers map NULL to EBADF identically, so
no caller can distinguish them. SELECT's foreign entry takes the existing
s == NULL branch: silently not ready, count unaffected, rest of the mask served,
no error on the call. A distinguishable refusal would turn SELECT into an
existence oracle for descriptor numbers. No new errno anywhere.

The token is still authenticated at the boundary, not for ownership but to
protect do_socket's s->apptok = r->apptok stamp: without it a client could not
USE a foreign socket but could still CREATE one labelled as another app's.
RQ_INITAPI is exempt explicitly and not by ordering, since it arrives with no
token and writes apptok as its output.

tools/check-sock-lookup.sh (CI job sock-lookup-callers) fails the build on an
unclassified sock_lookup caller. The rule it encodes is not "add a check" but "a
new resolution point must be classified by a person" -- req_socket never had one
and nobody saw sel_scan. It carries its own positive control: finding zero call
sites exits 2, because a broken pattern and a clean tree are otherwise
identical. Scope is production code; tests carry no marker at all, so none can
be mistaken for coverage.

The parked SELECT re-scan needed its own test. Every other SELECT test uses the
poll form, which never parks, so nsfsel_on_notify -- a different path reading
the identity from a different source -- would have been covered by nothing. The
identity is captured in sel_alloc, so busy implies an identity and there is no
window in which a SELCB is parked with a zero identity, which would have
defaulted fail-open.

g_cur_ascb/g_cur_asid are set at dispatch entry and deliberately NOT cleared: a
stale read denies (fail-closed, visible) where a cleared read would skip the
check (fail-open, silent).

sock_lookup keeps its signature and nsfsoc.c and below learn nothing. NSFRQE
stays frozen at 64 bytes, the anchor layout does not move, ANCVERNO stays 3.

Host 3414 -> 3469 PASS / 0 FAIL. Revert test in three states with exactly the
ownership assertions moving (0 -> 12 FAIL -> 0), the middle state rendering the
hole positively rather than as an absence. Guard verified to discriminate.
Alias scan 246 unique <= 8 chars; cross-build clean, 6 modules + 53 test
modules.

NOT LIVE-VERIFIED. Everything here is in the Phase-1 NSF module, where the
identity is zero and the check is inert, so every host assertion concerns the
zero-identity path except the new ones that pass an identity explicitly. The
inherited-child case is met as no-regression only -- the real gate is a
cross-AS accept, and it is live.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant