Feat/retro console support#3
Open
mattsp1290 wants to merge 11 commits into
Open
Conversation
Plan to cross-compile netty for the Nintendo 3DS and validate a 3DS client conversing with a server on this machine's LAN IP. The socket layer is the entire porting risk (reliability protocol is platform-agnostic Nim). Plan front-loads a gating compile/link spike, then real when-defined(ds3) branches for: socket-fd creation (bypass Nim createNativeSocket's SOCK_CLOEXEC + IPPROTO_UDP, which SOCU rejects), acInit/socInit network bring-up, and getAddrInfo bypass on bind+send via inet_aton. Cross-compile config + libdl/librt stubs + build script copy the configy reference pattern. Incorporates two Opus review passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cross-compile netty for the 3DS (devkitARM + libctru) and provide a 3DS client + LAN-bound server for the converse test. All changes are behind `when defined(ds3)`; desktop builds and `nimble test` are unaffected. The reliability protocol is untouched — only the socket layer needed work, in src/netty/ctru3ds.nim and guarded branches in netty.nim: - create the fd via direct libctru socket(AF_INET,SOCK_DGRAM,0): SOCU rejects IPPROTO_UDP and Nim's createNativeSocket ORs in the undefined SOCK_CLOEXEC - acInit + bounded WiFi-association wait + socInit (1MiB page-aligned buffer) in newReactor; socExit/acExit via addExitProc - bypass libctru's incomplete getAddrInfo: raw bind(INADDR_ANY) and inet_aton+sendto for bind/send (rawSend and sendMagic) - bypass net.recvFrom / getLocalAddr, which reference struct sockaddr_in6 (libctru has no IPv6), with raw recvfrom/getsockname - nim.cfg @if ds3 block, libdl/librt stubs, -DAF_UNIX=1, --opt:none Verification: - Bar 1 (compile+link) VERIFIED: verify/ds3/spike.nim and examples/client_3ds.nim build to valid .3dsx; spike findings in verify/ds3/SPIKE-RESULTS.md. Desktop nimble test green. - Real-network converse VERIFIED on desktop: client -> this machine's LAN IP -> server(0.0.0.0) round-trips "you said:hi". - Bar 2 (3DS client <-> host converse) PENDING HARDWARE: the ds3 runtime FFI paths have no on-device coverage (Citra/Azahar don't emulate SOC). Procedure and first-checks in verify/ds3/README.md. Reviewed by two Opus plan reviews + a RedOwl implementation review (FFI signatures confirmed against libctru headers; no Critical findings). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First on-device run hit `bind() failed on 3DS`. SOCU rejects bind(INADDR_ANY:0) — the libctru examples bind gethostid() on a specific port, never INADDR_ANY/0. For a UDP client (ephemeral port 0) the right fix is to skip the explicit bind entirely: sendto auto-binds an ephemeral port. bindAny3ds now binds gethostid() for the server case, and 3DS socket errors include errno for diagnosis. Bar 2 PASS on a real 3DS: client 10.0.0.91 <-> server 10.0.0.135:1999 round-tripped "hi"/"you said:hi". The full ds3 path is now exercised on hardware (bring-up, socket(,0), skip-bind, inet_aton+sendto, non-blocking recvfrom, connId matching). See .agents/plans/3ds-support/RESULTS.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
INADDR_ANY:0 bind is rejected by SOCU; client skips bind, server binds gethostid(). Reflects the verified on-device behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Plan for an on-device test harness to close the §07 robustness items the basic converse skipped: large-message fragmentation, sustained ping-pong (in-order/no-loss/no-dup), clock/timing sanity, and idle/timeout/reconnect. A host echo server + a 3DS phase-machine client, with two-signal verification (sdmc result file + host log) and the polish items (server IP from sdmc:/netty_server.txt, optional SMDH, README pointer). Incorporates two Opus reviews. Fixes applied before commit: - B2 pipelined: do NOT anchor msg.sequenceNum to 0 — recvSequenceNum is cumulative (~303 by then); anchor to first observed seq + local index, key integrity to the local index. (Was a guaranteed false-FAIL.) - D2 silent-timeout: measure from conn.lastActiveTime, not silentStart (the latter gives ~7s and false-FAILs the [9,12] window). - Crash-safety: hold the Connection ref (never c.connections[0], which timeouts mutate -> IndexDefect); flush the report incrementally so a later-phase crash still leaves a partial artifact. - Corroboration: server emits an EXACT TOTAL on connection death so the host check is equality, not an approximate periodic STAT. - Grounding fixes: lastActiveTime set at netty.nim:347 (fires on ACKs too); Reactor.time is private so timing uses epochTime()/getMonoTime(). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements .agents/plans/3ds-robustness/: a shared phase machine plus host echo server and a 3DS client that exercise netty's reliability layer — large-message fragmentation, sustained ping-pong (lock-step + pipelined), clock/timing sanity, and idle/timeout/reconnect. - examples/robustness_phases.nim shared, pure-netty phases (no gfx/IO) so the desktop verifier and the 3DS client run identical logic - examples/server_robustness.nim verbatim echo + EVT/STAT + exact TOTAL - examples/client_robustness_desktop.nim host verifier (cmdline ip) - examples/client_3ds_robustness.nim 3DS run: sdmc:/netty_server.txt config, incremental sdmc:/netty_robustness.txt report, gfx shell - client_3ds.nim now reads the host IP from sdmc:/netty_server.txt (polish) - build_3ds.sh: gated optional SMDH icon (assets/netty_icon.png) — cosmetic, never required; README gains a 3DS section Embeds the reviewer fixes: pipelined ordering anchors to the first observed msg.sequenceNum + a local index (not 0; recvSequenceNum is cumulative); the silent-timeout is measured from conn.lastActiveTime; phases hold the Connection ref (never index connections[0]); report flushes incrementally; server emits an exact TOTAL for equality corroboration. Desktop-verified end to end (loopback): OVERALL=PASS, host TOTAL=809 matches the device's sent counts exactly. On-device 3DS run pending hardware. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3DS (10.0.0.91) over WiFi vs host echo server: all four phases PASS. Two-signal corroboration exact (host TOTAL=809 = 3+300+500+6, +1 reconnect). Real WiFi RTT 6-27ms, zero loss/dupes/gaps, silentTimeout=10.0s. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cross-compile netty for the PS Vita via VitaSDK and prove it through the compile+link gate, mirroring the shipped 3DS port. The whole porting risk is the socket layer; a gating spike (verify/vita) front-loaded it and decided the path. Spike verdict: Path B (direct sceNet FFI). Path A (Nim std/net under -d:vita) fails to compile -- nativesockets' createNativeSocket references SOCK_CLOEXEC, IPPROTO_RAW, IPPROTO_ICMPV6, none defined in VitaSDK newlib. Even though the raw BSD symbols link, the wrappers still need sceNetInit and the compat layer is historically incomplete. Path B is self-contained and mirrors ctru3ds.nim 1:1. Implementation: - src/netty/vitanet.nim: minimal sceNet/sceNetCtl/sysmodule FFI. Net bring-up (sysmodule(NET) + sceNetInit(1 MiB pool) + sceNetCtl link-state poll) via a once-guarded ensureNet() with addExitProc teardown. UDP socket via sceNetSocket, non-blocking via sceNetSetsockopt(SO_NBIO) (the fd is a sceNet id, not a newlib fd, so fcntl/setBlocking is skipped). Numeric address build via sceNetInetPton/ sceNetHtons into SceNetSockaddrIn -- imported BY NAME because its layout differs from BSD sockaddr_in (1-byte sin_family, sin_len + sin_vport). - src/netty.nim: each socket site is now a 3-way when ds3/vita/else, parallel to the existing ds3 branches (newReactor create/bind/getLocalAddr, rawSend, sendMagic, readParts recv). - nim.cfg: @if vita: block. -Wl,-q mandatory (reloc data-abort otherwise); -d:release (not 3DS's --opt:none); -DSOCK_CLOEXEC=0/IPPROTO_RAW/IPPROTO_ICMPV6 compile fixup; -lpthread (std/exitprocs); librt stub via -L. (-lrt fires, SDK lacks it); -ldl resolves against the SDK's real libdl (no stub). - scripts/build_vita.sh: graceful no-op without VitaSDK; nim c -d:vita -> vita-elf-create -> make-fself -> mksfoex -> .vpk. - examples/client_vita.nim (+ .nim.cfg): converse exerciser, psvDebugScreen vendored from the SDK samples (shim includes debugScreen.c directly -- this SDK gates the impl behind DEBUG_SCREEN_CODE_INCLUDE). - verify/vita/: the gating spike, SPIKE-RESULTS.md, README. Plan + RESULTS under .agents/plans/vita-support/. Bar 1 (link gate) PASSES: client_vita.vpk builds, vita-elf-create consumes the -Wl,-q relocations. Desktop tests and the 3DS build are unaffected. Bars 2/3 (Vita3K + physical hardware converse) are runtime-gated and pending hardware. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A vpk built by scripts/build_vita.sh, installed via VitaShell, completed the full netty round-trip over real WiFi from a physical Vita (10.0.0.220): server logged `GOT: hi from 10.0.0.220` and the Vita screen showed `REPLY: you said:hi`. This is the plan's definition of done. Empirically retires the runtime risks the link gate could not (sceNetInit + WiFi, SO_NBIO non-blocking recv, clock advancing / librt stub not shadowing clock_gettime, -Wl,-q relocations on a real loader). Host half independently validated by a host-to-host converse first. Bar 2 (Vita3K) left inconclusive and off the critical path; basic-converse scope, robustness is a separate follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extend the shared robustness phases (examples/robustness_phases.nim) with a Vita napMs branch (sceKernelDelayThread, microseconds) and add a Vita robustness client mirroring client_3ds_robustness.nim: runs the four §07 phases (large-message fragmentation, sustained lockstep + pipelined ping-pong, clock/timing sanity, idle/timeout/reconnect) against the verbatim-echo server_robustness.nim, printing per-phase PASS/FAIL via psvDebugScreen and best-effort writing ux0:data/netty_robustness.txt. Validated end-to-end on desktop against server_robustness.nim: OVERALL=PASS (large 2/4/9 parts; lockstep 300/300; pipelined 500/500 zero dupes/gaps/mism; monotonic clock; 18s heartbeat survival; ~10s idle timeout; reconnect), host log corroborating TOTAL msgs=809 largest=4096. Hardware run pending. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The four §07 robustness phases passed on a physical Vita (NETY00002) against the verbatim-echo server, with three agreeing signals: on-device ux0:data/netty_robustness.txt (OVERALL=PASS, all phases), on-screen OVERALL=PASS, and the host log showing the Vita (10.0.0.220) echoed 809 msgs / 11788 bytes / largest 4096 byte-for-byte identical to a known-PASS desktop reference. Exercises fragmentation/reassembly (4096 B -> 9 parts), 300 lockstep + 500 pipelined (in-order, no dup/loss), clock-rate sanity, and idle/timeout/reconnect. No netty core or vitanet changes needed. Vita writeFile to ux0:data/ also confirmed working. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.