Feature/rust write support and fixes - #96
Open
hetii wants to merge 14 commits into
Open
Conversation
FATX_TIME_TO_HOUR masked the hour with 0xf (4 bits) and FATX_TIME_TO_MINUTE with 0x1f (5 bits). In the on-disk FAT time word the hour occupies 5 bits (0-23) and the minute 6 bits (0-59), so timestamps with hour >= 16 or minute >= 32 were silently corrupted on both read and write. Fix the masks (0x1f hour, 0x3f minute) in both the unpack and pack macros. No open issue tracks this defect; it is distinct from the host byte-order work tracked in mborgerson#62 (these masks corrupt timestamps even on little-endian hosts).
A sectors_per_cluster of 0 read from the superblock caused a division by zero (SIGFPE) in the geometry computation, and garbage values produced nonsense filesystems (blank root directories) on undersized or foreign images. Validate the field right after reading the superblock: it must be a non-zero power of two, at most 1024. Fixes mborgerson#55. Addresses the crash component of mborgerson#26 (the truncated 1 GB Chihiro images read garbage superblocks); the remaining part of mborgerson#26 is supporting the Chihiro partition layout itself.
fatxfs had no statfs callback, so df reported 0/0/0 and applications could not see free space before writing. Add fatx_get_fs_stat() to libfatx (total/free cluster counts derived from the FAT, so the numbers can never drift negative) and wire it into a .statfs implementation in fatxfs (block size = cluster size). Verified: df now tracks writes and deletions exactly; filling the volume shows 100% used and deleting returns the space. No open issue tracks this; reported by users as df showing zero/negative values when copying to a mounted volume.
The Rust crates were read-only. This ports the libfatx write path to the fatx crate and wires it into fatx-fuse: - fatx: cluster allocator with a per-fs rotating hint (like fatx_alloc_cluster, but without the thread-unsafe static), write-through FAT persistence, directory-slot claim/erase (keeps an end-of-directory terminator, grows directories by a cluster when full), create/unlink/ rmdir/rename/truncate/write path-level API, and statfs-style free-space reporting derived from the FAT (used = total - free can never go negative). - fatx: freshly allocated clusters are zero-filled so grown files never expose stale device bytes; out-of-space fails with Error::NoSpace before any structure is mutated; the hour/minute mask fix is mirrored in the Rust Time decoder, and packing helpers (to_fatx_encoding, now) are added. - fatx-fuse: create/mknod/mkdir/unlink/rmdir/rename/setattr(truncate)/ write/fsync/statfs FUSE ops; writes enabled by default with a new --read-only flag; the inode tracker learns rename/forget. Verified end-to-end against the C implementation: an image formatted by fatxfs (C), written through fatx-fuse (Rust), reads back byte-identical (md5) via fatxfs (C), including cross-directory renames. Closes mborgerson#73.
Building all components (libfatx, fatxfs against FUSE 2.x, the Rust crates), formatting and mounting images with fatxfs and fatx-fuse, partition/drive-letter selection, and pyfatx extraction — the per-component READMEs assume this knowledge.
fatx_write_dir() built the raw directory entry in an uninitialized stack struct and only filled the fields it knew about, so the unused part of the 42-byte filename field (and any padding) leaked arbitrary stack memory to disk. Fill the struct with 0xFF (the conventional FATX name-fill) before populating it. Also add a README note distinguishing FATX-OG (original Xbox, what this project implements) from FATX360/XTAF (Xbox 360) — the two on-disk formats share a name but are mutually incompatible.
Two mount-time usability fixes: - Resolve the device and log paths with realpath() before fuse_main(). libfuse daemonizes the process and chdir()s to /, so a relative path resolved after that point silently refers to a different location - the mount appears to succeed but the directory shows up empty. Fixes mborgerson#64. - When neither --drive nor --offset/--size is given, probe offset 0 for a FATX signature and mount the whole file as a bare image when found. Xbox Memory Unit dumps and single-partition images carry the superblock at offset 0, so they now mount with no extra options; disks without the signature keep defaulting to drive C. Addresses the auto-detection request in mborgerson#74. Verified: an XMU-style image mounts read/write with no options from a nested relative path, and a retail disk image still defaults to drive C.
--list probes every known retail partition offset for the FATX signature and prints what is mountable. --all mounts every detected partition under the mount point, one subdirectory per drive letter (x y z c e f), each as its own FUSE session; Ctrl-C unmounts everything cleanly. Supporting changes in the fatx crate: an F partition entry (0x1DD156000, 'F takes all' convention of oversized disks) and support for partition size 0 meaning 'extends to the end of the device'. Verified on a retail-formatted 8 GB image: all five partitions listed and mounted, files written into C and E through --all read back correctly via the C fatxfs driver mounted per partition. Closes mborgerson#76.
The Python bindings already wrap the full libfatx API, but the CLI could only list and bulk-extract. Add --cat, --get, --put (with overwrite), --rm, --mkdir, --mv and --extract-to DIR, so common one-file tasks no longer require writing Python. Addresses mborgerson#31. Verified end-to-end on a retail-formatted image: put -> cat/get round-trip (byte-identical), mkdir, mv, rm.
All FATX on-disk structures are little-endian, but the library read and wrote raw structs, so every multi-byte field (superblock, FAT entries, directory-entry cluster/size/timestamps, refurb info) was wrong on big-endian hosts. Introduce fatx_le16/32/64 helpers (no-ops on little-endian, __builtin_bswap on big-endian per __BYTE_ORDER__) and apply them at every disk I/O boundary. Fixes mborgerson#62. No behavior change on little-endian hosts (helpers compile away); verified by a full format/write/remount/md5 regression on x86_64, including a cross-check against an independent FATX implementation.
…endent crash Read the XBpartitioner-style partition table from sector 0 (magic '****PARTINFO****', up to 14 LE entries of name[16]/flags/lba_start/ lba_size). When present, --drive letters resolve through the table (entries map to letters by start LBA; entries past the retail region are F and G), which is the only way F/G drives get non-default sizes. Falls back to the fixed retail layout when no table exists. Closes mborgerson#75. Also fix a crash uncovered while testing: fatxfs allocated struct fatx_fs with malloc, but fs->log_handle is only assigned when --log is given, so the first debug print dereferenced uninitialized heap memory - a segfault that depends on prior heap contents (likely behind 'random' crash reports). Use calloc. Verified: a 12 GiB image with a table defining 1 GiB F and tail G mounts every letter through the table (C and Rust drivers), files written to G via one driver read back via the other.
Port the sector-0 partition-table support to the Rust crates: probe/ --list/--all and single mounts prefer table entries over the fixed layout (FatxFsConfig::offset_size added). With a table present, --all mounts all seven letters including custom-sized F and G. Part of mborgerson#75.
… image docs Three small items from the remaining issue backlog: - fatx_disk.c used plain fseek() (long offset) in two places, so on 32-bit builds disk-size queries and refurb-sector writes truncate offsets past 2 GiB even with _FILE_OFFSET_BITS=64. Use fseeko()/_fseeki64() like the rest of the codebase. Part of the 32-bit correctness work for mborgerson#54. - test.sh used fallocate, which fails on filesystems without preallocation support - notably Docker volumes mounted from a Windows host over 9p. truncate creates a sparse file everywhere, including macOS (the script already used it there). Fixes the failure mode reported in mborgerson#79. - Document mounting QCOW2/VHD/VMDK images through qemu-nbd in USAGE.md, which covers the use cases of mborgerson#39 without teaching every tool about image formats. Verified: full test.sh run passes (format, write, remount, sha256).
…upport Addresses the missing-features list in mborgerson#37: - Right-click 'Extract to...' on any file or directory extracts it ( recursively for directories) to a chosen local directory, reading through fatx_read in 1 MiB chunks. - The partition row's Size column now shows 'used / total MiB' derived from fatx_get_fs_stat (FAT-backed, cannot drift negative). - Partitions are resolved through an XBpartitioner table when present (custom F/G locations); without a table, F falls back to the fixed offset extending to the end of the disk. Unformatted slots are skipped with a clear message instead of 'Failed to open disk'. - Fix E partition length (0x131F00000 overshot the F boundary by ~12 MiB; the correct retail length is 0x1312D6000). Smoke-tested headless (QT_QPA_PLATFORM=offscreen) against a retail image and an XBpartitioner-table image with custom F/G.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hello.
This PR is the result of early work I started on my project https://github.com/hetii/reversefs
It is intended to address a significant portion of the reported issues and missing features.
I admit that this work is based on the Fable model, so it requires human quality control.
I also encourage you to check out my project https://hetii.github.io/reversefs/
It is intended to be able to mount Xbox OG and Xbox 360 drives and emulate them using a local filesystem.