Skip to content

security(core): harden tar extraction and cover archive path guards - #283

Merged
parthrohit22 merged 1 commit into
Second-Origin:devfrom
parthrohit22:security/tar-extraction-data-filter
Aug 8, 2026
Merged

security(core): harden tar extraction and cover archive path guards#283
parthrohit22 merged 1 commit into
Second-Origin:devfrom
parthrohit22:security/tar-extraction-data-filter

Conversation

@parthrohit22

Copy link
Copy Markdown
Collaborator

Summary

Uploaded archives are untrusted input — arbitrary users hand PARTHA a tarball and it extracts it to disk. The tar path called extractall(destination) with no filter.

The explicit member loop above it already rejects traversal, links and devices, so this was not exploitable as written. Two things still made it worth fixing:

  1. Nothing enforced containment at write time. The loop and the extraction were independent; if they ever disagreed — a member whose name passes _is_safe_child but resolves differently at write time — the extraction would win. That is precisely when a second control earns its place.
  2. Python 3.14 flips the default silently. The DeprecationWarning fired 1002 times per test run. On 3.14 the behaviour changes underneath the code rather than at a decision point.

While adding it I found the bigger gap: the traversal and link guards had no test coverage at all. Removing either one would have kept CI green.

Linked issue

None — found by triaging the deprecation warnings in the backend test run.

What changed

Backend

  • app/storage/local.py_safe_extract_tar now calls extractall(destination, filter="data"). Available since 3.12; backend requires >=3.12,<3.14.
  • tests/test_ingestion_resource_budgets.py — four new tests: tar .. traversal, tar absolute path, tar symlink member, zip .. traversal.
  • alembic.inipath_separator = os, settling the second warning.

No contract, schema, migration, or API change.

Acceptance criteria completed

  • Tar extraction hardened at write time, not only by pre-scan.
  • Previously-untested path guards now covered.
  • Attacks verified rejected and no file written outside the destination.
  • Two of three backend deprecation warnings eliminated.
  • All CI gates pass.

Testing performed

$ .venv/bin/python -m pytest tests
888 passed, 4 skipped in 50.36s          # was 884; +4 new tests

$ ruff check app          -> All checks passed!
$ ruff format --check app -> 126 files already formatted
$ mypy app                -> Success: no issues found in 126 source files

Direct exploit check against the real LocalStorage._safe_extract_tar, asserting both the rejection and that nothing landed on disk:

traversal  rejected -> Archive contains unsafe paths.                       leaked=False
absolute   rejected -> Archive contains unsafe paths.                       leaked=False
symlink    rejected -> Archive contains unsupported link or device entries. leaked=False

Deprecation warnings, before and after:

before: DeprecationWarning: tar archives ... 3.14 will filter        -> gone
before: DeprecationWarning: No path_separator found in configuration -> gone
still:  StarletteDeprecationWarning: httpx with starlette.testclient -> left alone (see below)

Honest note: I intended to prove the new tests fail with the guards removed, but the mutation step was interrupted mid-run. I verified the source file was unmodified (git diff shows only the one-line filter="data" change) and substituted the direct exploit harness above, which exercises the same guards without editing application code. So the tests are demonstrated to pass against real attacks, not demonstrated to fail without the guards. Slightly weaker evidence; stating it rather than glossing it.

Screenshots

Not applicable — backend only.

Security and data considerations

This is the security-relevant change in this PR, and it is defence in depth rather than a fix for a live vulnerability — the pre-scan already blocked these cases, and no exploit existed against dev. No PoC is included beyond the standard traversal/symlink shapes already public in CPython's own documentation.

Ingestion is the highest-value attack surface PARTHA has: it accepts untrusted archives from any authenticated user and writes them to the analysis sandbox. Adding coverage to guards that had none materially reduces the chance a future refactor silently removes containment.

No auth, owner-scoping, egress, secret-handling or migration changes. alembic.ini gains a configuration key only; no migration is edited and no schema moves.

Dependencies and blocked work

None.

Scope changes or remaining work

The third warning — Using httpx with starlette.testclient is deprecated; install httpx2 — is deliberately not addressed here. It requires swapping a test dependency, which needs owner approval per AGENTS.md, and it is test-infrastructure only with no runtime impact. Happy to file it as an issue.

Contributor checklist

  • Follows the branch naming and commit conventions
  • Targets dev
  • Verified locally with the commands above
  • No secrets, .env, dist/, or generated artifacts committed
  • No applied migration edited
  • Docs/behaviour claims match real repository state

Uploaded archives are untrusted input, and the tar path called extractall
without a filter. The explicit member loop already rejects traversal, links
and devices, but nothing enforced that at write time, and Python 3.14 will
change the default silently.

Pass filter=data so CPython applies its own extraction hardening as defence
in depth: absolute paths and .. traversal stripped, links, devices and
setuid metadata refused as members are written. Available since 3.12 and
the backend requires >=3.12,<3.14.

The traversal and link guards had no test coverage at all, so a regression
would have been silent. Add four tests covering tar traversal, tar absolute
path, tar symlink and the equivalent zip traversal, verified to reject the
attack with no file written outside the destination.

Also set path_separator=os in alembic.ini. Alembic warns when it is unset
and falls back to legacy splitting on spaces, commas and colons; os is the
correct reading of the single . entry and is stable when the fallback goes.
@parthrohit22
parthrohit22 merged commit cf352b6 into Second-Origin:dev Aug 8, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant