Cherry-pick three high-value upstream fixes

[virtaava]

Summary

Three targeted cherry-picks from bytedance/deer-flow upstream main (which we are 362 commits behind, with major restructure). All paths re-mapped from upstream's backend/packages/harness/deerflow/ layout to our backend/src/ layout. Each commit credits the original upstream author and PR.

Commits

1. [security] fix(uploads): require explicit opt-in for host-side document conversion

Upstream: 80e210f5 (bytedance#2332). CVE-class. Without this gate, every uploaded PDF/PPT/Excel/Word file is fed through markitdown on the host process. markitdown depends on a stack of PDF/Office parsers; bugs there are sandbox-escape vectors when the input is attacker-controlled.

• New config flag uploads.auto_convert_documents (default false)
• config.example.yaml documents the option with the security rationale
• Conversion call site in upload_files() gated on the flag

2. fix(memory): cache corruption, thread-safety, and caller mutation bugs

Upstream: 898f4e8a (bytedance#2251). Three bugs all present in our fork's backend/src/agents/memory/updater.py:

• Bug 1 — caller mutation: _apply_updates(current_memory, ...) mutates in place, so a subsequent save failure leaves the cached object partially mutated. Fix: copy.deepcopy(current_memory) before passing.
• Bug 2 — cache races: _memory_cache reads/writes without a lock; the background memory-update timer thread races with HTTP reload calls. Fix: new _cache_lock (threading.Lock) wrapping every access.
• Bug 3 — silent caller-dict mutation: _save_memory_to_file() did memory_data["lastUpdated"] = ... on the caller's dict. Fix: shallow copy ({**memory_data, "lastUpdated": ...}).

Local adaptation: upstream split storage from updater into a separate storage.py which our fork doesn't have. Same three patterns appear in our consolidated updater.py and were patched in place.

3. fix(subagent): event loop conflict in SubagentExecutor.execute()

Upstream: e5b14906 (bytedance#1965). When execute() is called from inside an already-running event loop (an async parent agent), asyncio.run() creates a new loop that conflicts with asyncio primitives bound to the parent loop (httpx clients, etc.).

• New get_isolated_loop_pool() lazy thread pool
• execute() detects a running loop and submits the work to that pool
• Cleanup of the isolated loop in finally (cancel pending tasks, shutdown asyncgens, restore previous loop)

Local adaptation: our fork uses lazy pool init via get_*_pool() helpers — the new isolated pool follows the same pattern.

Why these three

From the 362-commit lag survey, these were the highest-value fixes that we certainly care about:

• The security fix is exploitable today on any deployment exposed to untrusted uploads.
• The memory bugs cause silent data loss in the long-term memory store we already rely on.
• The subagent fix prevents event-loop crashes in async parent agents — which is exactly the call pattern Hermes uses.

What this PR does NOT do

• Does not attempt the broader upstream sync (the major refactor 76803b82 would conflict with most files in our fork).
• Does not touch present_file_tool.py (upstream f4c17c66) — partially overlaps with our open PR fix: middleware crashes when run is created without a context dict #1 (the middleware context fallback) and is better folded into that helper if we adopt the same pattern there.
• Does not bring in the subagent feature work (30d619de, ac04f270) or the LLM circuit breaker (4d4ddb3d) — those are nice but not urgent.

Verified

• Python syntax check on all four changed files passes
• Diffs reviewed against upstream commits to confirm semantic equivalence
• Runtime smoke-test on this branch is non-trivial because virtaava/main is one commit behind your local main (the unpushed feat: harness ACI tools commit + your config.yaml aren't here), but the deerflow-langgraph service starts cleanly with the cherry-picks once you have a valid config.yaml in place

Suggested merge order

PR #1 (middleware context fallback) first, then this one — they don't conflict, but the order keeps the history readable.

🤖 Generated with Claude Code