Secure attachments#82
Open
adambalogh wants to merge 4 commits into
Open
Conversation
Stop flattening user content to text in the enclave. Convert OpenAI-format content parts (text / image_url / file) into LangChain v1 standard content blocks so images and PDFs reach the provider natively instead of being dropped. Text-only content still collapses to a plain string. No new dependencies: the pinned langchain-* versions already translate standard image/file blocks to each provider's native API.
… attachments in request hash - validate_attachments(): reject image/PDF parts when the target model's LangChain profile explicitly lacks support (fails open for unknown models), and enforce a 30 MB inline attachment cap. Wired into create_chat_completion so it covers both the direct and OHTTP-inner paths. - Request hashing now canonicalizes multimodal user content, replacing inline base64 with a sha256 digest so the signed request commits to the exact attachment bytes without bloating the hashed payload.
Contributor
There was a problem hiding this comment.
Pull request overview
Enables native multimodal (image / PDF) attachments through the enclave instead of flattening user content to text, adds a per-request attachment cap and per-model capability gating, and changes request-hash canonicalization to digest attachment bytes rather than inlining megabytes of base64. A companion design doc describes the broader OHTTP attachment flow.
Changes:
llm_backend.py: replaces user-content flattening with a converter to LangChain v1 standardimage/fileblocks, plusvalidate_attachments()(capability gating + 30 MiB size cap) andAttachmentValidationError.chat_controller.py: callsvalidate_attachments()before dispatch and adds_canonical_user_content()to substitute attachment base64 with asha256digest in the hashed request payload.- Adds tests for conversion (text/image/PDF/URL), provider compatibility, validation, and canonical hashing; adds a design doc for native attachments over OHTTP.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
tee_gateway/llm_backend.py |
New multimodal content-block conversion, attachment size cap, and capability-aware validation. |
tee_gateway/controllers/chat_controller.py |
Pre-dispatch attachment validation and digest-based canonicalization of user content for request hashing. |
tee_gateway/test/test_tee_core.py |
Tests for converter output, cross-provider acceptance, validation behavior, and canonical hashing. |
docs/native-attachments-design.md |
Design proposal for native attachments over the OHTTP private path. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+53
to
+56
| # Hard cap on total inline (base64) attachment bytes per request, enforced | ||
| # regardless of model. Inline base64 rides inside the encrypted payload, so this | ||
| # bounds the request size the enclave will accept. | ||
| MAX_ATTACHMENT_BYTES = 30 * 1024 * 1024 # 30 MB |
Comment on lines
+673
to
+676
| if "base64" in block: | ||
| entry["sha256"] = hashlib.sha256( | ||
| block["base64"].encode("utf-8") | ||
| ).hexdigest() |
Comment on lines
+33
to
+35
| validate_attachments, | ||
| AttachmentValidationError, | ||
| _convert_content_part, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.