Skip to content

feat(oci): support conditional object uploads - #267

Open
felix0102 wants to merge 7 commits into
thanos-io:mainfrom
felix0102:feat/oci-conditional-writes
Open

feat(oci): support conditional object uploads#267
felix0102 wants to merge 7 commits into
thanos-io:mainfrom
felix0102:feat/oci-conditional-writes

Conversation

@felix0102

Copy link
Copy Markdown
Contributor

What this changes

Adds conditional write support to the OCI object storage provider:

  • expose object ETags through Attributes
  • support WithIfMatch
  • support WithIfNotExists
  • recognize OCI precondition failures
  • add tests for conditional request headers and errors

Why

OCI Object Storage supports If-Match and If-None-Match: *,
but the objstore OCI provider does not currently expose these through
ObjectUploadOption. This prevents consumers from safely performing
optimistic-concurrency updates.

Notes

IfNotMatch is not advertised because OCI PutObject only accepts *
for If-None-Match.

@felix0102

Copy link
Copy Markdown
Contributor Author

I found and fixed an OCI multipart-specific issue while running the acceptance test against a real OCI Object Storage bucket.

OCI SDK's UploadStream always uses multipart uploads for non-empty streams. Passing If-Match through that path caused the condition to be applied to UploadPart, resulting in a 412 IfMatchFailed even when the destination object's ETag was correct.

Conditional uploads now use a single PutObject request, while unconditional uploads continue to use UploadStream/multipart.

Verified against a real OCI bucket:

go test -v -count=1 ./objtesting
-run '^TestObjStore_AcceptanceTest_e2e$/^oci$'

Result:
PASS
--- PASS: TestObjStore_AcceptanceTest_e2e/oci

This verifies IfNotExists, invalid IfMatch, valid IfMatch, content preservation after failed conditions, and successful conditional replacement.

@felix0102
felix0102 force-pushed the feat/oci-conditional-writes branch from d631d7f to 04153b2 Compare August 5, 2026 03:38
@felix0102
felix0102 marked this pull request as draft August 5, 2026 04:14
@felix0102
felix0102 marked this pull request as ready for review August 5, 2026 04:15
@felix0102
felix0102 force-pushed the feat/oci-conditional-writes branch from 04153b2 to 0a3ed48 Compare August 5, 2026 04:23
@felix0102
felix0102 force-pushed the feat/oci-conditional-writes branch from d6f2d60 to cbbe9c2 Compare August 6, 2026 01:42
@felix0102

Copy link
Copy Markdown
Contributor Author

Hi @alsenz, this PR adds OCI support for the conditional upload API introduced in #178. Since you designed and implemented that API, would you be willing to take a look and provide technical feedback on the OCI implementation when available? Thank you!

@felix0102

Copy link
Copy Markdown
Contributor Author

Hi maintainers — a gentle follow-up on this PR.

I have now confirmed that this change also addresses the OCI connection leak previously reported in grafana/loki#17818.

I reproduced the issue against a real OCI Object Storage bucket using Instance Principal authentication. With the existing GetObject-based Attributes() implementation and the HTTP transport limited to five connections per host:

  • requests 1–5 completed quickly;
  • request 6 was blocked for approximately 6.7 seconds, waiting for a connection to become available.

After changing OCI metadata operations to use HeadObject, all ten requests completed in approximately 6–31 ms, with request 6 completing in 9.3 ms. This confirms that the HeadObject change prevents the response-body leak and connection-pool starvation.

The OCI provider unit tests and the full acceptance test against a real OCI Object Storage bucket also pass.

This PR is now needed by grafana/loki#23710 for two reasons:

  1. OCI conditional uploads are required to support Loki's GetAndReplace workflow safely.
  2. The HeadObject changes resolve the OCI metadata connection leak that previously blocked feat: Add support for Oracle OCI object storage grafana/loki#17818.

Could a maintainer please review this PR and let me know if any further changes are required for approval? I am available to address feedback promptly. If the implementation is ready, merging it would unblock the corresponding Grafana Objstore synchronization and the Loki OCI backend PR.

Thank you!

@alsenz

alsenz commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Hey @felix0102, I'm not a maintainer (or at least I haven't noticed being made one!) but I will give this a review for you this evening when my work finishes, might expedite being accepted.

@alsenz alsenz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full disclosure - I'm not super familiar with the OCI libary and can't integration test this but with some very minor comments looks good to me.

Once thing I noticed - I think the README would also need an update to flag that objstore supports OCI now!

Comment thread providers/oci/oci_test.go Outdated
Comment thread providers/oci/oci.go
Signed-off-by: felix0102 <felix0102@qq.com>
Signed-off-by: felix0102 <felix0102@qq.com>
Signed-off-by: felix0102 <felix0102@qq.com>
Signed-off-by: felix0102 <felix0102@qq.com>
Signed-off-by: felix0102 <felix0102@qq.com>
@felix0102
felix0102 force-pushed the feat/oci-conditional-writes branch from 6ffbef9 to e251cd9 Compare August 8, 2026 09:41
@felix0102

Copy link
Copy Markdown
Contributor Author

Hi @fpetkovski — this PR adds OCI support for the conditional-upload API and already has an approval from @alsenz. Would you be able to review it when you have time? I’d appreciate any feedback. Thank you!

Comment thread providers/oci/oci.go Outdated
Comment thread providers/oci/oci.go
Comment thread providers/oci/oci.go Outdated
Comment thread providers/oci/oci.go Outdated
@felix0102
felix0102 force-pushed the feat/oci-conditional-writes branch from 8359c9a to dd57942 Compare August 10, 2026 11:00
@felix0102
felix0102 requested a review from GiedriusS August 10, 2026 11:04
@felix0102

Copy link
Copy Markdown
Contributor Author

Sorry for the noise. The previous rebase caused many review comments to become outdated. I’ve handled all feedback. Please let me know if you want me to revisit anything.

@felix0102

Copy link
Copy Markdown
Contributor Author

Hi @GiedriusS — gentle follow-up on this PR. I’ve addressed all feedback from the previous review, the branch is clean, and all required checks are green. Could you please take another look when you have time and let me know if anything else is needed for approval? Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants