feat(Storage)!: Enable full object checksum validation for resumable uploads - #15769
feat(Storage)!: Enable full object checksum validation for resumable uploads#15769mahendra-google wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the upload validation mechanism in the Google Cloud Storage V1 client. It introduces UploadValidationMode.RejectAndThrow to allow the server to reject mismatched objects before creation, deprecates UploadValidationMode.DeleteAndThrow, and removes UploadValidationMode.ThrowOnly along with the UploadValidationException class. It also introduces a HashingStream within CustomMediaUpload to calculate CRC32C hashes on the fly and inject the x-goog-hash header. The feedback highlights that removing ThrowOnly and UploadValidationException are breaking changes for existing client applications, and recommends restoring them with an [Obsolete] attribute to maintain backward compatibility.
I am having trouble creating individual review comments. Click here to see my feedback.
apis/Google.Cloud.Storage.V1/Google.Cloud.Storage.V1/UploadValidationMode.cs (29-35)
Removing ThrowOnly from the UploadValidationMode enum is a breaking change for existing client applications. To maintain backward compatibility, please restore ThrowOnly and mark it as [Obsolete], mapping it to the new validation behavior.
/// <summary>
/// Obsolete. Use <see cref="RejectAndThrow"/> instead.
/// Previously, the hash of the data was computed while uploading, and if the resulting object had a different hash, an exception was thrown, but the object remained present in Storage.
/// </summary>
[Obsolete("ThrowOnly is deprecated. Use RejectAndThrow instead, as the server now rejects the object before creation.")]
ThrowOnly = 1,
/// <summary>
/// Obsolete. Use <see cref="RejectAndThrow"/> instead.
/// Previously, the object was uploaded and then deleted if the hash mismatched.
/// The server now rejects mismatched objects automatically before creation.
/// </summary>
[Obsolete("DeleteAndThrow is deprecated. Use RejectAndThrow instead, as the server now rejects the object before creation.")]
DeleteAndThrow = 2,apis/Google.Cloud.Storage.V1/Google.Cloud.Storage.V1/UploadValidationException.cs (23-26)
Removing the public class UploadValidationException is a breaking change for existing client applications that catch this exception.
To maintain backward compatibility, please restore this class and mark it as [Obsolete]. This allows existing code to compile with a warning rather than failing with a compilation error.
|
Let's keep this one in draft until we have updated the core libraries to the newest dependency. |
This PR transitions upload object checksum integrity validation from client-side to the server-side.
Previously,
CRC32Cvalidation occurred after the upload is completed, requiring the client to delete the object (usingDeleteAndThrowupload validation mode) or leave a corrupted object in the bucket (usingThrowOnlyupload validation mode) upon hash mismatches. By leveraging the newLastRequestExecutingevent ingoogle-api-dotnet-clientcore library, this implementation calculatesCRC32Cincrementally during streaming and injects thex-goog-hash: crc32c=...header on the final chunk. If a checksum mismatch occurs, the server rejects the upload with an HTTP400 Bad Request, ensuring invalid objects are never created in the bucket.