InvalidArgumentError("Expected 1 buffers in array of type FixedSizeBinary(4), got 2"

[alamb]

Describe the bug
There is some issue with decoding fixedsizebinary I found while working on tests for other

To Reproduce

Add this test to the parquet tests:

--- a/parquet/src/arrow/arrow_writer/mod.rs
+++ b/parquet/src/arrow/arrow_writer/mod.rs
@@ -2161,8 +2161,11 @@ mod tests {
             )
             .unwrap();

-            let data = DictionaryArray::<K>::new(keys, Arc::new(values));
-            let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(data)]).unwrap();
+
+            let data: ArrayRef = Arc::new(DictionaryArray::<K>::new(keys, Arc::new(values)));
+            one_column_roundtrip(Arc::clone(&data), true);
+
+            let batch = RecordBatch::try_new(Arc::new(schema), vec![data]).unwrap();
             roundtrip(batch, None);
         }

Fails like this

---- arrow::arrow_writer::tests::test_fixed_size_binary_in_dict stdout ----

thread 'arrow::arrow_writer::tests::test_fixed_size_binary_in_dict' (467576) panicked at parquet/src/arrow/buffer/offset_buffer.rs:133:48:
called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Expected 1 buffers in array of type FixedSizeBinary(4), got 2")

I am pretty sure this is related to disabling dictionary pages

Expected behavior
Test should pass

Additional context

[alamb]

The test was originally added by @albertlockett in this PR

• feat: Support round trip reading/writing Arrow type Dictionary(_, FixedSizeBinary(_)) to Parquet #7446

[alamb]

I found this while working on tests for

• [Parquet] Add test for reading/writing long UTF8 StringViews #9246

[albertlockett]

I'd be happy to look into this @alamb. I imagine I'll have some time next week

[alamb]

Thanks @albertlockett 🙏 -- there is no special urgency from me, I just wanted to try and capture this issue that happened to find while writing tests for something different

[vigneshsiva11]

Hi @alamb, I’d like to work on this issue.

From looking into the failure, it seems the problem happens during the Parquet roundtrip when rebuilding a DictionaryArray whose values are a FixedSizeBinaryArray. The error suggests that the resulting FixedSizeBinaryArray ends up with two buffers, even though FixedSizeBinary should only have a single data buffer.

My current guess is that somewhere in the Parquet → Arrow decoding path, FixedSizeBinary is being handled similarly to variable-length binary types (Binary or LargeBinary), which expect an offset buffer. However, since FixedSizeBinary has a fixed width, it should only use the data buffer.

My plan is to:

1. Trace how the array is reconstructed during the Parquet roundtrip when dictionary encoding is used.
2. Locate where the ArrayData for FixedSizeBinary is created and check how the buffers are assigned.
3. Make sure FixedSizeBinary arrays are built with only the data buffer (no offset buffer).
4. Add a regression test using a DictionaryArray with FixedSizeBinary values to confirm the roundtrip works correctly.

Does this approach sound reasonable, or is there a specific part of the codebase you’d recommend focusing on? Happy to adjust based on feedback.

[alamb]

I would personally start with adding the regression test and then fix the bug. Otherwise this is good

[vigneshsiva11]

hi @alamb Thanks for the suggestion!

That makes sense — I'll start by adding a regression test that reproduces the failure and then work on fixing the bug once the test is in place.