Summary
Historically, Pixels handled SHORT (SMALLINT), INT, and LONG (BIGINT)
columns through a single code path:
- Writing went through one
IntegerColumnWriter.
- Values were materialized into
long[] via LongColumnVector, regardless of
the logical type.
The main reason was performance rather than simplicity. On the older 64-bit
CPUs and JVMs that Pixels originally targeted, 32-bit integer arithmetic was
not faster than 64-bit long arithmetic (and in some cases was slower due
to how the JIT/hardware handled sub-word operations). Since narrowing to int
or short gave no speed benefit, it was cleaner to keep everything as long
and avoid the extra vector/reader/writer classes.
On modern CPUs and recent JVMs (e.g., Java 23), 32-bit and 16-bit integer
operations have comparable performance to 64-bit ones. This removes the
original justification for the unified path, so we can now use narrower
in-memory representations to save memory without performance degradation.
Summary
Historically, Pixels handled
SHORT(SMALLINT),INT, andLONG(BIGINT)columns through a single code path:
IntegerColumnWriter.long[]viaLongColumnVector, regardless ofthe logical type.
The main reason was performance rather than simplicity. On the older 64-bit
CPUs and JVMs that Pixels originally targeted, 32-bit integer arithmetic was
not faster than 64-bit
longarithmetic (and in some cases was slower dueto how the JIT/hardware handled sub-word operations). Since narrowing to
intor
shortgave no speed benefit, it was cleaner to keep everything aslongand avoid the extra vector/reader/writer classes.
On modern CPUs and recent JVMs (e.g., Java 23), 32-bit and 16-bit integer
operations have comparable performance to 64-bit ones. This removes the
original justification for the unified path, so we can now use narrower
in-memory representations to save memory without performance degradation.