Remove Vectorization 2.0 Implementation

[footka]

Description

This work item proposes the removal of the Vectorization 2.0 implementation from the codebase. The goal is to retain Vectorization 1.0 functionality (controlled by _rowsets_enabled) while removing the Vectorization 2.0 components (controlled by use_rich_format_ and _enable_rich_vector_format).

Technical Context

Vectorization 1.0:

1. Operators process multiple rows per call: get_next_row -> get_next_batch.
2. Expressions compute a batch at a time: expr->eval -> expr->eval_batch.
3. Storage decodes data in batches into ObDatum objects.

Vectorization 2.0:

Vectorization 2.0 is an extension of 1.0, focusing on changes to the "physical representation per column" and the "computation kernel."

Changes in Physical Representation:

• 1.0: Each column is a fixed ObDatum datums[256].
• 2.0: Dynamically selects different formats (IVector/Rich Format) based on data type and source:
  • VEC_FIXED: Fixed-length basic types (int, float, etc.)
  • VEC_CONTINUOUS: Multiple variable-length varchars stored in a contiguous memory block.
  • VEC_DISCRETE: For example, varchar expression results from operator computations stored in discrete memory.
  • VEC_UNIFORM: Compatible with the 1.0 ObDatum representation.
  • VEC_UNIFORM_CONST

Changes in Computation Kernel:

eval_batch_func_ -> eval_vector_func_: The latter supports multiple columnar memory formats and type specialization, aiming to reduce format conversion overhead, branching, and data access costs, and provides opportunities for SIMD optimization in some operators.

Proposed Code Changes

The following directories and files are identified for removal/modification:

src/sql/engine/vector/**

src/sql/engine/expr/vector_cast/**

# Various vec operators
src/sql/engine/aggregate/ob_exec_hash_struct_vec.*
src/sql/engine/basic/ob_limit_vec_op.*
src/sql/engine/join/ob_join_vec_op.*

# Rich Aggregate new framework
src/sql/engine/aggregate/
# Should retain ObAggregateProcessor, delete share::aggregate::Processor

src/sql/dtl/ob_dtl_vectors_buffer.cpp
src/sql/dtl/ob_dtl_vectors_buffer.h

# storage
src/storage/blocksstable/encoding/*
src/storage/access/ob_aggregated_store_vec.*
src/storage/access/ob_pushdown_aggregate_vec.*

Environment

• Version Information: (Required) Please provide the output of bin/observer -V. If this test involves changes to the obproxy version, also provide the output of obproxy -V.
• Is this an RD-provided temporary version?: No / [Specify if provided by xxx]
• Reproducibility: Consistent / Intermittent / Uncertain / Other
• Test Changes:
  • Keep vec 1.0: _rowsets_enabled
  • Remove vec 2.0: use_rich_format_, _enable_rich_vector_format
• Are there other anomalies or changes in the environment?: No / Clog disk was full / Manual operations performed / Injected xx exception
• Environment Details: Please provide connection strings for the problematic tenant and the sys tenant, along with deployment information. Indicate if the scene has been preserved.

Steps to Reproduce / Scenario Description

(Required) Please describe the scenario or steps to reproduce. If there are SQL statements for reproduction, please attach them.

Note regarding a related pool: "This pool is not very useful in practice; the real overhead is in Session allocation. Sessions are now essentially dynamically allocated."

Log Information

(Please provide relevant logs)

Other

For issues reported by external business teams, please correctly select the "Work Item Type" on the right based on the actual situation:

• Customer Service Feedback: Specifically refers to issues before production deployment or during POC, clearly identified as product bugs requiring fixes and release.
• Online Issue: Issues that have occurred in a production environment, clearly identified as product bugs, but not classified as a fault.
• Online Fault: Issues that have occurred in a production environment, clearly identified as product bugs, causing business errors or negative impact on the customer side (the interpretation of whether it constitutes a fault belongs to the delivery service team).