[fix](streaming-job) fix streaming job properties not parsed after FE restart#2
Open
JNSimba wants to merge 53 commits into
Open
[fix](streaming-job) fix streaming job properties not parsed after FE restart#2JNSimba wants to merge 53 commits into
JNSimba wants to merge 53 commits into
Conversation
# Conflicts: # build.sh
… restart After FE restart, StreamingJobProperties constructor was only called via gsonPostProcess() without validate(). When properties map was non-empty (e.g. max_interval=1), the old code skipped the isEmpty() branch and left maxIntervalSecond=0, causing timeoutMs=0 and every task to timeout immediately. Fix: parse properties directly in the constructor with safe fallback to defaults, so properties are always correctly initialized regardless of whether validate() is called. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
JNSimba
pushed a commit
that referenced
this pull request
Apr 21, 2026
### What problem does this PR solve? Fix memory corruption in PredicateColumnType::insert_duplicate_fields for string types Fixed a bug in the pointer arithmetic when inserting duplicate string values. The original code used dst += i * str.size() which incorrectly multiplied by the loop index, causing: - Iteration 0: no advance (correct by accident) - Iteration 1: advance by 1×size (correct) - Iteration 2: advance by 2×size (wrong - overwrites beyond allocated memory) This resulted in memory corruption and incorrect string data when inserting more than 2 duplicates. The fix changes to dst += str.size() to correctly advance the pointer by one string length per iteration. Added unit tests covering: - String type with multiple duplicate insertions - Integer type duplicates - LargeInt (Int128) type duplicates Introduced by apache#60530 ==13605==ERROR: AddressSanitizer: use-after-poison on address 0x7d639d26bc30 at pc 0x55e8b71df91e bp 0x7b4e8fc1e510 sp 0x7b4e8fc1dcd0 WRITE of size 6 at 0x7d639d26bc30 thread T620 (ls_normal [work) #0 0x55e8b71df91d in __asan_memcpy (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x2205791d) #1 0x55e8cce8b1ab in doris::PredicateColumnType<(doris::PrimitiveType)23>::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/predicate_column.h:327:17 #2 0x55e8b7db4e83 in doris::ColumnNullable::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/column_nullable.cpp:357:29 #3 0x55e8ccf8ba2c in doris::segment_v2::DefaultValueColumnIterator::_insert_many_default(doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2167:14 #4 0x55e8ccf3c705 in doris::segment_v2::DefaultValueColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, bool*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2152:5 apache#5 0x55e8cd4f9313 in doris::segment_v2::ColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.h:330:16 apache#6 0x55e8cd4f9313 in doris::segment_v2::SegmentIterator::_read_columns_by_index(unsigned int, unsigned short&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2291:13 apache#7 0x55e8cd50621b in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2643:5 apache#8 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*)::$_0::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2519:9 apache#9 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2518:19 apache#10 0x55e8cd37cca1 in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/lazy_init_segment_iterator.h:46:33 apache#11 0x55e8cccc0b05 in doris::Status doris::BetaRowsetReader::_next_batch<doris::Block>(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:113:35 apache#12 0x55e8ccc98668 in doris::BetaRowsetReader::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:56:55 apache#13 0x55e8ca62ee8f in doris::VCollectIterator::Level0Iterator::_refresh() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.h apache#14 0x55e8ca611036 in doris::VCollectIterator::Level0Iterator::refresh_current_row() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:523:24 apache#15 0x55e8ca601a08 in doris::VCollectIterator::Level0Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:496:14 apache#16 0x55e8ca604ea2 in doris::VCollectIterator::Level1Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:713:27 apache#17 0x55e8ca608b20 in doris::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader>>>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:189:9 apache#18 0x55e8ca5dc1ae in doris::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:152:9 apache#19 0x55e8ca5d7b9d in doris::BlockReader::init(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:226:19 apache#20 0x55e8d02df6d6 in doris::OlapScanner::_open_impl(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/olap_scanner.cpp:317:32 apache#21 0x55e8d02cd697 in doris::Scanner::open(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner.h:80:16 apache#22 0x55e8d02b576a in doris::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:185:5 apache#23 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:75:17 apache#24 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:74:27 apache#25 0x55e8d02bf603 in bool std::__invoke_impl<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 apache#26 0x55e8d02bf603 in std::enable_if<is_invocable_r_v<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 apache#27 0x55e8d02bf603 in std::_Function_handler<bool (), doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 apache#28 0x55e8d02bed85 in std::function<bool ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 apache#29 0x55e8d02bed85 in doris::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l>>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:425:25 apache#30 0x55e8d02aa844 in doris::PrioritizedSplitRunner::process() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/prioritized_split_runner.cpp:102:35 apache#31 0x55e8d0272674 in doris::TimeSharingTaskExecutor::_dispatch_thread() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:556:77 apache#32 0x55e8d3fa8ad1 in std::function<void ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 apache#33 0x55e8d3fa8ad1 in doris::Thread::supervise_thread(void*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:460:5 apache#34 0x55e8b71ddd26 in asan_thread_start(void*) (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x22055d26) apache#35 0x7f539b835ac2 in start_thread nptl/pthread_create.c:442:8 apache#36 0x7f539b8c78cf misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 0x7d639d26bc30 is located 816 bytes inside of 4096-byte region [0x7d639d26b900,0x7d639d26c900) allocated by thread T620 (ls_normal [work) here: #0 0x55e8b71e1b34 in malloc (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x22059b34) #1 0x55e8b78d2140 in doris::DefaultMemoryAllocator::malloc(unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/allocator.h:90:55 #2 0x55e8b78d2140 in doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>::alloc(unsigned long, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/allocator.cpp:337:23 #3 0x55e8b7ccdfaf in doris::Arena::Chunk::Chunk(unsigned long, doris::Arena::Chunk*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:59:63 #4 0x55e8b7ccdad9 in doris::Arena::_init_head_if_needed() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:131:24 apache#5 0x55e8b7ccd758 in doris::Arena::alloc(unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/arena.h:148:9 apache#6 0x55e8cce8b118 in doris::PredicateColumnType<(doris::PrimitiveType)23>::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/predicate_column.h:325:32 apache#7 0x55e8b7db4e83 in doris::ColumnNullable::insert_duplicate_fields(doris::Field const&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/core/column/column_nullable.cpp:357:29 apache#8 0x55e8ccf8ba2c in doris::segment_v2::DefaultValueColumnIterator::_insert_many_default(doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, unsigned long) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2167:14 apache#9 0x55e8ccf3c705 in doris::segment_v2::DefaultValueColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&, bool*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.cpp:2152:5 apache#10 0x55e8cd4f9313 in doris::segment_v2::ColumnIterator::next_batch(unsigned long*, doris::COW<doris::IColumn>::mutable_ptr<doris::IColumn>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/column_reader.h:330:16 apache#11 0x55e8cd4f9313 in doris::segment_v2::SegmentIterator::_read_columns_by_index(unsigned int, unsigned short&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2291:13 apache#12 0x55e8cd50621b in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2643:5 apache#13 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*)::$_0::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2519:9 apache#14 0x55e8cd4a0b82 in doris::segment_v2::SegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/segment_iterator.cpp:2518:19 apache#15 0x55e8cd37cca1 in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/segment/lazy_init_segment_iterator.h:46:33 apache#16 0x55e8cccc0b05 in doris::Status doris::BetaRowsetReader::_next_batch<doris::Block>(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:113:35 apache#17 0x55e8ccc98668 in doris::BetaRowsetReader::next_batch(doris::Block*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/rowset/beta_rowset_reader.h:56:55 apache#18 0x55e8ca62ee8f in doris::VCollectIterator::Level0Iterator::_refresh() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.h apache#19 0x55e8ca611036 in doris::VCollectIterator::Level0Iterator::refresh_current_row() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:523:24 apache#20 0x55e8ca601a08 in doris::VCollectIterator::Level0Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:496:14 apache#21 0x55e8ca604ea2 in doris::VCollectIterator::Level1Iterator::ensure_first_row_ref() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:713:27 apache#22 0x55e8ca608b20 in doris::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader>>>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/vcollect_iterator.cpp:189:9 apache#23 0x55e8ca5dc1ae in doris::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:152:9 apache#24 0x55e8ca5d7b9d in doris::BlockReader::init(doris::TabletReader::ReaderParams const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/storage/iterator/block_reader.cpp:226:19 apache#25 0x55e8d02df6d6 in doris::OlapScanner::_open_impl(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/olap_scanner.cpp:317:32 apache#26 0x55e8d02cd697 in doris::Scanner::open(doris::RuntimeState*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner.h:80:16 apache#27 0x55e8d02b576a in doris::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:185:5 apache#28 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:75:17 apache#29 0x55e8d02bf603 in doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:74:27 apache#30 0x55e8d02bf603 in bool std::__invoke_impl<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 apache#31 0x55e8d02bf603 in std::enable_if<is_invocable_r_v<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 apache#32 0x55e8d02bf603 in std::_Function_handler<bool (), doris::ScannerScheduler::submit(std::shared_ptr<doris::ScannerContext>, std::shared_ptr<doris::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 apache#33 0x55e8d02bed85 in std::function<bool ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 apache#34 0x55e8d02bed85 in doris::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l>>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.cpp:425:25 apache#35 0x55e8d02aa844 in doris::PrioritizedSplitRunner::process() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/prioritized_split_runner.cpp:102:35 apache#36 0x55e8d0272674 in doris::TimeSharingTaskExecutor::_dispatch_thread() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:556:77 apache#37 0x55e8d3fa8ad1 in std::function<void ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 apache#38 0x55e8d3fa8ad1 in doris::Thread::supervise_thread(void*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:460:5 Thread T620 (ls_normal [work) created by T0 here: #0 0x55e8b71c5e91 in pthread_create (/mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be+0x2203de91) #1 0x55e8d3fa7b8a in doris::Thread::start_thread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::function<void ()> const&, std::shared_ptr<doris::Thread>*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.cpp:412:15 #2 0x55e8d02849fd in doris::Status doris::Thread::create<void (doris::TimeSharingTaskExecutor::*)(), doris::TimeSharingTaskExecutor*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, void (doris::TimeSharingTaskExecutor::* const&)(), doris::TimeSharingTaskExecutor* const&, std::shared_ptr<doris::Thread>*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/util/thread.h:47:16 #3 0x55e8d026e6b3 in doris::TimeSharingTaskExecutor::_create_thread() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:671:12 #4 0x55e8d026b628 in doris::TimeSharingTaskExecutor::_try_create_thread(int, std::lock_guard<std::mutex>&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:360:25 apache#5 0x55e8d026967a in doris::TimeSharingTaskExecutor::init() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/task_executor/time_sharing/time_sharing_task_executor.cpp:221:27 apache#6 0x55e8d025a85a in doris::TaskExecutorSimplifiedScanScheduler::start(int, int, int, int) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/exec/scan/scanner_scheduler.h:288:9 apache#7 0x55e8d368d649 in doris::WorkloadGroup::upsert_thread_pool_no_lock(doris::WorkloadGroupInfo*, std::shared_ptr<doris::CgroupCpuCtl>) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group.cpp:571:38 apache#8 0x55e8d368f14d in doris::WorkloadGroup::upsert_task_scheduler(doris::WorkloadGroupInfo*) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group.cpp:667:12 apache#9 0x55e8d36bd223 in doris::WorkloadGroupMgr::create_internal_wg() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/workload_group/workload_group_manager.cpp:900:5 apache#10 0x55e8d32bbd3d in doris::ExecEnv::_create_internal_workload_group() /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:475:5 apache#11 0x55e8d32b27fc in doris::ExecEnv::_init(std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:424:5 apache#12 0x55e8d32ad7ec in doris::ExecEnv::init(doris::ExecEnv*, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::vector<doris::StorePath, std::allocator<doris::StorePath>> const&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&) /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/runtime/exec_env_init.cpp:199:17 apache#13 0x55e8b722cc8b in main /mnt/disk3/pipeline/repo/selectdb-core_master/selectdb-core/be/src/service/doris_main.cpp:531:14 apache#14 0x7f539b7cad8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
JNSimba
pushed a commit
that referenced
this pull request
Apr 27, 2026
…pache#62759) Related PR: apache#58002 Problem Summary: `arrow::Decimal256Array::Value()` returns raw bytes (`const uint8_t*`), we previously reinterpret_cast it to `Decimal256*` which is a misaligned access . ```text ../src/core/data_type_serde/data_type_decimal_serde.cpp:376:21: runtime error: reference binding to misaligned address 0x1338f3345b12 for type 'const FieldType' (aka 'const Decimal<integer<256, int>>'), which requires 8 byte alignment 0x1338f3345b12: note: pointer points here 00 c2 3e 20 00 00 10 63 2d 5e c7 6b 05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ^ #0 0x56203d1eb751 in doris::DataTypeDecimalSerDe<(doris::PrimitiveType)35>::read_column_from_arrow(doris::IColumn&, arrow::Array const*, long, long, cctz::time_zone const&) const be/build_ASAN/../src/core/data_type_serde/data_type_decimal_serde.cpp:375:25 #1 0x56203d9c6d6b in doris::DataTypeNullableSerDe::read_column_from_arrow(doris::IColumn&, arrow::Array const*, long, long, cctz::time_zone const&) const be/build_ASAN/../src/core/data_type_serde/data_type_nullable_serde.cpp:350:26 #2 0x562055dd83b5 in doris::FromRecordBatchToBlockConverter::convert(doris::Block*) be/build_ASAN/../src/format/arrow/arrow_block_convertor.cpp:131:9 #3 0x562055dd9229 in doris::convert_from_arrow_batch(std::shared_ptr<arrow::RecordBatch> const&, std::vector<std::shared_ptr<doris::IDataType const>, ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
StreamingJobPropertiesconstructor was called viagsonPostProcess()withoutvalidate(). When properties map was non-empty (e.g.max_interval=1), the old code skipped theisEmpty()branch and leftmaxIntervalSecond=0, causingtimeoutMs=0and every streaming task to timeout immediately.validate()is called.StreamingMultiTblTask.isTimeout()for easier future debugging.Test plan
StreamingJobPropertiesTest- covers empty properties, explicit properties, bad values, and validate() behaviortest_streaming_mysql_job_restart_fe_with_props- creates job withmax_interval=5, restarts FE, verifies new data is consumed after restart🤖 Generated with Claude Code