RocksDB Cloud: unbounded max_open_files (-1) can exhaust process fd limit

[thweetkomputer]

Problem

RocksDBCloudDataStore::OpenCloudDB sets max_open_files = 0 before DB::Open() (to avoid slow open / LRU-eviction failures during open), then restores it to -1 right after open succeeds:

• store_handler/eloq_data_store_service/rocksdb_cloud_data_store.cpp:679 — options.max_open_files = 0;
• store_handler/eloq_data_store_service/rocksdb_cloud_data_store.cpp:811 — SetDBOptions({{"max_open_files", "-1"}})

With max_open_files = -1, the table cache is unbounded: every SST file ever touched keeps its table reader and file descriptor open until the file is deleted by compaction. The number of open fds grows roughly as data size / target_file_size_base (e.g. ~16k files for 1 TB at 64 MB per file). On hosts with a modest ulimit -n (default 1024, commonly 65535), a large enough dataset can exhaust the process fd limit and crash or wedge the data store service. brpc and the AWS SDK also consume fds from the same budget, making the headroom smaller in practice.

Proposal

Make the post-open max_open_files value configurable instead of hardcoding -1, e.g. a rocksdb_cloud_max_open_files flag/ini option (default could stay -1 for compatibility):

1. Bounded max_open_files makes the table cache an LRU with that capacity, capping fds.
2. When bounded, evicting a table reader also drops its index/filter blocks (they live in the table reader heap by default), so a reopened file pays extra I/O — on RocksDB Cloud this can mean an S3 round trip if the file fell out of the local SST cache. To keep bloom filters (added in the recent rocksdb_cloud_enable_bloom_filter change) effective under a bounded table cache, consider also setting BlockBasedTableOptions::cache_index_and_filter_blocks = true (optionally with pin_l0_filter_and_index_blocks_in_cache) so filter/index memory is managed by the block cache independently of fd lifetime.
3. Alternatively/additionally, log a startup warning when RLIMIT_NOFILE looks too small for the current SST file count.

Notes

• The bloom filter config change also replaces options.table_factory with a fresh BlockBasedTableOptions when the filter is enabled; any table-factory-wide settings (e.g. block cache tuning) added later elsewhere would be silently discarded. Worth consolidating table options in one place while touching this code.