Skip to content

feat: consume brpc Redis maxclients support #560

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
thweetkomputer merged 6 commits into from
Sep 1, 2026
Merged

feat: consume brpc Redis maxclients support #560

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1e209fb
feat: consume brpc Redis maxclients support
thweetkomputer Aug 28, 2026
1bea324
fix: adapt EloqStore module to updated brpc
thweetkomputer Aug 28, 2026
f362f4d
build: pin merged brpc and EloqStore revisions
thweetkomputer Aug 28, 2026
a9a2f37
docs: scope maxclients file-limit behavior
thweetkomputer Aug 28, 2026
63640da
fix: stop applying maxclients to DataStoreService
thweetkomputer Aug 28, 2026
1d12310
build: update brpc Redis rejection fix
thweetkomputer Aug 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 0 additions & 10 deletions core/src/data_substrate.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
*/

#include <gflags/gflags.h>
#include <sys/resource.h>
#include <sys/sysinfo.h>

#include <filesystem>
Expand Down Expand Up @@ -667,15 +666,6 @@ bool DataSubstrate::LoadCoreAndNetworkConfig(const INIReader &config_reader)
!CheckCommandLineFlagIsDefault("maxclients")
? FLAGS_maxclients
: config_reader.GetInteger("local", "maxclients", FLAGS_maxclients);
struct rlimit ulimit
{
};
ulimit.rlim_cur = core_config_.maxclients;
ulimit.rlim_max = core_config_.maxclients;
if (setrlimit(RLIMIT_NOFILE, &ulimit) == -1)
{
LOG(WARNING) << "Failed to set maxclients.";
}
core_config_.bootstrap =
!CheckCommandLineFlagIsDefault("bootstrap")
? FLAGS_bootstrap
Expand Down
1 change: 1 addition & 0 deletions docs/01-architecture-overview.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ All knobs follow *gflag overrides ini section* (`[local]`, `[cluster]`, `[store]
| `auto_redirect` | false | Redirect object commands to remote owner NG internally |
| `enable_key_cache` | false | Key cache (non-MVCC only) |
| `enable_io_uring` / `raft_log_async_fsync` | false | IO engine options |
| `maxclients` | 500000 | API-server connection admission limit; does not modify process file-descriptor limits |
| `max_standby_lag` | 400000 | Max primary→standby message lag |

Build-time options are described in the repo `CLAUDE.md` (`WITH_DATA_STORE`, `WITH_LOG_STATE`, `WITH_LOG_SERVICE`, `EXT_TX_PROC_ENABLED`, `ELOQ_MODULE_ENABLED`).
Expand Down
2 changes: 1 addition & 1 deletion store_handler/eloq_data_store_service/eloqstore
Open in desktop
Submodule eloqstore updated 2 files
+1 −1 cmake/dependencies.cmake
+6 −1 include/eloqstore_module.h
19 changes: 0 additions & 19 deletions store_handler/eloq_data_store_service/main.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@
#include <brpc/server.h>
#include <gflags/gflags.h>
#include <gflags/gflags_declare.h>
#include <sys/resource.h>
#include <sys/sysinfo.h>
#include <unistd.h>

Expand Down Expand Up @@ -88,8 +87,6 @@ DEFINE_bool(bootstrap,
"Init data store config file and exit. (Only support bootstrap one "
"node now.)");

DEFINE_uint32(maxclients, 300000, "maxclients");

static bool CheckCommandLineFlagIsDefault(const char *name)
{
gflags::CommandLineFlagInfo flag_info;
Expand Down Expand Up @@ -205,22 +202,6 @@ int main(int argc, char *argv[])
std::filesystem::create_directories(data_path);
}

// Set maxclients
uint32_t maxclients =
!CheckCommandLineFlagIsDefault("maxclients")
? FLAGS_maxclients
: config_reader.GetInteger("local", "maxclients", FLAGS_maxclients);

struct rlimit ulimit
{
};
ulimit.rlim_cur = maxclients;
ulimit.rlim_max = maxclients;
if (setrlimit(RLIMIT_NOFILE, &ulimit) == -1)
{
LOG(WARNING) << "Failed to set maxclients.";
}

std::string ds_config_file_path = data_path + "/dss_config.ini";

EloqDS::DataStoreServiceClusterManager ds_config;
Expand Down
2 changes: 1 addition & 1 deletion third_party/src/brpc
Open in desktop
Submodule brpc updated 13 files
+8 −0 docs/cn/server.md
+8 −0 docs/en/server.md
+79 −4 src/brpc/acceptor.cpp
+21 −0 src/brpc/acceptor.h
+58 −2 src/brpc/server.cpp
+17 −0 src/brpc/server.h
+81 −18 src/bthread/eloq_module.cpp
+51 −0 src/bthread/eloq_module.h
+4 −0 src/bthread/ring_module.h
+101 −13 src/bthread/task_group.cpp
+14 −2 src/bthread/task_group.h
+1 −0 test/baidu_thread_local_unittest.cpp
+172 −0 test/brpc_server_unittest.cpp
5 changes: 5 additions & 0 deletions tx_service/include/tx_service.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1068,6 +1068,11 @@ class TxServiceModule : public eloq::EloqModule
public:
TxServiceModule() = default;

eloq::ModuleType Type() const override
{
return eloq::ModuleType::kTxService;
}

void Init(std::vector<std::unique_ptr<TxProcessor>> *tx_processors)
{
tx_processors_ = tx_processors;
Expand Down
Loading