feat(geo): add GEOSEARCHSTORE command - #7984
Conversation
Code Review by Qodo
🟠 Medium 1.
|
PR Summary by QodoAdd GEOSEARCHSTORE command with Redis-compatible store/count semantics
AI Description
Diagram
High-Level Assessment
Files changed (2)
|
🤖 Augment PR SummarySummary: Adds Redis-compatible Changes:
Technical Notes: The implementation reuses 🤖 Was this summary useful? React with 👍 or 👎 |
| if (shard->shard_id() == dest_shard) { | ||
| ZSetFamily::ZParams zparams; | ||
| zparams.override = true; | ||
| ZSetFamily::OpAdd(t->GetOpArgs(shard), zparams, dest_key, ScoredMemberSpan{smvec}); |
There was a problem hiding this comment.
src/server/geo_family.cc:500 — ZSetFamily::OpAdd’s result is ignored and store_cb always returns OpStatus::OK, so an OUT_OF_MEMORY/other write failure could still reply with smvec.size() and leave dest_key not updated. That would be a silent correctness issue for the GEO* STORE paths.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
Code Review by Qodo
Context used✅ Compliance rules (platform):
12 rules✅ Cross-repo context Not relevant to this PR:
romange/helio🔴 Action Required 1.
|
c6687f5 to
fdea81a
Compare
| # Geo | ||
| (b"GEOADD", 4, 10), (b"GEOHASH", 2, 6), (b"GEOPOS", 2, 6), (b"GEODIST", 3, 4), | ||
| (b"GEOSEARCH", 6, 12), (b"GEORADIUS", 5, 12), (b"GEORADIUS_RO", 5, 10), (b"GEORADIUSBYMEMBER", 4, 11), | ||
| (b"GEOSEARCH", 6, 12), (b"GEOSEARCHSTORE", 7, 13), (b"GEORADIUS", 5, 12), (b"GEORADIUS_RO", 5, 10), (b"GEORADIUSBYMEMBER", 4, 11), |
There was a problem hiding this comment.
@rounaknandanwar Thanks for your contribution.
Unfortunately, this is not enough. You have to add/update fuzzer seeds. Please take a look at an example here: https://github.com/dragonflydb/dragonfly/tree/main/fuzz/seeds/resp
There was a problem hiding this comment.
@vyavdoshenko I have fixed it with the last commit.
6c96511 to
d52fde4
Compare
vyavdoshenko
left a comment
There was a problem hiding this comment.
Must fix:
- Replica score corruption (broken replication for GEOSEARCHSTORE)
The manual journal path serializes ZADD scores viaabsl::StrCat(6 significant digits). Since GEOSEARCHSTORE isNO_AUTOJOURNAL, these lossy entries are its only replication mechanism.
master> GEOSEARCHSTORE dst src FROMLONLAT 15 37 BYRADIUS 500 km STOREDIST
master> ZSCORE dst Catania -> 56.4412578701582
replica> ZSCORE dst Catania -> 56.4413
# default mode: geohash 3479099956230698 -> 3479100000000000 (corrupted coordinates)
Fix: round-trip double formatting in the OpAdd journal_update path (also fixes the same latent bug in ZRANGESTORE/ZDIFFSTORE). Add a replication test asserting exact ZSCORE equality.
- GEORADIUS / GEORADIUSBYMEMBER STORE now double-journal
They shareGeoStoreToDest(which now setsjournal_update=true) but are still auto-journaled.
master> GEORADIUS src 15 37 200 km STORE dst
replica INFO commandstats -> cmdstat_del:1, cmdstat_zadd:1, cmdstat_georadius:1
# replica applies the store twice: manual DEL+ZADD, then the replayed command
Fix: add CO::NO_AUTOJOURNAL to both radius commands (also fixes #7996) - but only together with fix 1, otherwise the cross-shard crash becomes silent score corruption. Alternatively, keep journal_update=false on the radius path in this PR.
- New fuzz seeds are malformed RESP
geo_ops2.resp:$4for 5-byte keysgdst2/gdst3(must be$5);*9for the 8-element... FROMMEMBER Palermo ...command (must be*8).georadius_ops.resp:*10for the 9-element... FROMMEMBER catania ... ASCcommand (must be*9) - this swallows the next*12header and desyncs every seed after it, so the intended GEOSEARCHSTORE coverage never executes.
Should fix:
- Validate BYRADIUS/BYBOX at parse time.
GEOSEARCH src FROMLONLAT 15 37 BYRADIUS -5 kmnever replies and pins a proactor thread at 100% CPU (pre-existing, but this PR's new fuzz tokens make the fuzzer hit it). Negative BYBOX in store mode deletes dest and returns 0 instead of erroring. Reference errors: "radius cannot be negative" / "height or width cannot be negative". - Sign the commits - all 4 are unsigned (DCO only); merge is blocked on it.
- Drop the comment above
CmdGeoSearchStore- contains a non-ASCII em-dash and a competitor product reference. - Test coverage: DESC, COUNT n ANY, BYBOX+STOREDIST, FROMMEMBER+STOREDIST, dest==src, dest TTL cleared, empty result with existing src (case-3 empty path); assert the exact count/members in the
europe_boxblock instead ofEXPECT_GE(ZCARD, 1)and remove the deadrespassignment.
|
@rounaknandanwar |
d52fde4 to
815a363
Compare
@vyavdoshenko I have fixed all 3 in the last commit. |
|
@rounaknandanwar |
2827103 to
37a2efe
Compare
|
@rounaknandanwar |
37a2efe to
e2e76fd
Compare
@vyavdoshenko Yes its ready to review |
| << CI{"GEOPOS", CO::READONLY, -2, 1, 1}.HFUNC(GeoPos) | ||
| << CI{"GEODIST", CO::READONLY, -4, 1, 1}.HFUNC(GeoDist) | ||
| << CI{"GEOSEARCH", CO::READONLY, -7, 1, 1}.HFUNC(GeoSearch) | ||
| << CI{"GEOSEARCHSTORE", CO::JOURNALED | CO::DENYOOM | CO::NO_AUTOJOURNAL, -8, 1, 2}.HFUNC( |
There was a problem hiding this comment.
Require READ permission for the source key
Declaring key positions 1..2 is correct for routing, but because this command is marked JOURNALED, the ACL validator treats both keys as write keys. The second key is read-only.
A user with +GEOSEARCHSTORE +ZRANGE %W~secret %RW~out can copy secret into out and read it without having %R~secret, which is a confidentiality bypass.
Please add per-key ACL metadata or explicitly require READ permission for src and WRITE permission for dest.
There was a problem hiding this comment.
Added the ACL fix for GEOSEARCHSTORE: dest requires WRITE, src requires READ (validator.cc + tests).
For other commands, RequiredKeyPermissions() still uses the old default (all keys read on READONLY, all keys write on JOURNALED), so behaviour is unchanged except GEOSEARCHSTORE. The same read+write gap still exists on COPY/RENAME/*STORE etc.; I have only special-cased this command here.
I personally think a better long-term approach is per-key ACL on CommandId at registration, since this one needs special handling for all the read+write commands - which can become a mess in future. Let me know what you think.
Happy to do a follow-up for the broader mixed-key ACL work if you want it tracked separately (and keep the scope of this PR limited to geosearchstore implementation).
| constexpr auto kGeoSearchStoreGrammar = Compile(Options( | ||
| OneOf(kFromMemberLonglatErr, Action("FROMMEMBER", ParseGeoSearchFromMember), | ||
| Action<GeoSearchParse>("FROMLONLAT", ParseLongLat)), | ||
| OneOf(kByRadiusBoxErr, Action("BYRADIUS", ParseGeoSearchByRadius), |
There was a problem hiding this comment.
Reject negative radius and box dimensions before executing the search
These parsers accept negative dimensions. With an existing source, BYRADIUS -1 m reaches geohashEstimateStepsByRadius() and loops forever because the negative value remains below MERCATOR_MAX while being doubled.
A negative BYBOX produces no matches, so GEOSEARCHSTORE deletes an existing destination and returns 0 instead of reporting an error.
Please validate the dimensions in the shared parsers and add regression tests.
There was a problem hiding this comment.
I have added the checks and tests to validate those.
e2e76fd to
be47d04
Compare
Signed-off-by: rounaknandanwar <rounak.nandanwar@gmail.com>
Signed-off-by: rounaknandanwar <rounak.nandanwar@gmail.com>
Signed-off-by: rounaknandanwar <rounak.nandanwar@gmail.com>
Signed-off-by: rounaknandanwar <rounak.nandanwar@gmail.com>
Signed-off-by: rounaknandanwar <rounak.nandanwar@gmail.com>
Signed-off-by: rounaknandanwar <rounak.nandanwar@gmail.com>
Signed-off-by: rounaknandanwar <rounak.nandanwar@gmail.com>
Signed-off-by: rounaknandanwar <rounak.nandanwar@gmail.com>
96d04bf to
af39591
Compare
Signed-off-by: rounaknandanwar <rounak.nandanwar@gmail.com>
Signed-off-by: rounaknandanwar <rounak.nandanwar@gmail.com>
Adds GEOSEARCHSTORE (dest + src keys, same search options as GEOSEARCH, optional STOREDIST).
Most of the work was already in GeoSearchStoreGeneric() via GEORADIUS ... STORE — this wires up the dedicated command and parser. Closes #3883.
Also fixed a couple of Redis mismatches in the shared store/search path:
Tests in geo_family_test.cc. Ran geo_family_test locally + manual redis-cli checks.