Skip to content

Fix KQL wildcard literal failing to lex escaped specials with spaces#6383

Open
munzzyy wants to merge 1 commit into
elastic:mainfrom
munzzyy:fix/kql-wildcard-escaped-spaces
Open

Fix KQL wildcard literal failing to lex escaped specials with spaces#6383
munzzyy wants to merge 1 commit into
elastic:mainfrom
munzzyy:fix/kql-wildcard-escaped-spaces

Conversation

@munzzyy

@munzzyy munzzyy commented Jul 5, 2026

Copy link
Copy Markdown

Pull Request

Issue link(s):

Resolves #6282

Summary - What I changed

export-rules fails on a KQL rule when its query has a wildcard value that mixes unquoted spaces with an escaped special character, like app_message : *\: No such file or directory. Kibana accepts this query fine, but the KQL grammar here can't lex it, so export throws a syntax error and skips the rule (matches the screenshots in the issue).

The cause is in WILDCARD_LITERAL in lib/kql/kql/kql.g. It has four patterns for wildcard values containing spaces, and each one only allows a fixed set of raw characters as "safe" - anything backslash-escaped (\:, \(, \", etc.) falls outside that set and breaks the match, even though UNQUOTED_CHAR already knows how to handle those same escapes for the no-space case.

I extended each pattern so a "safe unit" is either one of those escape pairs or a raw safe character, matching what UNQUOTED_CHAR already does. No other behavior changes.

One thing worth flagging: if the wildcard value contains a bare or/and/not as a full word (like "No such file or directory" in the original report), the grammar still splits the query there - that's pre-existing behavior unrelated to this bug (I confirmed it happens on current main too, with or without an escaped char involved), so I left it alone to keep this change scoped to the actual reported failure.

How To Test

>>> import kql
>>> kql.parse(r"app_message: *\: no such file present")
FieldComparison(field=Field(name='app_message'), value=Wildcard(value='*: no such file present'))

Before this change, that raises kql.errors.KqlParseError.

Added test_wildcard_with_spaces_and_escaped_special_chars to tests/kuery/test_parser.py. Ran the full tests/kuery suite locally (72 tests, all passing) and ruff check / ruff format --check on the changed test file.

Checklist

  • Added a label for the type of pr: bug
  • Added the meta:rapid-merge label if planning to merge within 24 hours
  • Secret and sensitive material has been managed correctly
  • Automated testing was updated or added to match the most common scenarios
  • Documentation and comments were added for features that require explanation

Contributor checklist

WILDCARD_LITERAL only treated a fixed set of raw characters as safe, so
a wildcard value that mixed unquoted spaces with an escaped special char
(e.g. an escaped colon: *\: no such file) couldn't be lexed at all and
export-rules would error out instead of producing valid KQL.

Extend the grammar so each "safe unit" in a wildcard also accepts the
same backslash-escape pairs UNQUOTED_CHAR already understands, add a
regression test, and confirm the existing kuery test suite (72 tests)
still passes.

Resolves elastic#6282
@cla-checker-service

cla-checker-service Bot commented Jul 5, 2026

Copy link
Copy Markdown

💚 CLA has been signed

@eric-forte-elastic

Copy link
Copy Markdown
Contributor

@munzzyy is this substantially different from my commit from 2 weeks ago? main...6282-dac-export-rules-fails-to-export-kql-rules-when-spaces-are-not-quoted

Thanks!

@munzzyy

munzzyy commented Jul 6, 2026

Copy link
Copy Markdown
Author

Structurally it's the same fix - each unit inside the spaced-wildcard patterns becomes "escape pair or plain non-special char" - but there are two real differences:

  1. The 6282 branch predates [Bug] KQL does not properly escape leading forward slash #6001 (4292cd8, merged Jul 2), which changed wildcard values to be unescaped in the AST. That branch's new test asserts the backslash survives (Wildcard(r"*\: No such file*")), which was true before [Bug] KQL does not properly escape leading forward slash #6001 but isn't now. I cross-ran the suites locally to be sure: that test file against this PR's branch fails its new test (values come back unescaped), and this branch's test file (which includes the two [Bug] KQL does not properly escape leading forward slash #6001 regression tests) against 6282 fails test_escaped_colon_with_leading_slash_wildcard and test_escaped_wildcard_is_literal, since the branch is missing the parser.py side of [Bug] KQL does not properly escape leading forward slash #6001. (test_optimization also fails in both cross directions - that's just drift between the two branch points, unrelated to either fix.) So 6282 would need a rebase plus reworked assertions; this PR was written against post-[Bug] KQL does not properly escape leading forward slash #6001 main and its tests assert the unescaped values.

  2. The escape unit differs: 6282 takes \[^\s\r\n] (any escaped non-whitespace), this PR whitelists the same escapes UNQUOTED_CHAR already accepts (\[trn\():<>"*{}]). Where that shows: an undefined escape like \q inside a spaced wildcard - any-non-space accepts it, the whitelist rejects it the same way the rest of the grammar does.

One thing 6282 has that this doesn't: the package version bumps - happy to add those here. And no attachment on my end - if you'd rather rebase your branch and land it that way, I'll close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DaC] export-rules Fails to Export KQL rules when spaces are not quoted

2 participants