[IIS] Add grok pattern for s-ip + cookie + referer + split-bytes log layout#20048
[IIS] Add grok pattern for s-ip + cookie + referer + split-bytes log layout#20048brian-mckinney wants to merge 3 commits into
Conversation
…layout The documented IIS access log field layout at packages/iis/docs/README.md:69 (s-ip, cs(cookie), cs(Referer), sc-bytes, cs-bytes as separate fields) has no matching grok pattern in the ingest pipeline, so any log using this exact layout fails ingestion with a grok/number-conversion error instead of parsing cleanly. Adds the missing pattern (SDH-beats#7343).
Elastic Docs Style Checker (Vale)Summary: 2 suggestions found 💡 Suggestions (2): Optional style improvements. Apply when helpful.
The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
This comment has been minimized.
This comment has been minimized.
TL;DR
Remediation
Investigation detailsRoot CauseThis is a test fixture completeness failure in the IIS pipeline test suite. The PR adds The Buildkite command was Evidence
Verification
Follow-upIf the generated expected document shows the new grok pattern maps fields incorrectly, adjust What is this? | From workflow: PR Buildkite Detective Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not. |
Adds the missing test-iis-access-cookie-bytes-split.log-expected.json fixture generated via elastic-package test pipeline --generate, which resolves the failing IIS pipeline test.
🚀 Benchmarks reportTo see the full report comment with |
|
✅ All changelog entries have the correct PR link. |
💚 Build Succeeded
History
|
Summary
Fixes ingestion failures for IIS access logs using the field layout documented at
packages/iis/docs/README.md:69:None of the 10 grok patterns shipped in
packages/iis/data_stream/access/elasticsearch/ingest_pipeline/default.yml(v1.24.6) implement this exact combination (s-ipwithouts-sitename, pluscs(cookie),cs(Referer), andsc-bytes/cs-bytesas two separate fields). Every pattern fails to match, ingestion falls through toon_failure, and a stray numeric-conversion attempt on a desynced token surfaces aserror.message: "For input string: ..."instead of a clean parse.Reported in SDH-beats#7343 — customer configured IIS with exactly this documented layout and every event failed ingestion. Root cause and field-by-field trace are in that issue.
s-ip-based pattern, adding theiis.access.cookiecapture betweenuser_agent.originalandhttp.request.referrer)Test plan
date timepair as a singleTIMESTAMP_ISO8601match)elastic-package test pipeline --generate --data-streams accessstill needs to run against a live stack to generatetest-iis-access-cookie-bytes-split.log-expected.json— I don't have a stack running locally, so this is left for CI/a reviewer withelastic-package stack upavailable. Marking draft until that's filled in.