[stormshield] Add outcome, identity and HTTP mappings and extend the metadata allow-list

packages/stormshield/_dev/build/docs/README.md

@@ -9,15 +9,15 @@ The StormShield SNS integration for Elastic enables you to collect and analyze s
 
 This integration is compatible with the following third-party vendor systems:
 - Stormshield Network Security (SNS) firewalls running the unified firmware base.
-- SNS v4.x, which is the primarily documented and tested version for these log formats.
+- SNS 4.x and SNS 5.x, which share the same WELF key-value audit log format.
 
 ### How it works
 
 This integration collects logs from Stormshield SNS firewalls by receiving syslog data over `udp` or `tcp`. You deploy an Elastic Agent on a host that is configured as a syslog receiver. Once the agent is configured to listen on the appropriate port, you set up your Stormshield devices to forward logs to that agent. The agent then forwards the events to your Elastic deployment, where they are parsed and mapped to the Elastic Common Schema (ECS) for analysis and visualization.
 
 ## What data does this integration collect?
 
-The StormShield SNS integration collects a variety of security and system events from SNS instances. Each data stream provides specific insights into network activity:
+The StormShield SNS integration collects a variety of security and system events from SNS instances into a single `log` data stream. The collected log families provide specific insights into network activity:
 - Firewall logs: These provide detailed records of filter and NAT policy decisions, including source and destination IP addresses, ports, and protocols.
 - Audit and administrative logs: These capture events related to administrative logins, configuration changes through `serverd`, and system-level operations.
 - Alarm logs: These contain security-specific alerts generated by the SNS Intrusion Prevention System (IPS) and detection engines.
@@ -68,36 +68,40 @@ To configure Stormshield Network Security (SNS) to forward logs to the Elastic A
 
 1. Log in to the Stormshield Network Security web administration interface.
 2. Navigate to the **CONFIGURATION** tab.
-3. Go to **NOTIFICATIONS > LOGS - SYSLOG - IPFIX**.
+3. Go to **NOTIFICATIONS → LOGS - SYSLOG - IPFIX**.
 4. In the **Syslog** section, make sure the service is enabled by switching the status to **ON**.
 5. Select a **Profile** slot to edit (e.g., Profile 1).
 6. In the **Syslog server** field, select or create a host object corresponding to the Elastic Agent host.
 7. Enter the **Port** number matching your Elastic Agent configuration (e.g., `514` for UDP or `601` for TCP).
 8. Select the **Protocol** (**UDP** or **TCP**) that matches your integration input.
-9. Choose the **Format**: **RFC5424** is recommended for modern logging, though **Legacy** (BSD) is also supported.
+9. Set the **Format** to **RFC5424**. The integration expects RFC5424 messages; the **Legacy** and **Legacy long** formats are not supported at this time.
 10. In the **Advanced properties** section, enable the specific log families you want to send. Make sure **Filter Policy**, **Administration**, **System**, and **Alarms** are active.
 11. Click **Apply** to save the syslog configuration.
 
 #### Enabling traffic logs for specific rules
 
-1. Navigate to **Configuration > Security policy > Filter - NAT**.
+1. Navigate to **Configuration → Security policy → Filter - NAT**.
 2. Identify the security rules for which you want to collect logs.
 3. Double-click a rule to open its properties.
 4. Go to the **Action** menu or the General tab.
 5. Set the **Log level** to **Standard (connection log)**.
 6. Click **OK** and then click **Apply** at the top of the interface to deploy the policy change.
 
+#### Collecting flow records over IPFIX (optional)
+
+SNS firewalls can also export network flow records over the IPFIX protocol, configured in the same **NOTIFICATIONS → LOGS - SYSLOG - IPFIX** module (default port `4739`). Flow records are not handled by this integration: to collect them, use the [NetFlow Records](https://www.elastic.co/docs/reference/integrations/netflow) integration, which supports IPFIX. Both integrations can run on the same Elastic Agent to combine audit logs with flow analytics.
+
 #### Vendor resources
 
-- [Stormshield SNS Syslog Configuration](https://documentation.stormshield.eu/SNS/v4/en/Content/User_Configuration_Manual_SNS_v4/Logs-syslog/Syslog_tab.htm)
-- [Stormshield SNS Log Configuration Documentation](https://documentation.stormshield.eu/SNS/v4/en/Content/Description_of_Audit_logs/Configure_logs.htm)
+- [Stormshield SNS Syslog Configuration](https://documentation.stormshield.eu/SNS/v5/en/Content/User_Configuration_Manual_SNS_v5/Logs-syslog/Syslog_tab.htm)
+- [Stormshield SNS Log Configuration Documentation](https://documentation.stormshield.eu/SNS/v5/en/Content/Description_of_Audit_logs/Configure_logs.htm)
 - [FireMon - Stormshield Device Configuration Guide](https://docs.firemon.com/feature/Content/ADMINISTRATION/DEVICE/Devices/StormShield/Stormshield_Network_Security.htm?TocPath=Administration|Device|Devices|Choose+a+Device+to+Onboard|_____51)
 
 ### Set up steps in Kibana
 
 To set up the integration in Kibana, follow these steps:
 
-1. In Kibana, navigate to **Management > Integrations**.
+1. In Kibana, navigate to **Management → Integrations**.
 2. Search for and select **Stormshield SNS**.
 3. Click **Add Stormshield SNS**.
 4. Configure the inputs based on your Stormshield SNS configuration.
@@ -159,11 +163,11 @@ Perform the following actions on your Stormshield device to generate logs:
 #### 2. Check data in Kibana
 
 To verify the data in Kibana:
-1. Navigate to **Analytics > Discover**.
+1. Navigate to **Analytics → Discover**.
 2. Select the `logs-*` data view.
 3. Enter the following KQL filter in the search bar: `data_stream.dataset : "stormshield.log"`
 4. Verify logs appear and expand an entry to confirm the presence of fields like `event.dataset`, `source.ip`, and `message`.
-5. Navigate to **Analytics > Dashboards** and search for "Stormshield SNS" to view pre-built visualizations.
+5. Navigate to **Analytics → Dashboards** and search for "Stormshield SNS" to view pre-built visualizations.
 
 ## Troubleshooting
 
@@ -203,6 +207,42 @@ For more information on architectures that can be used for scaling this integrat
 
 You'll find events from Stormshield SNS logs in the `log` data stream. This data stream includes logs of the following types: traffic, filter, protection, and system events.
 
+##### ECS event categorization
+
+Each log family (identified by the `logtype` field) is mapped to the ECS categorization fields. Alarms raised by the intrusion prevention engine produce `event.kind: alert`. The periodic statistics families (`monitor`, `filterstat`, `count`, `routerstat`, `authstat`, `ipsecstat`) produce `event.kind: metric`, so health telemetry can be excluded from detection rule scopes with a single filter. All other families produce `event.kind: event`.
+
+| logtype | event.kind | event.category | event.type |
+|---|---|---|---|
+| alarm | alert | intrusion_detection, network | info |
+| auth | event | authentication | start |
+| authstat | metric | authentication | info |
+| connection | event | network, session | connection, end |
+| count, filterstat, ipsecstat, routerstat | metric | network | info |
+| date | event | host | change |
+| dmrouting, routing | event | network, host | change |
+| filter | event | network | connection |
+| ftp | event | network | info |
+| monitor | metric | host | info |
+| plugin | event | network | protocol |
+| pop3, smtp | event | email, network | info |
+| pvm | event | vulnerability | info |
+| restapi | event | web, configuration | access |
+| sandboxing | event | network | info |
+| server | event | configuration | access |
+| ssl | event | network | connection |
+| system | event | host | info |
+| vpn | event | network | info |
+| web | event | web, network | access |
+| xvpn | event | network, session | connection |
+
+For the families that carry a rule action, `event.type` additionally receives `allowed` when the action is `pass` and `denied` when the action is `block` (matched case-insensitively). An empty action field corresponds to a rule set to Log, and no modifier is added. Unknown log families fall back to `event.kind: event` with `event.category: network` and `event.type: info`.
+
+For the `auth`, `server` and `xvpn` families, `event.outcome` is derived from the vendor `error` field: `success` when the error code is `0`, `failure` otherwise.
+
+For alarm logs, `event.code` carries the Stormshield alarm identifier, `event.risk_score` the `risk` field (1-100), and `event.severity` the vendor alarm level from `pri`, where 1 is a major alarm and 4 a minor alarm. This scale is specific to the alarm family. In most other families, the appliance hardcodes the `pri` field to 5.
+
+`network.direction` is computed from the source and destination IP addresses, treating private (RFC 1918) address ranges as internal networks.
+
 ##### `log` fields
 
 You'll find a list of all exported fields in the following table:
@@ -218,5 +258,5 @@ The following is a sample event for the `log` data stream:
 ### Vendor documentation links
 
 You'll find additional details about Stormshield SNS logs in these official resources:
-* [Stormshield SNS Log Configuration Documentation](https://documentation.stormshield.eu/SNS/v4/en/Content/Description_of_Audit_logs/Configure_logs.htm)
+* [Stormshield SNS Log Configuration Documentation](https://documentation.stormshield.eu/SNS/v5/en/Content/Description_of_Audit_logs/Configure_logs.htm)
 * [Stormshield Official Website](https://www.stormshield.com)

packages/stormshield/changelog.yml

@@ -1,4 +1,37 @@
 # newer versions go on top
+- version: "1.7.0"
+  changes:
+    - description: Set `event.outcome` on authentication, server and SSL VPN logs from the vendor error code, map `domain` and `usergroup` to `user.domain` and `user.group.name`, the HTTP method and return code of web logs to `http.request.method` and `http.response.status_code`, the system daemon name to `process.name`, and set `observer.product`.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/20085
+    - description: Preserve the reputation, packet-capture, Ethernet protocol, SSO agent, IKE peer and industrial-protocol vendor fields under `stormshield.metadata` instead of leaving them as undeclared fields.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/20085
+- version: "1.6.0"
+  changes:
+    - description: Set ECS event categorization fields (`event.kind`, `event.category`, `event.type`) for every log family, including `event.kind` `metric` for the periodic statistics families.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/20031
+    - description: Map `rcvd` to `destination.bytes`, sum `network.bytes`, map interface fields to `observer.ingress` and `observer.egress`, compute `network.direction`, and populate `related.hosts`.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/20031
+    - description: Map alarm identifiers and levels to `event.code`, `event.risk_score` and `event.severity`, and HTTP proxy fields to `url.original` and `url.domain`.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/20031
+    - description: Fix the count sub-pipeline, which failed with a Painless runtime error on every count log, and skip malformed counters instead of failing the whole document.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/20031
+    - description: Add an Alarms dashboard and pipeline test samples for the alarm, auth, web, vpn, xvpn, count and filter log families.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/20031
+- version: "1.5.3"
+  changes:
+    - description: Fix ingest pipeline descriptions and remove field definitions that never appear in events (`stormshield.tz`, `stormshield.duration`, `stormshield.msg`, `stormshield.origdst`, `stormshield.origdstport`).
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/20030
+    - description: Correct the syslog format and SNS version compatibility statements in the documentation, and document IPFIX flow collection through the NetFlow integration.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/20030
 - version: "1.5.2"
   changes:
     - description: Remove top level note from docs