Skip to content

Backport missing custom-documentation fields to 8.19 #754

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nicholasberlin wants to merge 9 commits into
base: 8.19
Choose a base branch
from
+308 −2
Open

Backport missing custom-documentation fields to 8.19 #754

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
678110b
Add security event metrics to custom documentation (#652)
AsuNa-jp Jul 18, 2025
6422f88
Enhance process event metrics (#653)
nicholasberlin Jul 23, 2025
3107631
Add network.direction to docs (#659)
ricardo-estc Aug 13, 2025
0e1f949
Add kernel_process_events fields (#716)
matthewh-elastic Jan 12, 2026
9506e63
Add process.Ext.trusted to custom docs for already-running process ev…
gabriellandau Oct 24, 2025
b1a9026
Add missing field in the ASM API events custom documentation (#726)
AsuNa-jp Mar 11, 2026
18c6daf
Fix Linux and macOS behavior alert custom doc for subtechnique fields…
nicholasberlin May 4, 2026
a82dffb
Add mappings for process and security event sources metrics
nicholasberlin Jul 10, 2026
9e02926
Add sources arrays to metrics sample_event.json
nicholasberlin Jul 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions custom_documentation/doc/endpoint/alerts/linux/linux_malicious_behavior_alert.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,9 @@ This alert is generated when a Malicious Behavior alert occurs.
| threat.technique.name |
| threat.technique.reference |
| threat.technique.subtechnique |
| threat.technique.subtechnique.id |
| threat.technique.subtechnique.name |
| threat.technique.subtechnique.reference |
| user.Ext.real.id |
| user.Ext.real.name |
| user.id |
Expand Down
3 changes: 3 additions & 0 deletions custom_documentation/doc/endpoint/alerts/macos/macos_malicious_behavior_alert.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,9 @@ This alert is generated when a Malicious Behavior alert occurs.
| threat.technique.name |
| threat.technique.reference |
| threat.technique.subtechnique |
| threat.technique.subtechnique.id |
| threat.technique.subtechnique.name |
| threat.technique.subtechnique.reference |
| user.Ext.real.id |
| user.Ext.real.name |
| user.id |
Expand Down
1 change: 1 addition & 0 deletions custom_documentation/doc/endpoint/api/windows/windows_api_asm.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ This event is generated when ETW AttackSurfaceMonitor events are generated.
| host.os.version |
| message |
| process.Ext.api.behaviors |
| process.Ext.api.metadata.security_descriptor |
| process.Ext.api.name |
| process.Ext.api.parameters.device |
| process.Ext.api.parameters.io_control_code |
Expand Down
12 changes: 12 additions & 0 deletions custom_documentation/doc/endpoint/metrics/metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,11 @@ This is an internal state management document that includes metrics on Endpoint'
| Endpoint.metrics.documents_volume.overall.suppressed_count |
| Endpoint.metrics.documents_volume.process_events.sent_bytes |
| Endpoint.metrics.documents_volume.process_events.sent_count |
| Endpoint.metrics.documents_volume.process_events.sources.sent_bytes |
| Endpoint.metrics.documents_volume.process_events.sources.sent_count |
| Endpoint.metrics.documents_volume.process_events.sources.source |
| Endpoint.metrics.documents_volume.process_events.sources.suppressed_bytes |
| Endpoint.metrics.documents_volume.process_events.sources.suppressed_count |
| Endpoint.metrics.documents_volume.process_events.suppressed_bytes |
| Endpoint.metrics.documents_volume.process_events.suppressed_count |
| Endpoint.metrics.documents_volume.registry_events.sent_bytes |
Expand All @@ -83,6 +88,11 @@ This is an internal state management document that includes metrics on Endpoint'
| Endpoint.metrics.documents_volume.registry_events.suppressed_count |
| Endpoint.metrics.documents_volume.security_events.sent_bytes |
| Endpoint.metrics.documents_volume.security_events.sent_count |
| Endpoint.metrics.documents_volume.security_events.sources.sent_bytes |
| Endpoint.metrics.documents_volume.security_events.sources.sent_count |
| Endpoint.metrics.documents_volume.security_events.sources.source |
| Endpoint.metrics.documents_volume.security_events.sources.suppressed_bytes |
| Endpoint.metrics.documents_volume.security_events.sources.suppressed_count |
| Endpoint.metrics.documents_volume.security_events.suppressed_bytes |
| Endpoint.metrics.documents_volume.security_events.suppressed_count |
| Endpoint.metrics.documents_volume.volume_device_events.sent_bytes |
Expand Down Expand Up @@ -118,6 +128,8 @@ This is an internal state management document that includes metrics on Endpoint'
| Endpoint.metrics.system_impact.file_events.week_ms |
| Endpoint.metrics.system_impact.kernel_api_events.week_idle_ms |
| Endpoint.metrics.system_impact.kernel_api_events.week_ms |
| Endpoint.metrics.system_impact.kernel_process_events.week_idle_ms |
| Endpoint.metrics.system_impact.kernel_process_events.week_ms |
| Endpoint.metrics.system_impact.ldap_client_events.week_idle_ms |
| Endpoint.metrics.system_impact.ldap_client_events.week_ms |
| Endpoint.metrics.system_impact.library_load_events.week_idle_ms |
Expand Down
1 change: 1 addition & 0 deletions ...doc/endpoint/network/macos/macos_network_connection_attempted_and_disconnect.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ This event is generated when a connection is attempted or a request to terminate
| host.os.type |
| host.os.version |
| message |
| network.direction |
| network.transport |
| network.type |
| process.Ext.ancestry |
Expand Down
2 changes: 2 additions & 0 deletions ...m_documentation/doc/endpoint/process/windows/windows_process_already_running.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ This event is generated for a process that was already running before Endpoint's
| process.Ext.token.elevation_level |
| process.Ext.token.integrity_level_name |
| process.Ext.token.security_attributes |
| process.Ext.trusted |
| process.Ext.trusted_descendant |
| process.Ext.windows.zone_identifier |
| process.args |
| process.args_count |
Expand Down
3 changes: 3 additions & 0 deletions ...m_documentation/src/endpoint/data_stream/alerts/linux/linux_malicious_behavior_alert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,9 @@ fields:
- threat.technique.name
- threat.technique.reference
- threat.technique.subtechnique
- threat.technique.subtechnique.id
- threat.technique.subtechnique.name
- threat.technique.subtechnique.reference
- user.Ext.real.id
- user.Ext.real.name
- user.id
Expand Down
3 changes: 3 additions & 0 deletions ...m_documentation/src/endpoint/data_stream/alerts/macos/macos_malicious_behavior_alert.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,9 @@ fields:
- threat.technique.name
- threat.technique.reference
- threat.technique.subtechnique
- threat.technique.subtechnique.id
- threat.technique.subtechnique.name
- threat.technique.subtechnique.reference
- user.Ext.real.id
- user.Ext.real.name
- user.id
Expand Down
1 change: 1 addition & 0 deletions custom_documentation/src/endpoint/data_stream/api/windows/windows_api_asm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ fields:
- host.os.version
- message
- process.Ext.api.behaviors
- process.Ext.api.metadata.security_descriptor
- process.Ext.api.name
- process.Ext.api.parameters.device
- process.Ext.api.parameters.io_control_code
Expand Down
12 changes: 12 additions & 0 deletions custom_documentation/src/endpoint/data_stream/metrics/metrics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,11 @@ fields:
- Endpoint.metrics.documents_volume.overall.suppressed_count
- Endpoint.metrics.documents_volume.process_events.sent_bytes
- Endpoint.metrics.documents_volume.process_events.sent_count
- Endpoint.metrics.documents_volume.process_events.sources.sent_bytes
- Endpoint.metrics.documents_volume.process_events.sources.sent_count
- Endpoint.metrics.documents_volume.process_events.sources.source
- Endpoint.metrics.documents_volume.process_events.sources.suppressed_bytes
- Endpoint.metrics.documents_volume.process_events.sources.suppressed_count
- Endpoint.metrics.documents_volume.process_events.suppressed_bytes
- Endpoint.metrics.documents_volume.process_events.suppressed_count
- Endpoint.metrics.documents_volume.registry_events.sent_bytes
Expand All @@ -90,6 +95,11 @@ fields:
- Endpoint.metrics.documents_volume.registry_events.suppressed_count
- Endpoint.metrics.documents_volume.security_events.sent_bytes
- Endpoint.metrics.documents_volume.security_events.sent_count
- Endpoint.metrics.documents_volume.security_events.sources.sent_bytes
- Endpoint.metrics.documents_volume.security_events.sources.sent_count
- Endpoint.metrics.documents_volume.security_events.sources.source
- Endpoint.metrics.documents_volume.security_events.sources.suppressed_bytes
- Endpoint.metrics.documents_volume.security_events.sources.suppressed_count
- Endpoint.metrics.documents_volume.security_events.suppressed_bytes
- Endpoint.metrics.documents_volume.security_events.suppressed_count
- Endpoint.metrics.documents_volume.volume_device_events.sent_bytes
Expand Down Expand Up @@ -125,6 +135,8 @@ fields:
- Endpoint.metrics.system_impact.file_events.week_ms
- Endpoint.metrics.system_impact.kernel_api_events.week_idle_ms
- Endpoint.metrics.system_impact.kernel_api_events.week_ms
- Endpoint.metrics.system_impact.kernel_process_events.week_idle_ms
- Endpoint.metrics.system_impact.kernel_process_events.week_ms
- Endpoint.metrics.system_impact.ldap_client_events.week_idle_ms
- Endpoint.metrics.system_impact.ldap_client_events.week_ms
- Endpoint.metrics.system_impact.library_load_events.week_idle_ms
Expand Down
1 change: 1 addition & 0 deletions ...endpoint/data_stream/network/macos/macos_network_connection_attempted_and_disconnect.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ fields:
- host.os.type
- host.os.version
- message
- network.direction
- network.transport
- network.type
- process.Ext.ancestry
Expand Down
2 changes: 2 additions & 0 deletions ...cumentation/src/endpoint/data_stream/process/windows/windows_process_already_running.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ fields:
- process.Ext.token.elevation_level
- process.Ext.token.integrity_level_name
- process.Ext.token.security_attributes
- process.Ext.trusted
- process.Ext.trusted_descendant
- process.Ext.windows.zone_identifier
- process.args
- process.args_count
Expand Down
60 changes: 60 additions & 0 deletions custom_schemas/custom_endpoint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -533,6 +533,36 @@
type: long
description: Total size of suppressed documents

- name: metrics.documents_volume.process_events.sources
level: custom
type: object
description: An array of Process Event document statistics per source

- name: metrics.documents_volume.process_events.sources.source
level: custom
type: keyword
description: Process Event document source name

- name: metrics.documents_volume.process_events.sources.sent_count
level: custom
type: long
description: Number of sent Process Event documents from source

- name: metrics.documents_volume.process_events.sources.sent_bytes
level: custom
type: long
description: Total size of Process Event sent documents from source

- name: metrics.documents_volume.process_events.sources.suppressed_count
level: custom
type: long
description: Number of suppressed Process Event documents from source

- name: metrics.documents_volume.process_events.sources.suppressed_bytes
level: custom
type: long
description: Total size of suppressed Process Event documents from source

- name: metrics.documents_volume.library_events.sent_count
level: custom
type: long
Expand Down Expand Up @@ -573,6 +603,36 @@
type: long
description: Total size of suppressed documents

- name: metrics.documents_volume.security_events.sources
level: custom
type: object
description: An array of Security Event document statistics per source

- name: metrics.documents_volume.security_events.sources.source
level: custom
type: keyword
description: Security Event document source name

- name: metrics.documents_volume.security_events.sources.sent_count
level: custom
type: long
description: Number of sent Security Event documents from source

- name: metrics.documents_volume.security_events.sources.sent_bytes
level: custom
type: long
description: Total size of Security Event sent documents from source

- name: metrics.documents_volume.security_events.sources.suppressed_count
level: custom
type: long
description: Number of suppressed Security Event documents from source

- name: metrics.documents_volume.security_events.sources.suppressed_bytes
level: custom
type: long
description: Total size of suppressed Security Event documents from source

- name: metrics.documents_volume.dns_events.sent_count
level: custom
type: long
Expand Down
62 changes: 62 additions & 0 deletions package/endpoint/data_stream/metrics/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,37 @@
type: long
description: Number of sent documents
default_field: false
- name: metrics.documents_volume.process_events.sources
level: custom
type: object
description: An array of Process Event document statistics per source
default_field: false
- name: metrics.documents_volume.process_events.sources.sent_bytes
level: custom
type: long
description: Total size of Process Event sent documents from source
default_field: false
- name: metrics.documents_volume.process_events.sources.sent_count
level: custom
type: long
description: Number of sent Process Event documents from source
default_field: false
- name: metrics.documents_volume.process_events.sources.source
level: custom
type: keyword
ignore_above: 1024
description: Process Event document source name
default_field: false
- name: metrics.documents_volume.process_events.sources.suppressed_bytes
level: custom
type: long
description: Total size of suppressed Process Event documents from source
default_field: false
- name: metrics.documents_volume.process_events.sources.suppressed_count
level: custom
type: long
description: Number of suppressed Process Event documents from source
default_field: false
- name: metrics.documents_volume.process_events.suppressed_bytes
level: custom
type: long
Expand Down Expand Up @@ -343,6 +374,37 @@
type: long
description: Number of sent documents
default_field: false
- name: metrics.documents_volume.security_events.sources
level: custom
type: object
description: An array of Security Event document statistics per source
default_field: false
- name: metrics.documents_volume.security_events.sources.sent_bytes
level: custom
type: long
description: Total size of Security Event sent documents from source
default_field: false
- name: metrics.documents_volume.security_events.sources.sent_count
level: custom
type: long
description: Number of sent Security Event documents from source
default_field: false
- name: metrics.documents_volume.security_events.sources.source
level: custom
type: keyword
ignore_above: 1024
description: Security Event document source name
default_field: false
- name: metrics.documents_volume.security_events.sources.suppressed_bytes
level: custom
type: long
description: Total size of suppressed Security Event documents from source
default_field: false
- name: metrics.documents_volume.security_events.sources.suppressed_count
level: custom
type: long
description: Number of suppressed Security Event documents from source
default_field: false
- name: metrics.documents_volume.security_events.suppressed_bytes
level: custom
type: long
Expand Down
22 changes: 20 additions & 2 deletions package/endpoint/data_stream/metrics/sample_event.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -658,7 +658,16 @@
"suppressed_count": 0,
"suppressed_bytes": 0,
"sent_count": 259,
"sent_bytes": 422247
"sent_bytes": 422247,
"sources": [
{
"source": "diagnostic_4624",
"sent_count": 4,
"sent_bytes": 10240,
"suppressed_count": 0,
"suppressed_bytes": 0
}
]
},
"library_events": {
"suppressed_count": 0,
Expand All @@ -670,7 +679,16 @@
"suppressed_count": 0,
"suppressed_bytes": 0,
"sent_count": 201,
"sent_bytes": 602914
"sent_bytes": 602914,
"sources": [
{
"source": "load_module",
"sent_count": 2,
"sent_bytes": 5000,
"suppressed_count": 1,
"suppressed_bytes": 2500
}
]
},
"registry_events": {
"suppressed_count": 0,
Expand Down
12 changes: 12 additions & 0 deletions package/endpoint/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2998,6 +2998,12 @@ Metrics documents contain performance information about the endpoint executable
| Endpoint.metrics.documents_volume.overall.suppressed_count | Number of suppressed documents | long |
| Endpoint.metrics.documents_volume.process_events.sent_bytes | Total size of sent documents | long |
| Endpoint.metrics.documents_volume.process_events.sent_count | Number of sent documents | long |
| Endpoint.metrics.documents_volume.process_events.sources | An array of Process Event document statistics per source | object |
| Endpoint.metrics.documents_volume.process_events.sources.sent_bytes | Total size of Process Event sent documents from source | long |
| Endpoint.metrics.documents_volume.process_events.sources.sent_count | Number of sent Process Event documents from source | long |
| Endpoint.metrics.documents_volume.process_events.sources.source | Process Event document source name | keyword |
| Endpoint.metrics.documents_volume.process_events.sources.suppressed_bytes | Total size of suppressed Process Event documents from source | long |
| Endpoint.metrics.documents_volume.process_events.sources.suppressed_count | Number of suppressed Process Event documents from source | long |
| Endpoint.metrics.documents_volume.process_events.suppressed_bytes | Total size of suppressed documents | long |
| Endpoint.metrics.documents_volume.process_events.suppressed_count | Number of suppressed documents | long |
| Endpoint.metrics.documents_volume.registry_events.sent_bytes | Total size of sent documents | long |
Expand All @@ -3006,6 +3012,12 @@ Metrics documents contain performance information about the endpoint executable
| Endpoint.metrics.documents_volume.registry_events.suppressed_count | Number of suppressed documents | long |
| Endpoint.metrics.documents_volume.security_events.sent_bytes | Total size of sent documents | long |
| Endpoint.metrics.documents_volume.security_events.sent_count | Number of sent documents | long |
| Endpoint.metrics.documents_volume.security_events.sources | An array of Security Event document statistics per source | object |
| Endpoint.metrics.documents_volume.security_events.sources.sent_bytes | Total size of Security Event sent documents from source | long |
| Endpoint.metrics.documents_volume.security_events.sources.sent_count | Number of sent Security Event documents from source | long |
| Endpoint.metrics.documents_volume.security_events.sources.source | Security Event document source name | keyword |
| Endpoint.metrics.documents_volume.security_events.sources.suppressed_bytes | Total size of suppressed Security Event documents from source | long |
| Endpoint.metrics.documents_volume.security_events.sources.suppressed_count | Number of suppressed Security Event documents from source | long |
| Endpoint.metrics.documents_volume.security_events.suppressed_bytes | Total size of suppressed documents | long |
| Endpoint.metrics.documents_volume.security_events.suppressed_count | Number of suppressed documents | long |
| Endpoint.metrics.event_filter.active_global_count | The number of active global event filters | long |
Expand Down
Loading