ThreatQ Integration | Fixes Issue Pulling Sources & Attributes#20042
ThreatQ Integration | Fixes Issue Pulling Sources & Attributes#20042zach-threatq wants to merge 6 commits into
Conversation
|
💚 CLA has been signed |
ReviewersBuildkite won't run for external contributors automatically; you need to add a comment:
NOTE: https://github.com/elastic/integrations/blob/main/.buildkite/pull-requests.json contains all those details. |
Updated the changelog to reflect the correct pull request link for version 1.39.2.
andrewkroh
added
Integration:ti_threatq
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
![vera-review-bot[bot]](https://avatars.githubusercontent.com/in/3297022?s=60&v=4){kind=small}
| value: {{page_size}} | ||
| - set: | ||
| target: url.params.fields | ||
| value: {{threat_library_fields}} |
There was a problem hiding this comment.
Severity: 🟠 High confidence: high path: packages/ti_threatq/data_stream/threat/agent/stream/httpjson.yml.hbs:35
The new url.params.fields value is rendered unquoted; the default begins with *, which YAML parses as an alias node, so the agent config fails to parse. Single-quote the value.
Details
The added set transform renders value: {{threat_library_fields}} without quotes. The threat_library_fields var defaults to *,sources,attributes,attributes.name,attributes.value, so the rendered agent config line becomes value: *,sources,attributes,attributes.name,attributes.value. In YAML an unquoted scalar that begins with * is interpreted as an alias node reference (the * sigil), not a plain string — the parser looks for an anchor name and fails on the immediately following ,, producing a config parse error. With the default configuration the httpjson input cannot start. Every other value in this template that can contain special characters is single-quoted (e.g. value: '[[.cursor.cursor_mark]]'); page_size is safe unquoted only because it is an integer. The tenable_sc url.params.fields precedent renders unquoted only because its value starts with a letter.
Recommendation:
Single-quote the rendered value so a leading * (or commas) is treated as a plain string:
- set:
target: url.params.fields
value: '{{threat_library_fields}}'🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
|
No issues across the latest commits 0123fc5, 6280f3e. Review summaryIssues found across earlier commits aa6759b — 1 high
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
|
|
Is this change backward compatible with earlier versions of ThreatQ? Let's say a user is connected to an earlier version, then will the new query param on the GET request work the same?
Can you point me at the documentation for the API that this integration is using? I was looking at https://helpcenter.threatq.com/assets/HTML/api/ThreatQ_REST_API_Reference_v6.16.0.html and didn't see the API endpoint. |
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
The As far as documentation goes, I don't believe there are public docs outlining the endpoints in full detail, just high level. |
Even if it's high level info, can you point me toward where I would find a doc for |
| description: >- | ||
| Comma-separated list of fields to retrieve from the Threat Library. | ||
| An asterisk (*) can be used to retrieve all default fields. In ThreatQ v6.12 and later, | ||
| sources & attributes are no longer part of the default fields and must be explicitly requested. |
There was a problem hiding this comment.
| sources & attributes are no longer part of the default fields and must be explicitly requested. | |
| sources and attributes are no longer part of the default fields and must be explicitly requested. |
| # newer versions go on top | ||
| - version: "1.39.2" | ||
| changes: | ||
| - description: Restores `sources` & `attributes` fields after recent updates to the ThreatQ Platform's API. |
There was a problem hiding this comment.
| - description: Restores `sources` & `attributes` fields after recent updates to the ThreatQ Platform's API. | |
| - description: Restores `sources` and `attributes` fields after recent updates to the ThreatQ Platform's API. |
2 participants
Bug Fix
Proposed commit message
fix: issue returning sources & attributes from ThreatQ
ThreatQ recently updated the default API response from the Threat Library to exclude
attributesandsourcesby default. The original API response can be maintained by adding thefieldsURL parameter. In addition to the default fields (*), we can ask for theattributesandsourcesfields, which will restore full functionality of the integration & mappings. This commit also specifically asks for attribute names and values to de-clutter the API response.Checklist
changelog.ymlfile.Author's Checklist
N/A
How to test this PR locally
Using a ThreatQ instance on the latest version, ensure that the list of
sourcesandattributesproperly get parsed into an Elastic index record with the default integration configuration.Related issues
N/A
Screenshots
N/A