diff --git a/packages/greenhouse/_dev/build/docs/README.md b/packages/greenhouse/_dev/build/docs/README.md index 4bac41f7d00..b6f67bd2510 100644 --- a/packages/greenhouse/_dev/build/docs/README.md +++ b/packages/greenhouse/_dev/build/docs/README.md @@ -56,6 +56,28 @@ To find a user's ID: - **Batch Size**: Number of events per API request (100-500, default: 500) - **Performer IDs Filter**: Filter by specific user IDs - **Event Types Filter**: Filter by event type (data_change_update, data_change_create, data_change_destroy, harvest_access, action) + - **Enrich rejected application events**: When enabled, look up the rejection reason and rejection notes/comments from the Harvest API and add them to the event (default: disabled) + +### Enabling rejection enrichment (optional) + +When a candidate or prospect is rejected, Greenhouse audits it as an `action` event with `event.target_type` set to the literal string `Candidate or Prospect rejected` — this event does **not** include an application or candidate ID, and does not include the rejection reason or notes/comments entered at rejection time. Enabling **Enrich rejected application events** looks these up from the Harvest API and correlates them back to the audit event. + +1. On the same Harvest V3 (OAuth) API credential used for audit log access, grant the following read permissions (under **Configure > Dev Center > API Credentials** on the credential): + - **Rejection details** → *List rejection details* + - **Rejection reasons** → *List rejection reasons* and *Show rejection reasons* + - **Applications** → read access, needed for the `GET /v3/applications/{id}` lookup this integration performs + - **Activity Feed** → read access, needed for the `GET /v3/applications/{id}/activity_feed` lookup this integration performs to retrieve the rejection notes +2. Enable the **Enrich rejected application events** setting on the integration. + +Because the "Candidate or Prospect rejected" audit event has no application ID, the integration correlates it to a second audit event that Greenhouse records for the same rejection — one with `event.target_type` of `RejectionDetails` — by matching the `request.id` shared by both events. That event's `event.meta.application_id` is then used to look up the rejection reason and notes: + +- **Exactly one matching `RejectionDetails` event**: the rejection reason and notes are fetched and added to the event under `greenhouse.audit.event.rejection` (`application_id`, `candidate_id`, `reason.id`/`reason.name`/`reason.type`, `notes`, `rejected_at`). The notes are also copied to `event.reason`. +- **No matching event found**: the audit event is still indexed, with `greenhouse.audit.event.rejection.error` explaining that no matching `RejectionDetails` event was found for that request. This can happen if the two events land on different pages of a poll; increasing **Batch Size** reduces how often this occurs. +- **More than one matching event** (for example, a bulk rejection of several candidates in one action, which shares a single `request.id` across all of them): the integration does not guess which pairs with which. The audit event is indexed with `greenhouse.audit.event.rejection.error` and `greenhouse.audit.event.rejection.ambiguous_application_ids` listing the candidates, for manual follow-up. + +Any of the error cases above also adds the tag `greenhouse-rejection-enrichment-failed`, and the underlying rejection audit event is never dropped. + +Because this issues up to two extra API requests per rejection on top of the audit log polling, be mindful of Greenhouse's rate limits (50 general requests per 10 seconds) if your organization rejects applications in bulk. ## Logs @@ -93,6 +115,14 @@ If no events are being collected: 2. Check that there have been events in the last 30 days 3. Review any filter settings that might be excluding events +### Rejection Enrichment Errors + +If rejection events are tagged with `greenhouse-rejection-enrichment-failed` and `greenhouse.audit.event.rejection.error` is populated: +1. Verify the OAuth credential has been granted **Rejection details** (*List rejection details*), **Rejection reasons** (*List rejection reasons*, *Show rejection reasons*), **Applications**, and **Activity Feed** read permissions +2. Check that the authorizing user has permission to view the affected application +3. If the error mentions no matching `RejectionDetails` event was found, try increasing **Batch Size** so the two related audit events are less likely to land on different pages +4. If the error mentions an ambiguous match, the rejection was likely part of a bulk-reject action; `greenhouse.audit.event.rejection.ambiguous_application_ids` lists the candidates involved for manual follow-up + ## Logs reference ### audit diff --git a/packages/greenhouse/changelog.yml b/packages/greenhouse/changelog.yml index 0a1cb430a6f..5ba0a86c61e 100644 --- a/packages/greenhouse/changelog.yml +++ b/packages/greenhouse/changelog.yml @@ -1,4 +1,9 @@ # newer versions go on top +- version: "0.2.0" + changes: + - description: Add an optional "Enrich rejected application events" setting that, for a "Candidate or Prospect rejected" audit event, correlates it by timestamp to the matching Application via the Harvest API and adds the rejection reason and rejection notes/comments to the event as `greenhouse.audit.event.rejection`. + type: enhancement + link: https://github.com/elastic/integrations/pull/17347 - version: "0.1.0" changes: - description: Initial release. diff --git a/packages/greenhouse/data_stream/audit/_dev/test/pipeline/test-audit-events.json b/packages/greenhouse/data_stream/audit/_dev/test/pipeline/test-audit-events.json index a879a96a8b9..f397ecd9f1d 100644 --- a/packages/greenhouse/data_stream/audit/_dev/test/pipeline/test-audit-events.json +++ b/packages/greenhouse/data_stream/audit/_dev/test/pipeline/test-audit-events.json @@ -14,6 +14,15 @@ }, { "message": "{\"request\":{\"id\":\"3456cID\",\"type\":\"jobs#destroy\"},\"performer\":{\"meta\":{\"name\":\"HR Manager\",\"username\":\"hr.manager@omniva-corp.com\"},\"id\":55555,\"ip_address\":\"172.16.0.100\",\"type\":\"user\"},\"organization_id\":456,\"event\":{\"meta\":{\"name\":[\"Software Engineer\",null]},\"target_type\":\"Job\",\"target_id\":789,\"type\":\"data_change_destroy\"},\"event_time\":\"2023-06-04T08:15:30.500Z\"}" + }, + { + "message": "{\"request\":{\"id\":\"7890dID\",\"type\":\"people#change_application_state\"},\"performer\":{\"meta\":{\"name\":\"Recruiter One\",\"username\":\"recruiter.one@omniva-corp.com\"},\"id\":22222,\"ip_address\":\"198.51.100.5\",\"type\":\"user\"},\"organization_id\":123,\"event\":{\"meta\":null,\"target_type\":\"Candidate or Prospect rejected\",\"type\":\"action\",\"rejection\":{\"application_id\":\"987654\",\"candidate_id\":\"123123\",\"reason\":{\"id\":\"555\",\"name\":\"Not a fit\",\"type\":\"We rejected them\"},\"notes\":\"Candidate lacked required experience with distributed systems.\",\"rejected_at\":\"2023-06-05T09:00:00.000Z\"}},\"event_time\":\"2023-06-05T09:00:00.000Z\"}" + }, + { + "message": "{\"request\":{\"id\":\"7891eID\",\"type\":\"people#change_application_state\"},\"performer\":{\"meta\":{\"name\":\"Recruiter Two\",\"username\":\"recruiter.two@omniva-corp.com\"},\"id\":33333,\"ip_address\":\"198.51.100.6\",\"type\":\"user\"},\"organization_id\":123,\"event\":{\"meta\":null,\"target_type\":\"Candidate or Prospect rejected\",\"type\":\"action\",\"rejection\":{\"error\":\"ambiguous: 2 RejectionDetails events share request.id 7891eID (likely a bulk rejection)\",\"ambiguous_application_ids\":[\"111111\",\"222222\"]}},\"event_time\":\"2023-06-05T09:05:00.000Z\"}" + }, + { + "message": "{\"event\":{\"meta\":{\"application_id\":293344484002,\"id\":221864428002},\"target_id\":221864428002,\"target_type\":\"RejectionDetails\",\"type\":\"data_change_create\"},\"event_time\":\"2023-06-05T09:00:00.048Z\",\"organization_id\":123,\"performer\":{\"id\":22222,\"ip_address\":\"198.51.100.5\",\"meta\":{\"name\":\"Recruiter One\",\"username\":\"recruiter.one@omniva-corp.com\"},\"type\":\"user\"},\"request\":{\"id\":\"7890dID\",\"type\":\"people#change_application_state\"}}" } ] } diff --git a/packages/greenhouse/data_stream/audit/_dev/test/pipeline/test-audit-events.json-expected.json b/packages/greenhouse/data_stream/audit/_dev/test/pipeline/test-audit-events.json-expected.json index 4463639ee4e..2347eb0253a 100644 --- a/packages/greenhouse/data_stream/audit/_dev/test/pipeline/test-audit-events.json-expected.json +++ b/packages/greenhouse/data_stream/audit/_dev/test/pipeline/test-audit-events.json-expected.json @@ -337,6 +337,209 @@ "full_name": "HR Manager", "id": "55555" } + }, + { + "@timestamp": "2023-06-05T09:00:00.000Z", + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "action", + "category": [ + "iam" + ], + "id": "7890dID", + "kind": "event", + "original": "{\"request\":{\"id\":\"7890dID\",\"type\":\"people#change_application_state\"},\"performer\":{\"meta\":{\"name\":\"Recruiter One\",\"username\":\"recruiter.one@omniva-corp.com\"},\"id\":22222,\"ip_address\":\"198.51.100.5\",\"type\":\"user\"},\"organization_id\":123,\"event\":{\"meta\":null,\"target_type\":\"Candidate or Prospect rejected\",\"type\":\"action\",\"rejection\":{\"application_id\":\"987654\",\"candidate_id\":\"123123\",\"reason\":{\"id\":\"555\",\"name\":\"Not a fit\",\"type\":\"We rejected them\"},\"notes\":\"Candidate lacked required experience with distributed systems.\",\"rejected_at\":\"2023-06-05T09:00:00.000Z\"}},\"event_time\":\"2023-06-05T09:00:00.000Z\"}", + "reason": "Candidate lacked required experience with distributed systems.", + "type": [ + "info" + ] + }, + "greenhouse": { + "audit": { + "event": { + "rejection": { + "application_id": "987654", + "candidate_id": "123123", + "notes": "Candidate lacked required experience with distributed systems.", + "reason": { + "id": "555", + "name": "Not a fit", + "type": "We rejected them" + }, + "rejected_at": "2023-06-05T09:00:00.000Z" + }, + "target_type": "Candidate or Prospect rejected", + "type": "action" + }, + "performer": { + "meta": {}, + "type": "user" + }, + "request": { + "id": "7890dID", + "type": "people#change_application_state" + } + } + }, + "organization": { + "id": "123" + }, + "related": { + "ip": [ + "198.51.100.5" + ], + "user": [ + "22222", + "recruiter.one@omniva-corp.com" + ] + }, + "source": { + "ip": "198.51.100.5" + }, + "tags": [ + "preserve_original_event", + "forwarded", + "greenhouse-audit" + ], + "user": { + "email": "recruiter.one@omniva-corp.com", + "full_name": "Recruiter One", + "id": "22222" + } + }, + { + "@timestamp": "2023-06-05T09:05:00.000Z", + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "action", + "category": [ + "iam" + ], + "id": "7891eID", + "kind": "event", + "original": "{\"request\":{\"id\":\"7891eID\",\"type\":\"people#change_application_state\"},\"performer\":{\"meta\":{\"name\":\"Recruiter Two\",\"username\":\"recruiter.two@omniva-corp.com\"},\"id\":33333,\"ip_address\":\"198.51.100.6\",\"type\":\"user\"},\"organization_id\":123,\"event\":{\"meta\":null,\"target_type\":\"Candidate or Prospect rejected\",\"type\":\"action\",\"rejection\":{\"error\":\"ambiguous: 2 RejectionDetails events share request.id 7891eID (likely a bulk rejection)\",\"ambiguous_application_ids\":[\"111111\",\"222222\"]}},\"event_time\":\"2023-06-05T09:05:00.000Z\"}", + "type": [ + "info" + ] + }, + "greenhouse": { + "audit": { + "event": { + "rejection": { + "ambiguous_application_ids": [ + "111111", + "222222" + ], + "error": "ambiguous: 2 RejectionDetails events share request.id 7891eID (likely a bulk rejection)" + }, + "target_type": "Candidate or Prospect rejected", + "type": "action" + }, + "performer": { + "meta": {}, + "type": "user" + }, + "request": { + "id": "7891eID", + "type": "people#change_application_state" + } + } + }, + "organization": { + "id": "123" + }, + "related": { + "ip": [ + "198.51.100.6" + ], + "user": [ + "33333", + "recruiter.two@omniva-corp.com" + ] + }, + "source": { + "ip": "198.51.100.6" + }, + "tags": [ + "preserve_original_event", + "forwarded", + "greenhouse-audit", + "greenhouse-rejection-enrichment-failed" + ], + "user": { + "email": "recruiter.two@omniva-corp.com", + "full_name": "Recruiter Two", + "id": "33333" + } + }, + { + "@timestamp": "2023-06-05T09:00:00.048Z", + "ecs": { + "version": "8.11.0" + }, + "event": { + "action": "data_change_create", + "category": [ + "configuration", + "iam" + ], + "id": "7890dID", + "kind": "event", + "original": "{\"event\":{\"meta\":{\"application_id\":293344484002,\"id\":221864428002},\"target_id\":221864428002,\"target_type\":\"RejectionDetails\",\"type\":\"data_change_create\"},\"event_time\":\"2023-06-05T09:00:00.048Z\",\"organization_id\":123,\"performer\":{\"id\":22222,\"ip_address\":\"198.51.100.5\",\"meta\":{\"name\":\"Recruiter One\",\"username\":\"recruiter.one@omniva-corp.com\"},\"type\":\"user\"},\"request\":{\"id\":\"7890dID\",\"type\":\"people#change_application_state\"}}", + "type": [ + "creation" + ] + }, + "greenhouse": { + "audit": { + "event": { + "meta": { + "application_id": 293344484002, + "id": 221864428002 + }, + "target_id": "221864428002", + "target_type": "RejectionDetails", + "type": "data_change_create" + }, + "performer": { + "meta": {}, + "type": "user" + }, + "request": { + "id": "7890dID", + "type": "people#change_application_state" + } + } + }, + "organization": { + "id": "123" + }, + "related": { + "ip": [ + "198.51.100.5" + ], + "user": [ + "22222", + "recruiter.one@omniva-corp.com" + ] + }, + "source": { + "ip": "198.51.100.5" + }, + "tags": [ + "preserve_original_event", + "forwarded", + "greenhouse-audit" + ], + "user": { + "email": "recruiter.one@omniva-corp.com", + "full_name": "Recruiter One", + "id": "22222" + } } ] } diff --git a/packages/greenhouse/data_stream/audit/agent/stream/cel.yml.hbs b/packages/greenhouse/data_stream/audit/agent/stream/cel.yml.hbs index 27ec1143313..f2a2b3f4faf 100644 --- a/packages/greenhouse/data_stream/audit/agent/stream/cel.yml.hbs +++ b/packages/greenhouse/data_stream/audit/agent/stream/cel.yml.hbs @@ -16,8 +16,10 @@ resource.url: {{url}} state: initial_interval: {{initial_interval}} token_url: {{token_url}} + harvest_url: {{harvest_url}} client_id: {{client_id}} client_secret: {{client_secret}} + enrich_rejections: {{enrich_rejections}} {{#if user_id}} user_id: "{{user_id}}" {{/if}} @@ -51,6 +53,49 @@ Endpoints: - Response: {paging: {pit_id, next_search_after}, results: [...]} - next_search_after is null on the last page. +Rejection enrichment (optional, enrich_rejections): + + A candidate/prospect rejection is audited as an "action" event with + event.target_type == "Candidate or Prospect rejected" and + event.meta == null — this event carries no application or candidate + ID. However, the same API request also audits a second, separate + event with event.target_type == "RejectionDetails" and + event.type == "data_change_create", whose event.meta.application_id + gives the Application ID. Both events share the same request.id, so + within a single page of audit results, a rejection event is + correlated to its RejectionDetails sibling by matching request.id, + and the resulting application_id is used to look up the rejection + reason and notes from the Harvest API. The result is merged into the + rejection event as event.rejection before it is emitted. This reuses + the same bearer token as the audit log request, so the OAuth client + must also be granted read access to these Harvest V3 endpoints. + + Correlation by request.id: + - Exactly one RejectionDetails event shares the rejection event's + request.id: proceed to the Harvest API lookups below. + - None found: event.rejection.error records that no matching + RejectionDetails event was found for that request.id. This can + happen if the two events land on different audit-log pages; + increasing batch_size reduces how often pages split a request. + - More than one found (a bulk-reject action can audit multiple + "Candidate or Prospect rejected" and "RejectionDetails" events + under the same shared request.id, with no way to tell which + pairs with which): no attempt is made to guess. event.rejection + .error records the ambiguity and + event.rejection.ambiguous_application_ids lists the candidates. + + GET https://harvest.greenhouse.io/v3/applications/{application_id} + - Provides rejection_reason {id, name, type}, candidate_id, and + rejected_at. + + GET https://harvest.greenhouse.io/v3/applications/{application_id}/activity_feed + - Provides the "Candidate Rejected" activity, whose body holds the + rejection notes/comments entered at rejection time. + + A failed or inconclusive lookup does not drop the underlying audit + event: it is emitted with event.rejection.error describing why no + reason/notes could be attached. + State: auth_token — shallow-replaced object holding token and expires. Cleared to {} on 401 so the fields are absent, not sentinel-valued. @@ -168,7 +213,124 @@ program: |- ).as(next_max_event_time, { "auth_token": auth.auth_token, - "events": results.map(e, {"message": e.encode_json()}), + "events": results.filter(rd, + rd.?event.target_type.orValue("") == "RejectionDetails" && + rd.?event.type.orValue("") == "data_change_create" && + rd.?event.meta.application_id.hasValue() + ).as(rejection_details_events, + results.map(e, + ( + state.enrich_rejections && + e.?event.type.orValue("") == "action" && + e.?event.target_type.orValue("") == "Candidate or Prospect rejected" + ).as(is_rejection, + {"message": ( + is_rejection ? + rejection_details_events.filter(rde, + rde.?request.id.orValue("") == e.?request.id.orValue("") + ).as(matches, + (matches.size() == 1) ? + string(matches[0].event.meta.application_id).as(application_id, + request( + "GET", + state.harvest_url.trim_right("/") + "/v3/applications/" + application_id + ).with( + {"Header": {"Authorization": ["Bearer " + auth.auth_token.token]}} + ).do_request().as(app_resp, + (app_resp.StatusCode == 200) ? + app_resp.Body.decode_json().as(app, + request( + "GET", + state.harvest_url.trim_right("/") + "/v3/applications/" + application_id + "/activity_feed" + ).with( + {"Header": {"Authorization": ["Bearer " + auth.auth_token.token]}} + ).do_request().as(feed_resp, + (feed_resp.StatusCode == 200) ? + feed_resp.Body.decode_json().as(feed, + feed.?activities.orValue([]).filter(a, + a.?subject.orValue("") == "Candidate Rejected" + ).as(rejection_activities, + e.with({ + "event": e.event.with({ + "rejection": { + "application_id": application_id, + ?"candidate_id": (app.?candidate_id.orValue(null) == null) ? + optional.none() + : + optional.of(string(app.candidate_id)), + ?"reason": (app.?rejection_reason.orValue(null) == null) ? + optional.none() + : + optional.of({ + "id": string(app.rejection_reason.id), + "name": app.rejection_reason.name, + "type": app.rejection_reason.?type.name.orValue(""), + }), + ?"notes": (size(rejection_activities) > 0) ? + rejection_activities[0].?body + : + optional.none(), + ?"rejected_at": app.?rejected_at, + }, + }), + }) + ) + ) + : + e.with({ + "event": e.event.with({ + "rejection": { + "application_id": application_id, + ?"reason": (app.?rejection_reason.orValue(null) == null) ? + optional.none() + : + optional.of({ + "id": string(app.rejection_reason.id), + "name": app.rejection_reason.name, + "type": app.rejection_reason.?type.name.orValue(""), + }), + ?"rejected_at": app.?rejected_at, + "error": "activity_feed fetch failed: " + string(feed_resp.StatusCode), + }, + }), + }) + ) + ) + : + e.with({ + "event": e.event.with({ + "rejection": { + "application_id": application_id, + "error": "application fetch failed: " + string(app_resp.StatusCode), + }, + }), + }) + ) + ) + : (matches.size() == 0) ? + e.with({ + "event": e.event.with({ + "rejection": { + "error": "no RejectionDetails audit event found sharing request.id " + e.?request.id.orValue(""), + }, + }), + }) + : + e.with({ + "event": e.event.with({ + "rejection": { + "error": "ambiguous: " + string(matches.size()) + " RejectionDetails events share request.id " + e.?request.id.orValue("") + " (likely a bulk rejection)", + "ambiguous_application_ids": matches.map(m, string(m.event.meta.application_id)), + }, + }), + }) + ) + : + e + ).encode_json()} + ) + ) + ), "cursor": { // Keep window_start_time fixed until the final // page succeeds so the whole window can be diff --git a/packages/greenhouse/data_stream/audit/elasticsearch/ingest_pipeline/default.yml b/packages/greenhouse/data_stream/audit/elasticsearch/ingest_pipeline/default.yml index 64660b5324b..0cc54bddc13 100644 --- a/packages/greenhouse/data_stream/audit/elasticsearch/ingest_pipeline/default.yml +++ b/packages/greenhouse/data_stream/audit/elasticsearch/ingest_pipeline/default.yml @@ -187,6 +187,17 @@ processors: field: error.message value: 'Failed to convert target_id to string: {{{ _ingest.on_failure_message }}}' + # Rejection enrichment (Harvest API, requires enrich_rejections) + - set: + field: event.reason + copy_from: greenhouse.audit.event.rejection.notes + ignore_empty_value: true + - append: + field: tags + value: greenhouse-rejection-enrichment-failed + allow_duplicates: false + if: ctx.greenhouse?.audit?.event?.rejection?.error != null + # Related fields - append: field: related.user diff --git a/packages/greenhouse/data_stream/audit/fields/fields.yml b/packages/greenhouse/data_stream/audit/fields/fields.yml index 3d3d6461629..01267c94beb 100644 --- a/packages/greenhouse/data_stream/audit/fields/fields.yml +++ b/packages/greenhouse/data_stream/audit/fields/fields.yml @@ -71,3 +71,49 @@ type: flattened description: > The before and after values from data change events, or other relevant data for the event. + - name: rejection + type: group + description: > + Rejection details looked up from the Harvest API for a "Candidate or Prospect rejected" audit event. This audit event does not include an application ID; it is obtained by correlating a sibling "RejectionDetails" audit event that shares the same request.id. Only populated when the `enrich_rejections` setting is enabled. + fields: + - name: application_id + type: keyword + description: > + The ID of the Application matched to this rejection event via its sibling "RejectionDetails" audit event. + - name: candidate_id + type: keyword + description: > + The ID of the candidate whose application was rejected. + - name: ambiguous_application_ids + type: keyword + description: > + When more than one "RejectionDetails" audit event shares the rejection event's request.id (for example, a bulk rejection), the Application IDs of all of them. No reason or notes are attached in this case, since it is not possible to tell which one corresponds to this rejection event. + - name: reason + type: group + description: > + The rejection reason selected in Greenhouse. + fields: + - name: id + type: keyword + description: > + The ID of the rejection reason. + - name: name + type: keyword + description: > + The name of the rejection reason. + - name: type + type: keyword + description: > + The category of the rejection reason, for example "We rejected them" or "They rejected us". + - name: notes + type: match_only_text + description: > + The rejection notes/comments entered when the application was rejected, sourced from the Harvest API activity feed. + - name: rejected_at + type: date + description: > + The timestamp when the application was rejected, as reported by the Harvest API. + - name: error + type: keyword + description: > + Error message if the rejection enrichment lookup against the Harvest API failed. diff --git a/packages/greenhouse/data_stream/audit/manifest.yml b/packages/greenhouse/data_stream/audit/manifest.yml index ab78713cb94..70c9f5ee360 100644 --- a/packages/greenhouse/data_stream/audit/manifest.yml +++ b/packages/greenhouse/data_stream/audit/manifest.yml @@ -45,6 +45,15 @@ streams: multi: false show_user: true required: false + - name: enrich_rejections + type: bool + title: Enrich rejected application events + multi: false + required: true + show_user: true + default: false + description: >- + When an audit event records a candidate or prospect being rejected, look up the rejection reason and rejection notes/comments from the Harvest API and add them to the event. This audit event does not include an application ID, but Greenhouse also audits a separate "RejectionDetails" event under the same request.id that does; the Application is identified by correlating the two via request.id, and if that correlation is not exact, the event is emitted with an error instead of a guess. This requires the OAuth client to also have read access to the Harvest "Applications" and "Activity Feed" endpoints, and results in up to two additional API calls per rejection event. - name: enable_request_tracer type: bool title: Enable request tracing diff --git a/packages/greenhouse/docs/README.md b/packages/greenhouse/docs/README.md index 87af52dbfe3..47f885139d8 100644 --- a/packages/greenhouse/docs/README.md +++ b/packages/greenhouse/docs/README.md @@ -56,6 +56,28 @@ To find a user's ID: - **Batch Size**: Number of events per API request (100-500, default: 500) - **Performer IDs Filter**: Filter by specific user IDs - **Event Types Filter**: Filter by event type (data_change_update, data_change_create, data_change_destroy, harvest_access, action) + - **Enrich rejected application events**: When enabled, look up the rejection reason and rejection notes/comments from the Harvest API and add them to the event (default: disabled) + +### Enabling rejection enrichment (optional) + +When a candidate or prospect is rejected, Greenhouse audits it as an `action` event with `event.target_type` set to the literal string `Candidate or Prospect rejected` — this event does **not** include an application or candidate ID, and does not include the rejection reason or notes/comments entered at rejection time. Enabling **Enrich rejected application events** looks these up from the Harvest API and correlates them back to the audit event. + +1. On the same Harvest V3 (OAuth) API credential used for audit log access, grant the following read permissions (under **Configure > Dev Center > API Credentials** on the credential): + - **Rejection details** → *List rejection details* + - **Rejection reasons** → *List rejection reasons* and *Show rejection reasons* + - **Applications** → read access, needed for the `GET /v3/applications/{id}` lookup this integration performs + - **Activity Feed** → read access, needed for the `GET /v3/applications/{id}/activity_feed` lookup this integration performs to retrieve the rejection notes +2. Enable the **Enrich rejected application events** setting on the integration. + +Because the "Candidate or Prospect rejected" audit event has no application ID, the integration correlates it to a second audit event that Greenhouse records for the same rejection — one with `event.target_type` of `RejectionDetails` — by matching the `request.id` shared by both events. That event's `event.meta.application_id` is then used to look up the rejection reason and notes: + +- **Exactly one matching `RejectionDetails` event**: the rejection reason and notes are fetched and added to the event under `greenhouse.audit.event.rejection` (`application_id`, `candidate_id`, `reason.id`/`reason.name`/`reason.type`, `notes`, `rejected_at`). The notes are also copied to `event.reason`. +- **No matching event found**: the audit event is still indexed, with `greenhouse.audit.event.rejection.error` explaining that no matching `RejectionDetails` event was found for that request. This can happen if the two events land on different pages of a poll; increasing **Batch Size** reduces how often this occurs. +- **More than one matching event** (for example, a bulk rejection of several candidates in one action, which shares a single `request.id` across all of them): the integration does not guess which pairs with which. The audit event is indexed with `greenhouse.audit.event.rejection.error` and `greenhouse.audit.event.rejection.ambiguous_application_ids` listing the candidates, for manual follow-up. + +Any of the error cases above also adds the tag `greenhouse-rejection-enrichment-failed`, and the underlying rejection audit event is never dropped. + +Because this issues up to two extra API requests per rejection on top of the audit log polling, be mindful of Greenhouse's rate limits (50 general requests per 10 seconds) if your organization rejects applications in bulk. ## Logs @@ -93,6 +115,14 @@ If no events are being collected: 2. Check that there have been events in the last 30 days 3. Review any filter settings that might be excluding events +### Rejection Enrichment Errors + +If rejection events are tagged with `greenhouse-rejection-enrichment-failed` and `greenhouse.audit.event.rejection.error` is populated: +1. Verify the OAuth credential has been granted **Rejection details** (*List rejection details*), **Rejection reasons** (*List rejection reasons*, *Show rejection reasons*), **Applications**, and **Activity Feed** read permissions +2. Check that the authorizing user has permission to view the affected application +3. If the error mentions no matching `RejectionDetails` event was found, try increasing **Batch Size** so the two related audit events are less likely to land on different pages +4. If the error mentions an ambiguous match, the rejection was likely part of a bulk-reject action; `greenhouse.audit.event.rejection.ambiguous_application_ids` lists the candidates involved for manual follow-up + ## Logs reference ### audit @@ -108,6 +138,15 @@ If no events are being collected: | event.dataset | Event dataset | constant_keyword | | event.module | Event module | constant_keyword | | greenhouse.audit.event.meta | The before and after values from data change events, or other relevant data for the event. | flattened | +| greenhouse.audit.event.rejection.ambiguous_application_ids | When more than one "RejectionDetails" audit event shares the rejection event's request.id (for example, a bulk rejection), the Application IDs of all of them. No reason or notes are attached in this case, since it is not possible to tell which one corresponds to this rejection event. | keyword | +| greenhouse.audit.event.rejection.application_id | The ID of the Application matched to this rejection event via its sibling "RejectionDetails" audit event. | keyword | +| greenhouse.audit.event.rejection.candidate_id | The ID of the candidate whose application was rejected. | keyword | +| greenhouse.audit.event.rejection.error | Error message if the rejection enrichment lookup against the Harvest API failed. | keyword | +| greenhouse.audit.event.rejection.notes | The rejection notes/comments entered when the application was rejected, sourced from the Harvest API activity feed. | match_only_text | +| greenhouse.audit.event.rejection.reason.id | The ID of the rejection reason. | keyword | +| greenhouse.audit.event.rejection.reason.name | The name of the rejection reason. | keyword | +| greenhouse.audit.event.rejection.reason.type | The category of the rejection reason, for example "We rejected them" or "They rejected us". | keyword | +| greenhouse.audit.event.rejection.rejected_at | The timestamp when the application was rejected, as reported by the Harvest API. | date | | greenhouse.audit.event.target_id | The ID of the element that was edited or accessed. | keyword | | greenhouse.audit.event.target_type | The resource name for data changes, Harvest access, or the event action type for other actions. | keyword | | greenhouse.audit.event.type | The type of event: data_change_update, data_change_create, data_change_destroy, harvest_access, or action. | keyword | @@ -204,3 +243,60 @@ An example event for `audit` looks as following: } } ``` + +When **Enrich rejected application events** is enabled, a `Candidate or Prospect rejected` audit event is enriched with a `greenhouse.audit.event.rejection` object, correlated to its Application via a sibling `RejectionDetails` audit event sharing the same `request.id`: + +```json +{ + "greenhouse": { + "audit": { + "event": { + "rejection": { + "application_id": "987654", + "candidate_id": "123123", + "notes": "Candidate lacked required experience with distributed systems.", + "reason": { + "id": "555", + "name": "Not a fit", + "type": "We rejected them" + }, + "rejected_at": "2023-06-05T09:00:00.000Z" + }, + "target_type": "Candidate or Prospect rejected", + "type": "action" + } + } + }, + "event": { + "reason": "Candidate lacked required experience with distributed systems." + } +} +``` + +If the correlation is inconclusive, `greenhouse.audit.event.rejection` instead contains an `error` (and, for an ambiguous match, `ambiguous_application_ids`) with no `reason`/`notes` attached: + +```json +{ + "greenhouse": { + "audit": { + "event": { + "rejection": { + "ambiguous_application_ids": [ + "111111", + "222222" + ], + "error": "ambiguous: 2 RejectionDetails events share request.id 7891eID (likely a bulk rejection)" + }, + "target_type": "Candidate or Prospect rejected", + "type": "action" + } + } + }, + "tags": [ + "preserve_original_event", + "forwarded", + "greenhouse-audit", + "greenhouse-rejection-enrichment-failed" + ] +} +``` diff --git a/packages/greenhouse/manifest.yml b/packages/greenhouse/manifest.yml index fa5b99bca6f..90b5230035e 100644 --- a/packages/greenhouse/manifest.yml +++ b/packages/greenhouse/manifest.yml @@ -1,7 +1,7 @@ format_version: "3.3.2" name: greenhouse title: "Greenhouse" -version: "0.1.0" +version: "0.2.0" description: Collect audit logs from Greenhouse ATS with Elastic Agent. type: integration categories: @@ -48,6 +48,14 @@ policy_templates: required: true default: https://auth.greenhouse.io/token description: The OAuth 2.0 token endpoint URL for obtaining access tokens. + - name: harvest_url + type: text + title: Greenhouse Harvest API URL + multi: false + show_user: false + required: true + default: https://harvest.greenhouse.io + description: The base URL of the Greenhouse Harvest API, used to look up rejection details when "Enrich rejected application events" is enabled. - name: client_id type: text title: OAuth Client ID