chore: include the build-duration-milliseconds github runner monitoring in the reusable workflow

start-build-monitor/README.md

@@ -1,6 +1,6 @@
 # start-build-monitor
 
-This composite GitHub Action starts a lightweight background CPU and memory monitor at the beginning of a job. When used together with [`submit-build-status`](../submit-build-status/), resource utilization metrics are automatically included at the end in the [CI Analytics](https://confluence.camunda.com/display/HAN/CI+Analytics) record for that job.
+This composite GitHub Action starts a lightweight background CPU and memory monitor at the beginning of a job. When used together with [`submit-build-status`](../submit-build-status/), resource utilization metrics and the job duration are automatically included at the end in the [CI Analytics](https://confluence.camunda.com/display/HAN/CI+Analytics) record for that job.
 
 ## Usage
 
@@ -25,11 +25,11 @@ jobs:
           gcp_credentials_json: "${{ secrets.YOUR_GCP_CREDENTIALS }}"
 ```
 
-Jobs that do **not** include `start-build-monitor` are unaffected — `submit-build-status` simply omits the resource fields, which appear as `NULL` in BigQuery like any other optional field.
+Jobs that do **not** include `start-build-monitor` are unaffected — `submit-build-status` simply omits the resource and duration fields, which appear as `NULL` in BigQuery like any other optional field.
 
 ## Behavior
 
-Starts a polling loop in the background (interval: 5s) that samples CPU and memory usage once per interval until `submit-build-status` stops it at job end.
+Starts a polling loop in the background (interval: 5s) that samples CPU and memory usage once per interval until `submit-build-status` stops it at job end. It also records the start time with millisecond precision. `submit-build-status` uses this timestamp to calculate the duration, from this action's start until resource collection begins. Keep this action as the **first job step** so the reported duration covers the entire job.
 
 ### Metric sources
 

start-build-monitor/start-build-monitor.sh

@@ -2,8 +2,13 @@
 
 INTERVAL=5  # seconds
 # NOTE: these paths are shared with the monitor stop script(s) — if changed, update both
-LOG_FILE=/tmp/_monitor-start.log
-PID_FILE=/tmp/_monitor-start.pid
+LOG_FILE="${BUILD_MONITOR_LOG_FILE:-/tmp/_monitor-start.log}"
+PID_FILE="${BUILD_MONITOR_PID_FILE:-/tmp/_monitor-start.pid}"
+START_TIME_FILE="${BUILD_MONITOR_START_TIME_FILE:-/tmp/_monitor-start.epoch-millis}"
+
+# Remove state from a previous job before establishing this job's start time.
+rm -f "$PID_FILE" "$LOG_FILE" "$START_TIME_FILE"
+date +%s%3N > "$START_TIME_FILE"
 
 # CPU and memory cgroup controllers are independently enabled, so detect them separately.
 # Cgroup-based metrics are scoped to this container — unaffected by other pods on the node.
@@ -103,4 +108,4 @@ INTERVAL_US=$(( INTERVAL * 1000000 ))
 ) >> "$LOG_FILE" 2>&1 &
 
 echo $! > "$PID_FILE"
-echo "Resource monitor started (PID=$(cat $PID_FILE), cpu=${CPU_SOURCE}(${CPU_LIMIT_CORES} cores) mem=${MEM_SOURCE}(${MEM_TOTAL_KB}KB), interval=${INTERVAL}s)"
+echo "Resource monitor started (PID=$(cat "$PID_FILE"), cpu=${CPU_SOURCE}(${CPU_LIMIT_CORES} cores) mem=${MEM_SOURCE}(${MEM_TOTAL_KB}KB), interval=${INTERVAL}s)"

submit-build-status/README.md

@@ -11,7 +11,7 @@ This composite GHA can be used in any repository that was set up to provide cred
 | Input name           | Description                                        |
 |----------------------|----------------------------------------------------|
 | build_status         | String representing the build status that should be submitted to CI Analytics, e.g. `"success"`, `"failed"`, `"cancelled"` |
-| build_duration_millis | Optional number (positive) that indicates the duration of the build in milliseconds |
+| build_duration_millis | Optional non-negative build duration in milliseconds. Overrides the automatic duration from `start-build-monitor`. |
 | user_reason          | Optional string (200 chars max) the user can submit to indicate the reason why a build has ended with a certain build status , e.g. `"flaky-tests"` |
 | user_description     | Optional string (1000 chars max) the user can submit to provide details on the user_reason, e.g. a list of flaky tests |
 | gcp_credentials_json | Credentials for a Google Cloud ServiceAccount allowed to publish to Big Query formatted as contents of credentials.json file |
@@ -47,6 +47,12 @@ All data submitted by this action is stored as one record in the Big Query table
 | user_reason      | STRING     | NULLABLE   | Based on user input |
 | user_description | STRING     | NULLABLE   | Based on user input |
 
+#### Build duration
+
+When [`start-build-monitor`](../start-build-monitor/) is the first step in a job, this action automatically submits the elapsed time in milliseconds. The duration starts when the monitor action runs and ends when this action collects its metrics, before Google authentication and the BigQuery request. This excludes telemetry-submission overhead.
+
+An explicit `build_duration_millis` input takes precedence over the automatic duration. Explicit values must be non-negative integers no greater than 72 hours (`259200000` milliseconds); invalid values fail the action's input validation. If the monitor did not run, its timestamp is invalid, or the calculated duration is outside that range, the duration field is omitted without failing submission.
+
 
 ### Integration
 
@@ -70,6 +76,9 @@ jobs:
   successful-job:
     runs-on: ubuntu-22.04
     steps:
+    # Must remain first to collect the automatic duration and resource metrics.
+    - uses: camunda/infra-global-github-actions/start-build-monitor@main
+
     # Needed to create a workspace so submit-build-status can store files!
     - uses: actions/checkout@v4
 

submit-build-status/action.yml

@@ -8,7 +8,7 @@ inputs:
     description: String representing the build status that should be submitted to CI Analytics, e.g. "success", "failed", "cancelled".
     required: true
   build_duration_millis:
-    description: Optional number (positive) that indicates the duration of the build in milliseconds.
+    description: Optional non-negative build duration in milliseconds. Overrides the duration collected by start-build-monitor.
     required: false
   user_reason:
     description: Optional string (200 chars max) the user can submit to indicate the reason why a build has ended with a certain build status , e.g. "flaky-tests".
@@ -36,22 +36,12 @@ inputs:
 runs:
   using: composite
   steps:
-  - name: Echo inputs
-    shell: bash
-    run: |
-      echo "Inputs"
-      echo "-----"
-      echo "Build status: ${{ inputs.build_status }}"
-      echo "Build duration (in ms): ${{ inputs.build_duration_millis }}"
-      echo "User reason: ${{ inputs.user_reason }}"
-      echo "User description: ${{ inputs.user_description }}"
-      echo "Job name override: ${{ inputs.job_name_override }} (default job name: $GITHUB_JOB)"
-      echo "BQ table name (for testing): ${{ inputs.big_query_table_name }}"
-
   - name: Validate inputs
+    id: validate
     shell: bash
     env:
       BUILD_STATUS: "${{ inputs.build_status }}"
+      BUILD_DURATION_MILLIS: "${{ inputs.build_duration_millis }}"
       USER_REASON: "${{ inputs.user_reason }}"
       USER_DESCRIPTION: "${{ inputs.user_description }}"
     run: |
@@ -69,6 +59,40 @@ runs:
         echo "Specified user description has more than 1000 characters!"
         exit 1
       fi
+      VALIDATED_BUILD_DURATION_MILLIS=$(bash "$GITHUB_ACTION_PATH/validate-build-duration.sh" "$BUILD_DURATION_MILLIS")
+      printf 'build_duration_millis=%s\n' "$VALIDATED_BUILD_DURATION_MILLIS" >> "$GITHUB_OUTPUT"
+
+  - name: Collect resource usage metrics
+    id: monitor
+    if: always()
+    shell: bash
+    run: |
+      bash "$GITHUB_ACTION_PATH/stop-and-collect-build-monitor.sh" "$GITHUB_OUTPUT"
+
+  - name: Resolve build duration
+    id: duration
+    shell: bash
+    env:
+      EXPLICIT_BUILD_DURATION_MILLIS: "${{ steps.validate.outputs.build_duration_millis }}"
+      MONITOR_BUILD_DURATION_MILLIS: "${{ steps.monitor.outputs.build_duration_millis }}"
+    run: |
+      BUILD_DURATION_MILLIS=$(bash "$GITHUB_ACTION_PATH/resolve-build-duration.sh" \
+        "$EXPLICIT_BUILD_DURATION_MILLIS" "$MONITOR_BUILD_DURATION_MILLIS")
+      printf 'build_duration_millis=%s\n' "$BUILD_DURATION_MILLIS" >> "$GITHUB_OUTPUT"
+
+  - name: Echo inputs
+    shell: bash
+    env:
+      BUILD_DURATION_MILLIS: "${{ steps.duration.outputs.build_duration_millis }}"
+    run: |
+      echo "Inputs"
+      echo "-----"
+      echo "Build status: ${{ inputs.build_status }}"
+      echo "Build duration (in ms): $BUILD_DURATION_MILLIS"
+      echo "User reason: ${{ inputs.user_reason }}"
+      echo "User description: ${{ inputs.user_description }}"
+      echo "Job name override: ${{ inputs.job_name_override }} (default job name: $GITHUB_JOB)"
+      echo "BQ table name (for testing): ${{ inputs.big_query_table_name }}"
 
   - name: Login to Google Cloud
     id: auth
@@ -77,22 +101,21 @@ runs:
       credentials_json: '${{ inputs.gcp_credentials_json }}'
       token_format: 'access_token'
 
-  - name: Collect resource usage metrics
-    id: monitor
-    shell: bash
-    run: |
-      bash "$GITHUB_ACTION_PATH/stop-and-collect-build-monitor.sh" "$GITHUB_OUTPUT"
-
   - name: Submit build status to CI Analytics
     shell: bash
     env:
       BUILD_BASE_REF: "${{ github.base_ref && format('refs/heads/{0}', github.base_ref) || github.event.merge_group.base_ref }}"
       BUILD_HEAD_REF: "${{ github.head_ref && format('refs/heads/{0}', github.head_ref) || github.event.merge_group.head_ref }}"
+      BUILD_DURATION_MILLIS: "${{ steps.duration.outputs.build_duration_millis }}"
       ACCESS_TOKEN: "${{ steps.auth.outputs.access_token }}"
       RUNNER_INFO_RUNS_ON_FILE: "/home/runner/.camunda-arc-runner-info/runs-on"
     run: |
       # ── Build monitor: get resource usage metrics (might be empty) ──
       MONITOR_FIELDS="${{ steps.monitor.outputs.monitor_fields }}"
+      BUILD_DURATION_FIELD=""
+      if [ -n "$BUILD_DURATION_MILLIS" ]; then
+        BUILD_DURATION_FIELD=$(printf ', "build_duration_milliseconds": "%s"' "$BUILD_DURATION_MILLIS")
+      fi
 
       # ── Read runs-on label info from self-hosted runner metadata (might be empty) ──
       RUNNER_TYPE_FIELD=""
@@ -129,7 +152,7 @@ runs:
         "runner_os": "$(echo $RUNNER_OS | tr '[:upper:]' '[:lower:]')"
         ${{ (env.BUILD_BASE_REF == '') && ' ' || format(', "build_base_ref": "{0}"', env.BUILD_BASE_REF) }}
         ${{ (env.BUILD_HEAD_REF == '') && ' ' || format(', "build_head_ref": "{0}"', env.BUILD_HEAD_REF) }}
-        ${{ (inputs.build_duration_millis == '') && ' ' || format(', "build_duration_milliseconds": "{0}"', inputs.build_duration_millis) }}
+        $BUILD_DURATION_FIELD
         ${{ (inputs.user_reason == '') && ' ' || format(', "user_reason": "{0}"', inputs.user_reason) }}
         ${{ (inputs.user_description == '') && ' ' || format(', "user_description": "{0}"', inputs.user_description) }}
         $MONITOR_FIELDS

submit-build-status/resolve-build-duration.sh

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+
+EXPLICIT_BUILD_DURATION_MILLIS="${1:-}"
+MONITOR_BUILD_DURATION_MILLIS="${2:-}"
+
+if [ -n "$EXPLICIT_BUILD_DURATION_MILLIS" ]; then
+  printf '%s\n' "$EXPLICIT_BUILD_DURATION_MILLIS"
+else
+  printf '%s\n' "$MONITOR_BUILD_DURATION_MILLIS"
+fi

submit-build-status/stop-and-collect-build-monitor.sh

@@ -8,78 +8,111 @@ fi
 
 MONITOR_FIELDS=""
 # NOTE: these paths are shared with start-build-monitor.sh -- if changed, update both
-PID_FILE=/tmp/_monitor-start.pid
-LOG_FILE=/tmp/_monitor-start.log
+PID_FILE="${BUILD_MONITOR_PID_FILE:-/tmp/_monitor-start.pid}"
+LOG_FILE="${BUILD_MONITOR_LOG_FILE:-/tmp/_monitor-start.log}"
+START_TIME_FILE="${BUILD_MONITOR_START_TIME_FILE:-/tmp/_monitor-start.epoch-millis}"
+NET_DEV_FILE="${BUILD_MONITOR_NET_DEV_FILE:-/proc/net/dev}"
+MAX_BUILD_DURATION_MILLIS=259200000  # 72 hours
+BUILD_DURATION_MILLIS=""
+
+is_valid_timestamp() {
+  [[ "$1" =~ ^[0-9]{1,18}$ ]]
+}
+
+if [ -f "$START_TIME_FILE" ]; then
+  START_TIME_MILLIS=$(cat "$START_TIME_FILE")
+  END_TIME_MILLIS=$(date +%s%3N)
+  if is_valid_timestamp "$START_TIME_MILLIS" && is_valid_timestamp "$END_TIME_MILLIS"; then
+    BUILD_DURATION_MILLIS=$((10#$END_TIME_MILLIS - 10#$START_TIME_MILLIS))
+    if [ "$BUILD_DURATION_MILLIS" -ge 0 ] && [ "$BUILD_DURATION_MILLIS" -le "$MAX_BUILD_DURATION_MILLIS" ]; then
+      echo "Build duration: ${BUILD_DURATION_MILLIS}ms"
+    else
+      echo "Build duration is outside the supported range (0-${MAX_BUILD_DURATION_MILLIS}ms); duration omitted" >&2
+      BUILD_DURATION_MILLIS=""
+    fi
+  else
+    echo "Build start or end timestamp is invalid; duration omitted" >&2
+  fi
+else
+  echo "Build start timestamp not found; duration omitted" >&2
+fi
 
 if [ -f "$PID_FILE" ]; then
   MONITOR_PID=$(cat "$PID_FILE")
   kill "$MONITOR_PID" 2>/dev/null || true
-  rm -f "$PID_FILE"
   echo "Resource monitor stopped (PID=$MONITOR_PID)"
-  echo "Raw monitor log values (CPU, memory usage ratio) per interval:"
-  echo "::group::Resource monitor raw log"
-  cat "$LOG_FILE"
-  echo "::endgroup::"
 
-  # Parse monitor log: header lines start with '#', data lines are "cpu_ratio mem_ratio".
-  # Both are already 0-1 ratios computed in start-build-monitor.
-  read -r AVG_CPU CPU_MAX P90_CPU P95_CPU AVG_MEM MEM_MAX P90_MEM P95_MEM COUNT < <(
-    awk '
-      function ceil(x) { return (x == int(x)) ? x : int(x) + 1 }
-      function sort_num(a, n,   i,j,tmp) {
-        for (i = 1; i <= n; i++)
-          for (j = i + 1; j <= n; j++)
-            if (a[i] > a[j]) { tmp = a[i]; a[i] = a[j]; a[j] = tmp }
-      }
-      function pctile(sorted, n, p,   idx) {
-        idx = ceil(p * n)
-        if (idx < 1) idx = 1
-        if (idx > n) idx = n
-        return sorted[idx]
-      }
-      /^#/   { next }
-      NF < 2 { next }
-      {
-        cv = $1 + 0; mv = $2 + 0
-        if (cv < 0) cv = 0; if (cv > 1) cv = 1
-        if (mv < 0) mv = 0; if (mv > 1) mv = 1
-        cpu_sum += cv; mem_sum += mv; count++
-        cpu[count] = cv; mem[count] = mv
-        if (count == 1 || cv > cpu_max) cpu_max = cv
-        if (count == 1 || mv > mem_max) mem_max = mv
-      }
-      END {
-        if (count > 0) {
-          sort_num(cpu, count); sort_num(mem, count)
-          printf "%.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %d\n",
-            cpu_sum/count, cpu_max, pctile(cpu,count,0.90), pctile(cpu,count,0.95),
-            mem_sum/count, mem_max, pctile(mem,count,0.90), pctile(mem,count,0.95),
-            count
+  if [ -f "$LOG_FILE" ]; then
+    echo "Raw monitor log values (CPU, memory usage ratio) per interval:"
+    echo "::group::Resource monitor raw log"
+    cat "$LOG_FILE"
+    echo "::endgroup::"
+
+    # Parse monitor log: header lines start with '#', data lines are "cpu_ratio mem_ratio".
+    # Both are already 0-1 ratios computed in start-build-monitor.
+    read -r AVG_CPU CPU_MAX P90_CPU P95_CPU AVG_MEM MEM_MAX P90_MEM P95_MEM COUNT < <(
+      awk '
+        function ceil(x) { return (x == int(x)) ? x : int(x) + 1 }
+        function sort_num(a, n,   i,j,tmp) {
+          for (i = 1; i <= n; i++)
+            for (j = i + 1; j <= n; j++)
+              if (a[i] > a[j]) { tmp = a[i]; a[i] = a[j]; a[j] = tmp }
+        }
+        function pctile(sorted, n, p,   idx) {
+          idx = ceil(p * n)
+          if (idx < 1) idx = 1
+          if (idx > n) idx = n
+          return sorted[idx]
+        }
+        /^#/   { next }
+        NF < 2 { next }
+        {
+          cv = $1 + 0; mv = $2 + 0
+          if (cv < 0) cv = 0; if (cv > 1) cv = 1
+          if (mv < 0) mv = 0; if (mv > 1) mv = 1
+          cpu_sum += cv; mem_sum += mv; count++
+          cpu[count] = cv; mem[count] = mv
+          if (count == 1 || cv > cpu_max) cpu_max = cv
+          if (count == 1 || mv > mem_max) mem_max = mv
         }
-      }
-    ' "$LOG_FILE"
-  ) || true  # read returns 1 when awk produces no output (empty log); || true prevents set -e from killing the step
+        END {
+          if (count > 0) {
+            sort_num(cpu, count); sort_num(mem, count)
+            printf "%.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %d\n",
+              cpu_sum/count, cpu_max, pctile(cpu,count,0.90), pctile(cpu,count,0.95),
+              mem_sum/count, mem_max, pctile(mem,count,0.90), pctile(mem,count,0.95),
+              count
+          }
+        }
+      ' "$LOG_FILE"
+    ) || true  # read returns 1 when awk produces no output (empty log); || true prevents set -e from killing the step
 
-  if [ -n "$COUNT" ] && [ "$COUNT" -gt 0 ]; then
-    read -r _ac _cm _p90c _p95c _am _mm _p90m _p95m < <(awk "BEGIN {printf \"%.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f\\n\", $AVG_CPU*100,$CPU_MAX*100,$P90_CPU*100,$P95_CPU*100,$AVG_MEM*100,$MEM_MAX*100,$P90_MEM*100,$P95_MEM*100}") || true
-    echo "CPU: avg=${_ac}%, p90=${_p90c}%, p95=${_p95c}%, max=${_cm}%  |  Mem: avg=${_am}%, p90=${_p90m}%, p95=${_p95m}%, max=${_mm}%  (${COUNT} samples)"
-    MONITOR_FIELDS=$(printf ', "cpu_usage_ratio_avg": %s, "cpu_usage_ratio_p95": %s, "memory_usage_ratio_avg": %s, "memory_usage_ratio_p95": %s' \
-      "$AVG_CPU" "$P95_CPU" "$AVG_MEM" "$P95_MEM")
+    if [ -n "$COUNT" ] && [ "$COUNT" -gt 0 ]; then
+      read -r _ac _cm _p90c _p95c _am _mm _p90m _p95m < <(awk "BEGIN {printf \"%.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f\\n\", $AVG_CPU*100,$CPU_MAX*100,$P90_CPU*100,$P95_CPU*100,$AVG_MEM*100,$MEM_MAX*100,$P90_MEM*100,$P95_MEM*100}") || true
+      echo "CPU: avg=${_ac}%, p90=${_p90c}%, p95=${_p95c}%, max=${_cm}%  |  Mem: avg=${_am}%, p90=${_p90m}%, p95=${_p95m}%, max=${_mm}%  (${COUNT} samples)"
+      MONITOR_FIELDS=$(printf ', "cpu_usage_ratio_avg": %s, "cpu_usage_ratio_p95": %s, "memory_usage_ratio_avg": %s, "memory_usage_ratio_p95": %s' \
+        "$AVG_CPU" "$P95_CPU" "$AVG_MEM" "$P95_MEM")
+    else
+      echo "Resource monitor log had no data rows -- metrics omitted"
+    fi
   else
-    echo "Resource monitor log had no data rows -- metrics omitted"
+    echo "Resource monitor log not found -- metrics omitted" >&2
   fi
 fi
 
+# Remove monitor state only after the log and start timestamp have been collected.
+rm -f "$PID_FILE" "$LOG_FILE" "$START_TIME_FILE"
+
 # ── Network bytes: cumulative tx/rx summed across non-loopback interfaces. ──
 # Runner pods are ephemeral (one job per pod), so the counters span the job;
 # on long-lived runners they are cumulative since interface-up.
 TX_BYTES=""; RX_BYTES=""
-if [ -r /proc/net/dev ]; then
+if [ -r "$NET_DEV_FILE" ]; then
   read -r TX_BYTES RX_BYTES < <(
     awk -F'[: ]+' '
       NR > 2 && $2 != "lo" { rx += $3 + 0; tx += $11 + 0 }
       END { printf "%.0f %.0f\n", tx, rx }
-    ' /proc/net/dev
+    ' "$NET_DEV_FILE"
   ) || true
   if [ -n "$TX_BYTES" ]; then
     echo "Network: egress=${TX_BYTES}B, ingress=${RX_BYTES}B"
@@ -89,3 +122,4 @@ if [ -r /proc/net/dev ]; then
 fi
 
 printf 'monitor_fields=%s\n' "$MONITOR_FIELDS" >> "$OUTPUT_FILE"
+printf 'build_duration_millis=%s\n' "$BUILD_DURATION_MILLIS" >> "$OUTPUT_FILE"