Skip to content

ci: keyless Harbor robot + K8s creds via GitHub OIDC->Vault (retire admin HARBOR_* + KUBECONFIG_PROD) #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mattmattox merged 1 commit into from
Jul 2, 2026
Merged

ci: keyless Harbor robot + K8s creds via GitHub OIDC->Vault (retire admin HARBOR_* + KUBECONFIG_PROD) #16

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 72 additions & 31 deletions .github/workflows/pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,9 @@ jobs:
name: Build Docker Image
runs-on: ubuntu-latest
needs: [test-server, test-web]
permissions:
contents: read
id-token: write # GitHub OIDC -> Vault (keyless Harbor login)
outputs:
image-tag: ${{ steps.meta.outputs.tags }}
image-digest: ${{ steps.build.outputs.digest }}
Expand All @@ -152,22 +155,29 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Validate Harbor credentials
# Keyless: mint a GitHub OIDC token and exchange it at Vault for the
# repo-scoped Harbor robot (replaces org admin HARBOR_USERNAME/PASSWORD).
- name: Vault login (GitHub OIDC) + Harbor robot
if: github.event_name != 'pull_request'
run: |
if [ -z "${{ secrets.HARBOR_USERNAME }}" ] || [ -z "${{ secrets.HARBOR_PASSWORD }}" ]; then
echo "ERROR: Harbor credentials (HARBOR_USERNAME, HARBOR_PASSWORD) not configured"
echo "Please configure these secrets in repository settings"
exit 1
fi
uses: hashicorp/vault-action@v3
with:
url: https://vault.support.tools
method: jwt
path: github-actions
role: gha-kubetty
jwtGithubAudience: https://github.com/SupportTools
exportEnv: true
secrets: |
secret/data/harbor/kubetty-ci-robot username | HARBOR_ROBOT_USER ;
secret/data/harbor/kubetty-ci-robot password | HARBOR_ROBOT_TOKEN

- name: Log in to Harbor
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.HARBOR_USERNAME }}
password: ${{ secrets.HARBOR_PASSWORD }}
username: ${{ env.HARBOR_ROBOT_USER }}
password: ${{ env.HARBOR_ROBOT_TOKEN }}

- name: Extract metadata
id: meta
Expand Down Expand Up @@ -207,24 +217,32 @@ jobs:
actions: read
contents: read
security-events: write
id-token: write # GitHub OIDC -> Vault (keyless Harbor login)
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Validate Harbor credentials
run: |
if [ -z "${{ secrets.HARBOR_USERNAME }}" ] || [ -z "${{ secrets.HARBOR_PASSWORD }}" ]; then
echo "ERROR: Harbor credentials (HARBOR_USERNAME, HARBOR_PASSWORD) not configured"
echo "Please configure these secrets in repository settings"
exit 1
fi
# Keyless: exchange the GitHub OIDC token at Vault for the repo-scoped
# Harbor robot (replaces org admin HARBOR_USERNAME/PASSWORD).
- name: Vault login (GitHub OIDC) + Harbor robot
uses: hashicorp/vault-action@v3
with:
url: https://vault.support.tools
method: jwt
path: github-actions
role: gha-kubetty
jwtGithubAudience: https://github.com/SupportTools
exportEnv: true
secrets: |
secret/data/harbor/kubetty-ci-robot username | HARBOR_ROBOT_USER ;
secret/data/harbor/kubetty-ci-robot password | HARBOR_ROBOT_TOKEN

- name: Log in to Harbor
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.HARBOR_USERNAME }}
password: ${{ secrets.HARBOR_PASSWORD }}
username: ${{ env.HARBOR_ROBOT_USER }}
password: ${{ env.HARBOR_ROBOT_TOKEN }}

- name: Extract image tag
id: extract-tag
Expand Down Expand Up @@ -333,9 +351,12 @@ jobs:
# Stage 6: Deploy to Production
deploy-production:
name: Deploy to Production
runs-on: ubuntu-latest
runs-on: self-hosted-linux
needs: [build, helm-validate]
if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
permissions:
contents: read
id-token: write # GitHub OIDC -> Vault (keyless, namespace-scoped K8s token)
concurrency:
group: deploy-production
cancel-in-progress: false
Expand Down Expand Up @@ -380,19 +401,39 @@ jobs:
with:
version: 'v3.15.2'

- name: Validate Kubernetes credentials
run: |
if [ -z "${{ secrets.KUBECONFIG_PROD }}" ]; then
echo "ERROR: KUBECONFIG_PROD secret not configured"
echo "Please configure this secret in repository settings"
exit 1
fi

- name: Configure kubeconfig
# Keyless: exchange the GitHub OIDC token at Vault; exportToken makes
# VAULT_TOKEN available so the next step can mint a short-lived,
# namespace-scoped Kubernetes token (replaces org admin KUBECONFIG_PROD).
- name: Vault login (GitHub OIDC)
uses: hashicorp/vault-action@v3
with:
url: https://vault.support.tools
method: jwt
path: github-actions
role: gha-kubetty
jwtGithubAudience: https://github.com/SupportTools
exportToken: true

- name: Mint short-lived Kubernetes deploy token
env:
VAULT_ADDR: https://vault.support.tools
run: |
mkdir -p $HOME/.kube
echo "${{ secrets.KUBECONFIG_PROD }}" | base64 -d > $HOME/.kube/config
chmod 600 $HOME/.kube/config
KUBE_TOKEN=$(curl -sf -H "X-Vault-Token: ${VAULT_TOKEN}" \
-X PUT -d '{"kubernetes_namespace":"arc-runners-supporttools"}' \

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Generate the deploy token in the target namespace

The deploy job mints Vault Kubernetes credentials for arc-runners-supporttools, but every subsequent Helm/kubectl operation targets kubetty-gateway-prd. HashiCorp documents kubernetes_namespace as the namespace where the credentials are generated (and the response reports that service account namespace), so with the namespace-scoped token described here this job will authenticate as a service account scoped to the runner namespace and fail to deploy into kubetty-gateway-prd; request credentials for kubetty-gateway-prd instead, or explicitly use a cluster-wide binding if that is intended. See https://developer.hashicorp.com/vault/api-docs/secret/kubernetes#generate-credentials.

Useful? React with 👍 / 👎.

"${VAULT_ADDR}/v1/kubernetes-onprem/creds/kubetty" \
| jq -r '.data.service_account_token')
test -n "$KUBE_TOKEN" && test "$KUBE_TOKEN" != null || { echo "::error::failed to mint K8s token"; exit 1; }
echo "::add-mask::$KUBE_TOKEN"
# Build the kubeconfig at the same path the deploy steps already use.
# In-cluster runner => talk to a1-ops-prd's own apiserver + in-cluster CA.
mkdir -p "$HOME/.kube"
kubectl config set-cluster onprem \
--server=https://kubernetes.default.svc:443 \
--certificate-authority=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
--embed-certs=true
kubectl config set-credentials kubetty-ci --token="${KUBE_TOKEN}"
kubectl config set-context ci --cluster=onprem --user=kubetty-ci --namespace=kubetty-gateway-prd
kubectl config use-context ci

- name: Extract image tag
id: extract-tag
Expand Down
Loading