Published on 00/00/0000
Last updated on 00/00/0000
Published on 00/00/0000
Last updated on 00/00/0000
Share
INSIGHTS
7 min read
Published on 07/07/2020
Last updated on 06/18/2024
OIDC issuer discovery for Kubernetes service accounts
Share
Applications running in Kubernetes Pods are authenticated against the Kubernetes API with their corresponding ServiceAccount tokens. These JWT tokens are usually mounted into containers as files. JWT tokens are signed by the Kubernetes cluster's private key, and can be validated only with the TokenReview API. This API is not widely recognized and, to access it, external systems must first authenticate against Kubernetes to review ServiceAccounts. This configuration and access review process is considerably more complex than necessary, not to mention that it leaves out widely accepted standards like OIDC.
Kubernetes already has an OIDC integration, namely inbound authentication of users against the Kubernetes API. The new integration, which is what this blog post is about, wires OIDC in the opposite direction; the Service Account Issuer Discovery feature enables the federation of Kubernetes service account tokens issued by a cluster (the identity provider) with external systems (relying parties) based on the OIDC Discovery Spec. Projected Service Account Tokens are required for this feature to be enabled. Projected service account JWTs differ from "traditional" tokens in that they expire, have a proper issuer, and their audience fields come filled out so that they act like proper JWTs.
Today's post is going to be rather technical, since we'll be discussing authenticating Kubernetes applications with external systems through OIDC issuer discovery. We'll use Vault on Kubernetes as the OIDC consumer and a simple client application running in the cluster to access the Vault instance with a projected ServiceAccount token.
To be able to fetch the public keys and validate the JWT tokens against the Kubernetes cluster's issuer we have to allow external unauthenticated requests. To do this, we bind this special role (
In another terminal we need to configure the JWT Auth backend to federate Kubernetes JWT tokens with the OIDC endpoint:
Preparation
We need to create a Kubernetes cluster where theServiceAccountIssuerDiscovery feature gate is enabled. We are going to use kind to prepare our test cluster with some extra kubeadm patches to enable Service Account Token Volume Projection:
Some software you will be required to installed on your machine during this tutorial:
- kubectl
- kind
- curl
- jq
- step
- vault
kind create cluster --config - <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
featureGates:
ServiceAccountIssuerDiscovery: true
networking:
apiServerPort: 6443
kubeadmConfigPatches:
- |
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
extraArgs:
service-account-issuer: https://localhost:6443
service-account-jwks-uri: https://localhost:6443/openid/v1/jwks
service-account-signing-key-file: /etc/kubernetes/pki/sa.key
service-account-key-file: /etc/kubernetes/pki/sa.pub
EOFkubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
serviceAccountName: default
containers:
- image: nginx:alpine
name: oidc
volumeMounts:
- mountPath: /var/run/secrets/tokens
name: oidc-token
volumes:
- name: oidc-token
projected:
sources:
- serviceAccountToken:
path: oidc-token
expirationSeconds: 7200
audience: vault
EOFstep CLI:
kubectl exec nginx -- cat /var/run/secrets/tokens/oidc-token | step crypto jwt inspect --insecure{
"header": {
"alg": "RS256",
"kid": "Rt3TBA31bh3rH67PQbKImg2ldwhPqBTWF2w1Hxqi84c"
},
"payload": {
"aud": ["vault"],
"exp": 1592924135,
"iat": 1592916935,
"iss": "https://localhost:6443",
"kubernetes.io": {
"namespace": "default",
"pod": {
"name": "nginx",
"uid": "aa977398-8a06-4106-8563-972f9ecadd55"
},
"serviceaccount": {
"name": "default",
"uid": "b2680b48-75df-476f-9d95-2a0441c2bb83"
}
},
"nbf": 1592916935,
"sub": "system:serviceaccount:default:default"
},
"signature": "..."
}kubectl exec nginx -- cat /var/run/secrets/tokens/oidc-token | step crypto jwt inspect --insecure{
"header": {
"alg": "RS256",
"kid": "Rt3TBA31bh3rH67PQbKImg2ldwhPqBTWF2w1Hxqi84c"
},
"payload": {
"aud": ["vault"],
"exp": 1592924135,
"iat": 1592916935,
"iss": "https://localhost:6443",
"kubernetes.io": {
"namespace": "default",
"pod": {
"name": "nginx",
"uid": "aa977398-8a06-4106-8563-972f9ecadd55"
},
"serviceaccount": {
"name": "default",
"uid": "b2680b48-75df-476f-9d95-2a0441c2bb83"
}
},
"nbf": 1592916935,
"sub": "system:serviceaccount:default:default"
},
"signature": "..."
}system:service-account-issuer-discovery) with a ClusterRoleBinding to unauthenticated users (make sure that this is safe in your environment, but only public keys are visible on this URL):
kubectl create clusterrolebinding oidc-reviewer --clusterrole=system:service-account-issuer-discovery --group=system:unauthenticatedkubectl exec nginx -- cat /var/run/secrets/kubernetes.io/serviceaccount/ca.crt > kubernetes_ca.crtcurl --cacert kubernetes_ca.crt https://localhost:6443/.well-known/openid-configuration | jq{
"issuer": "https://localhost:6443",
"jwks_uri": "https://localhost:6443/openid/v1/jwks",
"response_types_supported": ["id_token"],
"subject_types_supported": ["public"],
"id_token_signing_alg_values_supported": ["RS256"]"jwks_uri") to view public keys:
curl --cacert kubernetes_ca.crt https://localhost:6443/openid/v1/jwks | jq
{
"keys": [
{
"use": "sig",
"kty": "RSA",
"kid": "Rt3TBA31bh3rH67PQbKImg2ldwhPqBTWF2w1Hxqi84c",
"alg": "RS256",
"n": "vL0tjBqLDFTyqOCPBQC5Mww_3xkhlkWmeklPjSAhFuqL0U-Oie9E1z8FuhcApBaUs7UEPzja02PEZd4i1UF2UDoxKYEG9hG5vPseTXwN_xGnbhOaBdfgQ7KDvqV-WHfmlrnnCizi1VmNAHsoAg6oZMiUdOuk8kCFxpe0N6THmBKNSKnqoRnhSL4uwHSBWJ5pEyWAqyL8KYaaGYhc2MVUs3I8e-gtQE6Vlwe75_QSp9uIZNZeFr5keqiXhz8BWL76ok-vY8UZ8-rH2VIN5LzXkCvhIFI9W_UBzziSnb9l5dgSQCwGf18zVgT0yJjCz0Z9YE9A1Wgeu-LLrJz3gxR8Hw",
"e": "AQAB"
}
]
}Configuring Vault as an OIDC consumer
We will use the Vault's JWT/OIDC Auth Method to consume the projected Service Account tokens from Kubernetes and validate them with the help of the OIDC Discovery endpoint exposed above.vault server -dev
vault server -dev
vault auth enable jwt
vault write auth/jwt/config \
oidc_discovery_url=https://localhost:6443 \
oidc_discovery_ca_pem=@kubernetes_ca.crt \
bound_issuer=https://localhost:6443
vault write auth/jwt/role/demo \
role_type=jwt \
bound_audiences=vault \
bound_subject="system:serviceaccount:default:default" \
user_claim=sub \
policies=defaultJWT=$(kubectl <span class="hljs-built_in">exec</span> nginx -- <span class="hljs-built_in">cat</span> /var/run/secrets/tokens/oidc-token)
curl http://127.0.0.1:8200/v1/auth/jwt/login --data <span class="hljs-string">"{\"jwt\": \"<span class="hljs-variable">$JWT</span>\", \"role\": \"demo\"}"</span> | jq{
"request_id": "c635533b-cfad-ba2d-c421-77eb18b45cd6",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": null,
"wrap_info": null,
"warnings": null,
"auth": {
"client_token": "s.TLRJddMCIo6d3BM70TjmVhkc",
"accessor": "koYXrht8K7rTlWZwgaBDGnBe",
"policies": ["default"],
"token_policies": ["default"],
"metadata": {
"role": "demo"
},
"lease_duration": 2764800,
"renewable": true,
"entity_id": "87b90fff-c019-5ebb-93e3-51677f538a53",
"token_type": "service",
"orphan": true
}
}VAULT_TOKEN=$(curl http://127.0.0.1:8200/v1/auth/jwt/login --data "{\"jwt\": \"$JWT\", \"role\": \"demo\"}" | jq -r .auth.client_token)
curl -H "X-Vault-Token: ${VAULT_TOKEN}" http://127.0.0.1:8200/v1/auth/token/lookup-self | jq{
"request_id": "5c8a033d-8f6f-a360-25ca-1ff32f5a69b8",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": {
"accessor": "2Q6PSJ1L9FLYcqBxNZA5tuuu",
"creation_time": 1592919300,
"creation_ttl": 2764800,
"display_name": "jwt-system:serviceaccount:default:default",
"entity_id": "bb1159b8-7b1c-bf88-509a-130e6666818b",
"expire_time": "2020-07-25T15:35:00.512007+02:00",
"explicit_max_ttl": 0,
"id": "s.nJm1aUQ6JsB39Yv3xankAXMe",
"issue_time": "2020-06-23T15:35:00.512019+02:00",
"meta": {
"role": "demo"
},
"num_uses": 0,
"orphan": true,
"path": "auth/jwt/login",
"policies": ["default"],
"renewable": true,
"ttl": 2764797,
"type": "service"
},
"wrap_info": null,
"warnings": null,
"auth": null
}Automating configuration and client access with Bank-Vaults
The JWT auth configuration of Vault and client access can be automated with the help of Bank-Vaults. This was introduced in a recent PR that added support for projected ServiceAccount tokens. The Bank-Vaults repository contains a fully-fledged Kubernetes OIDC federation example, where the OIDC endpoint is exposed internally, inside the cluster, on a special URL:https://kubernetes.
To set up a kind cluster with a JWT authenticated Vault instance, and run a client example, we have to check the repository and apply some manifests.
Note: This example requires kurun to be installed (brew install banzaicloud/tap/kurun), because the example container is built directly from Go code found in the repository at kubectl apply time.
Other requirements:
- go
- helm (v3)
git clone git@github.com:banzaicloud/bank-vaults.git
cd bank-vaults
# Create the OIDC issuer enabled cluster for in-cluster use
kind create cluster --config hack/kind.yaml
# Install the Banzai vault-operator
helm repo add banzaicloud-stable https://kubernetes-charts.banzaicloud.com
helm upgrade --install vault-operator banzaicloud-stable/vault-operator
# Create the Vault instance configured automatically for OIDC/JWT Auth
kubectl apply -f operator/deploy/rbac.yaml
kubectl apply -f operator/deploy/cr-oidc.yaml
# Run the example cluent which authenticates with a projected ServiceAccount JWT
kurun apply -f hack/oidc-pod.yaml
# Check the logs to make sure it works
kubectl logs -f oidcLearn more about Bank-Vaults:
- Secret injection webhook improvements
- Backing up Vault with Velero
- Vault replication across multiple datacenters
- Vault secret injection webhook and Istio
- HSM support
- Injecting dynamic configuration with templates
- OIDC issuer discovery for Kubernetes service accounts
- Show all posts related to Bank-Vaults
Subscribe to
the Shift!
Get emerging insights on emerging technology straight to your inbox.
