Signing Container Images with Cosign and Sigstore

Supply chain attacks are the category everyone talks about and almost nobody defends against properly. The XZ Utils backdoor showed how a carefully placed malicious contributor can compromise software that millions of systems trust. Image signing doesn't prevent a compromised build environment, but it does guarantee that the image running in your cluster is the exact one that passed through your CI pipeline — not a tampered copy sitting in the registry.

The attack surface is simpler than people assume: an attacker who can push to your registry — through compromised credentials, a misconfigured robot account, or a MITM between your CI runner and the registry — can swap a legitimate image with a malicious one. Your tag still says v1.4.2. Your deployment YAML hasn't changed. But you're running code you didn't build.

Cosign, part of the Sigstore project, gives you cryptographic proof that a given image digest was signed by a key you control. Combined with Kyverno's verifyImages admission rule, you get a closed loop: images that weren't signed by your CI pipeline are rejected at the Kubernetes API server before they ever schedule.

What You'll Build

• A Cosign keypair stored as a GitHub Actions secret
• A GitHub Actions workflow that builds, signs, and pushes an image
• Local verification of the signature
• A Kyverno ClusterPolicy that rejects unsigned images at deploy time
• Keyless signing with Sigstore Fulcio (no key management)

Step 1: Install Cosign

bash
Copy
1# macOS
2brew install cosign
3
4# Linux
5curl -O -L "https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64"
6chmod +x cosign-linux-amd64
7sudo mv cosign-linux-amd64 /usr/local/bin/cosign
8
9cosign version

You should see something like:

GitVersion:    v2.2.3
GitCommit:     abc1234...
BuildDate:     2024-01-15T00:00:00Z
Platform:      linux/amd64

Step 2: Generate a Keypair

bash
Copy
cosign generate-key-pair

Cosign prompts you for a passphrase, then writes two files:

Enter password for private key:
Enter password for private key again:
Private key written to cosign.key
Public key written to cosign.pub

cosign.key is the private key, encrypted with your passphrase. cosign.pub is the public key — safe to share, safe to commit to your repository.

The private key never leaves your environment. It goes into GitHub Actions secrets, not into the image or the registry. This is the only rule that matters for key hygiene: if cosign.key lands in version control, rotate immediately.

Store both values as GitHub Actions secrets in your repository settings:

• COSIGN_PRIVATE_KEY — paste the full contents of cosign.key, including the -----BEGIN ENCRYPTED COSIGN PRIVATE KEY----- header and footer
• COSIGN_PASSWORD — the passphrase you used during key generation

The public key (cosign.pub) should be committed to your repository. Verifiers — your Kyverno policy, your teammates, your auditors — need the public key to check signatures. Keeping it in the repo alongside your code is the natural place.

Step 3: Sign in GitHub Actions

The critical design decision here: sign the digest, not the tag. Tags are mutable — latest today can point to a different image layer set tomorrow. A digest (sha256:...) is a cryptographic hash of the image manifest. It is immutable by definition. Signing a tag gives you a false sense of security; signing a digest gives you a cryptographic guarantee.

The docker/build-push-action outputs the digest of the pushed image. We capture that and pass it directly to cosign sign.

yaml
Copy
1name: Build and Sign
2
3on:
4  push:
5    branches: [main]
6
7jobs:
8  build-sign:
9    runs-on: ubuntu-latest
10    permissions:
11      contents: read
12      packages: write
13      id-token: write  # Required for keyless signing
14
15    steps:
16      - uses: actions/checkout@v4
17
18      - name: Install Cosign
19        uses: sigstore/cosign-installer@v3.4.0
20
21      - name: Login to GHCR
22        uses: docker/login-action@v3
23        with:
24          registry: ghcr.io
25          username: ${{ github.actor }}
26          password: ${{ secrets.GITHUB_TOKEN }}
27
28      - name: Build and Push
29        id: build-push
30        uses: docker/build-push-action@v5
31        with:
32          context: .
33          push: true
34          tags: ghcr.io/${{ github.repository }}:latest
35
36      - name: Sign the image
37        env:
38          COSIGN_PRIVATE_KEY: ${{ secrets.COSIGN_PRIVATE_KEY }}
39          COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
40        run: |
41          cosign sign --key env://COSIGN_PRIVATE_KEY \
42            ghcr.io/${{ github.repository }}@${{ steps.build-push.outputs.digest }}

The --key env://COSIGN_PRIVATE_KEY syntax tells Cosign to read the private key from the environment variable rather than a file. This is the correct way to handle secrets in CI — no temp files, no disk writes.

The id-token: write permission is there already because we'll need it for keyless signing in Step 6. It has no effect when using key-based signing.

When the workflow runs, Cosign pushes the signature as an OCI artifact to the same registry. From the registry's perspective, signatures are just another artifact referencing the image manifest by digest — no separate signature storage, no external database.

Step 4: Verify the Signature Locally

Once the workflow runs and the image is signed, pull the public key from your repository and verify:

bash
Copy
cosign verify \
  --key cosign.pub \
  ghcr.io/yourorg/yourimage@sha256:<digest>

On success, Cosign outputs the verification payload as JSON:

json
Copy
1[
2  {
3    "critical": {
4      "identity": {
5        "docker-reference": "ghcr.io/yourorg/yourimage"
6      },
7      "image": {
8        "docker-manifest-digest": "sha256:abc123..."
9      },
10      "type": "cosign container image signature"
11    },
12    "optional": {
13      "Bundle": {
14        "SignedEntryTimestamp": "...",
15        "Payload": {
16          "body": "...",
17          "integratedTime": 1713964800,
18          "logIndex": 123456789,
19          "logID": "..."
20        }
21      }
22    }
23  }
24]

The critical.image.docker-manifest-digest field must match the digest you're verifying. If someone has replaced the image in the registry, the digest won't match the signature, and verification fails hard:

Error: no signatures found for image
main.go:62: error during command execution: no signatures found for image

Cosign exits non-zero on failure. Wire this into any script or pipeline step where you need to gate on signature verification before deployment.

Step 5: Enforce with Kyverno

Local verification is useful for debugging. What you actually want is enforcement at the cluster boundary — images that can't be verified should never schedule. Kyverno's verifyImages rule handles this at the admission controller level.

yaml
Copy
1apiVersion: kyverno.io/v1
2kind: ClusterPolicy
3metadata:
4  name: verify-image-signature
5spec:
6  validationFailureAction: Enforce
7  background: false
8  rules:
9    - name: check-image-signature
10      match:
11        any:
12          - resources:
13              kinds:
14                - Pod
15      verifyImages:
16        - imageReferences:
17            - "ghcr.io/yourorg/*"
18          attestors:
19            - count: 1
20              entries:
21                - keys:
22                    publicKeys: |-
23                      -----BEGIN PUBLIC KEY-----
24                      <contents of cosign.pub>
25                      -----END PUBLIC KEY-----

Apply it:

bash
Copy
kubectl apply -f kyverno-verify-image.yaml

The imageReferences glob determines which images the policy covers. Be specific — ghcr.io/yourorg/* covers your org's images without blocking third-party images you pull from other registries. If you want to cover all images in the cluster, use *, but be prepared to sign everything including base images.

background: false is non-negotiable for verifyImages rules. Background reconciliation runs outside the admission path and cannot make outbound network calls to verify signatures against the registry. Admission-time verification is the only mode that works. Kyverno will warn you if you set background: true here, but you want to be explicit.

Test enforcement with an image that wasn't signed by your key:

bash
Copy
# Try to deploy an unsigned image
kubectl run unsigned --image=ghcr.io/yourorg/yourimage:untagged

Error from server: admission webhook "mutate.kyverno.svc-fail" denied the request:
image signature verification failed

The pod never reaches the scheduler. The admission webhook rejects it and returns the error directly to kubectl. This is the behavior you want — fail closed, fail loud.

Step 6: Keyless Signing with Sigstore Fulcio

Key-based signing works, but key management is friction. You need to rotate the key when someone leaves the team. You need to protect the private key in secrets management. You need to distribute the public key to every verifier. Keyless signing eliminates all of that.

Keyless signing uses GitHub Actions OIDC to prove identity. Instead of a long-lived keypair, Cosign requests a short-lived certificate from Sigstore's Fulcio CA. The certificate is tied to your GitHub Actions workflow URL. The signing event is recorded in the Rekor transparency log. No key to manage, no key to rotate, no key to lose.

In your GitHub Actions workflow, replace the Sign the image step:

yaml
Copy
      - name: Sign the image (keyless)
        run: |
          cosign sign \
            ghcr.io/${{ github.repository }}@${{ steps.build-push.outputs.digest }}

No --key flag. No secret references. The id-token: write permission you already set is all Cosign needs to get the OIDC token from GitHub Actions and exchange it for a Fulcio certificate.

Verify a keyless signature by specifying the expected identity instead of a key:

bash
Copy
cosign verify \
  --certificate-identity-regexp="https://github.com/yourorg/yourrepo" \
  --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
  ghcr.io/yourorg/yourimage@sha256:<digest>

The --certificate-identity-regexp matches the workflow URL recorded in the certificate. This is what Cosign uses to confirm the signature came from your specific repository and workflow, not just any GitHub Actions run.

For Kyverno enforcement of keyless signatures, replace the keys entry with a keyless entry:

yaml
Copy
1      verifyImages:
2        - imageReferences:
3            - "ghcr.io/yourorg/*"
4          attestors:
5            - count: 1
6              entries:
7                - keyless:
8                    subject: "https://github.com/yourorg/yourrepo/.github/workflows/build.yml@refs/heads/main"
9                    issuer: "https://token.actions.githubusercontent.com"

The subject must match the exact workflow ref that signed the image. This is more precise than key-based signing — it ties the signature to a specific workflow file at a specific branch, not just "anyone with the private key."

Verification

Once your signing workflow is running and your Kyverno policy is applied, a few commands worth keeping in your runbook:

bash
Copy
1# List all signatures for an image (returns the OCI reference where signatures are stored)
2cosign triangulate ghcr.io/yourorg/yourimage:latest
3
4# Verify and output the full payload
5cosign verify --key cosign.pub ghcr.io/yourorg/yourimage@sha256:<digest> | jq .
6
7# Check that Kyverno policy is active
8kubectl get clusterpolicy verify-image-signature
9
10# Describe policy to see current enforcement status
11kubectl describe clusterpolicy verify-image-signature

cosign triangulate is useful for debugging registry issues — it shows you exactly where Cosign is looking for the signature artifact. If verification fails unexpectedly, check that the signature OCI artifact exists at the returned reference.

Common Mistakes

Signing the tag instead of the digest. Tags are mutable. If you run cosign sign ghcr.io/yourorg/yourimage:latest, you're signing a pointer that can be changed. Sign and verify by digest: @sha256:.... The docker/build-push-action outputs the digest; always use it.

Committing cosign.key to the repository. The private key file is the secret. It must only exist in secrets management — GitHub Actions secrets, Vault, AWS Secrets Manager. If it lands in version control, even in a private repo, rotate immediately: generate a new keypair, update the secret, and update the public key in the repo.

Using a registry that doesn't support OCI artifact referrers. Cosign stores signatures as OCI artifacts referencing the image by digest. Modern registries support this: GHCR, ECR, Docker Hub v2, Google Artifact Registry, Azure Container Registry. Older self-hosted registries or pull-through caches sometimes don't store OCI referrer artifacts. Test your registry with cosign triangulate before building your enforcement policy around it.

Setting background: true on verifyImages rules. Background scanning runs in the Kyverno controller outside the admission path. It cannot make outbound network calls to verify signatures at the registry. The rule silently does nothing in background mode. Always set background: false for verifyImages.

Assuming keyless signing works in air-gapped environments. Keyless signing requires two public Sigstore services: Fulcio (the CA) and Rekor (the transparency log). In air-gapped or restricted-network clusters, both must be unavailable by definition. Sigstore's scaffolding project lets you run private Fulcio and Rekor instances. This is a non-trivial operational commitment — factor it in before choosing keyless over key-based for regulated environments.

Cleanup

bash
Copy
kubectl delete clusterpolicy verify-image-signature

# Delete the private key file — never commit it
rm cosign.key

The public key (cosign.pub) can stay in your repository. It's worthless to an attacker without the corresponding private key.

Image signing is one layer of a supply chain security posture. It answers the question "did this image come from our CI pipeline?" — it doesn't answer "is our CI pipeline itself trustworthy?" For that you need separate controls: pinned actions, signed commits, artifact attestations, SBOM generation. Cosign supports attestations through the cosign attest command — the same signing infrastructure, applied to SBOM or vulnerability scan results attached to the image. That's a natural next step once key-based or keyless signing is working reliably.

Official References

• Cosign Documentation — Official Sigstore docs covering key-based and keyless signing, verification, and attestations
• Cosign GitHub Repository — Source, releases, and the full list of supported registries and features
• Sigstore Policy Controller — Sigstore's own Kubernetes admission controller for enforcing image signatures (alternative to Kyverno)
• Kyverno verifyImages — Kyverno's image verification rule reference, covering both key-based and keyless attestors
• Rekor Transparency Log — How keyless signatures are recorded and auditable in the Rekor append-only log