Skip to Content
GuidesWorkflowsEnrollmentVault PKI Integration Guide

Vault PKI Integration Guide

HashiCorp Vault’s PKI secrets engine provides enterprise-grade certificate management for Agent Harbor deployments. This guide covers integration with Vault for automated certificate issuance and rotation.

Overview

Vault PKI provides:

  • Dynamic certificates: Issued on-demand with configurable TTLs
  • Policy-based access: Fine-grained control over certificate issuance
  • Audit logging: Complete audit trail of all certificate operations
  • Integration options: AppRole, Kubernetes, OIDC authentication

Architecture

┌──────────────────────────────────────────────────────────────┐
│                     Vault Cluster                            │
│                                                              │
│  ┌─────────────────┐    ┌─────────────────────────────────┐  │
│  │   PKI Engine    │    │         Policy Engine           │  │
│  │  (Root + Int)   │    │  (ah-executor, ah-access-point) │  │
│  └────────┬────────┘    └─────────────────────────────────┘  │
│           │                                                  │
│           │ Issue Certificate                                │
│           ▼                                                  │
│  ┌─────────────────┐                                         │
│  │  PKI Role       │ ◄── allowed_domains, key_usage, ttl     │
│  │  (executor)     │                                         │
│  └────────┬────────┘                                         │
│           │                                                  │
└───────────┼──────────────────────────────────────────────────┘
            │
            │ HTTPS/mTLS
            ▼
┌───────────────────────┐        ┌───────────────────────┐
│     Access Point      │◄──────►│       Executor        │
│  (ah agent access-    │  mTLS  │     (ah agent)        │
│       point)          │        │                       │
└───────────────────────┘        └───────────────────────┘

Vault Setup

1. Enable PKI Secrets Engine

# Enable PKI for root CA
vault secrets enable -path=pki pki
vault secrets tune -max-lease-ttl=87600h pki
 
# Generate root CA
vault write pki/root/generate/internal \
  common_name="Agent Harbor Root CA" \
  ttl=87600h
 
# Enable PKI for intermediate CA
vault secrets enable -path=pki_int pki
vault secrets tune -max-lease-ttl=43800h pki_int
 
# Generate intermediate CSR
vault write -format=json pki_int/intermediate/generate/internal \
  common_name="Agent Harbor Intermediate CA" \
  | jq -r '.data.csr' > pki_int.csr
 
# Sign intermediate with root
vault write -format=json pki/root/sign-intermediate \
  csr=@pki_int.csr \
  format=pem_bundle \
  ttl=43800h \
  | jq -r '.data.certificate' > intermediate.pem
 
# Set signed intermediate
vault write pki_int/intermediate/set-signed \
  certificate=@intermediate.pem

2. Configure PKI Roles

# Role for access point certificates
vault write pki_int/roles/access-point \
  allowed_domains="access-point.example.com,localhost" \
  allow_subdomains=true \
  allow_localhost=true \
  max_ttl=72h \
  key_type=ec \
  key_bits=256 \
  key_usage="DigitalSignature,KeyEncipherment" \
  ext_key_usage="ServerAuth" \
  allowed_uri_sans="spiffe://example.org/ah/serve"
 
# Role for executor certificates
vault write pki_int/roles/executor \
  allowed_domains="executor.example.com" \
  allow_subdomains=true \
  max_ttl=24h \
  key_type=ec \
  key_bits=256 \
  key_usage="DigitalSignature,KeyEncipherment" \
  ext_key_usage="ClientAuth" \
  allowed_uri_sans="spiffe://example.org/ah/agent/*" \
  allow_any_name=false

3. Create Policies

# Policy for access point
cat > ah-access-point-policy.hcl << 'EOF'
path "pki_int/issue/access-point" {
  capabilities = ["create", "update"]
}
 
path "pki_int/cert/ca" {
  capabilities = ["read"]
}
EOF
 
vault policy write ah-access-point ah-access-point-policy.hcl
 
# Policy for executors
cat > ah-executor-policy.hcl << 'EOF'
path "pki_int/issue/executor" {
  capabilities = ["create", "update"]
}
 
path "pki_int/cert/ca" {
  capabilities = ["read"]
}
EOF
 
vault policy write ah-executor ah-executor-policy.hcl

4. Configure Authentication

# Enable AppRole auth
vault auth enable approle
 
# Create role for access point
vault write auth/approle/role/access-point \
  token_policies="ah-access-point" \
  token_ttl=1h \
  token_max_ttl=4h \
  secret_id_ttl=720h
 
# Get role ID and secret ID
vault read auth/approle/role/access-point/role-id
vault write -f auth/approle/role/access-point/secret-id
 
# Create role for executors
vault write auth/approle/role/executor \
  token_policies="ah-executor" \
  token_ttl=1h \
  token_max_ttl=4h \
  secret_id_ttl=720h

Kubernetes Auth (For K8s deployments)

# Enable Kubernetes auth
vault auth enable kubernetes
 
# Configure Kubernetes auth
vault write auth/kubernetes/config \
  kubernetes_host="https://kubernetes.default.svc:443"
 
# Create role for executor pods
vault write auth/kubernetes/role/executor \
  bound_service_account_names=ah-executor \
  bound_service_account_namespaces=agent-harbor \
  policies=ah-executor \
  ttl=1h

Agent Harbor Configuration

Access Point with Vault

ah agent access-point \
  --fleet-listen 0.0.0.0:4433 \
  --server-identity vault \
  --vault-addr https://vault.example.com:8200 \
  --vault-pki-path pki_int/issue/access-point \
  --vault-role-id $ROLE_ID \
  --vault-secret-id $SECRET_ID \
  --vault-cert-cn access-point.example.com \
  --vault-cert-ttl 24h \
  --executor-san-uri-prefix "spiffe://example.org/ah/agent/"

Executor Enrollment with Vault

ah agent enroll \
  --remote-server https://access-point.example.com:4433 \
  --identity vault \
  --vault-addr https://vault.example.com:8200 \
  --vault-pki-path pki_int/issue/executor \
  --vault-role-id $EXECUTOR_ROLE_ID \
  --vault-secret-id $EXECUTOR_SECRET_ID \
  --vault-cert-cn executor-1.example.com \
  --vault-cert-san-uri "spiffe://example.org/ah/agent/executor-1" \
  --name executor-1

Environment Variables

Instead of command-line flags, use environment variables for sensitive values:

export VAULT_ADDR=https://vault.example.com:8200
export VAULT_ROLE_ID=your-role-id
export VAULT_SECRET_ID=your-secret-id
 
ah agent enroll \
  --remote-server https://access-point.example.com:4433 \
  --identity vault \
  --vault-pki-path pki_int/issue/executor \
  --name executor-1

NixOS Deployment

Access Point with Vault

{ config, pkgs, ... }:
{
  imports = [ /path/to/agent-harbor/nix/nixos-modules ];
 
  # Secrets management (using sops-nix)
  sops.secrets."vault/role-id" = { };
  sops.secrets."vault/secret-id" = { };
 
  services.agent-harbor.access-point = {
    enable = true;
    settings = {
      fleetListen = "0.0.0.0:4433";
      serverIdentity = "vault";
      vault = {
        address = "https://vault.example.com:8200";
        pkiPath = "pki_int/issue/access-point";
        certCn = "access-point.example.com";
        certTtl = "24h";
      };
      executorSanUriPrefix = "spiffe://example.org/ah/agent/";
    };
    # Secrets injected at runtime
    environmentFile = config.sops.secrets."vault/env".path;
  };
}

Executor with Vault

{ config, pkgs, ... }:
{
  sops.secrets."vault/executor-role-id" = { };
  sops.secrets."vault/executor-secret-id" = { };
 
  services.agent-harbor.executor = {
    enable = true;
    settings = {
      remoteServer = "https://access-point.example.com:4433";
      identity = "vault";
      vault = {
        address = "https://vault.example.com:8200";
        pkiPath = "pki_int/issue/executor";
        certCn = config.networking.hostName;
      };
    };
    environmentFile = config.sops.secrets."vault/executor-env".path;
  };
}

Certificate Rotation

The Vault identity provider handles certificate rotation automatically:

  1. Initial issuance: Certificate requested on startup
  2. TTL monitoring: Provider tracks certificate expiry
  3. Renewal threshold: Renewal triggered at 2/3 of TTL
  4. Seamless rotation: New cert used for new connections

Rotation Configuration

# Set renewal threshold (default: 2/3 of TTL)
ah agent enroll \
  --identity vault \
  --vault-cert-ttl 24h \
  --vault-renewal-threshold 0.75 \  # Renew at 75% of TTL
  ...

Monitoring Rotation

# Check current certificate
ah agent status --show-cert
 
# View rotation logs
journalctl -u ah-executor -f | grep -i "certificate\|rotation\|renew"

High Availability

Vault Cluster Configuration

For production, deploy Vault in HA mode:

# Agent Harbor automatically handles Vault leader failover
ah agent access-point \
  --vault-addr https://vault.example.com:8200 \
  --vault-retry-max 5 \
  --vault-retry-delay 2s \
  ...

Multiple Vault Clusters

For multi-region deployments:

# Primary region
ah agent access-point \
  --vault-addr https://vault-us-east.example.com:8200 \
  ...
 
# Failover region (separate configuration)
ah agent access-point \
  --vault-addr https://vault-us-west.example.com:8200 \
  ...

Security Best Practices

1. Use Short-Lived Credentials

# AppRole with short secret ID TTL
vault write auth/approle/role/executor \
  secret_id_ttl=24h \
  secret_id_num_uses=1  # One-time use

2. Restrict PKI Roles

# Limit allowed SANs strictly
vault write pki_int/roles/executor \
  allowed_domains="executor.internal.example.com" \
  allow_subdomains=false \
  allowed_uri_sans="spiffe://example.org/ah/agent/*" \
  enforce_hostnames=true

3. Enable Audit Logging

vault audit enable file file_path=/var/log/vault/audit.log

4. Use Namespaces (Enterprise)

# Isolate Agent Harbor PKI in namespace
vault namespace create agent-harbor
vault secrets enable -namespace=agent-harbor -path=pki pki

Troubleshooting

Permission Denied

error: permission denied for path pki_int/issue/executor

Solutions:

  • Check policy is attached to token: vault token lookup
  • Verify policy path matches PKI path
  • Check namespace if using Vault Enterprise

Certificate CN/SAN Rejected

error: common name not in allowed domains

Solutions:

  • Update PKI role allowed_domains
  • Enable allow_any_name for testing (not recommended for production)
  • Check allowed_uri_sans pattern matches

Vault Unavailable

error: failed to connect to vault: connection refused

Solutions:

  • Verify Vault address and port
  • Check network connectivity and firewall
  • Ensure Vault is unsealed: vault status
  • Check TLS certificate if using HTTPS

Token Expired

error: vault token expired

Solutions:

  • Verify token_ttl and token_max_ttl in AppRole
  • Check system time synchronization
  • Re-authenticate to obtain new token

Migration from Other Providers

From Files to Vault

  1. Deploy Vault and configure PKI
  2. Update access point to use Vault identity
  3. Gradually migrate executors
  4. Remove old certificate files

From SPIFFE to Vault

If moving from SPIRE to Vault:

  1. Configure Vault with SPIFFE-compatible URI SANs
  2. Update --executor-san-uri-prefix to match
  3. Deploy new executors with Vault identity
  4. Decommission SPIRE infrastructure

Next Steps

Enrollment Troubleshooting GuideTroubleshooting