wishforge-ide/docs/deployment-secrets.md
eust-w b47a07e8d2 chore: baseline snapshot of the Codex-delivered code before review
Imports the previously untracked working tree as-is so that the review
changes that follow (requirements -> design -> implementation) can be
reviewed as diffs and reverted if needed. No content changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 13:18:04 +08:00

12 KiB

Versioned application secrets in the deployment process

Application secrets use DeploymentSecretBroker from @wishforge/runtime. This broker is instantiated only in the isolated deployment Runner/controller. Coding workers, verifiers, the browser and the control-plane API do not receive its Vault token or secret values. It is independent of the model BYOK Vault transit service: VAULT_ADDR, VAULT_TOKEN, VAULT_TRANSIT_KEY and WF_VAULT_KEY are never fallback credentials for this broker.

The control plane stores version references and an approved public policy digest. It does not store application secret material. A release cannot silently change its secret mapping after approval: deploymentSecretPolicyDigest must be part of the Environment/Release binding and signed Runner contract. The deployer supplies this as expectedPolicyDigest; prepare() rejects a non-empty secret set without it.

Operator-owned mapping

Configure these only in the deployment process:

NODE_ENV=production
WF_DEPLOYMENT_SECRET_MAP_FILE=/run/wishforge/application-secrets.json
WF_DEPLOYMENT_VAULT_ORIGIN=https://vault-deployment.example.internal
WF_DEPLOYMENT_VAULT_TOKEN_FILE=/run/secrets/deployment-vault-read-token

Mount the map and token file read-only. Grant that token only the specific KV v2 read paths assigned to this deployment environment. It needs no write, delete, transit, policy-management or provider-account privileges. Rotate it independently from application values and model-provider credentials. A lost/expired token fails closed; the broker does not obtain credentials from user login caches.

Example operator map:

{
  "schemaVersion": 1,
  "bindings": [{
    "tenantId": "tenant-a",
    "environmentId": "production-a",
    "ref": "database@version-7",
    "mount": "applications",
    "path": "tenant-a/production-a/database",
    "version": 7,
    "fields": {
      "DATABASE_URL": "url",
      "DATABASE_PASSWORD": "password"
    }
  }]
}

fields maps application environment names to Vault object fields. A release passes only its authorized secretVersionRefs, never Vault URLs, paths, tokens, field selectors or plaintext. Reference strings are exact opaque operator names; explicit version is mandatory and positive. Use version-bearing names when registering them. The broker never requests the latest version. Paths use safe explicit segments; traversal, encoded slashes and caller-defined URLs are rejected. Duplicate mappings, duplicate environment keys and conflicts with public application configuration are rejected. Process-control keys such as NODE_OPTIONS, LD_PRELOAD, PATH, WF_*, DOCKER_* and VAULT_* cannot be injected through this mechanism.

The fixed origin requires HTTPS in production. Only test/development with explicit WF_DEPLOYMENT_VAULT_ALLOW_LOOPBACK_HTTP=true can use HTTP on a loopback address. Redirects are rejected. Each exact-version read has a 10-second timeout, a 256 KiB response bound, at most 64 selected fields per reference and 128 selected fields in total. Selected string values have a 16 KiB limit; total selected material is limited to 128 KiB. Missing, deleted, destroyed, non-string or version-mismatched values fail closed with generic errors.

The protocol is Vault KV v2's versioned GET endpoint. The broker checks the returned version, deletion time and destroyed flag instead of assuming a successful HTTP response represents the requested live version. HashiCorp KV v2 API.

Public plan and approval

An operator can inspect the plan without reading the Vault token or contacting Vault:

node --import tsx scripts/deployment-secret-plan.ts \
  tenant-a production-a reviewed-release-id database@version-7

The output contains the policy digest, authorized references/versions, application key names, a deterministic immutable Secret name and environment-reference wiring. It contains no Vault path, token or secret value. The policy digest covers tenant, environment, fixed Vault origin, mount/path/version and field mappings. It excludes token rotation. A human operator applies this digest through the environment's protected configuration before requesting/approving a release. Changing a path or field while retaining a reference name changes the digest and invalidates that release's ability to retrieve material.

The broker re-reads the operator map after Vault I/O. A concurrent mapping change cannot silently replace the plan under an in-flight retrieval. Immutable runtime identity must include secretVersionRefs and deploymentSecretPolicyDigest, not short-lived tokens or plaintext.

Integration contract

const scope = {
  tenantId: release.tenantId,
  environmentId: release.environmentId,
  releaseId: release.id,
  secretVersionRefs: release.secretVersionRefs ?? [],
  publicConfigKeys: Object.keys(release.config ?? {}),
  expectedPolicyDigest: release.deploymentSecretPolicyDigest,
};
const prepared = await broker.prepare(scope);
try {
  // Public render: prepared.public.kubernetesEnv references a Secret by name/key.
  // Private operation: submit this document via stdin, without logging or saving it.
  await prepared.material.withKubernetesSecret(namespace, async secret => {
    await submitPrivateSecretThroughStdin(secret);
  });
} finally {
  prepared.material.dispose();
}

The Kubernetes document has immutable: true, scoped annotations and base64 data. Its contents are private material despite base64 encoding. It must never be included in rendered, release JSON, evidence, logs, source files, build artifacts or shell arguments. Apply it through the deployment process's protected stdin/HTTP path, then render the application using prepared.public.kubernetesEnv. Reconcile the existing immutable Secret's ownership and identity before reusing it. Namespace, cluster context and deployment credentials remain operator-controlled.

Compose rendering uses prepared.public.composeEnvironment: application keys map to ${WF_DEPLOY_APP_SECRET_n:?application-secret-required} placeholders. Inside material.withComposeEnvironment(callback), pass the private placeholder environment to the isolated Docker Compose child process only. Do not write an .env file, substitute plaintext into the saved Compose document, put values in argv or run a render command whose output exposes expanded values. Preserve only the explicitly needed operator Docker environment along with the placeholder values. An application secret must not become an inherited deployer-control environment variable.

withValues(callback) is available for a deployment adapter with another private in-memory API. Callbacks must not return or retain material, and consumers must never log their inputs. JSON serialization and Node inspection of the material capsule return only a redacted summary. Callback failures are converted to generic errors so stderr or exception messages cannot reveal values. dispose() zeroes retained buffers; JavaScript strings and copies cannot offer guaranteed cryptographic erasure.

Application injection necessarily makes values available to the target application and its trusted container runtime. Kubernetes Secret storage and Docker container configuration can persist them. Those systems need encryption at rest, RBAC, protected daemon access and operator retention controls. “Private in-memory material” describes the WishForge broker/control-plane boundary; it does not claim that an application runtime can use secrets without receiving them.

Validation and acceptance boundary

Unit tests cover exact versions, mapping drift, approval-digest requirements, scope, duplicate keys, reserved process keys, bounded responses, deleted versions, generic errors and public/private serialization. The optional integration test starts a temporary loopback HTTP fixture that implements only the KV v2 read response; it checks request headers/version and redirect refusal, then removes its own temporary files and closes its sockets.

No real Vault, customer account, production secret or Kubernetes/Docker target is accessed by these tests. Actual acceptance still requires a separately provisioned Vault KV v2 mount/read policy and TLS trust, version retention, immutable Kubernetes Secret ownership, Compose private child-environment injection and rollback using the exact retained secret versions. A missing old Vault version blocks rollback instead of substituting a newer secret. Provider BYOK authorization remains a separate system.

Implemented control-plane and adapter wiring

Environment creation accepts only secretVersionRefs and deploymentSecretPolicyDigest; non-empty references require an operator-reviewed sha256:... digest. Existing environments use PATCH /v1/environments/{id}/deployment-secrets with exactly those fields (digest may be null only when clearing references). This endpoint requires human Owner/Admin and refuses changes while a deployment owns the environment. Unknown fields, including secret values, are rejected by the HTTP schema. The console has a reference/digest-only configuration dialog.

Release and approval bindings, rollback compatibility, signed Runner payloads and runtime immutable identity include this policy digest. Changing the digest with unchanged reference names invalidates an old approval. The control plane does not fetch values from Vault to compute or verify the operator's selected digest; the independent deployer compares it against its protected public plan before execution.

ComposeDeploymentAdapter and KubernetesDeploymentAdapter now call plan() before rendering. Dry-run calls neither prepare() nor Vault and executes no commands. A missing broker or mismatched digest returns unconfigured. Standalone render() for non-empty references requires an explicit matching public plan.

Compose saves only placeholder configuration and invokes Compose with a separate child environment containing the placeholder values. It never changes the parent process environment or writes an .env file. The callback returns only the exit code; secret-bearing stdout/stderr and exception details do not enter the receipt. The material capsule is disposed on both success and failure.

Kubernetes uses the fixed context and namespace, sends an immutable Secret only through kubectl stdin, and renders the application with valueFrom.secretKeyRef. Before reusing an existing Secret, it compares namespace, name, immutable/type, managed owner, tenant/environment/release/policy annotations and exact data in the deployment process. Conflicts stop before candidate application. If create returns an error, it queries the named Secret before deciding whether the operation already succeeded. No Secret is force-replaced or automatically deleted. Public deployment manifests and result evidence never contain Secret data. Prepared material is released in finally blocks.

Focused adapter fixtures cover safe dry-run, exact private Compose child injection, exception redaction/disposal, Kubernetes stdin/namespace/ownership checks and policy drift before retrieval. API fixtures cover human-only configuration, approval invalidation, signed payload binding and active-release exclusion. Real production Vault/Kubernetes/retained-version rollback remain deployment acceptance requirements; any local fake-Vault/Compose result must be reported separately from those claims.

The final local Compose/Caddy acceptance run also passed the complete 10-check fixture, including a loopback fake Vault KV v2 version 3 response, the real Compose child environment, and an application endpoint returning only a secretConfigured boolean. Its public report is /private/tmp/wishforge-compose-final-acceptance.json; no secret value is included. The command wrapper preserves the private placeholder environment while fixing the fixture Docker configuration, and teardown uses harmless placeholder values rather than retrieving secrets again. This confirms local Colima injection and blue/green behavior; it remains distinct from real Vault authorization/TLS, Kubernetes Secret storage/RBAC and customer production acceptance.