Secret rotation

Replacing a value with no downtime and no manual work: env-master changes the password in the target system itself, verifies it, and only then shows it to your services.

Rotation is enabled by a policy on an entry: one entry, one policy. Until a policy exists, the value lives as long as whoever put it there.

Four phases

create → set → test → finish

• create — produce the new value (for postgres — generate a password; the roles from usernames must already exist in the database);
• set — apply it where it is supposed to take effect;
• test — make sure it works;
• finish — move the current label so consumers start reading the new value.

A run always starts from the beginning. A background worker ticks every 30 seconds and takes a 2-minute lease on the policy, so two runs of the same policy never overlap: a manual run takes the very same lease as the worker.

There is no resuming a run from a phase, and none is planned: an interrupted run is never continued — the next run always starts over from create. For postgres resuming would save nothing: its create does not touch the database, it only generates a password, so starting over costs one extra ALTER USER on the same user. A rotator whose create allocates something limited (for example, access keys capped per user) would need resume designed from scratch together with the rotator itself.

What the postgres rotator actually does

The only built-in rotator is postgres. Phase by phase, no generalities:

Three consequences follow, and they save an evening of debugging:

1. The rotator does not create users. You create the roles from usernames yourself and grant them yourself. The rotator only changes their password.
2. The admin DSN must be a URL — postgres://admin:pw@host:5432/db. The test phase substitutes the rotated user and the new password into it, and to do that it parses it as a URL; the host=… user=… format will not work at that phase.
3. The rotated user must connect to the same place as the admin. Host, port, database and parameters for test come from the admin DSN — only the name and the password are replaced. If the role has no CONNECT privilege on that database, the run fails at test even though the password has already been changed.

A superuser is not required: any role that your PostgreSQL version allows to change the password of exactly these users will do — usually CREATEROLE with admin rights over them.

Setup in five steps

A continuation of the quickstart: the BASE, TOKEN, PROJECT and ORG variables come from there.

1. Create the roles in PostgreSQL

CREATE ROLE app_a LOGIN PASSWORD 'temporary-a';
CREATE ROLE app_b LOGIN PASSWORD 'temporary-b';
GRANT CONNECT ON DATABASE billing TO app_a, app_b;
-- then your own grants on schemas and tables, identical for both roles

The passwords here are temporary: the very first run replaces them.

2. Store the admin DSN as a separate entry

curl -s "$BASE/projects/$PROJECT/entries?env=production" \
  -X PUT -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' \
  -d '{"key":"DB_ADMIN_DSN","kind":"secret","visibility":"restricted",
       "value":"postgres://rotator:pw@db.internal:5432/billing"}'

restricted is not over-caution: the rotator reads this entry regardless of visibility, and people never need it on any screen. Keeping an admin DSN masked means showing it to everyone allowed to look at values.

3. Create the entry you are going to rotate

ENTRY=$(curl -s "$BASE/projects/$PROJECT/entries?env=production" \
  -X PUT -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' \
  -d '{"key":"DB_PASSWORD","kind":"secret","visibility":"masked","value":"s3cret"}' \
  | jq -r .id)

The kind must be secret. A variable (var) stores its value in plaintext — a generated password would end up visible to anyone who sees the table.

4. Enable the policy

single-user here is not because dual-credential does not work (it does — see "Strategies"), but because it is simpler for a first run: one role, one value, fewer moving parts.

POLICY=$(curl -s $BASE/entries/$ENTRY/rotation-policy \
  -X PUT -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' \
  -d '{
    "rotator": "postgres",
    "strategy": "single-user",
    "intervalSeconds": 604800,
    "config": {
      "adminDsnEntryRef": "my-org/billing-api/production/DB_ADMIN_DSN",
      "usernames": ["app_a"]
    }
  }' | jq -r .id)

adminDsnEntryRef is a reference org/project/environment/KEY, not a value: the DSN stays an entry with its own permissions and its own access log. The reference must point inside the same organization, and you need at least the member role on its project — otherwise the policy will not save.

5. Verify with a first run instead of waiting for the schedule

curl -s $BASE/rotation-policies/$POLICY/run-now \
  -X POST -H "authorization: Bearer $TOKEN" | jq '.status, .phase, .error'

There is no reason to wait a week to find a typo in the DSN. What the errors of that first run mean is in "When a run fails" below.

Strategies: which one to pick

The difference is only about new connections. PostgreSQL checks the password when a connection is established, so ALTER USER does not drop connections that are already open — in either strategy. The window in which a new connection with the old password fails exists only for single-user, and it lasts from the set phase until your service re-reads the value.

What goes into the entry: a password or a connection string

With dual-credential the active role changes every run, so a bare password is useless to the service: it has no way to learn which name to connect with. Such a policy publishes the whole connection string:

postgres://app_b:pg_xxxxxxxx@db.internal:5432/billing

The name and the password arrive together, in one version of one entry — that is, atomically. Host, port, database and parameters come from the admin DSN: the rotated role connects to the same place as the admin. Name such entries accordingly — DATABASE_URL, not DB_PASSWORD.

With single-user the name never changes, so by default only the password is published. If a ready-made URL suits your service better, the form lets you pick the connection string there too — in the config this is the valueFormat field:

The test phase connects with exactly the value the service will receive: rebuilding it would mean testing something other than what reaches the consumer.

Two identical names in usernames are not rejected when the policy is saved — the run fails at the set phase with cannot determine the inactive username.

What your services see during a run

Until a run reaches finish, services get the previous value: a machine is served the version under the current label, and only finish moves it. An unfinished run in the feed is the normal course, not a breakage.

After finish the new value is not picked up instantly:

• the SDK keeps entries in a cache for 15 minutes (cacheTtlMs), so the usual delay is up to fifteen minutes;
• refresh() drops the cache and re-reads immediately — calling it from the entry.rotated webhook turns rotation into an event for the service instead of a wait;
• a process that read the value once into a variable at startup will never learn about the change. That is an argument not against rotation, but for reading secrets through the SDK.

Details are in the SDK documentation.

The pending label

Between create and finish the new value is already stored under the pending label. This is not an implementation detail: if the process crashes after changing the password in the database but before writing it to env-master, the password is lost — and it is already in effect. The value becomes durable before it starts being applied.

Consumers keep reading current at that point and notice nothing.

If a run fails after create, the new value stays under the pending label — it already exists in the target system and must not be lost. The next run creates its own value and stores it as the next version; the old pending value is never applied. Such a version stays in the entry's history as an ordinary version created by rotation-engine that was never current.

How to read a run

A run has a status and a phase — both are visible in the feed and in the GET /rotation-runs response.

failed almost always means "the value is unchanged": the current label moves at the very end, after a successful test. The single exception is a failure inside the finish phase after the label has moved: the run is then marked failed while services already receive the new value. That is why "rotation failed" in the feed does not read as "the secret did not change" — the phase on the run record says more than the status.

The pending and rolled-back statuses are reserved in the type, but the engine never sets them today.

A run whose process died does not stay running forever: starting the next run of the same policy closes the previous unfinished one as failed with the reason "interrupted and superseded by a newer run". There is deliberately no time threshold — it would guess how long a rotation "should" take, and that is a property of the target system. Until the next run starts, the cabinet feed marks such a run as stalled by age.

When a run fails

The engine only ever surfaces its own messages. A database driver error is replaced with "Rotation failed — see server logs for details": it may contain the entire connection string. The details are in the server log, in the rotation run failed record with policyId and runId.

A failed run raises a rotation.failed event, a webhook, and a metric. The value stays the same — consumers don't notice the failure while the old value is still valid.

Policy

The full request:

curl -s $BASE/entries/$ENTRY/rotation-policy \
  -X PUT -H "authorization: Bearer $TOKEN" -H 'content-type: application/json' \
  -d '{
    "rotator": "postgres",
    "strategy": "dual-credential",
    "intervalSeconds": 604800,
    "config": {
      "adminDsnEntryRef": "my-org/billing-api/production/DB_ADMIN_DSN",
      "usernames": ["app_a", "app_b"]
    }
  }'

• strategy — dual-credential or single-user;
• intervalSeconds — no less than 60;
• adminDsnEntryRef — an org/project/environment/KEY reference to the entry holding the administrator DSN; the rotator reads its value, the DSN itself is not stored in the config;
• enabled — optional, true by default;
• valueFormat — optional: "password" or "dsn". The default is "dsn" for dual-credential with two roles and "password" in every other case;
• activeUsername is added to the config by the rotator itself — do not set it by hand.

PUT replaces config as a whole. When editing an existing policy, read it with GET first and send activeUsername back — otherwise the rotator starts counting from the first name in the list and changes the password of the user your services are running as right now.

To pause rotation without deleting the policy, set enabled: false. To drop it entirely — DELETE /entries/:id/rotation-policy; the value and the version history remain, it just stops changing on a schedule.

Permissions: the project admin configures, pauses and runs; the viewer watches. An organization role by itself grants no access to rotation: it changes the value of a secret, and that is the data plane.

Manual runs

An out-of-schedule run is POST /rotation-policies/:id/run-now; history is GET /rotation-runs?policyId=....

A manual run while a run of the same policy is in progress is rejected with 409 CONFLICT — the tick and run-now share one policy lease. run-now still executes a paused (enabled: false) policy: it is an explicit request, not a schedule. Every manual run is written to the audit log as a rotation.run.requested event with the name of whoever triggered it.

The organization-wide summary is GET /orgs/:id/rotations: the latest runs (up to 5 per policy, 50 in total) across the projects where you have a role; the organization owner and admins see every project. The same feed lives in the cabinet, the "Rotations" section: the state of each run, the phase it failed at, and the error text.

Policies cannot be managed from the cabinet yet — only via the PUT /entries/:id/rotation-policy request shown above.

Rollback

One action (POST /entries/:id/rollback, permission — member): the previous value is published as a new version. History stays monotonic; a repeated rollback moves forward instead of swinging the value back and forth. If there is no previous version — 409 CONFLICT.

A rollback changes the value in env-master, not in the target system: the password in the database stays the one rotation wrote there. Rolling back a rotated entry only makes sense if you restore the password in the database too.

Available rotators

There is exactly one rotator — postgres, with both strategies (dual-credential and single-user). MySQL, Redis and Mongo appear in code comments as examples of extension points — there are no built-in rotators for them: a policy with rotator: "mysql" is rejected with an Unknown rotator error. The engine skeleton is not tied to a specific DBMS; adding a rotator means implementing the four phase methods.