Managing the Sealing Key
Sealed Secrets provide an effective means of managing secret information for your Kubernetes cluster, allowing you to check your secret information into a Version Control System (VCS) like Git. This allows you to manage your secrets via GitOps along with the rest of your cluster manifests.
However, the security of your system and its secrets depends on safely managing the
sealing key. The sealing key is the key used by the
Sealed Secret controller to decrypt your sealed secrets when they are applied to your cluster. As described in Managing Secrets, at cluster creation time the sealing key may be provided by you or automatically generated by
wkp. In either case, it is the one piece of information that must be controlled. Anyone with access to the sealing key can decrypt any sealed secret that was encrypted with the corresponding certificate.
To ensure you can decrypt your sealed secrets in the event of a catastrophic cluster failure, save your sealing key in a secure location such as a password manager or a dedicated secret management system like Hashicorp Vault.
#Key Renewal and Multiple Sealing Keys
By default, the Sealed Secrets controller periodically performs an operation called
key renewal which creates a new sealing key and corresponding certificate (default period: 30 days). After a key renewal, any secrets sealed with
kubeseal will be encrypted with the new certificate unless an older one is explicitly specified on the command line. The controller will then use the new sealing key to decrypt it.
If left alone, this will generate a growing set of sealing keys associated with the controller. The controller will use the appropriate key to decrypt each sealed secret depending on which certificate was used to create it. This means that any backup of sealing keys must take into account the new keys generated via
key renewal. You have three options:
- Turn off key renewal (not recommended)
- Periodically add any new sealing keys to your secret backup storage
- Periodically re-encrypt your existing sealed secrets with the current sealing key (recommended)
The third option is supported via
kubeseal with the
Once you perform this operation for each sealed secret, you are free to label any preexisting sealing key secret as
All the keys may be located via:
A restart of the sealed secret controller will ignore any compromised keys.
As discussed in Managing Secrets, key renewal (automatic or manual) is not a substitute for rotating your actual secrets. In particular, it's important to keep in mind that the security leak caused by a breached sealing key can not be corrected merely by removing and regenerating the sealing key. You need to assume that all secrets encrypted with the sealing key are compromised and rotate your secret information.
#Recoverying sealing keys from the cluster
For option 2 above (or if you did not back up your keys), you can recover your keys as long as your cluster is operational. The following command will extract the keys from your cluster:
You can apply them to a cluster as part of disaster recovery via:
If the controller is already running, restart the controller to install the recovered keys:
#Recoverying secrets after a cluster failure
If your cluster has failed and you have sealed secrets stored in version control or on your filesystem, you can recover your secret information provided you backed up your sealing keys as described above. You can decrypt your sealed secrets using: