External etcd

Supported Configurations

Running the control plane as a container with:

Host Nodes
Private Nodes

When using this backing store option, etcd is deployed on the host cluster in the same namespace as the vClustervClusterAn open-source software product that creates and manages virtual Kubernetes clusters inside a host Kubernetes cluster. vCluster improves isolation and multi-tenancy capabilities while reducing infrastructure costs.Related: Virtual Cluster, Host Cluster control plane pod. vCluster deploys etcd with a StatefulSet, Service, and headless Service.

controlPlane:
  backingStore:
    etcd:
      deploy:
        enabled: true

Customize the resources

You can customize the resources that is deployed for etcd. Here are some basic examples, but more options exist in the configuration.

Set resource requests for the StatefulSet

controlPlane:
  backingStore:
    etcd:
      deploy:
        enabled: true
        statefulSet:
          resources:
            requests:
              cpu: 20m
              memory: 150Mi

Add annotations to each etcd component

controlPlane:
  backingStore:
    etcd:
      deploy:
        enabled: true
        statefulSet:
          annotations:
            app.kubernetes.io/part-of: "etcd"
            cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
        service:
          annotations:
            app.kubernetes.io/part-of: "etcd"
        headlessService:
          annotations:
            app.kubernetes.io/part-of: "etcd"

Set the security context of the StatefulSet

controlPlane:
  backingStore:
    etcd:
      deploy:
        enabled: true
        statefulSet:
          security:
            podSecurityContext:
              allowPrivilegeEscalation: false

Migration options

Migrate to embedded etcd

Enterprise-Only Feature

This feature is an Enterprise feature. See our pricing plans or contact our sales team for more information.

You can migrate from deployed etcdetcdA distributed key-value store that provides reliable storage for Kubernetes cluster data. In vCluster, etcd can be deployed externally or embedded within the vCluster pod.Related: Control Plane to embedded etcd without data loss. This migration simplifies your deployment and reduces resource consumption.

Migration guide available

For step-by-step instructions on migrating from deployed to embedded etcd while preserving all your data, see the complete migration guide.

vcluster.yaml
controlPlane:
  backingStore:
    etcd:
      embedded:
        enabled: true
        migrateFromDeployedEtcd: true

Deploying with migrateFromDeployedEtcd: true retains the external etcd StatefulSet to perform the migration. After a successful migration, you should see the Successfully migrated etcd database to embedded etcd log message in vCluster. To remove the external etcd after migration, delete migrateFromDeployedEtcd: true from the vcluster.yaml:

vcluster.yaml after etcd migration
controlPlane:
  backingStore:
    etcd:
      embedded:
        enabled: true

Config reference

`deploy` required object

Deploy defines to use an external etcd that is deployed by the helm chart

`enabled` required boolean false

Enabled defines that an external etcd should be deployed.

`statefulSet` required object

StatefulSet holds options for the external etcd statefulSet.

`enabled` required boolean true

Enabled defines if the statefulSet should be deployed

`enableServiceLinks` required boolean true

EnableServiceLinks for the StatefulSet pod

`image` required object

Image is the image to use for the external etcd statefulSet

`registry` required string registry.k8s.io

Registry is the registry of the container image, e.g. my-registry.com or ghcr.io. This setting can be globally overridden via the controlPlane.advanced.defaultImageRegistry option. Empty means docker hub.

`repository` required string etcd

Repository is the repository of the container image, e.g. my-repo/my-image

`tag` required string 3.6.4-0

Tag is the tag of the container image, and is the default version.

`imagePullPolicy` required string

ImagePullPolicy is the pull policy for the external etcd image

`env` required object[] []

Env are extra environment variables

`extraArgs` required string[] []

ExtraArgs are appended to the etcd command.

`resources` required object

Resources the etcd can consume

`limits` required object

Limits are resource limits for the container

`requests` required object map[cpu:20m memory:150Mi]

Requests are minimal resources that will be consumed by the container

`pods` required object

Pods defines extra metadata for the etcd pods.

`annotations` required object {}

Annotations are extra annotations for this resource.

`labels` required object {}

Labels are extra labels for this resource.

`highAvailability` required object

HighAvailability are high availability options

`replicas` required integer 1

Replicas are the amount of pods to use.

`scheduling` required object

Scheduling options for the etcd pods.

`nodeSelector` required object {}

NodeSelector is the node selector to apply to the pod.

`affinity` required object {}

Affinity is the affinity to apply to the pod.

`tolerations` required object[] []

Tolerations are the tolerations to apply to the pod.

`priorityClassName` required string

PriorityClassName is the priority class name for the the pod.

`podManagementPolicy` required string Parallel

PodManagementPolicy is the statefulSet pod management policy.

`topologySpreadConstraints` required object[] []

TopologySpreadConstraints are the topology spread constraints for the pod.

`security` required object

Security options for the etcd pods.

`podSecurityContext` required object {}

PodSecurityContext specifies security context options on the pod level.

`containerSecurityContext` required object {}

ContainerSecurityContext specifies security context options on the container level.

`persistence` required object

Persistence options for the etcd pods.

`volumeClaim` required object

VolumeClaim can be used to configure the persistent volume claim.

`enabled` required boolean true

Enabled enables deploying a persistent volume claim.

`accessModes` required string[] [ReadWriteOnce]

AccessModes are the persistent volume claim access modes.

`retentionPolicy` required string Retain

RetentionPolicy is the persistent volume claim retention policy.

`size` required string 5Gi

Size is the persistent volume claim storage size.

`storageClass` required string

StorageClass is the persistent volume claim storage class.

`volumeClaimTemplates` required object[] []

VolumeClaimTemplates defines the volumeClaimTemplates for the statefulSet

`addVolumes` required object[] []

AddVolumes defines extra volumes for the pod

`addVolumeMounts` required object[]

AddVolumeMounts defines extra volume mounts for the container

`name` required string

This must match the Name of a Volume.

`readOnly` required boolean

Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.

`mountPath` required string

Path within the container at which the volume should be mounted. Must not contain ':'.

`subPath` required string

Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).

`mountPropagation` required string

mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.

`subPathExpr` required string

Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive.

`annotations` required object {}

Annotations are extra annotations for this resource.

`labels` required object {}

Labels are extra labels for this resource.

`service` required object

Service holds options for the external etcd service.

`enabled` required boolean true

Enabled defines if the etcd service should be deployed

`annotations` required object {}

Annotations are extra annotations for the external etcd service

`headlessService` required object

HeadlessService holds options for the external etcd headless service.

`annotations` required object {}

Annotations are extra annotations for the external etcd headless service

Customize the resources​

Set resource requests for the StatefulSet​

Add annotations to each etcd component​

Set the security context of the StatefulSet​

Migration options​

Migrate to embedded etcd​

Config reference​

deploy required object ​

enabled required boolean false ​

statefulSet required object ​

enabled required boolean true ​

enableServiceLinks required boolean true ​

image required object ​

registry required string registry.k8s.io ​

repository required string etcd ​

tag required string 3.6.4-0 ​

imagePullPolicy required string ​

env required object[] [] ​

extraArgs required string[] [] ​

resources required object ​

limits required object ​

requests required object map[cpu:20m memory:150Mi] ​

pods required object ​

annotations required object {} ​

labels required object {} ​

highAvailability required object ​

replicas required integer 1 ​

scheduling required object ​

nodeSelector required object {} ​

affinity required object {} ​

tolerations required object[] [] ​

priorityClassName required string ​

podManagementPolicy required string Parallel ​

topologySpreadConstraints required object[] [] ​

security required object ​

podSecurityContext required object {} ​

containerSecurityContext required object {} ​

persistence required object ​

volumeClaim required object ​

enabled required boolean true ​

accessModes required string[] [ReadWriteOnce] ​

retentionPolicy required string Retain ​

size required string 5Gi ​

storageClass required string ​

volumeClaimTemplates required object[] [] ​

addVolumes required object[] [] ​

addVolumeMounts required object[] ​

name required string ​

readOnly required boolean ​

mountPath required string ​

subPath required string ​

mountPropagation required string ​

subPathExpr required string ​

annotations required object {} ​

labels required object {} ​

service required object ​

enabled required boolean true ​

annotations required object {} ​

headlessService required object ​

annotations required object {} ​

Customize the resources

Set resource requests for the StatefulSet

Add annotations to each etcd component

Set the security context of the StatefulSet

Migration options

Migrate to embedded etcd

Config reference

`deploy` required object

`enabled` required boolean false

`statefulSet` required object

`enabled` required boolean true

`enableServiceLinks` required boolean true

`image` required object

`registry` required string registry.k8s.io

`repository` required string etcd

`tag` required string 3.6.4-0

`imagePullPolicy` required string

`env` required object[] []

`extraArgs` required string[] []

`resources` required object

`limits` required object

`requests` required object map[cpu:20m memory:150Mi]

`pods` required object

`annotations` required object {}

`labels` required object {}

`highAvailability` required object

`replicas` required integer 1

`scheduling` required object

`nodeSelector` required object {}

`affinity` required object {}

`tolerations` required object[] []

`priorityClassName` required string

`podManagementPolicy` required string Parallel

`topologySpreadConstraints` required object[] []

`security` required object

`podSecurityContext` required object {}

`containerSecurityContext` required object {}

`persistence` required object

`volumeClaim` required object

`enabled` required boolean true

`accessModes` required string[] [ReadWriteOnce]

`retentionPolicy` required string Retain

`size` required string 5Gi

`storageClass` required string

`volumeClaimTemplates` required object[] []

`addVolumes` required object[] []

`addVolumeMounts` required object[]

`name` required string

`readOnly` required boolean

`mountPath` required string

`subPath` required string

`mountPropagation` required string

`subPathExpr` required string

`annotations` required object {}

`labels` required object {}

`service` required object

`enabled` required boolean true

`annotations` required object {}

`headlessService` required object

`annotations` required object {}