annotation meaning in kubernetes

Well-Known Labels, Annotations and Taints

Kubernetes reserves all labels and annotations in the kubernetes.io and k8s.io namespaces.

This document serves both as a reference to the values and as a coordination point for assigning values.

Labels, annotations and taints used on API objects

Apf.kubernetes.io/autoupdate-spec.

Type: Annotation

Example: apf.kubernetes.io/autoupdate-spec: "true"

Used on: FlowSchema and PriorityLevelConfiguration Objects

If this annotation is set to true on a FlowSchema or PriorityLevelConfiguration, the spec for that object is managed by the kube-apiserver. If the API server does not recognize an APF object, and you annotate it for automatic update, the API server deletes the entire object. Otherwise, the API server does not manage the object spec. For more details, read Maintenance of the Mandatory and Suggested Configuration Objects .

app.kubernetes.io/component

Type: Label

Example: app.kubernetes.io/component: "database"

Used on: All Objects (typically used on workload resources ).

The component within the application architecture.

One of the recommended labels .

app.kubernetes.io/created-by (deprecated)

Example: app.kubernetes.io/created-by: "controller-manager"

The controller/user who created this resource.

app.kubernetes.io/instance

Example: app.kubernetes.io/instance: "mysql-abcxzy"

A unique name identifying the instance of an application. To assign a non-unique name, use app.kubernetes.io/name .

app.kubernetes.io/managed-by

Example: app.kubernetes.io/managed-by: "helm"

The tool being used to manage the operation of an application.

app.kubernetes.io/name

Example: app.kubernetes.io/name: "mysql"

The name of the application.

app.kubernetes.io/part-of

Example: app.kubernetes.io/part-of: "wordpress"

The name of a higher-level application this object is part of.

app.kubernetes.io/version

Example: app.kubernetes.io/version: "5.7.21"

The current version of the application.

Common forms of values include:

• semantic version
• the Git revision hash for the source code.

applyset.kubernetes.io/additional-namespaces (alpha)

Example: applyset.kubernetes.io/additional-namespaces: "namespace1,namespace2"

Used on: Objects being used as ApplySet parents.

Use of this annotation is Alpha. For Kubernetes version 1.28, you can use this annotation on Secrets, ConfigMaps, or custom resources if the CustomResourceDefinition defining them has the applyset.kubernetes.io/is-parent-type label.

Part of the specification used to implement ApplySet-based pruning in kubectl . This annotation is applied to the parent object used to track an ApplySet to extend the scope of the ApplySet beyond the parent object's own namespace (if any). The value is a comma-separated list of the names of namespaces other than the parent's namespace in which objects are found.

applyset.kubernetes.io/contains-group-resources (alpha)

Example: applyset.kubernetes.io/contains-group-resources: "certificates.cert-manager.io,configmaps,deployments.apps,secrets,services"

Part of the specification used to implement ApplySet-based pruning in kubectl . This annotation is applied to the parent object used to track an ApplySet to optimize listing of ApplySet member objects. It is optional in the ApplySet specification, as tools can perform discovery or use a different optimization. However, as of Kubernetes version 1.28, it is required by kubectl. When present, the value of this annotation must be a comma separated list of the group-kinds, in the fully-qualified name format, i.e. <resource>.<group> .

applyset.kubernetes.io/id (alpha)

Example: applyset.kubernetes.io/id: "applyset-0eFHV8ySqp7XoShsGvyWFQD3s96yqwHmzc4e0HR1dsY-v1"

Use of this label is Alpha. For Kubernetes version 1.28, you can use this label on Secrets, ConfigMaps, or custom resources if the CustomResourceDefinition defining them has the applyset.kubernetes.io/is-parent-type label.

Part of the specification used to implement ApplySet-based pruning in kubectl . This label is what makes an object an ApplySet parent object. Its value is the unique ID of the ApplySet, which is derived from the identity of the parent object itself. This ID must be the base64 encoding (using the URL safe encoding of RFC4648) of the hash of the group-kind-name-namespace of the object it is on, in the form: <base64(sha256(<name>.<namespace>.<kind>.<group>))> . There is no relation between the value of this label and object UID.

applyset.kubernetes.io/is-parent-type (alpha)

Example: applyset.kubernetes.io/is-parent-type: "true"

Used on: Custom Resource Definition (CRD)

Use of this label is Alpha. Part of the specification used to implement ApplySet-based pruning in kubectl . You can set this label on a CustomResourceDefinition (CRD) to identify the custom resource type it defines (not the CRD itself) as an allowed parent for an ApplySet. The only permitted value for this label is "true" ; if you want to mark a CRD as not being a valid parent for ApplySets, omit this label.

applyset.kubernetes.io/part-of (alpha)

Example: applyset.kubernetes.io/part-of: "applyset-0eFHV8ySqp7XoShsGvyWFQD3s96yqwHmzc4e0HR1dsY-v1"

Used on: All objects.

Use of this label is Alpha. Part of the specification used to implement ApplySet-based pruning in kubectl . This label is what makes an object a member of an ApplySet. The value of the label must match the value of the applyset.kubernetes.io/id label on the parent object.

applyset.kubernetes.io/tooling (alpha)

Example: applyset.kubernetes.io/tooling: "kubectl/v1.28"

Use of this annotation is Alpha. For Kubernetes version 1.28, you can use this annotation on Secrets, ConfigMaps, or custom resources if the CustomResourceDefinitiondefining them has the applyset.kubernetes.io/is-parent-type label.

Part of the specification used to implement ApplySet-based pruning in kubectl . This annotation is applied to the parent object used to track an ApplySet to indicate which tooling manages that ApplySet. Tooling should refuse to mutate ApplySets belonging to other tools. The value must be in the format <toolname>/<semver> .

apps.kubernetes.io/pod-index (beta)

Example: apps.kubernetes.io/pod-index: "0"

Used on: Pod

When a StatefulSet controller creates a Pod for the StatefulSet, it sets this label on that Pod. The value of the label is the ordinal index of the pod being created.

See Pod Index Label in the StatefulSet topic for more details. Note the PodIndexLabel feature gate must be enabled for this label to be added to pods.

cluster-autoscaler.kubernetes.io/safe-to-evict

Example: cluster-autoscaler.kubernetes.io/safe-to-evict: "true"

When this annotation is set to "true" , the cluster autoscaler is allowed to evict a Pod even if other rules would normally prevent that. The cluster autoscaler never evicts Pods that have this annotation explicitly set to "false" ; you could set that on an important Pod that you want to keep running. If this annotation is not set then the cluster autoscaler follows its Pod-level behavior.

config.kubernetes.io/local-config

Example: config.kubernetes.io/local-config: "true"

Used on: All objects

This annotation is used in manifests to mark an object as local configuration that should not be submitted to the Kubernetes API.

A value of "true" for this annotation declares that the object is only consumed by client-side tooling and should not be submitted to the API server.

A value of "false" can be used to declare that the object should be submitted to the API server even when it would otherwise be assumed to be local.

This annotation is part of the Kubernetes Resource Model (KRM) Functions Specification, which is used by Kustomize and similar third-party tools. For example, Kustomize removes objects with this annotation from its final build output.

internal.config.kubernetes.io/* (reserved prefix)

This prefix is reserved for internal use by tools that act as orchestrators in accordance with the Kubernetes Resource Model (KRM) Functions Specification. Annotations with this prefix are internal to the orchestration process and are not persisted to the manifests on the filesystem. In other words, the orchestrator tool should set these annotations when reading files from the local filesystem and remove them when writing the output of functions back to the filesystem.

A KRM function must not modify annotations with this prefix, unless otherwise specified for a given annotation. This enables orchestrator tools to add additional internal annotations, without requiring changes to existing functions.

internal.config.kubernetes.io/path

Example: internal.config.kubernetes.io/path: "relative/file/path.yaml"

This annotation records the slash-delimited, OS-agnostic, relative path to the manifest file the object was loaded from. The path is relative to a fixed location on the filesystem, determined by the orchestrator tool.

This annotation is part of the Kubernetes Resource Model (KRM) Functions Specification, which is used by Kustomize and similar third-party tools.

A KRM Function should not modify this annotation on input objects unless it is modifying the referenced files. A KRM Function may include this annotation on objects it generates.

internal.config.kubernetes.io/index

Example: internal.config.kubernetes.io/index: "2"

This annotation records the zero-indexed position of the YAML document that contains the object within the manifest file the object was loaded from. Note that YAML documents are separated by three dashes ( --- ) and can each contain one object. When this annotation is not specified, a value of 0 is implied.

kubernetes.io/arch

Example: kubernetes.io/arch: "amd64"

Used on: Node

The Kubelet populates this with runtime.GOARCH as defined by Go. This can be handy if you are mixing ARM and x86 nodes.

kubernetes.io/os

Example: kubernetes.io/os: "linux"

Used on: Node, Pod

For nodes, the kubelet populates this with runtime.GOOS as defined by Go. This can be handy if you are mixing operating systems in your cluster (for example: mixing Linux and Windows nodes).

You can also set this label on a Pod. Kubernetes allows you to set any value for this label; if you use this label, you should nevertheless set it to the Go runtime.GOOS string for the operating system that this Pod actually works with.

When the kubernetes.io/os label value for a Pod does not match the label value on a Node, the kubelet on the node will not admit the Pod. However, this is not taken into account by the kube-scheduler. Alternatively, the kubelet refuses to run a Pod where you have specified a Pod OS, if this isn't the same as the operating system for the node where that kubelet is running. Just look for Pods OS for more details.

kubernetes.io/metadata.name

Example: kubernetes.io/metadata.name: "mynamespace"

Used on: Namespaces

The Kubernetes API server (part of the control plane ) sets this label on all namespaces. The label value is set to the name of the namespace. You can't change this label's value.

This is useful if you want to target a specific namespace with a label selector .

kubernetes.io/limit-ranger

Example: kubernetes.io/limit-ranger: "LimitRanger plugin set: cpu, memory request for container nginx; cpu, memory limit for container nginx"

Kubernetes by default doesn't provide any resource limit, that means unless you explicitly define limits, your container can consume unlimited CPU and memory. You can define a default request or default limit for pods. You do this by creating a LimitRange in the relevant namespace. Pods deployed after you define a LimitRange will have these limits applied to them. The annotation kubernetes.io/limit-ranger records that resource defaults were specified for the Pod, and they were applied successfully. For more details, read about LimitRanges .

addonmanager.kubernetes.io/mode

Example: addonmanager.kubernetes.io/mode: "Reconcile"

To specify how an add-on should be managed, you can use the addonmanager.kubernetes.io/mode label. This label can have one of three values: Reconcile , EnsureExists , or Ignore .

• Reconcile : Addon resources will be periodically reconciled with the expected state. If there are any differences, the add-on manager will recreate, reconfigure or delete the resources as needed. This is the default mode if no label is specified.
• EnsureExists : Addon resources will be checked for existence only but will not be modified after creation. The add-on manager will create or re-create the resources when there is no instance of the resource with that name.
• Ignore : Addon resources will be ignored. This mode is useful for add-ons that are not compatible with the add-on manager or that are managed by another controller.

For more details, see Addon-manager .

beta.kubernetes.io/arch (deprecated)

This label has been deprecated. Please use kubernetes.io/arch instead.

beta.kubernetes.io/os (deprecated)

This label has been deprecated. Please use kubernetes.io/os instead.

kube-aggregator.kubernetes.io/automanaged

Example: kube-aggregator.kubernetes.io/automanaged: "onstart"

Used on: APIService

The kube-apiserver sets this label on any APIService object that the API server has created automatically. The label marks how the control plane should manage that APIService. You should not add, modify, or remove this label by yourself.

There are two possible values:

• onstart : The APIService should be reconciled when an API server starts up, but not otherwise.
• true : The API server should reconcile this APIService continuously.

service.alpha.kubernetes.io/tolerate-unready-endpoints (deprecated)

Used on: StatefulSet

This annotation on a Service denotes if the Endpoints controller should go ahead and create Endpoints for unready Pods. Endpoints of these Services retain their DNS records and continue receiving traffic for the Service from the moment the kubelet starts all containers in the pod and marks it Running , til the kubelet stops all containers and deletes the pod from the API server.

kubernetes.io/hostname

Example: kubernetes.io/hostname: "ip-172-20-114-199.ec2.internal"

The Kubelet populates this label with the hostname of the node. Note that the hostname can be changed from the "actual" hostname by passing the --hostname-override flag to the kubelet .

This label is also used as part of the topology hierarchy. See topology.kubernetes.io/zone for more information.

kubernetes.io/change-cause

Example: kubernetes.io/change-cause: "kubectl edit --record deployment foo"

Used on: All Objects

This annotation is a best guess at why something was changed.

It is populated when adding --record to a kubectl command that may change an object.

kubernetes.io/description

Example: kubernetes.io/description: "Description of K8s object."

This annotation is used for describing specific behaviour of given object.

kubernetes.io/enforce-mountable-secrets

Example: kubernetes.io/enforce-mountable-secrets: "true"

Used on: ServiceAccount

The value for this annotation must be true to take effect. This annotation indicates that Pods running as this ServiceAccount may only reference Secret API objects specified in the ServiceAccount's secrets field.

node.kubernetes.io/exclude-from-external-load-balancers

Example: node.kubernetes.io/exclude-from-external-load-balancers

Kubernetes automatically enables the ServiceNodeExclusion feature gate on the clusters it creates. With this feature gate enabled on a cluster, you can add labels to particular worker nodes to exclude them from the list of backend servers. The following command can be used to exclude a worker node from the list of backend servers in a backend set:

controller.kubernetes.io/pod-deletion-cost

Example: controller.kubernetes.io/pod-deletion-cost: "10"

This annotation is used to set Pod Deletion Cost which allows users to influence ReplicaSet downscaling order. The annotation value parses into an int32 type.

cluster-autoscaler.kubernetes.io/enable-ds-eviction

Example: cluster-autoscaler.kubernetes.io/enable-ds-eviction: "true"

This annotation controls whether a DaemonSet pod should be evicted by a ClusterAutoscaler. This annotation needs to be specified on DaemonSet pods in a DaemonSet manifest. When this annotation is set to "true" , the ClusterAutoscaler is allowed to evict a DaemonSet Pod, even if other rules would normally prevent that. To disallow the ClusterAutoscaler from evicting DaemonSet pods, you can set this annotation to "false" for important DaemonSet pods. If this annotation is not set, then the ClusterAutoscaler follows its overall behavior (i.e evict the DaemonSets based on its configuration).

kubernetes.io/ingress-bandwidth

Example: kubernetes.io/ingress-bandwidth: 10M

You can apply quality-of-service traffic shaping to a pod and effectively limit its available bandwidth. Ingress traffic to a Pod is handled by shaping queued packets to effectively handle data. To limit the bandwidth on a Pod, write an object definition JSON file and specify the data traffic speed using kubernetes.io/ingress-bandwidth annotation. The unit used for specifying ingress rate is bits per second, as a Quantity . For example, 10M means 10 megabits per second.

kubernetes.io/egress-bandwidth

Example: kubernetes.io/egress-bandwidth: 10M

Egress traffic from a Pod is handled by policing, which simply drops packets in excess of the configured rate. The limits you place on a Pod do not affect the bandwidth of other Pods. To limit the bandwidth on a Pod, write an object definition JSON file and specify the data traffic speed using kubernetes.io/egress-bandwidth annotation. The unit used for specifying egress rate is bits per second, as a Quantity . For example, 10M means 10 megabits per second.

beta.kubernetes.io/instance-type (deprecated)

Node.kubernetes.io/instance-type.

Example: node.kubernetes.io/instance-type: "m3.medium"

The Kubelet populates this with the instance type as defined by the cloud provider. This will be set only if you are using a cloud provider. This setting is handy if you want to target certain workloads to certain instance types, but typically you want to rely on the Kubernetes scheduler to perform resource-based scheduling. You should aim to schedule based on properties rather than on instance types (for example: require a GPU, instead of requiring a g2.2xlarge ).

failure-domain.beta.kubernetes.io/region (deprecated)

Failure-domain.beta.kubernetes.io/zone (deprecated), pv.kubernetes.io/bind-completed.

Example: pv.kubernetes.io/bind-completed: "yes"

Used on: PersistentVolumeClaim

When this annotation is set on a PersistentVolumeClaim (PVC), that indicates that the lifecycle of the PVC has passed through initial binding setup. When present, that information changes how the control plane interprets the state of PVC objects. The value of this annotation does not matter to Kubernetes.

pv.kubernetes.io/bound-by-controller

Example: pv.kubernetes.io/bound-by-controller: "yes"

Used on: PersistentVolume, PersistentVolumeClaim

If this annotation is set on a PersistentVolume or PersistentVolumeClaim, it indicates that a storage binding (PersistentVolume → PersistentVolumeClaim, or PersistentVolumeClaim → PersistentVolume) was installed by the controller . If the annotation isn't set, and there is a storage binding in place, the absence of that annotation means that the binding was done manually. The value of this annotation does not matter.

pv.kubernetes.io/provisioned-by

Example: pv.kubernetes.io/provisioned-by: "kubernetes.io/rbd"

Used on: PersistentVolume

This annotation is added to a PersistentVolume(PV) that has been dynamically provisioned by Kubernetes. Its value is the name of volume plugin that created the volume. It serves both users (to show where a PV comes from) and Kubernetes (to recognize dynamically provisioned PVs in its decisions).

pv.kubernetes.io/migrated-to

Example: pv.kubernetes.io/migrated-to: pd.csi.storage.gke.io

It is added to a PersistentVolume(PV) and PersistentVolumeClaim(PVC) that is supposed to be dynamically provisioned/deleted by its corresponding CSI driver through the CSIMigration feature gate. When this annotation is set, the Kubernetes components will "stand-down" and the external-provisioner will act on the objects.

statefulset.kubernetes.io/pod-name

Example: statefulset.kubernetes.io/pod-name: "mystatefulset-7"

When a StatefulSet controller creates a Pod for the StatefulSet, the control plane sets this label on that Pod. The value of the label is the name of the Pod being created.

See Pod Name Label in the StatefulSet topic for more details.

scheduler.alpha.kubernetes.io/node-selector

Example: scheduler.alpha.kubernetes.io/node-selector: "name-of-node-selector"

Used on: Namespace

The PodNodeSelector uses this annotation key to assign node selectors to pods in namespaces.

topology.kubernetes.io/region

Example: topology.kubernetes.io/region: "us-east-1"

Used on: Node, PersistentVolume

See topology.kubernetes.io/zone .

topology.kubernetes.io/zone

Example: topology.kubernetes.io/zone: "us-east-1c"

On Node : The kubelet or the external cloud-controller-manager populates this with the information from the cloud provider. This will be set only if you are using a cloud provider. However, you can consider setting this on nodes if it makes sense in your topology.

On PersistentVolume : topology-aware volume provisioners will automatically set node affinity constraints on a PersistentVolume .

A zone represents a logical failure domain. It is common for Kubernetes clusters to span multiple zones for increased availability. While the exact definition of a zone is left to infrastructure implementations, common properties of a zone include very low network latency within a zone, no-cost network traffic within a zone, and failure independence from other zones. For example, nodes within a zone might share a network switch, but nodes in different zones should not.

A region represents a larger domain, made up of one or more zones. It is uncommon for Kubernetes clusters to span multiple regions, While the exact definition of a zone or region is left to infrastructure implementations, common properties of a region include higher network latency between them than within them, non-zero cost for network traffic between them, and failure independence from other zones or regions. For example, nodes within a region might share power infrastructure (e.g. a UPS or generator), but nodes in different regions typically would not.

Kubernetes makes a few assumptions about the structure of zones and regions:

• regions and zones are hierarchical: zones are strict subsets of regions and no zone can be in 2 regions
• zone names are unique across regions; for example region "africa-east-1" might be comprised of zones "africa-east-1a" and "africa-east-1b"

It should be safe to assume that topology labels do not change. Even though labels are strictly mutable, consumers of them can assume that a given node is not going to be moved between zones without being destroyed and recreated.

Kubernetes can use this information in various ways. For example, the scheduler automatically tries to spread the Pods in a ReplicaSet across nodes in a single-zone cluster (to reduce the impact of node failures, see kubernetes.io/hostname ). With multiple-zone clusters, this spreading behavior also applies to zones (to reduce the impact of zone failures). This is achieved via SelectorSpreadPriority .

SelectorSpreadPriority is a best effort placement. If the zones in your cluster are heterogeneous (for example: different numbers of nodes, different types of nodes, or different pod resource requirements), this placement might prevent equal spreading of your Pods across zones. If desired, you can use homogenous zones (same number and types of nodes) to reduce the probability of unequal spreading.

The scheduler (through the VolumeZonePredicate predicate) also will ensure that Pods, that claim a given volume, are only placed into the same zone as that volume. Volumes cannot be attached across zones.

If PersistentVolumeLabel does not support automatic labeling of your PersistentVolumes, you should consider adding the labels manually (or adding support for PersistentVolumeLabel ). With PersistentVolumeLabel , the scheduler prevents Pods from mounting volumes in a different zone. If your infrastructure doesn't have this constraint, you don't need to add the zone labels to the volumes at all.

volume.beta.kubernetes.io/storage-provisioner (deprecated)

Example: volume.beta.kubernetes.io/storage-provisioner: "k8s.io/minikube-hostpath"

This annotation has been deprecated since v1.23. See volume.kubernetes.io/storage-provisioner .

volume.beta.kubernetes.io/storage-class (deprecated)

Example: volume.beta.kubernetes.io/storage-class: "example-class"

This annotation can be used for PersistentVolume(PV) or PersistentVolumeClaim(PVC) to specify the name of StorageClass . When both the storageClassName attribute and the volume.beta.kubernetes.io/storage-class annotation are specified, the annotation volume.beta.kubernetes.io/storage-class takes precedence over the storageClassName attribute.

This annotation has been deprecated. Instead, set the storageClassName field for the PersistentVolumeClaim or PersistentVolume.

volume.beta.kubernetes.io/mount-options (deprecated)

Example : volume.beta.kubernetes.io/mount-options: "ro,soft"

A Kubernetes administrator can specify additional mount options for when a PersistentVolume is mounted on a node.

volume.kubernetes.io/storage-provisioner

This annotation is added to a PVC that is supposed to be dynamically provisioned. Its value is the name of a volume plugin that is supposed to provision a volume for this PVC.

volume.kubernetes.io/selected-node

This annotation is added to a PVC that is triggered by a scheduler to be dynamically provisioned. Its value is the name of the selected node.

volumes.kubernetes.io/controller-managed-attach-detach

If a node has the annotation volumes.kubernetes.io/controller-managed-attach-detach , its storage attach and detach operations are being managed by the volume attach/detach controller .

The value of the annotation isn't important.

node.kubernetes.io/windows-build

Example: node.kubernetes.io/windows-build: "10.0.17763"

When the kubelet is running on Microsoft Windows, it automatically labels its Node to record the version of Windows Server in use.

The label's value is in the format "MajorVersion.MinorVersion.BuildNumber".

service.kubernetes.io/headless

Example: service.kubernetes.io/headless: ""

Used on: Service

The control plane adds this label to an Endpoints object when the owning Service is headless.

service.kubernetes.io/topology-aware-hints (deprecated)

Example: service.kubernetes.io/topology-aware-hints: "Auto"

This annotation was used for enabling topology aware hints on Services. Topology aware hints have since been renamed: the concept is now called topology aware routing . Setting the annotation to Auto , on a Service, configured the Kubernetes control plane to add topology hints on EndpointSlices associated with that Service. You can also explicitly set the annotation to Disabled .

If you are running a version of Kubernetes older than 1.28, check the documentation for that Kubernetes version to see how topology aware routing works in that release.

There are no other valid values for this annotation. If you don't want topology aware hints for a Service, don't add this annotation.

kubernetes.io/service-name

Example: kubernetes.io/service-name: "my-website"

Used on: EndpointSlice

Kubernetes associates EndpointSlices with Services using this label.

This label records the name of the Service that the EndpointSlice is backing. All EndpointSlices should have this label set to the name of their associated Service.

kubernetes.io/service-account.name

Example: kubernetes.io/service-account.name: "sa-name"

Used on: Secret

This annotation records the name of the ServiceAccount that the token (stored in the Secret of type kubernetes.io/service-account-token ) represents.

kubernetes.io/service-account.uid

Example: kubernetes.io/service-account.uid: da68f9c6-9d26-11e7-b84e-002dc52800da

This annotation records the unique ID of the ServiceAccount that the token (stored in the Secret of type kubernetes.io/service-account-token ) represents.

kubernetes.io/legacy-token-last-used

Example: kubernetes.io/legacy-token-last-used: 2022-10-24

The control plane only adds this label to Secrets that have the type kubernetes.io/service-account-token . The value of this label records the date (ISO 8601 format, UTC time zone) when the control plane last saw a request where the client authenticated using the service account token.

If a legacy token was last used before the cluster gained the feature (added in Kubernetes v1.26), then the label isn't set.

endpointslice.kubernetes.io/managed-by

Example: endpointslice.kubernetes.io/managed-by: "controller"

Used on: EndpointSlices

The label is used to indicate the controller or entity that manages the EndpointSlice. This label aims to enable different EndpointSlice objects to be managed by different controllers or entities within the same cluster.

endpointslice.kubernetes.io/skip-mirror

Example: endpointslice.kubernetes.io/skip-mirror: "true"

Used on: Endpoints

The label can be set to "true" on an Endpoints resource to indicate that the EndpointSliceMirroring controller should not mirror this resource with EndpointSlices.

service.kubernetes.io/service-proxy-name

Example: service.kubernetes.io/service-proxy-name: "foo-bar"

The kube-proxy has this label for custom proxy, which delegates service control to custom proxy.

experimental.windows.kubernetes.io/isolation-type (deprecated)

Example: experimental.windows.kubernetes.io/isolation-type: "hyperv"

The annotation is used to run Windows containers with Hyper-V isolation.

ingressclass.kubernetes.io/is-default-class

Example: ingressclass.kubernetes.io/is-default-class: "true"

Used on: IngressClass

When a IngressClass resource has this annotation set to "true" , new Ingress resource without a class specified will be assigned this default class.

kubernetes.io/ingress.class (deprecated)

Used on: Ingress

storageclass.kubernetes.io/is-default-class

Example: storageclass.kubernetes.io/is-default-class: "true"

Used on: StorageClass

When a single StorageClass resource has this annotation set to "true" , new PersistentVolumeClaim resource without a class specified will be assigned this default class.

alpha.kubernetes.io/provided-node-ip (alpha)

Example: alpha.kubernetes.io/provided-node-ip: "10.0.0.1"

The kubelet can set this annotation on a Node to denote its configured IPv4 address.

When kubelet is started with the --cloud-provider flag set to any value (includes both external and legacy in-tree cloud providers), it sets this annotation on the Node to denote an IP address set from the command line flag ( --node-ip ). This IP is verified with the cloud provider as valid by the cloud-controller-manager.

batch.kubernetes.io/job-completion-index

Type: Annotation, Label

Example: batch.kubernetes.io/job-completion-index: "3"

The Job controller in the kube-controller-manager sets this as a label and annotation for Pods created with Indexed completion mode .

Note the PodIndexLabel feature gate must be enabled for this to be added as a pod label , otherwise it will just be an annotation.

batch.kubernetes.io/cronjob-scheduled-timestamp

Example: batch.kubernetes.io/cronjob-scheduled-timestamp: "2016-05-19T03:00:00-07:00"

Used on: Jobs and Pods controlled by CronJobs

This annotation is used to record the original (expected) creation timestamp for a Job, when that Job is part of a CronJob. The control plane sets the value to that timestamp in RFC3339 format. If the Job belongs to a CronJob with a timezone specified, then the timestamp is in that timezone. Otherwise, the timestamp is in controller-manager's local time.

kubectl.kubernetes.io/default-container

Example: kubectl.kubernetes.io/default-container: "front-end-app"

The value of the annotation is the container name that is default for this Pod. For example, kubectl logs or kubectl exec without -c or --container flag will use this default container.

kubectl.kubernetes.io/default-logs-container (deprecated)

Example: kubectl.kubernetes.io/default-logs-container: "front-end-app"

The value of the annotation is the container name that is the default logging container for this Pod. For example, kubectl logs without -c or --container flag will use this default container.

kubectl.kubernetes.io/last-applied-configuration

Example: see following snippet

Used on: all objects

The kubectl command line tool uses this annotation as a legacy mechanism to track changes. That mechanism has been superseded by Server-side apply .

endpoints.kubernetes.io/over-capacity

Example: endpoints.kubernetes.io/over-capacity:truncated

The control plane adds this annotation to an Endpoints object if the associated Service has more than 1000 backing endpoints. The annotation indicates that the Endpoints object is over capacity and the number of endpoints has been truncated to 1000.

If the number of backend endpoints falls below 1000, the control plane removes this annotation.

batch.kubernetes.io/job-tracking (deprecated)

Example: batch.kubernetes.io/job-tracking: ""

Used on: Jobs

The presence of this annotation on a Job used to indicate that the control plane is tracking the Job status using finalizers . Adding or removing this annotation no longer has an effect (Kubernetes v1.27 and later) All Jobs are tracked with finalizers.

job-name (deprecated)

Example: job-name: "pi"

Used on: Jobs and Pods controlled by Jobs

controller-uid (deprecated)

Example: controller-uid: "$UID"

batch.kubernetes.io/job-name

Example: batch.kubernetes.io/job-name: "pi"

This label is used as a user-friendly way to get Pods corresponding to a Job. The job-name comes from the name of the Job and allows for an easy way to get Pods corresponding to the Job.

batch.kubernetes.io/controller-uid

Example: batch.kubernetes.io/controller-uid: "$UID"

This label is used as a programmatic way to get all Pods corresponding to a Job. The controller-uid is a unique identifer that gets set in the selector field so the Job controller can get all the corresponding Pods.

scheduler.alpha.kubernetes.io/defaultTolerations

Example: scheduler.alpha.kubernetes.io/defaultTolerations: '[{"operator": "Equal", "value": "value1", "effect": "NoSchedule", "key": "dedicated-node"}]'

This annotation requires the PodTolerationRestriction admission controller to be enabled. This annotation key allows assigning tolerations to a namespace and any new pods created in this namespace would get these tolerations added.

scheduler.alpha.kubernetes.io/tolerationsWhitelist

Example: scheduler.alpha.kubernetes.io/tolerationsWhitelist: '[{"operator": "Exists", "effect": "NoSchedule", "key": "dedicated-node"}]'

This annotation is only useful when the (Alpha) PodTolerationRestriction admission controller is enabled. The annotation value is a JSON document that defines a list of allowed tolerations for the namespace it annotates. When you create a Pod or modify its tolerations, the API server checks the tolerations to see if they are mentioned in the allow list. The pod is admitted only if the check succeeds.

scheduler.alpha.kubernetes.io/preferAvoidPods (deprecated)

This annotation requires the NodePreferAvoidPods scheduling plugin to be enabled. The plugin is deprecated since Kubernetes 1.22. Use Taints and Tolerations instead.

node.kubernetes.io/not-ready

Type: Taint

Example: node.kubernetes.io/not-ready: "NoExecute"

The Node controller detects whether a Node is ready by monitoring its health and adds or removes this taint accordingly.

node.kubernetes.io/unreachable

Example: node.kubernetes.io/unreachable: "NoExecute"

The Node controller adds the taint to a Node corresponding to the NodeCondition Ready being Unknown .

node.kubernetes.io/unschedulable

Example: node.kubernetes.io/unschedulable: "NoSchedule"

The taint will be added to a node when initializing the node to avoid race condition.

node.kubernetes.io/memory-pressure

Example: node.kubernetes.io/memory-pressure: "NoSchedule"

The kubelet detects memory pressure based on memory.available and allocatableMemory.available observed on a Node. The observed values are then compared to the corresponding thresholds that can be set on the kubelet to determine if the Node condition and taint should be added/removed.

node.kubernetes.io/disk-pressure

Example: node.kubernetes.io/disk-pressure :"NoSchedule"

The kubelet detects disk pressure based on imagefs.available , imagefs.inodesFree , nodefs.available and nodefs.inodesFree (Linux only) observed on a Node. The observed values are then compared to the corresponding thresholds that can be set on the kubelet to determine if the Node condition and taint should be added/removed.

node.kubernetes.io/network-unavailable

Example: node.kubernetes.io/network-unavailable: "NoSchedule"

This is initially set by the kubelet when the cloud provider used indicates a requirement for additional network configuration. Only when the route on the cloud is configured properly will the taint be removed by the cloud provider.

node.kubernetes.io/pid-pressure

Example: node.kubernetes.io/pid-pressure: "NoSchedule"

The kubelet checks D-value of the size of /proc/sys/kernel/pid_max and the PIDs consumed by Kubernetes on a node to get the number of available PIDs that referred to as the pid.available metric. The metric is then compared to the corresponding threshold that can be set on the kubelet to determine if the node condition and taint should be added/removed.

node.kubernetes.io/out-of-service

Example: node.kubernetes.io/out-of-service:NoExecute

A user can manually add the taint to a Node marking it out-of-service. If the NodeOutOfServiceVolumeDetach feature gate is enabled on kube-controller-manager , and a Node is marked out-of-service with this taint, the Pods on the node will be forcefully deleted if there are no matching tolerations on it and volume detach operations for the Pods terminating on the node will happen immediately. This allows the Pods on the out-of-service node to recover quickly on a different node.

node.cloudprovider.kubernetes.io/uninitialized

Example: node.cloudprovider.kubernetes.io/uninitialized: "NoSchedule"

Sets this taint on a Node to mark it as unusable, when kubelet is started with the "external" cloud provider, until a controller from the cloud-controller-manager initializes this Node, and then removes the taint.

node.cloudprovider.kubernetes.io/shutdown

Example: node.cloudprovider.kubernetes.io/shutdown: "NoSchedule"

If a Node is in a cloud provider specified shutdown state, the Node gets tainted accordingly with node.cloudprovider.kubernetes.io/shutdown and the taint effect of NoSchedule .

feature.node.kubernetes.io/*

Example: feature.node.kubernetes.io/network-sriov.capable: "true"

These labels are used by the Node Feature Discovery (NFD) component to advertise features on a node. All built-in labels use the feature.node.kubernetes.io label namespace and have the format feature.node.kubernetes.io/<feature-name>: "true" . NFD has many extension points for creating vendor and application-specific labels. For details, see the customization guide .

nfd.node.kubernetes.io/master.version

Example: nfd.node.kubernetes.io/master.version: "v0.6.0"

For node(s) where the Node Feature Discovery (NFD) master is scheduled, this annotation records the version of the NFD master. It is used for informative use only.

nfd.node.kubernetes.io/worker.version

Example: nfd.node.kubernetes.io/worker.version: "v0.4.0"

Used on: Nodes

This annotation records the version for a Node Feature Discovery's worker if there is one running on a node. It's used for informative use only.

nfd.node.kubernetes.io/feature-labels

Example: nfd.node.kubernetes.io/feature-labels: "cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-hardware_multithreading,kernel-version.full"

This annotation records a comma-separated list of node feature labels managed by Node Feature Discovery (NFD). NFD uses this for an internal mechanism. You should not edit this annotation yourself.

nfd.node.kubernetes.io/extended-resources

Example: nfd.node.kubernetes.io/extended-resources: "accelerator.acme.example/q500,example.com/coprocessor-fx5"

This annotation records a comma-separated list of extended resources managed by Node Feature Discovery (NFD). NFD uses this for an internal mechanism. You should not edit this annotation yourself.

service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval: "5"

The cloud controller manager integration with AWS elastic load balancing configures the load balancer for a Service based on this annotation. The value determines how often the load balancer writes log entries. For example, if you set the value to 5, the log writes occur 5 seconds apart.

service.beta.kubernetes.io/aws-load-balancer-access-log-enabled (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-access-log-enabled: "false"

The cloud controller manager integration with AWS elastic load balancing configures the load balancer for a Service based on this annotation. Access logging is enabled if you set the annotation to "true".

service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-access-log-enabled: example

The cloud controller manager integration with AWS elastic load balancing configures the load balancer for a Service based on this annotation. The load balancer writes logs to an S3 bucket with the name you specify.

service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-access-log-enabled: "/example"

The cloud controller manager integration with AWS elastic load balancing configures the load balancer for a Service based on this annotation. The load balancer writes log objects with the prefix that you specify.

service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: "Environment=demo,Project=example"

The cloud controller manager integration with AWS elastic load balancing configures tags (an AWS concept) for a load balancer based on the comma-separated key/value pairs in the value of this annotation.

service.beta.kubernetes.io/aws-load-balancer-alpn-policy (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-alpn-policy: HTTP2Optional

The AWS load balancer controller uses this annotation. See annotations in the AWS load balancer controller documentation.

service.beta.kubernetes.io/aws-load-balancer-attributes (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-attributes: "deletion_protection.enabled=true"

service.beta.kubernetes.io/aws-load-balancer-backend-protocol (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp

The cloud controller manager integration with AWS elastic load balancing configures the load balancer listener based on the value of this annotation.

service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled: "false"

The cloud controller manager integration with AWS elastic load balancing configures the load balancer based on this annotation. The load balancer's connection draining setting depends on the value you set.

service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout: "60"

If you configure connection draining for a Service of type: LoadBalancer , and you use the AWS cloud, the integration configures the draining period based on this annotation. The value you set determines the draining timeout in seconds.

service.beta.kubernetes.io/aws-load-balancer-ip-address-type (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-ip-address-type: ipv4

service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. The load balancer has a configured idle timeout period (in seconds) that applies to its connections. If no data has been sent or received by the time that the idle timeout period elapses, the load balancer closes the connection.

service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. If you set this annotation to "true", each load balancer node distributes requests evenly across the registered targets in all enabled availability zones . If you disable cross-zone load balancing, each load balancer node distributes requests evenly across the registered targets in its availability zone only.

service.beta.kubernetes.io/aws-load-balancer-eip-allocations (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-eip-allocations: "eipalloc-01bcdef23bcdef456,eipalloc-def1234abc4567890"

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. The value is a comma-separated list of elastic IP address allocation IDs.

This annotation is only relevant for Services of type: LoadBalancer , where the load balancer is an AWS Network Load Balancer.

service.beta.kubernetes.io/aws-load-balancer-extra-security-groups (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-extra-security-groups: "sg-12abcd3456,sg-34dcba6543"

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. The annotation value is a comma-separated list of extra AWS VPC security groups to configure for the load balancer.

service.beta.kubernetes.io/aws-load-balancer-healthcheck-healthy-threshold (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-healthcheck-healthy-threshold: "3"

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. The annotation value specifies the number of successive successful health checks required for a backend to be considered healthy for traffic.

service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: "30"

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. The annotation value specifies the interval, in seconds, between health check probes made by the load balancer.

service.beta.kubernetes.io/aws-load-balancer-healthcheck-path (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-healthcheck-path: /healthcheck

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. The annotation value determines the path part of the URL that is used for HTTP health checks.

service.beta.kubernetes.io/aws-load-balancer-healthcheck-port (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: "24"

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. The annotation value determines which port the load balancer connects to when performing health checks.

service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: TCP

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. The annotation value determines how the load balancer checks the health of backend targets.

service.beta.kubernetes.io/aws-load-balancer-healthcheck-timeout (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-healthcheck-timeout: "3"

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. The annotation value specifies the number of seconds before a probe that hasn't yet succeeded is automatically treated as having failed.

service.beta.kubernetes.io/aws-load-balancer-healthcheck-unhealthy-threshold (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-healthcheck-unhealthy-threshold: "3"

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. The annotation value specifies the number of successive unsuccessful health checks required for a backend to be considered unhealthy for traffic.

service.beta.kubernetes.io/aws-load-balancer-internal (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-internal: "true"

The cloud controller manager integration with AWS elastic load balancing configures a load balancer based on this annotation. When you set this annotation to "true", the integration configures an internal load balancer.

If you use the AWS load balancer controller , see service.beta.kubernetes.io/aws-load-balancer-scheme .

service.beta.kubernetes.io/aws-load-balancer-manage-backend-security-group-rules (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-manage-backend-security-group-rules: "true"

service.beta.kubernetes.io/aws-load-balancer-name (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-name: my-elb

If you set this annotation on a Service, and you also annotate that Service with service.beta.kubernetes.io/aws-load-balancer-type: "external" , and you use the AWS load balancer controller in your cluster, then the AWS load balancer controller sets the name of that load balancer to the value you set for this annotation.

See annotations in the AWS load balancer controller documentation.

service.beta.kubernetes.io/aws-load-balancer-nlb-target-type (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "true"

service.beta.kubernetes.io/aws-load-balancer-private-ipv4-addresses (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-private-ipv4-addresses: "198.51.100.0,198.51.100.64"

service.beta.kubernetes.io/aws-load-balancer-proxy-protocol (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"

The official Kubernetes integration with AWS elastic load balancing configures a load balancer based on this annotation. The only permitted value is "*" , which indicates that the load balancer should wrap TCP connections to the backend Pod with the PROXY protocol.

service.beta.kubernetes.io/aws-load-balancer-scheme (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-scheme: internal

service.beta.kubernetes.io/load-balancer-source-ranges (deprecated)

Example: service.beta.kubernetes.io/load-balancer-source-ranges: "192.0.2.0/25"

The AWS load balancer controller uses this annotation. You should set .spec.loadBalancerSourceRanges for the Service instead.

service.beta.kubernetes.io/aws-load-balancer-ssl-cert (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012"

The official integration with AWS elastic load balancing configures TLS for a Service of type: LoadBalancer based on this annotation. The value of the annotation is the AWS Resource Name (ARN) of the X.509 certificate that the load balancer listener should use.

(The TLS protocol is based on an older technology that abbreviates to SSL.)

service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-2-2017-01

The official integration with AWS elastic load balancing configures TLS for a Service of type: LoadBalancer based on this annotation. The value of the annotation is the name of an AWS policy for negotiating TLS with a client peer.

service.beta.kubernetes.io/aws-load-balancer-ssl-ports (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "*"

The official integration with AWS elastic load balancing configures TLS for a Service of type: LoadBalancer based on this annotation. The value of the annotation is either "*" , which means that all the load balancer's ports should use TLS, or it is a comma separated list of port numbers.

service.beta.kubernetes.io/aws-load-balancer-subnets (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-subnets: "private-a,private-b"

Kubernetes' official integration with AWS uses this annotation to configure a load balancer and determine in which AWS availability zones to deploy the managed load balancing service. The value is either a comma separated list of subnet names, or a comma separated list of subnet IDs.

service.beta.kubernetes.io/aws-load-balancer-target-group-attributes (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: "stickiness.enabled=true,stickiness.type=source_ip"

service.beta.kubernetes.io/aws-load-balancer-target-node-labels (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-target-node-labels: "kubernetes.io/os=Linux,topology.kubernetes.io/region=us-east-2"

Kubernetes' official integration with AWS uses this annotation to determine which nodes in your cluster should be considered as valid targets for the load balancer.

service.beta.kubernetes.io/aws-load-balancer-type (beta)

Example: service.beta.kubernetes.io/aws-load-balancer-type: external

Kubernetes' official integrations with AWS use this annotation to determine whether the AWS cloud provider integration should manage a Service of type: LoadBalancer .

There are two permitted values:

If you deploy a Service of type: LoadBalancer on AWS, and you don't set any service.beta.kubernetes.io/aws-load-balancer-type annotation, the AWS integration deploys a classic Elastic Load Balancer. This behavior, with no annotation present, is the default unless you specify otherwise.

When you set this annotation to external on a Service of type: LoadBalancer , and your cluster has a working deployment of the AWS Load Balancer controller, then the AWS Load Balancer controller attempts to deploy a load balancer based on the Service specification.

pod-security.kubernetes.io/enforce

Example: pod-security.kubernetes.io/enforce: "baseline"

Value must be one of privileged , baseline , or restricted which correspond to Pod Security Standard levels. Specifically, the enforce label prohibits the creation of any Pod in the labeled Namespace which does not meet the requirements outlined in the indicated level.

See Enforcing Pod Security at the Namespace Level for more information.

pod-security.kubernetes.io/enforce-version

Example: pod-security.kubernetes.io/enforce-version: "1.28"

Value must be latest or a valid Kubernetes version in the format v<major>.<minor> . This determines the version of the Pod Security Standard policies to apply when validating a Pod.

pod-security.kubernetes.io/audit

Example: pod-security.kubernetes.io/audit: "baseline"

Value must be one of privileged , baseline , or restricted which correspond to Pod Security Standard levels. Specifically, the audit label does not prevent the creation of a Pod in the labeled Namespace which does not meet the requirements outlined in the indicated level, but adds an this annotation to the Pod.

pod-security.kubernetes.io/audit-version

Example: pod-security.kubernetes.io/audit-version: "1.28"

pod-security.kubernetes.io/warn

Example: pod-security.kubernetes.io/warn: "baseline"

Value must be one of privileged , baseline , or restricted which correspond to Pod Security Standard levels. Specifically, the warn label does not prevent the creation of a Pod in the labeled Namespace which does not meet the requirements outlined in the indicated level, but returns a warning to the user after doing so. Note that warnings are also displayed when creating or updating objects that contain Pod templates, such as Deployments, Jobs, StatefulSets, etc.

pod-security.kubernetes.io/warn-version

Example: pod-security.kubernetes.io/warn-version: "1.28"

Value must be latest or a valid Kubernetes version in the format v<major>.<minor> . This determines the version of the Pod Security Standard policies to apply when validating a submitted Pod. Note that warnings are also displayed when creating or updating objects that contain Pod templates, such as Deployments, Jobs, StatefulSets, etc.

rbac.authorization.kubernetes.io/autoupdate

Example: rbac.authorization.kubernetes.io/autoupdate: "false"

Used on: ClusterRole, ClusterRoleBinding, Role, RoleBinding

When this annotation is set to "true" on default RBAC objects created by the API server, they are automatically updated at server start to add missing permissions and subjects (extra permissions and subjects are left in place). To prevent autoupdating a particular role or rolebinding, set this annotation to "false" . If you create your own RBAC objects and set this annotation to "false" , kubectl auth reconcile (which allows reconciling arbitrary RBAC objects in a manifest ) respects this annotation and does not automatically add missing permissions and subjects.

kubernetes.io/psp (deprecated)

Example: kubernetes.io/psp: restricted

This annotation was only relevant if you were using PodSecurityPolicy objects. Kubernetes v1.28 does not support the PodSecurityPolicy API.

When the PodSecurityPolicy admission controller admitted a Pod, the admission controller modified the Pod to have this annotation. The value of the annotation was the name of the PodSecurityPolicy that was used for validation.

seccomp.security.alpha.kubernetes.io/pod (non-functional)

Kubernetes before v1.25 allowed you to configure seccomp behavior using this annotation. See Restrict a Container's Syscalls with seccomp to learn the supported way to specify seccomp restrictions for a Pod.

container.seccomp.security.alpha.kubernetes.io/[NAME] (non-functional)

Snapshot.storage.kubernetes.io/allow-volume-mode-change.

Example: snapshot.storage.kubernetes.io/allow-volume-mode-change: "true"

Used on: VolumeSnapshotContent

Value can either be true or false . This determines whether a user can modify the mode of the source volume when a PersistentVolumeClaim is being created from a VolumeSnapshot.

Refer to Converting the volume mode of a Snapshot and the Kubernetes CSI Developer Documentation for more information.

scheduler.alpha.kubernetes.io/critical-pod (deprecated)

Example: scheduler.alpha.kubernetes.io/critical-pod: ""

This annotation lets Kubernetes control plane know about a Pod being a critical Pod so that the descheduler will not remove this Pod.

Annotations used for audit

• authorization.k8s.io/decision
• authorization.k8s.io/reason
• insecure-sha1.invalid-cert.kubernetes.io/$hostname
• missing-san.invalid-cert.kubernetes.io/$hostname
• pod-security.kubernetes.io/audit-violations
• pod-security.kubernetes.io/enforce-policy
• pod-security.kubernetes.io/exempt

See more details on Audit Annotations .

kubeadm.alpha.kubernetes.io/cri-socket

Example: kubeadm.alpha.kubernetes.io/cri-socket: unix:///run/containerd/container.sock

Annotation that kubeadm uses to preserve the CRI socket information given to kubeadm at init / join time for later use. kubeadm annotates the Node object with this information. The annotation remains "alpha", since ideally this should be a field in KubeletConfiguration instead.

kubeadm.kubernetes.io/etcd.advertise-client-urls

Example: kubeadm.kubernetes.io/etcd.advertise-client-urls: https://172.17.0.18:2379

Annotation that kubeadm places on locally managed etcd Pods to keep track of a list of URLs where etcd clients should connect to. This is used mainly for etcd cluster health check purposes.

kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint

Example: kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: https://172.17.0.18:6443

Annotation that kubeadm places on locally managed kube-apiserver Pods to keep track of the exposed advertise address/port endpoint for that API server instance.

kubeadm.kubernetes.io/component-config.hash

Example: kubeadm.kubernetes.io/component-config.hash: 2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae

Used on: ConfigMap

Annotation that kubeadm places on ConfigMaps that it manages for configuring components. It contains a hash (SHA-256) used to determine if the user has applied settings different from the kubeadm defaults for a particular component.

node-role.kubernetes.io/control-plane

A marker label to indicate that the node is used to run control plane components. The kubeadm tool applies this label to the control plane nodes that it manages. Other cluster management tools typically also set this taint.

You can label control plane nodes with this label to make it easier to schedule Pods only onto these nodes, or to avoid running Pods on the control plane. If this label is set, the EndpointSlice controller ignores that node while calculating Topology Aware Hints.

Example: node-role.kubernetes.io/control-plane:NoSchedule

Taint that kubeadm applies on control plane nodes to restrict placing Pods and allow only specific pods to schedule on them.

If this Taint is applied, control plane nodes allow only critical workloads to be scheduled onto them. You can manually remove this taint with the following command on a specific node.

node-role.kubernetes.io/master (deprecated)

Example: node-role.kubernetes.io/master:NoSchedule

Taint that kubeadm previously applied on control plane nodes to allow only critical workloads to schedule on them. Replaced by the node-role.kubernetes.io/control-plane taint. kubeadm no longer sets or uses this deprecated taint.

Was this page helpful?

Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it on Stack Overflow . Open an issue in the GitHub repo if you want to report a problem or suggest an improvement .

Kubernetes Annotations and Labels: What’s the Difference?

Ambassador Labs

Both labels and annotations are ways to attach metadata to objects in Kubernetes . But when should you use one versus the other?

The Kubernetes documentation is somewhat opaque on this subject:

Labels are key/value pairs that are attached to objects, such as pods. Labels are intended to be used to specify identifying attributes of objects that are meaningful and relevant to users, but do not directly imply semantics to the core system. Labels can be used to organize and to select subsets of objects. Labels can be attached to objects at creation time and subsequently added and modified at any time … Labels allow for efficient queries and watches and are ideal for use in UIs and CLIs. Non-identifying information should be recorded using annotations .

The definition of annotations is shorter, but still somewhat mysterious:

[Use] Kubernetes annotations to attach arbitrary non-identifying metadata to objects. Clients such as tools and libraries can retrieve this metadata.

The actual difference between annotations and labels is actually quite simple:

Labels are for Kubernetes, while annotations are for humans.

Tweet this.

What Are Labels in Kubernetes?

Labels are used in conjunction with selectors to identify groups of related resources. Because selectors are used to query labels, this operation needs to be efficient. To ensure efficient queries, labels are constrained by RFC 1123. RFC 1123, among other constraints, restricts labels to a maximum 63 character length. Thus, labels should be used when you want Kubernetes to group a set of related resources.

What Are Annotations in Kubernetes?

Annotations are used for “non-identifying information” i.e., metadata that Kubernetes does not care about. As such, annotation keys and values have no constraints. Thus, if you want to add information for other humans about a given resource, then annotations are a better choice.

Example of Using Annotations and Labels

The following simple example uses both annotations and a selector. The annotations are used to add information about the Kubernetes service for humans and are not used by Kubernetes. The selector is used by the Kubernetes service to identify resources that the service will route to.

Written by Richard Li

CEO, Ambassador Labs. Makers of the Kubernetes Developer Control Plane, CNCF Telepresence, and CNCF Emissary Ingress.

More from Richard Li and Ambassador Labs

externalTrafficPolicy=local on Kubernetes

How to preserve the source ip in kubernetes.

Why You Need an API Gateway For Microservices

An api gateway is one of the most critical components in a microservice architecture because it acts as a single entry point for all client….

Anita Ihuman

Request-Driven (RESTful) vs Event-Driven in Microservices

As technology advances, so do the techniques and tools used for building applications. apis have become increasingly prominent over the….

Envoy vs NGINX vs HAProxy: Why the open source Ambassador API Gateway chose Envoy

Nginx, haproxy, and envoy are all battle-tested l4 and l7 proxies. so why did we choose envoy as the core proxy as we developed the open…, recommended from medium.

Sumit Sagar

Understanding and Setting Up Istio in Kubernetes

Introduction.

Matthew Kennedy

Wise Engineering

Avoiding Kubernetes Pod Topology Spread Constraint Pitfalls

Now in AI: Handpicked by Better Programming

Kube-Proxy: What is it and How it Works

Purushotham Reddy

Kube-green: An operator to reduce CO2 footprint of your clusters

DeveloperSteve

Exploring the Kubernetes 1.28 2nd release (Planternetes)

Kubernetes, the tour-de-force of open-source collaboration, stands tall and proud in the tech forest, its branches stretching wide to….

Mutha Nagavamsi

Kubernetes 1.27: Pod Memory Protection

Kubernetes 1.27: memorythrottlingfactor is awesome⚡.

Text to speech

• Engineering
• Cost Per Customer Analysis
• Unit Cost Analysis
• SaaS COGS Measurement
• Cloud Cost Optimization
• Kubernetes Cost Analysis
• Migration Cost Monitoring
• Engineering Cost Awareness
• Tagging And Cost Allocation
• Content Library
• Case Studies
• Cloud Cost Assessment

CloudZero Advisor

Discover how CloudZero helps engineering and finance get on the same team — and unlock cloud cost intelligence to power cloud profitability

Explore CloudZero

Discover the power of cloud cost intelligence

Give your team a better cost platform

Give engineering a cloud cost coach

Learn more about CloudZero and who we are

Learn more about CloudZero's pricing

Take a customized tour of CloudZero

Explore CloudZero by feature

Build fast with cost guardrails

Drive accountability and stay on budget

Manage all your discounts in one place

Organize spend to match your business

By Use Case

Understand your cloud unit economics and measure cost per customer

Discover and monitor your real Kubernetes and container costs

Measure and monitor the unit metrics that matter most to your business

Allocate cost and gain cost visibility even if your tagging isn’t perfect

Identify and measure your software COGS

Decentralize cost decisions to your engineering teams

Automatically identify wasted spend, then proactively build cost-effective infrastructure

Monitor your AWS cost and track progress in real-time as you move to the cloud

All Your Cloud Spend, In One View

CloudZero ingests data from AWS, GCP, Azure, Snowflake, Kubernetes, and more

Discover the best cloud cost intelligence resources

Browse webinars, ebooks, press releases, and other helpful resources

Discover the best cloud cost intelligence content

Learn how we’ve helped happy customers like SeatGeek, Drift, Remitly, and more

Check out our best upcoming and past events

Gauge the health and maturity level of your cost management and optimization efforts

Compare pricing and get advice on AWS services including EC2, RDS, ElastiCache, and more

How SeatGeek Measures Cost Per Customer

Discover how SeatGeek decoded its AWS bill and measures cost per customer

How Skyscanner Creates A Cost-Aware Culture

Learn how Skyscanner decentralized cloud cost to their engineering teams

How Malwarebytes Measures Cost Per Customer

Learn how Malwarebytes measures cloud cost per product

How Remitly Shifts Cloud Costs Left

Learn how Remitly built an engineering culture of cost autonomy

How Ninjacat Combines AWS And Snowflake Spend

Discover how Ninjacat uses cloud cost intelligence to inform business decisions

How Smartbear Uses Cloud Cost To Inform GTM Strategies

Learn Smartbear optimized engineering use and inform go-to-market strategies

Kubernetes Annotations Vs. Labels: What's The Difference?

This guide covers Kubernetes labels and annotations in detail, the differences between the two, and best practices when using labels and annotations.

As a Kubernetes environment becomes more complex, it can become harder to manage. You can reduce that overwhelm by organizing your resources in a way that makes them easier to group, filter, and track.

You can do this by tagging your Kubernetes components. Kubernetes labels and annotations are two ways to accomplish this. Both are forms of Kubernetes metadata. Not sure what that is and how annotations and labels differ?

Here’s a straightforward guide to understanding Kubernetes (K8s) labels and annotations, including:

Table Of Contents

What Are Kubernetes Annotations?

What are kubernetes labels, what are the differences between labels and annotations in kubernetes.

• Best Practices Kubernetes Labels And Annotations
• How To Track, Understand, And Control Your Kubernetes Costs Hassle-Free

Kubernetes annotations are a type of metadata that you attach to your Kubernetes objects, such as ReplicaSets and Pods. In particular, annotations are key-value maps.

Annotations let you organize your application into sets of attributes that correspond to how you think about that application. You can organize, label, and cross-index each Kubernetes resource to reflect the groups that make sense for your use case.

Annotations attach extra metadata to an object, but you cannot use that data to select an object. That’s because this metadata is non-identifying, meaning it is not necessarily meant to identify an object in Kubernetes. Yet, clients, including libraries and tools, can make use of this data.

Depending on the annotation, metadata can be large or small, structured or unstructured, and can contain diverse characters — in fact, more than are permitted by Kubernetes labels (more on this in the next section).

Now, picture this:

Credit : Strings in Kubernetes keys and values

Kubernetes annotations must meet the following characteristics and syntax requirements:

• An annotation comprises a key and a value. The key is a unique identifier for the annotation, and the value is the information the annotation is associated with. Annotations store additional or contextual information about the data being stored, such as descriptions.
• Even Kubernetes annotations require that your keys and values be strings. This means the keys or values cannot be numeric, boolean, list, or other types.
• Valid annotation keys have two segments: an optional prefix and name, separated by a slash (/).
• You must include the name segment, which should be 63 characters or fewer, starting and ending with an alphanumeric character ( [a-z0-9A-Z] ) with dashes (-), underscores (_), dots (.), and alphanumerics between.
• A prefix is optional. But if you specify it, it must be a DNS subdomain. That is, it should be a series of DNS labels separated by dots (.), with no more than 253 characters in total, followed by a slash (/). If you omit the prefix, the annotation Key is presumed private to the user.
• Automated system components, such as kubectl, kube-scheduler, kube-apiserver, kube-controller-manager , or other third-party automation, that add annotations to end-user objects must specify a prefix.
• The prefixes kubernetes.io/ and k8s.io/ are reserved for Kubernetes core components.

Want to see an example Kubernetes annotation? Here's a manifest for a Pod that has the annotation imageregistry: https://hub.docker.com/

The following are some examples of details you can record in annotations:

• Fields that a declarative configuration layer manages. By attaching the fields as annotations, you get to differentiate them from default values that clients or servers set. This also distinguishes them from auto-generated fields and fields that auto-sizing or auto-scaling tools set.
• More information about builds, releases, images, teams, updates, etc. This includes details such as timestamps, image hashes, owner team, git branch, release IDs, registry address, and PR numbers.
• Pointers to monitoring, logging, analytics, or audit repositories.
• Data that's usable for debugging purposes, such as the name, version, and build information of a tool.
• Provenance information about a user, tool, or system, including URLs of related objects coming from other components in the same ecosystem.
• Config, checkpoints, or other lightweight information from a rollout tool.
• Contact details of the project team, such as phone or pager numbers. Or, directory entries that point to where to find that information, like a particular website.
• End-user or customer directives and instructions to alter behavior or extend capabilities.

Now, here's the thing. An external database or directory could store this type of information instead of in an annotation. But then again, that would make it harder to build shared client libraries and other tools, such as for deployments and management.

Labels in Kubernetes are key-value pairs containing metadata and are used to identify objects. An object can have its own key-value labels. In addition, you can attach labels to a K8s object when creating it. You can then add or modify the label as needed over time.

You can use labels in a variety of ways, including:

• Identify objects by properties users can understand (Kubernetes itself does not use labels).
• Group objects into subsets by defining and organizing them.
• Identify objects within a cluster.
• More easily modify configurations in bulk.
• Track, allocate, and manage Kubernetes costs.

Kubernetes labels must meet the following characteristics and syntax requirements:

• A key must be unique for a specific object.
• The segment must have up to 63 characters, while the prefix can have up to 253 characters.
• Unless its length is 0, it must have alphanumeric characters [a-z0-9A-Z] at the start and at the end.
• Characters like dot (.), dashes (-), and underscore (_) are allowed (internally).
• The prefix should be a sequence of DNS labels that are separated by a dot and preceded by a slash.

Using the prefix enables you and automatic system components, such as kube-scheduler, as well as third-party tools, to manage resources.

The biggest difference between annotations and labels in Kubernetes is this: while labels attach identifying metadata to a K8s object, annotations attach additional information that is not identifying.

As you’ve probably already noticed in the definition sections previously, there are several other differences, including the number of characters an annotation can have vs. a label and use cases.

But just to recap, here’s a quick side-by-side comparison of Kubernetes labels vs. annotations:

What Are Some Best Practices For Using Kubernetes Labels And Annotations?

By labeling or annotating your Kubernetes resources, you can keep track of specific components in your rapidly expanding K8s environment.

If you want to select, group, or filter specific K8s objects, you can use labels. You can also use annotations to provide more information about a particular object.

However, this is not always the case, particularly if you miss a couple of crucial best practices for tagging.

Below is a quick list of Kubernetes annotations and labels best practices to always keep in mind when tagging your K8s components.

• You can refine your visibility into your Kubernetes objects by attaching labels and annotations to more of your Kubernetes components.
• Maintain a consistent labeling convention to avoid mixups, duplications, and a lot of rework.
• Keeping your labels and annotations short will prevent complexity, particularly for labels, which K8s uses internally for operations.
• Simplify tagging by using Kubernetes-recommended labels and annotations as early as possible. This includes details about the object's name, owner, environment, and version.
• Build consensus by following the correct character set and syntax for each type.
• Do not add confidential metadata to annotations or labels.
• As a backup, use labels and annotations with other Kubernetes tagging methods.
• Organize resources in a way that makes sense for object queries. This way, your team can more easily detect and troubleshoot issues in bulk.

Track, Understand, And Control Your Kubernetes Costs With a Robust Platform

Tags and tag management can be endless, complicated, and tedious. Yet, if your cost allocation tags aren't perfect , it's hard to figure out who, what, and why your Kubernetes costs are changing. Well, not if you've got CloudZero.

If your labels and annotations aren't perfect, we get it. So we built CloudZero from scratch to deliver accurate and actionable Kubernetes cost intelligence . With CloudZero, you can collect cost data from tagged, untagged, and untaggable Kubernetes resources hassle-free.

Besides breaking down your Kubernetes costs by individual customer , team, product, feature, and more granularity, CloudZero also delivers your cost insights by various K8s concepts , such as per pod, cluster, namespace, environment, service, etc, — down to the hour.

And did we mention that tags are not required?

Cody Slingerland

Cody Slingerland, a FinOps certified practitioner, is an avid content creator with over 10 years of experience creating content for SaaS and technology companies. Cody collaborates with internal team members and subject matter experts to create expert-written content on the CloudZero blog.

Author: Cody Slingerland

Stay in the loop, join thousands of engineers who already receive the best aws and cloud cost intelligence content., recent posts.

CloudZero is the only solution that enables you to allocate 100% of your spend in hours — so you can align everyone around cost dimensions that matter to your business.

For Your Business Goals

By provider.

This website uses cookies. By continuing to browse, you agree to our Privacy Policy.

Best practices guide for kubernetes labels and annotations.

Kubernetes is the de facto container-management technology in the cloud world due to its scalability and reliability. It also provides a very flexible and developer-friendly API, which is the foundation of its control plane.

The effectiveness of the Kubernetes API comes from how it manages the Kubernetes resources via metadata: labels and annotations. Metadata is essential for grouping resources, redirecting requests and managing deployments. In addition, is is also used to troubleshoot Kubernetes applications .

In this blog post, you will learn the basics and best practices of using labels and annotations.

Kubernetes Labels

Kubernetes labels are the metadata information attached to the Kubernetes resources to group, view, and operate. Labels are in the format of key and value string pairs, where each key should be unique.

Let’s take a look at them in action:

In the previous command, you retrieved the labels of the minikube node, which include information related to the operating system, hostname, and the minikube version running on the node. You can use the labels for retrieving and filtering the data from the Kubernetes API.

Let’s assume you want to get all the pods running the Kubernetes dashboard . You can use the selector k8s-app=kubernetes-dashboard over labels with the following command:

The hidden gem of Kubernetes labels is that they are heavily used with the Kubernetes itself, such as scheduling pods to nodes, managing replicas of deployments, and network routing of services.

Let’s look at some labels and how they are used as selectors in Kubernetes by checking the spec of the kubernetes-dashboard service:

Kubernetes uses the labels defined in the selector section to distribute the incoming requests to the kubernetes-dashboard service. With a similar approach, replica sets track the number of pods to maintain replicas running on the cluster. Now let’s check the selector of the replica set for the dashboard:

The matchLabels indicate that there will be enough pods with the mentioned labels in the cluster. When you release a new version, it will create a new pod-template-hash , and replica set controllers will create new pods instead.

Kubernetes Annotations

Kubernetes annotations are the second way of attaching metadata to the Kubernetes resources. They are pairs of key and value strings that are similar to labels, but which store arbitrary non-identifying data. For instance, you can keep the contact details of the responsible people in the deployment annotations. Similarly, you can attach logging, monitoring, or auditing information for the resources in the annotations format.

The main difference between annotations and labels is that annotations are not used to filter, group, or operate on the resources. Rather, they are used to easily access additional information about the Kubernetes resources.

For instance, CRI socket or volume controller annotations show how the node works, instead of its characteristics, in the following example:

Client tools and Kubernetes users can retrieve the metadata and operate accordingly. You can imagine the data kept in annotations to be stored in Excel sheets or databases; however, they are attached to the resources. Therefore, there is no selector implementation like labels in the Kubernetes API.

Best Practices

Now that we’ve covered the fundamentals of Kubernetes labels and annotations, it’s time to explore the best practices for using them most beneficially.

Use the Correct Syntax

Annotations and labels are key-value pairs. Keys consists of two parts: an optional (but highly suggested) prefix and name:

• Prefix: If specified, the prefix should be a DNS subdomain no longer than 253 characters and ending with a slash. For example: k8s.komodor.com/
• Name: This is required and limited to 63 characters.

When the prefix is omitted, you can assume that labels or annotations are private for your cluster and user. When the prefix and name are used together, you should store the data to be used with multiple clients, similar to the following:

• app.kubernetes.io/version
• app.kubernetes.io/component
• helm.sh/chart

Using the correct syntax for labels and annotations makes it easier to communicate within your team and use the cluster with client tools and libraries such as kubectl, Helm, and operators. Therefore, it is suggested to choose a prefix for your company and sub-prefixes for your projects. This company-wide consensus will help you utilize labels and annotations to their full power.

Learn When to Use Labels and Annotations

As mentioned earlier, the main difference between labels and annotations is whether they are identifiers or not. If you want to attach information to group resources and filter, you should keep the data as labels. Use annotations if the metadata is not an identifier, but rather additional data related to the Kubernetes resources.

For instance, the following pod has two labels and two annotations:

In the demo pod, labels classify it as being an nginx application running in production. Annotations show the owner and communication data. If you plan to group pods by owners in the future, it is suggested to move komodor.com/owner to labels.

Using labels and annotations with the correct use cases is vital to have an easy-to-operate cluster with automated tools. Therefore, ensure that your labels and annotations are not overlapping in terms of data and usage.

Exploit the Standard Labels and Annotations

Kubernetes reserves all the labels and annotations with the key kubernetes.io domain name and keeps a list of well-known ones in the official documentation . You may have seen some of them in the Kubernetes dashboard or resource definitions, such as:

The main advantage of this metadata is that the Kubernetes machinery automatically fills values of the standard labels and annotations. Thus, it is suggested to use the well-known labels and annotations in your daily operations and client tools, such as Helm, Terraform, or kubectl.

Use Labels for Release Management

Releasing distributed microservices applications to the cloud is not straightforward, as you have an excessively high number of small applications—each with its own version. Therefore, most developers only change the version of a single application out of a hundred and test the rest of the system. Fortunately, you can use labels for grouping and filtering the applications running on Kubernetes.

Let’s assume you have a backend service that has multiple pods running behind it with the labels version:v1 and app:backend . You can deploy a new set of backend instances to the cluster and change the service label selector to version:v2 and app:backend . Now, all requests coming to the backend service will reach v2 instances. Luckily, switching back to v1 is pretty easy, as you only need to change the service specification.

This procedure is also known as the Blue/Green deployment strategy. In addition, you can easily implement A/B testing and canary release strategies with the help of Kubernetes labels.

Learn How To Manipulate Labels for Troubleshooting

The last best practice is for the Kubernetes operators who need to debug applications running inside the cluster. Let’s assume you have a deployment with the following selector labels:

• app.kubernetes.io/name: my-complex-app
• app.kubernetes.io/instance: prod-1
• app.kubernetes.io/version: "1.1.0"

All pods of the deployment will also have the same set of labels. Unfortunately, you cannot change and modify the pods, but you can change the labels of the selector in order to not match current pods. It will make the running pods orphaned, and you can exec into them for debugging.

Kubernetes will create new pods with the new labels, and your production setup will continue living as expected—with an additional pod that you’ll want to analyze further for troubleshooting. You can interfere with the operations of Kubernetes and troubleshoot your applications when you know how labels are designed and used by Kubernetes.

In this blog post, we covered the fundamentals of Kubernetes labels and annotations through examples and best practices that are essential to bringing the power of metadata tools to light. Using the correct syntax with the intended aim will make your labels and annotations more meaningful and maintainable. In addition, you can exploit the standard labels of Kubernetes with prepopulated data in your applications. Finally, labels are helpful for cloud-native release management and application debugging.

In order to gain overall control and visibility into your Kubernetes clusters, check out Komodor and our Kubernetes-native troubleshooting solution. This will simplify the complex and distributed environment of Kubernetes and help you understand what is actually happening in your clusters.

Sign up for a free trial to see how you troubleshootiing intelligently while leveraging your existing stack can make a difference.

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

Latest Blogs

Unveiling komodor’s network mapping capability.

Thanks to the power of the open-source community, and our friends over at Otterize, we have now enhanced our K8s offering for developers with another visual aid to streamline operations and troubleshooting.

Kubernetes Community Days Munich Recap

Guy and I had some fascinating conversations with folks from the local community and from all over Europe. We had a lot of fun and also learned a lot.

Using Helm Dashboard and Intents-Based Access Control for Pain-Free Network Segmentation

Learn how to use Komodor's and Otterize's OSS projects in tandem to ease the pain of network segmentation, without the messiness of a Service Mesh.

Cloud Training Program

Learn Cloud With Us

Kubernetes Labels | Labels And Annotations In Kubernetes – Everything You Need To Know

May 10, 2023 by Piyush Jain Leave a Comment

Kubernetes is a container orchestration tool, provides a platform for automating deployment, scaling, and management of our containers. Kubernetes Labels and Kubernetes Annotations are one of the main components. They both provide a way for adding additional metadata to our Kubernetes Objects.

Are you new to Kubernetes? Check out our blog Kubernetes for Beginners  to know in detail.

In this blog, we will be covering:

• What is Kubernetes Labels?
• Creating Labels

Label Selectors

• Service and Labels
• What are Annotations?
• Creating Annotations
• Kubernetes Labels vs Annotations

What Is Kubernetes Labels?

Labels in Kubernetes are intended to be used to specify identifying attributes of objects that are meaningful and relevant to users but are not used by the Kubernetes itself. Labels are fundamental qualities of the object that will be used for grouping, viewing, and operating. Each object can have a set of key/value labels defined. Each Key must be unique for a given object.

Labels have a simple syntax, where both the key and value are represented by strings. Label keys can be broken down into two parts: an optional prefix and a name , separated by a slash(/). The prefix can be a DNS sub-domain with a 253-character limit. The key name is required and must be shorter than 63 characters , beginning and ending with an alphanumeric character ( a-z0-9A-Z ) with dashes ( – ), underscores ( _ ), dots ( . ), and alphanumerics between.

Label values  are strings with a maximum length of 63 characters . The contents of the labels follow the same rules as for label keys. Here are some examples of labels in Kubernetes 1) key – kubernetes.io/cluster-service value – true 2) key – appVersion value – 1.0.0

Also Read:  Our blog post on P ersistent Storage in Kubernetes . Click here

Creating Labels In Kubernetes

Labels are attached to the Kubernetes objects and always defined in the metadata section of the configuration files. The commonly used key and label conventions are:

Also Read:  our blog post on Kubernetes Scheduler

Here’s the configuration file for a Pod that has two labels “ environment: dev ” and “ app: <name of the app> “

Save the file as pod.yaml and then run the following command:

To see the labels use the following command:

Read More : About Kubernetes cluster setup : A Complete Step-by-Step Guide

Label selectors are used for filter Kubernetes objects based on a set of labels. Selectors use a simple Boolean language. There are two kinds of selectors: Equality based and Set based.

Equality based

Equality-based selectors allow filtering by label keys and values. Three kinds of operators are used: =, ==, != Example: If we wanted to list all the Pods that had the environment label set to dev, we can use the selector flag:

Check Out:  Our blog post on M ulti Container Pod Kubernetes . Click here

Set-based label selectors allow filtering keys according to a set of values. Three kinds of operators used: in, notin, exists Example: If we wanted to list all the Pods that had the environment label set to dev or production:

Install Docker   on windows, ubuntu, mac with easy steps.

S ervices And Labels

A Service sits in front of our Pods and distributes requests to them. They route traffic across the Pods. Services are the abstraction that allows pods to die and replicate in Kubernetes without impacting the application. They use Kubernetes Labels and Selectors to match a set of Pods.

Also read: our blog on Kubernetes Networking Services

What Are Kubernetes Annotations?

Annotations in Kubernetes provide a place to store non-identifying metadata for Kubernetes Objects which can be used to get a more elaborate context for an object. Annotations are also key/value pairs like Labels. Annotation keys use the same format as Label keys.

Why Annotations Are Used?

• Keep track of a “reason” for the latest upgrade to an object.
• Communicate a specialized scheduling policy to a specialized scheduler.
• Extend data about the last tool to update the resource and how it was updated (used for detecting changes by other tools and doing a smart merge).
• Attach build, release, or image information that isn’t appropriate for labels.
• Enable the Deployment object to keep track of ReplicaSets that it is managing for rollouts.
• Phone or pager numbers of persons responsible, or directory entries that specify where that information can be found, such as a team web site.

Also read: What is the difference between Container vs VM

Creating Annotations In Kubernetes

Annotations are defined in the common metadata section in every Kubernetes object. Here’s the configuration file for a Pod that has the annotation.

Save the above file with annotations.yaml and run the following command:

Also Check:  Our blog post on Kubernetes pod . Click here

To see the annotations, use the following command:

Also Read:  what is helm Kubernetes ?

Difference Between Labels And Annotations

Kubernetes Labels and Kubernetes Annotations are used to add the metadata to our Kubernetes objects. But there is a difference between both of them. Kubernetes Labels allow us to do a grouping of our objects so that we can perform queries for viewing and operating.

Kubernetes Annotations are used for adding non-identifying metadata to Kubernetes objects. This metadata information is only for the user. Annotations can hold any kind of information that is useful and can provide context to DevOps teams. Examples include phone numbers of persons responsible for the object or tool information for debugging purposes.

Related Post

• Kubernetes Architecture | An Introduction to Kubernetes Components
• Kubernetes for Beginners
• Visit our YouTube channel on “Docker & Kubernetes”
• Certified Kubernetes Administrator (CKA) Certification Exam: Everything You Must Know
• Certified Kubernetes Administrator (CKA) Certification: Step By Step Activity Guides/Hands-On Lab Exercise

Join FREE Masterclass

Discover the Power of Kubernetes, Docker & DevOps – Join Our Free Masterclass . Unlock the secrets of Kubernetes, Docker, and DevOps in our exclusive, no-cost masterclass. Take the first step towards building highly sought-after skills and securing lucrative job opportunities. Click on the below image  to Register Our FREE Masterclass Now!

Share This Post with Your Friends over Social Media!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

"Learn Cloud From Experts"

• Partner with Us
• Terms and Conditions
• Privacy Policy
• Certified Kubernetes Administrator (CKA)
• Azure Administrator [AZ-104]
• Azure Solutions Architect [AZ-305]
• AWS Solutions Architect [SAA-C03]
• Azure Data Engineer [DP-203]
• Oracle Integration Cloud
• Oracle DBA To Cloud DBA [1Z0-1093]

Get in touch with us

8 Magnolia Pl, Harrow HA2 6DS, United Kingdom

Email : [email protected]

• Stack Overflow Public questions & answers
• Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers
• Talent Build your employer brand
• Advertising Reach developers & technologists worldwide
• Labs The future of collective knowledge sharing
• About the company

Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

What is the difference between. annotations and labels in in Kubernetes?

I'm trying to wrap my head around the difference between annotations and labels.

My understanding of annotations is that it is metadata that adds key-value pairs that cannot be used by Kubernetes for identifying/filtering the resource.

Labels on the other hand are metadata key-value pairs that can be used by Kubernetes to identify/filter the resource.

Is this right? If this is so, then what is the practical use of annotations? Is it something to do with performance? Where labels are under the scanner of Kubernetes for filters and annotations are purely for adding metadata that is just informational?

But I've seen in cases where deployments needing Nginx or ingress capabilities using annotations. Then how does this get searched or used. Why are labels not used here instead?

When do we use annotations over labels and vice-versa? What are the pros and cons of each?

My understanding is rather limited here, however reading the official docs has not really helped me understand the use case of when do I use annotations vs labels.

• kubernetes-apiserver

5 Answers 5

Labels are key/value pairs that can be attached to Kubernetes objects such as Pods and ReplicaSets. They can be arbitrary, and are useful for attaching identifying information to Kubernetes objects. Labels provide the foundation for grouping objects. Annotations, on the other hand, provide a storage mechanism that resembles labels: annotations are key/value pairs designed to hold nonidentifying information that can be leveraged by tools and libraries. -- Kubernetes up & running, Chapter 6

Labels are used to identify resources

Examples of what labels can do:

find all pods that have a value associated with the key

kubectl get pods -l key=val,key2=val2

merge and stream logs of the various pod that share the same label

kubectl logs -l key=val

The reason why labels are used as selectors as opposed to annotations is because most Kubernetes implementation index labels in etcd.

Annotations are used to store data about the resource itself

This usually consists of machine-generated data, and can even be stored in JSON form.

• last updated
• sidecar injection configuration etc

• another important difference is that opposed to annotations , the labels' values are limited to 63 characters . –  Wolfson Feb 21, 2022 at 17:39

Labels are metadata assigned to objects for identification purposes. For instance, a service selects the backend pod using the labels on pods.

Annotations are additional metadata that can be open-ended. It may be used for documentation purposes, or it can be used for configuring an object. For instance, the Nginx ingress controller reads those annotations on the running pod and uses them to configure the underlying NGinx instance. How annotations are used is completely up to the implementation.

Labels are indexed in Etcd and can be searched on. Annotations cannot.

I would like to add my perspective to understand better.

Labels - Key-value pairs that you can associate with any K8s object. It is meant to be consumed by developers or Admins to select or filter objects.

• Select all pods with a specific value and bring them down.

Annotation - Key-value pairs that you can associate with any K8s object. This is not meant to be used by devs/admins and is not queryable. It is primarily used by the object to configure itself. Just to compare this with Java/Spring App Development, It's like passing some spring properties that is used by one of the Spring beans to configure itself.

• Role to be used by the Service Account
• Target type or port to be used by Ingress Object.

Labels have additional limitations as explained here: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set

Valid label value: must be 63 characters or less (can be empty), unless empty, must begin and end with an alphanumeric character ([a-z0-9A-Z]), could contain dashes (-), underscores (_), dots (.), and alphanumerics between.

So if it is a simple value you can set it as a label, if it's a URL or more complex value then store it as metadata / annotation.

The search syntax is also different for labels and annotations.

Your Answer

Sign up or log in, post as a guest.

Required, but never shown

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct .

Not the answer you're looking for? Browse other questions tagged kubernetes deployment devops admin kubernetes-apiserver or ask your own question .

• The Overflow Blog
• If you want to address tech debt, quantify it first
• Fighting comment spam at Facebook scale (Ep. 602)
• Featured on Meta
• Moderation strike: Results of negotiations
• Our Design Vision for Stack Overflow and the Stack Exchange network
• Call for volunteer reviewers for an updated search experience: OverflowAI Search
• Temporary policy: Generative AI (e.g., ChatGPT) is banned
• Discussions experiment launching on NLP Collective

Hot Network Questions

• What do Americans say instead of “can’t be bothered”?
• Is the 'a nice' in 'a nice to have' grammatical?
• What are the balance implications of removing spell lists?
• What adhesive solution should use for walls with powdery surface?
• Is naturalism the null hypothesis?
• std::chrono compatible clock using CLOCK_MONOTONIC_RAW
• When in Rome... or perhaps Ankh-Morpork?
• Is it safe to create a public ID by hashing a private key?
• Can analog bandwidth be modeled as an RC low-pass filter in this way?
• What is this transparent round "plate" brick with wiring inside?
• How can a country force an ambassador to leave
• Feminist linguistics?
• Is there a socially-optimal way to drive on a busy interstate?
• What would happen if the Panama Isthmus was turned into an island chain?
• I've always learned that data standardization is not necessary for OLS regression, but then recommended for neural networks. Intuitively, why is that?
• LaTeX macro for summing numbers
• How to ensure that an optimising compiler will terminate?
• Correcting how a student writes symbols
• Sustainable eating habits as a pilot
• Estimate of Minkowski sum
• Coworker hiding/stealing lab material - advice?
• Ask for a reduction in conference registration fees
• Is quadrature still considered part of numerical analysis?
• People who can't receive O negative blood?

Your privacy

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy .

What is Annotations in Kubernetes?

There are two way using you can attach metadata to Kubernetes objects.

• annotations

Kubernetes annotations is used to attach arbitrary non-identifying metadata to objects. Clients such as tools and libraries can retrieve this metadata.

In contrast, annotations are not used to identify and select objects. The metadata in an annotation can be small or large, structured or unstructured, and can include characters not permitted by labels.

Annotations, like labels, are key/value maps:

apiVersion: extensions/v1beta1

kind: Deployment

  annotations:

    kompose.cmd: ./kompose convert

    kompose.version: “”

Here are some examples of information that could be recorded in annotations:

• Build, release, or image information like timestamps, release IDs, git branch, PR numbers, image hashes, and registry address.
• Pointers to logging, monitoring, analytics, or audit repositories.

https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/

• Recent Posts

• Zabbix Tutorials: Working with Zabbix API - August 20, 2023
• 𝗗𝗲𝘃𝗢𝗽𝘀 𝗩𝘀 𝗦𝗥𝗘 𝗩𝘀 𝗣𝗹𝗮𝘁𝗳𝗼𝗿𝗺 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 - August 20, 2023
• Zabbix Tutorials: Example of zabbix_server.conf - August 20, 2023