How Does Kubernetes Work?

In the early 2010s, Google was running billions of container starts every week. Not because it had billions of users, but because it was constantly destroying and recreating the containers running its internal services, replacing old versions with new ones thousands of times per day. This churn would have been unmanageable if Google’s engineers were manually tracking which container was running on which machine, whether it was healthy, and what to do when a server crashed. So Google built the internal system that eventually became Kubernetes: a way to declare what you want your software to look like, and let the system figure out how to make it happen.

Today, Kubernetes runs inside every major cloud provider, in corporate data centers, and on developer laptops. It has become the default way to run production software at scale, with the CNCF’s 2024 survey finding that 78% of organizations surveyed were using Kubernetes in production, up from 83% in 2022 and 59% in 2020. Understanding how it works is essential for anyone who deploys code in the cloud era.

The short answer

Kubernetes is an orchestration platform that automatically schedules, scales, and recovers containerized applications across a cluster of machines. You define the desired state of your application in a configuration file: how many copies should run, how they should be accessible, and what resources they need. Kubernetes continuously compares that desired state against the actual state of your running containers and corrects any drift, whether that means restarting a crashed process, rescheduling a pod onto a healthier machine, or adding more copies when traffic spikes.

The full picture

Containers and why they needed orchestration

Before Kubernetes, the standard approach to deploying software was either running it directly on virtual machines or bare metal servers, or packaging it as a virtual machine image. Both approaches had problems. VMs are large and slow to start. Installing and configuring software on a bare metal server is error-prone and hard to reproduce.

Containers changed that. A container is a lightweight, standalone package that includes an application and all the libraries and configuration files it needs to run. Unlike a VM, a container does not include a full operating system kernel. It shares the host kernel, which makes it start in seconds rather than minutes and use a fraction of the memory. Docker, the dominant container platform, made containers accessible to everyday developers: a single command could build an image, and that same image would run identically on a laptop, in a data center, or in the cloud.

But containers introduced a new problem. Once you had tens or hundreds of containers running across many machines, you needed to answer questions that the container runtime itself did not handle. Which server should a new container run on? What happens when a server fails mid-execution? How do you route traffic to containers whose IP addresses keep changing? How do you roll out a new version without dropping requests? Kubernetes was built to answer exactly these questions.

The cluster: nodes, control plane, and data plane

A Kubernetes cluster is a group of machines that work together to run containerized workloads. Each machine in the cluster is called a node. Nodes can be virtual machines or physical servers. One or more nodes are designated as the control plane, which is the set of processes that manage the overall state of the cluster: scheduling, scaling, and responding to node failures. The remaining nodes are worker nodes, which run the actual application pods.

The control plane runs a set of tightly scoped services. The API server is the central interface: every command, whether from a developer running kubectl or from an automated script, goes through the API server. The scheduler watches for new pods with no assigned node and decides where they should run, balancing resource availability across the cluster. The controller manager runs background control loops that continuously check whether the actual state of the cluster matches the desired state and corrects mismatches. The etcd database stores the entire cluster state as a set of key-value pairs, acting as the single source of truth for the control plane.

Worker nodes each run a kubelet, a small agent that communicates with the API server and ensures that the containers described for a pod are running and healthy. They also run kube-proxy, a network proxy that maintains network rules on each node, enabling pods to communicate with each other and with services outside the cluster.

Pods, deployments, and the reconciliation loop

The fundamental unit of work in Kubernetes is the pod. A pod represents a single instance of a running process and is the smallest schedulable entity in the cluster. Most pods contain a single container, but they can contain multiple containers that need to share namespace, network, or storage. Containers inside a pod can communicate with each other via localhost, because they share the same network namespace.

Pods are ephemeral by design. They do not persist when the node they run on fails or is shut down. This is a deliberate choice: it means application developers do not need to think about which specific machine their code runs on. Kubernetes abstracts away the physical infrastructure.

When you want to run a pod, you do not create it directly. Instead, you create a Deployment, which is a higher-level object that manages a set of identical pod replicas. The Deployment tells Kubernetes how many copies of your pod should be running, what container image to use, how to update them (rolling update or recreate), and how many resources each pod is allowed to consume.

Behind every Deployment is the reconciliation loop: a continuous cycle of comparing desired state against actual state and correcting drift. When you apply a Deployment that specifies five replicas, the Deployment controller checks whether five pods are currently running. If only three are running, it creates two more. If six are running, it terminates three. If a node fails and takes five pods with it, the kube-controller-manager detects that those pods have stopped sending heartbeats and creates five replacement pods on healthy nodes. This loop runs continuously, and it is what gives Kubernetes its self-healing property.

Services and ingress: how traffic reaches your pods

Because pods are ephemeral, their IP addresses are unstable. When a pod restarts, Kubernetes assigns it a new IP from a pool. This means you cannot reliably route traffic to a pod by its IP alone. The Service abstraction solves this.

A Service is a stable network endpoint that groups all pods matching a label selector and provides a single DNS name and IP address for the group. When traffic arrives at a Service, kube-proxy routes it across the eligible pods, handling load balancing automatically. If a pod crashes and a new one takes its place, the Service continues routing to the same DNS name; clients do not need to know that the underlying pods changed.

Kubernetes Services come in several types. A ClusterIP Service exposes the application only within the cluster, accessible at an internal IP. A NodePort Service exposes the application on a static port on every node’s IP, enabling external access without a cloud load balancer. A LoadBalancer Service provisions external cloud infrastructure to route traffic into the cluster, which is how most production web applications are exposed to the internet.

For more complex routing needs, such as handling multiple subdomains or path-based routing to different services, Kubernetes supports an Ingress object, which acts as a layer 7 HTTP load balancer configured by rules rather than by port.

ConfigMaps, secrets, and storage

Application code often needs configuration that changes between environments: database connection strings, API keys, feature flags. Hard-coding these values into a container image defeats the purpose of a portable, reproducible container. Kubernetes provides two objects for separating configuration from code.

A ConfigMap stores non-sensitive configuration as key-value pairs or files. Pods can consume ConfigMap values as environment variables or mounted files. A Secret works identically but is designed for sensitive data such as passwords, TLS certificates, and API tokens. Secrets are encoded in base64, not encrypted by default, so production deployments typically integrate with an external secrets manager that injects the actual values at runtime.

Applications also need persistent storage. A pod’s filesystem is ephemeral; when the pod terminates, the files disappear. PersistentVolumes provide a storage resource that exists independently of any pod. When a pod needs storage, it claims a PersistentVolume through a PersistentVolumeClaim, and Kubernetes binds the claim to an available volume that meets the requirements, whether that is a local disk, a network file system, or a cloud provider’s block storage.

Why it matters

Before Kubernetes, the gap between writing code and running it reliably in production was enormous. A team that wanted to scale from one server to ten had to manually provision machines, configure load balancers, set up health checks, and establish deployment procedures. Scaling back down required reverse-engineering all of those steps. The operational complexity grew faster than the business complexity, and engineering teams spent more time managing infrastructure than building features.

Kubernetes compressed that gap. When your application is running on Kubernetes, scaling from one replica to twenty is a single command or a single configuration change. The system handles provisioning the underlying resources, distributing the load, and restarting any replicas that fail. This is not just a developer convenience. It changes the economics of running software. Companies that once needed a dedicated infrastructure team of five to keep a production system running can now run the same workload with a single engineer who writes configuration files.

The self-healing property is equally consequential. In a world without orchestration, a server failure meant pages went out, an engineer got out of bed at 2am, logged into a machine, diagnosed the failure, and manually restored service. According to a Gartner analysis, infrastructure and operations automation is a top priority for cloud-native strategies. On Kubernetes, the system detects the failure, reschedules the affected pods, and restores capacity automatically, often before anyone notices. This is the difference between infrastructure that requires constant human attention and infrastructure that reliably runs itself.

Common misconceptions

“Kubernetes is the same as Docker.”

This confuses two distinct layers. Docker is a tool for building and running containers. Kubernetes is a system for coordinating many containers across many machines. You can use Docker without Kubernetes (a single developer laptop running containers manually), and you can use Kubernetes without Docker (any OCI-compliant container runtime, including containerd and Podman, works with Kubernetes). Docker was the catalyst that made containers mainstream, but Kubernetes is what made container orchestration practical.

“Kubernetes is only for large companies.”

The tooling has matured to the point where small teams and even individual developers run Kubernetes in production. Managed Kubernetes services from AWS, Google Cloud, and Azure let you provision a production-ready cluster in minutes without managing the control plane yourself. There is also a thriving ecosystem of lightweight distributions, including k3s, which runs a full Kubernetes cluster on a single Raspberry Pi. The operational complexity that once made Kubernetes prohibitive for small teams has largely moved into managed services and well-documented defaults.

“If my pods are running, my application is healthy.”

A pod being in a Running state tells you that the container process started, not that the application inside is working correctly. The application could be deadlocked, returning 500 errors, or slowly running out of memory, while Kubernetes reports the pod as healthy because the container process is still technically running. Production Kubernetes deployments rely on liveness probes and readiness probes, which are HTTP checks, TCP socket tests, or exec commands that Kubernetes runs against your container to verify that the application is actually responding correctly, not just alive.

Key terms

Cluster: A set of machines (nodes) that Kubernetes uses to run workloads. Consists of one or more control plane nodes and any number of worker nodes.

Node: A single machine in a Kubernetes cluster, either physical or virtual. Worker nodes run pods; control plane nodes run the Kubernetes control plane services.

Pod: The smallest deployable unit in Kubernetes. Represents a single running instance of a process, containing one or more containers that share network and storage.

Deployment: A Kubernetes object that manages a set of identical pod replicas, handling rolling updates, scaling, and self-healing.

Service: A stable network endpoint that routes traffic to a set of pods matching a label selector, providing a consistent DNS name even as individual pods come and go.

ReplicaSet: A Kubernetes controller that ensures a specified number of pod replicas are running at any given time. Deployments manage ReplicaSets indirectly.

Namespace: A virtual cluster within a physical Kubernetes cluster. Namespaces provide scope for names, resource quotas, and access control policies, enabling multiple teams or projects to share a cluster without interfering with each other.

kubectl: The command-line tool for interacting with Kubernetes clusters. Developers use kubectl to apply configurations, inspect pod status, fetch logs, and execute commands inside running containers.

etcd: A distributed key-value store that Kubernetes uses as its source of truth for all cluster state, including pod assignments, Service endpoints, and configuration data.