Kubernetes for Developers#

Table of Contents#

1. 1. Developer Mental Model: App Runtime vs SRE Concerns
2. 2. Pods & Multi-Container Sidecars
3. 3. Deployments & Zero-Downtime Rolling Updates
4. 4. Services & Internal DNS Discovery
5. 5. Ingress & External HTTP Routing
6. 6. ConfigMaps & Secrets (12-Factor App)
7. 7. Health Probes: Liveness, Readiness & Startup
8. 8. Resource Requests, Limits & OOMKilled
9. 9. Developer Debugging & kubectl CLI Toolkit
10. 10. Graceful Shutdown & SIGTERM Handling

1. Developer Mental Model: App Runtime vs SRE Concerns#

Most Kubernetes tutorials drown software developers in cluster-level administration: provisioning etcd clusters, configuring Calico CNI networking overlays, tuning kube-scheduler algorithms, and configuring node autoscalers. As an application developer, you do not need to manage Kubernetes clusters. Your focus is the Application Runtime Contract.

Core Takeaway: Think of Kubernetes as a distributed operating system. A container is a process, a Pod is a virtual machine/sandbox, a Deployment is your process supervisor (like systemd), and a Service is internal DNS and load balancing.

2. Pods & Multi-Container Sidecars#

The Pod is the smallest deployable compute unit in Kubernetes. A Pod encapsulates one or more closely coupled containers that share:

1. Network Namespace: All containers in a Pod share the same IP address and port space. They communicate with each other over localhost (e.g. your app container calls a local Redis sidecar at localhost:6379).
2. Storage Volumes: Shared directories mounted across containers for log shipping, configuration sharing, or cache sharing.

Here is an example of a developer-friendly Pod specification running a web service alongside a sidecar proxy:

apiVersion: v1
kind: Pod
metadata:
  name: payment-service-pod
  labels:
    app: payment-service
    env: production
spec:
  containers:
    # Primary Application Container
    - name: payment-api
      image: registry.example.com/payment-service:v2.4.1
      ports:
        - containerPort: 8080
      env:
        - name: PORT
          value: "8080"
    # Sidecar Container (e.g., Log Forwarder or Envoy Proxy)
    - name: log-collector
      image: fluent/fluent-bit:2.2
      volumeMounts:
        - name: shared-logs
          mountPath: /var/log/app
  volumes:
    - name: shared-logs
      emptyDir: {}

3. Deployments & Zero-Downtime Rolling Updates#

In production, you never deploy standalone Pods directly because if a worker node crashes, bare Pods are not restarted. Instead, you declare a Deployment. The Deployment controller manages a ReplicaSet, ensuring the desired number of Pod instances are always running and orchestrating zero-downtime rolling updates.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
  labels:
    app: order-service
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1        # Spin up at most 1 extra pod during rollout (total 4)
      maxUnavailable: 0  # Ensure 3 pods are always healthy before killing an old one
  selector:
    matchLabels:
      app: order-service
  template:
    metadata:
      labels:
        app: order-service
    spec:
      containers:
        - name: order-api
          image: mycompany/order-service:1.2.0
          ports:
            - containerPort: 3000

Rolling Update Mechanics: When you update the image tag from 1.2.0 to 1.3.0, Kubernetes spins up a new pod with version 1.3.0. Only after its readiness probe passes does it route traffic to it and gracefully terminate an old 1.2.0 pod.

4. Services & Internal DNS Discovery#

Pods are ephemeral: their IP addresses change every time they are restarted or updated. A Service provides a stable IP address, stable DNS name, and round-robin load balancing across all matching Pods.

apiVersion: v1
kind: Service
metadata:
  name: catalog-service
spec:
  type: ClusterIP # Internal cluster-only IP
  selector:
    app: catalog-service # Matches labels on target pods
  ports:
    - name: http
      port: 80         # Port exposed by the Service
      targetPort: 5000 # Port your application listens on inside container

How DNS Works for Developers: Inside the cluster, any service can reach the catalog service simply by requesting:

http://catalog-service:80/api/items
# Or fully qualified domain name (FQDN):
http://catalog-service.default.svc.cluster.local/api/items

5. Ingress & External HTTP Routing#

While Services handle internal routing, an Ingress exposes HTTP/HTTPS routes from outside the cluster to your internal Services based on hostname or URL path.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - host: api.techtoday.click
      http:
        paths:
          - path: /users
            pathType: Prefix
            backend:
              service:
                name: user-service
                port:
                  number: 80
          - path: /orders
            pathType: Prefix
            backend:
              service:
                name: order-service
                port:
                  number: 80

6. ConfigMaps & Secrets (12-Factor App)#

Never bake configuration or credentials into your Docker image. The 12-Factor App methodology mandates decoupling configuration from code. In Kubernetes:

• ConfigMap: Stores non-sensitive configuration keys (log level, cache TTL, feature flags).
• Secret: Stores sensitive keys (API tokens, database passwords) base64-encoded or backed by vault integrations.

# ConfigMap Definition
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  LOG_LEVEL: "info"
  CACHE_ENABLED: "true"
---
# Injecting into Pod as Environment Variables
spec:
  containers:
    - name: web-app
      image: mycompany/web-app:latest
      env:
        - name: LOG_LEVEL
          valueFrom:
            configMapKeyRef:
              name: app-config
              key: LOG_LEVEL
        - name: DATABASE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: password

7. Health Probes: Liveness, Readiness & Startup#

Kubernetes needs to know when your application is healthy. Setting up proper probes prevents outages during rollouts and automatically recovers stuck processes:

1. Startup Probe: Checks if a slow-starting application (e.g. loading large AI weights or warming caches) has initialized. Other probes are disabled until this succeeds.
2. Readiness Probe: Checks if the container is ready to accept incoming traffic. If this fails (e.g. database pool saturated), Kubernetes stops sending requests to this pod without killing it.
3. Liveness Probe: Checks if the container process is alive. If this fails, Kubernetes kills and restarts the container.

readinessProbe:
  httpGet:
    path: /readyz
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5
  failureThreshold: 3

livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 15
  periodSeconds: 10
  failureThreshold: 3

Warning for Developers: Never connect to external dependencies (like databases or third-party APIs) inside your /healthz liveness probe! If the database is momentarily down, all your app pods will fail their liveness probe and crash simultaneously in a catastrophic restart loop.

8. Resource Requests, Limits & OOMKilled#

Every container should define resource requests (guaranteed reservation for scheduling) and limits (maximum allowed threshold):

resources:
  requests:
    cpu: "250m"      # 250 milli-CPUs = 0.25 vCPU core
    memory: "256Mi"  # 256 Megabytes
  limits:
    cpu: "1000m"     # Throttled if application consumes more than 1 core
    memory: "512Mi"  # OOMKilled (Exit Code 137) if exceeded!

9. Developer Debugging & kubectl CLI Toolkit#

These commands are the core developer diagnostic toolkit for troubleshooting issues in staging and production:

# 1. Inspect Pod Status & Restart Counts
kubectl get pods -l app=order-service

# 2. View Real-time Application Logs (like docker logs)
kubectl logs -f deployment/order-service --tail=100

# 3. View Previous Crashed Container Logs (critical for post-crash triage)
kubectl logs <pod-name> --previous

# 4. Detailed Event Diagnostics (reasons for ImagePullBackOff, CrashLoopBackOff)
kubectl describe pod <pod-name>

# 5. Execute an Interactive Shell inside a Running Container
kubectl exec -it <pod-name> -- /bin/sh

# 6. Forward Remote Service Port Directly to Localhost (test internal API locally)
kubectl port-forward svc/catalog-service 8080:80
# Now visit http://localhost:8080 in your local browser or Postman!

10. Graceful Shutdown & SIGTERM Handling#

When Kubernetes terminates a Pod (due to scaling down, rolling update, or node eviction), it follows a strict sequence:

1. The Pod is removed from the Service endpoints list (no new requests sent).
2. Kubernetes sends a SIGTERM signal to PID 1 inside the container.
3. Your application has terminationGracePeriodSeconds (default: 30s) to finish processing active in-flight HTTP requests and close database connections.
4. If the process is still running after the grace period, Kubernetes sends SIGKILL.

In Node.js, Python, or Go, always listen for SIGTERM:

# Python (FastAPI / Flask / Uvicorn) example
import signal, sys, time

def handle_sigterm(*args):
    print("Received SIGTERM: Draining connections and shutting down gracefully...")
    # Finish pending jobs and close db pool
    sys.exit(0)

signal.signal(signal.SIGTERM, handle_sigterm)