Deployment and Pod / Node Selection

Node Selector

apiVersion: v1
kind: Pod
metadata:
  name: tom-web-pod
spec:
  nodeSelector:
    gpu: "nvidia"
  containers:
  - name: tom-web-container
    image: thomaspk/dragons

The yaml file above shows how nodeSelector is used to ensure that this pod will only be scheduled on nodes that have this key value pair.

Node Affinity

apiVersion: v1
kind: Pod
metadata:
  name: tom-web-pod
spec:
  containers:
  - name: tom-web-container
    image: thomaspk/dragon
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:   # Use these rules for pod scheduling
        nodeSelectorTerms:                        # Does not apply to already executing pods.
        - matchExpressions:
          - key: gpu
            operator: In                          # operator 'Exists' checks if key defined.
            values:
            - nvidia
            - amd

The yaml file show above does the same as the Node Selector yaml, except that it now uses affinity rules rathern than using node selection of labels. With this yaml definition, thomaspk/dragon image based pods can be scheduled on nodes that must have either nvidia or amd set in their gpu label. This is a hard requirement compared to using the preferredDuringSchedulingIgnoredDuringExectuion affinity rule.

Deployment file

apiVersion: apps/v1           # We have to specify the apps API group
                              # as it is not part of the core API group.
kind: Deployment
metadata:
  name: web-deployment        # The name used to reference this object (Deployment)
  labels:
    app: myapp-label          # The labels to apply to this object (Deployment)
    tier: frontend
spec:
  replicas: 2                 # Create two pods for this Kubernetes object (Deployment)
  selector:
    matchLabels:              # Tells deployment to match pods
      app: myapp-selector     #   of this label (myapp-selector)
  template:                   # Describes a pod template
    metadata:
      labels:
        app: myapp-selector   # Label the pods that are created
    spec:                     # Specifications for the pod(s)
      containers:
        - image: thomaspk/web
          name: myweb-container
          ports:
            - containerPort: 8080
              name: http
              protocol: TCP

An example of this deployment can be found at Documents/code/java/spring/boot/k8-app. We have to specify the pod specification in the deployment file so that Kubernetes can create these pods if they do not exist.

Rolling Updates

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deploy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-deploy
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 2
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx-deploy
    spec:
      containers:
      - image: nginx:1.16
        name: nginx

The example above shows how to create a rolling update deployment, specifying the maxSurge and maxUnavailable values. The nginx image can be updated to a new version with the command $ kubectl set image deployment/nginx-deploy nginx=nginx:1.17 --record

ReplicaSet file

ReplicaSets are a newer compared to ReplicationController, so we should be using ReplicaSets vs ReplicationControllers.

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: frontend
  labels:
    app: tom-web-app
    tier: frontend
spec:
  replicas: 3
  selector:           # This is a mandatory field for ReplicaSet.
    matchExpressions:
      - key: app
        operator: In
        values:
          - tom-app
    matchLabels:
      tier: frontend    # See section 4.3.2 of Kubernetes In Action.pdf
  template:
    metadata:
      labels:
        tier: frontend
        app: tom-app
    spec:
      containers:
      - name: tom-app
        image: thomaspk/web
        ports:
        - containerPort: 8080
          protocol: TCP

An example where multiple selectors are used. Here the ReplicaSet requires pods with app in tom-app and tier=frontend.
Note that the selector is a mandatory field which allows it to understand what pods fall under its jurisdiction. It also means that it can control pods that were not created as part of this yaml file definition.
The important definitions for a ReplicaSet are: replicas, selector (with MatchExpressions) and the pod template. The definition of the pod template is required, because if the required pod does not already exist, the ReplicaSet needs to know how to create the pod.

Health Probes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8-java-app
  labels:
    app: myapp-label
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp-selector
  template:
    metadata:
      labels:
        app: myapp-selector
    spec:
      minReadySeconds: 30           # Optional. Minimum time by which pod should be ready.
      containers:
        - image: thomaspk/web
          name: myweb-container
          livenessProbe:
            httpGet:
              path: /actuator/health  # The URL for kubernetes to do a health probe
              port: 8080
            initialDelaySeconds: 5    # Probe only after an initial delay of 5 seconds
            timeoutSeconds: 1         # Response must be received in 1 second (UP or DOWN)
            periodSeconds: 10         # Probe every 10 seconds
            failureThreshold: 3       # Consider failed if 3 consecutive DOWN responses
          ports:
            - containerPort: 8080
              name: http
              protocol: TCP

Quota on Computing Resources

Namespace Resource Quota

apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-ResourceQuota
  namespace: dev
sepc:
  hard:
    pods: "10"
    requests.cpu: "4"
    requests.memory: 5Gi
    limits.cpu: "10"
    limits memory: 10Gi

A Resource Quota provides constraints that limit resource consumption by Namespace. It is a way to divide resources in a shared cluster between groups of users.

Container level Quota

apiVersion: v1
kind: Pod
metadata:
  name: tom-app
  labels:
    app: tom-app
    tier: frontend
spec:
  containers:
  - name: tom-web-app
    image: thomaspk/fortune
    ports:
    - containerPort: 8080
    resources:
      requests:         # The minimum amount of resources required for this container
        memory: "1Gi"
        cpu: 1
      limits:           # The maximum amount of resources to be allocated to this container
        memory: "2Gi"   #   If the application uses more than this persistently,
        cpu: 2          #   it will be terminated by Kubernetes.

The yaml file above shows how requests and limits are specified at the container level.
Kubernetes will throttle the computing resources given to the container based on these specifications.

Default Limits & Requests

apiVersion: v1
kind: LimitRange
metadata:
  name: computing-limit-range
spec:
  limits:
  - defaults:
      cpu: 1
      memory: 512Mi
    defaultRequests:
      cpu: 0.5
      memory: 256Mi
    type: Container

The yaml file show above is used to set the default cpu and memory requests and limits for all containers that do not have these explicitly specified.

Taints and Tollerations

apiVersion: v1
kind: Pod
metadata:
  name: tom-web-app
spec:
  containers:
  - name: nginx
    image: nginx
  tolerations:                # This applies at the pod level
  - key: "department"         # instead of the individual container level.
    operator: "Equals"
    value: "Marketing"
    effect: "NoSchedule"      # Do not place pod on node without required taint
    tolerationSeconds: 3600   # Optional parameter.  Take effect in an hour

The above yaml pod definition shows how this pod meant for the marketing department will run on nodes tainted with the key value pair department=Marketing. The effect can also have a value of PreferNoSchedule for a soft preference of this rule or NoExecute to stop execution of existing pods without a matching toleration.