Container#

A container is an isolated environment that includes dependencies and resources for an application to run properly unlike virtual machines. Containers don’t include the kernel and use a host kernel, hence making them lightweight.

POD#

In K8s, containers are part of a POD and there can be multiple Pods. The POD handles resources and communications of the containers. By default, containers in the same pod can communicate with each other. However, we can restrict that using an Network policies.

Node#

There are 2 types of nodes in a K8s cluster

• Master Node
• Worker Node

Master Node

The master node hosts the K8s control plane which serves as a management layer for K8s and is made up of multiple system processes. It manages things such as activities within the cluster and the overall state of the cluster as well as Schedule workloads and makes sure a desired configuration is maintained.Control plane is a collection of system processes that manage the cluster such as :

• Kube-Apiserver: Exposes the k8sAPI; runs on port 6443
• Kube-Scheduler: Assigns Pods to nodes based on resources and constraints; runs on port 10251
• Kube-Control-Manager: Runs controllers to maintain cluster state; runs on port 10252
• Cloud-Controller-Manger: Manages cloud specific integrations
• Etcd : Distributed key-value store for cluster data (often runs as a Pod instead of a system process); runs on port 2379 (client) and 2380 (peer communication)

Worker Nodes

Worker nodes execute the actual applications, and they receive instructions from the master nodes and ensure the desired state is achieved.

NOTE

When we install K8s on a system, it is considered a node.

Example graph of a worker node which includes 2 pods :

Cluster#

Everything in k8s runs inside the cluster (system process, nodes etc .) and the api acts as a gateway between the outside/inside environment unless something is deliberately exposed via services, for example.

Cluster graph

In the graph below is shown a cluster which includes a master node and multiple worker nodes (which include multiple pods.

Objects#

The API resource serves as an endpoint that houses a specific collection of api objects. In k8s api these objects include essential elements such as :

• Pods
• Services
• Etc…

Operations#

There are Operations that we can perform on the objects avaliable such as :

• GET Retrieves information or list resources
• POST Creates a new Resource
• PUT Updates an existing Resource
• PATCH Applies partial updates to a resource (changes)
• DELETE Removes a resource

Communication#

We communicate with the API by sending an API Call and receiving an API Response. When we communicate with k8s api using a tool such as kubectl to list available pods, for example, it can look like this under the hood :

GET /api/v1/namespaces/default/pods HTTP/1.1
Host: <API-SERVER-IP>:6443
Authorization: Bearer <JWT-TOKEN>
Accept: application/json

The application receives the instructions and responds with the results .

{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": {"resourceVersion": "12345"},
  "items": [
    {
      "metadata": {
        "name": "nginx-pod",
        "namespace": "default",
        "uid": "a1b2c3d4"
      },
      "spec": {
        "containers": [{"name": "nginx", "image": "nginx:latest"}]
      },
      "status": {
        "phase": "Running",
        "podIP": "10.244.0.2"
      }
    }
  ]
}

NOTE

We don’t have to specifically use kubectl in order to communicate with K8s api. We can also use tools such as curl to do the same. What matters is the format of the request, authentication and client validation.

Authentication#

K8s supports various methods for authentication, such as Client certificate and Bearer Tokens using authentication proxy or HTTP to verify user’s identity. Once the user is authenticated, K8s enforces authorization decisions using RBAC (Role-based access control). This technique involves assigning specific roles to Users/Groups or a process to determine which actions they can perform on which available resources.

Anonymous Requests

Every worker node in K8s runs a kubelet, a Mini-API usually on port 10250that accepts the request from the K8s API. In older k8s distributions (prev1.14), Kubelet is allowed anonymous access by default, meaning we can bypass the K8s api (Hence its verification and authentication rules) and directly communicate with a worker node. Any request made to the Kubelet without a valid client certificate is considered an anonymous request.

If anonymous access is allowed, it can be an issue because a user/process that can reach the kubelets APIcan make requests and receive responses which can reveal sensitive information.

Example of a request made to a kubelet to get list of pods using curl :

curl -k https://<NODE-IP>:10250/pods 

Anonymous request allows us to :

• List all pods
• Read container logs
• Bypassing K8s RBAC
• Cluster Reconnaissance

And if the kubelet is misconfigured to allow the exec feature anonymously, even RCE. To check if we are allowed to do remote code execution in a pod, we can use kubeletctl, a client for kubelet.

Minikube#

We will utilize minikube to setup a local testing environment. Its perfect for personal use due to being lightweight and fast.

Installation

1. We get the minkube image

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64

2. Install it using install

sudo install minikube-linux-amd64 /usr/local/bin/minikube

3. Execute minikube to start the cluster

minikube start --driver=docker

Setting up nginx

1. Verify minikube is working using kubelctl get pods and then we deploy nginx.

kubectl create deployment nginx --image=nginx:latest

2. Make it accessible

kubectl expose deployment nginx --port=80 --type=NodePort

3. Verify that nginx is running

minikube service nginx

Now we can simply go to the link provided in the results and see nginx web-server .