Threat Alert: Market-First Container Image Built to Attack Kubernetes Clusters

Aqua’s Team Nautilus has uncovered a container image that, for the first time, allows bad actors to find and exploit vulnerabilities in Kubernetes clusters. The attackers propagate this malware through a Docker Hub lookalike account intended to dupe developers into downloading malicious images. To the best of our knowledge, this is the first known case where cyber attackers use this method to exploit cloud native stacks by specifically targeting weaknesses in Kubernetes clusters.

Using Aqua’s Dynamic Threat Analysis (DTA) scanner, we found 11 malicious container images in a Docker Hub account (a complete list of images and IoCs provided below). We believe that attackers created this account in order to masquerade as legitimate software providers, and have unsuspecting developers download and use it as they would normally.

Anything but normal: How these malicious container images strike

The Container image entrypoint is set to execute DEEPCE, an Offensive Security Tool (OST) designed to attempt escaping the containerized environment and taking a hold of the host, which means this container image is designed to run in the target’s environment. The image also contains other OSTs that allow the attacker to conduct further malicious attacks against Kubernetes clusters, such as detecting vulnerabilities in API server, pods, kublet API, Docker API, Docker.sock, etc.

Under closer inspection, we found TeamTNT, again

Initially, our researchers detected a suspicious Docker Hub account – portaienr. Based on experience, our hunch was that TeamTNT created this account in order to masquerade as a legitimate Docker Hub account and software (Portainer). The account uses typosquatting, which relies on common mistakes (e.g., typos) by users when keying in a website address. So, anyone who may misspell portainer as portanier is sent to the attacker’s account and duped into pulling malicious images.

After closely examining the artifacts, we concluded that the account did belong to TeamTNT. We based our conclusion on several images that download malicious components from TeamTNT’s C2 server (teamtnt[.]red). Additionally, most images were using binaries commonly applied by TeamTNT in previous campaigns. For instance, bioset (MD5= 283e0172063d1a23c20c6bca1ed0d2bb) serves as a bind shell that allows the attacker to connect to the host after the malware deployed to the container, and ziggy (MD5= 03647c6ae05aa8e4475d8ade50016385) — better known as Tsunami Malware, which is TeamTNT’s weapon of choice.

To bolster our assertions, we decided to conduct a short OSINT investigation (a multi-method approach) in order to search for other attacks coming from this Docker Hub account.

OSINT investigation revealed other victims

We noticed a post on Reddit from someone who claimed that his Virtual Private Server (PVS) was recently attacked by TeamTNT. He said that his VPS provider reported that his hosting company had complained that malicious traffic was originating from his server’s IP address. Following a quick inspection, he discovered a new user (Hilde) and SSH keys on his machine. In addition, he found a malicious shell file (Docker MassAutoInfect V1.2 by (c) 2020 by hilde[@]TeamTNT[.]red), which correlates with other TeamTNT malicious scripts.

This victim further claimed that he was using Portainer and that he might have “made some mistakes” with Docker API or Portainer configurations. He also claimed that he had exposed, for no reason, the docker/sock for many containers. Founded on the Reddit user’s details, we learned that the VPS freely allowed inbound traffic to all ports, and port 2375 in particular. Based on the evidence provided by the victim, we surmised that by using Masscan or Shodan engine, TeamTNT’s crypto worm automatically detected that port 2375 was open to the world and deployed and ran the container image “portaienr/allink:latest” on the VPS.

Next, we focused our attention on one image in particular (portaienr/bobedpei:latest), since it was packed with self-crafted and open-source tools that were designed to find and exploit vulnerabilities in Kubernetes and Linux Container (LXC) environments. Further analysis showed that this image contained interesting binaries and scripts designed to enable an orchestrated attack against Kubernetes clusters. Based on the entrypoint commands and the scripts embedded in the image, only some of the tools would be used during an attack. We believe the attackers adds scripts as needed (downloaded from a remote source) to attack specific targets. This is also a known technique by TeamTNT when they attack in the wild.

Analyzing the container image bobedpei

Let’s start by breaking down this image into its components, based on the artifacts that were detected by DTA:

Scanning for Kubernetes vulnerabilities

The image contains a binary file and downloads a Python package designed to detect vulnerabilities in Kubernetes clusters.

• Kube-hunter Python package is Aqua’s open-source pen-testing tool. That’s right, our own tool designed to help legitimate security folks hunt for security weaknesses in their Kubernetes clusters has been repurposed for malicious purposes by cyber attackers.
• Peirates is another Kubernetes penetration tool that enables an attacker to escalate privilege and pivot through a Kubernetes cluster. It automates known techniques to steal and collect service accounts, obtain further code execution, and gain control of the cluster. This tool is found in the binary pei (MD5= 018d88b8203bdea0fe4dc5b4baa930c4).

Attacking the Containers

The image includes a couple of tools that are designed to detect and exploit vulnerabilities in containers.

• Bob (Break Out the Box) is used to exploit common container vulnerabilities, post-exploitation techniques, identify Kubernetes service account secrets and attempt to use them, identify metadata services endpoints, find and identify UNIX domain sockets, identify UNIX domain sockets which support HTTP, etc. This tool is found in the binary bob (MD5= 342ac1027703123bfa9099bb18335871).
• DEEPCE (Docker Enumeration, Escalation of Privileges and Container Escapes) is another tool which can collect data by performing enumeration on the environment (e.g., container name and ID, IP addresses and DNS server, docker version, mounts, passwords and sensitive files, environment variables, other containers on the network). This tool is also set to perform privilege escalation and Docker escape. The image also contains two scripts designed to execute deepce.sh The script docker.sh is designed to run deepce.sh on Docker containers, while lxc.sh is designed to execute deepce.sh on Linux containers.
• ed is used to identify and exploit accessible UNIX domain Sockets and Docker.sock in particular. Once detected, ed attempts to exploit the socket to escape the container. ed is a binary (MD5= fd3c237e7da41539eb0506810b7ec238).

Network manipulation tools:

The image contains legitimate network manipulation tools that can be abused by the attacker in order to exploit vulnerable network connections.

• Scapy is a packet manipulation tool for computer networks, that can forge or decode packets, send them on the wire, capture them, and match requests and replies. It can also handle tasks like scanning, tracerouting, probing, unit tests, attacks, and network discovery.
• UTscapy is also imported to allow adversaries to launch scapy campaigns. We speculate these tools are present to enable attackers to conduct an efficient attack within a dense environment of containers, namely Kubernetes.

A possible orchestrated attack against Kubernetes cluster

Although we have not identified a specific script that executes the attack, based on the tools stored in this container image, we can infer that TeamTNT can download scripts from remote sources and modify their attack according to the target. Based on the tools in the image, we can suggest several possible attack vectors (from inside and outside of the cluster) that can be performed on a Kubernetes cluster as it appears in the figure below:

Initial access

The tools in this container image (bobedpei) can detect possible initial access points:

• Misconfigured ports that enable anybody to write commands to the APIs. These tools scan default ports to identify any misconfigurations. For instance, open to the world Docker API running on port 2375, or misconfigured etcd API on port 2379.
• Misconfigured K8s Dashboard enables unauthenticated login.
• Misconfigured K8s API Server enables unauthenticated login.
• Misconfigured etcd database enables anyone to make actions in the database. For instance, allowing Read and Write to etcd DB.
• In all Kubernetes versions before v1.10.11, v1.11.5, and v1.12.3, authentication API-Server can be exploited based on CVE-2018-1002105.

Discovery

In addition to allowing initial access, this container image (bobedpei) enables extensive discovery possibilities:

• K8s Dashboard
• K8s API Server
• K8s etcd
• K8s pods
• Docker Socket
• Collecting information about container files, network, and interfaces, which can be further exploited for credential theft
• Network Scanning
• Provides Metadata (i.e., AWS, GCP)

Privileged escalation and lateral movement

Once the Kubernetes cluster and initial access are discovered, several tools allow further exploitation of a Kubernetes cluster:

• In case of SA permissions running on a pod, an attacker can exploit K8s API Server to:
  • Increase permissions by obtaining service account tokens
  • Running code on the pod
  • Using Kubectl exec for reverse shell
• An attacker can perform DNS or ARP Spoofing to hijack another pod’s networks connections
• An attacker can exploit /var/log host mount, so the pod can traverse read files on the host machine
• In older Kubernetes versions, stealing service account tokens by abusing open Kubelet API
• Stealing IAM credentials by scraping Provider Metadata (AWS, GCP)
• In GCS environments, an attacker can pull service account (K8s) tokens from bucket storage
• Using RunCPawn vulnerability (CVE-2019-5736) an attacker can attempt to perform a container escape
• An attacker can exploit a privileged container to create a new root user on the host operating system
• An attacker can escalate to root via membership to the docker group on a host and run a custom payload

Impact

In all Kubernetes versions older than v1.11.8, v1.12.6, and v1.13.4, an attacker can launch a Denial of Service attack against the API-Service by exploiting CVE-2019-1002100, if the user is allowed to make API patch requests.

Protecting your K8s cluster

We’ve uncovered how TeamTNT has expanded its attacks against cloud native stacks using a first-ever approach. By using a container image packed with tools to detect and exploit vulnerabilities in a Kubernetes cluster, and a Docker Hub account to host this image, it’s able to masquerade as legitimate Kubernetes software and trick innocent users into downloading malicious container images.

And this isn’t a theoretical exercise, nor is it an attack against an inconsequential honeypot. We have evidence of a real-life attack by this Docker Hub account. We also confirmed an actual attack in the wild by TeamTNT after they exploited a misconfigured Docker daemon.

To protect your K8s clusters, we recommend being very careful about the images you download from public registries — and make sure they are actually the images you wanted, also:

1. Scan images for Kubernetes vulnerabilities, and if you frequently download images from public sources, use a tool like DTA to find hidden malware.
2. Be sure to update your Kubernetes version — previous versions would have more known vulnerabilities.
3. Harden your clusters using kube-bench, kube-hunter, and Aqua KSPM.
4. Monitor your running workloads and resource use for suspicious behaviors and patterns (Aqua’s runtime protection does that automatically with features like Drift Prevention.

If you’d like to learn more security steps to defend your Kubernetes clusters, go to Holistic Kubernetes Security.