Researchers: Admin Shortcut May Have Opened the Door to Hackers
Microsoft’s Azure Security Center has detected a new hacking campaign targeting the Kubeflow platform on Kubernetes and uses the XMRig cryptominer to mine for monero across multiple clusters.
While Kubernetes has previously been targeted in various cryptomining operations, this is the first time Kubeflow was singled out as the attack vector, according to Microsoft.
The cryptomining campaign was first spotted in April when an image from a public repository was found on many separate Kubernetes clusters. An examination of the image found it was, in fact, running an XMRig cryptominer, which has been tied to similar attacks using botnets (see Kingminer Botnet Targeting SQL Servers for Cryptomining).
Code showing how hackers targeted Kubeflow with a cryptominer (Source: Microsoft)
“Looking at the various clusters that the above image ran on showed that most of them run Kubeflow. This fact implies that the access vector in this attacker is the machine-learning framework,” Yossi Weizman, security research software engineer at the Azure Security Center, notes in the report.
Kubeflow is an open-source project designed to make deployments of machine learning workflows on Kubernetes simple, portable and scalable. Since it is a containerized service, any attacker who gains access will have multiple ways to run their malicious image in the cluster, according to the report.
Weizman believes an admin’s attempt to make the setup more convenient may have exposed the dashboard to the internet, thus giving the attacker a pathway into the service. In this case the dashboard was exposed by the Istio ingress gateway, which is by default accessible only internally to the internet, Weizman notes.
“By exposing the service to the internet, users can access to the dashboard directly. However, this operation enables insecure access to the Kubeflow dashboard, which allows anyone to perform operations in Kubeflow, including deploying new containers in the cluster,” Weizman says.
This is not the first time this particular attack vector has been used, as the Azure Security Center has detected earlier instances when hackers used an exposed service to the internet as a gateway. Never before, however, has Kubeflow itself been targeted, Weizman notes.
Weizman has several recommendations to protect against this type of attack:
- Authentication and access control to the application;
- Monitor the public-facing endpoints of the cluster. Make sure that sensitive interfaces are not exposed to the internet in an unsecure method;
- Regularly monitor the runtime environment. This includes monitoring the running containers, their images and the processes that they run;
- Allow deployments of only trusted images and scan your images for vulnerabilities. The allowed images in the cluster can be restricted by using Azure Policy.
This is the second case of cryptomining malware being found in a Kubernetes cluster in the past few months. In April the Azure Security Center detected a new cryptomining campaign that targeted Kubernetes environments. At the time the attack scaled quickly so that within two hours a container with the code was deployed on tens of Kubernetes clusters.