Kubernetes has exploded onto the technology scene over the last couple of years, as many organizations, including major cloud companies, adopted it as the default way to orchestrate and scale their container-based workloads. But as with any new, rapidly developing technology, questions remain around how best to secure those Kubernetes deployments.
And as IT has become more interested in deploying with Kubernetes, we’ve started to see an increase in attacker interest in compromising Kubernetes clusters. If you want to avoid becoming the next data breach statistic, you must think like a hacker when securing your system. Here's how.
Lesson one: You’re always visible on the Internet
The first thing to realize when thinking about securing cloud-based Kubernetes clusters is that hackers can find you on the Internet with great ease. When you’re spinning up a development or test system, it may feel as if there’s no need to focus too much on security, since you’re not really advertising the system's presence. But with tools such as Shodan out there, it’s trivial for attackers to locate promising targets.
So it's easy for attackers to identify Kubernetes clusters, since they usually listen on a range of well-defined and somewhat distinctive ports. A good example of this is etcd, which Kubernetes uses as its cluster database. It listens on port 2379/TCP, which is indexed by Shodan, and so easily found.
[Special Coverage: KubeCon/CloudNativeCon]
A recent analysis shows that etcd services are turning up on the Internet without authentication. Clearly, this is a problem to consider when setting up your clusters, since an attacker who can successfully attack your cluster database can probably compromise your entire system.
Lesson two: Don't let them just stroll in through the front door
The Kubernetes API service acts as the front door to any cluster. It is generally exposed on every deployment, since it’s needed for management purposes. That's why ensuring its security is so important.
Luckily, most Kubernetes deployments provide authentication for this port. But it’s still possible to expose it inadvertently, as Tesla found out when it exposed the dashboard that forms part of it main Kubernetes API service to the Internet without authentication.
The other way you can expose this is if the “insecure API service” is enabled. As the name suggests, this isn’t something you want to expose on an untrusted network. But we’ve seen this happen before, so it’s definitely something you should check.
Lesson three: You have a host of ports to worry about
It’s not just the etcd service and Kubernetes API that you need to be cautious about leaving exposed. Depending on how your cluster is configured, other services might cause you trouble if a hacker gains access to them.
The Kubelet API, used by Kubernetes to manage containers on individual nodes and in some clusters, is available unauthenticated. That allows attackers to execute code in your containers and possibly take over your entire cluster.
This isn’t a new issue; the “Kubelet exploit” has been around since 2016. But recently we’ve seen evidence of attackers using it to compromise clusters. It's very much an active form of attack.
Lesson four: A compromised container equals a compromised cluster
You use Kubernetes to host sets of application containers, so it’s important to ensure that a compromise of one application running on the cluster doesn’t turn into a full-blown cluster compromise.
There are a few things to watch out for here, since attackers could take advantage of some of your cluster settings to escalate their privileges.
The default behavior of many Kubernetes clusters (where a token that provides access to the Kubernetes API mounts into each container) can cause security issues, especially if the token has cluster admin rights. This happens where role-based access control (RBAC) isn’t configured. In this configuration, an attacker who gains access to a single container in the cluster can easily escalate these privileges to gain full control of it.
Given that, it’s well worth locking this down as part of any production deployment. Ideally, not mounting a token at all is your best approach from a security perspective. But if you really need tokens, restrict the rights they have to cluster resources.
The other area to watch for here is the use of “privileged” containers. A container running as privileged essentially disables the security mechanisms provided by Docker and allows code to run on the underlying system. So whenever possible, avoid allowing any parts of your cluster to run with this setting.
Now go lock it down
Kubernetes is a rapidly developing technology that's also quite complex, so secure default settings are vital to deploying it safely. As you go forward, follow the advice above, make sure that you understand what you’re exposing, and follow good security practices when deploying your clusters. And for more on ensuring your cluster’s security I recommend the CIS Benchmark for Kubernetes.
For more on the cloud-native enterprise, see my keynote presentation, "A Hacker's Guide to Kubernetes and the Cloud," at KubeCon/CloudNativeCon on May 2-4 in Copenhagen. TechBeacon readers receive a 20% discount when they enter code KCCNEUTB.
Keep learning
Choose the right ESM tool for your needs. Get up to speed with the our Buyer's Guide to Enterprise Service Management Tools
What will the next generation of enterprise service management tools look like? TechBeacon's Guide to Optimizing Enterprise Service Management offers the insights.
Discover more about IT Operations Monitoring with TechBeacon's Guide.
What's the best way to get your robotic process automation project off the ground? Find out how to choose the right tools—and the right project.
Ready to advance up the IT career ladder? TechBeacon's Careers Topic Center provides expert advice you need to prepare for your next move.
