Posted under: Research and Analysis
After many decades as security professionals, it is depressing to have the same issues repeatedly. It’s kind of like we’re stuck in this hacker groundhog day. Get up, clean up after stupid users, handle a new attack, fill out compliance report, and then do it all over again. Of course, we all live in an asymmetrical world when it comes to security. The attackers only have to be right once, and they are in your environment. The defenders only have to be wrong once, and the attackers also gain a foothold. It’s not fair, but then again, no one said life was fair.
The most basic advice we give to anyone building a security program is to make sure you do the fundamentals well. You remember security fundamentals, right? Visibility for every asset. Maintain a strong security configuration and posture for those assets. Patch those devices efficiently and effectively when the vendor issues an update. Most practitioners nod their head about the fundamentals and then spend all day figuring out how the latest malware off the adversary assembly line works — or burning a couple of days threat hunting in their environment. You know, the fun stuff. The fundamentals are just… boring.
The fact is, the fundamentals work, not for every attack but a lot of them. So we’re going to provide a reminder of that in this series we are calling Infrastructure Hygiene: The First Line of Security. We can’t eliminate all of the risks, but shame on us if we aren’t making it harder for the adversaries to gain a foothold in your environment. It’s about closing the paths of least resistance and making the adversaries work to compromise your environment.
We want to thank our pals at Oracle for potentially licensing the paper. We appreciate a company that is willing to remind its folks about the importance of blocking and tackling instead of just focusing on the latest, shiniest widget.
Let’s start the discussion with a fundamental question. Why focus on infrastructure? Aren’t apps attacked as well? Of course, adversaries attack applications, and in no way, shape or form are we saying that application security isn’t critical. But you have to start somewhere, and we favor starting with the foundation, which means your infrastructure.
Your tech stack’s main components are networks, servers, databases, load balancers, switches, storage arrays, etc. We’re not going to focus on protecting devices, applications, or identity, but those are important as well.
We’d be remiss not to highlight that what is considered infrastructure will change as more of your environments move to the cloud and PaaS. If you’ve gone to immutable infrastructure, your servers are snippets of code deployed into a cloud environment through a deployment pipeline. If you are using a PaaS service, your service provider runs the database, and your maintenance requirements are different.
That’s one of the huge advantages of moving to the cloud and PaaS. It allows you to abuse the shared responsibility model, which means that you contract with the provider to handle some of the fundamentals, like keeping up with the latest versions of the software and ensuring availability. Notice we said some of the fundamentals, but not all. Ultimately you are on the hook to make sure the fundamentals happen well. That’s the difference between accountability and responsibility. The provider may be responsible for keeping a database up to date, but you are accountable to your management and board if it doesn’t happen.
Many Bad Days
As we go through the lists of thousands of breaches over the years, quite a few resulted from misconfigurations or not fixing known vulnerabilities. We can go back into the Wayback Machine to see a few examples of the bad things that happen when you screw up the fundamentals.
Let’s dig into three specific breaches here because you’ll get a good flavor of the downside of failing at infrastructure hygiene.
Equifax: This company left Internet-facing devices vulnerable to the Apache Struts attack unpatched, allowing remote code execution on these devices. The patch was available from Apache but didn’t apply it to all the systems. Even worse, their Ops team checked for unpatched systems and didn’t find any, even though there were clearly vulnerable devices. It was a definite hygiene fail, resulting in hundreds of millions of user identities stolen. Equifax ended up paying hundreds of millions of dollars to settle their liability. That’s a pretty lousy day.
Citrix: When a major technology component is updated, you should apply the patch. It’s not like the attackers don’t reverse engineer the patch to determine the vulnerabilities. This situation was particularly problematic in the event of a Citrix hack early 2020 because the attackers could do automated searches and find vulnerable devices. And the attackers did. Even more instructive were the initial mitigations suggested by Citrix instead of a patch weren’t reliable nor implemented widely within their customer base, leaving many organizations exposed. At the same time, widely distributed exploit code made it easy to exploit the vulnerability. Once Citrix did issue the patches, customers quickly adopted the patches and largely shut down the attack. Patching works, but only if you do it.
Target: The last example we’ll use is the famous Target breach from 2013. It’s an oldie but highlights the challenge extends beyond your infrastructure. If you recall, Target was compromised through an unpatched 3rd party vendor system, allowing the attackers to access their systems. It’s not enough to get your hygiene in order. You also need to scrutinize the hygiene of any external organization (or contractor) that has access to your network or systems. Target paid tens of millions of dollars to settle the claims and dealt with significant brand damage.
We don’t like poking at companies that have suffered breaches, but it’s essential to learn from these situations. And if anything, infrastructure hygiene is getting more complicated. The SolarWinds attack from late 2020 was an example where even doing the right thing and patching the tool ended up providing an entry to attackers. If you looked at that situation in a bubble, you might have asked, “why bother patching?”
That question indicates you learned the wrong lesson. Go back up a few paragraphs where it says, “you can’t eliminate every risk.” Supply chain attacks will happen, and candidly there isn’t much you can do about them besides focusing on detection and monitoring. But not patching a component opens you up to anyone with the exploit. Patching may expose you to a sophisticated supply chain attack only available to a handful of adversaries (typically nation-states). We play the odds and patch to make it difficult for them.
Not an Option
After all the proof of downside above, let’s say you are still resistant to practicing good infrastructure hygiene. Don’t take it from us; listen to your auditor because they will find (and report) all sorts of deficiencies if you can’t keep things configured strongly and patched. Let’s highlight a few regulatory mandates that require patching.
- PCI: Requirements 2, 6 and 11 mention patching.
- ISO 27001: Control A.12.6.1 deals with remediating vulnerabilities (which means patching).
- NIST SP 800-53 R3: The Configuration Management (CM), Risk Assessment (RA), and System and Information Integrity (SI) highlight the need to patch the infrastructure.
So you don’t have an option, do you?
Are you sold yet? Good. But if it were easy, everyone would do it and do it well. Maintaining strong infrastructure hygiene isn’t easy, so in the next post, we’ll dig into the how of infrastructure hygiene and help you build a consistent program to ensure you get it all done.