Takeaway: People, Process and Technology Shortcomings Equal Management Failures
Time for a fresh edition of “learn from how others get breached” focusing on Equifax.
The goal here is not blame, but rather to highlight specific missteps by an organization so that others can avoid making the same mistakes, hopefully making them less likely to fall victim to attacks.
Over the course of 76 days, attackers ran 9,000 queries against 51 databases, using encrypted communications to exfiltrate data, as well as their own remote desktop protocol and web shell software, together with leased Swiss servers as a staging area so that IP addresses didn’t trace back to China.
Another caveat is that although Equifax was allegedly hacked by nation-state attackers, all organizations should have solid defenses in place that make it difficult for any hackers to get in, be they part of a cybercrime gang or an intelligence agency, or a mercenary, hacktivist or bored teenager.
On to Equifax, which suffered a breach in 2017 that U.S. prosecutors say resulted in the theft of personally identifiable information for 145 million Americans. On Monday, the Justice Department unsealed an indictment charging four officers of the Chinese People’s Liberation Army with the intrusion (see: No Surprise: China Blamed for ‘Big Data’ Hack of Equifax).
Equifax: So Many Missteps
What went wrong at Equifax, giving attackers a way in – and leading to the ouster of senior managers and massive costs for the publicly traded firm? (See: Equifax’s Data Breach Costs Hit $1.4 Billion.)
Beyond details contained in the Monday indictment, a thorough, must-read report on the Equifax breach from the U.S. Government Accountability Office as well as subsequent House and Senate reports, plus the findings of the U.K. Information Commissioner’s Office – which imposed the maximum possible fine on the credit reporting agency – have highlighted that Equifax’s failings were many (see: Why Was Equifax So Stupid About Passwords?).
The short version of what went wrong is that beginning in March 2017, hackers found and then exploited an unpatched, critical Apache Struts flaw, using it to gain a beachhead inside Equifax’s network. Along the way, they found plaintext credentials being stored in text files, giving them administrator-level access to numerous databases. Over the course of 76 days, attackers ran 9,000 queries against 51 databases, using encrypted communications to exfiltrate data, as well as their own remote desktop protocol and web shell software, together with leased Swiss servers as a staging area so that IP addresses didn’t trace back to China.
Before the breach began, Equifax had allowed eight SSL certificates to expire, meaning that a tool it had for analyzing encrypted communications was not working. Once the security team renewed the certificates in August 2017, alarms began sounding, highlighting the malicious activity.
Did You Really Patch?
If there’s one thing that every organization should learn from the Equifax breach, it’s about patching.
“My key takeaway from this would be the need for organizations to gain assurance that their patch management process is working as expected,” says David Stubley, CEO at 7 Elements, a security testing firm and consultancy in Edinburgh, Scotland. “Basically, it’s all well and good knowing that you need to patch, but did you check that you actually did?”
Continuous vulnerability assessment and remediation is one of the cornerstones of the top 20 critical security control areas identified by the SANS Institute. “And for good reason, as unpatched critical flaws often offer a malicious actor a trivial route to gain a foothold,” Stubley tells me. “Without assurance activity focused on ensuring that patches have been applied in a timely manner, organizations are leaving themselves open and increasing the likelihood of a successful breach.”
Missing: ‘Solid Leadership and Processes’
What else should cybersecurity professionals take away from the Equifax breach? As a U.S. House of Representatives Committee on Oversight and Government Reform report into the breach noted, the breach “was entirely preventable.”
“Equifax had the staff they needed and the tools they needed. What they lacked were solid leadership and processes,” says Adrian Sanabria, an advocate at honeypot vendor Thinkst Applied Research, who counted 29 separate process and control failures at Equifax.
Source: Adrian Sanabria
“Most of the things on that list weren’t specific to stopping that one particular attack vector,” Sanabria says via Twitter. “They were general controls that would have caught/prevented a wide range of attacks.”
Notably, the organization had already invested in an array of security controls. As Stubley notes, Equifax even had a patch policy. But the leadership failed to ensure that the security team was following the processes and also understood them and their tools. That left attackers with a choice of errors to exploit.
They did a lot of things right. Unfortunately, it didn’t come close to balancing out what they were doing wrong.
— Adrian Sanabria (@sawaba) February 12, 2020
Still Essential: People, Process, Technology
A good IT program – as well as information security – is predicated on having the right people, processes and technology in place. From a security standpoint, Equifax failed on all three fronts.
As Daniel Bilar, the lead cybersecurity analyst at Visa, has stated: “There are management solutions to technical problems, but there are no technical solutions to management problems.”
There are management solutions to technical problems, but there are no technical solutions to management problems.
Insecure brittle code and recurrent rich attack surfaces are solely – not 50% not 80% but 100% – due to upper management (process) failures. Eg Agile & security.
— Daniel Bilar (@daniel_bilar) September 26, 2019
While cybercrime gangs and nation-state attackers might sound like they’re bringing different levels of resources to bear, attackers’ MO remains to find the easiest way in to any target.
So-called advanced persistent threat actors – again, nation-state groups – will keep on trying. But intelligence officials emphasize that if more organizations make it difficult for APT groups to get in, there will be fewer victims (see: Turla Teardown: Why Attribute Nation-State Attacks?).
Again, the impetus here isn’t to name and shame Equifax, but rather to drive other organizations to take a good, hard look at their own defense postures.
“They absolutely aren’t alone” when it comes to having made egregious IT mistakes, tweets British security researcher Kevin Beaumont (@Gossithedog). “A lot of it comes down to lack of IT resource, and legacy [systems].”
Timeline: Equifax’s Errors, Breach, Response
Based on the many reports about the Equifax breach as well as Congressional testimony, here’s a timeline of notable dates concerning Equifax’s data breach as well as its response.
- May 2016: Digital certificate for network scanning tool used by Equifax expires, leaving it unable to inspect encrypted traffic for signs of malicious activity.
- March 8, 2017: US-CERT issues alert about Apache Struts 2, advising all organizations to install a patch to fix a newly discovered vulnerability that would allow attackers to remotely execute commands and take control of the web application framework. Apache releases the patch on the same day.
- March 10: Equifax gets probed. “Unidentified individuals scanned the company’s systems to determine if the systems were susceptible” to the Struts flaw, GAO says. “As a result of this scanning, the unidentified individuals discovered a server housing Equifax’s online dispute portal that was running a version of the software that contained the vulnerability.” Using some type of apparently automated vulnerability exploit software, “the unidentified individuals subsequently gained unauthorized access to the Equifax portal and confirmed that they could run commands,” but stole no data, GAO says.
- May 13: Starting that day and lasting until July 30 – a nearly 80-day period – attackers queried 51 Equifax databases, running 9,000 queries that extracted “records containing the PII of at least 145.5 million consumers in the U.S. and nearly 1 million consumers outside of the U.S.,” GAO says, but doing so “in small increments to help avoid detection.” The attackers encrypted their communications to help disguise their activities.
- July 29: After obtaining a new digital certificate for a tool that scans encrypted network traffic for signs of malicious activity, Equifax’s security team detects unusual activity and blocks it.
- July 30: Equifax’s security team detects further unusual activity and takes the Apache Struts portal offline.
- Aug. 2: Equifax hires cybersecurity firm Mandiant to investigate the breach and alerts the FBI.
- Aug. 7: Equifax issues first public data breach notification.
- Sept. 7: The company launches “www.equifaxsecurity2017.com” website, after having registered it on Aug. 22, to handle consumers’ queries. The website, because it is not hosted on the official equifax.com domain, gets mistaken as a phishing site by some security firms. Equifax says it believes 143 million U.S. consumers’ PII was stolen, including dispute documents for 209,000 consumers, which contained PII for approximately 182,000 consumers. It says PII for U.K. and Canadian consumers was also exposed.
- Sept. 15: Equifax’s CIO and CSO “retire.”
- Sept. 26: Richard Smith, Equifax’s CEO, also “retires.” Smith later appears on Capitol Hill to answer extensive questions from lawmakers about the breach.
- Sept. 28: Equifax interim CEO Paulino do Rego authors an op-ed in the Wall Street Journal apologizing for the breach and promising stronger consumer protection services.
- Oct. 2: Equifax wraps up its initial investigation, reporting that investigators have found that attackers also accessed PII for 2.5 million more U.S. consumers, and revising U.S. breach victim count from 143 million to at least 145.5 million.
- Feb. 12, 2018: Equifax announces the hiring of a new CISO: Jamil Farshchi, who comes from Home Depot.
- March 1: Equifax identifies about 2.4 million U.S. consumers whose names and partial driver’s license information were stolen. It says some of these individuals were already included in the count of 145.5 million breach victims.
- Sept. 20: The U.K. ICO slams Equifax with the maximum possible penalty for violating British privacy law.
- May 10, 2019: Equifax says it has spent $1.4 billion as a result of the breach, counting all of its incident response, legal, investigative and corporate information security overhaul costs.
- July 22: Equifax agrees to pay $175 million to 48 states, the District of Columbia and Puerto Rico, as well as $100 million to the Consumer Financial Protection Bureau in civil penalties, as part of a deal also negotiated with the Federal Trade Commission.
- Jan. 13, 2020: A federal judge gives final approval to a settlement over a U.S. class-action lawsuit filed against Equifax. The minimum cost to Equifax of the agreement will be $1.38 billion, which includes $1 billion in security upgrades, as well as free credit monitoring for victims.
- Feb. 10: U.S. Justice Department unseals an indictment against four Chinese military officers, accusing them of perpetrating the Equifax hack.
Red Star Hacking
Learning from the above is not an academic concern.
“For years, we have witnessed China’s voracious appetite for the personal data of Americans, including the theft of personnel records from the Office of Personnel Management, the intrusion into Marriott hotels and Anthem health insurance company, and now the wholesale theft of credit and other information from Equifax,” Attorney General Bill Barr said in a Monday press conference. His remarks appear to be the first time that the White House has publicly attributed the Marriott breach to China.
“Wu Zhiyong, Wang Qian, Xu Ke and Liu Lei face charges of computer fraud, economic espionage, and wire fraud for their role in one of the largest thefts of personally identifiable information by state-sponsored hackers ever recorded,” https://t.co/KcZ8lOfpbd pic.twitter.com/65vDyh4HTx
— FBI (@FBI) February 10, 2020
Such attacks continue, even if many of them never come to light, at least via indictments. Dmitri Alperovitch, CTO of U.S. cybersecurity firm CrowdStrike, notes that the indictment unsealed this week represents “one of the few publicly disclosed cyber actions from PLA” since the 2015 cyber agreement that the Obama administration reached with Beijing. Experts say that agreement appears to have gone by the wayside since President Donald Trump took office three years ago, perhaps due to the U.S.-China trade war.
“Most activity in recent years has emanated from MSS and their contractors,” Alperovitch says, referring to China’s Ministry of State Security.
Officials say China continues to commission hack attacks to steal intellectual property and fuel the country’s ambitious expansion plans. In the case of PII, intelligence experts say Chinese spies could use it to out U.S. agents and accomplices or hone targets for blackmail. To date, even well-resourced organizations with large security programs, such as Equifax, have not been immune to their efforts. But based on how Equifax got its defenses wrong, hopefully others can now better prepare themselves to repel such attacks.