DDoS Detection, Mitigation, Orchestration, and Threat Intelligence
Consolidated Security & CGNAT
TLS/SSL Inspection
Web Application Firewall
Application Security & Load Balancing
Analytics & Management
CGNAT & IPv6 Migration
The recent AWS Route 53 DNS attack should make you consider who is responsible for availability when we outsource key services. The attack highlights how dependent businesses have become on our wide swath of communication service providers (CSPs) and how this reliance impacts our business availability. As with Route 53, hosting domain name system (DNS) services have been one of the fastest-growing areas of outsourcing. In a 2018 Harvard Library study, the authors pointed to the industry-wide risk associated with having 66 percent of domains entirely managed by a small number of external service companies.
We’re investigating reports of intermittent DNS resolution errors with Route 53 & our external DNS providers. We’re working towards resolution & will post updates here: https://health.aws.amazon.com/phd/status. 🚦— AWS Support (@AWSSupport) October 22, 2019
We’re investigating reports of intermittent DNS resolution errors with Route 53 & our external DNS providers. We’re working towards resolution & will post updates here: https://health.aws.amazon.com/phd/status. 🚦
— AWS Support (@AWSSupport) October 22, 2019
We saw the report’s risk assessment play out in the real-world with the AWS Route 53 DDoS attack, just as it did with the colossal 2016 Mirai IoT DYN DDoS attack. Failed or partially mitigated DNS attacks leave a wake of collateral damage on the rest of us innocent bystanders who are dependent on these services. Interestingly, both of these DNS attacks leveraged a common, yet challenging to defend, DNS attack strategy cleverly named “water torture.” We previously explained this attack strategy in our Investigating Mirai Botnet paper.
Attack Floods ISP recursive DNS servers with randomized queries to a victim base domain name, causing the ISP’s DNS servers to perform the attack on the victim’s authoritative DNS servers.
Water torture cyber attacks can be difficult to defend because attackers use volumes of fake queries that look real to DNS services. Attackers exploit the label structure of DNS’s fully qualified domain name (FQDN) to create legitimate-looking queries that exercise the whole domain name tree with resolution misses. In this attack, the query will have a legitimate first label, like a10networks.com managed by the victim’s designated authoritative server. A small pool of DDoS drones spray spoofed recursive requests to any of the millions of unwitting DNS resolvers exposed on the internet. These fake queries are composed of pseudo-randomized second, third, fourth, or up to 50th labels adding up to a total of 253 ASCII characters. That’s 253256 possible permutations; yikes, a big number.
These fake queries are randomized so that each request results in full DNS tree climb since the FQDN will not be in any of the DNS servers’ hit or miss cache. The whole process keeps going as long as the drones spew out more queries, or when one of the servers in the path falls over from resource exhaustion.We will discuss how to defend against “water torture” later in this blog.
Hopefully, we will find out the whodunit and their motivation in the AWS incident, but rarely are the providers the intended victims of the DDoS attack. Instead, the mark is usually a tenant downstream inside the CSP’s infrastructure that has caught the ire of an attacker. Apparently, the attacker saw DNS as a lower-effort alternative to a volumetric assault against AWS’s Shield volumetric defenses. It is natural for attackers to gravitate to the weakest link, and they will exploit weaknesses regardless of a direct or indirect path to the target. Unfortunately for the rest of us, indirect attacks have a large blast radius that absorbs everyone in the connected vicinity.
It is ironic. We outsource for many reasons. In some instances, we believe going to vendors, especially bigger ones, will give us better security, hoping that being a part of the masses – hiding in anonymity is better than going it alone. Unfortunately, being a part of the swarm has its consequences even when we are not the target.
AWS told customers, “…but these mitigations are also flagging some legitimate customer queries at this time.” Source: Securityweek.com
If you were one of those companies impacted by AWS Shield’s massive over response that blocked legitimate requests, you can get reimbursed for the lost availability but does that really overcome the downtime?
Figure 1: AWS Route 53 SLA
That level of compensation for your service downtime may seem a little underwhelming, but that is what we must accept when we relinquish availability to service companies.
It is a given that enterprises will continue to outsource IT functions to CSPs. AWS wasn’t perfect in its attack response, but it gets the importance of DDoS resilience. However, AWS isn’t your only vendor. Are your other CSPs taking enough responsibility for their subscriber’s defenses?
Unfortunately, many enterprises don’t have the resources, time, or people to operate their own DDoS defenses. The CSP is in the best position to defend its tenants. After all, it is the conduit to legitimate and attack-traffic delivery to your applications, services, and infrastructure. The CSP’s network edge is the right place to scrub the traffic, not downstream in our infrastructure, or intercepted by yet another service provider. If they do the right thing by taking the step to protect us, we might even be willing to help them build a business service that takes a bite out of the multi-billion-dollar cloud scrubbing industry. Interestingly, these types of services instill our loyalty,
“Since deploying intelligent, automated DDoS protection, Leaseweb also has seen an 11 percent reduction in support tickets. Leaseweb has improved the experience of both its customers and help desk agents, and lowered its OpEx.”, Bart van der Sloot, at Leaseweb Network.
However, that doesn’t mean we aren’t responsible for our businesses’ availability. When we outsource, we have to heighten our awareness of what constitutes an effective defense against our number-one asset, AVAILABILITY. With an understanding of threats and appropriate mitigation strategies, we are in a better position to ask the right questions of providers to assess how effective their defenses work in protecting the critical services you count on. As President Reagan said, “trust but verify.” Wise advice when your outsourced services matter.
DNS water torture attacks are not going away any time soon. For attackers, water torture is easy to initiate, and nothing inspires repetition like success.
For DNS authoritative servers to be resilient to water torture, the server has to be able to answer an infinite number of bogus queries without impeding legitimate requests. Infinite resources aren’t possible, and even AWS has limits, so let’s look at a different strategy. A10 Networks’ Thunder Threat Protection System (TPS) DDoS defense is one such alternative approach. Here is a partial list of some mitigation capabilities that can be used independently or in combinations to achieve DNS DDoS resilience
