AI vs. WAF: Cloudflare's Frontier Test Results

AI-Powered WAF Defense: A New Frontier

Cloudflare's blog post offers a compelling demonstration of how frontier AI models can be leveraged to dynamically test Web Application Firewalls (WAFs). The adaptive loop, where an LLM iteratively modifies attack payloads based on WAF responses without direct access to internal WAF logic, is a significant advancement over traditional static and dynamic testing methods. This approach mimics the adversarial nature of real-world attacks more closely, enabling the identification of subtle bypasses that might be missed by rule-based systems. The emphasis on human review and triage for non-blocked requests is crucial, ensuring that identified vulnerabilities are actionable and not just noise. The concrete example of the SSRF attack effectively illustrates the iterative mutation process and the identification of an edge-case bypass via trailing-dot representation. Furthermore, the article provides practical advice for customers on deploying their WAFs and patching software, reinforcing the defense-in-depth strategy.

However, a key limitation is the inherent 'black-box' nature of the LLM's interaction with the WAF in this test setup. While this mirrors real-world attacker constraints, it also means the LLM isn't necessarily exploring the most efficient or effective attack vectors from a purely vulnerability-exploitation standpoint, but rather those that are most likely to bypass the WAF's input filtering. The article mentions that the LLM had no visibility into source code or WAF rules, which is standard for dynamic testing, but the AI's 'hypothesis' generation, while useful for explaining the process, is still an interpretation of the LLM's internal reasoning, which can be opaque. The article also acknowledges that the findings were heavily concentrated in CMDi and SSRF, suggesting that other attack categories might have been more effectively blocked, or perhaps less effectively explored by the LLM in this specific configuration. Future work on white-box testing, as mentioned, will be critical to address these potential blind spots. The reliance on human review to filter out noise and validate findings is a strength but also a bottleneck, underscoring the ongoing need for skilled security analysts.

Key Points

• Frontier AI models were used to dynamically test Cloudflare's WAF, simulating hacker-like iterative attacks.
• The adaptive loop allowed LLMs to mutate attack payloads based on WAF responses without WAF internal knowledge.
• Testing covered six attack categories (XSS, SQLi, CMDi, SSRF, LFI, Log4j) across 1,107 attempts.
• The vast majority of attacks were blocked, but 49 WAF-relevant findings (mostly CMDi and SSRF) were identified for improvement.
• Findings led to new detections like SSRF - Obfuscated Host and SSRF - Restricted Protocol.
• The process highlights the importance of human review in triaging AI-generated test results.
• Customers are advised to deploy multiple layers of security and run Managed Rules in log-first mode before blocking.
• Future testing will involve white-box approaches where AI has knowledge of application vulnerabilities and WAF rules.

---

📖 Source: We tested our own WAF with frontier AI models. Here’s what we found