Cloudflare Container Leak: A Storage Layer Isolation Wake-Up Call

Below the VM Boundary: The Real Isolation Risk

This InfoQ article provides a detailed breakdown of a critical cross-tenant data exposure vulnerability in Cloudflare's Containers. The key insight is that the isolation failure wasn't at the virtual machine (Firecracker) boundary, but deeper within the storage allocator (dm-thin) due to a specific configuration (skip_block_zeroing). This highlights a fundamental challenge in multi-tenant cloud environments: ensuring isolation across all layers, not just the most visible ones. The article effectively explains the technical mechanism, demonstrating how residual data from previous containers could be read by subsequent ones. The researchers' findings of recovered data like directory structures and SQLite databases underscore the potential impact, even if direct modification or active disk reads were not possible. Cloudflare's rapid response and comprehensive remediation, including retiring disks and restarting VMs, showcase a robust incident response, but the extended cleanup timeline also indicates the complexity of addressing such deep-seated issues.

Key Points

• A cross-tenant data exposure vulnerability in Cloudflare Containers allowed residual disk blocks from previous customers to be recovered by new tenants.
• The root cause was not a VM boundary failure, but a storage allocator configuration (skip_block_zeroing with dm-thin) that skipped zeroing newly allocated blocks.
• Researchers recovered diverse data types, including directory structures and SQLite databases, from unmapped residual blocks.
• Cloudflare's remediation involved a rapid runtime fix followed by a comprehensive cleanup of existing container disks and host caches.
• The incident serves as a critical reminder for practitioners to verify isolation at all infrastructure layers, especially storage, in shared environments.
• Security practitioners emphasize the recurring nature of storage-layer isolation breaks and the need for explicit zero-on-allocate or wipe-on-release checks in control planes.

---

📖 Source: Cloudflare Fixes Cross-Tenant Data Exposure in Containers