Cloudflare prototypes 'Cache Transcoding' to compress CDN cache by an average of one-third, freeing up several petabytes of capacity.

Cloudflare has revealed that it is prototyping 'Cache Transcoding,' a mechanism that compresses data stored in its CDN cache, effectively increasing the capacity of existing storage. In initial tests, the storage size of the data to be compressed was reduced to about one-third of the original size on average, and it is possible to generate an effective cache capacity of several petabytes by increasing the CPU load by only a few percent.

How we could save petabytes of cache storage with Zstandard and Pingora | Cloudflare Blog

https://blog.cloudflare.com/cache-transcoding/

A CDN (Content Delivery Network) is a system that temporarily stores website HTML, JavaScript, and other files on servers closer to the user, reducing access to the origin server where the original data is stored. Users can retrieve data from a nearby cache, thus shortening communication time. However, for a network as large as Cloudflare's, the storage capacity required to store the cache becomes a major issue. According to Cloudflare, the prices of RAM and HDDs have risen sharply in the past year, increasing the need to use existing hardware more efficiently.

Traditional Cloudflare caching essentially inherits the compression method used for data sent from the origin server. If the origin server returns uncompressed HTML, the uncompressed data is stored on disk and remains uncompressed when moving between Cloudflare data centers. This meant there was still room to reduce the amount of data transferred between storage and data centers.

That's when Arshi Patel, who participated in development as an intern at Cloudflare, prototyped Cache Transcoding. Cache Transcoding is a mechanism that adds compression processing to 'Pingora,' a Rust-based proxy platform that Cloudflare uses for HTTP communication and caching. If a cache miss occurs with data to be compressed, the data received from the origin server is compressed using Zstandard (zstd) before being saved to disk.

Zstandard is a lossless compression algorithm that allows data to be restored to its original state when recovered. It is designed with an emphasis on balancing compression ratio and processing speed, and the prototype version of Cache Transcoding uses compression level 3. In past tests, Cloudflare has reported that Zstandard can compress files 42% faster than Brotli while maintaining almost the same file size, and can reduce file size by 11.3% at a speed comparable to gzip.

Compressed data maintains its compressed state even on disk. Cloudflare's 'Tiered Cache' is a mechanism that reduces access to the origin server by tiering multiple cache servers, but with Cache Transcoding, data is transferred in Zstandard format even when sent from a higher cache to a lower cache. Because it is decompressed at the final stage of returning data to the user, it reduces not only disk space but also the amount of data transmitted between Cloudflare's data centers. The compression process is performed only once when the data first enters the cache, and thereafter the capacity reduction effect continues each time the cache is reused.

Cloudflare's tests showed that the size of data to be compressed was reduced to approximately 1/2.8th, and the storage capacity required was reduced to roughly 1/3. Models using compression level 3 also saw additional CPU load limited to just a few percent. Reducing the amount of data stored on disk allows more content to be retained on the same server, and also reduces the frequency with which useful cache data is evicted due to insufficient capacity. Cloudflare estimates that this could translate to several petabytes of effective cache capacity across their entire platform.

However, while the test involved sending over one million requests to 10 cache servers, the two data sets used, approximately 195 KiB and 272 KiB, were intentionally designed to be easily compressed. Cloudflare says that in the future, they plan to conduct tests using a wider range of content and file sizes, as well as investigate higher Zstandard compression levels, compressed origin responses, and Range requests that perform partial retrieval.