Cloudflare is saving 100TB of memory by trimming the 1.1.1.1 DNS cache by tens of bytes at a time.
Cloudflare has announced that it has revised the data structure of its DNS cache used in public DNS resolvers such as '1.1.1.1,' reducing the total memory used across the system by approximately 100TB. Benchmarks show that memory usage per cache item has decreased by 56%, from 953 bytes to 420 bytes, and that writing to and reading from the cache has also become faster while reducing memory usage.
How we saved 100 terabytes of memory by optimizing 1.1.1.1's DNS cache | Cloudflare Blog
1.1.1.1 is Cloudflare's public DNS resolver that translates domain names into IP addresses. For example, when you type 'cloudflare.com' into your browser, your device queries the DNS resolver to find the IP address to connect to. 1.1.1.1 caches DNS responses obtained once, so that when the same query comes in, it can answer without having to query the upstream DNS server every time.
The DNS infrastructure supporting 1.1.1.1 at Cloudflare is called 'Big Pineapple.' Big Pineapple is used by multiple services, including Gateway DNS and DNS Firewall, in addition to 1.1.1.1, and maintains over 250 billion DNS cache entries at all times. With a scale of 250 billion entries, even if each cache entry only uses a mere 1 byte, Cloudflare as a whole would need more than 250GB of memory.
Cloudflare then examined each piece of information to be stored in the cache, removing unnecessary areas and duplicate information after saving.
The first things I worked on were Rust's 'Vec' and 'String' types. Vec holds a pointer to the data itself, the current length, and capacity information so that data can be added in the future. However, DNS responses do not change after being stored in the cache, so there is no need for additional capacity.
Cloudflare replaced the old cache with a fixed-size Box-type format, reducing the total size by 64 bytes from the 8 Vec and String elements in each cached record. Including the extra space that was already allocated, they were able to save over 15TB across more than 250 billion cached records.
There were also small inefficiencies in how DNS responses were stored. DNS responses have multiple sections, such as the answer, authoritative information, and additional information, but traditionally, separate lists were prepared for each section. Cloudflare changed this to a method that combines the three sections into one list and records two positional information points indicating the boundary of the sections, each using two bytes. This reduces the amount of data stored per cache entry by 28 bytes. Multiple boolean values are also grouped at the bit level, reducing the amount of space that Rust inserts to align memory positions.
Furthermore, DNS records typically store the domain name to which the record belongs. However, in reality, the queried domain name and the domain name in the record are almost always the same. Big Pineapple has changed its method to not store the same name twice, but instead restores it from the cached lookup key when needed. The complete name is only stored when a different domain name is required, such as through a CNAME record.
The way Rust handles DNS record types has also been revised. Previously, DNS record types were managed using an enumeration type called Rust's enum, and even small A and AAAA records used 144 bytes of memory to match the largest NAPTR record. The data required for an A record is 4 bytes, and for an AAAA record it's 16 bytes, and these two together account for more than 80% of Cloudflare's DNS traffic. By moving only the larger data types to a separate memory area, we were able to save 120 bytes per A and AAAA record.
However, moving data to separate memory areas increases the number of memory allocations, requiring the CPU to read data from distant locations one after another. Therefore, the final solution involved saving the data portion of DNS records as a byte sequence encoded in the communication format, and storing them together in a single contiguous memory area. Because the data is consolidated in memory, it becomes easier to read, and for many DNS records, the byte sequence can be copied directly when creating a response. Cloudflare's benchmarks showed that the change reduced cache lookup time by 5%, and the use of reusable temporary buffers improved the number of items that could be added to the cache per second by 13%.
Combining the five levels of optimization, the memory usage per cache item decreased from 953 bytes to 420 bytes, and the amount of memory allocated also decreased from 1.1KB to 461 bytes. Although the actual reduction rate is smaller because servers use memory for purposes other than caching, the 99th percentile (p99) of memory usage saw a 43% reduction in resident memory, from 9.3GB to 5.3GB. Across the entire server cluster running Big Pineapple, approximately 100TB of memory was freed up, equivalent to the RAM of about 130 Cloudflare 13th generation servers. Furthermore, cache write performance improved by 43%, and search times were reduced by 19%.
Cloudflare plans to allocate the approximately 100TB of freed-up space to expanding its DNS cache, increasing the number of answers that can be stored without increasing memory usage. They believe that increasing the number of answers that can be stored in the cache will reduce the number of queries to upstream DNS servers, and they are also considering further optimization of the Big Pineapple cache.
Related Posts:
in Web Service, Posted by log1d_ts