Why did Cloudflare Build its Own Reverse Proxy?
Cloudflare is moving from NGINX to Pingora, it solves the primary reverse proxy and caching needs and even for web server’s request handling.
NGINX as a reverse proxy has long been a popular choice for its efficiency and reliability. However, Cloudflare announced their decision to move away from NGINX to their homegrown open-source solution for reverse proxy, Pingora.
What is Reverse Proxy?
A reverse proxy sits in front of the origin servers and acts as an intermediary, receiving requests, processing them as needed, and then forwarding them to the appropriate server. It helps improve performance, security, and scalability for websites and web applications.
Imagine you want to visit a popular website like Wikipedia. Instead of going directly to Wikipedia’s servers, your request first goes to a reverse proxy server.
The reverse proxy acts like a middleman. It receives your request and forwards it to one of Wikipedia’s actual servers (the origin servers) that can handle the request.
When the Wikipedia server responds with the requested content (like a web page), the response goes back to the reverse proxy first. The reverse proxy can then do some additional processing on the content before sending it back to you.
What is the difference between Forward Proxy vs Reverse Proxy?
Understand, The role that proxies play in web architecture and consider using them to improve the performance, security, and scalability of your site.
Reverse Proxy is used for:
- Caching: The reverse proxy stores frequently requested content in its memory. So if someone else requests the same Wikipedia page, the reverse proxy can quickly serve it from the cache instead of going to the origin server again.
- Load balancing: If there are multiple Wikipedia servers, the reverse proxy can distribute incoming requests across them to balance the load and prevent any single server from getting overwhelmed.
- Security: The reverse proxy can protect the origin servers by filtering out malicious requests or attacks before they reach the servers.
- Compression: The reverse proxy can compress the content to make it smaller, reducing the amount of data that needs to be transferred to you.
- SSL/TLS termination: The reverse proxy can handle the encryption/decryption of traffic, offloading this work from the origin servers.
Why Does Cloudflare Have a Problem with NGINX?
While NGINX has been a reliable workhorse for many years, Cloudflare encountered several architectural limitations that prompted it to seek an alternative solution. One of the main issues was NGINX’s process-based model. Each request was handled by a separate process, which led to inefficiencies in resource utilization and memory fragmentation.
Another challenge Cloudflare faced was the difficulty in sharing connection pools among worker processes in NGINX. Since each process had its isolated connection pool, Cloudflare found itself executing redundant SSL/TLS handshakes and connection establishments, leading to performance overhead.
Furthermore, Cloudflare struggled with adding new features and customizations to NGINX due to its codebase being written in C, a language known for its memory safety issues.
In-Memory Caching vs. In-Memory Data Store
In-memory caching and in-memory data storage are both techniques used to improve the performance of applications by storing frequently accessed data in memory. However, they differ in their approach and purpose.
How Cloudflare Built Its Reverse Proxy “Pingora” from Scratch?
Being Faced with these limitations, Cloudflare considered several options, including forking NGINX, migrating to a third-party proxy like Envoy, or building their solution from scratch. Ultimately, they chose the latter approach, aiming to create a more scalable and customizable proxy that could better meet their unique needs.
| Feature | NGINX | Pingora |
|---|---|---|
| Architecture | Process-based | Multi-threaded |
| Connection Pooling | Isolated per process | Shared across threads |
| Customization | Limited by configuration | Extensive customization via APIs and callbacks |
| Language | C | Rust |
| Memory Safety | Prone to memory safety issues | Memory safety guarantees with Rust |
To address the memory safety concerns, Cloudflare opted to use Rust, a systems programming language known for its memory safety guarantees and performance. Additionally, Pingora was designed with a multi-threaded architecture, offering advantages over NGINX’s multi-process model.
With the help of multi-threading, Pingora can efficiently share resources, such as connection pools, across multiple threads. This approach eliminates the need for redundant SSL/TLS handshakes and connection establishments, improving overall performance and reducing latency.
DevOps vs SRE vs Platform Engineering – Explained
At small companies, engineers often wear multiple hats, juggling a mix of responsibilities. Large companies have specialized teams with clearly defined roles in DevOps, SRE, and Platform Engineering.
The Advantages of Pingora
One of the main advantages of Pingora is its shared connection pooling capability. By allowing multiple threads to access a global connection pool, Pingora minimizes the need for establishing new connections to the backend servers, resulting in significant performance gains and reduced overhead.
Cloudflare also highlighted Pingora’s multi-threading architecture as a major benefit. Unlike NGINX’s process-based model, which can lead to resource contention and inefficiencies, Pingora’s threads can efficiently share resources and leverage techniques like work stealing to balance workloads dynamically.
Pingora: A Rust Framework for Network Services
Interestingly, Cloudflare has positioned Pingora as more than just a reverse proxy. They have open-sourced Pingora as a Rust framework for building programmable network services. This framework provides libraries and APIs for handling protocols like HTTP/1, HTTP/2, and gRPC, as well as load balancing, failover strategies, and security features like OpenSSL and BoringSSL integration.
The selling point of Pingora is its extensive customization capabilities. Users can leverage Pingora’s filters and callbacks to tailor how requests are processed, transformed, and forwarded. This level of customization is particularly appealing for services that require extensive modifications or unique features not typically found in traditional proxies.
The Impact on Service Meshes
As Pingora gains traction, it’s natural to wonder about its potential impact on existing service mesh solutions like Linkerd, Istio, and Envoy. These service meshes have established themselves as crucial components in modern microservices architectures, providing features like traffic management, observability, and security.
While Pingora may not directly compete with these service meshes in terms of their comprehensive feature sets, it could potentially disrupt the reverse proxy landscape. Service mesh adopters might consider leveraging Pingora’s customizable architecture and Rust-based foundation for building their custom proxies or integrating them into their existing service mesh solutions.
Monorepos vs Microrepos: Which is better?
Find out why companies choose Monorepos over Microrepos strategies and how they impact scalability, governance, and code quality.
The Possibility of a “Vanilla” Pingora Proxy
Given Pingora’s extensive customization capabilities, some speculate that a “vanilla” version of Pingora, pre-configured with common proxy settings, might emerge in the future. This could potentially appeal to users who desire an out-of-the-box solution while still benefiting from Pingora’s performance and security advantages.
About Author
