Twitter open sourced its proxy for the memcached protocol called "Twemproxy," which was primarily developed to reduce open connections to company's cache servers. "Twemproxy had been running in production for six months, during which time we've increased the number of front-ends by 30%," blogged Twitter.
Twemproxy has been instrumental in allowing front-end and cache server pools to scale independently as capacity requirements change.
"At Twitter, we have thousands of front-end application servers, and we run multiple Unicorn instances on each one. Each instance establishes at least one connection to every cache server. As we add front-ends and increase the number of Unicorns per front-end, this connection count continues to increase. As a result the performance of the cache server degrades due to per-connection overhead," company said.
Adding, "We use Twemproxy to bound the number of connections on the cache servers by deploying it as a local proxy on each of the front-ends. The Unicorns on each front-end connect to the cache servers through the local proxy, which multiplexes and pipelines the front-end requests over a single connection to each cache server. This reduces the number of connections to each cache server by a significant factor," said Twitter.
With Twemproxy you can:
- "Maintain persistent server connections
- Keep connection count on cache servers low
- Pipeline requests and responses
- Use multiple server pools simultaneously
- Proxy the complete memcached ASCII protocol
- Easily configure server pools using a YAML file
- Use multiple hashing modes, including consistent hashing
- Increase observability using stats exposed on a monitoring port," explains Twitter.
You can get Twemproxy on GitHub.