Varnish and the state of web caching

So there’s lots of excitement around Varnish, which is a caching proxy that is built to be first and foremost a reverse-proxy, as opposed to squid, which does both forward and reverse. Acceleration (reverse-proxying) is obviously important to us at Flickr, as we use squid extensively.

I’m hoping to do some testing with Varnish once it’s stable and has the ability to manage a constantly full cache. After emailing with Poul Henning-Kamp (one of the main developers) he says that object replacement/eviction is indeed on the roadmap, so we shall see.

From what I can tell, Varnish sounds a little like the COSS filesystem that squid can use, in that it uses one big file to store objects in. In varnish, this is mmap’d into the process and the kernel does all of the disk work. Since replacement/eviction isn’t done yet, not sure if the mechanism is “cyclical” like COSS, but however it will work, it’ll probably see some big performance increases when compared to the standard ‘nested directories’ way that aufs does things in squid currently.

Woohoo!

I like making things go! At the moment, I'm SVP of Infrastructure and Operations at Etsy, and I'm currently pursuing a Master's degree in Human Factors and Systems Safety at Lund University.

2 Comments

  1. Nagita   •  

    Is this not production-ready? Why not test now?

  2. allspaw   •     Author

    I’m sure it’s production-ready for some places. For us, we use a cache that is always full, and therefore needs to evict objects from the cache based on on their usage. Varnish doesn’t yet (as of my emails with the author) have this ability.

    That, and having some more documentation on their VCL language would get me to test it. :)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>