Whew. That took longer than I thought.
Todd Hoff over at the High Scalability blog has an email interview with me about a book that I wrote, called “The Art of Capacity Planning: Scaling Web Resources“. I’m still just happy that I got it done at all, seeing how it was due the same week that my son was born.
This book happened because of a LOT of people. You know who you are, because you’re in the Acknowledgements.
Via kottke: some good examples of doing rough math in your head, causing you to guess about assumptions all along the way.
IMHO, being able to do this is one of the things that makes a good web ops person. The examples might be “useless”, but the process is invaluable.
James Hamilton’s excellent LADIS 2008 presentation has lots of great stuff in it about internet scale bits. Cool stats.
It’s hard to describe how tiring it is to hear someone quote Donald Knuth (or Tony Hoare) in the wrong context. I’m not the only one annoyed by this. In “Structured Programming with go to Statements”, Knuth says:
We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.
After having read Knuth’s paper containing this quote, I can agree that it’s certainly a brilliant piece of advice in the context of programming. What is irritating to me is the blanket application of this pearl of wisdom to anything that has to do with computers, especially systems performance, web operations and architecture decisions.
For the record: I firmly believe in these principles:
- Done >= perfect.
- Don’t waste time building elaborate simulations for what the future might bring to your capacity.
- Performance tuning is better left outside the capacity planning process.
But I think sometimes folks lean on the Knuth/Hoare way too much, in the wrong situation. This was meant to be a blog post of my own, but I think this article pretty much sums up my current feelings about it.
Our philosophy in Flickr Operations Engineering.
Here are the slides from my talk at the Velocity Conference.
Just listened to Artur railing against squid and preaching the virtues of varnish. He quoted what most people quoted, which is how varnish performs serving out of *memory*.
It must be nice to have a working set that small. Until someone can show me numbers of disk-intensive (meaning, full caches, LRU eviction churning all the time) varnish numbers, then squid does us quite fine.
From this article on BusinesWeek:
In addition, through its Data Center Solutions Group, Dell builds clouds, the large groups of computers whose collective power can be tapped remotely, via the Internet. Customers for DCS-built clouds include Yahoo! (YHOO), Facebook, and Akamai (AKAM) as well as big Chinese Internet companies like Baidu (BIDU), Tudou, Tencent Holdings, and Youku.
and also this article about Ebay’s new Project Echo:
eBay joins social networking site Facebook and online customer relationship management provider Salesforce.com in offering the former type of cloud computing – their online development platforms relate specifically to their respective applications.
Apparently, if you have servers, in a data center, somewhere, connected to the internet, you have a “cloud” that is part of the new evolution of unicorn and rainbow-powered magic. If you have an API, then even more so!
Please, people: not everything worth mentioning is part of a cloud. Ugh.
So now there’s chapters 1-4 on Safari RoughCuts. Which means if you don’t mind shelling out the dough, you can take a look at what I’ve been getting up early for every day for the past few months. The working title is “The Art of Capacity Planning” and it’s meant to be a no-nonsense description of the capacity planning process and considerations for web operations.
I still have two chapters to go before it’s all finished, but if you’re nice enough to take a look at what I’ve got thus far, I’d appreciate any feedback. I’m sure there could be typos and some graphs misaligned, but such is life with “drafts”.