Thoughts on systems safety, software operations, and sociotechnical systems.
If you consider that you and your users are in some sort of a ‘relationship’, then good communication is pretty important. The Rackspace datacenter outage reminds me yet again that we’re lucky to have a handful of servers in more than one datacenter that can communicate to users in the case where we’ve lost one […]
“Do you know when your database layer will fall over and die ? At how many QPS (queries per second) will your application fall prey to slowness, corruption, replication issues, or other sorts of badness ?” I asked that question of the audience when giving a talk on capacity planning at the MySQL conference last […]
I don’t think I know Jon Prall, but I’m sure I must have seen him “around” in this small world of social media websites. He’s got a list of his 85 rules of Web Operations, some of which I agree with wholeheartedly. Reminds me to get #42 and #43 done one of these days.
WebOps-related mailing lists have always had a problem with this vague term, and I suspect that commercial vendors exploit this confusion. 🙂 Wikipedia gives a pretty vague definition: “…is the process of testing or tracking (monitoring) how end-users interact with a website or web application.” People use the term to describe lots of things that […]
No matter what sort of capacity planning tricks you think you have up your sleeve, if it doesn’t involve the procurement process, then you can’t call it planning. Procurement is the part that happens after you know what kind and how much capacity you need. It’s the part where you, someone in some other group […]
(well, not really) A little while ago, in one of our clusters we replaced a boatload of PowerEdge 1425 webserver-class boxes with a much smaller number of HP DL145 G3 quad-core boxes, getting the same amount of oomph from 1/3 the boxes. Not too bad.
I think I’m the only person from flickr going this year, and I’ll be there just for Wednesday’s sessions. Looking forward to Theo‘s session, “Advanced Production Troubleshooting“, the session on Puppet, Doug Cutting’s talk on Hadoop, and maybe some others. Who wants to drink some beer and shoot the breeze about things like what a […]
Now that the news of this is out and it’s happening, I can say that planning for it was pretty damn cool. Not just from the capacity and operations standpoint, but watching my codehead coworkers deal with all of the API/background/migration stuff as well. Excellent.
Was talking with Joyce the other day and we talked about how there’s no real Web Ops conference out there currently. The closest is possibly MySQL Conference or maybe the PHP conferences, where ops/sysadmin types will go and attend the few talks about infrastructure topics, but the main audience seems to be developers. I could […]
She just referred to our largest suitcase (we’re packing for a weeklong trip) as a “single point of failure.”