From one door to another

Last week I gave 2 month’s notice – I’ll be leaving Flickr in January. When Stew and Cat asked me to join Flickr in January of 2005, I felt like it was time to go and do something different, so I said yes. Five years (and four billion photos) later, it’s again time to go...
Continue reading...

WebOps: Good prep for becoming a new parent?

I think I’ve said before somewhere that working in the field of web operations prepared me somewhat for being a parent. I thought the other day that I should write down some of this reasoning, because it’s pretty often that I’m reminded of similarities: High availability Having redundant infrastructure is WebOps 101. For my kids’...
Continue reading...

Automated Control paper by the RAD Lab folks

Wow, how did I miss this until now? In June, some smart people gathered in Barcelona for the First Workshop on Automated Control for Datacenters and Clouds (ACDC09) and jeez it looked like it was a good time, from a glance at the program. One of the cooler papers isĀ “Automatic exploration of datacenter performance regimes”...
Continue reading...

Extreme Automated Infrastructure

I’ve said it before that I’ve always been a huge fan of SystemImager, for super simple imaging. It has some shortcomings for config management, but those are solved with things like Chef or Puppet. With all of the great things being talked about surrounding ‘Automated Infrastructure’, I’ll point to something insanely cool: 1,190 nodes installed...
Continue reading...

SLAs, clouds, and whatnot

Excellent. Good work, Ben: ah, the mighty service level agreement! the tooth and claw by which the wily customer brings the vendor to heel. get the SLA right and you, the customer, can sit back and relax, safe in the knowledge that should there be an outage, you are covered. your business is protected from...
Continue reading...

Annoying To Me.

I can’t tell you how ripped I get when people say things like this: “cloud computing means getting rid of ops” If by “ops” you mean “people in data centers racking servers, installing OSes, running cables, replacing broken hardware, etc.” then sure, cloud computing aims to relieve you of those burdens. If you really think...
Continue reading...

Context and Operational Metrics

I really don’t think it can be overestimated how important context can be when it comes to troubleshooting or evaluating the health of an infrastructure. When starting to troubleshoot a complex problem, web ops 101 “best practices” usually start with asking at least these questions: When did this problem start? What changes, if any, (software,...
Continue reading...

Mechanical Analogies To Web Stuff, Part 2.

This is a ramble continued from before, which means it’s mostly a blog post for me, but maybe others might find it interesting. The last time I made an analogy between back-end web architectures and mechanical structures, I blathered on about what are basically structural limitations of individual components in a physical device, and how...
Continue reading...