Ops Meta-Metrics: Velocity 2010 Slides

As expected, Velocity was excellent this year. What an awesome time to be in this field.

Caveat for those who didn’t see/hear my talk: the graphs and numbers in the slides are, for the most part, made up. But they’re also in line with what I’ve seen at Flickr and Etsy.


  1. Excellent presentation. I especially liked the release log and correlation of release size and incident TTR.

  2. Hey John,

    Caught your presentation at Velocity, loved it. I had a question, one that I can’t seem to answer locally in Oz at the moment.

    We’re using a number of tools to measure site load time – from Gomez to http://www.webpagetest.org/ . In the past we’ve used Keynote and Webmetrics and probably more that I’ve forgotten.

    What I struggle with – and I’d love to know your approach is – every single tool gives a different load time. Even if we try to run the same test at the same time on the same day of the week I get a different number.

    How do I come up with a definitive number? How do you do it?

    It makes it very hard to judge whether a code change has improved site load time, or merely that we happened to run the test at a time when the test gave a quick response.

    How do you guys do it?


  3. Ed:

    The key is not to come up with “the” number, but “a” number. You run the tests all the time and watch for changes in trends. If you are measuring load time constantly, you’ll be able to see pretty much immediately when things change. Even if tool A says, “2.5 seconds” and tool B says, “3.5 seconds”, if the time doubles when you deploy, I think it’s pretty clear something went wrong.


Leave a Comment

Your email address will not be published. Required fields are marked *