“Measure twice, cut once.” was the craftsman rule, referred by several experts of software engineering. I always struggled with two important issues in following this rule. First is it is really hard to define perfect measurable parameters for software products. Second problem is with software engineers, they always feel cutting is so much fun than measuring.
Problem is that even if we can define some measurable parameters, it is much harder to measure them in timely manner and incorporate their results in existing product development. By the time we have user feedback and other results, it is often too late to apply them in existing development. Even if we spend time and effort in applying those changes in next release of product, we usually find that the whole echo-system – customer expectations, hardware, and competitive products – is very different by the time we have our new release ready. Practically I also agree with the fact that most of the interesting innovation in software products are result of cutting first and measure later. Software engineer loves to cut first, and fine tune later after real measurement.
Last year I took new role for building infrastructure for our cloud based offerings. I found several philosophical changes in patterns and practices we need to apply in this new world. First thing I am excited in this new role is having an opportunity to measure often and also enjoy the fun part of innovation and experimentation. Important task for us is to choose from large number of parameters that we can collect, and apply them in improving our realtime decision making process.
Really success of a cloud service lies in building right telemetry strategy, and applying measurements into actionable items. Telemetry provides a cheap and efficient mechanism for developers to experiment their ideas and quickly measure success and failure. Management also has opportunity to set right priority on customer issues, adjust their plans with higher success probability. If you are working or planning for cloud based services, make sure that strategy for telemetry is in place from very early stage. After all it is not fun in flying a plane in a dense cloud with no control panel and measuring instruments.