Wednesday, May 20, 2009

Questions that SaaS executives must be able to answer - KPIs that matter.

“There is much pleasure to be gained from useless knowledge” (Bertrand Russell)

It has been my experience that SaaS executives have trouble answering the most basic questions about their service operations, and mind you, this is what the business is all about.


Again and again, I keep coming back to the conclusion that the fact that state of SaaS Service Operations is so dire is due to the fact that on-demand companies are built on the first ‘S’ (software) and not the second 'S' (Service).

SaaS entrepreneurs are, in general, bright, creative, out-of-the-box thinkers. They are software developers and have no clue about IT practices and disciplines.


The age old premise “if you can't measure it you can't manage it” somehow escapes SaaS companies across the globe, until it becomes a huge problem.

Have you gone through the numbing process of presenting a specific customer with their real SLA adherence? I have. On average, it would take me a few hours of going through multiple sources of data to come up with (sometimes) accurate data.


Following are a number of questions (an incomplete list) that every SaaS executive should be able to answer in her sleep, or at least with a click of a button.

1. Availability management
  • What are your real uptime numbers?
  • How do the trialing twelve months (TTM) look like
  • Are we better than we were six months ago?
  • How many outages have you had in the last M months?
  • What is the breakdown, based on severity?
  • What is the breakdown, based on downtime causes?
  • How many service disruption incidents were repeated?
  • How quickly do you recover from outages?
  • How many days have gone by without a critical, major outage?
2. SLA Management
  • How does your availability match up to your customer commitments?
  • Which customers were affected most (even if they do not complain)?
3. Change Management
  • How often are changes made to the production environment?
  • What is the breakdown of changes by category?
  • What percent of changes did you have to roll back?
4. Asset management
  • What is the status of your inventory? What box is located where?
  • What function or customer would be impacted by a loss of a certain box?
  • When do your support/software contracts expire and what might it affect?
5. Cost Management

  • What are the actual costs of the operations?
  • How is the budget allocated among the various components?
  • How much does each new (N) customer(s) cost?
  • Are we getting the full value from our supply chain?
6. Churn Management
  • How many customers have you lost in the past 6, 12, 24 months?
  • Is your customer retention improving over time?
  • What percent is your customer churn out of your customer base?
  • What is the average retention time of your customers?
  • What is the breakdown, based on reasons for churn?

I am well aware of the fact that there are no integrated solutions for the SMB supporting a database for these crucial KPIs, but every company should have some form of repository capturing at least some of the data and a easy way of extracting it.

The important issue here is that SaaS companies should be aware of these KPIs and start asking these questions, even if they do not yet have all the answers.