"The user's going to pick dancing pigs over security every time" - Bruce Schneier"
In this post I am publishing an article by a guest contributor - Rashed Khan (rash799@hotmail.com) who points out interesting study results...
Software as a service (SaaS) for application delivery is a hot topic when it comes to questions of security. Adding SaaS components in any form is something that seems to generate acute anxiety in anyone who takes the time to consider it. Fears about the loss of privacy and other related security issues top the list of current concerns.
On the other hand, those who are already using SaaS solutions or have added elements of SaaS to their systems are considerably more confident about security issues than non-users. When it comes right down to it, SaaS appears to be something that one must experience in order to trust.
Forrester Research has recently completed a study that supports this premise. In companies where SaaS was already in use, having replaced a complete solution, concerns over security are noticeably lessened. This is also true in companies where the decision to replace a complete solution with SaaS had already been made and was about to be implemented.
By contrast, companies that were only contemplating or planning to augment their solutions with SaaS, or in companies that were using just a few SaaS components, anxieties over safety were still running high.
Miroslaw Lisserman, analyst at Forrester Research, believes this to be a strong validation of the future of SaaS technology. Lisserman had this to say about the findings: “To me, this means the following: SaaS solutions are more secure than perceived by many, since once SaaS applications are deployed and used, the security concerns decrease.” Apparently, SaaS technology performs so well that it has to be experienced to be believed.
Analyst Krishnan Subramanian, an independent researcher, feels that the security concerns related to the SaaS technology itself have been overworked. He said that the real issue related to this application has more to do with people. Regarding these concerns, Subramanian had this to say: “It is the responsibility of the SaaS vendors to educate users about their people-centric security practices. It is the responsibility of the SaaS users to get to know these details from the vendors.”
Moving away from concerns about the security of SaaS technology and turning attention instead to security concerns related to the technology's providers and users is a measure of the maturing of this technology. It's a sign that SaaS is ultimately coming into its own.
The growth of the sector itself testifies to this belief. There has been rapid expansion of SaaS solutions with Enterprise Resource Planning (ERP) software functions. Additionally, there is growing use of the ERP system by both small and mid-sized manufacturers. The manufacturing software is also used more frequently by industry distributors and in job shops.
Small companies who are part of large supply chains, along with the supply chain members they deal with, are all discovering significant benefits and greater functionality in SaaS-based ERP when employed as a comprehensive manufacturing software solution. Home-grown and standalone applications fall short by comparison, making SaaS both the wave of the future and an increasingly intelligent choice.
Software-as-Service as a disruptive trend and how it affects the traditional, ISVs and IT moving to the Cloud. Considerations in the transition to the new model and expertise on SaaS Service Operations - STORM™ and DevOps
Monday, December 12, 2011
Thursday, December 01, 2011
The Black Swan Event in SaaS Operations
"I find that the harder I work the more luck I seem to have." - Thomas Jefferson
Nassim Taleb’s eye-opening books 'Black Swan' and (to a lesser extent) 'Fooled by Randomness' discuss the rare, unexpected and almost impossible to predict events that have a major impact (and usually tend to be disastrous). He calls these events Black Swan events, and gives samples such as World War I, stock market crashes, the PC, the Internet, and 9/11.
Interestingly enough, all the Black Swan events are easily rationalized after the event, by hindsight.
The Black Swan analogy is borrowed from the notion that while one can induce a hypothesis from observational data - e.g. all swans are white - one cannot prove that hypothesis, since after observing numerous white swans, it takes only a single black swan to refute it. Karl Popper, the science philosopher, made that notion popular in his discussion of the Scientific Method (The Logic of Scientific Discovery).
SaaS and the Black Swan
Have you ever lost your database only to find out that the backup files were deleted the previous day? Have you ever hit a major problem with a component in the system, only to find out that the support contract expired last month?
My own experience and the experience of the numerous companies I have worked with, have taught me that the next Black Swan is just around the corner, lurking in the dark and will hit you when you least expect it to. Heck, that’s the nature of a Black Swan.
The systems we deal with are so complex and interdependent that one could never analyze (let alone predict) the interconnections that govern the behavior of the services we offer. Luckily, statistics are on our side, so that most SaaS applications are stable most of the time and on average, we can predict the behavior over time. But that is just what creates a Black Swan – we observe a certain behavior for so long, that we tend to accept it as a scientific fact; until it bites us in the behind.
Running a complex SaaS operation with dozens (or hundreds) of servers, network boxes, configuration files, erratic software and all the dependencies we have on our infrastructure providers (power, internet, hardware, communications) is like driving a high speed car on a congested highway, blindfolded. We have no appreciation of how much Lady Luck is involved.
Keep in mind that the longer good things happen, the harder is the effect of the Black Swan event - remember the dot.com and the real-estate bubbles; most of us are still licking the wounds.
The Butterfly Effect
All it takes is an overflowing log file, that incapacitates the disk, that will bring the system down. Or a minor, forgotten gadget installed on one of the servers whose license has expired. A pipeline of requests starts filling up and there goes the system.
How about setting up an image of a new VM, whose IP and the DNS IP were reversed by mistake. Put it in production and slowly the wrong DNS IP starts propagating in the system. After a while the servers are not communicating with each other and the system freezes.
These tend to be catastrophic events, since they are so hard to detect and resolve. Many times, restarting the whole system is the chosen quick solution, praying that the problem will resolve itself. But in these cases, the system will behave just as badly, and by the time one realizes what is happening, major damage to the customers and your brand has been done.
Words of Wisdom
Do not despair. I am not suggesting that since a Black Swan event is unpredictable, there’s nothing you can do about it. The opposite is true.
The first step is to internalize the fact that it will occur, as the famous quote goes “s**t happens”.
“Prepare for Failure” is my motto. Take into account that at any given moment something might break.
A number of practices should be implemented early on:
Change Management: To ensure that the events are indeed rare and that one may recover quickly with the knowledge of what went wrong.
Event Management: To be able to detect early on, what is hitting the fan, and respond to it.
Availability Management: Analyze your Single Points of Failure and impact of component failure. Build your backups, your DRP and practice recovery.
Incident Management: Make sure you cover these practices: Detection, Recording, Classification, Notification, Escalation, Investigation, Diagnosis, Restoration and Closure.
The Wise and the Smart ones
I was approached by a few (emphasis on few) CEOs and COOs that felt uncomfortable about the fact everything was going smoothly. Some were on the verge of fast growth and wanted to assure themselves that they were better prepared to hit the highway. Others had a feeling in their bones that “too good for too long” was a recipe for disaster, even if they did not read Nasssim Taleb’s book.
But many potential customers I spoke with assured me that they really do not need my services since they are doing very well, thank you. Some are still doing very well and others had a large hat to eat and many letters of regret to write their customers.
Nassim Taleb’s eye-opening books 'Black Swan' and (to a lesser extent) 'Fooled by Randomness' discuss the rare, unexpected and almost impossible to predict events that have a major impact (and usually tend to be disastrous). He calls these events Black Swan events, and gives samples such as World War I, stock market crashes, the PC, the Internet, and 9/11.
Interestingly enough, all the Black Swan events are easily rationalized after the event, by hindsight.
The Black Swan analogy is borrowed from the notion that while one can induce a hypothesis from observational data - e.g. all swans are white - one cannot prove that hypothesis, since after observing numerous white swans, it takes only a single black swan to refute it. Karl Popper, the science philosopher, made that notion popular in his discussion of the Scientific Method (The Logic of Scientific Discovery).
SaaS and the Black Swan
Have you ever lost your database only to find out that the backup files were deleted the previous day? Have you ever hit a major problem with a component in the system, only to find out that the support contract expired last month?
My own experience and the experience of the numerous companies I have worked with, have taught me that the next Black Swan is just around the corner, lurking in the dark and will hit you when you least expect it to. Heck, that’s the nature of a Black Swan.
The systems we deal with are so complex and interdependent that one could never analyze (let alone predict) the interconnections that govern the behavior of the services we offer. Luckily, statistics are on our side, so that most SaaS applications are stable most of the time and on average, we can predict the behavior over time. But that is just what creates a Black Swan – we observe a certain behavior for so long, that we tend to accept it as a scientific fact; until it bites us in the behind.
Running a complex SaaS operation with dozens (or hundreds) of servers, network boxes, configuration files, erratic software and all the dependencies we have on our infrastructure providers (power, internet, hardware, communications) is like driving a high speed car on a congested highway, blindfolded. We have no appreciation of how much Lady Luck is involved.
Keep in mind that the longer good things happen, the harder is the effect of the Black Swan event - remember the dot.com and the real-estate bubbles; most of us are still licking the wounds.
The Butterfly Effect
All it takes is an overflowing log file, that incapacitates the disk, that will bring the system down. Or a minor, forgotten gadget installed on one of the servers whose license has expired. A pipeline of requests starts filling up and there goes the system.
How about setting up an image of a new VM, whose IP and the DNS IP were reversed by mistake. Put it in production and slowly the wrong DNS IP starts propagating in the system. After a while the servers are not communicating with each other and the system freezes.
These tend to be catastrophic events, since they are so hard to detect and resolve. Many times, restarting the whole system is the chosen quick solution, praying that the problem will resolve itself. But in these cases, the system will behave just as badly, and by the time one realizes what is happening, major damage to the customers and your brand has been done.
Words of Wisdom
Do not despair. I am not suggesting that since a Black Swan event is unpredictable, there’s nothing you can do about it. The opposite is true.
The first step is to internalize the fact that it will occur, as the famous quote goes “s**t happens”.
“Prepare for Failure” is my motto. Take into account that at any given moment something might break.
A number of practices should be implemented early on:
Change Management: To ensure that the events are indeed rare and that one may recover quickly with the knowledge of what went wrong.
Event Management: To be able to detect early on, what is hitting the fan, and respond to it.
Availability Management: Analyze your Single Points of Failure and impact of component failure. Build your backups, your DRP and practice recovery.
Incident Management: Make sure you cover these practices: Detection, Recording, Classification, Notification, Escalation, Investigation, Diagnosis, Restoration and Closure.
The Wise and the Smart ones
I was approached by a few (emphasis on few) CEOs and COOs that felt uncomfortable about the fact everything was going smoothly. Some were on the verge of fast growth and wanted to assure themselves that they were better prepared to hit the highway. Others had a feeling in their bones that “too good for too long” was a recipe for disaster, even if they did not read Nasssim Taleb’s book.
But many potential customers I spoke with assured me that they really do not need my services since they are doing very well, thank you. Some are still doing very well and others had a large hat to eat and many letters of regret to write their customers.
Monday, October 03, 2011
CAC, LTV, MRR - Translating SaaS Financials into Actions
“If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is.” - John Louis von Neumann
I’m sure that most of you have seen the various metrics floating around with CMRR, CLTV, Churn Rate and ASC starring in equations that sometimes cause one to cringe while sipping the day’s first coffee.
Let’s look at one of the basic formulas for SaaS Financials:
CAC < CLTV
This simply says that if you want to become profitable one day, you must make sure that your Customer Acquisition Costs should be less than your Customer Lifetime Value. In other words, the total amount of revenue you will generate from a customer, throughout the years or months that they derive value from your SaaS offering, should be more than the cash you spend on acquiring that customer.
Simple? It almost doesn’t pass the DUH Test. But in this article we'll look more carefully at the implications.
Acquisition vs. Retention
There is a notion in the industry that the costs to acquire a new customer are 5 to 7 times more expensive than the costs to retain an existing customer. Whether one agrees with the numbers or not, it is widely accepted that acquisition is more expensive than retention, yet most SaaS companies will spend far more resources and executive attention on growth through new customers than keeping the current customers satisfied, or in other words, reducing the Churn and up-selling to the current base. In fact, in every company I have consulted, the issues of Churn Management and Operational Excellence were far down on the priority list.
I guess hunting is far more exciting than farming.
Therefore, we will examine on how to grow the right hand side of the equation - the CLTV, not on how to lower the left side - the CAC.
Breaking down the CLTV
CLTV = Lifetime * ARPU * Gross Margin.
I hope I am not losing you here. Take another sip from your Latte. It is not complicated – sixth grade math. Stick with me, the actionable items will follow shortly.
ARPU means the ‘Average Revenue Per User’ for the time period defined as Lifetime. So if you count by months, Lifetime would be the number of months the customer remains loyal, and the ARPU would be the average that the customer would pay per month. If your value is calculated by years (lucky bastard!) then Lifetime would be how many years you retain the customer and the ARPU is average revenue per year from the customer.
Gross Margin is the ratio of total Revenue to the Costs Of Goods Sold (COGS) – how much does it cost you to give service to your customer.
For the Gross Margin to grow, the COGS should shrink, or at least stay stable as your revenue grows. So the lower the COGS are, the more you retain for your Christmas party.
As a simple example, let’s assume that your average customer sticks around for 19 months, that the average monthly payment from a customer is $430 and that your gross margins are 72%, then the CLTV = 19 * $430 * 0.72 = $5882.4.
Just imagine that with a little effort you could cause the Lifetime to grow to 21, or the ARPU to $460 and multiply the new CLTV by the number of customers...
What can we do about it?
Without going into details of how the various numbers are calculated we can still learn much about these equations and derive actions from them.
The bottom line is that you want to have the highest CLTV value possible. Looking at the equation, it means that your Lifetime, Gross Margin and ARPU values should grow.
Needless to say that for each of these three values, books could be written. Nevertheless, the paragraphs below cover the main points and map the actions one can take, and their direct impact on the equation’s variables.
Lifetime
In order for this value to grow, a SaaS provider should invest managerial attention into customer retention, or lower Churn. That means improving your customer service and responsiveness. Meassure and monitor the support KPIs. Run a weekly Customer Success meeting with Support, Operations, Sales and PS. Build a community and best practices around your product to enhance loyalty.
Be as transparent with your service levels as you can. Award loyalty with small gifts. Document your Churn data and analyze it – understand the reasons customers leave you and determine the trends.
Provide meaningful SLAs and act on them.
Average Revenue Per User
Translate ARPU into: “up-selling you service”. This means a strong group of farmers in your sales team. Use software to monitor user behavior. Think of value-added services that you could sell for a fraction of the recurring cost. Identify and keep in touch with your champion inside the customer’s organization and seek opportunities to sell more services or branch out to new groups within the organization.
Gross Margin
Lowering COGS means an effective and efficient Service Operations. This starts with a good team of dedicated professionals, a rigorous set of practices such as Change Mgmt, Incident Mgmt, Event Mgmt, etc. and a robust monitoring and alerts infrastructure.
Automation and Delegation - maximize what silicon and your customers can do instead of having people on your end doing it. That means create as much automation as possible around manual processes. Provide self-help, self-registration and self-configuration for your customers to run.
Understand the financials of the hosting services you are using. Don’t stick to the current solution just because you have been doing it for a long time. Circumstances would have changed, new solutions are offered every month, and a fresh look might save a lot of recurring costs.
In conclusion, we looked at one of the equations that every venture capitalist (i.e. your board members) tells you to watch, and transformed it into actionable items that your company should deal with. To pass that threshold of profitability, it probably won’t happen with that “major deal we’re about to sign”, but with improvements across the board in every aspect that tilts the right side of the equation.
For some good readings on SaaS Financials there are Bessemer’s 5 Cs, and Joel York’s excellent articles on the financials of SaaS.
I’m sure that most of you have seen the various metrics floating around with CMRR, CLTV, Churn Rate and ASC starring in equations that sometimes cause one to cringe while sipping the day’s first coffee.
Let’s look at one of the basic formulas for SaaS Financials:
CAC < CLTV
This simply says that if you want to become profitable one day, you must make sure that your Customer Acquisition Costs should be less than your Customer Lifetime Value. In other words, the total amount of revenue you will generate from a customer, throughout the years or months that they derive value from your SaaS offering, should be more than the cash you spend on acquiring that customer.
Simple? It almost doesn’t pass the DUH Test. But in this article we'll look more carefully at the implications.
Acquisition vs. Retention
There is a notion in the industry that the costs to acquire a new customer are 5 to 7 times more expensive than the costs to retain an existing customer. Whether one agrees with the numbers or not, it is widely accepted that acquisition is more expensive than retention, yet most SaaS companies will spend far more resources and executive attention on growth through new customers than keeping the current customers satisfied, or in other words, reducing the Churn and up-selling to the current base. In fact, in every company I have consulted, the issues of Churn Management and Operational Excellence were far down on the priority list.
I guess hunting is far more exciting than farming.
Therefore, we will examine on how to grow the right hand side of the equation - the CLTV, not on how to lower the left side - the CAC.
Breaking down the CLTV
CLTV = Lifetime * ARPU * Gross Margin.
I hope I am not losing you here. Take another sip from your Latte. It is not complicated – sixth grade math. Stick with me, the actionable items will follow shortly.
ARPU means the ‘Average Revenue Per User’ for the time period defined as Lifetime. So if you count by months, Lifetime would be the number of months the customer remains loyal, and the ARPU would be the average that the customer would pay per month. If your value is calculated by years (lucky bastard!) then Lifetime would be how many years you retain the customer and the ARPU is average revenue per year from the customer.
Gross Margin is the ratio of total Revenue to the Costs Of Goods Sold (COGS) – how much does it cost you to give service to your customer.
For the Gross Margin to grow, the COGS should shrink, or at least stay stable as your revenue grows. So the lower the COGS are, the more you retain for your Christmas party.
As a simple example, let’s assume that your average customer sticks around for 19 months, that the average monthly payment from a customer is $430 and that your gross margins are 72%, then the CLTV = 19 * $430 * 0.72 = $5882.4.
Just imagine that with a little effort you could cause the Lifetime to grow to 21, or the ARPU to $460 and multiply the new CLTV by the number of customers...
What can we do about it?
Without going into details of how the various numbers are calculated we can still learn much about these equations and derive actions from them.
The bottom line is that you want to have the highest CLTV value possible. Looking at the equation, it means that your Lifetime, Gross Margin and ARPU values should grow.
Needless to say that for each of these three values, books could be written. Nevertheless, the paragraphs below cover the main points and map the actions one can take, and their direct impact on the equation’s variables.
Lifetime
In order for this value to grow, a SaaS provider should invest managerial attention into customer retention, or lower Churn. That means improving your customer service and responsiveness. Meassure and monitor the support KPIs. Run a weekly Customer Success meeting with Support, Operations, Sales and PS. Build a community and best practices around your product to enhance loyalty.
Be as transparent with your service levels as you can. Award loyalty with small gifts. Document your Churn data and analyze it – understand the reasons customers leave you and determine the trends.
Provide meaningful SLAs and act on them.
Average Revenue Per User
Translate ARPU into: “up-selling you service”. This means a strong group of farmers in your sales team. Use software to monitor user behavior. Think of value-added services that you could sell for a fraction of the recurring cost. Identify and keep in touch with your champion inside the customer’s organization and seek opportunities to sell more services or branch out to new groups within the organization.
Gross Margin
Lowering COGS means an effective and efficient Service Operations. This starts with a good team of dedicated professionals, a rigorous set of practices such as Change Mgmt, Incident Mgmt, Event Mgmt, etc. and a robust monitoring and alerts infrastructure.
Automation and Delegation - maximize what silicon and your customers can do instead of having people on your end doing it. That means create as much automation as possible around manual processes. Provide self-help, self-registration and self-configuration for your customers to run.
Understand the financials of the hosting services you are using. Don’t stick to the current solution just because you have been doing it for a long time. Circumstances would have changed, new solutions are offered every month, and a fresh look might save a lot of recurring costs.
In conclusion, we looked at one of the equations that every venture capitalist (i.e. your board members) tells you to watch, and transformed it into actionable items that your company should deal with. To pass that threshold of profitability, it probably won’t happen with that “major deal we’re about to sign”, but with improvements across the board in every aspect that tilts the right side of the equation.
For some good readings on SaaS Financials there are Bessemer’s 5 Cs, and Joel York’s excellent articles on the financials of SaaS.
Sunday, August 21, 2011
SaaS and SLA - State of the Art
"You can get assent to almost any proposition so long as you are not going to do anything about it." (Chapman, John Jay)
Lately, I have been approached by a number of frustrated CIOs, asking me about what can be expected from a typical SLA in the industry and which provider offers an SLA with some beef.
A Typical SLA
Let’s see what a basic SaaS SLA should look like:
• Service Availability
• System Response Time
• Customer Service Response Time
• Customer Service Availability
• Service Outage Resolution Time
• Failover Window For Disaster Recovery
• Reclaiming Customer Data
• Maintenance Notification
• Proactive Service Outage Notification
• RFO (Reason for Outage)
Nice. Now let’s see what a typical SLA in the SaaS industry looks like:
• Service Availability
Is that it? Yeah, that’s about it. (Sometimes you may find Customer Support response time as well, the Lord be praised). The standard SLA in the industry only discusses ‘uptime’ and even that is usually very iffy, with mostly zero or negligible penalties.
Recently I have been meeting with CXOs of successful SaaS companies and asking them what their SLAs offer. Not surprisingly, their answers were reflective of the typical SLA above. Some did not even offer an SLA and one said, half jokingly, that they (the customers) should say ‘thank you’ for even having the service available. When asked about the future of SLA in the industry, the collective answer was that nothing will probably change, customers will not demand better SLOs (Service Level Objectives) and that the whole issue was quite irrelevant. One CEO suggested that the only concern of the CIO is ease of integration.
Is that so? Or are these guys burying their heads in the sand? When I asked about how many dealt with CIOs (as compared to business units), only one said that he did, and that it was an unpleasant experience.
How would you explain this discrepancy between what CIOs want and what SaaS CXOs would offer? And why is the state of SLAs in the industry is so pitiful?
A Quick Historical Review
I think the answers lie in the history of SaaS and how it penetrated the market. Around 12 years ago we started seeing the first SaaS applications (although no one came up with the name until a few years later). SaaS mostly targeted the SMBs who had no access to the enterprise software that was available to the larger companies. Either from a cost, or complexity or support point of view, the on-premise applications were out of reach for the smaller companies. When they started becoming available over the Web, the SMB were so delighted to even have a solution they were not going to bitch about the service levels being offered in the contracts. They were just happy that the apps were available. So, SaaS companies offered a 99% uptime which seemed pretty good (except that it translated into four days of downtime!). Nobody could talk about performance, as the dependency on the customers’ own network and on their ISPs allowed the providers an easy escape from accountability.
The Corporate Business Unit
Even though SaaS initially targeted the SMB, the big breakthrough came from the business units that found freedom in circumventing IT and getting their needs answered quickly (and in the process, flipping a bird to IT). The heads of the business units were mostly concerned with features and did not care much about SLAs. Even if they did, they did not have the experience and knowledge, that IT has accumulated over the years, on what to demand, how to verify that their service levels are met, etc.
The New IT Manager
More than ten years have passed with SaaS slowly establishing itself as mainstream, and conquering more and more territories. Old habits die hard and the sad state of SLAs remained where it had been a decade ago. Now, SaaS is finally entering the enterprise through the front door. There is a new generation of CIOs that are not threatened by SaaS and understand the freedom it offers them. They want to get back into the driver’s seat, clean up the mess that a decentralized SaaS policy created and control what is entering their domain.
As for the CIOs with the old-timer’s attitude, the Cloud hype has forced them to pay attention. When the CEOs caught on (hey, we can save a lot of money here) the pressure was on the CIOs to start acquiring Cloud Applications – SaaS. And, like it or not, there are numerous integration issues that demand that IT be in the picture.
Slowly, we are seeing a shift in the market. More and more CIOs and IT managers are in the picture. And when they see the lack of real certification or the famished SLAs offered by the vendors, they are probably baffled, at best, if not furious.
I believe that gradually, as more CIOs enter the picture, the SaaS providers will have to prove themselves as more mature, attentive and accountable vendors. I think that the IT customers will step-up the pressure and changes will occur. SaaS providers will succumb to provide a serious document with real numbers and repercussions.
Lately, I have been approached by a number of frustrated CIOs, asking me about what can be expected from a typical SLA in the industry and which provider offers an SLA with some beef.
A Typical SLA
Let’s see what a basic SaaS SLA should look like:
• Service Availability
• System Response Time
• Customer Service Response Time
• Customer Service Availability
• Service Outage Resolution Time
• Failover Window For Disaster Recovery
• Reclaiming Customer Data
• Maintenance Notification
• Proactive Service Outage Notification
• RFO (Reason for Outage)
Nice. Now let’s see what a typical SLA in the SaaS industry looks like:
• Service Availability
Is that it? Yeah, that’s about it. (Sometimes you may find Customer Support response time as well, the Lord be praised). The standard SLA in the industry only discusses ‘uptime’ and even that is usually very iffy, with mostly zero or negligible penalties.
Recently I have been meeting with CXOs of successful SaaS companies and asking them what their SLAs offer. Not surprisingly, their answers were reflective of the typical SLA above. Some did not even offer an SLA and one said, half jokingly, that they (the customers) should say ‘thank you’ for even having the service available. When asked about the future of SLA in the industry, the collective answer was that nothing will probably change, customers will not demand better SLOs (Service Level Objectives) and that the whole issue was quite irrelevant. One CEO suggested that the only concern of the CIO is ease of integration.
Is that so? Or are these guys burying their heads in the sand? When I asked about how many dealt with CIOs (as compared to business units), only one said that he did, and that it was an unpleasant experience.
How would you explain this discrepancy between what CIOs want and what SaaS CXOs would offer? And why is the state of SLAs in the industry is so pitiful?
A Quick Historical Review
I think the answers lie in the history of SaaS and how it penetrated the market. Around 12 years ago we started seeing the first SaaS applications (although no one came up with the name until a few years later). SaaS mostly targeted the SMBs who had no access to the enterprise software that was available to the larger companies. Either from a cost, or complexity or support point of view, the on-premise applications were out of reach for the smaller companies. When they started becoming available over the Web, the SMB were so delighted to even have a solution they were not going to bitch about the service levels being offered in the contracts. They were just happy that the apps were available. So, SaaS companies offered a 99% uptime which seemed pretty good (except that it translated into four days of downtime!). Nobody could talk about performance, as the dependency on the customers’ own network and on their ISPs allowed the providers an easy escape from accountability.
The Corporate Business Unit
Even though SaaS initially targeted the SMB, the big breakthrough came from the business units that found freedom in circumventing IT and getting their needs answered quickly (and in the process, flipping a bird to IT). The heads of the business units were mostly concerned with features and did not care much about SLAs. Even if they did, they did not have the experience and knowledge, that IT has accumulated over the years, on what to demand, how to verify that their service levels are met, etc.
The New IT Manager
More than ten years have passed with SaaS slowly establishing itself as mainstream, and conquering more and more territories. Old habits die hard and the sad state of SLAs remained where it had been a decade ago. Now, SaaS is finally entering the enterprise through the front door. There is a new generation of CIOs that are not threatened by SaaS and understand the freedom it offers them. They want to get back into the driver’s seat, clean up the mess that a decentralized SaaS policy created and control what is entering their domain.
As for the CIOs with the old-timer’s attitude, the Cloud hype has forced them to pay attention. When the CEOs caught on (hey, we can save a lot of money here) the pressure was on the CIOs to start acquiring Cloud Applications – SaaS. And, like it or not, there are numerous integration issues that demand that IT be in the picture.
Slowly, we are seeing a shift in the market. More and more CIOs and IT managers are in the picture. And when they see the lack of real certification or the famished SLAs offered by the vendors, they are probably baffled, at best, if not furious.
I believe that gradually, as more CIOs enter the picture, the SaaS providers will have to prove themselves as more mature, attentive and accountable vendors. I think that the IT customers will step-up the pressure and changes will occur. SaaS providers will succumb to provide a serious document with real numbers and repercussions.
In short, the differentiator is no longer the fact that a vendor offers SaaS, nor the feature set, nor the pricing. To distinguish oneself, a SaaS vendor will have to excel in every aspect of the service and provide the assurances for the service levels that CIOs are expecting.
Monday, July 04, 2011
The CIO's Dilemma – Adopting SaaS as a Strategy
“Luke, you're going to find that many of the truths we cling to depend greatly on our own point of view” (Obi-Wan, Star Wars, episode VI)
'IT-Avoidance' Mechanism
SaaS adoption has become an outstanding success, not in the only SMB which it targeted originally, but at the business-unit level in the larger corporations. SaaS became the ultimate IT-avoidance mechanism for the business department heads that were tired of waiting for many months (or years) for their IT needs, weary of investing huge budgets just to find out that the software did not deliver what was expected, or was outdated by the time it was implemented. With SaaS, they could start a free trial immediately and gain value of the solution with minutes, hours or days. IT managers sometimes found out that their internal customers were using SaaS software many months after it was a done deal.
It’s All About Control
This paradigm shift from transitional on-premise to SaaS (which is somewhat reminiscent of the PC revolution that empowered the end users and removed some of the dependency they had on IT), was not looked upon favorably by IT managers.
I believe that the main reason for IT's resentment towards SaaS, is the loss of control partly based on real problems caused by IT-Avoidance and partly is based on an emotional response to the notion of various business units not “needing” IT as much as before.
My premise is that CIO’s must adopt SaaS – it delivers the goods and it is happening anyway – but for the adoption to be successful, they must regain control of the situation.
Security
IT usually brings up the ‘security’ excuse to kill SaaS deals, but I believe that many times the ‘security’ they are talking about is their 'job security', afraid to let go of assets that everyone is dependent on.
So let’s examine the real security issue. As I have mentioned in numerous talks and presentations, Cloud companies, as a rule, will do a much better job at data security and privacy than a hospital or a car manufacturer (or a bank, credit card company or NASA judging by the publications on the subject).
Still, there is a major issue regarding SaaS accounts when they are not controlled by IT. Any business manager can swipe a credit card, and order 40 seats for her staff to start using an HR app. The manager knows nothing of security, nor does she bother much with it - the point is to get productivity up. The users are provisioned, not by IT, but by the business unit. When an employee leaves the company to work for the competition, IT is supposed to disconnect that employee from all the assets in the company. But how can they de-provision the employee if they have no access (or knowledge) of the various SaaS applications that person was using? Who can guarantee that this employee will not access company data from home or from the new employer’s premises?...
Lack of Visibility
Not only does the IT manager have incomplete knowledge of who is using what, even if they know that an employee has a SaaS account, there is no way to know if that user is accessing the software, how it is being used and what, if any problems are there. There is no visibility into performance issues. IT also has no knowledge of what part of the organizations’ data is stored where. And could it be that some of the same data is residing at different SaaS providers, and could it be that information at one provider is inconsistent with some information at another provider?
Vendor Selection
One of the areas of expertise of IT is the ability to select software solutions and evaluate the vendors. The business units do not have that ability, and frankly, they don’t give a damn. They want quick solutions within their monthly budgets and all other topics regarding security, integration, service continuity, financial viability, and SLAs are stuff that IT traditionally dealt with (and hence took forever to make a decision). So, IT is not involved in the solution/vendor selection process exposing the enterpise to bad choices and their consequences.
Lack of Efficiency
It is not uncommon in large, distributed companies, that different departments are consuming the service from the same SaaS vendor (or different departments are using similar solutions from different vendors) with multiple contracts in place, and perhaps different integration schemes. Of course this reduces the chances for bulk discounts and is inefficient in all aspects of organizational learning and business intelligence.
Another aspect of control is the lack of ability to access, backup and analyze the company’s data or to impose regulatory constraints on the user.
Lack of Strategic Planning
The fact that each department is an independent SaaS consumer and that IT is not driving and controlling the company’s solution is a great impediment to multiyear strategic planning. The individual business units do not have a high-level view of the company’s needs and strategy.
The lack of strategic planning reduces the company’s ability to ensure security and to employ cross company data analysis (the data is distributed across multiple vendors) and may cause compliance and regulatory issues in the future.
What to do, what to do?
A following article will outline strategies to employ in order to get hold the SaaS situation. But it will suffice to say that IT needs to restore control and bring itself to the forefront. This means that, first and foremost, the CIO has to embrace SaaS and not fear it. Start by defining the strategic goals of Cloud computing in the organization. Understand who is consuming what in the organization. Review your upcoming upgrades and begin a process of considering SaaS to replace your on-premise solutions.
SaaS is not a threat but a wonderful opportunity for the enterprise and the IT organization. Don’t play a defensive game; rather, become a leader in this area for your company.
'IT-Avoidance' Mechanism
SaaS adoption has become an outstanding success, not in the only SMB which it targeted originally, but at the business-unit level in the larger corporations. SaaS became the ultimate IT-avoidance mechanism for the business department heads that were tired of waiting for many months (or years) for their IT needs, weary of investing huge budgets just to find out that the software did not deliver what was expected, or was outdated by the time it was implemented. With SaaS, they could start a free trial immediately and gain value of the solution with minutes, hours or days. IT managers sometimes found out that their internal customers were using SaaS software many months after it was a done deal.
It’s All About Control
This paradigm shift from transitional on-premise to SaaS (which is somewhat reminiscent of the PC revolution that empowered the end users and removed some of the dependency they had on IT), was not looked upon favorably by IT managers.
I believe that the main reason for IT's resentment towards SaaS, is the loss of control partly based on real problems caused by IT-Avoidance and partly is based on an emotional response to the notion of various business units not “needing” IT as much as before.
My premise is that CIO’s must adopt SaaS – it delivers the goods and it is happening anyway – but for the adoption to be successful, they must regain control of the situation.
Security
IT usually brings up the ‘security’ excuse to kill SaaS deals, but I believe that many times the ‘security’ they are talking about is their 'job security', afraid to let go of assets that everyone is dependent on.
So let’s examine the real security issue. As I have mentioned in numerous talks and presentations, Cloud companies, as a rule, will do a much better job at data security and privacy than a hospital or a car manufacturer (or a bank, credit card company or NASA judging by the publications on the subject).
Still, there is a major issue regarding SaaS accounts when they are not controlled by IT. Any business manager can swipe a credit card, and order 40 seats for her staff to start using an HR app. The manager knows nothing of security, nor does she bother much with it - the point is to get productivity up. The users are provisioned, not by IT, but by the business unit. When an employee leaves the company to work for the competition, IT is supposed to disconnect that employee from all the assets in the company. But how can they de-provision the employee if they have no access (or knowledge) of the various SaaS applications that person was using? Who can guarantee that this employee will not access company data from home or from the new employer’s premises?...
Lack of Visibility
Not only does the IT manager have incomplete knowledge of who is using what, even if they know that an employee has a SaaS account, there is no way to know if that user is accessing the software, how it is being used and what, if any problems are there. There is no visibility into performance issues. IT also has no knowledge of what part of the organizations’ data is stored where. And could it be that some of the same data is residing at different SaaS providers, and could it be that information at one provider is inconsistent with some information at another provider?
Vendor Selection
One of the areas of expertise of IT is the ability to select software solutions and evaluate the vendors. The business units do not have that ability, and frankly, they don’t give a damn. They want quick solutions within their monthly budgets and all other topics regarding security, integration, service continuity, financial viability, and SLAs are stuff that IT traditionally dealt with (and hence took forever to make a decision). So, IT is not involved in the solution/vendor selection process exposing the enterpise to bad choices and their consequences.
Lack of Efficiency
It is not uncommon in large, distributed companies, that different departments are consuming the service from the same SaaS vendor (or different departments are using similar solutions from different vendors) with multiple contracts in place, and perhaps different integration schemes. Of course this reduces the chances for bulk discounts and is inefficient in all aspects of organizational learning and business intelligence.
Another aspect of control is the lack of ability to access, backup and analyze the company’s data or to impose regulatory constraints on the user.
Lack of Strategic Planning
The fact that each department is an independent SaaS consumer and that IT is not driving and controlling the company’s solution is a great impediment to multiyear strategic planning. The individual business units do not have a high-level view of the company’s needs and strategy.
The lack of strategic planning reduces the company’s ability to ensure security and to employ cross company data analysis (the data is distributed across multiple vendors) and may cause compliance and regulatory issues in the future.
What to do, what to do?
A following article will outline strategies to employ in order to get hold the SaaS situation. But it will suffice to say that IT needs to restore control and bring itself to the forefront. This means that, first and foremost, the CIO has to embrace SaaS and not fear it. Start by defining the strategic goals of Cloud computing in the organization. Understand who is consuming what in the organization. Review your upcoming upgrades and begin a process of considering SaaS to replace your on-premise solutions.
SaaS is not a threat but a wonderful opportunity for the enterprise and the IT organization. Don’t play a defensive game; rather, become a leader in this area for your company.
Monday, May 30, 2011
Organizational Culture and Company DNA – What Makes a Successful SaaS Company.
“No people come into possession of a culture without having paid a heavy price for it” (James A. Baldwin)
It was always clear to me that success with SaaS was not about technology, but about execution. This week I got a clear reminder.
The SaaS CEO Forum
For the past few months I have been running a SaaS CEO Forum that meets every six weeks or so, each time hosted at a different SaaS company, by a member of the Forum . The forum consists of a select group of successful SaaS companies, that have been selling their service for a number of years and are dealing with issues such as growth, operations, sales, marketing and customer satisfaction. Every meeting has a theme such as running a fabulous inside sales teams, SaaS Service Operations, knowledge-as-a-service, etc.. Yesterday the forum was hosted by Avinoam Nowogrodski, founder and CEO of Clarizen, a fast growing, market leader on collaborative project management. Beyond the very interesting review of the company’s clockwork marketing and sales operation, Avinoam gave a presentation on what makes a SaaS company successful.
Successful SaaS Company
No one can argue with the success of Clarizen, having grown 400% year over year, with an ever-growing community of happy customers, so it was worthwhile listening to Avinoam’s credo.
Clarizen’s CEO was talking about managing a company where execution is paramount and where Customer Success always comes first. He has been careful selecting an executive team that he regards as ‘A’ players and nurturing a culture of Respect, Modesty, Openness, and Accountability.
Among the factors Avinoam mentioned was “checking your ego at the door”, willingness to take risks, and acceptance of mistakes as part of the ever changing environment and conditions. Delegating Authority, Hands-on in your domain, Transparency and above all – Measure, Measure, Measure every aspect of the company’s operation; sales, marketing, conversion between each stage of the pipeline, responsiveness, costs.
The SaaS Angle – Fast Forward
While I agree wholeheartedly with all the above criteria being critical for a successful company, I asked for the SaaS angle. The answer was obvious even before I finished asking the question: “Pace”. In a SaaS company everything is fast-forwarded. The cycles in almost every aspect shorten and therefore the margins of error are ever so narrow. In a company that caters for the SMB in a low-touch model, the sales cycles are measured in days, not quarters, the software releases are shortened to weeks. The discovery of bugs usually occur within hours (or minutes) after a new version is introduced. Hence, Openness and Transparency are paramount and there is no time for ego games or controlling vital information (I have written about these aspects in the past: Transparency & Communications).
To reiterate – in a fast-pace, ever-changing, 24X7 environment, the need to feel the operational pulse, the need for responsiveness and open communications, the need for a listening ability and accountability are vital for success. Time spent on BS, on analysis-paralysis, on political games, on territorial squabbles, is time spent away from making sure your customers are successful, and that will be evident on the company’s bottom line and eventually, on the quarterly bonuses.
It was always clear to me that success with SaaS was not about technology, but about execution. This week I got a clear reminder.
The SaaS CEO Forum
For the past few months I have been running a SaaS CEO Forum that meets every six weeks or so, each time hosted at a different SaaS company, by a member of the Forum . The forum consists of a select group of successful SaaS companies, that have been selling their service for a number of years and are dealing with issues such as growth, operations, sales, marketing and customer satisfaction. Every meeting has a theme such as running a fabulous inside sales teams, SaaS Service Operations, knowledge-as-a-service, etc.. Yesterday the forum was hosted by Avinoam Nowogrodski, founder and CEO of Clarizen, a fast growing, market leader on collaborative project management. Beyond the very interesting review of the company’s clockwork marketing and sales operation, Avinoam gave a presentation on what makes a SaaS company successful.
Successful SaaS Company
No one can argue with the success of Clarizen, having grown 400% year over year, with an ever-growing community of happy customers, so it was worthwhile listening to Avinoam’s credo.
Clarizen’s CEO was talking about managing a company where execution is paramount and where Customer Success always comes first. He has been careful selecting an executive team that he regards as ‘A’ players and nurturing a culture of Respect, Modesty, Openness, and Accountability.
Among the factors Avinoam mentioned was “checking your ego at the door”, willingness to take risks, and acceptance of mistakes as part of the ever changing environment and conditions. Delegating Authority, Hands-on in your domain, Transparency and above all – Measure, Measure, Measure every aspect of the company’s operation; sales, marketing, conversion between each stage of the pipeline, responsiveness, costs.
The SaaS Angle – Fast Forward
While I agree wholeheartedly with all the above criteria being critical for a successful company, I asked for the SaaS angle. The answer was obvious even before I finished asking the question: “Pace”. In a SaaS company everything is fast-forwarded. The cycles in almost every aspect shorten and therefore the margins of error are ever so narrow. In a company that caters for the SMB in a low-touch model, the sales cycles are measured in days, not quarters, the software releases are shortened to weeks. The discovery of bugs usually occur within hours (or minutes) after a new version is introduced. Hence, Openness and Transparency are paramount and there is no time for ego games or controlling vital information (I have written about these aspects in the past: Transparency & Communications).
To reiterate – in a fast-pace, ever-changing, 24X7 environment, the need to feel the operational pulse, the need for responsiveness and open communications, the need for a listening ability and accountability are vital for success. Time spent on BS, on analysis-paralysis, on political games, on territorial squabbles, is time spent away from making sure your customers are successful, and that will be evident on the company’s bottom line and eventually, on the quarterly bonuses.
Saturday, May 07, 2011
System Integrators’ Cloud Strategies – React or Lead?
“A leader takes people where they want to go. A great leader takes people where they don’t necessarily want to go, but ought to be.” (Rosalynn Carter)
Recently a number of medium-sized System Integrators (SIs) have approached me to help them either define their SaaS/Cloud strategy, validate their strategy or help them with the realization of their strategy.
It seems that, the shoe has dropped. It took two years of relentless Cloud hype for the System Integrators to finally understand that major changes are occurring which will impact their customers and therefore – them.
New Breed of System Integrators
There is a new breed of Cloud System Integrators (up to a year ago they were called “SaaS System Integrators”) including companies such as Appirio, Astadia, Bluewolf & Model Metrics, that are dedicated to providing SaaS based services. Being newcomers, they are quite small compared to the large players, but their growth rates are phenomenal (e.g. Appirio has been growing in triple digits since 2008). So far they are have not been big enough to pose a visible threat to the old timers, but I suspect that the Dinosaurs are starting to feel a bit uncomfortable with the quick mammals that are infiltrating their territories.
Lack of Strategy – Tactical Approach
Having studied approaches of the major players in the market I have come to some interesting observations:
Accenture is a Salesforce.com partner. Ernst & Young are helping implement EmployU and HumanWave. Deloitte is with Workday and Genpact is a Netsuite implementor.
Capgemini is working with AWS to provide a Cloud Computing COE.
Considering the size and nature of the business of these players, it seems that these services are not high on the priority lists of the big SIs. It is quite obvious (to me) that these giants have not defined a SaaS strategy, rather they are reacting in an opportunistic manner to the market - akin to a “me too” tactic, just to have something 'Cloudy' or 'SaaSy' on their web site.
Cloud Strategy – React or Lead
As mentioned, many SIs have adopted tactical approaches, at the Cloud Service level, rather than a strategic approach. Further analysis led us to the conclusion that by zooming out, grouping and mapping the above Cloud Services. we can define five Cloud Strategies for System Integrators:
Using this prism, we could say that a number of players have chosen a strategic path:
IBM’s GBS and Wipro are offering services that we define as SaaS One-stop-shop for ISVs (strategy 3). KPMG is offering Roadmap for SaaS Adoption (strategy 5) while Infosys is acting as a SaaS Aggregator for enterprise IT (strategy 1). Smaller players such as iProcess are offering BpassS (Strategy 2)
Even though adoption rates are growing very fast, the hype of Cloud Computing is a couple of years ahead of reality, especially in the larger organizations that are the natural customers of the leading SIs. Some SIs have chosen to play it safe: not invest up front in new technologies and methodologies and watch the market carefully. Since they will still be milking many fat cows for a few years to come, the decision not to decide could be considered a smart, conservative strategy.
Others, though, have taken leadership positions, risking investments without a clear date on the ROI. They have the advantage of defining the market trends and have a better chance of emerging as the de-facto leaders in a few years when all the players, that are currently sitting on the fence, will have to define their strategy and see what bones are left to pick.
Recently a number of medium-sized System Integrators (SIs) have approached me to help them either define their SaaS/Cloud strategy, validate their strategy or help them with the realization of their strategy.
It seems that, the shoe has dropped. It took two years of relentless Cloud hype for the System Integrators to finally understand that major changes are occurring which will impact their customers and therefore – them.
New Breed of System Integrators
There is a new breed of Cloud System Integrators (up to a year ago they were called “SaaS System Integrators”) including companies such as Appirio, Astadia, Bluewolf & Model Metrics, that are dedicated to providing SaaS based services. Being newcomers, they are quite small compared to the large players, but their growth rates are phenomenal (e.g. Appirio has been growing in triple digits since 2008). So far they are have not been big enough to pose a visible threat to the old timers, but I suspect that the Dinosaurs are starting to feel a bit uncomfortable with the quick mammals that are infiltrating their territories.
Lack of Strategy – Tactical Approach
Having studied approaches of the major players in the market I have come to some interesting observations:
- There are around 15 Cloud services that SIs could offer their customers. Examples are: billing/integration/SSO services, PaaS development (Azure, Google Apps Engine, Force.com), SaaS-oriented testing, training, 24X7 NOC, etc.
- Judging by my engagement with some of the SIs, most are not even aware of what those services are, and are therefore concentrating on a few obvious choices.
- Most of the SIs are offering only one or two Cloud services from the list, and therefore:
- Most large SIs do not have a SaaS/Cloud strategy, or, their strategy is to wait and see how the market develops.
Accenture is a Salesforce.com partner. Ernst & Young are helping implement EmployU and HumanWave. Deloitte is with Workday and Genpact is a Netsuite implementor.
Capgemini is working with AWS to provide a Cloud Computing COE.
Considering the size and nature of the business of these players, it seems that these services are not high on the priority lists of the big SIs. It is quite obvious (to me) that these giants have not defined a SaaS strategy, rather they are reacting in an opportunistic manner to the market - akin to a “me too” tactic, just to have something 'Cloudy' or 'SaaSy' on their web site.
Cloud Strategy – React or Lead
As mentioned, many SIs have adopted tactical approaches, at the Cloud Service level, rather than a strategic approach. Further analysis led us to the conclusion that by zooming out, grouping and mapping the above Cloud Services. we can define five Cloud Strategies for System Integrators:
- SaaS Aggregator – Provide Applications on the cloud
- SaaS One-stop-Shop for Software Vendors
- Cloud/SaaS Adoption for IT
- Private cloud technologies
- BPasS - Could Integration
Using this prism, we could say that a number of players have chosen a strategic path:
IBM’s GBS and Wipro are offering services that we define as SaaS One-stop-shop for ISVs (strategy 3). KPMG is offering Roadmap for SaaS Adoption (strategy 5) while Infosys is acting as a SaaS Aggregator for enterprise IT (strategy 1). Smaller players such as iProcess are offering BpassS (Strategy 2)
Even though adoption rates are growing very fast, the hype of Cloud Computing is a couple of years ahead of reality, especially in the larger organizations that are the natural customers of the leading SIs. Some SIs have chosen to play it safe: not invest up front in new technologies and methodologies and watch the market carefully. Since they will still be milking many fat cows for a few years to come, the decision not to decide could be considered a smart, conservative strategy.
Others, though, have taken leadership positions, risking investments without a clear date on the ROI. They have the advantage of defining the market trends and have a better chance of emerging as the de-facto leaders in a few years when all the players, that are currently sitting on the fence, will have to define their strategy and see what bones are left to pick.
Friday, December 24, 2010
Automating SaaS Operations
"The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency" - Bill Gates
This is an article which I posted at the Nolio blog site as a guest blogger back in 2009. Not only is is relevant today, but perhaps even more so...
The Next Killer App
If I had a great idea for the next killer app (I have, actually) and if I had unlimited funds (I don’t, actually) I would have built the software as an on-demand offering.
I would have spent half my funds on building the operational support systems – provisioning, billing, retention policy, self-service, report generator, etc. The other half would be invested in building instrumentation, redundancy, automation, integration, application level monitoring, silent upgrades, customer notifications, and so on.
The rest of the money (you may wonder about my math, but hey, I’ve got unlimited funds) would go towards building the actual application.
Most SaaS vendors out there (and they are growing fast) have chosen the predictable path of building the application first, and worrying about serviceability later. This is the fastest way of getting to market with low costs. The next step is choosing some viable hosting solution and off we go, offering the world our ever better CRM.
Growth
Many months and dozens of customers later, reality hits with all the issues of servicing the software, rapid growth and dealing with labor intensive tasks that are the humdrum of daily life in a SaaS operation. Provisioning/de-provisioning, configuration changes, customized reports, and the most dreaded – upgrades, task the team as a whole, especially when the product is successful and the number of customers is growing daily.
It is not that SaaS executives, architects and engineers are lacking in any way. On the contrary, they are mostly smart, inventive, and creative and have a deep understanding of their customers’ needs in the specific domain. The problem is that they are product people, not service people. Practically none of them come from IT and cannot envision the life of a service operations engineer.
At this point, automation becomes crucial to the survival of the business.
Whether it is built into the next version (many architectures make this quite difficult) or done externally, automation is needed to reduce costs, physical labor, frustration and mainly, error-prone manual procedures. Repeatability, which is a derivative of automation, is also crucial.
Automation is needed across the board. Be it in setting up a new server, or building a new application instance. It could be a manual procedure regarding provisioning of application resources, or building a seamless upgrade procedure.
Outages happen. How quickly can you recover from a service disruption and ensure that the recovery does not create it own problems? Automation not only provides the routines for quick recovery, but instills a discipline of thinking out the necessary steps, discovering dependencies and planning ahead. An added benefit of automation is that it documents the process so you can go back and review the best and worst of your procedures.
In my next post I will take a closer look at the SaaS Upgrade Nightmare.
This is an article which I posted at the Nolio blog site as a guest blogger back in 2009. Not only is is relevant today, but perhaps even more so...
The Next Killer App
If I had a great idea for the next killer app (I have, actually) and if I had unlimited funds (I don’t, actually) I would have built the software as an on-demand offering.
I would have spent half my funds on building the operational support systems – provisioning, billing, retention policy, self-service, report generator, etc. The other half would be invested in building instrumentation, redundancy, automation, integration, application level monitoring, silent upgrades, customer notifications, and so on.
The rest of the money (you may wonder about my math, but hey, I’ve got unlimited funds) would go towards building the actual application.
Most SaaS vendors out there (and they are growing fast) have chosen the predictable path of building the application first, and worrying about serviceability later. This is the fastest way of getting to market with low costs. The next step is choosing some viable hosting solution and off we go, offering the world our ever better CRM.
Growth
Many months and dozens of customers later, reality hits with all the issues of servicing the software, rapid growth and dealing with labor intensive tasks that are the humdrum of daily life in a SaaS operation. Provisioning/de-provisioning, configuration changes, customized reports, and the most dreaded – upgrades, task the team as a whole, especially when the product is successful and the number of customers is growing daily.
It is not that SaaS executives, architects and engineers are lacking in any way. On the contrary, they are mostly smart, inventive, and creative and have a deep understanding of their customers’ needs in the specific domain. The problem is that they are product people, not service people. Practically none of them come from IT and cannot envision the life of a service operations engineer.
At this point, automation becomes crucial to the survival of the business.
Whether it is built into the next version (many architectures make this quite difficult) or done externally, automation is needed to reduce costs, physical labor, frustration and mainly, error-prone manual procedures. Repeatability, which is a derivative of automation, is also crucial.
Automation is needed across the board. Be it in setting up a new server, or building a new application instance. It could be a manual procedure regarding provisioning of application resources, or building a seamless upgrade procedure.
Outages happen. How quickly can you recover from a service disruption and ensure that the recovery does not create it own problems? Automation not only provides the routines for quick recovery, but instills a discipline of thinking out the necessary steps, discovering dependencies and planning ahead. An added benefit of automation is that it documents the process so you can go back and review the best and worst of your procedures.
In my next post I will take a closer look at the SaaS Upgrade Nightmare.
Thursday, December 16, 2010
2 x E-cube = S-cube - Simple math for SaaS Scalability Success
"The circumstances of human society are too complicated to be submitted to the rigor of mathematical calculation" (Marquis De Custine)
Recently I posted a discussion on the link between success with SaaS and scalability and how, therefore, the Service Operations needs to be geared to handle scale and deal with fast growth.
On a recent consultation engagement, I gave a presentation to the board members of a SaaS company that decided to penetrate the SMB, following their hardships in securing big deals in the enterprise market.
As part of the thought process and brainstorming session I came up with a marketing catchphrase to make a point. At the time I did not think much about it, but as I was working on the board presentation, I realized that there it was much deeper than I had originally thought.
I would like to share this with you.
The First Cube
What I drew on the white-board was roughly this artistic creation:
Announcing: “E-cube is the winning formula!”
- Easy to Buy
- Easy to Implement
- Easy to Use
Easy to Buy - Their product is very easy to implement, and easy to on-board a new organization, (although it is a complex product). There is a technical issue that involves an architectural change, so that became part of the plan.
Easy to Use - the product has a great user interface and intuitive flow. They need to add tutorial videos.
Easy to Buy – That was lacking and that was what we decided to focus our efforts on. It included a lead generation program, a no-touch free trial and an improved landing site.
The Second Cube
As I was working on what it would take from the company’s end to deliver scalability, I realized there was another E-cube involved:
- Easy to Sell
- Easy to Scale
- Easy to Maintain
It is all nice and well if you can bring thousands of leads to your site with the E-cube formula, but if you cannot convert these leads into paying customers and then give them the best service on a controlled budget, you will have not achieved your scalability goals.
So, Easy to Sell means - a simple pricing model and a top-notch insides-sales team (not easy to come by though) backed up by funnel management software.
Easy to Scale means - a solid, configurable, multi-tenant architecture, self service features and automated procedures.
Easy to Maintain means – a full featured Operations Support System, Service Operations practices and the discipline to enforce them.
So with two E-cube guidelines one can achieve SaaS Scalability Success.
Ergo: 2 x E3 = S3
Q.E.D.
Thursday, December 09, 2010
The SaaS Consumer’s Point of View – Negotiating an Agreement
“The food was superb, the atmosphere was great, the service was outstanding; it was those goddamned customers that had to ruin it all”. (Morris Green, restaurateur, NYC, 1987)
This short article is an introduction to an interesting blog post that was published recently by Derek Singleton from ERP Software Advice, but before I let you go, I want to take this opportunity to talk about the customer’s perspective.
My writings, presentations and webinars have been mostly dedicated to the point of view of the software provider, whether SaaS or SaaS-to-be. I would like to present a different point of view.
Throughout the years of being on the software provider’s side I have learned a thing two on what makes a happy customer and that is the guiding light I have been trying to follow for years.
I believe that if we understand the customer’s perspective we have a better chance of providing a good service and creating a happy, loyal customer base.
Who is the customer?
SaaS adoption has been mostly done in a haphazard fashion throughout the years. Many of the early adopters were business managers at the department level in the enterprise.
Even within smaller companies, the decision to consume SaaS was usually a point solution, for a particular issue to handle and not as part of a well thought process and methodology.
In many cases the IT department and CIOs were kept out of the loop in defining needs, selecting the service, negotiating the deals and the process of provisioning and de-provisioning. In the extreme, IT found out about their company consuming a web application only when a user would call the help desk and ask for support.
As more CIOs are ‘getting it’, as more IT departments are becoming cloud-oriented they are becoming that target customers, rather than the end users. They are usually better equipped (once their fears are neutralized) to judge the provider, the application, the integration and to negotiate a better deal for the organization.
These IT professionals should be planning a Roadmap for SaaS so that the consumed applications become part of a coherent plan rather than something the cat dragged in.
We should start focusing on this new generation of SaaS customers.
An interesting article covering the negotiations with a provider is therefore presented.
Saturday, November 13, 2010
SaaS, Scalability and the Three Little Pigs
“We have the wolf by the ears, and we can neither hold him, nor safely let him go” - Thomas Jefferson
A few days ago I gave a talk at a SaaS Business Challenges conference and I would like to share the main theme with my readers.
It’s the Service, Dummy!
When SaaS burst upon the scene about ten years ago, the first consumers were of two types: Those that used on-demand as an ideology, having foreseen the cloud revolution early on, and those that had no choice, because the other option was an expensive, time consuming and complex solution.
Back then, there were few SaaS application or service choices, so customers had to be very forgiving about the service levels and were willing to put up with reduced functionality, outages and low response times. SLAs, if they even existed, were non- binding and lacked both depth and breadth.
These days, however, when there are dozens of SaaS applications for every need, the differentiator is no longer the functionality. Most SaaS applications offer a similar set of capabilities, and as applications change on a bi-weekly or monthly basis, features are added on an on-going basis.
Even if one comes up with a revolutionary solution and provides the only SaaS offering of its kind, it is safe to say that within a year, three new SaaS companies will offer the same, or an improved set of capabilities.
So what differentiates one SaaS offering from another? It’s the Service, dummy!
Features don’t make a loyal customer – outstanding service does.
Scalability
There aren’t many SaaS companies out there that service a few dozen Fortune 200 companies while keeping the profitability high. The general rule is that margins are small and that profitability is achieved through hundreds or thousands of customers.
Giving great service to 20 or 30 customers is a no-brainer. Just throw more bodies at the problem and you will achieve a highly satisfied customer base.
But what happens when these numbers multiply rapidly? You’ll soon find out that what worked for a few dozens might collapse at the next order of magnitude.
When I approach SaaS companies at the stage when they already have a growing customer base and warn them about the perils of scaling up, the usual response I get is:
“I wish I will have to deal with that problem”, meaning that they would love to have 200 hundred customers that cause strain on the infrastructure and operations and deal with those ‘good’ problems then. It is only human to postpone these issues when they are not burning your behind.
To those CEOs I would say: “Pack your stuff, return the money to your investors and go look for another job elsewhere”.
Your investors did not give you their hard earned dollars for a proof of concept.
They invested in you because they believed that you will bring in thousands of customers.
My best clients are SaaS companies that come to me when they are in pain. They start loosing customers because they did not build an operation capable of handling the scale which they had wished for.
As I have stated time and again – SaaS companies usually consist of outstanding, creative developers that build great technology, but they don’t have the IT and/or operational experience. They lack the know-how and especially the methodology for building a successful, scalable operation.
Building Operations for Scalability
I have written much about these issues and they can all be found in previous blog postings.
Suffice to say that the setup you need to build for a scalable service operations includes:
· Methodology – the framework of practices, templates, workflows and tools
· Operations Support Systems – everything else your engineering team left out of the product.
· Executive buy-in and awareness -define the metrics, and provide tools to capture and analyze those metrics.
Can a Huff and Puff Blow Your House Down?
So what has all this to do with the Three Little Pigs?
The first little pig built a house of straw, since he didn't have time to invest in operational infrastructure and wanted to make a quick exit. He ended his career as wolf poop.
The second little pig invested in a better infrastructure but did not pay attention to the practices and processes. He lasted longer in his wooden structure but was huffed and puffed and blown away by the competition.
The third little pig took his time and invested executive attention in doing it right from the start. He lived happily ever after.
So even if you cannot afford a brick house from day one, you should have the blueprints ready and the determination to add the bricks when they become available.
By the time the Scalability Wolf arrives, you should have a sturdy enough structure and react quickly to threats, to survive and prosper.
Thursday, June 10, 2010
Private Clouds – What’s in a Name?
“Happiness is like a cloud, if you stare at it long enough, it evaporates” - Sarah McLachlan
Remember when all of our secretaries and stewardesses turned into office managers and flight attendants overnight? Remember when all the co-los and server-hosting companies became cloud providers overnight?
(To alleviate all suspicions – as my readers know, I am an ardent advocate for Cloud computing and SaaS in particular, so this post is not about arguing the merits of this constructive and disruptive trend).
Cloud means different things to different people – but mainly it means “a cool way to market my same old, tired stuff”. I worked with a co-lo provider a couple of years back. A few months ago I went to their site to check prices and lo and behold: They became a Cloud Provider! They had clouds splashed all over their site and every solution they sold was a Cloud solution. I called up a sales person and asked about their elasticity and time units. Turns out they were elastic in one direction – you could always order more servers – and you only had to commit to one year in advance. On a geological time scale that is quite flexible. When I laughed and asked what was ‘cloudy’ about their offering, the guy got confused and said that his manager will get back to me.
So what is all this newspeak about Private Clouds?
Guess what is happening to the good ol’ data center? As David Linthicum aptly puts it in his latest blog post: “the reality is that ‘private cloud’ is just another term for on-premise systems”.
Point number one is that I find the term an Oxymoron:
Cloud means that it is ‘somewhere out there’; location is transparent. Cloud means sharing; resources are transparent. The physical server I used this morning might be used by someone else this afternoon.
Private means it is in my back yard and only I get to play in the sandbox.
Point number two is that even if we apply the Cloud concepts to the enterprise, it will be relevant to a very small number of very big players. Those enterprises that are truly global and distributed. They could take advantage of the peaks and troughs, of the "follow the sun" model, of the large numbers and justify the investment of a Cloud player.
And what about the rest of us mortals? Let’s assume that a certain percentage of our IT services will not transition to the cloud – be it regulation, compliance, perceived loss of control, or the illusion of maintaining job security. We should be using Cloud-enabling technologies, to make a smarter use of our resources and data centers. That means virtualization, automation, orchestration and auto-provisioning technologies. We only get the silver lining – not the cloud.
So the Private Cloud, in essence, is a wonderful opportunity for the big vendors out there to sell to the enterprises - new equipment, new systems and new services.
Remember when all of our secretaries and stewardesses turned into office managers and flight attendants overnight? Remember when all the co-los and server-hosting companies became cloud providers overnight?
(To alleviate all suspicions – as my readers know, I am an ardent advocate for Cloud computing and SaaS in particular, so this post is not about arguing the merits of this constructive and disruptive trend).
Cloud means different things to different people – but mainly it means “a cool way to market my same old, tired stuff”. I worked with a co-lo provider a couple of years back. A few months ago I went to their site to check prices and lo and behold: They became a Cloud Provider! They had clouds splashed all over their site and every solution they sold was a Cloud solution. I called up a sales person and asked about their elasticity and time units. Turns out they were elastic in one direction – you could always order more servers – and you only had to commit to one year in advance. On a geological time scale that is quite flexible. When I laughed and asked what was ‘cloudy’ about their offering, the guy got confused and said that his manager will get back to me.
So what is all this newspeak about Private Clouds?
Guess what is happening to the good ol’ data center? As David Linthicum aptly puts it in his latest blog post: “the reality is that ‘private cloud’ is just another term for on-premise systems”.
Point number one is that I find the term an Oxymoron:
Cloud means that it is ‘somewhere out there’; location is transparent. Cloud means sharing; resources are transparent. The physical server I used this morning might be used by someone else this afternoon.
Private means it is in my back yard and only I get to play in the sandbox.
Point number two is that even if we apply the Cloud concepts to the enterprise, it will be relevant to a very small number of very big players. Those enterprises that are truly global and distributed. They could take advantage of the peaks and troughs, of the "follow the sun" model, of the large numbers and justify the investment of a Cloud player.
And what about the rest of us mortals? Let’s assume that a certain percentage of our IT services will not transition to the cloud – be it regulation, compliance, perceived loss of control, or the illusion of maintaining job security. We should be using Cloud-enabling technologies, to make a smarter use of our resources and data centers. That means virtualization, automation, orchestration and auto-provisioning technologies. We only get the silver lining – not the cloud.
So the Private Cloud, in essence, is a wonderful opportunity for the big vendors out there to sell to the enterprises - new equipment, new systems and new services.
Friday, May 21, 2010
The SaaS VP Operations as Product Manager
SaaS Operations Support Systems
“'Tis not enough to help the feeble up, but to support them after” - William Shakespeare
Every SaaS company needs to deal with numerous functions that are not necessarily part of the technological stack that originally came with the application, namely the Operations Support Systems.
If a SaaS start-up had unlimited time and funds to plan and build the perfect solution, they would probably all be in the Caribbean islands doing what people with unlimited time and funds do.
Just Do It!
Because of the nature of monetizing SaaS, companies try to get to market as soon as possible, getting subscriptions fees streaming in. Sometimes, they even launch with a half baked solution that will provide added value to the customers at the expense of future operational headaches.
The logic behind this is that dealing with a scalability problem, is a good thing. In other words - who doesn’t want to reach the stage when too many customers are taxing the team? We’ll deal with that when it becomes a problem.
These days, numerous PaaS offerings or SaaS frameworks offer built-in operational support systems functionality, but many of the necessary features are not supported.
Most of the SaaS application in the market were not built with these frameworks for various reasons. The most prevalent are that they were not available a few years ago, and that engineers have a tendency to build everything from scratch, or using frameworks (e.g. LAMP, Java, .Net, RoR) that they are familiar with.
When the typical SaaS service is launched, it lacks functionality that would support the scalability of the service operation. It is left to the Service Operations team to deal with all the 'maturity' functionality that the product lacks.
Let us examine some of the Operations Support Systems functions that are typically not handled by the application.
On-boarding new customers
One would expect most SaaS systems to have automatic provisioning. While that is true in many cases, a lot of the systems allow the customer to define users, but creating a new customer entity is left to the Support or Operations team. That may require generating a new database or schema, or setting up storage, etc. If the company is adding one customer a week, it may be manageable, but at a higher rate this is a taxing job and error prone.
De-provisioning
While it is expected that some level of automatic provisioning is provided with the product, very few SaaS applications provide a simple (never mind automatic) mechanism for removing existing customers. Very few SaaS developers design with of the prospect of loosing a customer in mind. Beyond the task of removing dependencies from the database, there are issues of releasing resources, and removing customizations.
Billing
Such a basic function, (one would think, for a company that lives or dies by subscriptions) is typically lacking from most SaaS infrastructures. While the situation is improving dramatically with the advent of PaaS and SaaS development frameworks, many systems still start out with excel sheets and a lot of manual work. Customer data must be extracted by some ad-hoc solution form the application database or is duplicated in the CRM, which causes endless synchronization errors. Ad-hoc solutions are implemented (either in-house or SaaS billing solutions) as the billing becomes ever more complex but until that occurs the brunt of the work falls upon the Operations team.
Metering is a whole new layer of complexity, if the billing is more complex than a fixed amount per seat per month.
Retention Policy
Some SaaS companies plan ahead for resource consumption by their customers, bur many start dealing with the issue only when they begin running out of storage, or when their storage costs are starting to hurt. Most customers want to retain their data forever on the provider’s disks, but that is impossible. So a retention policy must be defined and followed (such as delete files that are older than X months or larger than Y gigabytes).
The problem is that the application does not support that, so manual work is requried or ad-hoc solutions have to be built around the product.
Failover and Backup
Automatic recovery and failover are rarely built into the initial solution and are usually managed by the Operations team via building complex solutions with networking boxes.
This is true for backup and recovery mechanisms as well. The Operations team frequently has to build mechanisms around the production to support backup and recovery and most often they require much manual labor.
Application Monitoring
Well designed software is packed with instrumentation that is easy to turn on and off and easy to monitor and interpret. Not all SaaS systems have that built-in capability and the Operations team has to create an application-specific, monitoring infrastructure that can detect and react quickly to service degradation.
Seamless Upgrade
Upgrades are the recurring nightmare of any Operations team, especially on applications that have a demanding uptime SLA. Few SaaS systems are designed with that goal in mind. It usually takes a level of maturity and a sizeable customer base to get Engineering to consider revising the code to allow some sub-systems a no-downtime upgrade.
The problem is that is usually requires major revisions of the code if the system was not designed a-priori to handle a silent upgrade.
SLA Management
Whether you are using automated SLM or not (chances are you are not), you need to take into consideration the various aspects of your service that need to be metered, tracked and compared against a set of Service Level Objectives.
End User Broadcasting
Sometimes it is necessary to communicate with all of you users, or a sub group of them in real time. What a better option is there that to pop up a message on the end-user’s browser that you can compose on the spot or pull from a list of canned messages? The operations team need an integrated solution with the product to be able to do that.
Operations Console
And to tie it all up, a separate application that controls all the operational aspects of the service is needed. Included should be: provisioning/de-provisioning customers, password management, real-time login view of current users, real-time view of application usage, customer communication console, production environment control, etc.
There are many other operational features that are typically not found in a SaaS application as it leaves the factory floor. Just to mention a few: Integration, Reporting engine and reporting Database, Security, Scale-up and Scale-out capabilities, Sandbox, Status Page.
Development and Product management Experience Required
The VP Operations (or whatever title the job carries) is required to be a product manager of sorts, and a background in software development is almost a must. The Operations manager should be highly involved in the product roadmap and insist on having a say in defining future releases. There will be a contention between investing in Serviceability versus Functionality. Since the paying customers require more functionality, it is usually an uphill battle to gain service upgrades to the product.
While building an organic set of solutions into the product may practically take years, Operations cannot continue to throw bodies at solving scalability and downtime issues. So beyond influencing the product group on the direction in which the application should be developed, the Operations manager needs to build a set of tools addressing the Operations Support Systems needs as stated above. One option is to nurture a relationship with VP Engineering and get resources from her group to build well defined solutions that could each be completed in a couple of weeks of work. This is especially true in early stage companies where the Operations team is small and the engineers take the brunt of many of the operations’ tasks. The VP Engineering would appreciate the need as members of her team are feeling the pain as well.
In a more established company, the VP Operations must make sure that there are coding/scripting capabilities in the team, so simple projects and tools could be developed within the team, with minimal aid from the Engineering group.
“'Tis not enough to help the feeble up, but to support them after” - William Shakespeare
Every SaaS company needs to deal with numerous functions that are not necessarily part of the technological stack that originally came with the application, namely the Operations Support Systems.
If a SaaS start-up had unlimited time and funds to plan and build the perfect solution, they would probably all be in the Caribbean islands doing what people with unlimited time and funds do.
Just Do It!
Because of the nature of monetizing SaaS, companies try to get to market as soon as possible, getting subscriptions fees streaming in. Sometimes, they even launch with a half baked solution that will provide added value to the customers at the expense of future operational headaches.
The logic behind this is that dealing with a scalability problem, is a good thing. In other words - who doesn’t want to reach the stage when too many customers are taxing the team? We’ll deal with that when it becomes a problem.
These days, numerous PaaS offerings or SaaS frameworks offer built-in operational support systems functionality, but many of the necessary features are not supported.
Most of the SaaS application in the market were not built with these frameworks for various reasons. The most prevalent are that they were not available a few years ago, and that engineers have a tendency to build everything from scratch, or using frameworks (e.g. LAMP, Java, .Net, RoR) that they are familiar with.
When the typical SaaS service is launched, it lacks functionality that would support the scalability of the service operation. It is left to the Service Operations team to deal with all the 'maturity' functionality that the product lacks.
Let us examine some of the Operations Support Systems functions that are typically not handled by the application.
On-boarding new customers
One would expect most SaaS systems to have automatic provisioning. While that is true in many cases, a lot of the systems allow the customer to define users, but creating a new customer entity is left to the Support or Operations team. That may require generating a new database or schema, or setting up storage, etc. If the company is adding one customer a week, it may be manageable, but at a higher rate this is a taxing job and error prone.
De-provisioning
While it is expected that some level of automatic provisioning is provided with the product, very few SaaS applications provide a simple (never mind automatic) mechanism for removing existing customers. Very few SaaS developers design with of the prospect of loosing a customer in mind. Beyond the task of removing dependencies from the database, there are issues of releasing resources, and removing customizations.
Billing
Such a basic function, (one would think, for a company that lives or dies by subscriptions) is typically lacking from most SaaS infrastructures. While the situation is improving dramatically with the advent of PaaS and SaaS development frameworks, many systems still start out with excel sheets and a lot of manual work. Customer data must be extracted by some ad-hoc solution form the application database or is duplicated in the CRM, which causes endless synchronization errors. Ad-hoc solutions are implemented (either in-house or SaaS billing solutions) as the billing becomes ever more complex but until that occurs the brunt of the work falls upon the Operations team.
Metering is a whole new layer of complexity, if the billing is more complex than a fixed amount per seat per month.
Retention Policy
Some SaaS companies plan ahead for resource consumption by their customers, bur many start dealing with the issue only when they begin running out of storage, or when their storage costs are starting to hurt. Most customers want to retain their data forever on the provider’s disks, but that is impossible. So a retention policy must be defined and followed (such as delete files that are older than X months or larger than Y gigabytes).
The problem is that the application does not support that, so manual work is requried or ad-hoc solutions have to be built around the product.
Failover and Backup
Automatic recovery and failover are rarely built into the initial solution and are usually managed by the Operations team via building complex solutions with networking boxes.
This is true for backup and recovery mechanisms as well. The Operations team frequently has to build mechanisms around the production to support backup and recovery and most often they require much manual labor.
Application Monitoring
Well designed software is packed with instrumentation that is easy to turn on and off and easy to monitor and interpret. Not all SaaS systems have that built-in capability and the Operations team has to create an application-specific, monitoring infrastructure that can detect and react quickly to service degradation.
Seamless Upgrade
Upgrades are the recurring nightmare of any Operations team, especially on applications that have a demanding uptime SLA. Few SaaS systems are designed with that goal in mind. It usually takes a level of maturity and a sizeable customer base to get Engineering to consider revising the code to allow some sub-systems a no-downtime upgrade.
The problem is that is usually requires major revisions of the code if the system was not designed a-priori to handle a silent upgrade.
SLA Management
Whether you are using automated SLM or not (chances are you are not), you need to take into consideration the various aspects of your service that need to be metered, tracked and compared against a set of Service Level Objectives.
End User Broadcasting
Sometimes it is necessary to communicate with all of you users, or a sub group of them in real time. What a better option is there that to pop up a message on the end-user’s browser that you can compose on the spot or pull from a list of canned messages? The operations team need an integrated solution with the product to be able to do that.
Operations Console
And to tie it all up, a separate application that controls all the operational aspects of the service is needed. Included should be: provisioning/de-provisioning customers, password management, real-time login view of current users, real-time view of application usage, customer communication console, production environment control, etc.
There are many other operational features that are typically not found in a SaaS application as it leaves the factory floor. Just to mention a few: Integration, Reporting engine and reporting Database, Security, Scale-up and Scale-out capabilities, Sandbox, Status Page.
Development and Product management Experience Required
The VP Operations (or whatever title the job carries) is required to be a product manager of sorts, and a background in software development is almost a must. The Operations manager should be highly involved in the product roadmap and insist on having a say in defining future releases. There will be a contention between investing in Serviceability versus Functionality. Since the paying customers require more functionality, it is usually an uphill battle to gain service upgrades to the product.
While building an organic set of solutions into the product may practically take years, Operations cannot continue to throw bodies at solving scalability and downtime issues. So beyond influencing the product group on the direction in which the application should be developed, the Operations manager needs to build a set of tools addressing the Operations Support Systems needs as stated above. One option is to nurture a relationship with VP Engineering and get resources from her group to build well defined solutions that could each be completed in a couple of weeks of work. This is especially true in early stage companies where the Operations team is small and the engineers take the brunt of many of the operations’ tasks. The VP Engineering would appreciate the need as members of her team are feeling the pain as well.
In a more established company, the VP Operations must make sure that there are coding/scripting capabilities in the team, so simple projects and tools could be developed within the team, with minimal aid from the Engineering group.
Thursday, March 18, 2010
Change Management and the Sanctity of Production
“Most people are afraid of change. We love it!” (Sign of a beggar on a street in San Francisco, 2004)
(Note: This article is part of the STORM™ methodology)
Change is the greatest cause for service interruption in any IT operations. Period. Stop. Exclamation mark.
In the extreme, one might argue that change is the cause for every service interruption if one counts hardware or software malfunctions as a change as well.
SaaS operations tend to suffer more from a lack of proper change management for two reasons: First, the consequences of a service outage for a company whose entire existence depends on its service, is dire. Second, SaaS engineers, as I have written in numerous posts, lack the discipline that is more inherent in IT departments.
And yet, my experience has been that in most SaaS companies, changes are unsupervised, undocumented, unauthorized, unplanned, (sometimes unnecessary), underestimated and un_____ (fill in the blanks).
The importance of Change Management cannot be overstated and it is the first practice that I have implemented at companies that I worked at (or for). I wince when I recall the casualness which I have witnessed at various SaaS companies about making changes in the production system. I can quote my former boss, Mansur Salame, CEO of Contactual, saying that “production should be treated as sacred, with utmost respect appropriate to holy places” (or something of the sort). And, boy, were we sacrilegious back in those days!
In the Chapter on Change Management in the upcoming book, I will present my STORM™ adaptation with much detail.
In this post I will outline some guidelines for sane change management.
Sixty Seconds on Change Management
Below are listed the objects that comprise a comprehensive Change Management practice.
RFC – Request for Change document. Must initiate the process, regardless how small or major the changes are. It should include the what, the why, the when, the risk, the potential impact (on customers or components), and a checklist of notifications and tests that should/should not be done.
A Change Window must be defined, clearly notating what type of changes to what subsystems are allowed at which days, during what hours.
Change Calendar – which might be implemented in a number of static or automatic formats, must represent the ‘Change Window’, and depict all planned changes by the company, service providers and customers, and must be part of the RFC process.
Change Advisory Board or CAB is the pre-determined group or people who scrutinize and approve the RFC. The CAB may be large or small but it should include at least one person who is not involved in the request and planning process. The CAB may meet on a recurring schedule or as needed.
A Change Record is a record describing a change that occurred in production or the eco-system. It could be implemented as a database or excel or within a ticketing system. It should include the what, when and impact (on customers or components). Much important information could be derived from this data store that pertains to the Incident and Availability Management practices.
The Maintenance Plan is a detailed document defining the pre-requisite tasks, the maintenance tasks, rollback tasks and post-maintenance tasks. Each task should have a description, an owner, a time and duration. In most cases the plan must be scrutinized to the lowest detail level, and practiced in a Pre-Production environment that should mimic the production environment as much as possible.
The sixty seconds are over. This was just a teaser. Obviously there are templates, workflows and a sleuth of details that tie all of these objects into a well-oiled practice. The book will expand on the details and include the workflows, the templates and methods for automation.
To summarize; as the market matures and competition thrives, the big differentiator will be the second ‘S’ in SaaS and customers will become less and less forgiving. If a SaaS company does not practice a robust Change Management practice it will end up paying in a frustrated staff and customer churn.
(Note: This article is part of the STORM™ methodology)
Change is the greatest cause for service interruption in any IT operations. Period. Stop. Exclamation mark.
In the extreme, one might argue that change is the cause for every service interruption if one counts hardware or software malfunctions as a change as well.
SaaS operations tend to suffer more from a lack of proper change management for two reasons: First, the consequences of a service outage for a company whose entire existence depends on its service, is dire. Second, SaaS engineers, as I have written in numerous posts, lack the discipline that is more inherent in IT departments.
And yet, my experience has been that in most SaaS companies, changes are unsupervised, undocumented, unauthorized, unplanned, (sometimes unnecessary), underestimated and un_____ (fill in the blanks).
The importance of Change Management cannot be overstated and it is the first practice that I have implemented at companies that I worked at (or for). I wince when I recall the casualness which I have witnessed at various SaaS companies about making changes in the production system. I can quote my former boss, Mansur Salame, CEO of Contactual, saying that “production should be treated as sacred, with utmost respect appropriate to holy places” (or something of the sort). And, boy, were we sacrilegious back in those days!
In the Chapter on Change Management in the upcoming book, I will present my STORM™ adaptation with much detail.
In this post I will outline some guidelines for sane change management.
Sixty Seconds on Change Management
Below are listed the objects that comprise a comprehensive Change Management practice.
RFC – Request for Change document. Must initiate the process, regardless how small or major the changes are. It should include the what, the why, the when, the risk, the potential impact (on customers or components), and a checklist of notifications and tests that should/should not be done.
A Change Window must be defined, clearly notating what type of changes to what subsystems are allowed at which days, during what hours.
Change Calendar – which might be implemented in a number of static or automatic formats, must represent the ‘Change Window’, and depict all planned changes by the company, service providers and customers, and must be part of the RFC process.
Change Advisory Board or CAB is the pre-determined group or people who scrutinize and approve the RFC. The CAB may be large or small but it should include at least one person who is not involved in the request and planning process. The CAB may meet on a recurring schedule or as needed.
A Change Record is a record describing a change that occurred in production or the eco-system. It could be implemented as a database or excel or within a ticketing system. It should include the what, when and impact (on customers or components). Much important information could be derived from this data store that pertains to the Incident and Availability Management practices.
The Maintenance Plan is a detailed document defining the pre-requisite tasks, the maintenance tasks, rollback tasks and post-maintenance tasks. Each task should have a description, an owner, a time and duration. In most cases the plan must be scrutinized to the lowest detail level, and practiced in a Pre-Production environment that should mimic the production environment as much as possible.
The sixty seconds are over. This was just a teaser. Obviously there are templates, workflows and a sleuth of details that tie all of these objects into a well-oiled practice. The book will expand on the details and include the workflows, the templates and methods for automation.
To summarize; as the market matures and competition thrives, the big differentiator will be the second ‘S’ in SaaS and customers will become less and less forgiving. If a SaaS company does not practice a robust Change Management practice it will end up paying in a frustrated staff and customer churn.
Sunday, January 31, 2010
SLA Consequences to Service Operations
"What, me worry?" (Alfred E. Neuman)
In my previous post I discussed some basic concepts about SLAs, SLOs and penalties. As promised, I am addressing the ‘who cares?’ question.
From a Service Operations point of view, you may shrug your shoulders and claim that these are issues with the Legal and Finance departments. Most likely you were brought on board later in the game and never viewed an SLA until your were forced to do so.
As the person responsible for keeping all the services up and running, it may be best to keep the SLA to a minimum. After all, a document containing vague language, with little commitment and liability would be hard to wave in front of your face when the service levels drop.
I will argue that vagueness will play against you. A tough SLA will require the company to adhere to the high service levels they are committed to, and yes, pay the penalties for breaching these agreements. Keep in mind that if your service level drops one time too many, the legalese you will be hiding behind will not save your butt when customers drop from the service or simply do not renew.
I would take it even one step further. I advocate that the Service Ops managers bonuses are tied to achieving those SLOs that will keep a smile on the customers’ faces. (typically up-time and response time, but in some cases there are other objectives that are crucial to the customers). The carrot and the stick should work nicely to assure that you are doing the utmost to live up to the agreements.
Commitments
Another issue that concerns you (Service Operations) is that Sales are making commitments that you are suppose to keep, usually without you ever knowing about it. Operations needs to initiate a fact finding effort to learn what is there. You need to know what you are capable of providing. Everybody likes to state that they are five nines (99.999% uptime) but how many companies out there really are? You need to monitor and test your service over a substantial period of time before you commit to those numbers.
Another point in favor of having a good grasp of your SLA is that you, as a consumer of services would be conscience of your requirements vis-à-vis your service providers.
That will include the hosting services, your ISP, your communications provider, and whatever cloud services you are using. In my past positions as VP Service Operations I have been appalled by the contracts that my predecessors have signed with service providers. Some of them had no consequences to service level degradation. Others had ridiculous clauses such as 'for every hour of downtime, the credit would be for one hour prorated service cost' which meant that there was no real penalty. Another contract stated that we could get out of the agreement if for three months in a row(!) the service provided was available for less that 75% of the time.
We would have been out of business by then.
Where are those damn SLAs?
As we have seen, SLAs that are broad and meaningful will be complex. Add to that various service levels such as Standard, Gold and Platinum and the fact that some customers have negotiated special terms for themselves, and you are dealing with a mean, slimy problem.
To compound that problem, nine times out of ten, these documents are sitting on someone’s laptop in a PDF format with perhaps a hard copy in a dusty folder, in the cabinet below the espresso machine.
Imagine the exercise of figuring out if an SLA was breached for a particular customer, and if that breach carries a penalty.
I have painfully gone through that exercise too many times, and believe you me - I had much better things to attend to following a service outage. The process was extremely slow, finding the various documents, looking up the terms and comparing the events with them.
Then a calculation was needed as to how much credit was due. And all this was done for a single customer. Multiply that by the number of customers that may have been affected and you have just wasted many good hours of Solitaire.
SLA Management Tools
There are multiple tools out there (some are offered as SaaS) to manage your SLAs. Many of them provide a full cycle of defining SLOs, creating SLAs, generating the documents, monitoring performances against obligations, computing compensation and generating reports. I have not used any of them (although I used to work at an SLM ISV), so am not about to promote any single one, but there are very slick solutions available.
If you are at an early stage, it would be hard sell for you to justify to management that you need to start paying for a service that possibly no one in the company comprehends.
Typically, when a SaaS company starts out there are very simple, non-abiding, fixed SLAs, so there is very little attention paid to this aspect of the business.
As with any aspect of SaaS Service Operations, scalability issues hit you when you least expect them.
As most (all?) SaaS companies do not start with Service Level Management software, by the time it becomes a burden they will have many dozens, or hundreds of such SLAs. The effort of converting them to an automated system is daunting.
Therefore, you can start structuring your existing and future SLAs into a simple excel, or DB so that they are easily accessible, and comparable.
An example of a typical SLA would be stored in a table such as below.
The values for the various SLOs in the table were automatically populated from the definitions in the pre-defined Platinum and Gold tables (which state the default values for these SLAs). They may be overridden by specific values, following negotiations for a particular customer.
In the book I will elaborate on the structures and the tools and how to automate the compensation computations.
In my previous post I discussed some basic concepts about SLAs, SLOs and penalties. As promised, I am addressing the ‘who cares?’ question.
From a Service Operations point of view, you may shrug your shoulders and claim that these are issues with the Legal and Finance departments. Most likely you were brought on board later in the game and never viewed an SLA until your were forced to do so.
As the person responsible for keeping all the services up and running, it may be best to keep the SLA to a minimum. After all, a document containing vague language, with little commitment and liability would be hard to wave in front of your face when the service levels drop.
I will argue that vagueness will play against you. A tough SLA will require the company to adhere to the high service levels they are committed to, and yes, pay the penalties for breaching these agreements. Keep in mind that if your service level drops one time too many, the legalese you will be hiding behind will not save your butt when customers drop from the service or simply do not renew.
I would take it even one step further. I advocate that the Service Ops managers bonuses are tied to achieving those SLOs that will keep a smile on the customers’ faces. (typically up-time and response time, but in some cases there are other objectives that are crucial to the customers). The carrot and the stick should work nicely to assure that you are doing the utmost to live up to the agreements.
Commitments
Another issue that concerns you (Service Operations) is that Sales are making commitments that you are suppose to keep, usually without you ever knowing about it. Operations needs to initiate a fact finding effort to learn what is there. You need to know what you are capable of providing. Everybody likes to state that they are five nines (99.999% uptime) but how many companies out there really are? You need to monitor and test your service over a substantial period of time before you commit to those numbers.
Another point in favor of having a good grasp of your SLA is that you, as a consumer of services would be conscience of your requirements vis-à-vis your service providers.
That will include the hosting services, your ISP, your communications provider, and whatever cloud services you are using. In my past positions as VP Service Operations I have been appalled by the contracts that my predecessors have signed with service providers. Some of them had no consequences to service level degradation. Others had ridiculous clauses such as 'for every hour of downtime, the credit would be for one hour prorated service cost' which meant that there was no real penalty. Another contract stated that we could get out of the agreement if for three months in a row(!) the service provided was available for less that 75% of the time.
We would have been out of business by then.
Where are those damn SLAs?
As we have seen, SLAs that are broad and meaningful will be complex. Add to that various service levels such as Standard, Gold and Platinum and the fact that some customers have negotiated special terms for themselves, and you are dealing with a mean, slimy problem.
To compound that problem, nine times out of ten, these documents are sitting on someone’s laptop in a PDF format with perhaps a hard copy in a dusty folder, in the cabinet below the espresso machine.
Imagine the exercise of figuring out if an SLA was breached for a particular customer, and if that breach carries a penalty.
I have painfully gone through that exercise too many times, and believe you me - I had much better things to attend to following a service outage. The process was extremely slow, finding the various documents, looking up the terms and comparing the events with them.
Then a calculation was needed as to how much credit was due. And all this was done for a single customer. Multiply that by the number of customers that may have been affected and you have just wasted many good hours of Solitaire.
SLA Management Tools
There are multiple tools out there (some are offered as SaaS) to manage your SLAs. Many of them provide a full cycle of defining SLOs, creating SLAs, generating the documents, monitoring performances against obligations, computing compensation and generating reports. I have not used any of them (although I used to work at an SLM ISV), so am not about to promote any single one, but there are very slick solutions available.
If you are at an early stage, it would be hard sell for you to justify to management that you need to start paying for a service that possibly no one in the company comprehends.
Typically, when a SaaS company starts out there are very simple, non-abiding, fixed SLAs, so there is very little attention paid to this aspect of the business.
As with any aspect of SaaS Service Operations, scalability issues hit you when you least expect them.
As most (all?) SaaS companies do not start with Service Level Management software, by the time it becomes a burden they will have many dozens, or hundreds of such SLAs. The effort of converting them to an automated system is daunting.
Therefore, you can start structuring your existing and future SLAs into a simple excel, or DB so that they are easily accessible, and comparable.
An example of a typical SLA would be stored in a table such as below.
The values for the various SLOs in the table were automatically populated from the definitions in the pre-defined Platinum and Gold tables (which state the default values for these SLAs). They may be overridden by specific values, following negotiations for a particular customer.
Cust.
|
Calia
|
…
|
Google
|
Cust ID
|
123
|
…
|
213
|
SLSLA
|
Gold
|
…
|
Platinum
|
Uptime
|
99.9
|
…
|
99.99
|
Response time
| under 6 sec |
…
|
under 4 sec
|
Support Response time
|
2 hours
|
…
|
30 min
|
Support Avail.
|
12X6
|
…
|
24x7
|
Major Outage Resolution
|
1 hour
|
…
|
30 min
|
Partial outage resolution
|
4 hours
|
…
|
2 hours
|
Minor Outage Resolution
|
12 hours
|
…
|
6 hours
|
Maint. Notification
|
10 days
|
…
|
2 weeks
|
FTP
|
12 hrs
|
…
|
6 hours
|
Outage Notif.
|
Email 1 hours
|
…
|
email + call 30 min
|
In the book I will elaborate on the structures and the tools and how to automate the compensation computations.
Subscribe to:
Posts (Atom)