Estimated read time: 3 Minutes
Author: Dr. Manzoor Mohammed
This is the method the engineering team in a large SaaS company I worked with, used to gain senior leadership confidence. They only had 10 minutes and a couple of slides to tell their story. But they knew execs cared about metrics that impact the business.
They reported 4 simple metrics for the last 12 months at monthly granularity. The charts were reported on the same slide so you could see relationships between them. Below are the metrics they reported. They excluded any on-premise stats because they didn't control all the levers. I've also added in my suggestions:
- Revenue
- Revenue loss
- Costs
- Customer experience
1. Revenue
The easy one. You may want to filter this to only include revenue that your infrastructure is responsible for supporting, it is a great business demand indicator.
2. Revenue loss
They included lost revenue due to contracts delayed because of concerns about service stability. You could also add service credits, contracts not signed due to reputation, revenue delayed due to capacity not being available etc. That gives a fantastic picture of what is at stake.
3. Costs
They were going through a major cloud optimisation programme. They were reporting just cloud costs. You can also add tooling & licensing costs which are driven by infrastructure and approach to capacity and performance. The more you include the bigger the number and the more important it becomes.
4. Customer Experience
They reported P1s. I've seen other companies use multiple metrics including Customer Satisfaction Score (CPSAT), synthetic monitoring and actual performance as measured on their estate. Each of these metrics has its strengths and weaknesses.
It may make sense to look at all four metrics and see which one is telling a consistent story, e.g. You could use synthetic monitoring or actual performance stats. You could also report incidents, complaints, etc. What you report depends on what drives 1 and 2.
The organisation carried out a cloud optimisation programme and thought the programme was responsible for customer incidents. However, looking at the data showed the poor customer experience existed before the optimisation programme started. Spikes in poor performance turned out to be due to the infrastructure teams configuration of underlying VM infrastructure.
