Insights

Scaling opens great opportunities for growing businesses but can also become a constant uphill battle. Challenges can either come from the teams that are delivering their high-quality products or from the IT systems that drive the business demand. Left unchecked, both these areas can drive costs uncontrollably, increasing resources without cost consideration or through over working our systems so that they become un-performant.

To avoid this, you need a better way to observe teams and systems holistically to ensure you can meet business demand while remaining in control. 
  

Pros and Cons of horizontal scaling   

In a horizontal scaling architecture new nodes or machines are added to the infrastructure to cope with new/increased demands. In this sense, scaling teams this way adds more team members.
Scaling horizontally can give your teams the much-needed resource that they require to complete day to day tasks such as delivery. Freeing up the teams’ time allows them to pursue more avenues of work like pushing delivery into a different area, improving services, optimising resources and discover new improvements.

Usually, this is the case for teams with key roles in all areas, allowing them to grow horizontally - but what happens if that is not the case?

This situation can cause delivery teams to split their focus off delivery and onto training – this can cause delivery to suffer as more time is being spent training new stakeholders up to the standards required for delivery. In this situation the attempt of horizontally scaling fails at its objective. Teams produce less work and there is a reduction in quality. One other area where this type of scaling can fail to achieve its goal is within the team’s morals, resulting in an increase in stress on the delivery teams as they have more responsibilities with less time to do them.

Pros and Cons of Vertical scaling  

While horizontal scaling an architecture adds additional machines or nodes, vertical scaling adds more resources to existing machines. In this sense, scaling teams vertically improves the already existing teams without the need for additional members.

Vertically scaling a team can add to it: increased experience, knowledge, and ideas. It can also bring in more governance and ensure quality control of a product or teams’ delivery. It will allow the experienced team to find new areas of work or of development while voicing the concerns of the team and their achievements – an area that can be overlooked if the delivery teams does not have the support.

However, vertical scaling is not always all singing and all dancing.. What it fails to address is the increased workload, as more senior members of the team should be managing or strategising, and not be so involved in the delivery If they find themselves in such a situation then vertical scaling could have failed.

A key observation in teams scaling vertically is increasing workload or promising workload that is unattainable with the current number of delivery team members.

Is the answer to do both scaling methodologies?

Well, it is not quite that simple. It is about creating a balance of the two – horizontal scaling will take more time invested to achieve the delivery standards of the current work force whose story can be conversed to stakeholders through vertical scaling. The issue here however is vertical scaling can cost teams budgets while horizontal scaling can cost a company’s resources.

Depending on where you are within the team structure you may see things from a different point of view. All of these have validity to their objectives and points:

1. Strategist/budget holder: If work throughput needs to be increased, then you would likely suggest making teams or the system more efficient. Thus, saving on buying additional resourcing and getting the most out of what you currently have – a solution that will maximise your current structure.
2. Delivery Manager: your concern is making sure that the team is performant and delivering a high quality of work. You therefore want the most bang for your buck, bringing in more experienced or skilled team members to ensure increased demand is managed well and met.
3. Delivery Team Member: teams’ costs are, perhaps, not within your scope, you may suggest expanding horizontally adding in additional resourcing to ensure that increased workloads can be handled.

Although all 3 examples are a valid solution you need to take a step back to understand what the issue is and how you can allow teams to keep up with increasing demands. This shows us that not all situations are the same, therefore our understanding needs to be challenged to ensure the best outcome.

Are there lessons to be learnt for systems scaling?

Of course, it is easy to say that a team is much like a system, they both have a capacity of work they can achieve and a throughput of work governing their daily capacity. Each team members could also be seen as a system, in their own right, with individual capacity and daily throughput demand. 
If we apply that same observation around your position in the team structure to systems, we can see that;

1. Strategist/budget Holder: the push may be to get the most out of the systems that we have, having a finger on the budgets pulse. However, this may stretch team time finding efficiencies that do not exist – causing lost time and money. It may also cause an environment where teams are pushed to their absolute limits creating an environment that yields fewer outcomes.
2. Delivery Manager: to meet a project or system deadline anything seen to be un-performant may be tackled through scaling on larger instances. This short-term -vertically scaled- fix will allow the project to go live but will be causing an increased cost.
3. Delivery Team Member: without governance on delivery teams, we may see large increases in horizontal spend, scaling up machines to ensure that the demand of the system is met. What has also been observed is that this can also extend into the test environment where demand in testing causes more instances to be spun up, without thinking about the need for another. This can lead to a high throughput system, but it is by no means efficient and there is a large cost to this high capacity.

Again, a holistic approach here is needed, that will allow teams to Scale, Serve and Save – driving business growth. Tackling this issue without a holistic approach may lead to short term improvements but what has been observed is that the entire system is not considered leading to instability, poor scalability and even poor team moral. All these catches may be risks that business are willing to move ahead with, but each risk should be known so that an informed decision can be made.

Scaling costs

Uncontrollable costs are also a factor on why there needs to be governance across the teams. Without someone to guide teams into best decisions costs will rise quickly and with very little explanation. Many of the cost improvements are small in the cloud and therefore each small decision can add up, like team growth and cost.

A simple rule for scaling a system is that the cost increases should have a linear relationship with the growing business demand – if you find yourself increasing scaling four-fold while demand increase is doubled then we will have opportunity to save costs within the system.