Capacitas has been involved in a number of different performance testing projects for various years now. One of our customers with which we have been involved for over 9 years follows the Agile methodology to produce their eCommerce software. Agile development has a number of benefits for different stakeholders; customers, business people, development and testing team because new software can be released more frequently. Agile performance testing allows you to change code quickly and deliver software more frequently without risking end-user experience.
The managed performance testing service that we provide to this client is a critical part of their software development life cycle and this is why our involvement starts from the very beginning.
Sprint Planning
At the start of every sprint cycle we attend sprint planning sessions which are attended by the Scrum Master, product owners, development and testing team, where they review product backlogs (new features) and decide on which one they want to work on during the coming sprint cycle.
Once they all agree on which backlog items to work on, it is broken down into stories to facilitate (i) the tracking of the sprint progress and (ii) the allocation of tasks to the team members. Each story is then given a number of points based on their complexity using Scrum poker points and the development team agree on how many points they can work on by looking at previous sprints and the current availability of their resources.
During the sprint planning session, Capacitas assesses the risk (high, medium or low) of each new feature being introduced so that we can have an idea of the performance testing effort required. As this particular customer has a number of development teams working in the same sprint, it is not always possible to test all of the new features in the same way, we therefore look at which of the new features might cause a performance issue and agree with the development team to do more thorough testing around this area.
Planning and Testing
Once the sprint planning sessions are completed, we start writing a test plan that details the test cases and we also define thresholds that should not be exceeded for the test case to have a ‘Pass’ status.
During the sprint, we attend the daily scrum meetings to discuss progress and raise any issues that might be delaying the work. Before testing the new features, we usually create baselines that encompass the area of change so that when we come to test the new features, we have something to compare it to and decide whether it is good enough to be released in the production environment.
As there are a lot of changes being checked in everyday, we have to run daily performance tests to make sure that performance of the application and the platform are stable; we do this by running a series of Single Transaction Regression, Ramp and SOAK tests using the load testing tool available. These tests show us response times, service times, application bottlenecks, memory leaks amongst other metrics. If ever we find any defects during the performance testing process, we raise them in a defect tracking system which gives better visibility to the people involved in the sprint and once the defect is fixed and the performance is acceptable, it is then closed.
Regression and Post-release
At the end of every sprint we go through the regression testing phase where all new development is stopped, only bug fixing and testing is done to make sure good quality software is delivered.
As soon as all the bugs are fixed and the software is ready to be released, the regression cycle then stops and the software is released in the production environment. Capacitas also helps the client in post release activities such as monitoring the application and infrastructure when the new release is in the production environment.
The performance testing results also allows Capacitas to use the results to carry out capacity planning exercises to make sure that the client is able to provide a good service to their customers at all times.
Discover the core principles and best practice approaches for ensuring good performance whilst increasing the velocity of delivery - download here.
