The UK National Lottery web site crashed on January 9, 2016. This blog discusses the likely impact and measures to avoid crashes in the future.
Camelot Group have been operating the United Kingdom’s national lottery since 1994 and have a license to do so until 2023.
In the six months to 26th September 2015, Camelot sold £3.6 billion in ticket sales, a 4.2% increase in the same period the previous year. Tickets for the national lottery are available through retailers (currently around 37,000) and the national lottery website. Players can pay in cash at retailers, via an on-line account or via the Pingit app from Barclays (35% of all Camelot direct channel sales are via this payment method).
Demand for national lottery tickets increase when the prizes reach a particular threshold. For example Euromillions creates extra demand over a normal draw and roll-over draws create even greater interest. Special events e.g. when a hundred millionaires were created to coincide with the London 2012 Olympics, also creates additional sales.
The Outage
On Saturday 9th January 2016 the national lottery saw “unprecedented” demand and newspapers reported that Camelot expected 400 sales per second in the lead up to sales closing.
The web site was unable to cope with this demand and users were greeted with the following page:
Reputation and Revenue Impact
The reputational damage that this causes is difficult to quantify but Camelot’s capability was questioned with a series of frustrated tweets:
However, this needs to be put into context as the majority of sales go through retailers, Camelot report that over 80% of sales go through retail channels. There is also evidence that when websites are down and other channels are available then customers will use the other channels or in this case, buy a ticket well before the draw closes. In fact that was suggested by other customers tweeting in response to complaints.
The potential sales loss for one hour of down time on the website is estimated to be up to £600,000, however this would be offset by customers going to their local retail outlets and seems less significant compared to the fact that it is reported that 40 million tickets were sold for the draw on the 9th January – assuming a cost of £2 per ticket then that suggests that £80 million of sales were achieved.
What can organisations do to avoid web site crashes?
Accurately predicting demand can be difficult, newspaper reports in the week leading up to the draw on the 9th January showed that Camelot were predicting 200 sales per second in the final hour whereas on the night they expected that the demand would be double that. Camelot undertook a Demand Management approach by reminding customers that while the website was struggling customers could go to their local retail outlets. Under the circumstances that is a perfectly reasonable approach.
Organisations can undertake a number of mitigation steps in order to deal with the risk of a web site crash. A Performance Engineering approach can be followed where each component of the system can be tested to see how much demand it can handle.
Complex IT systems must be tested and modelled in order to prepare for peaks
IT Systems like the National Lottery’s can be complex, there are web servers, application servers and database servers – each of those has a number of finite resources that can be bottlenecks, e.g. CPU, memory, disk I/O, network interfaces etc. In addition there are backbone networks, retailer terminals, third party systems (e.g. payments) and even restrictions on software licenses that can restrict throughput. System configuration settings and application bottlenecks add to the complexity.
Some of these components may not be able to be tested in a “laboratory” environment and therefore other approaches have to be followed. These could include Service Assurance where the service is tested in the production environment, typically during quiet periods; and Supply and Demand Modelling where observations in production and testing are combined to give an end-to-end view of the ability to meet business outcomes (i.e. sales per second). Each of these approaches can achieve the desired outcome but it is when they are all followed, in an integrated way, where success is most likely.
If you would like to learn more about our Prepare for Peak and Performance Testing solutions, please click below, to see our latest Ebook.
