Rapid digital transformation

Our client’s digital mental health services support thousands of young people and adults toward better mental health and emotional wellbeing. We partnered with them to launch two core mental health and well-being platforms.

The client has a designated platform for children and young people, plus an adult equivalent. Together, these offer a range of services including early intervention support and access to a team of experienced counsellors, removing the traditional barriers to entry such as waiting lists, sub-threshold assessments, stigma, etc.

The platforms receive more than 46,000 every month. This is a critical service for many individuals as well as for their partner organizations such as NHS commissioners.

Our collaboration achieved rapid success with lasting impact. Our work helped our client improve a critical service that relies on trust, speed and professionalism to get people the help they need, when they need it.

As a result, the updated system was able to handle more than 3,000 times more users at any one time than previously.

A sensitive subject matter and a critical timeline meant that the client needed a partner they could trust from Day One. The work required a team with proven technical capability, flexibility, the capacity to share knowledge with the client’s team and build-in-house capability.

The client’s Chief Technology and Product Officer tasked us with relaunching both platforms’ websites, and with bolstering the ability of his internal team in agile ways of working and a culture of continuous improvement.

This would empower the client to serve a growing user base with even greater efficiency.

With speed a critical success factor, our teams came together to quickly assess the existing infrastructure and trace performance issues in high-risk areas of the codebase, which allowed us to pinpoint issues.

Following this, we implemented a scaling solution and a load-testing framework to improve performance of the apps. From the number of users the system could comfortably support, to the average API response time the client wanted to achieve, we benchmarked our progress with real-time data and continued to tweak until we were confident that performance could not just meet but exceed people’s expectations.

We continually shared our development expertise as part of a pairing policy between our engineering teams. Our changes have had lasting impact. This means the client’s team will continue to reap the rewards of agile ways of working long after our involvement.

Using Stackdriver and Locust to diagnose performance issues

Measuring app status, capacity and performance enabled the joint team to rapidly identify major performance improvement opportunities. Using Stackdriver for code hotspots and service logs and Locust to simulate load, the team determined a baseline performance score to measure improvements.

Resolving system crashes and other performance issues

We tested each service in isolation to resolve frequent site crashes in the cloud-based system caused by too many containers overloading the nodes. We fixed memory leaks in the codebase for a number of performance critical services.

Future-proofing through daily testing

We ran daily tests via Locust and a Grafana dashboard to ensure development changes had the desired impact on response times, failed requests and load capacity.

Where the changes didn’t have the intended impact, we leveraged the logging carried out with Stackdriver to diagnose and resolve issues before the next round of tests.

Ultimately, our thorough evaluation of the technical requirements for the client meant a lasting impact on its services — not merely providing quick fixes for the problems at hand.

Performance objectives

• The number of concurrent users the system was capable of handling grew from 30 to 960, exceeding the target of 840.

• The average API response time fell from 1600 ms to 480 ms, exceeding the target of 500 ms.

• The rate of system errors at load reduced from 0.8% to 0.000178% (a 450-fold improvement).

• Seamlessly released both platforms.

Growing capability

• The first release launched in two months. The second release followed three weeks later.

• The client facilitated the second release with a much small team. Just one-third of the team members we supplied on the original release were needed.

• We enabled the client to release continuously.

 

2 months

480 ms

To deploy the first release

Average API response time

450x

Improvement in rate of system errors at load