Agile, Peer-Reviewed - Rocket Mortgage applies Scaled Scrum.
The Impacts of Agile are shown in an academic paper.
In general, I don’t think the agile community evaluates itself enough in peer-reviewed papers. Whilst there is lots of anecdotal evidence, there is little in the way of scientific studies into agile methods and their impact on key metrics around cycle time, value delivered and ultimately company success.
However, a new paper has been published less than a month ago looking exactly at this. ‘Rocket Mortgage Delivers Twice the Value at Half the Cost at Scale’ was published with the help of the influential Michael Simmons and Jeff Sutherland as Authors. It describes a case study of Rocket Mortgage, and its application of Scaled Scrum on top of Scaled Agile to drive further improvements in their delivery.
Rocket Mortgage is a mortgage loan provider, based in the US. They are a very large and successful company - with revenue of $15.75bn in 2021, and net profits of almost $1bn. Despite their success, in 2019 they make a company decision to adopt agile in order to try and ship faster. Interestingly, the paper describes that shipping faster is the number one reason that companies seek to adopt agile.
From the paper directly: “A Scaled Agile transition organised teams into collections of release trains centred around business value streams with a common Scaled Agile methodology. These groups of release trains, called "streams," were strategically aligned around similar business and technology capabilities, explicitly covering the mission-critical areas of Product Engineering, Data Intelligence, Infrastructure and Operations, Security, and Enterprise Services. This restructuring happened all at once: the organization shifted from technology capability-centered platforms in the third quarter of 2018 to 23 value-focused release trains in the fourth quarter of 2018 (42 as of April 2021).”
This agile transformation had some massive impacts - and the paper has some amazing stats to demonstrate this. Whereas before on average teams took 83 days to release a feature, this was cut to 33 days post the agile transformation. The team delivered double the number of features they were delivering before agile - 833 vs 414. The team were also more predictable, delivering 88% of their commitments rather than the approximate 40% that had been the case pre-scaled agile transformation - critical for helping to effectively plan work collectively across teams.
Given this initial transformation, the paper looks at how the team continued to optimise their agile implementation. This outlines a number of issues that many companies find when implementing agile.
What were the challenges slowing Rocket Mortgage down?
Scrum Basics - as the team had scaled, the teams had fallen into a number of anti-patterns around Scrum. In particular, there was a lack of uniformity in the roles between the PO, SM and the developers. The team seeked to use training to re-teach Scrum and remove all roles that were not in the Scrum Guide.
Deployment Process - long and manual efforts in order to deploy slowed down the ability for the team to deliver value quickly. Infrastructure teams had to be involved to release, and it had to be done out of hours. The team moved to being able to release autonomously.
Testing and Quality Gates - testing was done by specific QAs in the team, which created a bottleneck in the release process. The team moved these QAs to specific roles elsewhere in the company and put the responsibility for testing on the whole team. Quality Gates were introduced to ensure code quality which were automatic.
Meeting Structure - the scrum of scrums meetings were held before the daily scrums, and were run as status updates. This MetaScrum was instead moved after the daily updates and focussed solely on the blockers for the teams.
What were the outcomes? Using a Scaled Scrum approach on top of Scaled Agile to solve these problems led to a further reduction in cycle time, higher quality, and improved customer and team satisfaction.
This paper is significant in that it outlines clearly how Scaled Agile and Scaled Scrum was implemented, and has a metric driven approach to evaluating success. However, it is once again an example of a single company’s implementation. A cross company comparison of agile implementations would help to further understand the impacts implementing agile has on feature delivery.
References