Scrum & Legacy Applications

While Scrum can be used in any environment, using it for development of a legacy system is particularly difficult. While the development team may be open to transitioning to Scrum, the team will encounter problems at all stages of development – from requirements management through to testing.

First, the product owner is likely to have a list of product backlog items (PBIs) that cover all areas of the application. When the product owner orders the PBIs by business value it will likely involve the development team working across all modules of the application, not just one. This is not an issue in itself, however when coupled with a lack of documentation of the application and the code (something common for legacy systems), the team will waste time reverse-engineering what each feature does and how it was coded. After a module is understood, the team will make the single required change instead of maximizing their new-found knowledge by working on every PBI in the area. While this is inefficient for the developers, this inefficiency multiplies when it comes to testing of this module, as the whole module or workflow now needs to be regression tested as a result of the single change.

Second, the code is likely to be “spaghetti” meaning it has been re-written, extended and fixed so many times by different developers with different styles and a different level of understanding of what it should do, that it now requires refactoring. However, if little documentation exists – in—line or in the form of design or requirements documentation, it will be hard to ensure the feature still works and that existing workflows are not broken as a result of refactoring. More likely, neither the product owner or the project manager will want to invest time refactoring code that “already works”. If this is the case, then any changes will need to be performed with surgical precision to ensure no defects are introduced.

Third, we encounter the issue of lack of automated tests. Without nightly smoke tests there is no way to detect bugs being introduced. This can result in the creation of new technical debt, as well as the change not being accepted by the product owner. However, an even bigger risk is there is simply not enough time to perform the regression testing manually if no automated regression tests exist, especially if the PBIs cover multiple areas of the system.

To effectively adopt Scrum, the team will need to:

1. Identify key workflows and document them,
2. Create automated tests that can run these workflows to ensure they are not broken as a result of changes. These tests would be executed nightly so freshly checked in code can be tested immediately and bugs identified within the Sprint.
3. Maximize efficiencies by working on multiple PBIs in the same Sprint. This can be an issue if the same code needs to be checked out and worked on by multiple people.
4. Spend a portion of the Sprint refactoring code, especially when it is messy or is subject to multiple or major changes.

The above is not easy and comes at a large cost many organizations are unwilling to pay for a legacy system. Typically, a legacy system has few new customers and revenue is generated by annual maintenance fees rather than the more lucrative sale of licenses. As a result, management will usually be unwilling to invest in automated testing, refactoring working code and spending time reverse-engineering functionality. Subsequently the team will be unable to adopt agile successfully and will only be able to adopt some practices.

In conclusion, while all this sounds negative, there are some more realistic options for the team. 

First the product owner should consider a major re-write, preferably in a new technology. For example, re-writing some areas of the application as a web application instead of a client-server application. This gives the team an opportunity to re-write the code and potentially redesign the whole feature, while giving management something “sellable” with a potential revenue source. 

If this cannot be done, the team should instead consider adopting Kanban as an agile methodology. This methodology has less overhead, removing all Sprint based ceremonies.

What is Agile?

In today’s organizations, there is a belief all projects could benefit from being agile, and agility will bring the dual benefits of speed and lower costs. This is often not the case. Agile methodology has a lot of overhead including the various ceremonies that are required each day and each sprint. By the time you factor this in, agile/scrum will likely cost more than if the project had been completed using a waterfall methodology.

However, this is only the case on a project where the outcomes are known. Where the outcomes are unknown, the benefits of agile are greater. Agile allows the team to work when they don’t yet know the full or complete requirements of the product, or where external risks may cause the project to take twists and turns in its journey. This is where agile truly shines. The only certainty required is enough for a two-week sprint.

Having said this, even agile projects benefit from having more certainty than one sprint’s worth. Without a vision for the product the team run the risk of needing to re-factor greatly each sprint as they discover what they have previously built isn’t compatible with the next sprint. To avoid this, the product owner should develop a vision for the software and outline the key features at the beginning of the project. This will help the team better understand how the product will iterate and grow. Agile principles still apply in regard to not building something unneeded, but the software can be designed to be extended.

Refactoring may still be necessary, and this emphasizes the importance of automated testing. If tests are built as the functionality is added then if the feature is re-factored, these tests should still apply and can be used to determine if the refactoring has broken the software. Without these tests, regression testing of refactoring would become an odious task, bringing even more work to the agile product than a waterfall project where all testing would be done at the end.

So what is agile? It isn’t doing something faster and cheaper, it is about reducing risk and uncertainty. It produces something of value earlier, but that something is not the full product.