March 10, 2014

Agile

Agile is a set of values from which an entire set of software delivery methodologies and practices have been derived. All Agile practices follow the same fundamental principles as set down in the Agile Manifesto and Principles, however, because Agile is flexible to meet the needs of individuals, teams and organisations, each work place implements it differently with varying degrees of success.

We will look at these guiding values and how they have developed into a cohesive end to end methodology for delivering working software. It is important to understand the intention behind Agile and to learn in depth the resulting methodologies, practices and culture that was intended.

The finance industry needs people who really understand the Agile process and can navigate teams through the flexibility that Agile provides to produce the results required in finance. Learning about Agile and getting qualified in the appropriate certification will significantly increase your chances of employment in the finance sector.

The guiding values – The Agile Manifesto

The original guiding values are defined in the Manifesto and represent the foundation of all the Agile processes, principles and methodologies. I recommend you learn these principles.

Source: Agile Manifesto

The signatories at the bottom of the manifesto

Take the time to review the signatories at the bottom of the manifesto. Google these names and you will see that many of them have contributed hugely to the Agile community. In fact many of the books and resources that we will explore as we learn more about Agile are written by the people on this list.

The principles behind the Manifesto

There are 12 principles behind the manifesto, these are less known but still form the basis for all of the work we cover
when examining, Extreme Programming, Scrum, and the Scaled Agile Framework.

  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  • Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
  • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
  • Business people and developers must work together daily throughout the project.
  • Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
  • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
  • Working software is the primary measure of progress.
  • Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
  • Continuous attention to technical excellence and good design enhances agility.
  • Simplicity–the art of maximizing the amount of work not done–is essential.
  • The best architectures, requirements, and designs emerge from self-organizing teams.
  • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

A methodology for complex software

As building modern software is so complex and has too many variables to take into account, a deterministic approach does not work. We look to scientific methods of discovery to let the software unfold as we learn more about the bespoke business space our software will be made for. Agile gives us an iterative approach to software. This is the scientific empirical approach.

Using an empirical approach we see the following characteristics compared to waterfall methodologies.

These values are achieved by transparency, frequent demonstrations of working software, feedback, short repeated development cycles allowing changing in direction (adaptability), early and frequent releases,
removing risks early through visible deployed working software.

Three pillars of Agile (from Scrum)

The Agile iterations allows Agile practitioners to be transparent in all that they do, use the transparency to inspect what has happened and adapt to make things better in the future. Agile methodologies have built into them the tools to improve the way you, your team, your customers and your organisation work together to achieve success. It is a self-improving system.

The three pillars of Scrum are:

  • Transparency
  • Inspect
  • Adapt

Declaration of Interdependence – Principles for management

The declaration of interdependence is a set of six management principles intended for project managers of software development projects. The principles are:

We …

  • increase return on investment by — making continuous flow of value our focus.
  • deliver reliable results by — engaging customers in frequent interactions and shared ownership.
  • expect uncertainty and manage for it through — iterations, anticipation and adaptation.
  • unleash creativity and innovation by — recognizing that individuals are the ultimate source of value, and creating an environment where they can make a difference.
  • boost performance through — group accountability for results and shared responsibility for team effectiveness.
  • improve effectiveness and reliability through — situationally specific strategies, processes and practices.

Further Reading: Declaration of Interdependence.

Agile Planning – Turning what is fixed on its head

In the old waterfall approach, features were fixed by large scoping documents. Budget was fixed and so was the release date. The idea was that software could be deterministically solved. As we found that this was not the case, a new way of planning was needed.

Agile projects are planned with fixed cost and time and variable scope (or feature set).
This means that if things take longer because the development team hit problems along the way or if things go really well, it is the number of features that grows or shrinks in a given time and cost. Working software still gets delivered with Agile, and what is delivered is the highest priority features.

The understanding that building software is such a complex undertaking that a fixed cost, time and feature set is not possible is key to success.

If all three elements of a project are fixed then when problems arise the only area that can change is quality. Agile solves this problem.

Trust

Another key proponent in being successful in an Agile environment is fostering trust. If the stakeholder knows they are going to get variable scope for their money, how can they know what they are getting in advance? This is where transparency is so important.

Agile teams strive for excellence and communicate this with the stakeholders in short cycles (typically 1 to 4 weeks) and show them progress. This allows the stakeholders to have input early, understand problems as they arise and visually see the quality they are paying for. It brings the stakeholders into the team and makes everyone responsible for getting the best possible results from a complex development process.

The stakeholder gets a team, which they are part of, which delivers the best and highest priority features first and increases the likelihood of success. The amount of risk from the short cycle feedback process is hugely reduced which allows the stakeholders to really understand their investment and to have trust in the development teams.

How the Agile implementations fit together

In this course we will discuss the most popular implementations of the Agile values and how these fit together to give a framework for the different aspects of software development.

The key methodologies are:

Extreme Programming

Extreme Programming (XP) can be thought of as the developer’s best practices. They encompass the good practices required to keep the code base clean and are specific to the team and how the team interacts with each other and manages the quality of their deliverable: working software that is fit for purpose.

See the page on Extreme Programming for more details and further explanation.

XP is closely focused on the practices and tools that the development team can use in their day to day development. These include things like continuous deployment and test driven development.

Scrum

Scrum is also focused at the development team level; however, Scrum provides a set of job roles, artefacts and a process for delivering software. Scrum naturally builds upon the team interactions and coding practices of XP to provide a practical methodology to deliver software using an empirical approach.

See the Scrum page for more details.

Kanban

Kanban comes from Lean approach to software and can be used wherever there is a list of work to be done. For example, in Agile we have a list of features or stories to deliver. It differs from Scrum, in that there is no iteration. Features are pulled from the backlog as resource becomes available to complete the work.

Kanban works well with teams whose backlog changes very frequently and stories are created that need immediate work but were impossible to predict. A typical environment that Kanban works well is in support teams where live product bugs are found and need quickly resolving. Kanban approach is used in SAFe at the Portfolio level.

Scaled Agile Framework (SAFe)

Scaled Agile Framework is relatively new compared to the other implementations. SAFe takes the Agile approach and implements it at an organisational level, allowing management of multiple teams using Agile values throughout all levels of an organisation. The framework takes the naturally scalable features of the single team frameworks and provides a framework to manage these at an enterprise level. The framework adds or replaces elements of other frameworks that don’t scale well.

See the Scaled Agile Framework page for more details.

Agile’s uptake

Agile has grown from a grass roots developer culture into a fully recognised system for delivering software. Agile development teams are becoming the norm across most industries. However, because Agile has been implemented from a bottom up approach, the implementation that different organisations and different teams are using varies vastly. As a result of this, teams have had varying degrees of success.

All across different industries organisations are slowly formalising their Agile approaches and moving towards the more popular methodologies such as Kanban, Scrum and XP. Most organisations are still in a primitive phase of using these methodologies together in a cohesive system and are even more primitive in their use of Agile as a scaled methodology to run the organisation beyond the individual development teams.

There is a huge amount of energy and focus being given to changing to a more successful Agile approach and demand for people with experience of how Agile can work well. It is both for this reason, and because working in a non-Agile way simply doesn’t work on complex IT projects, I strongly recommend you only consider Agile based projects on your quest for excellence.

In Finance, Goldman Sachs has implemented Agile projects at the team level successfully across the enterprise using Scrum, XP and Kanban, Standard Bank has implemented Agile projects successfully at the team level using Scrum and XP, BNP Paribas has implemented a Lean Agile framework across the entire organisation from the top down. NYSE is still using waterfall approaches to development however they are coming under pressure from their suppliers and staff to change this approach.

Many Hedge Funds are moving to an Agile development approach to cut costs or using Cloud technologies to replace in-house teams with SaaS developed by Agile teams.

Feel free to submit more examples of Agile in the your organisations.

Further reading for general Agile knowledge

Simon Powers

Simon Powers is an Agile Coach specialising in large scale transformations and agile adoption. He has a background in very large enterprise architecture which has led on to organisational design and agile process refinement. Simon is the founder of Adventures with Agile.