Our Agile Values

At Viderity, we are big fans of Agile ways of working. There are various methods branded “Agile,” with Scrum™ being the most well-known, although people sometimes incorrectly conflate Agile and Scrum. However, Agile is not a method. Rather, it is a set of principles stated in the Agile Manifesto, and a resultant way of thinking about how best to develop software. A given method can be more or less adherent to these principles, and so be more or less “agile”.

 

The awkward truths of software development

At its heart, Agile is about acknowledging and addressing some awkward truths about most, if not all, software development activity:

1. Requirements change

We often assume that with enough up-front analysis, solution requirements can be fully described and documented before development starts. The reality is that modern software projects are so complex and the stakeholders in a typical development project so diverse that requirements almost always change throughout a project.  We improve our understanding of the solution as we work through a project, so our very best understanding of the requirements exists at the last day of the project. Our methods should embrace and harness this learning-by-doing, not sweep it under a carpet of denial or change control governance designed to prevent even legitimate change.

2. You cannot plan the perfect solution

We often want to believe that we can design the correct technical solution on paper before we write a line of code. Get the best architects and developers in a room, the argument goes, and they can thrash out a great design that will save us rework later.

This argument is flawed in three ways:

• If the requirements are not fully understood up front, it is impossible to design the right end-state solution wholly up front.
  • Modern software development is far too complex to describe a fully consistent and optimal design at a moderate level of detail without the aid of development tools and techniques like code analysis, refactoring and test automation.
  • The bottleneck in any complex software development project is not the time it takes to type out the code, but a myriad of other activities such as detailed design (even if just in a developer’s head) and various levels of quality assurance. Trying to perfect a design to avoid reworking the code is unnecessary.

All of that is not to say that we do not need architecture. There are some decisions that are expensive to reverse, and so make sense to make up front. We like to say that a good architect is someone who can tell you which decisions you have to make now and which ones you can defer. Our aim is to defer decisions to the latest responsible moment to maximize flexibility and return on learning.

3. Large batches don’t work

Inherent in the two previous points is the common belief that working in large batches is more efficient than working in small ones. Isn’t it better to stay focused on requirements analysis and get it all out of the way, before we start figuring out how to build those requirements? Shouldn’t we be thinking about all the requirements in the greatest possible detail and then designing a solution that encompasses all of them in one hit?  This is the sort of thinking that got American carmakers into trouble a few decades ago. Lean thinking teaches us that large batch sizes are problematic. For example, what happens if we misunderstood a core requirement that was the basis for a lot of design and development activity and the misunderstanding was overlooked during requirements sign-off? In a large-batch scenario, the client may not see the manifestation of this requirement until most or all of the solution has been built, months or even years later, having already done a lot of wasteful or harmful work.  Furthermore, the overhead of producing, managing and re-consuming the documentation that is required for a large batch approach is significant.

In Agile projects, we try to write down only what is necessary, believing that a conversation is a more useful and efficient way to transfer knowledge from a business analyst to a developer than a document that has to be written in great detail, then read months later when the original context is obscured. At the end of the project, that document has little value: the client only cares about the software solution it results in.  To realize this efficiency, we need to be working in small batches, taking a requirement from analysis to build in a matter of days or weeks and being prepared to learn from this exercise, possibly resulting in some restructuring of the technology (what developers call refactoring) at a later date.

4. Estimating software development is very complex

Project managers faced with complex, long-term projects will often take a reductionist approach to planning: break the work down into small chunks of activity, estimate each, identify the dependencies, and add it all back up again to get a project plan.   Unfortunately, this is pretty much impossible in an environment of great uncertainty (see the first two points above). Both the identification of activities and their individual estimates are subject to too much variation. Alas, we often see project plans that try, making wild claims that designing such-and-such a module will take four hours, whilst building it will take six. We might imagine such a guesstimate would be subject to a 100-1000% margin of error, making the plan useless.

Instead, if we can break deliverables (e.g., a requirement going through an analysis-design-build-test cycle) into sufficiently small and similarly-sized chunks, and if we can start delivering these very early in the project, we can use a moving average throughput rate as a simple way to let past performance guide our planning and forecasting. This type of radical simplification also helps reduce the sometimes significant overhead of managing the plan and can give us useful data about where we have bottlenecks in our process and how to remove those.  There is usually still a Gantt chart: activities such as training, stakeholder communications and product marketing are often much better understood and so can often be estimated and planned with greater certainty. On this plan, however, development is shown as a series of boxes that deliver software at regular intervals (say, every month). These become milestones on which other aspects of the plan can be hung. What happens inside these black boxes is left to a development process more focused on flow and efficiency than up-front planning.

5. Budgets are limited

Few projects have unlimited time or budget (if only…). Most start with some idea of scope that needs to be delivered. As time, budget, or both, invariably become squeezed, it is natural to try to apply the less popular levers of project management, such as overtime or increased pressure on developers, to try to get the project in on time.  We should be reporting on progress against the stated goals of the project, and we should be trying to optimize our delivery approach as we learn how well it is working (though improving the system and process the team works within will produce order-of-magnitude greater improvements than trying to address the problem at the level of the individual). However, by treating scope, time and cost as constant, we fall victim to the dirty secret of software development: A good developer can write almost anything in weekend or two. He or she can probably even jerry-rig it so that it passes external quality control. Look at the code, however, and it will be a tangle of shortcuts and compromises. It will not be maintainable, scalable or robust.

The majority of the lifetime cost of most enterprise systems is incurred in maintenance, and most new projects have hopes of providing a long-lived platform that can be extended and improved over time. Compromise quality from the beginning, and maintenance costs soar whilst the business starts thinking about how to replace the solution wholesale in another expensive project.  In times of pressure, we usually don’t sacrifice quality because we consciously decide to, but because it is the hardest to measure. Agile (and the Software Craftsmanship movement) try to address this balance by suggesting that it is usually better to consider scope to be the variable. If we deliver incrementally (remember: small batches) and prioritize our work so that the most important requirements (as we understand them at the time) are fully built, tested and accepted early in the project, we have the option of pruning scope to deliver a higher quality solution. Cutting cost is about doing less for less, not doing more for less.

Our Agile Principles

In working with our clients delivering complex, multi-faceted software development projects, we face all of these challenges and more. We have found that neither the Agile Manifesto nor the core practices of Scrum™ alone fully encapsulate how we need to work. In part, this is because working in a professional services context is a little different to working in the type of long-running product development team that a lot of Agile literature takes as a starting point. For example, our teams often have to deal with things like fixed-price contracts, clients who have limited time to act as “product owners” guiding development, and teams that do not survive intact from project to project.

However, we believe that a set of core values are universal to all of our projects. In true Agile fashion, these have evolved over time (and continue to do so), but at the time of writing they are:

• Focus on business value: Let every team member be guided by an understanding of the client’s business and the strategic goals of the project
  •Quality: Build quality in from the beginning and manage quality proactively throughout the project
  • Simplicity: Design and build the simplest (and most elegant) solution that will meet the requirements
  • Collaboration: Flatten the team and work hand-in-hand with the business
  • Feedback: Build feedback loops into the project and use this to improve throughout the project, both as a team and as individuals
  • Commitment: Commit to the business as a team, and stand by those commitments
  • Visibility: Provide the business with up-to-date, meaningful metrics of our progress and early access to the solution we are building

We teach these values to our new hires and describe them to our clients at the beginning of our projects. The exact process we follow on a given project will be tailored to the circumstances, but these values remain and we use them as the guiding light by which we make decisions about how best to tailor our approach.