The right wrong thing
User stories should tell you the right thing to make, while requirements analysis should tell you how to make the thing right. However, the latter process gets some things right and other things wrong. It can reveal literal and obvious things, such as what platform you need your software to run on, how much memory it has, what can be considered an acceptable response time, or how many concurrent users you expect to reach. These numbers can, of course, be wrong. An agile development practice almost expects such answers to change, even if they are correct at the time, yet these facts are knowable at the time. They can be detected, measured, and calculated, even if they are wrong at the end of the project.
Other things are entirely unknowable such as unexpected market shifts or secret projects that are suddenly and publicly announced. No requirements analysis can gather the unforeseeable. Agile methods can prepare you to handle these, but they can’t predict them any more accurately than traditional requirements-gathering exercises. And if they don’t happen, why do we still think that agile approaches are superior?
What’s potentially knowable but often missed are the requirements we unearth when project components come online; the results of interactions between parts previously developed in isolation. These are the emergent properties and complexities. A requirements-gathering phase can occasionally pick up on a few of these if seasoned developers are present, but even if you have experienced experts onboard, you should not hope to discover more than a slim majority of them during any planning phase. For this reason, projects should have a pre-production phase. A more fruitful requirements-gathering process can occur during a prototyping or proof-of-concept development stage.
Turning to the making phase faster has two benefits. First, it creates people with wisdom. In the future, during preliminary requirements gathering on a different project, they will know what requirements there will be for the things they typically work on.
Second, this change allows you to rapidly reach the stage in development where the requirements you missed during requirements-gathering make themselves known. They appear as integration blockers or arise when completing detailed design work, occasionally manifesting as insurmountable bugs at the lower levels.
Code enforces its requirements better than documentation because conflicts in documentation need to be actively sought out. In code, conflicts rapidly become compilation or implementation impediments. There’s nothing quite as immediate and obvious as being in the middle of developing a feature and realising it’s not possible to implement it with the current API or data layout.
Christopher Alexander’s process mirrors both these benefits. Planning at the site resembles eliciting requirements by working on prototypes, while continuous integration parallels the production and reviewing of building mock-ups. The builders using Alexander’s processes were given fast feedback on what didn’t work, and became better architects at a faster rate because of it. Again, this leads to wiser builders and the swifter discovery of and recovery from emergent problems.
This process scales with experience. As the critical elements of the project increase in size, the depth of knowledge and the breadth of expertise needed by the development team grow in proportion. For large projects, the developers need to be experienced at building at least medium-sized projects. The larger the project, the more the work must be repetitive for the whole project to be completed successfully. This repetition is not like a factory line but more akin to learning how to mortice a door or frame a window; repetition with variation leads to mastery. Working as part of a team and adapting what you have learned to support the project provides purpose. Meanwhile, understanding and improving the process at the site as a trusted team member gives a sense of autonomy.
Learning is essential, and this is why teams need to stick together. It’s also why we need teams in the first place. No one can do all jobs well. Some members have to carry the knowledge for specialist areas to save time for the group. If everyone has to learn every part of a production chain, then no one knows anything intimately enough to make insightful improvements.
No process is magic; not even Christopher Alexander could invent a system by which a complete novice could build a town from nothing. But he did produce a process by which a beginner could decide to forge a village, and from that process would emerge a community of buildings and an architect.
Furthermore, writing code also has other benefits. Counterintuitively, code is simpler to change than specification. It’s not easier to alter, but it is more straightforward to be sure your work is done. Code takes time to perfect, incorporating many keystrokes and moments of grumbling over failed compilations and red test runs. But even though it takes longer to get right, it is simpler to change because when every test is green, you know you have finished. It gives you immediate feedback on repercussions and adds value as part of the change. It’s also in revision control, so there’s a record of the change. This which helps decode whether any newly introduced usage patterns were part of a vital feature or an unexpected behaviour added by accident.
Many of these aspects are not mirrored in the physical building site, but moving mock-ups around lets you see the impact of alternatives quickly and supplies immediate feedback on the repercussions. Feedback comes much faster than when drawing with a pencil on paper. Unfortunately, these physical processes of making changes do not commonly produce historical records, so decisions and reasoning of physical construction can easily be mislaid.
Problems with acting early
The problems with the action-first approach are mirrored in physical development too. When requirements change on a grand scale, such as building regulations for physical construction or hardware availability for software construction, much of the completed work is wasted and more must be done to move the project back to a new starting point. The source of requirements can vanish, such as when a construction no longer needs parking or a piece of software no longer needs a feature because the operating system now takes care of that aspect. Of course, there are also new requirements, conspicuous in hindsight, leading to regrets that there should have been a little more forethought. Then there are the horrible kinds of change whereby the requirements were gathered, but misinterpreted. The literal requirements remain unchanged, but the solution must accommodate the new interpretation.
An up-front analysis could have revealed some of these potential sources of outside interference, but performing it later leaves these risks unexplored until you can better discern their importance. Consequently, neglecting any up-front requirements analysis is a foolish way to develop anything, yet doing everything at the start is equally daft. The point must always be to reconsider your motivations. What is the state of the world, and what do you currently know about the problem? Each step is a place to stop and assess where you are now and where you need to go next.