Documentation Driven Development

One of the greatest takeaways I've had from the time spent working at NetSVS, a startup company, has been the pitfalls of not prioritizing documentation. I may be biased, as my academic background in information science likely predisposes me to a more favorable outlook of documentation, but I feel that my concerns are valid regardless. I first started contemplating this matter when I became aware of how much resistance, conscious or not, there is on the part of developers to avoid documentation. Furthermore, this is seemingly reinforced by the implicitly suggested mindset of the Agile development environment we work in.

The developers frequently tend to have a 'Build Now, Document Later' mentality which, when combined with the presence of more than one task needing documentation, results in little to no completion of the associated documentation. This outcome is partially the consequence of not being held accountable for documentation. Additionally to blame however, is the process leading to this result. A developer will complete the build of their software, mark it as complete, and then progress to building their next piece of software with the intention to do the documentation later. After another build is completed the same sequence of events occurs and the documentation from the prior build is further deprioritized or forgotten. When the developer completes their assigned builds they may then start some of the documentation they initially put off, or rather seek to work on something else. In either of these cases the beginning of a new sprint restarts the cycle as the developer's backlog of documentation is further put off in order to address the new tasks on their plate.

The role of the Agile sprint in this process is not only indicating the next phase of development, but also resetting the developer's mental clock to the beginning of their available time needed to complete their work. The combination of the developer's preference for building software over documenting it with their perceived reset of the clock results in a pseudo-rational decision that they can afford to push off the documentation for their new tasks because they will have time to complete it later. However, due to the mental conception of what constitutes a 'reset' or the 'beginning' of the new sprint, the developer has forgotten the previous documentation that they still need to complete and failed to factor it into their decision. Furthermore, this phenomena is implicitly suggested and pushed by the very language of the Agile methodology. This is seen by the two words most significantly associated with the methodology, 'agile' and 'sprint'. Each are heavily associated with the idea of speed, with 'agile' triggering mental associations with its root word 'agility' which implies avoiding obstacles and impediments to progress, while 'sprint' brings with it the association of getting something done as fast as possible and taking the shortest route to a goal. The union of the associations that result from these two significant terms prioritizes completing the minimum necessary to be considered 'done' and avoiding anything else which is not an equally high priority. Typically the output of this process is the Minimum Viable Product (MVP).

Though it is undoubtedly an effective strategy for rapid development of an MVP, the process and mentality outlined above results not only in the need to retroactively document the product, but in poorer quality documentation as well, if it is even done. Because the developer is not documenting immediately after or during development the subject being documented is not as fresh in their mind and more is likely to be missing or inaccurate in the documentation. This becomes a significant issue to anyone who is performing maintenance, or is being otherwise brought into the project without having the developer's knowledge and experience of the project's inner workings.

As a result of this train of thought, a solution came to me which I later discovered was known as Documentation Driven Development. This development process is typified by not just an emphasis on documentation, but rather prioritizing and completing documentation prior to the software build. Not only does this ensure the documentation is completed in a timely manner, but it is more user-centric and future oriented. The features desired by the user and in effect the user guide, form the basis for the documentation and by completing them prior to starting development they become the project specifications. This ensures not only that the documentation is accurate, but that the project's development is planned out as well. Thinking through the project's algorithms prior to building them also enables unique use cases and potential issues to be ironed out at a stage where changing something significant doesn't mean hacking something together on the fly or redoing previously completed portions of the project. In an Agile-like development environment where features can change part way through development, this methodology is implemented by simply changing the documentation to reflect the changed features and then having the developer make the changes they deem necessary to make the build reflect the documentation. Consequently, everything everything ends up clearly specified for the developer.