Writing "minimum viable" technical design documents

There are few standards in the engineering department where I work, which enables folks to determine their own best ways of working. But sometimes people new to the company ask for reference material, and that's the basis for this blog post.

We don't have a lot of recurring business-as-usual engineering requests; we typically automate that stuff quickly. This means that many systems and features we set out to build are for problems we haven't faced before. And while every new problem is new in it's own way, there are commonalities, as well. Broadly, there are requirements, dependencies, deadlines... and risk.

Technical design documents (TDD) are great for validating assumptions and raising confidence in an approach while it's still easy to change the approach. In my experience, they are produced much less frequently than they should be, considering their benefit.

A common reason for deciding to jump into development without a TDD is that the problem is not complex enough to warrant one. I've seen this, and Joel has written extensively on it. It seems to me that the perceived effort of creating a TDD is the significant factor. The effort itself coming from having to produce a "process doc" (gross) while also being unfamiliar with how to do so. No one's interested in this combo. So let's break it down.

There's always a design

A project can start without a design, but it will inevitably end up with one. A poor design results in technical debt, impacting all subsequent work. A considered design is the basis for all software architecture, and is critical for successful and sustained software engineering. A considered design is a leading indicator of a healthy codebase, and a healthy codebase is a leading indicator of a productive team.

There's no one-size-fits-all

Architecture is about capturing and communicating the significant technical design decisions that are happening on your team. Therefore, the purpose of a technical design document  is to identify the most significant aspects of the problem you are looking to solve, and the most significant aspects to the approach you want to take in your solution. It's almost entirely subjective, but it doesn't need to be hard, or messy. It can be easy, and it can be fun. It's essentially problem-solving (which engineers love), without the messy code getting in the way.

Get your head in the game

The intention is to get feedback as soon as possible, and ultimately to validate the right (and right-sized) approach. The faster that feedback and validation can be achieved, the better. Emphasize the starting point, the desired end point, and how you plan to get there. The more detail, the better. In your communication, use the best tools for the job. Diagrams are fast and effective, and if you are not familiar with whipping up entity maps or flow diagrams, you should use every opportunity to practice!

Let risk be your guide

If risks are low, then it's reasonable for the doc to be small. It's a small effort that confirms assumptions and caveats, and validates a known and familiar approach, leaving the engineering to dive in to coding the solution with peace of mind.

If the scope is larger, and risk is higher, then go deeper. Clarify the existing system. Brainstorm several potential solutions and weight the options, or determine a strategy for figuring out which is best.

Minimum viable

The following sections are necessary for understanding any sort of problem, right-sizing it for effective delivery, and validating the approach. I can fit it all in half a Google Doc page, or comfortably in an Asana ticket, and it keeps me honest.

Summary

High-level overview of the problem and proposed solution that can be understood by any engineer on your team. Keep it simple and jargon-free, but leverage your domain context.

Context

Tell a story. Like all good stories, use details to describe how we got here, and why it's now important to do something different. Consider sharing a desired future state, even while acknowledging it is out of scope, to set orientation for folks on the team.

Goals

Understand the problem deeply, then scope it down to what is most critical. Too many goals or fuzzy goals are an easy tell that there's more work to be done. There are many effective ways to captures what needs to be done, and your team can figure out which method they prefer. Here are some that I frequently use:

• A simple list of functional requirements (what it does) and non-functional requirements (what qualities should be considered or promoted) are often enough for small scope tasks.
• Acceptance criteria is effective in managing expectations with product stakeholders.
• Key scenarios or user stories covering how different types of users will interact with the system.

In all situations, engineering should understand and buy-in that the goals are a good fit for the current leg of work.

Caveats

Specifically what is not being done (non-goals). The lower and upper bounds for the delivered solution; even arbitrarily so. Considering when and how it will fail. Caveats are a superpower in clarifying fuzzy requirements.

Approach

If the problem has a small scope and you are fairly confident in a solution, go ahead and detail it out. Leverage well-known patterns and include enough detail so that you'd feel comfortable handing this off to another engineer to implement. Include data structures, libraries, algorithms, database models, etc. For more complex problems this can get large, so look to decompose into smaller pieces with their own technical design.

Estimate

Estimates are notoriously difficult to get right, but the exercise of estimating incentivizes some due diligence. Break down the problem, assign estimates to each, tally them up, and double it. Keep all estimates under two weeks, and preferably under one. If this doesn't seem possible, continue to break down the problem.

Going further

The following sections are useful for initiatives with more complex architectural concerns. These documents will always be longer, but should not exceed 10 pages or so, including diagrams.

Glossary

List all special words particular to the problem or domain. This can include technologies, domain language, new concepts or ideas, etc.

Existing solution

If there is an existing solution, provide entity diagram of the system and flow diagrams for the most important user stories.

Proposed approach

Consider both an "ideal" solution and a "cheap" solution, and pick what you think best fits the context and constraints on your team. Consider the following questions:

• What are the major components in the system and their relationships?
• How do the major components talk with each other? How will they fail, and how will the system recover?
• What data stores are used and what guarantees are required? Is a migration strategy required?
• What quality attributes are being promoted with this approach?
• What are the trade-offs?
• How will this approach affect maintainability, testability, monitoring?
• How will this affect the rest of the team or company?
• What are the relative costs for the given approach?

Paths not taken

Come up with at least a couple feasible options that you seriously considered. Keep it brief, and include only enough detail so others understand why it was rejected. Avoid future "did you consider" discussions.

Open questions

This is the "known unknowns" section, indicating where you require more insight or information. Other engineers will look here to help, or can add their own questions, risks, or concerns.

Make it easy and fun

Templates and checklists

It's difficult to face down an empty page. A template as a starting point or guideline really can help folks out, and it also creates a space for useful reminders or tips that can change over time, depending on the team. Useful templates target the 80% case, and encourage engineers to deviate.

Write what you know

The intention of a review culture is to socialize knowledge and good practices across the team. Technical design review ought to feel similar to code review, and in that vein, the goal for an engineer is to clearly communicating what you think is important, and leave it up to your peers to contribute what they think is important.

Keep it simple

Use short sentences, simple words, bulleted lists, concrete examples, and diagrams wherever possible. Remove unnecessary words, run your text through Grammarly, and avoid business-speak or jargon. Stick with a familiar font and size and with familiar domain language to reduce the cognitive load of the engineering reviewing your document.

Make it funny

There's no need to be stuffy or formal. You are working with your team, and they know who you are. Feel comfortable in your use of emojis, gifs, or overly-specific details. It's not necessary to be funny or creative, but it certainly can brighten up someone else's day!

Addendum

Well, I think I've hit upon a pretty good starting point for my own TDD template. Big ups to this article over at freecodecamp.org, where I pulled a few ideas. I'm also a big fan of "Design it! From Programmer to Software Architect." by Michael Keeling. I'm always referring to it. And I recently came across this post, which is forcing me to reconsider some of my section names...