Posted On: 2019-09-09
While reflecting how I approach my daily tasks, I noticed that each approach I use can be grouped into one of two very different processes. The first of these is highly iterative: it focuses on incremental revisions and tends to be very goal-oriented. The other one is more improvisational, and is often used on tasks where the goal cannot be neatly described from the outset. Using the correct process for a particular task has a tendency to yield better results, both in terms of quality of the final product and also in terms of productively using time. As such, I thought it would be worthwhile to describe these two processes, and their respective strengths, in case they are useful for you as well.
The first of these, the iterative process, is very goal-oriented. It begins by establishing that goal in detail before planning out and implementing the approach to achieve it. Once implemented, the results are evaluated, new goals are defined and the entire process repeated, incrementally improving across each iteration. Once complete, the resulting work is then transferred into a more permenant form (if applicable.)
The first phase of the process is to define the goal - not in terms of what to do, but rather in terms of why it is important and how to know when it's done. This is the opportunity for those working on it to "buy-in" to it: ideally, they should be able to understand the value chain from this individual task through to the company mission, all the way to how that improves the world. While it's not always necessary to spell these details out (company veterans likely already know about the company mission) it's important to have the vision of the task well understood.
Detailed test criteria should be articulated at this stage, since that will be used to verify that the task is completed. Ideally, the test criteria should match the vision for the task, since a mismatch between the two can make it difficult to accurately evaluate the task (while it is possible to revise these after work has begun, any work done without accurate success criteria is often unusable.)
Once it is clear what the goal is, the next phase is to plan how to achieve it. This plan is intended to be a first draft, so it is often quick, loose, and simple. If it can be described in a single sentence, that is usually good. Additional caveats or disclaimers ("this might not work if ...") can be used to describe known risks or permutations to the plan, but it's not necessary. One could say that the goal of this first plan is to intentionally aim at the naive implementation - short, simple, and a quick turn-around time are all desirable features.
With the first plan in mind, it's time to implement it. The implementation will likely reflect the looseness of the plan, so it's often helpful to approach it as a prototype. The objective of this phase is not to pass the original success criteria, but rather to get to a place where the plan's efficacy can be measured. As such, if the task is eligible for automated testing, some initial setup of those tests may occur but it's best to limit the number of tests (usually to one or two.)
The focus of the next phase, evaluation, is not to empirically measure the implementation, rather, it is to learn something about the implementation (what are its flaws, its quirks, its strengths, etc.) This phase can take as long or as short as desired - it is not necessary to learn everything at this time, but the evaluation phase typically provides value more densely than other phases (it has a better time to value ratio.) Some of the most important insights this phase can provide are the places where reality is the opposite of what was expected, so defining expectations before starting can be useful.
After evaluating comes the iteration phase - the namesake of this process. At this point, the task should be much better understood than when the first plan was developed. As such, this phase involves starting over, but this time with a new goal: pick something that can be improved in the current implementation, and go through all the phase again - goal-defining, planning, implementation, and evaluation. One important wrinkle in this is that the evaluation should include both the new and original goals: regardless of whether a change technically works, the original goal should still drive all decisions about the task as a whole. Last of all, one should even repeat the iteration phase - after evaluating, pick a new goal and iterate on that. One should continue iterating as many times as necessary - specifically, one should not stop iterating before one is confident that the current implementation meets the defined success criteria (both qualitative and quantitative.) Continued iteration after that point is polish (not a bad thing, but it can be easy to overdo it.)
Once iterations are complete, there is one final and (unfortunately) herculean task ahead: replicate the working implementation in a final product. The exact way to go about doing this varies based on the nature of the task (replicating code is different from replicating a painting, for example) but, regardless of how, it should eventually result in one final implementation. This phase is the biggest reason why the focus of evaluation was learning: if one has successfully learned over dozens (or even hundreds) of iterations, then applying what was learned can be the means through which one will achieve that level of quality in this final version.
In contrast to iteration, which is about learning how to achieve a specific goal through repeated refinement, improvisation is about creating an implementation that works well enough on the first try. It begins with a blank canvas (or paper or whatever.) From there, one starts the implementation - carefully and precisely. Each individual action (typed word/brush stroke/etc) is evaluated in real-time as it happens, and any deviations from the desired outcome are remedied immediately. This is repeated until the work is complete, at which point the whole is evaluated, and revisions are made as necessary.
By following the process exactly as described, I sometimes find that my improvisation can deviate rather dramatically - for example, a drawing of a character may (somewhere along the way) turn into a drawing of a piece of furniture. Starting with an outline (instead of a blank canvas) usually mitigates this, though it's important to note that substantial deviation from the outline is normal for this improvisational process (for example, an outline of a character may have a different face shape than the improvised version.) Other constraints and guides can also be used during the improvisational process, to improve its ability to conform to some particular objective (as an example, structural elements like topic sentences and conclusions can guide improvisational writing so that it becomes more essay-like.)
One important thing to note about the process is that one does not necessarily need to implement it continuously. It may be interrupted by breaks for other supporting activities (such as research) as well as unrelated activities. Taking unrelated breaks can be a good chance to stretch/come back with a fresh perspective, but there is also a switching cost to doing so. As such, it can sometimes be favorable to save the break until that fresh perspective is most valuable (for example, when stuck or after addressing all known issues.)
There are several factors that are relevant when choosing which of these two processes to use. The biggest of these is the role of the goal: iterative processes are best suited for situations where deviating from the goal comes with a substantial cost. Having success criteria clearly defined up-front is key for staying on-task, even across many iterations. Conversely, when the goal is nebulous or when other, similar work is acceptable, then the improvisational approach may be better suited. In particular, loose goals (like "write a good blog post") can benefit from the improvisational process's penchant for wandering.
Another factor to consider is the role of learning. If one is seeking to actively improve skills, then the iterative process provides much better support for such improvement. Having a phase dedicated to evaluating one's work from the perspective of learning is a key strength of the iterative approach.
Finally, deadlines can have an impact on which of these processes is most suitable. In particular, the improvisational approach can provide a "good enough" product when a deadline is suddenly shortened or the task for some reason goes overlong. The iterative approach is less suited to this, particularly so because all iterations prior to the final one are made as prototypes. (One can mitigate this somewhat by iterating directly in the final product, but that may require specialized tools that support it.)
Personally, I tend to use the iterative process for most of the tasks I undertake. Programming is the most obvious place where I use it, but I have used this approach for a wide variety of tasks. One extreme example of this is that, while growing up, I typically applied the iterative process to conversing: I would often simulate my response in my head and repeatedly revise it until it finally met all my goals (prosody, meaning, impact, etc.) Another is my approach to playing games: I often regard the electronic games of my childhood as having played a large role in my learning how to learn, since I would always try to understand why a particular outcome occurred in a game (this itself is thanks to my ever-patient older brother, who kindly let me play his games and mentored me through many of the early mistakes I made.)
I typically reserve the improvisational process for tasks that I do not know how to effectively evaluate or tasks that I do not expect to improve in. As a result, these tend to be tasks that I approach more impulsively, often attempting to complete them in one sitting. Reflecting on it, it appears that I use the improvisational approach for writing blog posts: until recently, I hadn't thought much about trying to improve on them, and even now I am not sure that I have the necessary critical language for evaluating the prosody of my writing. (Though I should be able to at least evaluate the structure - I can see some flaws that, with research, I should be able to describe and hopefully resolve.)
Iterative and improvisational processes are both useful tools for addressing tasks. Each one is suited to different constraints and offer different benefits, so it is valuable to choose deliberately. Contemplating this has encouraged me to look for ways to use the iterative approach whenever I want to improve, and I hope that this post will likewise help you.