Posted On: 2019-06-17
Productivity in software development has always been difficult to measure. Whether it's measured in lines of code per hour or story points per sprint, productivity is always focused on quantifying the usable output of the development process. Such an approach, unfortunately, ignores the inherent messiness of code as a creative work: it is indifferent to the learning and experimentation that is often a prerequisite of quality code. Historically, I've regarded the pursuit of productivity measures as the (completely understandable) folly of the business side of software: measuring output is an essential part of estimating whether developers can meet deadlines/milestones.* More recently, however, I have discovered that this same mindset, measuring progress only in terms of success, subconsciously influences how I perceive my work on a day-to-day basis (even though I frequently acknowledge the inadequacy of only recognizing success.)
At this point, I figure that I owe an explanation for why measuring success is inadequate. I will start by picking on the process of measuring lines of code, as this is a common (naive) approach to measuring productivity. I am relying on it as it is easy to explain, but it is also fairly well understood to be inadequate, so it's something of a straw-man. If you're already familiar with why measuring lines of code is ineffective, feel free to skip the next paragraph - the remainder of this section will focus on features that are common across many forms of measuring productivity.
Measuring lines of code is the process of taking a single, arbitrary unit of measurement (a line of code) and counting the number of them that appear in the code base for a piece of software. The closest analog for those who aren't familiar with software is counting sentences to measure a written work. The assumption is that, as code progresses towards a finished state, it will necessarily increase in the number of lines of code. This is just as true for software as it is for a novel: as one writes a novel, it certainly does increase in number of sentences, however, as one edits a novel, the number of sentences decreases, as one removes the unnecessary or combines multiple sentences to convey more meaning with less text. In software the process refactoring is similar to editing: it involves reducing the number of lines of code as a part of an effort to increase the overall quality of the code. Much like editing, refactoring is an essential part of development, and how much of it will be required varies greatly based on the nature of the project and the skills of the people involved. Also like editing, refactoring tends to occur in large bursts, rather than (or in addition to) occurring naturally over the course of development. As a result, the activity that most improves the code's quality often has a near-zero or negative productivity score when measured using lines of code.*
While the preceding argument against using lines of code is one that is often articulated, I'd like to dive into another, less explored reason why lines of code are inadequate for measuring productity. Consider again the analogy of the writer: there are many situations in which a writer will write something only to discover it just isn't working. Perhaps someone is not acting in-character, or perhaps a particular in-character action is writing the story into a corner. In such a situation a writer may discard this work, or flag it for major revision. None of those sentences will ever appear in the final version, so should we even count this as making progress? As an outsider looking in, it seems reasonable to claim this is progress: the writer is discovering what doesn't work so that they can better create something that will work in the future. Perhaps this revealed the need to further refine a particular character, or revealed a pitfall of an existing character design. Whatever the cause, gaining that knowledge will make the final product better, so it seems reasonably to consider this to be progress.
Software developers also experience similar kinds of road-blocks and set-backs: perhaps an algorithm isn't behaving as initially expected or perhaps an unanticipated edge case makes a particular approach unsuitable. Again, I think it is reasonable to say that the developer is making progress when they discover this. Beyond that, however, they also make progress outside of the code as they work on trying to understand the issue and come up with a resolution. This is the place, in my opinion, where measuring success most clearly falls down: as the developer thinks, scribbles notes, or tests hypotheses, they aren't adding new lines of code we can measure.** Nonetheless, this is often the most important part of the process: without the time spent working through the problem, it would be impossible to actually code the solution.
Everything I've written up until now largely reflects things that I've believed for many years, nonetheless, this past week, I've had an experience that reminded me of the importance of actively remembering these lessons. The first half of the week was spent as I normally do - writing blog posts, trying to implement the mechanics I want to showcase in the next prototype, and trying to put some time into creating the some portion of the art required to support those mechanics. Yet, by the middle of the week, I was despairing - I had made no progress on the mechanics, and the art, while marginally progressing, had yet to be included into the project. In short - I had no demonstrable progress, so I felt like I had failed.
I began to worry that perhaps I had too much scope, and might need to cut back, so I wrote up everything I knew and would need to accomplish the one mechanic I was focused on, as well as a couple sketches for workarounds as a backup. I also spent some time identifying my roadblocks (I hadn't checked code in for days, since nothing worked, and my fork was missing nearly all the work I'd done the past week on the trunk.) Armed with the knowledge of what to do, and the acceptance that I may have to compromise my goals just to get it to work at all, I set out to make Thursday as productive as I could: I wanted something to show that the whole week wasn't a waste.
And then I completely implemented the mechanic, exactly as I wanted, in a single day.
By the end of Thursday, I was looking back on the day, trying to understand what happened, how did it all work so well? At first, I thought about how much planning I'd done the day before, and how I'd cleared out my roadblocks before starting to work - these are both true and definitely did help. It was only later, as I kept mulling over what had happened, that I saw the missing piece: all week, I had tried and failed. By the time I was implementing it on Thursday, I already knew what didn't work, so I could focus on implementing what did. Thursday, of course, also did include a couple small things that didn't work, but I was able to save myself so much time by already knowing which pieces I would have to write from scratch versus which pieces I could rely on modifying existing code to accomplish. (As a quick example, I could not use negative amounts of contact damage to heal - in theory it should work, but after chasing it most of Tuesday, I could see clearly there was more to rewrite than I would actually be able to preserve.)
Perhaps the most eye-opening thing for me was that even I, who often advises others to perceive failures and canceled projects as mere stepping stones to a better future self, still fell prey to erroneously conflating the absence of success with a lack of progress. Now, as I look back on all of this, I am hopeful that this experience will help me in the future - the next time I worry about not making progress, I can look back on the lessons here and remember that everything is progress, it's just hard to see that when I'm in the moment.