Tuesday 27 July 2010

Some ideas for programmer metrics

Lines of code is a terrible metric of programmer productivity, prone to all sorts of gaming of the system, and here's a great story to illustrate: -2000 lines of code (+ some nice discussion).  Bill Atkinson ends up subverting the simplistic management by entering -2000 under the lines of code box, signifying the net number of lines he contributed during the week.

Of course, what this didn't capture was the qualitative detail of his work: identifying bugs, deleting, adding, fixing, abstracting, simplifying.  No single number can capture that.

Indeed, the great W Edwards Deming warned -- but who takes notice? -- that taking any measure (simple or complex) and rewarding or punishing according to it is asking to be gamed.  Doing so made #3 in his list of seven deadly diseases:
1. Lack of constancy of purpose
2. Emphasis on short-term profits
3. Evaluation by performance, merit rating, or annual review of performance
4. Mobility of management
5. Running a company on visible figures alone
6. Excessive medical costs
7. Excessive costs of warranty, fueled by lawyers who work for contingency fees
That said, it's not to say that metrics can't be useful as a source of insight.  I wonder whether a simple multi-dimensional metric on lines-of-code could be helpful.

Suggestion:
1. Lines of own code added
2. Lines of own code deleted
3. Lines of others' code added
4. Lines of others' code deleted
5. Lines of code deleted by others
I envisage that collecting this kind of data on a weekly basis (say) could show interesting patterns for individuals over time, and corroborate gut feelings about who is refactoring vs adding functionality vs breaking stuff etc.

Another aspect to consider is the scope of these metrics. Does one measure individuals, pairs, or whole teams? If the metrics were implemented for the whole team as a means of self-assessment changes over time could be due to external factors (changes in the kind of functionality requested, crunch times, etc.) or internal factors (learning, change of personnel, change of practices, etc.).