Some ideas for programmer metrics

Lines of code is a terrible metric of programmer productivity, prone to all sorts of gaming of the system, and here's a great story to illustrate: -2000 lines of code (+ some nice discussion).  Bill Atkinson ends up subverting the simplistic management by entering -2000 under the lines of code box, signifying the net number of lines he contributed during the week.

Of course, what this didn't capture was the qualitative detail of his work: identifying bugs, deleting, adding, fixing, abstracting, simplifying.  No single number can capture that.

Indeed, the great W Edwards Deming warned -- but who takes notice? -- that taking any measure (simple or complex) and rewarding or punishing according to it is asking to be gamed.  Doing so made #3 in his list of seven deadly diseases:

1. Lack of constancy of purpose
2. Emphasis on short-term profits
3. Evaluation by performance, merit rating, or annual review of performance
4. Mobility of management
5. Running a company on visible figures alone
6. Excessive medical costs
7. Excessive costs of warranty, fueled by lawyers who work for contingency fees

That said, it's not to say that metrics can't be useful as a source of insight.  I wonder whether a simple multi-dimensional metric on lines-of-code could be helpful.

Suggestion:

1. Lines of own code added
2. Lines of own code deleted
3. Lines of others' code added
4. Lines of others' code deleted
5. Lines of code deleted by others

I envisage that collecting this kind of data on a weekly basis (say) could show interesting patterns for individuals over time, and corroborate gut feelings about who is refactoring vs adding functionality vs breaking stuff etc.

Another aspect to consider is the scope of these metrics. Does one measure individuals, pairs, or whole teams? If the metrics were implemented for the whole team as a means of self-assessment changes over time could be due to external factors (changes in the kind of functionality requested, crunch times, etc.) or internal factors (learning, change of personnel, change of practices, etc.).

C# is *slowly* turning into Eiffel

When I first read Bertrand Meyer's fabulous tome, Object-oriented Software Development, (2nd edition). in 1997 I guessed that it was ten years ahead of its time.  Seems like I wasn't too far off the mark, with .NET 4 introducing language independent support for Design by Contract, as code contracts, and patching up some of the covariance and contravariance issues of previous versions.

How long it will take for most programmers to use the techniques well is anyone's guess, but having mainstream tools that support these approaches is a good thing.

Hygiene and Motivation

In Nine Things Developers Want, Rob references Herzberg's Two Factor Theory:

In the 1950s a researcher named Frederick Herzberg studied 200 engineers and accountants in the US. He asked them a few simple questions and came up with what is one of the most widely-accepted theories on job satisfaction called Two Factor Theory.
His theory breaks job satisfaction into two factors:

• hygiene factors such as working conditions, quality of supervision, salary, safety, and company policies
• motivation factors such as achievement, recognition, responsibility, the work itself, personal growth, and advancement

Hygiene factors are necessary to ensure employees don’t become dissatisfied, but they don’t contribute to higher levels of motivation. Motivation factors are what create motivation and job satisfaction by fulfilling a person’s need for meaning and personal growth.

In his cracking TED Talk Dan Pink delves into some of the behavioral research that explores the consequences of using extrinsic factors such as monetary rewards as motivational tools:



Bottom line: extrinsic rewards tend to narrow focus and hamper creativity.

Fartlek development

The amusingly named Fartlek training is a form of athletic training in which high intensity work and low intensity work are alternated in the course of each training session. The two forms of work are complimentary in that they stress the you in different ways (anaerobic and aerobically) and allow you a chance to recover.

I think that this is a good model for software development in software start-ups.

Priority one: Build something that people want, fast

The main risk in a software start-up is making something that people don't want. So you want to find out whether your initial idea has merit, fast. So -- if at all possible -- you build a small prototype, and test it out in the marketplace, fast.

In order to satisfy the need for speed, you cut a few corners, and because it's small (and your team is good) you can get away with this approach (for now).

If no-one wants your prototype, you scrap it, and you haven't wasted time on lots of process that was supposed to pay off in the long-run (because there is no long-run).

On the other hand, if it the prototype is promising, you will learn from the market feedback how to make it better, and it will start to grow. This is where it is critical to switch gears and slow down (at least for a while). Because if you do not, you will find yourself in the realm of cowboy development, the bugs will start to bite, and the pace of development will drop as your team gradually spends more and more time debugging, less time developing, resulting in a drop of confidence -- "I don't want to quickly make this change because it might break something" -- and loss of tempo.

Priority two: Restore quality (and sanity)

The alternative is to switch modes. Fred Brooks said "build one to throw away", but at a minimum you should review what has been done to date and investigate and prioritize the very real risks of continuing with a prototype. These might well include:

1. Customer or prospective customer loses data or has it corrupted
2. Customer or prospective customer experiences bugs, crashes, or loss of service
3. A crash or a bug during a demo
4. The software does not scale, so performance degrades over time
5. Development slows so that critical new features can not be delivered in a timely fashion

Improving the quality of the software process is what is needed for most of these, and many of the kinds of measures that help have largely been formalized in the Agile development processes:

• Pair programming: Two heads are better than one, plus it offers real time code reviews, mentoring, camaraderie and friendly competition, less web-surfing
• Test-driven development: Writing automated tests clarifies the design and as they build up offer an early warning system when a change breaks something unexpectedly (Design by Contract gives similar benefits)
  
• Continuous integration: Frequently "checking-in" and automatically running the automated test-suite identifies incompatible changes made by different programmers or pairs sooner rather than later; saves you from merge hell
• Refactoring: By reviewing code, eliminating duplication, reducing coupling, and improving the use of abstraction, your software base will become more concise, readable and maintainable
  
• Short iterations / planning game: By maintaining a list of feature requests and bug-fixes that the developers estimate and the internal customer prioritizes (within a budget based on the amount of work accomplished in the previous iteration), a sustainable rhythm is established, and healthy prioritization is forced
  
• Sustainable pace: Working more than an eight hour day / forty-hour week leads rapidly to damaging the code base, loss of morale, and bull sessions as people are present in body, but not in spirit -- avoid the "Death march" scenario
  

Additionally, a design review will be needed to knock the existing architecture and code into shape, and tests will needed to be added retrospectively if you didn't practice test-driven development in the initial prototype phase.

For a web-based service additional infrastructural elements and diagnostics should be reviewed and added.

All of this is manageable as long as the prototype is small, but the longer these steps are delayed, the greater the costs that must be paid. It's a lot like going into debt. You not only have to repay the loan, but also the accumulated interest. Some people call this design debt. (And the interest rates on design debt tend to be high.)

And then the cycle turns

While making incremental changes continued agile iterations is probably the way to go, slow and steady (but not slowing down). When the time comes to make a leap forward, knocking up a quick prototype is a great way to vary the routine, and makes business sense.

Conclusion
Almost everyone who learns to program first experiences the joys of "cowboy development" on small personal projects, but when they "turn pro" or start working on larger projects they soon learn that it doesn't scale beyond what you can easily hold in your head.

Some people think that up-front design and analysis will save the day, but this leads to waterfall with its 1/3 design, 1/3 development, 1/3 testing rule-of-thumb when done well. But then you discover that the requirements were wrong (they always are), and another long iteration is required.

Smart people rediscover short iterations, and lately these have been articulated as Agile Methods, and also include other ways to make the computer (e.g. automated tests) and human nature (e.g. pair programming) work effectively and more satisfactorily. The cost is a bit of deferred gratification by having to do some things up front (e.g. writing tests, refactoring) that pay off over time.

However, it would be wrong to say that an Agile approach is the best for everything, at every scale. In particular, in knocking up a quick prototype one may well be justified in reverting to some of the cowboy practices for a short period. Note that it is not compulsory to drop all the agile practices when doing so -- do what suits the situation.

The tricky bits are:

1. Not leaving it too long
2. Paying off the accumulated technical debt
3. Getting back into the discipline of following the practices
4. Combating the likely organizational pressure to continue at the previous pace!

So good luck with the fartlek development*, and let me know what you think.


*My esteemed colleague -- and former Olympian -- Ben Loft points out that interval training may offer a better analogy, but the name isn't as memorable.

Mullenweg's 12 rules for getting from 1 to 100,000 users

Matt "Wordpress, Akismet, Ping-o-matic" Mullenweg has 12 rules for getting to a large number of users:

1. You have to be the most passionate person involved.
2. Get off the computer - the act of writing things down on paper frees the mind, allows for the juices to really flow.
3. Obsess about the details, down to the space between two letters.
4. Do your own support. You have to be able to feel the pain of your users. Document everything. Make it as easy as possible for your users to contact you.
5. Blog every step of the way. Keep all of your users in the loop at all times - they will love you. Communicate with them and put them in the driver's seat.
6. Have a great tagline. If you can't describe what you are doing in less than 5 words, edit it, shave it down.
7. Frame everything you're talking about in a context for your users. What are you going to do for them?
8. Get out of version 1.0 as fast as possible. Most people make their successes on something different from where they started. Be flexible. User feedback is the most valuable asset. Don't let yourself be too led by your first users. Listen to the silent majority. Keep the majority in mind.
9. Track yourself.
10. Know what to do if you are successful.
11. Start strong, end strong. People don't often remember what was in the middle.
12. Be a pain killer, not a vitamin.

The 46-minute podcast of Matt's talk.

Excellent food for thought. One that caught my eye was #6. The problem: Get the shiny draft mission statement down to a snappy under-five-words tag-line.

What's important?

Computer scientist and mathematician Richard Hammond -- he of the Hamming Code -- used to ask his colleagues two questions. In order:

1. What the most important issues in your field?, and the follow-up
2. Why aren't you working on those?

Often this induced a cold-shoulder response, but others were grateful for the nudge.

I have been meditating on what the equivalent question should be in a business, as opposed to Science. My tentative questions are:

1. What are the most important issues facing our customers?
   
2. Why aren't we working on those?
   

From this it follows that deeply understanding our customers' issues is of paramount importance.

The Programming Djinni Grants Wishes

As a commercial product develops the wish-list grows longer and longer. User suggestions and feedback are collated and prioritized. Pretty soon you have a very large list.

The tragedy of the C cases
Here's the problem: Lots of worthy but little items are continually pushed to the bottom of the pile, by the latest big feature.

I call this "the tragedy of the C cases". These items of functionality may languish, if proper discipline is respected, never bubbling to the top.

There are disciplined ways around this use Scrum and XP style planning, but these require buy-in from the non-programmers and may not be achievable within your organization.

The magic approach
Here's a simple and fun alternative for a small company or department: One day per month -- customarily on the 23rd (or the nearest weekday) -- one or more programmers turn into djinnis who attempt to grant reasonable and achievable wishes to one lucky person in the company.

This breaks up the usual routine (a good thing as per the Hawthorne effect), is engaging for the lucky person -- who sees some immediate results -- and it is also good for the programmers, who get the satisfaction of working directly with a tangible person for a change rather than through a mediated priority list. And the software gets nicely polished.

Try it: See how you go; let me know.