Happy New Year / Kids and Maths

Happy New Year!

This morning -- January 1 2009 -- Jake and I went for a walk around the park and greeted various joggers and bike-riders and walkers with hellos, often resulting in replies of "Happy New Year!".

On the way back Jake -- who is almost five -- said to me, "Daddy, do you want to hear something that I know that you don't know?"

"Sure", I said.

"There are two fours in eight, and four twos in eight!"

Actually I did know that, but didn't say it, and instead felt very proud, and asked Jake how many twos there are in six, and how many threes.  I also asked him how he got to his first result: He "figured it out".  So I told him that he was noticing patterns in numbers, and that that is a good thing to do.

Just for the record, without much prompting from me, basically just encouraging him to count forwards and backwards, counting on fingers, tallying, and recognizing written numbers, my boy spontaneously noticed a pattern illustrating not only a feel for division, but also for symmetry, or more specifically the commutativity of multiplication (a*b = b*a always) albeit in this one case.  And this before noticing commutativity of addition (a+b = b+a always), and even the trick of "counting on".

I'll take this as evidence that (at least for Jake) encouraging an awareness of numbers and patterns is not a bad way to go. This contrasts with my experience of doing pages of simple sums and subtractions (4+3= ...) at age four, which apparently I enjoyed!  

Not a bad new year's present for a dad!

Peter Sbarski has submitted

Congratulations to Peter Sbarski on submitting his PhD thesis (just in time for Christmas). Peter developed the layout algorithms used in Authink's software, and more besides. His supervisors were Prof Kim Marriott of Moansh University and Austhink founder Dr Tim van Gelder, but Peter was kind enough to also thank me for my help and sometime mentoring:

Dan, I must thank you for all your advice and work that you have put into this thing. I know that my thesis wouldn't be the same without your help. I really do appreciate everything that you have done and I'll always remember it. In fact, I feel so very lucky that I got to work with you. A huge chunk of this thesis belongs to you.

[Quoted with permission.]

It should be pointed out that Peter was kind enough to include me as a co-author on some of the papers that came out of his thesis work, which I appreciate.

Peter is a bright fellow and collaborating with him and Kim has been a pleasure. I wish him all the best, and hope to work with him again one day.

Programming Languages as Cars, Religions

I owe part of my vanity rank to my contribution to Michael Vanier's amusing If programming languages were cars ... rant:

• Ada is a tank. A butt-ugly tank that never breaks down. People laugh uncontrollably if you tell them you drive Ada, but really, do you want to be driving a sports car in a war zone?
  

• Assembly Language is a bare engine; you have to build the car yourself and manually supply it with gas while it's running, but if you're careful it can go like a bat out of hell.

  Assembly Language: you are the car.

• Basic is a simple car useful for short drives to the local shops. Once popular with learner drivers, it has recently been stripped down to a shell and rebuilt by a major manufacturer, The new version has been refurbished for longer journeys, leaving only cosmetic similarities to the original model. [from Przemyslaw Wrzos]

• C is a racing car that goes incredibly fast but breaks down every fifty miles.

• Cobol is reputed to be a car, but no self-respecting driver will ever admit having driven one.

• C# is a competing model of family station wagons. Once you use this, you're never allowed to use the competitors' products again.

• C++ is a souped-up version of the C racing car with dozens of extra features that only breaks down every 250 miles, but when it does, nobody can figure out what went wrong.

• Eiffel is a car that includes a built-in driving instructor with a French accent. He will help you quickly identify and learn from your mistakes, but don't you dare argue with him or he'll insult you and throw you out of the car. [From Daniel Prager with some embellishments]

• etc. etc.
  

Now amz (and friends) have been inspired to write: If programming languages were religions.... Not quite as pithy, but possibly more accurate! Cut-down version:

C would be Judaism - it's old and restrictive, but most of the world is familiar with its laws and respects them. The catch is, you can't convert into it - you're either into it from the start, or you will think that it's insanity. Also, when things go wrong, many people are willing to blame the problems of the world on it.

Java would be Fundamentalist Christianity - it's theoretically based on C, but it voids so many of the old laws that it doesn't feel like the original at all. Instead, it adds its own set of rigid rules, which its followers believe to be far superior to the original. Not only are they certain that it's the best language in the world, but they're willing to burn those who disagree at the stake.

C++ would be Islam - It takes C and not only keeps all its laws, but adds a very complex new set of laws on top of it. It's so versatile that it can be used to be the foundation of anything, from great atrocities to beautiful works of art. Its followers are convinced that it is the ultimate universal language, and may be angered by those who disagree. Also, if you insult it or its founder, you'll probably be threatened with death by more radical followers.

C# would be Mormonism - At first glance, it's the same as Java, but at a closer look you realize that it's controlled by a single corporation (which many Java followers believe to be evil), and that many theological concepts are quite different. You suspect that it'd probably be nice, if only all the followers of Java wouldn't discriminate so much against you for following it.

Lisp would be Zen Buddhism - There is no syntax, there is no centralization of dogma, there are no deities to worship. The entire universe is there at your reach - if only you are enlightened enough to grasp it. Some say that it's not a language at all; others say that it's the only language that makes sense.

Haskell would be Taoism - It is so different from other languages that many people don't understand how can anyone use it to produce anything useful. Its followers believe that it's the true path to wisdom, but that wisdom is beyond the grasp of most mortals.

Erlang would be Hinduism - It's another strange language that doesn't look like it could be used for anything, but unlike most other modern languages, it's built around the concept of multiple simultaneous deities.

Ruby would be Neo-Paganism - A mixture of different languages and ideas that was beaten together into something that might be identified as a language. Its adherents are growing fast, and although most people look at them suspiciously, they are mostly well-meaning people with no intention of harming anyone.

Python would be Humanism: It's simple, unrestrictive, and all you need to follow it is common sense. Many of the followers claim to feel relieved from all the burden imposed by other languages, and that they have rediscovered the joy of programming. There are some who say that it is a form of pseudo-code.

COBOL would be Ancient Paganism - There was once a time when it ruled over a vast region and was important, but nowadays it's almost dead, for the good of us all. Although many were scarred by the rituals demanded by its deities, there are some who insist on keeping it alive even today.

APL would be Scientology - There are many people who claim to follow it, but you've always suspected that it's a huge and elaborate prank that got out of control.

Visual Basic would be Satanism - Except that you don't REALLY need to sell your soul to be a Satanist...

Although I have nothing to add -- at this time -- I did enjoy many of the comments on proggit.

Waffles are go!

My quest for waffles is over. Last Saturday I saw a modern waffle-maker at K-Mart with my beloved and gingies, and -- despite being there to purchase gifts to place under the Wishing Tree, and not stuff for ourselves -- I had to have it. Now it is mine:


The recipe was complicated, including sifting, separating eggs and beating the whites until they stiffened, folding, and -- most challengingly -- leaving the mixture to sit for 10 minutes.

Then you cook each waffle for exactly 3 minutes:


cool them on a rack:


Now cut them up and sprinkle with sugar, Belgian style:


And admire the complexity of the waffle-creation process:


Now enjoy!


Hmmm: They tasted like ... waffles! I.e. Good. The general consensus was that they are even better with ice-cream and Maple syrup. There was so much mixture that we were able to freeze some for later. I think that they should reheat quite nicely in the toaster.

Interesting articles on programming and management

The following are informative and frequently humorous resources for those of us working professionally in software development:

• The Hacker FAQ: Explains hackers to managers
• The Manager FAQ: Explains managers to hackers
• How to be a Programmer: Answers questions for those at various stages of the programming career-ladder

Anyone who fits the hacker archetype described in the Hacker FAQ may also find The Bipolar Lisp Programmer interesting.

My quest for waffles

As a child we had an old-fashioned waffle-iron. It was a two-handled affair that you would pour the batter into, close, and then cook over the stove. I favored my waffles with maple syrup and ice-cream.

Now that my children are eating solid food and enjoying occasional Sunday pancake (really crepe) breakfasts, I find myself pining for waffles, and wanting to share the experience with them.

So I am on a quest for a waffle iron, and I am torn: Do I go for authenticity and purchase a waffle iron of metal -- perhaps even an expensive Italian one that brands the maker's-mark into the waffle -- or a modern jaffle-like contraption that promises more consistent results, and easier cleaning?

Yesterday I bought a waffle at the local food-hall, and while adequate and not cheap, it was re-heated and not ideal.

Stay tuned. Meanwhile here is an article about the ancient and noble history of waffles.

Promoted: The Kibitzer becomes the Kibitzed

Around the start of the month I was promoted from Lead Software Developer to Chief Technology Officer at Austhink Software. [Look ma, I' m a CTO! If only we had an executive washroom. ;-)] I will still be hands on -- designing, programming, pairing, etc. -- but with the new title comes some additional power and responsibilities. In particular, I now have a hand in setting technology strategy plus additional leadership duties.

An interesting effect of the promotion has been that where previously I was the kibitzer -- giving unsolicited and sometimes unwelcome advice -- I have now become the kibitzed! The guys on my team -- Ben and Aaron -- are becoming increasingly assertive in stating to me what they think should be done. And that's fine with me. It seems to me that the allied challenge is to become a better communicator: Both in listening closely to the kibitzs (advice) and kvetchs (complaints), and in clearly articulating the over-arching priorities, so that we can all work together in setting and executing our plans.

Other than that, my motto since making the change has been: "Review, re-focus, and get on with it." The review part has been about taking stock of where we are, the re-focus is about articulating concrete changes in approach, and getting on with it is all about moving forwards.

So I expect that there will be a change tone in future entries, arising from my new perspective at work. OTOH: Whenever I run out of ideas I can always look back over my old blogs for inspiration!

What Ella wants for her 3rd birthday party

1. Cupcakes: No icing
2. Pink icy-poles
3. Mozzarella
4. Cheese sticks
5. Apricots
6. Snakes
7. Chocolate frogs
8. Lollipops
9. Birthday cake

Cross-promotion

My patchwork quilt obsessed partner has started her own blog: Patch Andi.

We're a blogging family now!

What's your vanity rank?

A vanity search is when you type your own name into a search engine.

Now, most of us vanity searchers have learned that we are not alone -- there are pretenders out there using our names! So I propose a new term: "Vanity rank". To determine your vanity rank, simply do a vanity search and find the first link that refers to you rather than some doppelganger.

What's my vanity rank? At time of writing -- using Google -- it is 2 as Daniel Prager, and 1 as Dan Prager.

Please look out for my coming book, "How to boost your vanity rank", Narcissus Press, a subsidiary of Mirror Corp.

Why Scheme rocks

Scheme is a modern Lisp dialect. It's what I program in for fun, and to learn new stuff.

Why is it so great? In a word, meta-programming. I can bend the language to my will.

For example, I have just added my own tracing to function definitions, without touching the insides of the function. Instead of using the built-in (define ...) I use my own 12-line* macro (define-with-trace ...). Now all I need to do is figure out how to redefine define.

Here is a more poetic endorsement.


* Plus 4 lines of auxiliary code.

Let My Textbooks Go!

There are three problems with prescribed textbooks:

1. A generally low standard: Feyman's anecdote. Has anything changed since the 60's?
   
2. An inflated sticker price: A NY Times article explains how textbook publishers are like big pharma.
   
3. Limited content: So learn how to search the interweb!
   

Textbooks are starting to be given away for free. One free one that I have bought -- because I wanted a dead leaves version -- "one of the great classics of Computer Science" - Paul Graham.

This is free as in free beer, which addresses point 2. Point 1, the generally low standard, might be helped by more collaborative work on the content, taking a leaf out of the free software movement.

We'll see ...

Unit testing vs Scenario testing

Miško Hevery gives some excellent advice on designing testable software here, and elaborated here:

• Unit Testing as the name implies asks you to test a Class (Unit) in isolation.
• If your code mixes Object Construction with Logic you will never be able to achieve isolation.
• In order to unit-test you need to separate object graph construction from the application logic into two different classes
• The end goal is to have either: classes with logic OR classes with “new” operators.

Here's the explanation / summary:

Asking for your dependencies instead of constructing them withing the application logic is called “Dependency Injection” and is nothing new in the unit-testing world. But the reason why Dependency Injection is so important is that within unit-tests you want to test a small subset of your application.

The requirement is that you can construct that small subset of the application independently of the whole system. If you mix application logic with graph construction (the new operator) unit-testing becomes impossible for anything but the leaf nodes in your application.

Without Dependency Injection the only kind of testing you can do is scenario-testing, where you instantiate the whole application and than pretend to be the user in some automated way.

My emphasis.

Now, this is not to say that you don't need scenario testing, but I have found by bitter experience that trying to mock bits and pieces in scenario tends to be more trouble than it is worth: You end up with large and unwieldy pieces of testing infrastructure.

My current recommendation is that you need to test in both isolation and integration:

1. For isolation (unit testing) use TDD and dependency injection, or DBC
2. For integration, use scenario-based testing on the actual application using some kind of scripting interface.

Other specialist tools may be useful. For example, for highly graphic applications it is invaluable to be able to step-through the scenario tests to visually confirm what is happening on-screen. Visual diagnostics are also a great help for this "semi-manual" testing.

Amazon's recommendation system is too good

Amazon makes it too easy to reliably find -- and purchase -- good books.

The histogram of star-ratings, the user reviews, the cross-referencing with other books bought by people who bought a particular item, and what people ended up buying who looked at all make it easy to track down top books in any given area. Not to mention the access to second-hand books, especially for out-of-print items.

Coupled with the recent strength of the Aussie dollar and the exorbitant cost of purchasing locally, Amazon has proved a recent drain on my finances and a burden to my bookshelves.

So I am resolved: I am not ordering any more books from Amazon for the rest of 2008.

Tickle the Mickle

The kids and I invented a new game today, called "Tickle the Mickle".

1. A player announces, "I am the Mickle!", and lies supine (tummy up) on the couch or floor.
2. Everyone else says, "Tickle the Mickle!", and starts tickling the Mickle.
3. This continues until the Mickle says "Stop!".
4. Someone else takes a turn to be the Mickle.

In a competitive game the winner is the Mickle who can endure the most tickling.

Alan Kay on Research into Learning and Teaching

SecretGeek asked Alan Kay about the paper "The Camel Has Two Humps", and received a fascinating reply. Excerpts:

Notion 1: Good science can rarely be pulled off in an environment with lots of degrees of freedom unless the cause and effect relationships are really simple. Trying to assess curricula, pedagogy, teaching, and the learners all at once has lots of degrees of freedom and is *not* simple.

So for example we've found it necessary to test any curriculum idea over three years of trials to try to normalize as much as possible to get a good (usually negative) result.

Notion 2: Most assessments of students wind up assessing almost everything but. This is the confusions of "normal" with "reality".

For example, in our excursions into how to help children learn powerful ideas, we observed many classrooms and got some idea of "what children could do". Then I accidentally visited a first grade classroom (we were concerned with grades 3-6) in a busing school whose demographic by law was representative of the city as a whole. However, every 6 year old in this classroom could really do math, and not just arithmetic but real mathematical thinking quite beyond what one generally sees anywhere in K-8 [kindergarten and grades 1 through 8].

This was a huge shock, and it turned out that an unusual teacher was the culprit. She was a natural kindergarten and first grade teacher who was also a natural mathematician. She figured out just what to do with 6 year olds and was able to adapt other material as well for them. The results were amazing, and defied all the other generalizations we and others had made about this age group.

This got me to realize that it would be much better to find unusual situations with "normal" populations of learners but with the 1 in a million teacher or curriculum.

Kay gives two examples of quite famous teachers who have written books about their methods:

I found Tim Gallwey [Inner Tennis, Inner Work, etc.], who could teach anyone (literally) how to play a workable game of tennis in 20 minutes, and observed him do this with many dozens of learners over several years.

I found Betty Edwards [Drawing with the Right Side of the Brain] who could teach (again literally) anyone to draw like a 2nd year art student in one intense week.

It is also an interesting social question as to why it is ok to celebrate amazing teachers in film -- e.g. Jaime Escalante in Stand and Deliver -- but not to look more closely at their methods and try to work with them to re-jig teacher identification, training and school curricula.

My Virtual Museum of Modern Art

You can create your own modern art at museumr.

Science at Home

In our current home arrangement my kids share a bedroom, and have a separate toy-room. The spare bedroom is my dearest's sewing room, although it often reverts to junk-room state. We use the dining-room as a kind of study.

My son would like a room to permanently set up electric train tracks. The toy room isn't big enough so he uses the floor in the "good room", but we make him pack it up at night.

I have said for some time that I would like a home dojo (martial arts area), preferably in a basement so as not to frighten the neighbours. As something of a homage, I would call it The Dungeon.

Additionally, there is the prospect of "doing science at home". Now, when I did year 12 chemistry I was fortunate to have a skilled laboratory partner, Steve Alcorn, who led the experiments while I took the lead on the calculation. However, my general experience at school and later at Uni. was that lab. work was dull, except on one occasion when I re-designed an experiment to make it work. That was a blast; the creativity and imagination were re-injected.

Reading Oliver Sacks's memoir Uncle Tungsten conveys the joy (and obsessionality!) of designing and performing one's owns experiments. In my case, as a teenager I satisfied that desire in the more abstracted world of exploratory computer programming, but I would like to expose my kids to the more concrete worlds of chemistry, electronics, and -- probably -- robotics first. Biological experiments can wait until they are older!

Apparently chemistry sets don't come with chemicals anymore. Here's a link to The Illustrated Guide to Home Chemistry Experiments, which sounds like a great place to start. Apparently you can get in trouble with the authorities for daring to do Science at home, so perhaps I should secrete "The Lab" behind the Dungeon.

Of course I don't really need a bigger house to do all this great stuff, just time, a little discipline (for cleaning up) and more importantly the patience to wait for the kids to show sparks of interest. Here's a nice story of a parent engaging a child in learning. For now, I'll try to encourage the kids to keep messing around with play-dough and paints, and try not to gripe too much about the mess.

What can go wrong with Agile approaches

Over the years, most (competent) software developers discover that an iterative approach -- Prototype; demonstrate; modify; demonstrate; rinse and repeat -- is essential for successful software development. In recent years this kind of approach has been formalized and elaborated as the Agile family of methods: XP, Scrum etc.

Here Gojko Adzic reports on a talk given by Henrik Kniberg -- author of Scrum and Xp from the Trenches -- on how software development can be screwed up anyway. He presented a long list of issues that have cropped up in Agile projects and -- interestingly -- had audience members hold up cards to signify how much of a problem each pathology was to them.

A good professional development exercise would be to go through a subset of the list with red/green/cards, and talking about the whys, wherefores, and possible remedies.

My Suggestion for a Privacy Statement

Most privacy statements for web-sites include something about the use of cookies. Recently I was peripherally involved in the re-drafting of one such section.

No, not that kind of cookie

Here's my "draft", which could be used as a honeypot to see if anyone ever reads these things:

We do not use cookies to determine the identity of individual users, unless you log in, in which case we track you using all available means, including, but not limited to: satellite, your mobile phone, secret service operatives, cookies, ice cream, fudge, radio-receivers implanted in your molars, and miniature remote-control insects. Depending on what we catch you doing we may use this information for blackmail or our own amusement at in-house video-nights. However, rest assured that we do not permanently store any of the gathered information.

Naturally, this is not the version that we are going forward with.

Some excellent OO programming tips (and discussion)

Some classy advice from munificent on a proggit thread:

• Do not copy and paste. Move code around, but don't duplicate it. Disabling Ctrl-C on IDEs would improve the world's programming better than anything I can think of.
• Decouple decouple decouple. Of all of the things you can do to improve code, reducing coupling will, by far, have the greatest impact at making your life better.
• Don't store what you can calculate. The fewer member variables, the better. Avoid storing any redundant data unless you want to fix synchronization bugs forever.
• Make classes immutable when possible. It's easiest to use something you can't break.
• Don't get too excited about inheritance. If you aren't using polymorphism, you probably don't need to use inheritance.
• Use all of the features of your language. Delegates (function pointers), static (i.e. non-member) functions, etc. are just as useful with OOP as within straight procedural.
• If you have a hard time naming a class, it's poorly defined. That being said, don't let lack of a name stop you, you can always find and replace later. Likewise, rename classes if their purpose changes over time.
• Never call anything "FooManager". Objects are supposed to manage themselves. Get rid of the manager and put the code in Foo.
• Don't be afraid to make little classes. Making a class just to bundle a couple of function args is perfectly fine. A few months later you'll find you've all sorts of other usefulness to it.
• Refactor constantly. Coding without refactoring is gardening without weeding and pruning. You can't just keep packing new plants into your yard.

There is some good further discussion and qualifications in the sub-comments.

Alan Kay's Reading List

Alan "Smalltalk" Kay gives a list of books for people interested in the ideas and philosophies that influenced Squeak. It looks like a good list for anyone interested in software invention.

I have read a smattering of the books and some other works by authors on the list, and they were generally outstanding. I will try to sample the remainder (over time).

A witty remark

The other day I was standing in my bedroom, and went to take a step, only to find that I could not. My son, Jake, aged 4 1/2, was lying full length and prone on the floor with arms extended and both hands securely clamped around my left ankle. Andi looked on. This had happened before.

Jake ordered me to drag him along the floor: "Walk!", came the command from below.

"I refuse to walk anywhere while there's a manacle attached to my leg", I replied.

"That's not a manacle; that's a boy-acle!", said my beloved.

Laws of Logins

Privacy issues aside, when it comes to logins, I want convenience.

When using a web-page that provides a service, I want it to:

1. Allow me to try or taste the service without entering a username and password (or I won't it use it).
2. Never supply username and password if the service is once only (or I won't use it).
   
3. Allow me to login and use a nice administrative interface if it is a site that allows me to create content. For this, I not only accept having to supply login details; I want to be able to login.
   

Of course having to remember multiple logins for multiple sites (point 3) is a PITA. Can OpenID help with this?

Order-dependence of design decisions

When the proggit community was asked for their best piece of Best piece of insight for computer programming/software engineering? some excellent responses were elicited, including this one from hamsterboy:

The quality of the design of a piece of software depends on the order in which you make the design decisions. 5 years from now, the earliest decisions will have the most far-reaching consequences, and will be the most difficult to re-think.

This can be taken incorrectly as an indictment of incremental approaches or, more acutely, as justification for periodic re-design.

Recently I have been working on a port /re-design of a 2.5 year-old product and it has been an absolute pleasure to be able to take advantage of the insights learned the first time round.

Where did the red hair come from?

Here's a picture of my kids, both of whom are gingies:

Ella (left) and Jake

Since neither I nor Andi (Mum) is a red-head we are incessantly asked "Where did the red hair come from?"

Popular answers include:

• Postman Pat
• Alien abduction
  
• They are freaky mutants
  

Incidentally, none of their grand- or great-grand-parents were red-heads, but they have a few carrot-headed cousins.

Apparently the genetics of red-hair is not simple. In particular, it is not just a case of a recessive gene hiding away for a few generations.

Andi and I are not planning any further children. I have suggested that we try for a non-redhead, but the danger of a third gingy has proven to scary.

Jake's 10 Commandments

As part of this year's Shavuot celebrations at my son's kindergarten every child devised his or her own version of the 10 commandments. This is what Jake came up with:

Jake's 10 Commandments

1. No jumping on other people.
2. After eating at kindergarten please have a drink to wash down the food that is stuck in your teeth.
3. No poo-ing in your undies.
4. I will not bend my glasses.
5. My sister cannot slap me.
6. I will not take the mattress off my bed and throw it on the floor.
7. Do not slam the door on somebody's fingers.
8. No touching the stove when it is on.
9. No screaming.
10. No tipping water out of the bath.

Signs of Good Design

It is a good sign when something you design solves problems not originally envisaged. This indicates depth and robustness.

Another good sign is when a third party of high standing makes a recommendation.

Both these signs are apparent in the increasing adoption of the Erlang language and libraries for distributed computing problems. Originally designed for telecommunications software, Erlang is now being used by Amazon, Facebook, and others as part of their infrastructure.

Also, Steve Vinoski, an authority on CORBA, an older technology for distributed computing, has been singing the praises of Erlang for largely "getting it right". In response Joe Armstrong (inventor of Erlang) has thanked Steve for going down the wrong path and living to tell the tale. Here's Joe on Steve, and Steve on Joe.

I hope that everyone working in distributed computing can take note of the lessons that Steve (and Joe) learned the hard way, without necessarily repeating all the hard yards.

Remember: It is always a good time to learn from other people's mistakes (and your own).

Got Art?

Last night my beloved and I splurged on a piece of modern art. In the past we've made our own -- in a "Jackson Pollock" style -- but Andrea spotted this work by local artist Gary Solomon while driving home yesterday:

Go 'Pies!

A screech of brakes, much excitement, a quick discussion of finances, and we're newly minted patrons of the arts. Now we just need more wall-space (and money) to grow our collection.

Programming Yin and Programming Yang

Q: Why is programming fun?
A: It is a form of play and exploration in which a "one-time" "small" effort (writing the program) yields a large reward (the program does something). It combines the artistic joy of creation with the scientific reward of nutting out a puzzle.

Q: Why is programming hell?
A1: When the program fails to produce the expected results a diagnostic process of "debugging" follows. As the program grows in size and power and (hence) complexity, this debugging phase dominates the programmer's time, and brings mainly relief rather than reward.

A2: When requirements change the program may need to be re-jigged to accommodate them, and this re-jigging, similar to debugging lacks the immediate rewards.

* * *

So, to increase the rewards of programming -- and incidentally productivity and profitability -- techniques are sought to reduce the dross (debugging and re-jiggering), and increase the rewards (features and elegance).

On the Yang side of programming I include design, architecture, algorithm invention, and implementation of new features.

On the Yin side I include practices like Design By Contract (DBC), Test Driven Development (TDD), and Continuous Integration (CI). These provide no up-front reward, but pay for themselves over time.

Interestingly, all three involve using programming to write better programs. DBC and TDD may be regarded -- with some licence -- as simple forms of meta-programming. We are writing program fragments to help test our programs. Also, the act of making tacit assumptions explicit helps us to think about and hence improve the underlying design.

By putting in place these effective (Yin) measures, we can be bolder (Yang) in adding new functionality, having built a software safety net as we step forward.

True meta-programming, writing programs that write programs brings us back to the Yang side.
We are now truly working at a higher level of abstraction, with correspondingly more leverage, but when bugs appear in our meta-programs the debugging gets harder too ...

Fred Brookes on Online Collaboration

DL Weinreb, one of the co-creators of Common Lisp, took some notes at the OOPSLA 2007 conference. One of the talks was by given by Fred Brooks:

Fredrick Brooks Jr., author of the classic book “The Mythical Man-Month”, talked about telecollaboration. Most of the talk was about collaboration itself, and under what circumstances it’s a good thing: not always! His main point is that collaboration is great for determining system requirements and brainstorming about possible approaches, but that you really need a single system architect in order to achive conceptual integrity. The system architect can delegate parts of the architecture to others (e.g. the user interface czar), but he distinguishes sharply between delegating design (OK) and sharing design (not OK).

Readers of The MMM, will not be surprised by Brooks's reiteration of the need for a single designer to ensure a unified vision, but it's nice to have strong position brought to light in a new context.

For me, Brooks's pronouncement raises a couple of questions:

• To what extent does his advice apply to non-software enterprises?
• To what extent should collaborative software endeavour to encourage "good practice" through constraining its users?
  

Algorithms vs Architecture

There are many facets to software design. Some, like user-interface design, are apparent to the end user. Others take place almost entirely under the hood. These are the bones on which the software is built.

Two significant under-the-hood aspects are algorithms and software architecture.

Algorithms form a major strand of computer science. They are concerned with
the nitty-gritty of getting the computer to do stuff. The major aspects of an
algorithm include: what it does, its domain of valid inputs, and its time- and
space-efficiency.

Software architecture, on the other hand, is concerned with the high-level structure and conventions of a software system, and is more of a topic for industry than academia. Good architectural choices contribute to the overall performance, resilience, and extensibility of a system.

We need both! And interestingly, you neglect either one at your peril.

For example, recently when we noticed a significant slow-down in performance the issue turned out to be an architecture-algorithm interaction. While the algorithm was O(n^2) in time, an imperfection in the architecture led to it being executed as each item was added to the structure, making it effectively O(n^3), and correspondingly sluggish.

As usual, when the problem is tricky to track down, it is often due to the interaction of two separate-seeming factors.

Just a Little Bit Proud

What with the recent passing of Charlton Heston a newcomer had to step in and play the lead role of Moses in our local kindergarten's model Passover Seder* later today, and my son Jake has scored the honour, leaving me just a little bit proud.

Charlton Heston as Moses

Jake as Moses

Here are the lyrics to one of the more amusing Passover songs that the children will be singing today, recalling the second of the ten plagues that Moses called down upon Pharaoh (and the Egyptians):

The Frog Song (Shirley Cohen)

One morning when Pharoah awoke in his bed
There were frogs in his bed, and frogs on his head
Frogs on his nose and frogs on his toes
Frogs here, frogs there
Frogs were jumping everywhere.

Fittingly, both of his grandfathers will be in attendance, and I look forward to hearing reports, seeing photos and -- technology permitting -- video of Jake with his "overflowing beard" in action.

*The Passover Seder is a special meal in which the story of the Jews' escape from slavery in Egypt is retold and celebrated with food, wine and singing. The model Seder is the kids' version where they learn how it all works. It's a bit like a Christmas pageant, but with more food.

Interview Questions for Candidate Software Developers

Basically I am looking for evidence of the following

1. Smart
   
2. Gets things done
3. Communicates effectively
   
4. Passionate about software development
   
5. Has a solid technical foundation
6. Quick learner
   
7. Cultural fit (sense of humor, down to earth, enjoys working with others)
   

In the past I have written down my questions for candidate software developers 0n scraps of paper, which get lost. Here are some somewhat open questions:

Work history

1. Why are you interested in this job / company?
2. Why now?
3. Tell me about a tough lesson from your last job.
4. What are the key things that you have learned about maintaining and enhancing a pre-existing software system?
   

Object-Orientation

1. What is the difference between a class and an object?
2. What is an interface? Why are interfaces useful?
3. What is inheritance? Compare and contrast single and multiple inheritance.

Programming Language Theory

1. What is recursion?
2. What is an advantage of iteration over recursion?
3. What is an advantage of recursion over iteration?
4. *What is a closure?
5. *What is a continuation?

Agile (assumes some [claimed] knowledge)

1. Have you worked in an "agile team" before?
2. Have you read up about Agile / XP / Scrum?
3. Explain difference between waterfall and iterative development
4. Explain some of the XP practices (e.g. pair-programming, test-driven development, continuous integration, re-factoring)
5. What's Scrum about? How does it work?

Testing

1. Difference between manual and automated testing?
2. Which kinds of automated testing have you used?
3. In which areas is it trickier to test automatically
4. Test first or code first? Why?
5. *What is Design By Contract?
   

Motivation

1. Favorite things about working in IT / software development?
2. Worst things?
3. What do you want to learn more about?
4. Let me solve it and report back, or let's figure this out together?
   
5. What are the key elements that make for a good team?
   
6. Tell me about a cool tool, language, or technique that you have learned and applied recently.
7. Favorite and least favorite programming languages that you have used professionally?
8. Why?
9. What's wrong with [favorite]?
10. What's good about [least favorite]?

Of course, a lot of the interview is about gauging the tone (affect) of the response and following up on interesting responses.

Additionally, I usually do a pair-programming exercise / challenge with technical candidates, usually with a little design phase and some test-driven development. This tells me a lot, but it is time-consuming and can be quite draining (especially if it doesn't go well!).

Cross-ferretization

Cross-ferretization, n. The art of stealing ferreting out ideas from other fields and applying them to one's own.

In programming, there are many generic job descriptions:

• programmer
• software developer
  
• software engineer
• software architect
• computer scientist
• hacker
• etc.
  

Each of these has its own connotations, some overlapping, some contradictory.

Recently I have been delving back into (physical) architecture, and finding out a bit more about that field.

(Physical) architects are the generalists in a once-unified field that has splintered into civil engineers, draughtsmen, builders, interior designers, etc. Consequently their training delves into many fields, and in practice they work with many specialists, especially on large projects.

So it seems reasonable that there is a role for (software) architects in the (virtual) world, although such titles seem to be largely self-bestowed and not really as well-defined as (physical) architects.

Another reason for the interest in architecture in software development is the influence of Christopher Alexander and his notions of patterns and pattern languages on programming. This gave rise, largely via the famous Gang-of-Four book, to the notion of patterns in software (although the notion of pattern languages) was not as widely imported.

For what it is worth, Alexander's original writings have always resonated with me in a way that the Gang-of-Four's have not. So the other day when I spied an attractively packaged little book aimed at (physical) architects and students of architecture, entitled 101 Things I Learned in Architecture School by Matthew Frederick, I purchased it with little hesitation.

Quick review: "101" is not just a book for architects, but for anyone with an interest in any kind of design. Many of the "things" are of general applicability-- I especially like #29, about the importance of being process- rather than product-oriented, and #45, which contrasts simplicity with complexity and informed-simplicity. Such of the ideas are really very general. Others are specific to architecture, either practical points or theoretical tidbits, often admitting useful analogies. Each thing is accompanied by an illustration.

Every field needs a book like this.

Bit rot vs Re-usable code

The intellectual property in a software company is diffuse. The explicit IP can be found in patents, papers, documents and, above all, the code base. The tacit IP exists in people's heads.

It is the tacit IP that allows for innovation and change. Indeed, a good way of understanding the possibility of change is the extent to which the maintainers of the software have a theory of how it works. If they do, they can change and extend it. If they cannot they are likely damaging the conceptual integrity of the code-base.

In The Worthlessness of Code James Turner advocates regularly rewriting to ensure that the software development team understands what they are working with, and in-particular, so that the original designers effectively refresh and transfer their knowledge to new hires.

In the comments the rebuttals of "continuous re-writing", and "re-usable code" are raised. Personally, I sympathize with Turner. I used to have a motto: Before re-use, use. This mocked the scenario of people trying to writing libraries or frameworks that would be used multiply, before being deployed successfully even once. Oy! The better alternative is to extract, abstract and generalize after an initial success or two.

What about libraries? The good ones do have a long life. This comment from the Reddit comments on Turner's article draws an important distinction:

How about "reusable code is an asset, single-use code is a liability"?

I like this idea. There is the abstracted, cleaned stuff that ends up in libraries, and gets re-used, and there is the more scripty stuff that glues things together and gets the short-term job done.

This raises the possibility of a new, somewhat informal metric: Ratio of multiple-use (quality code) to single-use (get-it-done code).