Thursday, June 28, 2007

Software Development vs. Engineering

I recently read Kelly Waters' blog entry about the 'uncommon sense' of agile development. Kelly's argument is that when you strip Scrum of its jargon, the ideas are self evident andjust common sense. So why is there so much controvery then? I think one of the problems is that software has been saddled with an analogy between software and civil engineering, e.g. contruction of bridges, buildings and such things. This way of looking at things continues to influence organizations to this day.

I believe one of the key reasons for the emphasis on up-front requirements and up-front design phases in the world of software stems from the idea that developing software is like engineering or architecture. In these areas one spends a lot of time on design, and only once the design has been completed does one actually engage in the activity of construction. If one takes this analogy to be valid, then developing a piece of software iteratively looks foolish: Would it be reasonable to design a skyscraper by first building a bungalo and using the feedback from that to build a two story house, etc. until one has a skyscraper? Of course not. I think that for the most part the approach that's currently used in bridge building and architecture makes good sense for those disciplines, but it doesn't translate nearly as well to the world of software. In my opinion, there are three primary dimensions that make software a rather different animal:

1) One can use engineering formulas in a practical way to validate a design ahead of time, say to make sure a bridge will stand up to a certain load of traffic. Software however is not subject to the contraints of physics, so the applicability of this kind of technique is limited to the fairly narrow area of algorithms. It may make sense to make sure a scheduling or path finding algorithm is correct for example, but does that kind of idea apply to a software system in general? Are there any engineering formulas that would insure that a design for accounting software will meet its users requirements?

2) Models are not nearly as useful in software development as they are in engineering. Models and simulations of bridges and buildings get pretty close to the reality of what one is trying to build without actually having to build it. One can look at a scale-model of a building or a 3-d computer simulation and get a lot of really valuable feedback. Things like ERD diagrams and various UML constructs, e.g. class diagrams, sequence diagrams, are far less kinesthetic - I'd say they're even less expressive than an engineering blueprint. Does some collection of classes and methods really represent a good solution for a given problem? Yes? No? Maybe? I've found it very difficult to anticipate from such diagrams what it will be like to actually work with the code.

3) Finally, buildings and bridges are designed to resist change. Once you've built something like that, it will remain that way until it needs to be replaced - every aspect of the effort is focused on that goal. Would anyone consider adding a few extra stories to an existing skyscraper, or adding a lane to a bridge that's already been built? On the other hand, most software systems must stand up to constant change during their life spans and those changes are often substantial. One's approach to software design should be flexible and adapt to future changes.

Monday, June 11, 2007

Little Shop Of Sudoku Horrors

I've recently run across a few examples of unorthodox Sudoku solver programs which are kind of entertaining in a way, but also frightening. Here's an example using sql, and here's another example using regular expressions. I've spent just enough time looking at these solutions to be horrified. I certainly wouldn't want to have to maintain such code, and it confirms my general opinion that sql and regular expressions are appropriate for a limited range of situations and should not be used for general programming. Of course it's kind of amusing to see that it's possible to write a sudoku solver with these technologies in the same way that this is amusing.

Additional sudoku horrors: Sudoku in cobol! Shortest sudoku solvers! These are kind of cool, but also teach the lesson that writing maintainable code isn't just about making it as concise as possible!

Reference: Peter Norvig's python code. This code is certainly clean and compact (I do take issue a bit with his constraint propagation. You can have a look at my java code example for more details). In any case, much nicer than the craziness above.