Saturday, November 24, 2007

Pair Programming Redux

I've been working for a few years in a mostly XP pair programming environment. Here's my list of pros and cons of pair programming based on this experience:

Pros:
  • Unlike most other projects I've worked on, I've seen an overall improvement in the code quality over the past few years. I've never seen that happen on any other project. As people learn better ways of doing things, pairing helps to disseminate that knowledge in a way that meetings, presentations, and code reviews just don't.
  • In my experience, there hasn't been a problem with people who are working together arguing and consequently getting nothing done. On the contrary, pairing seems to have encouraged an atmosphere of cooperation and friendliness.
  • Sometimes under time pressure one can't resist the urge to copy and paste some code or hack in some functionality without developing tests first. The mutual supervision of pair programming really does seem to have a positive effect on these kinds of transgressions. It's a lot harder to take a nasty shortcut under the watchful eye of the person you're pairing with.
  • I would tend to agree with the principle that working in pairs doesn't really hurt overall productivity. The reason seems to be that the team having a common understanding of the code and the business generally trumps the value of having two people typing code in separately. Integration is the real difficulty in many software projects, so whatever gain in the amount of code written might come from separating pairs, that gain seems to be offset by the greater consistency of code written when pairing.
Cons:
  • Hygiene: Pair programming in our environment means sharing one keyboard and mouse, and sitting in close proximity to another person all day long as well as switching pairs frequently. We often have had problems with people spreading colds around the office. I think developers in an XP shop should have their own personal wireless mouse/keyboard combination as well as their own chair that they don't have to share with anyone else.
  • Ergonomics: The reality is that sitting at a computer is not a natural position for the body and you can do damage to your back, neck, shoulders and head, and of course your wrists over time. Pairing tends to encourage bad ergonomic habits, because it's inconvenient to change the position of the keyboard tray, monitor, chair, etc when it's time to take one's turn at the keyboard. I really believe that these things need to be considered if you want to do pair programming long term.
  • Personal space: Some people enjoy having some peace and quiet as well as a space where they can keep their things. As much as I am a fan of pair programming, I think developers should have their own desks away from the common bull pen they can retreat to from time to time.
My conclusion is that pair programming does work, but it requires some care. First, developers have to respect each other and be willing to compromise. Also, pair programming 100% of the time doesn't work. Sometimes when facing a design problem it helps to go off and work separately, then get back together and discuss later. Also, pair programming - constantly communicating, asking questions, explaining and justifying one's own ideas throughout the day - is very demanding. After a period of time, burnout can occur. When that happens, I think it's a good idea for people to be able to work on something alone for a while.

Thursday, November 15, 2007

Demystefying Stubs, Fakes, and Mock Objects

This blog entry is my attempt to explain the different kinds test-only objects that can be substituted for the real thing in automated tests: Stubs, Fakes, and Mocks. It can be easy to get confused, so I hope this entry helps!

Stubs, fakes, and mocks are all objects that replace the objects that would normally be used in the actual application. What purpose do they serve? First, using such test-only objects promotes good design. Writing code so that classes have dependencies on interfaces and abstract classes which can be implemented either by the full-fledged production code or by a test-only class of some kind helps to reduce the overall coupling in an application. Test-only objects frequently come up when an application is accessing another system: Making sure that system is available and produces the same response to a given input all the time may be a problem (consider a service that provides the current price for stocks for example). Also, simply configuring something like a relational database or a web server so that tests can run against it can be an issue. More generally, using a test-only object makes it easier to set up the initial conditions for tests. Such objects can make it simpler to provide the initial state that the function under test will respond to. The time it takes for tests to run can also be an factor; using simple test-only objects can speed up the time it takes for tests to run substantially. Finally, in some cases one may want to develop some logic that depends on classes that haven't been written yet or that you don't have access to. Say for example another team is working on that functionality and you don't have access to their code. In such cases, you can write code against objects that, for the purposes of testing your own logic, implement the interfaces you expect to see in the yet-to-be-written API. The core idea behind the use of test-only objects is that we often want to write tests for the application logic we're currently working on without having to worry about th behaviour of some of the external code our logic is using.

Let's consider an example. I once wrote a program that allowed a user to schedule polling of different sensors - e.g. read sensor 'A' once an hour, sensor 'B' every minute, and sensor 'C' every second. I wanted to test my scheduling logic, but I wanted to make it independent from the actual sensors: I didn't want to have my application connected to real sensors just to make the my unit tests run. Also the scheduling system of course obtained the computer's system time to see whether it was time to read a given sensor. I also wanted my tests to be independent of the actual system time: What I wanted was to set the time in my test and make sure my program responded by reading the correct sensors - and avoided reading the wrong ones. I didn't want to have to set the actual computer's system time inside of my tests! So, for both of these cases, the business of actually connecting to the sensors and setting and reading the time, I created special test-only objects to stand-in for the ones that would be used in the actual application.

If you're interested in ways to instrument your code, and the trade-offs involved, so that you can substitute these kinds of test-only objects Jeff Langr's Don't Mock Me article is a great reference. I should note that his use of the word "mock" is more generic than the one that's often used. He means "mock" in the general sense of any kind of test object. As we'll see a bit further down, mock objects often have a specific meaning that's different from stubs and fakes.

Now, on to Stubs, Fakes, and Mocks.

Stub: A stub is an object that always does the same thing. It's very simple and very dumb. In our example above of polling sensors, the system time seems like a useful entity to replace with a stub. Let's suppose our scheduling code was in a class called Scheduler. This class might have a method called getSystemTime(). For the purpose of testing, we might create a TestingScheduler class that extends Scheduler and overrides the getSystemTime() method. Now you can set the system time in the constructor of this test-specific class, e.g:

public class TestingScheduler extends Scheduler {
public TestingScheduler(int timeInMillisForTest) {
this.timeInMillisForTest = timeInMillisForTest;
}

public int getSystemTime() {
return timeInMillisForTest;
}
}

When a TestingShcheduler object is used as part of a test, the rest of the Scheduler logic works normally, but it's now getting the time that's been set in the test instead of the actual system time.

Fake: A fake is a more sophisticated kind of test object. The idea is that the object actually displays some real behaviour, yet in some essential ways it is not the real thing. Fakes can be used to speed up the time it takes tests to run and/or to simplify configuration. For example, a project I am currently working on is using Oracle's Toplink as an object-relational mapper (ORM). This allows data in Java objects to be transparently saved to and retrieved from a relational database. To make tests that use this framework run faster, a much simplified memory-only implementation of Toplink's interfaces was implemented. This version doesn't know about transactions and doesn't actually persist data, but it works well enough to allow many of our tests to run against it - and since the actual Oracle database isn't involved, the tests run over an order of magnitude faster. Going back to the scheduler example, we developed a piece of software that could behave as though it was a real sensor. That way we were able to run a variety of fairly complicated tests to make sure our application could communicate with sensors correctly without actually having to hook up the tests to a real sensor. Any time you write code that simulates an external service - some sensors, a Web server, or what have you, you're creating a fake.

You can find a simple example of a fake in the TestNode class in my loop finder example. The TestNode implements the Node interface for the purposes of the unit tests. Classes that are actually part of the application have their own, more complex implementation, of this interface - but we're not interested in testing their implementation of the Node interface here. This allows us to write tests that can run in isolation from the rest of the application. From the perspective of the overall design, this approach helps us to reduce coupling between classes. The LoopFinder class only depends on the Node interface rather than on any specific implementation. That's an example of how making code easier to test concomitantly improves the design.

Mock: Mock objects can be the most confusing to understand. First of all, one can argue that the two types of test classes mentioned above are mocks. After all, they both "mock out" or "simulate" a real class. In fact mocks are a certain kind of stub or fake. However, the additional feature mock objects offer on top of acting as simple stubs or fakes is that they provide a flexible way to specify more directly how your function under test should actually operate. In this sense they also act as a kind of recording device: They keep track of which of the mock object's methods are called, with what kind of parameters, and how many times. If your function under test fails to exercise the mock as specified in the test, the test fails. That's why developing using mock objects is often called "interaction testing." You're not only writing a test which confirms that state after a given method call matches the expected values; you're also specifying how the objects in the function under test, which of course have been replaced with mocks, ought be exercised within a given test.

To sum up: A mock object framework can make sure a) that the method under test, when executed, will in fact call certain functions on the mock object (or objects) it interacts with and b) that the method under test will react in an appropriate way to whatever the mock objects do - this second part is not any different from what stubs and fakes offer.

We've already seen how stubs and fakes can be used, so let's create a hand-rolled example of the kind of thing that mock object frameworks can help with. Let's go back to the scheduler we've already talked about. Let's say the scheduler processes a queue of ScheduledItem objects (ScheduledItem might be an interface) . If it's time to run one of these items, the scheduler calls the item's execute method. In our test, we can create a queue of mock items such that only one of them is supposed to be executed. A simple way of implementing this mock item might look something like this:

public interface ScheduledItem {
public void execute();
public int getNextExecutionTime();
}

public class MockScheduledItem implements ScheduledItem {
private boolean wasExecuted;
private int nextRun;

public MockScheduledItem(int nextRun) {
this.nextRun = nextRun;
}

public void execute() {
wasExecuted = true;
}

public int getNextExecutionTime() {
return nextRun;
}

public boolean getWasExecuted() {
return wasExecuted;
}
}

Our test might look something like this:
public void testScheduler_MakeSureTheRightItemIsExecuted() {
//setup
MockScheduledItem shouldRun = new MockScheduledItem(1000)
MockScheduledItem shouldNotRun = new MockScheduledItem(2000)
Scheduler scheduler = new TestingScheduler(1100);
scheduler.add(shouldNotRun);
sheduler.add(shouldRun);

//execute
scheduler.processQueue();

//verify
assertTrue(shouldRun.getWasExecuted());
assertFalse(shouldNotRun.getWasExecuted());
}
That's a really simple, hand-rolled, example of a mock object. The test just makes sure that the processQueue method ran the execute method on the first item, but not for the second one. Of course this example is very simple. We could make it a little fancier by counting the number of times the execute method is called and make sure it's only called once during the test. Then we could start to implement functionality that makes sure functions belonging to a given mock object are called with particular arguments, in a particular order, etc. Mock object frameworks support this kind of functionality out of the box. You can take any class in your application and create a mock version of that class to be used as part of a test. There are a bunch of mocking frameworks for many different programming languages.

Before you dive in, consider my word of caution: In the great spectrum between pure black-box and pure white-box testing, using mock objects is about as "white-box" as it gets. You're saying things along the lines of "I want to make sure that when I call function X on object A (the function and object under test), that functions Y on object B and function Z on object C will be called in that order, with certain specific arguments." When the test you write is making sure that something *happened* as a result of your test, it tends to be easier to understand what the test is trying to do. On the other hand, if your test is just making sure that some functions were called, what does that really mean? Potentially very little. Also, because your mock objects are basically fakes or stubs, you are not guaranteed that the behaviour of the actual objects that are being mocked out will be consistent with the mocks. In other words, you can create a mock version of an object that adds one to its argument whereas the real function subtracts one. If you change the behaviour of a given function that is being mocked in a test somewhere, you have to be careful to make sure to adjust the mock accordingly. If you don't, you'll wind up with passing tests, but you may still have introduced a bug into the application. This kind of problem tends to become more likely as the sophistication of the fake implementation increases - and pure fakes also suffer from the same weakness. I do think that creating mock tests where the specified interactions become complicated and the mock itself is a sophisticated fake that can respond to a wide variety of interactions compounds the likeliness of running into this kind of problem. Also, on a more basic level, simply refactoring code can be difficult with mock objects. The mock frameworks sometimes use strings to represent the mock object's methods internally, so renaming a method using a refactoring tool may not actually update the mock, and your tests would suddenly fail just because you renamed a function. Of course even a slightly more complicated refactoring, like breaking up a method into two can also cause mock objects to fail trivially, telling you that yes indeed, you actually changed some code.

As you can tell, I am not a huge fan of extensive use of mock objects in the sense of specifying interactions. I believe that such objects can indeed be useful in specific cases, but that's not how I think about writing my code. When I write a test, I try to keep it simple and concentrate on what I can expect to happen as a result of running that test, not specifically what the execution path of the function under test will look like. There are of course cases where this type of interaction testing is useful. I think the scheduler example above is a good case in point. You want your test to make sure a method is called, but thats it; you're not interested in what the real implementation of that method may do. All in all, I tend to prefer to stick with simple hand-rolled stubs, fakes, and mocks in my TDD practice. Your mileage may vary. Martin Fowler has written about the distinction betwen mocks and stubs/fakes also.

I know that when I first encountered "mock objects", I had some trouble figuring out exactly what it meant and what all the fuss was about. If you've found this blog entry because you were experiencing the same confusion, I hope it's been of some help.

Additional Links:

Monday, November 12, 2007

Even Grandmasters Get The Blues

I read an article some months ago about Vladimir Kramnik's incredible mistake in a game against the computer opponent Deep Fritz. Kramnik was at that point the undisputed world champion in chess - apparently the first person to achieve that status since Kasparov. In this game however, he made a truly astounding blunder. Deep Fritz had made a move that threatened an immediate check mate. Even someone like me who barely knows the rules of chess can see this fairly easily, but somehow Kramnik missed it. Here's a diagram of what happened:



With Deep Fritz's queen at E4 threatening to check mate at H7 with the very next move, Kramnik, instead of defending, blithely moved his queen from A7 to E3. In an instant, the game was over. Here is an account of the events:

Kramnik played the move 34...Qe3 calmly, stood up, picked up his cup and was about to leave the stage to go to his rest room. At least one audio commentator also noticed nothing, while Fritz operator Mathias Feist kept glancing from the board to the screen and back, hardly able to believe that he had input the correct move. Fritz was displaying mate in one, and when Mathias executed it on the board Kramnik briefly grasped his forehead, took a seat to sign the score sheet and left for the press conference, which he dutifully attended.

It's a fascinating thing. How could Kramnik have made such an error? There could be lots of explanations. He might have been ill or in a bad mood. He might have been experiencing a lot of stress because his opponent was a computer: It must be psychologically difficult to cope with the idea of an opponent with perfect memory who will never make a reading mistake or get tired; an opponent who can not be intimidated with an agressive move, or confused, or tricked. Never the less, the fact is that this mistake really did happen.

For me this might be an interesting example of a phenomenon I'll call the "expert's blunder," which can apply to anything, and in particular, to software development. A beginner in any field has a very small number of things he or she can keep track of. As one learns more and more about a given area however, one has to think of an ever expanding tree of concepts and ideas. Anything you do as an expert is heavier and more difficult. The more you know, the more tools are available to you, the more effort is required to choose which tool or approach to use next. It becomes easier to overlook something completely simple that a beginner would spot right away. As Kramnik was playing this ill-fated game, I think he wasn't really looking at the current state of the board, but rather at his own mental picture, a cloud of variations and possibilities. He might have got out of sync with his place in the actual game.

I've noticed this kind of thing happening to me when I've found myself implementing a more complicated piece of code than was necessary. Here's a case I blogged about some time ago for example: My original blog entry, followed by my realization that my code was over-designed. Here I too was getting ahead of myself, thinking of classes and subclasses and interfaces instead of just solving the problem in the most straightforward way. I think this kind of mistake is a good example of why TDD (test-driven development) and pair programming are useful tools in software development. The more we focus on the simple step-by-step design process of TDD and the more we constantly subject our code to critique as we do in pair programming, the less likely we are to develop bloated and over-designed code - however well-intentioned we might have been in writing it in the first place. I think this also promotes the idea of pairing a seasoned veteran with a recent grad: The value flows in both directions, not just from the expert to the novice. Not only beginners can write bad code. By virtue of their experience, people with more knowledge can be just as susceptible. At the very least, it's safe to say that one should always pause and just "look at the board" every so often.

Saturday, November 10, 2007

Fun Introductory Software Projects

I've taught some evening/weekend courses in software development over the years. Usually I've taught adults, but sometimes I've had the chance to work with some talented kids too - from about 12 to 17 years of age. I have to admit, working with the kids was great. Someone recently suggested that I write up some of the projects that were done during these courses. I've always had students come up with their own ideas: I'm a big believer that if you come up with your own idea for something to work on, you'll be genuinely motivated. Therefore, I really suggest coming up with your own concept if you're looking for a project to work on... In case you're looking for some inspiration though, here are some of the types of things people in my courses have done. Have fun!

  • A version of the famous "snake" video game.
  • A really neat original 80's style arcade game which involved shooting a mothership and picking up space junk floating around on the screen to augment one's own spaceship. Let me tell you, it's quite an experience asking about how the swarming bullets were done and getting the reply "Oh, it's a very simple algorithm, but the swarming is emergent behaviour" from a 13/14 year old.
  • A simple contact manager which you can enter people's information and pictures into. This program used a file to store its data, so all of the storage/retrieval routines had to be written from scratch as opposed to using a database program, which proved quite instructive to the developer.
  • A very impressive network multi-player game along the lines of Warcraft (albeit much simpler). The programmer did good work with path-finding and making sure all players were in sync. The players were happy faces of different colors which became sad faces as you attacked them.
  • An online pizza-ordering application. I recall some good discussions about the user interface and whether it was ok to allow people to order who didn't want to enter in their credit-card information.
  • A chat program along the lines of ICQ/Instant Messenger
  • A "Towers of Hanoi" program in which the user could select the number of disks. The user could play the game him/herself or let the computer solve the puzzle. A challenging bonus (which in this case was not implemented) would be to incorporate a "hint for next move" feature.

Monday, October 08, 2007

Estimating Software Development

This blog entry was prompted by some thoughts I wanted to express about what it meant to make estimates for software projects in an agile way, and also by Steve Mcconnell's recent blog about Building a Fort.

The traditional way of estimating software development uses what I'd call a bottom-up approach. You discuss the requirements for the project in great detail with the customer, breaking the project into small pieces. You then estimate how long each of the pieces will take to develop, and then you add everything up to get the estimate for the entire project. In general, I don't think this is really a good idea for software projects. First of all, requirements really do change as time goes on, and an up-front estimate doesn't take that into account. Second of all, communication about requirements using words is a tricky business. I've witnessed, and engaged in, so many discussions where people seemed to agree when in fact they had very different visions. The converse is also often the case: It seems as if people completely disagree, and keep arguing about various points, when in fact they actually have a lot in common. They just don't realize it. So communication is really tough, especially when it comes to software projects where there is always so much room for interpretation. It's very likely that the breakdown of requirements won't be as accurate as it may seem at first. Finally, estimating the amount of time needed for a whole bunch of small requirements without having actually done any real work yet is dangerous too. Will something take two hours, two days, or two weeks? It's hard to say, but it's easy to give something a value and then to assume that it must be accurate. After all, the process to get that value was time consuming and very detailed!

In agile software development we try to be more realistic. I see it as as combining a top-down approach with a less detailed bottom-up strategy. The top-down way of estimating is simple. Try to compare apples with apples. If you (or your company) are asked to develop something, base the estimate on a similar project you've done fairly recently with comparable technology. That will give you a wonderful first-order estimate without breaking down the requirements at all. Next comes the agile version of bottom-up estimating. This involves breaking requirements down into a list that the customer prioritizes. For a smaller project, these chunks may be about a week's worth of work - the maximum size of most XP stories. For larger projects, they each might be about a month of work - the size of a Scrum sprint. The point is to provide an estimate that is based more on the sense of the scope and difficulty of each broad set of requirements without analyzing them to death. Once you've estimated each of the items on this prioritized list - it's called the Product Backlog in Scrum - you can compare the total estimate you arrive at with the earlier estimate using the top-down approach. Combining these two techniques should produce the kind of estimate that can be used to bid on or evaluate whether to go ahead with a project.

The important thing is that after each month-long iteration, or even earlier depending on the size of the project, you'll be able to compare the original estimate with the actual development time. After the first month, would the customer agree that you've completed what they expected to see? If you fall short, how much more time does it wind up taking for the the customers to say, 'yes, this is what we thought we would see for the first iteration'? That ratio is called the velocity. Say you took 25 days to complete what was estimated to take 20 days. That means your velocity is 20/25 = 0.8, or 80%. You can use the information to plan the next iteration, and so on as the project goes ahead.

There are a couple of nice things about this way of doing things. The first one is that you don't have to argue so much with the customer about what a requirement means. Say you estimated 20 days for feature X. After 20 days you present your software to the client. The client says, 'no, this isn't complete. You missed this and that'. I've seen my fair share of projects go completely off the rails when arguments like that break out, but with this agile approach, it doesn't matter so much. You discuss the missing pieces with the customer. Let's say, using our earlier numbers, that you take an additional 5 days to get everything done in a way the customer is happy with. That means, in a certain sense, that your velocity of 80% represents the difference between your original understanding of the requirements and the customer's expectations. Instead of creating arguments and finger-pointing, you can simply continue to monitor velocity and use it in your planning and budgeting. That's the other good thing - it's an adaptive approach. The customer is allowed to update the product backlog after each iteration - changing priorities, adding some new requirements or perhaps removing some that no longer make sense or just happened to emerge naturally as other features were being developed. The developers can then review and update their estimates and provide an ever more accurate planning picture. It's not very hard to track the velocity as a curve that makes the progress being made very clear to all parties involved.

Monday, September 17, 2007

Palisaro Wall Tutorial (flash using haXe)

I've written a tutorial for a flash game called Palisaro that was developed by a friend of mine, Eda-qa. I have a feeling flash perhaps wasn't designed for interactive applications as my friend tells me tweaking his code for performance was one of the most time-consuming aspects of getting the game working. This tutorial is about recognizing walls around areas, and I think it's interesting because the solution may be a bit counter-intuitive. When we were originally discussing it by e-mail, I was surprised the algorithm didn't focus on the wall elements themselves, but maybe that's just me. In any case, if you're interested in developing an application in flash, you may find this tutorial helpful. I found haXe to be a very nice language to look at and it definitely comes across as a language of choice for anyone with a C++/Java background who wants to write flash applications.

Wednesday, August 22, 2007

How to Hire (Programmers)

I've been interviewed for software development positions many times since I started working in the so-called IT industry. Generally I've been disappointed by the kinds of questions people ask. Either they ask lots of soft questions or they ask very academic "puzzle" types of questions. The problem with the soft questions is that a good talker can come up with convincing answers even if he/she is utterly incompetent. On the other hand, the puzzle questions leave me cold. It's quite possible to be good at solving such puzzles without being a good developer, and vice versa. Often the puzzle questions, if they have anything to do with programming at all, emphasize how to optimize certain algorithms. The truth is that an over-emphasis on optimization is one of the most common problems I see. It makes code much harder to work with, often without a tangible benefit. Don't get me wrong, I'm not saying that being able to communicate with people isn't important or that one ought to hire dumb people, but I think the industry lacks a sense of the overall qualities to look for. Thankfully, Marc Andreessen, of Netscape fame, has written (in my opinion) the definitive guide to hiring software developers.

http://blog.pmarca.com/2007/06/how_to_hire_the.html

Thursday, July 12, 2007

Tinkerers, Platonists, and Communicators

I've written on this topic before but lately I've been thinking some more about the differences of perspective people have about software development and I came up with the idea that there are three pure personality types in software development: Tinkerers, Platonists, and Communicators. That's my attempt to neologize a la Martin Fowler by the way! Each type tends to focus on one aspect of development over others, and I think we all tend to fall into the trap emphasizing our own inclinations and downplaying the importance of the other perspectives. I suspect most of us have some attributes of each type but that developers can probably be identified as mostly leaning toward one of these types more than the others. Here then are my descriptions of each type!

The Tinkerer: The tinkerer is someone who generally thinks of programming as an adjunct to other things. Some examples: A stock broker might write some macros in excel to crunch some financial numbers. An engineer might need to write some drivers for a piece of hardware s/he designed. A network administrator might cook up some scripts in perl to help manage repetitive tasks or analyse network traffic. The tinkerer's sweet spot is in getting things configured, finding ad-hoc optimizations, and writing quick and dirty hacks that get the job done. The big weakness of the tinkerer is scaling software to the point where it has to be maintained on an ongoing basis, especially if multiple developers become involved. Tinkerers may not to recognize such risks and are generally comfortable with a certain amount of messiness. Tinkerers are great people to have on a team when you need to figure out how to make something work - they love finding out about all of the different options a piece of software or hardware has and can find a way to make something work even if the tools that are being used don't make it easy. I think Linus Torvalds is an archetypal tinkerer.

The Platonist: The platonist is in many ways the diametric opposite of the tinkerer. They're not interested in nitty gritty details. They enjoy working with pure and abstract ideas and they want to solve problems in the most general way possible. They tend not to be happy with solutions that are "good enough for now." Platonists are generally not inclined toward empirical or iterative approaches to developing software because they see software as something like a math problem - something to be analyzed completely and solved in its entirety. Joel Spolsky jokingly referred to the kind of people I'm talking about here as "Architecture Astronauts," which highlights their main weaknesses: Platonists can fall into the trap of spending far too much time trying to fully analyze something leading to the infamous analysis paralysis that software teams are often subject to. Platonists also tend to produce designs that are so abstract it's difficult to actually work with them. You know you're dealing with software designed by a platonist if it can theoretically do anything you want it to but it's hard to figure out how to make it print hello world. A good platonist is great to have around when you need to apply or develop a sophisticated algorithm. They enjoy most being set loose on an intellectually challenging but well defined problem. I think Peter Norvig is a platonist and of course Dijkstra was the original platonist!

The Communicator: Communicators think of software develoment more as human language designed for human consumption than as an abstract mathematical concept. They focus on expressing ideas clearly in code. Both tinkerers and platonists would tend to leave a piece of code alone once it's working properly, but to a communicator it's not good enough for code merely to be correct. It also has to be maintainable. Communicators can focus so much on the quality of their code that they might miss the point that a given situation calls for a well-defined algorithm. They also don't enjoy figuring out how to hack something just to make it work, which wether they like it or not, is often necessary. Another problem I've seen with this style of development is that communicators can become overly enthusiastic in their efforts to shape code, so instead of writing code that is concise and simple they wind up producing bloated class hierarchies and abusing design patterns. I think communicators may share this type of weakness with the platonists. Good communicators are great to have around as people who remove duplication from code and generally clean it up. They can be helpful in translating a sophisticated algorithm into code that can be readily understood and if there is some gnarly code in the system, they at least will write adaptors that encasulate that code behind a clean interface. Developers who are good communicators often work well interacting with users and customers to translate and organize requirements that may seem arbitrary and complex into code that is consistent and simple. Kent Beck comes across as a very prototypical communicator. After all refactoring, one of the main practices in XP, concerns itself with the idea of altering the structure of code without changing what it does. I think Martin Fowler is another good example of a communicator.

All three of these "personas" have some strengths and some weaknesses. Also if you find that you tend strongly toward one of these poles, it may be worthwhile to expand your consciousness a bit and to force yourself to develop in the other directions as well!

Thursday, June 28, 2007

Software Development vs. Engineering

I recently read Kelly Waters' blog entry about the 'uncommon sense' of agile development. Kelly's argument is that when you strip Scrum of its jargon, the ideas are self evident andjust common sense. So why is there so much controvery then? I think one of the problems is that software has been saddled with an analogy between software and civil engineering, e.g. contruction of bridges, buildings and such things. This way of looking at things continues to influence organizations to this day.

I believe one of the key reasons for the emphasis on up-front requirements and up-front design phases in the world of software stems from the idea that developing software is like engineering or architecture. In these areas one spends a lot of time on design, and only once the design has been completed does one actually engage in the activity of construction. If one takes this analogy to be valid, then developing a piece of software iteratively looks foolish: Would it be reasonable to design a skyscraper by first building a bungalo and using the feedback from that to build a two story house, etc. until one has a skyscraper? Of course not. I think that for the most part the approach that's currently used in bridge building and architecture makes good sense for those disciplines, but it doesn't translate nearly as well to the world of software. In my opinion, there are three primary dimensions that make software a rather different animal:

1) One can use engineering formulas in a practical way to validate a design ahead of time, say to make sure a bridge will stand up to a certain load of traffic. Software however is not subject to the contraints of physics, so the applicability of this kind of technique is limited to the fairly narrow area of algorithms. It may make sense to make sure a scheduling or path finding algorithm is correct for example, but does that kind of idea apply to a software system in general? Are there any engineering formulas that would insure that a design for accounting software will meet its users requirements?

2) Models are not nearly as useful in software development as they are in engineering. Models and simulations of bridges and buildings get pretty close to the reality of what one is trying to build without actually having to build it. One can look at a scale-model of a building or a 3-d computer simulation and get a lot of really valuable feedback. Things like ERD diagrams and various UML constructs, e.g. class diagrams, sequence diagrams, are far less kinesthetic - I'd say they're even less expressive than an engineering blueprint. Does some collection of classes and methods really represent a good solution for a given problem? Yes? No? Maybe? I've found it very difficult to anticipate from such diagrams what it will be like to actually work with the code.

3) Finally, buildings and bridges are designed to resist change. Once you've built something like that, it will remain that way until it needs to be replaced - every aspect of the effort is focused on that goal. Would anyone consider adding a few extra stories to an existing skyscraper, or adding a lane to a bridge that's already been built? On the other hand, most software systems must stand up to constant change during their life spans and those changes are often substantial. One's approach to software design should be flexible and adapt to future changes.

Monday, June 11, 2007

Little Shop Of Sudoku Horrors

I've recently run across a few examples of unorthodox Sudoku solver programs which are kind of entertaining in a way, but also frightening. Here's an example using sql, and here's another example using regular expressions. I've spent just enough time looking at these solutions to be horrified. I certainly wouldn't want to have to maintain such code, and it confirms my general opinion that sql and regular expressions are appropriate for a limited range of situations and should not be used for general programming. Of course it's kind of amusing to see that it's possible to write a sudoku solver with these technologies in the same way that this is amusing.

Additional sudoku horrors: Sudoku in cobol! Shortest sudoku solvers! These are kind of cool, but also teach the lesson that writing maintainable code isn't just about making it as concise as possible!

Reference: Peter Norvig's python code. This code is certainly clean and compact (I do take issue a bit with his constraint propagation. You can have a look at my java code example for more details). In any case, much nicer than the craziness above.

Wednesday, May 30, 2007

Standish Group Project Data

There are some interesting articles/discussions on InfoQ concerning data about project successes and failures.

Interview with Jim Johnson of Standish Group
Why Were Project Failures Up and Cost Overruns Down in 1998

Wednesday, May 23, 2007

The Way Of Testivus

I recently read The Way Of Testivus. It was so refreshingly light-hearted, yet with a depth of wisdom at its light heart. Very enjoyable indeed. I'd like to rave about it some more, but really it speaks for itself. Just read and enjoy. Less Dogma! More Karma!

Saturday, May 12, 2007

Solving Sudoku Using TDD

Thanks to some encouragement from Ravi suggesting I try my hand at a Sudoku solver, I've written up my experiences writing one using TDD. My main goal was to see if using TDD would be helpful. I thought it was. However, you can judge for yourself. I think the experience confirmed my opinion that using TDD is not a magic wand that allows algorithms to just "emerge," but on the other hand it is a useful tool to move forward in small steps and to design code to be as intention-revealing as possible.

Tuesday, April 24, 2007

TDD Is Not An Algorithm Generator!

[Update: more on this topic]

In his blog, Ravi compares Ron Jeffries working out a sudoku solver using TDD (test-driven development) with Peter Norvig's implementation.

Now, what can we learn from the two efforts? Peter does not use TDD to solve the problem, yet his solution is compact and complete, whereas Ron's TDD effort remains just a partial solution. Does that mean that TDD is inferior to design up front? I do not believe that's the point Ravi was trying to make, but perhaps he was trying to say that TDD is not an algorithm generator. In that respect, I completely agree. In fact, one of the first rules I teach my students when I am doing a TDD workshop or teaching a course is precisely that TDD is not an algorithm generator! Solving sudoku is just the kind of problem you want to find an algorithm for first, then implement that algorithm. This is not a question of up-front design. It's just common sense: It's perfectly reasonable to know conceptually how to solve sudoku puzzles before writing any code. Else how would you know whether any of the intermediate methods that you so diligently write tests for will be useful in the end?

There is a real gap between a design, which is a particular structure of code, and an algorithm, which is a description of how to solve a problem. An algorithm can be expressed in any form you choose. It can be a mathematical notation or it can be written down in plain english, or it can live in your head, but if an algorithm is required to complete a story, you should certainly know what it is before you engage in writing your production code. If you doubt what I'm saying, consider a story where I ask you to write code to encrypt and decrypt files. Would you just start hacking out methods (using TDD or otherwise)? You'd be lucky to develop a trivial cipher. You need to choose a cryptographic algorithm before you begin any programming. Once you do start with the programming, I would certainly recommend using TDD as a way of designing the code.

Let's say you're faced with a problem where you want to find the right algorithm yourself, either because you have to or just because you enjoy the challenge. Writing some code to solve a simpler part of the problem may be a good way to start. That approach worked for Peter in his solution to Sudoku - I doubt it's guaranteed to work all the time. In any case, when you're hunting for an algorithm, any code you write should be considered provisional. The main purpose is not to produce high quality maintainable code, it's just to see if getting part way toward a solution will trigger any additional insights. Again, it's once you've understood the correct algorithm that you can begin writing the production code.

I certainly think Ron's example is not a very good indication of how TDD should work. I am also disheartened that Ron doesn't clearly make the point that he is looking for an algorithm. Therefore his efforts to refactor his code during the tdd process, are far too premature. Based on Peter's solution, I'd say Ron never even gets half way toward the correct algorithm and wastes his last few articles on refactoring to objects instead of doing anything particularly useful or enlightening. When you're in an investigative mode where you don't even know how to solve your problem, that harly seems like the time to solidify the design. Even if you're doing agile development, keep in mind that for each XP story, you should be certain that you know at least informally in your own mind the steps needed to solve the problem. Trying to offer TDD as an algorithm generator is dopey and it's just going to make it easy for people not to take TDD seriously as a valid design technique.

So what is the purpose of TDD then? One goal of TDD is to reduce the need to determine ahead of time which classes and methods you're going to implement for an entire story. There's a large body of shared experience in the developer community that trying to anticipate such things tends to lead to paralysis where nothing useful gets done and/or produces bloated, over-designed code. Instead, you can develop one aspect of the story at a time, using each test to keep yourself moving forward and refactoring the design as you go along - think of TDD then as a ratchet or a belay device in climbing. One of the principles of agile development is that it's generally not a good idea to try to comprehensively understand a whole project (or a big part of it) up-front, as is often done in waterfall-ish methodologies. However, that doesn't mean you need to begin every story blindly without understanding how to solve that particular problem.

Updates: UncleBob makes a good comment on reddit: http://programming.reddit.com/info/1kth0/comments/c1lkvh; Read about my TDD Sudoku effort

Sunday, April 22, 2007

Snake-Oil Salesmen, Process Skepticism, and Why I am Not a "Post-Agilist"

The optimist thinks that this is the best of all possible worlds, and the pessimist knows it.
-- J. Robert Oppenheimer, "Bulletin of Atomic Scientists"

While I am a strong proponent of agile methods, I've witnessed a number of people I respect writing about being skeptical of agile processes and complaining about evangelists who try to sell methodology without really knowing what they're talking about. For example, Jonathan Kohl has written about Post-Agilism where he states :
Personally, I like Agile methods, but I have also seen plenty of failures. I've witnessed that merely following a process, no matter how good it is, does not guarantee success. I've also learned that no matter what process you follow, if you don't have skilled people on the team, you are going to find it hard to be successful.

Ravi Mohan has written of Agile as a religion, "with its holy books, many patriarchs, its churches, and commandments." In his blog, Ravi also frequently decries the lack of actual programming experience demonstrated by many self-professed gurus to back up their claims.

On a certain level I agree. It seems evident that any movement, as it grows beyond a certain point, becomes vulnerable to distortion, both by the well meaning but ignorant, and also by those who cynically aim to make a buck from whatever is trendy today. Agile is no exception. One thing I stress whenever I talk about agile methods is that the agile manifesto is self-referencing. The first statement (Ravi would perhaps say commandment!) of the manifesto is clear:

Individuals and interactions over processes and tools

Those "processes and tools" include the methods of Scrum and XP! If you're working on an agile project and the people on the project are trying to blindly follow some agile practice when it clearly doesn't make sense, they are violating the spirit of agile development: Ipso facto, you must use your knowledge of what's actually happening on your particular project, your team, at a particular point in time to decide what to do next. Everything else is just a guideline.

However, things are unfortunately not so simple because it is often not clear when you should abandon a given practice. Let's say you think that something isn't testable and you give up test-first development; then you decide that a given feature really requires a lot of up-front design and you give up iterations. Before long you've entirely given up all the essential aspects of what agile software development is about! Following an agile practice is therefore always a delicate balancing act. You must respond empirically to reality, but you must learn the practical value of agile practices. I've found that many of these practices are worth defending. For example, at times I have been tempted to abandon test-first development when testing became difficult, but upon deeper consideration I realized that there was something wrong with the way I was approaching my problem, and that was the real culprit that made the test-first approach problematic. By re-thinking my approach, I found a way back to test-first development and its benefits.

I guess I would like to emphasize the following point: If you are interested in agile development, you must take the time to learn the value of the practices you're using. Blindly doing anything will never do any good. However, discomfort I have with notion of "post-agilism" is that the malaise people are experiencing, that's causing them to react, is rooted in fundamental problems: Managers will read articles about agile and impose it on their teams; so-called gurus will go into a company dispensing advice without properly understanding what's going on, perhaps without really knowing anything of value in the first place; teams will get caught up in zealous behaviour and start following practices blindly; people will splinter into factions, arguing over minutiae and jealously guarding their particular version of the Truth; incompetence will have its day. Thus dogmatism will set in. The way I see it, these things are inevitable. They are inevitable in the same way that corruption is inevitable in any social or political order. As with corruption, one can only do one's best to manage such problems, to reduce them to a minimum, but one will never get rid of them entirely.

I fear that creating a new movement/idea and calling it "post-agilism" is just sweeping these basic problems under the rug. If the agile movement is a case of taking some pragmatic ideas and codifying them into a general system, or "going meta" with them, as Bob Martin has written, then "post-agilism" is a case of going "meta-meta." I can't see how this will lead to any good. Before long, there will be "post-agilist" gurus and dogma, and because of its second order nature, the attendant problems will be much harder to deal with and untangle. I hope that the agile community will get itself out of this spiral and instead will establish ways to do research and to educate people so that as much as possible software development will be about common sense and about what works instead of dogma. Things will never be perfect, but hopefully we can work toward a system that represents the best possible balance and thus minimizes the inevitable corruption that accompanies any human endeavour.

An agile development process ought to be self-correcting. If that's not working, I'm not sure how embracing "post-agilism" will help: For better and for worse, I think the agile approach represents the best of all possible worlds.

Wednesday, April 18, 2007

TDD: The Bowling Tracker

Some years ago I purchased Bob Martin's book Agile Software Development, Principles, Patterns, and Practices. At the start of the book, he brings up a TDD exercise using 10 pin bowling. At the time I thought it would be a good idea to try my hand at the exercise so I avoided reading that chapter in the book; looking at someone else's ready-made solution to a problem tends to create mental blinders so I always try to do things on my own first, then look at the solution in the back, so to speak. Also, if my solution turns out to be inferior, I can learn a lot more from thinking about the differences between the two. I can't remember any more if I ever actually did the exercise or just forgot about it. Recently however, I was thinking about developing this example for potential use in a presentation or seminar about TDD for Eamug. Here then is my implementation along with all of the tests - so feel free to have a look!

Here are my general observations about the process:
  • It took longer than I thought! I expected to spend a couple of hours, but it actually took over a day. I estimate the total development time was about 10 hours!
  • Because all of the classes except BowlingLane emerged from refactoring, none of the classes used by BowlingLane had unit tests for their methods. Since none of those methods was public - they were only ever called by BowlingLane which provides the public API, I suppose that's ok. I think my rule is that if a method of a class comes about as a result of refactoring, then there should already be a test for that functionality at a higher level - so one doesn't need to worry about writing a specific test for that method.
  • Despite the simplicity of the problem, I was surprised to find quite a few gotchas that broke my tests. I definitely think that using TDD to generate this functionality both produced better code and saved time over trying to hack the entire thing out.
PS: Coincidentally Curtis Schofield made a link available to Bob Martin and Robert Koss' original bowling TDD example so feel free to have a look their implementation as well. They focus on just getting the scoring right, so they don't distinguish between an unplayed frame and a scratch where both throws were gutter balls; they also don't seem to handle displaying an empty cumulative score if the player makes a strike or a spare but hasn't made the subsequent throws required to settle the score for the current frame. I suppose that logic could go into display-specific code they didn't bother writing since their emphasis was just on the scoring itself. I might try their approach of just having an array of ints of size 21, which would make the scoring code a lot simpler. Then I'd have to put the smarts around displaying frames properly and displaying an empty cumulative score when appropriate into code that is specific to the display. The nice thing is that now that I have my external API I could in principle completely re-write the back end and as long as I my tests passed, I'd be all right.

PPS: For bonus marks, I just had a look over my code and I'm pretty sure there is at least one bug! Can you find it? :) Sadly even TDD doesn't remove the possibility of bugs, but I can think of a test that would expose it!

Tuesday, April 17, 2007

EAMUG #2!

We had another presentation for Eamug this evening! This time it was about TDD (Test-Driven Development) and Curtis Schofield was the presenter. He did an interesting and decidedly manic presentation and had a very fun presenting style. I think he had a slide at one point that said "Current development practices are teh suck!" or something along those lines, and his presentation was interspersed with random photos of pineapples among other things. It made me feel like I am too didactic when I do presentations - I'm so intent on making sure I am expressing myself clearly that I sometimes forget to just enjoy and have fun and that's a good lesson for me in any future presentations I do. We'll see if I can actually learn that lesson! The only problem I had with the presentation is that the code he was demonstrating - writing a templating engine in ruby - was too sophisticated for a short presentation. Not only was everything in Ruby which was the least well-known language among the attendees but it was ruby on steroids, full of crazy dynamic reflection meta-programming stuff. I had a lot of trouble following what was going on. On the other hand, I learned about some features of Ruby that I didn't know about, so I guess it wasn't entirely a bad thing. It's always tough when you're trying to use an example from a real project rather than a toy project - conveying the context becomes difficult. On the other hand toy projects are by definition not representative of real world issues. I think introducing TDD in a one hour presentation still requires a toy project though because otherwise it's just too difficult to get everything across clearly. In any case, there was a lively discussion afterward and generally a good feeling in the room and I was really happy to be there.

Monday, April 02, 2007

Zen And The Art of Software Development

I don't know who deserves the credit for this thought, but a few years ago I read (or someone might have said to me) that no matter how complex an application becomes, it must nevertheless start with a single line of code. Every application, no matter how sophisticated it may be, starts off as a simple application. I think that's a rather deep statement and worth thinking about. Here's my contribution: If you look a random piece of code, say about 10 lines, from *any* application, what you see will look much the same. There may be some kind of looping construct, or assignment, or function calls, or some conditional logic. That's true whether you're looking at a Web app, or video game code, or an operating system kernel! That's why I believe so much in the power of simplicity and of abstraction. No matter how devilishly complex or clever your algorithm may need to be, it can be abstracted away into some kind of function call. As for the code around that algorithm, it's probably going to be good old prosaic application logic. That's why using TDD as a design tool is so valuable. It helps to tackle complexity by thinking about interfaces, not gnarly implementation details. It pushes you to organize your code so it is as orthogonal as possible - so that modules of code can work independently and any combination of ways in which that code is executed will still work properly. If your code is like that, you don't *need* to test every possible combination - which is impossible anyway. You can count on things working because you've removed the duplication from your code. That's my rant for today, thanks for reading! :)

Saturday, March 31, 2007

Renovation Cost Overruns

I was listening to CBC radio today and they were talking about renovations. It turns out that the average cost of a renovation in Canada is about $5000 and the average cost overrun is 88%! I don't know about other people out there, but I tend to be somewhat ashamed of the poor track record software projects have - they always seem to be late and over budget and more buggy than one would like. It's kind of comforting to note that ours is not the only industry where such things happen. If I were hiring someone to do a renovation, I'd definitely implement the plan as a Scrum project, getting fresh estimates on a regular basis and tracking progress against the original estimate. I'd also have multiple "releases" i.e. I'd make sure one job was competed in its entirety before I agreed to go ahead with the next one.

Monday, March 26, 2007

Costing A Project Using Scrum and XP

At the last EAMUG meeting Mike Boroditsky came up with some good questions about doing an initial costing of a project. Let's say a potential customer comes along and asks you how much she would have to pay for a piece of software she has in mind. How does one go about providing such an estimate in an agile context? If agile software is about iterations and feedback, is there any way to give a customer an up-font estimate? The idea in XP is that every project balances 4 principal variables: Time, Cost, Scope, and Quality. XP fixes the time, cost, and quality. Therefore only scope can be varied during the project. Based on an initial estimate, the developer will commit to a particular schedule, but both the developer and customer must be flexible with respect to scope - the nature of the work that ends up being done. See here for a nice elaboration. I think the reality in agile projects is that the time dimension can also stretch as long as the customer is involved in managing the schedule.

Here's how it might work in context of Scrum/XP: The project starts up with one (or perhaps several) initial meetings. In these meetings the developer and the customer get together and build up a product backlog. The product backlog is essentially a list of features sorted in order of descending priority. During this process, the developer and customer work together to establish a shared understanding of what the customer is trying to accomplish. Once an initial product backlog has been created, the developer goes over each item in the backlog and provides an estimate. Since each item in the product backlog may expand into multiple stories when it's tackled as part of a sprint, such estimates as not as reliable as etimates for stories. Nevertheless, one should be able to generate reasonable estimates based on how long similar features took on past projects and on a sense of the complexity of given items. Once these estimates are added up, one can tally them up and provide a total time estimate for the
customer. These estimates will be tested against reality in the very first iteration; the burndown chart will make progress clearly visible to all. If progress is totally off-pace from the initial estimate, then that triggers an immediate signal that something is wrong.

How about fudge factors? I've heard people say they expect some estimates to be too high and others to be too low so that the average should work out to be about right. My experience suggests that estimates are far more likely to be too low than too high. Therefore it may be a good idea to apply a fudge factor to the project. In an agile context where everything is out in the open, it would be important to share this information with the customer. The developer may say to the customer, "the estimates in the product backlog suggest this project will take 3 months to develop but it's very easy for such estimates to be off by as much as a factor of 2. Let's budget for up to 6 months." Note however that the estimates are not being inflated - the budget is being adjusted as a matter of risk management but the estimates are not being changed at all.

Watching the burndown chart (which is a visual representation of velocity), one can quickly develop a sense of whether the project is on track. After each iteration, the customer can re-prioritize, add, and remove items. All changes are put back into the product backlog and the developer re-estimates all the items after every iteration. The "red line" allows the customer to clearly see what low priority items are falling out of scope. In an agile environment, the customer is not expected to be able to come up with perfect requirements up front. The customer is allowed to learn more about what she wants as the project goes on and as she is able to interact with the actual working application. The developer is also not expected to understand the requirements perfectly. It is assumed that the developer will misunderstand some requirements such that the application will need to be refined. It's a process of give and take - but again, the overall success of each iteration is assessed by the customer.

What about quality of estimates? The more familiar a developer is with a given type of requirement - the domain as well as the technology - and with a particular client, the better the estimate. If you've worked with a client before on a similar project, your estimates are going to be much more accurate. The higher the estimate, the more likely it is to be innacurate. If you think something will take 1 day, it may take a bit longer. If you think something will take a month, it becomes difficult to understand where that number is coming from. Therefore I suggest that the largest estimate for any given item on a product backlog be one week. If you think something will take more than a week, then you should find a way to break that requirement up into smaller pieces. It's important to realize anything that will require research is highly uncertain and should be approached with care.

I think the value of this approach is that it lets the customer see progress very clearly. After every iteration, the customer gets new features as part of a working application. Also, if the customer is unhappy with the progress being made, every iteration is an opportunity to change things, or in the extreme case to cancel the project before it becomes too expensive. Importantly. the developer does have the tools to provide an up-front estimate of time and money for a project.

EAMUG

I've started an agile group here in Edmonton, EAMUG - or Edmonton Agile Methods Users Group! So far I've met some good people and I presented at our first meeting, which I think went pretty well. Rod Potter, who works at IBM, made a fantastic room available for us at their innovation centre. I hope we'll have more good meetings in the future!

Thursday, March 22, 2007

Orcs And Elves

From John Carmack's blog:
O&E development was started high-end first with the low-end versions done afterwards. I should have known better (Anna was certainly suspicious), because it is always easier to add flashy features without introducing any negatives than it is to chop things out without damaging the core value of a game.
I think this is true of all development, not just games, and it's why adding a lot of flashy stuff up-front is a bad idea. Unfortunately, users often are much more impressed with the bells and whistles than with core functionality. Striking the right balance is one of the hardest things about software development. Agile approaches emphasize a vertical style of development - get it working end to end, then add more advanced capabilities, but it's often hard to do that with users often demanding cosmetic features that are not necessarily easy to implement and impose a debt load for ongoing development.

Wednesday, March 07, 2007

What Is a Successful Project?

What Is a Successful Project?

A few years ago I would have promptly answered that a successful project is on time, on budget, and does what the customer intended it to do. Over the course of the last few years though, I've learned that things aren't quite so simple. Developing software is a human activity and it's taken me a while to figure out that 'success' is a profoundly subjective notion.

Let's say youre a painter and you've been hired to paint a house. You go in, do the job, and when the customers look at what you've done, all you hear are complaints: They can see a few rivulets of paint near the ceiling, the paint job is not perfectly even everywhere, etc... The fact is that there's no such thing a s a perfect paint job so it's up to the customer and the painter to come to an understanding of what's reasonable and what's not. Trying to settle the matter confrontationally is unpleasant for both sides and should only be a measure of last resort.

The same is true in software, only more so. The degree of sophistication to which any given requirement can be implemented can vary enormously and it's important to come to a general understanding with a customer about what is expected and how much time and money will be involved. As a simple example, let's consider Web form validation. A simple way to validate data entered on a Web site is to have the application issue error messages that are displayed when a form is submitted. However, in addition to this basic type of validation, the customer may want validation to occur 'live' as the user is filling in fields on a form - in practice this rquires either client-side validation with Javascript or asynchronous server calls using 'ajax'. The customer may also want automatic filtering on fields to prevent the user from entering incorrect data - a typical case in point is filtering provinces/states once a country has been selected, and then additionally filtering cities once a province/state has been selected. Depending on what the customer really wants, a story may simply state the validation rules for a given form or it may describe these kinds of things in detail. In the first case, XP would indicate the simplest approach applies and the customer would have to draft a subsequent story to enhance validation. In the second case, fancy validation would have to be considered a motherhood story - something that is an essential part of the application right from the start and cuts across all stories (other examples of typical motherhood stories include Security, Performance, Failover, etc.). Thus, the Fancy Validation motherhood story would be an aspect of development for every form and support for it would have to be built into the code framework fairly early on.

Every development activity can be approached horizonally or vertically. Implementing fancy validation right away across the board would be an example of a horizontal approach. The vertical approach would suggest getting substantial parts of the application working satisfactorily before contemplating such niceties. Agile development tends to emphasize a vertical (get it working end-to-end first) approach, but it's essential to pay attention to the customer. Perhaps the customer can be convinced to alter priorities; on the other hand, the customer may know exactly what he or she wants and having raised the matter and given appropriate feedback with respect to risk and cost, etc, it would be the developer's job to carry out the customer's instructions.

In the previous paragraph I was trying to describe the kind of situation where the developer may think that everything is ok but the customer isn't happy. It's even possible the customer is being unreasonable but the bottom line is that a project where the customer is not happy cannot be a success, even if it appears to meet the stated requirements and is also on time and on budget. Now let's suppose the developer and the customer meet to discuss a new project. The developer provides an estimate of 6 months. 2 months into the project, the developer shows the customer the burndown chart which clearly indicates, given the progress made so far, that the project is much more likely to take about a year. There could be many reasons for this to happen. Perhaps as per the earlier discussion, the customer wants features to be implemented in a more sophisticated way than anticipated. In any case, the customer decides that despite the delay, the project is worth continuing. At the end of a year the project is delivered to, and accepted by, the customer. On one hand this project has exceeded its initial estimate by 100%! On the other hand the customer made the decision to continue and is satisfied with the result. It's probably reasonable to suggest that this project is a success, in spite of the budget overrun.

While being on time and on budget is important, and it's also important to satisfy the stated requirements for a project, the reality of software development is that constant communication is essential. Ultimately only the customer and the developer can together decide whether a project should be deemed a success or a failure. There doesn't exist any infallible objective measure of success. If you ignore customer expectations, you may end up delivering something on time and on budget that the users aren't happy with, and then you'll spend a huge amount of time arguing over whose fault it all was - was development not getting the requirements right or was the customer not being specific enough: The kind of confrtonational quagmire one often runs into in waterfall-ish projects. Agile software development provides the tools to resolve these problems, but both the customer and developer sides need to put in the work to strike the right balance between "lots of cool bells and whistles" and "a lean, focused, no-frills piece of software that does what it's supposed to do." It's also a good idea to always think about the difference between the horizontal and vertical styles of development. Some features are best implemented horizontally and others vertically. In an agile development context such decisions ought to considered frequently and reviewed often. I hope this helps you to make your next project a success - whatever that means!