Sunday, July 30, 2006

Ravi's Post About Boring Enterprise Software

I really enjoy reading Ravi Mohan's blog. Recently I read an entertaining post about enterprise software development, and how truly boring it is, in response to a blog posted originally by Martin Fowler. Here it is, "But Martin, Enterprise Software Is Boring!" Hehe. :)

In reading Ravi's post, I thought about a few things, like what makes software development fun and interesting. Ultimately, I came to the conclusion that software has a number of dimensions to it. The first cool thing about software is that you get to build something incredibly interactive. Software has a magical quality to it. It's almost mystical. Most things in life, you can kind of see how they work; they're physical. But on a computer, you just click on some keys for a while and all sorts of things may appear on the screen. You can make a video game, a business application, a compiler. It's amazing! The first computer I remember using was the Commodore 64. I learned the "poke" command and would poke in random numbers. Sometimes the screen would change color; sometimes nothing would happen; sometimes the system would just go wonky. The amount of technology the goes into making that sort of thing possible is truly vertiginous - from the design itself to physical manufacturing of all of the circuits and I/O to the logic of the operating system and device drivers, and finally to the development environment and compiler or interpreter. A few hundred dollars worth of hardware and software that anyone can pick up at a local electronic outlet represents the penultimate technological achievement of our age. They're so ubiquitous that we tend to forget how extraordinary computers really are. When we program, we have a kind of deep connection to this incredible phenomenon.

I think there are different personalities in the software dev world. Some dimensions of software development are only barely visible to the end user. In XP, when making a new piece of software they are often labeled "motherhood stories." Performance is one of them, perhaps the most common. In business software, often called enterprise software, it is often easier to spend 10, 20, or 50k on improved hardware rather than optimize memory or cpu. From a user's point of view, the software either performs adequately or it doesn't. How one gets there doesn't matter much to the user. Some developers really enjoy the challenge of finding creative solutions to such problems, and reasonably often, simply buying hardware won't cut it. Buying hardware is the brute force solution. It's how Alexander The Great "untied" the Gordian Knot and went on to conquer most of the known world at the time. When it works, it works. However, in the world of operating system design, compilers, video games, and embedded systems, it is often not a reasonable solution. Here, the cleverness of systems and framework programmers comes into full force. The user requirement is very simple: Make it work with this much memory and this much cpu. The hacker then goes off and happily works away behind the scenes to make that requirement a reality. It can be an enormous amount of work that the end user never fully appreciates. Just as none of use truly appreciates the wonder of being able to flick and switch and voila, the lights are on, the TV is on, we can check our e-mail. For the every day business programmer, most such techniques are not only un-necessary, they would probably be counterproductive; using them would lead to a less maintainable system.

Another aspect that some developers are attracted to in such "hardcore" systems is novelty. I suspect that developing one's first compiler must be very interesting. I myself only ever touched the surface of compiler-writing in a University course, but I can see how the sophistication of optimization can become tantalizing. However, I think thay novelty is something that never lasts, no matter what domain one is working with. I once worked with a very smart software developer who had been in the game for 20 years. He had his own company and had written numerous compilers and interpreters. For him it was routine, and he never wanted to do it again. Ultimately, everyone must make that decision in life. If you're only interested in novelty and the excitement of new problems, the research field is probably the best place to go. Even there, you will encounter the tedium of having to publish papers and cite references. If you're a professional software developer, one way or the other you will spend a certain amount of time learning new things, but more time building software to a customer's specification. If all you do is develop operating systems, or compilers, or quantum simulation technologies, whatever it is, soon enough you will have a standard set of tools and your main focus will be to figure out how the building blocks and tools you have fit into what your customer wants. In that respect, I think all professional software developers are in the same boat. Our main job is not to tackle a sexy new problem every day; it's to understand what are customers/users want and to provide a quality product in a timely fashion.

For me, the interesting thing about software development has to do with understanding the business needs of the customer. One ability I've had to hone is the capacity to understand a customer's business on the fly. I've worked in many different areas, and within a few months of starting a project I've had to get to the point where I could probably get hired to do my customers' job, usually at an entry level I'm sure - Still, being able to understand the fundamentals of various businesses, the motivations, the potential efficiencies, that's a real skill, as valid in its own right as being able to optimize C code to operate in under an mb of ram. Another area is modelling.

Being able to model a piece of software so it's maintainable, so new features fit in nicely, while dealing with the fact that many business rules are quite arbitrary, is a difficult skill. It means walking a fine line between generaling code to avoid duplication while avoiding over-design that leads to crufty frameworks that won't accomodate tomorrow's strange and inconsistent variations. More "technical" software projects are often far neater and more orthogonal, and therefore more amenable to a general analysis. As an analogy, consider how some differential equations can be solved analytically, but most can't. There is a neatness of solving somthing completely, the the messy reality of the world is that you come up with partial solutions that suit you in practice. Anyway, I can see how a tendency to construct a complex mental model of the whole system right up front can be a good thing when developing a compiler, while it actually might be a very nasty trait when developing certain business applications.

I guess my point is that people tend to vary quite a bit in the problem domains they're interested in or are good at solving. We tend to assume that whatever our particular area of interest or expertise is, that's the really hard thing to do. The truth is that sofware is a difficult thing to master for a reason, precisely because there are so many dimensions, and I'm sure I've only scratched the surface in this post!

Sunday, July 23, 2006

Software Testing

These are some notes from an evening course I've taught at the
University Of Calgary in Continuing Education for a few years.
Eventually I hope to organize them better and merge them into
something more whole and coherent, but for now here they are so
I won't forget where I put them!


Unit Testing and Refactoring
============================

The topic of this class is TDD, or Test-Driven Development. TDD is
about developing high quality code, keeping the design simple, and
having a regression framework in place for code maintenance. However,
TDD is not sufficient to insure the quality of commercial or
enterprise applications. In general terms, I break down testing
into three general categories:

I) Developer Tests (TDD)
II) QA Tests
III) User Acceptance Tests

Each category is somewhat different from the others. The primary
goal of developer testing is a strong design, high code quality, and
low defect rates.

I) Developer Tests (TDD)

In TDD, a developer always writes a test first, before writing any
code. The steps in TDD are always the same:

1) Write a test
2) Make sure the test fails as expected. If there are new classes or
methods involved, then the test won't even compile in a language
like Java or C#.Net. If the function already exists, but the
code inside has been modified, then make sure the test
initially fails the way you expect it to fail. Let's say you
have a function which calculates change for a purchase and you
are writing a test to cause it to break change down differently
depending on the quanities of available denominations; so whereas
it currently returns one quarter, it is supposed to return two
dimes and a nickle. Make sure the test fails initially by returning
a quarter (as opposed to simply crashing or actually returning the
right change before you've modified the code.
3) Write the new code and make sure the tests now all pass.
4) Examine the design and refactor any duplication (I'll discuss
refactoring in more detail in another class).

That's it, now rinse and repeat!

I'd like to make it clear that the goals of TDD are somewhat
different from the goals of other kinds of testing. TDD is
a development activity. The goal of TDD is first and foremost
to drive the design of the code itself. Using TDD ought to
generate code in which independent concerns are expressed in
separate pieces of code. Such separation makes it easier to
test the code. If the code is hard to test, that implies the
design of the code is not optimal. If the code is easy to
test, then the design of the code is better. Better code means
it's easier to add new features and it also means the QA people
will find fewer defects, and especially should find almost no
defects related to basic functionality.

You can look at an application as a bunch of nested boxes. The
innermost box is the code framework you're using to develop on
top of. It's there before you've written a single line of your
own code. Then you develop code around that kernel. The code
tends to become organized in ever wider layers, although some
"inner" layers may depend on outer layers, so the layering
is rarely "perfect." Still, good code generally has this sort of
hierarchical structure:
_____________
[ __A__ ]
[ [ B ] ]
[ [ [C] ] ]
[ [_____] ]
[_____________] etc...

If you're writing tests for code in B the general approach is
as follows: If necessary, you can "mock" out functionality in C
by subsituting a mock/stub/fake instead of the real code in C.
This kind of thing is done when the real code in C accesses
external resources which are irrelevant to testing the logic in B.
Having extensively tested B by itself, you would then write
a comparatively small number of tests against the application
as a whole making sure that the code in B is actually used by
the application. Such "sanity" tests will make sure the code in B
really has been included in the app and works for a few basic
cases. You can think of such "functional" tests as poking lines
through the outer layer of the application all the way through.
Note, do not confuse a functional test in TDD with a FIT test,
which is a functional integration test. The extensive testing has
already been done, so even though these functional tests do not
test every path through B, they make sure it's properly fitted
into the application as a whole.

A functional test through B looks something like this:
____\__________
[ _\_A___ ]
[ [ \B ] ]
[ [ [\C] ] ]
[ [_______] ]
[_______________] etc...


You can see it's hard to write enough tests like this to properly
cover B. That's what the unit tests are for.

Here is an eample unit test in Java-like syntax (modified from real
Java to make the example more readable):

public void
testCalculateChange_WithChangeLevelling() {
//setup
VendingMachine vm = new VendingMachine();
vm.setNumberOfQuarters(10);
vm.setNumberOfDimes(100);
vm.setNumberOfNickles(100);

//execute
int[] change = vm.makeChange(25);

//assert
assertEquals("use nickles and dimes", [10, 10, 5], change);
}

Note: Before implementing the "levelling" algorithm, the test should
return [25] instead of the expected [10, 10, 5]. Note that this test
just tests the makeChange function. It does not worry about how the
amount of change itself if calculated. A functional test would
be more complicated because more setup would be required, but
there are generally fewer of these per feature:

public void
testFunctionalTest_Purchase_WithChangeLevelling() {
//setup
VendingMachine vm = new VendingMachine();
vm.setNumberOfQuarters(10);
vm.setNumberOfDimes(100);
vm.setNumberOfNickles(100);
vm.addItem("Mars Bar", "$0.75", 15); //15 mars bars; 75 cents

//execute
int[] change = vm.purchase("Mars Bar", "$1.00");

//assert
assertEquals("use nickles and dimes", [10, 10, 5], change);
}


II) QA testing is done with a build of the application provided by
the developers. Therefore QA testing is done after a certain amount of
code has been developed. QA testing is usually done by QA
professionals often with support from users or business people to
make sure the developed software really does work and meets the user
requirements. Some QA testing can be automated with scripts. On
our project, we use a tool called Watir (http://wtr.rubyforge.org/)
to script interactions with the Web application as if a real user
were clicking on buttons and entering data.

III) User acceptance tests are generally written before the
application code has been developed. The format of the tests is
a table with the inputs entered in by the user and expected outputs.
Later, such tests are linked in with the application and executed
to determine whether they've passed. FIT (http://fit.c2.com/)
and Fitnesse (http://fitnesse.org/) are tools commonly used in
the XP community for such "functional integration" testing.