Sunday, July 30, 2006

Ravi's Post About Boring Enterprise Software

I really enjoy reading Ravi Mohan's blog. Recently I read an entertaining post about enterprise software development, and how truly boring it is, in response to a blog posted originally by Martin Fowler. Here it is, "But Martin, Enterprise Software Is Boring!" Hehe. :)

In reading Ravi's post, I thought about a few things, like what makes software development fun and interesting. Ultimately, I came to the conclusion that software has a number of dimensions to it. The first cool thing about software is that you get to build something incredibly interactive. Software has a magical quality to it. It's almost mystical. Most things in life, you can kind of see how they work; they're physical. But on a computer, you just click on some keys for a while and all sorts of things may appear on the screen. You can make a video game, a business application, a compiler. It's amazing! The first computer I remember using was the Commodore 64. I learned the "poke" command and would poke in random numbers. Sometimes the screen would change color; sometimes nothing would happen; sometimes the system would just go wonky. The amount of technology the goes into making that sort of thing possible is truly vertiginous - from the design itself to physical manufacturing of all of the circuits and I/O to the logic of the operating system and device drivers, and finally to the development environment and compiler or interpreter. A few hundred dollars worth of hardware and software that anyone can pick up at a local electronic outlet represents the penultimate technological achievement of our age. They're so ubiquitous that we tend to forget how extraordinary computers really are. When we program, we have a kind of deep connection to this incredible phenomenon.

I think there are different personalities in the software dev world. Some dimensions of software development are only barely visible to the end user. In XP, when making a new piece of software they are often labeled "motherhood stories." Performance is one of them, perhaps the most common. In business software, often called enterprise software, it is often easier to spend 10, 20, or 50k on improved hardware rather than optimize memory or cpu. From a user's point of view, the software either performs adequately or it doesn't. How one gets there doesn't matter much to the user. Some developers really enjoy the challenge of finding creative solutions to such problems, and reasonably often, simply buying hardware won't cut it. Buying hardware is the brute force solution. It's how Alexander The Great "untied" the Gordian Knot and went on to conquer most of the known world at the time. When it works, it works. However, in the world of operating system design, compilers, video games, and embedded systems, it is often not a reasonable solution. Here, the cleverness of systems and framework programmers comes into full force. The user requirement is very simple: Make it work with this much memory and this much cpu. The hacker then goes off and happily works away behind the scenes to make that requirement a reality. It can be an enormous amount of work that the end user never fully appreciates. Just as none of use truly appreciates the wonder of being able to flick and switch and voila, the lights are on, the TV is on, we can check our e-mail. For the every day business programmer, most such techniques are not only un-necessary, they would probably be counterproductive; using them would lead to a less maintainable system.

Another aspect that some developers are attracted to in such "hardcore" systems is novelty. I suspect that developing one's first compiler must be very interesting. I myself only ever touched the surface of compiler-writing in a University course, but I can see how the sophistication of optimization can become tantalizing. However, I think thay novelty is something that never lasts, no matter what domain one is working with. I once worked with a very smart software developer who had been in the game for 20 years. He had his own company and had written numerous compilers and interpreters. For him it was routine, and he never wanted to do it again. Ultimately, everyone must make that decision in life. If you're only interested in novelty and the excitement of new problems, the research field is probably the best place to go. Even there, you will encounter the tedium of having to publish papers and cite references. If you're a professional software developer, one way or the other you will spend a certain amount of time learning new things, but more time building software to a customer's specification. If all you do is develop operating systems, or compilers, or quantum simulation technologies, whatever it is, soon enough you will have a standard set of tools and your main focus will be to figure out how the building blocks and tools you have fit into what your customer wants. In that respect, I think all professional software developers are in the same boat. Our main job is not to tackle a sexy new problem every day; it's to understand what are customers/users want and to provide a quality product in a timely fashion.

For me, the interesting thing about software development has to do with understanding the business needs of the customer. One ability I've had to hone is the capacity to understand a customer's business on the fly. I've worked in many different areas, and within a few months of starting a project I've had to get to the point where I could probably get hired to do my customers' job, usually at an entry level I'm sure - Still, being able to understand the fundamentals of various businesses, the motivations, the potential efficiencies, that's a real skill, as valid in its own right as being able to optimize C code to operate in under an mb of ram. Another area is modelling.

Being able to model a piece of software so it's maintainable, so new features fit in nicely, while dealing with the fact that many business rules are quite arbitrary, is a difficult skill. It means walking a fine line between generaling code to avoid duplication while avoiding over-design that leads to crufty frameworks that won't accomodate tomorrow's strange and inconsistent variations. More "technical" software projects are often far neater and more orthogonal, and therefore more amenable to a general analysis. As an analogy, consider how some differential equations can be solved analytically, but most can't. There is a neatness of solving somthing completely, the the messy reality of the world is that you come up with partial solutions that suit you in practice. Anyway, I can see how a tendency to construct a complex mental model of the whole system right up front can be a good thing when developing a compiler, while it actually might be a very nasty trait when developing certain business applications.

I guess my point is that people tend to vary quite a bit in the problem domains they're interested in or are good at solving. We tend to assume that whatever our particular area of interest or expertise is, that's the really hard thing to do. The truth is that sofware is a difficult thing to master for a reason, precisely because there are so many dimensions, and I'm sure I've only scratched the surface in this post!


Ravi said...

Nice to know my post provoked you to think so much about the nature of software development. :-)

"Still, being able to understand the fundamentals of various businesses, the motivations, the potential efficiencies, that's a real skill, as valid in its own right as being able to optimize C code to operate in under an mb of ram."

Good Point. In 10 years of working in enterprise software, I've seen exactly 3 people who have this (very valuable) skill.

However, I think it is important that one doesn't get trapped into a false dichotomy. I don't think these skills are necessarily mutually exclusive in that one doesn't have to practice "Customer Affinity" *at the cost of* technical excellence. One skill is NOT a substitute for the other. That was my point.

I think, in the end it is all about what skills you enjoy developing and exercising.

I like understanding how business works. I like writing "tough" software. I like making people who use my software happy.

I do NOT enjoy grinding through 500 jsp pages :-) when the architecture settled down 8 years ago and I am being treated as a "cheap Indian programmer" who gets to do the boring work. :-)

In short, I agree with most of what you say. Having said that, given a choice I *still* prefer acquiring and using specialized knowledge and write programs that require high levels of skill, then the n+1th J2EE app.

FountainHead said...

Vlad, you must be an author! Give me the copy rights - We can publish "Chicken Soup for the Programmer's soul" . Excellently written,Vlad! I am proud to be working with ya