Wednesday, May 11, 2005

Software quality

Software quality is a tricky thing. There are many schools of thought. The formal methods camp says the only way to get quality software is to formally spec it and test against the spec using some formal testing method such as model-based testing. The agile camp says that the developers have to build quality in on a day by day basis based on feedback from the customer. There are problems with both approaches.

The formal methods approach assumes you can specify things formally for all systems. There are some systems where formal specs are impossible and there are a large number of systems in which formal specs are too expensive (most all IT applications come to mind).

The agile approach assumes the customer can tell when they get something they want. In some agile methods it is encouraged that the customer write the acceptance tests. This will work for some systems, but there are a large number of systems where this will not do (banking and pacemakers come to mind)

Is there a middle ground? I think there are ways to combine some of the formal methods approaches with the agile approaches for the subset that require it. Lets face it quality is in the mind of the buyer, not some objective thing (although there are people who think defects/LOC is meaningful. Just to dismiss this think of two fictional word processors. Both have the same LOC count and have 1 defect each. The first word processor's bug is that it crashes just before it saves. The second just after. Which has higher quality. Only an idiot or zealot, is there a difference, would say neither.

I say where it is warranted (two examples are when the cost of a bug is more than system development, the past has shown that the system needs better testing) you can make the acceptance tests in an agile approach use model-based testing. I haven't seen it work, so I am skeptical, but hopeful.