CUISINE

download
screenshots
home


Quick Intro
User Guide
System Design
File Formats
Terminology
Object Layout
Debugging Guide
Stability
Timeline View
Frame Server

Comments on automated tests

I like lots of XP, but not so much the tests. The big disadvantages for me are:

Tests are a pain to write. It's faster not to write them. I think it's more fun to write code and not write the tests.

Nearly all test suites don't guarantee good code. They only bound the behavior to the extent that you can write good tests. I don't want to write good tests or maintain them. I want to write clear, short, high-level application code which says no more than what it does. Documentation will provide even higher-level descriptions of what the code does (as well as hold any discussion, motivation, or other accessory to the code).

The plain alternative to automated testing is using the code, and noticing if it screws up. But that's not a complete test, since even if the surface functionality looks right for a few execution paths, you don't know if the program's internals are corrupt. So I add logging to my code. Tons of logging (about 2% of my lines of code are log calls, as of 2002/11/26). Logging output is a debugging tool and another form of documentation. Some log channels are even used for profiling.

Moreover, I like transparent programs, meaning programs that don't keep a lot of private structures. Where possible, any data the program deals with shows up either in the GUI or the data files. Anything else is probably available in some log channel.

A benefit of automated testing that I'm missing is the one where if I change some code, I'll quickly learn what other code it breaks since I'll see some faraway tests fail. That's a good feature against which I have no argument. But as long as I'm not doing automated testing, hopefully I'll remember the risk I'm taking (modifications can break faraway code without me noticing) and write code that's even more tightly modular.

I may start doing more tests, especially for the modules I hope to release as separate libraries.

[on sanity checks] C extensions and kernel drivers should be packed with checks. I had to abandon development of a dvgrab replacement called pour because of too many /dev/dv1394 crashes. I found a way to make pygtk segfault because it was missing a check. The Python wisdom is that it should be impossible to cause a segfault in Python (hinted at here by the creator of Python himself)

What else

Yes, this page is really supposed to say how you might go debugging various types of bugs. When I get to writing that part, the page will explain how I named my logging channels, how to turn them on and off, how to narrow down a strange error to the right program module, etc.

drewp@cuisine.bigasterisk.com.