Unit Testing

Disclaimer: I'm no expert in testing, wouldn't even say I have much experience in it or do it consistently, but the times when I've done it, been disciplined about it, and tried to do it right, it really paid off, and that's why I'm writing this entry...

Verifying behavior / Demonstrating correctness

This is the obvious one. An automated test is a great way to verify that a program behaves as expected. An automated test is a great way to verify that a program continues to behave as expected. If expectations happen to change (and they always do), go ahead and modify the tests. Of course its never just that easy but you get the point... You write a test to confirm your expectations of a program.

Notice how I said "your expectations"... "Yeah of course my program works, I know it'll return B when I pass in A"... And there's the inherent bias when writing your own tests. Reducing that bias is essential in effective testing, but I always find it difficult to do. Hehe maybe the more and more experience I get at unit testing will bias me towards NOT trusting my own code. Isn't that strange... as I start to trust my code less and less, I'll write more tests, and therefore my code will be better. I think test-first development helps to reduce the bias because you're writing test code first... before the production code has had a chance to bewitch you into thinking its beautiful and perfect... but alas I find test-first development very hard to stick to after the first few days of development... kind of like going to the gym right after work...

So let's rephrase it: You write a test to confirm your expectations, the business analyst's expectations, the users' expectations, the runtime environment's expectations, etc. of the program. Poor little unit test, that's a lot for you to shoulder all by yourself... wouldn't it then be nice if if everybody agreed on expectations... that's probably harder than the hardest programming problem out there.

Handle a range of inputs (expected and unexpected):

Unit tests help you to determine the range of input values that do work and compare them against the range of input values that should work. Which values should you be handling that are not currently working? What unexpected values can I send to my code? How does it respond? Null objects, boundary conditions, bad types, etc. As you do this research (I believe unit tests involve a lot of research), you can add more cases to your code, deal with unexpected values... which makes your code more robust. Can your code handle the values you expected to come from the external system? I think this is where unit tests shine. You can verify your code conforms to the interface agreed upon between you and the external system. This can save you from many embarassing errors and arguments during integration tests.

As I write a unit test I'm always amazed at input values I forgot to consider...

Program consistency / Expose conflicting requirements and inconsistencies:

I believe unit tests can aid in exposing conflicting requirements or inconsistencies. Do two different parts of the program interpret the data the same way? Is a requirement driving one feature in the program compatible / consistent with one driving another feature that interacts with it. A lot of times as your developing and the spec is not fully complete (is it ever 100% complete), different business analysts may tell you different things. Or a consultant may tell you an external system returns a set of values when in fact it returns something completely different. When the code under test is executed, these inconsistencies can quickly surface.

Just recently a consultant told me that a certain field in a message coming from an external system could be one of two values, both strings. I had written code that depends on the field and had defined constants for these values. Days later when I had finally got a sample message from the external system for testing (by which time I completely forgot about this field and its values), I ran it through a unit test and it failed. I looked at the message and saw that the field was not one of the two values, in fact they weren't even strings, but integers! The consultant had very limited time (because working with developers was not billable time, but that's another story). I ended up having to do my own research about the external system and actually look at some of their code to find out the list of values for this field. I modified my code and then the unit test passed. If it had gone into productino like this it would have failed miserably.

Now I'm not saying that unit tests are a panacea for miscommunication, but in certain instances they can help big time.

Forces you to deal with exceptions (or your users will) / Forces you to eat your own dogfood:

Guess what... your code can throw exceptions... and it should... but nasty HTTP 500 errors, pages with stack traces that piss off the end user, threads silently dying, etc. are nightmares! Unit tests help you exercise different paths through code and therefore force you to think more about error handling. What will you do when the error occurs? Simply log it? Set a boolean flag? Return an error code? Throw an exception? I guess its hard to explain this in words, but having to execute your own code helps you make these kinds of decisions.

Application exceptions: What application exceptions need to be caught? When do they actually occur? Which ones make sense and are useful to the user and which ones just unnecessarily interrupt program flow? How should they be named (and perhaps placed in an exception class hierarchy) so clients of your code understand what the error means, so readability is increased. Remember those crazy things called use cases? Ideally a use case should describe exceptional and alternative flows. I will admit I've only formally dealt with use cases in academic situations (at work I've always worked with a monolithic specs, no specs, email design discussions, but never formal use cases), but understanding at a system level what exceptions occur help me to better define my domain classes and exceptions: it helps me express intent more clearly.

Runtime exceptions: Count your lucky stars if you get runtime exceptions during unit testing... Now you know some of the truly unexpected situations that your code allows. You can handle these directly (not usually the best way) or perform checks in your code so they don't occur (like checks for null objects). This directly helps to make your code more robust.

For example I'm writing some classes right now that map fields from one set of JAXB generated objects (that conform to an external company's schemas) to another set of JAXB generated objects (that conform to our schemas... you must be wondering why I'm not using stylesheets. Personally I'd rather deal with Java code than XSLT and I don't know yet if it will be worse in performance than stylesheets). If an XML element is empty, JAXB will convert that to a null value. As I was reading values from the JAXB object, I got a NullPointerException because I wasn't checking for nulls and my unit test blew up. Mind you this code is going to be executed within a web service, so I didn't want an uninformative SOAP fault to be returned.

So it made me think about the end user.. Hmmm we don't want them to get a useless, generic SOAP fault... hmmm what is the root cause of this error? Certain fields are required and therefore can't be null. We somehow need to check for these required fields and throw an appropriate exception / fault that indicates what required fields are missing and what object they belong to. I was able to add the following method to my abstract base class and allow each
of my mapping classes to override it:

protected void getRequiredFields(...) throws RequiredFieldMissingException

Now in each class I was able to place the code for the required fields in the overridden method. Then a template method in the abstract base class called this method before further processing. In one fell swoop all of my subclasses now have required field validation!! The template pattern is great when used appropriately.

This unit test caused me to refactor my code to make it much more readable, maintainable, robust, extensible... Just because of a NullPointerException, and it didn't take much time !!

Catch stupid mistakes before they become costly (and embarassing) / Inadvertent testing:

try {

doSomethingStupid();

catch(stupid mistakes) {

logger.log(Level.INFO, "Stupid is as stupid does");
}

Null objects, improperly initialized objects, missing resources, wrong logical operator usage (I don't want to tell you how many times I've done this), infinite while loops, unintended recursion (what you don't think this can happen?), array out of bounds, class cast exceptions, you get the picture.

One of the benefits of writing unit tests is inadvertent testing. Sometimes you're testing one feature and it expects things to be in a certain state. When you run the test things horribly fail but not because the feature didn't work as expected... Because you forgot to set up some objects (preconditions) or a path in the code was exercised that you weren't expecting. These kinds of failures are great because it gives you a better understanding of your code, helps you to come up with more unit tests, and helps you to specify the preconditions. Why did that object need to be created? Was it really necessary, can it be refactored - placed into another method or class?... Its get you thinking more and more...

Side Note 1: Any semi-competent developer should know how to build truth tables and finite state machines. They've always helped me clearly enumerate the different cases and write clear code).

Side Note 2: On another side I've noticed that management and customers seem to like "automated test suite" more than "unit tests"...

Scenario 1:
Wha wha what! You're writing your own unit tests?? Look at the Gantt chart, it says development here!! Anyway isn't that what we have a test team for? Wha wha what! You
wrote more lines of test code than production code? I think that's against the law son...

Scenario 2:
So the automated test suite runs by itself AND tells me what works and what doesn't? I can go to the breakroom for coffee and be productive all at once? Why didn't we do this before? Anybody can run it? Yes (even you sir).

So next time you're working on a new project, tell your team lead or manager that you plan on having an automated test suite... It just sounds cooler and is an easier sell during meetings and presentations..