If people only knew how hard I work to gain my mastery, it wouldn't seem so wonderful at all.
- Michelangelo

Aristotle is the man usually credited with the invention of the logical device known as a syllogism. The syllogism takes two (hopefully) undeniable premises set forth in such a way that the conclusion is inescapable. Here is its classic formulation:

Premise 1:
All men are mortals.
Premise 2:
Socrates is a man.
Conclusion:
Therefore, Socrates is mortal.

On that simple foundation (premise plus premise yielding to conclusion) is built much of the last 2000 years of Western thought.

But new times call for new thinking, and I'd like to offer this updated syllogism:

Premise 1:
All programs have bugs.
Premise 2:
You write programs.
Conclusion:
Therefore, your program has bugs.

This syllogism's regrettably inescapable conclusion leads me to this month's topic: testing.

Testing is not something that comes naturally to most of us. I'd say the surest mark of an inexperienced programmer is an unwillingness to comment on his or her code, but it's followed closely behind by an aversion to testing it.

It's probably not coincidental that both of these tasks suffer from the "banana" problem - named after the little girl who told her teacher, "I know how to spell banana. I just don't know when to stop."

Everyone knows they should test their code, but where do you begin - and where do you stop? I think it will be helpful to peel this particular banana by looking more closely at what we mean by the term testing.

Testing and Debugging
While there are some obvious connections between debugging and testing, they're two separate disciplines. Debugging tries to get the program running. Testing is more comprehensive. Debugging tells us that there are no obvious errors. Testing probes that contention, seeing if the correct execution of our program can be depended on, or is merely a happy coincidence.

The best testing is planned into the design of our program. Such programs are written so that, at any point during the development (and perhaps even afterwards), tests can be run.

In this article I'm going to relate my own testing techniques. Because I use Extended Fusebox for developing applications, the test methods I use will be Extended Fusebox-centric, but good testing principles don't subscribe to any one methodology and can be adapted to what you find most helpful.

Levels of Testing
The most basic level of testing is the unit test. This is testing done at the level of the component (in Fusebox, this would be a fuse). What unit testing tells us is that this "unit" works as it should. It makes no larger claims about the application.

Very often unit testing is skipped or is done in an ad hoc fashion. Whatever time savings may accrue usually turns out to be a false economy. What should have been a simple case of a failed test on a single unit becomes an elusive bug in a larger component, costing far more in money and project delays to find and fix it.

Integrated testing describes the attempts to "wire together" the various lower-level units to see if, together, they work as advertised. It may be done at a subassembly level (a circuit application in Fusebox) if the application size warrants it, and it's always done on the release candidate for the finished application.

Acceptance testing is done with the client to verify that the application does what it's supposed to do. An acceptance test plan provides clear guidance on what tests should be run and what these tests should reveal in order for the project to be successfully finished. If you've ever been plagued with feature creep after delivery, acceptance tests provide a fixed point against which to evaluate any requests. It provides protection against our own version of the banana problem: "I know how to program an application. I just don't know when to stop."

Test What?
At all three levels, we must know what to test. Otherwise we're likely to simply click the "Browse" button on ColdFusion Studio and approve it if we don't see any obvious errors. In the remainder of this article, I'm going to restrict myself to discussing unit testing. What, though, should we unit test for?

You may want to consider the idea of "testing against a contract." As the name suggests, this means that you have a contract against which to test. We can test against a contract at unit, integrated, and acceptance test levels, though the format of the contract may be quite different. Here's a contract for a simple fuse (unit level):

If I receive an "itemID", I update the "Item" table with "price" and "quantity" passed to me. Otherwise, I insert a new item in the database.

If you use Fusedocs, this contract is likely to seem very familiar. It's the statement of responsibilities found in every Fusedoc. (For more reading on the Fusedoc specification, go to www.halhelms.com.)

Such a statement of responsibilities is important not only for testing, but for writing the code. Without it, how will the programmer know what to write - how to fulfill the contract, as it were?

Unlike the ones we're all too familiar with, these contracts aren't lengthy, obfuscated documents. They should be written in such a way that a competent coder can write the code without knowing anything more about the application. The more modular the code, the shorter the contract needs to be and the easier it is to write to and test against.

While Fusedocs make excellent contracts at unit level, I rely on prototyping and client feedback via DevNotes (more info at www.halhelms.com) to provide guidance on what the tests should test for.

Test Harnesses
No matter what level of testing - unit, integrated, or acceptance - successful tests are those that can be run again and again. It is key to successful projects - certainly successful large projects - that we have a high degree of confidence in the code that's already written. When you write all the code yourself, this may not seem much of an issue, but when other people are involved, it becomes a necessity.

We are all used to ad hoc tests - tests involving alterations to the code itself, such as <cfoutput>ing variable values. Here's an example of an ad hoc test:

<!--- TEST CODE HERE--->
<cfoutput>
<cflock scope="SESSION" timeout="5">
The value of session.userID is #session.userID#<br>
</cflock>
</cfoutput>

While this is a very helpful means of testing, we want a more flexible and powerful technique. This technique is the test harness. The harness "fits over" our code, letting us run automated tests whenever we wish. Here's one example of the simplest test harness: <cfinclude template="myCFMLpage.cfm">

It's nothing more than a file that includes the file to be tested. Of course, in real life, you'd probably never encounter so simple a test harness. After all, real code files often expect arguments to be passed in, and will break if they're absent. So, here's a more realistic - though still simple - test harness:

<cfset self="FuseTest.cfm">

<cfapplication name="TEST" session management="Yes" clientmanagement= "Yes">

<cflock scope="SESSION" timeout="10" type ="EXCLUSIVE">
<cfset session.userID = '111'> </cflock>

<cfinclude template="myCFMLpage.cfm">

Fusebox links and forms return to that main page (the fusebox). But right now, we don't want them returning there. When we're doing unit testing, we may be sitting on a beach somewhere warm and not have access to the fusebox.

Even if this much-to-be-desired scenario isn't the case, the fusebox isn't the ideal file to use when testing. It doesn't provide us with the kind of feedback we need for testing. Luckily, we can create a special file (I call mine FuseTest) that gives us more information. But I'm getting ahead of myself. I need to explain a little more about my Extended Fusebox methodology so you can follow what's happening.

Using Extended Fusebox, I don't hard-code links to the central file, but instead use the alias "self". Here's an example:

<a href="#self#?fuseaction=some_fuse action">click me</a>

<form action="#self#?fuseaction= some_other_fuseaction" method="post">
..
</form>

This looks odd at first sight. Fuse- boxers are more likely to recognize the above code in this formulation:

<a href="index.cfm?fuseaction= some_fuseaction">click me</a>

<form action="index.cfm?fuseaction= some_other_fuseaction" method="post">
..
</form>

There are two reasons for using the alias. First, we want our fusebox file to be the default page called when a user enters "www.my-Domain.com," but how do we know that the default page on the ser-ver(s) on which our applications will be deployed will be index.cfm? Since we don't, using the alias "self" lets us resolve this at deployment.

The other reason is for just this case - when we're testing. Instead of having the fuse submit to the fusebox, we want it to submit to a page that will display the variables being created by and/or sent with this fuse. Thus we can verify that our code is adhering to the contract set out for it. All that's required is pointing "self" to FuseTest.cfm. FuseTest. cfm uses Dan Switzer's excellent tag, Debug.cfm, available at Allaire's Developer Exchange, http://devex.allaire.com/developer/gallery.

When the test harness is run, the resulting fuse will head back to FuseTest, which will call Dan's custom tag and we'll see exactly what our fuse has been up to (see Figure 1).

Now I can test the output of my fuse with the contract in my Fusedoc to make sure the fuse is doing what it's supposed to.

In addition to setting "self", you also need to set any parameters the file being unit tested is expecting. How do you know what needs to be set, apart from poring over the file? Well, if you use Fusedoc, the "attributes" section will provide you with information. Regardless of how you get the information, you'll need to have these params set prior to testing the unit file.

Finally, you call the actual file to be tested, "myCFMLpage.cfm", in the example I gave. I save these test harnesses using a "tst" (or "tst_" if you like using underscores) prefix. They can go in the same directory as the actual files themselves, ready to be used whenever tests need to be run.

Testing Parameter Values
We can also test how robust our code is in dealing with unexpected parameter values. Suppose, for the sake of illustration, we are expecting a single parameter to be passed to our fuse. Here's the attributes section of the Fusedoc from "myCFMLpage.cfm":

--> attributes.quantity: an INTEGER

If this is completely foreign to you, here's a translation: a parameter called quantity, of the scope "attributes", will be passed into the fuse. The datatype of this variable will be an integer. Here's a snippet from "myCFMLpage.cfm":

#Evaluate( 19.95 * attributes.quantity)#

What kinds of things do we test for? Well, among others, we want to know how the application will handle things if the "quantity" passed in is one of these values:

• -1: A negative number
• 2.6: A noninteger number
• 0
• "hi there": A nonnumeric value
• 3: A valid value

We can rework our test harness to loop through each of these values, looking to see how the fuse handles it. Here's what the test harness would look like: <cfset quantity_List = "-1,2.6,0,hi there,3">

<cfloop list="#quantity_List#" index="aValue">
<cfset SetVariable( 'attributes.quantity', aValue )>

<cfoutput>
<b>Evaluation where attributes.quantity = #aValue#</b>
<br>
</cfoutput>

<cftry>
<cfinclude template="myCFMLpage.cfm">
<cfcatch>
<cfoutput>
#cfcatch.message#
</cfoutput>
</cfcatch>

</cftry>
<hr>
</cfloop>

We start by putting all the values we want to try in a list. Then we loop over the list, enclosing our file to be unit tested in <cftry><cfcatch> blocks to trap any errors. Figure 2 shows the output we get.

Of course, in a real-world case, you usually wouldn't have only one variable being passed in. In that case you can use nested loops to test for every possible permutation:

<cfset attributes_quantity = "-1,2.6,0,hi there">
<cfset attributes_myName = "Adam,Steve,Nat,Hal">

<cfloop list="#attributes_quantity#" index="aQuantity">
<cfloop list="#attributes_myName#" index="aMyName">
<cfset attributes.quantity = aQuantity>
<cfset attributes.myName = aMyName>
<cfoutput>
<b>Results with following parameter values:<br>
quantity: #aQuantity#<br>
myName: #aMyName#<br></b>
<br>
<cftry>
<cfinclude template="myCFMLpage.cfm">

<cfcatch>
#cfcatch.message#<br>
</cfcatch>
</cftry>
</cfoutput>
<hr>
</cfloop>
</cfloop>

Waiting for <CFDWIM>
Some time ago I asked Sim Simeonev, Allaire's chief architect, if he would add just one tag to ColdFusion. All I wanted was a <CFDWIM> tag.

"DWIM?" he asked.

"Yes, DWIM. It's an acroynm for Do What I Mean."

Sim promised me he would work on it. Until then, we're going to have to test our code. Regardless of the methodology you use for writing code, you need to ensure that your code works, and is robust enough to handle unwanted situations.

I've found that making myself write test harnesses influences the way I write code. Now I write it with an eye to testing it, and that shift in thinking has helped me write better code. One thing is sure: testing applications at the unit level allows the entire project to proceed faster and with less risk of running into error landmines when we later integrate the code. I hope you find some of these techniques and ideas help you write better code.

• Figure 1
• Figure 2