A big challenge when building Web applications is dealing with scalability and performance. Sometimes these issues are forgotten until after the application is being used by the public and under heavy load. Performance tuning usually happens as an afterthought when all the code is already completed.

While I was developing the latest version of Hot Banana, I faced the following challenge. Hot Banana is a ColdFusion Web content management system that uses ColdFusion Components (CFCs) extensively. It queries a database, processes the data, and serves up dynamic Web pages every time a user visits a Web site powered by Hot Banana. There is a lot of data to process in order to generate a single Web page. The dynamic navigation structure needs to be generated from data in the database. All the dynamic content needs to be pulled from the database and processed. Dynamic images and links need to be processed as well.

As we added more and more functionality and control to our Web pages I was faced with the challenge of speeding up the time it takes to generate a Web page. It was taking more than a full second to generate a page on our development server, which was completely unacceptable. I was forced to take a look at the different pieces that make up a Web page and find ways to cut corners.

First I looked at the entire tree structure of all the Web pages on the Web site, what we call the navigation structure. The navigation structure is used to build dynamic menus on the Web page, the site map, and more. Even though every page on the Web site uses the same navigation, every page was building it from scratch every time. We were already using a ColdFusion Component to manage the navigation structure. This CFC was being created by every Web page plus the administration area. This CFC provides a number of functions for interacting with the navigation. Most of the time was being spent searching through the navigation to return results.

To solve this, I decided to keep the navigation CFC stored in a shared scope. I could have stored only the data in shared memory, but by keeping the CFC itself in memory, the Web pages wouldn't need to create and initialize it every time. Then it would be able to use its internal indexes to search through the navigation and return information quickly. Sometimes you might just want to keep data in shared memory with a CFC, but since our navigation CFC is the only CFC accessing and managing the data, it made sense to keep the data inside the CFC and keep the CFC in shared memory. I could have put the navigation CFC in any ColdFusion scope, and there are a number to choose from.

Sometimes, the SESSION scope would make sense for storing data and objects. For example, this would allow each visitor to the Web site to potentially have a different navigation structure. This could have made sense if visitors to the Web site were able to customize the navigation of the Web site. Since all visitors would be seeing the same navigation, the SESSION scope wasn't the right decision.

Originally, I put the navigation CFC into the APPLICATION scope. Hot Banana can support more than one Web site on a single server. Each Web site is running under its own application, so there can be separate SESSION management across different Web sites. Since each Web site would have its own instance of the navigation CFC and its own application, I thought the APPLICATION scope would be a good choice.

But there was something I hadn't considered: our administration area is kept under a different application than the public Web site. When administrators logged in and added a Web page, I needed to refresh the navigation CFC being used by the Web site. Unfortunately, since this CFC was kept in the other APPLICATION scope of the Web site, there was no simple way to access it.

In the end, I decided to put everything related to Hot Banana into the SERVER scope. Since Hot Banana uses a number of different APPLICATION scopes at once, I wanted the convenience of being able to access any CFCs and data from any Web site. The SERVER scope is the most broadly shared scope available. It's shared across all applications that run on the Web server. By putting data into this scope, you allow all ColdFusion applications on the server to have access to these CFCs and data. This isn't always appropriate. If you can guarantee you have exclusive use of the server, you won't have any problems. Otherwise, you'll need to carefully think about the implication of having these CFCs available to the whole server.

I created a single ColdFusion struct variable within the SERVER scope, SERVER.HOTBANANA. I then stored all the cached CFCs and data in this struct. This prevents my SERVER variables from interfering with other ColdFusion applications that want to use the SERVER scope. Inside this structure, I have a struct for each Web site on the server. This lets each site have its own instance of the navigation CFC. Now, the administration area knows where to look to find a navigation CFC for a particular Web site.

You can see there's no limit to the ways you can organize your data, and each decision has its own pros and cons. It's important to make these decisions carefully. It helps to centralize the management of cached CFCs so that you can easily change your decision in the future.

Originally, to create an instance of the navigation CFC, I was using the CreateObject function, along with an initialization call:

objNavigation = CreateObject("component", "hotbanana.navigation");
objNavigation.init();

Instead, I made a global custom GetObject function that would handle the job of caching and managing CFCs (see Listing 1). We use an initialization function, so the GetObject function calls this function for us so that it only happens once.

By passing "true" as the second parameter in this function, you can specify whether or not to use a cached version of the CFC. This function assumes that you will need only one instance of each CFC. You may not want to use this style of caching all the time. For example, you may want to have many different instances of a single CFC to manage different data. You should utilize different strategies and methods of caching CFCs to take advantage of the different ways your application uses CFCs.

You might feel inclined to start caching every CFC you can. This is great - until the Web server starts running out of memory. If the amount of data being cached is great, and there are many applications using cached data, this can easily become a problem. It's important to think about how often the data is being used, and to try to balance the trade-off between performance and memory usage. If some data is rarely accessed, caching it won't have a significant impact on performance.

You may also want to design an algorithm that limits the number of CFCs cached in memory. For example, there may be a CFC for every Web page on a Web site. Your algorithm could cache only the top 10 requested Web pages. This would allow a Web site to get very large without having as much impact on the server's memory.

Caching Different Types of Data
There's no limit to the types of data you can cache within a CFC. Data that takes time to generate or retrieve is a good candidate for caching. Queries, XML documents, structs, and even other objects are worth caching. You may already be using the CACHEDWITHIN attribute of CFQUERY to cache queries in memory. This method is easy to use but lacks control. You can clear all queries at once only by using CFOBJECTCACHE. There is also a limit to the number of queries you can cache.

By caching queries as instance variables within CFCs you can have a lot more control over when to refresh the data. You also have no limit to the number of queries you cache. Parsing, outputting, and manipulating ColdFusion XML documents can be heavy on the server. By keeping an XML document in memory, you have the option of manipulating the document directly without needing to parse in or write out an XML document every time. You can improve performance by accessing such documents directly, the same way you would access structs.

Using cached CFCs as instance variables within other cached CFCs is a very efficient way of architecting an application. With Hot Banana, there are dozens of CFCs that interact with each other. This could create a tree of CFCs, with many being created unnecessarily. By reusing cached CFCs we create more of an interacting web of CFCs. This actually reduces the amount of memory taken up by the application because no unnecessary CFCs will be created.

You can even use shared CFCs to cache data updates as well. We were keeping track of visitors by updating the database every time someone visited, but I found the hit to the database was becoming too much. To solve this, I put a shared CFC in memory for each Web site. This CFC was in charge of remembering log updates (in an array), then writing all the updates to the database once an hour.

If you're generating dynamic HTML output, or any string output, you can use CFSAVECONTENT to cache the output within the CFC. This could even be the only data that you need to cache in your application, especially when it takes the greatest amount of time to generate the string.

Caching the results of CFHTTP can definitely improve performance. Sometimes you may be required to cache these results. If your application is pulling in an RSS feed, often you will be allowed to request the feed only once an hour. If this is the case, you will certainly need to cache the results. You can just look at any function and decide if the result of that function can be cached. It can be easy to implement this style of caching. For example, let's say you have a function that returns a query (see Listing 2). This function needs to hit the database only once. The result will be available in memory as long as the CFC exists. You can apply this strategy to any function or block of code you want to cache.

Refreshing the Cache
Caching data is great, just as long as the data doesn't change. In any application, though, data is going to be changing at different times, sometimes constantly. We need to consider when it is appropriate to cache data, when we will want to refresh the cache, and how we want to manage refreshing cached data and CFCs.

One extreme strategy is to wipe the whole cache every time any of the data changes. This can be unnecessarily excessive. The benefit is that it's a very easy strategy to maintain. If all the cached data is in a single struct in memory, performing a single StructClear() can wipe the entire cache at once. It's the same strategy ColdFusion uses with cached queries, by letting us use only CFOBJECTCACHE to clear out all cached queries. This strategy can be appropriate if data doesn't change very often - and if a performance hit is acceptable when changes do occur.

At the other extreme, you may want to refresh only the exact piece of data that has changed. You can achieve this if you centralize all the actions on a piece of data within a single CFC, as you may very well be doing anyway. Then, when the data gets updated, you can clear out and regenerate it.

There are other strategies between these two extremes. You may want to clear out the cache periodically, say, once an hour. Or you may want to clear the cache based on certain events that happen within the system. For example, a cached Web page could be refreshed any time an administrator loads the administration area for that page.

During development you may want to disable caching of CFCs. Otherwise, as you're making changes, your code will still be accessing the old, cached versions. It's important to note that if you do this you will definitely need to test your code with caching turned back on. Bugs relating to caching issues are always the most confusing, frustrating, and difficult bugs to narrow down and fix - especially if users of the system aren't aware of the caching going on behind the scenes.

Concurrency Issues
If only one person uses your application at a time, you likely won't run into any problems. Unfortunately for us, ColdFusion can use more than one thread at a time. This means that while one Web page is trying to update cached data, another Web page could be trying to retrieve this data at the same time. There are absolutely no guarantees about the order in which things will happen in this scenario.

Say you have a cached CFC that contains an array of data with three items in it. The CFC has two functions, resetArray() and getItemThree(). When the function resetArray() is called, it first sets this.array = ArrayNew(1), then it fills the array with data. Later, in getItemThree(), there is code that first calls resetArray(), then returns the third element in this.array (see Listing 3).

What would happen if getItemThree() were called by two different users at the same time? The answer is, we don't know. It's possible that everything will work fine. However, it's also possible that immediately after this.array = ArrayNew(1) is called, and before this.array[3] is set, the other getItemThree() function will try to access this.array[3] and find that this array element does not exist.

This is a very difficult situation to test. You won't realize there are problems with your code until it's being used by a large number of people. Even when you do know there is a problem, it's difficult to be able to re-create the problem, let alone prove that the problem has been resolved. How do we prevent problems like this? Luckily, ColdFusion provides us with CFLOCK. What we can do is put CFLOCK tags throughout our code (see Listing 4).

This will ensure that only one user can call this function at a time. It will guarantee that when we access this.array[3], it will still be initialized from the resetArray() function. When one user is trying to call resetArray() and the other is trying to return this.array[3], one will have to wait for the other to finish.

We might be tempted to put CFLOCK tags around all the code in our cached CFCs. This might allow us to be certain things will be kept consistent, but it may have an impact on the performance of the server. Most of the time you won't have problems with two users accessing a function at the same time. It's only when the cached data is being manipulated or accessed that you will need to be concerned. You should try to put as little code as possible inside CFLOCK tags.

A nice way to achieve this is through the use of getter and setter functions. These are special types of functions whose only role is to set and return variables in a CFC. Anytime you want to access these variables, you call these functions. Then you can put CFLOCK tags inside these two functions. In the getter function, we can use the CFLOCK attribute type="readonly". This way, more than one getter function can run simultaneously with no problems and no performance implications. Only the setter function will need type="exclusive".

It's not just instance variables that we need to be concerned about. Local variables in our CFC's functions can be a problem too. If you don't declare your local variables at the top of the function with the var keyword, they actually become instance variables of the CFC. This means that when someone is in the middle of a function, all of a sudden all the variables being used by that function could suddenly change. The function could be in the middle of a loop, when all of a sudden it finds itself at the start of the loop. The only way to solve this is to be sure you declare all your variables at the top of every function using the var keyword. This is generally a good idea anyway, as it ensures that your variables won't be overwriting variables from other functions.

Conclusion
After putting these practices into play, putting the core structure of Hot Banana in shared memory, and caching as much data as possible, I was able to reduce the load time of a Web page substantially. The time was reduced from over 1,000 milliseconds to as low as 10 milliseconds for a single Web page. This yielded a 99% improvement in performance, resulting in a very stable and scalable system.

Caching CFCs in shared scopes is a good way to improve the performance and scalability of your ColdFusion applications. There are definitely a lot of things to watch out for, but if you do it carefully and diligently, you will be rewarded with a big performance payoff.