ColdFusion MX Hardware Performance

Jan. 7, 2003 12:00 AM

Quite frequently, the question has been posed: "What kind of server should I set up to run ColdFusion MX?" Of course, any answer given by a software vendor, like Macromedia, would include a brief set of system requirements that detail what the minimum and recommended platforms are for the software.

While this serves its purpose to inform you of your platform requirements, it doesn't really provide insight into what hardware would truly run the software at its optimum. To offer a response to this elusive question, a series of tests were performed on several platforms to illustrate just what makes ColdFusion MX fly.

The CPU and How It Relates to ColdFusion
During the tests, it was obvious that MX scales well with multiple processors, clock speed, and L2 Cache. Okay - hold on; you might say, "What the heck are clock speed and L2 Cache?" Before we get into the test results, here's a brief CPU tutorial as it relates to MX.

When you request a template in ColdFusion, the CPU will perform all associated operations for that template. To accomplish this as quickly as possible, the CPU needs to keep the maximum number of instructions or amount of data relating to that operation available in its internal cache. There are generally between two and three "levels" of cache on the CPU. The idea behind splitting cache into multiple levels of a hierarchy is to continue to provide even faster memory.

The L1 Cache is relatively small and holds any data relevant to the operation. The L2 Cache holds excess data that couldn't be stored in the L1 Cache. It's relevant to the current operation, and perhaps the consecutive operation as well. However, the L2 Cache is significantly larger.

When the CPU has maximized its internal cache, and therefore must use external resources, such as RAM or memory, it slows the operation down considerably. In comparison, some of today's memory runs at 166MHz, while the CPU's internal cache usually runs at the same speed as the CPU. For example, if your CPU is a 2.0GHz processor, then the internal cache is also running at 2.0GHz - that's over 12 times faster than your memory.

This shows just how important it is that the L1/L2 Cache be as large as possible. Let's say you have a CFMX Web site consisting of one template. The first time the template is requested, it is pulled from main memory, which is slow (remember: over 12 times slower than the CPU), but then the CPU stores it into its cache. The next time the template is requested, it is served directly in the cache, and we now have a cache "hit."

Performance is improved tremendously, as the CPU doesn't have to go out to memory to perform its operations. It is for this reason that a fast cache is necessary for good performance in CFMX. But what happens when you have, say, 200 templates?

In this scenario, it is obvious that all of the templates won't fit into the small cache (usually 512KB or less for L2, and 128KB or less for L1). Thus, the CPU must go to main memory more frequently, which will reduce performance. If you can reduce the number of instances that the CPU must go to main memory in order to fetch data it needs, performance will improve dramatically - this is where a larger L2 Cache to store more useful data will help.

Now that you have an understanding of what the L1 and L2 Caches are in a CPU, we can discuss the other component with which ColdFusion MX seems to scale well - clock speed. To put it simply, clock speed is the speed in hertz at which the processor runs. I'm sure you're all aware of the different speeds of available processors, like 500MHz, 1GHz, etc. ColdFusion MX scales well with faster clocked CPUs. So if you upgrade from say, a P4 1.6GHz to a P4 2.0GHz, you'll see an increase in performance.

The Tests
Creating a Web-based test that will constantly work a CPU to the point of showing results between different clock speeds isn't as easy as you might think. This holds true because Web templates are relatively small, and it takes many clients to create a situation where the CPU is working as hard as possible without saturating the machine.

To construct the test, a load test was run on a rather small database, on a real-world application. The application used was FuseTalk Community Edition (made by FuseTalk Inc.), which is architected for ColdFusion MX. As a side note, one might assume that this application was chosen for this test because it's my employer's product. That's not entirely true. The main reason is that it's a real production CFMX application leveraging one of the major new features in CFMX - CFCs.

Have you heard enough about these lately? While they are fairly simple to write, they have drastically changed the way most ColdFusion applications are written. Since ColdFusion MX recently came into the scene, there are very few commercial applications designed and written for MX. FuseTalk Community Edition was designed and written with CFCs, and its entire backbone is based on a CFC API of sorts. Obviously, this design will be used by many people within their own applications, since it's going to provide the most scalability with ColdFusion MX. So it makes a useful test case.

The load test itself was recorded with the Microsoft Web Application Stress tool. It's a free tool that works quite well for loading up an application. You can use it to record a browser session with your application, loading any part of the application you want to load test. The major parts of the application were hit to construct this test recording, such as loading categories, threads, messages, and logging in. When recording the test patterns, the Web Stress tool was instructed not to record delay, so that we could stress the application by playing back the results as fast as possible.

The Web Stress tool was set to a stress level of 20 threads with a socket multiplier of 1. During the 30 minutes, the Web Stress tool replayed the test pattern, and simulated anywhere from 200-700 users depending on how frequently the user would refresh a page. The operating system used for the tests was Windows 2000 Server SP3. Apache 2.0.39 was used as the Web server. No additional tuning was done to the default install.

ColdFusion MX Tuning
To get the most out of the platform, you have to tune the software, of course. ColdFusion MX runs well as an out-of-the-box product, but it runs even better with a bit of tuning here and there. The adjusted settings are:

Trusted cache: On
Template cache size: 200
Limit the number of cached queries: 300

For more information on tuning the ColdFusion Administrator, check out Charlie Arehart's article "Toward Better CF Server Administration - Part 1: Performance," (CFDJ, Vol. 4, issue 1).

Next up is the JRE. The stock JRE in ColdFusion MX is Sun 1.3.1. The JRE was changed to BEA JRockit 7.0 for 1.3.1 J2SE. This is a free JRE, distributed by BEA (www.bea.com). The JRockit JRE is noticeably faster than the Sun JRE, by a good 20% from what I have seen so far. The JRE memory settings were set to 128MB minimum and 400MB maximum in the ColdFusion Administrator under the Java and JVM tab. The BEA JRockit JRE is free, distributed by BEA.

Processed Templates
This test will indicate the number of templates each platform had processed in the 30-minute test time (see Figure 1).

The results may surprise some people, especially those still running the outdated P3 Xeon platform. While the P3 Xeon was a solid performer in its day, it's now outperformed by single CPU desktop processors. As you can see, the Dual AMD Athlon 1800 surpasses everything here, and does so with a small hit to the wallet. At approximately $700 for the CPUs, 512MB of memory, and motherboard, you really can't go wrong.

What we're seeing in these results is the advantage of pure clock speed and L2 Cache scaling. The Intel Pentium 4 2.4B and 1.8A surpassed the Dual Xeon, and the Intel Pentium 4 1.6. Both the 2.4B and 1.8A have a 512K L2 Cache, giving them a decent performance increase over the P4 1.6 with it having only 256K of L2 Cache. Clock speed also plays a part; as you can see, ColdFusion MX scales well with the clock speed advantage that the 2.4B has over the 1.8A.

Another important point to note is how well ColdFusion MX scales with multiple CPUs. The Dual 1800 performed 66% better than the single 1800.

One thing to consider: you could just upgrade to a more current dual CPU platform, and run only one CPU. As your need for more power increases, just add another CPU. Besides the advantage of the faster CPUs available today, you'll benefit from the most current memory and motherboard architectures.

Average Request Time
This test outlines the average request time for the test period. All request times for each request are recorded, and then averaged at the end of the 30-minute test period. Again, the raw clock speed and Cache scaling play a significant part in the single processor differences. The Dual AMD setup shows its strength again here (see Figure 2).

Conclusions
There are a number of conclusions to draw from these tests:

L2 Cache plays a factor in the single CPU tests: If you noticed, the Intel Pentium 4 1.8A pulls ahead of the AMD 1800+. This is because of L2 Cache. The 1.8A has 512K, while the AMD 1800+ has 256K. As soon as the CPU has to release L2 Cache memory to fetch the data needed for the next operation, performance degrades. The Intel P4 1.8A will have to do this less often than the AMD 1800, hence the performance advantage.

We have clock speed, which is shown by the differences between the Intel Pentium 4 2.4B and the 1.8A: The CPUs are virtually identical in architecture, so this is the advantage of higher clock speed. The Dual Xeon 500 platform is outmuscled by clock speed. It has plenty of L2 cache, but it's only running at 500MHz, which is slow by today's standards.

Last, but certainly not least, is the advantage of a second CPU, illustrated by the AMD 1800+ tests: Adding another CPU is most certainly going to give the highest gains, especially the fact that you're getting something modern. Adding a second CPU will not only add a CPU to process another operation simultaneously, but it will add another 512K of L2 cache. If AMD isn't your game, a dual P4 Xeon will more than likely perform as well, if not better, than the Dual AMD 1800+.

For a fairly good price, you can feed ColdFusion MX what it needs - a faster CPU and a decent amount of L2 Cache. If you're looking for the highest performing hardware for your ColdFusion MX Server, it may be time to turn that old Xeon or Dual P3 into a high-performance development machine and bring those product servers up to date.

Related Links

▪ Figure 1

▪ Figure 2

About Jason Clark
Jason Clark, cofounder and CTO of e-Zone Media Inc., has more than 10 years of experience in programming specifically and the IT sector in general.