Welcome!

You will be redirected in 30 seconds or close now.

ColdFusion Authors: Yakov Fain, Maureen O'Gara, Nancy Y. Nee, Tad Anderson, Daniel Kaar

Related Topics: ColdFusion

ColdFusion: Article

Compilation and Precompilation in CFMX Templates

Compilation and Precompilation in CFMX Templates

Last month I introduced a precompile "batch" file that allowed you to manually compile CFMX templates, saving the penalty paid by the first user to browse a CFMX template after it was created or edited.

But there will be curious folks (and bit-twiddlers) among you who will want to know more - maybe lots more. How much time is this really saving? If it's compiled to disk, how and when does CFMX read it into memory to execute it? What's the cost of that? What happens with CFINCLUDEd files? Where does the compiled code go? Can I look at it? Can I just delete the generated class files instead? How do I determine which class file was generated for which CF template? Can I distribute the compiled code on other servers without the source code?

I'll address these questions and more in this conclusion.

Evaluating the Execution Time
Before going on, it may be helpful to explain something about determining the impact in terms of execution (and other startup) times for running a template. It will be very helpful to turn on the "Report Execution Times" option in the ColdFusion Administrator settings for "Debugging Settings". This will produce a display (see Table 1) at the bottom of a page when executed in a browser.

As in previous releases, you will only see this debugging information if the Administrator has enabled it. Also, if any IP addresses are listed in the "Debugging IP Addresses" page of the Administrator, then you will only see it if you are in that list. If there are no values in that list, all who execute CF templates on the server will see this info. Finally, if you still don't see this information, be sure that your template (or the application.cfm that is called before it) does not have <CFSETTING ShowDebugOutput="no">.

This particular example shows a file called test.cfm, which included a file called test.html, and also had an application.cfm that was called. It also shows a separate value for "startup, parsing, compiling, loading, & shutdown". We'll get to that in a moment. Keep an eye on this set of timing information when trying to evaluate the impact of various choices and settings.

Compilation Doesn't Eliminate All the Waits
So, after reading my previous article you now know how to precompile a template. You've paid the cost of compiling the code to its underlying Java byte code and now it should execute without any parsing/compiling delay (assuming it's not been edited). What if you restart the server? Will the first user really experience no further delay? Not exactly.

You may recall that I said that CFMX writes the compiled code to disk. Did you wonder how it gets from disk into memory, for CFMX to execute? I mentioned at the outset of the previous article that even in ColdFusion 5 and before, the first execution of a template after a server restart would cause CF to interpret the template into "pcode" and then load that into a "template cache". The code was then executed from memory all day, rather than being read from disk (unless the cache filled, in which case the least recently used template would be flushed from the cache). This was a useful solution for making code run as quickly as possible when executed.

CFMX faces the same dilemma. While the compilation of code to a Java class file eliminates the interpretation step the first time the code is executed after each restart, the fact remains that the compiled code needs to be read from disk and loaded into ColdFusion's memory - in that same template cache that existed in previous releases.

The precompile batch file from the previous release does not perform this "load" of the compiled code into CFMX's template cache, so you can't save that time. Similarly, when the server has just been restarted, it will need to be "loaded" into memory. It's just the first user to run the code after restart (or a compile or precompile) that experiences that load time. So be aware that precompiling won't eliminate the entire wait. There's still the time needed to load the code into the template cache the first time it's executed after restart.

Also, be careful in evaluating the impact of these steps. If you watch the precompile.bat file process, you may notice that compiling even a single template seems to take longer than if you simply let CF compile it automatically. The compiler process in the precompile.bat file has some load time itself on the first template it compiles. If only one template is precompiled, you see that time reflected in the time it takes to compile. If multiple files are precompiled at once, you see each individual template taking less time than if it was compiled alone.

Handling Included Files
In my example above I included a test.html file, but what if I had included a cfm file? Do you wonder if there might be an effect on compilation if code in the included file changes? Would you expect ColdFusion to have to recompile the including template?

It does not. Precompiling the including file after the included file has changed isn't necessary. Indeed, if you look at the compile time reported by the precompile tool it will be reported as 0ms. This reiterates the point that CFINCLUDE is a runtime directive, not a compile time one. Many misconstrue that. (Further proof is demonstrated by the fact that the value of the TEMPLATE attribute naming a file in CFINCLUDE can in fact be a variable. It's clearly a runtime directive.)

What's the difference? Say you have a template that includes another and that included file changes often during the day. You will notice a delay in loading the file. But it would have been true in CF 5 and before. Compilation isn't the cause, and precompilation is not the answer. You just need to get CF to reexecute the template so that it runs the CFINCLUDE to detect that changed file and load it into memory. Again, it's a runtime issue.

I've done this and watched the time reported in the debug output on the first execution of the calling template (after the included file has changed). In one example it went from 1000 ms (reported as execution time for the calling template) to 0ms from then on. Note that this change is in the runtime of the template, not its compile time. So it's not really a compilation issue.

What can you do, especially if some automated process is causing updates to the included file? You could add to that automated process (or create another) that simply executes the calling page once after the included file has changed. This could be kicked off via CFSCHEDULE or CFHTTP. The point isn't to save the resulting output in any way (which is an option with these tags), but merely to cause execution of (or in other words to "touch") the calling page so that it does what it needs to do to interpret the change in the included file.

Where Does the Compilation Go?
I also mentioned in the previous article that CFMX compiles your cfm (and cfc) templates into .class files, which are Java byte code files. If you're curious, the files are written to (and executed from) the cfclasses subdirectory of [cfusionmx]\wwwroot\WEB-INF\ directory where CFMX is installed. This occurs whether you are using another Web server or have located your file outside the default wwwroot location. CFMX compiles and runs the code from this cfclasses directory, regardless of the location of the source file.

This is the reason why in the previous article we needed to specify the -webinf directive in the batch file, to tell it to place compiled code there. If it's not placed there, it won't run in CFMX.

Be warned, though, that the file names for these class files may not be at all apparent. A CF template named Setsession.cfm might lead to a class file named cfsetsession2ecfm1011928409.class.

All templates from all directories end up in this one cfclasses subdirectory. They're not stored here in any subdirectories related to their original location. Instead, CF includes a hash of the directory name in that set of numbers after the file name. Keep that in mind when trying to associate a given class file with its original cfm template. The hashing process is a bit convoluted. (See my blog entry of Oct. 5 for more details, at cfmxplus.blogspot.com.)

Perhaps the easiest way to detect which class file goes with which source file is to simply edit the file and then execute (or precompile) it. Look in the cfclasses directory for the most recently created class file. Assuming your server is not too busy with many compilations taking place, it should be pretty easy to associate the classname with the CF source code name.

Saving Java Source Code Produced by CFMX
So that's where the compiled source code goes. But what about seeing the actual uncompiled Java source code that your CF template is converted into? Normally it's of no concern to CF developers what CFMX is doing under the covers in converting our CFML to Java.

For the ardently curious among you, did you know that you can ask CFMX to save the Java code it creates, in source form? You can. It's an undocumented feature, and while I've had no trouble doing it, I must warn that this is something you do at your own risk. Neither I, nor SYS-CON, nor Macromedia accept responsibility for your trying this.

The setting can only be enabled by someone with administrative control of the server, and the setting is also server-wide. It will add a slight additional time to the compile process, so it's not something you'd want to turn on in production. It probably ought not be left on in development either.

You need to edit the file web.xml in the [cfusionmx]\wwwroot\WEB-INF directory. There, if you're familiar with XML files, you'll find a parameter called "coldfusion.compiler.saveJava". Change its value from false to true. Save the file. Restart the server.

Now, whenever a new or recently edited file is compiled (whether automatically by CFMX or by our precompile.bat file), CFMX will also create a ".java" file along with the ".class" file. This ".java" file will be found in that same [cfusionmx]\wwwroot\WEB-INF\cfclasses\ directory as the ".class" files (and will be subject to that same issue of the curious file naming mentioned above).

The Idea of Deleting the Generated Class Files
Some have proposed that instead of precompiling their code they'd just as soon delete the underlying Java class file that was created when it was last compiled. That may seem like overkill, but there are times when it might be worth trying. Just note that, as the previous sections discussed, finding the class file that's associated with a given source template can be challenging. While some may simply delete all the class files, that's certainly overkill.

Don't forget that in the previous article I mentioned there is a -f directive you can pass to the compile process (by modifying the precompile.bat file). That will force a recompile of a file even if CF doesn't think it's necessary. Sometimes that solves the same problem that deleting the class file would solve.

The Dream of Distributing Compiled Code Without Source
Finally, all this talk of creating compiled Java code has excited some. They wonder if they might finally be able to distribute the compiled code without the associated source files. From an intellectual property protection standpoint, this makes sense. Being able to do so is important to those who sell ColdFusion applications. They want their clients to be able to use the code, without them (or their competitors) seeing the actual source code.

Unfortunately, this is not going to happen with the current release of CFMX (if at all). Macromedia has designed the process so that the source code must be present for the file to be loaded. That's an interesting key point. Some were excited by a demo by Ben Forta at last year's DevCon where he was able to delete a source code file and the template continued to run. Some have even noticed that that can still work. Not always, though. Only if the trusted cache setting is enabled.

With what you know from our previous discussion of loading templates and using the trusted cache setting, you may be able to figure out what's happening. If you run a CF template, it's loaded into memory (after being compiled if it changed). Once it's in memory, that's where CF runs it from. And if the trusted cache setting is enabled, then CFMX no longer looks to the source code file to determine if it's changed. Indeed, it also doesn't look to see if it's been deleted.

But if you restart the server, then CF will try to load the template from disk and even with the trusted cache enabled, the first time the template is run, it will look to see if the CF source template has changed since it was last compiled to disk. Only if they disabled that would we be able to truly distribute the source code.

Macromedia folks have further asserted that they won't enable that feature because it's inappropriate to assume that code compiled on one machine would necessarily work on another machine. It's important to note that your ColdFusion code is more than just the underlying Java byte code it's been compiled into. There are libraries that come with CFMX that are called when your code is run. You would at least need to distribute those as well. Whether Macromedia will address this problem for commercial solution developers is still unclear.

Conclusion
The process of compiling and understanding the compilation process for CFMX templates is something that's not well documented. I don't even want to claim to fully understand it. I've encountered some things in preparing this article that still confound me, and since it's undocumented, maybe we can't expect to fully understand it. I just hope that the information shared here will help you, and that it may also motivate Macromedia to decide what aspects of the process they want to make more public.

You could argue that an Admin interface would be a useful addition in doing the compilation process. Perhaps someone has already started working on that. I welcome comments, questions, and concerns you may have about this compilation process. Feel free to e-mail me or post comments below the online version of this article at the www.sys-con.com site.

SIDEBAR
Trusted Cache Still Applies
I mentioned in the previous article that ColdFusion automatically detects any new or newly edited templates and then automatically compiles them (in CFMX, or interprets them in CF 5 and before). It does this by checking to see if a CF template about to be executed is new or newly updated. And it does this before each invocation of a template, by every user.

That's a rather costly operation, but it's the cost of being able to freely make code changes and see them reflected immediately. If your production environment is designed so that you don't make changes to your code except on a scheduled basis (perhaps overnight), you should enable the "Trusted Cache" setting in the ColdFusion Administrator.

By enabling "Trusted Cache", you're telling ColdFusion that once it loads a program into memory, it should no longer check to see if its underlying source code template has been updated. This will remain in effect until the server is restarted. Sadly, disabling it and re-enabling it does not cause the template cache to be flushed.

One thing that has changed about the "trusted cache" is the specification of its size. Prior to CFMX, the size setting (on the same page in the Administrator) was set in bytes, and it was difficult to calculate an accurate size. Now, the setting is the total number of templates to cache, rather than their size. It still defaults to 1,024.

More Stories By Charlie Arehart

A veteran ColdFusion developer since 1997, Charlie Arehart is a long-time contributor to the community and a recognized Adobe Community Expert. He's a certified Advanced CF Developer and Instructor for CF 4/5/6/7 and served as tech editor of CFDJ until 2003. Now an independent contractor (carehart.org) living in Alpharetta, GA, Charlie provides high-level troubleshooting/tuning assistance and training/mentoring for CF teams. He helps run the Online ColdFusion Meetup (coldfusionmeetup.com, an online CF user group), is a contributor to the CF8 WACK books by Ben Forta, and is frequently invited to speak at developer conferences and user groups worldwide.

Comments (1) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Sven 11/14/02 04:59:00 AM EST
@ThingsExpo Stories
The BPM world is going through some evolution or changes where traditional business process management solutions really have nowhere to go in terms of development of the road map. In this demo at 15th Cloud Expo, Kyle Hansen, Director of Professional Services at AgilePoint, shows AgilePoint’s unique approach to dealing with this market circumstance by developing a rapid application composition or development framework.

ARMONK, N.Y., Nov. 20, 2014 /PRNewswire/ --  IBM (NYSE: IBM) today announced that it is bringing a greater level of control, security and flexibility to cloud-based application development and delivery with a single-tenant version of Bluemix, IBM's platform-as-a-service. The new platform enables developers to build ap...

"BSQUARE is in the business of selling software solutions for smart connected devices. It's obvious that IoT has moved from being a technology to being a fundamental part of business, and in the last 18 months people have said let's figure out how to do it and let's put some focus on it, " explained Dave Wagstaff, VP & Chief Architect, at BSQUARE Corporation, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
The major cloud platforms defy a simple, side-by-side analysis. Each of the major IaaS public-cloud platforms offers their own unique strengths and functionality. Options for on-site private cloud are diverse as well, and must be designed and deployed while taking existing legacy architecture and infrastructure into account. Then the reality is that most enterprises are embarking on a hybrid cloud strategy and programs. In this Power Panel at 15th Cloud Expo (http://www.CloudComputingExpo.com), moderated by Ashar Baig, Research Director, Cloud, at Gigaom Research, Nate Gordon, Director of T...
The Internet of Things is not new. Historically, smart businesses have used its basic concept of leveraging data to drive better decision making and have capitalized on those insights to realize additional revenue opportunities. So, what has changed to make the Internet of Things one of the hottest topics in tech? In his session at @ThingsExpo, Chris Gray, Director, Embedded and Internet of Things, discussed the underlying factors that are driving the economics of intelligent systems. Discover how hardware commoditization, the ubiquitous nature of connectivity, and the emergence of Big Data a...
SYS-CON Events announced today that Windstream, a leading provider of advanced network and cloud communications, has been named “Silver Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Windstream (Nasdaq: WIN), a FORTUNE 500 and S&P 500 company, is a leading provider of advanced network communications, including cloud computing and managed services, to businesses nationwide. The company also offers broadband, phone and digital TV services to consumers primarily in rural areas.
“In the past year we've seen a lot of stabilization of WebRTC. You can now use it in production with a far greater degree of certainty. A lot of the real developments in the past year have been in things like the data channel, which will enable a whole new type of application," explained Peter Dunkley, Technical Director at Acision, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that IDenticard will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. IDenticard™ is the security division of Brady Corp (NYSE: BRC), a $1.5 billion manufacturer of identification products. We have small-company values with the strength and stability of a major corporation. IDenticard offers local sales, support and service to our customers across the United States and Canada. Our partner network encompasses some 300 of the world's leading systems integrators and security s...
DevOps Summit 2015 New York, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that it is now accepting Keynote Proposals. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.
"People are a lot more knowledgeable about APIs now. There are two types of people who work with APIs - IT people who want to use APIs for something internal and the product managers who want to do something outside APIs for people to connect to them," explained Roberto Medrano, Executive Vice President at SOA Software, in this SYS-CON.tv interview at Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Nigeria has the largest economy in Africa, at more than US$500 billion, and ranks 23rd in the world. A recent re-evaluation of Nigeria's true economic size doubled the previous estimate, and brought it well ahead of South Africa, which is a member (unlike Nigeria) of the G20 club for political as well as economic reasons. Nigeria's economy can be said to be quite diverse from one point of view, but heavily dependent on oil and gas at the same time. Oil and natural gas account for about 15% of Nigera's overall economy, but traditionally represent more than 90% of the country's exports and as...
The Internet of Things is a misnomer. That implies that everything is on the Internet, and that simply should not be - especially for things that are blurring the line between medical devices that stimulate like a pacemaker and quantified self-sensors like a pedometer or pulse tracker. The mesh of things that we manage must be segmented into zones of trust for sensing data, transmitting data, receiving command and control administrative changes, and peer-to-peer mesh messaging. In his session at @ThingsExpo, Ryan Bagnulo, Solution Architect / Software Engineer at SOA Software, focused on desi...
"At our booth we are showing how to provide trust in the Internet of Things. Trust is where everything starts to become secure and trustworthy. Now with the scaling of the Internet of Things it becomes an interesting question – I've heard numbers from 200 billion devices next year up to a trillion in the next 10 to 15 years," explained Johannes Lintzen, Vice President of Sales at Utimaco, in this SYS-CON.tv interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
"For over 25 years we have been working with a lot of enterprise customers and we have seen how companies create applications. And now that we have moved to cloud computing, mobile, social and the Internet of Things, we see that the market needs a new way of creating applications," stated Jesse Shiah, CEO, President and Co-Founder of AgilePoint Inc., in this SYS-CON.tv interview at 15th Cloud Expo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events announced today that Gridstore™, the leader in hyper-converged infrastructure purpose-built to optimize Microsoft workloads, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Gridstore™ is the leader in hyper-converged infrastructure purpose-built for Microsoft workloads and designed to accelerate applications in virtualized environments. Gridstore’s hyper-converged infrastructure is the industry’s first all flash version of HyperConverged Appliances that include both compute and storag...
Today’s enterprise is being driven by disruptive competitive and human capital requirements to provide enterprise application access through not only desktops, but also mobile devices. To retrofit existing programs across all these devices using traditional programming methods is very costly and time consuming – often prohibitively so. In his session at @ThingsExpo, Jesse Shiah, CEO, President, and Co-Founder of AgilePoint Inc., discussed how you can create applications that run on all mobile devices as well as laptops and desktops using a visual drag-and-drop application – and eForms-buildi...
We certainly live in interesting technological times. And no more interesting than the current competing IoT standards for connectivity. Various standards bodies, approaches, and ecosystems are vying for mindshare and positioning for a competitive edge. It is clear that when the dust settles, we will have new protocols, evolved protocols, that will change the way we interact with devices and infrastructure. We will also have evolved web protocols, like HTTP/2, that will be changing the very core of our infrastructures. At the same time, we have old approaches made new again like micro-services...
Code Halos - aka "digital fingerprints" - are the key organizing principle to understand a) how dumb things become smart and b) how to monetize this dynamic. In his session at @ThingsExpo, Robert Brown, AVP, Center for the Future of Work at Cognizant Technology Solutions, outlined research, analysis and recommendations from his recently published book on this phenomena on the way leading edge organizations like GE and Disney are unlocking the Internet of Things opportunity and what steps your organization should be taking to position itself for the next platform of digital competition.
The 3rd International Internet of @ThingsExpo, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that its Call for Papers is now open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
As the Internet of Things unfolds, mobile and wearable devices are blurring the line between physical and digital, integrating ever more closely with our interests, our routines, our daily lives. Contextual computing and smart, sensor-equipped spaces bring the potential to walk through a world that recognizes us and responds accordingly. We become continuous transmitters and receivers of data. In his session at @ThingsExpo, Andrew Bolwell, Director of Innovation for HP's Printing and Personal Systems Group, discussed how key attributes of mobile technology – touch input, sensors, social, and ...