Click here to close now.

Welcome!

You will be redirected in 30 seconds or close now.

ColdFusion Authors: Yakov Fain, Maureen O'Gara, Nancy Y. Nee, Tad Anderson, Daniel Kaar

Related Topics: ColdFusion

ColdFusion: Article

COSMOS: Managing the ColdFusion Experience

COSMOS: Managing the ColdFusion Experience

Far too often we listen to the naysayers who tell us that something can't be done and give poorly founded reasons as to why our troubles persist. The ColdFusion Application Server is no exception to their folly. If you ask people for the drawbacks of ColdFusion, most will reply "speed" or "stability." Let me be the first to tell you that it does not have to be that way.

The system described in this article was built to change the way we think about our code and applications. COSMOS was designed to change our perceptions of the ColdFusion Application Server and to enhance the ColdFusion experience.

If you have ever looked in the /cfusion/log/ directory you've probably seen one or more of the many ColdFusion-generated error/information logs. These text files can easily grow to hundreds of MB and contain the best indicators of "what happened." As with any other service or application, a regular review of system logs should be a part of normal administration. Unfortunately, because of their large size and the fact that the data is segmented into so many logs, it's difficult to get a complete picture of performance, problems, and failure.

Developers who work on a dedicated server can use the ColdFusion Administrator to view these logs. This can be accomplished by clicking on "Log Files" and then downloading the entire log via a browser. Unfortunately, this is usually not possible given the size of most logs and the remote connection speed.

For shared developers, the critical information is unavailable due to the nature of the shared environment and security. In most cases, developers know only what a site user tells them or what they trap using CFTRY/CFCATCH and CFERROR. Even with these mechanisms in place, the larger picture is unavailable and the majority of performance issues go unnoticed and unattended.

The above issues hinder administrators and developers alike. The result is:

  • No true time or site correlation for ColdFusion Application Server events.
  • Time is wasted attempting to data mine text.
  • Site administrators, developers, and business owners don't know there is a problem.
  • A negative stigma is created based on a lack of timely and organized information.

    To be successful, a solution must have several characteristics:

  • Run autonomously, centrally, and constantly
  • Contain error lookup with "clean code" examples
  • Return all logs and bounced e-mails
  • Bring symmetry to the generated data through trending, aggregation, and normalization
  • Be fast without affecting performance on the managed server

    The solution is COSMOS. Written mainly with Cold-Fusion, it's an integration of ASP, DOS, Perl, ADSI, and Call-XML. It's a remote management platform that leverages the file system, registry, metabase, service controls, and performance counters. Currently, COSMOS contains over 16-million server events aggregated into an MS-SQL database. Captured within a maximum of 40 seconds, these events include all of the following:

    • Application errors
    • CF Application Server stop/starts
    • Hung threads
    • Long-running templates
    • Missing templates
    • Scheduled task results
    • Undeliverable e-mails
    • Mail sent
    How does this affect you? By returning timely, accurate, and relevant information, a developer can see immediately where improvement is needed. At Hostcentric, it's now possible to access several views, graphs, and aggregations that provide a new perspective on the ColdFusion experience.

    There are over 20 COSMOS reports available to a dedicated client, most of which are also available for shared customers. The following is a list of some reports with a brief description of how they impact the development and maintenance cycle.

    Information Listings
    There are several listings available, each with similar characteristics. A listing allows the user to select the maximum number of records to view per screen and how far back to examine data. It also allows the user to progress backward from that point to review previous messages. The majority of listings allow filtering to a single IIS root. They also provide direct access to the complete original error and a corrected code context lookup.

    General Application Error Listing
    Application errors are the best view into the progress and developmental completeness of a site (see Figure 1). A well-coded site generates no application errors. This listing provides a top-down view of the most recent application errors for all IIS roots. By clicking on the error message on the right, a popup window displays the error message as displayed to a site visitor.

    General Missing Template
    This applies to all .cfm templates requested by the Web server, but not found. In most cases, the developer doesn't even know that people are getting "404 File Not Found" messages. If a search engine indexes your site or a user bookmarks a page, a change in the site causes missed business. The solution is to use the default missing template handler in ColdFusion Administrator or to add a CFERROR TYPE="REQUEST" in your site's Application.cfm.

    Long-Running Template Listing
    This applies to the processing time for pages that take longer than expected. The determination of how long is too long is configured in the logging/settings section of ColdFusion Administrator. A typical setting is 45 seconds, though anything taking that long would most likely be canceled or ignored by the calling client. In addition, a script running for 45 seconds could help identify a performance bottleneck for the application server. By default, CF Administrator doesn't enable this counter. Over time, a development team should ratchet this value as low as possible to get the best diagnostics.

    Undeliverable CFMAIL Listing
    When ColdFusion is unable to deliver a message to the server specified in a CFMAIL script, the original template is renamed and filed in the /cfusion/ mail/undeliver/ directory. An error message is also written to the Mail.log or Error.log describing the problem that prevents proper delivery. This listing binds those two pieces of information together.

    The following popup allows an administrator to correct and resend the message from the original server. This function is indispensable for any business that relies on CFMAIL to reliably carry e-mail, and can't accept undelivered messages.

    Hung Thread Listing
    This is probably the greatest indicator of a performance and stability problem. Hung threads are ColdFusion's method of alerting us that it was unable to completely process the requested template. This is usually the result of code or database issues. CF4.x and above has an option in the Administrator to have CF "restart at x unresponsive requests."

    When the hung thread count matches the defined threshold, ColdFusion reaches a critical point and will stop/restart itself to avoid excessive downtime. Constant examination of hung threads is necessary to avoid application server failure. At the end of this article I've included three links that help to define more fully the causes of hung threads.

    Scheduled Task Listing
    Most scheduled tasks run completely unnoticed until someone realizes that a critical function has not processed in days. This listing is not much to look at but, under the hood, a huge modification and improvement has been created for the executive service.

    As always, COSMOS can determine if your task started, succeeded, or failed based on the logs. Furthermore, COSMOS will allow you to define a target string in the page HTML and record the generated content from the target URL to the database. If a scheduled task does not return the defined string, an e-mail containing the content and diagnostics can be generated at the time of failure. In addition, the actual HTTP response (CFHTTP.FILECONTENT) is zipped and written to the database.

    Aggregation and Stratification
    More commonly called a GROUPING, the next series of graphs were created to help identify the greatest problems quickly. By examining the data based on time, date, and IIS root, we can gather a greater understanding of where faults exist.

    Application Log Stratification by IIS Root
    Over a selectable time span, this graph allows you to see which sites are having the greatest incidence of errors (see Figure 2). By clicking on the blue horizontal bar on the right, you're driven back to the general application error listing but with an additional sort parameter that isolates errors created by the target root.

    Time/Error Graph
    Especially useful in determining if your day is getting better or worse, this graph breaks down the server errors by 10 minute increments over a selectable date span. This is often used to diagnose a recurring failure point over a multiple day or week period.

    Application Errors Stratified by Date
    Similar to the previous idea, this graph groups the number of errors by the date that they occurred (see Figure 3). This helps to identify programming trends and can easily indicate a "bad day" for an application. By clicking on the blue bar, your browser is taken to the application log stratification by IIS root. Clicking on the "Time Graph" button brings you to the next graph.

    Long-Running Template Aggregation by IIS Root
    Similar to the previous root aggregations, this has several prominent exceptions. Because a long-running page has a value associated with the processing time, I've included a column for the sum and average values. Using this display, it's possible to extract the templates most often run beyond acceptable limits, thus demanding the greatest processing time. This affects performance, though not necessarily a failure, and is a fantastic indicator of templates that need to be addressed before they become a stability issue.

    Hung Thread Aggregation by IIS Root
    This graph will often tell which application is responsible for killing the server. Over a selectable data span you can easily see which sites are causing CF to lose processing threads and tie up resources. The blue horizontal bar links back to the hung thread listing for a given root.

    The Big and the Bad
    With the thousands of errors, tasks, and events returned each hour, it's easy to become overwhelmed. In addition, not all errors have the same weight on the application server or urgency to a business owner. To resolve this problem a system of alerts and probes runs in the background. Operating at several intervals, the most relevant problems are quickly pulled out of the pool and matched with a type and severity. Once a probe identifies an event or error candidate, the alert has an option of paging, e-mailing, or calling (using CallXML) the administrator.

    A Final Look
    When did your application server last crash and why?

  • Event Chronology: As the first view that brings together data from multiple sources (see Figure 4), it provides a chronological view of all application errors, hung threads, long-running templates, and application server failures. The graph threads events are based on time in order to provide a trace leading up to a failure.
  • Spectral Analysis: This graph is unique because it rapidly identifies problems that would otherwise slip under the wire (see Figure 5). The three colors representing CF stops (red), starts (green), and hung threads (purple) are graphed relative to a 24-hour time line. By viewing all hung threads that led to server failures, a complete understanding of the root performance issues is garnered.

    Summary
    Tonight at 1 a.m. your database is going to run out of space and begin throwing application errors. Maybe your mail server stops relaying your order confirmations. There are a thousand permutations to a preventable and containable failure. Will your customers be the first to let you know?

    In the end, owners and developers, shared and dedicated, all have the same concerns: stability and performance. Using this system makes that realization no more than a few seconds away.

    Related Articles

  • http://allaire.com/Handlers/index.cfm?ID=8627&Method=Full
  • http://allaire.com/Handlers/index.cfm?ID=2497&Method=Full
  • http://allaire.com/Handlers/index.cfm?ID=1505&Method=Full
  • http://allaire.com/Handlers/index.cfm?ID=1540&Method=Full
  • http://support.microsoft.com/support/kb/articles/q174/4/96.asp
  • More Stories By Tim Nettleton

    Tim Nettleton is a senior engineer at Hostcentric’s Orlando division. He has worked with ColdFusion for four years and recently spoke at DevCon 2001 in Orlando, Florida.

    Comments (5) View Comments

    Share your thoughts on this story.

    Add your comment
    You must be signed in to add a comment. Sign-in | Register

    In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


    Most Recent Comments
    Craig Rosenblum 05/29/03 02:24:00 PM EDT

    I am not interested in the whole system mainly right now, the log component.

    It is a good idea, to help manage logs, and make sure we are really on top of errors.

    And it does look nice as well.

    jurgen koch 03/11/02 04:48:00 PM EST

    ok...let me retract the attitude from my immediately previous post and clarify the situation with more fact that flame.

    I contacted the articles author, Timothy Nettleton ([email protected]), who was very courteous and replied almost immediately to explain that while Cosmos will probably not be released for sale by Hostcentric, they can provide you remote access to Cosmos through its web interface at cosmos.hostcentric.net

    You will have to work out details re: how Hostcentric will obtain access to your CF log files for parsing, etc. but the good news is that Cosmos is available to the public...and the pricing is very reasonable.

    embarrased by my previous rant,
    Jurgen

    jurgen koch 03/11/02 01:19:00 PM EST

    I contacted Hostcentric (the ISP that the author of the article worked for) and the impression that I got was that
    COSMOS is only available to their colocation, etc. clients, and while they did develop the application, they have no plans to release it to the public.

    Kinda makes me wonder why CFDJ published the article at all. Just to tease CF Administrators?

    Hey CFDJ, want to pay me to write an article about the really cool CF apps I have developed, but can't/won't sell, or release code, logic, or the application to the public? Thanks for nothing.

    Greg Correll 02/15/02 12:55:00 PM EST

    Is cosmos available to the public?

    Matt McDonald 01/17/02 12:16:00 AM EST

    Is cosmos available to the public?

    @ThingsExpo Stories
    The WebRTC Summit 2014 New York, to be held June 9-11, 2015, at the Javits Center in New York, NY, announces that its Call for Papers is open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 16th International Cloud Expo, @ThingsExpo, Big Data Expo, and DevOps Summit.
    SYS-CON Events announced today that Cisco, the worldwide leader in IT that transforms how people connect, communicate and collaborate, has been named “Gold Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Cisco makes amazing things happen by connecting the unconnected. Cisco has shaped the future of the Internet by becoming the worldwide leader in transforming how people connect, communicate and collaborate. Cisco and our partners are building the platform for the Internet of Everything by connecting the...
    15th Cloud Expo, which took place Nov. 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA, expanded the conference content of @ThingsExpo, Big Data Expo, and DevOps Summit to include two developer events. IBM held a Bluemix Developer Playground on November 5 and ElasticBox held a Hackathon on November 6. Both events took place on the expo floor. The Bluemix Developer Playground, for developers of all levels, highlighted the ease of use of Bluemix, its services and functionality and provide short-term introductory projects that developers can complete between sessions.
    SYS-CON Events announced today that robomq.io will exhibit at SYS-CON's @ThingsExpo, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. robomq.io is an interoperable and composable platform that connects any device to any application. It helps systems integrators and the solution providers build new and innovative products and service for industries requiring monitoring or intelligence from devices and sensors.
    Temasys has announced senior management additions to its team. Joining are David Holloway as Vice President of Commercial and Nadine Yap as Vice President of Product. Over the past 12 months Temasys has doubled in size as it adds new customers and expands the development of its Skylink platform. Skylink leads the charge to move WebRTC, traditionally seen as a desktop, browser based technology, to become a ubiquitous web communications technology on web and mobile, as well as Internet of Things compatible devices.
    The list of ‘new paradigm’ technologies that now surrounds us appears to be at an all time high. From cloud computing and Big Data analytics to Bring Your Own Device (BYOD) and the Internet of Things (IoT), today we have to deal with what the industry likes to call ‘paradigm shifts’ at every level of IT. This is disruption; of course, we understand that – change is almost always disruptive.
    WebRTC is an up-and-coming standard that enables real-time voice and video to be directly embedded into browsers making the browser a primary user interface for communications and collaboration. WebRTC runs in a number of browsers today and is currently supported in over a billion installed browsers globally, across a range of platform OS and devices. Today, organizations that choose to deploy WebRTC applications and use a host machine that supports audio through USB or Bluetooth can use Plantronics products to connect and transit or receive the audio associated with the WebRTC session.
    Docker is an excellent platform for organizations interested in running microservices. It offers portability and consistency between development and production environments, quick provisioning times, and a simple way to isolate services. In his session at DevOps Summit at 16th Cloud Expo, Shannon Williams, co-founder of Rancher Labs, will walk through these and other benefits of using Docker to run microservices, and provide an overview of RancherOS, a minimalist distribution of Linux designed expressly to run Docker. He will also discuss Rancher, an orchestration and service discovery platf...
    SYS-CON Events announced today that Alert Logic, the leading provider of Security-as-a-Service solutions for the cloud, has been named “Bronze Sponsor” of SYS-CON's 16th International Cloud Expo® and DevOps Summit 2015 New York, which will take place June 9-11, 2015, at the Javits Center in New York City, NY, and the 17th International Cloud Expo® and DevOps Summit 2015 Silicon Valley, which will take place November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
    Sonus Networks introduced the Sonus WebRTC Services Solution, a virtualized Web Real-Time Communications (WebRTC) offer, purpose-built for the Cloud. The WebRTC Services Solution provides signaling from WebRTC-to-WebRTC applications and interworking from WebRTC-to-Session Initiation Protocol (SIP), delivering advanced real-time communications capabilities on mobile applications and on websites, which are accessible via a browser.
    SYS-CON Events announced today that Aria Systems, the leading innovator in recurring revenue, has been named “Bronze Sponsor” of SYS-CON's @ThingsExpo, which will take place on June 9–11, 2015, at the Javits Center in New York, NY. Proven by the world’s most demanding enterprises, including AAA NCNU, Constant Contact, Falck, Hootsuite, Pitney Bowes, Telekom Denmark, and VMware, Aria helps enterprises grow their recurring revenue businesses. With Aria’s end-to-end active monetization platform, global brands can get to market faster with a wider variety of products and services, while maximizin...
    SYS-CON Media announced today that @WebRTCSummit Blog, the largest WebRTC resource in the world, has been launched. @WebRTCSummit Blog offers top articles, news stories, and blog posts from the world's well-known experts and guarantees better exposure for its authors than any other publication. @WebRTCSummit Blog can be bookmarked ▸ Here @WebRTCSummit conference site can be bookmarked ▸ Here
    Wearable technology was dominant at this year’s International Consumer Electronics Show (CES) , and MWC was no exception to this trend. New versions of favorites, such as the Samsung Gear (three new products were released: the Gear 2, the Gear 2 Neo and the Gear Fit), shared the limelight with new wearables like Pebble Time Steel (the new premium version of the company’s previously released smartwatch) and the LG Watch Urbane. The most dramatic difference at MWC was an emphasis on presenting wearables as fashion accessories and moving away from the original clunky technology associated with t...
    SYS-CON Events announced today that Vitria Technology, Inc. will exhibit at SYS-CON’s @ThingsExpo, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Vitria will showcase the company’s new IoT Analytics Platform through live demonstrations at booth #330. Vitria’s IoT Analytics Platform, fully integrated and powered by an operational intelligence engine, enables customers to rapidly build and operationalize advanced analytics to deliver timely business outcomes for use cases across the industrial, enterprise, and consumer segments.
    SYS-CON Events announced today that Solgenia will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY, and the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Solgenia is the global market leader in Cloud Collaboration and Cloud Infrastructure software solutions. Designed to “Bridge the Gap” between Personal and Professional Social, Mobile and Cloud user experiences, our solutions help large and medium-sized organizations dr...
    SYS-CON Events announced today that Liaison Technologies, a leading provider of data management and integration cloud services and solutions, has been named "Silver Sponsor" of SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York, NY. Liaison Technologies is a recognized market leader in providing cloud-enabled data integration and data management solutions to break down complex information barriers, enabling enterprises to make smarter decisions, faster.
    Connected devices and the Internet of Things are getting significant momentum in 2014. In his session at Internet of @ThingsExpo, Jim Hunter, Chief Scientist & Technology Evangelist at Greenwave Systems, examined three key elements that together will drive mass adoption of the IoT before the end of 2015. The first element is the recent advent of robust open source protocols (like AllJoyn and WebRTC) that facilitate M2M communication. The second is broad availability of flexible, cost-effective storage designed to handle the massive surge in back-end data in a world where timely analytics is e...
    SYS-CON Events announced today that Akana, formerly SOA Software, has been named “Bronze Sponsor” of SYS-CON's 16th International Cloud Expo® New York, which will take place June 9-11, 2015, at the Javits Center in New York City, NY. Akana’s comprehensive suite of API Management, API Security, Integrated SOA Governance, and Cloud Integration solutions helps businesses accelerate digital transformation by securely extending their reach across multiple channels – mobile, cloud and Internet of Things. Akana enables enterprises to share data as APIs, connect and integrate applications, drive part...
    SYS-CON Events announced today that CommVault has been named “Bronze Sponsor” of SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY, and the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. A singular vision – a belief in a better way to address current and future data management needs – guides CommVault in the development of Singular Information Management® solutions for high-performance data protection, universal availability and sim...
    Cloud is not a commodity. And no matter what you call it, computing doesn’t come out of the sky. It comes from physical hardware inside brick and mortar facilities connected by hundreds of miles of networking cable. And no two clouds are built the same way. SoftLayer gives you the highest performing cloud infrastructure available. One platform that takes data centers around the world that are full of the widest range of cloud computing options, and then integrates and automates everything. Join SoftLayer on June 9 at 16th Cloud Expo to learn about IBM Cloud's SoftLayer platform, explore se...