Welcome!

You will be redirected in 30 seconds or close now.

ColdFusion Authors: Yakov Fain, Maureen O'Gara, Nancy Y. Nee, Tad Anderson, Daniel Kaar

Related Topics: ColdFusion

ColdFusion: Article

Search Terms Using a Reporting CFC

What Are They Looking For?

If I ran a candle shop and my visitors began to ask for blue-and-white-striped candles, you can bet I'd move whatever stock of blue-and-white-stripers I had into a prominent position, and I'd be sure that I had a good inventory of blue-and-white-striped candles on hand and on order. I would be avidly reading CSDJ (Candle Store Developer's Journal) every month, of course, and keeping abreast of market trends, for who would be fool enough to ignore what the customers themselves were asking for?

We all do the same thing with our Web sites, whether they exist for e-commerce or to provide information about a business or organization and its services and activities. I'm sure most Webmasters spend as much time as I do poring over logs and referrer reports, paying particular attention to which search terms or keywords visitors have used on Google and other search engines to arrive at their sites. The sites that I manage vary in size and in traffic, but are all of a scale such that they happily reside on the hosting company servers. I have about a dozen sites at CrystalTech, one of several hosting companies that pay particular attention to accommodating ColdFusion programmers. Such companies typically also provide a robust statistical reporting package as well as access to raw logs that allow you to do as much rooting around as you wish.

On many of my sites, there are areas with internal search facilities, employing either SQL-match searching, or, more likely, the Verity search features that are offered with ColdFusion MX. For these sites, I have created a ColdFusion Component (CFC) that collects the search terms used by visitors in my Verity search forms and generates a report of which search terms have been used, either upon request or on an automated schedule.

A Word on Verity
You'll note that I've titled the CFC discussed in this article "verityreport.cfc," even though there is nothing about the component that is inherently tied to Verity. Indeed, the component could be used entirely to track SQL searches. What my title betrays, I think, is how much I value Verity and the extent to which, when I think "search" in a ColdFusion application, I am quite likely thinking of employing Verity. That's because the features of Verity that are included with ColdFusion MX provide a range of powerful search options for CF Web sites. Developers can create index collections defined by URL paths in a domain, or defined by one or more fields in a database; searches can be made against these indices, either in isolation or in combination. The collections contain fields that allow parameters to be passed to the search results. The parameters then allow those results to display meaningful titles or summaries, as well as to include links to detail templates based on a primary key or other identifier. Plus, the search algorithm provides site visitors with an interface that is somewhat similar to what they experience in popular search engines, without requiring the programmer to supply the search logic, as would be the case with a masked SQL search.

That's already pretty inviting, but we're only describing the basics of Verity's powers in the current version of ColdFusion MX (version 6.1). The little that has been publicly discussed about what Verity may add in Blackstone is very intriguing. Macromedia ColdFusion product manager Tim Buntel's Weblog (www.buntel.com/blog/index.cfm) has suggested possibilities such as the ability to have categorized and context-driven searches and results, the ability to search within the context of a prior search, more customization, spelling suggestions for searches, and, generally, more intelligent searching.

This means that, using Blackstone, CF developers will likely be able to offer site-internal searches that are reasonably equivalent to popular search engines, which have a level of sophistication that site visitors have now come to expect. CF's powerful Verity search tool will become even more powerful and useful for site visitors.

First Things First
I mentioned that my Web site workload revolves around smaller-sized sites that reside on hosting company servers. ColdFusion Components are just as valuable for code reuse in that context as they are for developers serving multiple sites that call CFCs stored in a single directory. Write the CFC once, and drop it into the appropriately mapped directory of any hosted site that needs it. (By the way, you can get a good grounding in the principles of CFCs from the excellent series of articles Jeff Houser has recently been writing for CFDJ. Highly recommended!)

When I began writing this CFC, which I call "verityreport.cfc" (see Listing 1 - all listings for this article can be downloaded from www.sys.con.com/coldfusion/sourcec.cfm). I wanted a component that would make sense for a variety of my Web sites with search facilities. The basic functions would be (1) to collect information about which search terms site visitors used and (2) to report what those search terms were, for a given period of time. I also wanted to be able to use the CFC without much additional programming, particularly for smaller sites that might or might not have a client's administrative back end or content management system (CMS). I decided to add a presentation method that would generate and e-mail a weekly report to the appropriate parties, as well as provide ready-made reporting when a client's administrative interface does actually exist.

The first step was to create a database table to store the search terms entered by visitors (note that throughout this article, and the accompanying code listings, the database calls are for MySQL; with little or no modification, they can be made to serve other databases). I created both a field for the search term and a date/time field for the date and time when the search term was entered by a user, so that I could report the terms according to a given set of date parameters. Also, I added an auto-incremented primary key field.

I call the table "tbl_searchterms," and the three fields are "searchterm_id," "thesearchterm," and "thesearchdate."

The first method of the CFC, named "collect SearchTerm," is the one called to write a search term and the time it was called to the database. But, before we look at that, let's take a peek at my typical Application.cfm template, which sets some of the values for arguments that will be called in the CFC. The more I use custom tags and CFCs, the more each of my sites' Application.cfm templates is likely to have parameter-setting lines like this:


<CFSET request.dsn = "mydsn">
<CFSET request.dsnusername = "mydsnusername">
<CFSET request.dsnpassword = "mydsnpassword">
<CFSET request.mailto = "[email protected]">
<CFSET request.mailfrom = "[email protected]">
<CFSET request.thissite= "mysite.com">
<CFSET request.mailserver = "mail.mysite.com">

The "collectSearchTerms" method uses those request variables to call the database for the insert of the search term and the time it was called into the database and uses the variable passed in when the method was called to populate the field "thesearchterm." The date, as you can see, is drawn from the component's one line of constructor code (constructor code in a component is the code that precedes any of the individual methods' code, and may be used by one or more of the methods of the component).

This recording method is called with the following code, placed at the point in the site's search facility where the visitor's search term is collected for use against the collection (or database, if it is an SQL search rather than a Verity collection search that is being executed). The following code will typically be found on the action page that lists search results.


		<CFOBJECT 
		component="mycfcmapping.verityreport" 
		name="verityreportcfc">
		<CFINVOKE 
		component="#verityreportcfc#"
		method="collectSearchTerm">
		<CFINVOKEARGUMENT 
			name="thesearchterm"
			value="#session.keyword#">
		</CFINVOKE>

The variable "thesearchterm" can be derived from a session variable, as in this example, or from a client or other variable, as dictated by the particular methodology you use to produce a page of search results.

This simple function of the component, invoked from whatever points your site visitors are entering their search terms into, creates a database of search terms associated with the times at which they were entered.

Returning the Data
The second method, "getSearchTerms," gathers data according to the two parameters that set the date range we want our report to encompass. This method also supplies default values, drawn from the current time that we set in the constructor code, and, once again, utilizes the datasource request variables we set in the Applicaton.cfm template.

The query that is returned by this method limits the search by start and end dates (defaulting to the most recent week if no arguments are passed in for those values), and orders instances of "thesearchterm," first by the highest frequency of occurrence and then alphabetically.

It would be a simple matter to call this method, passing in dates if we wished to use a span other than the default and displaying the query results in whatever way we wished. For the manner in which I wanted to use this component, however, I didn't want to have to create any additional code in order for the component to serve its most basic use. Therefore, I created a presentation method that could run without any additional parameters having to be passed in, or could be used in a bit of a fancier way if it were to be included in an administrative back end used by a client.

The third method, "displayMailReport," can be run with this simple invocation (most likely run automatically through CFSCHEDULE or some other server scheduler set to run it once a week):


<><CFOBJECT
component="mycfcmapping.verityreport"
name="verityreportcfc">
<CFINVOKE
component="#verityreportcfc#"
method="displaymailreport">

The "displayMailReport" method called in this manner will e-mail the ordered report once a week to the e-mail address specified as "request.mailto" in the Application.cfm template. Or, it may be called with a form that asks for more detail, allowing for a user-specified date range as well as one or more user-specified e-mail recipients. Note one very handy feature of a method that invokes another method within a component: you can pass all the arguments wholesale from the calling method to the called method with this single parameter:

argumentcollection="#ARGUMENTS#".

Listing 2 provides an example of calling the "displayMailReport" method through a form. A screenshot of the report it can generate - ---in this case showing some recent search terms used on my blog, poorlcio.com - is shown in Figure 1.

It Could Be So Much Better!
Since I created this CFC, I have found it very useful to receive the weekly generated e-mails with the reports of search terms used by my various sites' visitors. Ihave also found it handy to use the form version in Listing 2 to generate screen reports or to send e-mail reports to myself or others.

But, like a lot of you, the second I make a mental note that something is done, I immediately start thinking of all the ways I could improve it. A future version of this CFC and its implementation would probably expand the database to include a field identifying which template in a site with multiple search blanks had generated this particular search, and to return that information as part of the reporting. Also, I'm already tinkering with a related CFC that will record and report which database records have been the most frequently called into template pages that are populated from databases - nothing to do with Verity or searches, but certainly information that tells us what our site visitors have been looking for and utilizing.

When we get an even more enriched version of Verity with Blackstone, I'll no doubt be taking another look at this CFC to mine the value of the new Verity features - not only for site visitors when they explore our sites, but also for us developers as we observe their paths through our sites. If this leads me to add a Verity-specific feature or two to my CFC , it would finally live up to its name.

More Stories By James Edmunds

James Edmunds is a freelance Internet developer and arts administration consultant living in New Iberia, Louisiana. After a career in journalism that included writing for national publications such as Newsweek and serving as editor for an alternative weekly newspaper he founded in southern Louisiana, James began to pursue a second career working with arts groups. Though he had no technology background, his interest in harnessing the power of the Internet to serve the interests of the arts led him into Internet development, an arena in which he has now gone beyond the arts to serve a general business clientele.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
One of the bewildering things about DevOps is integrating the massive toolchain including the dozens of new tools that seem to crop up every year. Part of DevOps is Continuous Delivery and having a complex toolchain can add additional integration and setup to your developer environment. In his session at @DevOpsSummit at 18th Cloud Expo, Miko Matsumura, Chief Marketing Officer of Gradle Inc., will discuss which tools to use in a developer stack, how to provision the toolchain to minimize onboa...
With an estimated 50 billion devices connected to the Internet by 2020, several industries will begin to expand their capabilities for retaining end point data at the edge to better utilize the range of data types and sheer volume of M2M data generated by the Internet of Things. In his session at @ThingsExpo, Don DeLoach, CEO and President of Infobright, will discuss the infrastructures businesses will need to implement to handle this explosion of data by providing specific use cases for filte...
SYS-CON Events announced today that Pythian, a global IT services company specializing in helping companies adopt disruptive technologies to optimize revenue-generating systems, has been named “Bronze Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. Founded in 1997, Pythian is a global IT services company that helps companies compete by adopting disruptive technologies such as cloud, Big Data, advanced analytics, and DevO...
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Avere delivers a more modern architectural approach to storage that doesn’t require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbuilding of data centers ...
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management...
The Quantified Economy represents the total global addressable market (TAM) for IoT that, according to a recent IDC report, will grow to an unprecedented $1.3 trillion by 2019. With this the third wave of the Internet-global proliferation of connected devices, appliances and sensors is poised to take off in 2016. In his session at @ThingsExpo, David McLauchlan, CEO and co-founder of Buddy Platform, will discuss how the ability to access and analyze the massive volume of streaming data from mil...
SYS-CON Events announced today that Alert Logic, Inc., the leading provider of Security-as-a-Service solutions for the cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Alert Logic, Inc., provides Security-as-a-Service for on-premises, cloud, and hybrid infrastructures, delivering deep security insight and continuous protection for customers at a lower cost than traditional security solutions. Ful...
SYS-CON Events announced today that Interoute, owner-operator of one of Europe's largest networks and a global cloud services platform, has been named “Bronze Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. Interoute is the owner-operator of one of Europe's largest networks and a global cloud services platform which encompasses 12 data centers, 14 virtual data centers and 31 colocation centers, with connections to 195 ad...
SYS-CON Events announced today that Men & Mice, the leading global provider of DNS, DHCP and IP address management overlay solutions, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. The Men & Mice Suite overlay solution is already known for its powerful application in heterogeneous operating environments, enabling enterprises to scale without fuss. Building on a solid range of diverse platform support,...
Fortunately, meaningful and tangible business cases for IoT are plentiful in a broad array of industries and vertical markets. These range from simple warranty cost reduction for capital intensive assets, to minimizing downtime for vital business tools, to creating feedback loops improving product design, to improving and enhancing enterprise customer experiences. All of these business cases, which will be briefly explored in this session, hinge on cost effectively extracting relevant data from ...
WebSocket is effectively a persistent and fat pipe that is compatible with a standard web infrastructure; a "TCP for the Web." If you think of WebSocket in this light, there are other more hugely interesting applications of WebSocket than just simply sending data to a browser. In his session at 18th Cloud Expo, Frank Greco, Director of Technology for Kaazing Corporation, will compare other modern web connectivity methods such as HTTP/2, HTTP Streaming, Server-Sent Events and new W3C event APIs ...
There will be new vendors providing applications, middleware, and connected devices to support the thriving IoT ecosystem. This essentially means that electronic device manufacturers will also be in the software business. Many will be new to building embedded software or robust software. This creates an increased importance on software quality, particularly within the Industrial Internet of Things where business-critical applications are becoming dependent on products controlled by software. Qua...
SYS-CON Events announced today that iDevices®, the preeminent brand in the connected home industry, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. iDevices, the preeminent brand in the connected home industry, has a growing line of HomeKit-enabled products available at the largest retailers worldwide. Through the “Designed with iDevices” co-development program and its custom-built IoT Cloud Infrastruc...
Companies can harness IoT and predictive analytics to sustain business continuity; predict and manage site performance during emergencies; minimize expensive reactive maintenance; and forecast equipment and maintenance budgets and expenditures. Providing cost-effective, uninterrupted service is challenging, particularly for organizations with geographically dispersed operations.
Join us at Cloud Expo | @ThingsExpo 2016 – June 7-9 at the Javits Center in New York City and November 1-3 at the Santa Clara Convention Center in Santa Clara, CA – and deliver your unique message in a way that is striking and unforgettable by taking advantage of SYS-CON's unmatched high-impact, result-driven event / media packages.
As enterprises work to take advantage of Big Data technologies, they frequently become distracted by product-level decisions. In most new Big Data builds this approach is completely counter-productive: it presupposes tools that may not be a fit for development teams, forces IT to take on the burden of evaluating and maintaining unfamiliar technology, and represents a major up-front expense. In his session at @BigDataExpo at @ThingsExpo, Andrew Warfield, CTO and Co-Founder of Coho Data, will dis...
Silver Spring Networks, Inc. (NYSE: SSNI) extended its Internet of Things technology platform with performance enhancements to Gen5 – its fifth generation critical infrastructure networking platform. Already delivering nearly 23 million devices on five continents as one of the leading networking providers in the market, Silver Spring announced it is doubling the maximum speed of its Gen5 network to up to 2.4 Mbps, increasing computational performance by 10x, supporting simultaneous mesh communic...
Eighty percent of a data scientist’s time is spent gathering and cleaning up data, and 80% of all data is unstructured and almost never analyzed. Cognitive computing, in combination with Big Data, is changing the equation by creating data reservoirs and using natural language processing to enable analysis of unstructured data sources. This is impacting every aspect of the analytics profession from how data is mined (and by whom) to how it is delivered. This is not some futuristic vision: it's ha...
The cloud promises new levels of agility and cost-savings for Big Data, data warehousing and analytics. But it’s challenging to understand all the options – from IaaS and PaaS to newer services like HaaS (Hadoop as a Service) and BDaaS (Big Data as a Service). In her session at @BigDataExpo at @ThingsExpo, Hannah Smalltree, a director at Cazena, will provide an educational overview of emerging “as-a-service” options for Big Data in the cloud. This is critical background for IT and data profes...
With the Apple Watch making its way onto wrists all over the world, it’s only a matter of time before it becomes a staple in the workplace. In fact, Forrester reported that 68 percent of technology and business decision-makers characterize wearables as a top priority for 2015. Recognizing their business value early on, FinancialForce.com was the first to bring ERP to wearables, helping streamline communication across front and back office functions. In his session at @ThingsExpo, Kevin Roberts...