Welcome!

You will be redirected in 30 seconds or close now.

ColdFusion Authors: Yakov Fain, Jeremy Geelan, Maureen O'Gara, Nancy Y. Nee, Tad Anderson

Related Topics: ColdFusion

ColdFusion: Article

A Cure for Arachnophobia

A Cure for Arachnophobia

Barely a week goes by without someone asking me about ColdFusion and search engine-friendly URLs. This is one of those topics that ColdFusion developers have been discussing for a long time - I first started a thread on this subject on the Allaire Developer's Forum close to five years ago. As this topic keeps coming up (and because three of you e-mailed me to ask about it this morning), I decided to scrap the column I was writing in favor of an explanation of all this once and for all.

Search Engines and ColdFusion
Here's the problem. Many search engines index sites by spidering them - starting at a known point, indexing that retrieved page, parsing it for embedded URLs, and then retrieving and indexing them too. They keep doing this until they have retrieved and indexed all linked pages in a site.

At least that's how it's supposed to work. Where things get complicated is when dynamic pages are indexed, or rather, when they're not. Some spiders won't index URLs that are dynamic, the theory being that since the content changes all the time there's no point in indexing it - after all, it may not even be the same content on subsequent accesses. So this link would be followed and indexed:

<A HREF="catalog.cfm">Catalog</A>

but this one may not:

<A HREF="catalog.cfm?dept=10">Catalog</A>

You can see the problem: ColdFusion developers (as well as ASP, PHP, Perl, Python, and JSP developers, among others) build dynamic pages. That's why we use ColdFusion - if all content was static we'd use plain old HTML without any server-side processing at all. We use ColdFusion because we want and need dynamic content - content that spiders may choose to ignore. If search engine indexing is a requirement for you, that can pose a real problem.

The Basic Solution
The solution to this problem is actually quite simple. What is it that tips off the spider, telling it that the page is dynamic? It's the query string portion of the URL, starting with the question mark. That question mark separates the URL (the file to be retrieved or the script to be executed) from any passed parameters (usually in name=value pairs).

The solution is to simply not have a query string - no question mark, no values passed after it, and no name=value pairs. Simple, eh? Yep, until you actually need passed parameters. Then what?

Well, look at this URL:

http://host/path/catalog.cfm/10

What happens when this is processed? catalog.cfm is the file to be executed and anything after the file name is ignored altogether. Even though the Web server and ColdFusion ignore it, that doesn't mean you have to. Using CGI variables (like PATH_INFO) you can access the complete URL - even parts of it that may be ignored by other processes or applications. The following snippet will display the complete path (minus any host and protocol information, although that's available in another CGI variable if needed).

<CFOUTPUT>#CGI.PATH_INFO#</CFOUTPUT>

Using the above URL this will display:

/path/catalog.cfm/10

We now know how to get the passed information. Next you need to remove the path and script name. You could do this by searching for the CFM file, but there's an easier way using yet another CGI variable, this time SCRIPT_NAME, which contains the name of the script being executed (here /path/catalog.cfm):

<CFSET query_string_length=Len(CGI.PATH_INFO)-Len(CGI.SCRIPT_NAME)>
<CFSET query_string=Right(CGI.PATH_INFO, query_string_length)>

The first <CFSET> gets the length of PATH_INFO and subtracts the length of SCRIPT_NAME from it, giving us the length of the section at the end (the data we want). The second <CFSET> uses Right() to extract the desired data, saving it in a variable named query_string. Displaying query_string would result in this output:

/10

That's how you get the data off the end of the URL.

Note: Depending on the Web server and OS being used you may find different CGI variables or different values in them. You can use <CFDUMP VAR="#CGI#"> to see all the CGI variables available to you (and you'll be able to adapt the code here accordingly).

Making It All Work
Now that the basic concept is clear, let's take this one step further. How can you pass multiple parameters? Well, you could pass three parameters, each separated by a /:

http://host/path/catalog.cfm/10/0/B12

But you would have no way of knowing which was which, what their names should be, and they'd always have to be in order (something you typically don't worry about when working with URL parameters).

While the technique is sound, the implementation can be improved by simulating name=value pairs. Look at this example:

http://host/path/catalog.cfm/dept/10/user/0/item/B12

There are now six values passed to the URL; each set of two is a name and value pair. Here it's dept=10, user=0, and item=B12. The advantage of this enhancement is that the variable name is known, the order of pairs is irrelevant, and the URL is easy to construct.

All that is required now is a simple way to extract these values and turn them into real URL values (after all, your application code shouldn't have to worry about manipulating this data; as far as it's concerned these are URL parameters plain and simple). This is where ColdFusion's list functions come in handy:

<!--- Extract "query_string" from full path --->
<CFSET query_string_length=Len(CGI.PATH_INFO)-Len(CGI.SCRIPT_NAME)>
<CFSET query_string=Right(CGI.PATH_
INFO, query_string_length)>

<!--- How many items in "query_string? --->
<CFSET items=ListLen(query_string, "/")>

<!--- Loop through list, pair of items at a time --->
<CFLOOP FROM="1" TO="#items#" STEP="2" INDEX="i">
<!--- Save this URL parameter --->
<CFPARAM NAME="URL.#ListGetAt(query_string, i, "/")#"
DEFAULT="#ListGetAt(query_string, i+1, "/")#">

</CFLOOP>

The first two <CFSET> statements simply extract the virtual query_string (as we saw earlier). Next we need to know how many items query_string contains. All ColdFusion list functions take an optional delimiter as a parameter, and so ListLen(query_string, "/") returns the number of items delimited by / (in this case, six).

Next, a <CFLOOP> is used to loop through the items, from one to however many there are, stepping two at a time (because every two is a set). Within the loop <CFPARAM> is used to create the variables. The value passed to NAME is the first of the pair (i) and the value passed to DEFAULT is the second (i+1). For the first set of values (/dept/10) the <CFPARAM> would be:

<CFPARAM NAME="URL.dept" DEFAULT="10">

There you have it. By the time the </CFLOOP> is reached, a set of URL parameters would have been created. This code could be placed once at the top of a page, and all other code could refer to URL parameters without knowing they were actually faked. In addition, as <CFPARAM> was used (instead of <CFSET>), you wouldn't overwrite variables if URL parameters actually did exist.

Clean, safe, and spider friendly, too.

Custom Tags to the Rescue
Of course, you wouldn't want all that code in every page - this type of processing is ideally suited for custom tags - thus my <CF_FakeURL> tag. Listing 1 shows the complete code.

Let's take a quick look at the tag. It starts with comments and a description, as all code should. Then a <CFPARAM> is used to define the default delimiter (a slash). Next the virtual query string is extracted using two <CFSET> tags (as we did earlier).

For this code to work there must always be an even number of elements (there are two for every parameter, one for name and one for value). The next <CFIF> statement uses the MOD operator to make sure the number of items in the list is a multiple of two - if this is not the case the list is not processed.

Within the <CFLOOP> the two values (current and next) are extracted, and then <CFPARAM> creates the URL variable.

Now all you need to do is call <CF_FakeURL> in your page and any faked URL parameters will be available for you to use. When you create your URLs just change this:

file.cfm?LName=Ben&FName=Forta

to this:

file.cfm/LName/Ben/FName/Forta

Simple as that.

Summary
Arachnophobia is the irrational fear of spiders - a fear that many ColdFusion developers seem to be suffering from. As you can see, with a little ingenuity and some good old CFML, there's a very workable solution to help you overcome this fear. Enjoy!

More Stories By Ben Forta

Ben Forta is Adobe's Senior Technical Evangelist. In that capacity he spends a considerable amount of time talking and writing about Adobe products (with an emphasis on ColdFusion and Flex), and providing feedback to help shape the future direction of the products. By the way, if you are not yet a ColdFusion user, you should be. It is an incredible product, and is truly deserving of all the praise it has been receiving. In a prior life he was a ColdFusion customer (he wrote one of the first large high visibility web sites using the product) and was so impressed he ended up working for the company that created it (Allaire). Ben is also the author of books on ColdFusion, SQL, Windows 2000, JSP, WAP, Regular Expressions, and more. Before joining Adobe (well, Allaire actually, and then Macromedia and Allaire merged, and then Adobe bought Macromedia) he helped found a company called Car.com which provides automotive services (buy a car, sell a car, etc) over the Web. Car.com (including Stoneage) is one of the largest automotive web sites out there, was written entirely in ColdFusion, and is now owned by Auto-By-Tel.

Comments (6) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Daniel Leroux 12/16/03 02:09:53 PM EST
Vikas Tailor 07/26/03 04:18:00 PM EDT

Hey all,

I used this great strategy and all of a sudden it stopped working! Could it be possible that the web server administrator no longer allows this? For example, go to http://www.linkexchanged.com/directory.cfm/Arts

This use to work, but no longer does.

Thoughts?

Laura Schneiderman 04/23/03 11:01:00 AM EDT

We've noticed that Norton firewall scrambles CGI variables. This CF_FakeURL tag relies on CGI variables. Has anyone else mentioned this problem?

Stephen Cassady 07/08/02 11:39:00 AM EDT

MX has many undocumented changes for 5.0 to MX.

One includes the new wal URLs are examined. I have a similiar script I picked up from the developers excange which allowed me to do http://www.site.com/page.cfm/var1.value1/var2.value2/var3.value3

And it also doesn't work now. Massive blow to the way everyting works, and, ironically, to the now indexed - but non-functional pages in all the search engines.

Arrgh. And there are other undocumented changes too (part of the CFIDE directory beeing in the site map - something we removed for security issues - for auto JavaScript from things like CFFrom, and a CFFILE problem).

Stephen Cassady
[email protected]
http://www.lopedia.com

Seth Aaronsen 06/30/02 06:30:00 PM EDT

We have successfully used CF_FakeURL on CF 5.0

We are having problems getting CF_FakeURL to work with CFMX standalone. This has been documented in the latest allaire message boards with the similar tag from Fusebox -- CF_FormUrl2AttributesSearch. CFMX will not process the strings http://www.mysite.com/page.cfm/variableid/value -- Please try this and you'll see.

We have tried to tweak the settings in the C:\CFusionMX\wwwroot\WEB-INF\web.xml doc by adding,

CfmServlet
*

but this does not work for us.

Do you have some suggestions?

Jill Crawford 02/22/02 09:51:00 AM EST

Thank you for the continued information that proves to be invaluable.

I have created URLs like http://www.yoursite.com/index.cfm/productID=item similiar to what you have suggested but and have lost the query string (everything after .cfm) in the log files for IIS since doing.

Apparently ColdFusion no longer believes these are URL parameters and does not return them to IIS to write as the cs-uri-query. Is there some way to capture the paramaters to the log file?

I would greatly appreciate any assistance you can give or point me towards.

@ThingsExpo Stories
The IoTs will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform. In his session at @ThingsExpo, Craig Sproule, CEO of Metavine, will demonstrate how to move beyond today's coding paradigm and share the must-have mindsets for removing complexity from the development proc...
SYS-CON Events announced today that Ericsson has been named “Gold Sponsor” of SYS-CON's @ThingsExpo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. Ericsson is a world leader in the rapidly changing environment of communications technology – providing equipment, software and services to enable transformation through mobility. Some 40 percent of global mobile traffic runs through networks we have supplied. More than 1 billion subscribers around the world re...
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
SYS-CON Events announced today that Peak 10, Inc., a national IT infrastructure and cloud services provider, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Peak 10 provides reliable, tailored data center and network services, cloud and managed services. Its solutions are designed to scale and adapt to customers’ changing business needs, enabling them to lower costs, improve performance and focus inter...
You deployed your app with the Bluemix PaaS and it's gaining some serious traction, so it's time to make some tweaks. Did you design your application in a way that it can scale in the cloud? Were you even thinking about the cloud when you built the app? If not, chances are your app is going to break. Check out this webcast to learn various techniques for designing applications that will scale successfully in Bluemix, for the confidence you need to take your apps to the next level and beyond.
There is an ever-growing explosion of new devices that are connected to the Internet using “cloud” solutions. This rapid growth is creating a massive new demand for efficient access to data. And it’s not just about connecting to that data anymore. This new demand is bringing new issues and challenges and it is important for companies to scale for the coming growth. And with that scaling comes the need for greater security, gathering and data analysis, storage, connectivity and, of course, the...
So, you bought into the current machine learning craze and went on to collect millions/billions of records from this promising new data source. Now, what do you do with them? Too often, the abundance of data quickly turns into an abundance of problems. How do you extract that "magic essence" from your data without falling into the common pitfalls? In her session at @ThingsExpo, Natalia Ponomareva, Software Engineer at Google, will provide tips on how to be successful in large scale machine lear...
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry's single source for the cloud. Fusion's advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including cloud...
In his session at @ThingsExpo, Chris Klein, CEO and Co-founder of Rachio, will discuss next generation communities that are using IoT to create more sustainable, intelligent communities. One example is Sterling Ranch, a 10,000 home development that – with the help of Siemens – will integrate IoT technology into the community to provide residents with energy and water savings as well as intelligent security. Everything from stop lights to sprinkler systems to building infrastructures will run ef...
Digital payments using wearable devices such as smart watches, fitness trackers, and payment wristbands are an increasing area of focus for industry participants, and consumer acceptance from early trials and deployments has encouraged some of the biggest names in technology and banking to continue their push to drive growth in this nascent market. Wearable payment systems may utilize near field communication (NFC), radio frequency identification (RFID), or quick response (QR) codes and barcodes...
The increasing popularity of the Internet of Things necessitates that our physical and cognitive relationship with wearable technology will change rapidly in the near future. This advent means logging has become a thing of the past. Before, it was on us to track our own data, but now that data is automatically available. What does this mean for mHealth and the "connected" body? In her session at @ThingsExpo, Lisa Calkins, CEO and co-founder of Amadeus Consulting, will discuss the impact of wea...
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
The IoT has the potential to create a renaissance of manufacturing in the US and elsewhere. In his session at 18th Cloud Expo, Florent Solt, CTO and chief architect of Netvibes, will discuss how the expected exponential increase in the amount of data that will be processed, transported, stored, and accessed means there will be a huge demand for smart technologies to deliver it. Florent Solt is the CTO and chief architect of Netvibes. Prior to joining Netvibes in 2007, he co-founded Rift Technol...
We’ve worked with dozens of early adopters across numerous industries and will debunk common misperceptions, which starts with understanding that many of the connected products we’ll use over the next 5 years are already products, they’re just not yet connected. With an IoT product, time-in-market provides much more essential feedback than ever before. Innovation comes from what you do with the data that the connected product provides in order to enhance the customer experience and optimize busi...
SYS-CON Events announced today that Stratoscale, the software company developing the next generation data center operating system, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Stratoscale is revolutionizing the data center with a zero-to-cloud-in-minutes solution. With Stratoscale’s hardware-agnostic, Software Defined Data Center (SDDC) solution to store everything, run anything and scale everywhere...
Angular 2 is a complete re-write of the popular framework AngularJS. Programming in Angular 2 is greatly simplified – now it's a component-based well-performing framework. This immersive one-day workshop at 18th Cloud Expo, led by Yakov Fain, a Java Champion and a co-founder of the IT consultancy Farata Systems and the product company SuranceBay, will provide you with everything you wanted to know about Angular 2.
SYS-CON Events announced today that Men & Mice, the leading global provider of DNS, DHCP and IP address management overlay solutions, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. The Men & Mice Suite overlay solution is already known for its powerful application in heterogeneous operating environments, enabling enterprises to scale without fuss. Building on a solid range of diverse platform support,...
Whether your IoT service is connecting cars, homes, appliances, wearable, cameras or other devices, one question hangs in the balance – how do you actually make money from this service? The ability to turn your IoT service into profit requires the ability to create a monetization strategy that is flexible, scalable and working for you in real-time. It must be a transparent, smoothly implemented strategy that all stakeholders – from customers to the board – will be able to understand and comprehe...
Increasing IoT connectivity is forcing enterprises to find elegant solutions to organize and visualize all incoming data from these connected devices with re-configurable dashboard widgets to effectively allow rapid decision-making for everything from immediate actions in tactical situations to strategic analysis and reporting. In his session at 18th Cloud Expo, Shikhir Singh, Senior Developer Relations Manager at Sencha, will discuss how to create HTML5 dashboards that interact with IoT devic...
Artificial Intelligence has the potential to massively disrupt IoT. In his session at 18th Cloud Expo, AJ Abdallat, CEO of Beyond AI, will discuss what the five main drivers are in Artificial Intelligence that could shape the future of the Internet of Things. AJ Abdallat is CEO of Beyond AI. He has over 20 years of management experience in the fields of artificial intelligence, sensors, instruments, devices and software for telecommunications, life sciences, environmental monitoring, process...