|By Ben Forta||
|January 9, 2002 12:00 AM EST||
Barely a week goes by without someone asking me about ColdFusion and search engine-friendly URLs. This is one of those topics that ColdFusion developers have been discussing for a long time - I first started a thread on this subject on the Allaire Developer's Forum close to five years ago. As this topic keeps coming up (and because three of you e-mailed me to ask about it this morning), I decided to scrap the column I was writing in favor of an explanation of all this once and for all.
Search Engines and ColdFusion
Here's the problem. Many search engines index sites by spidering them - starting at a known point, indexing that retrieved page, parsing it for embedded URLs, and then retrieving and indexing them too. They keep doing this until they have retrieved and indexed all linked pages in a site.
At least that's how it's supposed to work. Where things get complicated is when dynamic pages are indexed, or rather, when they're not. Some spiders won't index URLs that are dynamic, the theory being that since the content changes all the time there's no point in indexing it - after all, it may not even be the same content on subsequent accesses. So this link would be followed and indexed:
but this one may not:
You can see the problem: ColdFusion developers (as well as ASP, PHP, Perl, Python, and JSP developers, among others) build dynamic pages. That's why we use ColdFusion - if all content was static we'd use plain old HTML without any server-side processing at all. We use ColdFusion because we want and need dynamic content - content that spiders may choose to ignore. If search engine indexing is a requirement for you, that can pose a real problem.
The Basic Solution
The solution to this problem is actually quite simple. What is it that tips off the spider, telling it that the page is dynamic? It's the query string portion of the URL, starting with the question mark. That question mark separates the URL (the file to be retrieved or the script to be executed) from any passed parameters (usually in name=value pairs).
The solution is to simply not have a query string - no question mark, no values passed after it, and no name=value pairs. Simple, eh? Yep, until you actually need passed parameters. Then what?
Well, look at this URL:
What happens when this is processed? catalog.cfm is the file to be executed and anything after the file name is ignored altogether. Even though the Web server and ColdFusion ignore it, that doesn't mean you have to. Using CGI variables (like PATH_INFO) you can access the complete URL - even parts of it that may be ignored by other processes or applications. The following snippet will display the complete path (minus any host and protocol information, although that's available in another CGI variable if needed).
Using the above URL this will display:
We now know how to get the passed information. Next you need to remove the path and script name. You could do this by searching for the CFM file, but there's an easier way using yet another CGI variable, this time SCRIPT_NAME, which contains the name of the script being executed (here /path/catalog.cfm):
<CFSET query_string=Right(CGI.PATH_INFO, query_string_length)>
The first <CFSET> gets the length of PATH_INFO and subtracts the length of SCRIPT_NAME from it, giving us the length of the section at the end (the data we want). The second <CFSET> uses Right() to extract the desired data, saving it in a variable named query_string. Displaying query_string would result in this output:
That's how you get the data off the end of the URL.
Note: Depending on the Web server and OS being used you may find different CGI variables or different values in them. You can use <CFDUMP VAR="#CGI#"> to see all the CGI variables available to you (and you'll be able to adapt the code here accordingly).
Making It All Work
Now that the basic concept is clear, let's take this one step further. How can you pass multiple parameters? Well, you could pass three parameters, each separated by a /:
But you would have no way of knowing which was which, what their names should be, and they'd always have to be in order (something you typically don't worry about when working with URL parameters).
While the technique is sound, the implementation can be improved by simulating name=value pairs. Look at this example:
There are now six values passed to the URL; each set of two is a name and value pair. Here it's dept=10, user=0, and item=B12. The advantage of this enhancement is that the variable name is known, the order of pairs is irrelevant, and the URL is easy to construct.
All that is required now is a simple way to extract these values and turn them into real URL values (after all, your application code shouldn't have to worry about manipulating this data; as far as it's concerned these are URL parameters plain and simple). This is where ColdFusion's list functions come in handy:
<!--- How many items in "query_string? --->
<!--- Loop through list, pair of items at a time --->
<!--- Extract "query_string" from full path --->
The first two <CFSET> statements simply extract the virtual query_string (as we saw earlier). Next we need to know how many items query_string contains. All ColdFusion list functions take an optional delimiter as a parameter, and so ListLen(query_string, "/") returns the number of items delimited by / (in this case, six).
<CFSET items=ListLen(query_string, "/")>
<CFLOOP FROM="1" TO="#items#" STEP="2" INDEX="i">
<!--- Save this URL parameter --->
<CFPARAM NAME="URL.#ListGetAt(query_string, i, "/")#"
DEFAULT="#ListGetAt(query_string, i+1, "/")#">
<!--- How many items in "query_string? --->
<!--- Loop through list, pair of items at a time --->
Next, a <CFLOOP> is used to loop through the items, from one to however many there are, stepping two at a time (because every two is a set). Within the loop <CFPARAM> is used to create the variables. The value passed to NAME is the first of the pair (i) and the value passed to DEFAULT is the second (i+1). For the first set of values (/dept/10) the <CFPARAM> would be:
<CFPARAM NAME="URL.dept" DEFAULT="10">
There you have it. By the time the </CFLOOP> is reached, a set of URL parameters would have been created. This code could be placed once at the top of a page, and all other code could refer to URL parameters without knowing they were actually faked. In addition, as <CFPARAM> was used (instead of <CFSET>), you wouldn't overwrite variables if URL parameters actually did exist.
Clean, safe, and spider friendly, too.
Custom Tags to the Rescue
Of course, you wouldn't want all that code in every page - this type of processing is ideally suited for custom tags - thus my <CF_FakeURL> tag. Listing 1 shows the complete code.
Let's take a quick look at the tag. It starts with comments and a description, as all code should. Then a <CFPARAM> is used to define the default delimiter (a slash). Next the virtual query string is extracted using two <CFSET> tags (as we did earlier).
For this code to work there must always be an even number of elements (there are two for every parameter, one for name and one for value). The next <CFIF> statement uses the MOD operator to make sure the number of items in the list is a multiple of two - if this is not the case the list is not processed.
Within the <CFLOOP> the two values (current and next) are extracted, and then <CFPARAM> creates the URL variable.
Now all you need to do is call <CF_FakeURL> in your page and any faked URL parameters will be available for you to use. When you create your URLs just change this:
Simple as that.
Arachnophobia is the irrational fear of spiders - a fear that many ColdFusion developers seem to be suffering from. As you can see, with a little ingenuity and some good old CFML, there's a very workable solution to help you overcome this fear. Enjoy!
|Daniel Leroux 12/16/03 02:09:53 PM EST|
|Vikas Tailor 07/26/03 04:18:00 PM EDT|
I used this great strategy and all of a sudden it stopped working! Could it be possible that the web server administrator no longer allows this? For example, go to http://www.linkexchanged.com/directory.cfm/Arts
This use to work, but no longer does.
|Laura Schneiderman 04/23/03 11:01:00 AM EDT|
We've noticed that Norton firewall scrambles CGI variables. This CF_FakeURL tag relies on CGI variables. Has anyone else mentioned this problem?
|Stephen Cassady 07/08/02 11:39:00 AM EDT|
MX has many undocumented changes for 5.0 to MX.
One includes the new wal URLs are examined. I have a similiar script I picked up from the developers excange which allowed me to do http://www.site.com/page.cfm/var1.value1/var2.value2/var3.value3
And it also doesn't work now. Massive blow to the way everyting works, and, ironically, to the now indexed - but non-functional pages in all the search engines.
|Seth Aaronsen 06/30/02 06:30:00 PM EDT|
We have successfully used CF_FakeURL on CF 5.0
We are having problems getting CF_FakeURL to work with CFMX standalone. This has been documented in the latest allaire message boards with the similar tag from Fusebox -- CF_FormUrl2AttributesSearch. CFMX will not process the strings http://www.mysite.com/page.cfm/variableid/value -- Please try this and you'll see.
We have tried to tweak the settings in the C:\CFusionMX\wwwroot\WEB-INF\web.xml doc by adding,
but this does not work for us.
Do you have some suggestions?
|Jill Crawford 02/22/02 09:51:00 AM EST|
Thank you for the continued information that proves to be invaluable.
I have created URLs like http://www.yoursite.com/index.cfm/productID=item similiar to what you have suggested but and have lost the query string (everything after .cfm) in the log files for IIS since doing.
Apparently ColdFusion no longer believes these are URL parameters and does not return them to IIS to write as the cs-uri-query. Is there some way to capture the paramaters to the log file?
I would greatly appreciate any assistance you can give or point me towards.
With so much going on in this space you could be forgiven for thinking you were always working with yesterday’s technologies. So much change, so quickly. What do you do if you have to build a solution from the ground up that is expected to live in the field for at least 5-10 years? This is the challenge we faced when we looked to refresh our existing 10-year-old custom hardware stack to measure the fullness of trash cans and compactors.
Aug. 31, 2016 02:45 AM EDT Reads: 1,886
The emerging Internet of Everything creates tremendous new opportunities for customer engagement and business model innovation. However, enterprises must overcome a number of critical challenges to bring these new solutions to market. In his session at @ThingsExpo, Michael Martin, CTO/CIO at nfrastructure, outlined these key challenges and recommended approaches for overcoming them to achieve speed and agility in the design, development and implementation of Internet of Everything solutions wi...
Aug. 31, 2016 02:15 AM EDT Reads: 2,275
Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like “How is my application doing” but no id...
Aug. 31, 2016 02:00 AM EDT Reads: 1,986
Cloud computing is being adopted in one form or another by 94% of enterprises today. Tens of billions of new devices are being connected to The Internet of Things. And Big Data is driving this bus. An exponential increase is expected in the amount of information being processed, managed, analyzed, and acted upon by enterprise IT. This amazing is not part of some distant future - it is happening today. One report shows a 650% increase in enterprise data by 2020. Other estimates are even higher....
Aug. 31, 2016 01:45 AM EDT Reads: 3,079
Smart Cities are here to stay, but for their promise to be delivered, the data they produce must not be put in new siloes. In his session at @ThingsExpo, Mathias Herberts, Co-founder and CTO of Cityzen Data, will deep dive into best practices that will ensure a successful smart city journey.
Aug. 31, 2016 01:00 AM EDT Reads: 1,705
Identity is in everything and customers are looking to their providers to ensure the security of their identities, transactions and data. With the increased reliance on cloud-based services, service providers must build security and trust into their offerings, adding value to customers and improving the user experience. Making identity, security and privacy easy for customers provides a unique advantage over the competition.
Aug. 30, 2016 08:15 PM EDT Reads: 2,492
SYS-CON Events announced today that 910Telecom will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Housed in the classic Denver Gas & Electric Building, 910 15th St., 910Telecom is a carrier-neutral telecom hotel located in the heart of Denver. Adjacent to CenturyLink, AT&T, and Denver Main, 910Telecom offers connectivity to all major carriers, Internet service providers, Internet backbones and ...
Aug. 30, 2016 08:00 PM EDT Reads: 2,036
SYS-CON Events announced today that Pulzze Systems will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Pulzze Systems, Inc. provides infrastructure products for the Internet of Things to enable any connected device and system to carry out matched operations without programming. For more information, visit http://www.pulzzesystems.com.
Aug. 30, 2016 07:15 PM EDT Reads: 380
There is growing need for data-driven applications and the need for digital platforms to build these apps. In his session at 19th Cloud Expo, Muddu Sudhakar, VP and GM of Security & IoT at Splunk, will cover different PaaS solutions and Big Data platforms that are available to build applications. In addition, AI and machine learning are creating new requirements that developers need in the building of next-gen apps. The next-generation digital platforms have some of the past platform needs a...
Aug. 30, 2016 07:00 PM EDT Reads: 938
Data is an unusual currency; it is not restricted by the same transactional limitations as money or people. In fact, the more that you leverage your data across multiple business use cases, the more valuable it becomes to the organization. And the same can be said about the organization’s analytics. In his session at 19th Cloud Expo, Bill Schmarzo, CTO for the Big Data Practice at EMC, will introduce a methodology for capturing, enriching and sharing data (and analytics) across the organizati...
Aug. 30, 2016 06:15 PM EDT Reads: 381
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devices - comp...
Aug. 30, 2016 03:30 PM EDT Reads: 3,785
SYS-CON Events announced today Telecom Reseller has been named “Media Sponsor” of SYS-CON's 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
Aug. 30, 2016 02:30 PM EDT Reads: 1,063
SYS-CON Events announced today that Adobe has been named “Bronze Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. Adobe is changing the world though digital experiences. Adobe helps customers develop and deliver high-impact experiences that differentiate brands, build loyalty, and drive revenue across every screen, including smartphones, computers, tablets and TVs. Adobe content solutions are used daily by millions of co...
Aug. 30, 2016 02:00 PM EDT Reads: 3,825
Why do your mobile transformations need to happen today? Mobile is the strategy that enterprise transformation centers on to drive customer engagement. In his general session at @ThingsExpo, Roger Woods, Director, Mobile Product & Strategy – Adobe Marketing Cloud, covered key IoT and mobile trends that are forcing mobile transformation, key components of a solid mobile strategy and explored how brands are effectively driving mobile change throughout the enterprise.
Aug. 30, 2016 01:45 PM EDT Reads: 716
Pulzze Systems was happy to participate in such a premier event and thankful to be receiving the winning investment and global network support from G-Startup Worldwide. It is an exciting time for Pulzze to showcase the effectiveness of innovative technologies and enable them to make the world smarter and better. The reputable contest is held to identify promising startups around the globe that are assured to change the world through their innovative products and disruptive technologies. There w...
Aug. 30, 2016 01:30 PM EDT Reads: 901
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, will discuss the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
Aug. 30, 2016 01:15 PM EDT Reads: 2,076
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterpri...
Aug. 30, 2016 01:00 PM EDT Reads: 3,246
Almost two-thirds of companies either have or soon will have IoT as the backbone of their business in 2016. However, IoT is far more complex than most firms expected. How can you not get trapped in the pitfalls? In his session at @ThingsExpo, Tony Shan, a renowned visionary and thought leader, will introduce a holistic method of IoTification, which is the process of IoTifying the existing technology and business models to adopt and leverage IoT. He will drill down to the components in this fra...
Aug. 30, 2016 10:30 AM EDT Reads: 431
DevOps at Cloud Expo, taking place Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long dev...
Aug. 29, 2016 10:00 PM EDT Reads: 2,524
The 19th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Digital Transformation, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportuni...
Aug. 28, 2016 10:30 PM EDT Reads: 4,104