|By Joe Cronin||
|January 18, 2005 12:00 AM EST||
Do your end users complain about the quality of your ColdFusion application's search functionality?
Have you exceeded the 250,000-document limit of the search functionality that comes embedded in ColdFusion MX? Have the requirements of your application changed to include searching for content both inside and outside of the ColdFusion environment?
If you answered yes to any of the above questions, it sounds like your ColdFusion MX application could use some help. In surveys by industry analysts, such as Gartner, IDC, and Forrester, end users, executives, and developers alike consistently rank the ability to search as one of the most important features of all online applications. Yet search continues to be one of the most maligned utilities of Web sites, business applications, and you name it. The bottom line is that if your end users can't find what they're looking for with your application's search tools, you're not realizing the full value of your ColdFusion investment.
Early on, Allaire, and then Macromedia, understood the need to provide ColdFusion developers with the ability to integrate advanced search features into their applications. Since 1997, the search functionality embedded in ColdFusion has been provided by Verity, Inc. Verity was selected for a number of reasons: ease of integration into ColdFusion, advanced functionality, and Verity's recognized position as the market leader in the enterprise search space.
But even the best search tool is only as good as its implementation. This article includes tips on optimizing the Verity search included in ColdFusion MX 6.1. With this search, you can build applications with advanced, enterprise-class full-text search of up to 250,000 documents and/or database records within the ColdFusion environment (if you need a larger search, go directly to the end of this article for more information). In addition, this article describes how to quickly and easily add value to applications by enhancing the search within ColdFusion, and by extending search to content outside ColdFusion.
Search Within ColdFusion MX
To ensure that users of ColdFusion applications can find the specific information they need in databases and hundreds of file formats, Macromedia integrated Verity full-text search. This includes:
- Full-text search of all ColdFusion content
- The ability to search a wide range of document types, including HTML, binary documents, and database records
- The ability to limit search to specific groups, or collections of documents, in order to enable subject-specific searches
- Support for multiple languages, including most European and Asian languages
- Fielded search against index metadata
One of the strengths of Verity search solutions is that they can be configured to meet specific business and technical objectives. To optimize search with the ColdFusion environment, Macromedia implemented Verity to support content of the following basic data types:
- Text files such as HTML pages and CFML pages
- Binary documents
- Record sets returned from cfquery, cfldap, and cfpop queries
Typical Applications of Verity Search
By taking advantage of Verity's flexibility you can add significant value to your ColdFusion applications. Typical uses of Verity search within ColdFusion include:
- Indexing the content of a Web site and providing a generalized search mechanism, such as the familiar search box
- Indexing specific directories that contain documents on a specific topic in order to provide subject-based searching, or to limit the focus of searches to specific groups of documents
- Indexing cfquery record sets into a single Verity Collection and letting users search against the collection with a single query rather than requiring them to perform multiple database queries to return the same data
- Indexing cfldap and cfpop query results
- Indexing e-mail generated by ColdFusion application pages and making the resulting Verity Collection available for searching from your ColdFusion application pages
- Building Verity Collections with inventory data and making those collections available for searching from your ColdFusion application pages
- Supporting international users in a range of languages, using the cfindex, cfcollection, and cfsearch tags
One of the most obvious advantages of Verity search embedded in ColdFusion is its performance. For example, using Verity to index the output from database queries and then to perform searches against the indexed record sets is much faster than using SQL to search databases directly. Additional advantages of Verity over other search methods include:
- Superior relevancy of search results lists
- The display of document titles and summaries in search results lists
- Elimination of the need to programmatically create query constructs by allowing novice and expert users alike to use the same type of full-text queries they're used to using on the Web
- Indexing of database text fields, such as notes and product descriptions, that cannot be effectively indexed by native database tools
- The indexing and display of document URLs in results lists, a valuable document management feature
The good news is that Verity's advanced search features are straightforward to deploy within ColdFusion MX. In general, adding optimized osearch to your application involves three basic tasks:
- Creating a Verity Collection
- Indexing the content within your ColdFusion application
- Designing a search interface
There are pros and cons to using either the ColdFusion MX Administrator or CFML for deploying Verity search within ColdFusion applications. Refer to Table 2 to determine which is appropriate for your application and information environment.
Just as there is more than one method for deploying Verity search, you can configure your search implementation to meet specific business objectives. Primarily, you do this by running cfsearch or cfquery. Table 3 lists the advantages and uses of each.
Optimizing Search Relevancy
Once you've deployed the Verity search, you can also optimize its relevancy for your specific information environment. The ColdFusion implementation of Verity Query Language (VQL) uses operators and modifiers. These can either be used directly by advanced users, or implemented transparently so that they are applied automatically to all queries. The following are some of the more commonly used VQL operators:
For additional resources, see the ColdFusion MX documentation available at www.macromedia.com. All product questions and support for ColdFusion, including the Verity search integrated into ColdFusion MX, are provided by Macromedia.
Extending and Enhancing the Search in Macromedia ColdFusion MX
As advanced as the Verity search is within ColdFusion, you may eventually want to deploy ColdFusion applications with enhanced search capabilities that are not possible using Macromedia's implementation of Verity. This is not due to any inherent limitations on Verity's part as much as it is a result of the robust, advanced capabilities of ColdFusion that enable you to develop ever more powerful applications. Verity has found that the need for additional search features is largely driven by three key requirements, along with a less common fourth requirement:
1. Searching content outside of ColdFusion. The Verity search functionality within ColdFusion is limited to searching content and database records within your ColdFusion application. Many ColdFusion developers need to extend the search functionality to areas outside of ColdFusion, such as intranets, external Web sites, file servers, external databases, Microsoft Exchange, and third-party document management systems.
2. Additional Administrative Tools. To meet their specific application requirements, some developers require additional administrative tools.
3. Adding more advanced end-user interface options. Another common request among ColdFusion developers is the ability to add advanced search features not included with the ColdFusion implementation of Verity, such as spell checker/recommendations, and search-term highlighting.
4. Searching over 250,000 documents and/or records. Occasionally, developers will build applications that exceed the 250,000 documents and/or records that the ColdFusion implementation of Verity is limited to searching. For applications that must search large databases or repositories, developers need to extend this limit. This topic is not covered in this article. For information on overcoming the document number limitations of ColdFusion, contact [email protected].
Extending and Enhancing Search with Verity Ultraseek
The three most common ColdFusion search enhancements listed above can be accomplished with the addition of Verity Ultraseek. This downloadable search engine is easily integrated into applications, using its available Java API. It can also be readily deployed into mixed application environments, using its Web services interface, which supports both the .NET and J2EE platforms. A number of factors make Ultraseek the best choice for extending search outside of the ColdFusion environment and providing additional administration tools:
- Ultraseek's easy-to-implement, set-and-forget design requires extremely low ongoing maintenance and overhead
- Ultraseek's end-user interfaces (i.e., search boxes and results lists) are similar to those of the Verity search embedded in ColdFusion
- Ultraseek provides enterprise-class search at a price point in line with ColdFusion
ColdFusion Search-Specific Tags
Creating a Collection with the cfcollection tag
When using the cfcollection tag, you can specify the same attributes as in the ColdFusion MX Administrator:
- Action: (Optional) The action to perform on the collection (create, delete, repair, or optimize). The default value for the action attribute is list. For more information, see cfcollection in CFML Reference.
- Collection: The name of the new collection, or the name of a collection on which you will perform an action.
- Path: The location for the Verity collection.
- Language: (Optional) The language used to create the collection (English, by default).
<cfcollection action = "create" collection = "a_new_collection" path = "c:\CFusionMX\verity\collections\">
Indexing a Collection Using the cfindex tag
You can index a collection in CFML using the cfindex tag, which eliminates the need to use the ColdFusion MX Administrator.
- Collection: The name of the collection. If you are indexing an external collection (external = "Yes"), you must also specify the fully qualified path for the collection.
- Action: (Optional) Can be update (the default action), delete, purge, or refresh.
- Extensions: (Optional) The delimited list of file extensions that ColdFusion uses to index files if type="Path".
- Key: (Optional) The path containing the files you are indexing if type="path".
- URLpath: (Optional) The URL path for files if type="file" and type="path". When the collection is searched with cfsearch, the pathname is automatically prefixed to filenames and returned as the URL attribute.
- Recurse: (Optional) Yes or No. Yes specifies, if type = "Path", that directories below the path specified in the key attribute are included in the indexing operation.
- Language: (Optional) The language of the collection. English is the default.
You use the cfsearch tag to search an indexed collection. Searching a Verity Collection is similar to a standard ColdFusion query: both use a dedicated ColdFusion tag that requires a name attribute for their searches.
The following are important attributes for the cfsearch tag:
- Name: The name of the search query.
- Collection: The name of the collection(s) being searched. Use a fully qualified path for an external collection. Separate multiple collections with a comma; for example, collection = "sprocket_docs,CodeColl".
- Criteria: The search target (can be dynamic).
- RecordCount: The total number of records returned by the search.
- CurrentRow: The current row of the record set being processed by cfoutput.
- RecordsSearched: The total number of records in the index that were searched. If no records were returned in the search, this property returns a null value.
|Dave Hardwick 01/27/05 10:25:47 AM EST|
Ironic, the URL for the Ultraseek extension for CF provided a "404: File Not Found." When I searched for cfsearch or CFMX on Verity's Web site, it returned no results!
I was surprised this article did not mention KF2 Server. Crawling the site is a very powerful option that allows for regular expression type controls over what is and is not indexed, be it directories or filenames.
|Peter 01/20/05 03:53:53 AM EST|
Great article, glad to see Verity with CF getting some exposure. I agree that it is entirely about servicing the end user but have a problem that I can't seem to get around.
I have a handful of long (250+ page) pdfs which I can add to a verity collection and then search. What my end users really want, however, is not just to identify matching documents, but to identify the passages within those documents that match their search criteria.
Is anything along these lines possible with the Verity engine included with CF?
The explosion of connected devices / sensors is creating an ever-expanding set of new and valuable data. In parallel the emerging capability of Big Data technologies to store, access, analyze, and react to this data is producing changes in business models under the umbrella of the Internet of Things (IoT). In particular within the Insurance industry, IoT appears positioned to enable deep changes by altering relationships between insurers, distributors, and the insured. In his session at @ThingsExpo, Michael Sick, a Senior Manager and Big Data Architect within Ernst and Young's Financial Servi...
May. 23, 2015 04:00 AM EDT Reads: 4,516
15th Cloud Expo, which took place Nov. 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA, expanded the conference content of @ThingsExpo, Big Data Expo, and DevOps Summit to include two developer events. IBM held a Bluemix Developer Playground on November 5 and ElasticBox held a Hackathon on November 6. Both events took place on the expo floor. The Bluemix Developer Playground, for developers of all levels, highlighted the ease of use of Bluemix, its services and functionality and provide short-term introductory projects that developers can complete between sessions.
May. 23, 2015 04:00 AM EDT Reads: 6,017
The 3rd International @ThingsExpo, co-located with the 16th International Cloud Expo – to be held June 9-11, 2015, at the Javits Center in New York City, NY – is now accepting Hackathon proposals. Hackathon sponsorship benefits include general brand exposure and increasing engagement with the developer ecosystem. At Cloud Expo 2014 Silicon Valley, IBM held the Bluemix Developer Playground on November 5 and ElasticBox held the DevOps Hackathon on November 6. Both events took place on the expo floor. The Bluemix Developer Playground, for developers of all levels, highlighted the ease of use of...
May. 23, 2015 04:00 AM EDT Reads: 3,230
In the consumer IoT, everything is new, and the IT world of bits and bytes holds sway. But industrial and commercial realms encompass operational technology (OT) that has been around for 25 or 50 years. This grittier, pre-IP, more hands-on world has much to gain from Industrial IoT (IIoT) applications and principles. But adding sensors and wireless connectivity won’t work in environments that demand unwavering reliability and performance. In his session at @ThingsExpo, Ron Sege, CEO of Echelon, will discuss how as enterprise IT embraces other IoT-related technology trends, enterprises with i...
May. 23, 2015 03:00 AM EDT Reads: 4,030
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With "smart" appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user's habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps, abiding by privacy concerns and making the concept a reality. These challenges can't be addressed w...
May. 23, 2015 02:45 AM EDT Reads: 6,469
We’re no longer looking to the future for the IoT wave. It’s no longer a distant dream but a reality that has arrived. It’s now time to make sure the industry is in alignment to meet the IoT growing pains – cooperate and collaborate as well as innovate. In his session at @ThingsExpo, Jim Hunter, Chief Scientist & Technology Evangelist at Greenwave Systems, will examine the key ingredients to IoT success and identify solutions to challenges the industry is facing. The deep industry expertise behind this presentation will provide attendees with a leading edge view of rapidly emerging IoT oppor...
May. 23, 2015 02:30 AM EDT Reads: 4,655
The industrial software market has treated data with the mentality of “collect everything now, worry about how to use it later.” We now find ourselves buried in data, with the pervasive connectivity of the (Industrial) Internet of Things only piling on more numbers. There’s too much data and not enough information. In his session at @ThingsExpo, Bob Gates, Global Marketing Director, GE’s Intelligent Platforms business, to discuss how realizing the power of IoT, software developers are now focused on understanding how industrial data can create intelligence for industrial operations. Imagine ...
May. 23, 2015 02:00 AM EDT Reads: 4,928
SYS-CON Events announced today that Liaison Technologies, a leading provider of data management and integration cloud services and solutions, has been named "Silver Sponsor" of SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York, NY. Liaison Technologies is a recognized market leader in providing cloud-enabled data integration and data management solutions to break down complex information barriers, enabling enterprises to make smarter decisions, faster.
May. 23, 2015 01:30 AM EDT Reads: 5,071
The 17th International Cloud Expo has announced that its Call for Papers is open. 17th International Cloud Expo, to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, APM, APIs, Microservices, Security, Big Data, Internet of Things, DevOps and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal today!
May. 23, 2015 01:15 AM EDT Reads: 3,970
Collecting data in the field and configuring multitudes of unique devices is a time-consuming, labor-intensive process that can stretch IT resources. Horan & Bird [H&B], Australia’s fifth-largest Solar Panel Installer, wanted to automate sensor data collection and monitoring from its solar panels and integrate the data with its business and marketing systems. After data was collected and structured, two major areas needed to be addressed: improving developer workflows and extending access to a business application to multiple users (multi-tenancy). Docker, a container technology, was used to ...
May. 23, 2015 01:00 AM EDT Reads: 1,632
Hadoop as a Service (as offered by handful of niche vendors now) is a cloud computing solution that makes medium and large-scale data processing accessible, easy, fast and inexpensive. In his session at Big Data Expo, Kumar Ramamurthy, Vice President and Chief Technologist, EIM & Big Data, at Virtusa, will discuss how this is achieved by eliminating the operational challenges of running Hadoop, so one can focus on business growth. The fragmented Hadoop distribution world and various PaaS solutions that provide a Hadoop flavor either make choices for customers very flexible in the name of opti...
May. 23, 2015 12:30 AM EDT Reads: 3,402
For years, we’ve relied too heavily on individual network functions or simplistic cloud controllers. However, they are no longer enough for today’s modern cloud data center. Businesses need a comprehensive platform architecture in order to deliver a complete networking suite for IoT environment based on OpenStack. In his session at @ThingsExpo, Dhiraj Sehgal from PLUMgrid will discuss what a holistic networking solution should really entail, and how to build a complete platform that is scalable, secure, agile and automated.
May. 23, 2015 12:00 AM EDT Reads: 4,057
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades. With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo, June 9-11, 2015, at the Javits Center in New York City. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be
May. 22, 2015 11:30 PM EDT Reads: 1,512
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo in Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal an...
May. 22, 2015 11:30 PM EDT Reads: 2,271
The true value of the Internet of Things (IoT) lies not just in the data, but through the services that protect the data, perform the analysis and present findings in a usable way. With many IoT elements rooted in traditional IT components, Big Data and IoT isn’t just a play for enterprise. In fact, the IoT presents SMBs with the prospect of launching entirely new activities and exploring innovative areas. CompTIA research identifies several areas where IoT is expected to have the greatest impact.
May. 22, 2015 09:00 PM EDT Reads: 4,619
Wearable devices have come of age. The primary applications of wearables so far have been "the Quantified Self" or the tracking of one's fitness and health status. We propose the evolution of wearables into social and emotional communication devices. Our BE(tm) sensor uses light to visualize the skin conductance response. Our sensors are very inexpensive and can be massively distributed to audiences or groups of any size, in order to gauge reactions to performances, video, or any kind of presentation. In her session at @ThingsExpo, Jocelyn Scheirer, CEO & Founder of Bionolux, will discuss ho...
May. 22, 2015 09:00 PM EDT Reads: 4,913
Every day we read jaw-dropping stats on the explosion of data. We allocate significant resources to harness and better understand it. We build businesses around it. But we’ve only just begun. For big payoffs in Big Data, CIOs are turning to cognitive computing. Cognitive computing’s ability to securely extract insights, understand natural language, and get smarter each time it’s used is the next, logical step for Big Data.
May. 22, 2015 08:00 PM EDT Reads: 1,738
The 4th International Internet of @ThingsExpo, co-located with the 17th International Cloud Expo - to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA - announces that its Call for Papers is open. The Internet of Things (IoT) is the biggest idea since the creation of the Worldwide Web more than 20 years ago.
May. 22, 2015 08:00 PM EDT Reads: 1,234
There's no doubt that the Internet of Things is driving the next wave of innovation. Google has spent billions over the past few months vacuuming up companies that specialize in smart appliances and machine learning. Already, Philips light bulbs, Audi automobiles, and Samsung washers and dryers can communicate with and be controlled from mobile devices. To take advantage of the opportunities the Internet of Things brings to your business, you'll want to start preparing now.
May. 22, 2015 07:00 PM EDT Reads: 5,597
17th Cloud Expo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises are using some form of XaaS – software, platform, and infrastructure as a service.
May. 22, 2015 05:00 PM EDT Reads: 2,082