|By Joe Cronin||
|January 18, 2005 12:00 AM EST||
Do your end users complain about the quality of your ColdFusion application's search functionality?
Have you exceeded the 250,000-document limit of the search functionality that comes embedded in ColdFusion MX? Have the requirements of your application changed to include searching for content both inside and outside of the ColdFusion environment?
If you answered yes to any of the above questions, it sounds like your ColdFusion MX application could use some help. In surveys by industry analysts, such as Gartner, IDC, and Forrester, end users, executives, and developers alike consistently rank the ability to search as one of the most important features of all online applications. Yet search continues to be one of the most maligned utilities of Web sites, business applications, and you name it. The bottom line is that if your end users can't find what they're looking for with your application's search tools, you're not realizing the full value of your ColdFusion investment.
Early on, Allaire, and then Macromedia, understood the need to provide ColdFusion developers with the ability to integrate advanced search features into their applications. Since 1997, the search functionality embedded in ColdFusion has been provided by Verity, Inc. Verity was selected for a number of reasons: ease of integration into ColdFusion, advanced functionality, and Verity's recognized position as the market leader in the enterprise search space.
But even the best search tool is only as good as its implementation. This article includes tips on optimizing the Verity search included in ColdFusion MX 6.1. With this search, you can build applications with advanced, enterprise-class full-text search of up to 250,000 documents and/or database records within the ColdFusion environment (if you need a larger search, go directly to the end of this article for more information). In addition, this article describes how to quickly and easily add value to applications by enhancing the search within ColdFusion, and by extending search to content outside ColdFusion.
Search Within ColdFusion MX
To ensure that users of ColdFusion applications can find the specific information they need in databases and hundreds of file formats, Macromedia integrated Verity full-text search. This includes:
- Full-text search of all ColdFusion content
- The ability to search a wide range of document types, including HTML, binary documents, and database records
- The ability to limit search to specific groups, or collections of documents, in order to enable subject-specific searches
- Support for multiple languages, including most European and Asian languages
- Fielded search against index metadata
One of the strengths of Verity search solutions is that they can be configured to meet specific business and technical objectives. To optimize search with the ColdFusion environment, Macromedia implemented Verity to support content of the following basic data types:
- Text files such as HTML pages and CFML pages
- Binary documents
- Record sets returned from cfquery, cfldap, and cfpop queries
Typical Applications of Verity Search
By taking advantage of Verity's flexibility you can add significant value to your ColdFusion applications. Typical uses of Verity search within ColdFusion include:
- Indexing the content of a Web site and providing a generalized search mechanism, such as the familiar search box
- Indexing specific directories that contain documents on a specific topic in order to provide subject-based searching, or to limit the focus of searches to specific groups of documents
- Indexing cfquery record sets into a single Verity Collection and letting users search against the collection with a single query rather than requiring them to perform multiple database queries to return the same data
- Indexing cfldap and cfpop query results
- Indexing e-mail generated by ColdFusion application pages and making the resulting Verity Collection available for searching from your ColdFusion application pages
- Building Verity Collections with inventory data and making those collections available for searching from your ColdFusion application pages
- Supporting international users in a range of languages, using the cfindex, cfcollection, and cfsearch tags
One of the most obvious advantages of Verity search embedded in ColdFusion is its performance. For example, using Verity to index the output from database queries and then to perform searches against the indexed record sets is much faster than using SQL to search databases directly. Additional advantages of Verity over other search methods include:
- Superior relevancy of search results lists
- The display of document titles and summaries in search results lists
- Elimination of the need to programmatically create query constructs by allowing novice and expert users alike to use the same type of full-text queries they're used to using on the Web
- Indexing of database text fields, such as notes and product descriptions, that cannot be effectively indexed by native database tools
- The indexing and display of document URLs in results lists, a valuable document management feature
The good news is that Verity's advanced search features are straightforward to deploy within ColdFusion MX. In general, adding optimized osearch to your application involves three basic tasks:
- Creating a Verity Collection
- Indexing the content within your ColdFusion application
- Designing a search interface
There are pros and cons to using either the ColdFusion MX Administrator or CFML for deploying Verity search within ColdFusion applications. Refer to Table 2 to determine which is appropriate for your application and information environment.
Just as there is more than one method for deploying Verity search, you can configure your search implementation to meet specific business objectives. Primarily, you do this by running cfsearch or cfquery. Table 3 lists the advantages and uses of each.
Optimizing Search Relevancy
Once you've deployed the Verity search, you can also optimize its relevancy for your specific information environment. The ColdFusion implementation of Verity Query Language (VQL) uses operators and modifiers. These can either be used directly by advanced users, or implemented transparently so that they are applied automatically to all queries. The following are some of the more commonly used VQL operators:
For additional resources, see the ColdFusion MX documentation available at www.macromedia.com. All product questions and support for ColdFusion, including the Verity search integrated into ColdFusion MX, are provided by Macromedia.
Extending and Enhancing the Search in Macromedia ColdFusion MX
As advanced as the Verity search is within ColdFusion, you may eventually want to deploy ColdFusion applications with enhanced search capabilities that are not possible using Macromedia's implementation of Verity. This is not due to any inherent limitations on Verity's part as much as it is a result of the robust, advanced capabilities of ColdFusion that enable you to develop ever more powerful applications. Verity has found that the need for additional search features is largely driven by three key requirements, along with a less common fourth requirement:
1. Searching content outside of ColdFusion. The Verity search functionality within ColdFusion is limited to searching content and database records within your ColdFusion application. Many ColdFusion developers need to extend the search functionality to areas outside of ColdFusion, such as intranets, external Web sites, file servers, external databases, Microsoft Exchange, and third-party document management systems.
2. Additional Administrative Tools. To meet their specific application requirements, some developers require additional administrative tools.
3. Adding more advanced end-user interface options. Another common request among ColdFusion developers is the ability to add advanced search features not included with the ColdFusion implementation of Verity, such as spell checker/recommendations, and search-term highlighting.
4. Searching over 250,000 documents and/or records. Occasionally, developers will build applications that exceed the 250,000 documents and/or records that the ColdFusion implementation of Verity is limited to searching. For applications that must search large databases or repositories, developers need to extend this limit. This topic is not covered in this article. For information on overcoming the document number limitations of ColdFusion, contact [email protected].
Extending and Enhancing Search with Verity Ultraseek
The three most common ColdFusion search enhancements listed above can be accomplished with the addition of Verity Ultraseek. This downloadable search engine is easily integrated into applications, using its available Java API. It can also be readily deployed into mixed application environments, using its Web services interface, which supports both the .NET and J2EE platforms. A number of factors make Ultraseek the best choice for extending search outside of the ColdFusion environment and providing additional administration tools:
- Ultraseek's easy-to-implement, set-and-forget design requires extremely low ongoing maintenance and overhead
- Ultraseek's end-user interfaces (i.e., search boxes and results lists) are similar to those of the Verity search embedded in ColdFusion
- Ultraseek provides enterprise-class search at a price point in line with ColdFusion
ColdFusion Search-Specific Tags
Creating a Collection with the cfcollection tag
When using the cfcollection tag, you can specify the same attributes as in the ColdFusion MX Administrator:
- Action: (Optional) The action to perform on the collection (create, delete, repair, or optimize). The default value for the action attribute is list. For more information, see cfcollection in CFML Reference.
- Collection: The name of the new collection, or the name of a collection on which you will perform an action.
- Path: The location for the Verity collection.
- Language: (Optional) The language used to create the collection (English, by default).
<cfcollection action = "create" collection = "a_new_collection" path = "c:\CFusionMX\verity\collections\">
Indexing a Collection Using the cfindex tag
You can index a collection in CFML using the cfindex tag, which eliminates the need to use the ColdFusion MX Administrator.
- Collection: The name of the collection. If you are indexing an external collection (external = "Yes"), you must also specify the fully qualified path for the collection.
- Action: (Optional) Can be update (the default action), delete, purge, or refresh.
- Extensions: (Optional) The delimited list of file extensions that ColdFusion uses to index files if type="Path".
- Key: (Optional) The path containing the files you are indexing if type="path".
- URLpath: (Optional) The URL path for files if type="file" and type="path". When the collection is searched with cfsearch, the pathname is automatically prefixed to filenames and returned as the URL attribute.
- Recurse: (Optional) Yes or No. Yes specifies, if type = "Path", that directories below the path specified in the key attribute are included in the indexing operation.
- Language: (Optional) The language of the collection. English is the default.
You use the cfsearch tag to search an indexed collection. Searching a Verity Collection is similar to a standard ColdFusion query: both use a dedicated ColdFusion tag that requires a name attribute for their searches.
The following are important attributes for the cfsearch tag:
- Name: The name of the search query.
- Collection: The name of the collection(s) being searched. Use a fully qualified path for an external collection. Separate multiple collections with a comma; for example, collection = "sprocket_docs,CodeColl".
- Criteria: The search target (can be dynamic).
- RecordCount: The total number of records returned by the search.
- CurrentRow: The current row of the record set being processed by cfoutput.
- RecordsSearched: The total number of records in the index that were searched. If no records were returned in the search, this property returns a null value.
|Dave Hardwick 01/27/05 10:25:47 AM EST|
Ironic, the URL for the Ultraseek extension for CF provided a "404: File Not Found." When I searched for cfsearch or CFMX on Verity's Web site, it returned no results!
I was surprised this article did not mention KF2 Server. Crawling the site is a very powerful option that allows for regular expression type controls over what is and is not indexed, be it directories or filenames.
|Peter 01/20/05 03:53:53 AM EST|
Great article, glad to see Verity with CF getting some exposure. I agree that it is entirely about servicing the end user but have a problem that I can't seem to get around.
I have a handful of long (250+ page) pdfs which I can add to a verity collection and then search. What my end users really want, however, is not just to identify matching documents, but to identify the passages within those documents that match their search criteria.
Is anything along these lines possible with the Verity engine included with CF?
Companies can harness IoT and predictive analytics to sustain business continuity; predict and manage site performance during emergencies; minimize expensive reactive maintenance; and forecast equipment and maintenance budgets and expenditures. Providing cost-effective, uninterrupted service is challenging, particularly for organizations with geographically dispersed operations.
Feb. 12, 2016 06:00 PM EST
SYS-CON Events announced today that Interoute, owner-operator of one of Europe's largest networks and a global cloud services platform, has been named “Bronze Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. Interoute is the owner-operator of one of Europe's largest networks and a global cloud services platform which encompasses 12 data centers, 14 virtual data centers and 31 colocation centers, with connections to 195 ad...
Feb. 12, 2016 04:15 PM EST Reads: 425
Join us at Cloud Expo | @ThingsExpo 2016 – June 7-9 at the Javits Center in New York City and November 1-3 at the Santa Clara Convention Center in Santa Clara, CA – and deliver your unique message in a way that is striking and unforgettable by taking advantage of SYS-CON's unmatched high-impact, result-driven event / media packages.
Feb. 12, 2016 03:00 PM EST
There will be new vendors providing applications, middleware, and connected devices to support the thriving IoT ecosystem. This essentially means that electronic device manufacturers will also be in the software business. Many will be new to building embedded software or robust software. This creates an increased importance on software quality, particularly within the Industrial Internet of Things where business-critical applications are becoming dependent on products controlled by software. Qua...
Feb. 12, 2016 02:15 PM EST
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management...
Feb. 12, 2016 02:15 PM EST Reads: 449
With an estimated 50 billion devices connected to the Internet by 2020, several industries will begin to expand their capabilities for retaining end point data at the edge to better utilize the range of data types and sheer volume of M2M data generated by the Internet of Things. In his session at @ThingsExpo, Don DeLoach, CEO and President of Infobright, will discuss the infrastructures businesses will need to implement to handle this explosion of data by providing specific use cases for filte...
Feb. 12, 2016 01:00 PM EST Reads: 228
SYS-CON Events announced today that Pythian, a global IT services company specializing in helping companies adopt disruptive technologies to optimize revenue-generating systems, has been named “Bronze Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. Founded in 1997, Pythian is a global IT services company that helps companies compete by adopting disruptive technologies such as cloud, Big Data, advanced analytics, and DevO...
Feb. 12, 2016 12:30 PM EST Reads: 263
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Avere delivers a more modern architectural approach to storage that doesn’t require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbuilding of data centers ...
Feb. 12, 2016 12:30 PM EST Reads: 108
SYS-CON Events announced today that Alert Logic, Inc., the leading provider of Security-as-a-Service solutions for the cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Alert Logic, Inc., provides Security-as-a-Service for on-premises, cloud, and hybrid infrastructures, delivering deep security insight and continuous protection for customers at a lower cost than traditional security solutions. Ful...
Feb. 12, 2016 11:45 AM EST Reads: 453
Fortunately, meaningful and tangible business cases for IoT are plentiful in a broad array of industries and vertical markets. These range from simple warranty cost reduction for capital intensive assets, to minimizing downtime for vital business tools, to creating feedback loops improving product design, to improving and enhancing enterprise customer experiences. All of these business cases, which will be briefly explored in this session, hinge on cost effectively extracting relevant data from ...
Feb. 12, 2016 11:15 AM EST Reads: 141
SYS-CON Events announced today that iDevices®, the preeminent brand in the connected home industry, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. iDevices, the preeminent brand in the connected home industry, has a growing line of HomeKit-enabled products available at the largest retailers worldwide. Through the “Designed with iDevices” co-development program and its custom-built IoT Cloud Infrastruc...
Feb. 12, 2016 10:00 AM EST Reads: 133
As enterprises work to take advantage of Big Data technologies, they frequently become distracted by product-level decisions. In most new Big Data builds this approach is completely counter-productive: it presupposes tools that may not be a fit for development teams, forces IT to take on the burden of evaluating and maintaining unfamiliar technology, and represents a major up-front expense. In his session at @BigDataExpo at @ThingsExpo, Andrew Warfield, CTO and Co-Founder of Coho Data, will dis...
Feb. 12, 2016 09:30 AM EST Reads: 217
The Quantified Economy represents the total global addressable market (TAM) for IoT that, according to a recent IDC report, will grow to an unprecedented $1.3 trillion by 2019. With this the third wave of the Internet-global proliferation of connected devices, appliances and sensors is poised to take off in 2016. In his session at @ThingsExpo, David McLauchlan, CEO and co-founder of Buddy Platform, will discuss how the ability to access and analyze the massive volume of streaming data from mil...
Feb. 12, 2016 09:00 AM EST
WebSocket is effectively a persistent and fat pipe that is compatible with a standard web infrastructure; a "TCP for the Web." If you think of WebSocket in this light, there are other more hugely interesting applications of WebSocket than just simply sending data to a browser. In his session at 18th Cloud Expo, Frank Greco, Director of Technology for Kaazing Corporation, will compare other modern web connectivity methods such as HTTP/2, HTTP Streaming, Server-Sent Events and new W3C event APIs ...
Feb. 12, 2016 09:00 AM EST
SYS-CON Events announced today that Men & Mice, the leading global provider of DNS, DHCP and IP address management overlay solutions, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. The Men & Mice Suite overlay solution is already known for its powerful application in heterogeneous operating environments, enabling enterprises to scale without fuss. Building on a solid range of diverse platform support,...
Feb. 12, 2016 06:00 AM EST Reads: 255
Eighty percent of a data scientist’s time is spent gathering and cleaning up data, and 80% of all data is unstructured and almost never analyzed. Cognitive computing, in combination with Big Data, is changing the equation by creating data reservoirs and using natural language processing to enable analysis of unstructured data sources. This is impacting every aspect of the analytics profession from how data is mined (and by whom) to how it is delivered. This is not some futuristic vision: it's ha...
Feb. 12, 2016 05:45 AM EST Reads: 463
Silver Spring Networks, Inc. (NYSE: SSNI) extended its Internet of Things technology platform with performance enhancements to Gen5 – its fifth generation critical infrastructure networking platform. Already delivering nearly 23 million devices on five continents as one of the leading networking providers in the market, Silver Spring announced it is doubling the maximum speed of its Gen5 network to up to 2.4 Mbps, increasing computational performance by 10x, supporting simultaneous mesh communic...
Feb. 12, 2016 05:00 AM EST
The cloud promises new levels of agility and cost-savings for Big Data, data warehousing and analytics. But it’s challenging to understand all the options – from IaaS and PaaS to newer services like HaaS (Hadoop as a Service) and BDaaS (Big Data as a Service). In her session at @BigDataExpo at @ThingsExpo, Hannah Smalltree, a director at Cazena, will provide an educational overview of emerging “as-a-service” options for Big Data in the cloud. This is critical background for IT and data profes...
Feb. 12, 2016 02:30 AM EST Reads: 234
With the Apple Watch making its way onto wrists all over the world, it’s only a matter of time before it becomes a staple in the workplace. In fact, Forrester reported that 68 percent of technology and business decision-makers characterize wearables as a top priority for 2015. Recognizing their business value early on, FinancialForce.com was the first to bring ERP to wearables, helping streamline communication across front and back office functions. In his session at @ThingsExpo, Kevin Roberts...
Feb. 12, 2016 12:45 AM EST Reads: 413
Cognitive Computing is becoming the foundation for a new generation of solutions that have the potential to transform business. Unlike traditional approaches to building solutions, a cognitive computing approach allows the data to help determine the way applications are designed. This contrasts with conventional software development that begins with defining logic based on the current way a business operates. In her session at 18th Cloud Expo, Judith S. Hurwitz, President and CEO of Hurwitz & ...
Feb. 12, 2016 12:00 AM EST Reads: 291