|By Charlie Arehart||
|April 3, 2003 12:00 AM EST||
Amazon.com, the "earth's biggest selection," has exposed their entire catalog of products, product reviews and ratings, user-created book lists, and more via Web services. Whether you're new to Web services or not, and even if you don't care so much about accessing Amazon's data, it presents an interesting case study of doing Web services in CFMX. You might even be able to turn the opportunity into profit!
In this month's Journeyman column, I'll walk you through using Amazon Web Services (AWS), and along the way will point out some of the key points you need to know about CF (and Dreamweaver) to be able to use them. If you've not explored this feature of CFMX yet, the good news is that it's really quite easy. And if you have begun using Web services in CFMX, you might learn a thing or two along the way.
ColdFusion MX supports both the consumption and creation of Web services. In this article, we'll focus only on consuming them. There have been several articles both in this magazine and at the Macromedia DevNet site that cover more details about creating Web services (using CFCs). You may want to look for and refer to those for more background after reading this if you're entirely new to the process of working with Web services in general, or in CF.
But any CF developer should be able to follow along here quite easily. CF makes this just extraordinarily trivial to set up compared to many other environments.
A Quick Example
I want to start off quickly with an example of something that you can run right away without having to write any code. Save the code in Listing 1 to your CFMX server environment, placing it in a directory where you can execute other CFMX templates. This might be a subdirectory of your webroot, such as c:\cfusionmx\wwwroot. Save it as amazon-search-simple.cfm.
Don't worry if you don't understand entirely what's going on. Just save it and run it, then take a look at the code.
This simple example will retrieve a selection of books from Amazon's database, based on a "keyword" provided in the query string. In other words, to search for a book, enter ?keyword =somevalue on the URL when calling this template. I've designed it so that if you don't supply any, it will search for CF books by default. (Certainly, a more complete example would provide a form for the user to enter values instead.) Figure 1 shows a screenshot of how the results might appear.
This is a very simplistic example that leaves a few questions unanswered: Why were only 10 records returned? How do you request additional records? What's the sort order and how can that be changed? What if there are multiple authors? What other data is returned about the books? What other sort of searches can I do? We'll address those later.
For now, let's take a look at some fundamentals of setting up the code to call the Web service and process its results.
The Printed Documentation About the Web Service
Notice that the CFINVOKE is calling the Web service at the URL: http://soap.amazon.com/schemas2/AmazonWebServices.wsdl. Where did I get that value? It's provided in the documentation about the Web services, which is available via download as part of the "Developer's Kit" available at www.amazon.com/webservices.
That document, the "API and Integration Guide," offers a pretty complete discussion about many aspects of working with AWS, both in the SOAP Web service approach (that we're using here) as well as a pure XML approach for those who might prefer that. For a discussion of processing Web services in XML in an older form of AWS, see the sidebar about Jeffrey Houser's article in the Macromedia DevNet.
The Guide also lists all the available types of Web service methods that can be called, each of which enables different kinds of searches, such as by author name, artist/musician name, actor name, directory name, ASIN (Amazon's unique product IDs, which for a book is the same as its ISBN), UPC code, and more. Some of the listed methods include:
The Data Passed in to the Web Service Method
The Guide also explains that each method further requires that a specific argument be passed in on the request to execute that method, and it documents the argument name expected for each method, as well as the properties that the argument should include.
For instance, in the case of the KeywordSearchRequest used in Listing 1, the Guide shows that the expected name of the argument to be passed in is also KeywordSearchRequest, and it's to be composed of the following properties (with their data types):
- keyword (string)
- page (string)
- mode (string)
- tag (string)
- type (string)
- sort (string, optional)
- devtag (string)
For now, let's take a look again at Listing 1. In ColdFusion, the way to pass those properties as an argument called "Keyword SearchRequest" is to package them as a structure, which I do in the first few lines of code. Then in this example that structure is passed on the CFINVOKEARGUMENT tag:
While you could call the actual structure any name you want (I called it "aKeywordSearchRequest"), you must pass KeywordSearchRequest as the value in that tag's NAME attribute since that's what the Web service is expecting. (Note that you could also avoid the CFINVOKEARGUMENT tag by placing the attribute=value pair, KeywordSearchRequest="#aKeywordSearchRequest#", on the CFINVOKE tag itself. The two approaches achieve the same result.)
The Data Returned from the Web Services Method
Finally, the Guide also shows what results are returned, including the names of fields, arrays, and other objects that might be returned. In the case of a KeywordSearchRequest, the method returns a productInfo structure (referred to as a "node" in the Guide), whose keys describe the set of search results. The Guide explains that the productInfo node/structure contains the following data:
You'll notice that in Listing 1, I provide a name to refer to the ProductInfo results by way of the RETURNVARIABLE attribute of CFINVOKE, and I named it "aProductInfo." I could have called it anything I like. This is available as a structure in ColdFusion. I then use that to create output about the results in general, as in the first line inside the CFOUTPUT, which shows how many records were found (aProductInfo.TotalResults).
As stated above, the information about each record is found in a Details array within that ProductInfo result variable (in our case, the aProductInfo structure), and each element in the array will be a structure describing each product. The guide further describes the information returned for each product:
That's quite a bit of detail. Do you see now that this whole process is like performing a remote query against the Amazon database? We really get a lot of data back, including links to pictures, actual current prices, and more.
And this set of data is just what Amazon calls the "lite" set of data. The Guide also describes a "heavy" set of data that can provide much more information, such as sales rank, ratings, availability, features, tracks on CDs, reviews, similar products, and more!
You'll notice that in the simple example I have a CFLOOP looping over each element in the aProductInfo.details array, and for each I'm outputting just the product name and authors. Note, too, that the authors' value is itself another array, holding the name of each author, if any.
You might think that it would be useful to simply perform a CFDUMP on this chunk of data returned from the Web service. Sadly, you won't like the result. Because CF sees this aProductInfo variable as an object, rather than a structure with an array of other structures, if you try to dump it, you just get a Java-oriented description of the object.
This may bother you, though, because it means it can be difficult to know what exactly is returned by the Web service call. There's good news, however. Dreamweaver offers a mechanism to at least be able to see what kind of data will be returned, to help you build the code to output or otherwise process that returned data.
Using Dreamweaver MX to Explore the Web Service Definition
It can be cumbersome referring to the printed documentation to find out such information about the Web service as its available methods and their expected arguments, and the names and types of the returned results from each. Fortunately, Web services are self-documenting.
If you execute the URL of the Web service itself (the value entered in the CFINVOKE WEBSERVICE attribute) via a browser, that will generate an XML stream in return that does document how the Web service can be executed and the type of data that it will return. See Listing 2 for a small portion of the 1,000+ lines of XML returned that describe the AWS. You have to be pretty savvy in both XML and SOAP (and patient) to be able to read through that document to discover needed information.
Fortunately, Dreamweaver MX (DWMX) has a very useful feature for exploring Web service methods, arguments, and return results. From the Application panel, select the Components tab. Choose "Web Services" from the drop-down next to the "+" sign. See Figure 2:
Finally, select that "+" sign to the right of the drop-down menu where you just selected "Web services." At the prompt, paste in the URL of whatever Web service you care to browse. In our case, it's the URL in the CFINVOKE tag. In this same screen, be sure that "ColdFusion MX" is selected for the "Proxy Generator" drop-down.
When you select "ok," DWMX will go out to the Web service and examine that same XML stream we just discussed. But more important, when it's finished a few moments later, it will now have registered the Web service within DWMX for whatever "site" you had open at the time. Figure 2 happens to show a few Web services already having been registered on my site.
Further, if you expand any one of these Web services in DWMX, you will be shown all the Web service methods, as well as the arguments expected within each. In Figure 3, you can see I've selected the keyWordsearchRequest method, and its expected KeywordSearchRequest argument shown underneath it. A couple of things worth noting are that the methodname and the argumentname are preceded by other words.
In the first case, the ProductInfo refers to the data type of what's returned from the method. In the second case, the KeywordRequest refers to the data type of what's to be passed in the argument. But what are those and where do you find out more about each of them?
At the bottom of this list of methods for the Web service, as seen in Figure 3, there is another element listed as "structs." Expanding that, you'll see several entries, with one for each kind of data type (or node or structure) that is mentioned in that display of data types for methods and request arguments. Figure 4 shows the KeywordRequest and (the beginning of) the ProductInfo structs, expanded to show their properties.
The list in the KeywordRequest (technically, is com.amazon.soap.Keyword Request, at the top of Figure 4) maps to the same KeywordSearchRequest argument discussed at the opening of "The Data Passed in to the Web Service Method," above. And the ProductInfo item at the bottom of Figure 4 is the very ProductInfo described in "The Data Returned from the Web Service Method," above.
It's worth noting that the name of these structs is the same as the data type name indicated before the names of methods and arguments in the list above "structs." These struct names do not necessarily equate to the names of the arguments to be passed to the methods. In our example, Figure 3 showed that the method KeywordSearchRequest expected to be passed an argument of the same name. But that argument was indicated to be of a data type called simply KeywordRequest. That's what is listed in the "structs" section. And we need to then create a structure whose keynames map to those properties listed for KeywordRequest in Figure 4. It can be a little confusing. Hopefully this example will help get you straightened out as you explore this further.
Building Code with Dreamweaver
Finally, it's important to note that you can use all this information for more than just documentation. Click and drag a method name onto the code editing area in DWMX. If you're in "code view" of DWMX (View>Code), you'll see that DWMX will build a skeletal CFINVOKE and nested CFARGUMENT for you, with all the method names and appropriate argument names all properly filled in. Nifty!
Indeed, the CFINVOKE code in Listing 1 was generated with this very feature. All I needed to change was the VALUE attribute in the CFINVOKEARGUMENT, to specify the name of the structure I had built in the first few lines. And the keys in that structure were obtained by dragging and dropping the properties in the aforementioned KeywordRequest "struct."
Even dragging the KeywordRequest itself will create a line of code to create a new structure, since that is indeed what you must do to create this input argument structure. Unfortunately, the code generated is a CFSET doing a structnew(). I changed it in Listing 1 to all take place in a CFSCRIPT operation. Do whatever makes you comfortable. At least you now know that some of the code can be created for you.
When it comes time to build the output processing aspect, go to the ProductInfo "struct." Again, you can drag and drop the totalResults property name into your CFOUTPUT code. And when you see that there's a details array listed, that suggests you need to find the element of the same name in the "structs" area to find out what elements are within that data type.
If that shows that there's possibly an array of tracks within that, you'd then find an element describing those. Just note that in the case of an array like authors inside of ProductInfo, it shows that the data type for that is simply a string. That means there will be no further breakdown of elements for authors. It indicates that this is just an array of author names.
Before concluding this article, let's take just a brief look at some of the ways you might modify and enhance the example in Listing 1.
Enhancing the Example
We pointed out previously that the simple example we showed in Listing 1 left quite a bit unanswered. Let's address a few of these points quickly. First, did you notice that the Amazon search mechanism returned only 10 records at a time? That's by design, and it currently cannot be changed. What we can do is cause it to return another "page" of 10 records by modifying the page property in the KeyWordSearchRequest argument:
Of course, you'll want to change that programmatically, to facilitate paging or scrolling through the results. A greatly enhanced example in Listing 3 addresses that as well as a few more points.
Did you wonder what sort order the results were shown in? The Web service defaults to sorting the results in "Featured Item" order for the keyword search we were doing. You can change that sort order if you'd like. I've commented out a line of code that would cause an alternative sort by sales rank:
Note also that the aKeywordRequest structure is also where the "lite" or "heavy" type of result can be requested in the "type" property. And finally there's a "mode" property that has significance in indicating the kind of product you're searching for. In our case, it's set to "books." The AWS Guide offers more information on finding out valid values for the "mode" property. Just be careful not to make a mistake in that sort of designation. If you mistakenly left it as "book" rather than "books," you'd get a rather unclear error: "Could not perform Web service invocation 'KeywordSearchRequest' because AxisFault fault".
That's a fault in the underlying Axis processing, not from Amazon it seems, but it's an error to be careful about either way. It's worth noting that CF's Web services foundation is built upon the Apache Axis project, and while the base version of CF and the first two updaters continued to use a .9 release of that Axis project, the Updater 3 has resolved that, giving us the full version. It may not impact using AWS, but it's worth noting for other Web services.
Be aware also of some possible surprises about the data as you're processing results from a Web service, such as in the Details array in our example. For instance, we've said before that the authors field was an array. In the simple example, I was processing only the first author. You should be prepared to handle all of them. The enhanced code in Listing 3 does that.
Also note that a book might not have any authors at all. This is an interesting situation, because while you may think you can use CF's isdefined() function to check for that, it will not work. Sadly, you really need to test for a null value in that property, and CF offers no means to do that. You'll notice that I use CFTRY/CFCATCH processing so that if I refer to a field that has no values, instead of getting an error, I just continue processing.
The enhanced version of the example, in Listing 3, does that and more, performing scrolling through records and showing how to handle multiple or zero authors. There are still more things that could be done, such as having a form for inputs and controlling other aspects like the page, etc. It could also perform caching of the Web service, so as not to visit it too often when data may not change. This isn't a trivial problem, but it is one worth exploring.
One last point to note is that if you want to quickly and easily explore the data that would be returned from the Amazon Web services, you can always use the XML service I alluded to above. Again, see the API for more information, but with it you can enter a URL such as this in your browser: http://xml.amazon.com/onca/xml2?dev-t= yourdevtag&t=syndic8-20 &KeywordSearch=ColdFusion&page= 1&mode=books&type=lite&f=xml
So now you know how to access the Amazon Web service, as well as find documentation both in print and electronically to understand better what is possible with that service and its methods. And you've learned how to take advantage of Dreamweaver MX to make browsing Web service properties, and even building code, easier.
Now you just need to explore all the possibilities that Amazon Web Services provides. We've really only scratched the surface. As I alluded to previously, there is also a mechanism that would allow you to enable one-click and other shopping cart functionality on your site that can be executed against the Amazon servers to allow your visitors to make purchases. And if you obtain and use that Associates ID, you can earn from 5-15% on their purchases. Wow! Check out the Guide for more information.
If you're more a developer than a budding retailer, consider how you might still turn AWS to profit. First of all, you could build vastly enhanced mechanisms to help retailers show off certain products. Remember that there are reviews, ratings, similar products, and a whole lot more that you can tap into with AWS. Apply a little Flash and Flash Remoting, and you can build something that may truly be unique.
Consider also that you can perform searches by seller, so that you could help a seller create better inventory or other product management and integration systems to sync their own data with that about their transactions at Amazon. The sky really is the limit.
Be sure to read the API and Integration Guide. It's not too long and offers lots of useful information. You'll also want to download that Developer's Kit as it has lots of examples (though none currently in CF, which I hope to address with them.) There is also a Discussion Board at that www.amazon.com/webservices site that's friendly, supportive (including AWS staffers), and not generating more than several posts a day. Sometimes there are very useful discussions that take place.
Finally, I'll remind you to seek out some other articles (and books) on the topic of Web services in CFMX, and Amazon Web Services in particular. Don't be afraid to do a Google search and see where it takes you. Even a resource shows using them in ASP or PHP may still share new perspectives that could apply to your using Web ser-vices in CFMX. That's the cool thing about Web services: they don't care what kind of client you are. They're there to serve us all. Enjoy.
|Mark 07/11/03 01:21:00 PM EDT|
This code doesn't work! It's giving me a huge error message if Amazon doesn't have anything match the query results. Help!
|Charlie Arehart 07/11/03 04:15:00 PM EDT|
Mark, you're overreacting a bit, don't you think. Yes, the code does work. It just has the error you note when you pass it something it doesn't find. So the truth is it doesn't "always" work. I'm not meaning to split hairs.
I had explained in the article that if some unexpected result came back (such as if there were no authors and you tried to loop over them), there could be an error just like you're seeing, and I spoke about using CFTRY to handle that.
I'll grant that I hadn't tested what would happen if the entire search result came back empty. I'll make the same assertion I did in the article: slap a CFTRY around it.
In this case, since the error arises on the CFINVOKE itself, put it in a CFTRY and CFCATCH any error. If it works when there is a result but not when there s not, this will suffice to prevent the error.
If you want to go further and detect and handle different errors from the CFINVOKE (such as if the service is down, etc.), you can. I'll offer a simple example here but I'll leave expanding on it as an "exercise for the reader". I wasn't writing a production read example. Just meaning to give you enough to take the ball and run with it.
Try this in place of the CFINVOKE:
- Where Are RIA Technologies Headed in 2008?
- The Next Programming Models, RIAs and Composite Applications
- AJAX World RIA Conference & Expo Kicks Off in New York City
- Constructing an Application with Flash Forms from the Ground Up
- Building a Zip Code Proximity Search with ColdFusion
- Personal Branding Checklist
- CFEclipse: The Developer's IDE, Eclipse For ColdFusion
- Has the Technology Bounceback Begun?
- Adobe Flex 2: Advanced DataGrid
- i-Technology Viewpoint: We Need Not More Frameworks, But Better Programmers
- Web Services Using ColdFusion and Apache CXF
- Passing Parameters to Flex That Works