Exploring Amazon Web Services with ColdFusion MX

Apr. 3, 2003 12:00 AM

Amazon.com, the "earth's biggest selection," has exposed their entire catalog of products, product reviews and ratings, user-created book lists, and more via Web services. Whether you're new to Web services or not, and even if you don't care so much about accessing Amazon's data, it presents an interesting case study of doing Web services in CFMX. You might even be able to turn the opportunity into profit!

In this month's Journeyman column, I'll walk you through using Amazon Web Services (AWS), and along the way will point out some of the key points you need to know about CF (and Dreamweaver) to be able to use them. If you've not explored this feature of CFMX yet, the good news is that it's really quite easy. And if you have begun using Web services in CFMX, you might learn a thing or two along the way.

ColdFusion MX supports both the consumption and creation of Web services. In this article, we'll focus only on consuming them. There have been several articles both in this magazine and at the Macromedia DevNet site that cover more details about creating Web services (using CFCs). You may want to look for and refer to those for more background after reading this if you're entirely new to the process of working with Web services in general, or in CF.

But any CF developer should be able to follow along here quite easily. CF makes this just extraordinarily trivial to set up compared to many other environments.

A Quick Example
I want to start off quickly with an example of something that you can run right away without having to write any code. Save the code in Listing 1 to your CFMX server environment, placing it in a directory where you can execute other CFMX templates. This might be a subdirectory of your webroot, such as c:\cfusionmx\wwwroot. Save it as amazon-search-simple.cfm.

Don't worry if you don't understand entirely what's going on. Just save it and run it, then take a look at the code.

Sidebar
Before you try to browse this URL to execute the request, note that Amazon expects developers to obtain their own "developer tag." This is free and can be done easily at www.amazon.com/webservices. You'll get one on the spot. Apply that value in the one line of code:

aKeywordRequest.devtag="yourtag";

This simple example will retrieve a selection of books from Amazon's database, based on a "keyword" provided in the query string. In other words, to search for a book, enter ?keyword =somevalue on the URL when calling this template. I've designed it so that if you don't supply any, it will search for CF books by default. (Certainly, a more complete example would provide a form for the user to enter values instead.) Figure 1 shows a screenshot of how the results might appear.

This is a very simplistic example that leaves a few questions unanswered: Why were only 10 records returned? How do you request additional records? What's the sort order and how can that be changed? What if there are multiple authors? What other data is returned about the books? What other sort of searches can I do? We'll address those later.

For now, let's take a look at some fundamentals of setting up the code to call the Web service and process its results.

The Printed Documentation About the Web Service
Notice that the CFINVOKE is calling the Web service at the URL: http://soap.amazon.com/schemas2/AmazonWebServices.wsdl. Where did I get that value? It's provided in the documentation about the Web services, which is available via download as part of the "Developer's Kit" available at www.amazon.com/webservices.

That document, the "API and Integration Guide," offers a pretty complete discussion about many aspects of working with AWS, both in the SOAP Web service approach (that we're using here) as well as a pure XML approach for those who might prefer that. For a discussion of processing Web services in XML in an older form of AWS, see the sidebar about Jeffrey Houser's article in the Macromedia DevNet.

Sidebar
Jeffrey Houser has done another article on integrating Amazon Web services with CF, at www.macromedia.com/devnet/mx/ coldfusion/articles/wsamazon.html. That article is based on an older form of obtaining Amazon Web services as pure XML. Though the service he calls and the code he wrote still work, the Amazon Developer's Kit describes a newer approach to requesting pure XML responses from AWS. There's also an option to pass in an XSLT file to modify the output from that XML-based mechanism. In either case, the approach I describe in this article saves you having to perform any XML parsing or processing. That's one benefit of SOAP-based Web services processing!

The Guide also lists all the available types of Web service methods that can be called, each of which enables different kinds of searches, such as by author name, artist/musician name, actor name, directory name, ASIN (Amazon's unique product IDs, which for a book is the same as its ISBN), UPC code, and more. Some of the listed methods include:

KeywordSearchRequest
AuthorSearchRequest
ArtistSearchRequest
ActorSearchRequest

You see the pattern to their names.

The Data Passed in to the Web Service Method
The Guide also explains that each method further requires that a specific argument be passed in on the request to execute that method, and it documents the argument name expected for each method, as well as the properties that the argument should include.

For instance, in the case of the KeywordSearchRequest used in Listing 1, the Guide shows that the expected name of the argument to be passed in is also KeywordSearchRequest, and it's to be composed of the following properties (with their data types):

keyword (string)
page (string)
mode (string)
tag (string)
type (string)
sort (string, optional)
devtag (string)

The Guide further explains the meaning of each of those properties. I'll discuss some of them later.

For now, let's take a look again at Listing 1. In ColdFusion, the way to pass those properties as an argument called "Keyword SearchRequest" is to package them as a structure, which I do in the first few lines of code. Then in this example that structure is passed on the CFINVOKEARGUMENT tag:

While you could call the actual structure any name you want (I called it "aKeywordSearchRequest"), you must pass KeywordSearchRequest as the value in that tag's NAME attribute since that's what the Web service is expecting. (Note that you could also avoid the CFINVOKEARGUMENT tag by placing the attribute=value pair, KeywordSearchRequest="#aKeywordSearchRequest#", on the CFINVOKE tag itself. The two approaches achieve the same result.)

The Data Returned from the Web Services Method
Finally, the Guide also shows what results are returned, including the names of fields, arrays, and other objects that might be returned. In the case of a KeywordSearchRequest, the method returns a productInfo structure (referred to as a "node" in the Guide), whose keys describe the set of search results. The Guide explains that the productInfo node/structure contains the following data:

TotalResults: Number of results (products) generated by search

ErrorMsg: The error message that is returned if the call fails

Details: An array describing the details about each specific product

You'll notice that in Listing 1, I provide a name to refer to the ProductInfo results by way of the RETURNVARIABLE attribute of CFINVOKE, and I named it "aProductInfo." I could have called it anything I like. This is available as a structure in ColdFusion. I then use that to create output about the results in general, as in the first line inside the CFOUTPUT, which shows how many records were found (aProductInfo.TotalResults).

As stated above, the information about each record is found in a Details array within that ProductInfo result variable (in our case, the aProductInfo structure), and each element in the array will be a structure describing each product. The guide further describes the information returned for each product:

ASIN: Amazon.com Standard Item Number

ProductName: Name or title of product

Catalog: Product category (book, music, etc.)

Artists: Musician(s) of a CD or cassette

Authors: Author(s) of a book

URL: Link to product detail page at Amazon.com

ReleaseDate: Date that the product was released to market

Manufacturer: Manufacturer of product; can represent publishers (for books), labels (for music), or studios (for films)

ImageUrlSmall: Small product image or cover art

ImageUrlMedium: Medium product image or cover art

ImageUrlLarge: Large product image or cover art

ListPrice: Usually represents the price of the product suggested by the manufacturer

OurPrice: The Amazon.com product price

Status: Indicates if we encountered any errors while processing the service request

That's quite a bit of detail. Do you see now that this whole process is like performing a remote query against the Amazon database? We really get a lot of data back, including links to pictures, actual current prices, and more.

And this set of data is just what Amazon calls the "lite" set of data. The Guide also describes a "heavy" set of data that can provide much more information, such as sales rank, ratings, availability, features, tracks on CDs, reviews, similar products, and more!

You'll notice that in the simple example I have a CFLOOP looping over each element in the aProductInfo.details array, and for each I'm outputting just the product name and authors. Note, too, that the authors' value is itself another array, holding the name of each author, if any.

You might think that it would be useful to simply perform a CFDUMP on this chunk of data returned from the Web service. Sadly, you won't like the result. Because CF sees this aProductInfo variable as an object, rather than a structure with an array of other structures, if you try to dump it, you just get a Java-oriented description of the object.

This may bother you, though, because it means it can be difficult to know what exactly is returned by the Web service call. There's good news, however. Dreamweaver offers a mechanism to at least be able to see what kind of data will be returned, to help you build the code to output or otherwise process that returned data.

Using Dreamweaver MX to Explore the Web Service Definition
It can be cumbersome referring to the printed documentation to find out such information about the Web service as its available methods and their expected arguments, and the names and types of the returned results from each. Fortunately, Web services are self-documenting.

If you execute the URL of the Web service itself (the value entered in the CFINVOKE WEBSERVICE attribute) via a browser, that will generate an XML stream in return that does document how the Web service can be executed and the type of data that it will return. See Listing 2 for a small portion of the 1,000+ lines of XML returned that describe the AWS. You have to be pretty savvy in both XML and SOAP (and patient) to be able to read through that document to discover needed information.

Fortunately, Dreamweaver MX (DWMX) has a very useful feature for exploring Web service methods, arguments, and return results. From the Application panel, select the Components tab. Choose "Web Services" from the drop-down next to the "+" sign. See Figure 2:

Finally, select that "+" sign to the right of the drop-down menu where you just selected "Web services." At the prompt, paste in the URL of whatever Web service you care to browse. In our case, it's the URL in the CFINVOKE tag. In this same screen, be sure that "ColdFusion MX" is selected for the "Proxy Generator" drop-down.

When you select "ok," DWMX will go out to the Web service and examine that same XML stream we just discussed. But more important, when it's finished a few moments later, it will now have registered the Web service within DWMX for whatever "site" you had open at the time. Figure 2 happens to show a few Web services already having been registered on my site.

Further, if you expand any one of these Web services in DWMX, you will be shown all the Web service methods, as well as the arguments expected within each. In Figure 3, you can see I've selected the keyWordsearchRequest method, and its expected KeywordSearchRequest argument shown underneath it. A couple of things worth noting are that the methodname and the argumentname are preceded by other words.

In the first case, the ProductInfo refers to the data type of what's returned from the method. In the second case, the KeywordRequest refers to the data type of what's to be passed in the argument. But what are those and where do you find out more about each of them?

At the bottom of this list of methods for the Web service, as seen in Figure 3, there is another element listed as "structs." Expanding that, you'll see several entries, with one for each kind of data type (or node or structure) that is mentioned in that display of data types for methods and request arguments. Figure 4 shows the KeywordRequest and (the beginning of) the ProductInfo structs, expanded to show their properties.

The list in the KeywordRequest (technically, is com.amazon.soap.Keyword Request, at the top of Figure 4) maps to the same KeywordSearchRequest argument discussed at the opening of "The Data Passed in to the Web Service Method," above. And the ProductInfo item at the bottom of Figure 4 is the very ProductInfo described in "The Data Returned from the Web Service Method," above.

It's worth noting that the name of these structs is the same as the data type name indicated before the names of methods and arguments in the list above "structs." These struct names do not necessarily equate to the names of the arguments to be passed to the methods. In our example, Figure 3 showed that the method KeywordSearchRequest expected to be passed an argument of the same name. But that argument was indicated to be of a data type called simply KeywordRequest. That's what is listed in the "structs" section. And we need to then create a structure whose keynames map to those properties listed for KeywordRequest in Figure 4. It can be a little confusing. Hopefully this example will help get you straightened out as you explore this further.

Building Code with Dreamweaver
Finally, it's important to note that you can use all this information for more than just documentation. Click and drag a method name onto the code editing area in DWMX. If you're in "code view" of DWMX (View>Code), you'll see that DWMX will build a skeletal CFINVOKE and nested CFARGUMENT for you, with all the method names and appropriate argument names all properly filled in. Nifty!

Indeed, the CFINVOKE code in Listing 1 was generated with this very feature. All I needed to change was the VALUE attribute in the CFINVOKEARGUMENT, to specify the name of the structure I had built in the first few lines. And the keys in that structure were obtained by dragging and dropping the properties in the aforementioned KeywordRequest "struct."

Even dragging the KeywordRequest itself will create a line of code to create a new structure, since that is indeed what you must do to create this input argument structure. Unfortunately, the code generated is a CFSET doing a structnew(). I changed it in Listing 1 to all take place in a CFSCRIPT operation. Do whatever makes you comfortable. At least you now know that some of the code can be created for you.

When it comes time to build the output processing aspect, go to the ProductInfo "struct." Again, you can drag and drop the totalResults property name into your CFOUTPUT code. And when you see that there's a details[] array listed, that suggests you need to find the element of the same name in the "structs" area to find out what elements are within that data type.

If that shows that there's possibly an array of tracks[] within that, you'd then find an element describing those. Just note that in the case of an array like authors[] inside of ProductInfo, it shows that the data type for that is simply a string. That means there will be no further breakdown of elements for authors. It indicates that this is just an array of author names.

Before concluding this article, let's take just a brief look at some of the ways you might modify and enhance the example in Listing 1.

Enhancing the Example
We pointed out previously that the simple example we showed in Listing 1 left quite a bit unanswered. Let's address a few of these points quickly. First, did you notice that the Amazon search mechanism returned only 10 records at a time? That's by design, and it currently cannot be changed. What we can do is cause it to return another "page" of 10 records by modifying the page property in the KeyWordSearchRequest argument:

aKeywordRequest.page="1";

Of course, you'll want to change that programmatically, to facilitate paging or scrolling through the results. A greatly enhanced example in Listing 3 addresses that as well as a few more points.

Did you wonder what sort order the results were shown in? The Web service defaults to sorting the results in "Featured Item" order for the keyword search we were doing. You can change that sort order if you'd like. I've commented out a line of code that would cause an alternative sort by sales rank:

aKeywordRequest.sort="+salesrank";

Note also that the aKeywordRequest structure is also where the "lite" or "heavy" type of result can be requested in the "type" property. And finally there's a "mode" property that has significance in indicating the kind of product you're searching for. In our case, it's set to "books." The AWS Guide offers more information on finding out valid values for the "mode" property. Just be careful not to make a mistake in that sort of designation. If you mistakenly left it as "book" rather than "books," you'd get a rather unclear error: "Could not perform Web service invocation 'KeywordSearchRequest' because AxisFault fault".

That's a fault in the underlying Axis processing, not from Amazon it seems, but it's an error to be careful about either way. It's worth noting that CF's Web services foundation is built upon the Apache Axis project, and while the base version of CF and the first two updaters continued to use a .9 release of that Axis project, the Updater 3 has resolved that, giving us the full version. It may not impact using AWS, but it's worth noting for other Web services.

Be aware also of some possible surprises about the data as you're processing results from a Web service, such as in the Details array in our example. For instance, we've said before that the authors field was an array. In the simple example, I was processing only the first author. You should be prepared to handle all of them. The enhanced code in Listing 3 does that.

Also note that a book might not have any authors at all. This is an interesting situation, because while you may think you can use CF's isdefined() function to check for that, it will not work. Sadly, you really need to test for a null value in that property, and CF offers no means to do that. You'll notice that I use CFTRY/CFCATCH processing so that if I refer to a field that has no values, instead of getting an error, I just continue processing.

The enhanced version of the example, in Listing 3, does that and more, performing scrolling through records and showing how to handle multiple or zero authors. There are still more things that could be done, such as having a form for inputs and controlling other aspects like the page, etc. It could also perform caching of the Web service, so as not to visit it too often when data may not change. This isn't a trivial problem, but it is one worth exploring.

One last point to note is that if you want to quickly and easily explore the data that would be returned from the Amazon Web services, you can always use the XML service I alluded to above. Again, see the API for more information, but with it you can enter a URL such as this in your browser: http://xml.amazon.com/onca/xml2?dev-t= yourdevtag&t=syndic8-20 &KeywordSearch=ColdFusion&page= 1&mode=books&type=lite&f=xml

Conclusion
So now you know how to access the Amazon Web service, as well as find documentation both in print and electronically to understand better what is possible with that service and its methods. And you've learned how to take advantage of Dreamweaver MX to make browsing Web service properties, and even building code, easier.

Now you just need to explore all the possibilities that Amazon Web Services provides. We've really only scratched the surface. As I alluded to previously, there is also a mechanism that would allow you to enable one-click and other shopping cart functionality on your site that can be executed against the Amazon servers to allow your visitors to make purchases. And if you obtain and use that Associates ID, you can earn from 5-15% on their purchases. Wow! Check out the Guide for more information.

If you're more a developer than a budding retailer, consider how you might still turn AWS to profit. First of all, you could build vastly enhanced mechanisms to help retailers show off certain products. Remember that there are reviews, ratings, similar products, and a whole lot more that you can tap into with AWS. Apply a little Flash and Flash Remoting, and you can build something that may truly be unique.

Consider also that you can perform searches by seller, so that you could help a seller create better inventory or other product management and integration systems to sync their own data with that about their transactions at Amazon. The sky really is the limit.

Be sure to read the API and Integration Guide. It's not too long and offers lots of useful information. You'll also want to download that Developer's Kit as it has lots of examples (though none currently in CF, which I hope to address with them.) There is also a Discussion Board at that www.amazon.com/webservices site that's friendly, supportive (including AWS staffers), and not generating more than several posts a day. Sometimes there are very useful discussions that take place.

Finally, I'll remind you to seek out some other articles (and books) on the topic of Web services in CFMX, and Amazon Web Services in particular. Don't be afraid to do a Google search and see where it takes you. Even a resource shows using them in ASP or PHP may still share new perspectives that could apply to your using Web ser-vices in CFMX. That's the cool thing about Web services: they don't care what kind of client you are. They're there to serve us all. Enjoy.

Related Links

About Charlie Arehart
A veteran ColdFusion developer since 1997, Charlie Arehart is a long-time contributor to the community and a recognized Adobe Community Expert. He's a certified Advanced CF Developer and Instructor for CF 4/5/6/7 and served as tech editor of CFDJ until 2003. Now an independent contractor (carehart.org) living in Alpharetta, GA, Charlie provides high-level troubleshooting/tuning assistance and training/mentoring for CF teams. He helps run the Online ColdFusion Meetup (coldfusionmeetup.com, an online CF user group), is a contributor to the CF8 WACK books by Ben Forta, and is frequently invited to speak at developer conferences and user groups worldwide.