Searching Yahoo using REST
REST is often considered the simplest web service architecture. For the most part, REST operations work just like standard web page requests, with no fancy extras. A REST application simply makes a query via a URL, just like a normal web page request. The HTTP server returns a document with the results of the request. This returning document is usually in a proprietary XML format (though XML isn’t a requirement; the service can return any data structure). In fact, if your browser is capable of displaying XML, you can enter the URL generated by our first client into the URL bar and see the raw XML results that Yahoo! generates.
A Rails-based REST client needs to:
-
Connect to the web service with a standard GET or POST request (depending on the service requirements), using the NET library
-
Store the results as an in-memory REXML document
-
Parse the results with the REXML library for use in the Rails application
To demonstrate, let’s build a simple controller that looks for the top three results of a Yahoo! search, using Yahoo!’s REST interface. Yahoo offers a web service API for many of their services, including the popular Yahoo! search engine. The Yahoo! search service API is currently free, but it does limit the number of requests you can make (currently to 5,000 requests per IP address per day). To use the Yahoo API (and to test the sample code), you’ll need a free Yahoo! Developer’s Key. Get your key directly from Yahoo at http://api.search.yahoo.com/webservices/register_application.
Once you have your developer key, we’re ready to build a simple controller (saved as code_controller.rb in the app/controllers folder):
class CodeController < ApplicationController
def yahootest
query = CGI.escape("SEARCH TEXT") # URL-encoded search value
yahookey = "YOUR YAHOO DEVELOPER KEYs # Your Yahoo! dev key
url = "http://api.search.yahoo.com/" + # The URL to the Yahoo!
"WebSearchService/V1/webSearch?" + # Search service
"appid=#{yahookey}&query=#{query}" +
"&results=3&start=1"
result = Net::HTTP.get(URI(url)) # make the actual HTTP request
@doc = REXML::Document.new result # turn the results into a
# REXML document
end
end
Then we build a view that displays the search results in a simple view. Save the following code in a file called yahootest.rhtml in the app/code/views folder:
<% @doc.root.each_element do |res| %> <b>Title:</b> <%= res[0].text.to_s %><br> <b>Summary:</b> <%= res[1].text.to_s %><br> <b>Link:</b> <a href=”<%= res[2].text.to_s %>”><%= res[2].text.to_s %></a> <br><br> <% end %>
Believe it or not, we’re done. In just a few short lines of code, we’ve built a complete web service client for your Rails application. You should be able to test your application with your local version of Webrick at the URL http://localhost:3000/code/yahootest.
Yahoo! search results for our REST web service request
Now that we have the client working, let’s look at what is going on with the code. First, even though there are no “require” statements, the client is using several important libraries: NET, CGI and REXML. All three libraries come standard with the Ruby 1.8.4 distribution and should be automatically loaded and ready for use by your Rails application. You don’t have to do anything special to use them in your code. You just need to know that they are available.
I’m guessing you’ve heard of the NET and CGI libraries before, since Rails uses these libraries to accomplish a lot of its web-friendly magic, but you may not be familiar with the REXML library. REXML is a pure Ruby XML Processor with lots of nice features, including full support for Xpath 1.0. For more information on REXML, including full documentation, visit http://www.germanesoftware.com/software/rexml/.
We first use the CGI library to escape our search term, ensuring that the search term is safe for use in our HTTP GET request:
query = CGI.escape("SEARCH TEXT")
Next, we save our Yahoo! Developer Key in a variable and build the URL with parameters specified in the documentation for the Yahoo! API. For additional parameters and options, see the online Yahoo! documentation at http://developer.yahoo.net/:
yahookey = "YOUR YAHOO DEVELOPER KEY"
url = "http://api.search.yahoo.com/WebSearchService/" \
"V1/webSearch?appid=#{yahookey}&query=#{query}&" \
"results=3&start=1"
Note: You cna see the XML documents that Yahoo! returns by using this URL in any browser that can render XML.
Understanding the URL isn’t too difficult: we’re calling version 1 (V1) of the webSearch service, passing it four parameters: the appid (the developer key), the query (the search string), the number of results we want (3), and start, which tells the Yahoo! web service the number of the first result we want it to return. In a larger example, we might use this parameter to help page through a long set of results; right now, we just want the first three results, so we start with result 1. Our next step is to use the NET library to make the HTTP GET request and store the returning document in a variable:
result = Net::HTTP.get(URI(url))
After we have our results stored in a local variable, we use the REXML library to convert the results into a REXML document:
@doc = REXML::Document.new result
Finally, our view uses the REXML library to parse the XML document and display the results. REXML offers a number of ways to access tag and attribute values. Here we use array-like methods:
<% @doc.root.each_element do |res| %> <b>Title:</b> <%= res[0].text.to_s %><br> <b>Summary:</b> <%= res[1].text.to_s %><br> <b>Link:</b> <a href=”<%= res[2].text.to_s %>”><%= res[2].text.to_s %></a> <br><br> <% end %>
We use the each_element method on the root element of the document (ResultSet) to access each result (Result). Since Element is a subclass of Parent, we can then use Array-like methods to access the children that we intend to display (Title, Summary, and Url). See the online documentation for additional information about how to access elements and attributes with REXML. Because it is so direct and conceptually simple, many developers feel that RESTas we implemented itis the easiest type of web service to use. While that’s probably true of many other languages, it could be said that building REST clients in Ruby actually involves more work than building clients with SOAP or XML-RPC. That’s because the SOAP and XML-RPC Ruby libraries handle all the low-level details for you: you don’t need to manage communication to and from the server, build the XML request, or in many cases even parse the XML response. These all happen automatically, leaving you free to work with your results and focus on your business logic.
So let’s move on to the next web service type, SOAP, and see just how easy it is to write SOAP clients for Rails.
Comments
Leave a Reply
You must be logged in to post a comment.