I’ve recently been checking out YQL, Yahoo’s new gift to developers! The idea behind it is to unify the complicated world of APIs into one solid SQL-like abstraction; flickr feeds, search, weather and RSS feeds (etc). Additionally, it can retrieve HTML, XML, CSV, ATOM and more, all returned as structured data via either the JSON or XML format.

The fact that it has the capability to return JSON and even JSONP means cross-domain web querying is now a piece of cake. Just imagine being able to have access to the entire internet from the comfort of the client-side!

If you have a Yahoo account you should definitely check out the console; there’s a bunch of examples to choose from on the side. Try this query:

SELECT * FROM html WHERE url='http://news.bbc.co.uk' AND xpath="//*[@id='livestats200']//ul[contains(@class,'popstoryList')]/li/a" LIMIT 1

The above query should return the most read article on the BBC website (at the time of querying). You can use XPATH to retrieve specific nodes within the targetted document. It would be cool if YQL supported CSS selectors as many more people know the syntax but unfortunately we’re stuck with XPATH… It’s not too hard to learn though, if you’re new to it then this page is worth a glance: http://www.w3schools.com/XPath/xpath_syntax.asp.

I’ve started development on a “CSS to XPATH” converter for JavaScript, it works quite well, but it’s not quite robust enough to deal with complicated selectors yet.

I’ve also written a small interface for working with YQL within your JavaScript applications (framework agnostic). Have a look:

// YQL serves JSONP (with a callback) so all we have to do
// is create a script element with the right 'src':
function YQLQuery(query, callback) {
    this.query = query;
    this.callback = callback || function(){};
    this.fetch = function() {
 
        if (!this.query || !this.callback) {
            throw new Error('YQLQuery.fetch(): Parameters may be undefined');
        }
 
        var scriptEl = document.createElement('script'),
            uid = 'yql' + +new Date(),
            encodedQuery = encodeURIComponent(this.query.toLowerCase()),
            instance = this;
 
        YQLQuery[uid] = function(json) {
            instance.callback(json);
            delete YQLQuery[uid];
            document.body.removeChild(scriptEl);
        };
 
        scriptEl.src = 'http://query.yahooapis.com/v1/public/yql?q='
                     + encodedQuery + '&format=json&callback=YQLQuery.' + uid; 
        document.body.appendChild(scriptEl);
 
    };
}

Use it like this:

// Alert the latest post title from Ajaxian.com
 
// Construct your query:
var query = "select * from rss where url='feeds2.feedburner.com/ajaxian' limit 1";
 
// Define your callback:
var callback = function(data) {
    var post = data.query.results.item;
    alert(post.title);
};
 
// Instantiate with the query:
var ajaxianPosts = new YQLQuery(query, callback);
 
// If you're ready then go:
ajaxianPosts.fetch(); // Go!!
 
/* Callback & query can be defined as properties also:
   ajaxianPosts.query = 'select * from...';
   ajaxianPosts.callback = function(){}; */

Also, for you jQuery junkies, here’s a plugin:

$.YQL = function(query, callback) {
 
    if (!query || !callback) {
        throw new Error('$.YQL(): Parameters may be undefined');
    }
 
    var encodedQuery = encodeURIComponent(query.toLowerCase()),
        url = 'http://query.yahooapis.com/v1/public/yql?q='
            + encodedQuery + '&format=json&callback=?';
 
    $.getJSON(url, callback);
 
};
 
// Usage:
$.YQL("select * from rss where url='feeds2.feedburner.com/ajaxian' limit 1", function(data) {
    var post = data.query.results.item;
    alert(post.title);
});

Even if you’re not planning on using it in a project it’s definitely worth a look and Yahoo’s console makes it incredibly easy to play around with it.

Thanks for reading! Please share your thoughts with me on Twitter. Have a great day!