Using jQuery and YQL to get an RSS feed from a site

Add to FacebookAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to TwitterAdd to TechnoratiAdd to Yahoo BuzzAdd to Newsvine

Finding an RSS feed on a site can sometimes be a hassle.  Browsers have started introducing a feature that will include an RSS icon in the address bar when the browser finds the site has an RSS feed.  I thought it would be nice to allow users to just enter in a web address and let us do the work finding the RSS feed for the site.  I looked into how browsers were accomplishing this and found that browsers look for a <link> tag in the <head> of the document with the type attribute of “application/rss+xml”.

Now that we know what to look for, how do we go about determining if the website provided has the correct <link> tag?  I decided to let YQL do the work for me.  I loaded up the YQL Query Console and began testing some queries.  On the right hand side, there are different Data Tables you can use.  I scrolled down to “data” and selected the “html” option which allows you to query the html of any site.

The example that you start out with looks similar to this:

select * from html where url="http://finance.yahoo.com/q?s=yhoo" and xpath='//div[@id="yfi_headlines"]/div[2]/ul/li/a'

From this example it is easy to see which URL is being queried and the XPath that is being used to find elements on the page.  If you aren’t familiar with  XPath, check out the W3 XPath documentation.  So now we just plug in the URL of the site we want to search and change the XPath to whatwe need to use to find the <link> elements. Now the query looks like this:

select * from html where url="https://lanitdev.wordpress.com/" and xpath='//link[@type="application/rss+xml"]'

In the query console, be sure to select the JSON radio button and then click the “Test” button.  You will see JSON output that looks like the following:

feeds({
    "query":{
    "count":"1",
    "created":"2010-02-26T09:08:43Z",
    "lang":"en-US",
    "updated":"2010-02-26T09:08:43Z",
    "uri":"http://query.yahooapis.com/v1/yql?q=select+*+from+html+where+url%3D%22http%3A%2F%2Flanitdev.wordpress.com%22+and%0A++++++xpath%3D%27%2F%2Flink%5B%40type%3D%22application%2Frss%2Bxml%22%5D%27",
    "results":{
        "link":{
            "href":"https://lanitdev.wordpress.com/feed/",
            "rel":"alternate",
            "title":"The Lanit Development Blog RSS Feed",
            "type":"application/rss+xml"
        }
    }
}
});

We can see from the JSON that we found 1 feed and we get all the information to go along with it.

Now that we have a proper YQL query, let’s use some jQuery to create a form that retrieves these feeds for us.  I am going to create a simple form where a user inputs a url and then outputs a list of RSS feeds or a message stating that an RSS feed couldn’t be found.

The form is straightforward.  I just have a textbox and a button.  I also have an unordered list which I’ll use to display the list of RSS feeds.

<table>
    <tr>
        <td>
            <label><input type="text" id="website" /></label>
        </td>
        <td>
            <input id="getrss" type="button" value="Get RSS" />
        </td>
    </tr>
</table>

<ul id="rsslist">
</ul>

Next is the jQuery.  We will attach an event to the button that will hit the YQL URL and return JSON that we can parse and get the data we need.  The YQL URL we will be using is found in the YQL Query Console next to where you entered the query.  There is a box called “The REST Query” which has the query that we will use.  The only modification I made to the rest query was that I took out the URL that was in there and replaced with with the jQuery value from the textbox.

$(function(){
    $("#getrss").click(function(){
        $("#rsslist").empty();
        $.ajax({
            type: "GET",
            url: "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22"  + $("#website").val() + "%22%20and%0A%20%20%20%20%20%20xpath%3D'%2F%2Flink%5B%40type%3D%22application%2Frss%2Bxml%22%5D'&format=json&diagnostics=false",
            dataType: "json",
            success: function(data){
                if (data.query.count == 1){
                    $("#rsslist").append("<li><a  href='" + data.query.results.link.href + "'>" + data.query.results.link.title + "</a></li>");
                }
                else if (data.query.count > 1){
                    for (var i = 0; i < data.query.results.link.length; i++)
                    {
                        var link = data.query.results.link[i];
                        $("#rsslist").append("<li><a  href='" + link.href + "'>" + link.title + "</a></li>");
                    }
                } else {
                    $("#rsslist").append("<li>No RSS feed found</li>");
                }
             },
             error: function(){
                 console.log("error");
             }
        });
     });
})

When the AJAX request finishes successfully we check to see if any feeds are returned and if there are, we loop through them and display them.

I created a demo page using jsbin.com so you can see this in action. View Demo

I hope this small example gives you an idea how you can use jQuery and YQL to accomplish other interesting challenges.

Advertisements

Bing Maps API – Getting Started

Lanit had recently been researching getting a mapping API set up for use in various projects. Ultimately, due to ease of use and licensing, we settled on the Bing Maps API.

The goal of the first project using these maps was to plot a fairly large number of items with known locations on a simple map of Missouri.

Starting out, you must first include a reference to the bing maps in your page:

<script type='text/javascript' src='http://ecn.dev.virtualearth.net/mapcontrol/mapcontrol.ashx?v=6.2&mkt=en-us'/>

After that, create some objects that you’re going to plot. For our implementation, all the needed information is stored on the page -so these properties are just getting values from these.

function Item($container) {
    var item = this;
    var lngBox = $container.find("input.lng");
    var latBox = $container.find("input.lat");
    item.getLat = function() { return latBox.val(); };
    item.getLng = function() { return lngBox.val(); };
    item.Icon = “img/iconhere”;
    item.Title = “Title Here”;
}

Item.prototype.plot = function(map) {
    var item = this;
    item.getPosition(function(latLng) {
    var thisPin = new VEShape(VEShapeType.Pushpin, latLng);
    thisPin.SetCustomIcon(item.Icon);
    thisPin.SetTitle(Item.Title);
    var pinPoint = map.AddShape(thisPin);
});};

It’s worth noting here the “plot” is expecting an object of type “VEMap”, and will then position a point of type VEShape at the position of the item.

For our implementation, we had static addresses with Latitudes and Longitudes already saved. If your implementation will need to geocode addresses, the “GetLat” and “GetLng” functions will need to be replaced with calls to the geocode service.

After creating these objects, we’ll load the map on page load and plot the objects

var settings = {
              mapOpts: {container: "map", // id of map 
             containerheight: 550, 
           width: 600, 
zoom: 7, 
center: new VELatLong(38.208142, -92.381362) },
elementClick: function() { }// this code is run when a plotted item is clicked
};

$(function() {

    $("#" + settings.mapOpts.container).css({
        'width': settings.mapOpts.width, // the API will not automatically set the height/width of the container object – this must be done before the maps are loaded.
        'height': settings.mapOpts.height,
        'position': 'relative'
    });

    // creates a map in the dom object by ID
    var map = new VEMap(settings.mapOpts.container);

    // attach an event to the item clicks -
    map.AttachEvent("onclick", function(e) {
        if (e.elementID != null) { // call the click function using the Pin element
            var shape = map.GetShapeByID(e.elementID);
            settings.elementClick(shape);
        }
    });

    // this renders the map to the page
    map.LoadMap(settings.mapOpts.center, settings.mapOpts.zoom, null, false, null, false);

    // disable zooming on mouse wheel
    map.AttachEvent("onmousewheel", function(e) {
        var mouseWheel = -e.mouseWheelChange / 2;
        window.scrollBy(0,mouseWheel);
        return true;
    });

    // plot the items on the map
    var items = $("#itemContainer").find("div.itemContainer");
    items.each(function() {
        var item = new Item($(this));
        item.plot(map);
   });
});

You can see some implementation decisions here – the pages were pretty unusable with the default functionality to scroll on mouse wheel, so that was disabled.

SQL Select Multiple Columns into Comma-Separated List

I needed to select a field from multiple rows in one table into one column in another table.  I wanted the one column to be a comma-separated list as well.

One way to do this would be to use cursors, but the following post showed an easier way using XML Path:  http://bytes.com/topic/sql-server/answers/431513-getting-data-multiple-rows-into-one-column.

I ended up doing something like this:


SELECT  State.Name,
       (SELECT  City.Name AS [text()]
        FROM City
        WHERE City.StateID = State.StateID  && City.Population > 100000
        FOR XML Path(")) AS LargeCities
FROM State

Of course, this just gives me the list of city names all run together like this:  “DenverBoulderPuebloColorado Springs”.

I wanted to separate these by commas so I changed the query to “Select City.Name + ‘, ‘ AS [text()], but this gives me an extra comma on the end like this: “Denver, Boulder, Pueblo, Colorado Springs, “.

Of course, I could have used SUBSTRING(“the inner select statement”, 0, LEN(“the inner select statement”) – 2) and been fine, but I didn’t want to rerun the query and it would be harder to ready anyway.

What I ended up doing was inverting the positioning of the “City.Name” and the “, “.  This gave me the following:  “, Denver, Boulder, Pueblo, Colorado Springs”.

Then, all I had to do with the SUBSTRING command was this:  SUBSTRING(“the inner select statement”, 2, 10000).  The “2” as the starting position got me past the first comma, and there was no trailing comma to worry about, since each city was preceded by a comma rather than followed by a comma.  So, the final query looks like this:


SELECT  State.Name,
        SUBSTRING((SELECT ', ' +  City.Name AS [text()]
                   FROM City
                   WHERE City.StateID = State.StateID  && City.Population > 100000
                   FOR XML Path("))), 2, 10000) AS LargeCities
FROM State

and would give the following results:

State Name      LargeCities
Colorado               Denver, Boulder, Pueblo, Colorado Springs

This strategy could be applied to regular code as well when building a comma-separated list.


string list = "";
for (int i = 0; i < items.count; i++)
{
   list += ", " + items[i].Name;
}
list = list..Substring(2);

Loading images last with jQuery

Add to FacebookAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to TwitterAdd to TechnoratiAdd to Yahoo BuzzAdd to Newsvine

There are lots of ways to make your webpages faster and more responsive. YSlow is a great tool to help you find many great ways to make a particular page faster.

One of the best things you can do is reduce the number of requests (css/js/images/etc) to the server. Typically, this would mean that you would combine files – merge all of your JS and CSS (and minify while you are at it), and use CSS Sprites to combine images.

One major problem of using CSS Sprites is that it can be quite painful to maintain. Over time, if you want to add or change some of your images – you basically need to rebuild and replace the combined images and all of the CSS rules specifying coordinates. Sometimes, this makes the CSS Sprite technique unreasonable to implement.

In one such case, we had about 50 images in one application that were causing the page to take a long time to load. These images were previews of some different design choices that the user could make. The design choices themselves (and their previews) were database driven so that we can add new designs through an admin interface. So, CSS Spriteing the previews would seriously hamper that flexibility.

One other design consideration was that the previews weren’t that important – the page was fully functional and usable without the images. In fact, the designs weren’t even visible until you toggled the design menu.

There is a lazy loader plugin for jQuery already available here – but it didn’t fit our needs. Instead of skipping images in order to get the page working as soon as possible (and initiate the load once the page is usable) – it is made to skip loading offscreen images until they are scrolled into view. It might have somewhat worked for our needs – but I thought it was better to load the images as soon as possible, instead of waiting for the design menu to be expanded to initiate the load. That way, most of the time the designs would be visible by the time they open the menu – but it wouldn’t interfere with the rest of the interface.

My solution was to set the src for all of the previews to a single animated loading image – like one you can get here. Then, I set a custom attribute on the image for the real preview’s url. Finally, some jQuery code runs after the page is done loading which replaces each src attribute with the url in the custom attribute, which will load the real image.

Sample HTML:

<ul>
    <li templateid="7bcf8f23-fdd0-45c5-a429-d2ffb59e47f0" class="selected"><span>3D Dots
        Dark</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/7bcf8f23-fdd0-45c5-a429-d2ffb59e47f0/preview.jpg"
            class="deferredLoad" alt="3D Dots Dark" />
    </li>
    <li templateid="b1a09e28-629e-472a-966e-fc98fc269607"><span>3D Dots Lite</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/b1a09e28-629e-472a-966e-fc98fc269607/preview.jpg"
            class="deferredLoad" alt="3D Dots Lite" />
    </li>
    <li templateid="e121d26a-9c8f-466f-acc7-9a79d5e8cfa9"><span>Beauty</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/e121d26a-9c8f-466f-acc7-9a79d5e8cfa9/preview.jpg"
            class="deferredLoad" alt="Beauty" />
    </li>
    <li templateid="322e4c7a-33e7-4e05-bb72-c4076a83a3d0"><span>Black and White</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/322e4c7a-33e7-4e05-bb72-c4076a83a3d0/preview.jpg"
            class="deferredLoad" alt="Black and White" />
    </li>
    <li templateid="57716da9-91ef-4cf0-82f1-722d0770ad7f"><span>Blank</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/57716da9-91ef-4cf0-82f1-722d0770ad7f/preview.jpg"
            class="deferredLoad" alt="Blank" />
    </li>
    <li templateid="a79e1136-db47-4acd-be3e-2daf4522796d"><span>Blue Leaves</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/a79e1136-db47-4acd-be3e-2daf4522796d/preview.jpg"
            class="deferredLoad" alt="Blue Leaves" />
    </li>
    <li templateid="03cb737d-4da7-46d5-b4e4-5ad4b4a3aaf4"><span>Blue Open</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/03cb737d-4da7-46d5-b4e4-5ad4b4a3aaf4/preview.jpg"
            class="deferredLoad" alt="Blue Open" />
    </li>
    <li templateid="899dff2f-38ba-44f7-9fe2-af66e62674a4"><span>Compass</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/899dff2f-38ba-44f7-9fe2-af66e62674a4/preview.jpg"
            class="deferredLoad" alt="Compass" />
    </li>
</ul>

Sample javascript:

$(function(){
        $("img.deferredLoad").each(function() {
            var $this = $(this);
            $this.attr("src", $this.attr("deferredSrc")).removeClass("deferredLoad");
        });
});