Using jQuery and YQL to get an RSS feed from a site

Add to FacebookAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to TwitterAdd to TechnoratiAdd to Yahoo BuzzAdd to Newsvine

Finding an RSS feed on a site can sometimes be a hassle.  Browsers have started introducing a feature that will include an RSS icon in the address bar when the browser finds the site has an RSS feed.  I thought it would be nice to allow users to just enter in a web address and let us do the work finding the RSS feed for the site.  I looked into how browsers were accomplishing this and found that browsers look for a <link> tag in the <head> of the document with the type attribute of “application/rss+xml”.

Now that we know what to look for, how do we go about determining if the website provided has the correct <link> tag?  I decided to let YQL do the work for me.  I loaded up the YQL Query Console and began testing some queries.  On the right hand side, there are different Data Tables you can use.  I scrolled down to “data” and selected the “html” option which allows you to query the html of any site.

The example that you start out with looks similar to this:

select * from html where url="http://finance.yahoo.com/q?s=yhoo" and xpath='//div[@id="yfi_headlines"]/div[2]/ul/li/a'

From this example it is easy to see which URL is being queried and the XPath that is being used to find elements on the page.  If you aren’t familiar with  XPath, check out the W3 XPath documentation.  So now we just plug in the URL of the site we want to search and change the XPath to whatwe need to use to find the <link> elements. Now the query looks like this:

select * from html where url="https://lanitdev.wordpress.com/" and xpath='//link[@type="application/rss+xml"]'

In the query console, be sure to select the JSON radio button and then click the “Test” button.  You will see JSON output that looks like the following:

feeds({
    "query":{
    "count":"1",
    "created":"2010-02-26T09:08:43Z",
    "lang":"en-US",
    "updated":"2010-02-26T09:08:43Z",
    "uri":"http://query.yahooapis.com/v1/yql?q=select+*+from+html+where+url%3D%22http%3A%2F%2Flanitdev.wordpress.com%22+and%0A++++++xpath%3D%27%2F%2Flink%5B%40type%3D%22application%2Frss%2Bxml%22%5D%27",
    "results":{
        "link":{
            "href":"https://lanitdev.wordpress.com/feed/",
            "rel":"alternate",
            "title":"The Lanit Development Blog RSS Feed",
            "type":"application/rss+xml"
        }
    }
}
});

We can see from the JSON that we found 1 feed and we get all the information to go along with it.

Now that we have a proper YQL query, let’s use some jQuery to create a form that retrieves these feeds for us.  I am going to create a simple form where a user inputs a url and then outputs a list of RSS feeds or a message stating that an RSS feed couldn’t be found.

The form is straightforward.  I just have a textbox and a button.  I also have an unordered list which I’ll use to display the list of RSS feeds.

<table>
    <tr>
        <td>
            <label><input type="text" id="website" /></label>
        </td>
        <td>
            <input id="getrss" type="button" value="Get RSS" />
        </td>
    </tr>
</table>

<ul id="rsslist">
</ul>

Next is the jQuery.  We will attach an event to the button that will hit the YQL URL and return JSON that we can parse and get the data we need.  The YQL URL we will be using is found in the YQL Query Console next to where you entered the query.  There is a box called “The REST Query” which has the query that we will use.  The only modification I made to the rest query was that I took out the URL that was in there and replaced with with the jQuery value from the textbox.

$(function(){
    $("#getrss").click(function(){
        $("#rsslist").empty();
        $.ajax({
            type: "GET",
            url: "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22"  + $("#website").val() + "%22%20and%0A%20%20%20%20%20%20xpath%3D'%2F%2Flink%5B%40type%3D%22application%2Frss%2Bxml%22%5D'&format=json&diagnostics=false",
            dataType: "json",
            success: function(data){
                if (data.query.count == 1){
                    $("#rsslist").append("<li><a  href='" + data.query.results.link.href + "'>" + data.query.results.link.title + "</a></li>");
                }
                else if (data.query.count > 1){
                    for (var i = 0; i < data.query.results.link.length; i++)
                    {
                        var link = data.query.results.link[i];
                        $("#rsslist").append("<li><a  href='" + link.href + "'>" + link.title + "</a></li>");
                    }
                } else {
                    $("#rsslist").append("<li>No RSS feed found</li>");
                }
             },
             error: function(){
                 console.log("error");
             }
        });
     });
})

When the AJAX request finishes successfully we check to see if any feeds are returned and if there are, we loop through them and display them.

I created a demo page using jsbin.com so you can see this in action. View Demo

I hope this small example gives you an idea how you can use jQuery and YQL to accomplish other interesting challenges.

Advertisements

Bing Maps API – Getting Started

Lanit had recently been researching getting a mapping API set up for use in various projects. Ultimately, due to ease of use and licensing, we settled on the Bing Maps API.

The goal of the first project using these maps was to plot a fairly large number of items with known locations on a simple map of Missouri.

Starting out, you must first include a reference to the bing maps in your page:

<script type='text/javascript' src='http://ecn.dev.virtualearth.net/mapcontrol/mapcontrol.ashx?v=6.2&mkt=en-us'/>

After that, create some objects that you’re going to plot. For our implementation, all the needed information is stored on the page -so these properties are just getting values from these.

function Item($container) {
    var item = this;
    var lngBox = $container.find("input.lng");
    var latBox = $container.find("input.lat");
    item.getLat = function() { return latBox.val(); };
    item.getLng = function() { return lngBox.val(); };
    item.Icon = “img/iconhere”;
    item.Title = “Title Here”;
}

Item.prototype.plot = function(map) {
    var item = this;
    item.getPosition(function(latLng) {
    var thisPin = new VEShape(VEShapeType.Pushpin, latLng);
    thisPin.SetCustomIcon(item.Icon);
    thisPin.SetTitle(Item.Title);
    var pinPoint = map.AddShape(thisPin);
});};

It’s worth noting here the “plot” is expecting an object of type “VEMap”, and will then position a point of type VEShape at the position of the item.

For our implementation, we had static addresses with Latitudes and Longitudes already saved. If your implementation will need to geocode addresses, the “GetLat” and “GetLng” functions will need to be replaced with calls to the geocode service.

After creating these objects, we’ll load the map on page load and plot the objects

var settings = {
              mapOpts: {container: "map", // id of map 
             containerheight: 550, 
           width: 600, 
zoom: 7, 
center: new VELatLong(38.208142, -92.381362) },
elementClick: function() { }// this code is run when a plotted item is clicked
};

$(function() {

    $("#" + settings.mapOpts.container).css({
        'width': settings.mapOpts.width, // the API will not automatically set the height/width of the container object – this must be done before the maps are loaded.
        'height': settings.mapOpts.height,
        'position': 'relative'
    });

    // creates a map in the dom object by ID
    var map = new VEMap(settings.mapOpts.container);

    // attach an event to the item clicks -
    map.AttachEvent("onclick", function(e) {
        if (e.elementID != null) { // call the click function using the Pin element
            var shape = map.GetShapeByID(e.elementID);
            settings.elementClick(shape);
        }
    });

    // this renders the map to the page
    map.LoadMap(settings.mapOpts.center, settings.mapOpts.zoom, null, false, null, false);

    // disable zooming on mouse wheel
    map.AttachEvent("onmousewheel", function(e) {
        var mouseWheel = -e.mouseWheelChange / 2;
        window.scrollBy(0,mouseWheel);
        return true;
    });

    // plot the items on the map
    var items = $("#itemContainer").find("div.itemContainer");
    items.each(function() {
        var item = new Item($(this));
        item.plot(map);
   });
});

You can see some implementation decisions here – the pages were pretty unusable with the default functionality to scroll on mouse wheel, so that was disabled.

Loading images last with jQuery

Add to FacebookAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to TwitterAdd to TechnoratiAdd to Yahoo BuzzAdd to Newsvine

There are lots of ways to make your webpages faster and more responsive. YSlow is a great tool to help you find many great ways to make a particular page faster.

One of the best things you can do is reduce the number of requests (css/js/images/etc) to the server. Typically, this would mean that you would combine files – merge all of your JS and CSS (and minify while you are at it), and use CSS Sprites to combine images.

One major problem of using CSS Sprites is that it can be quite painful to maintain. Over time, if you want to add or change some of your images – you basically need to rebuild and replace the combined images and all of the CSS rules specifying coordinates. Sometimes, this makes the CSS Sprite technique unreasonable to implement.

In one such case, we had about 50 images in one application that were causing the page to take a long time to load. These images were previews of some different design choices that the user could make. The design choices themselves (and their previews) were database driven so that we can add new designs through an admin interface. So, CSS Spriteing the previews would seriously hamper that flexibility.

One other design consideration was that the previews weren’t that important – the page was fully functional and usable without the images. In fact, the designs weren’t even visible until you toggled the design menu.

There is a lazy loader plugin for jQuery already available here – but it didn’t fit our needs. Instead of skipping images in order to get the page working as soon as possible (and initiate the load once the page is usable) – it is made to skip loading offscreen images until they are scrolled into view. It might have somewhat worked for our needs – but I thought it was better to load the images as soon as possible, instead of waiting for the design menu to be expanded to initiate the load. That way, most of the time the designs would be visible by the time they open the menu – but it wouldn’t interfere with the rest of the interface.

My solution was to set the src for all of the previews to a single animated loading image – like one you can get here. Then, I set a custom attribute on the image for the real preview’s url. Finally, some jQuery code runs after the page is done loading which replaces each src attribute with the url in the custom attribute, which will load the real image.

Sample HTML:

<ul>
    <li templateid="7bcf8f23-fdd0-45c5-a429-d2ffb59e47f0" class="selected"><span>3D Dots
        Dark</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/7bcf8f23-fdd0-45c5-a429-d2ffb59e47f0/preview.jpg"
            class="deferredLoad" alt="3D Dots Dark" />
    </li>
    <li templateid="b1a09e28-629e-472a-966e-fc98fc269607"><span>3D Dots Lite</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/b1a09e28-629e-472a-966e-fc98fc269607/preview.jpg"
            class="deferredLoad" alt="3D Dots Lite" />
    </li>
    <li templateid="e121d26a-9c8f-466f-acc7-9a79d5e8cfa9"><span>Beauty</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/e121d26a-9c8f-466f-acc7-9a79d5e8cfa9/preview.jpg"
            class="deferredLoad" alt="Beauty" />
    </li>
    <li templateid="322e4c7a-33e7-4e05-bb72-c4076a83a3d0"><span>Black and White</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/322e4c7a-33e7-4e05-bb72-c4076a83a3d0/preview.jpg"
            class="deferredLoad" alt="Black and White" />
    </li>
    <li templateid="57716da9-91ef-4cf0-82f1-722d0770ad7f"><span>Blank</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/57716da9-91ef-4cf0-82f1-722d0770ad7f/preview.jpg"
            class="deferredLoad" alt="Blank" />
    </li>
    <li templateid="a79e1136-db47-4acd-be3e-2daf4522796d"><span>Blue Leaves</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/a79e1136-db47-4acd-be3e-2daf4522796d/preview.jpg"
            class="deferredLoad" alt="Blue Leaves" />
    </li>
    <li templateid="03cb737d-4da7-46d5-b4e4-5ad4b4a3aaf4"><span>Blue Open</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/03cb737d-4da7-46d5-b4e4-5ad4b4a3aaf4/preview.jpg"
            class="deferredLoad" alt="Blue Open" />
    </li>
    <li templateid="899dff2f-38ba-44f7-9fe2-af66e62674a4"><span>Compass</span>
        <img src="/static/img/ajax-loader-small.gif" deferredsrc="/resources/899dff2f-38ba-44f7-9fe2-af66e62674a4/preview.jpg"
            class="deferredLoad" alt="Compass" />
    </li>
</ul>

Sample javascript:

$(function(){
        $("img.deferredLoad").each(function() {
            var $this = $(this);
            $this.attr("src", $this.attr("deferredSrc")).removeClass("deferredLoad");
        });
});

Accessible Custom AJAX and .NET

Add to FacebookAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to TwitterAdd to TechnoratiAdd to Yahoo BuzzAdd to Newsvine

One general rule in making an accessible web application is that you shouldn’t change content of the page with javascript. This is because screen readers have a tough time monitoring the page and notifying the user of dynamic changes. Usually, the reason you would use Ajax is exactly that – do something on the server, and then update some small part of the page without causing the whole page to refresh. That means if you have an “ajaxy” page and you care about accessibility, you have to provide a non-ajax interface as a fallback for screen readers.

Ajax.NET, Microsoft’s library for Ajax, makes this fallback easy to implement. Microsoft has you define your AJAX properties in an abstraction layer – the page XAML – which means that the framework can decide to not render the AJAX code to certain browsers (and screen readers) that will not support it, and instead use the standard postback method.

The problem with Ajax.NET is that the communication can be bloated (mostly because of the abstraction layer, it sends more post values than you might need – like the encrypted viewstate value), which negates many of the benefits of an Ajax solution. I really wanted to roll my own ajax communications to make them as lightweight as possible.

My solution was to write the page in a standard .NET postback manner, and then use a user-defined property that would allow javascript to replace the postback in javascript/jQuery with an Ajax call.

Here’s my code:

$(function() {
            if (serverVars.uiVersion != "accessible") { // use js/ajax for checking/unchecking where possible
                var $todos = $("#chkToDo");
               $todos.removeAttr("onclick"); // remove postback
                $todos.click(
                    function() {
                        //some stuff altering the document, and an ajax call to report the change to the db
                    }
                );
            }

This works great, although you need to be careful about your server-side events. In my case, I had an OnCheckChanged event to handle the postback/accessible mode. Even though checking/unchecking the box no longer fired an autopostback – ASP.NET will still fire the checkchanged event if you postback later for some other reason (e.g. – a linkbutton elsewhere on the page) after the checked status had changed. So, if a user had changed the state of a checkbox, then clicked another link button on the page – instead of sending the user to the intended page,my app just refreshed the whole page(because my CheckChanged event redirected to reload the page – which caused it to skip the ‘click’ event of the linkbutton). Once I realized this was happening, it was easy enough to fix – I just needed to only run the event logic if the user was in accessibility mode. I spent a little time running in circles on that one though, at first I thought my client side document changes were causing a ViewState validation error on the server.

Some Good Uses for the ASP.NET .ASHX/”Generic Handler”

Add to FacebookAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to TwitterAdd to TechnoratiAdd to Yahoo BuzzAdd to Newsvine

I’ve been developing ASP.NET applications since 2002.  Up until recently, I’d overlooked a pretty useful part of the ASP.NET Framework : Generic Handlers.

Generic Handlers are basically HTTP Handlers that can process requests to exactly one url (Http Handlers are classes that need to be registered in the web.config for what urls they run on, and often manipulate requests/responses that go to/come from .aspx pages). They are “closer to the wire” than .ASPX WebForms or .ASMX WebServices.  They are built with only two requirements, implement:

public void ProcessRequest(HttpContext context);

(this contains the logic for when this page is requested) and, implement:

public bool IsReusable;

(this tells ASP.NET whether it can use one instance to serve multiple requests).

Generic Handlers are similar to web forms – they are reachable by a standard get/post to the url they are located in.  ASP.NET starts you off with context object (passed into the ProcessRequest() function you implement), which you then use to communicate with the request (form and query vals, etc) and the response object (to write info to the browser).  You can decide in what capacity you want that context to have access to the session:read/write, readonly, or none. Unlike web forms, generic handlers skip some of the niceties: the designer/xaml/servercontrol model, themes, viewstate, etc. Thus, generic handlers can be much more efficient for many tasks that make no use of those features.

Here’s a few common tasks where a generic handler would be more suitable than a web form:

  • Writing out the contents of files stored in the db/elsewhere for download
  • Writing out manipulated images, generated graphics, etc
  • Writing out generated PDFs/CSVs etc
  • Writing out data without the webservice overhead
    • xml for other systems or sites
    • html for simple AJAX get requests to replace page content
    • JSON for more advanced ajax data communication
  • Callback urls for data sent from other sites through a http post
  • Building a “REST API”
  • A url that outputs status info about the website, for use by a content switch or other monitoring system
  • Writing out dynamic JS/CSS based on server variables, browser capabilities, etc

Basically, anywhere I found myself before creating an ASPX page, but clearing everything on the .ASPX and using Response.Write()/Response.BinaryWrite() to talk to the browser or another system from the code behind, I should have been using a .ASHX handler instead and saving my webserver some work.

Here is a sample GenericHandler to get you started. It writes out a simple js file that gives you some useful globals you can use in other js files. Its meant to just be an example of the types of things you can do with generic handlers, and doesn’t necessarily produce what you might want in your own dynamic JS file.


    public class SiteJSGlobals : System.Web.IHttpHandler, System.Web.SessionState.IReadOnlySessionState
    {
        public void ProcessRequest(HttpContext context)
        {
            context.Response.ContentType = "text/javascript";

            context.Response.Write("var _g_username='" + context.User.Identity.Name + "';\n");
            context.Response.Write("var _g_usertype='" + context.Session["UserType"] + "';\n");
            context.Response.Write("var _g_browserheight=" + context.Request.Browser.ScreenPixelsHeight + ";\n");
            context.Response.Write("var _g_browserwidth=" + context.Request.Browser.ScreenPixelsWidth + ";\n");
            context.Response.Write("var _g_queryparam='" + context.Request["queryparam"] + "';\n");
            context.Response.Write("var _g_developmentmode=" + context.Request.Url.Host.ToLower().StartsWith("localhost") + ";\n");
            context.Response.Write("var _g_approoturl='" + context.Request.ApplicationPath + "';\n");
        }
        public bool IsReusable
        {
            get
            {
                return false;
            }
        }
    }