Selenium Regression Testing Part II – Tips and Tricks

Add to FacebookAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to TwitterAdd to TechnoratiAdd to Yahoo BuzzAdd to Newsvine

In my last post, I talked about how you can use Selenium to do real regressions tests for web applications. It’s a great way to automate testing the real user experience, and not just the backend stuff.

That said, Selenium is a relatively new technology, and it’s not without its issues. When building your first test you might find a lot of times where it’s a trial and error process. There are so many different ways to do the same test, it can be overwhelming. Often times, one or more of the ways you try won’t work. I’m going to try to list some of the common problems I ran into and tips I found below.

  • Selenium Commands
    • waitForElementPresent
      • this is will block your test proceeding until it finds the identified element.  It checks once a second for 30 seconds.
      • this was the most common way I dealt with ‘ajax’ type interactions where it takes an unknown period of time for something to show up
      • I also use it generally instead of verifyElementPresent – it does basically the same thing with a little wiggle room
    • mouseDown/mouseUp/mousePressed/click
      • mostly equivalent, but sometimes you need to match the desired event to a javascript handler
      • try click first.  If it doesn’t work the way you want, move on to mouseDown and so on.
    • waitForFrameToLoad/selectFrame
      • important if you use iFrames (modal dialog, etc.)
      • the selenium selectors only hit the current frame, otherwise you have to select the correct frame
      • an easy way to get back to the root window is to do selectFrame null
    • type vs. typeKeys
      • type fills out an input in code – if it works, use this.  You can use this, and then fire a single typeKeys for the last character if you need an event to be triggered.
      • typeKeys fires the key events on the element
        • has some idiosyncracies – certain letters (I’m looking at you, ‘y’) are reserved to do other special keypresses
        • necessary if you are using a wysiwyg ‘designmode’ type box instead of a standard input
    • verifyX vs. assertX
      • if verify fails, the test continues (and is marked as errored).  If assert fails, the test aborts.
      • Usually verify is better, unless one task blocks a future one from functioning correctly
    • runScript vs. Eval vs. Expression
      • runScript inserts the javascript you provide into the current frame/window.  Useful for those things selenium doesn’t support – like moving a cursor around and selecting text in a wysiwyg
      • Eval runs javascript in the context of selenium. Handy for complex checks – use waitForEval (for instance, checking the css background image property of a particular element)
        • Use this.browserbot.findeElement(“selenium selector”) to find elements the way selenium would
        • Use window.X To access the current frame/window context objects
      • Expression is similar to Eval, but uses Selenium’s custom expression format instead of javascript (but you can combine with javascript by using  javascript{}
        • storedVars[‘key’] allows you to get to a variable you created with a Selenium ‘store’ expression
    • selectPopUp
      • useful for checking stuff in a popup that was initiated
      • Easiest to get by the html title of the popup, but do a ‘pause’ first to let it load
  • Selenium Selectors and XPath
    • In general, be as abstract as possible.
      • Don’t select individual server generated ids (hand crafted html ids are ok if you don’t expect them to change)
      • Don’t select on complicated relationships ( /div[0]/div[2]/a[4] ) – your html structure will change and you’ll have to maintain it
      • Select links by the simple link=text when possible – easy to read/maintain, unlikely to change
      • Use //that (any decendant) instead of /this/that where possible
      • .  references ‘this’ element.  Helps to select something with a particular text:   //div[@id=’publish-private-shares’]//p[.=’This is pretty cool.’]
      • Contains() is useful if you don’t know the exact text (for instance, when an element has multiple css classes):     //div[@id=’pageContent’ and contains(@class,’contenteditable’) and h2=’Goals’]/p[1]
  • Selenium RC
    • While you can use Selenium IDE to create a c# version of your tests – if you do so, you have two tests to maintain.  You can run your ‘selenese’ tests directly with RC, too.
      • I’ve written a simple csharp console project that automatically finds the correct javapath and fires up the test when you run it.  If people ask in the comments, I’ll post it.
    • Last I checked, Chrome and Safari-Windows don’t work.  Chrome is supposed to be fixed in Selenium RC 1.0.4
  • Sauce RC
    • This is a great UI to help test multiple browsers, but there are a couple of issues
      • Firefox works, but only in dual window mode
      • IE works, but only in single window mode.
      • The ‘timeout’ setting implies a default timeout per action in your test, but it is actually the timeout for your entire test run.  Since it defaults to 30 seconds, you’ll probably want to change it, or else your tests will suddenly die for no reason with no explanation/log.

I’m sure there is probably more I’ve forgotten, so leave a comment if you get stuck and I’ll try to help out if I can.

Extended CheckBoxList Control

A common task in programming is to retrieve all of the selected items or values from a CheckBoxList control. It is unfortunate that no native properties provide this already. However, extending the CheckBoxList to provide these is simple enough.

The extension I created provides four new properties:

  • SelectedValuesString – A comma-delimited string of the selected values.
  • SelectedValuesAsIntegers – An Integer List of the selected values.
  • SelectedValuesAsStrings – A string List of the selected values.
  • SelectedItemTextAsStrings – A string list of the text for the selected items.
public class ExtendedCheckBoxList : CheckBoxList
    public string SelectedValuesString
            return String.Join(",",SelectedValuesAsStrings);

    public IEnumerable SelectedValuesAsIntegers
            int temp;
            return this.Items.Cast<ListItem>()
                 .Where(i => i.Selected && int.TryParse(i.Value,out temp)).Select(i => int.Parse(i.Value));
    public IEnumerable SelectedValuesAsStrings
            return this.Items.Cast<ListItem>().Where(i => i.Selected).Select(i=>i.Value);

    public IEnumerable SelectedItemTextAsStrings
            return this.Items.Cast<ListItem>().Where(i => i.Selected).Select(i => i.Text);

Databinding Nested Repeaters

Many of you might already know about this, but I’ll post it for those of you like me who didn’t. Occasionally I’ll nest a repeater inside of another repeater, and when I do I always attach an ItemDataBound event handler to the parent repeater so I can set the DataSource of the child repeater.

<asp:Repeater ID="rptManufacturers" runat="server" OnItemDataBound="rptManufacturers_ItemDataBound">
        <%# Eval("Name") %>
        <asp:Repeater runat="server" ID="rptModels">
                <li><%# Eval("ModelName") %></li>
protected void rptManufacturers_ItemDataBound(object sender, RepeaterItemEventArgs e)
        Manufacturer man = (Manufacturer)e.Item.DataItem;
        Repeater rptModels = (Repeater)e.Item.FindControl("rptModels");
        rptModels.DataSource = man.Models;

I always found it annoying that I had to create a ItemDataBound function when the only thing I needed to do was bind the child repeater. However, recently I found out you don’t need to bother with all the above code. You can just do this:

<asp:Repeater ID="rptManufacturers" runat="server">
        <%# Eval("Name") %>
        <asp:Repeater runat="server" ID="rptModels" DataSource='<%# Eval("Models") %>'>
                <li><%# Eval("ModelName") %></li>

This is a really simple solution when the only thing you need to do is bind a child control ( Repeater, DataGrid, GridView, etc ). The solution doesn’t really apply if you need to do more logic on the ItemDataBound event.

Automatic Update of Datamodel In Linq

One of the issues that we came across fairly quickly when converting some projects to Linq was how unusable the default Linq to SQL utility was. The interface worked extremely well for 2 or 3 tables, but quickly became unmanageable with much more than that.

We began looking for other solutions, and discovered a command line utility called SQLMetal that can be used to generate a DataModel from a connection string and/or an XML file.

The solution we settled on uses SQLMetal to generate XML markup from the database, then uses XSL Transformations to make desired property/model changes (Private properties, Delay loaded, etc), and then uses SQLMetal to generate a code file from this XML file.

To start, we created a batch file called updateModel.bat and placed it in the project:

"C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\SQLMetal.exe" /conn:"CONNECTIONSTRING" 
      /timeout:0 /namespace:MODELNAMESPACE /context:DATACONTEXTNAME /language:csharp /pluralize 
"%~msxsl.exe" "%~dp0DataModel.dbml" "%~dp0ModifyDbml.xslt" -o "%~dp0DataModel.dbml"
"C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\SQLMetal.exe" /code:"%~dp0DataModel.cs"
    /namespace:MODELNAMESPACE /context:DATACONTEXTNAME /language:csharp 
    /pluralize %DataModel.dbml

The output of SQLMetal in the first line of this file is an XML file called DataModel.dbml that looks something like this :

  <Table Name="dbo.Person" Member="Persons">
    <Type Name="Person">
      <Column Name="PersonID" Type="System.Int32" DbType="Int NOT NULL IDENTITY" IsPrimaryKey="true" IsDbGenerated="true" CanBeNull="false"></Column>
      <Column Name="AddressID" Type="System.Int32" DbType="Int NOT NULL" CanBeNull="false"></Column>
      <Column Name="Email" Type="System.String" DbType="VarChar(255) NOT NULL" CanBeNull="false"></Column
      <Column Name="NameFirst" Type="System.String" DbType="VarChar(255) NOT NULL" CanBeNull="false"></Column>
      <Column Name="NameLast" Type="System.String" DbType="VarChar(255) NOT NULL" CanBeNull="false"></Column>
<Association Name="FK_Person_Address" Member="Address" ThisKey="AddressID" OtherKey="AddressID" Type="Address" IsForeignKey="true"></Association>

The second line of this script uses a utility called msxsl.exe (note that this requires MSXML). This program uses a file called ModifyDbml.xslt to perform an XSL tranformation on the DataModel.dbml file.

<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0"
  <xsl:output method="xml" indent="yes"/>

  <xsl:template match="dbml:Database/dbml:Table/dbml:Type[@Name = 'Person']/dbml:Column[@Name = 'AddressID']">
      <xsl:apply-templates select="@* | node()" />
       <xsl:attribute name="AccessModifier">Private</xsl:attribute>

<xsl:template match="@* | node()">
      <xsl:apply-templates select="@* | node()"/>

This will make it so the “AddressID” property of “Person” is a private property – and this is where all such DataModel changes should be stored. Note that any changes made directly to the DataModel after this point will be lost each time the files are generated.

The final line of this script generates a DataModel.cs file from the updated XML file.

Finally, we looked for a way to call this script to update from within visual studio. To do this, we went to Tools -> External Tools -> Add, and used the following arguments:

This allows you to highlight the updateModel.bat from Visual Studio and go to “Tools->CMD Prompt” to update your DataModel.

Threading with Impersonation in an ASP.NET Project

Add to FacebookAdd to DiggAdd to Del.icio.usAdd to StumbleuponAdd to RedditAdd to BlinklistAdd to TwitterAdd to TechnoratiAdd to Yahoo BuzzAdd to Newsvine

Every once in a while, you might run into a need to do something that takes some time in a web app, but doesn’t require user interaction. Maybe you are processing an uploaded file (rescaling images, unzipping, etc). Maybe you are rewriting some statistical data based on new posts. Basically, something that takes minutes or hours – but isn’t that important to be interactive with the user.

You could set up a “job” in a database to be run the next time your timer runs (see If you don’t have a timer yet, though, that can be overkill if you don’t care that multiple jobs may run at once.

In my case, I needed to export a large volume of data to a zip file. I asked up front for an email address – and the user will receive a link to the completed zip in an email later. The job would only be performed by admins, and even then only about once a year – so there was no need to schedule the job – I could just fire it off when the user requested it.

Any easy way to do this is to use the .NET threading objects in System.Threading. Because I need to save a file, I also have one additional issue – Threads don’t automatically run under the same account that the site does, so I had to include code to impersonate a user that has write permissions.

Here’s a bit of code to get you started:

// param class to pass multiple values
private class ExportParams
            public int UserID { get; set; }
            public string Email { get; set; }
            public string ImpersonateUser { get; set; }
            public string ImpersonateDomain { get; set; }
            public string ImpersonatePassword { get; set; }

        protected void btnExport_Click(object sender, EventArgs e)
//  .... code to get current app user, windows user to impersonate .....

            Thread t = new Thread(new ParameterizedThreadStart(DoExport));
            t.Start(new ExportParams(){
                ImpersonateUser = username,
                ImpersonateDomain = domain,
                ImpersonatePassword = password
             // show user 'processing' message .....

        private void DoExport(object param)
            ExportParams ep = (ExportParams)param;

            using(var context = Security.Impersonate(ep.ImpersonateUser , ep.ImpersonateDomain,
             ep.ImpersonatePassword ))
            // do the work here..............

Here’s the relevant part of the Security class that does the impersonation:

using System.Runtime.InteropServices;
using System.Security.Principal;
// .....
public class Security {
public const int LOGON_TYPE_INTERACTIVE = 2;
        public const int LOGON_TYPE_PROVIDER_DEFAULT = 0;
        // Using this api to get an accessToken of specific Windows User by its user name and password
        [DllImport("advapi32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
        static public extern bool LogonUser(string userName, string domain, string passWord, int logonType, int logonProvider, ref IntPtr accessToken);

        public static WindowsImpersonationContext Impersonate()
            return Impersonate("DEFAULT_USER","DEFAULT_DOMAIN","DEFAULT_PASSWORD");
        public static WindowsImpersonationContext Impersonate(string username, string domain, string password)
            IntPtr accessToken = IntPtr.Zero;
            var success = LogonUser(username, domain, password, LOGON_TYPE_INTERACTIVE, LOGON_TYPE_PROVIDER_DEFAULT, ref accessToken);

            if (!success)
                return null;

            WindowsIdentity identity = new WindowsIdentity(accessToken);

            return identity.Impersonate();
// ..........

Radio button within a repeater problem

Recently I was developing a system to create tests and test questions.  For these tests our client wanted multiple choice questions.  To implement this I decided to have a list of textboxes for the answer text, and a radio button for each textbox to select the correct answer.   I knew that a RadioButtonList couldn’t have anything other than a radio button and text, so I went with a repeater.

<asp:Repeater ID="rptRadios" runat="server">
            <asp:RadioButton ID="rbRadio" runat="server" GroupName="RadioGroup" />
            <asp:TextBox runat="server" ID="txtRadio"></asp:TextBox>

Doing it this way caused the group name of each radio button to be inconsistent, because of the repeater. After a while of researching and not finding any good solutions I decided to try changing the group name of the radio buttons using jQuery.

$("input:radio").attr('name', 'RadioGroup');

That gave me the radio button functionality that I wanted, but it prevented me from getting the selected radio button on postback. So I decided to just implement the radio button functionality manually.

var radios = $("input:radio"); {
     $(this).attr('checked', 'checked');
     return true;

Which gave me the correct functionality and I could still get the selected radio button and textbox on postback. Probably not the most elegant solution, but I couldn’t find any other way to do it.

Also I needed to make sure at least one of the radio buttons was selected so I added a CustomValidator that called a javascript function.

function ValidateRadioButtons(sender, args) {
     args.IsValid = $("input:radio:checked").size() > 0;

Registration Code Generation

I was recently assigned the task of creating unique registration codes for a product. The requirements were as follows:

  • 12 digits long
  • 3 groups of 4, separated by hyphens
  • No duplicates, of course
  • Easy for anyone to recognize and enter

I began by using the GUID.NewGuid() method. This gave me a long hexadecimal string of random characters between 0 and 9, a-f. By taking various substrings from this and then making sure there were no collisions with any existing registration code, I could have a very randomized registration code generator.

However, upon further reflection–and recalling difficulty I myself have had in the past trying to enter the right codes–I came up with the following issues:

  • It’s really difficult to distinguish between a zero and the letter “O”. The people trying to enter the registration code would not know that it was generated from a GUID that only had the letters A-F, so I wanted to eliminate this confusion for the user.
  • It’s really difficult to distinguish between the lowercase “L” -“l” and the number one (“1”) in some fonts.
  • You probably want to avoid some obvious combinations in the code like “KKK” or “666”. Even though our code would contain groups of 4 characters, it seemed best to still avoid those.

We opted to make the registration code all upper case. We also decided to change the ones and zeroes to “h’s” and “k’s. We skipped “g” since it goes “below the line” in the lower case version. We also skipped “i” since that can look like a lowercase “l” or a “1” if, for whatever reason, the printing isn’t clear. So, with all of that, here’s the code:

public string CreateRegistrationCode()
    string code = "";
    while (code == "") //repeat until you don't get a collision
         //replace zeroes and ones to eliminate confusion between o's and L's.  
         //Also get rid of the hyphens in the GUID
         string newGuidString = Guid.NewGuid().ToString().Replace("-", "").Replace("0", "h").Replace("1", "k");
         code = newGuidString.Substring(0, 4) 
                 + "-" + newGuidString.Substring(4, 4) 
                 + "-" + newGuidString.Substring(8, 4);
         code = code.ToUpper().Replace("KKK", "HHH").Replace("666", "777"); // just to avoid an undesirable code

         var codeExists = (from r in DataContext.Registrations
                                  where r.Code == code
                                  select r).Any(); //test to see if the registration code already exists.  

         if (codeExists)
               code = "";

    return code;