Selenium Regression Testing Part II – Tips and Tricks

In my last post, I talked about how you can use Selenium to do real regressions tests for web applications. It’s a great way to automate testing the real user experience, and not just the backend stuff.

(see Luke’s last post here: Web Application Functional Regression Testing Using Selenium)

That said, Selenium is a relatively new technology, and it’s not without its issues. When building your first test you might find a lot of times where it’s a trial and error process. There are so many different ways to do the same test, it can be overwhelming. Often times, one or more of the ways you try won’t work. I’m going to try to list some of the common problems I ran into and tips I found below.

• Selenium Commands
  • waitForElementPresent
    • this is will block your test proceeding until it finds the identified element.  It checks once a second for 30 seconds.
    • this was the most common way I dealt with ‘ajax’ type interactions where it takes an unknown period of time for something to show up
    • I also use it generally instead of verifyElementPresent – it does basically the same thing with a little wiggle room
  • mouseDown/mouseUp/mousePressed/click
    • mostly equivalent, but sometimes you need to match the desired event to a javascript handler
    • try click first.  If it doesn’t work the way you want, move on to mouseDown and so on.
  • waitForFrameToLoad/selectFrame
    • important if you use iFrames (modal dialog, etc.)
    • the selenium selectors only hit the current frame, otherwise you have to select the correct frame
    • an easy way to get back to the root window is to do selectFrame null
  • type vs. typeKeys
    • type fills out an input in code – if it works, use this.  You can use this,and then fire a single typeKeys for the last character if you need an event to be triggered.
    • typeKeys fires the key events on the element
      • has some idiosyncracies – certain letters (I’m looking at you,‘y’) are reserved to do other special keypresses
      • necessary if you are using a wysiwyg ‘designmode’ type box instead of a standard input
  • verifyX vs. assertX
    • if verify fails, the test continues (and is marked as errored).  If assert fails, the test aborts.
    • Usually verify is better, unless one task blocks a future one from functioning correctly
  • runScript vs. Eval vs. Expression
    • runScript inserts the javascript you provide into the current frame/window.  Useful for those things selenium doesn’t support – like moving a cursor around and selecting text in a wysiwyg
    • Eval runs javascript in the context of selenium. Handy for complex checks – use waitForEval (for instance, checking the css background image property of a particular element)
      • Use this.browserbot.findeElement(“selenium selector”) to find elements the way selenium would
      • Use window.X To access the current frame/window context objects
    • Expression is similar to Eval, but uses Selenium’s custom expression format instead of javascript (but you can combine with javascript by using  javascript{}
      • storedVars[‘key’] allows you to get to a variable you created with a Selenium ‘store’ expression
  • selectPopUp
    • useful for checking stuff in a popup that was initiated
    • Easiest to get by the html title of the popup, but do a ‘pause’ first to let it load
• Selenium Selectors and XPath
  • In general, be as abstract as possible.
    • Don’t select individual server generated ids (hand crafted html ids are ok if you don’t expect them to change)
    • Don’t select on complicated relationships ( /div[0]/div[2]/a[4] ) – your html structure will change and you’ll have to maintain it
    • Select links by the simple link=text when possible – easy to read/maintain, unlikely to change
    • Use //that (any decendant) instead of /this/that where possible
    • .  references ‘this’ element.  Helps to select something with a particular text:   //div[@id='publish-private-shares']//p[.='This is pretty cool.']
    • Contains() is useful if you don’t know the exact text (for instance, when an element has multiple css classes):     //div[@id='pageContent' and contains(@class,'contenteditable') and h2='Goals']/p[1]
• Selenium RC
  • While you can use Selenium IDE to create a c# version of your tests – if you do so, you have two tests to maintain.  You can run your ‘selenese’ tests directly with RC, too.
    • JAVAPATHjava.exe –jar SELENIUMPATHselenium-server.jar –htmlSuite “*BROWSER” “BASESITEURL” “SUITEFILEPATH” “RESULTSFILEPATH”
    • I’ve written a simple csharp console project that automatically finds the correct javapath and fires up the test when you run it.  If people ask in the comments, I’ll post it.
  • Last I checked, Chrome and Safari-Windows don’t work.  Chrome is supposed to be fixed in Selenium RC 1.0.4
• Sauce RC
  • This is a great UI to help test multiple browsers, but there are a couple of issues
    • Firefox works, but only in dual window mode
    • IE works, but only in single window mode.
    • The ‘timeout’ setting implies a default timeout per action in your test, but it is actually the timeout for your entire test run.  Since it defaults to 30 seconds, you’ll probably want to change it, or else your tests will suddenly die for no reason with no explanation/log.

I’m sure there is probably more I’ve forgotten, so leave a comment if you get stuck and I’ll try to help out if I can.

For some helper classes for Selenium 2 with ASP.NET Forms, see the following post: Simplifying C# Selenium 2 Tests for ASP.NET WebForms