Agile Zone is brought to you in partnership with:

I am a programmer and architect (the kind that writes code) with a focus on testing and open source; I maintain the PHPUnit_Selenium project. I believe programming is one of the hardest and most beautiful jobs in the world. Giorgio is a DZone MVB and is not an employee of DZone and has posted 638 posts at DZone. You can read more from them at their website. View Full User Profile

Selenium is not a panacea

09.07.2010
| 9607 views |
  • submit to reddit

Selenium is a great tool for acceptance testing of web applications: it works with real browsers, and drives them to exercise your application of choice in the same way a real user would do. But it is incredibly fast in performing this real world tests, as the interaction from the fake user (Selenium itself) and the server is instantaneous.

Basically, it's like putting a robot at work with a list of instructions, knowing that it will be much faster than any human in this repetitive job. Selenium is also more and more becoming mandatory for acceptance testing, since it gives you the capability to test web applications built via Javascript, while similar tools like Zend_Test or HttpUnit are not so evolved.  However, it's not so simple: there are many gotchas in using Selenium to drive JavaScript-powered applications.

Our application will be the case study for this article. The environment comprehends PHPunit test cases that extend the Selenium one (PHPUnit_Extension_SeleniumTestCase), and run against a sandboxed instance of the application which can be thrown away at the end of the test suite execution. A Selenium RC server is also necessary, as it would expose the HTTP proxy server needed for the client tests to work. I'm writing this article so that I can refer to it in the future when it comes the time to work again with Selenium, and to avoid repeating the same errors and false assumptions all over again.

Basic issues

Let's start with the typical issues that a developer encounters when he starts using Selenium.

For starters, the generated tests created via recording in Selenium IDE are garbage: they use DOM ids and every kind of reference to the elements physical structure that will break every day from here to the eternity. You can reuse their flow (maybe), but not their CSS selectors.

Remember that you need a Selenium RC server to run a real suite, and you must exclude the acceptance test from the build where this harness is not available. If you want to run the whole test suite on a Continuos Integration server that does not have a a graphical interface, follow the related article.

Last but not least, when JS frameworks get in the way, it's not so simple to model the user interaction. They generate a lot of markup, which you will find hard to navigate into. There are alternative to fighting with the DOM.

Accessing the widgets

So navigating the DOM is difficult, and it may break very often since JavasScript frameworks employ generated markup and ids.
A colleague of mine, during a sleepless night, had the brilliant idea of setting up a series of Proxy objects for the JavaScript widgets and DOM nodes which act as Test Helpers, to use $this->getEval() of Selenium test cases to execute JavaScript commands. Essentially, this infrastructure lets you work with PHP objects that resemble the JavaScript framework's ones, and forward calls and field access to the JavaScript objects created by the browser.

It works like this: a test harness built over Selenium translates the JavaScript data with window.objclone() into a JSON structure that can be read on the PHP side with json_decode(). We intend to implement also __call() on proxies to be able to call the JS widgets methods (at least the ones without parameters).

There is a trade-off between the approaches of executing code on the browser side or on the test suite side:

  • more logic in PHP makes the test code easier to work with, and to wrap with helper methods. It is also possible to dump values for debugging.
  • More logic in JavaScript means reusing the JavaScript framework (ExtJS in our case) and you can grab the objects defined in the client (window.Ext for example).

Thus we are seeking a combination of the two approaches.

The goal is to build a DSL for acceptance testing: tests will be written with this higher-level API where not only there is a Page Object pattern, but also objects for all the widgets that link to each other, and skip the various DOM nodes and references that get in the way.

With this DSL in place, there will be no need for JavaScript gurus to read an acceptance test or to slightly modify one; nor for writing ones with already available components for ExtJS created by the most able programmers.

Events

DOM events are also not simple to manage, since you are never sure if you're generating the right event on the right element via PHP code.

Remember that clicks are not your only tool: there are many other events that an trigger an action by the Ajax widgets, like mousedown, blur and change. Don't rely on Selenium IDE to discover the right events for you: it will seldomly record everything you produce, and often only the wrong, obvious event that does not work alone when transported in your tests.

Simulating text

$this->type() is available in SeleniumTestCase subclasses for inserting text into widgets. There are many other methods that generate the related events, such as keyPress() and keyDown().

These methods can even be used to produce special characters like \r, \t, to simulate the pressure of the Enter or Tab key, which is quite useful when widgets respond to this event.

The scope

Keep an eye on the window object, which is not the default anymore. For example, you'll have to call window.Ext or window.$ when writing code for $this->getEval().

Since every JavaScript snippet submerged in PHP code passes from getEval(), you should be careful in copying and pasting code from the Firebug or Chrome console to your test cases.

Final recommendations

When writing tests for Selenium, work in very little iterations, which are quick if you execute one test at the time. Selenium will perform all the user interaction by responding just after the page has finished loading itself or its components. You'll slowly gain confidence and become able to skip intermediate steps.

You should also always use a mockup, even when working with Acceptance Test-Driven Development. When you have the mocked up test, you can move to the back end to make it work with more than one data set, following the rules of Red-Green-Refactor on your server-side code. However, exploring the DOM in your brain memory is not a good idea for writing tests if you do not have a quick live application available (always in a sandbox) as a reference.

If creating a sandbox is too complicated, work on your Phing/Ant build scripts to automate it. This will pay back in the future, not only for the sake of easy testing but as a mean to quickly build your application.

Published at DZone with permission of Giorgio Sironi, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)