Introduction

In a previous post about unit testing your Javascript, I explained how you can unit test your own Javascript library code with the QUnit test suite. Now that we can unit test our Javascript, it would be cool to automate the process. We’ll accomplish that by using JsTestDriver, a Javascript test runner, built in Java.

What is a “test runner”?

A test runner is something that runs your tests. In my previous blog post on the subject, we wrote an HTML file which then contained references to the library & scripts to run. By opening that HTML file in a web browser, we ran our tests. In this case, the test runner was the browser itself, and the HTML file was a wrapper for your tests.

Having a browser as a test runner is good, because you are then testing your code in that specific browser. However, it is tedious work to open up the HTML wrapper in each and every browser you have available. Testing takes ages this way. What would be better is some sort of “thing” that can run your test code in a lot of browsers, preferably in parallel, with as little effort as possible. That’s where JsTestDriver comes in sight.

How does JsTestDriver work?

JsTestDriver simply works by starting a server, listening on a port (usually port 4224), where you connect browsers to. All the kinds of browsers you want your code tested agains, should connect to the JsTestDriver server. JsTestDriver then “captures” your browser, and puts it available for running tests in. Then you execute your test suite, and each captured browser will execute your test code. Here’s a small schema I copied from their site:

This kind of setup makes it easier to test your code against multiple browsers. Usually you won’t have different browsers on different OS’es available. What can help, is building virtual machines in VirtualBox and installing the browsers in them. Then all you have to do is point these browsers to the same JsTestRunner server instance, and you have your own little test farm.

Setting JsTestDriver server up

First thing you need to do, is download JsTestDriver from the project page over at google. At the time of writing, the most recent version was JsTestDriver-1.2.2.jar. I have a dedicated folder for it, where I also added the Code Coverage plugin. But more on that plugin later.

Get the server running by entering the following in your shell:

	java -jar /path/to/JsTestDriver-1.2.2.jar --port 4224

Now you have to leave your shell open to keep the server running. If all went well, you should now be able to surf to http://localhost:4224 and see the following:

You can now go ahead, and add your browser to the list of browsers that will execute the test suite, by clicking the link “Capture This Browser”. You should now see this in your browser:

Now your browser is ready and waiting for tests to come in.

Preparing your test suite

JsTestDriver can work with multiple Javascript test suites, but adheres to the JUnit standard. Therefor, if you want to keep on using QUnit, you’ll have to use an adapter. Luckily there’s one readily available, and using the adapter is very easy.

To use the adapter, first go to http://code.google.com/p/js-test-driver/source/browse/#svn/trunk/JsTestDriver/contrib/qunit/src and download equiv.js and QUnitAdapter.js to your test suite. Next you have to tell JsTestRunner where to find these files by adding them into your config file.

For each set of Javascript unit tests you want to run, you have to create a config file for JsTestDriver. In this config file, you specify the location of your files, and optionally which plugins you want to use. Here’s my config file:

server: http://localhost:4224

load:
# Add these lines to load the equiv function and adapter in order, before the tests
- external/equiv.js
- external/QUnitAdapter.js

# Load jQuery external dependency for the TAG library:
- external/jquery.js

# Load the files we want to test:
- lib/tag.js
- lib/tag/*.js

# Load the tests:
- tests/tag.js
- tests/tag/*.js

As you can see, first I load the files necessary for the QUnit adapter. Then I load jQuery, because it’s a dependency for my own library. Next all files of my library are loaded, and lastly the unit tests. Save this file as “jsTestDriver.conf” alongside your other files.

Limitations

The QUnitAdapter does a nice job converting the QUnit tests to something JsTestDriver can understand. However, it is not possible to do asynchronous tests with JsTestDriver. This means that any test where you use the “start()” and “stop()” functionality of QUnit, you will get an error. Keep this in mind when running your tests. They may run fine when you use the browser as your test runner, but fail when you use JsTestDriver.

Running your tests

Now you have to open up a new shell, and just enter the following:

java -jar /path/to/JsTestDriver-1.2.2.jar --config /path/to/jsTestDriver.conf --tests all

If you’re familiar with PHPUnit, you should see similar output:

A dot represents a passed test. An “F” means a Failed test, and “E” indicates that an error happened.

If you have more browsers installed, just go ahead and capture them by surfing to http://localhost:4224. If you then run your test suite again, you’ll see just how many of your tests passed/failed/errored in each browser.

Code coverage

Unit tests are all great, but it’s even better if you had an idea about which parts of your code were actually executed, and which parts were skipped. There is a plugin for this available at the project website. Download, and copy coverage-1.2.2.jar in a “plugins” directory alongside the JsTestDriver-1.2.2.jar file.

Now we need to add it to the jsTestDriver.conf for our test suite:

server: http://localhost:4224

load:
# Add these lines to load the equiv function and adapter in order, before the tests
- external/equiv.js
- external/QUnitAdapter.js

# Load jQuery external dependency:
- external/jquery.js

# Load the files we want to test:
- lib/tag.js
- lib/tag/*.js

# Load the tests:
- tests/tag.js
- tests/tag/*.js

# Code coverage
plugin:
- name: "coverage"
  jar: "/path/to/plugins/coverage-1.2.2.jar"
  module: "com.google.jstestdriver.coverage.CoverageModule"

With the config updated, it’s time to run the tests again. We run the same command as previously mentioned, but with the added option to specify a code coverage output folder.

java -jar /path/to/JsTestDriver-1.2.2.jar --config /path/to/jsTestDriver.conf --tests all --testOutput /path/to/coverage/output

If the specified output directory doesn’t exist yet, it will be created, and in it you will now find some XML files and a jsTestDriver.conf-coverage.dat file in the directory you specified. These files contain all the data of the coverage report, exported into the a format that can be used by other programs to show you a nice overview. If you have LCOV installed on your system, you can convert the .dat file easily to a readable HTML file by entering this command:

genhtml /path/to/jsTestDriver.conf-coverage.dat.

If you now open up the generated index.html file in your favorite browser, you get something like this:

With this coverage report, you can see which parts of your code were executed during the unit tests, and more importantly, which parts weren’t. It’s up to you now, to update your unit tests to get your coverage percentage higher. We would all love to achieve 100% coverage, but I think that if you have about 70% coverage, your library is already well tested.

Conclusion

With JsTestDriver, it’s possible to quite easily automate your Javascript unit testing in more browsers. The process of capturing browsers, and then distributing the unit tests in parallel to them certainly speeds up the whole unit testing itself. Code coverage is very nice to have, as it presents you with a document you can show your client or employer.

The only downside I have discovered so far, is the lack of the ability to create asynchronous tests. Especially with so much website functionality relying on AJAX to update the interface, you really miss this feature. On the other hand, you can construct your code in such a way, that not testing AJAX functionality doesn’t necessarily result in big chunks of code without coverage. Only thing left is to get as many Virtual Machines and browsers connected to the JsTestDriver server, and get testing!