Add end to end testing to your continuous integration

You already know those technologies ? Jump directly to how to setup Travis CI & SauceLabs for Protractor.

Reminders

What are Travis CI, SauceLabs & Protractor ?

Protractor is an end-to-end test framework for AngularJS applications. It can also test any non-angular applications since it’s a wrapper around WebDriverJs (which controls the Selenium Server) and Jasmine (which brings the testing framework).

Via Selenium, it will send commands to web browsers (like “click here” then “check if this is displayed”, “populate that field”, “submit the form” then “check if we are logged in” …).

SauceLabs is a cross-browser automation tool built on top of Selenium WebDriver. It lets you run your end-to-end tests on multiple browsers and operating systems, in the cloud (you can run tests in multiple vms in parallel). It integrates very well in CI tools such as Travis CI.

Travis CI is an open-source hosted, distributed continuous integration service used to build and test projects hosted at GitHub. You configure it with a simple .travis.yml file where you specify your CI workflow (you can launch test scripts as well as deploy scripts).

It will try to build your project and run your tests on each push (on any branches, including pull-requests). It supports many languages and you’ll see a lot of open-source projects using it.

If you want to know more – some resources:

• Introduction to SauceLabs
• Introduction to Protractor

Why use SauceLabs with Travis CI ?

By default, Travis CI doesn’t provide extended features for e2e testing. You’ll see that on SauceLabs, you can use your Protractor/Selenium tests “as is” (just add a little config) and get lots of interesting informations such as extended logs, screencast (video playback) …

Setup Travis CI & SauceLabs for Protractor

This part takes for granted that you already know how to run protractor tests in local.

Pre-requisites

Create a SauceLabs account

• For open source accounts
• For regular accounts

Download Travis cli

Follow these instructions to download the Travis Command Line Interface.

Setup SauceLabs credentials

Go to your SauceLabs account and retrieve your ACCESS KEY. Then, at the root of your project:

travis login

#the following will encrypt and add the tokens to your .travis.yml

travis encrypt SAUCE_USERNAME=[your-SauceLabs-login] --add
travis encrypt SAUCE_ACCESS_KEY=[your-SauceLabs-AccessKey] --add

Enable Sauce Connect addon for Travis CI

Travis CI integration with SauceLabs automatically sets up a tunnel required to get started testing with. To do that we need to enable the Sauce Connect addon for Travis CI.

To enable Sauce Connect for Travis CI, add the following to your .travis.yml file:

addons:
  sauce_connect: true

That way, the e2e tests launched on your Travis CI will be executed on SauceLabs’s Selenium server which in return will have access to the test server you’ll be running from your Travis CI – all this, via Sauce Connect’s encrypted tunnel.

Upgrade your protractor config

Add the following to your protractor config file. This is a basic config. It won’t affect your local tests, it will only activate when running on Travis CI:

if (process.env.TRAVIS) {
  config.sauceUser = process.env.SAUCE_USERNAME;
  config.sauceKey = process.env.SAUCE_ACCESS_KEY;
  config.capabilities = {
    'browserName': 'chrome',
    'tunnel-identifier': process.env.TRAVIS_JOB_NUMBER,
    'build': process.env.TRAVIS_BUILD_NUMBER
  };
}

Feel free to customize it (if you don’t work with protractor but only Selenium WebDriver, it’s about the same).

Don’t forget to add the task running protractor to the tests to be run by Travis CI on the .travis.yml file, in the script section.

Tips

Keep in mind you’ll need a server to run your e2e tests against (this is not unit testing), so you will have to launch an instance of this server in the before_script part of the .travis.yml file.

Since you are doing tests, this server might have to be in “test” mode (providing mock results on API endpoints for example).

Even if your app is full front-end and doesn’t rely on a backend you built, you will have to provide a server to run your e2e tests on (this is my case on topheman/vanilla-es6-jspm).

Good thing to know: when you develop in ES6 using runtime transpiling (like babel-runtime), a lot of different scripts are loaded (and evaled browser-side), which can raise timeouts SauceLabs-side (that you didn’t have on your local computer when running your e2e tests).

To avoid that, run your e2e tests against a bundled version of your website (all scripts/assets bundled/concateneted in one or a few files). This will relieve the traffic on the sauce tunnel and the VM, SauceLabs-side (that may not be as powerfull as your computer).

Resources

I set up Travis CI with SauceLabs for Protractor on my latest project: topheman/vanilla-es6-jspm, to have my e2e test being a part of my continuous integration workflow.

Take a look at the commit where I added the feature, it could give you some ideas of implementation …

Tophe

Edit: Using React ? Here is one of my latest projects topheman/react-es6-redux where I also setup SauceLabs (amongst other things like unit-tests, code coverage on es6+ …)