Thursday, April 18, 2019

Headless Browser Testing with Selenium


1) What is a headless browser?
2) Benefits of headless testing
3) Limitations of headless testing
4) Demo introduction
5) Chrome
6) Firefox
7) HtmlUnit
8) JBrowser
9) PhantomJS
10) Summary

1. What is a headless browser?

A headless browser is a very popular term in the testing community which refers to a web browser running without Graphical User Interface (GUI). A headless browser can access any website but unlike normal browsers (which you currently use) nothing will appear on the screen. Everything is done on the backend side invisible to the user.

Recent popularity rise of such browsers led to the creation of a new term: headless testing. It is running a UI-based browser test using a headless browser.

In this article, I'm going to explain when is it worth to use headless browser and what are the actual limitations. As usual, I have also prepared a Java demo with 5 most popular headless browsers used with Selenium WebDriver - Chrome, Firefox, HtmlUnit, PhantomJS, and JBrowser.

2. Benefits of headless testing

Why do people use headless browsers?

a) Speed - since headless browsers don’t have a UI, they are faster than real browsers. 
b) Dependencies - sometimes you don't have full access to test environment and headless browsers are much easier to setup on Continuous Integration servers like Jenkins
c) Data extraction - if your task is to scrape some data from website headless browser would do it much faster
d) Performance monitoring - headless browsers are often used to measure the client-side performance
e) Layout testing - headless browsers are useful when testing layout (for example with Galen framework). You can even automate screen captures for layout checks

3. Limitations of headless testing

When you should not use headless browsers?

a) Functional tests - your users are not using a headless browser. You need a real browser to provide real feedback.
b) Debugging - headless browsers are hard to debug. You can't visually follow your tests step by step. 

4. Demo introduction

This time demo will be done using pure Selenium which you can copy paste into your project. I'll reuse page objects which have been fully introduced in Selenium vs FluentLenium post. In order to keep this post at a reasonable length, I'll skip a few implementation details. Please refer to my GitHub project for details.

Each headless driver would run the following scenario:
  1. Open Awesome Testing blog
  2. Search for 'public speaking' term
  3. Verify that something has been found
  4. Click on the first found article
  5. Verify the comments section is present in the article
Inheritance will be used to make this method available for each headless test.

Just for reference here is how search method implementation looks like.

And final assertion...

5. Chrome

Google Chrome is perhaps the most popular headless browser. In order to use ChromeDriver, the following dependency is required.

Setup requires adding two chrome switches into Chrome options: --headless, --disable-gpu. 

For details and FluentLenium example, please refer to my Headless Chrome post.

6. Firefox

Firefox browser doesn't require an introduction as well. In order to use FirefoxDriver, the following dependency is required.

Setup requires adding one argument into Firefox: --headless.

For details and FluentLenium example, please refer to my Headless Firefox post.

7. HtmlUnit

HtmlUnitDriver is a WebDriver compatible driver for HtmlUnit headless browser. This project is under SeleniumHQ organization so there is no need to worry about maintenance efforts. 

The following dependency is needed.

HtmlUnitDriver doesn't require to install any browser or set a path to driver executable. It's the most lightweight solution of all presented here. This test will work on every platform without any configuration efforts.

8. JBrowser

jBrowserDriver is A programmable, embeddable web browser driver compatible with the Selenium WebDriver spec. It has a few interesting features that I'd like to present in a separate post one day. No spoilers though :)

The following dependency is required.

Unfortunately, jBrowserDriver is the only driver which failed my test so I'm a bit worried about its stability. There are also no guarantees that it will be supported in the long term. For example, it still doesn't work with Java 11.

9. PhantomJS

PhantomJS is no longer and I'm adding it only for reference. I can't see any reasons to use it in 2019.


And test...

10. Summary 

Headless browsers give us a few interesting benefits (speed being by far the most important), but you have to remember about their limitations. There is no guarantee that website working perfectly with headless mode would work in a real browser.

If you are about to start your journey with headless Selenium I recommend to use Chrome or Firefox. You can easily debug any issues by commenting --headless switch and see actual browser behavior. For more experienced users I recommend HtmlUnitDriver which we use a lot in FluentLenium project and can confirm that it is a stable solution.