Saturday, January 18, 2020

Practical test strategy for Spring & React application



Contents:

1) Introduction
2) System overview
3) Functional test strategy
4) Non-functional test strategy

1. Introduction

There are two leading approaches for tests: isolated and end to end (E2E). As usual, both approaches have upsides and downsides and it's impossible to say which one is better. Everything is dependent on application context and architecture.

There are though few guidelines which can be followed most of the time:
  • high test coverage should be achieved for a single application. Other parts of the system (i.e. connected microservices) should be stubbed/mocked
  • important business cases should be tested in the E2E manner without any mocks. E2E tests should give us a high amount of confidence that our system will work on the production
  • isolated tests should make up the majority of your overall test count
  • E2E test count should be low because they may be flaky
It remains to be seen whether or not contract tests would make a list of best practices. I certainly recommend adding them to the list.

Enough of theory, let's move into applying it in real life.

2. System overview

Let's say we want to test a very simple system with:

To complicate things a bit let's also assume that our backend is interacting with external API served by the other service (which we have limited control of).


The application I wrote is very simple, it only allows to CRUD (create, read, update, delete) a user but it's about to overcome Facebook soon as the most popular social media platform so we need to prepare a comprehensive test strategy.


3. Functional test strategy

We need the following functional tests:


Isolated tests should be in the same repository as applications and they should run in pipelines after each commit. For E2E tests we need a separate repository. They should run as often as possible. Results should be recorded and displayed on a screen placed in a visible area.

For simplicity let's assume we are not covering non-functional characteristics at this point.

4. Non-functional test strategy

We need the following performance tests:
  • Our API backend should be tested in isolation with tools like Gatling or Locust. JMeter is not recommended because we want to store performance tests as a code
  • We measure website performance using Lighthouse on test environment and production. Results should be recorded and displayed on a screen placed in a visible area.
In the beginning, we assume that the application should handle 100 simultaneous users. Tests should prove that we can handle 200. 

2 comments:

  1. Hey,

    Helpful article - I believe it has covered all of the essential layers of the *test pyramid*. Classic! I believe you might be already familiar with "The Practical Test Pyramid" by Ham Vocke, but I thought it might be useful to share it here as well: https://martinfowler.com/articles/practical-test-pyramid.html

    I have a question, though, regarding the "For E2E tests we need a separate repository": why is that? How is that beneficial? From my point of view, it is much easier to avoid decoupling from the AUT while having them in the same one, as well as to prevent flakiness by using e.g., shared data test IDs. I understand that the separate one might seem to be the right choice from the "being independent" point of view, but like I said, what is the actual benefit?

    Also, since this is a React on the front-end, why not using Cypress for the E2E as well? Is there any particular reason behind using Selenium?

    ReplyDelete
    Replies
    1. Hi, thanks for your comment :)

      Regarding repositories:
      I'm trying to follow very simple pattern. When tests are independent from external factors (i.e. fully controlled from tested application) I add them in the same repository. When they touch external systems I recommend separating them in some way. External repository is the most flexible solution. You can for example manage dependencies without any conflicts. Alternatively you can create separate Maven/Gradle module.

      Regarding E2E tests and Selenium:
      Selenium is still the most powerful testing tool. It allows you to write in any language and test any browser. For E2E I don't want to be sandboxed in what Cypress is offering.

      Delete