Reaching peak efficiency in Parallelism without flakiness utilizing nested describe blocks in Playwright

Introduction

Have you worked with interdependent tests in Playwright that cause flakiness when run in parallel mode? In this blog, we'll explore how to reach peak efficiency while running tests in parallel without flakiness, utilizing nested describe blocks in Playwright.

Getting started

Playwright allows the user to run all the tests in parallel. But with parallelism comes a set of problems the user faces since there will be some tests that can affect the execution of another test.

Consider a scenario where the functionality of our test is to verify the count of emails. Since all the tests are running in parallel, there's a high probability that the other tests will also create a new email during execution. This will break our test since the count of emails will increase due to the interaction by the other tests, and we cannot reliably predict what the count will be.

Different execution modes in Playwright

There are three execution modes in Playwright, which can be configured as:

1. Parallel: All tests run in parallel to each other. If one test fails, all the other tests are not affected.

test.describe ("Parallel execution mode", () = {

test.describe.configure({ mode: "parallel" });

  test ("Test A", async () => {
    ...
  });

  test ("Test B", async () => {
    ...
  });
});

2. Serial: All tests are executed sequentially, meaning each test is executed one after the other. If one test fails, all subsequent tests are skipped.

Note: Executing tests in serial mode is not recommended. It is usually better to isolate our tests so that they can be run independently.

test.describe ("Serial execution mode", () = {

test.describe.configure({ mode: "serial" });

  test ("Test A", async () => {
    ...
  });

  test ("Test B", async () => {
    ...
  });
});

3. Default: All tests are executed sequentially, meaning each test is executed one after the other. If one test fails, it does not affect the execution of the other tests.

Note: Executing tests in default mode is not recommended. Even though default mode is preferred over serial mode, it is usually better to isolate our tests so that they can be run independently.

test.describe ("Default execution mode", () = {

test.describe.configure({ mode: "default" });

  test ("Test A", async () => {
    ...
  });

  test ("Test B", async () => {
    ...
  });
});

Nested describe blocks

Describe block is a logical way to group together tests that are similar. Playwright doesn’t limit us to having only one describe block per spec. We can have multiple describe blocks within a spec, which can also be nested within each other. These are called nested describe blocks.

Achieving peak parallelism for interdependent tests

Now that we have an understanding of what the different execution modes and nested describe blocks are, let’s see how to achieve peak efficiency by applying these concepts. Let’s try to visualize the problem of running interdependent tests in parallel, with the help of the following scenario. Consider five tests named T1, T2, T3, T4, and T5. Here T1, T2, and T3 are interdependent when run in parallel while T4 and T5 are independent of all others.

In such cases, we can group the tests T1, T2, and T3 into a single describe block and configure them to run in default or serial mode. We can have a separate describe block for T4 and T5, which will run parallel to the first describe block. Let’s visualize this example better with the help of a time chart.

This is how we can reach peak efficiency while working with any number of interdependent tests. We just need a clear idea about the tests that can affect each other causing failures.

test.describe("This is the outer describe block", () => {

  test.describe.configure({ mode: "parallel" });
  
  test.describe("This describe block contains Test 1, Test 2, and Test 3", () => {

    test.describe.configure({ mode: "default" });

    test("Test 1", async () => {
      .
      .
      .
    });
 
    test("Test 2", async () => {
      .
      .
      .
    });
 
    test("Test 3", async () => {
      .
      .
      .
    });
  });

  test.describe("This describe block contains Test 4, and Test 5", () => {

    test.describe.configure({ mode: "parallel" });

    test("Test 4", async () => {
      .
      .
      .
    });
 
    test("Test 5", async () => {
      .
      .
      .
    });
  });
})

When to avoid nested describe blocks

It is advisable to avoid too many nested describe, test, and step blocks, as this can make the code unreadable. It is also advisable to limit the maximum nesting of blocks to two.