Scaling Playwright tests with tag annotations

On the business-to-business side of Zoopla we have a SaaS (Software as a Service) product called Alto which is a sales and lettings CRM for estate agents. The product comprises of multiple applications, each a microfrontend. Each microfrontend uses Playwright for its automated browser based tests. These tests include:

• Accessibility checks using Axe
• Functional tests to ensure our applications work as intended
• Visual regression tests using screenshots
• Smoke tests to cover key customer journeys

Alto has scaled a lot over the past few years; it's healthy for teams to revisit technical decisions and patterns to constantly iterate and improve the efficiency of the codebase. In the front-end space, we've spent a lot of time improving our implementation of Playwright.

The Problem

Each microfrontend has its own test directory which contains the spec files for the above test types. It was structured like so:

tests
    accessibility
        application.spec.ts
    functional
        components
            component.spec.ts
        page1.spec.ts
        page2.spec.ts
        page3.spec.ts
    smoke
        desktop.spec.ts
        mobile.spec.ts
    utils
        helperFunctions.spec.ts
    visual
        desktop.spec.ts
        mobile.spec.ts

Each type of test had its own directory. Our Playwright configuration would use projects to define which suite of tests to run:

// playwright.config.ts
export default config: PlaywrightTestConfig = {
    projects: [
		{
			name: 'accessibility',
			testDir: 'tests/accessibility',
		},
		{
			name: 'functional',
			testDir: 'tests/functional',
		},
		{
			name: 'smoke',
			testDir: 'tests/smoke',
		},
		{
			name: 'visual',
			testDir: 'tests/visual',
		},
	],
};

We'd use the project filter to only run the desired test type:

"scripts": {
    "test:a11y": "playwright test --project=accessibility",
    "test:functional": "playwright test --project=functional",
    "test:smoke": "playwright test --project=smoke",
    "test:visual": "playwright test --project=visual",
},

This worked great! We were able to run the suite of tests filtered by type which is a neat way of only running what we need when we need it.

So why change it?

As the codebases scaled we noticed each type of test was repeating a large chunk of code. While each test type would have a different outcome, we would run the same set of steps and duplicate a lot of code:

// Functional
test("foo bar", async ({ page }) => {
  await page.goto("/foo/bar");
  await page.locator("text=Foo");
  page.locator("text=My Button").click();

  // Assertions
  await expect(page.locator("text=Bar")).toBeVisible();
  await expect(page.locator("text=Bar")).toBeEnabled();
});

// Accessibility
test("foo bar", async ({ page }) => {
  await page.goto("/foo/bar");
  await page.locator("text=Foo");
  page.locator("text=My Button").click();

  await injectAxe(page);

  // Check for accessibility errors and warnings
  await checkA11y(page, undefined, {});
});

// Visual
test("foo bar", async ({ page }) => {
  await page.goto("/foo/bar");
  await page.locator("text=Foo");
  page.locator("text=My Button").click();

  // Screenshot
  await page.screenshot({
    animations: "disabled",
    path: `foo-bar.png`,
    fullPage: true,
  });
});

We want to avoid duplication as much as possible:

1. As the codebase scales, the spec files become larger and more convoluted
2. If we update the UI, we need to update the tests in multiple places
3. We have more code to maintain

We had already made use of helper functions to abstract the more complex steps but as the codebase scales, these files would grow in size so we wanted a more scalable pattern.

The Solution

One of our engineers recommended we look at Playwright's Tag Annotations. While there's only a few lines of documentation to read, we found this feature of Playwright to be incredibly powerful for our use case.

Instead of using projects (as defined above in our configuration file) we use tags to dictate which tests to run. This means we're able to run different test types from combined spec files. For example:

// page1.spec.ts
describe("foo bar", () => {
  test("@accessibility");
  test("@functional");
  test("@visual");
  test("@smoke");
});

We'd remove projects from the configuration and specify the tag like so:

"scripts": {
    "test:a11y": "playwright test -g @accessibility",
    "test:functional": "playwright test -g @functional",
    "test:smoke": "playwright test -g @smoke",
    "test:visual": "playwright test -g @visual",
},

Now if we run pnpm test:functional it will only run the tests tagged with @functional within the application.

The Outcome

As a result, we were able to massively restructure our test files, starting with the test directories:

tests
    components
        component.spec.ts
    page1.spec.ts
    page2.spec.ts
    page3.spec.ts
    utils
        helperFunctions.spec.ts

Revisiting the repeated example above, our spec file would look like so:

// page1.spec.ts
describe("foo bar", () => {
  beforeEach(({ page }) => {
    await page.goto("/foo/bar");
    await page.locator("text=Foo");
    page.locator("text=My Button").click();
  });

  test("@functional", async ({ page }) => {
    await expect(page.locator("text=Bar")).toBeVisible();
    await expect(page.locator("text=Bar")).toBeEnabled();
  });

  test("@visual", async ({ page }) => {
    await page.screenshot({
      animations: "disabled",
      path: `foo-bar.png`,
      fullPage: true,
    });
  });

  test("@accessibility", async ({ page }) => {
    await injectAxe(page);
    await checkA11y(page, undefined, {});
  });
});

We felt structuring our tests this way significantly improves our:

1. Maintainability: the steps are only written once
2. Readability: the grouping of code is more logical
3. Scalability: the test directory is a lot cleaner and well structured
4. Quality: it's easier to spot if we've missed a test type

We're now also exploring page object models to further improve any duplication around navigating our applications.

Image source

• Post image by Angèle Kamp on Unsplash