Our Playwright testing standards at Houseful

By Boyana Staneva

20 November 2023

Multiple teams across the Houseful business use Playwright for frontend test automation. While building out our Playwright frameworks, we are aiming to have code that is easy to read, follow and debug. To help us achieve that, as a team we pulled together to create test standards across our repositories that use Playwright.

Having test standards aligned across repositories provides us with benefits when it comes to readability, usability and barrier to entry. For example:

Improves reusability - functions / locators / shared steps / other test code can be easily reused. It is straightforward for someone working on a test to find the functions and elements they need. Leading to less duplication and the need to rework once a code review has started.
Eases reviews - code can be reviewed quicker. It reduces the mental load of reviewing code as you can easily figure out what the test code is doing.
Quicker onboarding - naming conventions help new starters get onboarded quickly and feel comfortable contributing to the codebase.

Here are the test standards and guidelines we adhere to when creating Playwright tests at Houseful.

Playwright Guidelines

Data Creation in e2e tests

Running end-to-end (e2e) tests is expensive. They are costly in terms of resources, can take a lot of effort to setup and can be sensitive to small changes anywhere in the flow, which can make them frail and difficult to maintain over time. Consider other options before going down this route.

If you have to make an e2e test:

DO 👍

Each test has its own data creation. i.e. it creates all the data it needs to complete the checks
Each test has a teardown step i.e. clearing the data

DO NOT ⚔️

Rely on existing data to perform the test
Leave uncleared state after a test

Page Object Model (POM)

Each page should have a corresponding POM file to help the maintainability and scalability of our tests. The POM file should contain all selectors and functions that relate to the given POM.

All interactions should be done via page objects, i.e. no selectors in your tests.

All assertions should be done in your test, i.e. no assertions in the POM.

You can read more about how we leverage POM in our blog on software testing design patterns

DO 👍

// POM file './pom/foo'
// add all locators and functions related to the page. 
// allowing all tests to reuse 

import { Locator, Page } from '@playwright/test';

export class FooPage {
	readonly page: Page;
	readonly pageTitle: Locator;
	readonly buttonFoo: Locator;

	constructor(page: Page) {
		this.page = page;
		this.pageTitle = page.locator('text=My Page');
		this.buttonFoo = page.locator('text=Foo');
	}
}

// tests/foo.spec.ts
// call the POM to use the locators and functions 
// do all assertions needed for the test 

import { FooPage } from './pom/foo';

let fooPage: FooPage;

describe('displays foo bar', () => {
	beforeEach(({ page }) => {
		fooPage = new FooPage(page);
	});

	test('bar is visible', async () => {
		await fooPage.buttonFoo.click();
		await expect(fooPage.titlePage).toBeVisible();
	});
});

DO NOT ⚔️

// tests/foo.spec.ts
// include loctors directly in test

import { Locator, Page } from '@playwright/test';

describe('displays foo bar', () => {
	test('bar is visible', async () => {
		await page.locator('text=Foo').click();
		await expect(page.page.locator('text=My Page')).toBeVisible();
	});
});

Test Structure - Arrange, Act, Assert

Follow the AAA (Arrange, Act, Assert) pattern when structuring the tests. In most cases the Arrange step can be included in a Before block.

Consider including comments defining each section to ease readability.

DO 👍

// arrange, create a let property

await createProperty()

// act, raise a charge

await raiseCharge()

// assert, confirm charge has been raised

expect(charge).ToBe('raised')

Linter

Install and use linting rules. We use eslint-playwright-plugin

The recommended configuration will help enforce some of the guidelines described in this blog post.

npm install -D eslint-plugin-playwright

Avoid Conditionals

Avoid having conditional logic inside the test files. Tests should be deterministic, meaning that we should aim to have tests which will return pre-determined results. Tests with conditionals can get hard to maintain and read. Additionally, these kinds of tests can be flaky unless you are absolutely certain the state of the application has stabilised at the point of making assertions.

Having conditionals can be a sign a test is doing too much, and can be broken down.

DO NOT ⚔️

import { FooPage } from './pom/foo';

let fooPage: FooPage;

describe('Conditional test', () => {
  beforeEach(({ page }) => {
    fooPage = new FooPage(page);
  });

  test('bar is visible', async () => {
    const isButtonVisible = await fooPage.buttonFoo.isVisible();

    if (isButtonVisible) {
      // If buttonFoo is visible, click it to make pageTitle visible
      await fooPage.buttonFoo.click();
      // Then check if pageTitle is visible
      await expect(fooPage.pageTitle).toBeVisible();
    } else {
	  //Else just check if pageTitle is visible
    await expect(fooPage.pageTitle).toBeVisible();
}

  });
});

DO 👍

// Have separate specs for each scenario. 
// With setup steps needed to get to that state

// fooPageVisible.spec.js

  test('bar is visible initially', async () => {
  // setup your test in the state where bar is visible initially

  // assert bar is visible
    await expect(fooPage.pageTitle).toBeVisible();
  });

// fooBarButton.spec.js

import { FooPage } from './pom/foo';

let fooPage: FooPage;

test('bar is visible after clicking Foo button', async () => {
  // setup your test in the state where button is visible initially

  //perform button action
  await fooPage.buttonFoo.click();

  // assert bar is visible
  await expect(fooPage.pageTitle).toBeVisible();
  });

Waiting

Don’t use any arbitrary waits. This can cause flaky tests as you can rarely be certain the amount of wait time is enough. It can also unnecessarily increase the test run time . Instead try:

Use Playwright’s waitUntil: 'domcontentloaded'
Wait for specific network requests to resolve
Wait for the page state to settle, for example elements to be visible / not visible on the page

DO NOT ⚔️

await page.waitForTimeout(5000)

DO 👍

// define in a wait helpers file
// wait-helpers.ts

export const waitForAPIResponse = async (
  page: Page,
  url: string,
  statusCode: number,
): Promise<void> => {
  await page.waitForResponse((res) => res.url().includes(url) 
  && res.status() === statusCode);
};

// use in your test file
//tests/foo.spec.ts

import { Locator, Page } from '@playwright/test';
import { waitForAPIResponse } from '../../helpers/wait-helpers';


describe('displays foo bar', () => {
	test('bar is visible', async () => {
		await fooPage.buttonFoo.click();
await waitForNewAPIResponse(this.page, '/Accounting/GetRaisedCharges', 200);
await expect(fooPage.titlePage).toBeVisible();
	});
});

DO 👍

//tests/foo.spec.ts

import { Locator, Page } from '@playwright/test';
import { waitForAPIResponse } from '../../helpers/wait-helpers';


describe('displays foo bar', () => {
	test('bar is visible', async () => {
await page.goto(fooBarURL, {
    waitUntil: 'domcontentloaded'
});

	});
});

DO 👍

//tests/foo.spec.ts

import { Locator, Page } from '@playwright/test';
import { waitForAPIResponse } from '../../helpers/wait-helpers';


describe('navigate to foo bar', () => {
	test('page is loaded', async () => {
		await fooPage.buttonFoo.click();
		await expect(fooPage.titlePage).toBeVisible();
	});
});

Selectors

Avoid selectors tied to implementation and page structure.

Instead, we prioritise the below, based on testing-library guiding principles

getByRole (this aids accessibility, reflects how users and assistive technology perceive the page)
getByText
getByTestId (add them when needed)

DO NOT ⚔️

page.locator('.opt-u > div > .summary > div:nth-child(4) > div')

DO 👍

page.locator('#foo-button');

page.getByText('OK');

Tagging

Utilise tagging in Playwright to group tests together and have targeted runs. Some ways to tag tests include:

By test type (ex. Functional, visual,...)
By where the tests run in the pipeline (release, regression,...)
By feature (ex. Diaries, login, …)

You can read a bit more about how we use tagging in our blog post about tag annotations.

Do - By Test Type 👍

describe('displays foo bar', () => {
	// set out steps to run before each test type
	beforeEach(async ({ page }) => {
		await page.goto('/foo/bar');
		fooPage.buttonFoo.click();
		await fooPage.titlePage.waitFor();
	});

	// check for accessibility
	test('@accessibility', async ({ page }) => {
		await injectAxe(page);
		await checkA11y(page, undefined, a11yOpts);
	});

	// run assertions on the page
	test('@functional @smoke', async ({ page }) => {
		await expect(fooPage.buttonFoo).toBeVisible();
		await expect(fooPage.titlePage).toBeVisible();
	});

	// visual snapshot for desktop
	test('@visual desktop', async ({ page, captureScreenshot }) => {
		await captureScreenshot('foo-desktop.png');
	});

	// visual snapshot for mobile
	testMobile('@visual mobile', async ({ captureScreenshot }) => {
		await captureScreenshot('foo-mobile.png');
	});
});

Do - By Page / Feature / Where test is run 👍

//tests/foo.spec.ts

describe('@foobar @smoke navigate to foo bar', () => {
	test('page is loaded', async () => {
		await fooPage.buttonFoo.click();
		await expect(fooPage.titlePage).toBeVisible();
	});
});

Flaky tests

Flaky tests should be resolved as a priority. If you can’t resolve it at the time, mark them with .fixme. This will skip the test.

//tests/foo.spec.ts

describe('displays foo bar', () => {
	test.fixme('bar is visible', async () => {
		await fooPage.buttonFoo.click();
		await expect(fooPage.titlePage).toBeVisible();
	});
});

Parallelization and Repeatability

Build tests to run repeatedly without intervention. And run in parallel with other tests in the suite, i.e. without interfering with other tests.

Playwright runs tests in parallel out the box, through spinning up multiple worker processes that all run at the same time. Playwright can scale up the number of worker processes based on the resources available. By using large github runners, we can run more tests in parallel in our CI pipeline.

Naming Conventions

Variables

Declare in camelCase.

Booleans

Start with ‘is’, ‘has’, ‘are’, ‘have’. This helps spot that this is a boolean while skimming the code. Still declared in camelCase.

let isTurnedOn = false

Page Objects / Classes

Declare in PascalCase.

Use descriptive naming, which can help the reader quickly identify what page or page component this is covering. Use as much context as needed from your product to make the name meaningful.

DO 👍

export class AddWorksOrderModal

DO NOT ⚔️

export class newModal

Locators

Use descriptive naming, which can help the reader quickly identify what element the locator is targetting.

As an example, you can use a naming structure that contains “action / name of element” + “type of element”.

Defining type of element - These are your basic HTML element types, they’ll be defined and named in the design system, or as a team you can align on a consistent naming of the elements. Example: checkbox, tickbox, button, tooltip

Defining action / name Think about what action this element will perform when interacted with. Or any existing name/text of the element

DO 👍

//This element is a save button which sits within the context of properties

readonly savePropertyButton: Locator;

DO 👍

//This is a field for a reported date 
readonly reportedDateField: Locator;

Function names

Always start function names with a “verb”, followed by the “component context” that the function is interacting with i.e. what entity it is having an effect on.

DO 👍

getWorksOrder()

printTransactions()

deleteProperty()

Houseful Product & Technology Blog

Our Playwright testing standards at Houseful

Playwright Guidelines

Data Creation in e2e tests

Page Object Model (POM)

Test Structure - Arrange, Act, Assert

Linter

Avoid Conditionals

Waiting

Selectors

Tagging

Flaky tests

Parallelization and Repeatability

Naming Conventions

Variables

Booleans

Page Objects / Classes

Locators

Function names

Topics

About us

Careers

Get in touch