Playwright parameterization

Parameterization is a technique that allows the execution of the same test with various inputs. This guide addresses the problems that parameterization can solve, how to implement parameterization in Playwright, and its integration with Allure Report.

1. Preparation

Prerequisites

Make sure you have:

• Allure Report installed
• NodeJS installed

Dependency List

This guide uses the following packages:

• @playwright/test: 1.51.1
• allure-playwright: 3.2.0
• allure-js-commons: 3.2.0

Code sample

The source code used in this guide is available at https://github.com/allure-examples/guide-playwright-parameterization.

Setup

To run the examples in this guide:

1. Install Node.js

2. Install Allure Report

3. Download a fresh project with Playwright from Allure Start

4. Navigate to the project directory and install the dependencies using a package manager of your choice:

   npm

   npm install

   yarn

   yarn install

   pnpm

   pnpm install

5. Run the project:

   npm

   npm test

   yarn

   yarn test

   pnpm

   pnpm test

6. Open Alure Report:

   allure serve

2. Benefits of parameterization

Let’s start with a simple function. We want to check if adding two numbers gives the right answer:

JavaScript

import { expect, test } from "playwright/test";

const sum = (a, b) => {
  return a + b;
};

test("basic sum", () => {
  expect(sum(1, 2)).toBe(3);
});

TypeScript

import { expect, test } from "playwright/test";

const sum = (a: number, b: number) => {
  return a + b;
};

test("basic sum", () => {
  expect(sum(1, 2)).toBe(3);
});

Next, we would like to test this function with different data sets.

The first idea might be to use a loop in the test and check each result inside the loop. It could look like this:

JavaScript

import { expect, test } from "playwright/test";

test("loop sum", () => {
  [
    [1, 2, 3],
    [1, -1, 0],
    [0, 0, 0],
  ].forEach((testData) => {
    await expect(sum(testData[0], testData[1])).toBe(testData[2]);
  });
});

TypeScript

import { expect, test } from "playwright/test";

test("loop sum", () => {
  const testData: [number, number, number][] = [
    [1, 2, 3],
    [1, -1, 0],
    [0, 0, 0],
  ];

  for (const [a, b, expected] of testData) {
    expect(sum(a, b)).toBe(expected);
  }
});

This method has several problems:

1. If the test fails for one data set, it won’t check the rest.
2. The data is stored in an array, and there are no clear variable names, making it harder to understand. This isn’t very obvious here, but in a more organized test, it would be more noticeable.
3. The test report only shows the test name, not which specific data set caused the issue. There’s no way to make this clearer in this setup.

Playwright has a better solution: parameterization. We can use a forEach loop, like before, but this time we put the test inside the loop and the input inside the test function title:

JavaScript

import { expect, test } from "playwright/test";

[
  { x: 1, y: 2, expectedSum: 3 },
  { x: 1, y: -1, expectedSum: 0 },
  { x: 0, y: 0, expectedSum: 0 },
].forEach(({ x, y, expectedSum }) => {
  test("the sum of ${x} and ${y} should be ${expectedSum}", () => {
    expect(sum(x, y)).toBe(expectedSum);
  });
});

TypeScript

import { expect, test } from "playwright/test";

const testCases: { x: number; y: number; expectedSum: number }[] = [
  { x: 1, y: 2, expectedSum: 3 },
  { x: 1, y: -1, expectedSum: 0 },
  { x: 0, y: 0, expectedSum: 0 },
];

testCases.forEach(({ x, y, expectedSum }) => {
  test("the sum of ${x} and ${y} should be ${expectedSum}", () => {
    expect(sum(x, y)).toBe(expectedSum);
  });
});

This method fixes the issues we discussed earlier:

1. The test runs multiple times, and even if one fails, the others will still give the feedback.
2. The data is now accessed using clear variable names (x, y, and expectedSum), which makes it easier to understand.
3. In a test report, each test run is shown as a separate entry, making it clearer.

3. Parameterization in Allure Report

Allure Report integrates with Playwright, and supports parameterized automated tests. By default, the parameterized tests appear in Allure Report as separate tests without the distinction for the parameterized inputs.

To make the parameterized inputs appear in Allure Report, you should use the allure.parameter() function. It creates a new parameter with a given name and value and forces the new parameter to be displayed in the report, but it does not change or hide the existing parameters. For example:

JavaScript

import * as allure from "allure-js-commons";
import { expect, test } from "playwright/test";

[
  { x: 1, y: 2, expectedSum: 3 },
  { x: 1, y: -1, expectedSum: 0 },
  { x: 0, y: 0, expectedSum: 0 },
].forEach(({ x, y, expectedSum }) => {
  test("the sum of ${x} and ${y} should be ${expectedSum}", async () => {
    await allure.parameter("x", x);
    await allure.parameter("y", y);
    await allure.parameter("expectedSum", expectedSum);

    expect(sum(x, y)).toBe(expectedSum);
  });
});

TypeScript

import * as allure from "allure-js-commons";
import { expect, test } from "playwright/test";

const testCases: { x: number; y: number; expectedSum: number }[] = [
  { x: 1, y: 2, expectedSum: 3 },
  { x: 1, y: -1, expectedSum: 0 },
  { x: 0, y: 0, expectedSum: 0 },
];

testCases.forEach(({ x, y, expectedSum }) => {
  test("the sum of ${x} and ${y} should be ${expectedSum}", async () => {
    await allure.parameter("x", x.toString());
    await allure.parameter("y", y.toString());
    await allure.parameter("expectedSum", expectedSum.toString());

    expect(sum(x, y)).toBe(expectedSum);
  });
});

With that, Allure Report starts to show the parameters in the test results:

The allure.parameter() function has an additional options argument. If given, the argument must be an object with the following properties:

1. excluded (Boolean, by default, false) defines how Allure Report treats the parameter if it changes the value across test runs. For example, if you pass a current timestamp as a parameter value, it will differ for each test retry. By default, Allure Report treats the tests with such parameters as different ones and shows each rerun separately. To avoid this, you can set the excluded value to true. Read more in History and retries.

2. mode (string) defines how the parameter appears in Allure Report. Possible values:

   • "default": the parameter name and value appear in Allure Report as they are
   • "masked": the parameter name appears in Allure Report, but the value is hidden. You can use this mode for passwords, tokens, and other sensitive parameters.
   • "hidden": the parameter name and value don't appear in Allure Report. A common use case is to hide a parameter whose only purpose is to separate test results obtained from different test environments. You can learn more here.

   Note that the parameter with { mode: "masked"} or { mode:"hidden" } can still be extracted from the allure_results directory.

An example of the allure.parameter() function usage with the options object:

import { test } from "@playwright/test";
import * as allure from "allure-js-commons";

for (const login of ["johndoe", "[email protected]"]) {
  test(`Test Authentication as ${login}`, async () => {
    await allure.parameter("login", login, { mode: "masked" });
    await allure.parameter("time", new Date().toUTCString(), { excluded: true });
    // ...
  });
}

4. Conclusion

Allure Report works natively with parameterization in Playwright and doesn't need extra setup or code. This testing approach allows you to run the same test with different data sets. This helps cut down on repetition and makes things simpler by not needing to manually loop through data.

To make the test report clearer, you can also add parameters for your tests and control their appearance in Allure Report.