Getting started with Allure Playwright

Generate beautiful HTML reports using Allure Report and your Playwright tests.

INFO

Check out the example projects at github.com/allure-examples to see Allure Playwright in action.

Setting up

1. Prepare your project

1. Make sure Node.js is installed.

   Allure Playwright is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that.

2. Open a terminal and go to the project directory. For example:

   cd /home/user/myproject

3. Make sure Allure Report is installed. If it's not, follow the installation instructions. Note that Allure Report requires Java.

4. Install the Allure Playwright adapter and make sure that the Playwright framework itself is also listed in the project's dependencies.

   npm

   npm install --save-dev @playwright/test allure-playwright

   yarn

   yarn add --dev @playwright/test allure-playwright allure-js-commons

   pnpm

   pnpm install --dev @playwright/test allure-playwright

5. In the playwright.config.ts file, add Allure Playwright as a reporter.

   With default options

   export default defineConfig({
     // ...
     reporter: [["line"], ["allure-playwright"]],
   });

   With custom options

   export default defineConfig({
     // ...
     reporter: [
       ["line"],
       [
         "allure-playwright",
         {
           resultsDir: "allure-results",
         },
       ],
     ],
   });

   Some additional options can also be defined here, see Configuration.

2. Run tests

Run your Playwright tests the same way as you would run them usually. For example:

npm

npx playwright test

yarn

yarn run playwright test

pnpm

pnpx playwright test

This will save necessary data into allure-results or other directory, according to the configuration. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all.

3. Generate a report

Finally, run Allure to convert the test results into an HTML report. This will automatically open your browser to view the report.

allure serve allure-results

If necessary, replace allure-results with the path to the directory specified in the configuration.

There are some options that can affect how the report is generated. Run allure --help for the full list of options.

Writing tests

The Allure Playwright adapter extends the standard reporting features of Playwright by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized:

• Metadata Annotation: Enhance test reports with descriptions, links, and other metadata.
• Test Organization: Structure your tests into clear hierarchies for better readability and organization organize tests.
• Step Division: Break down tests into smaller test steps for easier understanding and maintenance.
• Parametrized Tests: Clearly describe the parameters for parametrized tests to specify different scenarios.
• Set labels globally: Use environment variables to set metadata and other labels.
• Attachments: Automatically capture screenshots and other files during test execution.
• Test Selection: Use a test plan file to select which tests to run, allowing for flexible test execution.
• Environment Details: Include comprehensive environment information to accompany the test report.

In most cases, Allure Playwright provides two different ways to use a feature: the Runtime API and the Metadata API.

• Runtime API: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically.

  Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early.

• Metadata API: add a metadata tag (beginning with @) into the test name. Allure Playwright will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs.

Add Metadata

Allure allows you to enrich your reports with a variety of metadata. This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the metadata reference section for an exhaustive list of what can be added.

Runtime API

import { test } from "@playwright/test";
import * as allure from "allure-js-commons";

test("Test Authentication", async () => {
  await allure.displayName("Test Authentication");
  await allure.owner("John Doe");
  await allure.tags("Web interface", "Authentication");
  await allure.severity("critical");
  // ...
});

Metadata API

import { test } from "@playwright/test";

test(
  "Test Authentication" +
    " @allure.label.owner:JohnDoe" +
    " @allure.label.tag:WebInterface" +
    " @allure.label.tag:Authentication" +
    " @allure.label.severity:critical",
  async () => {
    // ...
  },
);

Organize tests

As described in Improving navigation in your test report, Allure supports multiple ways to organize tests into hierarchical structures.

To specify a test's location in the behavior-based hierarchy:

Runtime API

import { test } from "@playwright/test";
import * as allure from "allure-js-commons";

test("Test Authentication", async () => {
  await allure.epic("Web interface");
  await allure.feature("Essential features");
  await allure.story("Authentication");
  // ...
});

Metadata API

import { test } from "@playwright/test";

test(
  "Test Authentication" +
    " @allure.label.epic:WebInterface" +
    " @allure.label.feature:EssentialFeatures" +
    " @allure.label.story:Authentication",
  async () => {
    // ...
  },
);

To specify a test's location in the suite-based hierarchy:

Runtime API

import { test } from "@playwright/test";
import * as allure from "allure-js-commons";

test("Test Authentication", async () => {
  await allure.parentSuite("Tests for web interface");
  await allure.suite("Tests for essential features");
  await allure.subSuite("Tests for authentication");
  // ...
});

Metadata API

import { test } from "@playwright/test";

test(
  "Test Authentication" +
    " @allure.label.parentSuite:TestsForWebInterface" +
    " @allure.label.suite:TestsForEssentialFeatures" +
    " @allure.label.subSuite:TestsForAuthentication",
  async () => {
    // ...
  },
);

Divide a test into steps

To create steps and sub-steps, you can use the step() function, see the reference.

Playwright's test.step() is also supported.

import { test } from "@playwright/test";
import * as allure from "allure-js-commons";
import { Status } from "allure-js-commons";

test("Test Authentication", async () => {
  await allure.step("Step 1", async () => {
    await allure.step("Sub-step 1", async (ctx) => {
      await ctx.parameter("foo", "1");
      // ...
    });
    await allure.step("Sub-step 2", async (ctx) => {
      await ctx.parameter("foo", "2");
      // ...
    });
  });
  await allure.logStep("Step 2", Status.SKIPPED);
});

Note that by default Allure displays automatically generated steps in the list, too, including those defined using Playwright's beforeAll(), beforeEach(), afterEach(), and afterAll() functions. This can be disabled using the detail setting.

Describe parametrized tests

A typical way to implement the parametrized tests pattern in Playwright is to define the test in a loop and use the loop variable in the test's title and body.

To display a parameter value in the test report, pass it to the parameter() function.

import { test } from "@playwright/test";
import * as allure from "allure-js-commons";

for (const login of ["johndoe", "[email protected]"]) {
  test(`Test Authentication as ${login}`, async () => {
    await allure.parameter("login", login);
    await allure.parameter("time", new Date().toUTCString(), { excluded: true });
    // ...
  });
}

test("Test Authentication With Empty Login", async ({ page }) => {
  await allure.parameter("login", "");
  await allure.parameter("auth_method", "password");
  // ...
});

Set labels globally

Any labels, including custom ones, can be set via the environment variables in your operating system. Here's an example (assuming you use the npm package manager):

MacOS/Linux

export ALLURE_LABEL_epic=WebInterface
npx playwright test

Windows

$Env:ALLURE_LABEL_epic = "WebInterface"
npx playwright test

Attach screenshots and other files

Reports generated by Allure can include any files attached to the test using allure.attachment() method. (Playwright's built-in TestInfo.attach() method is supported as well.)

For example, a popular way to make a report easier to understand is to attach a screenshot of the opened web page. See Attachments.

import { test } from "@playwright/test";
import * as allure from "allure-js-commons";
import { ContentType } from "allure-js-commons";

test("Test Authentication", async () => {
  // ...

  await allure.attachment("Text file", "This is the file content.", ContentType.TEXT);

  await allure.attachmentPath("Screenshot", "/path/to/image.png", {
    contentType: ContentType.PNG,
    fileExtension: "png",
  });
});

Select tests via a test plan file

If the ALLURE_TESTPLAN_PATH environment variable is defined and points to an existing file, Playwright will only run tests listed in this file.

Here's an example of running tests according to a file named testplan.json (assuming you use the npm package manager):

MacOS/Linux

export ALLURE_TESTPLAN_PATH=testplan.json
npx playwright test

Windows

$Env:ALLURE_TESTPLAN_PATH = "testplan.json"
npx playwright test

Environment information

For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, specify the information in the environmentInfo configuration parameter.

For example, it is a good idea to use this to remember the OS version and Node.js version retrieved from the os and process objects. This may help the future reader investigate bugs that are reproducible only in some environments.

import type { PlaywrightTestConfig } from "@playwright/test";
import * as os from "node:os";

const config: PlaywrightTestConfig = {
  reporter: [
    ["line"],
    [
      "allure-playwright",
      {
        environmentInfo: {
          os_platform: os.platform(),
          os_release: os.release(),
          os_version: os.version(),
          node_version: process.version,
        },
      },
    ],
  ],
};

export default config;

Note that if your launch includes multiple Playwright runs (see How it works), Allure Playwright will only save the environment information from the latest run.