Multistage Builds Allure 3

The Multistage Builds feature added in Allure Report 3 allows you to create unified reports from multiple sources of test run data. Instead of generating individual reports for every stage of a complex testing pipeline with multiple test runs and/or multiple test environments, you can save the full state of each stage as a stage archive (a .zip file with all test run data), and then build a single report from the resulting collection of archives.

Typical use cases

Building unified reports from:

• test runs in different environments and contexts
• multi-stage test pipelines, where each stage requires a separate test run
• CI/CD setups involving some or all of the above

What is a stage archive

A stage archive stores all test results generated by the Allure integration - everything you would find in a regular results folder - in an optimized format.

In addition to that, it captures the full state of the run, which includes important data produced by Allure Report 3 itself - data that does not normally go into the results folder.

For this reason, manually collecting result files from multiple test runs and building a single report from them wouldn't give you the same level of detail as a report built from a collection of stage archives will.

Here is what stage archives save and results folders don't:

• Test instability marks: if a history file is available during the test run, Allure will identify new, fixed, regressed, malfunctioned and flaky tests and store this information in the stage archive.

• Quality gates: if the test run failed any quality gate rules you defined for your project, the details are saved in the archive.

• Known issues: if a known issues file is configured during the test run, the archive will store that information.

• Environment management: if you configured environments, be it via the configuration file, or via the allure run --environment ... command, this information is also saved.

• Global errors and global attachments: if anything went wrong at the system level during the test run, the terminal output and error log are saved to the archive.

Any reports built from such stage archives include all this data.

Building multistage reports

To create a stage archive, use the allure run wrapper command, when running your tests.

1. Save the full state of your run into the stage1.zip file like so:

allure run --stage=stage_1 -- <test command>

For example, if npm is your test runner:

allure run --stage=stage_1 -- npm test

2. Generate a unified report from all stage archives matching the stage_*.zip pattern:

allure generate --stage=stage_*.zip

3. Open the resulting report with:

allure open

Environments in multistage reports

Environments are fully integrated with stage archives, and are the main intended use case for this feature: you can run your tests in entirely different environments, save full test run states and create unified reports that utilize all new advanced features of Allure Report 3.

Please refer to the Environments documentation for a detailed example of how to use these features together.