Test categories

Tests can be divided into categories based on their status and output messages. Each test result belongs to no more than one category. It can be one of the default categories or a custom category.

On the Categories tab in a test report, the tests are grouped by their category and then by their error message. The tests that do not belong to any category are not displayed on the tab.

Allure Report Categories tab

Allure also uses the information about test categories to build the Categories trend graph and display the Categories block on the Overview tab.

Default categories

Allure defines two default test categories, both of them including tests that did not finish successfully:

  • Product defects (corresponds to the Failed status) — tests that stopped because of an assertion error.
  • Test defects (corresponds to the Broken status) — tests that stopped because of any uncaught exception other than an assertion error.

Note that if a test result matches a custom category, it is considered neither a product defect nor a test defect.

Custom categories

You can define any number of custom categories by placing a categories file into the test results directory (see How it works). Allure checks each test against all the categories in the file, from top to bottom. The test gets assigned the first matching category. When no matches are found, Allure uses one of the default categories if the test is unsuccessful or no category otherwise.

Some Allure plugins can generate the file automatically based on the project configuration. Check your adapter's documentation.

Powered by
logo

Join our newsletter

Join our community

We aim to make Allure Report as reliable and user-friendly as possible, and together with the community, we're here to help when problems arise.

© 2024 Qameta Software Inc. All rights reserved.