Improving navigation in your test report

A big enough project can have hundreds or even thousands of tests. When collecting information about tests, Allure can organize the results into easy-to-grasp tree structures. With such a structure, a reader of the test report can easily navigate to the test results they need. The tree is generated on the Suites, Behaviors, or Packages tab of the report, depending on the data provided by test author.

This article describes hierarchies that are built based on the information about the tests themselves. For a hierarchy based on the test results, see Categories.

Tests hierarchies

Allure supports different types of hierarchies. You can choose the one that better aligns with the development workflow in your project. For each one, Allure expects test authors to assign specific fields to the tests. The exact functions that should be used for assigning the fields vary between different Allure adapters.

Behavior-based hierarchy

Epics, features and user stories are widely-used terms for describing a software's features and organizing tests for said features. Usually, an epic is a large set of features that the team aims to develop and test, and a feature is described as a set of user stories that describe how the software is expected to behave in different scenarios.

Allure adapters for all test frameworks provide ways to indicate a test's epic, feature and user story. If a test framework lets the author define some of these terms natively, e.g., in its own language-agnostic file format, then Allure tries its best to incorporate the provided data into the generated tree structure. See the specific Allure adapter's documentation for more details.

Allure Report tab behaviors

Suite-based hierarchy

A test suite is an abstract term for any group of tests, with its actual meaning depending on the specifics of your project.

Allure adapters for all test frameworks provide ways to indicate a test's suite, with most of them also providing ways to indicate parent suites and sub-suites, for the total of up to three levels of hierarchy. Note, however, that the implementation may differ in how the default values are chosen for these fields based on the data provided by the test frameworks itself. See the specific Allure adapter's documentation for more details.

Allure Report tab suites

Package-based hierarchy

In some projects, it is convenient to organize tests by packages, closely following the structure of the code. To place a test in the tree on the Packages tab, Allure normally uses the actual names of the module and the function in which the test is implemented.

Some Allure adapters allow overriding this behavior. However, it is recommended to always follow the hierarchical naming convention for packages, with subpackage names placed after package names, separated by dots.

In the tree, packages' common prefixes are shown as their parent packages.

Allure Report tab packages

Annotations API vs. Runtime API

Some adapters provide more than one way to assign fields. The two examples below are both written in Java using Allure JUnit 5, but demonstrate two different approaches to specifying a test's fields for the behavior-based hierarchy. The first example uses annotations, while the second one calls methods inside the function's body. Both examples produce the same hierarchy on the Behaviors tab.

Note that when using the methods like in the second example (or its analogs for other languages), it is recommended to call the functions as close to the beginning of the test body as possible. This way, the test result will be placed in the hierarchy correctly even if the test fails early.

Java
import io.qameta.allure.Epic; import io.qameta.allure.Feature; import io.qameta.allure.Story; import org.junit.jupiter.api.Test; import java.io.IOException; public class TestMyWebsite { @Test @Epic("Web interface") @Feature("Essential features") @Story("Authentication") public void testAuthentication() throws IOException { // ... } }
Java
import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; import java.io.IOException; public class TestMyWebsite { @Test public void testAuthentication() throws IOException { Allure.epic("Web interface"); Allure.feature("Essential features"); Allure.story("Authentication"); // ... } }

Omitting tree levels

In all the hierarchies described above, none of the fields are necessary.

To determine a test's location in the Suites or Behaviors hierarchy, Allure checks for the relevant fields in the predefined order:

For each provided field, Allure creates a subtree in the test results tree. If the field was not provided, the subtree creation is skipped.

In the example below (written in Java using Allure JUnit 5), the second test only has the feature field. In the report, its feature is displayed at the top level, essentially becoming this test's epic.

Java
import io.qameta.allure.Epic; import io.qameta.allure.Feature; import io.qameta.allure.Story; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; public class AllureNavigationTest { @Test @Epic("Web interface") @Feature("Essential features") @Story("Authentication") @DisplayName("Log in with username and password") public void testLogInWithUsernameAndPassword() { // ... } @Test @Feature("Mobile API") @DisplayName("API: Read user profile") public void testReadUserProfile() { // ... } }

Allure Report suites tab nesting

Powered by
logo

Join our newsletter

Join our community

We aim to make Allure Report as reliable and user-friendly as possible, and together with the community, we're here to help when problems arise.

© 2024 Qameta Software Inc. All rights reserved.