Test steps

Test steps are a mechanism for structured logging, supported by most Allure adapters.

A step begins and ends around a block of code, typically a function or a lambda function, with the step name describing what the code is trying to do: “Initializing test data,” “Connecting to the server,” etc. Unlike a simple text log entry that can only show you a single event, a step allows you to view the execution results of the entire code block. A step may also contain nested steps forming a tree-like structure.

In the test report, Allure displays steps in chronological order, preserving the nesting structure. For each step, it shows its separate duration, status, and, optionally, parameters and attached files.

Why use test steps?

Using steps in your test serves two purposes: to simplify test debugging and to improve test documentation.

Simplify debugging of your tests

Just like any other form of logging, steps help you see which sections of the test code were executed and which were not. But unlike a traditional log entry, each step is an interval, not an instant. That fact helps you find the code block that has started but hasn't finished successfully, judging by the steps' statuses. This is also useful when looking for code blocks that take too long to run, thanks to the duration being shown next to each step.

Once you've identified a troublesome code block, Allure will help you further investigate what happened by providing a comprehensive error message with the stack trace.

The report can be improved even further by adding attachment to the steps. One common practice is to log the state of a graphical user interface by attaching a screenshot on each step. For complex test scenarios, this will help understand where exactly something went wrong, without resorting to manual testing.

For most languages and test frameworks, Allure provides a way to set special annotations (attributes, decorators) to your test functions, thus automatically turning each call of the functions into a step or sub-step with minimal effort.

Document what your tests do

A list of well-described steps serves as a high-level description of a test. It can help your colleagues or manager understand what it does without the need to read the code.

To achieve this, we recommend to make sure that the steps cover the whole test, i.e. every line of code belongs to one step or another. If you use a BDD-oriented test framework like Cucumber or Robot Framework, then your code already follows this rule, and Allure will use the framework's steps automatically. Otherwise, split your test code into high-level steps yourself using functions or lambda functions. Optionally, mark some important events during the test using no-op steps.

In complex tests, high-level steps are often useful as a documentation, while lower-level steps provide information for debugging.

Steps created automatically

Whenever possible, Allure Report takes advantage of the concept of a step provided by the test framework itself.

For example, this is the case with Cucumber-JVM, Behave, SpecFlow and other BDD-oriented frameworks that use the Gherkin file format. The Allure adapter makes its best to reflect the structure of a Gherkin file the same way a native reporter would, while also allowing the test author to define arbitrary sub-steps in each step's code.

In addition to that, there is a variety of integration packages that provide automatic step creation on certain events, see the Modules page. In the example below, the sub-steps are created by the Allure integration package for the AssertJ library. Other integration packages exist for HTTP request libraries, SQL libraries, etc.

Steps created by your code

Depending on your programming language and test framework, Allure can provide one or more ways to define test steps and sub-steps. See the specific Allure adapter's documentation for more details.

In general, the supported styles include:

• steps as reusable functions,
• steps as lambda functions,
• steps as no-op calls.

If your Allure adapter supports multiple styles, you can use them interchangeably in the same test. For example, you may split your test into large functions, each one further split into lambda sub-steps, each one reporting multiple no-op steps.

Steps as reusable functions

This approach is also known as “decorated steps”, “annotated steps” or “attribute-based steps”. It involves declaring a function or a method that implements a step, marking it in some language-specific way (e.g., adding a special annotation in Java), and then calling the function one or multiple times during the execution of the test.

Using functions helps to avoid name conflicts between variables used in different steps. Also, this allows different tests to call the same step at some point, e.g., two tests can share a preparation routine.

If the function accepts arguments, some Allure adapters automatically interpret them as step parameters.

Java
@Step("Open the website")
void openWebsite(String url) {
    // ... browser navigation logic ...
}

Steps as lambda functions

When the reusability of step implementations is not a priority, it may be more convenient to define steps in place as lambda functions. When the individual steps are small, this approach often leads to a short and self-explanatory code of the test.

Java
Allure.step("Open the website", () -> {
    // ... browser navigation logic ...
});

Steps as no-op calls

Sometimes it can be important to report an instant event, similar to how a text log would include a text message. Most Allure adapters provide a way to do it in the form of a “no-op step”. Such a step does not contain executable code, and it never has sub-steps.

Note that when your test is expected to produce a huge amount of messages, it may be better to add them to the test result as a text attachment instead.

Java
// ... browser navigation logic ...
Allure.step("Website is opened");

Fixtures

A fixture, also known as a hook, is a piece of code that runs before or after a test or a group of tests. The code that runs before tests is often referred to as a “setup function”, and the code that runs after tests as a “teardown function”.

For the test frameworks with fixtures support, Allure considers each fixture a separate step and displays it in a special block above or below the list of test's own steps. Just like a normal step, a fixture can be split into sub-steps.

In some Allure adapters, including fixtures in the test report is optional and can be disabled in the configuration. Disabling fixtures can help keep the report clean and readable, but keep in mind that having all fixtures listed can be useful, e.g., for detecting timing issues.

Step statuses

Similarly to test statuses, each test step has its own status. The status of a step does not have to be the same as the status of the whole test — e.g., you can have a failed step within a passed test.

• If a step is implemented as a function or a lambda function, it usually gets assigned the Passed status if the code runs normally, or the Failed or Broken status if it is throws an exception. The exact behavior may vary for different Allure adapters.

  If you need to keep the test running after a step threw an exception, catch it outside the step, as shown in the example below.

• If a step is a no-op step, it gets assigned the Passed status by default.

  Some adapters provide a way to pass another status as an argument in the Runtime API, see the reference for your adapter. We recommend to use the same approximate meanings of statuses for consistency: for example, use Passed and Failed for reporting successful and unsuccessful actions, respectively.

The example below demonstrates how test statuses can help the reader understand the test results. Here, the test made three attempts to load some data from the internet. The step statuses indicate that the first two attempts failed, but the code caught the exceptions and proceeded to the third attempt which was successful.

Java
Allure.step("Load data from remote server", () -> {
    for (int attempt = 1; attempt <= 3; attempt++) {
        try {
            Allure.step("Attempt #%d".formatted(attempt), () -> {
                // ... data loading logic ...
            });
            break;

        } catch (Exception e) {
            if (attempt == 3) {
                throw e;
            }
        }
    }
});

Allure.step("Validate the response headers", () -> {
    // ... assertions ...
});

Allure.step("Validate the data", () -> {
    // ... assertions ...
});

Step durations

Allure Report always calculates the difference between the start time and the end time of a step. The result is displayed next to the step name in the test report.

In the example below, you can immediately see that the first two steps passed very fast, while the third one failed after a suspiciously round time interval (almost exactly three seconds). This is often because some operation gets interrupted by a timeout in either the test code or the product code.

Exceptions in steps

When you write steps as reusable functions or lambda functions, Allure takes care of logging any exception thrown by the functions. The exception's message and stack trace are attached to the step, and the step gets the Failed or Broken status depending on the exception type.

In the example below, the test make multiple attempts to run some code, each one wrapped in Allure.step() and then in a try block. The block here does not process assertion errors (the catch block is empty) and only serves for keeping the test running after them. However, the implementation of Allure.step() itself makes sure that the exception message will be displayed if you click on a failed step. (The full stack trace will be displayed if you click on the message.)

Java
String[] resolutions = {"1920x1080", "1280x720", "1024x768", "800x600"};
for (String resolution : resolutions) {
    try {
        Allure.step("Rendering at " + resolution, () -> {
            // ... rendering logic and assertions ...
        });
    } catch (AssertionError ignored) {
    }
}

Step parameters

Step parameters are a way to describe the data that the step worked with.

This is not to be confused with the test parameters that characterize the whole test.

The API for setting step parameters varies for different Allure adapters and different styles of step declaration. For example:

• For a reusable step function, some Allure adapters automatically display each argument as a step parameter.
• For a lambda step, some Allure adapters provide a special object that can be used to manipulate the list of parameters from within the step.
• For both styles, it is usually possible to specify a list of step parameters manually by calling a special Runtime API method.

For more details, see the reference for your adapter.

In the example below, the test uses parameters to describe the user profiles it creates. The set of parameters is different in each step.

Java
Allure.step("Create user Alice", step -> {
    step.parameter("Name", "Alice");
    step.parameter("Login", "alice");
    step.parameter("Email", "[email protected]");
    // ... user creation logic ...
});

// ... other steps ...

Step attachments

Allure Report supports attaching arbitrary files to a step. This can be used to describe either the input data the step processed (similar to step parameters) or the output data it received from somewhere.

For example:

• In a UI application test, you may want to save screenshots of the application after each action.
• In a web application test, you may want to save all HTTP requests and responses the test performs.
• In any kind of test, you may want to save logs produced by the system under test.

Attaching files to steps can be a way of telling the reader a chronological story, in which certain screenshots or other files illustrate certain moments of the test execution.

Some popular file types will be rendered on the test report page, others will be available for download if the reader clicks a link. See Attachments for more information.

Java
Allure.step("Visit the website", () -> {
    // InputStream screenshot = ...
    Allure.attachment("Screenshot of the main page", screenshot);
});

// ... other steps ...

Attachment-only steps

Allure Report supports a special case called an “attachment-only step”. It is a step that does not contain any sub-steps and contains exactly one attachment with the same name as the step itself. Allure Report will simplify such a step visually, showing just the attachment, while preserving its position in relation to other steps. Any parameters for the step will not be displayed.

In the example below, the “Screenshot of the filled form” attachment is wrapped in the “Screenshot of the filled form” step. This allows to place the screenshot after the earlier steps but before the later step.

Java
Allure.step("Visit the website", () -> {
    // ... browser navigation code ...
});

Allure.step("Screenshot of the main page", () -> {
    // InputStream screenshot = ...
    Allure.attachment("Screenshot of the main page", screenshot);
});

// ... other steps ...