Allure Cucumber-JVM reference

These are the functions that you can use to integrate your Cucumber-JVM tests with Allure.

Allure allows you to enrich your reports with a variety of metadata. This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the metadata reference section for an exhaustive list of what can be added.

You can assign metadata to a Cucumber-JVM features in two ways:

1. Using Gherkin tags: Use a naming convention to add metadata to test scenarios.
2. Runtime API: Use a method call within the body of a test step method to dynamically assign metadata during test runtime. This approach allows for the construction of strings and values on-the-fly.

When assigning metadata to your tests using Allure's Runtime API, it's crucial to consider the balance between control and potential risks. While the Runtime API offers more flexibility in handling metadata dynamically, it also comes with certain drawbacks. For instance, if a test fails before all the dynamic metadata is applied, the resulting report may lack important information, leading to reduced clarity and completeness. Therefore, it's essential to ensure that metadata is set early in the test to mitigate this risk.

WARNING

To use Gherkin tags for assigning data, be sure to upgrade to Allure Report 2.27 or higher or consider uploading your test results to Allure TestOps.

Metadata

Assign a test's description, links and other metadata.

Title

• Allure.getLifecycle().updateTestCase(Consumer<TestResult> update)

Set the test's title.

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {
        Allure.getLifecycle().updateTestCase(result ->
                result.setName("Some modified scenario name"));
        // ...
    }
}

Description

• Allure.description(String description)

Set the test's description.

You can either begin a Feature with a description in your Gherkin file or call the methods for updating the values dynamically.

Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes.

Gherkin description

Feature: Labels

  This test attempts to create a label with specified title

  Scenario: Create new label for authorized user
    When I open labels page
    And I create label with title "hello"
    Then I should see label with title "hello"

Runtime API

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {
        Allure.description("This test attempts to create a label with specified title");
        // ...
    }
}

Owner

Set the test's owner.

You can either use the @allure.label.owner tags in your Gherkin file or call the label() method for updating the values dynamically. Note that when using tags in Gherkin, the value cannot contain spaces.

Gherkin tags

@allure.label.owner=JohnDoe
Feature: Labels

  Scenario: Create new label for authorized user
    When I open labels page
    And I create label with title "hello"
    Then I should see label with title "hello"

Runtime API

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {
        Allure.label("owner", "JohnDoe");
        // ...
    }
}

Tag

Set the test's tags.

You can either use the tags in your Gherkin file or call the label() method for updating the values dynamically.

Gherkin tags

@ui @labels
Feature: Labels

  Scenario: Create new label for authorized user
    When I open labels page
    And I create label with title "hello"
    Then I should see label with title "hello"

Runtime API

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {
        Allure.label("tag", "ui");
        Allure.label("tag", "labels");
        // ...
    }
}

Severity

Set the test's severity.

Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”.

Gherkin tags

Feature: Labels

  @critical
  Scenario: Create new label for authorized user
    When I open labels page
    And I create label with title "hello"
    Then I should see label with title "hello"

Runtime API

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {
        Allure.label("severity", "critical");
        // ...
    }
}

Label

• Allure.label(String name, String value)

Allure labels are key-value pairs used to add additional meta-information to your tests. They can be considered as a way to categorize, organize, and search for test cases within your Allure reports. Labels can be used to specify things like the test case ID, the severity of the test, the feature being tested, or the story that relates to the test. This meta-information helps to create more detailed and structured reports.

Gherkin Tags

@allure.label.layer:web
@allure.label.owner:eroshenkoam
@allure.label.page:/{org}/{repo}/labels
Feature: Labels

  @critical
  @allure.label.jira:AE-2
  Scenario: Create new label for authorized user
    When I open labels page
    And I create label with title "hello"
    Then I should see label with title "hello"

Runtime API

import io.cucumber.java.en.Given;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {
        Allure.label("layer", "web");
        Allure.label("owner", "eroshenkoam");
        Allure.label("page", "/{org}/{repo}/labels");
        Allure.label("severity", "critical");
        Allure.label("jira", "AE-2");
        // ...
    }
}

Allure ID (Allure TestOps)

You can specify test case ID for your Cucumber-JVM tests using @allure.id tag or via Runtime API:

Gherkin Tags

Feature: Labels

  @allure.id:123
  Scenario: Create new label for authorized user
    When I open labels page
    And I create label with title "hello"
    Then I should see label with title "hello"

Link

• Allure.link(String url)
• Allure.link(String name, String url)
• Allure.link(String name, String type, String url)
• Allure.issue(String name, String url)
• Allure.tms(String name, String url)

Add a link related to the test.

Based on the type (which can be any string), Allure will try to load a corresponding link pattern to process the URL, as defined by the allure.link.*.pattern configuration option. If no pattern found for the given type, the URL is left unmodified.

The name will be used as the link's text. If it is omitted, the unprocessed URL will be used instead.

For convenience, Allure provides two shorthand functions with pre-selected link types: issue and tms.

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {
        Allure.link("Website", "https://dev.example.com/");
        Allure.issue("AUTH-123", "https://example.com/issues/AUTH-123");
        Allure.tms("TMS-456", "https://example.com/tms/TMS-456");
        // ...
    }
}

Behavior-based hierarchy

• Allure.epic(String value)
• Allure.feature(String value)
• Allure.story(String value)

Assign names of epics, features or user stories for a test, as part of Allure's behavior-based hierarchy.

You can either use the @allure.label.* tags in your Gherkin file or call the methods for updating the values dynamically. Note that when using tags in Gherkin, the values cannot contain spaces.

Gherkin tags

@allure.label.epic:WebInterface
@allure.label.feature:EssentialFeatures
@allure.label.story:Labels
Feature: My feature
  Scenario: My scenario
    Then my step

Runtime API

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {
        Allure.epic("Web interface");
        Allure.feature("Essential features");
        Allure.story("Labels");
        // ...
    }
}

Suite-based hierarchy

• Allure.suite(String value)

Assign the name of suite, as part of Allure's suite-based hierarchy.

You can either use the @allure.label.* tags in your Gherkin file or call the suite() and label() methods for updating the values dynamically. Note that when using tags in Gherkin, the values cannot contain spaces.

Gherkin tags

@allure.label.parent_suite:WebInterface
@allure.label.suite:EssentialFeatures
@allure.label.sub_suite:Labels
Feature: My feature
  Scenario: My scenario
    Then my step

Runtime API

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {
        Allure.label("parentSuite", "Tests for web interface");
        Allure.suite("Tests for essential features");
        Allure.label("subSuite", "Tests for labels");
        // ...
    }
}

Test steps

• @Step(String value="")
• Allure.step(String name)
• Allure.step(String name, Status status)
• Allure.step(String name, ThrowableRunnableVoid runnable)
• Allure.step(String name, ThrowableRunnable<T> runnable)
• Allure.step(ThrowableContextRunnableVoid<StepContext> runnable)
• Allure.step(String name, ThrowableContextRunnableVoid<StepContext> runnable)
• Allure.step(String name, ThrowableContextRunnable<T, StepContext> runnable)
• Allure.step(ThrowableContextRunnable<T, StepContext> runnable)

Define a test step with the given name.

There are three ways of defining a step.

• “An annotated step”: define a method containing a test step and decorate it with @Step().
• “A lambda step”: write a test step in a lambda function and pass it to Allure.step().
• “A no-op step”: just pass a name and an optional status to Allure.step() to immediately add a corresponding entry to the results as a sub-step of the current step.

When you are implementing a function for a lambda step, it can accept either no arguments or a single argument of class StepContext. This object provides the following methods:

• name() — override the step name during its execution.
• parameter() — indicate arbitrary parameters used for the step. The available signatures of this method are the same as for the test-wide implementation, see Parametrized tests.

Annotated steps

import io.cucumber.java.en.Then;
import io.qameta.allure.Step;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {
        step1();
        step2();
    }

    @Step("Step 1")
    public void step1() {
        subStep1();
        subStep2();
    }

    @Step("Sub-step 1")
    public void subStep1() {
        // ...
    }

    @Step("Sub-step 2")
    public void subStep2() {
        // ...
    }

    @Step("Step 2")
    public void step2() {
        // ...
    }
}

Lambda steps and no-op steps

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() {

        Allure.step("Step 1", (step) -> {
            // ...
            Allure.step("Sub-step 1");
            // ...
            Allure.step("Sub-step 2");
        });

        Allure.step("Step 2", (step) -> {
            // ...
        });
    }
}

Parametrized tests

• Allure.parameter(String name, T value)
• Allure.parameter(String name, T value, Boolean excluded)
• Allure.parameter(String name, T value, Parameter.Mode mode)
• Allure.parameter(String name, T value, Boolean excluded, Parameter.Mode mode)

Specify a name and value of a parameter that was used during this test. See Parametrized tests for more details.

WARNING

Allure Cucumber-JVM identifies tests based only on their names and Gherkin parameters. If you have multiple tests that only differ in dynamic parameters (i.e., the Allure.parameter() calls), they will be treated as one test for the purposes of history and retries.

If the excluded argument is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. This argument is only used by Allure TestOps.

The mode argument affects how the parameter will be displayed in the report. Available options are defined in the Parameter.Mode enumeration:

• Parameter.Mode.DEFAULT (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters.
• Parameter.Mode.MASKED — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters.
• Parameter.Mode.HIDDEN — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the allure_results directory if you publish it.

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

public class CucumberSteps {

    @Then("authorize as {string}")
    public void testAuthentication(String login) {
        Allure.parameter("auth_method", "password");
        Allure.parameter("login", login);
        // ...
    }

    @Then("authorize with empty login")
    public void testAuthenticationWithEmptyLogin() {
        Allure.parameter("login", "");
        Allure.parameter("auth_method", "password");
        // ...
    }
}

Attachments

Functions for adding attachments to test results.

Attaching content from variables

• Allure.addAttachment(String name, String content)
• Allure.addAttachment(String name, String type, String content)
• Allure.addAttachment(String name, String type, String content, String fileExtension)
• Allure.addAttachment(String name, InputStream content)
• Allure.addAttachment(String name, String type, InputStream content, String fileExtension)
• Allure.attachment(String name, String content)
• Allure.attachment(String name, InputStream content)

Add content as an attachment to the test result under the given name (defaults to a unique pseudo-random string).

TIP

You can use data produced by any function, not necessarily read from an actual file.

To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as type and, optionally, a filename extension as fileExtension. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file.

import io.cucumber.java.en.Then;
import io.qameta.allure.Allure;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() throws IOException {
        // ...
        Allure.attachment("data.txt", "This is the file content.");
        Allure.attachment("img.png", Files.newInputStream(Paths.get("/path/img.png")));
    }
}

Generating attachments dynamically

• @Attachment(String value="", String type="", String fileExtension="")

Define a function that returns the data you need to attach and annotate it with @Attachment. Call the function at any point during your test to attach the data to the test result.

The data will be processed as following:

• If the data type is byte[], Allure will use it without modification.
• If the data type is String, Allure will convert it to byte[].
• If the data is of any other type, Allure will call the toString() method and then convert the string to byte[].

To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as type and, optionally, a filename extension as fileExtension. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file.

import io.cucumber.java.en.Then;
import io.qameta.allure.Attachment;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

public class CucumberSteps {

    @When("^I open labels page$")
    public void openLabelsPage() throws IOException {
        // ...
        attachDataTXT();
        attachScreenshotPNG();
    }

    @Attachment(value = "data", type = "text/plain", fileExtension = ".txt")
    public String attachDataTXT() {
        return "This is the file content.";
    }

    @Attachment(value = "screenshot", type = "image/png", fileExtension = ".png")
    public byte[] attachScreenshotPNG() throws IOException {
        return Files.readAllBytes(Paths.get("/path/to/image.png"));
    }
}

INFO

Additionally, Allure provides a way to spawn threads that will add attachments to the test results asynchronously without blocking the test runner. While not recommended for the majority of cases, this approach can improve tests that otherwise have to wait for processing very large files, such as videos larger than 1 GB. Please refer to the source code for Allure.addByteAttachmentAsync() and Allure.addStreamAttachmentAsync() to learn more.

Results history

Flaky

• @Flaky

Indicate that the test is known to be unstable and can may not succeed every time. See Flaky tests.