Getting started with Allure JUnit 4

Generate beautiful HTML reports using Allure Report and your JUnit 4 tests.

Allure Report JUnit 4 Example

INFO

Check out the example projects at github.com/allure-examples to see Allure JUnit 4 in action.

Setting up

To integrate Allure into an existing JUnit 4 project, you need to:

Add Allure dependencies to your project.
Set up AspectJ.
Designate a location for Allure results storage.

Add Allure dependencies

MavenGradle (Kotlin)Gradle (Groovy)

xml

<!-- Define the version of Allure you want to use via the allure.version property -->
<properties>
    <allure.version>2.25.0</allure.version>
</properties>

<!-- Add allure-bom to dependency management to ensure correct versions of all the dependencies are used -->
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-bom</artifactId>
            <version>${allure.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<!-- Add necessary Allure dependencies to dependencies section -->
<dependencies>
    <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-junit4</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>

kts

// Define the version of Allure you want to use via the allureVersion property
val allureVersion = "2.25.0"
// ...
dependencies {
    // Import allure-bom to ensure correct versions of all the dependencies are used
    testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion"))
    // Add necessary Allure dependencies to dependencies section
    testImplementation("io.qameta.allure:allure-junit4")
}

groovy

// Define the version of Allure you want to use via the allureVersion property
def allureVersion = "2.25.0"

dependencies {
    // Import allure-bom to ensure correct versions of all the dependencies are used
    testImplementation platform("io.qameta.allure:allure-bom:$allureVersion")
    // Add necessary Allure dependencies to dependencies section
    testImplementation "io.qameta.allure:allure-junit4"
}

Configure AspectJ

AspectJ is required for registering Allure JUnit 4 as a listener for events that occur during the test execution. Additionally, Allure and some other frameworks (such as allure-assertj) leverage AspectJ for various integrations, including the @Step and @Attachment annotations.

MavenGradle (Kotlin)Gradle (Groovy)

xml

<!-- Define the version of AspectJ -->
<properties>
    <aspectj.version>1.9.21</aspectj.version>
</properties>

<!-- Add the following options to your maven-surefire-plugin -->
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.2.3</version>
    <configuration>
        <testFailureIgnore>false</testFailureIgnore>
        <argLine>
            -Dfile.encoding=${project.build.sourceEncoding}
            -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
        </argLine>
        <properties>
            <property>
                <name>listener</name>
                <value>io.qameta.allure.junit4.AllureJunit4</value>
            </property>
        </properties>
    </configuration>
    <dependencies>
        <dependency>
            <groupId>org.aspectj</groupId>
            <artifactId>aspectjweaver</artifactId>
            <version>${aspectj.version}</version>
        </dependency>
        <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-junit4-aspect</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
</plugin>

kts

// Define the version of AspectJ
val aspectJVersion = "1.9.21"

// Define configuration for AspectJ agent
val agent: Configuration by configurations.creating {
    isCanBeConsumed = true
    isCanBeResolved = true
}

dependencies {
    // Add aspectjweaver dependency
    agent("org.aspectj:aspectjweaver:${aspectJVersion}")

    // Add AspectJ integration for Allure
    testImplementation("io.qameta.allure:allure-junit4-aspect")
}

// Configure javaagent for test execution
tasks.test {
    useJUnit()
    jvmArgs = listOf(
        "-javaagent:${agent.singleFile}"
    )
}

groovy

// Define the version of AspectJ
def aspectJVersion = '1.9.21'

// Define configuration for AspectJ agent
configurations {
    agent {
        canBeResolved = true
        canBeConsumed = true
    }
}

dependencies {
    // Add aspectjweaver dependency
    agent "org.aspectj:aspectjweaver:$aspectJVersion"

    // Add AspectJ integration for Allure
    testImplementation "io.qameta.allure:allure-junit4-aspect"
}

// Configure javaagent for test execution
test {
    useJUnit()
    jvmArgs = [ "-javaagent:${configurations.agent.singleFile}" ]
}

Specifying Allure Results location

Allure, by default, saves test results in the project's root directory. However, it is recommended to store your test results in the build output directory.

To configure this, create an allure.properties file and place it in the test resources directory of your project, which is typically located at src/test/resources:

MavenGradle (Kotlin)Gradle (Groovy)

properties

allure.results.directory=target/allure-results

properties

allure.results.directory=build/allure-results

properties

allure.results.directory=build/allure-results

Run tests

Run your JUnit 4 tests the same way as you would run them usually. For example:

For Gradle:

MacOS/LinuxWindows

bash

./gradlew test

bash

gradlew test

For Maven:

MacOS/LinuxWindows

bash

./mvnw verify

bash

mvnw verify

After running the tests, Allure will gather the test execution data and store it in the allure-results directory. You can then generate an HTML report from these results using Allure's reporting tools.

Generate a report

Finally, convert the test results into an HTML report. This can be done by one of two commands:

allure generate processes the test results and saves an HTML report into the allure-report directory. To view the report, use the allure open command.
Use this command if you need to save the report for future reference or for sharing it with colleagues.
allure serve creates the same report as allure generate but puts it into a temporary directory and starts a local web server configured to show this directory's contents. The command then automatically opens the main page of the report in a web browser.
Use this command if you need to view the report for yourself and do not need to save it.

Writing tests

The Allure JUnit 4 adapter not only collects the data provided by JUnit 4's standard features, but also provides additional features for writing even better tests. This section lists the most notable ways to improve your tests, using both JUnit 4's and Allure JUnit 4's features.

With Allure JUnit 4, you can:

provide description, links and other metadata,
organize tests into hierarchies,
divide the test into smaller, easier-to-read test steps,
describe parameters used when running parametrized tests,
make the test save screenshots and other files during execution,
select which tests to run via a test plan file,
provide arbitrary environment information for the whole test report.

Specify description, links and other metadata

There is a lot of metadata you can add to each test so that it would appear in the report. See the reference for more details.

For each of the metadata fields, there are two ways to assign it: via an annotation before a test method or via a method call inside a test method's body. The second way is called “dynamic”, because it allows you to construct strings and other values at runtime before passing to the methods. Note, however, that it is highly recommended to assign all metadata as early as possible. Otherwise, there is a risk of the test failing before having all metadata set, which is bad for the test report's readability.

Annotations APIRuntime API

java

import io.qameta.allure.Description;
import io.qameta.allure.Issue;
import io.qameta.allure.Link;
import io.qameta.allure.Owner;
import io.qameta.allure.Severity;
import io.qameta.allure.TmsLink;
import io.qameta.allure.junit4.DisplayName;
import org.junit.Test;

import static io.qameta.allure.SeverityLevel.*;

public class TestMyWebsite {

    @Test
    @DisplayName("Test Authentication")
    @Description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.")
    @Severity(CRITICAL)
    @Owner("John Doe")
    @Link(name = "Website", url = "https://dev.example.com/")
    @Issue("AUTH-123")
    @TmsLink("TMS-456")
    public void testAuthentication() {
        // ...
    }
}

java

import io.qameta.allure.Allure;
import org.junit.Test;

import static io.qameta.allure.SeverityLevel.*;

public class TestMyWebsite {

    @Test
    public void testAuthentication() {
        Allure.getLifecycle().updateTestCase(result -> result.setName("Test Authentication"));
        Allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.");
        Allure.label("severity", "critical");
        Allure.label("owner", "John Doe");
        Allure.link("Website", "https://dev.example.com/");
        Allure.issue("AUTH-123", "https://example.com/issues/AUTH-123");
        Allure.tms("TMS-456", "https://example.com/tms/TMS-456");
        // ...
    }
}

Organize tests

As described in Improving navigation in your test report, Allure supports multiple ways to organize tests into hierarchical structures. Allure JUnit 4 provides functions to assign the relevant fields to tests either by adding annotations or “dynamically” (same as for the metadata fields).

To specify a test's location in the behavior-based hierarchy:

Annotations APIRuntime API

java

import io.qameta.allure.Epic;
import io.qameta.allure.Feature;
import io.qameta.allure.Story;
import org.junit.Test;

public class TestMyWebsite {

    @Test
    @Epic("Web interface")
    @Feature("Essential features")
    @Story("Authentication")
    public void testAuthentication() {
        // ...
    }
}

java

import io.qameta.allure.Allure;
import org.junit.Test;

public class TestMyWebsite {

    @Test
    public void testAuthentication() {
        Allure.epic("Web interface");
        Allure.feature("Essential features");
        Allure.story("Authentication");
        // ...
    }
}

To specify a test's location in the suite-based hierarchy:

java

import io.qameta.allure.Allure;
import org.junit.Test;

public class TestMyWebsite {

    @Test
    public void testAuthentication() {
        Allure.label("parentSuite", "Tests for web interface");
        Allure.suite("Tests for essential features");
        Allure.label("subSuite", "Tests for authentication");
        // ...
    }
}

A test's location in the package-based hierarchy is defined by the fully qualified names of the classes they are declared in, with common prefixes shown as parent packages.

Divide a test into steps

Allure JUnit 4 provides three ways of creating steps and sub-steps: “annotated steps”, “lambda steps” and “no-op steps”, see the reference.

Annotated stepsLambda stepsNo-op steps

java

import io.qameta.allure.Param;
import io.qameta.allure.Step;
import org.junit.Test;

public class TestMyWebsite {

    @Test
    public void testAuthentication() {
        testAuthenticationWith("https://1.example.com/", "alice", "qwerty");
        testAuthenticationWith("https://2.example.com/", "bob", "asdfgh");
        testAuthenticationWith("https://3.example.com/", "charlie", "zxcvbn");
    }

    @Step("Authenticate on {url}")
    void testAuthenticationWith(String url, String login, String password) {
        openWebPage(url);
        enterCredentials(login, password);
        checkIfLoggedInAs(login);
    }

    @Step("Visit {url}")
    void openWebPage(@Param("URL") String url) {
        // ...
    }

    @Step("Enter credentials")
    void enterCredentials(
            @Param("Login") String login,
            @Param("Password") String password
    ) {
        // ...
    }

    @Step("Check if I am logged in")
    void checkIfLoggedInAs(
            @Param("Login") String login
    ) {
        // ...
    }
}

java

import io.qameta.allure.Allure;
import org.junit.Test;

public class TestMyWebsite {

    private static final String[][] authenticationData = new String[][]{
            {"https://1.example.com/", "alice", "qwerty"},
            {"https://2.example.com/", "bob", "asdfgh"},
            {"https://3.example.com/", "charlie", "zxcvbn"},
    };

    @Test
    public void testAuthentication() {

        for (String[] data : authenticationData) {
            String url = data[0];
            String login = data[1];
            String password = data[2];

            Allure.step("Authenticate on %s".formatted(url), () -> {

                Allure.step("Visit %s".formatted(url), stepContext -> {
                    stepContext.parameter("URL", url);
                    // ...
                });

                Allure.step("Enter credentials", stepContext -> {
                    stepContext.parameter("Login", login);
                    stepContext.parameter("Password", password);
                    // ...
                });

                Allure.step("Check if I am logged in", stepContext -> {
                    stepContext.parameter("Login", login);
                    // ...
                });
            });
        }
    }
}

java

import io.qameta.allure.Allure;
import io.qameta.allure.model.Status;
import org.junit.Test;

public class TestMyWebsite {

    private static final String[][] authenticationData = new String[][]{
            {"https://1.example.com/", "alice", "qwerty"},
            {"https://2.example.com/", "bob", "asdfgh"},
            {"https://3.example.com/", "charlie", "zxcvbn"},
    };

    @Test
    public void testAuthentication() {

        for (String[] data : authenticationData) {
            String url = data[0];
            String login = data[1];
            String password = data[2];

            Allure.step("Begin testing %s...".formatted(url));
            try {
                // ...
                Allure.step("Visited %s.".formatted(url));

                // ...
                Allure.step("Entered credentials.");

                // ...
                Allure.step("I am logged in!");

            } catch (Exception e) {
                Allure.step("Failed!", Status.FAILED);
            }
        }
    }
}

Describe parametrized tests

When using the parametrized tests pattern, use the Allure JUnit 4's parameter() to add the parameters to the report, see the reference.

java

import io.qameta.allure.Allure;
import org.junit.Test;

public class TestMyWebsite {

    @Test
    public void testAuthenticationWithUsername() {
        Allure.parameter("login", "johndoe");
        // ...
    }

    @Test
    public void testAuthenticationWithEmail() {
        Allure.parameter("login", "[email protected]");
        // ...
    }
}

Attach screenshots and other files

You can attach any sorts of files to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point.

Allure JUnit 4 provides various ways to create an attachment, both from existing files or generated dynamically, see the reference.

java

import io.qameta.allure.Allure;
import org.junit.Test;

import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Paths;

public class TestMyWebsite {

    @Test
    public void testAuthentication() throws IOException {
        // ...
        Allure.attachment("data.txt", "This is the file content.");
        try (InputStream is = Files.newInputStream(Paths.get("/path/img.png"))) {
            Allure.attachment("image.png", is);
        }
    }
}

Select tests via a test plan file

WARNING

For this feature to work, the AspectJ integration for Allure Report must be added to the project.

If the ALLURE_TESTPLAN_PATH environment variable is defined and points to an existing file, JUnit 4 will only run tests listed in this file.

Here's an example of running tests according to a file named testplan.json:

MacOS/LinuxWindows

bash

export ALLURE_TESTPLAN_PATH=testplan.json
./gradlew test

powershell

$Env:ALLURE_TESTPLAN_PATH = "testplan.json"
gradlew test

Environment information

For the main page of the report, you can collect various information about the environment in which the tests were executed.

For example, it is a good idea to use this to remember the OS and Java versions. This may help the future reader investigate bugs that are reproducible only in some environments.

Allure Report Environments Widget

To provide environment information, put a file named environment.properties into the allure-results directory after running the tests. See the example in Environment file.

Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using Parametrized tests.

Getting started with Allure JUnit 4

Setting up

Add Allure dependencies

Configure AspectJ

Specifying Allure Results location

Run tests

Generate a report

Writing tests

Specify description, links and other metadata

Organize tests

Divide a test into steps

Describe parametrized tests

Attach screenshots and other files

Select tests via a test plan file

Environment information

Powered by

Join our newsletter

Allure TestOps

Company

Getting started with Allure JUnit 4 ​

Setting up ​

Add Allure dependencies ​

Configure AspectJ ​

Specifying Allure Results location ​

Run tests ​

Generate a report ​

Writing tests ​

Specify description, links and other metadata ​

Organize tests ​

Divide a test into steps ​

Describe parametrized tests ​

Attach screenshots and other files ​

Select tests via a test plan file ​

Environment information ​

Powered by

Join our newsletter

Allure TestOps

Company

Getting started with Allure JUnit 4

Setting up

Add Allure dependencies

Configure AspectJ

Specifying Allure Results location

Run tests

Generate a report

Writing tests

Specify description, links and other metadata

Organize tests

Divide a test into steps

Describe parametrized tests

Attach screenshots and other files

Select tests via a test plan file

Environment information