---
title: History and retries
description: Explore the history and retries of test results in Allure Report, and learn how to track test performance over time with detailed insights and trends.
---

# History and retries

We know that one of the reasons regular testing is important is because it helps understand how the quality of your product is changing over time. If a certain test fails now, it is useful to know whether it also failed yesterday or even last week.

In Allure Report, you can navigate between multiple runs of the same test — both across different reports via [Tests history](#tests-history) and within a single one via [Retries](#retries).

## Tests history

![Tests History](/images/diagrams/history.png "Tests History")

Each report includes test results collected during a new **test launch**. However, the tests themselves are usually not new at all: you most likely use the same test code each time, slightly updating it only when necessary. Using each test's [unique identifier](#making-sure-tests-are-identified-correctly), Allure Report is able to link its latest result to the results of the same test from previous launches. This collection of links is known as the **tests history**.

In a report with the tests history included, you can:

- see what statuses a test had previously,
- identify unstable tests (see [Test stability analysis](/docs/test-stability/)),
- find tests that changed their status since last report (see [Filter tests by marks](/docs/sorting-and-filtering/#filter-tests-by-marks)),
- see how various metrics change over time (see [Trend charts](/docs/visual-analytics/#trend-charts)).

For most CI servers that we support, a corresponding Allure Report plugin will automatically keep the history and let you seamlessly navigate to previous test report. The same can be done when building reports locally, too, see [How to enable history](#how-to-enable-history) below.

### How to enable history

Allure 2 and Allure 3 utilize different approach to working with history.

#### Allure Report 3

In Allure Report 3, to enable history, simply set `historyPath` to the desired location of the JSONL history file in the [configuration file](/docs/v3/configure/#_3-history-options):

```json
{
  "historyPath": "./history.jsonl"
}
```

Each time Allure Report 3 generates a report while history is enabled, it reads previous historical data from the history file and writes a new line with all current test launch data to it.

All historical data in Allure 3 is stored in this single file. For more details on the file structure, see [History files](/docs/how-it-works-history-files/#allure-report-3).

#### Allure Report 2

Each time Allure Report 2 generates a test report (see [How it works](/docs/how-it-works/)), it also generates data into the `history` subdirectory. The JSON files in this subdirectory store all the information that Allure needs to include this test report's data in the next test report, see [History files](/docs/how-it-works-history-files/#allure-report-2).

For Allure 2, enabling the history-related features is a matter of copying the `history` subdirectory into the test results directory before generating the next test report.

Here is an example of how to do it, assuming that your project is configured to use `allure-results` and `allure-report` directories:

1. Make sure you have the previous report generated in the `allure-report` directory.

2. Remove the `allure-results` directory.

3. Run the tests.

4. Copy the `allure-report/history` subdirectory to `allure-results/history`.

5. Generate the new report.

If you follow such a routine regularly, Allure will each time keep data from up to 20 latest reports in `allure-results/history`. This creates up to 20 history entries for each test and up to 20 columns in each chart.

## Retries

![Test Retries](/images/diagrams/retries.png "Test Retries")

When you run the same test multiple times, Allure produces a separate [test result file](/docs/how-it-works-test-result-file/) for each run. The test report then includes the status and other information from the latest file, with the other runs shown as **retries**.

A common reason to retry a test is if it failed for some environment-specific reasons (poor internet connection, no disk space available, etc.) and you need to run it again for more informative results. Some test frameworks may even retry failed tests automatically. In these cases, the total number of retries can become a useful metric in itself — you can see if it grows or declines on one of the [Trend charts](/docs/visual-analytics/#trend-charts).

Another case is when one or more tests are known to be unstable, but it is still important to differentiate between the “fails always” and “fails sometimes” states. In Allure Report, you can find all tests that have changed their status after a retry using a special filter, see [Filter tests by marks](/docs/sorting-and-filtering/#filter-tests-by-marks).

Note that unlike the latest attempt at each test, the retries do not get included in the [Tests history](#tests-history).

### How to keep retries

To make sure the information about retries gets added to the test report, just avoid removing your test results directory (which is usually called `allure-results`). If the Allure integration for the framework that you use has an option to clean the directory before each run, disable that option. Then, when building a report, Allure Report will include all the files present in the directory, i.e., all the tests runs since you last cleaned the directory.

Note, however, that keeping all the retries can skew the data for the Retries trend chart. The Allure Report plugins for CI servers always run tests with an empty `allure-results` directory, so that the report only includes automatic retries done by the test framework.

## Making sure tests are identified correctly

For test results to appear correctly in the History and Retries tabs, Allure needs to identify which results belong to the same test across different runs. It does this using an [internal identifier called “history ID”](/docs/how-it-works-test-identifiers/#historyid).

The history ID is automatically calculated by your framework's Allure integration. With some limitations and exceptions, discussed in the linked article, each integration assigns the history ID based on two factors:

- the test's fully-qualified name,
- the test's non-excluded [parameters](/docs/v3/readability/#parametrized-tests).

This allows Allure to correctly group together test runs of the exact same test with the exact same parameters:

- Tests results with the same history ID that are part of a single launch (located in the `allure-results` directory) are included in the Retries tab for the test in your report.

- Tets results with the same history ID from different launches - the ones you copy to `allure-results/history`, are linked on the History tab in your report.

Tip:
To ensure that the test report will properly deal with the history and retries, make sure to pass to Allure all parameters that distinguish different versions of a test.

Most Allure integrations take care of it automatically, but some may need additional code, such as [`Allure::parameter()`](/docs/phpunit-reference/#parametrized-tests) for Allure PHPUnit or [`@TestInstanceParameter`](/docs/testng-reference/#test-instance-parameters-via-dataprovider) for some configurations in Allure TestNG.

Below are some common issues related to history ID inconsistencies, that you may encounter when setting up your reporting workflows.

### Common pitfall: tests from different environments are missing in the report

Let's assume that you run a test on multiple platforms before collecting all the results into a single `allure-results` directory and building the report. Unless you make sure that different runs get different sets of [parameters](/docs/v2/readability/#parametrized-tests), Allure Report will show only one test result by default, moving other ones to its list [retries](#retries).

We recommend adding one or more special parameters that will make up a different combination for each environment. For example, you may want to add such parameters as “os_version”, “browser_version”, “java_version”, etc., depending on the technologies used by your test.

Be sure to check the “Parametrized tests” section of your Allure integration's documentation to see if it has any limitations. When an integration has multiple ways of declaring test parameters, it may or may not support all of them for the purposes of identifying a test. For example, Allure JUnit 5 and some other integrations can not distinguish tests that only differ in “dynamic” parameters, i.e., the parameters that were set after the test execution started.

### Common pitfall: a test's retries are displayed as separate tests

Normally, when you run a test with a set of parameters different from the previous times, Allure Report treats it as a new test. This means the runs are displayed in the test report separately instead of being grouped into a list of a single test's [retries](#retries). Additionally, Allure may fail to find links to the previous results in the test's [history](#tests-history).

For example, if a parameter derives from current date and time or from a randomized value, you will get seemingly duplicated test results when you run the test more than once.

If this is not the desired behavior for your test, consider removing such a parameter or marking it as `excluded`. See your integration's reference to find out if it allows marking parameters `excluded` and how to do it.

## Retrying tests from Allure

One popular feature request is the ability to rerun failed tests right from Allure Report. Some scenarios where it may be helpful include:

- checking if the test results are consistent,
- retrying flaky tests,
- following the TDD process with Allure.

Starting with [Allure Report 3](https://github.com/allure-framework/allure3) some of these scenarios are now possible.

The `allure run` command allows to run any framework-specific test command, for example:

```bash
allure run -- npm run test
```

This command includes an option to rerun failed tests up to a set number of times. The following command will rerun failed tests up to three times, and the results of multiple runs will be listed as retries in the generated report.

```bash
allure run --rerun 3 -- npm run test
```

If you need more, consider using [Allure TestOps](https://qameta.io/). It supports rerunning an arbitrary set of tests in a CI pipeline or in manual mode.