Test Identifiers

Allure integrations and Allure reporting tools use internal test identifiers to track tests across runs and correctly map historical results within reports.

This article explains how these internal identifiers work and what parts of your testing workflow may be affected by any changes to them.

TIP

This article frequently refers to concepts such as test cases, tests and test results. Please refer to the Glossary to learn what exactly these terms mean within Allure Report.

Identifier Types

There are four main identifiers that Allure tools use to track test results.

• fullName
• testCaseId
• historyId
• uuid

Here is an example of an Allure test result file (typically found within the allure-results directory of your project) where all these identifiers are stored:

{
  "name": "test_sum_keyword[sum_arguments1]",
  "status": "passed",
  "parameters": [
    {
      "name": "sum_arguments",
      "value": "[-1, 1, 0]"
    }
  ],
  "start": 1759304434025,
  "stop": 1759304434025,
  "uuid": "2f25f93b-9ef3-4e62-b4d7-e376db69e198",
  "historyId": "741e3c0379dae9c479f495d79cfc4a46",
  "testCaseId": "e3edbc90111b1e74614910797ec9e808",
  "fullName": "test.test_sum#test_sum_keyword",
  "labels": [
    {
      "name": "parentSuite",
      "value": "test"
    },
    {
      "name": "suite",
      "value": "test_sum"
    },
    {
      "name": "package",
      "value": "test.test_sum"
    }
  ]
}

All four are opaque identifiers that are calculated for each test result automatically and are not, usually, displayed in the generated reports.

However, as they are based on test framework metadata, they may change when you refactor your test code.

Two of them, fullName and testCaseId, are used to map test results to tests and test cases, so changes in them may cause problems in your workflow. The third one, historyId, is only used by older versions of Allure Report.

Below we discuss how each identifier is calculated and what it affects.

fullName

fullName identifies test cases and is always calculated according to two rules:

• it must be unique for every test case in the project
• for parametrized test cases it must not depend on parameter values

It is typically a text string consisting of concatenated test metadata supplied by the testing framework (for example, package name + class + test name).

Different Allure integrations use different metadata combinations to define fullName. Refer to the section for details.

If a test result has no testCaseId, Allure Report uses its fullName to identify the test for history and retries, see How Allure Report identifies tests.

fullName is the identifier used by Allure reporting tools to do selective test runs:

• Allure Report 3 uses fullName to identify which test cases to rerun.

Example

You run your JavaScript tests like so: npx allure run --rerun 5 -- npm test . This means that if any test cases during the initial run fail, you want Allure to rerun them up to 5 times. When the first run completes, Allure collects the fullName identifiers of failed tests and creates a test plan with these identifiers, which it then reruns.

• Allure TestOps uses fullName to run selected test cases from a test plan or launch.

• You can also create a test plan file with fullName identifiers manually and run it via your test framework’s Allure integration.

In all three cases, you may encounter problems, if the fullName identifier of some test changes, that is, if you refactor your code (for example, rename some test functions, rearrange test files), between the moment the identifier was saved into a test plan, and the moment the test plan is used to rerun tests.

If this is something that may happen in your project, the best practice is to assign ALLURE_ID labels to affected tests: if that label is present it overrides fullName as the main test case identifier for test plans, and it doesn’t change when you rearrange your code in any way.

TIP

The exact semantics of how each framework calculates fullName are subject to change. For example, the order in which strings are concatenated, or the characters used as separators may be revised at some point to account for some edge cases in naming. That's why this document goes into detail on what metadata is included into fullName for different frameworks, but not into how exactly it is constructed.

And for this same reason, we advise against building any custom integrations based on the exact structure of fullName.

testCaseId

testCaseId identifies test cases and is usually a hash of fullName. In some cases, it is derived from other, similarly unique test metadata. These cases are discussed below.

Like fullName, testCaseId is an opaque identifier not meant to be human readable.

In most cases, code refactoring that causes fullName for a test case to change will also change testCaseId.

Allure Report uses testCaseId to identify tests for history and retries, see How Allure Report identifies tests. Allure TestOps uses it to map test results to test cases.

TIP

Unlike fullName, you can set testCaseId via API in some framework integrations. For details, please consult the framework-specific documentation.

historyId

historyId is an identifier of individual tests that Allure integrations calculate from the test's metadata and parameters.

Allure Report 3 and Allure Report 2.45.0 and later don't use historyId: they identify tests themselves, as described below. The integrations still write it for the older versions of Allure Report, and Allure Report 2.46.1 and later use it to find the history recorded by those older versions.

uuid

uuid is unique for each test result, but is not reused across different runs in any way. No existing integration relies, and no custom integration should rely on it to map results to tests, so it won't be discussed further in this document.

How Allure Report identifies tests

To show a test's history and retries, Allure Report needs to know which test results belong to the same test. Allure Report 3 and Allure Report 2.45.0 and later decide it themselves, based on:

• the test result's testCaseId or, if it is not set, its fullName,
• the names and values of its parameters, except the excluded ones; the order of the parameters doesn't matter, and the values of hidden and masked parameters are taken into account too,
• in Allure Report 3, its environment, if the report is configured with environments.

Test results with the same identity are shown as retries of one test when they are part of the same launch, and as its history when they come from previous launches. Anything that changes the identity starts a new history for the test: a different testCaseId or fullName, an added or removed parameter, or a changed parameter value.

TIP

If you don't want a certain parameter to affect the test's identity, for example, a timestamp, mark that parameter as excluded. Most Allure integrations support this feature.

Updating an Allure integration can change the identifiers it calculates, too. The reference below mentions such changes in recent versions. For how older versions of Allure Report identified tests, see Making sure tests are identified correctly.

Test Identifiers Reference by Framework

This section documents what fullName and testCaseId depend on for each framework integration and, where relevant, which parameters the integration reports automatically.

Its purpose is to show what changes to a given test during refactoring may break the test history and/or its association with the test case.

If you find that some refactoring you're planning will affect the identifiers, you can set testCaseId to a previous value manually (this is supported by many integrations) and avoid these consequences.

C#

NUnit

• fullName depends on:
  • assembly name
  • class name (including namespace and generic type arguments)
  • test fixture arguments (if any)
  • method signature (name + generic arguments + parameter types)
• testCaseId depends on fullName
• parameters: the test method's arguments

xUnit

• fullName depends on:
  • assembly name
  • class name (including namespace)
  • method signature (name + generic arguments + parameter types)
• testCaseId depends on fullName
• parameters: the arguments of theories

Reqnroll/SpecFlow

• fullName depends on:
  • assembly name
  • folder path containing the feature file
  • featureInfo.Title - the feature title
  • scenarioTitle - the scenario name
• testCaseId depends on fullName

Dart

package:test and flutter_test

• fullName depends on:
  • test file path (relative to the package)
  • group names
  • test name
• testCaseId depends on fullName

Go

go test

• fullName depends on:
  • test file path (relative to the module)
  • test name, including the names of subtests
• testCaseId depends on fullName

Java

WARNING

The rules below describe Allure Java 3. Allure Java 3.0 changed how some identifiers are calculated: in Allure Java 2.x, the integrations for JUnit 4, TestNG and JBehave didn't set testCaseId, so after the upgrade, the history of these tests starts over. Parameters added at runtime are now part of the test's identity, too. See Migrating to Allure Java 3 for details.

Cucumber JVM

• fullName depends on:
  • feature file path (relative)
  • scenario line number
• testCaseId depends on:
  • feature file path
  • scenario name
• parameters: the values from the scenario outline's Examples row

JBehave

• fullName depends on:
  • story name
  • scenario title
• testCaseId depends on fullName
• parameters: the values from the examples table row

JUnit4

• fullName depends on:
  • package name
  • class name
  • method name
  • for Cucumber tests via JUnit4: feature file path or URI
• testCaseId depends on:
  • class name
  • method name

JUnit Jupiter

• fullName depends on:
  • methodSource: package, class name, method name
  • classSource: package, class name
  • classpathResourceSource: Resource path, optional line number
• testCaseId depends on JUnit Platform's unique ID of the test, without the part that identifies an invocation of a parameterized or repeated test
• parameters:
  • the arguments of @ParameterizedTest methods (requires extension autodetection)
  • the position of the invocation, for parameterized, repeated and other template tests (for example, changing the order of the values in a @ValueSource affects the history)

Karate

• fullName depends on:
  • feature file path (relative)
  • scenario line number
• testCaseId depends on:
  • feature file path
  • scenario name (or line number as fallback)

Scalatest

• fullName and testCaseId depend on:
  • suite ID
  • test name

Spock

• fullName depends on:
  • package name
  • class name
  • feature name (for data-driven features, the name of the feature, not of the iteration)
• testCaseId depends on fullName
• parameters: the data variables of data-driven features

TestNG

• fullName depends on:
  • fully qualified class name (includes package)
  • test method name
• testCaseId depends on:
  • class name
  • test method name
• parameters: the test method's parameters

JavaScript

WARNING

Since version 3.9.0, most Allure JavaScript integrations include the project name, taken from the name field of the nearest package.json file, in fullName and testCaseId. After upgrading from an older version, the history of the tests starts over.

AVA

• fullName and testCaseId depend on:

  • test file path (relative)
  • test title

  Excluded:

  • metadata tags/annotations embedded in the test title

Bun

• fullName and testCaseId depend on:

  • test file path
  • array of suite names + test name

  Excluded:

  • metadata tags/annotations embedded in test names

Cucumber-JS

• fullName depends on:
  • project name
  • relative path to the feature file
  • scenario name (as processed by Cucumber)
• testCaseId depends on:
  • project name
  • relative path to the feature file
  • scenario name (for scenario outlines, the name of the outline)

Cypress

• fullName and testCaseId depend on:

  • project name
  • spec file path
  • array of suite names + test name

  Excluded:

  • metadata tags/annotations embedded in test names

Jasmine

• fullName and testCaseId depend on:

  • project name
  • spec file path (relative, optional - can be undefined)
  • array of suite names
  • the spec/test name

  Excluded:

  • metadata tags/annotations embedded in spec name

Jest

• fullName and testCaseId depend on:

  • project name
  • test file path (relative to the Jest root directory)
  • all parent describe block names (suite hierarchy)
  • test name

  Excluded:

  • ROOT_DESCRIBE_BLOCK (Jest's internal root node)

Mocha / CodeceptJS

CodeceptJS uses Mocha under the hood, so identifier calculation rules are the same.

• fullName and testCaseId depend on:
  • project name
  • relative path to the test file
  • suite names
  • test name

Newman

• fullName and testCaseId depend on:
  • project name
  • array of parent collection/folder names
  • request name

node:test

• fullName and testCaseId depend on:

  • project name
  • relative path to the test file
  • suite names
  • test name

  Excluded:

  • metadata tags/annotations embedded in the test name

Playwright

• fullName depends on:
  • test file path
  • suite names
  • test name
• testCaseId depends on:
  • project name
  • test file path
  • suite names
  • test name

Before version 3.12.0, fullName depended on the test file path and the line and column numbers of the test function instead.

TestCafe

• fullName and testCaseId depend on:

  • fixture file path (relative)
  • fixture name
  • test name

  Excluded:

  • metadata tags/annotations embedded in the test name

Vitest

• fullName and testCaseId depend on:

  • project name
  • test file path (via relativeTestPath)
  • array of parent suite names
  • test name

  Excluded:

  • metadata tags/annotations embedded in spec name

WebdriverIO (WDIO)

• fullName depends on:
  • test file path (relative, URL-aware, strips position suffixes)
  • full test title (suite names + test title)
• testCaseId depends on:
  • full test title (suite names + test title)
  • name of the browser or device the test runs on

PHP

Behat

Allure Behat writes results in the older Allure 1 format, and Allure Report calculates fullName from them.

• fullName depends on:
  • regular scenario: Behat suite name, feature title, scenario title
  • scenario outline: Behat suite name, feature file path, line number
• testCaseId is not set (the tags configured via test_id_tag_prefix add TMS links, they don't affect the test's identity)

Codeception

• fullName depends on:
  • Cept: Codeception's built-in signature
  • Cest: Namespace, class name, method name
  • Unit: Namespace, class name, test name
  • Gherkin: Not set
• testCaseId depends on:
  • Codeception's built-in signature
  • parameter names

PHPUnit

• fullName depends on:
  • fully qualified class name
  • test method name
• testCaseId depends on:
  • fullName
  • non-excluded parameter names
• parameters: only the ones added via Allure::parameter(), see Parametrized tests

Python

Behave

• fullName and testCaseId depend on:

  • feature name
  • scenario name

  Excluded:

  • parameters (just the base scenario name used)

• parameters: scenario outline parameters (the Examples section)

Before version 2.15.1, Allure Behave didn't set testCaseId, so after upgrading from an older version, the history of the tests starts over.

pytest

• fullName and testCaseId depend on:

  • package/module path
  • test function name
  • class names - only if test is inside a class (can be nested classes)

  Excluded:

  • parameters

• parameters:

  • pytest parameters (e.g., provided via @pytest.mark.parametrize)
  • Allure parameters (except those with excluded set to True)

pytest-bdd

• fullName and testCaseId depend on:

  • relative file path to the .feature file
  • scenario name

  Excluded:

  • feature name (uses file path instead)
  • parameters (just the base scenario name)

• parameters:

  • pytest parameters (e.g., provided via @pytest.mark.parametrize)
  • scenario outline parameters from the Examples section of the Gherkin file
  • Allure parameters (except those with excluded set to True)

Robot Framework

• fullName depends on Robot Framework's built-in longname attribute, which is the full hierarchical test name (suite path + test name)
• testCaseId depends on fullName

Before version 2.15.2, Allure Robot Framework didn't set testCaseId, so after upgrading from an older version, the history of the tests starts over.

Ruby

Cucumber

• fullName depends on the scenario name
• testCaseId is not set (the allure_id label is used for test plans, it doesn't affect the test's identity)
• parameters: the values from the scenario outline's Examples row

RSpec

• fullName depends on example.full_description (RSpec's built-in property that includes the nested describe block descriptions plus the test description)
• testCaseId is not set (the allure_id label is used for test plans, it doesn't affect the test's identity)

Rust

Rust tests (allure-cargotest)

• fullName depends on:
  • module path
  • test function name
• testCaseId depends on fullName