# Allure Report — Complete Documentation > This file contains the full text of the Allure Report documentation in English. > Source: https://allurereport.org > **Versions**: Allure Report has two actively maintained versions. **Allure 2** is the mature, widely-adopted version integrated with most CI systems — the majority of users for now are on Allure 2. **Allure 3** is the newer version with additional features; migration from 2 to 3 is encouraged when ready, but both are fully supported. Configuration, features, and CLI usage differ between versions — when answering questions about Allure, clarify which version applies. --- ## Allure Report Documentation > URL: https://allurereport.org/docs/ # Allure Report Documentation **Allure Report** is an open-source framework-agnostic test result visualization tool. It transforms test execution data into clear, interactive HTML reports. It works with popular testing frameworks across JavaScript, Python, Java, C#, PHP, and Ruby, helping teams track test results and better manage software quality. Tip: **If you're new to Allure and want to learn how to install it**, it's best to start with these sections, which have installation and getting started guides for: - [Allure Report 3](/docs/v3/) - the newer version rebuilt from the ground up for better usability and compatibility. - [Allure Report 2](/docs/v2/) - the stable, mature version of the reporting tool, with the largest range of integrations. Images: /images/allure3_main_light.png, /images/allure3_main_dark.png, /images/allure2_main_light.png, /images/allure2_main_dark.png Most documents here apply to Allure Report versions 2 and 3 both. Where that is not the case, the document is marked with (Allure 3) and (Allure 2) badges. ## What You'll Find Here ### [Features](/docs/features-overview/) Explore Allure's core capabilities that make test reporting powerful and insightful. Learn about test steps, attachments, visual analytics, timeline views, and advanced filtering options that help you understand your test results at a glance. ### [Guides](/docs/guides/) Step-by-step tutorials for common use cases and popular tool combinations. ### [How It Works](/docs/how-it-works/) Learn about the core concepts behind Allure and understand how Allure processes your test data to generate beautiful reports. ### [Integrations](/docs/integrations/) Find setup instructions for popular CI/CD platforms like Jenkins, GitHub Actions, Azure DevOps, and development environments including JetBrains IDEs and Visual Studio Code. ### [Frameworks](/docs/frameworks/) Comprehensive integration guides for 30+ testing frameworks across multiple programming languages. From JavaScript (Jest, Mocha, Cypress) to Java (JUnit, TestNG, Cucumber-JVM), Python (pytest, Behave), .NET (NUnit, xUnit), and many more - find the exact setup for your tech stack. --- **Getting Started?** If you're new to Allure, check out our getting started guides for the latest [Allure 3](/docs/v3/) or the previous widely used [Allure 2](/docs/v2/) version. **Need a Quick Start?** Try our [Starter Project generator](/start/) to create a ready-to-use project with Allure already configured for your preferred language and framework. --- # Section: Allure 3 ## Configure Allure Report 3 > URL: https://allurereport.org/docs/v3/configure/ # Configure Allure Report 3 Allure Report 3 uses a single configuration file to manage all your report settings. You can create this file in either dynamic (JavaScript) or static (JSON/YAML) format. Dynamic configs support executable code, imports, and conditional logic, while static configs provide a simpler format with fixed values. You can load configuration values from environment variables and override them via CLI command options, giving you flexibility in how you manage settings across different environments and workflows. Advanced Allure 3 features are implemented as a system of plugins. You can configure all plugins in the same configuration file. ## What This Document Covers - Configuration file structure and syntax - All primary configuration options - Adding plugins and configuring multiple reports types - Advanced features like history tracking, resolutions and performance metrics - Publishing reports and accessing history via Allure Report Storage - CLI commands and overrides ## 1. Configuration File Basics ### File Name and Location Allure CLI looks for configuration files in the current working directory, in the following order (first file discovered gets parsed for configuration parameters): - `allurerc.js` - `allurerc.mjs` - `allurerc.cjs` - `allurerc.json` - `allurerc.yaml ` - `allurerc.yml` ### File Format and Syntax Allure 3 supports both dynamic configs (.mjs, .cjs, .js) and static configs (.json, .yml, .yaml). Static configs have limited functionality (no imports, no reading from environment variables, no executable code) but work with a global install of Allure Report 3. Dynamic config files are written in JavaScript. They require Allure 3 to be locally installed in your project and you can use the following in them: - Environment variables - Conditional logic - Computed values - Imported utilities Details: Dynamic and Static Configuration File Examples **allurerc.mjs:** ```javascript import { defineConfig } from "allure"; export default defineConfig({ name: "Allure Report", output: "./allure-report", historyPath: "./history.jsonl", qualityGate: { rules: [ { maxFailures: 5, fastFail: true, }, ], }, plugins: { awesome: { options: { reportName: "HelloWorld", singleFile: false, reportLanguage: "en", open: false, publish: true, }, }, allure2: { options: { reportName: "HelloWorld", singleFile: false, reportLanguage: "en", }, }, classic: { options: { reportName: "HelloWorld", singleFile: false, reportLanguage: "en", }, }, dashboard: { options: { singleFile: false, reportName: "HelloWorld-Dashboard", reportLanguage: "en", }, }, csv: { options: { fileName: "allure-report.csv", }, }, log: { options: { groupBy: "none", }, }, }, variables: { env_variable: "unknown", }, environments: { foo: { variables: { env_variable: "foo", env_specific_variable: "foo", }, matcher: ({ labels }) => labels.some(({ name, value }) => name === "env" && value === "foo"), }, bar: { variables: { env_variable: "bar", env_specific_variable: "bar", }, matcher: ({ labels }) => labels.some(({ name, value }) => name === "env" && value === "bar"), }, }, }); ``` **allurerc.json:** ```json { "name": "Allure Report", "output": "./allure-report", "historyPath": "./history.jsonl", "qualityGate": { "rules": [ { "maxFailures": 5, "fastFail": true } ] }, "plugins": { "awesome": { "options": { "reportName": "HelloWorld", "singleFile": false, "reportLanguage": "en", "open": false, "publish": true } }, "allure2": { "options": { "reportName": "HelloWorld", "singleFile": false, "reportLanguage": "en" } }, "classic": { "options": { "reportName": "HelloWorld", "singleFile": false, "reportLanguage": "en" } }, "dashboard": { "options": { "singleFile": false, "reportName": "HelloWorld-Dashboard", "reportLanguage": "en" } }, "csv": { "options": { "fileName": "allure-report.csv" } }, "log": { "options": { "groupBy": "none" } } }, "variables": { "env_variable": "unknown" } } ``` **allurerc.yaml:** ```yaml name: Allure Report output: ./allure-report historyPath: ./history.jsonl qualityGate: rules: - maxFailures: 5 fastFail: true plugins: allure2: options: reportName: HelloWorld singleFile: false reportLanguage: en classic: options: reportName: HelloWorld singleFile: false reportLanguage: en awesome: options: reportName: HelloWorld singleFile: false reportLanguage: en open: false publish: true dashboard: options: singleFile: false reportName: HelloWorld-Dashboard reportLanguage: en csv: options: fileName: allure-report.csv log: options: groupBy: none variables: env_variable: unknown ``` ### Dynamic Config File Capabilities and Prerequisites #### For Node.js Projects Add Allure as a development dependency to your project: ```bash cd your-project npm install -D allure ``` Then, wrap the configuration object with the `defineConfig` function: ```javascript import { defineConfig } from "allure"; export default defineConfig({ name: "My Allure Report", // ... configuration }); ``` #### For Other Types of Projects You can use a global installation of Allure Report and just export the configuration object directly: ```js export default { name: "My Allure Report", // ... configuration }; ``` This way, however, you won't get type hints. You can get around that limitation by initializing a dummy Node.js project and installing Allure to it: ```bash npm init --yes npm install -D allure ``` Then, wrap the configuration object in the `defineConfig` function, just as you would for any Node.js project (see the section above). #### Reading Environment Variables Since dynamic config files are executable JavaScript, you can use: ```javascript import { env } from "node:process"; export default { name: env.REPORT_NAME || "Default Report", output: env.CI ? "./ci-reports" : "./local-reports", }; ``` #### Importing Code When using dymanic files you can offload some executable logic to other JavaScript files and then import it to your config from them. You can also import code form other Node.js modules, including Allure 3 plugins. ### Default Top-Level Values Reference | Option | Default Value | | :-------------------- | :----------------------------------------------------------------------------------------------------------------------------------------- | | `name` | `"Allure Report"` | | `output` | `"./allure-report"` | | `open` | `false` | | `port` | `undefined` (random) | | `historyPath` | `undefined` | | `appendHistory` | `true` | | `historyLimit` | `undefined` (unlimited) | | `flakyDetection` | `undefined` (`historyDepth: 5`, `includePassedTests: false`) | | `categories` | `{ rules: [{ name: "Product errors", matchers: { statuses: ["failed"] } }, { name: "Test errors", matchers: { statuses: ["broken"] } }] }` | | `variables` | `{}` | | `environments` | `{}` | | `environment` | `undefined` | | `allowedEnvironments` | `undefined` | | `defaultLabels` | `{}` | | `hideLabels` | `undefined` | | `resolutions` | `undefined` | | `plugins` | `{ awesome: { options: {} } }` | | `qualityGate` | `{}` | | `performance` | `undefined` | | `dump` | `undefined` | | `globalAttachments` | `undefined` | ## 2. Basic Configuration Options ### `name` - **Type:** `string` - **Default:** `"Allure Report"` - **Description:** Display name for the report ```javascript { name: "My Test Suite Report" } ``` ### `output` - **Type:** `string` - **Default:** `"./allure-report"` - **Description:** Directory where the report will be generated ```javascript { output: "./reports/allure" } ``` ### `open` - **Type:** `boolean` - **Default:** `false` - **Description:** After generation completes, starts a local static file server and opens the report in the default browser. Use together with `port` to control the server port. ```javascript { open: true } ``` ### `port` - **Type:** `string` - **Default:** `undefined` (random available port) - **Description:** Port for the local server started when `open: true`. If omitted, a random available port is chosen. ```javascript { port: "3000" } ``` ### `hideLabels` - **Type:** `(string | RegExp)[]` - **Default:** `undefined` - **Description:** Hides labels from the report UI by label **name**. Does not filter tests out - only affects what is displayed. Labels whose name starts with `_` are always hidden regardless of this setting. ```javascript { hideLabels: ["package", "thread", /^allure_id/] } ``` ### `dump` - **Type:** `string` - **Default:** `undefined` - **Description:** When set, archives the processed test result state to a `.zip` file at the given path (`.zip` is appended automatically) **instead of** running report plugins. The archive can later be restored using the `--dump` flag on the `generate` CLI command. ```javascript { dump: "./snapshots/run-2025-10-20" // produces ./snapshots/run-2025-10-20.zip } ``` **Documentation:** See [Multistage Builds](/docs/multistage-builds/) for complete documentation. ### `globalAttachments` - **Type:** `string[]` - **Default:** `undefined` - **Description:** Glob patterns (relative to the working directory) for files to attach to the report globally, independent of any test result. Only files inside the working directory are included; paths resolving outside it are silently skipped. ```javascript { globalAttachments: ["./logs/*.log", "./screenshots/global/**/*.png"] } ``` **Documentation:** See [Global Errors and Attachments](/docs/global-errors-and-attachments/) for complete documentation. ## 3. History Options ### `historyPath` - **Type:** `string` - **Default:** `undefined` - **Description:** Path to the history file that tracks test results across multiple runs - **Format:** JSONL (JSON Lines) - each line contains data from one complete test run - **Used for:** - Trend charts showing test behavior over time - Flaky test detection - Historical comparisons ```javascript { historyPath: "./test-history/allure-history.jsonl" } ``` ### `appendHistory` - **Type:** `boolean` - **Default:** `true` - **Description:** Controls history accumulation behavior ```javascript { appendHistory: true } ``` ### `historyLimit` - **Type:** `number` - **Default:** `undefined` (unlimited) - **Description:** Caps how many past runs are considered. With `historyPath` (local history file), this prunes the file: when it exceeds this count, the oldest entries are dropped. With [`allureService`](#_11-report-publishing-allure-service) (Allure Report Storage), it instead caps how many previous entries are fetched, nothing is dropped remotely, older history remains stored and retrievable with a larger limit later. Has no effect if neither `historyPath` nor `allureService.accessToken` is set. ```javascript { historyLimit: 20 } ``` ### `flakyDetection` - **Type:** `{ historyDepth?: number; includePassedTests?: boolean }` - **Default:** `undefined` (the default values of both fields are used) - **Description:** Controls how Allure marks tests as [flaky](/docs/test-stability/#flaky-tests) based on their history. Has no effect if history is not available. - **Fields:** - `historyDepth` - the maximum number of previous results of a test that Allure analyzes. Only results with the **Passed**, **Failed**, or **Broken** status from the same environment are counted. Default: `5`. Set to `0` to analyze all available history, or to `-1` to disable history-based flaky detection. Must be an integer greater than or equal to `-1`. - `includePassedTests` - whether a test that passed in the current run can also be marked as flaky. Default: `false`, which means only tests with the **Failed** or **Broken** status can be marked. Tests that your Allure integration marks as flaky explicitly are always marked as flaky, regardless of these settings. ```javascript { flakyDetection: { historyDepth: 10, includePassedTests: true, }, } ``` **File Structure:** See [History File](/docs/how-it-works-history-files/#allure-report-3) document for details. ## 4. Categories Define custom categories to group and classify failed and broken tests by status, error message, labels, and other criteria. - **Type:** `{ rules: CategoryRule[] }` - **Default:** Two built-in categories - Product errors ([failed](https://allurereport.org/docs/test-statuses/#failed)) and Test errors ([broken](https://allurereport.org/docs/test-statuses/#broken)) - **Purpose:** Create a "Categories" tab in reports that groups test results by error type. All test results are matched against custom rules first, in the order defined; unmatched failed/broken tests fall back to the default categories. ### Structure ```js { categories: { rules: [ { name: "Category title", // Required id: "category-id", // Optional, stable identifier that allows to change name matchers: { ... } | (data) => boolean | [...], // Required, either object matcher or function matcher groupBy: ["severity", "owner"], // Optional, determines organization levels groupByMessage: true, // Optional, default: true groupEnvironments: true, // Optional expand: false, // Optional, default: false hide: false, // Optional, default: false } ] } } ``` You may also set `categories: false` to disable categories entirely (neither default categories nor custom rules will be applied). ### `matchers` `matchers` is mandatory and has to be one of the following: - **Object matcher** - a plain object with any combination of: - `statuses`: TestStatus[] - matches against [test status](/docs/test-statuses/), e.g. `["failed", "broken"]`. - `flaky`: boolean - determines whether the category should include [flaky](/docs/test-stability/#flaky-tests) tests. Requires [history](/docs/history-and-retries/#allure-report-3/) to work. - `labels`: { [labelName]: string | RegExp } - matches against [labels](/docs/how-it-works-test-result-file/#labels-array), as a regular expression. String input is converted into a RegExp object, so any regex metacharacters included in the string (like `.`, `*`, or `^`) will affect how the match behaves, e.g. `{ owner: /^alice/ }` will match all results where the `owner` label starts with `alice`. - `message`: (sub)string | RegExp - matches against the error message as a regular expression. Same as `labels`, any regex metacharacters in a provided string will affect the match. - `trace`: (sub)string | RegExp - matches against the stack trace as a regular expression. Same as `labels`, any regex metacharacters in a provided string will affect the match. - `transitions`: TestStatusTransition[] - matches against [transitions](/docs/test-stability/), e.g. `["new", "regressed"]`. Requires [history](/docs/history-and-retries/#allure-report-3/) to work. - `environments`: string[] - matches against [environments](/docs/environments/). Tests fitting all of these conditions will be included (AND logic applies here). - **Function matcher** - `(data) => boolean`, where `data` contains `status`, `labels`, `message`, `trace`, `flaky`, `duration`, `transition`, `environment`. Function matcher [requires dynamic config](/docs/v3/configure/#dynamic-config-file-capabilities-and-prerequisites) to work, and just like with object matchers, you have to enable [history](/docs/history-and-retries/#allure-report-3/) to match against `flaky` and `transition`, and set [environments](/docs/environments/) to match against `environment`. When using function matchers, you can additionally filter by `duration` (e.g. `(data) => data.duration > 5000` ), implement complex logic gates across multiple conditions and use arbitrary JavaScript expressions to fine-tune your categories. - **Array** - any mix of the above; the test matches if any matcher in the array matches (OR logic applies). ### `groupBy` `groupBy` is optional, controls the nesting hierarchy inside the category tree. Each element is either a built-in string selector or a custom label object: - Built-ins: `"flaky"`, `"owner"`, `"severity"`, `"transition"`, `"status"`, `"environment"`, `"layer"` - Custom labels: `{ label: "myCustomLabel" }` Example: `groupBy: ["severity", { label: "feature" }]` creates two levels of grouping - first by severity, then by feature - before showing individual test results. ### Example **Categories via object matchers:** ```js import { defineConfig } from "allure"; export default defineConfig({ name: "Allure Report 3", output: "./allure-report", historyPath: "./history.jsonl", //required for flaky and status transitions detection categories: { rules: [ { name: "Critical regressions by layer", id: "critical-regressions-by-layer", // this section determines what's included into the category matchers: { statuses: ["failed", "broken"], flaky: false, labels: { severity: "critical" }, message: /expected|API/, // accepts regular expressions trace: /AssertionError|timeout|unexpected/, // accepts regular expressions transitions: ["regressed", "malfunctioned"], environments: ["staging", "production"], }, // this section determines how the category is presented groupBy: ["layer", "owner", "status"], groupByMessage: true, groupEnvironments: true, expand: true, hide: false, }, ], }, }); ``` **Categories via function matchers:** ```js import { defineConfig } from "allure"; export default defineConfig({ name: "Allure Report 3", output: "./allure-report", historyPath: "./history.jsonl", //required for flaky and status transitions detection categories: { rules: [ { name: "Critical regressions by layer", id: "critical-regressions-by-layer", // this section determines what's included into the category matchers: (data) => (data.status === "failed" || data.status === "broken") && !data.flaky && data.labels.some((l) => l.name === "severity" && l.value === "critical") && /expected|API/.test(data.message ?? "") && /AssertionError|timeout|unexpected/.test(data.trace ?? "") && (data.transition === "regressed" || data.transition === "malfunctioned") && (data.environment === "staging" || data.environment === "production"), // this section determines how the category is presented groupBy: ["layer", "owner", "status"], groupByMessage: true, groupEnvironments: true, expand: true, hide: false, }, ], }, }); ``` **Documentation:** See [Categories Guide](/docs/categories/) for complete documentation. ## 5. Variables and Labels ### `variables` Display custom key-value metadata at the top of the report. - **Type:** `Record` - **Default:** `{}` - **Where shown:** Top of report in awesome plugin - **Purpose:** Display build/run metadata #### Example ```javascript { variables: { "App Version": "2.5.1", "Test Suite": "Regression v1.2", "Launch Date": "2025-10-20", "Build Number": "#1234", "Environment": "Staging" } } ``` ### `defaultLabels` Apply default labels to tests. - **Type:** `Record` - **Default:** `{}` - **Behavior:** Non-overriding - only applies if test doesn't have that label - **Purpose:** Provide fallback values for missing labels #### Example ```javascript { defaultLabels: { "severity": "normal", "owner": "unassigned", "layer": "unknown", "tags": ["needs-review"] } } ``` #### How It Works - Test with `severity: "critical"` → keeps `"critical"` (not overridden) - Test without `severity` → gets `"normal"` from defaults - Test without `tags` → gets `["needs-review"]` - Test with `tags: ["smoke"]` → keeps `["smoke"]` #### Label Values **Single value:** ```javascript "severity": "critical" ``` **Multiple values (array):** ```javascript "tags": ["smoke", "regression", "api"] ``` ## 6. Environments Group and categorize test results by environment with custom matching logic. - **Type:** `Record` - **Default:** `{}` - **Purpose:** Create an "Environments" menu in reports and split test result tree into environment-based groups, which you can switch between using that menu. ### Structure ```javascript { environments: { "environment-id": { // Key = ID: letters, digits, underscores, hyphens only name?: string, // Optional display name shown in the report; defaults to the ID matcher: ({ labels }) => boolean, // Required variables?: Record // Optional } } } ``` ### Matcher Function Receives test labels and returns `true` if the test belongs to this environment. Requires dynamic config to work. **Payload structure:** ```javascript { labels: [ { name: "os", value: "Windows" }, { name: "browser", value: "Chrome" }, // ... more labels ] } ``` **Note**: these labels should be set in the tests as there are no default values for environment labels. ### Example **OS-based Environments:** ```javascript { environments: { windows: { matcher: ({ labels }) => labels.find(({ name, value }) => name === "os" && value === "Windows"), variables: { "OS": "Windows 11", "Architecture": "x64" } }, macos: { matcher: ({ labels }) => labels.find(({ name, value }) => name === "os" && value === "macOS"), variables: { "OS": "macOS Sonoma", "Architecture": "arm64" } }, linux: { matcher: ({ labels }) => labels.find(({ name, value }) => name === "os" && value === "Linux"), variables: { "OS": "Ubuntu 22.04", "Architecture": "x64" } } } } ``` **Deployment Environments:** ```javascript { environments: { staging: { matcher: ({ labels }) => labels.find(l => l.name === "env" && l.value === "staging"), variables: { "Server": "staging.example.com", "Database": "staging-db" } }, production: { matcher: ({ labels }) => labels.find(l => l.name === "env" && l.value === "production"), variables: { "Server": "api.example.com", "Database": "prod-db-primary" } } } } ``` ### Environment Variables Optional environment-specific variables override or extend global variables when viewing that environment. **Merge behavior:** 1. Global `variables` shown in all views 2. Environment-specific `variables` override/add to globals when that environment is selected ### Use Cases - Group tests by OS (Windows/macOS/Linux) - Group by browser (Chrome/Firefox/Safari) - Group by deployment environment (staging/production) - Group by any custom label-based criteria ### `environment` **Note**: This is a top-level option. - **Type:** `string` - **Default:** `undefined` - **Description:** Force-assigns a single environment ID to all test results in this run, bypassing the matcher logic in `environments`. Use this when you don't need dynamic label-based matching and already know which environment you're running against. Takes precedence over any matched environment. ```javascript { environment: "staging" } ``` ### `allowedEnvironments` **Note**: This is a top-level option. - **Type:** `string[]` - **Default:** `undefined` - **Description:** Restricts which environment IDs are considered valid. When set, any environment ID - from the `environments` map, the `environment` override, or test result data - that is not in this list will cause a validation error at startup. Use this to enforce a fixed set of known environments and catch misconfigurations early. ```javascript { allowedEnvironments: ["staging", "production", "windows", "macos"] } ``` **Documentation:** See [Environments Guide](/docs/environments/) for complete documentation. ## 7. Resolutions Mark known, muted and accepted test failures, and link them to issue trackers. Resolved failures are highlighted in the report, and muted and accepted ones are ignored by quality gates. ### `resolutions` - **Type:** `object` - **Default:** `undefined` - **Description:** Resolution rules, issue link templates, and the path of the known issues file ```javascript { resolutions: { links: { jira: { urlTemplate: "https://jira.example.com/browse/%s", nameTemplate: "JIRA %s" }, }, rules: [ { resolution: "issue", issue: { id: "SHOP-42", type: "jira" }, messageRegexp: "Connection timeout", comment: "Payment gateway sandbox is slow", }, { resolution: "muted", testCaseId: ["103"], comment: "Muted until the widget redesign lands", }, ], }, } ``` | Field | Description | | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `rules` | Required. List of rules. Each rule has a `resolution` (`issue`, `muted` or `accepted`), one or more matchers (`messageRegexp`, `testCaseId`, `retryHash`, `environment`), an `issue` (for `issue` rules) and a `comment` (required for `muted` and `accepted` rules). | | `links` | Templates for issue links, by issue type: `urlTemplate` (required) and `nameTemplate`, each with exactly one `%s` placeholder for the issue ID. | | `knownIssuesPath` | Path of the known issues file that Allure writes after generating the report. Default: `known-issues.json` in the current working directory. Can be overridden with the `--known-issues` option. | **Documentation:** See [Resolutions](/docs/resolutions/) for complete documentation. ## 8. Plugin Configuration ### Plugin System Overview Allure 3 uses a plugin-based architecture. Each plugin generates a specific type of report or provides additional functionality. **Available Official Plugins:** - `awesome` - Modern UI (recommended) - `classic` - Classic-style UI - `allure2` - Allure 2-style UI - `dashboard` - Dashboard view with charts - `log` - Console log output - `csv` - CSV export - `slack` - Slack integration - `testplan` - Test plan generation - `jira` - Jira integration You can find the official plugins code in the [Allure 3 repository](https://github.com/allure-framework/allure3/tree/main/packages). ### Basic Plugin Configuration To enable a plugin in your project, install the plugin package: ``` npm add @allurereport/plugin-name ``` And add it to the `plugins` section of the config: ```javascript { plugins: { "name": { options: { // plugin-specific options } } } } ``` ### Plugin Configuration Properties #### `import` (optional) Custom module path for the plugin ```javascript { plugins: { "my-custom-id": { import: "@allurereport/plugin-awesome", options: {} } } } ``` **Use case:** Running multiple instances of the same plugin with different configurations #### `enabled` (optional) Enable or disable the plugin - **Type:** `boolean` - **Default:** `true` ```javascript { plugins: { awesome: { enabled: false, // Disable this plugin options: {} } } } ``` #### `options` (optional) Plugin-specific configuration object - **Type:** `Record` - **Default:** `{}` ```javascript { plugins: { awesome: { options: { singleFile: true, reportLanguage: "en" } } } } ``` ### Multiple Reports You can generate multiple reports by configuring multiple plugins: ```javascript { plugins: { "awesome-framework": { import: "@allurereport/plugin-awesome", options: { reportName: "Framework View", groupBy: ["framework", "language"] } }, "awesome-package": { import: "@allurereport/plugin-awesome", options: { reportName: "Package View", groupBy: ["package", "suite"] } }, dashboard: { options: { reportName: "Dashboard" } } } } ``` ### Default Plugin If no plugins are specified, Allure uses the `awesome` plugin by default: ```javascript // These are equivalent: export default { name: "Report" } export default { name: "Report", plugins: { awesome: { options: {} } } } ``` ## 9. Plugin Options ### Awesome Plugin **Recommended modern UI for Allure reports.** #### Common Options **`reportName`** (string) - Overrides the top-level `name` for this report **`singleFile`** (boolean, default: `false`) - Generate report as a single standalone HTML file - `true`: Single HTML file - `false`: Multiple files **`reportLanguage`** (default: `"en"`) - Sets the UI language **`groupBy`** (string[]) - Defines test result hierarchy organization - Array of label names **Common patterns:** ```javascript // BDD-style hierarchy groupBy: ["epic", "feature", "story"]; // Suite-based hierarchy groupBy: ["parentSuite", "suite", "subSuite"]; // Package-based hierarchy groupBy: ["package", "testClass", "testMethod"]; // Module-based groupBy: ["module", "parentSuite", "suite", "subSuite"]; // Custom hierarchy groupBy: ["framework", "language", "package"]; ``` **`filter`** (function) (Requires dynamic config) - Determines which test results are included in the report. - Receives a test result object; return a truthy value to include, falsy to exclude **Common patterns:** ```javascript // Include only tests with a specific label filter: ({ labels }) => labels.find(({ name, value }) => name === "framework" && value === "playwright"); // Exclude tests with a specific label filter: ({ labels }) => !labels.find(({ name, value }) => name === "language" && value === "java"); ``` **`charts`** (default: all charts included) - Built-in charts configuration: charts configured here will be included in the Awesome report itself. Do not confuse with [Dashboard](#dashboard-plugin), which is a separate charts-only report type. - For details on what the charts illustrate, please refer to the [full Charts documentation](/docs/visual-analytics/). - To populate most charts properly, [history](#_3-history-options) should be enabled. - If `charts` is omitted, all available chart types are included with their default options, which you can review below: Details: Full charts configuration example with all default options ```js // Charts configuration with all chart types and all options set to default values const allChartsDefaultValues = [ { type: "currentStatus", title: "Current status", statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses metric: "passed", // status used to calculate the central percentage value }, { type: "testResultSeverities", title: "Test results by severities", levels: ["blocker", "critical", "normal", "minor", "trivial"], //included severities statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses includeUnset: true, //whether to show the “No severity“ section }, { type: "statusDynamics", title: "Status dynamics", limit: 10, // number of shown runs, including the latest statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses }, { type: "statusTransitions", title: "Status transitions", limit: 10, // number of shown runs, including the latest }, { type: "testBaseGrowthDynamics", title: "Test base growth dynamics", statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses limit: 10, // number of shown runs, including the latest }, { type: "coverageDiff", title: "Coverage diff map", }, { type: "successRateDistribution", title: "Success rate distribution", }, { type: "problemsDistribution", title: "Problems distribution by environment", by: "environment", //can only be grouped by environments }, { type: "stabilityDistribution", title: "Stability distribution by features", threshold: 90, // acceptable stability level in % skipStatuses: ["skipped", "unknown"], //skipped test statuses groupBy: "feature", //by what label to group the tests together }, { type: "stabilityDistribution", title: "Stability distribution by epics", threshold: 90, // acceptable stability level in % skipStatuses: ["skipped", "unknown"], //skipped test statuses groupBy: "epic", //by what label to group the tests together }, { type: "stabilityDistribution", title: "Stability distribution by stories", threshold: 90, // acceptable stability level in % skipStatuses: ["skipped", "unknown"], //skipped test statuses groupBy: "story", //by what label to group the tests together }, { type: "durations", title: "Durations histogram", groupBy: "none", //whether to group the tests by layer (other labels not supported) }, { type: "durations", title: "Durations by layer histogram", groupBy: "layer", //whether to group the tests by layer (other labels not supported) }, { type: "durationDynamics", title: "Durations dynamics", limit: 10, // number of shown runs, including the latest }, { type: "statusAgePyramid", title: "Status age pyramid", limit: 10, // number of shown runs, including the latest }, { type: "testingPyramid", title: "Testing pyramid", layers: ["unit", "integration", "e2e"], //layers of the pyramid, bottom to top }, ]; // charts configuration used within the Awesome report charts: allChartsDefaultValues, ``` To change only some of the charts, start from the default list instead of copying it: the `allure` package exports it as `defaultChartsConfig`. For example, this configuration removes the coverage diff map and adds a stability distribution chart grouped by owner: ```javascript import { defaultChartsConfig, defineConfig } from "allure"; const charts = [ // all default charts except the coverage diff map ...defaultChartsConfig.filter((chart) => chart.type !== "coverageDiff"), // plus one more stability distribution chart { type: "stabilityDistribution", title: "Stability distribution by owners", threshold: 90, skipStatuses: ["skipped", "unknown"], groupBy: "owner", }, ]; export default defineConfig({ plugins: { awesome: { options: { charts } }, }, }); ``` #### Example Configuration ```javascript { plugins: { awesome: { options: { singleFile: false, reportLanguage: "en", reportName: "My Awesome Report", groupBy: ["epic", "feature", "story"], filter: ({ labels }) => labels.find(({ name, value }) => name === "framework" && value === "playwright") } } } } ``` #### Complete Options Reference For all available options including experimental features, see: - [AwesomePluginOptions source code](https://github.com/allure-framework/allure3/blob/main/packages/plugin-awesome/src/model.ts) ### Dashboard Plugin **Dashboard view with charts and visualizations.** #### Common Options **`reportName`** (string) - Dashboard title **`singleFile`** (boolean, default: `false`) - Generate as single file **`reportLanguage`** (default: `"en"`) - UI language **`layout`** (array of widget configurations) - Dashboard report charts configuration - For details on what different charts illustrate, please refer to the [full Charts documentation](/docs/visual-analytics/). - To populate most charts properly, [history](#_3-history-options) should be enabled. - If `layout` is omitted, all available chart types are included with their default options, which you can review below: - To change only some of the charts, start from `defaultChartsConfig`, the same way as for the [`charts`](#awesome-plugin) option of the Awesome report. **`filter`** (function) (Requires dynamic config) - Determines which test results are included in the dashboard. Receives a test result object; return a truthy value to include, falsy to exclude. #### Example Configuration Details: Full charts configuration example with all default options ```js // Charts configuration with all chart types and all options set to default values const allChartsDefaultValues = [ { type: "currentStatus", title: "Current status", statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses metric: "passed", // status used to calculate the central percentage value }, { type: "testResultSeverities", title: "Test results by severities", levels: ["blocker", "critical", "normal", "minor", "trivial"], //included severities statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses includeUnset: true, //whether to show the “No severity“ section }, { type: "statusDynamics", title: "Status dynamics", limit: 10, // number of shown runs, including the latest statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses }, { type: "statusTransitions", title: "Status transitions", limit: 10, // number of shown runs, including the latest }, { type: "testBaseGrowthDynamics", title: "Test base growth dynamics", statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses limit: 10, // number of shown runs, including the latest }, { type: "coverageDiff", title: "Coverage diff map", }, { type: "successRateDistribution", title: "Success rate distribution", }, { type: "problemsDistribution", title: "Problems distribution by environment", by: "environment", //can only be grouped by environments }, { type: "stabilityDistribution", title: "Stability distribution by features", threshold: 90, // acceptable stability level in % skipStatuses: ["skipped", "unknown"], //skipped test statuses groupBy: "feature", //by what label to group the tests together }, { type: "stabilityDistribution", title: "Stability distribution by epics", threshold: 90, // acceptable stability level in % skipStatuses: ["skipped", "unknown"], //skipped test statuses groupBy: "epic", //by what label to group the tests together }, { type: "stabilityDistribution", title: "Stability distribution by stories", threshold: 90, // acceptable stability level in % skipStatuses: ["skipped", "unknown"], //skipped test statuses groupBy: "story", //by what label to group the tests together }, { type: "durations", title: "Durations histogram", groupBy: "none", //whether to group the tests by layer (other labels not supported) }, { type: "durations", title: "Durations by layer histogram", groupBy: "layer", //whether to group the tests by layer (other labels not supported) }, { type: "durationDynamics", title: "Durations dynamics", limit: 10, // number of shown runs, including the latest }, { type: "statusAgePyramid", title: "Status age pyramid", limit: 10, // number of shown runs, including the latest }, { type: "testingPyramid", title: "Testing pyramid", layers: ["unit", "integration", "e2e"], //layers of the pyramid, bottom to top }, ]; // charts configuration used in a Dashboard report layout: allChartsDefaultValues, ``` #### Complete Options Reference For all available options, see: - [DashboardPluginOptions source code](https://github.com/allure-framework/allure3/blob/main/packages/plugin-dashboard/src/model.ts) **Note:** Dashboard generates a **standalone report** separate from other plugins. ### Allure2 Plugin **Allure 2-style UI** #### Common Options **`reportName`** (string) - Overrides the top-level `name` for this report **`singleFile`** (boolean, default: `false`) - Generate report as a single standalone HTML file **`reportLanguage`** (default: `"en"`) - Sets the UI language **`createdAt`** (number) - Report creation timestamp in milliseconds since epoch. Defaults to the current time if not set. #### Complete Options Reference For all available options, see: - [Allure2PluginOptions source code](https://github.com/allure-framework/allure3/blob/main/packages/plugin-allure2/src/model.ts) ### Classic Plugin **Classic UI.** #### Common Options **`reportName`** (string) **`reportLanguage`** (default: `"en"`) **`singleFile`** (boolean) #### Complete Options Reference For all available options, see: - [ClassicPluginOptions source code](https://github.com/allure-framework/allure3/blob/main/packages/plugin-classic/src/model.ts) ### CSV Plugin **Exports test results as a CSV file.** #### Options **`fileName`** (string, default: `"allure-report.csv"`) - Output file name for the CSV export **`separator`** (string, default: `","`) - Field separator character **`disableHeaders`** (boolean, default: `false`) - When `true`, omits the header row from the output **`fields`** (array) (Requires dynamic config) - Custom column definitions. Each entry has a `header` string and an `accessor` - either a key of the test result object or a function `(result) => string`. **`sort`** (function) (Requires dynamic config) - Sort function applied to test results before writing. Receives two test result objects; follows standard `Array.sort` comparison conventions. **`filter`** (function) (Requires dynamic config) - Determines which test results are included. Receives a test result object; return a truthy value to include, falsy to exclude. #### Example Configuration ```javascript { plugins: { csv: { options: { fileName: "results.csv", separator: ";", filter: ({ status }) => status === "failed", fields: [ { header: "Name", accessor: "name" }, { header: "Status", accessor: "status" }, { header: "Duration (ms)", accessor: ({ duration }) => String(duration ?? "") } ] } } } } ``` #### Complete Options Reference For all available options, see: - [CsvPluginOptions source code](https://github.com/allure-framework/allure3/blob/main/packages/plugin-csv/src/model.ts) ### Log Plugin **Prints test results to the console.** #### Options **`groupBy`** (`"suite" | "feature" | "package" | "suites" | "features" | "packages" | "none"`, default: `"suite"`) - Groups console output by the specified hierarchy before listing test results **`allSteps`** (boolean, default: `false`) - Show all steps in the output. By default only steps from failed tests are shown. **`withTrace`** (boolean, default: `false`) - Include stack traces for failed tests **`filter`** (function) (Requires dynamic config) - Determines which test results are printed. Receives a test result object; return a truthy value to include, falsy to exclude. **`qualityGateResults`** (boolean, default: `true`) - Print the [quality gate](/docs/quality-gate/) results after the test results **`qualityGateFilter`** (function, default: only failed rules) (Requires dynamic config) - Determines which quality gate results are printed. Receives a quality gate result object with the `success`, `rule`, `message`, `actual` and `expected` fields; return a truthy value to include, falsy to exclude. For example, `() => true` prints the passed rules too. #### Example Configuration ```javascript { plugins: { log: { options: { groupBy: "suites", allSteps: false, withTrace: true, filter: ({ status }) => status === "failed" || status === "broken" } } } } ``` ### Slack Plugin **Posts a test results summary to a Slack channel.** #### Options **`token`** (string) - Slack Bot User OAuth Token. Can also be provided via the `ALLURE_SLACK_TOKEN` environment variable. **`channel`** (string) - Slack channel ID to post to. Can also be provided via the `ALLURE_SLACK_CHANNEL` environment variable. The plugin posts a message containing a breakdown of test counts by status (failed, broken, passed, skipped, unknown) and a list of failed test names. #### Example Configuration ```javascript import { env } from "node:process"; export default { plugins: { slack: { options: { token: env.ALLURE_SLACK_TOKEN, channel: env.ALLURE_SLACK_CHANNEL, }, }, }, }; ``` #### Complete Options Reference For all available options, see: - [SlackPlugin source code](https://github.com/allure-framework/allure3/blob/main/packages/plugin-slack/src/plugin.ts) ### TestPlan Plugin **Generates a `testplan.json` file for re-running a subset of tests.** The file lists the tests that matched the filter - by default, all unsuccessful tests from the run. It can be passed to a supported test framework to re-execute only those tests. #### Options **`fileName`** (string, default: `"testplan.json"`) - Output file name **`filter`** (function) (Requires dynamic config) - Determines which test results are included in the test plan. Defaults to all unsuccessful (failed, broken, or otherwise non-passing) tests. #### Example Configuration ```javascript { plugins: { testplan: { options: { fileName: "rerun.json", filter: ({ status }) => status === "failed" } } } } ``` #### Complete Options Reference For all available options, see: - [TestPlanPlugin source code](https://github.com/allure-framework/allure3/blob/main/packages/plugin-testplan/src/plugin.ts) ### TestOps Plugin **Uploads test results to Allure TestOps.** #### Options **`endpoint`** (string, required) - Allure TestOps instance URL **`accessToken`** (string, required) - API access token for authentication **`projectId`** (string, required) - Target project ID in Allure TestOps **`launchName`** (string, required) - Display name for the launch created in TestOps **`launchTags`** (string[], required) - Tags to attach to the launch **`autocloseLaunch`** (boolean, default: `false`) - Automatically close the launch in TestOps after upload completes **`limit`** (number, default: unlimited) - Maximum number of test results to upload per batch **`filter`** (function) (Requires dynamic config) - Determines which test results are uploaded. Receives a test result object; return a truthy value to include, falsy to exclude. #### Example Configuration ```javascript import { env } from "node:process"; export default { plugins: { testops: { import: "@allurereport/plugin-testops", options: { endpoint: env.TESTOPS_URL, accessToken: env.TESTOPS_TOKEN, projectId: env.TESTOPS_PROJECT_ID, launchName: "Regression Suite", launchTags: ["regression", "nightly"], autocloseLaunch: true, }, }, }, }; ``` #### Complete Options Reference For all available options, see: - [TestOpsPluginOptions source code](https://github.com/allure-framework/allure3/blob/main/packages/plugin-testops/src/model.ts) ## 10. Quality Gate Controls the process exit code based on test result criteria, allowing test runs to pass or fail based on custom rules. ```javascript { qualityGate: { rules: [ { maxFailures: 5, }, ] } } ``` **Documentation:** See [Quality Gate Guide](/docs/quality-gate/) for complete documentation. **Note:** This is a complex feature with its own dedicated documentation. ## 11. Report Publishing (Allure Service) Connects Allure 3 to [Allure Report Storage](/docs/self-hosted-storage/) (or a compatible instance), so reports can be published and their history retrieved automatically after each run, instead of managing history files or hosting yourself. `allureService.accessToken` supplies the access token; `publish: true` on a plugin's `options` marks that plugin's report for publishing. ```javascript { plugins: { awesome: { options: { publish: true, }, }, }, allureService: { accessToken: "ars1...", }, } ``` **Documentation:** See [Allure Report Storage](/docs/self-hosted-storage/) for complete documentation, including how to generate an access token and run the service yourself. ## 12. Performance Metrics Titles, units, groups and the preferred direction of the [performance metrics](/docs/performance-metrics/) collected during the test run. ### `performance` - **Type:** `object` - **Default:** `undefined` - **Description:** `groups` defines the metric groups (`title` for each group ID); `metrics` defines the settings for each metric key: `title`, `unit`, `better` (`"lower"` or `"higher"`) and `group` ```javascript { performance: { groups: { web: { title: "Web pages" }, }, metrics: { "page.load": { title: "Page load", unit: "ms", better: "lower", group: "web" }, }, }, } ``` **Documentation:** See [Performance Metrics](/docs/performance-metrics/) for complete documentation. ## CLI Configuration Overrides Config options can be overridden from the command line: ```bash allure generate \ --config ./allurerc.mjs \ --output ./custom-output \ --name "Overridden Name" ``` **Overridable options:** - `--name` / `--report-name` - `--output` / `-o` - `--known-issues` (path of the [known issues file](/docs/resolutions/#known-issues-file)) - Plugin-specific options (varies by command) ### Plugin-Specific Commands Generate reports with specific plugins directly: #### Awesome Report ```bash allure awesome [options] ``` **Options:** - `--single-file` - `--logo` - `--theme` - `--report-language, --lang` - `--group-by, -g` #### Dashboard ```bash allure dashboard [options] ``` #### Classic Report ```bash allure classic [options] ``` #### CSV Export ```bash allure csv [options] ``` #### Log Output ```bash allure log [options] ``` ### Utility Commands #### History ```bash allure history [--history-path,-h] [--report-name,--name] ``` Generates history to specified folder. #### Test Plan ```bash allure testplan [--output,-o] ``` Generates testplan.json. ### Help Get detailed help for any command: ```bash allure --help ``` --- ## Appendix: Type Definitions Reference For reference, here are the main TypeScript interfaces that define the configuration structure: ### Config Interface ```typescript interface Config { name?: string; output?: string; historyPath?: string; resolutions?: ResolutionsConfig; performance?: PerformanceConfig; defaultLabels?: DefaultLabelsConfig; environments?: EnvironmentsConfig; variables?: ReportVariables; plugins?: Record; appendHistory?: boolean; qualityGate?: QualityGateConfig; allureService?: { accessToken?: string; }; // see Section 11: Report Publishing (Allure Service) } ``` ### Supporting Types ```typescript type DefaultLabelsConfig = Record; type ReportVariables = Record; type EnvironmentDescriptor = { name?: string; variables?: ReportVariables; matcher: (payload: { labels: TestLabel[] }) => boolean; }; type EnvironmentsConfig = Record; interface PluginDescriptor { import?: string; enabled?: boolean; options?: Record; } interface ResolutionsConfig { knownIssuesPath?: string; links?: Record; rules: ResolutionRule[]; } interface ResolutionMatcher { messageRegexp?: string; testCaseId?: string[]; retryHash?: string[]; environment?: string[]; } type ResolutionRule = | (ResolutionMatcher & { resolution: "issue"; issue: { id: string; type: string }; comment?: string; }) | (ResolutionMatcher & { resolution: "muted" | "accepted"; comment: string; }); interface PerformanceConfig { groups?: Record; metrics?: Record< string, { title: string; unit: string; better: "lower" | "higher"; group?: string } >; } ``` ## Additional Resources - [Allure 3 GitHub Repository](https://github.com/allure-framework/allure3) - [Charts and Visual analytics documentation](/docs/visual-analytics/) - [Environments Feature](/docs/environments/) - [Quality Gate Feature](/docs/quality-gate/) - [Resolutions Feature](/docs/resolutions/) - [Performance Metrics Feature](/docs/performance-metrics/) --- ## Introduction > URL: https://allurereport.org/docs/v3/ # Allure Report 3 Images: /images/allure3_main_light.png, /images/allure3_main_dark.png **Allure Report 3** is the newest version of the Allure reporting framework, rebuilt from the ground up with a new architecture and expanded capabilities. Like its predecessor, Allure 2, it supports many languages and frameworks and can be added to your testing workflow with little to zero configuration. It produces visually rich HTML reports based on your test results that can be opened anywhere and read by anyone, no deep technical knowledge required. Unlike its predecessor, this version is developed in TypeScript and introduces a modular plugin system for greater flexibility. A key addition is the Awesome plugin, which delivers an enhanced UI for better report visualization. ## What's New in Allure 3 Images: /images/allure3/multiple-reports-light.png, /images/allure3/multiple-reports-dark.png Allure 3 introduces several notable improvements: - The framework has been entirely rewritten in TypeScript, making it more extensible and easier to maintain. Its plugin system allows you to customize and extend reporting functionality to fit your specific needs. - Configuration is simpler: a single file manages all report settings, making it more convenient to handle multiple reports. - Reruns in case of test failure are now supported. The results of the reruns are all grouped together in a single report. - An all-new environments system that lets you create multiple-environment reports and link together tests from different environments. - You can now mark known, muted and accepted test failures with [resolutions](/docs/resolutions/) and link them to your issue tracker. - Allure 3 adds real-time reporting, which lets you view live updates during test execution using the watch command. - Allure 3 supports project-wide quality gate settings, enabling fast fails of test runs that generated a critical amount of failed tests. - Allure 3 can collect [performance metrics](/docs/performance-metrics/), such as page load times, show their trends across runs, and check them with quality gates. - Allure 3 introduces the [Allure Storage Service](/docs/self-hosted-storage/) for the first time - a built-in cloud storage for your test history. - Certain previously standalone test result readers are becoming part of Allure's core functionality with version 3. ## Get started with Allure - **Try the demo:** Play around with the different kinds of reports Allure 3 can offer in the [demo repository](https://github.com/allure-framework/allure3-demo). - **The quick start:** [**install Allure Report**](/docs/v3/install/) and the [integration for your framework](/docs/frameworks/), run your tests the same way as you do it now, make sure you've got results in the supported formats, then and [**run Allure Report**](/docs/v3/generate-report/) to turn those results into a beautiful HTML report. - **Ready for more?** Learn to use the API provided by the [integrations](/docs/frameworks/) to enable advanced Allure features like [steps](/docs/steps/), [attachments](/docs/attachments/), [test hierarchies](/docs/v3/navigation/), and more. - **Need a bigger picture?** Consider using [**Allure TestOps**](https://qameta.io/) - our paid offering that allows you to track both manual and automated test cases, aggregate test results from a wide range of sources and analyze them across multiple launches and projects. ## Control your data flow A testing workflow with Allure Report consists of two steps, both of which can be done locally, without sending anything over the wire. 1. **Collection phase**: while the tests are running, the test framework writes their results into a file or directory. 2. **Visualization phase**: the Allure Report command-line utility reads the test results and builds an HTML report. Once the HTML report is generated, you can safely remove the test results - for example, as part of a cleaning procedure before the next test run. ![How Allure Report works: the collection phase and the visualization phase](/images/diagrams/how-it-works-simplified.png "Allure Report The Collection Phase And The Visualization Phase") Allure recognizes test results in many formats, including [our own data format](/docs/how-it-works-test-result-file/). Its support can be added to many popular frameworks by using [Allure integrations](/docs/frameworks/), and some frameworks even support it natively. In this data format, each test result is stored as a separate JSON file with a randomly generated filename. The Allure's data format and the file-based data flow is: - **parallelizable**: you can run tests on multiple nodes as long as you merge all the results into a single directory before the visualization phase; - **framework-agnostic**: the framework-agnostic data format is perfect for projects that span across multiple programming languages and test frameworks; - **failure-tolerant**: you can build a report based on the test results that you have even if other tests did not write their files due to an unexpected issue; - **extensible**: feel free to filter or modify the test results between the two phases, thanks to the well-documented data format. See [How it works](/docs/how-it-works/) for a more detailed overview of the typical data flow in Allure Report. ## Organize reports your way A test report built by Allure is not locked to a specific service - it is just a small static HTML website. You can open a report on any computer using the Allure Report utility itself or even just a browser. This gives you maximum flexibility when in comes to sharing or publishing the report. - Send a report as a ZIP archive to your colleague that needs to see it. - Store the reports on your web server, with [links to previous reports](/docs/history-and-retries/). - Use the [integration plugins](/docs/integrations/) to embed reports into your CI tool's interface. - Publish automatically and keep history with the [Allure Storage Service](/docs/self-hosted-storage/). For more information, see [How to view a report](/docs/v3/view-report/). ## The Allure ecosystem The Allure Report utility itself is a part of a larger ecosystem, supported by both [Qameta Software](https://qameta.io/about-us/) and the broad community. The ecosystem includes: - [**Integrations**](/docs/frameworks/) that enable support for the Allure data format in various test framework. Using an Allure integration for your framework is recommended to enable more features, such as [steps](/docs/steps/), [attachments](/docs/attachments/), and [test hierarchies](/docs/v3/navigation/). - **Library integrations** for logging certain type of events during the test, e.g., HTTP requests, database queries, assertions, etc. - Coming soon to Allure 3: **IDE integrations** for building reports from [JetBrains IDEs](/docs/integrations-jetbrains/) or [Visual Studio Code](/docs/integrations-vscode/). - Coming soon to Allure 3: [**CI/CD integrations**](/docs/integrations/) for building and publishing reports as part of your pipeline. - [**Allure TestOps**](https://qameta.io/) - a paid Allure Report replacement that manages test results across multiple launches for even deeper analysis. Most of the tools (with the exception of Allure TestOps) are open source under Apache License 2.0. --- ## Installation > URL: https://allurereport.org/docs/v3/install/ # Install or upgrade Allure Report There is one recommended installation method for Allure Report 3. - [Install Allure Report 3 for Node.js](/docs/v3/install/) (any system) Note that Allure Report 3 is actively developed, and a lot of new features are introduced in newer versions. If you installed Allure Report a while ago, consider [upgrading it](/docs/v3/upgrade/). --- ## Upgrade Allure Report 3 > URL: https://allurereport.org/docs/v3/upgrade/ # Upgrade Allure Report We recommend to use the latest version of the Allure Report command-line utility. To upgrade your Allure Report installation to the newest version, follow the instructions below. Select the same method you used when installing the previous version. Tip: **Which version do I have now?** To find out the current installed version of Allure Report, open the terminal and run `allure --version`. If it displays the same version number as the latest Allure Report release, you do not need to upgrade for now. We also recommend running this command after an upgrade to make sure it was successful. ## Upgrade from a Node.js package [![Allure Report 3 npm latest version](https://img.shields.io/npm/v/allure?style=flat "Allure Report 3 npm latest version")](https://www.npmjs.com/package/allure) If you installed Allure Report [from a Node.js package](/docs/v3/install/) for your project, upgrade it via `npm`. **Global installation:** ``` npm update -g allure ``` **Project-specific installation:** ``` npm update allure ``` --- ## Migration from Allure Report 2 > URL: https://allurereport.org/docs/v3/migrate/ # Migrate from Allure Report 2 Allure Report 3 has been rebuilt from the ground up to offer an improved interface, more flexibility, and new long-requested features, while maintaining the maximum possible compatibility with test setups built for Allure Report 2. This guide will help you transition your projects smoothly. Tip: You will not have to change anything in the actual source code of your tests. Allure 3 maintains full compatibility with all official [Allure integrations](/docs/frameworks/). The only thing that changes is the CLI report generator itself. ## Install ### For Node.js Projects Add Allure 3 as a development dependency: ```bash cd your-project npm install -D allure ``` Run it like so: ```bash npx allure ``` ### For Non-Node.js Projects You have two options: **Option 1: Global Installation** Install Allure globally for command-line access: ```bash npm install -g allure ``` Run like any globally installed tool: ```bash allure ``` Tip: If you already have Allure Report 2 installed globally, you may encounter naming conflicts when installing Allure 3 on top of it. If you want to keep both, you can use shell aliases to differentiate them. Global installations, however, have some limitations. Allure 3 supports dynamic configuration files with executable code, which allow for very deep customizations. These won't work with a globally installed tool - it can only pick up static JSON or YAML configuration files. Static configs still let you configure every Allure 3 option - just with static values. Look for more information on static and dynamic config capabilities in the [configuration reference](/docs/v3/configure/#dynamic-config-file-capabilities-and-prerequisites), and if you decide you need the full package, proceed to: **Option 2: Dummy Node.js Project (Full Configuration Capabilities)** Create a minimal Node.js project to unlock dynamic configuration capabilities: ```bash npm init --yes npm install -D allure npx allure ``` ## Configure Configuration in Allure 3 is much more flexible than in version 2. Refer to the [configuration reference](/docs/v3/configure/) for detailed options and capabilities. ## Use ### What Stays the Same #### Framework Integrations All your existing Allure [integrations](/docs/frameworks/) continue to work unchanged. You can run tests exactly as you did before, and you will get the exact same result files as with Allure 2. #### Core Commands The main commands remain essentially the same: ```bash allure generate ./allure-results # Generate report from allure-results directory allure open # Open the generated report ``` Some command options change. You can always consult the built-in help to get the most up-to-date information on options: ```bash allure --help # All commands allure --help # Command-specific help ``` ### New Capabilities #### `allure run` wrapper command Allure 3 introduces a wrapper command that supports many new features, such as: - failed test reruns even when your testing framework doesn't natively support them - multistage report builds - collecting test results from multiple data sources into a unified report **Failed test reruns:** ```bash # This will rerun failed tests up to the specified number of times: allure run --rerun -- # You can use it instead of running `` directly, for example: allure run --rerun 3 -- npm test ``` **Multistage builds:** ```bash # This will save the full state of your run into the stage1.zip file allure run --dump=stage_1 -- # This will generate a report from all dump archives matching the "stage_*.zip" pattern allure generate --dump=stage_*.zip ``` ## Important Changes Requiring Action ### History Tracking **Allure 2** stored historical data files in an automatically generated folder, which you had to copy to the results directory for every new report generation. **Allure 3** simplifies this: now history is stored in a single JSONL file, that Allure writes all the necessary data to on each report generation. No more need to copy anything anywhere. To enable history in Allure 3, simply set the [`historyPath` config parameter](/docs/v3/configure/#historypath). ### Report Hierarchies and Grouping **Allure 2** automatically populates Behaviors, Categories, Packages, and Suites tabs with tests results grouped according to predefined logic. **Allure 3** can group test results by any labels in any hierarchy, but requires explicit configuration: - By default it creates a report with just one default hierarchy. - You can customize the default hierarchy however you want, and create additional hierarchies through [configuration](/docs/v3/configure/#awesome-plugin). Images: /images/allure3/multiple-reports-light.png, /images/allure3/multiple-reports-dark.png ### Timeline and Charts Location Timeline and charts are now accessed differently: - Click on **"Report"** in the top left corner of the homepage - You can [configure](/docs/visual-analytics/) which charts are displayed Images: /images/charts/charts-allure3-light.png, /images/charts/charts-allure3-dark.png ### Categories Default categories in Allure 3 work just the same as in Allure 2. If you have any custom categories configured in the `categories.json` file of your project, you'll have to migrate this configuration to the `categories`/`rules` field in your Allure 3 [configuration file](/docs/v3/configure/#_4-categories) like so: | Allure 2 (`categories.json`) | Allure 3 (`allurerc.mjs`) | Notes | | ---------------------------- | ------------------------- | -------------------------------------------------------------------------- | | `name` | `name` | Unchanged | | `matchedStatuses` | `matchers.statuses` | Same values; now nested inside `matchers` | | `messageRegex` | `matchers.message` | Accepts a string, substring, or `RegExp` - no longer requires regex syntax | | `traceRegex` | `matchers.trace` | Same as above | | `flaky` | `matchers.flaky` | Same values; now nested inside `matchers` | Example: **Old categories.json file:** ```json [ { "name": "Infrastructure problems", "messageRegex": ".*RuntimeException.*", "traceRegex": ".*DriverException.*", "matchedStatuses": ["broken", "failed"], "flaky": true } ] ``` **Matching allurerc.mjs file:** ```js import { defineConfig } from "allure"; export default defineConfig({ categories: { rules: [ { id: "infrastructure-problems", name: "Infrastructure problems", matchers: { message: /.*RuntimeException.*/, trace: /.*DriverException.*/, statuses: ["broken", "failed"], flaky: true, }, }, ], }, }); ``` Allure 3 adds a few more ways to customize your categories. To learn more, please refer to the main [Categories](/docs/categories/) documentation. ## Entirely New Features to Explore Allure 3 introduces significant new capabilities: - [Environments](/docs/environments/) - Create multiple-environment reports and organize tests by environments. - [Resolutions](/docs/resolutions/) - Mark known, muted and accepted test failures and link them to issue trackers. - [Performance Metrics](/docs/performance-metrics/) - Collect performance measurements, track them across runs and check them with quality gates. - [Quality Gate](/docs/quality-gate/) - Set up automated quality criteria for your test runs. ## What's Still in Development Most CI and dev tools integrations are still in development. Integrations that remain Allure 2-only are marked with the (Allure 2) badge in the integrations section of the documentation. Check the [integrations](/docs/integrations/) page to verify if your CI platform or tool is supported in Allure 3. --- ## How to generate a report > URL: https://allurereport.org/docs/v3/generate-report/ # How to Generate a Report ## Run Tests and Generate Reports Running tests and generating a report with Allure 3 is straightforward. Using the `run` command, you can execute your test suite and automatically generate a report: ```bash allure run -- ``` For example, if you're using `npm` as your test runner, the command would be: ```bash allure run -- npm test ``` To successfully generate a report, ensure that your test setup outputs results into an `allure-results` directory, which is automatically detected by Allure 3. This directory can be placed at any nested level within your project (e.g., `out/tests/allure-results`), provided it retains the correct name. After the tests complete, the report is generated automatically. Existing results from previous runs are ignored, as Allure 3 focuses solely on new data to ensure accurate and up-to-date reporting. ### Generate Reports Manually from Existing Results If you already have test results and wish to generate a report manually, use the `generate` command: ```bash allure generate ``` This generates a standard Allure 3 report. You can customize output settings, such as the destination directory, through the configuration file. ## Generate Reports with the Allure Awesome Plugin Allure 3 also features an advanced report generator plugin - Allure Awesome. It supports additional configuration options, such as generating the report as a single HTML file, setting the theme, custom branding and language of the generated report, and taking known issues into account. To explore all currently available Allure Awesome features, run: ```bash allure awesome --help ``` To manually generate a customized Allure Awesome report, use the `awesome` command: ```bash allure awesome [options] ``` --- ## Improving navigation > URL: https://allurereport.org/docs/v3/navigation/ # Improving Navigation in Your Test Report A big enough project can have hundreds or even thousands of tests. When collecting information about tests, Allure can organize the results into easy-to-grasp tree structures. With such a structure, a reader of the test report can easily navigate to the test results they need. Different hierarchies can be convenient for different roles in the same team: for example, a test engineer may pay more attention to the Behaviors tree, while a developer analyzes the Suites tree, etc. Allure 3 supports a variety of ways to organize the test results, all defined by the `groupBy` parameter in the tool's [configuration](/docs/v3/configure/#awesome-plugin). You can choose the one that better aligns with the development workflow in your project. For each hierarchy, Allure expects test authors to assign specific fields to the tests. The exact functions that should be used for assigning the fields vary between different [Allure integrations](/docs/frameworks/). Info: This article describes hierarchies that are built based on the information about the tests themselves. For a hierarchy based on the test errors, see [Categories](/docs/categories/). ## Behavior-based Hierarchy **Epics**, **features** and **user stories** are widely-used terms for describing a software's features and organizing tests for said features. Usually, an epic is a large set of features that the team aims to develop and test, and a feature is described as a set of user stories that describe how the software is expected to behave in different scenarios. Allure integrations for all test frameworks provide ways to indicate a test's epic, feature and user story. If a test framework lets the author define some of these terms natively, e.g., in its own language-agnostic file format, then Allure tries its best to incorporate the provided data into the generated tree structure. See the specific Allure integration's documentation for more details. Images: /images/allure3/navigation/behaviors-light.png, /images/allure3/navigation/behaviors-dark.png Typical [configuration](/docs/v3/configure/#awesome-plugin) for behavior-based hierarchy looks like this: ```js groupBy: ["epic", "feature", "story"]; ``` ## Suite-based Hierarchy A **test suite** is an abstract term for any group of tests, with its actual meaning depending on the specifics of your project. Allure integrations for all test frameworks provide ways to indicate a test's **suite**, with most of them also providing ways to indicate **parent suites** and **sub-suites**, for the total of up to three levels of hierarchy. Note, however, that the implementation may differ in how the default values are chosen for these fields based on the data provided by the test frameworks itself. See the specific Allure integration's documentation for more details. Images: /images/allure3/navigation/suites-light.png, /images/allure3/navigation/suites-dark.png Typical [configuration](/docs/v3/configure/#awesome-plugin) for suite-based hierarchy looks like this: ```js groupBy: ["parentSuite", "suite", "subSuite"]; ``` ## Package-based Hierarchy In some projects, it is convenient to organize tests by **packages**, closely following the structure of the code. To place a test in the tree on the Packages tab, Allure normally uses the actual names of the module and the function in which the test is implemented. Some Allure integrations allow overriding this behavior. However, it is recommended to always follow the hierarchical naming convention for packages, with subpackage names placed after package names, separated by dots. In the tree, packages' common prefixes are shown as their parent packages. Images: /images/allure3/navigation/packages-light.png, /images/allure3/navigation/packages-dark.png Typical [configuration](/docs/v3/configure/#awesome-plugin) for package-based hierarchy looks like this: ```js groupBy: ["package", "testClass", "testMethod"]; ``` --- ## Reading Allure charts > URL: https://allurereport.org/docs/v3/read-charts/ # Reading Allure Charts Allure Report 3 introduces many new chart types that help your team manage your project and provide different types of insights into your test suite: snapshots of current state, trends over time, project health indicators, and performance metrics. This guide shows where to start with charts depending on what you want to track. Images: /images/charts/charts-light.png, /images/charts/charts-dark.png ## Insights **Current state** — Get a snapshot of the latest run: - The more green on the [Test statuses](/docs/visual-analytics/#test-statuses-chart) and [Severity](/docs/visual-analytics/#severity-chart) charts, the better. - The less critical the severity values for failed tests, the better: see the [Severity](/docs/visual-analytics/#severity-chart) chart. - Check the [Testing pyramid](/docs/visual-analytics/#testing-pyramid) to verify your test distribution by layer matches your expectations. **Trends** — Spot patterns and changes over time: - Watch for upward trends in problem statuses on the [Status dynamics](/docs/visual-analytics/#status-dynamics) chart. - Monitor [Status transitions](/docs/visual-analytics/#status-transitions) and [Test base growth](/docs/visual-analytics/#test-base-growth-dynamics) for unexpected spikes. **Project health** — Identify risk areas and stability issues: - A deep and extensive [Status age pyramid](/docs/visual-analytics/#status-age-pyramid) indicates persistent issues worth prioritizing. - The greener the [Success rate distribution](/docs/visual-analytics/#success-rate-distribution), the better. - Unless tests were rewritten or intentionally deleted, there shouldn't be red on the [Coverage diff map](/docs/visual-analytics/#coverage-diff-map). - Check [Stability distribution](/docs/visual-analytics/#stability-distribution) charts to find problem areas. - Review [Problems by environment](/docs/visual-analytics/#problems-distribution-by-environment) to spot environment-specific and feature-specific issues. **Performance** — Assess test execution efficiency: - The faster most tests run, the better: see [Durations](/docs/visual-analytics/#durations-histogram) and [Duration dynamics](/docs/visual-analytics/#durations-dynamics). - Watch for degradation in the speedup coefficient on [Duration dynamics](/docs/visual-analytics/#durations-dynamics). ## Roles Different team members use Allure charts for different purposes. Here's where each role might start: - **QA Leads** rely on charts for strategic decisions: prioritizing work ([Success rate distribution](/docs/visual-analytics/#success-rate-distribution), [Status age pyramid](/docs/visual-analytics/#status-age-pyramid)), planning resources ([Test base growth](/docs/visual-analytics/#test-base-growth-dynamics)), and managing risk ([Stability distributions](/docs/visual-analytics/#stability-distribution), [Status dynamics](/docs/visual-analytics/#status-dynamics) and [Status transitions](/docs/visual-analytics/#status-transitions)). - **Automation Engineers** focus on performance and reliability: identifying optimization targets ([Durations](/docs/visual-analytics/#durations-histogram)), detecting regressions ([Duration dynamics](/docs/visual-analytics/#durations-dynamics)), and cleaning up flaky tests ([Stability distributions](/docs/visual-analytics/#stability-distribution) and [Status transitions](/docs/visual-analytics/#status-transitions)). - **Manual Testers** use charts to guide their testing efforts: finding high-risk areas ([Success rate distribution](/docs/visual-analytics/#success-rate-distribution)), understanding what changed ([Coverage diff map](/docs/visual-analytics/#coverage-diff-map)), and prioritizing by severity ([Severity chart](/docs/visual-analytics/#severity-chart)). - **Stakeholders** need high-level quality indicators: overall health ([Status chart](/docs/visual-analytics/#test-statuses-chart)), quality trends ([Status dynamics](/docs/visual-analytics/#status-dynamics)), and test strategy validation ([Testing pyramid](/docs/visual-analytics/#testing-pyramid)). ## What's Next Allure Report 3 charts are highly configurable. See the [Visual Analytics](/docs/visual-analytics/) document to learn how to customize the charts appearing in your reports. --- ## Improving readability of your test reports > URL: https://allurereport.org/docs/v3/readability/ # Improving Readability of Your Test Reports The goal of Allure is to help you make test reports as easy to understand as possible. Ideally, when your colleague sees a test failure in the report, we want them to quickly understand the context: how serious the problem can be, what features are affected by it, and who can help them with investigating it further and eventually fixing it. While sometimes the existence of the failure itself is enough to make some conclusions, it is not always the case. And even if the context seems obvious at the time of writing the test, the details may become forgotten when it fails after a few years of stable work. For that reason, it is a good practice to fill tests with additional information, describing everything that a future reader may need. With the Allure integration for your test framework, you can: - provide [description, links and other metadata](#description-links-and-other-metadata); - describe parameters used when running [parametrized tests](#parametrized-tests); - provide arbitrary [environment information](#environment-information) for the whole test report. Some Allure integrations may support additional features, see their respective documentation. ## Description, Links and Other Metadata You can assign some fields that will help reader understand what a test does. ### Title A human-readable title of the test. If not provided, the function name is used instead. Images: /images/allure3/metadata/title-light.png, /images/allure3/metadata/title-dark.png ### Description An arbitrary text describing the test in more detail than the title allows. If not provided, some Allure integrations may automatically extract the description from the programming language's native documentation comments. See the specific Allure integration's documentation for more details. The description is treated as Markdown text, so you can use some basic formatting in it. HTML tags are not allowed in it and will be removed when building the report. Info: Allure Report has the internal support for HTML-formatted descriptions, and some integrations may provide functions to use it. But please remember that doing so is unsafe and can potentially break the test report's layout or even create an XSS attack vector for your organization. Images: /images/allure3/metadata/description-light.png, /images/allure3/metadata/description-dark.png ### Tags Any number of short terms the test is related to. Usually it's a good idea to list relevant features that are being tested. Tags can then be used [for filtering](/docs/sorting-and-filtering/#filter-tests-by-tags). Images: /images/allure3/metadata/tags-light.png, /images/allure3/metadata/tags-dark.png ### Severity A value indicating how important the test is. This may give the future reader an idea of how to prioritize the investigations of different test failures. Severity can also be used [for filtering](/docs/sorting-and-filtering/#filter-tests-by-severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. ### Owner The team member who is responsible for the test's stability. For example, this can be the test's author, the leading developer of the feature being tested, etc. Images: /images/allure3/metadata/owner-light.png, /images/allure3/metadata/owner-dark.png ### Links List of links to webpages that may be useful for a reader investigating a test failure. You can provide as many links as needed. There are three types of links: - a standard web link, e.g., a link to the description of the feature being tested; - a link to an issue in the product's issue tracker; - a link to the test description in a test management system (TMS). It is recommended to configure the Allure integration in such a way that it accepts short identifiers of issues and TMS links and uses a URL templates to generate full URLs. For example, an identifier `BUG-123` can automatically turn into `https://bugs.example.com/BUG-123`. You can define your own types of links with their own URL templates. See the specific Allure integration's documentation for more details. Images: /images/allure3/metadata/links-light.png, /images/allure3/metadata/links-dark.png ### ID Unique identifier of the test. The ID can be used by additional tools such as [Allure TestOps](https://qameta.io/). ### Other Labels Other key-value pairs. The labels can be used by additional tools such as [Allure TestOps](https://qameta.io/). ## Parametrized Tests Sometimes you want to run the same test multiple times, each with different set of values. For instance, you may want to make sure that the system behavior does not change whether some string is empty or not. The pattern for writing tests with common logic but different values is called **parametrized tests**. The recommended way to implement this pattern depends on the test framework. Images: /images/allure3/metadata/parametrized-light.png, /images/allure3/metadata/parametrized-dark.png Allure collects and displays data about the parameters where possible, while also providing functions for you to manually override parameter names and values for the report. This has multiple benefits for the readability of the report: - You may customize how Allure displays the parameters in the report — for example, make one of the parameters hidden. - If a test parameter is a complex object that is not displayed correctly, you can customize the value for the report specifically. - If two tests are very similar to each other, even though their implementations are independent, you can assign them identical [titles](#title) and different parameters to help reader understand both the similarities and the differences of the two cases. Note, however, that when test implementations differ significantly, it is usually better to organize them into [suites, behaviors, or packages](/docs/v3/view-report/) instead. In the list of tests, the parameter values are shown next to each test's title, so you can tell the runs of a parametrized test apart without opening them. Masked parameter values are shown as ``, and hidden parameters are not shown. ## Environment Information Allure Report 3 comes with a new Environments feature that lets you create multiple-environment reports and organize test results by environments. It also links results of the same test in different environments. For example, you can use this to track the OS, browser, deployment environments, etc. This may help a future reader investigate bugs that are reproducible only in some environments. Images: /images/environments/environments_light.png, /images/environments/environments_dark.png Learn how it works and how to set it up [here](/docs/environments/). To display key-value pairs that apply to the whole run, such as the OS or Node.js version, provide an [environment file](/docs/how-it-works-environment-file/) (`environment.properties`) in the test results directory. Some Allure integrations can generate this file for you, see your [integration's documentation](/docs/frameworks/). The key-value pairs are displayed in the **Metadata** block on the report's main page. To display key-value pairs defined in the Allure Report configuration instead, use the [`variables`](/docs/v3/configure/#variables) option. They are displayed in the **Variables** block on the report's main page. --- ## How to view a report > URL: https://allurereport.org/docs/v3/view-report/ # How to View a Report Test reports generated with Allure Report are basically small HTML websites intended to be viewed in a web browser. This page describes how to open and view such a report once you have generated one or received one from your colleague. ## Open a Single HTML File If you (or another person responsible for generating the test reports) use the single-file mode of the [Allure Awesome plugin](/docs/v3/generate-report/#generating-reports-with-the-allure-awesome-plugin) (the `--single-file` option), then good news! Such a report is the easiest to view: just open the HTML file in your preferred web browser, and that's it. Consider switching to the single-file mode if you haven't already, so that opening test reports will be that easy every time. There could be, however, reasons not to do this — most notably, to avoid potential issues when browsing really huge projects (lots of test results, tons of attachments). In such a case, please keep reading below. ## Start a Local Web Server By default, a test report is not one file, but rather a directory containing various files (HTML, JSON and other types), sometimes distributed as a ZIP archive. The most reliable method of opening such a report is to use the `allure open` command in a terminal. 1. If you got the test report in a ZIP archive, unpack all its content into a directory on your device. 1. Run the `allure open` command, providing it with the path to the directory. For example: ```bash allure open ``` The command starts a local web server configured to show the report directory's contents. The command then automatically opens the main page of the report in a web browser. By default, the web server works at a local IP address (such as **127.0.1.1**) and a randomly generated port. You may override this behavior with the `--port` argument. Use the `--live` argument to reload pages on any file change in the served directory. 1. When you finish viewing the report, you may stop the server by pressing **Ctrl+C** in the terminal window. --- # Section: Allure 2 ## Introduction > URL: https://allurereport.org/docs/v2/ # Allure Report 2 Images: /images/allure2_main_light.png, /images/allure2_main_dark.png **Allure Report** is a popular open source tool for visualizing the results of a test run. It can be added to your testing workflow with little to zero configuration. It produces reports that can be opened anywhere and can be read by anyone, no deep technical knowledge required. - **As a QA analyst**, make your test results analysis both easier and more effective by [filtering and sorting test results](/docs/sorting-and-filtering/) according to your needs. Compare the results with [previous runs](/docs/history-and-retries/). Find [unstable tests](/docs/test-stability/). Analyze how defects are distributed across test results with [Categories](/docs/categories/). - **As a developer**, view error messages, stack traces, and other debugging information. Enrich your tests' output by [splitting them into steps](/docs/steps/) or [attaching runtime data](/docs/attachments/) such as screenshots, logs, or anything else. If a test was executed multiple times, [see the details for each attempt](/docs/history-and-retries/). Use the [timeline tool](/docs/timeline/) to find performance and parallelization issues of the tests. - **As a manager**, keep track of the quality of the entire project or [individual features and test suites](/docs/v2/navigation/). Use [visual analytics tools](/docs/visual-analytics/) to see how the project's quality changes over time. The test reports are [easy to open](/docs/v2/view-report/), easy to read and support multiple locales. ## How Allure is different Unlike framework-specific reporting tools, Allure Report supports **multiple languages and frameworks** and allows you to use any combination of them. For example, if you have to run separately your backend and frontend tests, you still can convert all results into a single test report — and get a better picture of what is happening in your project. At the same time, Allure Report has a whole **ecosystem of integrations** for various test frameworks and libraries. They enhance the reports with some data out-of-box and provide you with the API to add even more data. Check out if your favorite [framework](/modules/?category=Test+Framework) or [library](/modules/?category=Test+Library) is supported. Allure Report works with test results in the most simple yet flexible form: **just files**. No need to rely on a report server being online during the execution — in fact, no need to send any data over the wire at all. Just point Allure Report to a directory, and it will read the test results from it. And finally, Allure Report is very **easy to integrate**. You don't have to change your testing workflow to start using it. One run or multiple runs, online or offline, on one machine or a cluster — any configuration is supported, and the open data format allows you to extend the workflow if needed. ## Get started with Allure - **The quickest start:** [**install Allure Report**](/docs/v2/install/) and the [integration for your framework](/docs/frameworks/), run your tests the same way as you do it now, make sure you've got results in the supported formats, then and [**run Allure Report**](/docs/v2/generate-report/) to turn those results into a beautiful HTML report. - **Ready for more?** Learn to use the API provided by the [integrations](/docs/frameworks/) to enable advanced Allure features like [steps](/docs/steps/), [attachments](/docs/attachments/), [test hierarchies](/docs/v2/navigation/), and more. - **Need a bigger picture?** Consider using [**Allure TestOps**](https://qameta.io/) — our paid offering that allows you to track both manual and automated test cases, aggregate test results from a wide range of sources and analyze them across multiple launches and projects. ## Control your data flow A testing workflow with Allure Report consists of two steps, both of which can be done locally, without sending anything over the wire. 1. **Collection phase**: while the tests are running, the test framework writes their results into a file or directory. 2. **Visualization phase**: the Allure Report command-line utility reads the test results and builds an HTML report. Once the HTML report is generated, you can safely remove the test results — for example, as part of a cleaning procedure before the next test run. ![How Allure Report works: the collection phase and the visualization phase](/images/diagrams/how-it-works-simplified.png "Allure Report The Collection Phase And The Visualization Phase") Allure recognizes test results in many formats, including [our own data format](/docs/how-it-works-test-result-file/). Its support can be added to many popular frameworks by using [Allure integrations](/docs/frameworks/), and some frameworks even support it natively. In this data format, each test result is stored as a separate JSON file with a randomly generated filename. The Allure's data format and the file-based data flow is: - **parallelizable**: you can run tests on multiple nodes as long as you merge all the results into a single directory before the visualization phase; - **framework-agnostic**: the framework-agnostic data format is perfect for projects that span across multiple programming languages and test frameworks; - **failure-tolerant**: you can build a report based on the test results that you have even if other tests did not write their files due to an unexpected issue; - **extensible**: feel free to filter or modify the test results between the two phases, thanks to the well-documented data format. See [How it works](/docs/how-it-works/) for a more detailed overview of the typical data flow in Allure Report. ## Organize reports your way A test report built by Allure is not locked to a specific service — it is just a small static HTML website. You can open a report on any computer using the Allure Report utility itself or even just a browser. This gives you maximum flexibility when in comes to sharing or publishing the report. - Send a report as a ZIP archive to your colleague that needs to see it. - Store the reports on your web server, with [links to previous reports](/docs/history-and-retries/). - Use the [integration plugins](/docs/integrations/) to embed reports into your CI tool's interface. For more information, see [How to view a report](/docs/v2/view-report/). ## The Allure ecosystem The Allure Report utility itself is a part of a larger ecosystem, supported by both [Qameta Software](https://qameta.io/about-us/) and the broad community. The ecosystem includes: - [**Integrations**](/docs/frameworks/) that enable support for the Allure data format in various test framework. Using an Allure integration for your framework is recommended to enable more features, such as [steps](/docs/steps/), [attachments](/docs/attachments/), and [test hierarchies](/docs/v2/navigation/). - **Library integrations** for logging certain type of events during the test, e.g., HTTP requests, database queries, assertions, etc. - **IDE integrations** for building reports from [JetBrains IDEs](/docs/integrations-jetbrains/) or [Visual Studio Code](/docs/integrations-vscode/). - [**CI/CD integrations**](/docs/integrations/) for building and publishing reports as part of your pipeline. - [**Allure TestOps**](https://qameta.io/) — a paid Allure Report replacement that manages test results across multiple launches for even deeper analysis. Most of the tools (with the exception of Allure TestOps) are open source under Apache License 2.0. --- ## Installation > URL: https://allurereport.org/docs/v2/install/ # Install or upgrade Allure Report There are multiple methods of installing Allure Report. Most of them are dependent on the operating system you use: - [Install Allure Report for Windows](/docs/v2/install-for-windows/) - [Install Allure Report for macOS](/docs/v2/install-for-macos/) - [Install Allure Report for Linux](/docs/v2/install-for-linux/) - [Install Allure Report for Node.js](/docs/v2/install-for-nodejs/) (any system) Note that Allure Report is actively developed, and a lot of new features are introduced in newer versions. If you installed Allure Report a while ago, consider [upgrading it](/docs/v2/upgrade/). --- ## Install Allure Report for Linux > URL: https://allurereport.org/docs/v2/install-for-linux/ # Install Allure Report for Linux To get the reporting process working you need to install two components: - [Allure Report CLI tool](#install-the-allure-report-cli) - [Allure integration for your test framework](#install-the-test-framework-integration) ## Install the Allure Report CLI Allure Report can be installed on Linux from a Homebrew or DEB package or manually from an archive. ### Install from Homebrew [![Allure Report commandline Homebrew latest version](https://img.shields.io/homebrew/v/allure?style=flat "Allure Report commandline Homebrew latest version")](https://formulae.brew.sh/formula/allure) 1. Make sure [Homebrew](https://brew.sh/) is installed. 1. In a terminal, run this command: ``` brew install allure ``` 1. Run this command to see if it reports the latest version: ``` allure --version ``` ### Install from a DEB package [![Allure Report commandline latest version](https://img.shields.io/github/v/release/allure-framework/allure2?style=flat "Allure Report commandline latest version")](https://github.com/allure-framework/allure2/releases/latest) 1. Make sure that your Linux distribution supports installing software from DEB packages. 1. Go to [the latest Allure Report release on GitHub](https://github.com/allure-framework/allure2/releases/latest) and download the `allure-*.deb` file. 1. In a terminal, go to the directory with the file and install it. For example: ``` sudo dpkg -i allure_2.29.0-1_all.deb ``` 1. Run this command to see if it reports the latest version: ``` allure --version ``` ### Install from an archive [![Allure Report commandline latest version](https://img.shields.io/github/v/release/allure-framework/allure2?style=flat "Allure Report commandline latest version")](https://github.com/allure-framework/allure2/releases/latest) 1. Make sure Java version 8 or above installed, and its directory is specified in the `JAVA_HOME` environment variable. 1. Go to [the latest Allure Report release on GitHub](https://github.com/allure-framework/allure2/releases/latest) and download the `allure-*.zip` or `allure-*.tgz` archive. 1. Uncompress the archive into an installation directory of your choice. For example: ``` tar xf /home/user/Downloads/allure-2.29.0.tgz -C /home/user/tools ``` 1. Make sure Allure is in your `PATH`, so you can invoke it from the command line by just typing `allure`. There are various ways to do so, for example, via the `update-alternatives` utility or by modifying the `PATH` variable. **Using update-alternatives** Details: 1. Run the following two commands to register the new path to Allure Report and make it default. (Replace `/home/user/tools/allure-2.29.0/bin/allure` with the path to the `allure` binary file in your installation directory.) ``` sudo update-alternatives --install /ust/bin/allure allure /home/user/tools/allure-2.29.0/bin/allure 10 sudo update-alternatives --set allure /home/user/tools/allure-2.29.0/bin/allure ``` 2. If you have installed an older version before and do not need it anymore, unregister it, for example: ``` sudo update-alternatives --remove allure /home/user/tools/allure-2.28.0/bin/allure ``` **Modifying PATH** Details: 1. Open the script that your command shell loads when logging in (it can be located at `~/.profile`, `~/.bash_profile`, `~/.zprofile`, etc.) and add the following line to the end of it. (Replace `/home/user/tools/allure-2.29.0/bin` with the path to the `bin` subdirectory file in your installation directory.) ```bash export PATH=$PATH:/home/user/tools/allure-2.29.0/bin ``` 2. If you have previously added a path to another version of Allure Report to `PATH`, make sure to remove it. 3. Close the current terminal window, then open a new one. 1. Run this command to see if it reports the latest version: ``` allure --version ``` ## Install the test framework integration Look for the integration for your test framework in the list of [Allure integrations](/docs/frameworks/), and follow the setup instructions. --- ## Install Allure Report for macOS > URL: https://allurereport.org/docs/v2/install-for-macos/ # Install Allure Report for macOS To get the reporting process working you need to install two components: - [Allure Report CLI tool](#install-the-allure-report-cli) - [Allure integration for your test framework](#install-the-test-framework-integration) ## Install the Allure Report CLI Allure Report can be installed on Linux from the Homebrew package repository or manually from an archive. ### Install from Homebrew [![Allure Report commandline Homebrew latest version](https://img.shields.io/homebrew/v/allure?style=flat "Allure Report commandline Homebrew latest version")](https://formulae.brew.sh/formula/allure) 1. Make sure [Homebrew](https://brew.sh/) is installed. 1. In a terminal, run this command: ``` brew install allure ``` 1. Run this command to see if it reports the latest version: ``` allure --version ``` ### Install from an archive [![Allure Report commandline latest version](https://img.shields.io/github/v/release/allure-framework/allure2?style=flat "Allure Report commandline latest version")](https://github.com/allure-framework/allure2/releases/latest) 1. Make sure Java version 8 or above installed, and its directory is specified in the `JAVA_HOME` environment variable. 1. Go to [the latest Allure Report release on GitHub](https://github.com/allure-framework/allure2/releases/latest) and download the `allure-*.zip` or `allure-*.tgz` archive. 1. Uncompress the archive into an installation directory of your choice. For example: ``` tar xf /home/user/Downloads/allure-2.29.0.tgz -C /home/user/tools ``` 1. Open the script that your command shell loads when logging in (it can be located at `~/.profile`, `~/.bash_profile`, `~/.zprofile`, etc.) and add the following line to the end of it. (Replace `/home/user/tools/allure-2.29.0/bin` with the path to the `bin` subdirectory file in your installation directory.) ```bash export PATH=$PATH:/home/user/tools/allure-2.29.0/bin ``` 1. If you have previously added a path to another version of Allure Report to `PATH`, make sure to remove it. 1. Close and re-open the terminal window. 1. Run this command to see if it reports the latest version: ``` allure --version ``` ## Install the test framework integration Look for the integration for your test framework in the list of [Allure integrations](/docs/frameworks/), and follow the setup instructions. --- ## Install Allure Report for Node.js > URL: https://allurereport.org/docs/v2/install-for-nodejs/ # Install Allure Report for Node.js To get the reporting process working you need to install two components: - [Allure Report CLI tool](#install-the-allure-report-cli) - [Allure integration for your test framework](#install-the-test-framework-integration) ## Install the Allure Report CLI [![Allure Report commandline npm latest version](https://img.shields.io/npm/v/allure-commandline?style=flat "Allure Report commandline npm latest version")](https://www.npmjs.com/package/allure-commandline) Allure Report can be installed on any operating system using Node.js. 1. Make sure [Node.js](https://nodejs.org/) is installed. 1. Make sure Java version 8 or above installed, and its directory is specified in the `JAVA_HOME` environment variable. 1. In a terminal, run this command: ``` npm install -g allure-commandline ``` 1. Run this command to see if it reports the latest version: ``` allure --version ``` If the `allure` command does not work, it is most likely caused by the NPM's directory not being added to your operating system's `PATH` variable. To get the path that needs to be added, run this command: **Windows:** ```powershell npm config get prefix ``` **macOS:** ```bash echo $(npm config get prefix)/bin ``` **Linux:** ```bash echo $(npm config get prefix)/bin ``` ## Install the test framework integration Look for the integration for your test framework in the list of [Allure integrations](/docs/frameworks/), and follow the setup instructions. --- ## Install Allure Report for Windows > URL: https://allurereport.org/docs/v2/install-for-windows/ # Install Allure Report for Windows To get the reporting process working you need to install two components: - [Allure Report CLI tool](#install-the-allure-report-cli) - [Allure integration for your test framework](#install-the-test-framework-integration) ## Install the Allure Report CLI Allure Report can be installed on Windows from the [Scoop](https://scoop.sh/) package repository or manually from an archive. ### Install from Scoop [![Allure Report commandline latest version](https://img.shields.io/scoop/v/allure?style=flat "Allure Report commandline latest version")](https://scoop.sh/#/apps?q=allure) 1. Make sure Scoop is installed. See the [installation instructions on GitHub](https://github.com/ScoopInstaller/Install#readme). 1. Make sure Java version 8 or above installed, and its directory is specified in the `JAVA_HOME` environment variable. 1. In a terminal, run this command: ``` scoop install allure ``` 1. Run this command to see if it reports the latest version: ``` allure --version ``` ### Install from an archive [![Allure Report commandline latest version](https://img.shields.io/github/v/release/allure-framework/allure2?style=flat "Allure Report commandline latest version")](https://github.com/allure-framework/allure2/releases/latest) 1. Make sure Java version 8 or above installed, and its directory is specified in the `JAVA_HOME` environment variable. 1. Go to [the latest Allure Report release on GitHub](https://github.com/allure-framework/allure2/releases/latest) and download the `allure-*.zip` or `allure-*.tgz` archive. 1. Uncompress the archive into the directory of your choice. Remember the path to its `bin` subdirectory. You will need this path in a future step. 1. Make sure that the `allure` command resolves to the `allure` file from your installation directory. There are various ways to do so, for example, via the Control Panel or by running a PowerShell script. **Using Control Panel** Details: 1. Press **Win+R** and enter the command: `sysdm.cpl` to open the **System Properties** tool. 1. On the **Advanced** tab, click **Environment variables**. 1. In either the **User variables** or **System variables** list, double-click the `Path` variable to open the editing dialog. Note that editing the system variable requires administrator privileges and affects all users of the computer. 1. In the **Edit environment variable** dialog, click **New** to add a new line entry to the paths list. In the new line, specify the full path to the `bin` subdirectory from an earlier step, for example: `D:\Tools\allure-2.29.0\bin`. 1. If the list contains a path to a previously installed Allure version, delete it. 1. Click **OK** to save the changes. **Using PowerShell** Details: 1. Launch PowerShell and run the following three commands. (Replace `D:\Tools\allure-2.29.0\bin` with the path to the `bin` subdirectory from an earlier step.) ```powershell $AllureBinPath = "D:\Tools\allure-2.29.0\bin" $NewPath = (([Environment]::GetEnvironmentVariable("PATH", "User") -split ";") | ?{ $_ -and $_ -notlike "*\allure-*\bin" }) -join ";" [Environment]::SetEnvironmentVariable("PATH", "$NewPath;$AllureBinPath", "User") ``` 1. Close the current PowerShell window. 1. In a new PowerShell window, run this command to see if it reports the latest version: ``` allure --version ``` ## Install the test framework integration Look for the integration for your test framework in the list of [Allure integrations](/docs/frameworks/), and follow the setup instructions. --- ## Upgrade Allure Report > URL: https://allurereport.org/docs/v2/upgrade/ # Upgrade Allure Report We recommend to use the latest version of the Allure Report command-line utility. To upgrade your Allure Report installation to the newest version, follow the instructions below. Select the same method you used when installing the previous version. Tip: **Which version do I have now?** To find out the current installed version of Allure Report, open the terminal and run `allure --version`. If it displays the same version number as the latest Allure Report release, you do not need to upgrade for now. We also recommend running this command after an upgrade to make sure it was successful. ## Upgrade from Scoop [![Allure Report commandline latest version](https://img.shields.io/scoop/v/allure?style=flat "Allure Report commandline latest version")](https://scoop.sh/#/apps?q=allure) If you installed Allure Report [from Scoop](/docs/v2/install-for-windows/#install-from-scoop), use the following command to upgrade it. ``` scoop update allure ``` ## Upgrade from Homebrew [![Allure Report commandline Homebrew latest version](https://img.shields.io/homebrew/v/allure?style=flat "Allure Report commandline Homebrew latest version")](https://formulae.brew.sh/formula/allure) If you installed Allure Report from Homebrew (for [macOS](/docs/v2/install-for-macos/#install-from-homebrew) or [Linux](/docs/v2/install-for-linux/#install-from-homebrew)), use the following command to upgrade it. ``` brew upgrade allure ``` ## Upgrade from a DEB package [![Allure Report commandline latest version](https://img.shields.io/github/v/release/allure-framework/allure2?style=flat "Allure Report commandline latest version")](https://github.com/allure-framework/allure2/releases/latest) If you installed Allure Report [from a DEB package](/docs/v2/install-for-linux/#install-from-a-deb-package), you need to download and install a newer version of the DEB package. This will automatically replace the existing installation. 1. Go to [the latest Allure Report release on GitHub](https://github.com/allure-framework/allure2/releases/latest) and download the `allure-*.deb` file. 1. In a terminal, go to the directory with the file and install it. For example: ``` sudo dpkg -i allure_2.29.0-1_all.deb ``` ## Upgrade from a Node.js package [![Allure Report commandline npm latest version](https://img.shields.io/npm/v/allure-commandline?style=flat "Allure Report commandline npm latest version")](https://www.npmjs.com/package/allure-commandline) If you installed Allure Report [from a Node.js package](/docs/v2/install-for-nodejs/) for your project, upgrade it via `npm`. ``` npm update -g allure-commandline ``` ## Upgrade from an archive [![Allure Report commandline latest version](https://img.shields.io/github/v/release/allure-framework/allure2?style=flat "Allure Report commandline latest version")](https://github.com/allure-framework/allure2/releases/latest) If you installed Allure Report from an archive (for [Windows](/docs/v2/install-for-windows/#install-from-an-archive), [macOS](/docs/v2/install-for-macos/#install-from-an-archive) or [Linux](/docs/v2/install-for-linux/#install-from-an-archive)), follow the same installation instructions again to download and install a newer archive. When editing the environment variable on the last step, make sure to delete the path to the previous version. Optionally, remove the directory containing the previous Allure version if you don't need it. --- ## How to generate a report > URL: https://allurereport.org/docs/v2/generate-report/ # How to Generate a Report ## 1. Run Tests Run your tests as you would do normally: from the command line, from your IDE, etc. In addition to printing or saving the test results in its own format, the Allure Report integration receives the data from a test framework. The API used for this communication depends on the framework — for example, the Allure integration may be registered as a [Playwright reporter](https://playwright.dev/docs/test-reporters), a [Cucumber.js formatter](https://github.com/cucumber/cucumber-js/blob/main/docs/custom_formatters.md), etc. The integration saves data in the Allure file formats into a separate directory (`allure-results` by default). The directory path can be changed in the integration settings. The generated files include: - [test result files](/docs/how-it-works-test-result-file/) to describe execution of tests, - [container files](/docs/how-it-works-container-file/) to describe test fixtures. Tip: You may run this step multiple times, accumulating data into the same results directory. This is useful, for example, if different tests require different runner parameters or different environments. As a group, these runs are referred to as a **launch**. When a launch contains multiple runs of the same test, they are referred to as [retries](/docs/history-and-retries/#retries). ## 2. Add Launch Data If necessary, put files with additional data for this launch into the results directory. Additional files may include: - [environment file](/docs/how-it-works-environment-file/) to store some global data for all the tests in the report, - [executor file](/docs/how-it-works-executor-file/) to describe the CI job and the URL at which the report will be located, - [history files](/docs/how-it-works-history-files/) to enable trend charts and other history-related features, - [categories file](/docs/how-it-works-categories-file/) to group test results into custom categories. Skip this step if you don't need to enable any of these features, or if your Allure integration generates the files automatically. Check [your integration's documentation](/docs/frameworks/). ## 3. Generate Report Generate the HTML report by using one of the two Allure command-line commands. - `allure generate` processes the test results and saves an HTML report into the specified directory (`allure-report` by default). To view the report, use the `allure open` command. Use this command if you need to save the report for future reference or for sharing it with colleagues. - `allure serve` creates the same report as `allure generate` but puts it into a temporary directory and starts a local web server configured to show this directory's contents. The command then automatically opens the main page of the report in a web browser. Use this command if you need to view the report for yourself and do not need to save it. ## Initial configuration The initial configuration depends on the test framework you are using. In general, the following steps are required: 1. Install the Allure integration for your test framework. 1. Install the Allure command-line utility, see [Installation](/docs/v2/install/). Info: Depending on the framework you are using, the Allure integration may come with the Allure command-line utility included. In such a case, you may skip this step. 1. In your project's configuration, enable the Allure integration and set its parameters. See the [specific integration documentation](/docs/frameworks/) for more details. --- ## Improving navigation > URL: https://allurereport.org/docs/v2/navigation/ # Improving Navigation in Your Test Report A big enough project can have hundreds or even thousands of tests. When collecting information about tests, Allure can organize the results into easy-to-grasp tree structures. With such a structure, a reader of the test report can easily navigate to the test results they need. Allure generates different trees on the [Suites](#suite-based-hierarchy), [Behaviors](#behavior-based-hierarchy), or [Packages](#package-based-hierarchy) tabs of the report. Different hierarchies can be convenient for different roles in the same team: for example, a test engineer may pay more attention to the Behaviors tree, while a developer analyzes the Suites tree, etc. You can choose the one that better aligns with the development workflow in your project. For each hierarchy, Allure expects test authors to assign specific fields to the tests. The exact functions that should be used for assigning the fields vary between different [Allure integrations](/docs/frameworks/). Info: This article describes hierarchies that are built based on the information about the tests themselves. For a hierarchy based on the test errors, see [Categories](/docs/categories/). ## Behavior-based Hierarchy **Epics**, **features** and **user stories** are widely-used terms for describing a software's features and organizing tests for said features. Usually, an epic is a large set of features that the team aims to develop and test, and a feature is described as a set of user stories that describe how the software is expected to behave in different scenarios. Allure integrations for all test frameworks provide ways to indicate a test's epic, feature and user story. If a test framework lets the author define some of these terms natively, e.g., in its own language-agnostic file format, then Allure tries its best to incorporate the provided data into the generated tree structure. See the specific Allure integration's documentation for more details. Images: /images/navigation/behaviors-light.png, /images/navigation/behaviors-dark.png ## Suite-based Hierarchy A **test suite** is an abstract term for any group of tests, with its actual meaning depending on the specifics of your project. Allure integrations for all test frameworks provide ways to indicate a test's **suite**, with most of them also providing ways to indicate **parent suites** and **sub-suites**, for the total of up to three levels of hierarchy. Note, however, that the implementation may differ in how the default values are chosen for these fields based on the data provided by the test frameworks itself. See the specific Allure integration's documentation for more details. Images: /images/navigation/suites-light.png, /images/navigation/suites-dark.png ## Package-based Hierarchy In some projects, it is convenient to organize tests by **packages**, closely following the structure of the code. To place a test in the tree on the Packages tab, Allure normally uses the actual names of the module and the function in which the test is implemented. Some Allure integrations allow overriding this behavior. However, it is recommended to always follow the hierarchical naming convention for packages, with subpackage names placed after package names, separated by dots. In the tree, packages' common prefixes are shown as their parent packages. Images: /images/java/packages-light.png, /images/java/packages-dark.png ## Omitting Tree Levels In all the hierarchies described above, none of the fields are necessary. To determine a test's location in the **Suites** or **Behaviors** hierarchy, Allure checks for the relevant fields in the predefined order: - for [behavior-based hierarchy](#behavior-based-hierarchy): epic, feature, user story, - for [suite-based hierarchy](#suite-based-hierarchy): parent suite, suite, sub-suite. For each provided field, Allure creates a subtree in the test results tree. If the field was not provided, the subtree creation is skipped. In the example below (written in Java using Allure JUnit 5), the second test only has the `feature` field. In the report, its **feature** is displayed at the top level, essentially becoming this test's **epic**. ```java import io.qameta.allure.Epic; import io.qameta.allure.Feature; import io.qameta.allure.Story; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; public class AllureNavigationTest { @Test @Epic("Web interface") @Feature("Essential features") @Story("Authentication") @DisplayName("Log in with username and password") public void testLogInWithUsernameAndPassword() { // ... } @Test @Feature("Mobile API") @DisplayName("API: Read user profile") public void testReadUserProfile() { // ... } } ``` Images: /images/suites-omit-levels-light.png, /images/suites-omit-levels-dark.png --- ## Improving readability of your test reports > URL: https://allurereport.org/docs/v2/readability/ # Improving Readability of Your Test Reports The goal of Allure is to help you make test reports as easy to understand as possible. Ideally, when your colleague sees a test failure in the report, we want them to quickly understand the context: how serious the problem can be, what features are affected by it, and who can help them with investigating it further and eventually fixing it. While sometimes the existence of the failure itself is enough to make some conclusions, it is not always the case. And even if the context seems obvious at the time of writing the test, the details may become forgotten when it fails after a few years of stable work. For that reason, it is a good practice to fill tests with additional information, describing everything that a future reader may need. With the Allure integration for your test framework, you can: - provide [description, links and other metadata](#description-links-and-other-metadata); - describe parameters used when running [parametrized tests](#parametrized-tests); - provide arbitrary [environment information](#environment-information) for the whole test report. Some Allure integrations may support additional features, see their respective documentation. ## Description, Links and Other Metadata You can assign some fields that will help reader understand what a test does. ### Title A human-readable title of the test. If not provided, the function name is used instead. Images: /images/metadata/title-allure2-light.png, /images/metadata/title-allure2-dark.png ### Description An arbitrary text describing the test in more details than the title could fit. If not provided, some Allure integrations may automatically extract description from the programming language's native documentation comments. See the specific Allure integration's documentation for more details. The description will be treated as a Markdown text, so you can you some basic formatting in it. HTML tags are not allowed in such a text and will be removed when building the report. Info: Allure Report has the internal support for HTML-formatted descriptions, and some integrations may provide functions to use it. But please remember that doing so is unsafe and can potentially break the test report's layout or even create an XSS attack vector for your organization. Images: /images/metadata/description-allure2-light.png, /images/metadata/description-allure2-dark.png ### Tags Any number of short terms the test is related to. Usually it's a good idea to list relevant features that are being tested. Tags can then be used [for filtering](/docs/sorting-and-filtering/#filter-tests-by-tags). Images: /images/metadata/tags-allure2-light.png, /images/metadata/tags-allure2-dark.png ### Severity A value indicating how important the test is. This may give the future reader an idea of how to prioritize the investigations of different test failures. Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. Images: /images/metadata/severity-allure2-light.png, /images/metadata/severity-allure2-dark.png ### Owner The team member who is responsible for the test's stability. For example, this can be the test's author, the leading developer of the feature being tested, etc. Images: /images/metadata/owner-allure2-light.png, /images/metadata/owner-allure2-dark.png ### Links List of links to webpages that may be useful for a reader investigating a test failure. You can provide as many links as needed. There are three types of links: - a standard web link, e.g., a link to the description of the feature being tested; - a link to an issue in the product's issue tracker; - a link to the test description in a test management system (TMS). It is recommended to configure the Allure integration in such a way that it accepts short identifiers of issues and TMS links and uses a URL templates to generate full URLs. For example, an identifier `BUG-123` can automatically turn into `https://bugs.example.com/BUG-123`. You can define your own types of links with their own URL templates. See the specific Allure integration's documentation for more details. Images: /images/metadata/links-allure2-light.png, /images/metadata/links-allure2-dark.png ### ID Unique identifier of the test. The ID can be used by additional tools such as [Allure TestOps](https://qameta.io/). ### Other Labels Other key-value pairs. The labels can be used by additional tools such as [Allure TestOps](https://qameta.io/). ## Parametrized Tests Sometimes you want to run the same test multiple times, each with different set of values. For instance, you may want to make sure that the system behavior does not change whether some string is empty or not. The pattern for writing tests with common logic but different values is called **parametrized tests**. The recommended way to implement this pattern depends on the test framework. Images: /images/java/junit5/parametrized-allure2-light.png, /images/java/junit5/parametrized-allure2-dark.png Allure collects and displays data about the parameters where possible, while also providing functions for you to manually override parameter names and values for the report. This has multiple benefits for the readability of the report: - You may customize how Allure displays the parameters in the report — for example, make one of the parameters hidden. - If a test parameter is a complex object that is not displayed correctly, you can customize the value for the report specifically. - If two tests are very similar to each other, even though their implementations are independent, you can assign them identical [titles](#title) and different parameters to help reader understand both the similarities and the differences of the two cases. Note, however, that when test implementations differ significantly, it is usually better to organize them into [suites, behaviors, or packages](/docs/v2/view-report/) instead. ## Environment Information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version, programming language version, etc. This may help a future reader investigate bugs that are reproducible only in some environments. Images: /images/python/environment-allure2-light.png, /images/python/environment-allure2-dark.png To provide such information, put a special [environment file](/docs/how-it-works-environment-file/) into the test results directory after running the tests. The file will be used when building the report by the `allure` command. Some Allure integrations can generate this file automatically based on their configuration, see the specific integration's documentation for more details. Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#parametrized-tests). --- ## How to view a report > URL: https://allurereport.org/docs/v2/view-report/ # How to View a Report Test reports generated with Allure Report are basically small HTML websites intended to be viewed in a web browser. This page describes how to open and view such a report once you have generated one or received one from your colleague. Tip: When possible, it is recommended to configure a CI integration plugin in such a way that the test report would be available online without downloading. ## Open a Single HTML File If you (or another person responsible for generating the test reports) use the single-file mode (the `--single-file` option), then good news! Such a report is the easiest to view: just open the HTML file in your preferred web browser, and that's it. Consider switching to the single-file mode if you haven't already, so that opening test reports will be that easy every time. There could be, however, reasons not to do this — most notably, to avoid potential issues when browsing really huge projects (lots of test results, tons of attachments). In such a case, please keep reading below. ## Start a Local Web Server By default, a test report is not one file, but rather a directory containing various files (HTML, JSON and other types), sometimes distributed as a ZIP archive. The most reliable method of opening such a report is to use the `allure open` command in a terminal. 1. If you got the test report in a ZIP archive, unpack all its content into a directory on your device. 1. Run the `allure open` command, providing it with the path to the directory. For example: ```plain allure open /home/user/myproject/build/allure-report ``` The command starts a local web server configured to show the report directory's contents. The command then automatically opens the main page of the report in a web browser. By default, the web server works at a local IP address (such as **127.0.1.1**) and a randomly generated port. You may override this behavior with the `--host` and `--port` arguments. 1. When you finish viewing the report, you may stop the server by pressing **Ctrl+C** in the terminal window. ## Open Local Directory in a Web Browser ### Google Chrome Warning: This method is not recommended, as it involves setting parameters that make Google Chrome **vulnerable to a certain type of malicious scripts**. If you choose to use this method, make sure to close any tabs with content that you do not trust completely and not open them again until you revert the setting. To be able to correctly open test reports from local files in Google Chrome, you need to launch the browser with a certain command-line argument. This will affect all websites opened in Google Chrome until you restart it again without that command-line argument. 1. If you got the test report in a ZIP archive, unpack all its content into a directory on your device. 1. If Google Chrome is running, open the browser menu and select **Exit** to close all its windows. This is necessary because otherwise the command-line option on the next step won't affect the browser. 1. Open a terminal and run Google Chrome, specifying the `--allow-file-access-from-files` option and the path to your test report's `index.html` file. For example: **Windows:** ```plain %ProgramFiles%\Google\Chrome\Application\chrome.exe -allow-file-access-from-files /home/user/myproject/build/allure-report/index.html ``` **MacOS:** ```plain open -a "Google Chrome" --args --allow-file-access-from-files /Users/user/myproject/build/allure-report/index.html ``` **Linux:** ```plain google-chrome --allow-file-access-from-files /home/user/myproject/build/allure-report/index.html ``` 1. Once you are done with viewing the test report, open the browser menu and select **Exit** to close all its windows. This is necessary to make the browser “forget” the temporary setting before the next launch. ### Mozilla Firefox Warning: This method is not recommended, as it involves setting parameters that make Mozilla Firefox **vulnerable to a certain type of malicious scripts**. If you choose to use this method, make sure to close any tabs with content that you do not trust completely and not open them again until you revert the setting. To be able to correctly open test reports from local files in Mozilla Firefox, you need to change a certain global preference. This will affect all websites opened in Mozilla Firefox until you change the preference back. 1. If you got the test report in a ZIP archive, unpack all its content into a directory on your device. 1. In Mozilla Firefox, open a new tab and go to the address: `about:config`. 1. If a **Proceed with Caution** warning shows up, click **Accept the Risk and Continue**. 1. In the **Search preference name** box, enter: `security.fileuri.strict_origin_policy`. The preference with that name should appear on the page. 1. If the preference is set to `true`, double-click on the value to toggle it. Make sure that the preference is now set to `false`. 1. In a file explorer, navigate to your test report's directory and open its `index.html` file in Mozilla Firefox. 1. Once you are done with viewing the test report, switch back to the `about:config` tab and double-click on the value again. Make sure that the preference is now set to `true`. --- # Section: Features ## Agent Mode > URL: https://allurereport.org/docs/agent-mode/ # Agent Mode (Allure 3) Allure Agent Mode lets an AI coding agent run your existing test command and review agent-readable test reporting before it answers. Use it when an agent needs to make a claim about test coverage, a new or changed test, a failing run, flaky behavior, or regression safety. Instead of relying only on terminal output, the agent can review what ran, what passed or failed, what was skipped or retried, and what evidence was attached. Agent Mode does not replace Allure reports for humans. It adds an agent-readable layer on top of the same reporting workflow your team already uses. ## What Agent Mode does Agent Mode is useful when you ask a coding agent to: - review whether a feature is actually covered by tests; - add or improve a test and validate the intended scope; - debug a failing or flaky test with runtime evidence; - check whether a change is safe before claiming it is done; - enrich tests with useful steps, attachments, labels, parameters, or descriptions. The goal is not just to run a command. The goal is to make the agent base its answer on test reporting that can be reviewed. ## How Agent Mode works `allure agent` wraps the test command your project already uses: ```shell allure agent -- npm test checkout.spec.ts ``` Everything after `--` is still your real test command. Your test runner executes the tests, Allure adapters emit Allure results, and Allure captures command output and runtime evidence to write Agent Mode output for that run. A typical Agent Mode workflow is: 1. Choose a focused validation goal. 2. Run the smallest meaningful test scope through `allure agent`. 3. Review the generated Agent Mode output before reading source code. 4. Fix product code, tests, fixtures, or evidence quality when needed. 5. Rerun when the conclusion depends on the fix. 6. Report what ran, what was checked, what evidence supports the conclusion, and what remains uncertain. ### Live runs and existing results Agent Mode can give your coding agent reporting context in two ways: - wrap a live test command with `allure agent -- `; - inspect existing Allure results or dump artifacts with `allure agent inspect`. The live-command path creates fresh evidence: the agent runs the relevant test scope through `allure agent`, Allure captures the command output and runtime evidence, and Allure writes Agent Mode output for that run. The agent then inspects that output to analyze coverage, failures, findings, and limitations. Sometimes the useful evidence already exists. For example, a run may have happened in CI, on a teammate's machine, or earlier on your own machine. When that run preserved raw `allure-results` directories or Allure dump artifacts, Allure turns those artifacts into Agent Mode output without rerunning the tests. Both paths give the agent a reporting-first review surface before it reads long logs, generated HTML reports, or source code. If the original run did not preserve raw results or dump artifacts, inspection has less runtime context to work with. ### Agent output `allure agent` creates and prints an output directory when no output path is provided. Use that default unless your project needs an explicit output location. Agent output is the review surface your coding agent reads before it makes a test-quality claim. A typical output directory includes: ```text agent-output/ index.md manifest/ tests/ logs/ awesome/ ``` The exact file names and internal layout may change in future Allure versions, but the idea stays the same: `index.md` is the entry point, and the directory contains the run summary plus the supporting runtime details. Depending on what the run produced, Agent Mode output can give the agent access to: - the wrapped command, exit code, timing, and selected test scope; - passed, failed, skipped, broken, retried, and missing-test signals; - findings about weak evidence, scope mismatches, unexpected skips, or reporting problems; - per-test summaries with steps, labels, parameters, attachments, retries, and failure details; - stdout, stderr, logs, and process-level artifacts when they matter; - the generated human-report status and path, when a human-readable report was produced. You can inspect the same Agent Mode output yourself. For example, if you already have an `allure-results` directory from a local run or CI artifact, create a readable Agent Mode output directory with: ```shell allure agent inspect allure-results --output agent-output ``` Each run should have distinct Agent Mode output. Do not reuse one output directory for unrelated runs, and do not share one output path between parallel runs. After each run, the agent should report the generated `index.md` path. For final or user-facing runs, if a human-readable report was generated, the agent should also report its path or state that no human report was generated. ## Before you start ### Requirements To use Agent Mode, you need: - Allure Report 3.12.0 or newer with Agent Mode support; - a project test command, such as `npm test`, `pytest`, `mvn test`, or `./gradlew test`; - Allure results emitted by the tests in a location Allure can discover; - an AI coding agent that can run shell commands and read generated files in your repository. If your project does not meet the reporting requirement yet, see [Configure Allure reporting](#configure-allure-reporting) before enabling Agent Mode. Agent Mode does not require MCP, a hosted service, or a specific AI vendor. You can use it with Codex, Claude, OpenCode, custom scripts, CI-driven agents, or another agentic code workflow. Use the installed CLI as the source of truth for exact options: ```shell allure --version allure agent --help allure agent capabilities --json ``` If your project uses a wrapper, package script, or build-system command for Allure, run the checks through that wrapper. For reviewing Allure results that were already produced by a local run or CI job, use: ```shell allure agent inspect --help ``` ### Install or update Allure Install Allure Report 3 using the normal installation flow for your project or environment: - [Install Allure Report](/docs/v3/install/) - [Upgrade Allure Report](/docs/v3/upgrade/) Prefer the project-approved Allure command. In many repositories, that may be a package script, build task, wrapper, or local dependency rather than a global `allure` binary. ### Configure Allure reporting Agent Mode depends on useful Allure results. If your project does not emit Allure results yet, configure the appropriate Allure adapter for your test framework first. Find your test framework in the [Allure framework integrations](/docs/frameworks/) list and follow its setup instructions. In practice, `allure agent -- ` follows the same reporting constraints as `allure run -- `: the wrapped command must be the trusted project test command, it must run with the wrappers and environment that make Allure result emission work, and Allure must be able to discover the produced result directories. A report command cannot repair missing or empty results after the fact. Keep these boundaries clear: - **Allure results** are raw files emitted by framework adapters. - **Agent Mode output** is the Markdown and metadata generated for one `allure agent` run. - **Allure reports** are visual reports for humans. Do not write all three artifact types into the same directory. Framework result directories should usually keep `allure-results` as the final path component, such as `target/allure-results`, `build/allure-results`, or `out/allure-results`. ## Set up your coding agent ### Add Allure skills for coding agents If you use Codex, Claude, OpenCode, or another agent that supports local skills, add the Allure skills to the repository where your tests live: ```shell npx skills add allure-framework/skills ``` The repository currently includes these Allure skills: - **`allure-configure-reporting`** — configures Allure adapters, result directories, local report commands, evidence integrations, and CI artifact or report handling when a project does not emit useful Allure results yet. - **`allure-configure-agent-workflow`** — configures the local Agent Mode workflow once reporting exists by discovering project test commands, wrappers, result paths, supported `allure agent` options, and agent-output conventions. - **`allure-test-agent`** — helps agents do day-to-day test work with Agent Mode, including coverage review, test authoring, failure debugging, flaky-test investigation, evidence enrichment, and regression checks. ### Enable the local Agent Mode workflow After the skills are installed, configure the local Agent Mode workflow with the **`allure-configure-agent-workflow`** skill. If your agent supports direct skill invocation, call `allure-configure-agent-workflow` directly. Otherwise, ask your coding agent to enable Agent Mode for the project: **Codex:** ```shell codex "enable allure agent mode" ``` **Claude:** ```shell claude "enable allure agent mode" ``` **OpenCode:** ```shell opencode run "enable allure agent mode" ``` The agent should discover the local test commands, wrappers, result paths, supported `allure agent` options, and where generated Agent Mode output should be read from. If your project does not have Allure reporting yet, [finish the reporting setup first](#configure-allure-reporting), then return to Agent Mode. The expected result is: - a project guide at `docs/allure-agent-mode.md`; - a short root agent entry file, such as `AGENTS.md`, `CLAUDE.md`, or another project-specific instruction file, that links test-related work to `docs/allure-agent-mode.md`; - local notes about confirmed test commands, Allure wrappers, result paths, Agent Mode capabilities, output conventions, and known limitations. ## Try it on real test work There are many possible uses for Agent Mode. The main goal is always the same: give your coding agent test reporting it can inspect before it makes a claim. This can make agentic development work faster or more precise when the extra verification is useful. An agent that can review executed tests, skipped cases, retries, attachments, and report output has better ways to check its own work than an agent that only sees terminal text. The tradeoff is worth knowing up front: report-backed runs use more tokens and larger context than a plain terminal command. That is normal for evidence-rich agent work. Try it on the tasks below and judge the benefit in your own workflow. Here are a few real tasks you can try with Agent Mode. If you try them, please [share feedback with us](https://github.com/orgs/allure-framework/discussions) so we can learn where it helps most. **Codex:** ```shell codex "review checkout test coverage" ``` **Claude:** ```shell claude "add a test for expired-card checkout" ``` **OpenCode:** ```shell opencode run "debug the checkout retry test failure" ``` ### Review test coverage Use this when you want to know whether a feature, component, package, or test suite is actually covered at runtime. Expected agent behavior: choose a focused scope, run it through Agent Mode, compare intended coverage with observed runtime coverage, and report which tests ran, which cases were skipped or missing, what evidence looked weak, and which coverage claims are still uncertain. ### Add or improve a test Use this when you want the agent to add a missing test, strengthen weak assertions, or improve test evidence with meaningful steps, attachments, labels, parameters, or descriptions. Expected agent behavior: identify the behavior and risk, choose the test layer, add or improve the test, run the intended scope through Agent Mode, review the generated output, and only then explain what the new or changed test proves. ### Debug a failing test Use this when a test fails, hangs, or produces confusing terminal output. Agent Mode helps the agent review per-test evidence, findings, attachments, retries, logs, and process output before deciding what broke. Expected agent behavior: run the smallest failing scope, inspect the generated Agent Mode output before source code, separate product failures from test bugs, fixture or setup issues, environment problems, and reporting or adapter issues, then rerun when the fix needs validation. ### Investigate flaky behavior Use this when a test changes outcome across retries, environments, browsers, shards, or reruns. Expected agent behavior: inspect retries, timing, stale state, attachments, environment facts, and repeated evidence before recommending a fix. If the run is non-gating, quarantined, or too narrow, the agent should say so. ### Check regression safety Use this before accepting a change as safe. Ask the agent to run the smallest meaningful scope through Agent Mode and explain what the run actually proves. Expected agent behavior: select the smallest meaningful scope, run it through Agent Mode, review the output, and state exactly what is proven and what is not. The agent should avoid broad claims when the selected scope was only a smoke test, skipped important cases, lacked evidence, or depended on mocks or unavailable services. --- ## Viewing assertion failures: actual vs. expected > URL: https://allurereport.org/docs/assertion-diff/ # Viewing assertion failures: actual vs. expected (Allure 3) When a test fails due to an assertion error, Allure Report can show you exactly what went wrong by comparing the expected value (what your test anticipated) against the actual value (what your code produced). ## The error block Each failed test displays an error block containing the assertion message. Clicking the message expands the full stack trace below it. Images: /images/assertion-diff/error-block-light.png, /images/assertion-diff/error-block-dark.png ## The diff viewer If the test framework reports both an actual and an expected value, a **Show difference** button appears in the error block. Clicking it opens a comparison modal with two controls: **View mode** - choose how the comparison is laid out: - **Unified** - shows the expected value in grey, followed immediately by the diff. Added text is highlighted green, removed text red. - **Side-by-side** - shows expected on the left and actual (with the diff applied) on the right. **Compare by** - choose the granularity of the diff: - **Lines** (default) - highlights entire lines that differ, good for multi-line strings or JSON. - **Words** - highlights individual words, useful for prose or long sentences. - **Chars** - highlights individual characters, useful for spotting subtle typos or whitespace differences. Images: /images/assertion-diff/diff-viewer-light.png, /images/assertion-diff/diff-viewer-dark.png --- ## Attachments > URL: https://allurereport.org/docs/attachments/ # Attachments When you are investigating an unexpected test failure, sometimes it is convenient to not just read [the list of completed steps](/docs/steps/), but also see the same data the test saw. For example, if you could see a screenshot of the web page just before the failure, you often could quickly understand whether it looks the way it was expected by the test author. With Allure, you can attach any kinds of files to the whole test result or to the current test step or fixture (depending on the Allure integration implementation). In the report, Allure will provide both a link for downloading the attachment and a preview for supported media types. Images: /images/attachments/attachment-png-annotated-allure3-light.png, /images/attachments/attachment-png-annotated-allure3-dark.png, /images/attachments/attachment-png-annotated-allure2-light.png, /images/attachments/attachment-png-annotated-allure2-dark.png ### Screenshots Media types: `image/bmp`, `image/gif`, `image/jpeg`, `image/png`, `image/svg+xml`, `image/tiff`, `image/*`. Whenever a test interacts with a graphical interface (whether it is a desktop app, a mobile app, or a website), attaching screenshots to the test report increases its informativeness significantly. Just glancing at a screenshot answers a lot of questions, like "Is the website loaded properly?", "Is a form filled correctly?", "Aren't there any unexpected popups in the way?", etc. Images: /images/attachments/attachment-png-allure3-light.png, /images/attachments/attachment-png-allure3-dark.png, /images/attachments/attachment-png-allure2-light.png, /images/attachments/attachment-png-allure2-dark.png - A common approach is to take a screenshot as soon as you detect a failure — this is usually when the reader has the most questions. In a lot of programming languages, you can perform post-failure operations using a `catch` or `except` statement — these are great places to attach screenshots for the report. - If you don't mind using some more disk space, you can help the reader even more by taking screenshots on each [step](/docs/steps/) of your test, even before a failure. For complex test scenarios, this will help understand where exactly something went wrong, without resorting to manual testing. Note that in some situation, it may be even more informative to [attach a video](#videos). The functions for taking screenshots are different in each test framework, and some Allure integrations may attach each taken screenshot automatically. Generally, though, you need to get either a filename of the screenshot or its content as raw bytes and then use an Allure integration's function to attach it to the report. ### Visual comparisons Images: /images/attachments/attachment-imagediff-allure3-light.png, /images/attachments/attachment-imagediff-allure3-dark.png, /images/attachments/attachment-imagediff-allure2-light.png, /images/attachments/attachment-imagediff-allure2-dark.png Media type: `application/vnd.allure.image.diff`. A visual comparison, also known as a "visual diff", is a way to check if one state of a user interface is identical to another state or to an expected state. The comparison is done by analyzing two screenshots on a pixel-by-pixel basis. If the number of pixels that do not match is higher than a certain threshold, it is considered a failure. Visual comparisons can be performed by various test frameworks or additional utilities, such as [pixelmatch](https://github.com/mapbox/pixelmatch). It is a good idea to attach the result of a visual comparison to the test report for a human inspection. With Allure Report, the reader will be able to switch between the following display modes for each visual comparison: - **Diff** — view the pixel-level difference image, as produced by the test framework or utility, - **Actual** — view the actual screenshot captured during the test run, - **Expected** — view the expected reference screenshot, - **Side by side** — view the actual and expected images side by side, - **Overlay** — view the two original images on both sides of the mouse pointer. Some Allure integrations provide functions for attaching visual comparisons. Info: If the integration for your framework does not provide such a function yet, follow the steps below to construct the necessary data. You will need to use your programming language's JSON- and Base64-related functions. 1. Prepare all three images: the expected screenshot, the actual screenshot, and a visual comparison image. 1. Convert each image into a Base64 data URL, beginning with `data:image/png;base64,`. 1. Put the three strings into a JSON object under the keys `expected`, `actual`, and `diff`. 1. Encode the JSON object into a string or a byte array, depending on the Allure integration. 1. Attach the data and specify the `application/vnd.allure.image.diff` media type. **JavaScript (Mocha):** ```js const { allure } = require("allure-mocha/runtime"); const fs = require("fs"); it("attachment", () => { // ... // Read the three files and encode to base64 const expected = fs.readFileSync("expected.png", "base64"); const actual = fs.readFileSync("actual.png", "base64"); const diff = fs.readFileSync("diff.png", "base64"); // Wrap in a JSON, encode as string const content = JSON.stringify({ expected: `data:image/png;base64,${expected}`, actual: `data:image/png;base64,${actual}`, diff: `data:image/png;base64,${diff}`, }); // Attach to the test report allure.attachment("Screenshot diff", content, "application/vnd.allure.image.diff"); }); ``` **Java (JUnit 5):** ```java import io.qameta.allure.Allure; import org.json.JSONObject; import org.junit.jupiter.api.Test; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; import java.util.Base64; class TestImageDiff { @Test void testImageDiff() throws IOException { // ... // Read the three files as bytes byte[] expected = Files.readAllBytes(Paths.get("expected.png")); byte[] actual = Files.readAllBytes(Paths.get("actual.png")); byte[] diff = Files.readAllBytes(Paths.get("diff.png")); // Encode the data, wrap in a JSON, encode as bytes String content = new JSONObject() .put("expected", "data:image/png;base64," + Base64.getEncoder().encodeToString(expected)) .put("actual", "data:image/png;base64," + Base64.getEncoder().encodeToString(actual)) .put("diff", "data:image/png;base64," + Base64.getEncoder().encodeToString(diff)) .toString(); // Attach to the test report // (in Allure Java 2.x, use Allure.addAttachment() with the same arguments) Allure.attachment("Screenshot diff", "application/vnd.allure.image.diff", content); } } ``` **Python (pytest):** ```python import json import base64 from pathlib import Path import allure def test_attach_image_diff(): ... # Read the three files and encode to base64 expected = base64.b64encode(Path('expected.png').read_bytes()).decode() actual = base64.b64encode(Path('actual.png').read_bytes()).decode() diff = base64.b64encode(Path('diff.png').read_bytes()).decode() # Wrap in a JSON, encode as bytes content = json.dumps({ 'expected': f'data:image/png;base64,{expected}', 'actual': f'data:image/png;base64,{actual}', 'diff': f'data:image/png;base64,{diff}', }).encode() # Attach to the test report allure.attach(content, name='Screenshot diff', attachment_type='application/vnd.allure.image.diff') ``` **C# (NUnit):** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestImageDiff { [Test] public void Test_ImageDiff() { // ... AllureApi.AddScreenDiff("expected.png", "actual.png", "diff.png"); } } ``` ### Videos Media types: `video/mp4`, `video/ogg`, `video/webm`. Some test frameworks and libraries provide means to record videos of what happens in a window or on a webpage. Attaching such a video to the test report serves a purpose similar to [attaching a screenshot](#screenshots). The reader will be able to better understand how the system reacted to the user input or other events and what caused an unexpected behavior. Images: /images/attachments/attachment-video-allure3-light.png, /images/attachments/attachment-video-allure3-dark.png, /images/attachments/attachment-video-allure2-light.png, /images/attachments/attachment-video-allure2-dark.png In the test report, click on an attached video to play it in the browser. ### Text #### Plain text Media types: `text/plain`, `text/*`. Images: /images/attachments/attachment-txt-allure3-light.png, /images/attachments/attachment-txt-allure3-dark.png, /images/attachments/attachment-txt-allure2-light.png, /images/attachments/attachment-txt-allure2-dark.png #### Formatted text Media types: `text/html`, `text/plain`, `text/*`. Attaching texts to the test report can serve various purposes — from checking texts fround in a user interface under test to storing full logs of some complex operations. Allure supports two types of text attachments: plain text and formatted text. Formatted text is rendered as HTML in an isolated frame: scripts in the attachment can run, but they can't access the report or load data over the network. Images: /images/attachments/attachment-html-allure3-light.png, /images/attachments/attachment-html-allure3-dark.png, /images/attachments/attachment-html-allure2-light.png, /images/attachments/attachment-html-allure2-dark.png Some Allure integrations may automatically attach the content from `stdout` and `stderr` as plain text attachments. Check your integration's documentation for more details. ### Tables Images: /images/attachments/attachment-table-allure3-light.png, /images/attachments/attachment-table-allure3-dark.png, /images/attachments/attachment-table-allure2-light.png, /images/attachments/attachment-table-allure2-dark.png Media types: `text/csv`, `text/tab-separated-values`. You can present arbitrary data as tables in Allure Report. To do so, format the data as comma-separated values (CSV) or tab-separated values (TSV) and attach it to the test result. The table will be displayed on the test result page without need to download and open the file. ### URI lists (Allure 2) ![Attaching URI lists](/images/attachments/attachment-uri-list.png "Attaching URI lists") Media types: `text/uri-list`. Sometimes, when testing an application, especially a web application, you may want to collect and analyze a list of links. For example, you may want to check if all links on a website are valid. By attaching the list of links to the test report, you will help the future reader inspect them again manually when investigating a failure. The URI list format is just that — a list of URIs, one URI per line. Allure Report will make each link clickable in the test report. ### Documents Media types: `text/xml`, `application/xml`, `application/json`, `application/yaml`. If you attach an XML, JSON, or YAML document to your test report and specify the corresponding media type, Allure Report will display the document with syntax highlighting enabled. Images: /images/attachments/attachment-json-allure3-light.png, /images/attachments/attachment-json-allure3-dark.png, /images/attachments/attachment-json-allure2-light.png, /images/attachments/attachment-json-allure2-dark.png ### HTTP exchanges Media type: `application/vnd.allure.http+json`. An HTTP exchange attachment captures one complete HTTP request-response cycle — method, URL, headers, cookies, query parameters, and body on both sides, plus timing and, for failed requests, error details — as a single structured attachment. Allure Report renders it as a request/response viewer instead of a raw JSON dump, so the reader can inspect a call the same way they would in a browser's network tab. Images: /images/attachments/attachment-httpexchange-allure3-light.png, /images/attachments/attachment-httpexchange-allure3-dark.png, /images/attachments/attachment-httpexchange-allure2-light.png, /images/attachments/attachment-httpexchange-allure2-dark.png The format has a shared, versioned schema (currently schema version `1`). HTTP exchange attachments are usually produced automatically by integrations that wrap an HTTP client and capture its requests. In Allure Java 3.0 and newer, you can also build an exchange yourself and attach it with `Allure.addHttpExchange()`. The following integrations produce HTTP exchange attachments: - Java (Allure Java 3.0 and newer): - [Allure REST Assured](/docs/restassured-configuration/#configure-the-http-exchange-attachment) - [Allure Java HTTP Client](https://github.com/allure-framework/allure-java/tree/main/allure-java-httpclient) (`java.net.http.HttpClient`) - [Allure OkHttp](https://github.com/allure-framework/allure-java/tree/main/allure-okhttp3) - [Allure Apache HttpClient 4](https://github.com/allure-framework/allure-java/tree/main/allure-httpclient) and [Allure Apache HttpClient 5](https://github.com/allure-framework/allure-java/tree/main/allure-httpclient5) - [Allure Spring Web](https://github.com/allure-framework/allure-java/tree/main/allure-spring-web) (`RestTemplate`) - [Allure JAX-RS](https://github.com/allure-framework/allure-java/tree/main/allure-jax-rs) - [Allure gRPC](https://github.com/allure-framework/allure-java/tree/main/allure-grpc) - [Allure Karate](https://github.com/allure-framework/allure-java/tree/main/allure-karate) - JavaScript: - [Allure Axios](/docs/axios-reference/#attachment-schema) - [Allure Fetch](/docs/fetch-reference/#attachment-schema) - Go: - [Allure Go `httpexchange` package](/docs/go-reference/#http-exchange-attachments) - Rust: - [Allure Reqwest](/docs/reqwest-reference/#attachment-schema) See the corresponding integration's documentation for the exact field list and default redaction rules. ### Playwright traces Media type: `application/vnd.allure.playwright-trace`. A Playwright trace is Playwright's own [trace](https://playwright.dev/docs/trace-viewer) — a zip archive recording DOM snapshots, network activity, console logs, and actions during a test run. [Allure Playwright](/docs/playwright/#playwright-traces) detects the trace file Playwright produces and tags it with this media type so Allure Report can open it directly in Playwright's Trace Viewer, instead of offering it as a plain zip download. The archive itself isn't modified by Allure — only the attached media type is changed. Other integrations don't generate Playwright traces themselves, but provide a method to attach an existing trace archive with this same media type: [Allure Dart](/docs/dart-reference/#attachments) (`attachTrace`), [Allure Playwright Java](/docs/playwright-java-reference/#attachtrace) (`AllurePlaywright.attachTrace`), and [Allure Rust](/docs/rust-reference/#attachments) (`attach_trace`/`attach_trace_named`). Images: /images/attachments/attachment-playwright-trace-allure3-light.png, /images/attachments/attachment-playwright-trace-allure3-dark.png, /images/attachments/attachment-playwright-trace-allure2-light.png, /images/attachments/attachment-playwright-trace-allure2-dark.png - In Allure 3, clicking the attachment opens the trace in a new browser tab. - In Allure 2, the Trace Viewer is embedded inline in the attachment preview, with a download link alongside it. Both open the hosted viewer at `https://trace.playwright.dev/` and hand the trace data to it with `postMessage`. The trace bytes are never uploaded anywhere — they stay in the browser and are passed directly to the viewer page — but loading the viewer page itself requires internet access, since Allure Report doesn't bundle an offline copy of it. ### Other files Allure Report does not enforce any limitations on what files can be attached to a test report, even though it cannot provide previews for all of them. For any unsupported type, a **Click to download attachment** link will be displayed instead of a preview. Images: /images/attachments/attachment-zip-allure3-light.png, /images/attachments/attachment-zip-allure3-dark.png, /images/attachments/attachment-zip-allure2-light.png, /images/attachments/attachment-zip-allure2-dark.png If you believe Allure Report lacks support of an important media type, feel free to [start a discussion](https://github.com/orgs/allure-framework/discussions)! --- ## Categories > URL: https://allurereport.org/docs/categories/ # Categories Allure Report helps you track errors (and, optionally any other kind of test results) by dividing them into **categories** based on the statuses and output messages of the tests. Tip: Each result belongs to only one category. It can be one of the [default categories](#default-categories) Allure assigns automatically, or a [custom category](#custom-categories) you define yourself. On the **Categories** tab in a test report, the tests are grouped by their category, their error message, and optionally by several other [levels of categorization](#custom-categories). ## Default categories Allure defines two default error categories, both of which include tests that did not finish successfully: - **Product errors** - [Failed](/docs/test-statuses/#failed) tests. - **Test errors** - [Broken](/docs/test-statuses/#broken) tests. Images: /images/categories/categories-allure3-light.png, /images/categories/categories-allure3-dark.png, /images/categories/categories-allure2-light.png, /images/categories/categories-allure2-dark.png ## Custom categories You can define any number of custom categories. All test results are matched against your custom categories first, in the order you define them. Failed or broken tests that don't match any custom category are assigned to the default categories automatically. ### Allure Report 3 In Allure Report 3 you can configure custom categories in the `categories` field of the runtime [configuration file](/docs/v3/configure/). The examples below showcase the two main approaches for defining categories, with all available settings included. We will discuss the differences between these two approaches further down. **Categories via object matchers:** ```js import { defineConfig } from "allure"; export default defineConfig({ name: "Allure Report 3", output: "./allure-report", historyPath: "./history.jsonl", //required for flaky and status transitions detection categories: { rules: [ { name: "Critical regressions by layer", id: "critical-regressions-by-layer", // this section determines what's included into the category matchers: { statuses: ["failed", "broken"], flaky: false, labels: { severity: "critical" }, message: /expected|API/, // accepts regular expressions trace: /AssertionError|timeout|unexpected/, // accepts regular expressions transitions: ["regressed", "malfunctioned"], environments: ["staging", "production"], }, // this section determines how the category is presented groupBy: ["layer", "owner", "status"], groupByMessage: true, groupEnvironments: true, expand: true, hide: false, }, ], }, }); ``` **Categories via function matchers:** ```js import { defineConfig } from "allure"; export default defineConfig({ name: "Allure Report 3", output: "./allure-report", historyPath: "./history.jsonl", //required for flaky and status transitions detection categories: { rules: [ { name: "Critical regressions by layer", id: "critical-regressions-by-layer", // this section determines what's included into the category matchers: (data) => (data.status === "failed" || data.status === "broken") && !data.flaky && data.labels.some((l) => l.name === "severity" && l.value === "critical") && /expected|API/.test(data.message ?? "") && /AssertionError|timeout|unexpected/.test(data.trace ?? "") && (data.transition === "regressed" || data.transition === "malfunctioned") && (data.environment === "staging" || data.environment === "production"), // this section determines how the category is presented groupBy: ["layer", "owner", "status"], groupByMessage: true, groupEnvironments: true, expand: true, hide: false, }, ], }, }); ``` Both of these examples create a custom category that looks like this: Images: /images/categories/custom-allure3-light.png, /images/categories/custom-allure3-dark.png As shown in the examples above, custom categories are defined in the `categories` configuration field, which is an object with a `rules` array. Every rule in the `rules` array is a custom category definition, with the following available settings: - `name` (string, required) - the **category title** shown in the report. Can be changed freely without affecting category identity when `id` is set. - `id` (string, optional) - a **stable identifier** for the category. Used to track category identity across report runs, so changes to `name` doesn't break history or create duplicate entries. If omitted, `name` is used as the `id`. Must be a non-empty string with no leading/trailing whitespace or control characters. Two rules with the same `id` are merged. - `matchers` (required) - determines which **test results fall into the category** by looking for specified [metadata in your test results](/docs/how-it-works-test-result-file/). `matchers` can be: - **Object matcher** - a plain object with any combination of: - `statuses`: TestStatus[] - matches against [test status](/docs/test-statuses/), e.g. `["failed", "broken"]`. - `flaky`: boolean - determines whether the category should include [flaky](/docs/test-stability/#flaky-tests) tests. Requires [history](/docs/history-and-retries/#allure-report-3/) to work. - `labels`: { [labelName]: string | RegExp } - matches against [labels](/docs/how-it-works-test-result-file/#labels-array), as a regular expression. String input is converted into a RegExp object, so any regex metacharacters included in the string (like `.`, `*`, or `^`) will affect how the match behaves, e.g. `{ owner: /^alice/ }` will match all results where the `owner` label starts with `alice`. - `message`: (sub)string | RegExp - matches against the error message as a regular expression. Same as `labels`, any regex metacharacters in a provided string will affect the match. - `trace`: (sub)string | RegExp - matches against the stack trace as a regular expression. Same as `labels`, any regex metacharacters in a provided string will affect the match. - `transitions`: TestStatusTransition[] - matches against [transitions](/docs/test-stability/), e.g. `["new", "regressed"]`. Requires [history](/docs/history-and-retries/#allure-report-3/) to work. - `environments`: string[] - matches against [environments](/docs/environments/). Tests fitting all of these conditions will be included (AND logic applies here). - **Function matcher** - `(data) => boolean`, where `data` contains `status`, `labels`, `message`, `trace`, `flaky`, `duration`, `transition`, `environment`. Function matcher [requires dynamic config](/docs/v3/configure/#dynamic-config-file-capabilities-and-prerequisites) to work, and just like with object matchers, you have to enable [history](/docs/history-and-retries/#allure-report-3/) to match against `flaky` and `transition`, and set [environments](/docs/environments/) to match against `environment`. When using function matchers, you can additionally filter by `duration` (e.g. `(data) => data.duration > 5000` ), implement complex logic gates across multiple conditions and use arbitrary JavaScript expressions to fine-tune your categories. - **Array** - any mix of the above; the test matches if any matcher in the array matches (OR logic applies). - `groupBy` (array, optional) - controls the nesting hierarchy inside the category tree. Each element is either a built-in string selector or a custom label object: - Built-ins: `"flaky"`, `"owner"`, `"severity"`, `"transition"`, `"status"`, `"environment"`, `"layer"` - Custom labels: `{ label: "myCustomLabel" }` Example: `groupBy: ["severity", { label: "feature" }]` creates two levels of grouping — first by severity, then by feature — before showing individual test results. - `groupByMessage` (boolean, default true) - When true, adds an additional grouping level below `groupBy` levels, clustering tests that share the same error message. Long messages get a "show more/less" toggle in the UI. - `groupEnvironments` (boolean, optional) - controls whether to group tests from different environments together (showing the test name once, with environment-labeled leaves beneath it). - `expand` (boolean, default false) - whether the category is expanded by default in the UI. - `hide` (boolean, default false) - whether to exclude this category from the tree entirely (matched tests are still consumed by the rule, preventing them from matching later rules and default categories). You may also set `categories: false` to disable categories entirely (neither default categories nor custom rules will be applied). Details: Assorted custom categories configuration examples ```js import { defineConfig } from "allure"; export default defineConfig({ name: "Allure Report 3", output: "./allure-report", historyPath: "./history.jsonl", // required for flaky and transitions detection categories: { rules: [ // 1. Object matcher, all groupBy built-ins, groupByMessage, groupEnvironments explicit true { id: "critical-failures-by-layer", name: "Critical failures by layer", matchers: { statuses: ["failed"], labels: { severity: "critical" }, }, groupBy: ["severity", "owner", "layer", "status", "transition", "flaky"], groupByMessage: true, groupEnvironments: true, expand: true, hide: false, }, // 2. Custom label selector in groupBy, groupEnvironments explicit false { id: "failures-by-component", name: "Failures by component", matchers: { statuses: ["failed", "broken"], labels: { component: /.+/ }, }, groupBy: [{ label: "epic" }, { label: "feature" }], groupByMessage: true, groupEnvironments: false, expand: false, hide: false, }, // 3. Function matcher, groupByMessage false, groupEnvironments auto (omitted) { id: "long-running-tests", name: "Long running tests", matchers: (data) => data.duration > 5000, groupBy: ["owner"], groupByMessage: false, expand: false, hide: false, }, // 4. Array of matchers (OR logic), mix of object and function { id: "flaky-or-regressed", name: "Flaky or regressed", matchers: [{ flaky: true }, (data) => data.transition === "regressed"], groupBy: ["flaky", "transition"], groupByMessage: true, groupEnvironments: true, expand: true, hide: false, }, // 5. Message regex matching { id: "connection-errors", name: "Connection errors", matchers: { message: /ECONNRESET|connection failed|timeout/, statuses: ["broken"], }, groupBy: ["environment", "layer"], groupByMessage: true, groupEnvironments: false, expand: false, hide: false, }, // 6. Transition matcher, which includes new passed tests { id: "new-and-regressed-tests", name: "New and regressed tests", matchers: { transitions: ["new", "regressed"], }, groupBy: ["transition", "owner"], groupByMessage: false, groupEnvironments: true, expand: true, hide: false, }, // 7. Hidden category — consumes matched tests so they don't fall into defaults { id: "other-failed-and-broken-tests", name: "Other failed and broken tests", matchers: { statuses: ["failed", "broken"], }, groupBy: [], groupByMessage: false, groupEnvironments: false, expand: false, hide: true, // category won't be shown in the report }, ], }, plugins: { awesome: { options: { reportName: "Allure Report", singleFile: false, reportLanguage: "en", }, }, }, // environments configuration, set up for groupEnvironments to work environments: { staging: { matcher: ({ labels }) => labels.some(({ name, value }) => name === "env" && value === "staging"), }, production: { matcher: ({ labels }) => labels.some(({ name, value }) => name === "env" && value === "production"), }, }, }); ``` ### Allure Report 2 In Allure Report 2 you can define custom categories by placing a [categories file](/docs/how-it-works-categories-file/) into the test results directory (see [How it works](/docs/how-it-works-categories-file/)). Allure checks each test against all the categories in the file, from top to bottom. The test gets assigned the first matching category. When no matches are found, Allure uses one of the default categories if the test fails or no category otherwise. Some Allure plugins can generate the categories file automatically based on the project configuration. Check [your integration's documentation](/docs/frameworks/). --- ## Environments > URL: https://allurereport.org/docs/environments/ # Environments (Allure 3) Allure Report 3 offers a new way to organize test results by environments, allowing you to group together tests by operating systems, browsers, deployments - or any other ways of categorization, and compare results from different environments easily. This feature is based on test result labels: you can map certain labels (or combinations of labels) to a specific environment name, and Allure Report 3 will split all test results on the report home page into separate hierarchies, which you can switch between via the **Environments** dropdown menu. Images: /images/environments/environments_light.png, /images/environments/environments_dark.png Additionally, on the **Environments** tab of every test result you will be able to see available results of the same test from other environments. Images: /images/environments/environments-tab-light.png, /images/environments/environments-tab-dark.png ## Prerequisites - This feature requires you to use a [dynamic configuration file](/docs/v3/configure/#file-format-and-syntax) (`.mjs`, `.cjs`, `.js`) for Allure Report 3. - You should set the labels storing environment information in your tests code. Please refer to the documentation of the [framework integration](/docs/frameworks/) you're using to find out how. ## Basic configuration Environments are governed by the `environments` [configuration parameter](/docs/v3/configure/#_6-environments). The keys of the `environments` object are environment **IDs** — short identifiers used internally (latin letters, digits, underscores, and hyphens only, up to 64 characters). Each entry requires a `matcher` function and also accepts an optional `name` field for the display name shown in the report. If `name` is omitted, the ID is used as the display name. The examples below show how to assign environments based on a single label or multiple labels. Allure will look for labels named `os` and, in the two-label example, `browser`, and assign environments accordingly: **One label → One environment:** ```js { environments: { windows: { matcher: ({ labels }) => labels.find(({ name, value }) => name === "os" && value === "Windows"), }, macos: { matcher: ({ labels }) => labels.find(({ name, value }) => name === "os" && value === "macOS"), }, linux: { matcher: ({ labels }) => labels.find(({ name, value }) => name === "os" && value === "Linux"), } } } ``` **Two labels → One environment:** ```js { environments: { "windows-chrome": { matcher: ({ labels }) => labels.some(l => l.name === "os" && l.value === "Windows") && labels.some(l => l.name === "browser" && l.value === "Chrome"), }, "windows-firefox": { matcher: ({ labels }) => labels.some(l => l.name === "os" && l.value === "Windows") && labels.some(l => l.name === "browser" && l.value === "Firefox"), }, "ubuntu-chrome": { matcher: ({ labels }) => labels.some(l => l.name === "os" && l.value === "Ubuntu") && labels.some(l => l.name === "browser" && l.value === "Chrome"), }, "ubuntu-firefox": { matcher: ({ labels }) => labels.some(l => l.name === "os" && l.value === "Ubuntu") && labels.some(l => l.name === "browser" && l.value === "Firefox"), } } } ``` The above configurations result in environment menus like these: Images: /images/environments/dropdown-light.png, /images/environments/dropdown-dark.png To set a display name that is different from the ID, add the `name` field: ```js { environments: { prod_env: { name: "Production", // shown in the report; ID "prod_env" is used internally matcher: ({ labels }) => labels.find(({ name, value }) => name === "env" && value === "production"), }, } } ``` ### How matcher functions work - Allure reads [test result files](/docs/how-it-works-test-result-file/) and loads all `labels` from each test result. - The matcher function receives `{ labels: TestLabel[] }` from the Allure Report process. Each label is `{ name: string, value: string }`. - If the test result belongs to the environment, the matcher function returns `true`. - **First match wins** - tests results cannot be assigned to multiple environments, so the order of environments in the config file matters. - If Allure can't match a test result to any environment, it assigns it to the `default` environment, so no test results get lost. ## Environment variables You can add any metadata that applies to the whole report via the global [`variables` configuration parameter](/docs/v3/configure/#variables). Allure displays it at the top of the report homepage in the **Variables** section. In addition to global variables, you can set environment-specific variables, which can override existing global variables, or be displayed in addition to them, when you select a specific environment: ```js // Global vs Environment-Specific Variables { variables: { "App Version": "2.5.1", // Shown everywhere "Build Number": "#1234", // Shown everywhere "Database": "prod-db-primary" // Shown, unless overridden }, environments: { staging: { matcher: ({ labels }) => labels.some(({ name, value }) => name === "env" && value === "staging"), variables: { "Server": "staging.example.com", // Shown for staging "Database": "staging-db", // Overrides global variable for staging } }, production: { matcher: ({ labels }) => labels.some(({ name, value }) => name === "env" && value === "production"), variables: { "Server": "api.example.com", // Shown for production } } } } ``` Images: /images/environments/variables-light.png, /images/environments/variables-dark.png ## Forcing a single environment When you don't need label-based matching and already know which environment a run belongs to, you can use the `environment` [top-level configuration option](/docs/v3/configure/#environment) to assign all test results to one environment ID directly, bypassing matcher logic entirely: ```javascript export default { environment: "staging", }; ``` This is the config-file equivalent of the `--environment` CLI flag described below. The CLI flag takes priority if both are set. ## Restricting valid environments The `allowedEnvironments` [top-level configuration option](/docs/v3/configure/#allowedenvironments) lets you declare an explicit allowlist of environment IDs. Any environment ID — from the `environments` map, the `environment` override, or test result data — that is not in the list causes a validation error at startup: ```javascript export default { allowedEnvironments: ["staging", "production", "windows", "macos"], }; ``` Use this to catch environment misconfigurations early, especially in projects where the set of valid environments is fixed. ## CLI override and CI/CD workflows Allure CLI allows you to override environment assignment for the entire test launch by using the `allure run` wrapper command. Use `--environment` to specify the environment by its **ID** (the config key): ```bash allure run --environment="" -- ``` Or use `--environment-name` to specify it by **display name**: ```bash allure run --environment-name="" -- ``` `--environment` takes priority over `--environment-name` if both are provided. Both take priority over the `environment` value in the config file. For example: ```bash allure run --environment="prod_env" -- pnpm test allure run --environment-name="Production" -- pnpm test ``` Its main use is running tests in multiple environments in CI/CD workflows and creating combined reports with the help of the [multistage builds](/docs/multistage-builds/) feature: - You run your tests in different environments and save each run as a [dump archive](/docs/multistage-builds/#what-is-a-dump-archive). - Then you collect all dump archives and build a single multistage report from them. - Allure maps test results from all runs to their assigned to environments and you get a well-structured unified report, where you can easily see and compare how your tests perform in different contexts. Below is a snippet from the [Allure Report 3 Demo workflow](https://github.com/allure-framework/allure3-demo/blob/main/.github/workflows/build.yml) on GitHub, illustrating this idea. ```yaml jobs: test: name: Build strategy: matrix: os: [ubuntu-latest, macos-latest] runs-on: ${{ matrix.os }} steps: # ... # Test environment set up steps # ... - name: Test Project # One full test run per os saved as a dump archive run: pnpm allure run --config=./allurerc.mjs --environment="${{ matrix.os }}" --dump="allure-results-${{ matrix.os }}" -- pnpm test - name: Upload test results if: always() uses: actions/upload-artifact@v5 with: name: allure-results-${{ matrix.os }} path: ./allure-results-${{ matrix.os }}.zip compression-level: 0 report: runs-on: ubuntu-latest needs: test steps: # ... # Report generator set up steps # ... # Fetching the dump archive for the Ubuntu test run - name: Download allure results from Ubuntu uses: actions/download-artifact@v6 with: name: allure-results-ubuntu-latest path: ./ # Fetching the dump archive for the macOS test run - name: Download allure results from macOS uses: actions/download-artifact@v6 with: name: allure-results-macos-latest path: ./ - name: Build report # Building the multistage report from all fetched dump archives run: pnpm allure generate --dump="allure-results-*.zip" --output=./allure-report # ... # Report deployment steps # ... ``` Tip: While the `--environment` option overrides the label mapping rules in the config file, it will still pick up any variables specified for the environment matching the one you pass. This means you can define environment variables in the config even when using `--environment` to assign environments. ```js { environments: { staging: { // matcher redundant when using --environment flag variables: { "Server": "staging.example.com", "Database": "staging-db" } }, production: { variables: { "Server": "api.example.com", "Database": "prod-db-primary" } } } } ``` ## The old environments feature Allure 3 still supports adding environment metadata to a report using the [legacy Allure 2 method](/docs/how-it-works-environment-file/): if an `environment.properties` file is discovered in the results directory, Allure displays the data from it at the top of the report page in the **Metadata** section. Images: /images/environments/old-environments-light.png, /images/environments/old-environments-dark.png This can either complement or conflict with your environments configuration, so make sure to get rid of this file, or to harmonize it with your new environments set up - whichever suits you best. --- ## Export to CSV > URL: https://allurereport.org/docs/export-csv/ # Export to CSV Allure Report can generate CSV reports containing the same test data that goes into the HTML report, which you can open and analyze in external software, such as Microsoft Excel or Google Sheets. ![CSV export example](/images/export-csv/excel.png "CSV export example") ## Allure Report 3 Allure Report 3 has a CSV plugin, which can generate a CSV report along with or even instead of the HTML report automatically, if enabled in the [configuration file](/docs/v3/configure/#_9-plugin-configuration). ```js import { defineConfig } from "allure"; export default defineConfig({ name: "Allure Report", output: "./allure-report", plugins: { csv: { options: { fileName: "report.csv", }, }, }, }); ``` The CSV plugin accepts the following options: | Option | Description | Type | Default | | ---------- | ------------------------------- | --------------------------------------------------------------------------------- | ------------------- | | `fileName` | Name of the report file | `string` | `allure-report.csv` | | `fields` | Fields to include in the report | `{ header: string; accessor: keyof T \| ((result: T) => string \| undefined) }[]` | All fields | | `sort` | Sorting function | `(a: T, b: T) => number` | Default order | | `filter` | Filtering function | `(a: T) => boolean` | No filtering | You can also manually generate the CSV report via the CLI: ```bash allure csv [options] ``` For example, this will generate a `report.csv` file in the current directory from result files located in the `allure-results` subdirectory of the current directory: ```bash allure csv --output report.csv ./allure-results ``` Tip: If no `--output` option is provided to the `allure csv` command, it respects the `output` parameter of the configuration file, and saves `allure-report.csv` to the directory specified there. ## Allure Report 2 Allure Report 2 automatically generates CSV reports along with HTML reports. You can get these files by using the **Download CSV** file on top of the corresponding tab in the HTML report or just by browsing to the `data` subdirectory in the output directory. ### Suites data To get the suites CSV file, click the **Download CSV** button on the [Suites tab](/docs/v2/navigation/#suite-based-hierarchy), or just open the file directly as `data/suites.csv` in the output directory. Images: /images/export-csv/csv-suites-light.png, /images/export-csv/csv-suites-dark.png Each row in the file describes a single test result, with the following columns. - **Status** — the [test status](/docs/test-statuses/). Possible values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. - **Start Time**, **Stop Time** — information about when the test was executed, with seconds precision. - **Duration in ms** — how long it took to complete the test, with milliseconds precision. - **Parent Suite**, **Suite**, **Sub Suite** — the test's location in the suite-based hierarchy. - **Test Class**, **Test Method** — the test's class and method names, as reported by the Allure integration. - **Name** — the test's [title](/docs/v2/readability/#title). - **Description** — the test's [description](/docs/v2/readability/#description). ### Behaviors data To get the behaviors CSV file, click the **Download CSV** button on the [Behaviors tab](/docs/v2/navigation/#behavior-based-hierarchy), or just open the file directly as `data/behaviors.csv` in the output directory. Images: /images/export-csv/csv-behaviors-light.png, /images/export-csv/csv-behaviors-dark.png Each row in the file shows aggregate data about test results with identical behavior-based labels, with the following columns. - **Epic**, **Feature**, **Story** — the tests' location in the behavior-based hierarchy. - **FAILED**, **BROKEN**, **PASSED**, **SKIPPED**, **UNKNOWN** — number of tests with each [test status](/docs/test-statuses/) in the given epic, feature and story. ### Categories data To get the categories CSV file, click the **Download CSV** button on the [Categories tab](/docs/categories/), or just open the file directly as `data/categories.csv` in the output directory. Images: /images/export-csv/csv-categories-light.png, /images/export-csv/csv-categories-dark.png Each row in the file shows aggregate data about test results from the same category, with the following columns. - **Category** — the name of a test category. - **FAILED**, **BROKEN**, **PASSED**, **SKIPPED**, **UNKNOWN** — number of tests with each [test status](/docs/test-statuses/) in the given category. --- ## Export metrics > URL: https://allurereport.org/docs/export-metrics/ # Export metrics (Allure 2) Tip: This page describes how to export Allure Report 2 test statistics. To collect performance measurements, such as page load times, in Allure Report 3, see [Performance Metrics](/docs/performance-metrics/). In a test report directory, the `export` subdirectory contains files that can be imported to InfluxDB or Prometheus for building custom graphs. ## Export to InfluxDB The `export/influxDbData.txt` file contains aggregate test data in the [line protocol](https://docs.influxdata.com/influxdb/v2/reference/syntax/line-protocol/) format for [InfluxDB](https://www.influxdata.com/). When imported, it created the following measurements.
Measurement Tags
`launch_status` One tag per each possible [test status](/docs/test-statuses/): `failed`, `broken`, `passed`, `skipped`, `unknown`. Each value represents the number of tests with the status.
`launch_time` `duration` - the time it took to run all tests. \ `min_duration` - the duration of the fastest test run. \ `max_duration` - the duration of the slowest test run. \ `sum_duration` - all durations, combined. The values for `duration` and `sum_duration` may differ significantly if your test framework runs test in parallel or spends significant time preparing for each test.
`launch_problems` One tag per each non-empty [category](/docs/categories/). Each category's name is converted to lowercase, and the spaces are replaced with underscores, for example: `test_defects`, `product_defects`. Each value represents the number of tests in the category.
`launch_retries` `retries` - the number of unique test runs. \ `run` - the number of test runs that were [retries](/docs/history-and-retries/#retries) of other test runs.
## Export to Prometheus The `export/prometheusData.txt` file contains aggregate test data in the [text-based format](https://github.com/prometheus/docs/blob/main/docs/instrumenting/exposition_formats.md#text-based-format) format for [Prometheus](https://prometheus.io/). When imported, it created the following metrics.
Prefix Metrics
`launch_status_*` One metric per each possible [test status](/docs/test-statuses/): \ `launch_status_failed`, \ `launch_status_broken`, \ `launch_status_passed`, \ `launch_status_skipped`, \ `launch_status_unknown`. Each value represents the number of tests with the status.
`launch_time_*` `launch_time_duration` - the time it took to run all tests. \ `launch_time_min_duration` - the duration of the fastest test run. \ `launch_time_max_duration` - the duration of the slowest test run. \ `launch_time_sum_duration` - all durations, combined. The values for `launch_time_duration` and `launch_time_sum_duration` may differ significantly if your test framework runs test in parallel or spends significant time preparing for each test.
`launch_problems_*` One metric per each non-empty [category](/docs/categories/). Each category's name is converted to lowercase, the spaces are replaced with underscores, and the prefix is added, for example: `launch_problems_test_defects`, `launch_problems_product_defects`. Each value represents the number of tests in the category.
`launch_retries_*` `launch_retries_retries` - the number of unique test runs. \ `launch_retries_run` - the number of test runs that were [retries](/docs/history-and-retries/#retries) of other test runs.
--- ## Features > URL: https://allurereport.org/docs/features-overview/ # Features This chapter describes the key features of Allure Report. While all the features are worth learning, you may find it more productive to start with the ones that are the most useful specifically for you. Which ones are they? It depends on your role. ## For QA engineers As a QA engineer, you may want to start with features that will help you the most with the test results analysis: - [Agent Mode](/docs/agent-mode/) - let coding agents review test work with report-backed runtime context instead of terminal output alone. - [Test statuses](/docs/test-statuses/) - learn what statuses Allure supports and their meaning. - [Sorting and filtering](/docs/sorting-and-filtering/) - learn how to focus on what's important for you right now when you're reading the report. - [Environments](/docs/environments/) - see failed tests distribution across environments. - [Multistage builds](/docs/multistage-builds/) - build your reports in multiple stages without losing any important execution context. - [Categories](/docs/categories/) - figure out how errors are distributed across the test results and how many tests each error affects. - [Test stability analysis](/docs/test-stability/) - learn how to identify and tackle unstable tests, i.e., the tests that change their outcome for no apparent reason. - [History and retries](/docs/history-and-retries/) - access previous outcomes of the test you're currently focused on. - [Test steps](/docs/steps/) - divide a complex test into well-defined steps to better understand what went wrong and make a more precise conclusion about its outcome. - [Assertion diffs](/docs/assertion-diff/) - see exactly what value your test expected versus what it actually got, with a side-by-side or unified diff view. - [Quality gate](/docs/quality-gate/) - set a quality standard for your project. This feature fails a test run if it violates a defined set of rules, thus preventing the deployments that don't meet the desired quality standard. - [Resolutions](/docs/resolutions/) - mark failures caused by known bugs, as well as muted and accepted failures, so that you can focus on the new ones. - [Global errors and attachments](/docs/global-errors-and-attachments/) - capture process-level errors and outputs to diagnose test framework and infrastructure issues. ## For developers As a developer, you may find features that help you debug the most useful. Check these out: - [Agent Mode](/docs/agent-mode/) - give Codex, Claude, OpenCode, or another coding agent an agent-readable test report before it reviews coverage, changes tests, or debugs failures. - [Test steps](/docs/steps/) - a.k.a. a developer's best friends. Use them to log the execution results of code blocks in your tests. A failure in a test of any complexity becomes much easier to investigate when you can access all the contextual information the steps provide. This feature is the most powerful when combined with [attachments](/docs/attachments/). - [Attachments](/docs/attachments/) - enrich your test results with files (e.g., logs, HTML response dumps, screenshots), which, when a test fails, will help you get more context about the error. - [Assertion diffs](/docs/assertion-diff/) - instantly see the expected vs. actual values for a failed assertion without digging through logs. - [Global errors and attachments](/docs/global-errors-and-attachments/) - access complete `stdout`/`stderr` outputs, stack traces, and custom process-level diagnostic data to debug the test execution environment. - [History and retries](/docs/history-and-retries/) - access the previous results of a retried test to understand better what happened during each attempt. - [Self-hosted storage](/docs/self-hosted-storage/) - publish reports and keep cross-run history automatically from your CI pipeline, without relying on third-party hosting. - [Timeline](/docs/timeline/) - identify performance and parallelization issues of the tests. - [Performance metrics](/docs/performance-metrics/) - collect performance measurements, such as page load times, track them across runs, and fail the run when they get worse. ## For managers As a manager, you may be interested in features that help you better understand the state of the whole project or its essential parts: - [Visual analytics](/docs/visual-analytics/) - learn how to assess the current state of the project and its dynamic using charts and trend charts. - [Sorting and filtering](/docs/sorting-and-filtering/) - check how errors are distributed across the product features and what features are most affected. - [Export to CSV](/docs/export-csv/) - learn how to transfer the data from Allure Report to tools like Microsoft Excel or Google Sheets for further analysis. - [Export metrics](/docs/export-metrics/) - learn how to move the data from Allure Report to InfluxDB or Prometheus. ## The complete list Here is the complete list of Allure Report features described in this chapter: - [Agent Mode](/docs/agent-mode/) (Allure 3) - [Test steps](/docs/steps/) - [Attachments](/docs/attachments/) - [Test statuses](/docs/test-statuses/) - [Assertion diffs](/docs/assertion-diff/) (Allure 3) - [Sorting and filtering](/docs/sorting-and-filtering/) - [Environments](/docs/environments/) (Allure 3) - [Multistage builds](/docs/multistage-builds/) (Allure 3) - [Categories](/docs/categories/) - [Visual analytics](/docs/visual-analytics/) - [Test stability analysis](/docs/test-stability/) - [History and retries](/docs/history-and-retries/) - [Self-hosted storage](/docs/self-hosted-storage/) (Allure 3) - [Quality gate](/docs/quality-gate/) (Allure 3) - [Resolutions](/docs/resolutions/) (Allure 3) - [Global errors and attachments](/docs/global-errors-and-attachments/) - [Timeline](/docs/timeline/) - [Performance metrics](/docs/performance-metrics/) (Allure 3) - [Export to CSV](/docs/export-csv/) - [Export metrics](/docs/export-metrics/) --- ## Global Errors and Global Attachments > URL: https://allurereport.org/docs/global-errors-and-attachments/ # Global Errors and Global Attachments Starting with Allure Report 3, you can capture process-level information about your test runs, such as `stdout` and `stderr` outputs, test process errors and exit codes, and attach them to your test reports. Errors and attachments that your Allure integration records for the whole test run ([custom globals](#custom-globals)) are also shown in Allure Report 2.46.0 and newer, see [Allure Report 2](#allure-report-2). ## How to use To take advantage of this feature, you have to run your tests with the `allure run` wrapper command: ```bash allure run -- ``` For example, if `npm` is your test runner: ```bash allure run -- npm test ``` ## What data you can capture Global test run data appears in dedicated sections on the report homepage. ### Global Exit Code **Global exit code** is listed right under the report name and displays two values: the **actual exit code** (how the `allure run` process finished) and the **original exit code** (the exit code of the test process executed within `allure run`). These values may be different, for example, if you set [quality gate](/docs/quality-gate/) to tolerate a certain number of failed tests, or conversely, enforce some stricter quality standard than all tests in the run simply passing. Images: /images/globals/exit-code-light.png, /images/globals/exit-code-dark.png ### Global Errors **Global Errors** tab contains any process-level exceptions encountered during the test run, with stack traces for unexpected errors. Images: /images/globals/global-errors-light.png, /images/globals/global-errors-dark.png ### Global Attachments **Global Attachments** tab currently stores the entire `stdout` and `stderr` output generated by the test run. Images: /images/globals/global-attachments-light.png, /images/globals/global-attachments-dark.png ### Custom File Attachments You can attach additional files to the report as global attachments using the `globalAttachments` [configuration option](/docs/v3/configure/#globalattachments). It accepts an array of glob patterns (relative to the working directory) and attaches any matching files to the report independently of any test result: ```javascript export default { globalAttachments: ["./logs/*.log", "./screenshots/global/**/*.png"], }; ``` Only files inside the working directory are included; paths that resolve outside it are silently skipped. ### Custom Globals You can also configure your test pipeline to explicitly record errors and attachments as globals via the globals runtime API. It allows you to capture setup/teardown failures or any other information that doesn't belong to any individual test. Tip: This feature will be gradually rolled out for all framework integrations with runtime API support, starting with [Allure JavaScript](https://github.com/allure-framework/allure-js), [Allure Java](https://github.com/allure-framework/allure-java) 3.0, [Allure Dart](https://github.com/allure-framework/allure-dart), and [Allure Ruby](https://github.com/allure-framework/allure-ruby) 2.30. Please refer to your [integration's documentation](/docs/frameworks/) to see if it's already supported. In Allure Java 3.0 and newer, failures that don't belong to a single test, such as a failing `@BeforeAll` method or another fixture or lifecycle hook, are recorded as global errors automatically, instead of creating artificial test results. To record your own global error, call `Allure.globalError()` with a `Throwable` or a `StatusDetails` object: ```java import io.qameta.allure.Allure; Allure.globalError(new IllegalStateException("Test database snapshot is outdated")); ``` Allure Java writes global errors to the test results directory, so they appear in the report even if you don't use the `allure run` wrapper. In Allure Dart 1.3.0 and newer, when you use the drop-in imports, a failing `setUp`, `tearDown`, `setUpAll`, or `tearDownAll` callback is recorded as a global error, with a message such as `setUpAll failed: `. To record your own global error, call [`allure.globalError()`](/docs/dart-reference/#run-level-attachments-and-errors). In Allure Ruby 2.30 and newer, a failing `before(:suite)` or `after(:suite)` hook in RSpec, or a failing `BeforeAll` or `AfterAll` hook in Cucumber, is recorded as a global error. To record your own global errors and attachments, see [Run-level attachments and errors](/docs/rspec-reference/#run-level-attachments-and-errors) in the RSpec reference or the [Cucumber.rb reference](/docs/cucumberrb-reference/#run-level-attachments-and-errors). ## Allure Report 2 Allure Report 2.46.0 and newer shows the [custom globals](#custom-globals) recorded by your Allure integration. If the test run has global errors, the **Overview** page shows a warning that the run completed with errors, even if all the tests passed. The errors, with their stack traces, and the global attachments are listed on the **Run** tab, which only appears in reports that have global errors or attachments. ## How to turn the feature on and off - Default global errors and global attachments are on by default and may be turned off with the `--ignore-logs` option: ```bash allure run --ignore-logs -- ``` Additionally, the [Log plugin](/docs/v3/configure/#log-plugin) can limit the outputs saved as global attachments. - Custom globals captured via the globals runtime API can't be turned off except by removing them from the test code. ## Interactions with other features - [Quality gate](/docs/quality-gate/) results are not global errors: they are shown in the separate **Quality Gates** tab of the report. Using the `--ignore-logs` option to turn off global errors and attachments does not affect quality gates operation, and does not hide failed quality gates in the report. - [Multistage reports](/docs/multistage-builds/) list all global errors and all global attachments from all stages included in the report. --- ## History and retries > URL: https://allurereport.org/docs/history-and-retries/ # History and retries We know that one of the reasons regular testing is important is because it helps understand how the quality of your product is changing over time. If a certain test fails now, it is useful to know whether it also failed yesterday or even last week. In Allure Report, you can navigate between multiple runs of the same test — both across different reports via [Tests history](#tests-history) and within a single one via [Retries](#retries). ## Tests history ![Tests History](/images/diagrams/history.png "Tests History") Each report includes test results collected during a new **test launch**. However, the tests themselves are usually not new at all: you most likely use the same test code each time, slightly updating it only when necessary. Using each test's [unique identifier](#making-sure-tests-are-identified-correctly), Allure Report is able to link its latest result to the results of the same test from previous launches. This collection of links is known as the **tests history**. In a report with the tests history included, you can: - see what statuses a test had previously, - identify unstable tests (see [Test stability analysis](/docs/test-stability/)), - find tests that changed their status since last report (see [Filter tests by marks](/docs/sorting-and-filtering/#filter-tests-by-marks)), - see how various metrics change over time (see [Trend charts](/docs/visual-analytics/#trend-charts)). For most CI servers that we support, a corresponding Allure Report plugin will automatically keep the history and let you seamlessly navigate to previous test report. The same can be done when building reports locally, too, see [How to enable history](#how-to-enable-history) below. ### How to enable history Allure 2 and Allure 3 utilize different approach to working with history. #### Allure Report 3 In Allure Report 3, to enable history, simply set `historyPath` to the desired location of the JSONL history file in the [configuration file](/docs/v3/configure/#_3-history-options): ```json { "historyPath": "./history.jsonl" } ``` Each time Allure Report 3 generates a report while history is enabled, it reads previous historical data from the history file and writes a new line with all current test launch data to it. All historical data in Allure 3 is stored in this single file. For more details on the file structure, see [History files](/docs/how-it-works-history-files/#allure-report-3). #### Allure Report 2 Each time Allure Report 2 generates a test report (see [How it works](/docs/how-it-works/)), it also generates data into the `history` subdirectory. The JSON files in this subdirectory store all the information that Allure needs to include this test report's data in the next test report, see [History files](/docs/how-it-works-history-files/#allure-report-2). For Allure 2, enabling the history-related features is a matter of copying the `history` subdirectory into the test results directory before generating the next test report. Here is an example of how to do it, assuming that your project is configured to use `allure-results` and `allure-report` directories: 1. Make sure you have the previous report generated in the `allure-report` directory. 2. Remove the `allure-results` directory. 3. Run the tests. 4. Copy the `allure-report/history` subdirectory to `allure-results/history`. 5. Generate the new report. If you follow such a routine regularly, Allure will each time keep data from up to 20 latest reports in `allure-results/history`. This creates up to 20 history entries for each test and up to 20 columns in each chart. ## Retries ![Test Retries](/images/diagrams/retries.png "Test Retries") When you run the same test multiple times, Allure produces a separate [test result file](/docs/how-it-works-test-result-file/) for each run. The test report then includes the status and other information from the latest file, with the other runs shown as **retries**. A common reason to retry a test is if it failed for some environment-specific reasons (poor internet connection, no disk space available, etc.) and you need to run it again for more informative results. Some test frameworks may even retry failed tests automatically. In these cases, the total number of retries can become a useful metric in itself — you can see if it grows or declines on one of the [Trend charts](/docs/visual-analytics/#trend-charts). Another case is when one or more tests are known to be unstable, but it is still important to differentiate between the “fails always” and “fails sometimes” states. In Allure Report, you can find all tests that have changed their status after a retry using a special filter, see [Filter tests by marks](/docs/sorting-and-filtering/#filter-tests-by-marks). Note that unlike the latest attempt at each test, the retries do not get included in the [Tests history](#tests-history). ### How to keep retries To make sure the information about retries gets added to the test report, just avoid removing your test results directory (which is usually called `allure-results`). If the Allure integration for the framework that you use has an option to clean the directory before each run, disable that option. Then, when building a report, Allure Report will include all the files present in the directory, i.e., all the tests runs since you last cleaned the directory. Note, however, that keeping all the retries can skew the data for the Retries trend chart. The Allure Report plugins for CI servers always run tests with an empty `allure-results` directory, so that the report only includes automatic retries done by the test framework. ## Making sure tests are identified correctly For test results to appear correctly in the History and Retries tabs, Allure needs to identify which results belong to the same test across different runs. It does this using an [internal identifier called “history ID”](/docs/how-it-works-test-identifiers/#historyid). The history ID is automatically calculated by your framework's Allure integration. With some limitations and exceptions, discussed in the linked article, each integration assigns the history ID based on two factors: - the test's fully-qualified name, - the test's non-excluded [parameters](/docs/v3/readability/#parametrized-tests). This allows Allure to correctly group together test runs of the exact same test with the exact same parameters: - Tests results with the same history ID that are part of a single launch (located in the `allure-results` directory) are included in the Retries tab for the test in your report. - Tets results with the same history ID from different launches - the ones you copy to `allure-results/history`, are linked on the History tab in your report. Tip: To ensure that the test report will properly deal with the history and retries, make sure to pass to Allure all parameters that distinguish different versions of a test. Most Allure integrations take care of it automatically, but some may need additional code, such as [`Allure::parameter()`](/docs/phpunit-reference/#parametrized-tests) for Allure PHPUnit or [`@TestInstanceParameter`](/docs/testng-reference/#test-instance-parameters-via-dataprovider) for some configurations in Allure TestNG. Below are some common issues related to history ID inconsistencies, that you may encounter when setting up your reporting workflows. ### Common pitfall: tests from different environments are missing in the report Let's assume that you run a test on multiple platforms before collecting all the results into a single `allure-results` directory and building the report. Unless you make sure that different runs get different sets of [parameters](/docs/v3/readability/#parametrized-tests), Allure Report will show only one test result by default, moving other ones to its list [retries](#retries). We recommend adding one or more special parameters that will make up a different combination for each environment. For example, you may want to add such parameters as “os_version”, “browser_version”, “java_version”, etc., depending on the technologies used by your test. Be sure to check the “Parametrized tests” section of your Allure integration's documentation to see if it has any limitations. When an integration has multiple ways of declaring test parameters, it may or may not support all of them for the purposes of identifying a test. For example, Allure JUnit 5 and some other integrations can not distinguish tests that only differ in “dynamic” parameters, i.e., the parameters that were set after the test execution started. ### Common pitfall: a test's retries are displayed as separate tests Normally, when you run a test with a set of parameters different from the previous times, Allure Report treats it as a new test. This means the runs are displayed in the test report separately instead of being grouped into a list of a single test's [retries](#retries). Additionally, Allure may fail to find links to the previous results in the test's [history](#tests-history). For example, if a parameter derives from current date and time or from a randomized value, you will get seemingly duplicated test results when you run the test more than once. If this is not the desired behavior for your test, consider removing such a parameter or marking it as `excluded`. See your integration's reference to find out if it allows marking parameters `excluded` and how to do it. ## Retrying tests from Allure One popular feature request is the ability to rerun failed tests right from Allure Report. Some scenarios where it may be helpful include: - checking if the test results are consistent, - retrying flaky tests, - following the TDD process with Allure. Starting with [Allure Report 3](https://github.com/allure-framework/allure3) some of these scenarios are now possible. The `allure run` command allows to run any framework-specific test command, for example: ```bash allure run -- npm run test ``` This command includes an option to rerun failed tests up to a set number of times. The following command will rerun failed tests up to three times, and the results of multiple runs will be listed as retries in the generated report. ```bash allure run --rerun 3 -- npm run test ``` If you need more, consider using [Allure TestOps](https://qameta.io/). It supports rerunning an arbitrary set of tests in a CI pipeline or in manual mode. --- ## Multistage Builds > URL: https://allurereport.org/docs/multistage-builds/ # Multistage Builds (Allure 3) The Multistage Builds feature added in Allure Report 3 allows you to create unified reports from multiple sources of test run data. Instead of generating individual reports for every stage of a complex testing pipeline with multiple test runs and/or multiple test environments, you can save the full state of each stage as a dump archive (a `.zip` file with all test run data), and then build a single report from the resulting collection of archives. ## Typical use cases Building unified reports from: - test runs in different environments and contexts - multi-stage test pipelines, where each stage requires a separate test run - CI/CD setups involving some or all of the above ## What is a dump archive A dump archive stores all test results generated by the Allure integration - everything you would find in a regular results folder - in an optimized format. In addition to that, it captures the full state of the run, which includes important data produced by Allure Report 3 itself - data that does not normally go into the results folder. For this reason, manually collecting result files from multiple test runs and building a single report from them wouldn't give you the same level of detail as a report built from a collection of dump archives will. Here is what dump archives save and results folders don't: - [Test instability marks](/docs/test-stability/): if a [history file](/docs/how-it-works-history-files/#allure-report-3) is available during the test run, Allure will identify new, fixed, regressed, malfunctioned and flaky tests and store this information in the dump archive. - [Quality gates](/docs/quality-gate/): if the test run failed any quality gate rules you defined for your project, the details are saved in the archive. - [Resolutions](/docs/resolutions/): if resolutions are configured during the test run, the archive stores the resolutions assigned to the test results. - [Performance metrics](/docs/performance-metrics/): if the test run produced performance metrics, the archive stores them. - [Environment management](/docs/environments/): if you configured environments, be it via the configuration file, or via the `allure run --environment ...` command, this information is also saved. - [Global errors and global attachments](/docs/global-errors-and-attachments/): if anything went wrong at the system level during the test run, the terminal output and error log are saved to the archive. Any reports built from such dump archives include all this data. ## Building multistage reports To create a dump archive, use the `allure run` wrapper command, when running your tests. 1. Save the full state of your run into the `stage1.zip` file like so: ```bash allure run --dump=stage_1 -- ``` > For example, if `npm` is your test runner: > > ```bash > allure run --dump=stage_1 -- npm test > ``` 2. Generate a unified report from all dump archives matching the `stage_*.zip` pattern: ```bash allure generate --dump=stage_*.zip ``` 3. Open the resulting report with: ```bash allure open ``` ### Using the config file You can also produce a dump archive by setting the `dump` option in your [configuration file](/docs/v3/configure/#dump): ```javascript export default { dump: "./snapshots/stage_1", // produces ./snapshots/stage_1.zip }; ``` When `dump` is set, Allure processes the test results and saves the full run state to the archive **without generating a report**. You can then collect archives from all stages and build the final report with `--dump` on the generate command, same as above. Note that `dump` in the config and `--dump` on the `generate` command are complementary but opposite operations: the config option **writes** a dump archive; the CLI flag **reads** one. ## Environments in multistage reports Environments are fully integrated with dump archives, and are the main intended use case for this feature: you can run your tests in entirely different environments, save full test run states and create unified reports that utilize all new advanced features of Allure Report 3. Please refer to the [Environments documentation](/docs/environments/#cli-override-and-ci-cd-workflows) for a detailed example of how to use these features together. --- ## Performance Metrics > URL: https://allurereport.org/docs/performance-metrics/ # Performance Metrics (Allure 3) Besides test results, Allure Report 3 can collect numeric **performance metrics** from your test run: page load times, API response times, throughput, memory usage, bundle sizes, or any other measurement. The report shows them in a dedicated **Metrics** section and compares them with previous runs, and [quality gates](/docs/quality-gate/#metric-rules) can fail the run when a metric gets worse. See [View metrics in the report](#view-metrics-in-the-report) for where to find the Metrics section and what it shows. Tip: This feature is not related to [Export metrics](/docs/export-metrics/), which describes how to export Allure Report 2 test statistics to InfluxDB or Prometheus. ## Write metrics to the results directory Allure reads metrics from files in the [test results directory](/docs/how-it-works/), next to the test result files: - any file whose name ends with `-perf.json`, for example, `a1b2c3d4-perf.json`; - a file named `performance.json`. Each file contains a JSON array of measurements: ```json [ { "id": "home-page", "key": "page.load", "value": 1240, "start": 1792065109697, "stop": 1792065110937 }, { "id": "search-page", "key": "page.load", "value": 1515, "start": 1792065110937, "stop": 1792065112452 } ] ``` | Field | Type | Description | | ------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | `string` | Identifier of the measured object, for example, the page or the endpoint. Keep it the same across runs. | | `key` | `string` | Name of the metric. Measurements with the same key are aggregated in the report, and the key is used in the configuration and in quality gate rules. | | `value` | `number` | The measured value. | | `start` | `number` | Start time of the measurement, as a Unix timestamp in milliseconds. | | `stop` | `number` | End time of the measurement, as a Unix timestamp in milliseconds. | All fields are required. Allure ignores measurements with missing fields or a non-numeric `value`. Allure integrations don't write metric files, so you need to produce them yourself: from your tests, from a separate script, or by converting the output of a performance testing tool. For example, this Node.js script measures how long two pages take to load and saves the results as a metric file: ```javascript import { randomUUID } from "node:crypto"; import { mkdirSync, writeFileSync } from "node:fs"; const resultsDir = "allure-results"; const metrics = []; const measure = async (id, key, action) => { const start = Date.now(); await action(); const stop = Date.now(); metrics.push({ id, key, value: stop - start, start, stop }); }; await measure("home-page", "page.load", () => fetch("https://example.com/")); await measure("search-page", "page.load", () => fetch("https://example.com/?q=allure")); mkdirSync(resultsDir, { recursive: true }); writeFileSync(`${resultsDir}/${randomUUID()}-perf.json`, JSON.stringify(metrics)); ``` ## Configure metrics Metrics are shown in the report even without any configuration: under their keys, in the **Unconfigured** group. To give them readable titles and units, and to group related metrics together, add the `performance` section to the [configuration file](/docs/v3/configure/): **allurerc.mjs:** ```javascript import { defineConfig } from "allure"; export default defineConfig({ performance: { groups: { web: { title: "Web pages" }, api: { title: "API" }, }, metrics: { "page.load": { title: "Page load", unit: "ms", better: "lower", group: "web" }, "api.rps": { title: "API throughput", unit: "req/s", better: "higher", group: "api" }, }, }, }); ``` **allurerc.json:** ```json { "performance": { "groups": { "web": { "title": "Web pages" }, "api": { "title": "API" } }, "metrics": { "page.load": { "title": "Page load", "unit": "ms", "better": "lower", "group": "web" }, "api.rps": { "title": "API throughput", "unit": "req/s", "better": "higher", "group": "api" } } } } ``` **allurerc.yaml:** ```yaml performance: groups: web: title: Web pages api: title: API metrics: page.load: title: Page load unit: ms better: lower group: web api.rps: title: API throughput unit: req/s better: higher group: api ``` - `groups` - the groups to show the metrics in. Each key is a group ID, and `title` is the group name shown in the report. - `metrics` - settings for each metric key: - `title` - the name of the metric shown in the report. - `unit` - the unit added to the values, for example, `ms` or `req/s`. - `better` - `lower` if smaller values are better (durations, sizes), or `higher` if larger values are better (throughput, scores). The report uses it to highlight changes for the worse, and [quality gate](#check-metrics-with-quality-gates) delta rules use it to ignore changes for the better. - `group` - the ID of a group from `groups`. ## View metrics in the report When the report contains metrics, the **Report** menu in the top left corner gets the **Metrics** item: Images: /images/performance-metrics/metrics-menu-light.png, /images/performance-metrics/metrics-menu-dark.png It opens a table for each group, with a row for each metric: - **Count** - the number of measurements with this key in the current run. - **Total**, **Avg**, **Min**, **Max** - the sum, average, minimum and maximum of the values. - **Delta** - how much the average value has changed since the previous report. According to the `better` setting, a change for the worse is highlighted in red, and a change for the better in green. - **Trend** - a sparkline of the average values in the previous reports and the current one. Click a row to see the **Metric history**: the value and the change for each previous report. The **Current values** list at the bottom shows every individual measurement of the current run, with the file it was read from. Images: /images/performance-metrics/metrics-light.png, /images/performance-metrics/metrics-dark.png ## Track metrics over time To compare metrics with previous runs, Allure needs the test history: either a [history file](/docs/how-it-works-history-files/#allure-report-3) enabled with the `historyPath` option, or [Allure Report Storage](/docs/self-hosted-storage/). For each report, Allure saves the average value of each metric key to the history. The **Delta** and **Trend** columns, the metric history and the [delta quality gate rules](/docs/quality-gate/#metric-rules) are based on these saved values. With Allure Report Storage, history is kept per branch, and a branch that doesn't have enough reports of its own gets the main branch's reports added to its history (see [History depth and branches](/docs/self-hosted-storage/#history-depth-and-branches)). As a result, the first reports of a new branch compare their metrics with the latest report of the main branch. In a pull request or merge request pipeline, Allure uses the history of the target branch instead, on every run (see [History depth and branches](/docs/self-hosted-storage/#history-depth-and-branches)), so the delta rules check whether the change made the metrics worse than on the target branch. ## Check metrics with quality gates Allure Report has four [quality gate](/docs/quality-gate/) rules for metrics. They can fail a test run if a metric exceeds a limit or changes too much since the previous run: ```javascript import { defineConfig } from "allure"; export default defineConfig({ historyPath: "./history.jsonl", qualityGate: { rules: [ { metricMax: { key: "page.load", value: 1500 }, metricMaxDeltaPercent: { key: "api.rps", value: 10 }, }, ], }, }); ``` See [Metric rules](/docs/quality-gate/#metric-rules) for details. ## Metrics in multistage builds [Dump archives](/docs/multistage-builds/) store the metrics collected during each stage, and the report built from the archives shows the metrics from all stages. --- ## Quality Gate > URL: https://allurereport.org/docs/quality-gate/ # Quality Gate (Allure 3) Quality Gate is a built-in feature of Allure Report 3 that allows you to enforce a quality standard on your project, typically when it's about to advance from one deployment stage to another. This feature fails a test run if it violates the defined set of rules, thus preventing the deployment of a project that doesn't meet the desired quality standard. Allure Report has these built-in rules: | Rule name | Rule description | Rule function name | Units | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ----------------------------------------- | | `maxFailures` | Maximum allowed number of failed and broken tests | `maxFailuresRule` | Number of tests | | `minTestsCount` | Minimum required number of executed tests | `minTestsCountRule` | Number of tests | | `successRate` | Minimum required success rate: passed tests divided by passed, failed and broken tests (skipped and unknown tests are excluded) | `successRateRule` | Ratio (0.0 to 1.0) | | `newTests` | If `true`, the run must contain at least one new test, i.e., a test that isn't in the [history](/docs/history-and-retries/#tests-history) | `newTestsRule` | Boolean | | `maxDuration` | Maximum allowed duration for any single test | `maxDurationRule` | Milliseconds | | `allTestsContainEnv` | All tests must belong to the specified [environment](/docs/environments/) | `allTestsContainEnvRule` | Environment ID (string) | | `environmentsTested` | Every specified environment must have at least one test in the run | `environmentsTestedRule` | Array of environment IDs | | `metricMax` | Maximum allowed average value of a [performance metric](/docs/performance-metrics/) | `metricMaxRule` | Object, see [Metric rules](#metric-rules) | | `metricMin` | Minimum required average value of a performance metric | `metricMinRule` | Object, see [Metric rules](#metric-rules) | | `metricMaxDelta` | Maximum allowed change of a performance metric since the previous run | `metricMaxDeltaRule` | Object, see [Metric rules](#metric-rules) | | `metricMaxDeltaPercent` | Maximum allowed change of a performance metric since the previous run, in percent | `metricMaxDeltaPercentRule` | Object, see [Metric rules](#metric-rules) | The `newTests`, `metricMaxDelta` and `metricMaxDeltaPercent` rules compare the run with previous runs, so they require history: either a history file set with `historyPath`, or [Allure Report Storage](/docs/self-hosted-storage/). Without history, no test is considered new. All rules ignore earlier attempts of retried tests (only the latest attempt counts) and failures marked as **Muted** or **Accepted** with [resolutions](/docs/resolutions/). Failures linked to an **Issue** resolution are still counted. You can also use third-party rules and write entirely custom rules yourself. ## Basic Configuration ### Built-in Rules As other advanced features, you can configure Quality Gate in the runtime [configuration file](/docs/v3/configure). Depending on format, add these lines to it: **allurerc.mjs:** ```javascript import { defineConfig } from "allure"; export default defineConfig({ qualityGate: { rules: [ { : , }, ], }, }) ``` **allurerc.json:** ```json { "qualityGate": { "rules": [ { "": "" } ] } } ``` **allurerc.yaml:** ```yaml qualityGate: rules: - : ``` Tip: What config format to use? Generally, if you use only the built-in rules without any additional logic applied, you can stick to static `.json` or `.yaml` config files. If you want to customize anything, apply additional filters to analyzed test results, or use any custom rules, you need the dynamic `.mjs` or `.js` config, as you'll be adding executable code to it. For example: ```js import { defineConfig } from "allure"; export default defineConfig({ qualityGate: { rules: [ { maxFailures: 5, minTestsCount: 30, // executable code that validates only test results with critical severity label filter: (tr) => tr.labels.some((label) => label.name === "severity" && label.value === "critical"), }, ], }, }); ``` If either of the `maxFailures` and `minTestsCount` rules you configured here fail, your run will complete with exit code 1, and Quality Gate will print a message reporting what rule failed to the console. If no Quality Gate rules fail, the exit code will be 0. When Quality Gate is configured, it determines the exit code of `allure run`: if some tests fail, but their number stays within the limits you set, the exit code is still 0. The report shows both the actual exit code and the original exit code of the test process (see [Global exit code](/docs/global-errors-and-attachments/#global-exit-code)). The **Quality Gates** tab of the report lists the results of all rules, both failed and passed. For a rule that depends on specific tests, such as `maxFailures`, expand **Related test results** to see these tests. Images: /images/allure3/qualitygate-light.png, /images/allure3/qualitygate-dark.png ### Fast Fail If you want to terminate the test run immediately when at least one Quality Gate rule fails - add the `fastFail` property to the rule set: ```js import { defineConfig } from "allure"; export default defineConfig({ qualityGate: { rules: [ { maxFailures: 5, fastFail: true, }, ], }, }); ``` Not all rules make sense with `fastFail`. We do not recommend enabling `fastFail` in rulesets including rules like `minTestsCount` and `successRate`, because the values they check can only be calculated correctly after all test results are received, i.e., after the test run is already over. If you want to use these rules, we recommend moving them into a new rule set (see below) with `fastFail` set to false. ### Multiple Rule Sets You can also define multiple Quality Gate rule sets. This can be useful when you want to separate fast-failing rules from all others: ```js import { defineConfig } from "allure"; export default defineConfig({ qualityGate: { rules: [ { // use id field to make ruleset's rules easier to identify id: "fast-failing ruleset", maxFailures: 5, fastFail: true, }, { id: "non-fast-failing ruleset", successRate: 0.95, }, ], }, }); ``` ### Metric Rules The `metricMax`, `metricMin`, `metricMaxDelta` and `metricMaxDeltaPercent` rules check the [performance metrics](/docs/performance-metrics/) collected during the run. Each of them takes an object with the metric `key` and the limit `value`: ```js import { defineConfig } from "allure"; export default defineConfig({ historyPath: "./history.jsonl", qualityGate: { rules: [ { // the average page load time must not exceed 1500 metricMax: { key: "page.load", value: 1500 }, // the average API throughput must be at least 100 metricMin: { key: "api.rps", value: 100 }, // the average page load time must not grow by more than 200 since the previous run metricMaxDelta: { key: "page.load", value: 200 }, // the average API throughput must not drop by more than 10% since the previous run metricMaxDeltaPercent: { key: "api.rps", value: 10 }, }, ], }, }); ``` - All metric rules check the average value of all measurements with the given key in the current run. - A metric rule fails if the run has no measurements with the given key. - `metricMaxDelta` and `metricMaxDeltaPercent` compare the current average with the value saved in the history for the previous run, so they require history: either a [history file](/docs/how-it-works-history-files/#allure-report-3) set with `historyPath`, or [Allure Report Storage](/docs/self-hosted-storage/). If there is no previous value, the rule passes. With Allure Report Storage, the first reports of a new branch are compared with the main branch, and the reports of a pull request pipeline are compared with its target branch (see [Performance Metrics](/docs/performance-metrics/#track-metrics-over-time)). - If the metric has the `better` setting in the [`performance` configuration](/docs/performance-metrics/#configure-metrics), the delta rules only fail on changes for the worse: an increase for `better: "lower"`, a decrease for `better: "higher"`. Without it, a change in either direction counts. ## Run Allure with Quality Gates Enabled There are three Allure CLI commands that apply quality gates: - `allure run` - applies the quality gates marked with [`fastFail`](#fast-fail) as the test run proceeds and checks if it should be terminated early; once the test run finishes, it applies all of the quality gates to the final test results. - `allure generate` - applies all of the quality gates to the test results it reads. It generates the report (or writes the [dump archive](/docs/multistage-builds/), if `dump` is configured) first, and then exits with code 1 if any rule failed. - `allure quality-gate` - applies the quality gates against all test results from all matched directories. Doesn't generate reports. `allure run` and `allure generate` always use the rules specified in the configuration file. Tip: The `fastFail` rule sets can only stop the test run early if the Allure integration writes each test result as soon as the test finishes. Some integrations write all results at the end of the run; in this case, the rules are applied when the test run finishes. `allure quality-gate` uses the configuration by default, but also allows defining the rules via the CLI by using these options: - `--max-failures` - `--min-tests-count` - `--success-rate` If any of these are used, the configuration file is ignored and the rules from the CLI command are used instead. You can also assign an [environment](/docs/environments/) to all test results checked by the command, the same way as with `allure run`. This affects environment-related rules, such as `allTestsContainEnv`, and the environment shown in the quality gate results: - `--environment` - environment **ID** (the config key; letters, digits, underscores, and hyphens only) - `--environment-name` - environment **display name**; `--environment` takes priority if both are given For example: ```bash allure quality-gate --max-failures 5 --environment prod_env allure quality-gate --max-failures 5 --environment-name "Production" ``` ### Reruns Quality gates work with the [`--rerun`](/docs/history-and-retries/#retrying-tests-from-allure) option of `allure run`, which restarts the test command to retry failed tests: ```bash allure run --rerun 2 -- npm test ``` The number you pass is the total number of restarts available to the run. Each restart uses one: - When the test command finishes, Allure restarts it with a test plan that contains the failed and broken tests (except the ones with the **Muted** or **Accepted** [resolutions](/docs/resolutions/)) and the tests related to failed quality gate rules. The test plan is passed via the `ALLURE_TESTPLAN_PATH` environment variable, so the Allure integration must support test plans. - When a [`fastFail`](#fast-fail) rule set stops the test command early, the results are incomplete, so Allure restarts the full test command. The quality gates are applied to the latest attempt of each test. With `--rerun 0`, or without `--rerun`, a `fastFail` rule set stops the run without restarting it. ### Multi-Environment Builds If you run tests in several environments and build a single report from [dump archives](/docs/multistage-builds/), you can apply quality gates at either stage, or both: - **In each environment.** Add `qualityGate` to the configuration used for the test run in each environment. Each run checks its own test results and fails if they violate the rules. The results are saved to the dump archive, and the final report shows them per environment. If the configuration used to build the final report has no `qualityGate`, the rules aren't applied again. - **In the final report.** Add `qualityGate` to the configuration used to build the final report with `allure generate --dump ...`. The rules are applied to the combined test results from all environments. Use this if you need one pass/fail decision for the whole build, for example, for the [Quality Gate check](/docs/integrations-github-action/#quality-gate-check) of the GitHub Action, which reads the `quality-gate.json` file generated with the final report. For example, with a configuration file that applies quality gates when the `ALLURE_QUALITY_GATE` environment variable is set: ```js import { defineConfig } from "allure"; export default defineConfig({ ...(process.env.ALLURE_QUALITY_GATE ? { qualityGate: { rules: [{ maxFailures: 0 }] } } : {}), }); ``` Each environment then runs the tests without the quality gate: ```bash allure run --environment=linux --dump=allure-results-linux -- npm test ``` The final job applies the quality gate to the combined results: ```bash ALLURE_QUALITY_GATE=1 allure generate --dump "allure-results-*.zip" ``` ## View Quality Gate Results Quality gate results are available in several places: - **Terminal.** `allure run`, `allure generate` and `allure quality-gate` print the failed rules to the console. - **Report.** The **Quality Gates** tab of the report shows all rules, both failed and passed (see [Built-in Rules](#built-in-rules)). - **`allure log`.** The `allure log` command prints the failed rules after the test results. Add `--all-quality-gate-results` to also print the passed rules, or `--no-quality-gate-results` to hide quality gate results. When you use the [Log plugin](/docs/v3/configure/#log-plugin) in the configuration file, use its `qualityGateResults` and `qualityGateFilter` options instead. - **`quality-gate.json`.** The `quality-gate.json` file in the report directory contains the results of all rules in the JSON format. For each rule, it lists the IDs of the related test results; the details of these test results are in the `test-results.json` file in the same directory. CI integrations, such as the [GitHub Action](/docs/integrations-github-action/), use these files to report the quality gate status. ## Advanced Configuration The features in this section require dynamic config files (`.mjs`/`.js`) that can execute JavaScript code. ### Custom Filters You can apply custom filtering logic to the test results Quality Gate analyzes. The second ruleset in the example below validates only test results with critical severity label: if any such test fails, the entire run fails immediately, because `fastFail` is set to `true` for this ruleset. ```js import { defineConfig } from "allure"; export default defineConfig({ qualityGate: { rules: [ // first ruleset allows to have up to 10 failed tests in total, it won't fail in runtime and will report validation result in the end { id: "first-set", maxFailures: 10, }, // second ruleset fails immediately if there is at least one critical test failure { id: "second-set", maxFailures: 0, fastFail: true, // validate only test results with critical severity label filter: (tr) => tr.labels.some((label) => label.name === "severity" && label.value === "critical"), }, ], }, }); ``` ### `use` Field The `use` array in the Allure 3 config is a list of rule implementations, where each implementation is an object defining how a specific rule works. The default list includes the implementations of all the built-in rules, so if you're only using them, like in the examples from the [Basic Configuration](#basic-configuration) section, you can omit it. But whenever you need to introduce a custom rule or tweak an existing one, you must provide your own list, and **this list must include the implementations of all rules mentioned in your rule sets - both default and custom**. You should import built-in rule implementations from `allure/rules`, and custom rule implementations from whichever packages or files contain them: ```js import { defineConfig } from "allure"; import { maxFailuresRule } from "allure/rules"; import { ruleImplementation1, ruleImplementation2 } from "custom-rules"; export default defineConfig({ qualityGate: { rules: [ { maxFailures: 5, rule1: 1, rule2: 2, }, ], use: [ maxFailuresRule, // implements maxFailures, which is built-in, but you still have to list it, because you're also using custom rules ruleImplementation1, // implements custom rule1 ruleImplementation2, // implements custom rule2 ], }, }); ``` ### Customize a Rule Message If a rule fails, Allure prints a message to the console. You can customize this message to your liking by redefining the message function in the `use` array of the Quality Gate settings. ```js import { defineConfig } from "allure"; import { maxFailuresRule } from "allure/rules"; export default defineConfig({ qualityGate: { rules: [ { maxFailures: 5, }, ], use: [ { // don't forget to use rest spread operator to keep the rule's other fields intact ...maxFailuresRule, message: ({ expected, actual }) => `Custom message: expected ${expected}, got ${actual}`, }, ], }, }); ``` ### Use External Rules You can use third-party Quality Gate rules: - install the package with the rule to your project - import the rules from the package - provide them to the `use` field in the quality gate configuration: ```js import { defineConfig } from "allure"; import { rule1, rule2 } from "custom-rules-package"; export default defineConfig({ qualityGate: { rules: [ // define rulesets according the external rules signature ], use: [rule1, rule2], }, }); ``` Tip: If you want to use default and external rules together, make sure to import default rules from the `allure/rules` package and include them in the `use` array as well: ```js import { defineConfig } from "allure"; // import default rules at once import { qualityGateDefaultRules } from "allure/rules"; // or import them separately import { allTestsContainEnvRule, environmentsTestedRule, maxDurationRule, maxFailuresRule, minTestsCountRule, successRateRule, } from "allure/rules"; import { rule1, rule2 } from "custom-rules-package"; export default defineConfig({ qualityGate: { rules: [ // define rulesets according the external rules signature ], use: [...qualityGateDefaultRules, rule1, rule2], }, }); ``` If you don't re-assign `use` field, Allure will use only the default rules automatically without additional imports. ### Create Custom Rules You can create your own quality gate rules by implementing the `QualityGateRule` interface, where `T` is the type of the value that the rule is checking, and `K` is the type of the intermediate result that the rule is using to keep the state between the test results batches it processes. Warning: If you're using TypeScript, make sure to compile your custom rule file before use! Allure doesn't support on-the-fly TypeScript compilation. This is a simplified version of the built-in `maxFailures` rule: **TypeScript:** ```ts import { filterUnsuccessful } from "@allurereport/core-api"; import { type QualityGateRule } from "@allurereport/plugin-api"; export const maxFailuresRule: QualityGateRule = { rule: "maxFailures", message: ({ actual, expected }) => `The number of failed tests ${String(actual)} exceeds the allowed threshold value ${String(expected)}`, successMessage: ({ actual, expected }) => `The number of failed tests ${String(actual)} is within the allowed threshold value ${String(expected)}`, validate: async ({ trs, expected, state }) => { const previous = Number(state.getResult() ?? 0); const failedTrs = trs.filter(filterUnsuccessful); const testResults = failedTrs.map((tr) => tr.id); const actual = previous + failedTrs.length; state.setResult(actual, testResults); return { success: actual <= expected, actual, testResults, }; }, }; ``` **JavaScript:** ```js import { filterUnsuccessful } from "@allurereport/core-api"; export const maxFailuresRule = { rule: "maxFailures", message: ({ actual, expected }) => `The number of failed tests ${String(actual)} exceeds the allowed threshold value ${String(expected)}`, successMessage: ({ actual, expected }) => `The number of failed tests ${String(actual)} is within the allowed threshold value ${String(expected)}`, validate: async ({ trs, expected, state }) => { const previous = state.getResult() ?? 0; const failedTrs = trs.filter(filterUnsuccessful); const testResults = failedTrs.map((tr) => tr.id); const actual = previous + failedTrs.length; state.setResult(actual, testResults); return { success: actual <= expected, actual, testResults, }; }, }; ``` The rule object contains the following fields: **rule** - the rule name which is used to identify the rule in the configuration file; it is also used in the validation messages printed to the console and shown in the report **message** - function that formats the message shown if the rule fails; can be redefined in the configuration file: - takes an object with the following fields: - `actual`: the actual value calculated by the validate function. - `expected`: the expected value configured in the rule set. - must return a string to display. **successMessage** (optional) - function that formats the message shown if the rule passes. It takes the same arguments as `message`. If omitted, Allure uses a generic message with the actual and expected values. **validate** - validation function that implements the validation logic: - takes an object with the following fields: - `trs`: an array of test results. These can be all test results produced in the run or just a portion of it, i.e., a batch. Earlier attempts of retried tests and failures with the **Muted** or **Accepted** [resolutions](/docs/resolutions/) are already excluded. - `expected`: the expected value of type T configured in the rule set. - `state`: an object that keeps the state across multiple validate calls against different test result batches. See an example below. - `environment`: the ID of the environment the rules are applied to, if any. - `metrics`: an array of the [performance metrics](/docs/performance-metrics/) collected in the run. - `previousHistory`: an array of the metric values saved in the history file for previous runs. - must return a promise that resolves to an object with the following fields: - `success`: the validation result (true or false). - `actual`: the calculated value that the rule has checked. - `testResults`: an array of IDs of the test results related to the rule, for example, the failed tests counted by `maxFailures`. The report lists these tests under **Related test results**. Return an empty array if the rule doesn't depend on specific tests. To create a custom rule, define an object with the structure described above. #### `state` and Fast Fail Logic Allure Report applies the quality gate rules in two different ways: - **when the test run completes without triggering fast fail, and all test results are ready**: Allure simply applies all quality gate rules to the whole set of results. - **while the test run is in progress**: Allure receives batches of test results from the test framework as it progresses over the tests and checks them against quality gate rules from `fastFail` sets. If a rule from a `fastFail` set is violated, Allure terminates the test run and adds diagnostic data to the report. Some rules work the same way in both contexts. For example, you can define a rule which checks that all blockers pass: such a rule assesses test results one-by-one and is well-defined against any subset of the test results. Other rules, on the other hand, rely on some state that should persist across test result batches. The built-in `maxFailures` rule shown above is a good example: with `fastFail` enabled it must count failed tests as they appear for the whole duration of the run. Such rules must use the `state` object that is passed to the validate function. Its `getResult()` method returns the value saved by the previous call, and `setResult(value, testResults)` saves the new value together with the IDs of the related test results: ```ts import type { QualityGateRule } from "allure/rules"; export const myRule: QualityGateRule = { rule: "myRule", message: ({ expected, actual }) => `The custom rule has failed. Expected: ${expected} doesn't equal to ${actual}`, validate: async ({ trs, expected, state }) => { // there is no initial value in the state, so we use 0 as a fallback const previous = Number(state.getResult() ?? 0); const actual = previous + trs.length; state.setResult(actual, []); return { success: actual === expected, actual, testResults: [], }; }, }; ``` #### Import and Enable Custom Rule Import the rule to the configuration file, add it to the `use` list, along with other rules you also want to use: ```js import { defineConfig } from "allure"; import { maxFailuresRule } from "allure/rules"; import { myRule } from "./myRule.js"; export default defineConfig({ qualityGate: { rules: [ { maxFailures: 5, myRule: 10, }, ], use: [maxFailuresRule, myRule], }, }); ``` You can generally place the code of your custom rule anywhere in your project as long as Node can successfully resolve it. #### Built-in Rules Source Code You can find it in [this repository](https://github.com/allure-framework/allure3/blob/main/packages/core/src/qualityGate/rules.ts). ### Use Environment Variables in Rules If you need to pass the expected value of a rule from the outside (for example, from a workflow definition), you can use environment variables in dynamic configuration files: **Allure config:** ```js import { defineConfig } from "allure"; import { myRule } from "./myRule.js"; const { MY_RULE_VALUE } = process.env; export default defineConfig({ name: "Allure Report 3", qualityGate: { rules: [ { // use 100 as a fallback value when env variable is not set myRule: MY_RULE_VALUE ? Number(MY_RULE_VALUE) : 100, }, ], use: [myRule], }, }); ``` **CI:** ```yaml jobs: test: runs-on: "ubuntu-latest" steps: # ... - run: npx allure run -- npm test env: MY_RULE_VALUE: 10 # ... ``` --- ## Resolutions > URL: https://allurereport.org/docs/resolutions/ # Resolutions (Allure 3) Not every failed test needs a fresh investigation. A failure may already be reported as a bug, be temporarily ignored while someone works on a fix, or be expected in the current context. **Resolutions** let you mark such failures in the Allure [configuration file](/docs/v3/configure/). In the report, failures with a resolution get an icon in the test tree and are grouped on the **Resolutions** tab, and you can select them with the **Resolution** filter. The **Failed** and **Broken** counters only show the failures that still need attention. See [Resolutions in the report](#resolutions-in-the-report) for details. Images: /images/resolutions/resolutions-overview-light.png, /images/resolutions/resolutions-overview-dark.png A resolution is assigned by a **rule** that matches test results by their error message, test case ID, retry hash or environment. Resolutions apply only to failed and broken test results: a passed or skipped result never gets a resolution, even if it matches a rule. ## Resolution types There are three resolution types: | Resolution | Use it for | Required fields | Counted by [quality gates](#resolutions-and-quality-gates) | | ---------- | ------------------------------------------------------------------------------------- | --------------- | ---------------------------------------------------------- | | `issue` | A failure caused by a known bug that is tracked in an issue tracker. | `issue` | Yes | | `muted` | A failure you want to ignore temporarily, for example, while the test is being fixed. | `comment` | No | | `accepted` | A failure that is expected, for example, a feature that isn't available yet. | `comment` | No | ## Configure resolutions Add the `resolutions` section to the configuration file. It contains a list of `rules` and, optionally, `links` templates that turn issue IDs into clickable links: **allurerc.mjs:** ```javascript import { defineConfig } from "allure"; export default defineConfig({ resolutions: { links: { jira: { urlTemplate: "https://jira.example.com/browse/%s", nameTemplate: "JIRA %s", }, }, rules: [ { resolution: "issue", issue: { id: "SHOP-42", type: "jira" }, messageRegexp: "Connection timeout", comment: "Payment gateway sandbox is slow", }, { resolution: "muted", testCaseId: ["103"], comment: "Muted until the widget redesign lands", }, { resolution: "accepted", environment: ["windows"], messageRegexp: "^Export format", comment: "The v1 export format is expected on Windows", }, ], }, }); ``` **allurerc.json:** ```json { "resolutions": { "links": { "jira": { "urlTemplate": "https://jira.example.com/browse/%s", "nameTemplate": "JIRA %s" } }, "rules": [ { "resolution": "issue", "issue": { "id": "SHOP-42", "type": "jira" }, "messageRegexp": "Connection timeout", "comment": "Payment gateway sandbox is slow" }, { "resolution": "muted", "testCaseId": ["103"], "comment": "Muted until the widget redesign lands" }, { "resolution": "accepted", "environment": ["windows"], "messageRegexp": "^Export format", "comment": "The v1 export format is expected on Windows" } ] } } ``` **allurerc.yaml:** ```yaml resolutions: links: jira: urlTemplate: https://jira.example.com/browse/%s nameTemplate: JIRA %s rules: - resolution: issue issue: id: SHOP-42 type: jira messageRegexp: Connection timeout comment: Payment gateway sandbox is slow - resolution: muted testCaseId: ["103"] comment: Muted until the widget redesign lands - resolution: accepted environment: [windows] messageRegexp: ^Export format comment: The v1 export format is expected on Windows ``` Each rule contains: - `resolution` - `issue`, `muted` or `accepted`. - one or more [matchers](#matchers) that select the test results. - `issue` (only for `issue` rules) - an object with the issue `id` and the `type`, which must be one of the keys of [`links`](#issue-links). Each issue ID can be used in only one rule. - `comment` - an explanation shown in the report. Required for `muted` and `accepted` rules, optional for `issue` rules. Allure validates the resolutions configuration at startup. If the configuration is invalid (for example, a rule has no matchers, or a `muted` rule has no comment), the command fails with an error that lists every problem: ```plain The provided resolutions config is invalid: resolutions.rules[1].comment must be a non-empty string ``` ### Matchers A rule must contain at least one matcher. If a rule contains several matchers, a test result must satisfy **all** of them. Matchers that take a list are satisfied if **any** of the listed values matches. | Matcher | Type | Matches a test result if... | | --------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `messageRegexp` | `string` | its error message matches the JavaScript regular expression. The expression isn't anchored: `"timeout"` matches any message that contains `timeout`. | | `testCaseId` | `string[]` | its test case ID is in the list. Any of these IDs works: the test case ID calculated by Allure, the test's [ID](/docs/v3/readability/#id) label (`ALLURE_ID`), or the test case ID reported by the Allure integration. | | `retryHash` | `string[]` | its retry hash is in the list. The retry hash identifies a specific test with specific parameters in a specific [environment](/docs/environments/). | | `environment` | `string[]` | its [environment](/docs/environments/) ID is in the list. | To find the test case ID or the retry hash of a test, open the test in the report and hover over the copy icon next to its full name. The menu lets you copy the full name, the **Test case ID** or the **Retry hash**. Tip: Use `testCaseId` to match a test regardless of its parameters and environment, and `retryHash` to match only one specific variant of a parametrized test or only one environment. Note that the retry hash of the same test differs between environments. ### Issue links The `links` section defines one template for each issue type you use. The key (`jira` in the examples above) is the type you refer to in the `issue.type` field of the rules. - `urlTemplate` (required) - the URL of the issue, with exactly one `%s` placeholder for the issue ID. The resulting URL must be an absolute `http://` or `https://` URL. - `nameTemplate` (optional) - the link text, with exactly one `%s` placeholder for the issue ID. If omitted, the issue ID itself is used as the link text. With the configuration above, the `SHOP-42` issue is shown in the report as a **JIRA SHOP-42** link to `https://jira.example.com/browse/SHOP-42`. ### Rule priority A test result can get only one resolution. If it matches several rules, Allure picks the resolution in this order: 1. The first matching `issue` rule. 2. The `muted` flag set in the test result itself (see [Resolutions from test results](#resolutions-from-test-results)). 3. The `known` flag set in the test result itself. 4. The first matching `muted` rule. 5. The first matching `accepted` rule. For example, if a failure is linked to an issue and also matches a `muted` rule, the report shows it as linked to the issue. ## Resolutions from test results A test result file can also mark a failure as muted or known by itself, via the `muted` or `known` flag in its `statusDetails`. Allure Report 3 converts these flags into resolutions: - a result with `"muted": true` gets the **Muted** resolution with the comment "Muted from result"; - a result with `"known": true` gets the **Accepted** resolution with the comment "Accepted from result (known)". This works even if the configuration file has no `resolutions` section. ## Resolutions in the report Test results with a resolution are marked with an icon in the test tree: a bug for **Issue**, an eye for **Muted**, and a check mark for **Accepted**. **Resolution filter.** Use the **Resolution** dropdown above the test tree to show only the test results with the selected resolutions. Besides the three resolution types, it has the **No resolution** option, which shows the failed and broken test results that don't have a resolution yet - the failures that still need attention. Images: /images/resolutions/resolution-filter-light.png, /images/resolutions/resolution-filter-dark.png **Counters and status filters.** Failed and broken test results that have a resolution aren't included in the **Failed** and **Broken** counters on the right side of the summary, and the **Failed** and **Broken** status filters above the tree show only failures without a resolution. To see the resolved failures, use the **Resolution** filter. **Resolutions tab.** The **Resolutions** tab of the report groups the resolved test results: one group per issue, with its link, type and comment, then one **Muted** or **Accepted** group per comment. Images: /images/resolutions/resolutions-tab-light.png, /images/resolutions/resolutions-tab-dark.png **Test page.** The page of a resolved test result has a **Resolutions** tab with the resolution, the issue link (for `issue`) and the comment. Images: /images/resolutions/test-resolutions-light.png, /images/resolutions/test-resolutions-dark.png ## Resolutions and quality gates [Quality gate](/docs/quality-gate/) rules ignore failures with the **Muted** and **Accepted** resolutions: they don't count towards `maxFailures`, `successRate` or any other rule. Failures linked to an **Issue** still count, because the bug they point to is still there. The success rate shown in the report's pie chart excludes them as well. When you run tests with [`allure run --rerun`](/docs/quality-gate/#reruns), muted and accepted failures are not rerun. ## Known issues file Every time Allure generates a report with resolutions configured, it also writes a **known issues file**: a JSON file with the issues that failures in the run are linked to, and the failed and broken test results linked to each of them. You can process this file in your CI pipeline, for example, to post the list of known issues to a chat. By default, the file is written to `known-issues.json` in the current working directory. You can change the path in the configuration file or with the `--known-issues` command-line option; the path must end with `.json`: ```javascript import { defineConfig } from "allure"; export default defineConfig({ resolutions: { knownIssuesPath: "./out/known-issues.json", rules: [ // ... ], }, }); ``` ```bash allure generate --known-issues ./out/known-issues.json ./allure-results ``` The file is an output only: Allure doesn't read it back. To mark failures as known, use resolution rules. The file isn't part of the generated report, so it isn't published to [Allure Report Storage](/docs/self-hosted-storage/) either. ## Resolutions in multistage builds [Dump archives](/docs/multistage-builds/) store the resolutions assigned during each stage. When you build the final report from the archives, use the same `resolutions` configuration as in the stages, so that the final report shows the same resolutions. ## Migrating from the known issues file Earlier versions of Allure Report 3 supported known issues via a JSON file with a list of `historyId` values, configured with the top-level `knownIssuesPath` option and created with the `allure known-issue` command. Both have been replaced with resolutions: a configuration file that still contains the top-level `knownIssuesPath` option makes Allure fail with the "unsupported fields: knownIssuesPath" error. To migrate: 1. Remove the top-level `knownIssuesPath` option from the configuration file. 2. For each known issue, add an `issue` rule to `resolutions.rules`. Match the tests by `testCaseId`, `retryHash` or `messageRegexp` instead of `historyId`. 3. Add a `links` template for your issue tracker. 4. If you used the known issues file to ignore failures without linking them to issues, use `muted` or `accepted` rules instead. --- ## Self-hosted storage service > URL: https://allurereport.org/docs/self-hosted-storage/ # Self-hosted storage service (Allure 3) [**Allure Report Storage**](https://github.com/allure-framework/allure-report-storage) is a free, open-source (Apache-2.0) service that gives Allure 3 somewhere to publish HTML reports and keep cross-run history, without relying on third-party hosting. You run it yourself - as a Docker container or a Cloudflare Worker - and Allure 3 talks to it through the built-in `allureService` client. With [`allureService.accessToken`](/docs/v3/configure/#_11-report-publishing-allure-service) set and [`publish: true`](/docs/v3/configure/#_11-report-publishing-allure-service) on the relevant plugin(s), publishing is automatic: every report is uploaded, gets a shareable URL, and becomes part of that project's history. ## How it works Reports are organized by **repository and branch** rather than a fixed project ID, so history is scoped per branch by default, except in pull request pipelines (see [History depth and branches](#history-depth-and-branches)). Allure 3 detects `repo` and `branch` automatically: from CI-provided environment variables on a [supported CI server](https://github.com/allure-framework/allure3/tree/main/packages/ci), or otherwise from the current git repository name and checked-out branch. Each publish: 1. creates a draft report for that `repo`/`branch`; 2. uploads the generated report files; 3. marks the report complete and attaches a history data point. A completed report is immutable: it can only be deleted, not modified. The history data point is what powers [History and retries](/docs/history-and-retries/) across separate runs. Images: /images/allure-report-storage/history-tab-light.png, /images/allure-report-storage/history-tab-dark.png ## Deployment Run it via the [Docker guide](/docs/guides/self-hosted-storage-docker/) or the [Cloudflare Worker guide](/docs/guides/self-hosted-storage-cloudflare/). Both cover full setup step by step. Neither needs an external database or bucket by default: Docker uses a bundled SQLite database and local disk, the Worker uses Cloudflare D1 and R2. Where report files are stored is independent of where the service runs: - **Local filesystem** - default for Docker/Node; files go under the configured data directory. - **Any S3-compatible bucket** - AWS S3, R2's S3-compatible endpoint, or another provider such as MinIO. See the [Docker guide](/docs/guides/self-hosted-storage-docker/#s3-compatible-storage) for setup. - **Cloudflare R2** - used automatically via a native binding when running as a Worker. ## Connecting Allure 3 to the storage service Mint a report-access token using the bootstrap `ACCESS_TOKEN` set up during deployment: ```bash curl -sS -X POST http://localhost:3000/api/token \ -H "Authorization: Bearer storage_bootstrap_token" ``` This returns a token in the `ars1..` format. Unlike the bootstrap token, it's safe to hand to CI: it only authorizes publishing and history reads. It's also self-describing: the payload embeds the storage instance's URL, so Allure 3 needs no separate server-address setting. Tip: Running behind a reverse proxy By default, the embedded URL is the address the token request was sent to. If the service runs behind a reverse proxy or TLS termination, this may differ from the public URL that Allure 3 should use. In this case, set the service's `PUBLIC_URL` environment variable (Allure Report Storage 1.1.0 and later) to the public origin, for example `https://reports.example.com`, before minting tokens. See the [Docker](/docs/guides/self-hosted-storage-docker/#_1-start-the-service) and [Cloudflare Worker](/docs/guides/self-hosted-storage-cloudflare/#_2-configure-secrets) guides. The value must be an absolute HTTP or HTTPS origin, optionally with a port, but without a path, a query, a fragment, or credentials. Otherwise, the service does not start. Tokens minted before setting `PUBLIC_URL` keep the URL they were minted with. Mint new tokens and replace the old ones in your Allure 3 and CI configuration. Add it to the Allure 3 configuration, alongside `publish: true` on the plugin(s) to publish: ```javascript import { defineConfig } from "allure"; export default defineConfig({ name: "Allure Report", plugins: { awesome: { options: { publish: true, // explicitly specify which reports should be published }, }, }, allureService: { accessToken: "ars1...", }, }); ``` From here, running tests through Allure 3 publishes the report automatically. ## Choosing a main branch A repository's main branch defaults to `main`. If yours is named differently, set it explicitly (also using the bootstrap token) so other branches compare against the right baseline: ```bash curl -sS -X POST http://localhost:3000/api/projects/main-branch \ -H "Authorization: Bearer storage_bootstrap_token" \ -H "Content-Type: application/json" \ -d '{ "repo": "repo_name", "main_branch": "main_branch_name"}' ``` ## History depth and branches Every completed report's history point is kept permanently: nothing expires automatically, only explicit deletion removes it. Allure 3 fetches the **10 most recent** points per branch by default; set `historyLimit` to fetch further back: ```javascript export default defineConfig({ historyLimit: 50, }); ``` Unlike `historyPath` (which prunes older entries from a local file), `historyLimit` here only limits what's _fetched_: full history stays stored regardless. The two options are also mutually exclusive: `historyPath` is ignored if `allureService.accessToken` is set. A non-main branch's history is padded with the main branch's older reports when it doesn't have enough of its own to fill the requested count, extending its own timeline further back, never interleaved with it. A branch with one or two publishes still gets a populated trend chart this way. Main's own history is never padded, and a branch with enough reports of its own doesn't get padded either. This is based on branch _name_, not git ancestry: a git-orphan branch is treated the same as a real feature branch. Only `GET /api/history` (and therefore trend charts and the history tab) does this blending; the [reports tree page](#browsing-published-reports) always groups strictly by branch. Pull request and merge request pipelines work differently. If the CI server provides the pull request's target branch (for example, `GITHUB_BASE_REF` on GitHub Actions, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME` on GitLab CI, or `CHANGE_TARGET` on Jenkins), Allure reads the history of the target branch on every run. The report is still published under the pull request's own branch, not the target branch, but the pull request's own earlier runs don't appear in its history. As a result, tests added in the pull request are shown as new on every run, and the report shows how the change compares with the target branch. The same applies to [performance metrics](/docs/performance-metrics/): until a branch has enough reports of its own, its metric deltas, and the [delta quality gate rules](/docs/quality-gate/#metric-rules), compare its reports with the main branch's latest report. In a pull request pipeline, they always compare with the target branch's latest report. ## Browsing published reports ![Reports tree page](/images/allure-report-storage/reports-tree.png "Reports tree page") Every instance also serves a page listing completed reports for a repository, grouped by branch: ``` https:///reports/tree?repo= ``` Useful for finding a previous report without digging through CI logs. ## Deleting a report A completed report can only be deleted, not edited: this removes both its files and its history point, authenticated with a report-access token rather than the bootstrap token: ```bash curl -sS -X POST http://localhost:3000/api/report//delete \ -H "Authorization: Bearer ars1..." ``` Afterward, the report's URL stops resolving, and it disappears from `GET /api/history` and the [tree page](#browsing-published-reports). If it was one of main's reports, branches that were padded with it fall back to whatever remains. --- ## Sorting and filtering > URL: https://allurereport.org/docs/sorting-and-filtering/ # Sorting and filtering The [grouping options in Allure Report 3](/docs/v3/configure/#common-options) and the Behaviors, Suites, Packages, and Categories tabs in Allure 2 (see [Improving navigation in your test report](/docs/v2/navigation/)) provide various ways to sort and filter the list of tests. ## Sort tests - In Allure 3 use the **Sort by:** dropdown to select sort criteria and direction. - In Allure 2 click on a label in the header to change how the tests are sorted. Click on an active label again to reverse the sorting order. This affects both the sorting of groups (categories, suites, etc.) and the sorting of tests within the same group. Images: /images/sorting-and-filtering/sort-allure3-light.png, /images/sorting-and-filtering/sort-allure3-dark.png, /images/sorting-and-filtering/sort-allure2-light.png, /images/sorting-and-filtering/sort-allure2-dark.png The possible sorting criteria are: - the **order** of execution; - the **name**: the [title](/docs/v3/readability/#title) of a test or a group of tests; - the **duration** of execution; - the **status** tests finished with, see the list in [Filter tests by status](#filter-tests-by-status). ## Filter tests by name In the search box, enter some text to see only the tests that contain it text in their [title](/docs/v3/readability/#title). The filter is case-insensitive. Images: /images/sorting-and-filtering/filter-by-name-allure3-light.png, /images/sorting-and-filtering/filter-by-name-allure3-dark.png, /images/sorting-and-filtering/filter-by-name-allure2-light.png, /images/sorting-and-filtering/filter-by-name-allure2-dark.png ## Filter tests by tags - In Allure 3 click on the **Tags** button to select [tags](/docs/v3/readability/#tags) to filter by. - In Allure 2 enter `tag:` followed by a comma-separated list of [tags](/docs/v2/readability/#tags). The report will only show tests that have all the tags you've entered. The tag names are case-insensitive. Tags with whitespaces are not supported. Images: /images/sorting-and-filtering/filter-by-tags-allure3-light.png, /images/sorting-and-filtering/filter-by-tags-allure3-dark.png, /images/sorting-and-filtering/filter-by-tags-allure2-light.png, /images/sorting-and-filtering/filter-by-tags-allure2-dark.png ## Filter tests by status Click on the colored buttons to hide or show all the tests that finished with a specific [status](/docs/test-statuses/). The number on each button indicates the number of tests with this status present in the hierarchy. If there are no tests with a given status in the hierarchy, then the corresponding button reads zero and does nothing. Images: /images/sorting-and-filtering/filter-by-status-allure3-light.png, /images/sorting-and-filtering/filter-by-status-allure3-dark.png, /images/sorting-and-filtering/filter-by-status-allure2-light.png, /images/sorting-and-filtering/filter-by-status-allure2-dark.png ## Filter tests by marks - In Allure 3 click on the **Flaky tests**, **New tests** or **Retried tests** counters above the test list, or click **Transition** to select one or more transitions to filter by: **New**, **Fixed**, **Regressed**, **Malfunctioned**. - In Allure 2 click on a button with a mark icon to enable or disable a filter by a specific mark set by Allure. In Allure 3, if multiple transitions are selected, the report shows tests that have any of them. The **Flaky tests** and **Retried tests** filters work the same way: with both enabled, the report shows tests that are either flaky or retried. Different filters are combined with each other, for example, selecting the **Fixed** transition and the **Critical** [severity](#filter-tests-by-severity) shows only the critical tests that were fixed. In Allure 2, if one or more filters by marks are enabled, the report will only show those tests that have all selected marks. If no filters by marks are enabled, all tests in the hierarchy are shown. Images: /images/sorting-and-filtering/filter-by-marks-allure3-light.png, /images/sorting-and-filtering/filter-by-marks-allure3-dark.png, /images/sorting-and-filtering/filter-by-marks-allure2-light.png, /images/sorting-and-filtering/filter-by-marks-allure2-dark.png The possible marks are: **Flaky**, **Retry**, **New**, **Fixed**, **Regressed**, **Malfunctioned**. See [Test stability analysis](/docs/test-stability/) for more details. ## Filter tests by severity (Allure 3) In Allure 3 click on the **Severity** button to select one or more [severity](/docs/v3/readability/#severity) levels to filter by: **Blocker**, **Critical**, **Normal**, **Minor**, **Trivial**, or **No severity** for tests without a severity level. The report shows tests that have any of the selected levels. Images: /images/sorting-and-filtering/filter-by-severity-allure3-light.png, /images/sorting-and-filtering/filter-by-severity-allure3-dark.png The severity of a test is set via your [Allure integration](/docs/frameworks/), see its documentation for details. ## Filter tests by categories (Allure 3) In Allure 3 click on the **Categories** button to select categories to filter by. Images: /images/sorting-and-filtering/filter-by-categories-allure3-light.png, /images/sorting-and-filtering/filter-by-categories-allure3-dark.png See [Categories](/docs/categories/) for more details. ## Filter tests by environment (Allure 3) In Allure 3, if the report is configured with [environments](/docs/environments/), use the **Environment:** dropdown in the top right corner of the report to only show test results from a specific environment. Images: /images/sorting-and-filtering/filter-by-environment-allure3-light.png, /images/sorting-and-filtering/filter-by-environment-allure3-dark.png See [Environments](/docs/environments/) for more details. ## Need more? If you need to filter test results against a broader set of criteria, check out [Allure TestOps](https://qameta.io/). It allows you to filter by environment, layer, assignee, and other parameters. Add custom fields and use Allure Query Language to create even more advanced filters. Such filters are handy for collaborative work on test result analysis, especially if the number of tests is huge. --- ## Test steps > URL: https://allurereport.org/docs/steps/ # Test steps Test steps are a mechanism for structured logging, supported by most Allure integrations. A step begins and ends around a block of code, typically a [function](#steps-as-reusable-functions) or a [lambda function](#steps-as-lambda-functions), with the step name describing what the code is trying to do: “Initializing test data,” “Connecting to the server,” etc. Unlike a simple text log entry that can only show you a single event, a step allows you to view the execution results of the entire code block. A step may also contain nested steps forming a tree-like structure. In the test report, Allure displays steps in chronological order, preserving the nesting structure. For each step, it shows its separate duration, [status](#step-statuses), and, optionally, [parameters](#step-parameters) and [attached files](#step-attachments). Images: /images/steps/steps-allure3-light.png, /images/steps/steps-allure3-dark.png, /images/steps/steps-allure2-light.png, /images/steps/steps-allure2-dark.png ## Why use test steps? Using steps in your test serves two purposes: to simplify test debugging and to improve test documentation. #### Simplify debugging of your tests Just like any other form of logging, steps help you see which sections of the test code were executed and which were not. But unlike a traditional log entry, each step is an interval, not an instant. That fact helps you find the code block that has started but hasn't finished successfully, judging by the steps' [statuses](#step-statuses). This is also useful when looking for code blocks that take too long to run, thanks to the [duration](#step-durations) being shown next to each step. Once you've identified a troublesome code block, Allure will help you further investigate what happened by providing a comprehensive [error message](#exceptions-in-steps) with the stack trace. For a failed or broken test as a whole, the error also appears [directly in the steps section](#test-level-errors). The report can be improved even further by [adding attachment to the steps](#step-attachments). One common practice is to log the state of a graphical user interface by attaching a screenshot on each step. For complex test scenarios, this will help understand where exactly something went wrong, without resorting to manual testing. For most languages and test frameworks, Allure provides a way to [set special annotations (attributes, decorators)](#steps-as-reusable-functions) to your test functions, thus automatically turning each call of the functions into a step or sub-step with minimal effort. #### Document what your tests do A list of well-described steps serves as a high-level description of a test. It can help your colleagues or manager understand what it does without the need to read the code. To achieve this, we recommend to make sure that the steps cover the whole test, i.e. every line of code belongs to one step or another. If you use a BDD-oriented test framework like Cucumber or Robot Framework, then your code already follows this rule, and Allure will [use the framework's steps automatically](#steps-created-automatically). Otherwise, split your test code into high-level steps yourself using [functions](#steps-as-reusable-functions) or [lambda functions](#steps-as-lambda-functions). Optionally, mark some important events during the test using [no-op steps](#steps-as-no-op-calls). In complex tests, high-level steps are often useful as a documentation, while lower-level steps provide information for debugging. ## Steps created automatically Whenever possible, Allure Report takes advantage of the concept of a step provided by the test framework itself. For example, this is the case with [Cucumber-JVM](/docs/cucumberjvm/), [Behave](/docs/behave/), [SpecFlow](/docs/specflow/) and other BDD-oriented frameworks that use the Gherkin file format. The Allure integration makes its best to reflect the structure of a Gherkin file the same way a native reporter would, while also allowing the test author to define arbitrary sub-steps in each step's code. In addition to that, there is a variety of integration packages that provide automatic step creation on certain events, see the [Modules](/modules/) page. In the example below, the sub-steps are created by the Allure Behave integration. Other integration packages exist for HTTP request libraries, SQL libraries, etc. Images: /images/steps/steps-behave-allure3-light.png, /images/steps/steps-behave-allure3-dark.png, /images/steps/steps-behave-allure2-light.png, /images/steps/steps-behave-allure2-dark.png ## Steps created by your code Depending on your programming language and test framework, Allure can provide one or more ways to define test steps and sub-steps. See the specific [Allure integration](/docs/frameworks/)'s documentation for more details. In general, the supported styles include: - [steps as reusable functions](#steps-as-reusable-functions), - [steps as lambda functions](#steps-as-lambda-functions), - [steps as no-op calls](#steps-as-no-op-calls). If your Allure integration supports multiple styles, you can use them interchangeably in the same test. For example, you may split your test into large functions, each one further split into lambda sub-steps, each one reporting multiple no-op steps. #### Steps as reusable functions This approach is also known as “decorated steps”, “annotated steps” or “attribute-based steps”. It involves declaring a function or a method that implements a step, marking it in some language-specific way (e.g., adding a special annotation in Java), and then calling the function one or multiple times during the execution of the test. Using functions helps to avoid name conflicts between variables used in different steps. Also, this allows different tests to call the same step at some point, e.g., two tests can share a preparation routine. If the function accepts arguments, some Allure integrations automatically interpret them as [step parameters](#step-parameters). ```java @Step("Open the website") void openWebsite(String url) { // ... browser navigation logic ... } ``` #### Steps as lambda functions When the reusability of step implementations is not a priority, it may be more convenient to define steps in place as lambda functions. When the individual steps are small, this approach often leads to a short and self-explanatory code of the test. ```java Allure.step("Open the website", () -> { // ... browser navigation logic ... }); ``` #### Steps as no-op calls Sometimes it can be important to report an instant event, similar to how a text log would include a text message. Most Allure integrations provide a way to do it in the form of a “no-op step”. Such a step does not contain executable code, and it never has sub-steps. Note that when your test is expected to produce a huge amount of messages, it may be better to add them to the test result as a [text attachment](/docs/attachments/#text) instead. ```java // ... browser navigation logic ... Allure.step("Website is opened"); ``` ## Fixtures A fixture, also known as a hook, is a piece of code that runs before or after a test or a group of tests. The code that runs before tests is often referred to as a “setup function”, and the code that runs after tests as a “teardown function”. For the test frameworks with fixtures support, Allure considers each fixture a separate step and displays it in a special block above or below the list of test's own steps. Just like a normal step, a fixture can be split into sub-steps. In some Allure integrations, including fixtures in the test report is optional and can be disabled in the configuration. Disabling fixtures can help keep the report clean and readable, but keep in mind that having all fixtures listed can be useful, e.g., for detecting timing issues. Images: /images/steps/steps-fixtures-allure3-light.png, /images/steps/steps-fixtures-allure3-dark.png, /images/steps/steps-fixtures-allure2-light.png, /images/steps/steps-fixtures-allure2-dark.png ## Step statuses Similarly to [test statuses](/docs/test-statuses/), each test step has its own status. The status of a step does not have to be the same as the status of the whole test — e.g., you can have a failed step within a passed test. - If a step is implemented as a function or a lambda function, it usually gets assigned the [Passed](/docs/test-statuses/#passed) status if the code runs normally, or the [Failed](/docs/test-statuses/#failed) or [Broken](/docs/test-statuses/#broken) status if it is throws an exception. The exact behavior may vary for different Allure integrations. If you need to keep the test running after a step threw an exception, catch it outside the step, as shown in the example below. - If a step is a no-op step, it gets assigned the [Passed](/docs/test-statuses/#passed) status by default. Some integrations provide a way to pass another status as an argument in the Runtime API, see the reference for your integration. We recommend to use the same approximate meanings of statuses for consistency: for example, use [Passed](/docs/test-statuses/#passed) and [Failed](/docs/test-statuses/#failed) for reporting successful and unsuccessful actions, respectively. The example below demonstrates how test statuses can help the reader understand the test results. Here, the test made three attempts to load some data from the internet. The step statuses indicate that the first two attempts failed, but the code caught the exceptions and proceeded to the third attempt which was successful. Images: /images/steps/steps-catch-allure3-light.png, /images/steps/steps-catch-allure3-dark.png, /images/steps/steps-catch-allure2-light.png, /images/steps/steps-catch-allure2-dark.png ```java Allure.step("Load data from remote server", () -> { for (int attempt = 1; attempt <= 3; attempt++) { try { Allure.step("Attempt #%d".formatted(attempt), () -> { // ... data loading logic ... }); break; } catch (Exception e) { if (attempt == 3) { throw e; } } } }); Allure.step("Validate the response headers", () -> { // ... assertions ... }); Allure.step("Validate the data", () -> { // ... assertions ... }); ``` ## Step durations Allure Report always calculates the difference between the start time and the end time of a step. The result is displayed next to the step name in the test report. In the example below, you can immediately see that the first two steps passed very fast, while the third one failed after a suspiciously round time interval (almost exactly three seconds). This is often because some operation gets interrupted by a timeout in either the test code or the product code. Images: /images/steps/steps-durations-allure3-light.png, /images/steps/steps-durations-allure3-dark.png, /images/steps/steps-durations-allure2-light.png, /images/steps/steps-durations-allure2-dark.png ## Exceptions in steps When you write steps as [reusable functions](#steps-as-reusable-functions) or [lambda functions](#steps-as-lambda-functions), Allure takes care of logging any exception thrown by the functions. The exception's message and stack trace are attached to the step, and the step gets the [Failed](/docs/test-statuses/#failed) or [Broken](/docs/test-statuses/#broken) status depending on the exception type. In the example below, the test make multiple attempts to run some code, each one wrapped in `Allure.step()` and then in a `try` block. The block here does not process assertion errors (the `catch` block is empty) and only serves for keeping the test running after them. However, the implementation of `Allure.step()` itself makes sure that the exception message will be displayed if you click on a failed step. (The full stack trace will be displayed if you click on the message.) Images: /images/steps/steps-exceptions-allure3-light.png, /images/steps/steps-exceptions-allure3-dark.png, /images/steps/steps-exceptions-allure2-light.png, /images/steps/steps-exceptions-allure2-dark.png ```java String[] resolutions = {"1920x1080", "1280x720", "1024x768", "800x600"}; for (String resolution : resolutions) { try { Allure.step("Rendering at " + resolution, () -> { // ... rendering logic and assertions ... }); } catch (AssertionError ignored) { } } ``` ## Test-level errors (Allure 3) When a failed or broken test has an error - a message, stack trace, or actual/expected diff - Allure 3 injects a synthetic **error step** into the steps list to make that information visible. The error message becomes the step's title; the stack trace and diff are shown inside it. Where the error step appears depends on whether the error can be attributed to a specific step: - If one of the existing steps already failed with the same message or trace, the error step is nested **inside that step**, giving you the error detail exactly where the failure occurred. - Otherwise, the error step is appended **at the end of the steps list**. This means the steps section is the single place to look for error information, whether the test has zero steps or many. The "No test steps information available" placeholder is now shown only when there is truly no content: no steps and no test-level error. ## Step parameters Step parameters are a way to describe the data that the step worked with. Info: This is not to be confused with the [test parameters](/docs/v3/readability/#parametrized-tests) that characterize the whole test. The API for setting step parameters varies for different Allure integrations and different styles of step declaration. For example: - For a [reusable step function](#steps-as-reusable-functions), some Allure integrations automatically display each argument as a step parameter. - For a [lambda step](#steps-as-lambda-functions), some Allure integrations provide a special object that can be used to manipulate the list of parameters from within the step. - For both styles, it is usually possible to specify a list of step parameters manually by calling a special Runtime API method. For more details, see the reference for your integration. In the example below, the test uses parameters to describe the user profiles it creates. The set of parameters is different in each step. Images: /images/steps/steps-parameters-allure3-light.png, /images/steps/steps-parameters-allure3-dark.png, /images/steps/steps-parameters-allure2-light.png, /images/steps/steps-parameters-allure2-dark.png ```java Allure.step("Create user Alice", step -> { step.parameter("Name", "Alice"); step.parameter("Login", "alice"); step.parameter("Email", "alice@example.com"); // ... user creation logic ... }); // ... other steps ... ``` ## Step attachments Allure Report supports attaching arbitrary files to a step. This can be used to describe either the input data the step processed (similar to [step parameters](#step-parameters)) or the output data it received from somewhere. For example: - In a UI application test, you may want to save screenshots of the application after each action. - In a web application test, you may want to save all HTTP requests and responses the test performs. - In any kind of test, you may want to save logs produced by the system under test. Attaching files to steps can be a way of telling the reader a chronological story, in which certain screenshots or other files illustrate certain moments of the test execution. Some popular file types will be rendered on the test report page, others will be available for download if the reader clicks a link. See [Attachments](/docs/attachments/) for more information. Images: /images/steps/steps-attachment-1-allure3-light.png, /images/steps/steps-attachment-1-allure3-dark.png, /images/steps/steps-attachment-1-allure2-light.png, /images/steps/steps-attachment-1-allure2-dark.png ```java Allure.step("Visit the website", () -> { // InputStream screenshot = ... Allure.attachment("Screenshot of the main page", screenshot); }); // ... other steps ... ``` #### Attachment-only steps (Allure 2) Allure Report 2 supports a special case called an “attachment-only step”. It is a step that does not contain any sub-steps and contains exactly one attachment with the same name as the step itself. Allure Report will simplify such a step visually, showing just the attachment, while preserving its position in relation to other steps. Any [parameters](#step-parameters) for the step will not be displayed. In the example below, the “Screenshot of the filled form” attachment is wrapped in the “Screenshot of the filled form” step. This allows to place the screenshot after the earlier steps but before the later step. Images: /images/steps/steps-attachment-2-allure2-light.png, /images/steps/steps-attachment-2-allure2-dark.png ```java Allure.step("Visit the website", () -> { // ... browser navigation code ... }); Allure.step("Screenshot of the main page", () -> { // InputStream screenshot = ... Allure.attachment("Screenshot of the main page", screenshot); }); // ... other steps ... ``` --- ## Test stability analysis > URL: https://allurereport.org/docs/test-stability/ # Test stability analysis Since tests are used for determining the quality of your project, it is quite important for the tests themselves to be stable. Unstable behavior of a test may interfere with the test results, making the whole report less trustworthy and giving the QA team “alarm fatigue.” It may also have a measurable negative impact on the team's productivity, as the time spent on chasing non-existing bugs will not be spent on real ones. Ideally, each test should be consistent: it should yield a different status only when the system under test behaves differently. When a test fails, you should be able to reflect this in your team's priorities immediately instead of hoping that the test will pass again next time. Allure Report helps you find potentially unstable tests by assigning **test instability marks** to them. There are 5 types of instability marks: [Flaky](#flaky-tests), [Retried](#retried-tests), [Fixed (New passed)](#fixed-new-passed-tests), [Regressed (New failed)](#regressed-new-failed-tests) and [Malfunctioned (New broken)](#malfunctioned-new-broken-tests). The marks are displayed next to the tests in the [trees](/docs/v3/navigation/) and on the details panel. You can [filter tests](/docs/sorting-and-filtering/) by their marks. There is also a mark for **new** tests, which is not technically an instability mark, but Allure displays it in the same way. Images: /images/test-stability/marks-allure3-light.png, /images/test-stability/marks-allure3-dark.png, /images/test-stability/marks-allure2-light.png, /images/test-stability/marks-allure2-dark.png ## Flaky tests The **Flaky** mark is automatically added to a test based on its [tests history](/docs/history-and-retries/#tests-history), which must be enabled for Allure Report to add this mark. In Allure Report 3, a test is marked as **Flaky** if its status has recently been switching between [Passed](/docs/test-statuses/#passed) and [Failed](/docs/test-statuses/#failed) or [Broken](/docs/test-statuses/#broken). Allure compares the test's latest result with its previous results, giving more weight to the most recent ones. A single change of status, such as the first failure after a series of successful runs, is not enough for the mark. By default: - only tests that got the Failed or Broken status in the latest launch can be marked, - Allure analyzes up to 5 previous results of the test from the same environment, ignoring results with other statuses, such as Skipped. You can change this with the [`flakyDetection`](/docs/v3/configure/#flakydetection) configuration option. In Allure Report 2, the **Flaky** mark is automatically added to a test if all the following conditions are met: 1. the test got the [Failed](/docs/test-statuses/#failed) status at some point within the latest 5 launches, 1. the test got the [Passed](/docs/test-statuses/#passed) status at least one time since then, 1. the test got the [Failed](/docs/test-statuses/#failed) or [Broken](/docs/test-statuses/#broken) status in the latest launch. In both versions, some Allure integrations also provide a way to mark a problematic test as **Flaky** manually, see the documentation for your [Allure integration](/docs/frameworks/). Images: /images/test-stability/flaky-allure3-light.png, /images/test-stability/flaky-allure3-dark.png, /images/test-stability/flaky-allure2-light.png, /images/test-stability/flaky-allure2-dark.png For dealing with flaky tests in your project, we recommend a 4-step process: investigate everything, fix what can be fixed, retry the rest, and keep monitoring. 1. **Investigate why the tests are unstable.** A flaky test may indicate a wide range of problems with the test code, the test environment or the product itself. Various tools in Allure Report can help you in your test result analysis. - [Filter by Flaky tests](/docs/sorting-and-filtering/#filter-tests-by-marks) to see which features or suites contain more flaky tests than the others. For example, if the majority of flaky tests are related to a recently introduced feature in your application, you may need to investigate this feature more closely. - Investigate if the flaky tests have similar error messages and stack traces or not. For example, if you see the same timeout error across different flaky tests, it may be caused by an external server being unavailable and not some test-specific issues. - For individual tests that are not parts of larger trends, consider using [steps](/docs/steps/) and [attachments](/docs/attachments/) to narrow down the moments something goes wrong during the execution. 2. **If possible, fix the source of instability.** Once the source of instability is detected, we recommend investing your resources into fixing it. This is important even for less critical bugs, as the inability to reliably run the test can prevent you from finding a more critical problem in the future. If the instability is caused by the test code or the environment, fixing it may involve rearranging the test, rewriting some assertions, introducing mock servers or doubles instead of external services the test depends on, etc. When different tests have a lot of common codebase, it is possible that the underlying issue will improve multiple tests at once. Implementing all the changes may take some time, but it also can save a lot of time in the future, as the team won't have to analyze this flaky test result again. Sometimes, the source of instability is the product itself. For example, flaky tests may appear because of memory leaks or data races. If you can't fix the problem now, try to reproduce it reliably. You may need to write a new test for that. This way, even when all the flaky tests happen to look “green” in a future report, you will see at least one “red” test result to inform you that the issue is still present. 3. **If the test is still flaky, enable retries.** If you can't fix the source of flakiness right now, consider enabling an auto-retry mechanism. For example, set it up to run the test up to 5 times when it fails, thus increasing the flaky test's chances to pass — thus saving your time on future test analysis. For example, if your test tries to connect to an external server and gets a timeout error, it is likely caused by a temporary problem which may be solved by the time you auto-retry the test in a few seconds. Many test frameworks provide easy-to-use annotations or similar constructions for setting failing tests to retry automatically when they fail. In Allure Report 3 specifically, you can use the following command to rerun failed tests up to a set amount of times without any additional setup in the code. ```bash allure run --rerun -- ` ``` When you retry the whole test, its [test status](/docs/test-statuses/) will be determined by the latest attempt, with other attempts available via the [Retries tab](/docs/history-and-retries/#retries). Alternatively, implement a retry logic just for the unstable part of your test, logging each attempt with a [step status](/docs/steps/#step-statuses). This way, the test will only deal with known sources of instability and won't hide one that is yet unknown. 4. **Keep monitoring your flaky tests.** In each test report, make sure to analyze all flaky tests, even the ones you analyzed in previous reports. For example, it is possible that previously a test failed early because of an unavailable service, but now it fails for a completely different cause, and it is important to analyze the new cause. Consider defining [custom categories](/docs/categories/#custom-categories) to distinguish between known and unknown causes. ## Retried tests The **Retried** mark is automatically added to a test if it was run multiple times during the test launch and it got at least two different statuses between them. Unlike the other marks, Allure Report does not need access to earlier launches' history to add the **Retried** mark. It only takes into account all results of the test from the current launch. See [Retries](/docs/history-and-retries/#retries) for more details. Images: /images/test-stability/retried-allure3-light.png, /images/test-stability/retried-allure3-dark.png, /images/test-stability/retried-allure2-light.png, /images/test-stability/retried-allure2-dark.png Depending on the test framework configuration and the nature of the tests themselves, the **Retried** status may or may not indicate a problem. For example, if a test depends on an external web service, the [Passed](/docs/test-statuses/#passed) status with the **Retried** mark may be completely acceptable. On the other hand, a test that only works with the product's own functionality is probably expected to pass each time, so the **Retried** mark on it probably needs further investigation. Make sure to keep an eye on these tests every once in a while. Just like a [Flaky](#flaky-tests) test, a **Retried** test's instability may have multiple sources, some of which might've not been analyzed yet. ## Fixed (New Passed) tests The **New passed** mark is automatically added to a [Passed](/docs/test-statuses/#passed) test if it had any other status in the previous launch. The [tests history](/docs/history-and-retries/#tests-history) must be enabled for Allure Report to add this mark. Images: /images/test-stability/new-passed-allure3-light.png, /images/test-stability/new-passed-allure3-dark.png, /images/test-stability/new-passed-allure2-light.png, /images/test-stability/new-passed-allure2-dark.png When appearing unexpectedly, **New passed** tests usually should raise the same concerns as [New failed](#regressed-new-failed-tests) tests. Review any recent changes in the product, the test, or the environment to see if they could cause the change of the status. If they did, that's good news, but if not, this may mean the test is [Flaky](#flaky-tests) and that other passed tests might be less trustworthy, too. We also recommend re-running the tests (for example, on your local computer) to see if they pass consistently or if they go back to the previous status. In the latter case, see the recommendations for [Flaky tests](#flaky-tests). ## Regressed (New Failed) tests The **New failed** mark is automatically added to a [Failed](/docs/test-statuses/#failed) test if it had any other status in the previous launch. The [tests history](/docs/history-and-retries/#tests-history) must be enabled for Allure Report to add this mark. Images: /images/test-stability/new-failed-allure3-light.png, /images/test-stability/new-failed-allure3-dark.png, /images/test-stability/new-failed-allure2-light.png, /images/test-stability/new-failed-allure2-dark.png When confronted by a **New failed** test, start by reviewing recent changes in the product codebase, the test codebase, their dependencies or the external resources they rely on. Figure out if the new issue is unique or common across different tests. - If the issue seems to be caused by a change, treat is as a defect and try to fix it if possible. - If the issue is not related to any recent changes, this basically means that the test is on its way to being recognized as [Flaky](#flaky-tests). A good way to start the investigation is by running the test again (for example, on your local computer). See if it fails consistently, and if it does not, see the recommendations for [Flaky tests](#flaky-tests). ## Malfunctioned (New Broken) tests The **New broken** mark is automatically added to a [Broken](/docs/test-statuses/#broken) test if it was not broken in the previous launch. The [tests history](/docs/history-and-retries/#tests-history) must be enabled for Allure Report to add this mark. Images: /images/test-stability/new-broken-allure3-light.png, /images/test-stability/new-broken-allure3-dark.png, /images/test-stability/new-broken-allure2-light.png, /images/test-stability/new-broken-allure2-dark.png Just like a [New failed](#regressed-new-failed-tests) test, a **New broken** test can indicate a new bug or an instability. Check if the recent changes cause the issue. If that's not the case, see the recommendations for [Flaky tests](#flaky-tests). --- ## Test statuses > URL: https://allurereport.org/docs/test-statuses/ # Test statuses Every test in an Allure test report has one of the five statuses: [Passed](#passed), [Failed](#failed), [Skipped](#skipped), [Broken](#broken), [Unknown](#unknown). Allure uses colors to represent the statuses in the filters, lists and charts throughout the test report. ## Passed A passed test (green) is a test that finished successfully. This usually means that the tested scenario works as expected. Note that some Allure integrations allow you to indicate different statuses for different [test steps](/docs/steps/). Depending on how the test is written, a failure of a step does not necessarily cause a failure of the test as a whole. ## Failed A failed test (red) is a test that encountered an unexpected behavior in the system under test. This means that the test itself seems to be valid (not [Broken](#broken)), but its execution ended with a false assertion. ## Skipped A skipped test (gray) is a test that was included in the test plan but then not executed. This status is reported by the test framework itself and may be caused by various reasons. For example, some frameworks skip tests if they have a special annotation or similar construction, such as the `@Disabled` annotation in JUnit 5 or the `@pytest.mark.skip` decorator in pytest. Info: If you use Allure TestOps, you can unselect a test when creating a test plan. In such a case, the test will not be shown as **Skipped**, but instead will be excluded from the report entirely. ## Broken A broken test (yellow) is a test that failed because of a test defect. Unlike [Failed](#failed), this status means that the test was unable to check the product's behavior as it intended to, therefore, the failure may or may not indicate an actual product defect. For different test frameworks, the Allure integrations implement different criteria for differentiating between broken and failed tests. As a general rule, a test is considered broken if it causes an exception unrelated to the normal test framework behavior — for example, when it tries and fails to read a configuration file. Notably, some JavaScript frameworks make it harder to check the reason of a failure because they don't always provide the full name of an exception that occurred. The Allure integrations [make their best to deduce it](https://github.com/allure-framework/allure-js/blob/main/packages/allure-js-commons/src/sdk/utils.ts#L5-L25) based on the information available. Consider opening an issue in the integration's bug tracker if some of your failed test are accidentally marked as broken or vice versa. ## Unknown A test gets the “Unknown” status (violet) when the Allure integration did not explicitly set any other status for it, not even the [Skipped](#skipped) status. This may be caused by a bug or an incorrect usage of the Allure integration. Consider opening an issue in the integration's bug tracker if you encounter tests with an unknown status in your reports. ## Beyond the test statuses While the set of test statuses is fixed and cannot be extended, there are additional ways to sort and filter test results. - Keep the history from previous test reports to automatically add special marks to [newly passed, newly failed and newly broken tests](/docs/sorting-and-filtering/#filter-tests-by-marks). - Configure [custom categories](/docs/categories/#custom-categories) to differentiate tests based on their error messages. - Consider using [Allure TestOps](https://qameta.io/) to perform other action on test results, for example: mute a test for a period of time, retry a test in a new run, map a failure to an issue in a bug tracker, and more. --- ## Timeline > URL: https://allurereport.org/docs/timeline/ # Timeline The Timeline chart shows the order and the durations of all tests' executions. Images: /images/charts/timeline-allure3-light.png, /images/charts/timeline-allure3-dark.png, /images/charts/timeline-allure2-light.png, /images/charts/timeline-allure2-dark.png When a test framework runs tests in parallel, it does so using a set of worker processes. Each worker can only run one test at a time, but then the worker can be re-used to run another test. In the chart, each row represents a single worker process, and each rectangle on it represents a single test's execution. The color of each rectangle represents the test status, with the colors having the same meanings as on the [test statuses chart](/docs/visual-analytics/#test-statuses-chart). ## Where to see timeline - In Allure Report 3, click on **Report** in the top left corner of the home page and select **Timeline**. - In Allure Report 2, it is located on its own **Timeline** tab. ## Working with the timeline You can control the timeline chart to better analyze the data you are interested in. - Move the thumb of the slider above the chart to show only those tests that took longer than a certain threshold. - Move the thumbs of the range slider below the chart to zoom in to a time interval. The labels near the range slider indicate time passed since the beginning of the first test's execution. - Hover the mouse pointer over a rectangle to see the test's title and the time its execution started and finished (expressed in seconds and milliseconds since the beginning of the first test's execution). Click on a rectangle to see the test details. ## Recognizing common issues You can find the causes of various test failures or slowdowns just by looking at the timeline. For example: - When some tests are significantly slower than the others, the corresponding rectangles on the chart will be the largest. ![Allure Report Timeline durations](/images/charts/timeline-durations.png "Allure Report Timeline durations") - If your tests work on a web resource, and the resource goes offline for a while, a lot of seemingly unrelated tests may fail simultaneously. Such a situation can manifest itself as a set of failed (red) tests in one area. ![Allure Report Timeline period](/images/charts/timeline-verticals.png "Allure Report Timeline period") - Some test frameworks wait for all running tests to finish before attempting to run the failed ones again. This usually leads to noticeable areas of almost blank space. ![Allure Report Timeline retries](/images/charts/timeline-retries.png "Allure Report Timeline retries") - A test framework may spend too much time loading the environment before each test. This leads to wide horizontal gaps between tests. ![Allure Report Timeline gaps](/images/charts/timeline-gaps.png "Allure Report Timeline gaps") --- ## Visual Analytics > URL: https://allurereport.org/docs/visual-analytics/ # Visual Analytics A test report generated by Allure includes a number of charts built from test data. These charts provide different types of insights into your test suite: snapshots of current state, trends over time, project health indicators, and performance metrics. When someone is familiar with your project and looks at its test reports regularly, they can get a rough idea of the project's current status by just glancing at the latest test report. Here are some of the tricks a reader can adopt in a well-organized project: - The more green color there is on the charts, the better: see the [Test statuses](#test-statuses-chart), various [trend](#trend-charts) and [distribution](#success-rate-distribution) charts. - The less critical are the severity values for the failed tests, the better: see the [Severity](#severity-chart) chart. - The faster most tests run, the better: see the [Duration](#duration-chart) and the [Duration dynamics](#durations-dynamics) charts. - Unless the tests have been rewritten or intentionally deleted, there shouldn't be any red on the [coverage diff](#coverage-diff-map) map. - A deep and extensive [status age pyramid](#status-age-pyramid) indicates persistent issues worth prioritizing. Tip: All trend charts, the duration dynamics chart, the coverage diff map and the status age pyramid rely on historical data, so if you want to use them, make sure to [enable history](/docs/history-and-retries/#how-to-enable-history) first. ## Allure 3 vs Allure 2 Allure Report 3 has many more chart types than Allure Report 2, and still lacks a few that Allure 2 has. To indicate which version supports a certain chart type, they are marked with version badges throughout this document: (Allure 3) or (Allure 2), or both. ### Where to see charts - In Allure Report 3, click on **Report** in the top left corner of the home page. Or configure a charts-only report with the [Dashboard](/docs/v3/configure/#dashboard-plugin) plugin. - In Allure Report 2, charts are split between the **Overview** and **Graphs** tabs. Images: /images/charts/charts-allure3-light.png, /images/charts/charts-allure3-dark.png, /images/charts/charts-allure2-light.png, /images/charts/charts-allure2-dark.png ### Charts configuration Unlike Allure Report 2, all Allure Report 3 charts are configurable. You can choose to include them into your reports or not, and most of them are customizable. Refer to the example below for a full charts configuration example, and explore the following sections of this document to learn how each chart looks and works and what the configuration options affect. Details: Charts configuration examples for Allure Report 3 **Full charts configuration with all options:** ```js import { defineConfig } from "allure"; // Charts configuration with all chart types and all options set to default values const allChartsDefaultValues = [ { type: "currentStatus", title: "Current status", statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses metric: "passed", // status used to calculate the central percentage value }, { type: "testResultSeverities", title: "Test results by severities", levels: ["blocker", "critical", "normal", "minor", "trivial"], //included severities statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses includeUnset: true, //whether to show the “No severity“ section }, { type: "statusDynamics", title: "Status dynamics", limit: 10, // number of shown runs, including the latest statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses }, { type: "statusTransitions", title: "Status transitions", limit: 10, // number of shown runs, including the latest }, { type: "testBaseGrowthDynamics", title: "Test base growth dynamics", statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses limit: 10, // number of shown runs, including the latest }, { type: "coverageDiff", title: "Coverage diff map", }, { type: "successRateDistribution", title: "Success rate distribution", }, { type: "problemsDistribution", title: "Problems distribution by environment", by: "environment", //can only be grouped by environments }, { type: "stabilityDistribution", title: "Stability distribution by features", threshold: 90, // acceptable stability level in % skipStatuses: ["skipped", "unknown"], //skipped test statuses groupBy: "feature", //by what label to group the tests together }, { type: "stabilityDistribution", title: "Stability distribution by epics", threshold: 90, // acceptable stability level in % skipStatuses: ["skipped", "unknown"], //skipped test statuses groupBy: "epic", //by what label to group the tests together }, { type: "stabilityDistribution", title: "Stability distribution by stories", threshold: 90, // acceptable stability level in % skipStatuses: ["skipped", "unknown"], //skipped test statuses groupBy: "story", //by what label to group the tests together }, { type: "durations", title: "Durations histogram", groupBy: "none", //whether to group the tests by layer (other labels not supported) }, { type: "durations", title: "Durations by layer histogram", groupBy: "layer", //whether to group the tests by layer (other labels not supported) }, { type: "durationDynamics", title: "Durations dynamics", limit: 10, // number of shown runs, including the latest }, { type: "statusAgePyramid", title: "Status age pyramid", limit: 10, // number of shown runs, including the latest }, { type: "testingPyramid", title: "Testing pyramid", layers: ["unit", "integration", "e2e"], //layers of the pyramid, bottom to top }, ]; export default defineConfig({ name: "Allure Report", output: "./allure-report", // history file configuration is required to populate most charts historyPath: "./history.jsonl", plugins: { awesome: { options: { reportName: "Allure Report", singleFile: false, reportLanguage: "en", // charts configuration used within the Awesome report charts: allChartsDefaultValues, }, }, dashboard: { options: { reportName: "Dashboard", singleFile: false, reportLanguage: "en", // charts configuration used in a Dashboard report layout: allChartsDefaultValues, }, }, }, }); ``` **How to include selected charts only:** ```js import { defineConfig } from "allure"; export default defineConfig({ name: "Allure Report", output: "./allure-report", historyPath: "./history.jsonl", plugins: { awesome: { options: { reportName: "Allure Report", singleFile: false, reportLanguage: "en", charts: [ { type: "currentStatus", }, { type: "testResultSeverities", }, { type: "durations", }, { type: "testingPyramid", }, ], }, }, }, }); ``` Tip: If you do not configure charts in Allure Report 3, all available chart types will be included by default. ## Test statuses chart (Allure 3) (Allure 2) Test statuses chart is a pie diagram showing how many tests got each [status](/docs/test-statuses/) in the latest run. Images: /images/charts/statuses-allure3-light.png, /images/charts/statuses-allure3-dark.png, /images/charts/statuses-allure2-light.png, /images/charts/statuses-allure2-dark.png The numbers in the center of the pie chart indicates the percentage of test cases that passed relative to the total number of tests. On the right side of the chart, three additional metrics are shown: - **New tests** - the number of tests that appear in this report for the first time - **Flaky tests** - the number of unstable tests with historically inconsistent results - **Retries** - the total number of test that had to be rerun If there are no new, flaky or retried tests, these metrics are not shown. Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "currentStatus", title: "Current status", //chart name statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses metric: "passed", // status used to calculate the central percentage value } ``` ## Severity chart (Allure 3) (Allure 2) The severity chart groups tests by both their statuses and [severity](/docs/v3/readability/#severity). The assignment logic for tests without an explicitly specified [Severity](/docs/v3/readability/#severity) level varies between Allure versions: - In Allure 3 the test is added to the “No severity“ count and placed in the rightmost section. You can disable this section. - In Allure 2 the test is treated as having the “normal” severity level and is displayed in the middle section, while the tests with other severity levels are placed to the left or right of it. Images: /images/charts/severity-allure3-light.png, /images/charts/severity-allure3-dark.png, /images/charts/severity-allure2-light.png, /images/charts/severity-allure2-dark.png In some projects, it may be considered acceptable to have some number of “normal”, “minor” or “trivial” test failures, but not acceptable to have any “critical” or “blocker” ones. To roughly understand the current stability in such a project, a reader can just check how the different test statuses are distributed across the chart. As a general rule, the farther a red bar is placed to the left, the more attention it needs. The test status colors have the same meanings as on the [test statuses chart](#test-statuses-chart). Hover the mouse pointer over a column to see the counts of the tests with the corresponding status and severity. Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "testResultSeverities", title: "Test results by severities", //chart name levels: ["blocker", "critical", "normal", "minor", "trivial"], //included severities statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses includeUnset: true, //whether to show the “No severity“ section } ``` ## Trend charts The trend charts show how a certain value changed over time. Each section corresponds to a certain version of the test report, with the last line on the right corresponding to the current version. Info: See [Tests history](/docs/history-and-retries/#tests-history) to learn how to populate the trend charts with historical data. ### Status dynamics (Allure 3) (Allure 2) The status dynamics chart illustrates how the number of test results in each status changes over time (from older runs to the most recent one). The colors that indicate tests with each status are the same as in the [test statuses chart](#test-statuses-chart). Images: /images/charts/status-dynamics-allure3-light.png, /images/charts/status-dynamics-allure3-dark.png, /images/charts/status-dynamics-allure2-light.png, /images/charts/status-dynamics-allure2-dark.png This visualization makes it easy to: - Identify increases or decreases in **problem statuses** - Observe growth or reduction in the overall **test suite size** - Quickly compare historical runs with the latest report Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "statusDynamics", title: "Status dynamics", //chart name statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses limit: 10, // number of shown runs, including the latest } ``` ### Status transitions (Allure 3) The status transitions chart visualizes how test results change between consecutive executions by showing the number of tests that [transitioned from one status to another](/docs/test-stability/) over time. Images: /images/charts/transitions-light.png, /images/charts/transitions-dark.png Each time point displays bars representing different types of transitions: - **Fixed** - a test that was previously `failed` or `broken` and is now `passed` - **Regressed** - a test that was previously `passed` or `broken` and is now `failed` - **Malfunctioned** - a test that was previously `passed` or `failed` and is now `broken` A linear trend line is overlaid on the chart to indicate the overall direction of status changes: - A **positive slope** indicates that more tests are being fixed than broken - A **negative slope** indicates that breakages and regressions outweigh fixes The status transitions chart helps teams to: - Assess the amount of **regression** between test runs - Tune test result analysis strategies by taking into account the **ratio of recent failures** to long-standing (stale) failures - Monitor execution history for unexpected **spikes in transitions**, including both sudden breakages and mass fixes Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "statusTransitions", title: "Status transitions", //chart name limit: 10, // number of shown runs, including the latest } ``` ### Test base growth dynamics (Allure 3) The test base growth dynamics chart illustrates how the test suite evolves over time by showing the number of **newly added** and **deleted** tests for each test status. Images: /images/charts/test-base-light.png, /images/charts/test-base-dark.png The chart is displayed as a stacked bar chart with a trend line, where positive values indicate **added tests** and negative values indicate **deleted tests** Each bar is divided into colored segments representing test statuses (for example: **passed**, **failed**, **broken**, **unknown**, **skipped**), allowing you to see how changes in the test base are distributed across result types. A linear trend line is overlaid on the chart to highlight the overall direction and speed of test base changes: - A **positive slope** indicates: - that more tests are being added than removed - or hat the amount of new tests per run is growing - or that the amount of deleted tests per run is declining - A **negative slope** indicates: - that test deletions outweigh additions - or that the amount of new tests per run is declining - or that the amount of deleted tests per run is growing The test base growth dynamics chart helps teams to: - Assess whether the test suite is **growing or shrinking** over time - Tune test result analysis strategies by accounting for **recently introduced tests and failures** - Detect **accidental test removals** that may reduce coverage - Prevent **unexpected additions** of failed or broken tests - Understand whether changes in test stability are caused by regressions or by changes in the test base itself Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "testBaseGrowthDynamics", title: "Test base growth dynamics", //chart name statuses: ["passed", "failed", "broken", "skipped", "unknown"], //included statuses limit: 10, // number of shown runs, including the latest } ``` ### Retries trend (Allure 2) - The **Retries trend** chart shows the total number of [retries](/docs/history-and-retries/#retries) the test runner did for all the tests. ### Categories trend (Allure 2) - The **Categories trend** chart shows the number of test defects and product defects. See also: [Categories chart](#categories-chart). ## Coverage diff map (Allure 3) The coverage diff map visualizes how test coverage for application features changes between the previous and the latest test run. It highlights which features are affected by new, removed, enabled, and disabled tests in the most recent run. Images: /images/charts/coverage-diff-light.png, /images/charts/coverage-diff-dark.png This chart is a tree map built from test results that have `epic`, `feature` and `story` [labels](/docs/v3/navigation/#behavior-based-hierarchy). - Rectangles represent individual **features** - Hierarchy reflects the **feature structure** - Rectangle size represents the **number of tests** covering the feature - **Green** sections indicate the parts of your test base that had tests added or enabled in the most recent run, and the **red** sections are the ones that had tests deleted or disabled. The coverage diff map helps teams to ensure adequate coverage and prevent unintended test deletion. Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "coverageDiff", title: "Coverage diff map", //chart name } ``` ## Success rate distribution (Allure 3) The success rate distribution chart visualizes how test success rates are distributed across your test base, grouped by application features. Images: /images/charts/success-rate-light.png, /images/charts/success-rate-dark.png This chart is a tree map built from test results that have `epic`, `feature` and `story` [labels](/docs/v3/navigation/#behavior-based-hierarchy). - Rectangles represent individual **features** - Hierarchy reflects the **feature structure** - Rectangle size represents the **number of tests** covering the feature - **Rectangle color** represents the **average success rate** of those tests, with 100% being green, and the lower the success rate, the redder the rectangle The success rate distribution chart helps teams to: - Tune test result analysis by considering both **feature size** and **test reliability**: large features with a low success rate typically require the highest attention - Identify **poorly covered or unstable areas** of the system under test - Prioritize stabilization and additional testing efforts based on impact rather than raw failure counts Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "successRateDistribution", title: "Success rate distribution", //chart name } ``` ## Problems distribution by environment (Allure 3) The problems distribution by environment chart visualizes how test failures are distributed across features and execution [environments](/docs/environments/), helping identify environment-specific and feature-specific issues. Images: /images/charts/environments-light.png, /images/charts/environments-dark.png Only tests with a `feature` [label](/docs/v3/navigation/#behavior-based-hierarchy) are included in this chart. This chart is a **heat map**, where each cell is a feature–environment combination. - **Cell value** shows the percentage of failed tests relative to the total number of tests for that feature in the given environment - **Color** represents failure intensity: with 0% failures being green, and the higher the failure rate, the redder the rectangle The problems distribution by environment chart helps teams to: - **Detect environment-specific failures**, such as issues limited to a particular OS, browser, or configuration - **Assess overall environment health** by spotting environments with consistently high failure rates - **Prioritize test result analysis** by focusing on the most broken feature–environment combinations - Validate that expected functionality behaves consistently across supported environments Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "problemsDistribution", title: "Problems distribution by environment", //chart name by: "environment", //can only be grouped by environments } ``` ## Stability distribution (Allure 3) The stability distribution charts show how test stability is distributed across different categories of the test suite, such as layers, features, stories, epics, and other labels. Images: /images/charts/stability-light.png, /images/charts/stability-dark.png By default Allure creates three stability distribution charts organized by the following labels: - `epic` - `feature` - `story` Test stability is calculated as the percentage of **stable executions** within a certain category. - A **horizontal threshold line** indicates the acceptable stability level (90% by default). - Bars below the threshold highlight potentially unstable areas The stability distribution charts help teams to: - Identify **unstable areas** of the test suite across different dimensions - Detect **risk zones** where test reliability falls below an acceptable threshold - Prioritize stabilization efforts based on category-level trends rather than individual tests Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "stabilityDistribution", title: "Stability distribution by features", // chart name threshold: 90, // acceptable stability level in % skipStatuses: ["skipped", "unknown"], //skipped test statuses groupBy: "feature", //by what label to group the tests together groupValues: ["Authentication", "Payments"], //(optional) included values of the grouping label } ``` Tests can be grouped by the following labels: - `feature` - `epic` - `story` - `suite` - `severity` - `owner` - Custom labels using the following format: `groupBy: "label-name:"` ## Duration charts ### Durations histogram (Allure 3) (Allure 2) The main duration chart divides all the tests into duration buckets based on how long it took to complete them. The chart uses a logarithmic scale to make it easier to distinguish columns that represent similar values. Images: /images/charts/duration-allure3-light.png, /images/charts/duration-allure3-dark.png, /images/charts/duration-allure2-light.png, /images/charts/duration-allure2-dark.png If your project does not have any tests that take long time by design, this chart often displays the largest groups of tests to the left. If a test takes longer than others and thus is placed into the groups to the right, this may be an indicator of a problem within the test or even the product feature it works with: for example, an attempt to load a resource from a slow server. Also, a test's long duration is caused by a large number of retries. Note, however, that sometimes a test's duration can be affected by various external factors, such as the amount of free operating memory on the machine. It is usually a good idea to run the tests again, if possible, to confirm that a test's long duration is a persistent problem. The durations histogram chart helps teams to: - Understand **overall test performance** and execution time distribution - Identify **performance bottlenecks** Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "durations", title: "Durations histogram", // chart name groupBy: "none", //whether to group the tests by layer (other labels not supported) } ``` ### Durations by layer (Allure 3) This chart is a version of the durations histogram, with all tests grouped by the `layer` label (for example: `unit`, `integration`, `e2e`). Images: /images/charts/duration-layer-light.png, /images/charts/duration-layer-dark.png **Note:** If you don't set the `layer` label for your tests, this chart will be empty. No other labels are currently supported for this chart. When grouping by layer is enabled, the durations histogram is rendered as a **stacked bar chart**, where each stack segment represents a specific testing **layer**, showing its contribution to the total number of tests in each duration bucket. The layer grouping helps to: - Analyze performance **impact by testing layer** - Detect layers that disproportionately contribute to long-running tests Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "durations", title: "Durations by layer histogram", // chart name groupBy: "layer", //whether to group the tests by layer (other labels not supported) } ``` ### Durations dynamics (Allure 3) The durations dynamics chart shows how test execution durations change over time, comparing **execution time** with **serialized duration** and tracking the **speedup coefficient**. Images: /images/charts/duration-dynamics-light.png, /images/charts/duration-dynamics-dark.png Each bar is composed of: - **Serialized duration** - the total execution time if all tests were run sequentially - **Duration** - actual execution time of the test run A **trend line** overlays the chart to show how the **speedup coefficient** evolves over time. **Speedup Coefficient** The speedup coefficient reflects the effectiveness of parallel execution and resource utilization. Higher values indicate better parallelization and more efficient use of available resources. The durations dynamics chart helps teams to: - Assess **resource utilization** and the effectiveness of parallel test execution - Investigate **performance regressions** in test runs - Evaluate the **performance impact of recent changes** in the test suite or infrastructure Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "durationDynamics", title: "Durations dynamics", // chart name limit: 10, // number of shown runs, including the latest } ``` ## Status age pyramid (Allure 3) The status age pyramid visualizes how long tests in **failed**, **broken**, **skipped**, and **unknown** states persist over time, helping to understand the **longevity** of problematic or unused tests. Images: /images/charts/status-age-pyramid-light.png, /images/charts/status-age-pyramid-dark.png The chart is a vertically oriented stacked bar chart: - the top bar shows shows the numbers of **failed**, **broken**, **skipped**, and **unknown** tests in the current run - the bar for each older run shows the number of tests that had the exact same status in that older run **and** all the runs since it The chart is symmetrically divided: - **Right side** shows **skipped** and **unknown** tests - **Left side** shows **failed** and **broken** tests The pyramid layout makes it easy to compare how long different categories of problematic tests persist. For example, in the chart above, we can observe the following: - one test was skipped in every shown test run, making its skipped status at least 10 runs old. - another test started getting skipped 5 runs ago and is still being skipped, making its skipped status 5 runs old. Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "statusAgePyramid", title: "Status age pyramid", // chart name limit: 10, // number of shown runs, including the latest } ``` ## Testing pyramid (Allure 3) This chart visualizes the testing pyramid of your project. It displays the number and success rate of tests by **layer**. Images: /images/charts/testing-pyramid-light.png, /images/charts/testing-pyramid-dark.png By default the pyramid is built from the following values of the `layer` label: - `unit` - `integration` - `e2e` You can also configure custom layers to build the pyramid from. Details: Allure 3 configuration example You can set the following options for this chart in the [configuration file](#charts-configuration): ```js { type: "testingPyramid", title: "Testing pyramid", //chart name layers: ["unit", "integration", "e2e"], //layers of the pyramid, bottom to top } ``` ## Categories chart (Allure 2) The Overview tab in Allure Report 2 contains the **Categories** block that presents [categories of tests](/docs/categories/). This includes the custom categories, as well as the predefined categories “Product defects” and “Test defects”, assigned based on test statuses. Images: /images/charts/categories-allure2-light.png, /images/charts/categories-allure2-dark.png Each category is shown as a horizontal line, split into sections of different colors. Sections represent numbers of test with a certain status, with the colors having the same meanings as on the [test statuses chart](#test-statuses-chart). The number on a section indicates how many tests in the category ended with the corresponding status. The line that correspond to the predefined categories are not split into sections, since each test in such a category has the same status. Click on a category to view it in the Categories tab. Click **Show all** to open the Categories tab and explore the entire hierarchy. ## Suites and Features by stories (Allure 2) The Overview tab in Allure 2 contains the **Suites** and **Features by stories** blocks that present groups of tests from the [Suite-based hierarchy](/docs/v2/navigation/#suite-based-hierarchy) and [Behavior-based-hierarchy](/docs/v2/navigation/#behavior-based-hierarchy), respectively. Images: /images/charts/suites-allure2-light.png, /images/charts/suites-allure2-dark.png Images: /images/charts/stories-allure2-light.png, /images/charts/stories-allure2-dark.png Each group (a suite or a feature) is shown as a horizontal line, split into sections of different colors. Sections represent numbers of test with a certain status, with the colors having the same meanings as on the [test statuses chart](#test-statuses-chart). The number on a section indicates how many tests in the group ended with the corresponding status. In each block, only up to ten most attention-demanding groups (suites or features) are displayed. To choose which groups to show, Allure sorts them by the number of failed tests, then by the number of broken tests. Click on a suite or a story to open it in the Suites or Behaviors tab. Click **Show all** to open the Suites or Behaviors tab and explore the entire hierarchy. --- # Section: Guides ## Guides > URL: https://allurereport.org/docs/guides/ # Guides Let's practice using Allure Report together. Here is the list of available guides: - [Migrating to Allure Java 3](/docs/guides/allure-java-3-migration/) - [JUnit 5 parameterization](/docs/guides/junit5-parametrization/) - [Integrating screenshots and other attachments in Allure Report with Selenide and JUnit 5](/docs/guides/junit5-selenide-screenshots/) - [Integrating screenshots in Allure Report with Selenium and JUnit 5](/docs/guides/junit5-selenium-screenshots/) - [Setting up JUnit 5 with GitHub Actions](/docs/guides/junit5-github-actions/) - [Pytest parameterization](/docs/guides/pytest-parameterization/) - [Integrating screenshots in Allure Report with Pytest and Selenium](/docs/guides/pytest-selenium-screenshots/) - [Integrating screenshots in Allure Report with Pytest and Playwright](/docs/guides/playwright-pytest-screenshots/) - [Integrating videos in Allure Report with Pytest and Playwright](/docs/guides/pytest-playwright-video/) - [Playwright parameterization](/docs/guides/playwright-parameterization/) - [Publishing Reports to GitHub Pages](/docs/guides/github-pages/) - [Deploying Self-Hosted Storage with Docker](/docs/guides/self-hosted-storage-docker/) - [Deploying Self-Hosted Storage on Cloudflare Workers](/docs/guides/self-hosted-storage-cloudflare/) - [Allure Report 3: XCResults Reader ](/docs/guides/xcresults-reader/) Stay tuned, as there are more to come. --- ## Migrating to Allure Java 3 > URL: https://allurereport.org/docs/guides/allure-java-3-migration/ # Migrating to Allure Java 3 Allure Java 3.0 is a major release of the Allure integrations for JVM test frameworks (JUnit, TestNG, Cucumber-JVM, Spock, ScalaTest, Karate and others). It raises the minimum Java version, removes several legacy modules and Runtime API methods, changes how HTTP requests are recorded and how tests are identified in the report history. This guide lists everything you may need to change when upgrading from Allure Java 2.x. Tip: You do not need to change your Allure Report version. Results produced by Allure Java 3 can be viewed in both Allure Report 3 and Allure Report 2. ## 1. Check the requirements Allure Java 3 requires **Java 17 or newer**. The `allure-karate` and `allure-jooq` modules require **Java 21 or newer**. Some framework integrations also require newer framework versions: - TestNG: **7.10 or newer**. - Karate: **Karate 2** (`io.karatelabs:karate-core`). Allure Java 3 no longer supports Karate 1. If your tests run on an older Java or framework version, keep using Allure Java 2.x until you can upgrade. ## 2. Update the dependency version If you use `allure-bom`, update its version. All Allure modules will be upgraded together. **Maven:** ```xml 3.0.0 ``` **Gradle (Kotlin):** ```kts val allureVersion = "3.0.0" // ... dependencies { testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) } ``` **Gradle (Groovy):** ```groovy def allureVersion = "3.0.0" // ... dependencies { testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") } ``` If you use the [Allure Gradle plugin](/docs/integrations-gradle/) to add the adapters, it still uses Allure Java 2.x by default. Use plugin version 4.3.0 or newer and opt in to Allure Java 3 explicitly, see [Using Allure Java 3](/docs/integrations-gradle/#using-allure-java-3): ```kts allure { adapter { allureJavaVersion.set("3.0.0") } } ``` ## 3. Replace removed modules The following modules are no longer published in Allure Java 3. If your project depends on any of them, replace it as shown below. | Removed module | Replacement | | -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | `allure-junit5` | `allure-jupiter`, see [JUnit Jupiter](/docs/junit-jupiter/) | | `allure-junit5-assert` | `allure-jupiter-assert` | | `allure-cucumber4-jvm`
`allure-cucumber5-jvm`
`allure-cucumber6-jvm` | Upgrade to Cucumber-JVM 7 and use `allure-cucumber7-jvm`, see [Cucumber-JVM](/docs/cucumberjvm/) | | `allure-jbehave` (JBehave 4) | Upgrade to JBehave 5 and use `allure-jbehave5`, see [JBehave](/docs/jbehave/) | | `allure-spock` (Spock 1) | Upgrade to Spock 2 and use `allure-spock2`, see [Spock](/docs/spock/) | | `allure-okhttp` (OkHttp 2) | Upgrade to OkHttp 3 or newer and use `allure-okhttp3` | | `allure-attachments` | No replacement needed. HTTP integrations now produce [HTTP exchange attachments](/docs/attachments/#http-exchanges) without it | | `allure-test-filter` | No replacement needed. Test plan support is now part of `allure-java-commons`, which every adapter already depends on. Remove the dependency | | `allure-reader` | No replacement | ## 4. Replace removed Runtime API methods Several methods of the `Allure` class and its lifecycle were removed. Replace them as shown below. | Removed method | Replacement | | ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- | | `Allure.addAttachment(name, content)` | `Allure.attachment(name, content)` | | `Allure.addAttachment(name, type, content)` | `Allure.attachment(name, type, content)` | | `Allure.addAttachment(name, type, content, fileExtension)` | `Allure.attachment(name, type, content, AttachmentOptions.withFileExtension(fileExtension))` | | `Allure.addByteAttachmentAsync()`
`Allure.addStreamAttachmentAsync()` | `Allure.attachmentAsync(name, type, body)`, where `body` is a `CompletionStage` of an `InputStream` | | `Allure.addDescription(description)` | `Allure.description(description)` | | `Allure.addDescriptionHtml(descriptionHtml)` | `Allure.descriptionHtml(descriptionHtml)` | | `Allure.addLabels(labels...)` | `Allure.label(name, value)` for each label | | `Allure.addLinks(links...)` | `Allure.link(name, type, url)` for each link | | `Allure.getLifecycle().updateTestCase(update)` | `Allure.getLifecycle().updateTest(update)` | The lower-level `AllureLifecycle` methods that take UUIDs, such as `startStep(uuid, result)`, `stopStep(uuid)` or `getCurrentTestCase()`, were replaced with methods that take an `AllureExternalKey`. They are mostly used by custom integrations, see [For integration authors](#for-integration-authors). ## 5. Review behavior changes ### HTTP requests are recorded as HTTP exchanges All HTTP client integrations (REST Assured, OkHttp, Apache HttpClient, Spring Web, JAX-RS and others) now record each request and its response as a single [HTTP exchange attachment](/docs/attachments/#http-exchanges). Allure Report shows it as an interactive request/response viewer. In Allure Java 2.x, these integrations produced two separate HTML attachments rendered by FreeMarker templates. The templates and the methods for customizing them, such as `setRequestTemplate()` and `setResponseTemplate()` in [Allure REST Assured](/docs/restassured-configuration/), were removed. To control sensitive data and body sizes, use `configureHttpExchange()` instead. ### Attachments are wrapped in steps Attachments added via the Runtime API, such as `Allure.attachment()`, now appear in the report as a step with the attachment's name, placed in the order the attachments were created relative to other steps. ### Test history Allure Java 3 calculates [test identifiers](/docs/how-it-works-test-identifiers/) differently, which affects [history and retries](/docs/history-and-retries/) in two ways: - **History may start over after the upgrade.** Tests run with JUnit 4, TestNG and JBehave get new identifiers, so the first report generated after the upgrade shows them as new tests, without their previous history. Tests run with JUnit Jupiter, Cucumber-JVM and Spock keep their history. This happens in both Allure Report 3 and Allure Report 2. - **Runtime parameters now affect the test's identity.** Parameters added via `Allure.parameter()` are now taken into account, together with the parameters provided by the test framework. If a parameter's value changes between runs, such as a timestamp or a random ID, the test is treated as a new test on every run. Mark such parameters as excluded: ```java Allure.parameter("startedAt", startedAt, true); ``` ### Test plan environment variable The legacy `AS_TESTPLAN_PATH` environment variable is no longer supported. Use `ALLURE_TESTPLAN_PATH` to point Allure to a test plan file. ### Results directory cleanup The `allure.results.clean.before.run` and `allure.results.clean.only.once` configuration properties were removed. Allure Java no longer deletes existing files in the results directory. Use your build tool to clean it, for example, `mvn clean` or `gradle clean`. ### Jakarta APIs `allure-servlet-api` and `allure-jax-rs` now use the `jakarta.*` APIs instead of `javax.*`. If your project still uses `javax.servlet` or `javax.ws.rs`, migrate it to Jakarta EE before upgrading. ### For integration authors If you develop your own Allure integration on top of `allure-java-commons`, note that the `AllureLifecycle` API was reworked. Methods that identified tests, steps, fixtures and containers by UUID strings were removed. They were replaced with methods that take an `AllureExternalKey`, and test containers were replaced with scopes (`registerScope()`, `addTestToScope()` and `writeScope()`). The format of the result files written to disk is unchanged. ## 6. Explore what's new Allure Java 3 also brings new features, including: - [Global errors and attachments](/docs/global-errors-and-attachments/) — failures in fixtures and lifecycle hooks are reported for the whole test run instead of creating artificial test results, and you can report your own via `Allure.globalError()`. - [HTTP exchange attachments](/docs/attachments/#http-exchanges) with header, cookie, query and form parameter redaction. - Support for [JUnit 6](/docs/junit-jupiter/). - The `@Flaky` and `@Muted` annotations now work with [JUnit 4](/docs/junit4-reference/#flaky). --- ## Publishing Reports to GitHub Pages > URL: https://allurereport.org/docs/guides/github-pages/ # Publishing Reports to GitHub Pages With Allure Report and GitHub Actions, you can automatically generate test reports and publish them to GitHub Pages. To set this up, do the following: 1. [add actions for building and publishing reports](#_1-add-actions-for-building-and-publishing-reports). 2. [enable write access for workflow runs](#_2-enable-write-access-for-workflow-runs), 3. [set up publishing to GitHub Pages](#_3-set-up-publishing-to-github-pages). ## 1. Add actions for building and publishing reports At the end of your repository's testing workflow, add the steps that will [load test report history](#load-test-report-history), [build test report](#build-test-report) and [publish test report](#publish-test-report). **The two Allure Report engines need different steps for building the report.** [`simple-elf/allure-report-action`](https://github.com/simple-elf/allure-report-action), the community action used for Allure 2 below, always generates an Allure 2 report internally (it bundles its own copy of the Allure 2 command line tool), so it can't be reused for Allure 3. For Allure 3, install the `allure` CLI directly in the workflow instead. Here is a full example of a workflow file for a Java project. The example assumes that the branch used for GitHub Pages is called `gh-pages`. **Allure 2:** ```yaml name: Run tests and publish report on: [push] permissions: contents: write jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 - name: Set up JDK uses: actions/setup-java@v6 with: distribution: zulu java-version: 17 - name: Run tests run: ./gradlew clean test - name: Load test report history uses: actions/checkout@v7 if: always() continue-on-error: true with: ref: gh-pages path: gh-pages - name: Build test report uses: simple-elf/allure-report-action@v1.15 if: always() with: gh_pages: gh-pages allure_history: allure-history allure_results: build/allure-results - name: Publish test report uses: peaceiris/actions-gh-pages@v4 if: always() with: github_token: ${{ secrets.GITHUB_TOKEN }} publish_branch: gh-pages publish_dir: allure-history ``` **Allure 3:** ```yaml name: Run tests and publish report on: [push] permissions: contents: write jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 - name: Set up JDK uses: actions/setup-java@v6 with: distribution: zulu java-version: 17 - name: Run tests run: ./gradlew clean test - name: Build test report if: always() run: | npm install -g allure allure generate build/allure-results -o allure-report - name: Publish test report uses: peaceiris/actions-gh-pages@v4 if: always() with: github_token: ${{ secrets.GITHUB_TOKEN }} publish_branch: gh-pages publish_dir: allure-report ``` Warning: This Allure 3 example doesn't carry test history between runs - each report is generated fresh, with nothing to compare it against. For test history that works properly with Allure 3, use the [self-hosted storage service](/docs/self-hosted-storage/) - see the [Docker](/docs/guides/self-hosted-storage-docker/) or [Cloudflare Workers](/docs/guides/self-hosted-storage-cloudflare/) guides to set it up. ### Load test report history As discussed in [Tests history](/docs/history-and-retries/#tests-history), Allure Report can include some data from previous reports into each new report. **This step applies to Allure 2 only** - the Allure 3 example above doesn't carry history between runs; see the warning above. To load history for Allure 2, you need to provide it with the files published on GitHub Pages by previous runs. This step does it by doing a `git checkout` from the branch used for the GitHub Pages content. In the workflow file, set the following values for this step: - `name`: any human-redable name, e.g., “Load test report history”. - `uses`: [`actions/checkout`](https://github.com/marketplace/actions/checkout). - `if`: `always()`. - `continue-on-error`: `true`. - `with`: - `ref`: the branch used for the GitHub Pages content. - `path`: an arbitrary name for a directory to which the previous data will be saved. ### Build test report This step produces the HTML report. For Allure 2, it's based on data from both the current and previous test launches; for Allure 3, see the warning above about history. How it works depends on the engine: **Allure 2** runs the Allure Report utility via [`simple-elf/allure-report-action@v1.15`](https://github.com/marketplace/actions/allure-report-with-history). The new report, along with the copies of previous reports, will be saved into the directory specified in `allure_history`, ready to be published to GitHub Pages. Set the following values for this step: - `name`: any human-readable name, e.g., “Build test report”. - `uses`: `simple-elf/allure-report-action@v1.15`. - `if`: `always()`. - `with`: - `gh_pages`: the directory name to which the previous data was downloaded. Must be the same as the `path` value from the [Load test reports history](#load-test-report-history) step. - `allure_results`: path to the current test results directory. Depending on the framework you use and the Allure integration configuration, an appropriate path may be `allure-results`, `build/allure-results`, or some custom path. - `allure_history`: an arbitrary name for a directory to which the result will be saved. **Allure 3** doesn't have an equivalent action, so the workflow installs and runs the `allure` CLI directly: - `npm install -g allure` installs the CLI. - `allure generate build/allure-results -o allure-report` builds the report into the `allure-report` directory. As noted above, this doesn't give you test history - for that, set up the [self-hosted storage service](/docs/self-hosted-storage/) instead. ### Publish test report The final step of the workflow will push the generated directory to the branch used for GitHub Pages. This is supposed to trigger a second workflow run, called “pages build and deployment”, which, in turn, will update the actual contents on the GitHub Pages domain. (See [Set up publishing to GitHub Pages](#_3-set-up-publishing-github-pages) below.) This step is the same for both engines, aside from which directory gets published. In the workflow file, set the following values for this step: - `name`: any human-redable name, e.g., “Publish test report”. - `uses`: [`peaceiris/actions-gh-pages@v4`](https://github.com/marketplace/actions/github-pages-action). - `if`: `always()`. - `with`: - `github_token`: `${{ secrets.GITHUB_TOKEN }}`. See [Enable write access for GitHub Actions](#_2-enable-write-access-for-workflow-runs) for more details. - `publish_branch`: the branch used for the GitHub Pages content. - `publish_dir`: the directory to be published - the `allure_history` value from the [Build test report](#build-test-report) step for Allure 2, or the `allure generate` output directory (`allure-report` above) for Allure 3. ## 2. Enable write access for workflow runs When pushing files to the GitHub Pages branch, the workflow uses an authentication token generated by GitHub. However, the default configuration does not allow GitHub Actions to push files to the repository. The `permissions: contents: write` block already included in the example workflow above grants this for that one workflow only, which is the preferred, minimal-scope approach - prefer it over the repository-wide setting below when you can. (You may see older examples use the broader `permissions: write-all` instead - that grants write access to far more than this workflow needs, like issues and packages.) If you'd rather grant it repository-wide instead - or if you're still seeing errors like this in your workflow run's logs even with the `permissions` block in place: ```plain remote: Permission to ⟨USER⟩/⟨REPOSITORY⟩.git denied to github-actions[bot]. fatal: unable to access 'https://github.com/⟨USER⟩/⟨REPOSITORY⟩.git/': The requested URL returned error: 403 ``` To fix this, you need to grant write permissions to the token. 1. On the project's page on GitHub, go to **Settings → Actions → General**. 1. Under the **Workflow permissions** section, select the **Read and write permissions** option. 1. Click **Save**. ![Allure Github configure workflow permissions](/images/integrations/github/permissions.png "Allure Github configure workflow permissions") ## 3. Set up publishing to GitHub Pages After you've run the workflow for the first time, it will push the test report to the `gh-pages` branch. On a public repository, GitHub will often detect this and publish it automatically, without any further action needed. If that doesn't happen - or you'd rather configure it explicitly - here's how: 1. Make sure that the branch with the content exists (`gh-pages` in the example above). 1. On the project's page on GitHub, go to **Settings → Pages**. 1. Under the **Build and deployment** section, specify the options: - **Source**: “Deploy from a branch”. - **Branch**: the branch used for the GitHub Pages content. In the next dropdown list, select “/ (root)”. 1. Click **Save**. 1. Go to the **Actions** tab. 1. Make sure that the workflow run called “pages build and deployment” was automatically created. Once the run is completed, the test report should appear on the GitHub Pages domain. ![Allure Github configure github pages](/images/integrations/github/pages.png "Allure Github configure github pages") Once your CI is generating and publishing reports on every run, you might also want automatic pull request summaries - a results table, optional per-test breakdowns, and a Quality Gate check, all posted as PR comments. Take a look at the [Allure GitHub Action](/docs/integrations-github-action/) for that; it's a separate, complementary action you can add alongside the workflow above. --- ## Setting up JUnit 5 with GitHub Actions > URL: https://allurereport.org/docs/guides/junit5-github-actions/ # Setting up JUnit 5 with GitHub Actions When we write our tests, the basic principle is to make them as focused as possible to minimize the number of error sources for each test. Ideally, when a test fails, you should only have to look for the cause in one place. The test environment causes many failures, so it has become standard practice to run tests on a CI server. This allows us to keep the environment constant and avoid the hated phrase - “Hey, it works on my machine.” A CI server with tests also allows every team member to run tests easily without setting up the whole environment on each computer. Testing becomes a tool for the whole team, not just QA specialists. In this guide, we’ll demonstrate the benefits of running tests in a CI and show how to set up Allure Report with GitHub Actions and JUnit. ## 1. Preparation ### Prerequisites Make sure the following prerequisites are met: - [A GitHub repository is created](https://github.com/) and [GitHub Actions are enabled](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository). - [A Selenoid server is up and running](https://aerokube.com/selenoid/latest/) (for running UI tests). - [Docker is installed](https://docs.docker.com/get-started/get-docker/) (for Selenoid). ### Dependency List This guide uses the following packages: - [org.aspectj:aspectjweaver: 1.9.25](https://central.sonatype.com/artifact/org.aspectj/aspectjweaver/1.9.25) - [org.junit:junit-bom: 5.14.4](https://central.sonatype.com/artifact/org.junit/junit-bom/5.14.4) - [org.junit.jupiter:junit-jupiter-api](https://central.sonatype.com/artifact/org.junit.jupiter/junit-jupiter-api) - [org.junit.jupiter:junit-jupiter-engine](https://central.sonatype.com/artifact/org.junit.jupiter/junit-jupiter-engine) - [com.codeborne:selenide: 7.17.0](https://central.sonatype.com/artifact/com.codeborne/selenide/7.17.0) - [io.qameta.allure:allure-bom: 2.35.4](https://central.sonatype.com/artifact/io.qameta.allure/allure-bom/2.35.4) - [io.qameta.allure:allure-jupiter](https://central.sonatype.com/artifact/io.qameta.allure/allure-jupiter) - [io.qameta.allure:allure-selenide](https://central.sonatype.com/artifact/io.qameta.allure/allure-selenide) ### Code sample The complete source code used in this guide is available at [https://github.com/allure-examples/guide-junit5-github-actions](https://github.com/allure-examples/guide-junit5-github-actions). ### Setup You’ll need to create a Git repository with your project and upload it to GitHub. You can download a default project with JUnit, Gradle, and Allure from [Allure Start](/start/). The GitHub repository needs to have GitHub Actions (they are enabled by default; if not - follow the instructions from [here](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository)). If you want to run UI tests, you’ll also need to [set up a Selenoid browser farm](https://aerokube.com/selenoid/latest/) (or have access to one). If you’re setting it up yourself, your computer will need to have Docker installed. ## 2. Setting up GitHub Actions To begin with, you’ll need to create an Actions workflow. In your repository, create a folder called `.github/workflows`. GitHub recognizes any file with the `.yml` or `.yaml` extension in that folder as a workflow. So, let’s go ahead and create one with the following content: ```yaml name: Run tests on: [push] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up JDK uses: actions/setup-java@v4 with: distribution: zulu java-version: 17 - name: Run tests run: ./gradlew clean test ``` Let’s go over what this means. The `name` is going to be displayed in the Actions tab when you run this. `on: [push]` is the trigger that executes the workflow; in this case, it’s when we push to the repository. You can [add other triggers](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows), e.g. `on: [push, workflow_dispatch]`; now, the workflow will both run on push and allow you to trigger the workflow manually. The rest of the file describes the actual stuff that happens when the workflow is triggered: setting up the JDK and running our tests. Now, let’s commit our new workflow and push it to GitHub. This will create a new Action and immediately execute it: ![Newly created GitHub action](/images/guides/junit5-github-actions/new_action.png) Inside the workflow run, you’ll find a log from the tests that have been executed: ![Run tests log](/images/guides/junit5-github-actions/test_log.png) We’re not finished, though. As a general rule, people don’t run browser GUIs on CI/CD servers: it’s too resource-intensive, difficult to scale, and there is a lot of hassle with version management. This is why your GitHub virtual machine doesn’t have a browser, so all your GUI tests will throw errors. One solution is to use a remote browser on Selenoid, which is specifically created to run lots of browsers efficiently. The way you connect to it depends on your browser control tool. We’re using Selenide, and it just needs the address of our Selenoid server: ```java @BeforeAll static void setupSelenoid() { Configuration.remote = "https://your.selenoid.address/wd/hub"; } ``` However, it's considered a bad practice to embed URLs like this directly into the source code. Instead, let's pass it via an environment variable: ```java @BeforeAll static void setupSelenoid() { String selenideUrl = System.getenv("SELENIDE_URL"); if (selenideUrl != null && !selenideUrl.isEmpty()) { Configuration.remote = selenideUrl; } } ``` And propagate a GitHub variable into this environment variable in the workflow definition: ```yml name: Run tests on: [push] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up JDK uses: actions/setup-java@v4 with: distribution: zulu java-version: 17 - name: Run tests run: SELENIDE_URL="${{ vars.SELENIDE_URL }}" ./gradlew clean test ``` Now, as long as you have the `SELENIDE_URL` GitHub variable [defined in the repo](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/store-information-in-variables#creating-configuration-variables-for-a-repository), provided it points to an existing Selenoid server, your UI tests should also run on GitHub Actions. Excellent! Now, everyone can use tests - they will be executed whenever someone commits something. However, the output doesn’t tell us much about test results. Whether you want to debug or just get a general understanding of the state of the system, an Allure report would be much easier to read. ## 3. Integrating with Allure Report Running Allure Report on a CI server is the best way to make use of all its features. In particular, you will automatically have access to test run history. With test run history, you can [identify unstable tests](/docs/test-stability/), find [tests that changed their status since last report](/docs/sorting-and-filtering/#filter-tests-by-marks), and [visualize changes in metrics](/docs/visual-analytics/#trend-charts). For GitHub Actions to work with Allure Report, we’ll need to do a little more work. ### a. Create a branch We’ll need a new branch for GitHub Pages (below, we’ll assume it’s called `gh-pages`). Keep in mind, though - this is _not_ where the workflow files should be located. This is where the the files of our report will be pushed. That's why you may create an empty branch that isn't related to any other branches in the repository: ```bash git switch --orphan gh-pages git commit --allow-empty -m "create a branch for GitHub Pages" git push -u origin gh-pages git switch a-previous-branch ``` ### b. Modify the workflow Next, we need to update the workflow file - add the steps for loading report history, building the report, and publishing it to GitHub Pages. **The two Allure Report engines need different steps here** - [`simple-elf/allure-report-action`](https://github.com/simple-elf/allure-report-action), the community action used for the Allure 2 setup below, always generates an Allure 2 report internally (it bundles its own copy of the Allure 2 command line tool), so it can't be reused for Allure 3. For Allure 3, we install the `allure` CLI directly in the workflow instead. The file should look like this: **Allure 2:** ```yaml name: Run tests and publish the report on: [push] permissions: contents: write jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 - name: Set up JDK uses: actions/setup-java@v6 with: distribution: zulu java-version: 17 - name: Run tests run: SELENIDE_URL="${{ vars.SELENIDE_URL }}" ./gradlew clean test - name: Load test report history uses: actions/checkout@v7 if: always() continue-on-error: true with: ref: gh-pages path: gh-pages - name: Build test report uses: simple-elf/allure-report-action@v1.15 if: always() with: gh_pages: gh-pages allure_history: allure-history allure_results: build/allure-results - name: Publish test report uses: peaceiris/actions-gh-pages@v4 if: always() with: github_token: ${{ secrets.GITHUB_TOKEN }} publish_branch: gh-pages publish_dir: allure-history ``` **Allure 3:** ```yaml name: Run tests and publish the report on: [push] permissions: contents: write jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 - name: Set up JDK uses: actions/setup-java@v6 with: distribution: zulu java-version: 17 - name: Run tests run: SELENIDE_URL="${{ vars.SELENIDE_URL }}" ./gradlew clean test - name: Build test report if: always() run: | npm install -g allure allure generate build/allure-results -o allure-report - name: Publish test report uses: peaceiris/actions-gh-pages@v4 if: always() with: github_token: ${{ secrets.GITHUB_TOKEN }} publish_branch: gh-pages publish_dir: allure-report ``` A couple of things to note here: - `permissions: contents: write`: this allows the workflow to push commits into the GitHub Pages branch. (You may see older examples use the broader `permissions: write-all` - that grants write access to far more than this workflow needs, like issues and packages, so prefer the minimal `contents: write` shown here.) - `if: always()`: these lines ensure the report will be built and published even if there are failed tests. Warning: This Allure 3 example doesn't carry test history between runs - each report is generated fresh, with nothing to compare it against. For test history that works properly with Allure 3, use the [self-hosted storage service](/docs/self-hosted-storage/) - see the [Docker](/docs/guides/self-hosted-storage-docker/) or [Cloudflare Workers](/docs/guides/self-hosted-storage-cloudflare/) guides to set it up. ### c. Enable the deployment If you've run the workflow above, the report has been pushed to the `gh-pages` branch. On a public repository, GitHub will often detect this and publish it automatically, without any further action needed. If that doesn't happen - or you'd rather configure it explicitly - here's how: - Go to **Settings > Pages** - Under **Build and deployment**, set the **Source** option to **Deploy from a branch** - The **Branch** should be set to `gh-pages` (or whichever branch you’re using for Pages) ![GitHub Pages settings](/images/guides/junit5-github-actions/gh_pages_settings.png) - Click **Save** - Make sure the new workflow (**pages-build-deployment**) has been created under **Actions**: ![Pages deployment workflow](/images/guides/junit5-github-actions/pages_workflow.png) With that, your reports should appear at the domain that GitHub Pages has created for you (you can find the link under **Settings > Pages**): ![GitHub Pages URL](/images/guides/junit5-github-actions/gh_pages_url.png) The Allure 2 workflow above keeps history via the `simple-elf` action's `gh-pages`/`allure-history` folders, so its Overview page shows an interactive Trend chart out of the box. For Allure 3, the equivalent view - the Status dynamics chart under **Report > Graphs** - needs history data to show anything, which (as noted above) means setting up the self-hosted storage service rather than relying on the basic workflow here. Once history is present, either way, it looks like this: Images: /images/guides/junit5-github-actions/allure-report-with-history-allure3-light.png, /images/guides/junit5-github-actions/allure-report-with-history-allure3-dark.png, /images/guides/junit5-github-actions/allure-report-with-history-allure2-light.png, /images/guides/junit5-github-actions/allure-report-with-history-allure2-dark.png Once your CI is generating reports on every run, you might also want automatic pull request summaries - a results table, optional per-test breakdowns, and a Quality Gate check, all posted as PR comments. Take a look at the [Allure GitHub Action](/docs/integrations-github-action/) for that; it's a separate, complementary action you can add alongside the steps above. And with that, you’re ready to test in a standardized and stable setting. ## 4. Conclusion When working as a team, the best way to run tests is on a CI server; this allows everyone quick access to tests and ensures that test results are reproducible. GitHub Actions allows you to execute tests in a centralized setting, and Allure Report makes test results quickly accessible to the entire team. --- ## JUnit 5 parameterization > URL: https://allurereport.org/docs/guides/junit5-parametrization/ # JUnit 5 parameterization When writing automated tests, we often use parameterization, which means running the same test with different inputs. Let's find out what problems parameterization can solve and how to use it with Allure Report. ## 1. Preparation ### Prerequisites Make sure the following prerequisites are met: - [Java installed](https://openjdk.org/install/). - [Gradle installed](https://docs.gradle.org/current/userguide/installation.html). - [Allure installed](/docs/v3/install/) ### Dependency List This guide uses the following packages: - [org.junit:junit-bom:5.14.4](https://central.sonatype.com/artifact/org.junit/junit-bom/5.14.4) - [io.qameta.allure:allure-bom:2.35.4](https://central.sonatype.com/artifact/io.qameta.allure/allure-bom/2.35.4) ### Code sample The complete source code used in this guide is available at [https://github.com/allure-examples/guide-junit5-parametrization](https://github.com/allure-examples/guide-junit5-parametrization). ### Setup To run the examples in this guide, you need to have Java and Allure Report installed. Next, download a fresh project with JUnit from [Allure Start](/start/). ## 2. Why use parameterization We'll start with a straightforward example: adding two numbers together. ```java @Test public void testSum() { assertEquals(1 + 2, 3); } ``` How do we run this test on multiple sets of data? The first solution that comes to mind is just creating a loop inside the test. Something like this: ```java @Test public void testSum() { final int[][] data = new int[][]{ {1, 2, 3}, {-1, 1, 0}, {0, 0, 0} }; for (int[] datum : data) { assertEquals(datum[0] + datum[1], datum[2]); } } ``` This approach has many drawbacks: 1. If the test fails on one data set, it will never reach any others after it. 2. Data is accessed via an array, and there are no named variables, which hurts readability. This drawback isn't too obvious here, but if the test were better structured, it would definitely stand out. 3. A test report only shows the test's name, not the particular cycle of the loop. In this setup, there is no way to make it more readable. Images: /images/guides/junit5-parametrization/test-with-loop-allure3-light.png, /images/guides/junit5-parametrization/test-with-loop-allure3-dark.png, /images/guides/junit5-parametrization/test-with-loop-allure2-light.png, /images/guides/junit5-parametrization/test-with-loop-allure2-dark.png So, what would be a better way of doing things? JUnit5 already has an answer: [parameterization](https://docs.junit.org/5.14.3/writing-tests/parameterized-classes-and-tests.html). Let's use it to rewrite the previous test: ```java private static Stream dataProvider() { return Stream.of( Arguments.of(1, 2, 3), Arguments.of(-1, 1, 0), Arguments.of(0, 0, 0) ); } @ParameterizedTest @MethodSource("dataProvider") public void testSum(int x, int y, int sum) { assertEquals(x + y, sum); } ``` This approach allows us to solve the problems that we've talked about before: 1. The test is executed multiple times, and even if one of the runs fails, you'll still get feedback from all the others. 2. Data is accessed with variables (`x`, `y`, and `sum`), which is much more convenient. 3. In a report, multiple launches of the parameterized test are shown as separate test runs: Images: /images/guides/junit5-parametrization/test-with-parameters-allure3-light.png, /images/guides/junit5-parametrization/test-with-parameters-allure3-dark.png, /images/guides/junit5-parametrization/test-with-parameters-allure2-light.png, /images/guides/junit5-parametrization/test-with-parameters-allure2-dark.png ## 3. Parameterization in Allure Allure Report [integrates with JUnit5](https://allurereport.org/docs/junit-jupiter/) and supports parameterized automated tests. But before we use them, let's find out how they work in Allure. If you simply turn on the Allure integration, parameterized tests will unfortunately look like normal tests, just shown multiple times in the report: Images: /images/guides/junit5-parametrization/allure-without-parameters-allure3-light.png, /images/guides/junit5-parametrization/allure-without-parameters-allure3-dark.png, /images/guides/junit5-parametrization/allure-without-parameters-allure2-light.png, /images/guides/junit5-parametrization/allure-without-parameters-allure2-dark.png Why does this happen? Because Allure has to be told about the parameters. To understand what Allure sees, run a parameterized test and then look at the allure-results folder. There, open any `{uid}-result.json` file and find the `parameters` section: ```json { "testCaseName": "testSum(int, int, int)", "parameters": [ { "name": "UniqueId", "value": "[engine:junit-jupiter]/[class:org.example.junit_parameterization.SumTest]/[test-template:testSumParameterized(int, int, int)]/[test-template-invocation:#1]", "mode": "hidden" } ], "start": 1717700137076, "stop": 1717700137101 } ``` As you can see, this section does not mention `x`, `y`, or `sum`, which is why there is no section for parameters; they are just written as part of the method name. How do we solve this problem? If you want a detailed solution, you can look it up [in the documentation](https://allurereport.org/docs/junit-jupiter-reference/#parametrized-tests), but for now, let's just rewrite our parameterized test like this: ```java @ParameterizedTest @MethodSource("dataProvider") public void testSum(int x, int y, int sum) { Allure.parameter("x", x); Allure.parameter("y", y); Allure.parameter("sum", sum); assertEquals(x + y, sum); } ``` Now, if we re-run our example, the `parameters` section in the test results file will look like this: ```json { "testCaseName": "testSum(int, int, int)", "parameters": [ { "name": "UniqueId", "value": "[engine:junit-jupiter]/[class:org.example.junit_parameterization.SumTest]/[test-template:testSumAllurified(int, int, int)]/[test-template-invocation:#2]", "mode": "hidden" }, { "name": "x", "value": "-1" }, { "name": "y", "value": "1" }, { "name": "sum", "value": "0" } ] } ``` As you can see, parameters have finally appeared in the test results, which means the report should now look like this: Images: /images/guides/junit5-parametrization/allure-with-parameters-allure3-light.png, /images/guides/junit5-parametrization/allure-with-parameters-allure3-dark.png, /images/guides/junit5-parametrization/allure-with-parameters-allure2-light.png, /images/guides/junit5-parametrization/allure-with-parameters-allure2-dark.png ## 4. Conclusion Parameterization is a very useful testing technique that allows you to run the same test on multiple data sets without duplicating the test or complicating it with loops. This increases readability and makes test maintenance easier. Parameterization is fully supported in the Allure-JUnit5 integration. The Allure integrations are constantly improved, and in upcoming versions, you might be able to achieve the same result without explicitly writing the parameters. Still, the current approach provides the required functionality and allows you to check everything before building the report. --- ## Integrating screenshots and other attachments in Allure Report with Selenide and JUnit 5 > URL: https://allurereport.org/docs/guides/junit5-selenide-screenshots/ # Integrating screenshots and other attachments in Allure Report with Selenide and JUnit 5 Reading test results has to be a very quick process. Ideally, the reader should be able to understand the source of failure without opening any additional code, running the debugger, or bothering other team members. To achieve this, we must provide as much information as possible in as little space as possible; screenshots serve this purpose very well. Let’s try taking screenshots with Selenide in some JUnit tests and viewing them with Allure Report. ## 1. Preparation ### Prerequisites Make sure the following prerequisites are met: - [Java is installed](https://openjdk.org/install/). - [Gradle is installed](https://docs.gradle.org/current/userguide/installation.html). - [Allure is installed](/docs/v3/install/) ### Dependency List This guide uses the following packages: - [org.aspectj:aspectjweaver: 1.9.25](https://central.sonatype.com/artifact/org.aspectj/aspectjweaver/1.9.25) - [org.junit:junit-bom: 5.14.4](https://central.sonatype.com/artifact/org.junit/junit-bom/5.14.4) - [org.junit.jupiter:junit-jupiter-api](https://central.sonatype.com/artifact/org.junit.jupiter/junit-jupiter-api) - [org.junit.jupiter:junit-jupiter-engine](https://central.sonatype.com/artifact/org.junit.jupiter/junit-jupiter-engine) - [com.codeborne:selenide: 7.17.0](https://central.sonatype.com/artifact/com.codeborne/selenide/7.17.0) - [io.qameta.allure:allure-bom: 2.35.4](https://central.sonatype.com/artifact/io.qameta.allure/allure-bom/2.35.4) - [io.qameta.allure:allure-selenide: 2.35.4](https://central.sonatype.com/artifact/io.qameta.allure/allure-selenide/2.35.4) ### Code sample The complete source code used in this guide is available at [https://github.com/allure-examples/guide-junit5-selenide-screenshots](https://github.com/allure-examples/guide-junit5-selenide-screenshots). ### Setup To run the examples in this guide, you need to have Java and Allure Report installed. Next, download a fresh project with JUnit from [Allure Start](/start/). Then, [add Selenide](https://selenide.org/quick-start.html) to that project. With Gradle, that means adding the following lines to `build.gradle`: ```groovy dependencies { testImplementation 'com.codeborne:selenide:7.17.0' testImplementation 'io.qameta.allure:allure-selenide:2.35.4' } ``` ## 2. Screenshots in Selenide Selenide takes screenshots automatically on every test failure, which is very convenient. By default, those screenshots are stored in `build/reports/tests`. You can control the screenshot’s location with the `-Dselenide.reportsFolder=test-result/reports` system property. If you wish to take a screenshot at an arbitrary time, you can do it with just one line of code: ```java import static com.codeborne.selenide.Selenide.screenshot; String pngFileName = screenshot("my_file_name"); ``` This generates two files, PNG and HTML, in the screenshot folder. If you want to further customize the process of taking screenshots, you can add an extension to your test class: ```java public class MyTest { @RegisterExtension static ScreenShooterExtension screenshotEmAll = new ScreenShooterExtension(true).to("target/screenshots"); } ``` All of this is great for taking screenshots, but viewing them isn’t very convenient: you don’t have all your data in one place, which means it takes longer to resolve test failures. To remedy that, let’s attach our screenshot to an Allure report. ## 3. Integrating with Allure Report Allure Report shows you screenshots inside the test where they are taken. That is also where the error messages and other attachments are presented to you, so you have everything you need to analyze test results in one place: Images: /images/guides/junit-selenide-screenshots/screenshot-in-allure-allure3-light.png, /images/guides/junit-selenide-screenshots/screenshot-in-allure-allure3-dark.png, /images/guides/junit-selenide-screenshots/screenshot-in-allure-allure2-light.png, /images/guides/junit-selenide-screenshots/screenshot-in-allure-allure2-dark.png Allure stores attachments in the `allure-results` folder; links to them are written in the JSON file that stores test results. It looks like this: ```json { ... "attachments": [ { "name": "Screenshot", "type": "image/png", "source": "04ea4c96-5f27-42d6-8ab3-3cd390ca3d8a-attachment.png" } ], ... } ``` You don’t have to manage screenshot placement or anything like that; it’s all done automatically. To integrate Allure Report with Selenide, you need to use the `AllureSelenide` listener: ```java @BeforeAll static void setupAllureReports() { SelenideLogger.addListener("AllureSelenide", new AllureSelenide() .screenshots(true) ); } ``` Allure will now automatically attach the screenshots that Selenide makes by default on test failure. You don’t have to add any extra code to the tests themselves here. If you want to take a screenshot at an arbitrary time and attach it to Allure, you can either use the `Allure.attachment` method or the `@Attachment` annotation. With `@Attachment`, you define a method that returns the data type appropriate for your attachment type: ```java @Attachment(type = "image/png") public byte[] screenshot() throws IOException { String screenshotAsBase64 = Selenide.screenshot(OutputType.BASE64); return Base64.getDecoder().decode(screenshotAsBase64); } ``` To ensure the browser displays the attachment correctly, it is best to specify the attachment type in the annotation arguments. And with `Allure.attachment`: ```java public void screenshot() throws IOException { byte[] screenshot = Selenide.screenshot(OutputType.BYTES); try (InputStream is = new ByteArrayInputStream(screenshot)) { Allure.attachment("image.png", is); } } ``` Allure Report can also handle other types of attachments - text, tables, URI lists, documents (XML, JSON, YAML). Here’s an example: ```java Allure.addAttachment("response content", "text/xml", responseContent); ``` A report with such an attachment is going to look like this: Images: /images/guides/junit-selenide-screenshots/attachment-in-allure-allure3-light.png, /images/guides/junit-selenide-screenshots/attachment-in-allure-allure3-dark.png, /images/guides/junit-selenide-screenshots/attachment-in-allure-allure2-light.png, /images/guides/junit-selenide-screenshots/attachment-in-allure-allure2-dark.png You can consult the official documentation for more on [attachments with JUnit 5](https://allurereport.org/docs/junit-jupiter-reference/#attachments) and [attachment types](https://allurereport.org/docs/attachments/). ## 4. Conclusion Selenide takes screenshots by default after each failed test, which is when you need screenshots the most; you can also generate screenshots at arbitrary times. Allure Report integrates fully with Selenide and allows you to view screenshots together with other attachment types (text, documents, tables, video) inside the respective test case, providing the reader of your test with the maximum amount of data to resolve test failures. --- ## Integrating screenshots and other attachments in Allure Report with JUnit 5 and Selenium > URL: https://allurereport.org/docs/guides/junit5-selenium-screenshots/ # Integrating screenshots and other attachments in Allure Report with Selenium and JUnit 5 Reading test results needs to be quick. Ideally, the person looking at the results should be able to figure out what went wrong without checking extra code, using a debugger, or asking teammates for help. To make this happen, we should give a lot of information in a small space, and taking screenshots is a great way to do that. Let's try capturing screenshots with JUnit5 and Selenium and then viewing them using Allure Report. ## 1. Preparation ### Prerequisites Make sure the following prerequisites are met: - [Java](https://www.java.com/en/download/) - [Gradle](https://gradle.org/install/) - [Allure Report](/docs/v3/install/) ### Dependency List This guide uses the following packages: - [org.aspectj:aspectjweaver: 1.9.25](https://central.sonatype.com/artifact/org.aspectj/aspectjweaver/1.9.25) - [org.junit:junit-bom: 5.14.4](https://central.sonatype.com/artifact/org.junit/junit-bom/5.14.4) - [org.junit.jupiter:junit-jupiter-api: 5.14.4](https://central.sonatype.com/artifact/org.junit.jupiter/junit-jupiter-api) - [org.junit.jupiter:junit-jupiter-engine: 5.14.4](https://central.sonatype.com/artifact/org.junit.jupiter/junit-jupiter-engine) - [org.seleniumhq.selenium:selenium-java: 4.47.0](https://central.sonatype.com/artifact/org.seleniumhq.selenium/selenium-java/4.47.0) - [io.qameta.allure:allure-bom: 2.35.4](https://central.sonatype.com/artifact/io.qameta.allure/allure-bom/2.35.4) - [io.qameta.allure:allure-jupiter: 2.35.4](https://central.sonatype.com/artifact/io.qameta.allure/allure-jupiter/2.35.4) ### Code sample The source code used in this guide is available at [https://github.com/allure-examples/guide-junit5-selenium-screenshots](https://github.com/allure-examples/guide-junit5-selenium-screenshots). ### Setup To run the examples in this guide: 1. Install [Java](https://www.java.com/en/download/) 1. Install [Gradle](https://gradle.org/install/) 1. Install [Allure Report](/docs/v3/install/) 1. Download [the sample repository](https://github.com/allure-examples/guide-junit5-selenium-screenshots/) 1. Navigate to the repository and run the command: ```shell ./gradlew test ``` ## 2. Taking screenshots with Selenium To take screenshots with Selenium at an arbitrary time, you can use the `getScreenshotAs` method provided by the [`TakesScreenshot`](https://www.selenium.dev/selenium/docs/api/java/org/openqa/selenium/TakesScreenshot.html) interface. All Selenium [WebDrivers](https://www.selenium.dev/documentation/webdriver/) and [WebElements](https://www.selenium.dev/documentation/webdriver/elements/) implement this interface. For example: **A screenshot of a page:** ```java import java.io.File; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.StandardCopyOption; import org.openqa.selenium.OutputType; import org.openqa.selenium.TakesScreenshot; import org.openqa.selenium.WebDriver; public class ScreenshotUtils { public static void savePageScreenshot(WebDriver webDriver, Path path) { File screenshotFile = ((TakesScreenshot) webDriver).getScreenshotAs(OutputType.FILE); try { Files.copy(screenshotFile.toPath(), path, StandardCopyOption.REPLACE_EXISTING); } catch (IOException e) { System.out.println("Can't copy the screenshot file:" + e.getMessage()); } } } ``` **A screenshot of an element:** ```java import java.io.File; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.StandardCopyOption; import org.openqa.selenium.OutputType; import org.openqa.selenium.WebElement; public class ScreenshotUtils { public static void saveElementScreenshot(WebElement webElement, Path path) { File screenshotFile = webElement.getScreenshotAs(OutputType.FILE); try { Files.copy(screenshotFile.toPath(), path, StandardCopyOption.REPLACE_EXISTING); } catch (IOException e) { System.out.println("Can't copy the screenshot file:" + e.getMessage()); } } } ``` Whenever you call any of the example functions, a screenshot will be saved at `path`. ## 3. Automatic screenshots on failure To take screenshots automatically of every test failure, you can create a class implementing the JUnit5 [`TestExecutionExceptionHandler`](https://docs.junit.org/5.14.3/api/org.junit.jupiter.api/org/junit/jupiter/api/extension/TestExecutionExceptionHandler.html) interface and override the [`handleTestExecutionException`](https://docs.junit.org/5.14.3/api/org.junit.jupiter.api/org/junit/jupiter/api/extension/TestExecutionExceptionHandler.html#method-detail) method. When a test fails, an exception is raised, and the `handleTestExecutionException` method allows handling this exception. Inside the overridden method, you should get a `WebDriver` or `WebElement` object and use the function from the [previous section](#_2-taking-screenshots-with-selenium). For example: ```java import java.nio.file.Path; import java.util.Objects; import java.util.function.Function; import org.junit.jupiter.api.extension.ExtensionContext; import org.junit.jupiter.api.extension.TestExecutionExceptionHandler; import org.openqa.selenium.WebDriver; public class AutoScreenshotExtension implements TestExecutionExceptionHandler { // Function to extract WebDriver from the ExtensionContext private final Function driverSupplier; public AutoScreenshotExtension(Function driverSupplier) { this.driverSupplier = driverSupplier; } @Override public void handleTestExecutionException(final ExtensionContext context, final Throwable throwable) throws Throwable { final WebDriver webDriver = this.driverSupplier.apply(context); if (Objects.nonNull(webDriver)) { final Path path = Path.of("Screenshot on failure (" + throwable.getClass().getSimpleName() + ").png"); ScreenshotUtils.savePageScreenshot(webDriver, path); } throw throwable; } } ``` After that, you need to add your class implementing the `TestExecutionExceptionHandler` interface as an extension to your test class. In the example, there is [`WebdriverExtension`](https://github.com/allure-examples/guide-junit5-selenium-screenshots/blob/main/src/test/java/org/allurereport/guide/WebdriverExtension.java) that creates web drivers for the tests. This extension also provides the current instance of the web driver to `AutoScreenshotExtension` when a test fails. This is done via the lambda function that we are passing to the constructor of `AutoScreenshotExtension`: ```java import org.junit.jupiter.api.extension.RegisterExtension; public class ScreenshotTest { // Extension for WebDriver @RegisterExtension WebdriverExtension webDriverManager = new WebdriverExtension(); @RegisterExtension AutoScreenshotExtension screenshotManager = new AutoScreenshotExtension( c -> this.webDriverManager.getWebDriver(c)); // Test logic } ``` This way, on each test failure, a screenshot will be saved at `path`. ## 4. Integrating with Allure Report Taking screenshots helps understand why tests fail, but looking at them all separately can be time-consuming. When test information is scattered in different places, figuring out and fixing problems takes longer. To make this easier, screenshots should be added directly to Allure Reports. This way, all important test details are in one place for quicker analysis and troubleshooting. In Allure Reports, you can see the screenshots alongside the tests where they occurred. You’ll also find error messages and other important details there, making it easier to review test results in one spot: Images: /images/guides/junit5-selenium-screenshots/screenshot-attachment-allure3-light.png, /images/guides/junit5-selenium-screenshots/screenshot-attachment-allure3-dark.png, /images/guides/junit5-selenium-screenshots/screenshot-attachment-allure2-light.png, /images/guides/junit5-selenium-screenshots/screenshot-attachment-allure2-dark.png To attach screenshots to Allure with Selenium, you can use one of the following: - the [`@Attachment`](https://allurereport.org/docs/junit-jupiter-reference/#attachments) annotation - the [`Allure.attachment`](https://allurereport.org/docs/junit-jupiter-reference/#attachments) static method - the [`Allure.addAttachment`](https://allurereport.org/docs/junit-jupiter-reference/#attachments) static method With the `@Attachment` annotation, you must define a method that returns an array of bytes `byte[]` or a `String` representing the attachment data. To ensure the browser displays the attachment correctly, we recommend specifying the media type in the annotation arguments. For example: **@Attachment with a page screenshot:** ```java import io.qameta.allure.Attachment; import org.openqa.selenium.OutputType; import org.openqa.selenium.TakesScreenshot; import org.openqa.selenium.WebDriver; public class ScreenshotUtils { @Attachment(type = "image/png") public static byte[] attachPageScreenshot(WebDriver webDriver) { return ((TakesScreenshot) webDriver).getScreenshotAs(OutputType.BYTES); } } ``` **@Attachment with an element screenshot:** ```java import io.qameta.allure.Attachment; import org.openqa.selenium.OutputType; import org.openqa.selenium.WebElement; public class ScreenshotUtils { @Attachment(type = "image/png") public static byte[] attachElementScreenshot(WebElement webElement) { return webElement.getScreenshotAs(OutputType.BYTES); } } ``` With the `Allure.attachment` static method, you can use slightly modified functions from the [**Taking screenshots with Selenium**](#_2-taking-screenshots-with-selenium) section: **Allure.attachment with a page screenshot:** ```java import io.qameta.allure.Allure; import java.io.ByteArrayInputStream; import org.openqa.selenium.OutputType; import org.openqa.selenium.TakesScreenshot; import org.openqa.selenium.WebDriver; public class ScreenshotUtils { public static void attachPageScreenshot(WebDriver webDriver, String name) { byte[] screenshotBytes = ((TakesScreenshot) webDriver).getScreenshotAs(OutputType.BYTES); Allure.attachment(name, new ByteArrayInputStream(screenshotBytes)); } } ``` **Allure.attachment with an element screenshot:** ```java import io.qameta.allure.Allure; import java.io.ByteArrayInputStream; import org.openqa.selenium.OutputType; import org.openqa.selenium.WebElement; public class ScreenshotUtils { public static void attachElementScreenshot(WebElement webElement, String name) { byte[] screenshotBytes = webElement.getScreenshotAs(OutputType.BYTES); Allure.attachment(name, new ByteArrayInputStream(screenshotBytes)); } } ``` To take automatically screenshot on test failure with Allure, you can use the `TestExecutionExceptionHandler` implementation as described in [**Automatic screenshots on failure**](#_3-automatic-screenshots-on-failure) and update the `handleTestExecutionException` method. For example, for using the `Allure.attachment` static method: ```java import java.util.Objects; import java.util.function.Function; import org.junit.jupiter.api.extension.ExtensionContext; import org.junit.jupiter.api.extension.TestExecutionExceptionHandler; import org.openqa.selenium.WebDriver; public class AutoScreenshotExtension implements TestExecutionExceptionHandler { private final Function driverSupplier; public AutoScreenshotExtension(Function driverSupplier) { this.driverSupplier = driverSupplier; } @Override public void handleTestExecutionException(final ExtensionContext context, final Throwable throwable) throws Throwable { final WebDriver webDriver = this.driverSupplier.apply(context); if (Objects.nonNull(webDriver)) { final String attachmentName = "Screenshot on failure (" + throwable.getClass().getSimpleName() + ")"; ScreenshotUtils.attachPageScreenshot(webDriver, attachmentName); } throw throwable; } } ``` ## 5. Other attachment types Allure Report can also handle other types of attachments, like text, tables, URI lists, documents (XML, JSON, YAML): ```java Allure.addAttachment("response content", "text/xml", responseContent); ``` An example of a report with sush an attachment: Images: /images/guides/junit5-selenium-screenshots/attachment-in-allure-allure3-light.png, /images/guides/junit5-selenium-screenshots/attachment-in-allure-allure3-dark.png, /images/guides/junit5-selenium-screenshots/attachment-in-allure-allure2-light.png, /images/guides/junit5-selenium-screenshots/attachment-in-allure-allure2-dark.png You can consult the official documentation for more on [attachments with JUnit 5](https://allurereport.org/docs/junit-jupiter-reference/#attachments) and [attachment types](https://allurereport.org/docs/attachments/). ## 6. Conclusion With Selenium and JUnit5, you can capture screenshots whenever you want during your tests, or you can set up a function that automatically takes screenshots at specific moments, like when a test fails or when it passes. Allure Report helps you see these screenshots along with other types of files (like text, documents, tables, and videos) for each test case. This gives anyone looking at your tests plenty of information to understand and fix any issues that come up. --- ## Playwright parameterization > URL: https://allurereport.org/docs/guides/playwright-parameterization/ # Playwright parameterization Parameterization is a technique that allows the execution of the same test with various inputs. This guide addresses the problems that parameterization can solve, how to implement parameterization in Playwright, and its integration with Allure Report. ## 1. Preparation ### Prerequisites Make sure you have: - [Allure Report installed](/docs/v3/install/) - [NodeJS installed](https://nodejs.org/en/download/) ### Dependency List This guide uses the following packages: - [@playwright/test: 1.62.1](https://www.npmjs.com/package/@playwright/test/v/1.62.1) - [allure-playwright: 3.10.2](https://www.npmjs.com/package/allure-playwright/v/3.10.2) - [allure-js-commons: 3.10.2](https://www.npmjs.com/package/allure-js-commons/v/3.10.2) ### Code sample The source code used in this guide is available at [https://github.com/allure-examples/guide-playwright-parameterization](https://github.com/allure-examples/guide-playwright-parameterization). ### Setup To run the examples in this guide: 1. Install [Node.js](https://nodejs.org/en/download/) 1. Install [Allure Report](/docs/v3/install/) 1. Download a fresh project with Playwright from [Allure Start](/start/) 1. Navigate to the project directory and install the dependencies using a package manager of your choice: **npm:** ```shell npm install ``` **yarn:** ```shell yarn install ``` **pnpm:** ```shell pnpm install ``` 1. Run the project: **npm:** ```shell npm test ``` **yarn:** ```shell yarn test ``` **pnpm:** ```shell pnpm test ``` 1. Open Alure Report: ```shell allure serve ``` ## 2. Benefits of parameterization Let’s start with a simple function. We want to check if adding two numbers gives the right answer: **JavaScript:** ```js import { expect, test } from "playwright/test"; const sum = (a, b) => { return a + b; }; test("basic sum", () => { expect(sum(1, 2)).toBe(3); }); ``` **TypeScript:** ```ts import { expect, test } from "playwright/test"; const sum = (a: number, b: number) => { return a + b; }; test("basic sum", () => { expect(sum(1, 2)).toBe(3); }); ``` Next, we would like to test this function with different data sets. The first idea might be to use a loop in the test and check each result inside the loop. It could look like this: **JavaScript:** ```js import { expect, test } from "playwright/test"; test("loop sum", () => { [ [1, 2, 3], [1, -1, 0], [0, 0, 0], ].forEach((testData) => { expect(sum(testData[0], testData[1])).toBe(testData[2]); }); }); ``` **TypeScript:** ```ts import { expect, test } from "playwright/test"; test("loop sum", () => { const testData: [number, number, number][] = [ [1, 2, 3], [1, -1, 0], [0, 0, 0], ]; for (const [a, b, expected] of testData) { expect(sum(a, b)).toBe(expected); } }); ``` This method has several problems: 1. If the test fails for one data set, it won’t check the rest. 1. The data is stored in an array, and there are no clear variable names, making it harder to understand. This isn’t very obvious here, but in a more organized test, it would be more noticeable. 1. The test report only shows the test name, not which specific data set caused the issue. There’s no way to make this clearer in this setup. Playwright has a better solution: [parameterization](https://playwright.dev/docs/test-parameterize). We can use a `forEach` loop, like before, but this time we put the test inside the loop and the input inside the test function title: **JavaScript:** ```js import { expect, test } from "playwright/test"; [ { x: 1, y: 2, expectedSum: 3 }, { x: 1, y: -1, expectedSum: 0 }, { x: 0, y: 0, expectedSum: 0 }, ].forEach(({ x, y, expectedSum }) => { test(`the sum of ${x} and ${y} should be ${expectedSum}`, () => { expect(sum(x, y)).toBe(expectedSum); }); }); ``` **TypeScript:** ```ts import { expect, test } from "playwright/test"; const testCases: { x: number; y: number; expectedSum: number }[] = [ { x: 1, y: 2, expectedSum: 3 }, { x: 1, y: -1, expectedSum: 0 }, { x: 0, y: 0, expectedSum: 0 }, ]; testCases.forEach(({ x, y, expectedSum }) => { test(`the sum of ${x} and ${y} should be ${expectedSum}`, () => { expect(sum(x, y)).toBe(expectedSum); }); }); ``` This method fixes the issues we discussed earlier: 1. The test runs multiple times, and even if one fails, the others will still give the feedback. 1. The data is now accessed using clear variable names (`x`, `y`, and `expectedSum`), which makes it easier to understand. 1. In a test report, each test run is shown as a separate entry, making it clearer. ## 3. Parameterization in Allure Report Allure Report [integrates with Playwright](/docs/playwright/), and supports parameterized automated tests. By default, the parameterized tests appear in Allure Report as separate tests without the distinction for the parameterized inputs. Images: /images/guides/playwright-parameterization/allure-without-parameters-allure3-light.png, /images/guides/playwright-parameterization/allure-without-parameters-allure3-dark.png, /images/guides/playwright-parameterization/allure-without-parameters-allure2-light.png, /images/guides/playwright-parameterization/allure-without-parameters-allure2-dark.png To make the parameterized inputs appear in Allure Report, you should use the [`allure.parameter()`](/docs/playwright-reference/#parametrized-tests) function. It creates a new parameter with a given name and value and forces the new parameter to be displayed in the report, but it **does not** change or hide the existing parameters. For example: **JavaScript:** ```js import * as allure from "allure-js-commons"; import { expect, test } from "playwright/test"; [ { x: 1, y: 2, expectedSum: 3 }, { x: 1, y: -1, expectedSum: 0 }, { x: 0, y: 0, expectedSum: 0 }, ].forEach(({ x, y, expectedSum }) => { test(`the sum of ${x} and ${y} should be ${expectedSum}`, async () => { await allure.parameter("x", x.toString()); await allure.parameter("y", y.toString()); await allure.parameter("expectedSum", expectedSum.toString()); expect(sum(x, y)).toBe(expectedSum); }); }); ``` **TypeScript:** ```ts import * as allure from "allure-js-commons"; import { expect, test } from "playwright/test"; const testCases: { x: number; y: number; expectedSum: number }[] = [ { x: 1, y: 2, expectedSum: 3 }, { x: 1, y: -1, expectedSum: 0 }, { x: 0, y: 0, expectedSum: 0 }, ]; testCases.forEach(({ x, y, expectedSum }) => { test(`the sum of ${x} and ${y} should be ${expectedSum}`, async () => { await allure.parameter("x", x.toString()); await allure.parameter("y", y.toString()); await allure.parameter("expectedSum", expectedSum.toString()); expect(sum(x, y)).toBe(expectedSum); }); }); ``` Warning: `allure.parameter()` expects a string value. Passing a number directly works in Allure 2, but Allure 3 renders it as `#__unknown_value__#` instead of the actual value - always convert to a string first. With that, Allure Report starts to show the parameters in the test results: Images: /images/guides/playwright-parameterization/allure-with-parameters-allure3-light.png, /images/guides/playwright-parameterization/allure-with-parameters-allure3-dark.png, /images/guides/playwright-parameterization/allure-with-parameters-allure2-light.png, /images/guides/playwright-parameterization/allure-with-parameters-allure2-dark.png The `allure.parameter()` function has an additional `options` argument. If given, the argument must be an object with the following properties: 1. `excluded` (Boolean, by default, `false`) defines how Allure Report treats the parameter if it changes the value across test runs. For example, if you pass a current timestamp as a parameter value, it will differ for each test retry. By default, Allure Report treats the tests with such parameters as different ones and shows each rerun separately. To avoid this, you can set the `excluded` value to `true`. Read more in [History and retries](/docs/history-and-retries/#making-sure-tests-are-identified-correctly). 1. `mode` (string) defines how the parameter appears in Allure Report. Possible values: - `"default"`: the parameter name and value appear in Allure Report as they are - `"masked"`: the parameter name appears in Allure Report, but the value is hidden. You can use this mode for passwords, tokens, and other sensitive parameters. - `"hidden"`: the parameter name and value don't appear in Allure Report. A common use case is to hide a parameter whose only purpose is to separate test results obtained from different test environments. You can learn more [here](/docs/history-and-retries/#common-pitfall-tests-from-different-environments-are-missing-in-the-report). Note that the parameter with `{ mode: "masked"}` or `{ mode:"hidden" }` can still be extracted from the `allure_results` directory. An example of the `allure.parameter()` function usage with the `options` object: ```js import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login, { mode: "masked" }); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } ``` ## 4. Conclusion Allure Report works natively with parameterization in Playwright and doesn't need extra setup or code. This testing approach allows you to run the same test with different data sets. This helps cut down on repetition and makes things simpler by not needing to manually loop through data. To make the test report clearer, you can also add parameters for your tests and control their appearance in Allure Report. --- ## Integrating screenshots in Allure Report with Pytest and Playwright > URL: https://allurereport.org/docs/guides/playwright-pytest-screenshots/ # Integrating screenshots in Allure Report with Pytest and Playwright Interpreting test results should be fast and straightforward, allowing the root cause of a failure to be immediately clear without requiring code inspection, debugging, or external assistance. To achieve this, test reports must present detailed information in a concise and accessible format. Incorporating screenshots is an effective way to provide visual context for test failures. This guide explains how to capture screenshots in Pytest with Playwright and integrate them into Allure Report to enhance readability and analysis. ## 1. Preparation ### Prerequisites Make sure the following prerequisites are met: - [Python installed](https://www.python.org/downloads/) - [Allure Report installed](/docs/v3/install/) ### Dependency list This guide uses the following packages: - [Pytest 9.1.1](https://pypi.org/project/pytest/9.1.1/) - [allure-pytest 2.16.0](https://pypi.org/project/allure-pytest/2.16.0/) - [Pytest-playwright 0.9.0](https://pypi.org/project/pytest-playwright/0.9.0/) ### Code sample The complete source code used in this guide is available at [https://github.com/allure-examples/guide-pytest-playwright-screenshots](https://github.com/allure-examples/guide-pytest-playwright-screenshots). ### Setup To run the examples in this guide: 1. Install [Python](https://www.python.org/downloads/) 1. Install [Allure Report](/docs/v3/install/) 1. Download a fresh project with Pytest from [Allure Start](/start/) 1. Add [the Playwright Pytest plugin](https://playwright.dev/python/docs/intro#installing-playwright-pytest) to your project. For example, if you use `pip`: **Install Playwright Pytest:** ```shell pip install pytest-playwright ``` ## 2. Screenshots with Playwright Pytest To take a screenshot with Playwright Pytest at an arbitrary time, use the [`screenshot`](https://playwright.dev/python/docs/api/class-page#page-screenshot) method provided by the `Page` object. To get the `Page` object, you can use the `page` fixture: ```python def test_example(page): page.goto("https://playwright.dev/") try: assert "Playwright" in page.title() except AssertionError: page.screenshot(path="path/to/screenshot.png") raise ``` To take a screenshot of a particular element, use the [`screenshot`](https://playwright.dev/python/docs/api/class-locator#locator-screenshot) method of the `Locator` object, for example: ```python element = page.get_by_role("search") # or however you find the element # take the screenshot element.screenshot(path="path/to/screenshot.png") ``` ## 3. Automatic screenshots on failure To take screenshots automatically of every test failure, set the [`--screenshot`](https://playwright.dev/python/docs/test-runners#cli-arguments) option to `only-on-failure` in your project's [Pytest configuration](https://pytest.org/en/latest/reference/customize.html#command-line-options-and-configuration-file-settings). For example, if you use `pyproject.toml` as a configuration file, add the following: ```toml [tool.pytest.ini_options] ... addopts = [ ... "--screenshot", "only-on-failure" ] ``` Now, when a test fails, a screenshot file will be saved automatically in the test results directory of your project. Playwright Pytest deletes and recreates the directory with each test run. ## 4. Integrating with Allure Report While capturing screenshots is beneficial for diagnosing test failures, viewing them separately can be inefficient. When test data is dispersed across multiple locations, the process of identifying and resolving failures becomes more time-consuming. To streamline this workflow, screenshots should be directly attached to Allure Reports, ensuring all relevant test information is consolidated in a single location for efficient analysis and troubleshooting. Allure Report shows you screenshots inside the test in which they were taken. That is also where the error messages and other attachments are presented to you, so you have everything you need to analyze test results in one place: Images: /images/guides/playwright-pytest-screenshots/screenshot-in-scenario-allure3-light.png, /images/guides/playwright-pytest-screenshots/screenshot-in-scenario-allure3-dark.png, /images/guides/playwright-pytest-screenshots/screenshot-in-scenario-allure2-light.png, /images/guides/playwright-pytest-screenshots/screenshot-in-scenario-allure2-dark.png To attach screenshots to Allure, use either the [`allure.attach()`](/docs/pytest-reference/#attaching-content-from-variables) or the [`allure.attach.file()`](/docs/pytest-reference/#reading-attachments-from-files) functions. Use `allure.attach.file()` to attach an existing file: ```python import allure from pathlib import Path # Attach a screenshot of a page or an element def attach_screenshot(pageOrElement, path): name = Path(path).name pageOrElement.screenshot(path=path) allure.attach.file( path, name=name, attachment_type=allure.attachment_type.PNG, ) ``` Use `allure.attach()` if you don’t need to save the screenshot on disk: ```python import allure # Attach a screenshot of a page or an element def attach_screenshot(pageOrElement, name): allure.attach( pageOrElement.screenshot(), name=name, attachment_type=allure.attachment_type.PNG, ) ``` Tip: When you need to attach a screenshot you just took, prefer using the `allure.attach` function if possible. The reason is that when you save a screenshot on disk, it may not always be immediately available for reading because of OS caching. If that happens, you may end up with an empty file attached to Allure. If you also set the `--screenshot` configuration option as described in [**Automatic screenshots on failure**](#_3-automatic-screenshots-on-failure) section, Playwright Pytest will save screenshots locally. To automatically attach screenshots to Allure on each test failure, you can add a hook in the `conftest.py` file in your project's test root directory. The hook may look up the screenshots Playwright Pytest saved after the failed tests and attach these screenshots to Allure Report. For example: ```python import re import allure import pytest from pathlib import Path # A regular expression to match the file names of the screenshots taken on test failures SCREENSHOT_NAME_PATTERN = re.compile(r"^test-failed-\d+\.png$") @pytest.hookimpl(hookwrapper=True) def pytest_runtest_teardown(item, nextitem): yield try: artifacts_dir = item.funcargs.get("output_path") if artifacts_dir: artifacts_dir_path = Path(artifacts_dir) if artifacts_dir_path.is_dir(): for file in artifacts_dir_path.iterdir(): if file.is_file() and SCREENSHOT_NAME_PATTERN.match(file.name): allure.attach.file( str(file), name=file.name, attachment_type=allure.attachment_type.PNG, ) except Exception as e: print(f"Error taking screenthot: {e}") ``` ## 5. Other attachment types Allure Report can also handle other types of attachments - text, tables, URI lists, or documents (XML, JSON, YAML). For example: ```python allure.attach( response_content, name="response content", attachment_type=allure.attachment_type.XML ) ``` Allure Pytest makes some attachments automatically: 1. everything passed to `sys.stdout` - e.g. `print(...)` 2. everything passed to `sys.stderr` - e.g. `print(..., file=sys.stderr)` 3. logging, e.g. `logging.warning(...)` These outputs are attached as pseudo-files to the report: Images: /images/guides/pytest-screenshots/attachment-pytest-auto-allure3-light.png, /images/guides/pytest-screenshots/attachment-pytest-auto-allure3-dark.png, /images/guides/pytest-screenshots/attachment-pytest-auto-allure2-light.png, /images/guides/pytest-screenshots/attachment-pytest-auto-allure2-dark.png If you don’t want all of this to appear in the report, use the [`--allure-no-capture`](/docs/pytest-configuration/#allure-no-capture) option. Allure also allows you to [attach videos](/docs/attachments/#videos). You can consult the official documentation for more on [attachments with Pytest](/docs/pytest/#attach-screenshots-and-other-files) and [attachment types](/docs/attachments/). ## 6. Conclusion With Pytest and Playwright, you can capture screenshots at any point during your tests or configure Playwright to take them automatically upon test failures. Allure Report combines these screenshots along with other attachments (such as text, documents, tables, and videos) into a single view, providing comprehensive data to help diagnose test failures efficiently. --- ## Pytest parameterization > URL: https://allurereport.org/docs/guides/pytest-parameterization/ # Pytest parameterization When writing automated tests, parameterization is frequently used, allowing the same test to be executed with different inputs. This approach addresses various challenges in test automation. This guide explores the problems parameterization can resolve, Pytest parameterization, and its integration with Allure Report. ## 1. Preparation ### Prerequisites Make sure the following prerequisites are met: - [Python installed](https://www.python.org/downloads/) - [Allure Report installed](/docs/v3/install/) ### Dependency List This guide uses the following packages: - [Pytest 9.1.1](https://pypi.org/project/pytest/9.1.1/) - [allure-pytest 2.16.0](https://pypi.org/project/allure-pytest/2.16.0/) ### Code sample The complete source code used in this guide is available at [https://github.com/allure-examples/guide-pytest-parameterization](https://github.com/allure-examples/guide-pytest-parameterization). ### Setup To run the examples in this guide: 1. Install [Python](https://www.python.org/downloads/) 1. Install [Allure Report](/docs/v3/install/) 1. Download a fresh project with Pytest from [Allure Start](/start/) 1. In the project directory, use the script that corresponds to your operating system: **MacOS / Linux:** ```shell ./run.sh ``` **Windows:** ```powershell .\run.bat ``` ## 2. Why use parameterization Let's start with a simple example: we want to test if a sum of two numbers is calculated correctly. ```python def add(a, b): return a + b def test_sum(): s = add(1, 2) assert s == 3 ``` Now, how can we execute this test across various data sets? One approach is implementing a loop through a data set within the test itself. For instance: ```python def test_sum_loop(): data = [ [1, 2, 3], [-1, 1, 0], [0, 0, 0] ] for x, y, ttl in data: s = add(x, y) assert s == ttl ``` This approach has many disadvantages: 1. If an assertion for one specific input fails, the whole test fails immediately without reporting the results of the remaining cases. 1. We lose visibility on which specific input set caused the test failure, making it harder to debug and maintain. 1. A test report only shows the test name, not the particular cycle of the loop. In this setup, there is no option to make it more readable. Pytest framework provides a better alternative: [parameterization](https://docs.pytest.org/en/stable/how-to/parametrize.html). To parameterize a test with Pytest, you can use the [`@pytest.mark.parametrize`](https://docs.pytest.org/en/stable/how-to/parametrize.html#pytest-mark-parametrize-parametrizing-test-functions) decorator with parameter names and a list of test inputs. Pytest runs the test function with each test input creating separate test cases. With this feature, we can rewrite the previous test: ```python data = [ [1, 2, 3], [-1, 1, 0], [0, 0, 0] ] @pytest.mark.parametrize("x,y,ttl", data) def test_sum_parameterized(x, y, ttl): s = add(x, y) assert s == ttl ``` With Pytest, you can also [parameterize fixtures](https://docs.pytest.org/en/stable/how-to/fixtures.html#fixture-parametrize). Parameterized fixtures allow running the same test multiple times with different fixture values as test inputs. For example: ```python @pytest.fixture(params=[0, 1, 2]) def sum_arguments(request): yield prepare_test_data(request.param) def test_sum_simple(sum_arguments): x, y, ttl = sum_arguments assert add(x, y) == ttl def test_sum_keyword(sum_arguments): x, y, ttl = sum_arguments assert sum((x, y)) == ttl ``` In this example, we define a fixture `sum_arguments` and pass a list as the fixture `params` argument. The `prepare_test_data` function represents code that prepares the test data that goes to the test functions below. In the example repository, this function reads the data from the global variable. But it could have also retrieved it from another source: a file, a database, and so on. This approach helps if you need to parameterize multiple tests using the same set of data. Both parameterized tests and parameterized fixtures provide significant advantages: 1. Each test is executed multiple times with different inputs. If one run fails, you will still get feedback from all the others. 1. The test data is defined in one place and accessed through variables (`x`, `y`, and `sum`), which is much more convenient. 1. In a test report, multiple runs of a parameterized test with different inputs are shown separately. 1. The data preparation is separated from the test functions making the code more compact and easier to read. ## 3. Parameterization in Allure Allure Report [integrates with Pytest](/docs/pytest/) and supports parameterized automated tests. An example of parameterized tests in Allure Report: Images: /images/guides/pytest-parameterization/report-with-parameters-allure3-light.png, /images/guides/pytest-parameterization/report-with-parameters-allure3-dark.png, /images/guides/pytest-parameterization/report-with-parameters-allure2-light.png, /images/guides/pytest-parameterization/report-with-parameters-allure2-dark.png Allure Report recognizes all Pytest parameterization mechanisms including [fixtures](https://docs.pytest.org/en/stable/explanation/fixtures.html), the [`@pytest.mark.parametrize`](https://docs.pytest.org/en/stable/how-to/parametrize.html#pytest-mark-parametrize) annotation, and the [`pytest_generate_tests`](https://docs.pytest.org/en/stable/how-to/parametrize.html#basic-pytest-generate-tests-example) hook, and displays all test information correctly without adding any extra code. For tests with parameterized fixtures, the report shows the original values passed to the fixtures. ### Test name interpolation You can pass Pytest parameters to the test title that appears in Allure Report with [`@allure.title`](/docs/pytest-reference/#title): ```python import pytest import allure @pytest.mark.parametrize("username", ["j_doe", "admin"]) @allure.title("Check {username} can log in") def test_login(username): # test logic ``` The formatting rules for `@allure.title` are the same as for [`str.format()`](https://docs.python.org/3/library/string.html#formatstrings). ### Adding dynamic parameters You can manually add parameters and values for the test cases in Allure Report with the [`allure.dynamic.parameter()`](/docs/pytest-reference/#parametrized-tests) function: ```python def test_sum(): x = 1 y = 2 ttl = 3 allure.dynamic.parameter("x", x) allure.dynamic.parameter("y", y) allure.dynamic.parameter("sum", ttl) assert add(x, y) == ttl ``` The function adds a new parameter to the report, but it **does not** change or hide the existing parameters. `allure.dynamic.parameter()` also affects the [test history and retries](/docs/history-and-retries/#making-sure-tests-are-identified-correctly). By default, Allure Report separates all retries and history of a parameterized test based on its parameter values. Sometimes, you may want to avoid this behavior for a new parameter. To do so, set `excluded` to `True`. This is useful if you add a parameter, whose value changes on each run. The next example adds a timestamp to a test: ```python from datetime import datetime def test_sum(): x = 1 y = 2 ttl = 3 allure.dynamic.parameter("Timestamp", str(datetime.now()), excluded=True) assert add(x, y) == ttl ``` Without `excluded`, each new run would create a new entry in the report. See more info [here](/docs/history-and-retries/#common-pitfall-a-test-s-retries-are-displayed-as-separate-tests). Another useful argument of `allure.dynamic.parameter` is `mode`. It accepts `allure.parameter_mode.HIDDEN` or `allure.parameter_mode.MASKED` values. The `allure.parameter_mode.HIDDEN` mode hides the parameter from Allure Report. Note, that the parameter can still be extracted from the `allure_results` directory. A common use case is to hide a parameter whose only purpose is to separate test results obtained from different test environments. You can learn more [here](/docs/history-and-retries/#common-pitfall-tests-from-different-environments-are-missing-in-the-report). The `allure.parameter_mode.MASKED` mode obfuscates the parameter value in Allure Report. Use this mode for passwords, tokens, and other sensitive parameters. ### Dealing with large values When you parameterize tests, the values may become very large. For example, you may have: ```python @pytest.mark.parametrize(["user"], [ # You may have a larger object here { "name": "John", "surname": "Doe", "login": "j_doe", "role": "admin" }, ]) def test_user_login(user): login = user["login"] # test logic ``` Large parameterized values may cause problems: they can make the test report take much more disk space and are difficult to read. To avoid these problems, we recommend parameterizing with the map keys and look up the values in the test functions. We also recommend parameterizing fixtures as they allow custom initialization. For example: ```python USERS = { "j_doe": { "name": "John", "surname": "Doe", "login": "j_doe", "role": "admin" }, } @pytest.fixture(params=USERS.keys()) def user(request): # You can add a custom initialization here (for example, load data from DB ro something else) yield USERS[request.param] def test_user_login(user): login = user["login"] # If we want to display more data about the user, we should add it as an `excluded` Allure parameter allure.dynamic.parameter("Full name", f"{user["name"]} {user["surname"]}", excluded=True) # test logic ``` ## 4. Conclusion Allure Report natively supports parameterization in Pytest, requiring no additional configuration or extra code. This testing technique enables the execution of the same test with multiple data sets, eliminating redundancy and avoiding the complexity of manually looping through test cases. With parameterization, tests become more structured, readable, and easier to maintain. Additionally, Allure seamlessly captures and displays parameterized test runs in its reports, making it simple to analyze results and identify failures across different inputs. This improves both test coverage and debugging efficiency. --- ## Integrating videos in Allure Report with Pytest and Playwright > URL: https://allurereport.org/docs/guides/pytest-playwright-video/ # Integrating videos in Allure Report with Pytest and Playwright Reading test results should be a quick process, allowing the reader to understand failures without digging into code, running a debugger, or interrupting teammates. To achieve this, test reports must present detailed information in a concise and accessible format. While [attaching screenshots](/docs/attachments/#screenshots) and logs is standard practice, it is sometimes insufficient. In such cases, recording a video of the test execution can be invaluable. Videos improve issue reproduction, speed up debugging, and help automated test analysis. They also improve documentation accuracy by capturing intermittent bugs. In this guide, we will set up tests with Pytest, record them using Playwright Pytest, and attach videos to reports in Allure Report. ## 1. Preparation ### Prerequisites Make sure the following prerequisites are met: - [Python installed](https://www.python.org/downloads/) - [Allure Report installed](/docs/v3/install/) ### Dependency List This guide uses the following packages: - [Pytest 9.1.1](https://pypi.org/project/pytest/9.1.1/) - [allure-pytest 2.16.0](https://pypi.org/project/allure-pytest/2.16.0/) - [Pytest-playwright 0.9.0](https://pypi.org/project/pytest-playwright/0.9.0/) ### Code sample The complete source code used in this guide is available at [https://github.com/allure-examples/guide-pytest-playwright-video](https://github.com/allure-examples/guide-pytest-playwright-video). ### Setup To run the examples in this guide: 1. Install [Python](https://www.python.org/downloads/) 1. Install [Allure Report](/docs/v3/install/) 1. Download a fresh project with Pytest from [Allure Start](/start/) 1. Add [the Playwright Pytest plugin](https://playwright.dev/python/docs/intro#installing-playwright-pytest) to your project. For example, if you use `pip`: **Install Playwright Pytest:** ```shell pip install pytest-playwright ``` ## 2. Recording videos with Playwright Pytest Playwright Pytest records videos in the browser context and saves them as [WebM](https://en.wikipedia.org/wiki/WebM) files. To start recording manually, you can create a browser context with the [`new_context()`](https://playwright.dev/python/docs/api/class-browser#browser-new-context) method of the `Browser` object and pass the [`record_video_dir`](https://playwright.dev/python/docs/api/class-browser#browser-new-context-option-record-video-dir) parameter for it, for example: ```python from playwright.sync_api import sync_playwright with sync_playwright() as p: # Start a browser browser = p.chromium.launch() # Initialising a browser context with video recording context = browser.new_context(record_video_dir="path/to/videos") page = context.new_page() page.goto("https://playwright.dev") ``` ## 3. Automatic videos on failure To automatically save the video in case of a test failure, set the [`--video`](https://playwright.dev/python/docs/test-runners#cli-arguments) option to `retain-on-failure` in your project's [Pytest configuration](https://pytest.org/en/latest/reference/customize.html#command-line-options-and-configuration-file-settings). For example, if you use `pyproject.toml` as a configuration file, add the following: ```toml [tool.pytest.ini_options] ... addopts = [ ... "--video", "retain-on-failure", ] ``` For each test, Playwright Pytest creates a subdirectory in the `test-results` directory and saves the related video there. ## 4. Attaching videos to an Allure Report While capturing videos is beneficial for diagnosing test failures, viewing them separately can be inefficient. When test data is scattered across multiple locations, identifying and resolving issues becomes more time-consuming. To streamline this process, videos should be directly attached to Allure Reports, ensuring all relevant test information is consolidated in one place for efficient analysis and troubleshooting. Allure Report displays videos within the test where they were recorded, alongside error messages and other attachments, providing everything you need to analyze test results effectively. Images: /images/guides/pytest-playwright-video/pytest-playwright-video-attachment-allure3-light.png, /images/guides/pytest-playwright-video/pytest-playwright-video-attachment-allure3-dark.png, /images/guides/pytest-playwright-video/pytest-playwright-video-attachment-allure2-light.png, /images/guides/pytest-playwright-video/pytest-playwright-video-attachment-allure2-dark.png To add a local video file to Allure Report, use the [`allure.attach.file()`](/docs/pytest-reference/#reading-attachments-from-files) function: ```python import allure allure.attach.file( "file_name.webm", name="video attachment", attachment_type=allure.attachment_type.WEBM ) ``` To attach a local video automatically on every test failure, you can write a hook in the `conftest.py` file in your project's test root directory. Assuming you set the `--video` configuration option as described in [**Automatic videos on failure**](#_3-automatic-videos-on-failure) section, the hook may look up the saved videos and attach them to Allure Report. For example: ```python # Contents of conftest.py import pathlib import allure import pytest @pytest.hookimpl(hookwrapper=True) def pytest_runtest_teardown(item, nextitem): yield try: # Get the output directory for the test artifacts_dir = item.funcargs.get("output_path") if artifacts_dir: artifacts_dir_path = pathlib.Path(artifacts_dir) if artifacts_dir_path.is_dir(): for file in artifacts_dir_path.iterdir(): # Find the video file and attach it to Allure Report if file.is_file() and file.suffix == ".webm": allure.attach.file( file, name=file.name, attachment_type=allure.attachment_type.WEBM, ) except Exception as e: print(f"Error attaching video: {e}") ``` ## 5. Other attachment types Allure Report can also handle other types of attachments - images, text, tables, URI lists, or documents (XML, JSON, YAML). For example: ```python allure.attach( response_content, name="response content", attachment_type=allure.attachment_type.XML ) ``` Allure Pytest makes some attachments automatically: 1. everything passed to `sys.stdout` - e.g. `print(...)` 2. everything passed to `sys.stderr` - e.g. `print(..., file=sys.stderr)` 3. logging, e.g. `logging.warning(...)` These outputs are attached as pseudo-files to the report: Images: /images/guides/pytest-screenshots/attachment-pytest-auto-allure3-light.png, /images/guides/pytest-screenshots/attachment-pytest-auto-allure3-dark.png, /images/guides/pytest-screenshots/attachment-pytest-auto-allure2-light.png, /images/guides/pytest-screenshots/attachment-pytest-auto-allure2-dark.png If you don’t want these automatic Pytest attachments to appear in the report, use the [`--allure-no-capture`](/docs/pytest-configuration/#allure-no-capture) option. If you want to learn how attach screenshots, consult the guide for [attachments with Pytest and Playwright](/docs/guides/playwright-pytest-screenshots/) and the documentation for [attachment types](/docs/attachments/). ## 6. Conclusion With Pytest and Playwright, you can record videos of your tests to provide maximum visibility of your tests’ execution. Allure Report allows you to view those videos with other attachment types (screenshots, text, documents, tables) inside the respective test case, providing the test reader with the maximum amount of data to resolve failures. --- ## Integrating screenshots in Allure Report with Pytest and Selenium > URL: https://allurereport.org/docs/guides/pytest-selenium-screenshots/ # Integrating screenshots in Allure Report with Pytest and Selenium The process of interpreting test results should be optimized for speed and clarity. Ideally, the root cause of a failure should be immediately identifiable without requiring the reader to inspect additional code, execute a debugger, or seek assistance from other team members. To achieve this goal, test reports must convey comprehensive information in a concise format. One effective approach is the inclusion of screenshots, which provide visual context for test failures. This guide outlines the process of capturing screenshots using the pytest-selenium plugin and integrating them into test reports with Allure Report for enhanced readability and analysis. ## 1. Preparation ### Prerequisites Make sure the following prerequisites are met: - [Python installed](https://www.python.org/downloads/) - [Allure Report installed](/docs/v3/install/) ### Dependency list This guide uses the following packages: - [Pytest 9.1.1](https://pypi.org/project/pytest/9.1.1/) - [allure-pytest 2.16.0](https://pypi.org/project/allure-pytest/2.16.0/) - [pytest-selenium 4.1.0](https://pypi.org/project/pytest-selenium/4.1.0/) ### Code sample The complete source code used in this guide is available at [https://github.com/allure-examples/guide-pytest-selenium-screenshots](https://github.com/allure-examples/guide-pytest-selenium-screenshots/). ### Setup To run the examples in this guide: 1. Install [Python](https://www.python.org/downloads/) 1. Install [Allure Report](/docs/v3/install/) 1. Download a fresh project with Pytest from [Allure Start](/start/) 1. Add [the pytest-selenium plugin](https://pytest-selenium.readthedocs.io/en/latest/installing.html) to your project. For example, if you use `pip`: **Install pytest-selenium:** ```shell pip install pytest-selenium ``` ## 2. Screenshots with Selenium To take screenshots with pytest-selenium at an arbitrary time, use the methods provided by the `WebDriver` object, for example, [`save_screenshot`](https://selenium-python.readthedocs.io/api.html#selenium.webdriver.remote.webdriver.WebDriver.save_screenshot). To get the `WebDriver` object, you can use [the `selenium` fixture](https://pytest-selenium.readthedocs.io/en/latest/user_guide.html#quick-start): ```python def test_example(selenium): selenium.get("https://www.selenium.dev/") try: assert "Selenium" in selenium.title except AssertionError: selenium.save_screenshot("path/to/screenshot.png") raise ``` Whenever you call the `save_screenshot` method, a screenshot will appear at `path/to/screenshot.png`. You can also use other methods that the `WebDriver` object provides: - [`get_screenshot_as_file()`](https://selenium-python.readthedocs.io/api.html#selenium.webdriver.remote.webdriver.WebDriver.get_screenshot_as_file) - [`get_screenshot_as_png()`](https://selenium-python.readthedocs.io/api.html#selenium.webdriver.remote.webdriver.WebDriver.get_screenshot_as_png) - [`get_screenshot_as_base64()`](https://selenium-python.readthedocs.io/api.html#selenium.webdriver.remote.webdriver.WebDriver.get_screenshot_as_base64) To take a screenshot of a particular element, use the [`screenshot`](https://selenium-python.readthedocs.io/api.html#selenium.webdriver.remote.webelement.WebElement.screenshot) method of the `WebElement` object, for example: ```python element = selenium.find_element(By.NAME, "q") # or however you find the element # take the screenshot element.screenshot("path/to/element_screenshot.png") ``` You can also use these properties of `WebElement`: - [`screenshot_as_png`](https://selenium-python.readthedocs.io/api.html#selenium.webdriver.remote.webelement.WebElement.screenshot_as_png) - [`screenshot_as_base64`](https://selenium-python.readthedocs.io/api.html#selenium.webdriver.remote.webelement.WebElement.screenshot_as_base64) ## 3. Automatic screenshots on failure To take screenshots automatically of every test failure: 1. Write a hook that pytest-selenium will call to process the screenshots it takes. The hook code must be placed in the `conftest.py` file in your project's test root directory: **pytest-selenium:** ```python # Contents of conftest.py import base64 def pytest_selenium_capture_debug(item, report, extra): for log_type in extra: if log_type["name"] == "Screenshot": content = base64.b64decode(log_type["content"].encode("utf-8")) with open(f"{item.name}.png", "wb") as f: f.write(content) ``` 1. Set the `selenium_capture_debug` option to `failure` in [Pytest configuration](https://pytest.org/en/latest/reference/customize.html#command-line-options-and-configuration-file-settings). For example, if you use `pyproject.toml` as a configuration file, add the following: ```toml [tool.pytest.ini_options] ... selenium_capture_debug = "failure" ``` Now, when a test fails, a screenshot file will be saved automatically in your project. ## 4. Integrating with Allure Report While capturing screenshots is beneficial for diagnosing test failures, viewing them separately can be inefficient. When test data is dispersed across multiple locations, the process of identifying and resolving failures becomes more time-consuming. To streamline this workflow, screenshots should be directly attached to Allure Reports, ensuring all relevant test information is consolidated in a single location for efficient analysis and troubleshooting. Allure Report shows you screenshots inside the test in which they were taken. That is also where the error messages and other attachments are presented to you, so you have everything you need to analyze test results in one place: Images: /images/guides/pytest-screenshots/screenshot-in-scenario-allure3-light.png, /images/guides/pytest-screenshots/screenshot-in-scenario-allure3-dark.png, /images/guides/pytest-screenshots/screenshot-in-scenario-allure2-light.png, /images/guides/pytest-screenshots/screenshot-in-scenario-allure2-dark.png To attach screenshots to Allure, use either the [`allure.attach()`](/docs/pytest-reference/#attaching-content-from-variables) or the [`allure.attach.file()`](/docs/pytest-reference/#reading-attachments-from-files) functions. Use `allure.attach.file()` to attach an existing file: ```python import allure from pathlib import Path # Attach a screenshot of a page def attach_screenshot(driver, path): name = Path(path).name driver.get_screenshot_as_file(path) allure.attach.file( path, name=name, attachment_type=allure.attachment_type.PNG, ) # Attach a screenshot of an element def attach_element_screenshot(element, path): name = Path(path).name element.screenshot(path) allure.attach.file( path, name=name, attachment_type=allure.attachment_type.PNG, ) ``` Use `allure.attach()` if you don’t need to save the screenshot on disk: ```python import allure # Attach a screenshot of a page def attach_screenshot(driver, name): allure.attach( driver.get_screenshot_as_png(), name=name, attachment_type=allure.attachment_type.PNG, ) # Attach a screenshot of an element def attach_element_screenshot(element, name): allure.attach( element.get_screenshot_as_png(), name=name, attachment_type=allure.attachment_type.PNG, ) ``` Tip: When you need to attach a screenshot you just took, prefer using the `allure.attach` function if possible. The reason is that when you save a screenshot on disk, it may not always be immediately available for reading because of OS caching. If that happens, you may end up with an empty file attached to Allure. To automatically attach screenshots to Allure on each test failure, use a slightly modified version of the pytest-selenium hook we’ve written in the [**Automatic screenshots on failure**](#_3-automatic-screenshots-on-failure) section: **pytest-selenium:** ```python # Contents of conftest.py import allure import base64 from allure_commons.types import AttachmentType def pytest_selenium_capture_debug(item, report, extra): for log_type in extra: if log_type["name"] == "Screenshot": content = base64.b64decode(log_type["content"].encode("utf-8")) allure.attach( content, name="Screenshot on failure", attachment_type=AttachmentType.PNG, ) ``` ## 5. Other attachment types Allure Report can also handle other types of attachments - text, tables, URI lists, or documents (XML, JSON, YAML). For example: ```python allure.attach( response_content, name="response content", attachment_type=allure.attachment_type.XML ) ``` Allure Pytest makes some attachments automatically: 1. everything passed to `sys.stdout` - e.g. `print(...)` 2. everything passed to `sys.stderr` - e.g. `print(..., file=sys.stderr)` 3. logging, e.g. `logging.warning(...)` These outputs are attached as pseudo-files to the report: Images: /images/guides/pytest-screenshots/attachment-pytest-auto-allure3-light.png, /images/guides/pytest-screenshots/attachment-pytest-auto-allure3-dark.png, /images/guides/pytest-screenshots/attachment-pytest-auto-allure2-light.png, /images/guides/pytest-screenshots/attachment-pytest-auto-allure2-dark.png If you don’t want all of this to appear in the report, use the [`--allure-no-capture`](/docs/pytest-configuration/#allure-no-capture) option. Allure also allows you to [attach videos](/docs/attachments/#videos). You can consult the official documentation for more on [attachments with Pytest](/docs/pytest/#attach-screenshots-and-other-files) and [attachment types](/docs/attachments/). ## 6. Conclusion When using Pytest with Selenium, you can take screenshots at arbitrary times in your tests or instruct Selenium to take screenshots automatically on test failures. Allure Report allows viewing the screenshots and other test attachments (text, documents, tables, video) in one place, providing the reader with the maximum amount of data to resolve test failures. --- ## Deploying Self-Hosted Storage on Cloudflare Workers > URL: https://allurereport.org/docs/guides/self-hosted-storage-cloudflare/ # Deploying Self-Hosted Storage on Cloudflare Workers (Allure 3) Run [Allure Report Storage](/docs/self-hosted-storage/) as a Cloudflare Worker, backed by Cloudflare D1 (metadata) and R2 (report files). 1. [Create the Cloudflare resources](#_1-create-the-cloudflare-resources). 2. [Configure secrets](#_2-configure-secrets). 3. [Deploy](#_3-deploy). 4. [Validate it's running](#_4-validate-it-s-running). 5. [Use S3-compatible storage instead of R2](#_5-use-s3-compatible-storage-instead-of-r2) (optional). ## Requirements - [A Cloudflare account](https://dash.cloudflare.com/sign-up) - [`wrangler`](https://developers.cloudflare.com/workers/wrangler/), Cloudflare's CLI ## 1. Create the Cloudflare resources Create the D1 database and R2 bucket: ```bash corepack enable yarn dlx wrangler d1 create allure-report-storage yarn dlx wrangler r2 bucket create allure-report-storage ``` Update `wrangler.toml` with the `database_id` printed by the first command. The R2 bucket name must match `bucket_name` in `wrangler.toml` (`allure-report-storage` by default) or be updated there. ## 2. Configure secrets ```bash yarn dlx wrangler secret put ACCESS_TOKEN yarn dlx wrangler secret put SECRET ``` Both are plain strings you choose yourself when prompted, not a generated or fixed-format value; pick anything long and unguessable. Their use is narrow: `ACCESS_TOKEN` only authorizes minting a report-access token (`POST /api/token`) and setting a project's main branch, nothing else; `SECRET` is never sent anywhere, it just signs the report-access tokens issued from those `ACCESS_TOKEN` requests. Neither is what you give to Allure 3 directly - that's a separate, freshly minted `ars1..` token, covered in [Connecting Allure 3 to the storage service](/docs/self-hosted-storage/#connecting-allure-3-to-the-storage-service). If the public URL of the Worker differs from the URL that requests arrive at, for example behind TLS termination, set `PUBLIC_URL` under `[vars]` in `wrangler.toml` before minting tokens: ```toml [vars] PUBLIC_URL = "https://reports.example.com" ``` See [Running behind a reverse proxy](/docs/self-hosted-storage/#connecting-allure-3-to-the-storage-service) for details. ## 3. Deploy ```bash corepack enable yarn dlx wrangler deploy ``` Or run it locally first, entirely on your own machine. `wrangler dev` emulates D1 and R2 locally, with no real Cloudflare resources touched: ```bash yarn dlx wrangler dev ``` ## 4. Validate it's running ```bash curl https:///api/ping ``` A `{"pong":true,...}` response confirms the service is up. From here, [connect Allure 3 to the storage service](/docs/self-hosted-storage/#connecting-allure-3-to-the-storage-service). ## 5. Use S3-compatible storage instead of R2 Set `STORAGE_BACKEND=s3` (as a var or Worker secret, same as `ACCESS_TOKEN`/`SECRET`) to store report files in an S3-compatible bucket instead of the native R2 binding. | Variable | Required | Purpose | | ---------------------- | -------- | ------------------------------------------------------------------------------ | | `S3_BUCKET` | Yes | Bucket name | | `S3_REGION` | Yes | Bucket region (`auto` for Cloudflare R2) | | `S3_ENDPOINT` | Yes | S3-compatible endpoint URL | | `S3_ACCESS_KEY_ID` | No\* | Access key | | `S3_SECRET_ACCESS_KEY` | No\* | Secret key | | `S3_FORCE_PATH_STYLE` | Yes | `true` for most self-hosted S3-compatible servers (e.g. MinIO), `false` for R2 | | `S3_SESSION_TOKEN` | No | Session token, for temporary credentials | | `S3_PREFIX` | No | Prefix applied to every object key | | `S3_REPORTS_PREFIX` | No | Prefix for report files (default `files`) | | `S3_ASSETS_PREFIX` | No | Prefix for shared assets (default `assets`) | \* If omitted, the AWS SDK's default credential provider chain is used instead. ```bash yarn dlx wrangler secret put STORAGE_BACKEND yarn dlx wrangler secret put S3_BUCKET yarn dlx wrangler secret put S3_REGION yarn dlx wrangler secret put S3_ENDPOINT yarn dlx wrangler secret put S3_ACCESS_KEY_ID yarn dlx wrangler secret put S3_SECRET_ACCESS_KEY yarn dlx wrangler secret put S3_FORCE_PATH_STYLE ``` Cloudflare R2 example values: ```bash STORAGE_BACKEND=s3 S3_BUCKET=allure-report-storage S3_REGION=auto S3_ENDPOINT=https://.r2.cloudflarestorage.com S3_ACCESS_KEY_ID= S3_SECRET_ACCESS_KEY= S3_FORCE_PATH_STYLE=false ``` Bucket-specific R2 endpoints (`..r2.cloudflarestorage.com`) are normalized internally to the account-level form above. See [Self-hosted storage](/docs/self-hosted-storage/) for how to use it: publishing, branches, history, and deleting reports. --- ## Deploying Self-Hosted Storage with Docker > URL: https://allurereport.org/docs/guides/self-hosted-storage-docker/ # Deploying Self-Hosted Storage with Docker (Allure 3) Run [Allure Report Storage](/docs/self-hosted-storage/) in a container on any machine that can run Docker. 1. [Start the service](#_1-start-the-service). 2. [Validate it's running](#_2-validate-it-s-running). 3. [Use S3-compatible storage instead of the filesystem](#_3-use-s3-compatible-storage-instead-of-the-filesystem) (optional). ## Requirements - [Docker](https://docs.docker.com/get-docker/) - [Docker Compose](https://docs.docker.com/compose/install/) ## 1. Start the service Create a `compose.yaml` file using the published [`allure/allure-report-storage`](https://hub.docker.com/r/allure/allure-report-storage) image - no build step required. Check the [tags page](https://hub.docker.com/r/allure/allure-report-storage/tags) for the current version: ```yaml services: app: image: allure/allure-report-storage:v1.1.0 environment: ACCESS_TOKEN: ${ACCESS_TOKEN:-change-me} DATABASE_PATH: /data/reports.sqlite DATA_DIR: /data HOST: 0.0.0.0 MAIN_BRANCH: ${MAIN_BRANCH:-main} PORT: 3000 PUBLIC_URL: ${PUBLIC_URL:-} SECRET: ${SECRET:-change-me-secret} ports: - "3000:3000" restart: unless-stopped volumes: - report-data:/data volumes: report-data: ``` ```bash docker compose up -d ``` By default this stores report files on local disk and metadata in a bundled SQLite database, with no external database or bucket needed. Two environment variables matter: | Variable | Purpose | | -------------- | ------------------------------------------------------------------------------------ | | `ACCESS_TOKEN` | Bootstrap bearer token - mints report-access tokens and sets a project's main branch | | `SECRET` | Signing secret for the report-access tokens the service issues | Both are plain strings you choose yourself, not a generated or fixed-format value; pick anything long and unguessable. Their use is narrow: `ACCESS_TOKEN` only authorizes minting a report-access token (`POST /api/token`) and setting a project's main branch, nothing else; `SECRET` is never sent anywhere, it just signs the report-access tokens issued from those `ACCESS_TOKEN` requests. Neither is what you give to Allure 3 directly - that's a separate, freshly minted `ars1..` token, covered in [Connecting Allure 3 to the storage service](/docs/self-hosted-storage/#connecting-allure-3-to-the-storage-service). Tip: `compose.yaml` ships placeholder defaults (`change-me` / `change-me-secret`) so the service runs out of the box for a quick test. Replace both with unique, unguessable values for anything beyond that. If the service runs behind a reverse proxy or TLS termination, also set `PUBLIC_URL` to the public origin of the service, for example `PUBLIC_URL=https://reports.example.com` in the `.env` file next to `compose.yaml`, before minting tokens. See [Running behind a reverse proxy](/docs/self-hosted-storage/#connecting-allure-3-to-the-storage-service) for details. ## 2. Validate it's running ```bash curl http://localhost:3000/api/ping ``` A `{"pong":true,...}` response confirms the service is up. From here, [connect Allure 3 to the storage service](/docs/self-hosted-storage/#connecting-allure-3-to-the-storage-service). ## 3. Use S3-compatible storage instead of the filesystem Set `STORAGE_BACKEND=s3` to store report files in an S3-compatible bucket instead of on local disk. Metadata stays in SQLite either way. | Variable | Required | Purpose | | ---------------------- | -------- | ------------------------------------------------------------------------------ | | `S3_BUCKET` | Yes | Bucket name | | `S3_REGION` | Yes | Bucket region (`auto` for Cloudflare R2) | | `S3_ENDPOINT` | Yes | S3-compatible endpoint URL | | `S3_ACCESS_KEY_ID` | No\* | Access key | | `S3_SECRET_ACCESS_KEY` | No\* | Secret key | | `S3_FORCE_PATH_STYLE` | Yes | `true` for most self-hosted S3-compatible servers (e.g. MinIO), `false` for R2 | | `S3_SESSION_TOKEN` | No | Session token, for temporary credentials | | `S3_PREFIX` | No | Prefix applied to every object key | | `S3_REPORTS_PREFIX` | No | Prefix for report files (default `files`) | | `S3_ASSETS_PREFIX` | No | Prefix for shared assets (default `assets`) | \* If omitted, the AWS SDK's default credential provider chain is used instead. Cloudflare R2 example: ```bash STORAGE_BACKEND=s3 S3_BUCKET=allure-report-storage S3_REGION=auto S3_ENDPOINT=https://.r2.cloudflarestorage.com S3_ACCESS_KEY_ID= S3_SECRET_ACCESS_KEY= S3_FORCE_PATH_STYLE=false ``` Bucket-specific R2 endpoints (`..r2.cloudflarestorage.com`) are normalized internally to the account-level form above. See [Self-hosted storage](/docs/self-hosted-storage/) for how to use it: publishing, branches, history, and deleting reports. --- ## Allure Report 3 - XCResults Reader > URL: https://allurereport.org/docs/guides/xcresults-reader/ # Allure Report 3: XCResults Reader (Allure 3) Allure Report 3 introduces native support for Apple Xcode test results. You can now generate fully functional Allure reports directly from your XCTest and/or Swift Testing executions without additional conversion tools. This article is both an overview of features and a basic integration guide for your Xcode projects. Info: Note Allure 2 requires you to use the separate [xcresults CLI tool](https://github.com/eroshenkoam/xcresults) to export Xcode test results into a compatible format. The vast majority of its features are present in Allure 3, and many more advanced ones were added on top. Read on to find out the details. ## Getting Started ### Prerequisites - [Xcode IDE](https://developer.apple.com/xcode/) - [Xcode-generated test results](https://developer.apple.com/documentation/xcode/running-tests-and-interpreting-results) - [Allure 3](https://github.com/allure-framework/allure3) Tip: The full Xcode IDE is required for proper report generation. Xcode Command Line Tools Package is not sufficient. Moreover, if you install the full Xcode IDE on top of the CLI, make sure to set it as the active development environment via [xcode-select --switch](https://developer.apple.com/library/archive/technotes/tn2339/_index.html#//apple_ref/doc/uid/DTS40014588-CH1-SOURCECODE4) ### Sample Project You can explore various examples of Xcode tests and see how Allure displays the results in its reports here: https://github.com/allure-examples/allure3-xcresult ### Generate Report Allure Report can read Xcode test results and generate reports from them - simply use Swift Testing’s or XCTest’s standard features and Allure will automatically extract the structure, outcomes, and metadata from your test results. You can generate a basic Allure report from your .xcresult bundle directly, no additional changes to the code required: `npx allure generate -o ` And then open it like any other Allure report: `npx allure open ` Below are some examples of Xcode tests and how they will be displayed in Allure Report. **Swift Testing:** ```swift import Testing struct MathTests { @Test func addition() async throws { let result = 2 + 3 #expect(result == 5) } @Test func incorrectMultiplication() async throws { let result = 3 * 4 #expect(result == 11) // This will fail } } ``` **XCTest:** ```swift import XCTest final class MathTests: XCTestCase { func testAddition() throws { let result = 2 + 3 XCTAssertEqual(result, 5) } func testIncorrectMultiplication() throws { let result = 3 * 4 XCTAssertEqual(result, 11) // This will fail } } ``` Images: /images/guides/xcresults/xcresults-02-light.png, /images/guides/xcresults/xcresults-02-dark.png # Supported Features While any existing Xcode test results can be turned into an Allure Report, and most metadata will be picked up automatically, you might want to organize and label your tests a certain way to get the most out of Allure. Explore this section to learn how various features work, and how they are reflected in the resulting report. (Swift Testing) and (XCTest) badges will be used below to mark which framework(s) a particular feature applies to. ## 1. Setting Custom Names for Tests (Swift Testing) (XCTest) Unless otherwise specified, Allure uses test function names as names. You can also give a custom name to your test. In Swift Testing, you accomplish this using the `@Test` annotation, while XCTest employs a naming convention for activities that lets you add various metadata to tests, including custom names. We'll discuss the naming convention API in more detail [below](#_3-3-allure-api-metadata-naming-convention). **Swift Testing:** ```swift @Test("This name was assigned to the test via the @Test annotation") func renamedTest() async throws { } ``` **XCTest:** ```swift final class AllureApiActivities: XCTestCase { func testDisplayName() throws { XCTContext.runActivity(named: "allure.name:This name was set by an activity", block: { _ in }) } } ``` ## 2. Working With Outcomes ### Failed Assertions (Swift Testing) (XCTest) When a test result includes one or more failed assertions, it is marked as failed. The failures’ count and the details of the failures are listed on the result page. Images: /images/guides/xcresults/xcresults-15-light.png, /images/guides/xcresults/xcresults-15-dark.png Refer to [this example](https://allure-examples.github.io/allure3-xcresult/main/#f24e5564a18e8e57ddeda434b22069b5) to see how Allure handles multiple failed assertions specifically. ### Uncaught Exceptions (Swift Testing) (XCTest) When a test throws an uncaught exception, test frameworks mark it as failed and Allure captures the exception details in the failure report. Such tests will be labeled **Broken** in Allure report. Images: /images/guides/xcresults/xcresults-05-light.png, /images/guides/xcresults/xcresults-05-dark.png ### Skipping Tests (Swift Testing) (XCTest) Swift Testing provides native test skipping through the `.disabled()` trait. You can optionally include a description explaining why the test is disabled. These appear as **Skipped** in Allure report with the reason displayed. XCTest supports skipping via the `XCTSkip()` error. **Swift Testing:** ```swift @Test(.disabled("This description was provided via the disabled trait")) func skippedTestWithDescription() async throws { } ``` **XCTest:** ```swift func testSkippedTestWithDescription() throws { throw XCTSkip("This description was provided via XCTSkip") } ``` Images: /images/guides/xcresults/xcresults-16-light.png, /images/guides/xcresults/xcresults-16-dark.png ### Expected Failures (XCTest) XCTest's `XCTExpectFailure()` allows you to mark tests containing asserts that are known to fail. When an expected failure occurs, the test passes. If the test unexpectedly passes or fails in a different way, it's marked as failed. Allure distinguishes between expected and unexpected failures in the report. ```swift func testOneExpectedFailureWithMessage() throws { XCTExpectFailure("This message was set via XCTExpectFailure") XCTAssertEqual("foo", "bar") } func testUnmetExpectedFailure() throws { XCTExpectFailure() XCTAssertEqual("foo", "foo") // This passes, but we expected it to fail } ``` In the example above `testOneExpectedFailureWithMessage` will be marked as **passed** in Allure, as we expected the assertion inside it to fail, while `testUnmetExpectedFailure` will be displayed as failed - as we expected the assertion to fail, but it passed instead. `XCTExpectFailure()` works in a logical OR manner: if at least one assertion after this function fails, the whole test passes. Otherwise, the test fails. **Example**: code [sample](https://github.com/allure-examples/allure3-xcresult/blob/main/XcresultSamplesXCTests/ExpectedFailures.swift) and the [report generated based on it](https://allure-examples.github.io/allure3-xcresult/main/). ## 3. Test Grouping and Labeling Allure 3 can be configured to group test results into a convenient navigation tree by a wide variety of labels. This section discusses how these labels are mapped and processed by the tool. ### 3.1. Suites and Suite Hierarchy (Swift Testing) (XCTest) Allure has three levels of suites: - parentSuite - suite - subSuite The exact mapping of these levels depends on the test framework. #### Swift Testing **Suite hierarchy** is mapped to Allure's navigation tree, creating a nested organization in the Allure report matching your code structure: - The **module name** always becomes the **parentSuite** in Allure - The top-level Swift Testing suite becomes the **suite** in Allure - nested suites become **subSuites** (if there are more than two nested suites, they are joined with > and the result is assigned to **subSuite**) - **Unannotated suites** use struct names (`UnannotatedSuite` → `UnannotatedSubSuite`) - **Named suites** use custom names (`Named suite` → `Named sub-suite`) - **testClass** and **testMethod** labels always hold the original names of the class/struct and the method. ```swift // Unannotated suite - uses struct name struct UnannotatedSuite { struct UnannotatedSubSuite { struct UnannotatedSubSubSuite { @Test("testInSubSubSuite (defined in Suites.swift)") func testInSubSubSuite(){ } } @Test("testInSubSuite (defined in Suites.swift)") func testInSubSuite(){ } } @Test("testInTopLevelSuite (defined in Suites.swift)") func testInTopLevelSuite(){ } } // Named suite with @Suite annotation @Suite("Named suite") struct NamedSuites { @Suite("Named sub-suite") struct NamedSubSuite { @Test("testInNamedSubSuite (defined in Suites.swift)") func testInNamedSubSuite(){ } } @Test("testInNamedTopLevelSuite (defined in Suites.swift)") func testInNamedTopLevelSuite(){ } } ``` Images: /images/guides/xcresults/xcresults-06-light.png, /images/guides/xcresults/xcresults-06-dark.png #### XCTest For XCTest, Allure suites are based on the class structure of the test files: - The **module name** always becomes the **parent suite** in Allure - Each **class** becomes a **suite** in Allure - **testClass** label shows the class name - **suite** label typically matches the class name ```swift final class Activities: XCTestCase { func testOnePassedActivity() throws { XCTContext.runActivity(named: "Step 1", block: { _ in }) } func testOneActivityWithOneFailedAssertion() throws { XCTContext.runActivity(named: "Step 1") { _ in XCTAssertEqual("foo", "bar") } } ``` ### 3.2. Packages (Swift Testing) (XCTest) Allure automatically creates package labels using the format `.`: **Examples from the sample project:** - [`XcresultSamples.XcresultSamplesSwiftTests`](https://allure-examples.github.io/allure3-xcresult/main/#a07256b6c04ead7cfdd484ca63460056) (Swift Testing tests) These appear as `package` labels in Allure reports. ### 3.3. Activities and Steps (XCTest) XCTest supports organizing tests into activities within a single test function, which Allure then translates into [steps](/docs/steps/). Each activity becomes a step (or substep) in the test execution flow. ```swift func testSixNestedSteps() throws { XCTContext.runActivity(named: "Step 1") { _ in XCTContext.runActivity(named: "Step 1.1") { _ in } XCTContext.runActivity(named: "Step 1.2") { _ in } } XCTContext.runActivity(named: "Step 2") { _ in XCTContext.runActivity(named: "Step 2.1") { _ in } XCTContext.runActivity(named: "Step 2.2") { _ in } } } ``` Images: /images/guides/xcresults/xcresults-07-light.png, /images/guides/xcresults/xcresults-07-dark.png Tip: Allure has an API that allows to add native Allure metadata (which XCTest doesn’t support) to your tests by naming activities according to a certain convention, which we discuss in the section below. Allure doesn’t translate activities named according to the API convention into steps. ### 3.4. Allure API: Metadata Naming Convention (XCTest) XCTest activity names additionally serve as a vehicle to add native Allure metadata and labels to your tests. The `allure.*` prefixed activity names become rich metadata in the report: - `allure.name:` sets custom test names - `allure.label.*:` adds Allure labels - `allure.parameter.*:` adds parameter information with masking options - `allure.description.:` adds a description - `allure.link.*:` adds a link (there are convenient shorthands for adding links to bugs, which we discuss [below](#_5-issue-tracking)) - `allure.flaky` marks a test as [flaky](/docs/test-stability/#flaky-tests). ```swift func testDisplayName() throws { XCTContext.runActivity(named: "allure.name:This name was set by an activity", block: { _ in }) } func testLabel() throws { XCTContext.runActivity(named: "allure.label.owner:John Doe", block: { _ in }) } func testParameters() throws { XCTContext.runActivity(named: "allure.parameter.Regular parameter:Value 1", block: { _ in }) XCTContext.runActivity(named: "allure.parameter.Masked parameter[masked]:Value 2", block: { _ in }) } ``` Tip: Allure doesn’t translate such activities into test steps, so you shouldn’t put any actual test code in them. Make also sure that activities used to insert Allure metadata into tests are not nested, otherwise they won’t be properly recognized. This is a change from Allure 2’s xcresultstool CLI tool behavior, that parses even nested activities for metadata. **Example**: code [sample](https://github.com/allure-examples/allure3-xcresult/blob/main/XcresultSamplesXCTests/AllureApiActivities.swift) and [report](https://allure-examples.github.io/allure3-xcresult/main/) ### 3.5. Tags (Swift Testing) Allure 3 supports Swift Testing tags. Currently they are displayed among the labels on the test result page, and in future releases it will be possible to filter by them. ```swift extension Tag { @Tag static var fromSuite: Self @Tag static var fromSubSuite: Self @Tag static var fromTest: Self } @Suite(.tags(.fromSuite)) struct Tags { @Suite(.tags(.fromSubSuite)) struct SubSuite { @Test(.tags(.fromTest)) func hasTagsFromBothSuitesAndTest() { } @Test func hasTagsFromSuitesOnly() { } } @Test(.tags(.fromTest)) func hasTagsFromOneSuiteAndTest() { } } ``` Example: code [sample](https://github.com/allure-examples/allure3-xcresult/blob/3b79ff76b643ab36188aa38afc1d19eaffe146a7/XcresultSamplesSwiftTests/Tags.swift#L12) and [report](https://allure-examples.github.io/allure3-xcresult/main/#3483a7ad2fdf38ffa2ecf4f586a96a1d). ### 3.6. Test Repetitions (Swift Testing) (XCTest) Allure supports repeated test executions, if the source bundle contains them. You can find repetition results on the **Retries** tab of a test result page. Tip: ‘Retries’ is the term that Allure uses for repeated test executions. It has the same meaning as ‘repetitions’ in Xcode. Images: /images/guides/xcresults/xcresults-08-light.png, /images/guides/xcresults/xcresults-08-dark.png ### 3.7. Destinations (Devices) and Multiple Test Plans (Swift Testing) (XCTest) Allure automatically labels all Xcode tests with the destination (the physical or emulated device the tests run on), as well as with Test plan names, if such data exists in the source bundle. If your result bundles contain result data for multiple destinations or test plans, you can generate an Allure report with all records, provided you merge the bundles from different destinations and/or test plans into a single bundle with the `xcresulttool merge` command first and then use the merged bundle to generate your Allure report. Images: /images/guides/xcresults/xcresults-09-light.png, /images/guides/xcresults/xcresults-09-dark.png If the source bundle contains test results for more than one destination, Allure displays the Device Details parameter in addition to the Device parameter. Similarly, if the bundle contains test results for more than one test plan, Allure displays the Test Plan parameter. ## 4. Parameterized Tests (Swift Testing) Parameterized testing is a native feature of Swift Testing that allows running the same test with different input values. XCTest does not have built-in parameterized test support. ```swift struct Parameters { @Test(arguments: ["foo", ""], [1, 2]) func twoParametersOneFail(Text text: String, Count count: Int) async throws { #expect(!text.isEmpty) // Fails for ("", 1) and ("", 2) #expect(count > 0) } } ``` In Allure report each combination of parameters becomes a separate test result grouped under a single Suite. For each test result, parameter values are listed in the report. Images: /images/guides/xcresults/xcresults-14-light.png, /images/guides/xcresults/xcresults-14-dark.png **Example**: code [sample](https://github.com/allure-examples/allure3-xcresult/blob/main/XcresultSamplesSwiftTests/Parameters.swift) and [report](https://allure-examples.github.io/allure3-xcresult/main/). ## 5. Issue Tracking (Swift Testing) (XCTest) Issue tracking is a native feature of Swift Testing that allows linking tests to known issues. XCTest does not have built-in issue tracking support, but just like with custom names, the [Activity naming convention](#_3-4-allure-api-metadata-naming-convention) can be used to add links to issues. In Allure, if this metadata is given, it is transformed into clickable issue links. **Swift Testing:** ```swift @Test(.bug("https://github.com/allure-framework/allure3/issues/2", id: "2", "Issue #2")) func bugUrlNameAndId() async throws { } ``` **XCTest:** ```swift func testIssueLink() throws { XCTContext.runActivity(named: "allure.issue.Issue 2:https://github.com/allure-framework/allure3/issues/2", block: { _ in }) } ``` Images: /images/guides/xcresults/xcresults-11-light.png, /images/guides/xcresults/xcresults-11-dark.png ## 6. Attachments (XCTest) Attachments allow you to include additional data (text, screenshots, logs) with your test results for debugging and analysis. Swift Testing will support basic attachments in Swift 6.2, but currently only XCTest provides full attachment capabilities. ```swift func testTextAttachment() throws { XCTContext.runActivity(named: "allure.description:The attachment is visible in the 'Multiple destinations' report only", block: { _ in }) let attachment = XCTAttachment(string: "Lorem Ipsum") attachment.name = "A text attachment" self.add(attachment) } ``` Text attachments can include logs, debug information, or any string data. You can optionally provide a name for better organization. Unnamed attachments get default names in Allure reports. Images: /images/guides/xcresults/xcresults-17-light.png, /images/guides/xcresults/xcresults-17-dark.png Unlike Allure 2, in Allure 3 attachments are displayed chronologically, in the order they were created. Depending on type, attachments are handled differently. Images, for example, will be rendered within Allure report itself and can be conveniently downloaded. **Example**: code [sample](https://github.com/allure-examples/allure3-xcresult/blob/main/XcresultSamplesXCTests/Attachments.swift) and [report](https://allure-examples.github.io/allure3-xcresult/main/). ## 7. UI Testing (XCTest) UI testing is exclusively available through XCTest and provides capabilities for automating user interface interactions and capturing visual evidence. Allure recognizes image and video attachments generated by the XCTest UI testing and stores them in the relevant test steps. Images: /images/guides/xcresults/xcresults-13-light.png, /images/guides/xcresults/xcresults-13-dark.png **Example**: code [samples](https://github.com/allure-examples/allure3-xcresult/tree/main/XcresultSamplesUITests) and reports ([screenshots](https://allure-examples.github.io/allure3-xcresult/ui-screenshots) and [recordings](https://allure-examples.github.io/allure3-xcresult/ui-recording)). ## 8. Build Errors Allure processes build failures as well. If the source bundle contains a single run resulting in a build failure, it will be the only result in the Allure report. If the bundle has multiple runs, some with build failures and others with test results, build failures will be displayed alongside the test outcomes. Images: /images/guides/xcresults/xcresults-10-light.png, /images/guides/xcresults/xcresults-10-dark.png ## Links - [Samples project](https://github.com/allure-examples/allure3-xcresult) and [reports](https://allure-examples.github.io/allure3-xcresult/main) - Old [xcresults CLI tool](https://github.com/eroshenkoam/xcresults) --- # Section: How it works ## How it works > URL: https://allurereport.org/docs/how-it-works/ # How it works Allure Report consists of a framework integration and the `allure` command-line utility. After a simple initial configuration, creating a test report for your project takes no more than three steps. ![How Allure Report works, Allure Results, additional data, history data, serve vs generate](/images/diagrams/how-it-works.png "How Allure Report works, Allure Results, additional data, history data, serve vs generate") ## 1. Data Collection During Test Execution When you run tests normally - whether from command line, IDE, or CI pipeline - Allure captures detailed execution data through framework-specific integrations. These integrations hook into your testing [framework's lifecycle](/docs/frameworks/) naturally. The integration saves structured data files to a results directory (`allure-results` by default), building a complete record of test execution including steps, timings, attachments, and metadata Tip: Multiple test runs can accumulate data into the same results directory, making it easy to aggregate results from different environments, configurations, or test suites into a single **launch** that will become a report. ## 2. Metadata Enrichment Optionally, you can enrich your results with additional context through supplementary files, such as: - **Environment data** to capture global test conditions and configurations - **CI integration details** to link reports back to specific builds or jobs - **Historical data** to enable trend analysis and stability tracking - **Custom categorization rules** to automatically classify and group test failures ## 3. Report Generation Allure's report generator processes the collected data, analyzes test results, applies categorization rules, calculates trends, and builds an interactive HTML report. Static generation (`allure generate`) creates a portable HTML report that you can archive, share, or host anywhere, while local preview (`allure serve` in Allure 2 or `allure open` in Allure 3) sets up a local server and displays the report in your browser. The generator handles data aggregation across multiple test runs, automatically detects retries, and creates visual analytics. To share the report, you can use the following approaches. - You may download and archive your report, then send it via file hosting, email, Slack, or other available method. - If you have integrated the report with a CI tool, you may share a link to the job results page that embeds the report. - You may host the report on a web server and share the link to it. - In the following cases, we recommend considering [Allure TestOps](https://qameta.io/): - If the report needs to be available 24/7 in a centralized and secure way, - If the report needs to be provided to non-technical team members or management who do not have access to the CI tool, - If the archive is too large and difficult to send. --- ## Categories file > URL: https://allurereport.org/docs/how-it-works-categories-file/ # Categories file Warning: This page describes a data format used by Allure and its integrations for test frameworks. Depending on the test framework you use, this file may or may not be generated automatically based on the project configuration. Check [your integration's documentation](/docs/frameworks/). The `categories.json` file in the test results directory, if present, defines [custom categories](/docs/categories/#custom-categories). The data is stored as a JSON array of objects. Each object defines one custom category and can have the properties described below. The order of objects in the array is the order in which Allure will try to match a test result against the categories. Example: ```json [ { "name": "Ignored tests", "messageRegex": ".*ignored.*", "matchedStatuses": ["skipped"] }, { "name": "Infrastructure problems", "messageRegex": ".*RuntimeException.*", "matchedStatuses": ["broken"] } ] ``` ## name (string) The name of the category. ## messageRegex (string) The regular expression that the test result's message should match. ## traceRegex (string) The regular expression that the test result's trace should match. ## matchedStatuses (array) An array of statuses that the test result should be one of. Possible values are: “failed”, “broken”, “passed”, “skipped”, “unknown”. See [Test statuses](/docs/test-statuses/). ## flaky (boolean) Whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). --- ## Container file > URL: https://allurereport.org/docs/how-it-works-container-file/ # Container file Warning: This page describes a data format used by Allure and its integrations for test frameworks. Understanding this format is **not necessary** for using Allure in your project. For a more high-level description of the process of building a report, see [How it works](/docs/how-it-works/). A container file describes a set of initialization and finalization steps related to certain tests. It is used by the test frameworks that implement the concept of **test fixtures**, with one container normally describing one fixture. The container may refer to multiple [test results](/docs/how-it-works-test-result-file/) via the [children](#children-array) property, thus adding the initialization and finalization steps to each of them in the test report. The data is stored as a JSON object. Each file is named `{uuid}-container.json`, where `{uuid}` is a randomly generated identifier. This article describes the properties that can be found in the JSON object. Example: ```json { "uuid": "987b1a5d-fbab-4707-acff-ddfd41f9e511", "start": 1682502818632, "stop": 1682502818712, "children": [ "f82da25c-3565-4eb4-a1db-4a502fe35eea", "fc9e5cee-f4af-475c-94f2-5902ee7c9d2d", "e3592f29-c828-4816-8d3d-3759c6663b27" ], "befores": [ { "name": "Create temporary directory", "status": "passed", "start": 1682502818632, "stop": 1682502818632 }, { "name": "Create temporary files", "status": "passed", "start": 1682502818632, "stop": 1682502818632 } ], "afters": [ { "name": "Remove temporary files", "status": "passed", "start": 1682502818712, "stop": 1682502818712 }, { "name": "Remove temporary directory", "status": "passed", "start": 1682502818712, "stop": 1682502818712 } ] } ``` ### uuid (string) A unique identifier of the test result. ### start (integer) The time when the execution of the initialization steps started, in the [UNIX timestamp](https://en.wikipedia.org/wiki/Unix_time) format. Normally, it is equal or close to the earliest `start` time from the [`befores`](#befores-array). ### stop (integer) The time when the execution of the finalization steps finished, in the [UNIX timestamp](https://en.wikipedia.org/wiki/Unix_time) format. Normally, it is equal or close to the latest `stop` time from the [`afters`](#afters-array). ### children (array) The [`uuid`](/docs/how-it-works-test-result-file/#uuid-string) values of the test results for which this fixture was added. ### befores (array) An array of initialization steps that were executed before the tests. Note that the test report does not necessarily display these steps in the same order as defined in the file. Objects in the array have the following properties, each having the same format as the corresponding property of a test result: - Metadata - [`name`](/docs/how-it-works-test-result-file/#name-string) - [`parameters`](/docs/how-it-works-test-result-file/#parameters-array) - [`attachments`](/docs/how-it-works-test-result-file/#attachments-array) - Execution - [`status`](/docs/how-it-works-test-result-file/#status-string) - [`statusDetails`](/docs/how-it-works-test-result-file/#statusdetails-object) - [`stage`](/docs/how-it-works-test-result-file/#stage-string) - [`start`](/docs/how-it-works-test-result-file/#start-integer) - [`stop`](/docs/how-it-works-test-result-file/#stop-integer) - [`steps`](/docs/how-it-works-test-result-file/#steps-array) ### afters (array) An array of finalization steps that were executed after the tests. Note that the test report does not necessarily display these steps in the same order as defined in the file. Objects in the array have the following properties, each having the same format as the corresponding property of a test result: - Metadata - [`name`](/docs/how-it-works-test-result-file/#name-string) - [`parameters`](/docs/how-it-works-test-result-file/#parameters-array) - [`attachments`](/docs/how-it-works-test-result-file/#attachments-array) - Execution - [`status`](/docs/how-it-works-test-result-file/#status-string) - [`statusDetails`](/docs/how-it-works-test-result-file/#statusdetails-object) - [`stage`](/docs/how-it-works-test-result-file/#stage-string) - [`start`](/docs/how-it-works-test-result-file/#start-integer) - [`stop`](/docs/how-it-works-test-result-file/#stop-integer) - [`steps`](/docs/how-it-works-test-result-file/#steps-array) --- ## Environment file > URL: https://allurereport.org/docs/how-it-works-environment-file/ # Environment file Warning: This page describes a data format used by Allure and its integrations for test frameworks. Depending on the test framework you use, this file may or may not be generated automatically based on the project configuration. Check [your integration's documentation](/docs/frameworks/). The `environment.properties` file in the test results directory, if present, defines a set of key-value pairs to display in the test report. See [Environment information](/docs/v3/readability/#environment-information). The data is stored in the [`.properties`](https://en.wikipedia.org/wiki/.properties) format. Example: ```properties os_platform = linux os_release = 5.15.0-60-generic os_version = #66-Ubuntu SMP Fri Jan 20 14:29:49 UTC 2023 python_version = Python 3.10.9 ``` Note that this feature should be used for properties that do not change for the whole test launch. If you have properties that can be different for different tests, consider using [Parametrized tests](/docs/v3/readability/#parametrized-tests). ## New Environments feature in Allure Report 3 In addition to environment files, Allure Report 3 comes with a much more powerful Environments feature that lets you store more information on environments and create multiple-environment reports. Learn how it works and how to set it up [here](/docs/environments/). --- ## Executor file > URL: https://allurereport.org/docs/how-it-works-executor-file/ # Executor file Warning: This page describes a data format used by Allure and its integrations for test frameworks. Understanding this format is **not necessary** for using Allure in your project. For a more high-level description of the process of building a report, see [How it works](/docs/how-it-works/). The `executor.json` file in the test results directory, if present, should contain information about the CI build that produced the test report. It is automatically generated by various CI integration plugins. Example: ```json { "reportName": "Build #1234", "buildOrder": 1234, "reportUrl": "http://example.org/build#1234/AllureReport", "name": "Jenkins", "type": "jenkins", "buildName": "allure-report_deploy#1234", "buildUrl": "http://example.org/build#1234" } ``` ## Report properties - `reportName` (string) — the name to be shown on top of the Overview tab. When not set, it defaults to “Allure Report”. ## History-related properties These fields will be copied into the report's [history files](/docs/how-it-works-history-files/) so that future reports can link back to the current one. - `buildOrder` (integer) — the identifier for this build's columns on future [trend charts](/docs/visual-analytics/#trend-charts). - `reportUrl` (string) — the URL at which this test report will be located. ## Build properties These fields affect the “Executor” block in the test report. The purpose of the block is to show what CI build produced the current report, with an optional link to that CI build. - `name` (string) — the name of the CI server or plugin. - `type` (string) — the type of the CI tool. This affects the icon next to the name. Allowed values are: “jenkins”, “bamboo”, “teamcity”, “gitlab”, “github”, “circleci”, “bitbucket”, “azure”. - `buildName` (string) — the name of the CI build that produced the current report. Normally, a CI integration plugin includes a unique build identifier into this name. - `buildUrl` (string) — the URL of the CI build. --- ## Allure Report Glossary > URL: https://allurereport.org/docs/how-it-works-glossary/ # Allure Report Glossary ## Test Case A test case is a step-by-step instruction that defines how some functionality is tested. - A test case is usually represented by a function. - It can be parametrized. The following example defines two test cases: - `test_authentication` is parametrized - `test_signin_available` is parameterless ```python @pytest.mark.parametrize(["login", "role], [ ("John", "user"), ("Jane", "admin"), ]) def test_authentication(login, role): """Checks if a user with a specific role can sign in.""" # ... def test_signin_available(): """Checks a sign page can be accessed.""" # ... ``` ## Test A test is a specific sequence of actions that can run independently and produces a single test result. - A test case with no parameters produces exactly one test. - A parametrized test case produces one test per parameter combination. Our example defines three tests. Two are part of the `test_authentication` test case. The remaining one is the single test in the `test_signin_available` test case: - `test_authentication` (login is "John", role is "user") - `test_authentication` (login is "Jane", role is "admin) - `test_signin_available` ```python @pytest.mark.parametrize(["login", "role], [ ("John", "user"), ("Jane", "admin"), ]) def test_authentication(login, role): """Checks if a user with a specific role can sign in.""" # ... def test_signin_available(): """Checks a sign page can be accessed.""" # ... ``` ## Test Result A test result is the object containing detailed information about a test's execution: the outcome, duration, steps, errors, attachments, etc. - Allure Report converts any test results generated by test frameworks into its own format. - An Allure test result maps to a single `...-result.json` file, with optional attachment files. Consider the the following example: ```python @pytest.mark.parametrize(["login", "role], [ ("John", "user"), ("Jane", "admin"), ]) def test_authentication(login, role): """Checks if a user with a specific role can sign in.""" # ... def test_signin_available(): """Checks a sign page can be accessed.""" # ... ``` If we run the tests defined here, and require any failed tests to be retried several times until they pass, we would end up with a collection of test results that looks like this: ``` test case test_authentication (parameters: login, role) ├── test (login = "John", role = "user") │ ├── test result (attempt 1, failed) │ └── test result (attempt 2, passed) ← retry after failure └── test (login = "Jane", role = "admin") └── test result (attempt 1, passed) test case test_signin_available (no parameters) └── test ├── test result (attempt 1, failed) ├── test result (attempt 2, failed) ← retry after failure └── test result (attempt 3, passed) ← retry after failure ``` --- ## History files > URL: https://allurereport.org/docs/how-it-works-history-files/ # History files Warning: This page describes a data format used by Allure. Understanding this format is **not necessary** for using Allure in your project. For a more high-level description of working with history, see [Tests history](/docs/history-and-retries/#tests-history). The `history` directory in Allure Report 2 and the history JSONL file, which replaced it in Allure Report 3, are used for transferring simplified information between consecutive test reports. Historical data populates the History tab in the test details and [various charts](/docs/visual-analytics/). ## Allure Report 3 Allure report 3 stores all historical data in a single JSONL file, its location determined by the `historyPath` [configuration parameter](/docs/v3/configure/#_3-history-options). ### History File Format **File:** JSONL (JSON Lines) format - one JSON object per line **Location:** Configured via `historyPath` (default: `undefined`) **Structure:** Each line represents one complete test run Example: ```jsonl {"uuid":"abc-123","name":"Allure Report","timestamp":1697020800000,"knownTestCaseIds":["tc1","tc2"],"testResults":{...},"metrics":{},"url":""} {"uuid":"def-456","name":"Allure Report","timestamp":1697107200000,"knownTestCaseIds":["tc1","tc2","tc3"],"testResults":{...},"metrics":{},"url":""} ``` Each history entry contains: - `uuid` (string) — unique ID for this report run. - `name` (string) — report name. - `timestamp` (integer) — when this run happened, in the UNIX timestamp format. - `knownTestCaseIds` (array of strings) — array of test case IDs. - `testResults` (object) — object keyed by `historyId` containing test results. - `url` (string) — remote URL (e.g., CI job URL). Each test result in the `testResults` object contains: - `id` (string) — the unique identifier of this history item. - `name` (string) — the title of the test. Copied from [`name`](/docs/how-it-works-test-result-file/#name-string) in the test result file. - `fullName` (string) — a unique identifier based on the file name and the test name. Copied from [`fullName`](/docs/how-it-works-test-identifiers/#fullname) in the test result file. - `environment` (string) — environment name. Set by matching labels via rules defined in the [configuration file](/docs/v3/configure/#_6-environments). - `status` (string) — the status with which the test or step finished. Copied from [`status`](/docs/how-it-works-test-result-file/#status-string) in the test result file. - `start` (integer) — the time when the execution started, in the UNIX timestamp format. Copied from [`start`](/docs/how-it-works-test-result-file/#start-integer) in the test result file. - `stop` (integer) — the time when the execution finished, in the UNIX timestamp format. Copied from [`stop`](/docs/how-it-works-test-result-file/#stop-integer) in the test result file. - `duration` (integer) — the difference between the `start` and `stop` values. - `labels` (array) — array of test labels. Copied from [`labels`](/docs/how-it-works-test-result-file/#labels-array) in the test result file. - `url` (string) — currently unused. - `historyId` (string) — the [`historyId`](/docs/how-it-works-test-identifiers/#historyid) for this history item. - `reportLinks` (array) — currently unused. Failed tests also contain the following fields populated with error data copied from [`statusDetails`](/docs/how-it-works-test-result-file/#statusdetails-object) in the test result file: - `message` (string) — a brief description of why the test failed. - `trace` (string) — the complete stack trace showing where in the code the failure occurred. Single test result object example: ```json { "9b448b5eff623519556020f329494163.d41d8cd98f00b204e9800998ecf8427e": { "id": "74364e3f7162027e8e10fe7f3af56b5e", "name": "Creating new issue authorized user", "fullName": "io.qameta.allure.IssuesWebTest.shouldCreateIssue", "environment": "default", "status": "passed", "start": 1748616059263, "stop": 1748616060376, "duration": 1113, "labels": [ { "name": "framework", "value": "junit-platform" }, { "name": "language", "value": "java" }, { "name": "package", "value": "io.qameta.allure.IssuesWebTest" }, { "name": "testClass", "value": "io.qameta.allure.IssuesWebTest" }, { "name": "testMethod", "value": "shouldCreateIssue" }, { "name": "suite", "value": "io.qameta.allure.IssuesWebTest" } ], "url": "", "historyId": "9b448b5eff623519556020f329494163.d41d8cd98f00b204e9800998ecf8427e", "reportLinks": [] } } ``` ### Configuration ```javascript { historyPath: "./.allure/history.jsonl", appendHistory: true } ``` ## Allure Report 2 Allure Report 2 generates its `history` directory during test report generation. It can then be copied to the test results directory for the new report, either manually or by a CI integration plugin. The directory contains the following files: - [history.json](#historyjson) - [history-trend.json](#history-trendjson) - [duration-trend.json](#duration-trendjson) - [retry-trend.json](#retry-trendjson) - [categories-trend.json](#categories-trendjson) ### history.json The `history/history.json` file stores general information about runs of each test. In the root JSON object in the file, each key is some test's [`historyId`](/docs/how-it-works-test-identifiers/#historyid). Each value is an object containing: - `statistic` — the statistics related to the test (used for the [Test statuses chart](/docs/visual-analytics/#test-statuses-chart)). - `items` — information about specific runs of the test (used for the History tab in the test details). Example: ```json { "4da3226ec9761b158ba5f826fc8f8289": { "statistic": { "failed": 1, "broken": 0, "skipped": 0, "passed": 0, "unknown": 0, "total": 1 }, "items": [ { "uid": "74556baed1daf783", "reportUrl": "https://reports.example.com/1234", "status": "failed", "statusDetails": "AssertionError: Some fail reason\nassert False", "time": { "start": 1682358404995, "stop": 1682358404995, "duration": 0 } } ] } } ``` In the `statistic` object, there is a key corresponding to each possible [test status](/docs/test-statuses/). The value for each of these keys represents the number of times the test got this status. Additionally, there is the `total` key, with the value representing the total number of reports that include this test. The `items` array represents the test's data from different test reports. Each object in the array has the following properties: - `uid` (string) — the unique identifier of this history item. - `reportUrl` (string) — the URL of the test report from which this data is extracted. - `status` (string) — the status with which the test or step finished. Copied from [`status`](/docs/how-it-works-test-result-file/#status-string) in the test result file. - `statusDetails` (string) — the short text message with which the test finished. Copied from [`statusDetails.message`](/docs/how-it-works-test-result-file/#statusdetails-object) in the test result file. - `time` (object): - `start` (integer) — the time when the execution started, in the UNIX timestamp format. Copied from [`start`](/docs/how-it-works-test-result-file/#start-integer) in the test result file. - `stop` (integer) — the time when the execution finished, in the UNIX timestamp format. Copied from [`stop`](/docs/how-it-works-test-result-file/#stop-integer) in the test result file. - `duration` (integer) — the difference in seconds between the `start` and `stop` values. ### history-trend.json The `history/history-trend.json` file stores data for the [Trend chart](/docs/visual-analytics/#trend-charts). Example: ```json [ { "buildOrder": 1234, "reportName": "Report #1234", "reportUrl": "https://reports.example.com/1234", "data": { "failed": 0, "broken": 0, "passed": 32, "skipped": 0, "unknown": 0, "total": 32 } } ] ``` Each object in the array has the following properties: - `buildOrder` (integer) — the identifier provided by the build's [`executor.json`](/docs/how-it-works-executor-file/). - `reportUrl` (string) — the URL of the test report from which the data is extracted. - `reportName` (string) — the title of the report from which the data is extracted. - `data` (object) — in this object, there is a key corresponding to each category (see [Categories](/docs/categories/)). The value for each of these keys represents the number of tests in this category. The value for the `total` key indicates the total number of tests. ### duration-trend.json The `history/duration-trend.json` file stores data for the Duration trend chart, see [Trend charts](/docs/visual-analytics/#trend-charts). Example: ```json [ { "buildOrder": 1234, "reportName": "Report #1234", "reportUrl": "https://reports.example.com/1234", "data": { "duration": 78 } } ] ``` Each object in the array has the following properties: - `buildOrder` (integer) — the identifier provided by the build's [`executor.json`](/docs/how-it-works-executor-file/). - `reportUrl` (string) — the URL of the test report from which the data is extracted. - `reportName` (string) — the title of the report from which the data is extracted. - `data` (object): - `duration` (integer) — number of seconds it took to run all the tests. ### retry-trend.json The `history/retries-trend.json` file stores data for the Retries trend chart, see [Trend charts](/docs/visual-analytics/#trend-charts). Example: ```json [ { "buildOrder": 1234, "reportName": "Report #1234", "reportUrl": "https://reports.example.com/1234", "data": { "run": 39, "retry": 15 } } ] ``` Each object in the array has the following properties: - `buildOrder` (integer) — the identifier provided by the build's [`executor.json`](/docs/how-it-works-executor-file/). - `reportUrl` (string) — the URL of the test report from which the data is extracted. - `reportName` (string) — the title of the report from which the data is extracted. - `data` (object): - `run` (integer) — number of test runs that did not cause a retry. - `retry` (integer) — number of test runs that caused a retry. ### categories-trend.json The `history/categories-trend.json` file stores data for the Categories trend chart, see [Trend charts](/docs/visual-analytics/#trend-charts). Example: ```json [ { "buildOrder": 1234, "reportName": "Report #1234", "reportUrl": "https://reports.example.com/1234", "data": { "Product defects": 2, "Test defects": 1 } } ] ``` Each object in the array has the following properties: - `buildOrder` (integer) — the identifier provided by the build's [`executor.json`](/docs/how-it-works-executor-file/). - `reportUrl` (string) — the URL of the test report from which the data is extracted. - `reportName` (string) — the title of the report from which the data is extracted. - `data` (object) — in this object, there is a key corresponding to each category (see [Categories](/docs/categories/)). The value for each of these keys represents the number of tests in this category. --- ## Test Identifiers > URL: https://allurereport.org/docs/how-it-works-test-identifiers/ # Test Identifiers Allure integrations and Allure reporting tools use internal test identifiers to track tests across runs and correctly map historical results within reports. This article explains how these internal identifiers work and what parts of your testing workflow may be affected by any changes to them. Tip: This article frequently refers to concepts such as test cases, tests and test results. Please refer to the [Glossary](/docs/how-it-works-glossary/) to learn what exactly these terms mean within Allure Report. ## Identifier Types There are four main identifiers that Allure tools use to track test results. - [`fullName`](#fullname) - [`testCaseId`](#testcaseid) - [`historyId`](#historyid) - [`uuid`](#uuid) Here is an example of an Allure test result file (typically found within the `allure-results` directory of your project) where all these identifiers are stored: ```json { "name": "test_sum_keyword[sum_arguments1]", "status": "passed", "parameters": [ { "name": "sum_arguments", "value": "[-1, 1, 0]" } ], "start": 1759304434025, "stop": 1759304434025, "uuid": "2f25f93b-9ef3-4e62-b4d7-e376db69e198", "historyId": "741e3c0379dae9c479f495d79cfc4a46", "testCaseId": "e3edbc90111b1e74614910797ec9e808", "fullName": "test.test_sum#test_sum_keyword", "labels": [ { "name": "parentSuite", "value": "test" }, { "name": "suite", "value": "test_sum" }, { "name": "package", "value": "test.test_sum" } ] } ``` All four are opaque identifiers that are calculated for each test result automatically and are not, usually, displayed in the generated reports. However, as they are based on test framework metadata, they may change when you refactor your test code. Three out of four of them (`fullName`, `testCaseId`, `historyId`) are used to map test results to tests, so changes in them may cause problems in your workflow. Below we discuss how each identifier is calculated and what it affects. ### fullName `fullName` identifies [test cases](/docs/how-it-works-glossary/#test-case) and is always calculated according to two rules: - it must be unique for every test case in the project - for parametrized test cases it must not depend on parameter values It is typically a text string consisting of concatenated test metadata supplied by the testing framework (for example, package name + class + test name). Different Allure integrations use different metadata combinations to define `fullName`. Refer to the [section](#test-identifiers-reference-by-framework) for details. `fullName` is the identifier used by Allure reporting tools to do selective test runs: - Allure Report 3 uses `fullName` to identify which test cases to rerun. Tip: Example You run your JavaScript tests like so: `npx allure run --rerun 5 -- npm test` . This means that if any test cases during the initial run fail, you want Allure to rerun them up to 5 times. When the first run completes, Allure collects the `fullName` identifiers of failed tests and creates a test plan with these identifiers, which it then reruns. - Allure TestOps uses `fullName` to run selected test cases from a test plan or launch. - You can also create a test plan file with `fullName` identifiers manually and run it via your test framework’s Allure integration. In all three cases, you may encounter problems, if the `fullName` identifier of some test changes, that is, if you refactor your code (for example, rename some test functions, rearrange test files), between the moment the identifier was saved into a test plan, and the moment the test plan is used to rerun tests. If this is something that may happen in your project, the best practice is to assign `ALLURE_ID` labels to affected tests: if that label is present it overrides `fullName` as the main test case identifier for test plans, and it doesn’t change when you rearrange your code in any way. Tip: The exact semantics of how each framework calculates `fullName` are subject to change. For example, the order in which strings are concatenated, or the characters used as separators may be revised at some point to account for some edge cases in naming. That's why this document goes into detail on what metadata is included into `fullName` for different frameworks, but not into how exactly it is constructed. And for this same reason, we advise against building any custom integrations based on the exact structure of `fullName`. ### testCaseId `testCaseId` identifies [test cases](/docs/how-it-works-glossary/#test-case) and is usually a hash of `fullName`. In some rare cases, it is a hash derived from similarly unique test metadata. These cases will be discussed below. Like `fullName`, `testCaseId` is an opaque identifier not meant to be human readable. In most cases, code refactoring that causes `fullName` for a test case to change will also change `testCaseId`. Currently, it is only used to map test results to test cases in the Allure TesOps database, but in future releases its use will expand. Tip: Unlike `fullName`, you can set `testCaseId` via API in some framework integrations. For details, please consult the framework-specific documentation. ### historyId `historyId` identifies individual [tests](/docs/how-it-works-glossary/#test). It is used to track the history of test results for a specific test with the same set of parameters (and display previous results on the [History and Retries](/docs/history-and-retries/) tabs in Allure Report). - it depends on all of the same metadata that `fullName` and `testCaseId` depend on - unlike `fullName` and `testCaseId`, it additionally depends on parameters. Where a parametrized test case will have the same `fullName` and `testCaseId` across all parameter variations, `historyId` will be unique for every unique combination of parameters. And whenever a test parameter value changes, `historyId` for this test changes as well. Tip: If you don't want a certain parameter to affect `historyId`, mark that parameter as excluded. Most Allure integrations support this feature. ### uuid `uuid` is unique for each test result, but is not reused across different runs in any way. No existing integration relies, and no custom integration should rely on it to map results to tests, so it won't be discussed further in this document. # Test Identifiers Reference by Framework This section documents what the identifiers depend on for each framework integration. Its purpose is to show what changes to a given test during refactoring may break the test history and/or its association with the test case. If you find that some refactoring you're planning will affect the identifiers, you can set `testCaseId`/`historyId` to a previous value manually (this is supported by many integrations) and avoid these consequences. ## C# ### NUnit - `fullName` and `testCaseId` depend on: - assembly name - class name (including namespace and generic type arguments) - test fixture arguments (if any) - method signature (name + generic arguments + parameter types) - `historyId` depends on: - `fullName` - parameters ordered by name ### xUnit - `fullName` and `testCaseId` depend on: - assembly name - class name (including namespace) - method signature (name + generic arguments + parameter types) - `historyId` depends on: - `fullName` - parameters ordered by name ### Reqnroll/SpecFlow - `fullName` and `testCaseId` depend on: - assembly name - folder path containing the feature file - `featureInfo.Title` - the feature title - `scenarioTitle` - the scenario name - `historyId` depends on: - `fullName` - parameters ordered by name ## Java Warning: Allure Java 3.0 changed how test identifiers are calculated. After upgrading from Allure Java 2.x, the history of some tests starts over, and parameters added at runtime now affect the identifiers. The rules below describe Allure Java 2.x. See [Migrating to Allure Java 3](/docs/guides/allure-java-3-migration/#test-history) for details. ### Cucumber JVM - `fullName` depends on: - feature file path (relative) - scenario line number - `testCaseId` depends on: - feature file path - original scenario name - `historyId` depends on: - feature file path (relative) - scenario line number ### JBehave - `fullName` depends on: - story name - scenario title - `testCaseId` is not set - `historyId` depends on `fullName` and non-excluded parameter names and values ### JUnit4 - `fullName` depends on: - package name - class name - method name - for Cucumber tests via JUnit4: feature file path or URI - `testCaseId` is not set - `historyId` depends on `fullName` and non-excluded parameter names and values ### JUnit5 - `fullName` depends on: - methodSource: package, class name, method name - classSource: package, class name - classpathResourceSource: Resource path, optional line number - `testCaseId` equals JUnit Platform's uniqueId (stable for test templates) - `historyId` is a hash of JUnit Platform's `uniqueId` ### Karate - `fullName` depends on: - feature file path (relative) - scenario line number - `testCaseId` depends on: - feature file path - scenario name (or line number as fallback) - `historyId` depends on: - Cucumber's internal scenario unique ID ### Scalatest - `fullName`, `testCaseId` and `historyId` all depend on: - suite ID - test name ### Spock - `fullName` depends on: - package name - class name - feature/test name (or iteration name for parameterized tests) - `testCaseId` (Spock 2 only) depends on: - `fullName` - `historyId` depends on: - qualified name - parameter names and values (sorted) ### TestNG - `fullName` depends on: - fully qualified class name (includes package) - test method name - `testCaseId` is not set - `historyId` depends on `fullName` and non-excluded parameter names and values ## JavaScript ### Cucumber-JS - `fullName` and `testCaseId` depend on: - relative path to the feature file - scenario name (as processed by Cucumber) - `historyId` depends on `fullName` and non-excluded parameter names and values ### Cypress - `fullName` and `testCaseId` depend on: - spec file path - array of suite names + test name **Excluded:** - metadata tags/annotations embedded in test names - `historyId` depends on `fullName` and non-excluded parameter names and values ### Jasmine - `fullName` and `testCaseId` depend on: - spec file path (relative, optional - can be undefined) - array of suite names + test name - the spec/test name **Excluded:** - metadata tags/annotations embedded in spec name - `historyId` depends on `fullName` and non-excluded parameter names and values ### Jest - `fullName` and `testCaseId` depend on: - test name - all parent `describe` block names (suite hierarchy) **Excluded:** - ROOT_DESCRIBE_BLOCK (Jest's internal root node) - `historyId` depends on `fullName` and non-excluded parameter names and values ### Mocha / CodeceptJS CodeceptJS uses Mocha under the hood, so identifier calculation rules are the same. - `fullName` and `testCaseId` depend on: - relative path to the test file - suite names - test name - `historyId` depends on `fullName` and non-excluded parameter names and values ### Newman - `fullName`, `testCaseId` and `historyId` all depend on: - array of parent collection/folder names - request/test name ### Playwright - `fullName` depends on: - test file path - line number of the test function in the source file - column number of the test function in the source file - `testCaseId` depends on: - test file path - suite hierarchy - test name - `historyId` depends on `testCaseId` and non-excluded parameter names and values ### Vitest - `fullName` and `testCaseId` depend on: - test file path (via `relativeTestPath`) - array of parent suite names - test name **Excluded:** - metadata tags/annotations embedded in spec name - `historyId` depends on `fullName` and non-excluded parameter names and values ### WebdriverIO (WDIO) - `fullName` and `testCaseId` depend on: - test file path (relative, URL-aware, strips position suffixes) - test title - `historyId` depends on `fullName` and non-excluded parameter names and values ## PHP ### Behat - `fullName` depends on: - regular scenario: Feature title, scenario title - scenario outline: Feature file path, line number - `testCaseId`: - not automatically set - can be manually provided via `@TestCaseId` annotations (stored as label) - `historyId` is not set ### Codeception - `fullName` depends on: - Cept: Codeception's built-in signature - Cest: Namespace, class name, method name - Unit: Namespace, class name, test name - Gherkin: Not set - `testCaseId` depends on: - Codeception's built-in signature - parameter names ### PHPUnit - `fullName` depends on: - fully qualified class name - test method name - `testCaseId` depends on: - `fullName` - non-excluded parameter names - `historyId` depends on: - `testCaseId` - test method name - parameter names and values ## Python ### Behave - `fullName` and `testCaseId` depend on: - feature name - scenario name **Excluded:** - parameters (just the base scenario name used) - `historyId` depends on: - feature name - scenario name - scenario outline parameters (the `Examples` section) ### pytest - `fullName` and `testCaseId` depend on: - package/module path - test function name - class names - only if test is inside a class (can be nested classes) **Excluded:** - parameters - `historyId` depends on: - `fullName` - parameters ordered by name: - pytest parameters (e.g., provided via `@pytest.mark.parametrize`) - Allure parameters (except those with `excluded` set to `True`) ### pytest-bdd - `fullName` and `testCaseId` depend on: - relative file path to the `.feature` file - scenario name **Excluded:** - feature name (uses file path instead) - parameters (just the base scenario name) - `historyId` depends on: - `testCaseId` - parameters ordered by name: - pytest parameters (e.g., provided via `@pytest.mark.parametrize`) - scenario outline parameters from the `Examples` section of the Gherkin file - Allure parameters (except when `excluded` is set to `False`) ### Robot Framework - `fullName` equals Robot Framework's built-in `longname` attribute, which is the full hierarchical test name (suite path + test name) - `testCaseId` depends on: - suite names - test case name - `historyId` is a hash of `longname` ## Ruby ### Cucumber - `fullName` equals built-in `scenario.name` - `testCaseId`: - not automatically set - can be manually provided via `allure_id` labels for test plan support - `historyId` equals built-in `scenario.id` ### RSpec - `fullName` equals `example.full_description` (RSpec's built-in property that includes the nested describe block descriptions plus the test description) - `testCaseId`: - not automatically set - can be manually provided via `allure_id` labels for test plan support - `historyId` depends on RSpec’s built-in `example.id` and parameter names and values --- ## Test result file > URL: https://allurereport.org/docs/how-it-works-test-result-file/ # Test result file Warning: This page describes a data format used by Allure and its integrations for test frameworks. Understanding this format is **not necessary** for using Allure in your project. For a more high-level description of the process of building a report, see [How it works](/docs/how-it-works/). A test result file stores data related to a single run of a single test. The data is stored as a JSON object. Each file is named `{uuid}-result.json`, where `{uuid}` is a randomly generated identifier. This article describes the properties that can be found in the JSON object. Some of these properties can also be found in the objects describing test steps and sub-steps (see the [steps](#steps-array) section). Example: ```json { "uuid": "9d95e6e7-9cf6-4ca5-91b4-9b69ce0971f8", "historyId": "2b35e31882061875031701ba05a3cd67", "testCaseId": "43f8868a367ff70177a99838d39c5b33", "fullName": "com.example.web.essentials.AuthenticationTest.testAuthentication", "name": "testAuthentication()", "links": [ { "type": "link", "name": "Allure Examples", "url": "https://examples.com/" }, { "type": "issue", "name": "BUG-123", "url": "https://bugs.example.com/BUG-123" } ], "labels": [ { "name": "host", "value": "machine-1" }, { "name": "thread", "value": "306681-MainThread" }, { "name": "language", "value": "java" }, { "name": "framework", "value": "junit-platform" }, { "name": "epic", "value": "Web interface" }, { "name": "feature", "value": "Essential features" }, { "name": "story", "value": "Authentication" } ], "status": "passed", "start": 1682358426014, "stop": 1682358426014, "steps": [ { "name": "Step 1", "status": "passed", "start": 1682358426014, "stop": 1682358426014 }, { "name": "Step 2", "status": "passed", "start": 1682358426014, "stop": 1682358426014 } ] } ``` ## Identifiers ### uuid (string) A unique identifier of the test result. ### historyId (string) An identifier used by Allure Report. Two runs of the same test with the same set of parameters will always have the same `historyId`. Refer to the [`historyId` section](/docs/how-it-works-test-identifiers/#historyid) in the test identifiers reference to find out how this identifier works. ### testCaseId (string) An identifier used by Allure TestOps. Two runs of the same test will always have the same `testCaseId`. Refer to the [`testCaseId` section](/docs/how-it-works-test-identifiers/#testsaseid) in the test identifiers reference to find out how this identifier works. ## Metadata ### name (string) The [title](/docs/v3/readability/#title) of the test or the name of the step. ### fullName (string) A unique identifier based on the file name and the test name. Different Allure plugins can use different notations to generate this identifier. Refer to the [`fullName` section](/docs/how-it-works-test-identifiers/#fullname) in the test identifiers reference to find out how this identifier works. ### description (string) The [description](/docs/v3/readability/#description) of the test or step in Markdown format. ### descriptionHtml (string) The [description](/docs/v3/readability/#description) of the test or step in HTML format. ### links (array) An array of [links](/docs/v3/readability/#links) added to the test or step. Each object in the array has the following properties: - `name` (string) — the link's name that will be displayed in the test report, - `url` (string) — the full URL of the link, - `type` (string) — the type of the link, e.g., “issue” or “tms”. ### labels (array) An array of various labels added to the test or step. Each object in the array has the following properties: - `name` (string) — the name of the label. - `value` (string) — the value of the label. A test result can have multiple labels with the same name. For example, this is often the case when a test result has multiple tags. Many metadata fields and navigation-related fields are represented in the file as labels: | Label name | Link to the field description | | ------------- | ---------------------------------------------------------- | | `tag` | [Tags](/docs/v3/readability/#tags) | | `severity` | [Severity](/docs/v3/readability/#severity) | | `owner` | [Owner](/docs/v3/readability/#owner) | | `epic` | [Epic](/docs/v3/navigation/#behavior-based-hierarchy) | | `feature` | [Feature](/docs/v3/navigation/#behavior-based-hierarchy) | | `story` | [Story](/docs/v3/navigation/#behavior-based-hierarchy) | | `parentSuite` | [Parent suite](/docs/v3/navigation/#suite-based-hierarchy) | | `suite` | [Suite](/docs/v3/navigation/#suite-based-hierarchy) | | `subSuite` | [Sub-suite](/docs/v3/navigation/#suite-based-hierarchy) | | `package` | [Package](/docs/v3/navigation/#package-based-hierarchy) | Additionally, most Allure integrations set the following labels for every test: | Label name | Description | | ----------- | ------------------------------------------------------- | | `host` | Hostname of the machine on which the test was run. | | `thread` | The identifier of the thread in which the test was run. | | `language` | The programming language of the test. | | `framework` | The test framework of the test. | The test author can set additional labels with arbitrary names, see [Other labels](/docs/v3/readability/#other-labels). ### parameters (array) An array of [parameters](/docs/v3/readability/#parametrized-tests) added to the test or step. Each object in the array has the following properties: - `name` (string) — the name of the parameter. - `value` (string) — the value of the parameter. - `excluded` (boolean) — if true, Allure will not use the parameter when comparing the current result with the previous one in the history. - `mode` (string) — how the parameter will be shown in the report: - `default` — the parameter and its value will be shown in a table along with other parameters. - `masked` — the parameter will be shown in the table, but its value will be hidden. - `hidden` — the parameter and its value will not be shown in the test report. ### attachments (array) An array of [attachments](/docs/attachments/) added to the test or step. The content of each attachment is stored as a file in the results directory. The file is named `{uuid}-attachment.{ext}`, where `{uuid}` is a randomly generated identifier, and `{ext}` is the filename extension. Each object in the array has the following properties: - `name` (string) — the human-readable name of the attachment. - `source` (string) — the name of the file with the attachment's content. - `type` (string) — the [media type](https://en.wikipedia.org/wiki/Media_type) of the content. ## Execution ### status (string) The status with which the test or step finished. Possible values are: “failed”, “broken”, “passed”, “skipped”, “unknown”. See [Test statuses](/docs/test-statuses/). ### statusDetails (object) An object having the following properties: - `known` (boolean) — indicates that the test fails because of a known bug. - `muted` (boolean) — indicates that the result must not affect the statistics. See [Visual analytics](/docs/visual-analytics/). - `flaky` (boolean) — indicates that this test or step is known to be unstable and can may not succeed every time. See [Flaky tests](/docs/test-stability/#flaky-tests). - `message` (string) — the short text message to display in the test details, such as a name of the exception that led to a failure. - `trace` (string) — the full stack trace to display in the test details. ### stage (string) The stage in the lifecycle of the test or step. Possible values are: “scheduled”, “running”, “finished”, “pending”, “interrupted”. ### start (integer) The time when the execution of the test ot step started, in the [UNIX timestamp](https://en.wikipedia.org/wiki/Unix_time) format. Together with [`stop`](#stop-integer), this field is used to place the test on the [Timeline tab](/docs/timeline/) and to calculate its [duration](/docs/visual-analytics/#duration-charts). ### stop (integer) The time when the execution of the test ot step finished, in the [UNIX timestamp](https://en.wikipedia.org/wiki/Unix_time) format. Together with [`start`](#start-integer), this field is used to place the test on the [Timeline tab](/docs/timeline/) and to calculate its [duration](/docs/visual-analytics/#duration-charts). ### steps (array) An array of [test steps](/docs/steps/). Each step may contain a array of `steps`, too, which allows to represent a nested array of steps and sub-steps. Objects in the array have the following properties, each having the same format as the corresponding property of a test result: - Metadata - [`name`](#name-string) - [`parameters`](#parameters-array) - [`attachments`](#attachments-array) - Execution - [`status`](#status-string) - [`statusDetails`](#statusdetails-object) - [`stage`](#stage-string) - [`start`](#start-integer) - [`stop`](#stop-integer) - [`steps`](#steps-array) --- # Section: Integrations ## Integrations > URL: https://allurereport.org/docs/integrations/ # Integrations This chapter describes how to integrate Allure Report into various tools used for software development. ## Integrations with CI/CD tools Having Allure Report on a local machine is great, but what if we want to generate the reports automatically as part of a CI/CD process? Integrations with CI/CD systems allow just that. They download the correct version of Allure Report and generate the report during a pipeline run. Some integrations can embed the report into the pipeline results page so you and your colleagues may view it later. Check out the documentation page of each integration to learn more about its capabilities: - [Azure DevOps integration](/docs/integrations-azure/) - [Bamboo integration](/docs/integrations-bamboo/) (Allure 2) - [GitHub Action integration](/docs/integrations-github-action/) (Allure 3) - [Jenkins integration](/docs/integrations-jenkins/) - [TeamCity integration](/docs/integrations-teamcity/) (Allure 2) ## Integrations with build tools These integrations add Allure Report generation as a step in your build lifecycle. They download the Allure CLI automatically and expose goals or tasks for generating and serving reports: - [Gradle integration](/docs/integrations-gradle/) - [Maven integration](/docs/integrations-maven/) ## Integrations with development tools These integrations embed Allure Report commands into the IDE's UI: - [JetBrains IDE integration](/docs/integrations-jetbrains/) (Allure 2) - [Visual Studio Code integration](/docs/integrations-vscode/) (Allure 2) --- ## Azure DevOps integration > URL: https://allurereport.org/docs/integrations-azure/ # Azure DevOps integration With the Allure Report extension for Azure DevOps, you can set up automatic generation of reports after each test launch for your project. The reports are displayed in an **Allure Report** tab on the pipeline run page. By default, the extension generates reports with Allure Report 3. If needed, you can [switch to Allure Report 2](#using-allure-report-2) instead. ## 1. Install the extension For the integration to work, you need to install the official Allure Report extension to your Azure DevOps organization. 1. On the [Allure Report extension page](https://marketplace.visualstudio.com/items?itemName=qameta.allure-azure-pipelines) in the Visual Studio Marketplace, click **Get it free**. 1. If prompted, sign in as your organization's administrator. 1. Select your organization from the dropdown list. Click **Install**. ![Selecting organization for the extension installation](/images/integrations/azure/select-organization.png "Selecting organization for the extension installation") ## 2. Modify the pipeline Once the extension is installed, it makes the `PublishAllureReport@2` task available in Azure Pipelines. To publish Allure Report for a pipeline, add a step with this task after all the steps that produce test results: ```yaml steps: - script: ./gradlew clean test displayName: Run tests - task: PublishAllureReport@2 displayName: Publish Allure Report ``` With no parameters, the task generates a report from the test results in the `allure-results` directory and publishes it under the name “Allure Report”. ### Parameters Specify parameters in the `inputs` section of the step if the defaults do not fit your project: - `testResultsDir` — path to the directory with test results. Relative paths are resolved against the pipeline's working directory (`$(System.DefaultWorkingDirectory)`, the repository checkout root). Depending on the test framework and the Allure adapter configuration, an appropriate path may be `allure-results`, `build/allure-results`, or some custom path. Defaults to `allure-results`. - `reportName` — the report name to display in the Azure DevOps UI. Defaults to “Allure Report”. This value also takes precedence over the report name set in the [configuration file](#configuration-file). - `reportDir` — path to an Allure Report 3 report generated by an earlier step of the pipeline. If specified, the task publishes this report instead of generating a new one, and `testResultsDir` is ignored. The report must be generated in single-file mode, unless it was [published to a storage service](#reports-published-to-a-storage-service). A report published this way keeps the name it was generated with — the `reportName` parameter has no effect on it. - `allureVersion` — an Allure Report 2 version, e.g. `2.36.0`. If specified, the task generates the report with [Allure Report 2](#using-allure-report-2) instead of Allure Report 3. - `allureDownloadUrl` — a custom URL to download the Allure Report 2 distribution from, see [Using Allure Report 2](#using-allure-report-2). ```yaml - task: PublishAllureReport@2 displayName: Publish Allure Report inputs: testResultsDir: build/allure-results reportName: My Report ``` ### Configuration file If the working directory contains an [Allure Report 3 configuration file](/docs/v3/configure/) (`allurerc.js`, `allurerc.mjs`, etc.), the task generates the report according to it. Two things to keep in mind: - The task can only publish reports generated in single-file mode. Enable it in the configuration like so: ```js export default { name: "My Report", plugins: { awesome: { options: { singleFile: true, }, }, }, }; ``` Reports that are not generated in single-file mode are skipped with a warning. - If the configuration file imports packages, such as `@allurereport/core`, make sure your pipeline installs the project's dependencies before the task runs. If there is no configuration file, the task generates a single-file report with the default settings. ### Publishing multiple reports To publish several reports from a single pipeline run, add multiple task steps with different inputs: ```yaml - task: PublishAllureReport@2 displayName: Generate and publish a new report inputs: testResultsDir: path/to/allure-results reportName: Report 1 - task: PublishAllureReport@2 displayName: Publish an already generated report inputs: reportDir: path/to/already-generated-report ``` The **Allure Report** tab will then show the list of the published reports with their test statistics. Note that the second report in this example is displayed under the name it was generated with, not under a name from the pipeline definition. ### Reports published to a storage service If your Allure Report configuration enables publishing to a [storage service](/docs/self-hosted-storage/), then generating a report also uploads it to the service, where it gets a permanent URL. In this case, the report directory on the build agent contains a reference to the hosted copy, and the task publishes only this metadata to Azure DevOps instead of the report files. The **Allure Report** tab then displays the hosted report directly from its URL. Single-file mode is not required for such reports. For example, if your test step generates and publishes the report, point the task at the resulting report directory: ```yaml - script: npx allure run -- npm test displayName: Run tests and generate the report - task: PublishAllureReport@2 displayName: Publish Allure Report inputs: reportDir: allure-report ``` Since the tab loads such a report from the storage service, the report is only visible to users whose browsers can reach the service. Warning: The service must be reachable at a public HTTPS address: modern browsers do not allow pages on the public web, including Azure DevOps, to load content from private networks, so a report hosted on an intranet-only or `localhost` address will be blocked by the viewer's browser. ### Using Allure Report 2 To generate the report with Allure Report 2 instead of Allure Report 3, set the `allureVersion` parameter: ```yaml - task: PublishAllureReport@2 displayName: Publish Allure Report inputs: allureVersion: 2.36.0 ``` Version 2.36.0 is bundled with the extension and works without downloading anything. Any other version is downloaded from the [GitHub releases](https://github.com/allure-framework/allure2/releases/). Do not use versions older than 2.25.0, as they do not support the single-file mode required by the extension. If your build agents cannot access GitHub, use the `allureDownloadUrl` parameter to download the Allure Report 2 distribution from a custom location. The `{{allureVersion}}` placeholder in the URL is replaced with the `allureVersion` value: ```yaml - task: PublishAllureReport@2 displayName: Publish Allure Report inputs: allureVersion: 2.36.0 allureDownloadUrl: https://mirror.example.com/allure/allure-{{allureVersion}}.tgz ``` ## 3. View test reports After the configuration, each new run's page will include the **Allure Report** tab. If a single report was published, the tab displays it right away: Images: /images/integrations/azure/report-light.png, /images/integrations/azure/report-dark.png If there are several reports, the tab lists them along with their test statistics, and you can open any of them with the **View** button: Images: /images/integrations/azure/report-list-light.png, /images/integrations/azure/report-list-dark.png ## Migrating from version 1 of the extension Version 1 of the extension provided the `PublishAllureReport@1` task, which always generated reports with Allure Report 2. Version 2 adds the `PublishAllureReport@2` task without removing the old one: pipelines that reference `PublishAllureReport@1` keep working, and previously published reports remain viewable in the **Allure Report** tab. However, the version 1 task is frozen at its last release (1.5.0) and no longer receives fixes or features, so updating your pipelines is recommended. Info: The version 1 task is only available in organizations that had it installed before the extension was updated. If the extension was first installed at version 2 or later, `PublishAllureReport@1` fails to resolve. To migrate a pipeline: 1. Change the task reference from `PublishAllureReport@1` to `PublishAllureReport@2`. 1. Choose the report engine. All version 1 parameters (`testResultsDir`, `reportName`, `allureVersion`, `allureDownloadUrl`) work in version 2 with the same meaning, and if your step sets `allureVersion`, it continues to generate the same Allure Report 2 reports as before. However, if `allureVersion` is not set, the new task generates reports with [Allure Report 3](/docs/v3/) — a new report engine with a different look. To keep the reports as they were, set `allureVersion` explicitly: ```yaml - task: PublishAllureReport@2 displayName: Publish Allure Report inputs: allureVersion: 2.36.0 ``` ## Limitations - The task can only publish reports generated in single-file mode, unless they were [published to a storage service](#reports-published-to-a-storage-service). Reports generated with a [configuration file](#configuration-file) must have the mode enabled explicitly. - The maximum size of a report the tab can display is **512 MB**. The task fails if the generated report file exceeds this limit. Reports larger than 20 MB are uploaded in chunks and may take a bit longer to open in the tab. ## Troubleshooting ### The Allure Report tab shows an error instead of a report The error means that no report was attached to the pipeline run, even though the task ran. The most common cause is a [configuration file](#configuration-file) that does not enable single-file mode: the task cannot publish such reports and skips them, leaving a warning about single-file mode in the task log. Enable the mode in the configuration as shown above. Otherwise, check the log of the `PublishAllureReport` step for errors and warnings. ### The report is empty If the tab shows a report with no test results in it, the task generated the report from a wrong or empty directory. Check the following: - The `testResultsDir` value matches the directory your test step actually writes results to, keeping in mind that relative paths are resolved against `$(System.DefaultWorkingDirectory)`. - The steps that produce test results run before the `PublishAllureReport` step and actually wrote the result files. For example, make sure the pipeline does not skip the test step when tests fail. - The task log: an error mentioning the test results directory, such as `can't read directory`, means the directory did not exist when the task ran. --- ## Bamboo integration > URL: https://allurereport.org/docs/integrations-bamboo/ # Bamboo integration (Allure 2) With the Allure Report plugin for Bamboo, you can set up automatic generation of reports after each test launch for your project. ## Installation and configuration To enable Allure support in your Bamboo installation, do the following under the administrator account: 1. [install the plugin](#_1-install-the-plugin), 2. [configure global settings](#_2-configure-global-settings), 3. [install the executable](#_3-install-the-executable). ### 1. Install the plugin The recommended method of installing the Allure Report plugin is via the Bamboo web interface. However, if your Bamboo configuration does not have access to the internet, you can install the plugin by uploading a JAR file to Bamboo manually or specifying its URL. The selected installation method does not affect the functionality of the plugin. **To install the plugin from the Atlassian Marketplace directly:** 1. In Bamboo, click the gear icon in the top right corner and select **Manage apps**. 1. Click **Find new apps**. 1. Using the search box, find the “Allure Report for Bamboo” plugin. Click **Install** next to the plugin. 1. In the dialog that appears, click **Accept & install**. Wait until the plugin is downloaded and installed. **To install the plugin via a JAR file:** 1. On the [Allure Report plugin page in the Atlassian Marketplace](https://marketplace.atlassian.com/apps/1217177), click **Get it now**. 1. In the dialog that appears, click the **Get it now** button for the “Data Center” option. Save the JAR file to a local directory on your device. 1. In Bamboo, click the gear icon in the top right corner and select **Manage apps**. 1. Click **Upload app**. 1. Next to the **From my computer** label, click the **Browse** button. 1. In the dialog that appears, select the JAR file from your device. 1. Click **Upload**. Wait until the plugin is uploaded and installed. **To install the plugin from a custom URL:** 1. On the [Allure Report plugin page in the Atlassian Marketplace](https://marketplace.atlassian.com/apps/1217177), click **Get it now**. 1. In the dialog that appears, click the **Get it now** button for the “Data Center” option. Save the JAR file to a web server that is available to the Bamboo server. 1. In Bamboo, click the gear icon in the top right corner and select **Manage apps**. 1. Click **Upload app**. 1. Next to the **From this URL** label, specify the URL to the JAR file. 1. Click **Upload**. Wait until the plugin is downloaded and installed. ### 2. Configure global settings 1. In the Bamboo web interface, click the gear icon in the top right corner and select **Overview**. 1. In the menu on the left, select **Plan → Allure Report**. 1. Edit the settings: - **Download if no executable present** — if checked, Bamboo will automatically download the Allure Report utility to each build agent, based on the provided label (see [Install the executable](#_3-install-the-executable)). - **Enable report custom logo** — if checked, you will be able to replace the Allure logo with a custom image (see [Configuring a build plan](#configuring-a-build-plan)). - **Build Allure for all builds by default** — if checked, report generation will be enabled by default for all new build plans (see [Configuring a build plan](#configuring-a-build-plan)). - **Allure binary base url** — the base URL used for downloading the required version of the Allure Report utility, if the **Download if no executable present** setting is checked. Details: By default, the base URL is set to: - https\://github.com/allure-framework/allure2/releases/download/ This means that, to download Allure Report 2.24.0, the plugin will try the following URLs: - https\://github.com/allure-framework/allure2/releases/download/2.24.0/allure-2.24.0.zip - https\://github.com/allure-framework/allure2/releases/download/allure-commandline/2.24.0/allure-commandline-2.24.0.zip - **Allure local storage** — path to a local directory on the Bamboo server where the generated test reports must be stored. 1. Click **Save** to apply the settings. ### 3. Install the executable By default, the plugin adds the latest version of the Allure command-line utility to Bamboo as an “executable”. You may add any number of other versions as well. Once added, Bamboo will take care of downloading the utility to every machine that needs to build a test report. The recommended way to add executables is by letting Bamboo download them automatically from the official Allure releases page. However, if your Bamboo configuration does not have access to the internet, you can set up a custom URL from which the executable will be downloaded. 1. In Bamboo, click the gear icon in the top right corner and select **Overview**. 1. In the menu on the left, select **Build resources → Server capabilities**. 1. In the **Add capability** section, fill in the fields: - **Capability type** — “Executable”. - **Type** — “Allure Report”. - **Executable label** — `allure-⟨VERSION⟩`, e.g., `allure-2.24.0`. See the [Releases page on GitHub](https://github.com/allure-framework/allure2/releases/) for the list of available versions. - **Path** — local path to the Allure Report executable. Note that if you enable the **Download if no executable present** setting (see [Configure global settings](#_2-configure-global-settings)), the plugin will be able to download and save the executable to the path automatically, based on the specified label. If the **Download if no executable present** setting is disabled, make sure that the executable is located at the specified path at each Bamboo build agent that may need to generate the test report. ## Configuring a build plan To add test report generation to a build plan in Bamboo, you don't need to add a dedicated step, rather just [create an artifact](#_1-create-a-build-artifact) from an existing step and [enable report generation](#_2-enable-report-generation) in the plan configuration. ### 1. Create a build artifact 1. On the build plan page, click **Actions → Configure plan**. 1. In the menu on the left, select the job that runs the tests. Info: It is possible that a test launch for your project consists of multiple separate runs, one after another, accumulating the results in the same directory. In such a case, select the last job, e.g., the job after which the test results directory contains all necessary data. 1. Go to the **Artifacts** tab. 1. Click **Create artifact**. 1. Fill in the artifact settings: - **Name** — an identifier for this artifact, e.g., `allure-results`. - **Location** — leave empty. - **Copy pattern** — pattern describing the path to the test results directory (see [How it works](/docs/how-it-works/)). Depending on the framework you use and the Allure integration configuration, an appropriate pattern may be `allure-results/**`, `build/allure-results/**`, or some custom path. Note that the double asterisk at the end is necessary to indicate that the artifact should include all files and subdirectories. 1. Click **Create**. ![Allure Bamboo plugin artifacts page](/images/integrations/bamboo/artifacts.png "Allure Bamboo plugin artifacts page") ### 2. Enable report generation 1. On the build plan page, click **Actions → Configure plan**. 1. Go to the **Other** tab. 1. In the **Allure Reporting** section, make sure that the **Enable Allure Building** checkbox is checked. 1. Edit the settings: - **Build report only for failed builds** — if checked, the plugin will skip generating test reports for successful builds. - **Executable** — the Allure Report version to use for generating the report (see [Install the executable](#_3-install-the-executable)). - **Artifact name to use** — the name you specified on the [Create a build artifact](#_1-create-a-build-artifact) step. - **Custom logo URL** — a URL to a logo that will be displayed instead of the Allure Report logo in the test report. The logo must be an SVG file. This setting is only relevant if the **Enable report custom logo** setting is enabled (see [Configure global settings](#_2-configure-global-settings)). 1. Click **Save**. ## Viewing test reports After the configuration, each finished build's page will include the **Allure Report** tab. The tab contains a frame with the test report. Click **Download** to get the test report as a ZIP archive. Click **Expand** to open the test report on a separate page. ![Allure Bamboo plugin generate report](/images/integrations/bamboo/report.png "Allure Bamboo plugin generate report") --- ## GitHub Action integration > URL: https://allurereport.org/docs/integrations-github-action/ # Allure GitHub Action (Allure 3) Allure Report 3 comes with support for the [Allure GitHub Action](https://github.com/allure-framework/allure-action). With this action you can automatically post test run summaries as pull request comments in your GitHub repository. The summaries contain: - A visual summary table of the run and generated reports - Test run duration - Amount of new, flaky and retried tests in the run - Full report link, if `remote-href` is configured or the report was published to the [Allure Storage Service](/docs/self-hosted-storage/) Additionally, if your allure configuration includes [quality gate](/docs/quality-gate/) settings, you get a pass/fail Quality Gate GitHub Check on your pull requests. ## Prerequisites ### Allure Report Generation Workflow To work properly, this GitHub Action requires that your workflow use [Allure 3](https://github.com/allure-framework/allure3) and relevant [Allure integrations depending on your framework](/docs/frameworks/) to generate an HTML Allure Report. HTML reports produce the `{report-directory}/summary.json` file which the action reads, turns into a readable summary table, and posts as a pull request comment. Reports also include a `{report-directory}/test-results.json` file with each test's name, status, and duration, which the action uses to resolve individual tests for [Section Comments](#section-comments-sections). Tip: The action will not fully work if you're generating CSV reports, as the CSV plugin doesn't create a `summary.json` file. It may also not work with third party HTML report plugins, if they don't create a `summary.json`. ### GitHub Permissions Add `pull-requests` and `checks` permissions to your workflow to enable comments and checks for pull requests in your repository: ```yaml permissions: pull-requests: write checks: write ``` ## Configuration Add the action to your workflow right after the step at which you produce the Allure Report: ```yaml - name: Run tests run |- # run your tests that generate Allure Report data - name: Run Allure Action uses: allure-framework/allure-action@v0 with: # Path to the generated report directory # By default, it's set to `./allure-report` report-directory: "./allure-report" # GitHub Token that's used for posting the comments in Pull Requests github-token: ${{ secrets.GITHUB_TOKEN }} ``` ### Minimal Working Example ```yaml name: Tests with Allure Report on: pull_request: branches: [main] permissions: pull-requests: write checks: write jobs: test: runs-on: ubuntu-latest steps: # 1. Checkout code - name: Checkout uses: actions/checkout@v4 # 2. Setup your language/runtime - name: Setup Node.js uses: actions/setup-node@v4 with: node-version: "20" # 3. Install dependencies - name: Install dependencies run: npm ci # 4. Run tests (must use Allure reporter) - name: Run tests and generate report run: npx allure run -- npm run test # This generates allure-results/ directory and creates allure-report/ with summary.json # 5. Post summary to PR - name: Run Allure Action uses: allure-framework/allure-action@v0 with: report-directory: "./allure-report" github-token: ${{ secrets.GITHUB_TOKEN }} ``` ### Report URL (`remote-href`) To add a **View** link to the summary table and turn new/flaky/retry counts into clickable filter links, provide the base URL where your report is hosted: ```yaml - name: Run Allure Action uses: allure-framework/allure-action@v0 with: report-directory: "./allure-report" github-token: ${{ secrets.GITHUB_TOKEN }} remote-href: "https://my-org.github.io/my-repo/" ``` This is useful when publishing reports to GitHub Pages or any other static hosting. If the report was published to the [Allure Storage Service](/docs/self-hosted-storage/), the URL is embedded in the report automatically and `remote-href` is not needed. ### Section Comments (`sections`) By default, the action posts only the summary table. To also post collapsible comments that list individual tests, use the `sections` input: ```yaml - name: Run Allure Action uses: allure-framework/allure-action@v0 with: report-directory: "./allure-report" github-token: ${{ secrets.GITHUB_TOKEN }} sections: | new flaky ``` Supported values: `new`, `flaky`, `retry`, `all`. The input accepts comma- or newline-separated values. Each section is posted as a separate collapsible comment listing tests with their status, name, and duration, resolved from the `{report-directory}/test-results.json` file - keep the full report directory intact when passing it to the action, or affected tests are silently dropped from the list. If `remote-href` is configured, each test name links directly to its result in the report. Oversized lists are truncated and a **More** link to the filtered remote report is appended when available. Section comments from previous runs are automatically cleaned up when sections are removed. ### Workflows not triggered by pull requests (`pr-number`, `head-sha`) By default, the action finds the pull request to comment on from the `pull_request` event. For workflows triggered by other events, such as `issue_comment`, `workflow_dispatch`, or `workflow_run`, pass the pull request number via the `pr-number` input. When the `pull_request` event data is available, it takes precedence over `pr-number`. To attach the checks to the pull request's head commit instead of the commit that triggered the workflow, also pass the `head-sha` input. For example, in an `issue_comment` workflow, the `github.sha` context value points to the default branch, not to the pull request, so resolve the head commit first: ```yaml on: issue_comment: types: [created] permissions: pull-requests: write checks: write jobs: report: if: ${{ github.event.issue.pull_request && contains(github.event.comment.body, '/run-tests') }} runs-on: ubuntu-latest steps: - name: Resolve pull request head id: pr env: GH_TOKEN: ${{ github.token }} GH_REPO: ${{ github.repository }} PR_NUMBER: ${{ github.event.issue.number }} run: | head_sha="$(gh pr view "$PR_NUMBER" --json headRefOid --jq .headRefOid)" echo "head-sha=$head_sha" >> "$GITHUB_OUTPUT" # check out the pull request head, run your tests, and generate the report - name: Run Allure Action uses: allure-framework/allure-action@v0 with: report-directory: "./allure-report" github-token: ${{ github.token }} pr-number: ${{ github.event.issue.number }} head-sha: ${{ steps.pr.outputs.head-sha }} ``` The `pr-number` and `head-sha` inputs are available in Allure Action 0.11.0 and newer. ## Outputs ### Summary Table Comment The action posts a comment like this: Images: /images/integrations/action/summary-light.png, /images/integrations/action/summary-dark.png If you have multiple reports set up (for example one general report and one dashboard report), you'll get a separate table row for each report. The action scans `report-directory` recursively for all `summary.json` files. ### Section Comments If the `sections` input is configured, the action posts additional collapsible comments — one per enabled section — listing individual tests. See [Section Comments](#section-comments-sections) above for details. Images: /images/integrations/action/sections-light.png, /images/integrations/action/sections-dark.png Section comments are always posted collapsed, so they won't clutter your pull requests. ### Quality Gate Check If [quality gates](/docs/quality-gate/) are configured in your Allure setup, the action creates a GitHub Check: - ✅ **Success** - All quality gates passed - ❌ **Failure** - One or more quality gates failed The check includes details about which rules failed. To determine the outcome, the action looks for quality gate data in the `{report-directory}/quality-gate.json` file. --- ## Gradle integration > URL: https://allurereport.org/docs/integrations-gradle/ # Gradle integration The Allure Gradle plugin generates Allure reports from the raw results produced by your test tasks. It handles downloading the Allure CLI, instrumenting test tasks to write results, and providing tasks for generating and serving reports. - Requires Gradle 8.11 or newer - Requires Java 17 or newer ## Setting up Apply the umbrella plugin in your `build.gradle.kts`: ```kotlin plugins { id("io.qameta.allure") version "4.3.0" } repositories { // Required so Gradle can resolve allure-java adapter dependencies mavenCentral() } ``` Or in Groovy DSL (`build.gradle`): ```groovy plugins { id 'io.qameta.allure' version '4.3.0' } repositories { mavenCentral() } ``` The `io.qameta.allure` plugin is a shortcut that applies both `io.qameta.allure-adapter` and `io.qameta.allure-report` together. Apply them separately if you need finer control (see [Plugin breakdown](#plugin-breakdown)). ## Generating a report ### View the report locally ```shell ./gradlew allureServe ``` Opens the report in a local web server. By default it does not run your tests first. ### Run tests and view the report ```shell ./gradlew allureServe --depends-on-tests ``` ### Generate a static report ```shell ./gradlew allureReport ``` Output: `build/reports/allure-report/allureReport/` When using the extension `reportDir`, the output subdirectory is always named after the task (`allureReport`, `allureAggregateReport`, etc.), so the aggregate report lands at `build/reports/allure-report/allureAggregateReport/`. When using the `--report-dir` flag, the report is written directly to the specified path with no task-name subdirectory added. ### Run tests and generate a report in one command ```shell ./gradlew allureReport --depends-on-tests ``` ## Tasks added by the plugin | Task | Description | | ----------------------- | ----------------------------------------------------------------------------------------------- | | `allureReport` | Generates a static HTML report | | `allureServe` | Generates a report and opens it in a local web server | | `allureAggregateReport` | Generates an aggregate report across subprojects (requires `io.qameta.allure-aggregate-report`) | | `allureAggregateServe` | Serves an aggregate report (requires `io.qameta.allure-aggregate-report`) | | `downloadAllure` | Downloads and caches the Allure CLI | ## Selecting the Allure version Set `allure.version` to choose the runtime: ```kotlin allure { version = "3.16.1" // Allure 3 (default) // version = "2.42.1" // Allure 2 } ``` - Version `3.x` → Allure 3 (Node.js-based, provisioned automatically) - Version `2.x` → Allure 2 (`allure-commandline` zip, downloaded from Maven Central) - Default when unset: Allure 3, version `3.16.1` Any version string whose major number is 3 or higher selects Allure 3; major number 2 or lower selects Allure 2. ## Allure 3 runtime provisioning For Allure 3, the plugin provisions a private Node.js runtime automatically. You do **not** need Node.js installed on your machine. On first use, the plugin downloads Node.js and installs the `allure` npm package. The Node.js download is cached by Gradle and skipped on subsequent builds. The npm install runs again on a fresh checkout. ## Test framework autoconfiguration When you apply `io.qameta.allure-adapter`, the plugin scans your test classpath and automatically adds the matching allure-java adapter. No manual dependency declaration is needed for supported frameworks. Supported frameworks and their activation triggers: | Framework DSL name | Activates when this is on the classpath | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `junit4` | `junit:junit` | | `jupiter` | `org.junit.jupiter:junit-jupiter-api` (JUnit 5 and JUnit 6). The previous name, `junit5`, still works as a deprecated alias | | `junitPlatform` | `org.junit.platform:junit-platform-launcher` | | `testng` | `org.testng:testng` (6.14.3+ with Allure Java 2.x, 7.10.0+ with Allure Java 3.x) | | `assertj` | `org.assertj:assertj-core` | | `jbehave` | `org.jbehave:jbehave-core` (4.x). Deprecated, Allure Java 2.x only | | `jbehave5` | `org.jbehave:jbehave-core` (5.x) | | `karate` | `com.intuit.karate:karate-core` (Karate 1, Allure Java 2.x) or `io.karatelabs:karate-core` (Karate 2, Allure Java 3.x) | | `scalatest` | `org.scalatest:scalatest_2.12`, `scalatest_2.13`, or `scalatest_3` (Allure Java 3.x only) | | `spock` | `org.spockframework:spock-core` (uses `allure-spock2`; requires `useJUnitPlatform()` on the test task — without it, the adapter is on the classpath but captures no results) | | `cucumber4Jvm` | `io.cucumber:cucumber-core` (4.x). Deprecated, Allure Java 2.x only | | `cucumber5Jvm` | `io.cucumber:cucumber-core` (5.x). Deprecated, Allure Java 2.x only | | `cucumber6Jvm` | `io.cucumber:cucumber-core` (6.x). Deprecated, Allure Java 2.x only | | `cucumber7Jvm` | `io.cucumber:cucumber-core` (7.x) | The default allure-java version used for all adapters is `2.35.5`. AspectJ weaver version is `1.9.25.1`. To use Allure Java 3, see [Using Allure Java 3](#using-allure-java-3). ### What autoconfiguration does When a supported framework is detected: - Adds the allure adapter dependency to the test compile and runtime classpath - Adds the AspectJ weaver agent to the JVM arguments of all `Test` and `JavaExec` tasks (enables `@Step` and `@Attachment` support) - Adds `allure-descriptions-javadoc` to the annotation processor path (enables `@Description` from Javadoc) - Configures default test listeners where applicable (`jupiter`, `junitPlatform`, `testng`, `karate`) ### Disabling autoconfiguration To turn off all autoconfiguration: ```kotlin allure { adapter { autoconfigure.set(false) } } ``` To turn off individual parts: ```kotlin allure { adapter { autoconfigureListeners.set(false) // don't register default listeners autoconfigureJavadocDescriptions.set(false) // don't add the Javadoc annotation processor aspectjWeaver.set(false) // don't add AspectJ agent } } ``` To disable listeners for a specific framework only: ```kotlin allure { adapter { frameworks { jupiter { autoconfigureListeners.set(false) } } } } ``` ### Explicitly listing frameworks If you have multiple test frameworks on the classpath but only want to instrument one: ```kotlin allure { adapter { frameworks { jupiter // only jupiter will be configured; others are ignored } } } ``` Listing any framework explicitly in `frameworks {}` switches the plugin from fully-automatic mode to selective mode: only the adapters you name here are registered, and the automatic registration of all other adapters is suppressed. Each listed adapter is still activation-gated — its dependency is added only when the trigger dependency is detected on the classpath, so there is no cost if the trigger is absent. ## Using Allure Java 3 The plugin uses Allure Java 2.x by default. To switch all adapters to [Allure Java 3](/docs/guides/allure-java-3-migration/), set `allureJavaVersion`: ```kotlin allure { adapter { allureJavaVersion.set("3.0.0") } } ``` Allure Java 3 requires Java 17 or newer for the test JVM (Java 21 for Karate 2). The JVM running Gradle itself can be different, so set the Java toolchain or the test task's `javaLauncher` accordingly. If a detected framework is not supported by the selected Allure Java version, for example, Cucumber-JVM 6 or TestNG older than 7.10.0 with Allure Java 3, the build fails with a message explaining how to fix it. Upgrade the framework, or keep that project on Allure Java 2.x. With Karate 2, register the Allure listener in your runner explicitly, since the plugin can only register it automatically for Karate 1: ```java Runner.builder() .path("classpath:features") .listener(new io.qameta.allure.karate.AllureKarate()) .parallel(4); ``` ## Configuration reference ### Adapter options ```kotlin allure { adapter { // allure-java adapter version for all frameworks (set to "3.0.0" to use Allure Java 3) allureJavaVersion.set("2.35.5") // AspectJ weaver version aspectjVersion.set("1.9.25.1") // Where test tasks write raw Allure results resultsDir.set(layout.buildDirectory.dir("allure-results")) // Path to categories.json for Allure 2 (auto-detected from test/resources by default) categoriesFile.set(layout.projectDirectory.file("config/categories.json")) } } ``` ### Report options ```kotlin allure { report { // Base directory for all reports (each task gets its own subdirectory named after the task) reportDir.set(layout.buildDirectory.dir("my-reports")) // Generate a single self-contained HTML file instead of a directory singleFile.set(true) // Custom Allure 3 config file configFile.set(layout.projectDirectory.file("allurerc.mjs")) // Run test tasks before allureReport and allureServe dependsOnTests.set(true) } } ``` ### Environment variables for the Allure CLI ```kotlin allure { environment.put("JAVA_HOME", "/path/to/java_home") } ``` ## Task-level options (command line) Options shared by both `allureReport` and `allureServe`: ```shell # Run test tasks before generating the report ./gradlew allureReport --depends-on-tests ./gradlew allureServe --depends-on-tests # Use a custom Allure 3 config file (Allure 3 only) ./gradlew allureReport --config-file=allurerc.mjs ./gradlew allureServe --config-file=allurerc.mjs ``` To make `dependsOnTests` permanent without the flag, enable it in the `report` block. This applies to both `allureReport` and `allureServe`: ```kotlin allure { report { dependsOnTests.set(true) } } ``` To enable it for a single task only, configure that task: ```kotlin tasks.named("allureReport") { dependsOnTests() } ``` Options specific to `allureReport`: ```shell # Generate a single-file report ./gradlew allureReport --single-file=true # Output to a specific directory ./gradlew allureReport --report-dir=build/my-report # Delete the report directory before generating ./gradlew allureReport --clean ``` Options specific to `allureServe`: ```shell # Bind server to a specific port ./gradlew allureServe --port=8080 # Bind server to a specific host address (Allure 2 only) ./gradlew allureServe --host=192.168.1.10 ``` `--host` is not supported for Allure 3. Use `--port` only, or set `allure.version` to a `2.x` release if you need host binding. For Allure 3, `--port` is passed to the `allure open` subcommand (Allure 3 uses `generate` + `open` internally rather than a persistent `serve` process). ## Custom results directory By default, raw results go to `build/allure-results`. To change this: ```kotlin allure { adapter { resultsDir.set(layout.buildDirectory.dir("custom-allure-results")) } } ``` ## Single-file report To produce a self-contained HTML file instead of a directory: ```kotlin allure { report { singleFile.set(true) } } ``` Or for a one-off run: ```shell ./gradlew allureReport --single-file=true ``` This works for both Allure 2 and Allure 3. For `allureServe`, `singleFile` has no effect. ## Allure 3 configuration file For Allure 3, you can provide a custom config file to control report name, theme, environments, grouping, and plugins: ```kotlin allure { report { configFile.set(layout.projectDirectory.file("allurerc.mjs")) } } ``` Or per-invocation: ```shell ./gradlew allureReport --config-file=allurerc.mjs ``` When `configFile` is set, the plugin passes `--config ` to the Allure CLI and also passes `--output `. The `--output` flag takes precedence over any `output` field in your config file, so the report is always written to the task's `reportDir`. Allure 3 options such as `singleFile`, report name, and theme must be set in your config file; the Gradle-level `singleFile` property and `--single-file` flag are ignored when a custom `configFile` is in use. `configFile` is not supported for Allure 2. Setting it while using a `2.x` version fails with an error. ## Multi-project builds ### Aggregate report To generate a combined report from all subprojects, apply `io.qameta.allure-aggregate-report` to the root project and `io.qameta.allure-adapter` to each subproject that runs tests. Root `build.gradle.kts`: ```kotlin plugins { id("io.qameta.allure-aggregate-report") version "4.3.0" } repositories { mavenCentral() } ``` Each subproject's `build.gradle.kts`: ```kotlin plugins { `java` id("io.qameta.allure-adapter") version "4.3.0" } ``` Then run: ```shell ./gradlew allureAggregateReport # or view directly: ./gradlew allureAggregateServe ``` By default, `io.qameta.allure-aggregate-report` aggregates results from `allprojects` (the root project plus all subprojects). To customize which projects are included: ```kotlin // Remove all defaults and add specific projects configurations.allureAggregateReport.dependencies.clear() dependencies { allureAggregateReport(project(":module1")) allureAggregateReport(project(":module2")) } ``` ### Per-module results directories If a subproject writes results to a non-default location, configure it in that subproject: ```kotlin allure { adapter { resultsDir.set(layout.buildDirectory.dir("my-results")) } } ``` ### Adding external result directories To feed results from a directory not managed by the plugin (e.g., from a script or a non-JVM tool) into the report: ```kotlin // In a project that applies io.qameta.allure-adapter-base: plugins { id("io.qameta.allure-adapter-base") version "4.3.0" } dependencies { allureRawResultElements(files(layout.buildDirectory.dir("external-results"))) } ``` Or from the report side: ```kotlin dependencies { allureReport(files(layout.buildDirectory.dir("external-results"))) } ``` ## Allure 2 download customization (Allure 2) For Allure 2, the plugin downloads `io.qameta.allure:allure-commandline:{version}:zip` from Maven Central via Gradle dependency resolution. ### Custom download URL (air-gapped environments) ```kotlin allure { version.set("2.42.1") commandline { // Patterns: [organization], [group], [module], [version] downloadUrlPattern.set("https://my-mirror.example.com/[group]/[module]-[version].zip") } } ``` The plugin registers an exclusive Ivy repository for the resolved URL. The archive is always resolved as a zip; hardcode `.zip` in your URL pattern. ### Local allure-commandline archive To resolve Allure 2 from a local file instead of a repository: ```kotlin dependencies { allureCommandline(files("/path/to/allure-commandline.zip")) } ``` Use an absolute path — relative paths are unreliable in multi-project builds. ## Plugin breakdown The umbrella plugin `io.qameta.allure` is composed of smaller plugins you can apply individually: | Plugin ID | What it does | | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | `io.qameta.allure` | Applies `io.qameta.allure-adapter` + `io.qameta.allure-report` | | `io.qameta.allure-adapter` | Auto-instruments all `Test` and `JavaExec` tasks, adds allure-java adapters, AspectJ | | `io.qameta.allure-adapter-base` | Exposes `allureRawResultElements` configuration; no adapter autoconfiguration | | `io.qameta.allure-report` | Adds `allureReport` and `allureServe` tasks for the current project | | `io.qameta.allure-aggregate-report` | Adds `allureAggregateReport` and `allureAggregateServe` tasks; aggregates from the root project and all subprojects | | `io.qameta.allure-download` | Adds the `downloadAllure` task; provisions the Allure CLI | | `io.qameta.allure-base` | Adds the `allure {}` extension; the foundation all other plugins build on | ## Using in CI ### GitHub Actions Cache the Gradle build cache directory to avoid re-downloading the Allure CLI on every run: ```yaml - name: Run tests and generate Allure report run: ./gradlew test allureReport - name: Upload Allure report uses: actions/upload-artifact@v4 with: name: allure-report path: build/reports/allure-report/allureReport/ ``` To share the Gradle caches between runs (including the downloaded Allure CLI): ```yaml - uses: gradle/actions/setup-gradle@v4 ``` ### Jenkins Pipeline ```groovy pipeline { agent any stages { stage('Test') { steps { sh './gradlew test' } } stage('Report') { steps { sh './gradlew allureReport' publishHTML(target: [ reportDir: 'build/reports/allure-report/allureReport', reportFiles: 'index.html', reportName: 'Allure Report' ]) } } } } ``` Alternatively, if you use the [Allure Jenkins plugin](/docs/integrations-jenkins/), you can let Jenkins generate and display the report itself instead of calling `allureReport`. ### Running without running tests In CI pipelines where tests and reporting are separate stages, you may want to run only the report task against results already written to disk: ```shell ./gradlew allureReport ``` This reads existing `build/allure-results/` without re-running tests, which is the default behavior. --- ## Jenkins integration > URL: https://allurereport.org/docs/integrations-jenkins/ # Jenkins integration With the Allure Report plugin for Jenkins, you can add an "Allure Report" step to your build configuration, so that Jenkins will generate a test report automatically for each build. The plugin supports both **Allure 2** (Java-based, installed as a Jenkins global tool) and **Allure 3** (Node.js-based, resolved from PATH on the agent). You choose which version to use per job. ## Installation and configuration To enable Allure support in your Jenkins installation, do the following under the administrator account: 1. [install the plugin](#_1-install-the-plugin), 2. [set up the report generation tool](#_2-set-up-the-report-generation-tool). ### 1. Install the plugin The recommended method of installing the Allure Report plugin is via the Jenkins web interface. However, if your Jenkins configuration does not have access to the internet, you can install the plugin by uploading an HPI file to Jenkins manually. The selected installation method does not affect the functionality of the plugin. **To install the plugin from the internet directly:** 1. In the Jenkins web interface, go to **Manage Jenkins → Plugins → Available plugins**. 1. Using the search box, find the "Allure" plugin. Check the checkbox next to the plugin. 1. Click **Install without restart**. On the **Download progress** page, wait until each status is "Success". **To install the plugin via a file:** 1. On the [Releases page for Allure Report plugin](https://plugins.jenkins.io/allure-jenkins-plugin/releases/), find the version you want to choose (the latest version is recommended) and click **direct link**. Save the HPI file to a local directory on your device. 1. In the Jenkins web interface, go to **Manage Jenkins → Plugins → Advanced settings**. 1. Under the **Deploy Plugin** section, click **Browse** and select the HPI archive from your device. 1. Click the **Deploy** button to confirm the installation. On the **Download progress** page, wait until each status is "Success". After the installation, the plugin will appear in the **Manage Jenkins → Plugins → Installed plugins** section. ### 2. Set up the report generation tool The setup depends on which version of Allure you plan to use. #### Allure 3 (Node.js-based) Allure 3 expects the `allure` command to be available in the PATH on each Jenkins agent where reports will be generated. **To set up Allure 3 on Jenkins agents:** 1. On each agent that will generate Allure 3 reports, install the Allure CLI via npm: ```shell npm install -g allure ``` 2. Verify the installation: ```shell allure --version ``` Info: An Allure 3 (from PATH) installations section is available under **Manage Jenkins** → **Tools**. This section is an artifact of the Jenkins Tool management system, requires no configuration, and has no effect — the plugin always resolves the `allure` command from the system PATH automatically. You can safely ignore this section. #### Allure 2 (Java-based) Allure 2 is installed as a Jenkins global tool. Once added, Jenkins takes care of downloading it to every agent that needs to build a report. **Quick Setup (recommended for internet-connected instances):** The easiest way to configure Allure 2 is to use the Quick Setup button, which appears automatically in the job configuration whenever no Allure 2 installation has been set up yet. 1. Open any job's **Configure** page and add an **Allure Report** post-build action (see [Using Allure in a freestyle project](#using-allure-in-a-freestyle-project)). 1. Under the **Allure Report** section, if no Allure 2 installation is configured, a warning banner will appear. Click **Quick Setup (Install Latest Stable Allure 2)**. Info: The Quick Setup button is only visible to Jenkins administrators. Jenkins will download the latest stable release of Allure 2 from Maven Central and register it as a global tool. The tool will be downloaded to each agent on first use. **To install the tool manually (internet-connected):** 1. In the Jenkins web interface, go to **Manage Jenkins → Tools**. 1. Under the **Allure Commandline installations** section, click **Add Allure Commandline**. Make sure that the **Install automatically** checkbox is checked. 1. From the installer dropdown, select **From Maven Central (legacy)**. 1. Fill in the fields: - **Name** — a name to help you recognize this version of the tool, e.g., "2.39.0". - **Version** — the release of Allure Report to install. The latest version is recommended. 1. Click **Save**. **To install the tool from a custom URL (air-gapped environments):** 1. In the [allure-commandline directory at Maven Central](https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/), find the latest ZIP or TAR.GZ archive. Copy it to a web server that is available to the Jenkins server. 1. In the Jenkins web interface, go to **Manage Jenkins → Tools**. 1. Under the **Allure Commandline** section, click **Add Allure Commandline**. Make sure that the **Install automatically** checkbox is checked. 1. Click **X** to remove the existing installer block. 1. Click **Add installer → Extract \*.zip/\*.tar.gz**. 1. Fill in the fields: - **Name** — a name to help you recognize this version of the tool, e.g., "2.39.0". - **Download URL for binary archive** — the URL from which the utility will be downloaded. - **Subdirectory of extracted archive** — leave empty. 1. Click **Save**. ## Updating Allure integration Tip: Updating Allure in Jenkins requires updating not just the Allure **plugin** itself, which only manages the Jenkins integration, but also the **report generation tool**, which is the actual Allure Report software. Furthermore, we recommend updating the tool from time to time even if there are no pending plugin updates, as the tool usually gets more frequent updates than the plugin. ### 1. Update the plugin **To update via Jenkins interface:** 1. In the Jenkins web interface, go to **Manage Jenkins → Plugins**. 2. Click on the **Updates** tab. 3. If the Allure plugin appears in the list, check the checkbox next to "Allure". 4. Click **Update**. 5. On the next page, check **Restart Jenkins when installation is complete and no jobs are running**. 6. Wait for Jenkins to restart automatically. **To update manually via HPI file:** 1. Visit the [Allure Report plugin releases page](https://plugins.jenkins.io/allure-jenkins-plugin/releases/). 2. Download the latest `.hpi` file. 3. In Jenkins, go to **Manage Jenkins → Plugins → Advanced settings**. 4. Under "Deploy Plugin", click **Browse** and select your downloaded HPI file. 5. Click **Deploy**. 6. Restart Jenkins when prompted. ### 2. Update the report generation tool #### Allure 3 To update Allure 3, run the following command on each agent where it is installed: ```shell npm update -g allure ``` #### Allure 2 **If the tool was installed from the internet directly:** 1. Go to **Manage Jenkins → Tools**. 2. Find your existing Allure Commandline configuration in the **Allure Commandline** section. 3. Click on the tool name to expand its configuration. 4. Update the **Version** field to the latest Allure version (e.g., change from "2.34.0" to "2.39.0"). 5. Optionally, update the **Name** field to reflect the new version. 6. Click **Save**. **If the tool was installed from a custom URL:** 1. Go to **Manage Jenkins → Tools**. 2. Find your existing Allure Commandline configuration. 3. Update the **Download URL for binary archive** to point to the newer version at your mirror or at [Maven Central](https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/). 4. Optionally, update the **Name** field to reflect the new version. 5. Click **Save**. ## Using Allure in a freestyle project In Jenkins, a "freestyle project" is a build configuration for which you add and edit steps via the web interface. With the Allure Report plugin, the web interface gets an option to add a post-build action for building test reports. 1. In the Jenkins web interface, select a job you want to enable Allure Report for. 1. In the menu on the left, click **Configure**. Info: Before continuing, make sure that the build configuration: - contains a step that runs the project's tests, - has the Allure integration enabled for its test framework. 1. Under the **Post-build Actions** section, click **Add post-build action → Allure Report**. 1. In the **Allure Version** dropdown, select the version of Allure to use: - **Allure 2 (Java-based, auto-installed)** — uses a Jenkins global tool; requires a configured Allure Commandline installation (see [Set up the report generation tool](#_2-set-up-the-report-generation-tool)). - **Allure 3 (Node.js-based, requires 'allure' in PATH)** — uses the `allure` command from the agent's PATH; requires Allure 3 to be installed on the agent via npm. 1. In the **Results → Path** field, specify the path to the test results directory (see [How it works](/docs/how-it-works/)). Tip: If you have multiple build steps generating test results into multiple directories, use the **Add** button to specify more paths. 1. If you have more than one Allure 2 installation configured and are using Allure 2, use the **Allure CLI Installation** dropdown to select which one to use. If only one installation is configured, it is selected automatically. 1. Click **Advanced** if you need to configure any of the following: - **JDK** — the JDK to use for report generation. Applies to Allure 2 only. - **Allure Configuration** — path to a [configuration file](/docs/v3/configure/) for report generation. Applies to Allure 3 only. 1. Click **Save**. After the configuration, you can click **Build Now** to run the build configuration. ## Using Allure in a Jenkins Pipeline script If you store the job in a [Jenkins Pipeline script](https://www.jenkins.io/doc/book/pipeline/), you can use the [`allure`](https://www.jenkins.io/doc/pipeline/steps/allure-jenkins-plugin/) command to define a step for building test reports. It is recommended to use the Jenkins web interface for generating the command with the appropriate parameters. 1. In Jenkins, select a job you want to enable upload for. 1. In the menu on the left, click **Pipeline Syntax**. An interface for generating code snippets for Jenkins Pipeline will open. 1. In the **Sample Step** dropdown list, select "allure: Allure Report". 1. In the **Allure Version** dropdown, select **Allure 2** or **Allure 3** as needed. 1. In the **Results → Path** field, specify the path to the test results directory (see [How it works](/docs/how-it-works/)). Tip: If you have multiple build steps generating test results into multiple directories, use the **Add** button to specify more paths. 1. If you are using Allure 2 and you have more than one Allure 2 installation configured, use the **Allure CLI Installation** dropdown to select which one to use. 1. Click **Generate Pipeline Script**. In the area below the button, the generated command will appear. For Allure 2, for example: ```groovy allure includeProperties: false, jdk: '', results: [[path: 'build/allure-results']] ``` For Allure 3: ```groovy allure allureVersion: '3', includeProperties: false, results: [[path: 'build/allure-results']] ``` Copy the generated command into clipboard. 1. Open the Groovy script that you use for the project. It may be: - the script in the text box in **(job) → Configure → Pipeline**, - the **Jenkinsfile** under the project's version control system (see [the official documentation](https://www.jenkins.io/doc/book/pipeline/jenkinsfile/)). Info: Before continuing, make sure that the build configuration: - contains a step that runs the project's tests, - has the Allure integration enabled for its test framework. 1. Paste the generated command into the appropriate place of the script. **For Declarative Pipelines:** ```groovy // If you use the [Declarative Pipeline syntax](https://www.jenkins.io/doc/book/pipeline/syntax/#declarative-pipeline), find the stage that runs the tests and insert a new `always` block into that stage's `post` block. This will make Allure Report run after the test launch regardless of how many tests succeeded. pipeline { agent any stages { stage('Build') { steps { git 'https://github.com/eroshenkoam/allure-example.git' sh './gradlew clean test' } post { always { allure includeProperties: false, jdk: '', results: [[path: 'build/allure-results']] } } } } } ``` **For Scripted Pipelines:** ```groovy // If you use the [Scripted Pipeline syntax](https://www.jenkins.io/doc/book/pipeline/syntax/#scripted-pipeline), there are a lot of ways to organize the pipeline code in such a way that Allure Report runs after the tests, regardless of their results. One way to do so is to wrap all existing steps in a `try` statement, with the generated `allure` command placed in the `finally` block. node { stage('Build') { try { git 'https://github.com/eroshenkoam/allure-example.git' sh './gradlew clean test' } finally { allure includeProperties: false, jdk: '', results: [[path: 'build/allure-results']] } } } ``` 1. Save the script. In the case of a **Jenkinsfile**, make sure to commit the changes to the repository. --- ## JetBrains IDE integration > URL: https://allurereport.org/docs/integrations-jetbrains/ # JetBrains IDE integration (Allure 2) With the Allure plugin for the JetBrains IDEs, you can quickly generate and open test reports from the IDE's context menu. The plugin can be used in all the IDEs based on the IntelliJ platform, including IntelliJ IDEA, WebStorm, PyCharm, GoLand and others. In addition to Allure Report, the plugin also provides some functionality related to Allure TestOps. See [the Allure TestOps documentation](https://docs.qameta.io/allure-testops/ecosystem/intellij-plugin/). ## Installation and configuration Warning: Before installing the plugin, make sure that the Allure Report command-line tool is [installed in your operating system](/docs/v2/install/). 1. Launch the IDE. 1. Go to **File → Settings**. 1. In the settings navigation tree, select the **Plugins** section. 1. Go to the **Marketplace** tab. 1. In the search box, enter: `/organization:Qameta`. 1. Next to the **Allure TestOps Support** plugin, click **Install**. 1. Wait for the installation to complete and click **Apply**. 1. In the settings navigation tree, select the **Tools → Allure TestOps** section. 1. Make sure that the **Allure Report path** field contains the command or the full path to the Allure Report utility. Edit the path if necessary. 1. Click **OK**. ## Generate a test report 1. Run your tests as you would do normally. Assuming an [Allure integration](/docs/frameworks/) is configured correctly in your project, this must produce a test results directory at `allure-results`, `build/allure-results` or other location, depending on the configuration. 1. In the **Project** tool window, right-click on the test results directory and select **Allure Report: Generate**. The test report will be generated in a new directory, whose name will contain the suffix `-report-⟨DATE⟩_⟨TIME⟩`, for example: `allure-results-report-2023-05-23_08-37-17`. 1. To view the test report in a web browser, right-lick on the test report directory and select **Allure Report: Open Generated Report**. ## Automatically regenerate a test report To start a service that will regenerate the test report automatically: 1. Run your tests as you would do normally. Assuming an [Allure integration](/docs/frameworks/) is configured correctly in your project, this must produce a test results directory at `allure-results`, `build/allure-results` or other path, depending on the configuration. 1. In the **Project** tool window, right-click on the test results directory and select **Allure Report: Generate On-The-Fly**. The test report will be generated in a new directory, whose name will contain the suffix `-report`, for example: `allure-results-report`. The report will be also automatically opened in the web browser. Whenever you run your tests again, the plugin will automatically regenerate the test report. The page in the web browser will be refreshed automatically. Tip: If you use a wide monitor or multiple monitors, it may be convenient to keep the IDE and the web browser visible simultaneously while you are working on the code or the tests. This will allow you to see how your modifications affect the report. In the **Allure** tool window (**View → Tool Windows → Allure**), a list item will appear, describing the parameters of the local web server, for example: `⟨PROJECT⟩/build/allure-results → ⟨PROJECT⟩/build/allure-results-report`. Select it to enable additional actions: | Icon | Meaning | | ------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------- | | ![Allure Jetbrans IDE suspend icon](/images/integrations/jetbrains/suspend.svg "Allure Jetbrans IDE suspend icon") | Stop on the fly report generation. | | ![Allure Jetbrans IDE refresh icon](/images/integrations/jetbrains/refresh.svg "Allure Jetbrans IDE refresh icon") | Force regenerate report, regardless of whether new results are available. | | ![Allure Jetbrans IDE open report icon](/images/integrations/jetbrains/web.svg "Allure Jetbrans IDE open report icon") | Open the test report in the web browser. | | ![Allure Jetbrans IDE open report directory icon](/images/integrations/jetbrains/menu-open.svg "Allure Jetbrans IDE open report directory icon") | Open the rest report directory in the file manager. | --- ## Maven integration > URL: https://allurereport.org/docs/integrations-maven/ # Maven integration With the Allure Maven plugin, you can generate Allure reports as part of your Maven build by adding a `` block to your `pom.xml`. The plugin supports both **Allure 3** (the default, Node.js-based) and **Allure 2** (Java-based, legacy). - Requires JDK 17 or newer - Requires Maven 3.1.1 or newer ## Setting up Add the plugin to the `` section of your `pom.xml`: ```xml io.qameta.allure allure-maven 3.1.0 ``` After that, run your tests to produce Allure result files: ```shell mvn clean test ``` Your test framework's Allure adapter (e.g. `allure-jupiter`, `allure-testng`) writes raw result files to `target/allure-results/` during the test run. The plugin then reads those files and generates the report. ## Running the plugin as a build plugin To invoke Allure goals directly (e.g. `mvn allure:report`, `mvn allure:serve`), also declare the plugin in the `` section. Maven only resolves plugin prefixes from `` — a ``-only declaration supports `mvn site` but not direct invocations. ```xml io.qameta.allure allure-maven 3.1.0 ``` You can also use the `` declaration to bind a goal to a specific lifecycle phase, for example to generate the report automatically during `mvn verify`: ```xml io.qameta.allure allure-maven 3.1.0 verify report ``` ## Generating a report ### View the report locally ```shell mvn allure:serve ``` The plugin generates the report and opens it in a local web server. Press `Ctrl+C` to stop the server when done. For Allure 3, the report is written to the configured report directory (default: `target/site/allure-maven-plugin/`) before the server starts and remains there after it stops. To use a specific port: ```shell mvn -Dallure.serve.port=8080 allure:serve ``` To bind the server to a specific network interface (Allure 2 only, accepts an IPv4 address): ```shell mvn -Dreport.version=2.39.0 -Dallure.serve.host=192.168.1.10 allure:serve ``` `allure.serve.host` is not supported for Allure 3. Use `reportVersion 2.x` if host binding is required. ### Generate a static report ```shell mvn allure:report ``` The report is written to `target/site/allure-maven-plugin/index.html`. You can also generate the report as part of the standard Maven site lifecycle: ```shell mvn site ``` ## Plugin goals | Goal | Default phase | Description | | ------------------ | -------------------- | --------------------------------------------------------------- | | `allure:report` | `site` | Generates a static HTML report from `allure-results/` | | `allure:serve` | `site` | Generates a report and opens it in a local web server | | `allure:aggregate` | `site` | Aggregates results from all modules in a multi-module project | | `allure:install` | `generate-resources` | Downloads and caches the Allure CLI without generating a report | ## Selecting the runtime The `reportVersion` parameter controls which version of Allure is used to generate reports. | `reportVersion` value | Runtime used | | --------------------- | -------------------------- | | Not set (default) | Allure 3, version `3.17.0` | | `3.x` (e.g. `3.19.1`) | Allure 3 | | `2.x` (e.g. `2.39.0`) | Allure 2 | **In `pom.xml`:** ```xml 3.19.1 ``` **On the command line:** ```shell mvn -Dreport.version=3.19.1 allure:report ``` Any version string that does not start with `2.` or `3.` is rejected with an error. ## Allure 3 runtime provisioning For Allure 3, the plugin manages its own Node.js runtime. You do **not** need Node.js installed on your system. On first use, the plugin downloads Node.js and the Allure package and caches them under `${project.basedir}/.allure` (configurable via `allure.install.directory`). Subsequent runs reuse the cache and skip the download. ### Air-gapped and offline builds Maven offline mode (`mvn -o`) is supported. If the Node.js runtime and Allure package are already cached, the plugin uses them without network access. If either is missing while offline, the build fails with a clear error. To pre-populate the cache, run `mvn allure:install` once with network access before switching to offline mode. ### Custom Node.js download location To download Node.js from a mirror instead of `nodejs.org` (for example, in a restricted network): ```xml https://my-mirror.example.com/node/v%s/node-v%s-%s.%s ``` The URL pattern receives four arguments: `{version}`, `{version}`, `{os-classifier}`, `{archive-extension}`. To verify the download, the plugin also fetches `SHASUMS256.txt` from the same directory as the archive. Your mirror must serve this file next to the archive, with the matching checksum — if the file is missing or the checksum doesn't match, the installation fails. ### Custom Node.js version The default Node.js version is `24.14.1`. To use a different version: ```xml 22.0.0 ``` ### Custom npm registry ```xml https://npm.example.com ``` ### Local Allure 3 package archive To install Allure 3 from a local `.tgz` or `.tar.gz` archive instead of the npm registry (useful for air-gapped environments or testing custom builds): ```xml packages/allure-custom.tgz ``` Relative paths are resolved from the project root. This parameter is only supported for Allure 3. ## Allure 3 configuration For Allure 3, the plugin reads an `allurerc` config file to customize the report. You can provide it as an auto-detected file at the project root or point to it explicitly. ### Auto-detected config files The plugin looks for any of the following files at the project root, in this order: - `allurerc.js` - `allurerc.mjs` - `allurerc.cjs` - `allurerc.json` - `allurerc.yaml` - `allurerc.yml` The first match is used. Example `allurerc.yml`: ```yaml plugins: awesome: options: reportLanguage: en ``` ### Explicit config path Use `allure.config.path` to point to a config file anywhere on the filesystem. Relative paths are resolved from the project root. **In `pom.xml`:** ```xml config/allure3.yml ``` **On the command line:** ```shell mvn -Dallure.config.path=config/allure3.yml allure:report ``` When an explicit path is provided, auto-detection is skipped. This parameter is only supported for Allure 3. ### Keys managed by the plugin The plugin always controls `output` and `name` — setting these in your own config has no effect. Everything else in your config is preserved. Plugin-managed defaults (such as `historyPath` and `appendHistory` for history) are only applied when your config does not already set those keys, so your values always take precedence. ## Allure 2 download sources (Allure 2) For Allure 2, the plugin downloads the CLI using Maven's standard dependency resolution, so your repository mirrors, proxies, and credentials configured in `settings.xml` apply automatically. ### Custom download URL To download Allure 2 from a custom location (for example, a local mirror in a restricted network): ```xml https://my-mirror.example.com/allure-%s/allure-%s.zip 2.39.0 ``` The URL pattern receives the version string twice (once for the directory, once for the filename). This parameter is only supported for Allure 2 — use `nodeDownloadUrl` for the Allure 3 Node.js runtime instead. `allureDownloadUrl` is also respected by the `allure:install` goal when using Allure 2. ### Proxy support The plugin respects the proxy settings configured in your Maven `settings.xml`. No additional configuration is needed for Allure 2 downloads or Allure 3 Node.js and npm downloads. ## Trend history For the `allure:report` goal, the plugin preserves trend history across builds automatically. History is cached in the install directory and carried forward into each new report. History is **enabled by default**. To disable it: ```xml false ``` Or on the command line: ```shell mvn -Dallure.history.enabled=false allure:report ``` History preservation applies to `allure:report` only. The `allure:serve` and `allure:aggregate` goals are not affected. For Allure 3, if your `allurerc` file sets `historyPath` or `appendHistory`, your values take precedence over the plugin's defaults. ## Multi-module projects For multi-module Maven projects, use `allure:aggregate` to combine results from all modules into one report. In the parent `pom.xml`: ```xml io.qameta.allure allure-maven 3.1.0 aggregate true aggregate ``` Then run: ```shell mvn site ``` The aggregate goal collects `allure-results/` from each child module and produces a single combined report in the parent's `target/site/allure-maven-plugin/`. You can also invoke aggregate directly without the site lifecycle: ```shell mvn -Dreport.version=2.39.0 allure:aggregate ``` ### Per-module results directory If a module writes results to a non-default location, configure it using the `allure.results.directory` property in that module's `pom.xml`: ```xml my-results ``` The aggregate goal reads this property from each module's effective POM. The value must be a relative path (relative to the module's build directory) — absolute paths are rejected. ## Advanced configuration ### Custom results directory By default, the plugin reads results from `target/allure-results/`. To change it: ```xml my-results ``` Or on the command line: ```shell mvn -Dallure.results.directory=my-results allure:report ``` The value is resolved relative to the Maven build directory (`target`), so `my-results` maps to `target/my-results`. Use an absolute path to point outside the build directory. ### Multiple input directories (`allure:report` only) (Allure 2) To merge results from multiple directories into one report: ```shell mvn -Dallure.results.inputDirectories=/path/one,/path/two allure:report ``` ### Custom report output directory ```xml ${project.build.directory}/my-report ``` Or on the command line: ```shell mvn -Dallure.report.directory=target/my-report allure:report ``` ### Single-file report To generate a self-contained single HTML file instead of a directory: ```xml true ``` This works for both Allure 2 and Allure 3. For `allure:serve`, `singleFile` is ignored — use `allure:report` or `allure:aggregate` to produce a single-file report. Allure 3 serve logs a warning when `singleFile` is set; Allure 2 serve silently ignores it. ### Report generation timeout The default timeout for report generation is 60 seconds. To extend it: ```xml 300 ``` The `allure:serve` goal uses a separate timeout (default: 3600 seconds, one hour) that covers both generation and the time the server is running: ```xml 7200 ``` ### Allure properties The plugin passes properties to Allure at report generation time. You can use these to configure things like issue tracker URL patterns. **From a properties file** (default lookup path: `report.properties` in the project root, i.e. the directory from which you run Maven): ```properties allure.issues.tracker.pattern=https://jira.example.com/browse/%s allure.link.tms.pattern=https://tms.example.com/case/%s ``` **From plugin configuration:** ```xml https://jira.example.com/browse/%s ``` **From classpath** (both `allure.properties` and `report.properties` are discovered automatically from the test classpath). Properties from all sources are merged. Plugin configuration properties take highest precedence. Placeholder values like `${project.version}` in property values are expanded using Maven and system properties. ### Categories Place a `categories.json` file on the test classpath (e.g. under `src/test/resources/`) to define custom test failure categories. The plugin copies it into the results directory before invoking Allure. This works for both Allure 2 and Allure 3. ### Install directory The default install directory for the Allure runtime is `${project.basedir}/.allure`. To share one installation across multiple projects or put it outside the project tree: ```xml /opt/allure-cache ``` Or on the command line: ```shell mvn -Dallure.install.directory=/opt/allure-cache allure:report ``` ## Using in CI ### GitHub Actions In a GitHub Actions workflow, run your tests and then generate the Allure report. The install directory is cached between workflow runs to avoid re-downloading Node.js and the Allure package on every run. ```yaml - name: Run tests run: mvn test - name: Cache Allure runtime uses: actions/cache@v4 with: path: .allure key: allure-${{ hashFiles('pom.xml') }} - name: Generate Allure report run: mvn allure:report - name: Upload Allure report uses: actions/upload-artifact@v4 with: name: allure-report path: target/site/allure-maven-plugin/ ``` ### Jenkins Pipeline ```groovy pipeline { agent any stages { stage('Test') { steps { sh 'mvn clean test' } } stage('Report') { steps { sh 'mvn allure:report' publishHTML(target: [ reportDir: 'target/site/allure-maven-plugin', reportFiles: 'index.html', reportName: 'Allure Report' ]) } } } } ``` Alternatively, if you use the [Allure Jenkins plugin](/docs/integrations-jenkins/), you can let Jenkins generate and display the report itself instead of invoking `allure:report` in the pipeline. ### Sharing the Allure install directory across builds On CI agents that run multiple projects, point all of them at a shared install directory to avoid redundant downloads: ```shell mvn -Dallure.install.directory=/var/cache/allure allure:report ``` Or pre-populate the cache once per agent using the `allure:install` goal: ```shell mvn -Dallure.install.directory=/var/cache/allure allure:install ``` --- ## TeamCity integration > URL: https://allurereport.org/docs/integrations-teamcity/ # TeamCity integration (Allure 2) With the Allure Report plugin for TeamCity, you can add an “Allure Report” step to your build configuration, so that TeamCity will generate a test report automatically for each build. ## Installation and configuration To enable Allure support in your TeamCity installation, do the following under the administrator account: 1. [install the plugin](#_1-install-the-plugin), 2. [install the command-line tool](#_2-install-the-command-line-tool), 3. [configure serving artifacts](#_3-configure-serving-artifacts) (optional). ### 1. Install the plugin The recommended method of installing the Allure Report plugin is to use the one-click installation feature in JetBrains Marketplace. However, if your TeamCity configuration does not have access to the internet, you can install the plugin by uploading a ZIP archive to TeamCity manually. The selected installation method does not affect the functionality of the plugin. **To install the plugin from JetBrains Marketplace directly:** 1. In the TeamCity web interface, go to **Administration → Plugins** and click **Browse plugins repository**. 1. In the dialog that appears, click **Proceed**. This will enable the one-click installation feature for JetBrains Marketplace in your browser. 1. On the [Allure Report plugin page in JetBrains Marketplace](https://plugins.jetbrains.com/plugin/20544-allure-report), click **Get → Install to ⟨hostname⟩**. 1. Click **Install** to confirm the installation. 1. Click **Enable uploaded plugin**. 1. In the dialog that appears, click **Enable** to confirm the plugin activation. **To install the plugin via a ZIP archive:** 1. On the [Allure Report plugin page in JetBrains Marketplace](https://plugins.jetbrains.com/plugin/20544-allure-report), click **Get → Download**. Save the ZIP archive to a local directory on your device. 1. In the TeamCity web interface, go to **Administration → Plugins** and click **Upload plugin zip**. 1. In the dialog that appears, select the ZIP archive from your device. 1. Click the **Upload plugin zip** button to confirm the installation. 1. Click **Enable uploaded plugin**. 1. In the dialog that appears, click **Enable** to confirm the plugin activation. After the installation, the plugin will appear in the list under the **External plugins** section. ![Allure TeamCity plugin install](/images/integrations/teamcity/allure-plugin.png "Allure TeamCity plugin install") ### 2. Install the command-line tool The second necessary part of the installation process is adding the Allure command-line utility to TeamCity as an “agent tool” (see the [official TeamCity documentation](https://www.jetbrains.com/help/teamcity/installing-agent-tools.html)). Once added, TeamCity will take care of distributing the utility to every machine (“agent”) that needs to build a test report. The Allure Report plugin provides an easy way to download and add the latest version of the command-line tool from the official Allure server. However, if your TeamCity configuration does not have access to the internet, you can add the tool by uploading a ZIP archive to TeamCity manually. The selected installation method does not affect the functionality of the plugin. **To install the tool from the internet directly:** 1. In the TeamCity web interface, go to **Administration → Tools** and click **Install Tool**. 1. In the dialog that appears, select the **Allure** option. 1. Select the **Download** option. 1. Select the latest version of the Allure command-line utility from the dropdown list. 1. Click **Add**. **To install the tool via a ZIP archive:** 1. In the [allure-commandline directory at Maven Central](https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/), find and download the latest ZIP archive to a local directory on your device. Warning: Make sure to save the archive as **allure-commandline-⟨version⟩.zip**, for example: **allure-commandline-2.24.0.zip**. TeamCity may not recognize the command-line tool correctly if it has a different name. 1. In the TeamCity web interface, go to **Administration → Tools** and click **Install Tool**. 1. In the dialog that appears, select the **Allure** option. 1. Select the **Upload** option. 1. Select the ZIP archive from your device. 1. Click **Add**. After the installation, the version of the command-line tool will appear in the list under the **Allure** section. ![Allure TeamCity plugin configure Allure Commandline](/images/integrations/teamcity/allure-commandline.png "Allure TeamCity plugin configure Allure Commandline") ### 3. Configure serving artifacts Info: This step is required if you prefer the test reports to be available for browsing without downloading. If you prefer them to be available only as downloadable ZIP archives, skip this step. By default, TeamCity does not allow browsing the HTML artifacts generated by external plugins. This principle is called “artifacts domain isolation” (see the [official TeamCity documentation](https://www.jetbrains.com/help/teamcity/2022.10/teamcity-configuration-and-maintenance.html#artifacts-domain-isolation)). However, if you configure an additional domain for this purpose, TeamCity will let users browse the artifacts on that domain. This is the recommended approach. Alternatively, you can disable the artifacts domain isolation. In this case, TeamCity will let users browse the HTML artifacts, including Allure test reports, on the main domain of the TeamCity installation. **To enable viewing test reports on a separate domain:** 1. In the TeamCity web interface, go to **Administration → Global Settings**. 1. Under the **Artifacts Domain Isolation** section, check the **Enable isolation protection** checkbox. Warning: This setting applies to all the projects in your TeamCity installation, which may be considered unsafe for some projects. 1. Enter the **Artifacts URL**. Make sure to use a domain that is configured to point to the same IP address as the main TeamCity server URL. 1. Click **Save**. **To enable viewing test reports on the TeamCity domain:** 1. In the TeamCity web interface, go to **Administration → Global Settings**. 1. Under the **Artifacts Domain Isolation** section, uncheck the **Enable isolation protection** checkbox. 1. Click **Save**. ## Using Allure in a UI-based build configuration 1. In the TeamCity web interface, go to **Projects**. 1. In the tree on the left, select a project and a build configuration. 1. On the top of the page, click **Edit configuration**. 1. Select the **Build Steps** section. Info: Before continuing, make sure that the build configuration: - contains a step that runs the project's tests, - has the Allure integration enabled for its test framework. 1. Click **Add build step**. 1. On the runner type selection screen, select **Allure Report**. 1. Fill the build step creation form. - **Step name** — any name to identify the report building step, e.g., “Generate test report”. - **Execute step** — “Even if some of the previous steps failed”. - **Result directory** — the path to the directory in which the Allure integration test framework saves the test results. Most Allure integrations save the test results into `allure-results` unless specifically configured otherwise. For more details, check the “Run tests” section of the documentation for your [Allure integration](/docs/frameworks/). - **Report directory** — the name of the directory in which Allure should generate the HTML report. Note that the directory name cannot contain the slash character. - **Allure** — the version of the Allure command-line tool to build the test report with. The list of available options includes the versions installed on the [step 2](#_2-install-the-command-line-tool). It is recommended to use the latest available version. - **Publish mode**: - “Publish archived report” — Allure will create an `allure-report.zip` archive containing a directory which will be named according to the **Report directory** setting. - “Publish plain report” — Allure will create a directory named according to the **Report directory** setting. 1. Click **Save**. After the configuration, you can click **Run** to run the build configuration. ## Using Allure in a DSL-based build configuration If you use the [Configuration as Code](https://www.jetbrains.com/teamcity/features/configuration-as-code/) approach and store the build configuration in a [Kotlin DSL](https://www.jetbrains.com/help/teamcity/2023.05/kotlin-dsl.html) script, you can enable building test reports by copying a new `step` object into your `.teamcity/settings.kts` file. The new step must be placed after all the steps that run the tests. Info: Before continuing, make sure that the build configuration: - contains a step that runs the project's tests, - has the Allure integration enabled for its test framework. Inside the `step` object, define the properties: - `name` — any name to identify the report building step, e.g., “Generate test report”. - `type` — `allureReportGeneratorRunner`. - `executionMode` — `BuildStep.ExecutionMode.RUN_ON_FAILURE`. Use the `param()` function to set additional parameters for the Allure Report step. If a parameter is not set, its default value will be used. - `allure.version` — the version of the Allure command-line tool to build the test report with. It is recommended to use the latest available version. Set the parameter to `%teamcity.tool.allure-⟨version⟩%` to use a specific version of the command-line utility, foe example: `%teamcity.tool.allure-2.24.0%`. Skip this parameter or set it to `%teamcity.tool.allure.DEFAULT%` to use the version that is selected as the default one in the **Administation → Tools** section. - `allure.result.directory` — the path to the directory in which the Allure integration test framework saves the test results. Most Allure integrations save the test results into `allure-results` unless specifically configured otherwise. For more details, check the “Run tests” section of the documentation for your [Allure integration](/docs/frameworks/). Defaults to `allure-results`. - `allure.report.path.prefix` — the name of the directory in which Allure should generate the HTML report. Note that the directory name cannot contain the slash character. Defaults to `allure-report`. - `allure.publish.mode`: - `ARCHIVE` — Allure will create an `allure-report.zip` archive containing a directory which will be named according to the `allure.report.path.prefix` setting. - `PLAIN` — Allure will create a directory named according to the `allure.report.path.prefix` setting. Defaults to `ARCHIVE`. Here's an example of a project's `.teamcity/settings.kts` file: ```kotlin import jetbrains.buildServer.configs.kotlin.* import jetbrains.buildServer.configs.kotlin.buildSteps.python version = "2022.10" project { buildType(Build) } object Build : BuildType({ name = "Build" vcs { root(DslContext.settingsRoot) } steps { python { environment = venv {} command = pytest {} } step { name = "Generate test report" type = "allureReportGeneratorRunner" executionMode = BuildStep.ExecutionMode.RUN_ON_FAILURE param("allure.version", "%teamcity.tool.allure-2.24.0%") param("allure.result.directory", "allure-results") param("allure.report.path.prefix", "allure-report") param("allure.publish.mode", "PLAIN") } } }) ``` --- ## Visual Studio Code integration > URL: https://allurereport.org/docs/integrations-vscode/ # Visual Studio Code integration (Allure 2) With the Allure extension for the Visual Studio Code, you can quickly generate and open test reports from its context menu. The extension also provides a way to quickly upload test results to your Allure TestOps account, see the [Allure TestOps documentation](https://docs.qameta.io/allure-testops/ecosystem/vscode-plugin/). ## Installation Warning: Before installing the extension, make sure that the Allure Report command-line tool is [installed in your operating system](/docs/v2/install/). 1. Launch Visual Studio Code. 1. Open the **Extensions** view. 1. In the **Search Extensions in Marketplace** field, enter `qameta.allure-vscode`. 1. Click on the **Allure Support** extension, then click **Install**. 1. Wait for the installation to complete. ## Generating and serving test reports The extension provides three operations. - **[Generate and serve a test report](#generate-and-serve-a-test-report)**: given the test results, view a report in the browser. - [Just generate a test report](#generate-a-test-report): given the test results, generate a report directory. - [Just serve an existing test report](#serve-an-existing-test-report): given the report directory, view it in the browser. ### Generate and serve a test report 1. Run your tests as you would do normally. Assuming an [Allure integration](/docs/frameworks/) is configured correctly in your project, this must produce a test results directory at `allure-results`, `build/allure-results` or other location, depending on the configuration. 1. In the **Explorer** view, right-click on the test results directory and select **Allure Report: serve test results**. The test report will be generated in a temporary directory (outside of your project). A local web server for the directory will be automatically started. 1. In the **Allure** view, find the item with the same name as the test results directory. Hover the item to see the available actions: | Action | Description | | ------------------------------------------------------------------------------------- | ------------------------------------------------ | | ![Open browser](/images/integrations/vscode/open-browser.png "Open browser action") | Open the test report in the default web browser. | | ![Shutdown server](/images/integrations/vscode/shutdown.png "Shutdown server action") | Stop the local web server. | ### Generate a test report 1. Run your tests as you would do normally. Assuming an [Allure integration](/docs/frameworks/) is configured correctly in your project, this must produce a test results directory at `allure-results`, `build/allure-results` or other location, depending on the configuration. 1. In the **Explorer** view, right-click on the test results directory and select **Allure Report: generate report**. The test report will be generated in a new directory named `allure-report-⟨DATE⟩_⟨TIME⟩`, for example: `allure-report-2023-05-23_08-37-17`. 1. To start a local server for viewing the report, right-click on the test report directory and select **Allure Report: serve generated report**. ### Serve an existing test report 1. Make sure that you have a previously generated directory with the HTML test report. 1. In the **Explorer** view, right-click on the report directory and select **Allure Report: serve generated report**. 1. In the **Allure** view, find the item with the same name as the report directory. Hover the item to see the available actions: | Action | Description | | ------------------------------------------------------------- | ------------------------------------------------ | | ![Open browser](/images/integrations/vscode/open-browser.png) | Open the test report in the default web browser. | | ![Shutdown server](/images/integrations/vscode/shutdown.png) | Stop the local web server. | --- # Section: Frameworks ## AVA > URL: https://allurereport.org/docs/ava/ # Getting started with Allure AVA [![Allure AVA npm latest version](https://img.shields.io/npm/v/allure-ava?style=flat "Allure AVA npm latest version")](https://www.npmjs.com/package/allure-ava) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [AVA](https://avajs.dev/) tests. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure AVA requires AVA 8 or later. AVA 8 requires Node.js `^22.20`, `^24.12`, or `>=26`. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure AVA integration: **npm:** ```bash npm install --save-dev allure-ava ava ``` **yarn:** ```bash yarn add --dev allure-ava ava ``` **pnpm:** ```bash pnpm add -D allure-ava ava ``` 1. Call `installAllure()` from your AVA configuration file. For an ESM config (`ava.config.mjs`): ```js // ava.config.mjs import { installAllure } from "allure-ava"; await installAllure({ resultsDir: "allure-results", }); export default {}; ``` If your project does not use `"type": "module"` in its `package.json`, keep the configuration in CommonJS — name the file `ava.config.js` and export an async factory: ```js // ava.config.js module.exports = async () => { const { installAllure } = await import("allure-ava"); await installAllure({ resultsDir: "allure-results", }); return {}; }; ``` Note that AVA only reads `ava.config.js` and `ava.config.mjs`; a file named `ava.config.cjs` is silently ignored. Your test files themselves do not need any changes. Optionally, specify `resultsDir` and other options. See [Allure AVA configuration](/docs/ava-configuration/) for more details. Tip: If the tests run but the results directory is not created and the console prints `no test runtime is found`, AVA did not load your configuration file — check its name and location. ### 2. Run tests Run your AVA tests the same way you would run them usually: ```bash npx ava ``` This will save the necessary data into `allure-results` or another directory, according to the [configuration](/docs/ava-configuration/). If the directory already exists, new files will be added to the existing ones so that a future report will be based on them all. Tests declared with `test.skip()` or `test.todo()`, as well as tests excluded by `test.skipIf()` and `test.runIf()` conditions, appear in the report with the **Skipped** status. Tests declared with `test.failing()` follow AVA's own semantics: when such a test fails as expected, it counts as successful and appears in the report with the **Passed** status; if it unexpectedly passes, it is reported as **Broken** with AVA's "Test was expected to fail, but succeeded" error. If an AVA worker process crashes, times out, or exits unexpectedly, the tests affected by it appear in the report with the **Broken** status. You can still pass regular AVA CLI arguments: ```bash npx ava test/**/*.test.js --match "signing in" ``` ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure AVA integration extends the standard reporting features of AVA by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization, see [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Capture [files and other content](#attach-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure AVA provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@allure.`) into the test name. Allure AVA will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ### Add metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/ava-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.displayName("Test Authentication"); await allure.owner("John Doe"); await allure.tags("Web interface", "Authentication"); await allure.severity("critical"); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test( "Test Authentication" + " @allure.label.owner:JohnDoe" + " @allure.label.tag:WebInterface" + " @allure.label.tag:Authentication" + " @allure.label.severity:critical", (t) => { // ... t.pass(); }, ); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", (t) => { // ... t.pass(); }, ); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", (t) => { // ... t.pass(); }, ); ``` ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), you can use the `step()` function, see the [reference](/docs/ava-reference/#test-steps). ```js import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); t.pass(); }); ``` ### Describe parametrized tests To implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern in AVA, write the test inside a loop and use the variable parameters in both the title and the body. Each iteration is treated as a separate test run by Allure Report. To display a parameter value in the test report, pass it to the [`parameter()`](/docs/ava-reference/#parametrized-tests) function. ```js import * as allure from "allure-js-commons"; import test from "ava"; for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async (t) => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... t.pass(); }); } ``` ### Set labels globally Any [labels](/docs/ava-reference/#label), including custom ones, can be set via environment variables in your operating system. Here's an example: **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface npx ava ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" npx ava ``` ### Attach files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. Additionally, any messages logged with AVA's built-in `t.log()` are automatically added to the test result (or to the corresponding fixture, when logged in a hook) as a text attachment named "AVA logs". For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure AVA reference](/docs/ava-reference/#attachments). ```js import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); t.pass(); }); ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, AVA will only run tests listed in this file. Tests can be matched in two ways: - **By selector** — using the `path/to/file.test.js#test title` format. The path is relative to the project root (the closest directory with a `package.json`), and the title is the test name with any `@allure.*` metadata tags stripped out. - **By Allure ID** — using the `@allure.id:⟨VALUE⟩` metadata tag in the test name. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json npx ava ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" npx ava ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, edit the [`environmentInfo`](/docs/ava-configuration/#environmentinfo) object in the configuration. For example, it is a good idea to record the OS version and the Node.js version. This may help the future reader investigate bugs that are reproducible only in some environments. ```js // ava.config.mjs import { installAllure } from "allure-ava"; import * as os from "node:os"; await installAllure({ environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }); export default {}; ``` --- ## AVA configuration > URL: https://allurereport.org/docs/ava-configuration/ # Allure AVA configuration The [Allure AVA](/docs/ava/) integration behavior can be configured by passing an options object to `installAllure()` in your AVA configuration file. For example: ```js // ava.config.mjs import { installAllure } from "allure-ava"; import { Status } from "allure-js-commons"; import * as os from "node:os"; await installAllure({ resultsDir: "allure-results", links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: "https://jira.example.com/browse/%s", }, }, categories: [ { name: "Assertion failures", matchedStatuses: [Status.FAILED], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, globalLabels: { layer: "api", }, }); export default {}; ``` ## resultsDir Path to the directory where Allure AVA will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ## links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/ava-reference/#link)), you can specify the following: - `nameTemplate` — a template for generating the link name when it is not provided. - `urlTemplate` — a template for generating the full link address from a short identifier. Each template is either a string containing `%s` at the position where the identifier should be placed, or a function that accepts the identifier and returns the resulting string. For example, with the configuration above, `await allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/123`. Allure will display the link with the appropriate icon for the `issue` type. ## categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `description` — a category description, in plain text. - `descriptionHtml` — a category description, in HTML format. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). Warning: This option works with Allure Report 2, which reads the `categories.json` file that Allure AVA writes into the results directory. Allure Report 3 ignores that file — define the categories in the [Allure Report 3 configuration file](/docs/categories/) instead. ## environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ## globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ name, value }` label objects: ```js await installAllure({ globalLabels: [ { name: "layer", value: "api" }, { name: "team", value: "backend" }, ], }); ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```js await installAllure({ globalLabels: { layer: "api", team: "backend", }, }); ``` --- ## AVA reference > URL: https://allurereport.org/docs/ava-reference/ # Allure AVA reference These are the functions that you can use to integrate your AVA tests with Allure. In most cases, Allure AVA provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@allure.`) into the test name. Allure AVA will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. A metadata tag consists of a name and a value separated by `:` or `=`. If the value must contain spaces, wrap it in single, double, or backtick quotes, for example: `@allure.label.owner:"John Doe"`. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `allure.displayName(name: string): PromiseLike` Set the test's [title](/docs/v3/readability/#title). ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.displayName("Test Authentication!"); // ... t.pass(); }); ``` ### Description - `allure.description(markdown: string): PromiseLike` - `allure.descriptionHtml(html: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Use `description()` for Markdown content. For security purposes, any HTML formatting in it is not rendered: depending on the Allure Report version, the tags are either stripped or displayed as plain text. Use `descriptionHtml()` when you need to supply raw HTML directly instead of Markdown. ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.description("This test attempts to log into the website."); // ... t.pass(); }); ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.owner("John Doe"); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test("Test Authentication @allure.label.owner:JohnDoe", (t) => { // ... t.pass(); }); ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` - `@allure.label.tag:⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.tag("New UI"); await allure.tags("Essentials", "Authentication"); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test("Test Authentication @allure.label.tag:WebInterface @allure.label.tag:Authentication", (t) => { // ... t.pass(); }); ``` ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: `"trivial"`, `"minor"`, `"normal"`, `"critical"`, and `"blocker"`. **Runtime API:** ```js import * as allure from "allure-js-commons"; import { Severity } from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.severity(Severity.CRITICAL); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test("Test Authentication @allure.label.severity:critical", (t) => { // ... t.pass(); }); ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` - `@allure.label.⟨NAME⟩:⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for many of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.label("microservice", "UI"); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test("Test Authentication @allure.label.microservice:UI", (t) => { // ... t.pass(); }); ``` ### ID - `allure.allureId(value: string): PromiseLike` - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). The ID is used for test plan filtering by Allure TestOps. **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.allureId("123"); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test("Test Authentication @allure.id:123", (t) => { // ... t.pass(); }); ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` - `@allure.link.⟨TYPE⟩:⟨VALUE⟩` Add a [link](/docs/v3/readability/#links) related to the test. If the `url` is not a full URL (for example, it is just an issue identifier), Allure will try to construct the full address from a **link template** that corresponds to the `type` (which can be any string and defaults to `"link"`), as defined by the [`links`](/docs/ava-configuration/#links) configuration option. If no template is found for the given type, or the value is already a full URL, it is left unmodified. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.issue("AUTH-123", "Related issue"); await allure.tms("TMS-456", "Related TMS issue"); await allure.link("JIRA-777", "Related Jira issue", "jira"); await allure.link("https://example.com/", "Project website"); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test("Test Authentication @allure.link.issue:AUTH-123 @allure.link.tms:TMS-456", (t) => { // ... t.pass(); }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", (t) => { // ... t.pass(); }, ); ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). By default, Allure AVA does not assign suite labels to tests; the default grouping in the report's tree view is based on the test file path. Use these functions to organize tests into an explicit suite hierarchy. **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... t.pass(); }); ``` **Metadata API:** ```js import test from "ava"; test( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", (t) => { // ... t.pass(); }, ); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test step or sub-step](/docs/steps/) with the given `name`. The `step()` function accepts an asynchronous anonymous function as its second argument. The anonymous function can accept either no arguments or a single argument of type `StepContext`. This object provides the following methods: - `displayName()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are similar to the test-wide implementation, see [Parametrized tests](#parametrized-tests). To create a step without a body, call the `logStep()` function that accepts a name and an optional step status. ```js import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); t.pass(); }); ``` ## Hooks AVA hooks declared with `test.before()`, `test.beforeEach()`, `test.after()`, and `test.afterEach()` (including their `.always` variants) are reported as the test's setup and teardown **fixtures**, including their statuses and durations. If a hook throws, it is shown as a failed or broken fixture, and the tests it blocks are reported as broken. You can use Allure's functions inside hooks: - Labels, links, and parameters added in `before()` and `beforeEach()` hooks are applied to the test results they run for. Metadata added in `after()` and `afterEach()` hooks does not affect the test results. - Steps and attachments created in a hook are displayed inside the corresponding fixture. ```js import * as allure from "allure-js-commons"; import test from "ava"; test.beforeEach(async () => { await allure.parameter("database", "postgres"); }); test("Test Authentication", (t) => { // ... t.pass(); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` The [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern in AVA can be implemented by running a `test()` inside a loop. Each iteration is treated as a separate test run by Allure Report. The values that distinguish one iteration from another are called test parameters. To display a parameter value in the test report, pass it to the `parameter()` function. The `options` argument, if given, must be an object with two optional properties: `excluded` and `mode`. - If `excluded` is set to `true`, Allure will not use the parameter when comparing the current test result with previous ones in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. Note that even when you use the `"masked"` or `"hidden"` mode, it is still possible to extract the value from the `allure-results` directory if you publish it. ```js import * as allure from "allure-js-commons"; import test from "ava"; for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async (t) => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... t.pass(); }); } ``` ## Attachments - `allure.attachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` directly or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options in an object (as shown for the image attachment below) or just specify the media type and let Allure deduce the appropriate filename extension automatically (as shown for the text attachment below). In either case, the media type can be a value from the `ContentType` enumeration or any string. ```js import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import test from "ava"; test("Test Authentication", async (t) => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); t.pass(); }); ``` Additionally, any messages logged with AVA's built-in `t.log()` are automatically added to the test result (or to the corresponding fixture, when logged in a hook) as a text attachment named "AVA logs". ## Global attachments and errors - `allure.globalAttachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.globalAttachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` - `allure.globalError(details: StatusDetails): PromiseLike` Attach data or report an error that relates to the test run as a whole rather than to a specific test. Call these functions outside of any test or hook — for example, at the top level of a test file. The `globalAttachment()` and `globalAttachmentPath()` functions accept the same arguments as their test-level counterparts, see [Attachments](#attachments). ```js import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import test from "ava"; await allure.globalAttachment( "Run configuration", JSON.stringify({ region: "eu-west-1" }), ContentType.JSON, ); test("Test Authentication", (t) => { // ... t.pass(); }); ``` ## Sync API All functions in this reference are asynchronous and return a `PromiseLike`. If your test uses synchronous helpers or matcher integrations that do not support async, import from `allure-js-commons/sync` instead: ```js import * as allure from "allure-js-commons/sync"; import test from "ava"; test("Test Authentication", (t) => { allure.step("check result", () => { allure.parameter("mode", "sync"); }); t.pass(); }); ``` The sync facade is strict-sync only: `step()` must complete synchronously and must not return a `Promise`. --- ## Axios > URL: https://allurereport.org/docs/axios/ # Getting started with Allure Axios [![Allure Axios npm latest version](https://img.shields.io/npm/v/allure-axios?style=flat "Allure Axios npm latest version")](https://www.npmjs.com/package/allure-axios) Capture [Axios](https://axios-http.com/) HTTP requests and responses as attachments in your [Allure Report](https://allurereport.org/docs/) test reports. Allure Axios is compatible with any Allure JS framework integration, including [Allure Vitest](/docs/vitest/), [Allure Jest](/docs/jest/), [Allure Playwright](/docs/playwright/), [Allure Mocha](/docs/mocha/), and others. To enable the integration in your project: 1. Make sure you have an Allure integration set up for the test framework you use. See the instructions in the integration's documentation in [Frameworks](/docs/frameworks/). 1. Install `allure-axios`. **npm:** ```bash npm install --save-dev allure-axios ``` **yarn:** ```bash yarn add --dev allure-axios ``` **pnpm:** ```bash pnpm add -D allure-axios ``` 1. For every Axios instance used in your tests, call `instrumentAxios()` to start capturing requests. `instrumentAxios()` installs interceptors on the instance and returns a cleanup function. Call the cleanup function when you no longer want requests captured — typically in `afterEach` or `afterAll`. The example below shows usage in a Vitest test suite. ```ts import { instrumentAxios } from "allure-axios"; import axios, { type AxiosInstance } from "axios"; import { afterEach, beforeEach, describe, expect, it } from "vitest"; describe("Orders API", () => { let client: AxiosInstance; let restore: () => void; beforeEach(() => { client = axios.create({ baseURL: "https://api.example.com" }); restore = instrumentAxios(client); }); afterEach(() => { restore(); }); it("creates an order", async () => { const response = await client.post("/orders", { item: "book", qty: 1 }); expect(response.status).toBe(201); }); }); ``` Every HTTP request made through `client` will be added to the test result as an [HTTP exchange attachment](/docs/attachments/#http-exchanges), wrapped in a step of the same name. The attachment captures the full exchange in one place: the request method, URL, headers, query parameters, cookies, and body; the response status, headers, cookies (including attributes such as `Path`, `Domain`, `Expires`, `Secure`, `HttpOnly`, and `SameSite`), and body; the exchange start and stop timestamps; and, when the request fails, the error name and message. See the [attachment schema reference](/docs/axios-reference/#attachment-schema) for the exact field list. Info: To prevent secrets from leaking into reports, certain values are redacted by default — all cookies, any header or query parameter whose name suggests it contains a credential, and any URL-encoded form field whose name does the same. See [Allure Axios configuration](/docs/axios-configuration/) for the full list of defaults and how to customize the redaction rules. 1. Run your tests and generate a test report the same way as you would normally. --- ## Axios configuration > URL: https://allurereport.org/docs/axios-configuration/ # Allure Axios configuration This page describes the options that affect the behavior of the [Allure Axios](/docs/axios/) integration. All options are passed as the second argument to `instrumentAxios()`: ```ts instrumentAxios(client, { // options here }); ``` `instrumentAxios()` validates its first argument and throws `"allure-axios requires an Axios instance"` immediately if it is not a valid Axios instance. ## Customize the attachment name - `attachmentName: string | ((exchange: HttpExchange) => string)` Specify the name under which Allure Axios creates the attachment for each HTTP exchange. For the shape of the `HttpExchange` argument, see [Allure Axios reference](/docs/axios-reference/). By default, each attachment is named after the request method and full URL — for example, `GET https://api.example.com/orders`. The URL used here is the raw, resolved URL, including any query parameters in their original (unredacted) form. Redaction only applies to the structured fields inside the attachment (`query`, `headers`, `cookies`, `form`), not to the URL string in the attachment name or in `exchange.request.url`. To use the same static name for all attachments, pass a string: ```ts instrumentAxios(client, { attachmentName: "API request", }); ``` To derive the name dynamically from the exchange data, pass a function: ```ts instrumentAxios(client, { attachmentName: (exchange) => `${exchange.request.method} ${new URL(exchange.request.url).pathname}`, }); ``` ## Control body capture - `captureRequestBody: boolean` (default: `true`) - `captureResponseBody: boolean` (default: `true`) - `maxBodySize: number` (default: `65536`) By default, Allure Axios captures both the request and response body. Set `captureRequestBody` or `captureResponseBody` to `false` to skip one or both: ```ts instrumentAxios(client, { captureRequestBody: false, captureResponseBody: false, }); ``` `maxBodySize` sets the maximum number of bytes to capture from any single body. Bodies that exceed this limit are captured up to the limit and flagged as truncated in the attachment. The default is 64 KiB (65 536 bytes). ```ts instrumentAxios(client, { maxBodySize: 1024 * 1024, // 1 MiB }); ``` Any value less than `1` is treated as `0`, which captures body metadata but stores an empty `value`. For details on how captured bodies are encoded and what the attachment fields contain, see [Allure Axios reference](/docs/axios-reference/). ## Redaction Allure Axios replaces sensitive values with the sentinel string `__ALLURE_REDACTED__` before writing the attachment. The defaults are: | Category | Redacted by default | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Headers | `authorization`, `cookie`, `set-cookie`, and any header whose name contains `token`, `password`, `passwd`, `secret`, `api-key`/`api_key`/`apikey` (one pattern), or `session` | | Query parameters | Any parameter whose name contains the same patterns listed above | | Cookies | All cookies | | URL-encoded form fields | Any field whose name contains the same patterns listed above | The following options let you override the defaults for each category. Each accepts an array of `RedactionMatcher` values — strings, regular expressions, or functions. Providing an array replaces the defaults for that category entirely. See [Allure Axios reference](/docs/axios-reference/) for the full `RedactionMatcher` and `RedactionContext` type documentation. - `redactHeaders: RedactionMatcher[]` - `redactQueryParams: RedactionMatcher[]` - `redactCookies: RedactionMatcher[]` - `redactFormFields: RedactionMatcher[]` ```ts instrumentAxios(client, { // Only redact headers explicitly listed, nothing else redactHeaders: ["authorization", "x-api-key"], // Allow cookies through unredacted redactCookies: [], }); ``` ## Error options - `includeErrorStack: boolean` (default: `false`) When a request fails — whether due to a transport error, a cancellation, or an HTTP error response — Allure Axios records the error's name and message in the attachment. Set `includeErrorStack` to `true` to also capture the stack trace: ```ts instrumentAxios(client, { includeErrorStack: true, }); ``` - `throwAttachmentErrors: boolean` (default: `false`) By default, if writing the attachment fails — for example, because no Allure runtime is active during the request — the error is silently ignored. Set `throwAttachmentErrors` to `true` to have such errors rethrow instead. This is useful when debugging why attachments are not appearing: ```ts instrumentAxios(client, { throwAttachmentErrors: true, }); ``` Info: `throwAttachmentErrors` only takes effect when the HTTP request itself succeeds. If the request fails (due to a network error, cancellation, or HTTP error response) and writing the attachment also fails, the attachment error is still silently ignored so the original request error propagates cleanly. --- ## Axios reference > URL: https://allurereport.org/docs/axios-reference/ # Allure Axios reference ## Attachment schema Each HTTP exchange is recorded as an [HTTP exchange attachment](/docs/attachments/#http-exchanges) with content type `application/vnd.allure.http+json` and file extension `.httpexchange`. The root object has the following shape: | Field | Type | Description | | --------------- | -------- | --------------------------------------------------------------------------------------------- | | `schemaVersion` | `1` | Schema version. Always `1`. | | `start` | `number` | Unix timestamp in milliseconds when the request started. | | `stop` | `number` | Unix timestamp in milliseconds, recorded when the response or error was received. | | `request` | object | Captured request data. See [Request](#request). | | `response` | object | Captured response data. Present only when a response was received. See [Response](#response). | | `error` | object | Error information. Present only when the request failed. See [Error](#error). | For HTTP error responses (4xx and 5xx status codes), both `response` and `error` are present simultaneously: `response` contains the status, headers, and body of the error response, while `error` contains the `AxiosError` details. ### Request | Field | Type | Description | | --------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------- | | `method` | `string` | HTTP method in uppercase, e.g. `"GET"`. | | `url` | `string` | Resolved request URL, including query parameters in their original unredacted form. | | `headers` | `{name, value}[]` | Request headers after redaction. Omitted when there are no headers. | | `query` | `{name, value}[]` | Query parameters parsed from the URL, after redaction. Omitted when there are no query parameters. | | `cookies` | `Cookie[]` | Cookies parsed from the `Cookie` request header, after redaction. Omitted when there are no cookies. See [Cookie](#cookie). | | `body` | object | Request body. Present only when `captureRequestBody` is `true` and a body was sent. See [Body](#body). | ### Response | Field | Type | Description | | ------------ | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `status` | `number` | HTTP status code. | | `statusText` | `string` | HTTP status text. | | `headers` | `{name, value}[]` | Response headers after redaction. Omitted when there are no headers. | | `cookies` | `Cookie[]` | Cookies parsed from `Set-Cookie` response headers, after redaction. Omitted when there are no `Set-Cookie` headers. See [Cookie](#cookie). | | `body` | object | Response body. Present only when `captureResponseBody` is `true` and a body was received. See [Body](#body). | Response bodies are captured from `response.data` — the value after Axios applies its response transforms. When `responseType` is `'json'` (the default), Axios parses the response as JSON before Allure Axios sees it, so the captured `value` is the parsed object re-serialized with `JSON.stringify`. This produces compact JSON regardless of how the server formatted the original response. To capture the unmodified response bytes, set `responseType: 'arraybuffer'` or `responseType: 'text'` on the Axios request. ### Body | Field | Type | Description | | ------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `contentType` | `string` | Value of the `Content-Type` header. Absent when no `Content-Type` header is present. | | `encoding` | `"utf8" \| "base64"` | Encoding used for `value`. Present only for non-stream bodies. See [Body encoding](#body-encoding). | | `value` | `string` | The captured body content, encoded per `encoding`. Absent for stream and multipart bodies. | | `size` | `number` | Body size in bytes. For non-stream bodies, always present. For stream bodies, present only when a `Content-Length` header is available; absent otherwise. See [Body size](#body-size). | | `truncated` | `boolean` | `true` when the body exceeded `maxBodySize` and was cut off. Always `false` for stream bodies. | | `form` | `{name, value}[]` | Structured fields parsed from a `application/x-www-form-urlencoded` body, after redaction. See [URL-encoded form bodies](#url-encoded-form-bodies). | | `stream` | `{type: string}` | Present instead of `value` and `encoding` for stream and multipart bodies. See [Stream bodies](#stream-bodies). | #### Body encoding `encoding` depends on the JavaScript type of the body data, not primarily on the `Content-Type` header: - **`string`**, **`URLSearchParams`**, **plain objects** (JSON-serialized), and any other non-binary value → `"utf8"`. - **`ArrayBuffer`** and **typed arrays** → `"base64"`. - **`Blob`** → determined by `Blob.type`: `"utf8"` when `Blob.type` is `text/*`, `application/json`, `application/javascript`, `application/x-www-form-urlencoded`, `application/xml`, or ends in `+json` or `+xml`; `"base64"` for all other types. A missing or empty `Blob.type` is treated as text. #### URL-encoded form bodies When the content type is `application/x-www-form-urlencoded`, the body contains two representations of the same data: - `value`: the re-serialized URL-encoded string, with sensitive fields replaced by the redaction placeholder. - `form`: a structured `{name, value}[]` array with the same redaction applied. #### Stream bodies Bodies that cannot be read without consuming them are not captured. The `stream` field is present instead of `value`. The following are treated as streams: - **Node.js readable streams and async iterables** — objects with a `pipe`, `read`, or `Symbol.asyncIterator` property, excluding `URLSearchParams`, `ArrayBuffer`, and typed arrays. - **Objects from the `form-data` npm package**. - **Any body whose `Content-Type` is `multipart/form-data`**. `Blob` bodies are **captured**, not treated as streams — Allure Axios reads them via `arrayBuffer()`. Encoding is determined by `Blob.type`, not the request `Content-Type` header. ```json { "stream": { "type": "chunked" } } ``` The `type` is derived from the content type: `text/event-stream` → `"server-sent-events"`, `application/grpc` → `"grpc"`, all others → `"chunked"`. #### Body size The `size` field uses the `Content-Length` header when present. If `Content-Length` is absent, `size` is the actual byte count of the captured data. For truncated bodies, `size` may therefore be larger than the length of the stored `value`. ### Cookie Request cookies are parsed from the `Cookie` header. Response cookies are parsed from `Set-Cookie` headers. | Field | Type | Description | | ---------- | --------- | ----------------------------------------------------------------------- | | `name` | `string` | Cookie name. | | `value` | `string` | Cookie value, after redaction. | | `path` | `string` | `Path` attribute. Response cookies only. | | `domain` | `string` | `Domain` attribute. Response cookies only. | | `expires` | `string` | `Expires` attribute. Response cookies only. | | `httpOnly` | `boolean` | `true` when the `HttpOnly` attribute is present. Response cookies only. | | `secure` | `boolean` | `true` when the `Secure` attribute is present. Response cookies only. | | `sameSite` | `string` | `SameSite` attribute value. Response cookies only. | ### Error Present when the request failed — whether due to a transport error, cancellation, or an HTTP error response. | Field | Type | Description | | --------- | -------- | --------------------------------------------------------------------------------------------------------- | | `name` | `string` | The error's `name` property. | | `message` | `string` | The error message. | | `stack` | `string` | Stack trace. Present only when `includeErrorStack` is `true` and the thrown value is an `Error` instance. | ## Redaction types ### `RedactionMatcher` A `RedactionMatcher` controls whether a specific field value should be redacted. It can be any of the following: - A **string**: redacts any field whose name equals the string, case-insensitively. - A **`RegExp`**: redacts any field whose name matches the pattern. - A **function** `(name: string, value: string, context: RedactionContext) => boolean`: redacts based on any logic, with access to the field name, value, and surrounding context. ```ts instrumentAxios(client, { redactHeaders: [ "x-api-key", // exact name, case-insensitive /^x-internal-/i, // regex on the header name (name, value) => value.startsWith("sk-"), // match on the value itself ], }); ``` ### `RedactionContext` The `context` argument passed to a function `RedactionMatcher`: | Property | Type | Description | | ---------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | `kind` | `"header" \| "query" \| "cookie" \| "form"` | Which category is being evaluated. | | `name` | `string` | The field name. | | `value` | `string` | The field value. | | `url` | `string \| undefined` | The resolved request URL. | | `request` | `InternalAxiosRequestConfig \| undefined` | The Axios request config. | | `response` | `AxiosResponse \| undefined` | The Axios response. Present only when evaluating response-phase fields (response headers, response cookies). | --- ## Behat > URL: https://allurereport.org/docs/behat/ # Getting started with Allure Behat [![Allure Behat latest version](https://img.shields.io/packagist/v/allure-framework/allure-behat?style=flat "Allure Behat latest version")](https://packagist.org/packages/allure-framework/allure-behat) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Behat](https://behat.org/) tests. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Add the Allure Behat repository to your project's `composer.json`, for example: ```json { "repositories": [ { "url": "https://github.com/allure-framework/allure-behat.git", "type": "vcs" } ] } ``` 1. Add `allure-framework/allure-behat` to your project's development dependencies: ```bash php composer.phar require allure-framework/allure-behat:dev-master --dev ``` 1. In [`behat.yml`](https://docs.behat.org/en/latest/user_guide/configuration.html), add the Allure extension and enable the `allure` formatter: ```yaml default: formatters: pretty: true allure: true extensions: Allure\Behat\AllureFormatterExtension: {} ``` 1. Change the extension's [configuration options](/docs/behat-configuration/) if necessary. ### 2. Run tests Run your Behat tests the same way as you would run them usually. For example: ```bash vendor/bin/behat ``` This will save necessary data into `allure-results` or other directory, according to the [`output_path`](/docs/behat-configuration/#output-path) setting. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Behat integration extends the standard reporting features of Behat by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [issue links, TMS links and severity levels](#specify-metadata). - **Test Organization**: Structure your tests based on their [user-stories](#organize-tests) for better readability and organization. - **Step Division**: Break down steps into smaller [sub-steps](#divide-a-step-into-sub-steps) for easier understanding and maintenance. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. ### Specify metadata Specify issue links, TMS links and severity level for your test via Gherkin tags. See the [reference](/docs/behat-reference/) for more details. ```gherkin Feature: Labels @issue:https://issues.example.org/AUTH-123 @tms:https://tms.example.org/TMS-456 @severity:critical Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Organize tests Group tests based on their [user stories](/docs/v3/navigation/) by adding arbitrary Gherking tags. ```gherkin @UI @Labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Divide a step into sub-steps Split a Gherkin step into smaller [sub-steps](/docs/steps/). See the [reference](/docs/behat-reference/#test-sub-steps) for more details. ```php use Behat\Behat\Context\Context; use Yandex\Allure\Adapter\Support\StepSupport; class MyContext implements Context { use StepSupport; /** * @When I go to the registration form */ public function goToRegistrationForm() { $this->executeStep('Step 1', [$this, 'step1']); $this->executeStep('Step 2', [$this, 'step2']); } public function step1() { $this->executeStep('Step 1.1', function () { // ... }); $this->executeStep('Step 1.2', function () { // ... }); } public function step2() { $this->executeStep('Step 2.1', function () { // ... }); $this->executeStep('Step 2.2', function () { // ... }); } } ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. See the [reference](/docs/behat-reference/#attachments) for more details. ```php use Behat\Behat\Context\Context; use Yandex\Allure\Adapter\Support\AttachmentSupport; class MyContext implements Context { use AttachmentSupport; /** * @When I go to the registration form */ public function profileShouldBeCreated() { $this->addAttachment('/path/to/image.png', 'Screenshot', 'image/png'); } } ``` ### Select tests via a test plan file Danger: Test plan is currently not supported by the Allure Behat integration. ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and PHP version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/php/environment-allure3-light.png, /images/php/environment-allure3-dark.png, /images/php/environment-allure2-light.png, /images/php/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](/docs/behat-reference/#parametrized-tests). --- ## Behat configuration > URL: https://allurereport.org/docs/behat-configuration/ # Allure Behat configuration This page describes the configuration options that affect the behavior of [Allure Behat](/docs/behat/). To set the options, edit your project's [`behat.yml`](https://docs.behat.org/en/latest/user_guide/configuration.html) file. The `output_path` option is configured for the Allure formatter in the `formatters` section. All other options are configured for the Allure extension in the `extensions` section. ```yaml default: formatters: pretty: true allure: output_path: "%paths.base%/build/allure" extensions: Allure\Behat\AllureFormatterExtension: ignored_tags: "wip, skip" severity_key: "severity:" issue_tag_prefix: "issue:" test_id_tag_prefix: "tms:" ``` Info: Remember that in YAML, any values ending with a colon must be contained in quotation marks. ## output_path Path to the directory where Allure Behat will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `%paths.base%/allure-results`. ## ignored_tags A tag or a comma-separated list of tags that must not be interpreted as [user stories](/docs/behat-reference/#behavior-based-hierarchy). It is recommended to put here the tags that you use for filtering purposes, such as “wip” from the [example in the Behat documentation](https://docs.behat.org/en/latest/user_guide/configuration.html#global-filters). ## severity_key The tag prefix for specifying [severity levels](/docs/behat-reference/#severity) for tests. ## issue_tag_prefix The tag prefix for specifying [issue links](/docs/behat-reference/#link) for tests. ## test_id_tag_prefix The tag prefix for specifying [TMS links](/docs/behat-reference/#link) for tests. --- ## Behat reference > URL: https://allurereport.org/docs/behat-reference/ # Allure Behat reference These are the attributes and methods that you can use to integrate your Behat tests with Allure Report. ### Severity Set the test's [severity](/docs/v3/readability/#severity). To enable this, specify the [`severity_key`](/docs/behat-configuration/#severity-key) configuration option. Use this key as a prefix in the Gherkin tags, followed by one of the values: “trivial”, “minor”, “normal”, “critical”, or “blocker”. The example below assumes that you use have configured `severity:` as the prefix. ```gherkin Feature: Labels @severity:critical Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Link Add [links](/docs/v3/readability/#links) related to the test via Gherkin tags. Two types of links are supported: issue links and TMS links. To enable this, specify the [`issue_tag_prefix`](/docs/behat-configuration/#issue-tag-prefix) and [`test_id_tag_prefix`](/docs/behat-configuration/#test-id-tag-prefix) configuration options. The example below assumes that you use have configured `issue:` and `tms:` as the prefixes. ```gherkin Feature: Labels @issue:https://issues.example.org/AUTH-123 @tms:https://tms.example.org/TMS-456 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ## Behavior-based hierarchy Any Gherkin tag is considered the test's **user story** for the Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy), unless the tag has a known prefix or is ignored, according to the [configuration](/docs/behat-configuration/). ```gherkin @UI @Labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ## Test sub-steps - `StepSupport::executeStep($name, $logic, $title = null)` To split a Gherkin step into smaller [sub-steps](/docs/steps/): 1. In your context class, add the `StepSupport` trait. 1. In the step implementation, pass a lambda function or a method reference to `$this->executeStep()`. The function or method should accept no arguments. During the execution of a sub-step, you can call `$this->executeStep()` again to split it into more sub-steps. ```php use Behat\Behat\Context\Context; use Yandex\Allure\Adapter\Support\StepSupport; class MyContext implements Context { use StepSupport; /** * @When I go to the registration form */ public function goToRegistrationForm() { $this->executeStep('Step 1', [$this, 'step1']); $this->executeStep('Step 2', [$this, 'step2']); } public function step1() { $this->executeStep('Step 1.1', function () { // ... }); $this->executeStep('Step 1.2', function () { // ... }); } public function step2() { $this->executeStep('Step 2.1', function () { // ... }); $this->executeStep('Step 2.2', function () { // ... }); } } ``` ## Parametrized tests In Gherkin, a [`Scenario Outline`](https://docs.behat.org/en/latest/user_guide/writing_scenarios.html#scenario-outlines) (or a `Scenario Template`) implements the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. A scenario outline must contain an `Examples` table, from which Behat loads sets of parameters, one row after another. Behat runs all the outline's steps multiple times — one per each set of parameters. The example below shows a Gherkin file and a PHP implementation file of a test. In this example, In the test report, this test will include the three steps executed with the first set of parameter, then the three steps executed with the second set of parameters. ```gherkin Feature: User management Scenario Outline: Registration When I go to the registration form And I enter my details: "", "", "", "" Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ```php use Behat\Behat\Context\Context; class MyContext implements Context { /** * @When I go to the registration form */ public function goToRegistrationForm() { } /** * @When I enter my details: :login, :password, :name, :birthday */ public function enterDetails($login, $password, $name, $birthday) { } /** * @When I go to the registration form */ public function profileShouldBeCreated() { } } ``` ## Attachments - `AttachmentSupport::addAttachment($filePathOrContents, $caption, $type = null)` To create an [attachment](/docs/attachments/), add the `AttachmentSupport` trait to your context class and call `$this->addAttachment()` at any point during your test. Pass either the content string or the path from which the data will be read. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `$type`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. If not specified, Allure Behat will try to guess the type automatically based on the contents. ```php use Behat\Behat\Context\Context; use Yandex\Allure\Adapter\Support\AttachmentSupport; class MyContext implements Context { use AttachmentSupport; /** * @When I go to the registration form */ public function profileShouldBeCreated() { $this->addAttachment('/path/to/image.png', 'Screenshot', 'image/png'); } } ``` --- ## Behave > URL: https://allurereport.org/docs/behave/ # Getting started with Allure Behave [![Allure Behave pypi latest version](https://img.shields.io/pypi/v/allure-behave?style=flat "Allure Behave pypi latest version")](https://pypi.org/project/allure-behave/) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [behave](https://behave.readthedocs.io/en/stable/) tests. Info: Check out the example project at [github.com/allure-examples/behave-pip](https://github.com/allure-examples/behave-pip) to see Allure Behave in action. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. If necessary for your system configuration, activate the virtual Python environment for your project. For example, if the project uses a `venv` environment, the command to activate it may look like this: ```bash source .venv/bin/activate ``` This step is not necessary if you are using the system Python environment. 1. Install the Allure Behave integration. ```bash pip install allure-behave ``` 1. In the [Behave configuration file](https://behave.readthedocs.io/en/stable/behave/#configuration-files) (such as `.behaverc` or `behave.ini`), specify the Allure Behave as the output formatter and specify the path to the test results directory. For example: ```ini [behave] format=allure_behave.formatter:AllureFormatter outfiles=allure-results ``` ### 2. Run tests Run your behave tests the same way as you would run them usually. For example: ```bash python -m behave ``` This will save necessary data into `allure-results` or other directory, according to the `outfile` option. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Behave integration extends the standard reporting features of behave by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#specify-description-links-and-other-metadata). - **Test Organization**: Structure your tests into [clear hierarchies](#organize-tests) for better readability and organization. - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, you need to indicate to Allure Behave that a certain property needs to be assigned to the test result. Most properties can be assigned via Gherkin tags or via the Runtime API. - **Gherkin tags**: use [Gherkin tags](https://behave.readthedocs.io/en/latest/gherkin/#tags) to assign various data to a particular `Scenario` or a whole `Feature`. Note that due to a limitation in the Gherkin syntax, a tag cannot contain spaces. When using this approach, the data is guaranteed to be added to the test result regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add data to the test result during the execution of its steps. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/behave-reference/#metadata) for more details. **Gherkin tags:** ```gherkin Feature: Labels @critical @allure.label.owner:JohnDoe @allure.link:https://dev.example.com/ @allure.issue:UI-123 @allure.tms:TMS-456 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```python import allure from behave import then @then('I open labels page') def step_impl(context): allure.dynamic.link('https://dev.example.com/', name='Website') allure.dynamic.issue('UI-123') allure.dynamic.testcase('TMS-456') ... ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure Behave provides functions to assign the relevant fields to tests either by adding decorators or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Gherkin tags:** ```gherkin @allure.label.epic:WebInterface Feature: Labels @allure.label.story:Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Gherkin tags:** ```gherkin Feature: Labels @allure.label.parentSuite:WebInterface @allure.label.suite:EssentialFeatures @allure.label.subSuite:Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` To specify a test's location in the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy): **Gherkin tags:** ```gherkin Feature: Labels @allure.label.package:org.example @allure.label.testClass:TestMyWebsite @allure.label.testMethod:TestLabels() Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Divide a test into steps Allure Behave provides two ways of [creating steps and sub-steps](/docs/steps/): “decorated steps” and “context steps”, see the [reference](/docs/behave-reference/#test-steps). **Decorated steps:** ```python import allure from behave import then @then("I open labels page") def step_impl(context): step1() for val in ["val1", "val2", "val3"]: step2(val) @allure.step("Step 1") def step1(): ... @allure.step("Step 2 (with value {val})") def step2(val): ... ``` **Context steps:** ```python import allure from behave import then @then("I open labels page") def step_impl(context): with allure.step("Step 1"): ... for val in ["val1", "val2", "val3"]: with allure.step(f"Step 2 (with value {val})"): ... ``` ### Describe parametrized tests Allure Behave can display parameters passed to the tests via `Examples` tables, see the [reference](/docs/behave-reference/#parametrized-tests). ```gherkin Feature: User management Scenario Outline: Registration When I go to the registration form And I enter my details: , , , Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure Behave provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/behave-reference/#attachments). ```python import allure from behave import then @then("I open labels page") def step_impl(context): ... with open('/path/to/image.png', 'rb') as png_file: png_bytes = png_file.read() allure.attach(png_bytes, name="img", attachment_type=allure.attachment_type.PNG) ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, behave will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json python -m behave ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" python -m behave ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and Python version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/python/environment-allure3-light.png, /images/python/environment-allure3-dark.png, /images/python/environment-allure2-light.png, /images/python/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## Behave configuration > URL: https://allurereport.org/docs/behave-configuration/ # Allure Behave configuration Allure Behave accepts some options via the behave's “user-defined data”. All the options should be prefixed with `AllureFormatter.`. - To pass configuration options to the `behave` command-line utility, use the `-D` (or `--define`) flag. For example: ```plain python -m behave -D AllureFormatter.hide_excluded=true ``` - To save the configuration options permanently, add them to the `behave.userdata` section of the [behave configuration file](https://behave.readthedocs.io/en/stable/behave/#configuration-files). For example: ```ini [behave] format = allure_behave.formatter:AllureFormatter outfiles = allure-results [behave.userdata] AllureFormatter.link_pattern = {} AllureFormatter.issue_pattern = https://github.com/allure-framework/allure-python/issues/{} AllureFormatter.hide_excluded = true ``` ## link_pattern Define a **link pattern** — a template that can be used to construct full URLs from short identifiers, see the [reference](/docs/behave-reference/#link). The pattern must contain `{}` at the position where the identifier should be placed. The pattern will be applied to all links of type “link”, i.e. all links that are defined using the `@allure.link` tag or the `link()` function without providing a non-default `link_type`. ## issue_pattern Define an **issue pattern** — a template that can be used to construct full URLs from short identifiers, see the [reference](/docs/behave-reference/#link). The pattern must contain `{}` at the position where the identifier should be placed. The pattern will be applied to all links of type “issue”, i.e. all links that are defined using the `@allure.issue` tag, the `issue()` function or the `link()` function with `link_type="issue"`. ## hide_excluded This option affects how Allure behave treats scenarios that are skipped by behave, e.g., based on a [tag expression](https://behave.readthedocs.io/en/stable/behave/#configuration-files). By default, the test report will include such scenarios, with their test results marked as **Skipped** (see [Test statuses](/docs/test-statuses/)). However, if this configuration option is set to true, the skipped scenarios will be excluded from the test report entirely. --- ## Behave reference > URL: https://allurereport.org/docs/behave-reference/ # Allure Behave reference These are the attributes and methods that you can use to integrate your behave tests with Allure Report. In most cases, you need to indicate to Allure Behave that a certain property needs to be assigned to the test result. Most properties can be assigned via Gherkin tags or via the Runtime API. - **Gherkin tags**: use [Gherkin tags](https://behave.readthedocs.io/en/latest/gherkin/#tags) to assign various data to a particular `Scenario` or a whole `Feature`. Most of the tags require values. You can use either a colon or an equal sign to separate the value from the name, e.g., `@allure.label.epic:WebInterface` is identical to `@allure.label.epic=WebInterface`. Note that due to a limitation in the Gherkin syntax, the value cannot contain spaces. When using this approach, the data is guaranteed to be added to the test result regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add data to the test result during the execution of its steps. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `allure.dynamic.description(test_description: str)` Set the test's [description](/docs/v3/readability/#description). Allure Behave uses a scenario's description from the Gherkin file, if present. Alternatively, use Runtime API to set the description dynamically. Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. **Gherkin description:** ```gherkin Feature: Labels Scenario: Create new label for authorized user This test attempts to create a label with specified title When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```python import allure from behave import then @then("I open labels page") def step_impl(context): allure.dynamic.description("This test attempts to create a label with specified title") ... ``` ### Owner Set the test's [owner](/docs/v3/readability/#owner). **Gherkin tags:** ```gherkin @allure.label.owner:JohnDoe Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Tag Set the test's [tags](/docs/v3/readability/#tags). Any Gherkin tag is automatically added to the list of the test's tags, unless it matches one of the known patterns for other labels. **Gherkin tags:** ```gherkin @UI @Labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Severity Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Gherkin tags:** ```gherkin Feature: Labels @critical Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Label Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. **Gherkin tags:** ```gherkin @allure.label.layer:web @allure.label.owner:eroshenkoam @allure.label.page:/{org}/{repo}/labels @allure.label.jira:AE-2 Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Allure ID (Allure TestOps) Set the test's [ID](/docs/v3/readability/#id). **Gherkin tags:** ```gherkin Feature: Labels @allure.label.allure_id:123 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Link - `allure.dynamic.link(url: str, link_type: str = LinkType.LINK, name: str = None)` - `allure.dynamic.issue(url: str, name: str = None)` - `allure.dynamic.testcase(url: str, name: str = None)` Add a [link](/docs/v3/readability/#links) related to the test. - In a Gherkin file, use the `@allure.⟨TYPE⟩:⟨URL⟩` or `@allure.⟨TYPE⟩.⟨NAME⟩:⟨URL⟩` tags. If the `⟨URL⟩` does not start with “http” or “https”, it will be processed according to the [`link_pattern`](/docs/behave-configuration/#link-pattern) and [`issue_pattern`](/docs/behave-configuration/#issue-pattern) configuration options. The resulting URL will then be used as both the link target and the link name, unless a custom `⟨NAME⟩` is provided. - In the Runtime API, use the `allure.dynamic.link()` function with an optional `link_type` argument or its shorthand versions for the `issue` and `tms` link types. **Gherkin tags:** ```gherkin @allure.link.MyWebsite:https://dev.example.com/ @allure.issue.UI-123:https://issues.example.com/UI-123 @allure.tms.TMS-456:https://tms.example.com/TMS-456 Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```python import allure from behave import then @then("I open labels page") def step_impl(context): allure.dynamic.link("https://dev.example.com/", name="Website") allure.dynamic.issue("https://issues.example.com/UI-123", name="UI-123") allure.dynamic.testcase("https://tms.example.com/TMS-456", name="TMS-456") ... ``` ## Behavior-based hierarchy Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). Each function support setting one or more values at once. **Gherkin tags:** ```gherkin @allure.label.epic:WebInterface Feature: Labels @allure.label.story:CreateLabels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ## Suite-based hierarchy Assign the names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Gherkin tags:** ```gherkin @allure.label.parentSuite:WebInterface @allure.label.suite:EssentialFeatures @allure.label.subSuite:Labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ## Package-based hierarchy Assign the names of package, test class and test method, as part of Allure's [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy). **Gherkin tags:** ```gherkin @allure.label.package:org.example @allure.label.testClass:TestMyWebsite @allure.label.testMethod:TestLabels() Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ## Test steps - `@allure.step(title: str)` - `with allure.step(title: str)` Define a [test step](/docs/steps/) with the given `title`. There are two ways of defining a step. - **Decorated steps** Define a function or a method containing a test step and decorate it with `@allure.step()`. If it accepts arguments, you can include their values in the step title via **replacement fields**. A replacement field is a parameter name delimited by braces `{}`, as supported by the standard [`str.format()`](https://docs.python.org/3/library/stdtypes.html#str.format) method. - **Context steps** Write a test step in-place but use the `with allure.step()` statement to create its context. **Decorated steps:** ```python import allure from behave import then @then("I open labels page") def step_impl(context): step1() for val in ["val1", "val2", "val3"]: step2(val) @allure.step("Step 1") def step1(): ... @allure.step("Step 2 (with value {val})") def step2(val): ... ``` **Context steps:** ```python import allure from behave import then @then("I open labels page") def step_impl(context): with allure.step("Step 1"): ... for val in ["val1", "val2", "val3"]: with allure.step(f"Step 2 (with value {val})"): ... ``` ## Parametrized tests In Gherkin, a [`Scenario Outline`](https://behave.readthedocs.io/en/stable/tutorial/#scenario-outlines) (or a `Scenario Template`) implements the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. A scenario outline must contain an `Examples` table, from which behave loads sets of parameters, one row after another. Each set of parameters is being placed into the step declarations according to the placeholders, thus generating a new scenario based on the row. Behave then runs each of them independently, as if it was a separate `Scenario`. The data can then be captured by behave and passed as separate arguments to the Python code. Allure Behave automatically recognizes this pattern. No additional configuration is required. The example below shows a Gherkin file and a Python implementation file of a test. In this example, the four parameters for the “I enter my details...” step will be displayed in both instances of the scenario in the test report. ```gherkin Feature: User management Scenario Outline: Registration When I go to the registration form And I enter my details: , , , Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ```python from behave import then, when @when('I go to the registration form') def step_impl(context): ... @when('I enter my details: {login}, {password}, {name}, {birthday}') def step_impl(context, login, password, name, birthday): ... @then('the profile should be created') def step_impl(context): ... ``` ## Attachments Functions for [adding attachments](/docs/attachments/) to test results. ### Attaching content from variables - `allure.attach(body, name=None, attachment_type="text/plain", extension="attach")` Add `body` as an attachment to the test result under the given `name` (defaults to a unique pseudo-random string). The `body` must be of type `bytes` or `str`. Info: You can use data produced by any function, not necessarily read from an actual file. For attaching contents of existing files, use [`attach.file()`](#reading-attachments-from-files) instead. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. There are two ways to do this: - Pass the media type of the content as `attachment_type` and, optionally, a filename extension as `extension`. For example: ```python allure.attach('{"value":100}', attachment_type="application/json", extension="json") ``` Some popular media types are `image/png` and `image/jpeg` for screenshots and other images, `application/json` for JSON data, and `text/plain` for text files. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. - Pass a value from the `allure.attachment_type` class as `attachment_type`. For example: ```python allure.attach('{"value":100}', attachment_type=allure.attachment_type.JSON) ``` This will automatically set the media type and the appropriate filename extension. ```python import allure from behave import then @then("I open labels page") def step_impl(context): ... with open('/path/to/image.png', 'rb') as png_file: png_bytes = png_file.read() allure.attach(png_bytes, name="img", attachment_type=allure.attachment_type.PNG) ``` ### Reading attachments from files - `allure.attach.file(source, name=None, attachment_type=None, extension=None)` Same as [`attach()`](#attach), but the content is loaded from the existing `source` file. ```python import allure from behave import then @then("I open labels page") def step_impl(context): ... allure.attach.file( '/path/img.png', name="img", attachment_type=allure.attachment_type.PNG ) ``` --- ## Bun > URL: https://allurereport.org/docs/bun/ # Getting started with Allure Bun [![Allure Bun npm latest version](https://img.shields.io/npm/v/allure-bun?style=flat "Allure Bun npm latest version")](https://www.npmjs.com/package/allure-bun) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Bun](https://bun.sh/docs/cli/test) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Abun) to see Allure Bun in action. ## Setting up ### 1. Prepare your project 1. Make sure [Bun](https://bun.sh/) is installed. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure Bun integration: **npm:** ```bash npm install --save-dev allure-bun allure-js-commons ``` **yarn:** ```bash yarn add --dev allure-bun allure-js-commons ``` **pnpm:** ```bash pnpm add -D allure-bun allure-js-commons ``` **bun:** ```bash bun add --dev allure-bun allure-js-commons ``` Keep `allure-bun` and `allure-js-commons` on the same version. 1. Add the preload entry to your `bunfig.toml`: ```toml [test] preload = ["allure-bun/setup"] ``` This registers the integration before any test files are loaded. Your test files themselves do not need any changes — keep using `bun:test` as usual. Optionally, specify `resultsDir` and other options. See [Allure Bun configuration](/docs/bun-configuration/) for more details. ### 2. Run tests Run your Bun tests the same way you would run them usually: ```bash bun test ``` This will save the necessary data into `allure-results` or another directory, according to the [configuration](/docs/bun-configuration/). If the directory already exists, new files will be added to the existing ones so that a future report will be based on them all. Warning: Running tests with `--concurrent` or `--randomize` is not supported. Allure Bun will fail fast with a descriptive error if either flag is detected. All standard `bun:test` modifiers work as usual: `.skip`, `.todo`, `.if()`, `.skipIf()`, `.todoIf()`, and `.each()` — on both `test` and `describe`; plus `.failing` and `.serial` on `test` only. Two exceptions: `test.concurrent` throws an error, and `.only` (on both `test` and `describe`) is not supported — it causes the same queue-ordering error described in the warning below. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Bun integration extends the standard reporting features of Bun by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization, see [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Capture [files and other content](#attach-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure Bun provides two different ways to use a feature: the Runtime API and the Metadata API. Warning: Known limitation: always use `describe()` Always place tests inside `describe()` blocks. Mixing root-level `test()` calls with `describe()` blocks in the same file produces the error `allure-bun does not support concurrent tests`. This is a known limitation: Bun defers `describe()` callbacks, which causes allure-bun's registration queue to diverge from execution order — the same ordering mechanism it uses to prevent concurrent test execution. `test.skip` and `test.todo` are exempt. Files that contain only root-level tests with no `describe()` blocks are also unaffected. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Bun will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/bun-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("authentication", () => { test("Test Authentication", async () => { await allure.displayName("Test Authentication"); await allure.owner("John Doe"); await allure.tags("Web interface", "Authentication"); await allure.severity("critical"); // ... }); }); ``` **Metadata API:** ```ts import { describe, test } from "bun:test"; describe("authentication", () => { test( "Test Authentication" + " @allure.label.owner:JohnDoe" + " @allure.label.tag:WebInterface" + " @allure.label.tag:Authentication" + " @allure.label.severity:critical", async () => { // ... }, ); }); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("authentication", () => { test("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Authentication"); await allure.story("Login with username"); // ... }); }); ``` **Metadata API:** ```ts import { describe, test } from "bun:test"; describe("authentication", () => { test( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); }); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("authentication", () => { test("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); }); ``` **Metadata API:** ```ts import { describe, test } from "bun:test"; describe("authentication", () => { test( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); }); ``` ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), you can use the `step()` function, see the [reference](/docs/bun-reference/#test-steps). ```ts import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; import { describe, test } from "bun:test"; describe("authentication", () => { test("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); }); ``` ### Describe parametrized tests Bun's built-in `test.each()` makes it straightforward to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. Each row in the table is treated as a separate test run by Allure Report. To display a parameter value in the test report, pass it to the [`parameter()`](/docs/bun-reference/#parametrized-tests) function. **Using test():** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("authentication", () => { for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } }); ``` **Using test.each():** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("authentication", () => { test.each(["johndoe", "johndoe@example.com"])("Test Authentication as %s", async (login) => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); }); ``` ### Set labels globally Any [labels](/docs/bun-reference/#label), including custom ones, can be set via environment variables in your operating system. Here's an example: **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface bun test ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" bun test ``` ### Attach files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure Bun reference](/docs/bun-reference/#attachments). ```ts import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import { describe, test } from "bun:test"; describe("authentication", () => { test("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); }); ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Bun will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json bun test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" bun test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, edit the [`environmentInfo`](/docs/bun-configuration/#environmentinfo) object in the configuration. For example, it is a good idea to record the OS version and the Bun runtime version. This may help the future reader investigate bugs that are reproducible only in some environments. ```ts // preload.ts import type { ReporterConfig } from "allure-js-commons/sdk/reporter"; import * as os from "node:os"; globalThis.allureBunConfig = { environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), bun_version: Bun.version, }, } satisfies ReporterConfig; await import("allure-bun/setup"); ``` ```toml # bunfig.toml [test] preload = ["./preload.ts"] ``` --- ## Bun configuration > URL: https://allurereport.org/docs/bun-configuration/ # Allure Bun configuration The [Allure Bun](/docs/bun/) integration behavior can be configured through `globalThis.allureBunConfig` or the `ALLURE_BUN_CONFIG` environment variable. ## Providing configuration ### Using a custom preload file Use a custom preload file when you need configuration values that cannot be represented as JSON — for example, listener functions or link template functions. The custom preload file must import `allure-bun/setup` after setting the configuration: ```ts // preload.ts import { Status } from "allure-js-commons"; import type { ReporterConfig } from "allure-js-commons/sdk/reporter"; import * as os from "node:os"; globalThis.allureBunConfig = { resultsDir: "allure-results", links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: "https://jira.example.com/browse/%s", }, }, categories: [ { name: "foo", messageRegex: "bar", traceRegex: "baz", matchedStatuses: [Status.FAILED, Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), bun_version: Bun.version, }, globalLabels: { layer: "api", }, } satisfies ReporterConfig; await import("allure-bun/setup"); ``` Point `bunfig.toml` at your preload file instead of directly at `allure-bun/setup`: ```toml [test] preload = ["./preload.ts"] ``` ### Using the environment variable For simple deployments where all options can be expressed as JSON, pass configuration through the `ALLURE_BUN_CONFIG` environment variable: **MacOS/Linux:** ```bash export ALLURE_BUN_CONFIG='{"resultsDir":"allure-results","environmentInfo":{"node_version":"20"}}' bun test ``` **Windows:** ```powershell $Env:ALLURE_BUN_CONFIG = '{"resultsDir":"allure-results","environmentInfo":{"node_version":"20"}}' bun test ``` `globalThis.allureBunConfig` takes priority over `ALLURE_BUN_CONFIG` if both are set. ## resultsDir Path to the directory where Allure Bun will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. You can also set the output directory using the `ALLURE_RESULTS_DIR` environment variable. The `resultsDir` option takes priority over the environment variable if both are set. ## links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/bun-reference/#link)), you can specify the following: - `nameTemplate` — a template for generating the link name when it is not provided. - `urlTemplate` — a template for generating the link address when it is not provided. Both templates must contain `%s` at the position where the identifier should be placed. For example, with the configuration above, `await allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/123`. Allure will display the link with the appropriate icon for the `issue` type. ## categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). ## environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ## globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ name, value }` label objects: ```ts globalThis.allureBunConfig = { globalLabels: [ { name: "layer", value: "api" }, { name: "team", value: "backend" }, ], }; ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```ts globalThis.allureBunConfig = { globalLabels: { layer: "api", team: "backend", }, }; ``` --- ## Bun reference > URL: https://allurereport.org/docs/bun-reference/ # Allure Bun reference These are the functions that you can use to integrate your Bun tests with Allure. In most cases, Allure Bun provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Bun will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. Warning: Known limitations Always place tests inside `describe()` blocks. Mixing root-level `test()` calls with `describe()` blocks in the same file produces the error `allure-bun does not support concurrent tests`. This is a known limitation: Bun defers `describe()` callbacks, which causes allure-bun's registration queue to diverge from execution order — the same ordering mechanism it uses to prevent concurrent test execution. `test.skip` and `test.todo` are exempt. Files that contain only root-level tests with no `describe()` blocks are also unaffected. `.only` (on both `test` and `describe`) is not supported for the same reason: when only one test runs, allure-bun's queue expects the first registered test, not the `.only` test, and throws the same error. ## Sync API All functions in this reference are asynchronous and return a `PromiseLike`. If your test uses synchronous helpers or matcher integrations that do not support async, import from `allure-js-commons/sync` instead: ```ts import * as allure from "allure-js-commons/sync"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", () => { allure.step("check result", () => { allure.parameter("mode", "sync"); }); }); }); ``` The sync facade is strict-sync only: `step()` must complete synchronously and must not return a `Promise`. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `allure.displayName(name: string): PromiseLike` Set the test's [title](/docs/v3/readability/#title). ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.displayName("Test Authentication!"); // ... }); }); ``` ### Description - `allure.description(markdown: string): PromiseLike` - `allure.descriptionHtml(html: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Use `description()` for Markdown content. Any HTML formatting, if present, will be stripped for security purposes. Use `descriptionHtml()` when you need to supply raw HTML directly instead of Markdown. ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.description("This test attempts to log into the website."); // ... }); }); ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.owner("John Doe"); // ... }); }); ``` **Metadata API:** ```ts import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication @allure.label.owner:JohnDoe", async () => { // ... }); }); ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` - `@allure.label.tag:⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.tag("New UI"); await allure.tags("Essentials", "Authentication"); // ... }); }); ``` **Metadata API:** ```ts import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication @allure.label.tag:WebInterface @allure.label.tag:Authentication", async () => { // ... }); }); ``` ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: `"trivial"`, `"minor"`, `"normal"`, `"critical"`, and `"blocker"`. **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { Severity } from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.severity(Severity.CRITICAL); // ... }); }); ``` **Metadata API:** ```ts import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication @allure.label.severity:critical", async () => { // ... }); }); ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` - `@allure.label.⟨NAME⟩:⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for many of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.label("microservice", "UI"); // ... }); }); ``` **Metadata API:** ```ts import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication @allure.label.microservice:UI", async () => { // ... }); }); ``` ### ID - `allure.allureId(value: string): PromiseLike` - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.allureId("123"); // ... }); }); ``` **Metadata API:** ```ts import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication @allure.id:123", async () => { // ... }); }); ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string, defaults to `"link"`), Allure will try to load a corresponding **link template** to process the URL, as defined by the [`links`](/docs/bun-configuration/#links) configuration option. If no template is found for the given type, the URL is left unmodified. If the URL does not start with `"http"` or `"https"`, it will be processed according to the [`links`](/docs/bun-configuration/#links) configuration option. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.issue("AUTH-123", "Related issue"); await allure.tms("TMS-456", "Related TMS issue"); await allure.link("JIRA-777", "Related Jira issue", "jira"); await allure.link("https://example.com/", "Project website"); // ... }); }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); }); ``` **Metadata API:** ```ts import { describe, test } from "bun:test"; describe("example", () => { test( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); }); ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). This overrides the corresponding levels of the default suites hierarchy, which Allure Bun creates based on the `describe()` arguments. **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); }); ``` **Metadata API:** ```ts import { describe, test } from "bun:test"; describe("example", () => { test( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); }); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test step or sub-step](/docs/steps/) with the given `name`. The `step()` function accepts an anonymous function as its second argument, which can be synchronous or asynchronous. The anonymous function can accept either no arguments or a single argument of type `StepContext`. This object provides the following methods: - `displayName()` — override the step name during its execution. - `parameter(name: string, value: string, mode?: "default" | "masked" | "hidden")` — indicate arbitrary parameters used for the step. The third argument is the display mode (a plain string, not a `ParameterOptions` object). Unlike the test-wide `parameter()`, step parameters do not support the `excluded` option. To create a step without a body, call the `logStep()` function that accepts a name and an optional step status. ```ts import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` The [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern in Bun can be implemented by calling the built-in [`test.each()`](https://bun.sh/docs/test/writing#test-each) function or by running a `test()` inside a loop. In both cases, Allure Report recognizes each iteration as a separate test run. The values that distinguish one iteration from another are called test parameters. To display a parameter value in the test report, pass it to the `parameter()` function. The `options` argument, if given, must be an object with two optional properties: `excluded` and `mode`. - If `excluded` is set to `true`, Allure will not use the parameter when comparing the current test result with previous ones in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. Note that even when you use the `"masked"` or `"hidden"` mode, it is still possible to extract the value from the `allure-results` directory if you publish it. **Using test():** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } }); ``` **Using test.each():** ```ts import * as allure from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test.each(["johndoe", "johndoe@example.com"])("Test Authentication as %s", async (login) => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); }); ``` ## Attachments - `allure.attachment(name: string, content: Buffer | Uint8Array | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` directly or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options in an object (as shown for the image attachment below) or just specify the media type and let Allure deduce the appropriate filename extension automatically (as shown for the text attachment below). In either case, the media type can be a value from the `ContentType` enumeration or any string. ```ts import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import { describe, test } from "bun:test"; describe("example", () => { test("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); }); ``` --- ## Chai > URL: https://allurereport.org/docs/chai/ # Getting started with Allure Chai [![Allure Chai npm latest version](https://img.shields.io/npm/v/allure-chai?style=flat "Allure Chai npm latest version")](https://www.npmjs.com/package/allure-chai) Record [Chai](https://www.chaijs.com/) assertions as steps in your [Allure Report](https://allurereport.org/docs/). ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure Chai is tested against Node.js 20 and 22. Older versions may work, but we can't guarantee that. 1. Make sure you have Chai installed. Allure Chai supports Chai 4, 5, and 6. Chai 7 and later are not supported. 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Allure Chai is a plugin for Chai — it does not run tests on its own. Install it together with an Allure test runner integration. It is compatible with [allure-mocha](/docs/mocha/), [allure-jasmine](/docs/jasmine/), [allure-jest](/docs/jest/), [allure-vitest](/docs/vitest/), [allure-playwright](/docs/playwright/), and [allure-bun](/docs/bun/). For example, to use it with Mocha: **npm:** ```bash npm install --save-dev allure-chai allure-mocha chai mocha ``` **yarn:** ```bash yarn add --dev allure-chai allure-mocha chai mocha ``` **pnpm:** ```bash pnpm add -D allure-chai allure-mocha chai mocha ``` 1. Register `allureChai` with Chai before your tests run. The best place is a setup file that your test runner loads before the test suite. ```js import { allureChai } from "allure-chai"; import * as chai from "chai"; chai.use(allureChai); ``` How to load a setup file depends on your test runner. For example, with Mocha, add it via the `--require` flag or in `.mocharc.json`: ```json { "require": ["./test/setup.js"] } ``` With Vitest, add it to `setupFiles` in `vitest.config.ts`: ```ts export default defineConfig({ test: { setupFiles: ["allure-vitest/setup", "./test/setup.ts"], }, }); ``` Warning: Make sure you call `chai.use(allureChai)` after the Allure runner integration is initialized. The `allure-vitest/setup` and similar setup imports must come first. ### 2. Run tests Run your tests the same way you would normally. For example, with Mocha: **npm:** ```bash npm test ``` **yarn:** ```bash yarn test ``` **pnpm:** ```bash pnpm test ``` Each Chai assertion will be recorded as a step inside the test that made it. The test results will be saved to `allure-results` or whatever directory your runner integration is configured to use. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## How assertions appear in reports Once registered, Allure Chai automatically records every Chai assertion as a step. Each step is named after the assertion expression, so the report reads like the test code. A passing assertion produces a passed step; a failing assertion produces a failed step with the original error message, and the actual and expected values from the assertion error are captured in the step details. All three Chai assertion styles are supported. ### expect style ```js expect(response.status).to.equal(200); ``` Produces a step named: ``` expect(201).to.equal(200) ``` ### should style ```js import "chai/register-should"; response.status.should.equal(200); ``` Produces a step named: ``` expect(201).to.equal(200) ``` ### assert style ```js assert.equal(response.status, 200); ``` Produces a step named: ``` assert.equal(201, 200) ``` ### Chained assertions When you chain assertions, each assertion in the chain produces its own sibling step. The second step runs on the value returned by the first assertion, not on the original object: ```js expect({ id: 1, name: "Ada" }).to.have.property("name").that.equals("Ada"); ``` Produces two steps: ``` expect({"id":1,"name":"Ada"}).to.have.property("name") expect("Ada").to.equal("Ada") ``` ## Modifiers in step names Modifiers such as `not`, `include`, `deep`, `nested`, `own`, `ordered`, `any`, and `all` appear in the step name. For example: ```js const user = { name: "Alice" }; expect(user).to.not.have.property("password"); const body = { id: 2 }; expect(body).to.deep.equal({ id: 1, name: "Alice" }); expect([1, 2, 3]).to.include.any.members([1, 2]); ``` These produce steps named: ``` expect({"name":"Alice"}).to.not.have.property("password") expect({"id":2}).to.deep.equal({"id":1,"name":"Alice"}) expect([1,2,3]).to.include.any.members([1,2]) ``` Language chain properties like `and`, `but`, `with`, `that`, etc. have no effect on the step name. `to` always appears in `expect`/`should` step names. `be` and `have` are added automatically as prefixes where the assertion name requires them — you do not need to write them in your code for them to appear in the step name. Tip: `length` vs `lengthOf` `expect(arr).to.have.length(3)` produces `expect([...]).to.length(3)` — the `have` is dropped because `length` is a chainable method, not a `have`-prefixed assertion. Use `lengthOf` if you want `have` to appear: `expect(arr).to.have.lengthOf(3)` → `expect([...]).to.have.lengthOf(3)`. ## Custom assertions Allure Chai automatically instruments assertions added via `chai.Assertion.addMethod()`, `chai.Assertion.addProperty()`, and `chai.Assertion.addChainableMethod()`. No extra configuration is needed — custom assertions are recorded as steps in the same way as built-in ones. The `overwriteMethod`, `overwriteProperty`, and `overwriteChainableMethod` APIs are not intercepted. If a Chai plugin uses these to redefine an existing assertion after `chai.use(allureChai)`, the overwritten implementation will not be automatically wrapped, and that assertion will no longer be recorded as a step. ```js chai.Assertion.addMethod("positiveNumber", function () { this.assert( this._obj > 0, "expected #{this} to be a positive number", "expected #{this} to not be a positive number", ); }); expect(42).to.be.positiveNumber(); // Step: expect(42).to.positiveNumber() ``` If a custom assertion calls inner assertions inside a callback, those inner assertions are recorded as nested child steps of the outer assertion's step: ```js chai.Assertion.addMethod("satisfyEach", function (callback) { const items = chai.util.flag(this, "object"); items.forEach(callback); }); expect([{ enabled: true }]).to.satisfyEach((item) => { expect(item.enabled).to.equal(true); }); // Step: expect([{"enabled":true}]).to.satisfyEach([Function]) // Step: expect(true).to.equal(true) ``` ## Compatibility Allure Chai requires an Allure test runner integration to be active when tests run. It is compatible with the following integrations: | Integration | Supported | | -------------------------------------- | -------------------- | | [allure-mocha](/docs/mocha/) | Yes | | [allure-jasmine](/docs/jasmine/) | Yes | | [allure-jest](/docs/jest/) | Yes | | [allure-vitest](/docs/vitest/) | Yes — see note below | | [allure-playwright](/docs/playwright/) | Yes | | [allure-bun](/docs/bun/) | Yes | | allure-cypress | No | **Note for Vitest users**: Vitest's own `expect()` is Chai-based, but allure-vitest already records Vitest assertions as steps. To avoid duplicate steps, Allure Chai automatically skips Vitest's built-in assertions. The plugin is still useful when you bring in additional Chai assertions on top of Vitest's built-in ones — for example from a Chai plugin like `chai-http`. **Cypress is not supported**: Cypress bundles its own Chai instance. Allure Chai detects when `globalThis.Cypress` is present and the registered Chai instance matches Cypress's own, and opts out for that instance — registering `allureChai` against Cypress's Chai will have no effect. If you import a separate copy of Chai in the same project (e.g. for a plugin like `chai-http`), that instance is not affected by the Cypress check and will still be recorded. Use [allure-cypress](/docs/cypress/) for Cypress-native assertions. --- ## Chai reference > URL: https://allurereport.org/docs/chai-reference/ # Allure Chai reference ## allureChai - `allureChai: ChaiPlugin` The Chai plugin exported by `allure-chai`. Register it with Chai once before your tests run: ```js import { allureChai } from "allure-chai"; import * as chai from "chai"; chai.use(allureChai); ``` Once registered, every Chai assertion executed during the test run will be recorded as an Allure step. The plugin instruments the `chai.Assertion` prototype, the `chai.assert` interface, and the static methods used to define custom assertions. Calling `chai.use(allureChai)` more than once has no effect — the plugin guards against double-instrumentation. ## Step name format ### expect and should style Steps from `expect()` and `.should` assertions follow this pattern: ``` expect(⟨actual⟩).to[.⟨modifier⟩...][.have|.be].⟨assertion⟩[(⟨arguments⟩)] ``` - `⟨actual⟩` is the value passed to `expect()`, serialized as described in [Value serialization](#value-serialization). - Modifiers (`not`, `deep`, `nested`, `own`, `ordered`, `any`, `all`) are included between `to` and the assertion name when they are active on the assertion chain. `include` also appears as a modifier when `.include` (or `.contain`, `.contains`, `.includes`) is chained before another assertion such as `.members`. When multiple modifiers are active, they always appear in this fixed order: `not`, `include`, `deep`, `nested`, `own`, `ordered`, `any`, `all`. - Some assertions are prefixed with `have` or `be` to match their natural Chai expression. See [Assertion path prefixes](#assertion-path-prefixes) below. - Property assertions (those that take no arguments) omit the trailing parentheses. **Examples:** | Code | Step name | | --------------------------------------- | ----------------------------------------- | | `expect("a").to.equal("b")` | `expect("a").to.equal("b")` | | `expect(arr).to.have.lengthOf(3)` | `expect([...]).to.have.lengthOf(3)` | | `expect(x).to.be.null` | `expect(null).to.be.null` | | `expect(x).to.not.be.undefined` | `expect("foo").to.not.be.undefined` | | `expect(obj).to.deep.equal({a:1})` | `expect({...}).to.deep.equal({"a":1})` | | `expect(obj).to.have.own.property("x")` | `expect({...}).to.own.have.property("x")` | The step name always follows the structural pattern `to[.⟨modifier⟩...][.have|.be].⟨assertion⟩`, regardless of the order in which language chains appear in code. For example, the `have` in `.to.have.own.property(...)` is a language chain that gets dropped, and `have` is re-added structurally after the `own` modifier, yielding `.to.own.have.property(...)`. ### assert style Steps from `assert.*` calls follow this pattern: ``` assert.⟨name⟩(⟨arguments⟩) ``` All arguments are serialized and joined with `, `. **Examples:** | Code | Step name | | ----------------------------- | ----------------------------- | | `assert.equal("a", "b")` | `assert.equal("a", "b")` | | `assert.isTrue(false)` | `assert.isTrue(false)` | | `assert.lengthOf([1,2,3], 3)` | `assert.lengthOf([1,2,3], 3)` | ### Assertion name aliases Several Chai assertion aliases are normalized to their canonical names in step output: | Alias | Displayed as | | --------------------------------- | ----------------------- | | `eq`, `equals` | `equal` | | `eqls` | `eql` | | `contain`, `contains`, `includes` | `include` | | `exists` | `exist` | | `matches` | `match` | | `throws`, `Throw` | `throw` | | `greaterThan`, `gt` | `above` | | `greaterThanOrEqual`, `gte` | `least` | | `lessThan`, `lt` | `below` | | `lessThanOrEqual`, `lte` | `most` | | `instanceof` | `instanceOf` | | `haveOwnProperty` | `ownProperty` | | `haveOwnPropertyDescriptor` | `ownPropertyDescriptor` | | `respondsTo` | `respondTo` | | `satisfies` | `satisfy` | ### Assertion path prefixes Allure Chai adds `have` or `be` before the assertion name in `expect`/`should` steps to match the natural Chai expression. The prefixes are applied as follows: - **`.have.`** is added before: `keys`, `key`, `lengthOf`, `members`, `property`, `ownProperty`, `ownPropertyDescriptor`, `string`. The `.have.` prefix is omitted when `include` is an active modifier. For example, `expect([1,2,3]).to.include.members([2,3])` produces `expect([1,2,3]).to.include.members([2,3])`, not `expect([1,2,3]).to.include.have.members([2,3])`. - **`.be.`** is added before: `a`, `an`, `arguments`, `Arguments`, `empty`, `exist`, `extensible`, `false`, `finite`, `frozen`, `instanceOf`, `NaN`, `null`, `ok`, `sealed`, `true`, `undefined`. - No prefix is added for all other assertions. ## Value serialization Values appearing in step names (both `⟨actual⟩` and assertion arguments) are serialized as follows: - **Strings** are JSON-encoded, including surrounding quotes: `"hello"`. - **Numbers, booleans, `null`, `undefined`** are converted to their string representations: `42`, `true`, `null`, `undefined`. - **BigInts** are serialized as `⟨value⟩n`: `9007199254740993n`. - **Symbols** are serialized using `Symbol.prototype.toString()`: `Symbol(foo)`. - **Functions** are serialized as `[Function ⟨name⟩]`, or `[Function]` if the function has no name. Constructor functions that are subclasses of `Error` (i.e. where `value.prototype instanceof Error` is true) are serialized by name only: `TypeError`. Note that `Error` itself does not satisfy this check and serializes as `[Function Error]`. - **RegExp values** are serialized using `RegExp.prototype.toString()` and then JSON-encoded as a string: `"/^foo$/i"`. - **Error instances** are serialized as `{"name":"⟨name⟩","message":"⟨message⟩"}`, whether they appear as the top-level value or nested inside objects or arrays. - **Arrays and objects** are JSON-serialized up to two levels of nesting. Values nested deeper than two levels are replaced with `[Array]` or `[Object]`. Circular references are replaced with `[Circular]`. - **Serialized values longer than 160 characters** are truncated and suffixed with `... `. ## What is and is not recorded **Recorded as steps:** - All method assertions on `chai.Assertion.prototype` (e.g. `equal`, `include`, `throw`, `match`). - All property assertions on `chai.Assertion.prototype` that are not language chains or modifiers (e.g. `null`, `ok`, `empty`, `exist`). - Chainable methods used as assertions: `a`, `an`, `include`, `contain`, `contains`, `includes`, `length`. - All methods on `chai.assert` (e.g. `assert.equal`, `assert.isTrue`, `assert.throws`). `assert.*` methods are implemented on top of `expect` internally; those inner `expect` calls are suppressed so each `assert.*` call records exactly one step, not two. - Custom assertions added via `chai.Assertion.addMethod()`, `chai.Assertion.addProperty()`, and `chai.Assertion.addChainableMethod()`. Assertions already on the prototype when `chai.use(allureChai)` is called are instrumented at registration time; assertions added afterward are instrumented at the point of the `addMethod`/`addProperty`/`addChainableMethod` call. - Inner assertions called from a callback inside a custom assertion. These are recorded as nested child steps of the outer assertion's step. - When a failed assertion carries `actual` and `expected` on its error (as Chai's `AssertionError` does), those values are captured in the step's status details. **Not recorded:** - Language chain properties: `to`, `be`, `been`, `is`, `and`, `has`, `have`, `with`, `that`, `which`, `at`, `of`, `same`, `but`, `does`, `still`, `also`. These are cosmetic and have no effect on the report. - Modifier properties accessed as part of a chain: `not`, `deep`, `nested`, `own`, `ordered`, `any`, `all`, `itself`. These are not recorded as steps themselves. All except `itself` appear in the name of the assertion step that follows; `itself` is a silent chain that affects `.respondTo()` semantics but is never reflected in step names. - Internal `Assertion.prototype` members that are not user-facing assertions: `_obj`, `__flags`, `__methods`, `callable`, `iterable`, `numeric`, `assert`, `constructor`. These are skipped during instrumentation. - `AssertionError` on `chai.assert`. This is a constructor, not a callable assertion, and is not wrapped. - Vitest's built-in `expect()` assertions. Allure Chai detects Vitest-owned assertion chains and skips them automatically to avoid duplicating the steps that allure-vitest already records. - Assertions invoked through Cypress's built-in Chai instance. Allure Chai detects when `globalThis.Cypress` is present and the registered Chai instance matches Cypress's own, and opts out for that instance. A separately imported Chai instance used in the same project (e.g. for `chai-http` assertions) is not affected by the Cypress check and will still be recorded. --- ## Codeception > URL: https://allurereport.org/docs/codeception/ # Getting started with Allure Codeception [![Allure Codeception latest version](https://img.shields.io/packagist/v/allure-framework/allure-codeception?style=flat "Allure Codeception latest version")](https://packagist.org/packages/allure-framework/allure-codeception) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Codeception](https://codeception.com/) tests. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Make sure that your project uses PHP 8 and Codeception 5. 1. Add `allure-framework/allure-codeception` to your project's development dependencies: ```plain php composer.phar require allure-framework/allure-codeception --dev ``` 1. Enable the Allure extension for Codeception in [`codeception.yml`](https://codeception.com/docs/reference/Configuration). ```yaml namespace: Tests support_namespace: Support paths: tests: tests output: tests/_output data: tests/Support/Data support: tests/Support envs: tests/_envs extensions: enabled: - Qameta\Allure\Codeception\AllureCodeception ``` 1. Optionally, specify configuration options, as described in [Configuration](/docs/codeception-configuration/). ### 2. Run tests Run your Codeception tests the same way as you would run them usually. For example: ```plain vendor/bin/codecept run ``` This will save necessary data into the test results directory, according to the [`outputDirectory` setting](/docs/codeception-configuration/#outputdirectory). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Codeception integration extends the standard reporting features of Codeception by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#specify-description-links-and-other-metadata). - **Test Organization**: Structure your tests into [clear hierarchies](#organize-tests) for better readability and organization. - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure Codeception provides two different ways to use a feature: the Attributes API and the Runtime API. - **Attributes API**: add a PHP attribute to a test method or a whole class to add certain data to the test result. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/codeception-reference/#metadata) for more details. **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\Description; use Qameta\Allure\Attribute\DisplayName; use Qameta\Allure\Attribute\Issue; use Qameta\Allure\Attribute\Link; use Qameta\Allure\Attribute\Owner; use Qameta\Allure\Attribute\Severity; use Qameta\Allure\Attribute\TmsLink; class TestMyWebsite extends Unit { #[DisplayName('Test Labels')] #[Description('This test attempts to create a label with specified title.')] #[Severity(Severity::CRITICAL)] #[Owner('John Doe')] #[Link('My Website', 'https://example.com/')] #[Issue('UI-123')] #[TmsLink('TMS-456')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; use Qameta\Allure\Model\Severity; class TestMyWebsite extends Unit { public function testLabels() { Allure::displayName('Test Metadata'); Allure::description('This test attempts to create a label with specified title.'); Allure::severity(Severity::critical()); Allure::owner('John Doe'); Allure::link('My Website', 'https://example.com/'); Allure::issue('UI-123'); Allure::tms('TMS-456'); // ... } } ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure Codeception provides the API to assign the relevant fields to tests either by adding attributes or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\Epic; use Qameta\Allure\Attribute\Feature; use Qameta\Allure\Attribute\Story; class TestMyWebsite extends Unit { #[Epic('Web interface')] #[Feature('Essential features')] #[Story('Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { Allure::epic('Web interface'); Allure::feature('Essential features'); Allure::story('Labels'); // ... } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\ParentSuite; use Qameta\Allure\Attribute\SubSuite; use Qameta\Allure\Attribute\Suite; class TestMyWebsite extends Unit { #[ParentSuite('Web interface')] #[Suite('Essential features')] #[SubSuite('Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { Allure::parentSuite('Web interface'); Allure::suite('Essential features'); Allure::subSuite('Labels'); // ... } } ``` ### Divide a test into steps Allure Codeception provides three ways of [creating steps and sub-steps](/docs/steps/): “method-based steps”, “lambda steps” and “no-op steps”, see the [reference](/docs/codeception-reference/#test-steps). **Method-based steps:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; use Qameta\Allure\Attribute\DisplayName; use Qameta\Allure\StepContextInterface; class TestMyWebsite extends Unit { public function testLabels() { Allure::runStep([$this, 'logIn']); Allure::runStep([$this, 'createLabel']); Allure::runStep([$this, 'checkThatLabelExists']); } #[DisplayName('Log in')] function logIn(StepContextInterface $context) { $context->parameter('Email', 'johndoe@example.com'); $context->parameter('Password', 'qwerty'); // ... } #[DisplayName('Create label')] function createLabel(StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... } #[DisplayName('Check that label exists')] function checkThatLabelExists(StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... } } ``` **Lambda steps:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; use Qameta\Allure\StepContextInterface; class TestMyWebsite extends Unit { public function testLabels() { Allure::runStep(function (StepContextInterface $context) { $context->parameter('Email', 'johndoe@example.com'); $context->parameter('Password', 'qwerty'); // ... }, 'Log in'); Allure::runStep(function (StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... }, 'Create label'); Allure::runStep(function (StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... }, 'Check that label exists'); } } ``` **No-op steps:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { // ... Allure::addStep('Log in'); // ... Allure::addStep('Create label'); // ... Allure::addStep('Check that label exists'); } } ``` ### Describe parametrized tests If you use the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, call the `Allure::parameter()` function to add the parameters to the test report, see the [reference](/docs/codeception-reference/#parametrized-tests). **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testAuthenticationWithUsername() { Allure::parameter('login', 'johndoe'); Allure::parameter('password', 'qwerty'); // ... } public function testAuthenticationWithEmail() { Allure::parameter('login', 'johndoe@example.com'); Allure::parameter('password', 'qwerty'); // ... } } ``` **DataProvider + Runtime API:** ```php use Codeception\Attribute\DataProvider; use Codeception\Test\Unit; use Qameta\Allure\Allure; class ParametersProviderTest extends Unit { protected function dataProvider() { return [ ['login' => 'johndoe', 'password' => 'qwerty'], ['login' => 'johndoe@example.com', 'password' => 'qwerty'], ]; } #[DataProvider('dataProvider')] public function testAuthenticationWithData($login, $password) { Allure::parameter('Login', $login); Allure::parameter('Password', $password); // ... } } ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure Codeception provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/codeception-reference/#attachments). **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { // ... Allure::attachment('data.txt', 'This is the file content.', 'text/plain'); Allure::attachmentFile('data.txt', '/path/to/image.png', 'image/png'); } } ``` ### Select tests via a test plan file Danger: Test plan is currently not supported by the Allure Codeception integration. ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and PHP version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/php/environment-allure3-light.png, /images/php/environment-allure3-dark.png, /images/php/environment-allure2-light.png, /images/php/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## Codeception configuration > URL: https://allurereport.org/docs/codeception-configuration/ # Allure Codeception configuration The behavior of [Allure Codeception](/docs/codeception/) can be controlled by some configuration options in your `codeception.yml` file. For example: ```yaml namespace: Tests support_namespace: Support paths: tests: tests output: tests/_output data: tests/Support/Data support: tests/Support envs: tests/_envs extensions: enabled: - Qameta\Allure\Codeception\AllureCodeception config: Qameta\Allure\Codeception\AllureCodeception: outputDirectory: allure-results setupHook: Hooks\SetupHook linkTemplates: issue: https://issues.example.org/%s tms: https://tms.example.org/%s jira: https://jira.example.org/browse/%s ``` ## outputDirectory Path to the directory where Allure Codeception will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. The path is relative to the directory specified in `paths.output`, e.g. the directory will be created at `tests/_output/allure-results` in the example above. ## setupHook If provided, Allure Codeception will call the `__invoke()` method of the given class before running any tests. This can be used, for example, to copy your previous report's [history files](/docs/how-it-works-history-files/), to generate an [environment file](/docs/how-it-works-environment-file/), etc. Make sure the setup hook class can be loaded by PHP. If you use Composer's autoloading mechanism, see the [Composer documentation](https://getcomposer.org/doc/01-basic-usage.md#autoloading). ## linkTemplates A mapping of templates that can be used to construct full URLs from short identifiers, see the [reference](/docs/codeception-reference/#link). Each key must be a link type identifier, and each pattern must contain `%s` at the position where the identifier should be placed. --- ## Codeception reference > URL: https://allurereport.org/docs/codeception-reference/ # Allure Codeception reference These are the attributes and methods that you can use to integrate your Codeception tests with Allure Report. In most cases, Allure Codeception provides two different ways to use a feature: the Attributes API and the Runtime API. - **Attributes API**: add a PHP attribute to a test method or a whole class to add certain data to the test result. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `#[DisplayName(string $value)]` - `Allure::displayName(string $name)` Set the test's [title](/docs/v3/readability/#title). **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\DisplayName; class TestMyWebsite extends Unit { #[DisplayName('Test Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { Allure::displayName('Test Labels'); // ... } } ``` ### Description - `#[Description(string $value)]` - `Allure::description(string $description)` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\Description; class TestMyWebsite extends Unit { #[Description('This test attempts to create a label with specified title.')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { Allure::description('This test attempts to log into the website using a login and a password.'); // ... } } ``` ### Owner - `#[Owner(string $value)]` - `Allure::owner(string $value)` Set the test's [owner](/docs/v3/readability/#owner). **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\Owner; class TestMyWebsite extends Unit { #[Owner('John Doe')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { Allure::owner('John Doe'); // ... } } ``` ### Tag - `#[Tag(string $value)]` - `Allure::tag(string $value)` Set the test's [tags](/docs/v3/readability/#tags). **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\Tag; class TestMyWebsite extends Unit { #[Tag('UI'), Tag('Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { Allure::tag('UI'); Allure::tag('Labels'); // ... } } ``` ### Severity - `#[Severity(string $value)]` - `Allure::severity(Severity $value)` Set the test's [severity](/docs/v3/readability/#severity). For the Attributes API, use the constants from the `Severity` class. For the Runtime API, use the static methods on the `Severity` class to construct the values. **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\Severity; class TestMyWebsite extends Unit { #[Severity(Severity::CRITICAL)] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; use Qameta\Allure\Model\Severity; class TestMyWebsite extends Unit { public function testLabels() { Allure::severity(Severity::critical()); // ... } } ``` ### Label - `#[Label(string $name, ?string $value = null)]` - `Allure::label(string $name, string $value)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other APIs. You may also extend the Attributes API by defining your own attribute that will work as a `#[Label]` with a predefined `$name`. **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\Label; class TestMyWebsite extends Unit { #[Label('MyCustomLabel', 'value')] public function testLabels() { // ... } } ``` **Attributes API with a custom attribute:** ```php use Attribute; use Codeception\Test\Unit; use Qameta\Allure\Attribute\AbstractLabel; #[Attribute(Attribute::TARGET_CLASS | Attribute::TARGET_METHOD | Attribute::IS_REPEATABLE)] final class MyCustomLabel extends AbstractLabel { public function __construct(string $value) { parent::__construct('MyCustomLabel', $value); } } class TestMyWebsite extends Unit { #[MyCustomLabel('value')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { Allure::label('MyCustomLabel', 'value'); // ... } } ``` ### ID - `#[AllureId(string $value)]` Set the test's [ID](/docs/v3/readability/#id). ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\AllureId; class TestMyWebsite extends Unit { #[AllureId('123')] public function testLabels() { // ... } } ``` ### Link - `#[Link(?string $name = null, ?string $url = null, string $type = Link::CUSTOM)]` - `#[Issue(?string $name = null, ?string $url = null)]` - `#[TmsLink(?string $name = null, ?string $url = null)]` - `Allure::link(string $url, ?string $name = null, ?LinkType $type = null)` - `Allure::issue(string $name, ?string $url = null)` - `Allure::tms(string $name, ?string $url = null)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type`, Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`linkTemplates`](/docs/codeception-configuration/#linktemplates) configuration option. If no pattern found for the given type, the URL is left unmodified. When using the Runtime API, the `type` must be a value from the `LinkType` enumeration. When using the Attributes API, the `type` can be any string. Both APIs provide shorthands for the “issue” and “tms” link types. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\Issue; use Qameta\Allure\Attribute\Link; use Qameta\Allure\Attribute\TmsLink; class TestMyWebsite extends Unit { #[Link('My Website', 'https://example.com/')] #[Issue('UI-123')] #[TmsLink('TMS-456')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { Allure::link('https://example.com/', 'My Website'); Allure::issue('UI-123'); Allure::tms('TMS-456'); // ... } } ``` ## Behavior-based hierarchy - `#[Epic(string $value)]` - `#[Feature(string $value)]` - `#[Story(string $value)]` - `Allure::epic(string $value)` - `Allure::feature(string $value)` - `Allure::story(string $value)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\Epic; use Qameta\Allure\Attribute\Feature; use Qameta\Allure\Attribute\Story; class TestMyWebsite extends Unit { #[Epic('Web interface')] #[Feature('Essential features')] #[Story('Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { Allure::epic('Web interface'); Allure::feature('Essential features'); Allure::story('Labels'); // ... } } ``` ## Suite-based hierarchy - `#[ParentSuite(string $value)]` - `#[Suite(string $value)]` - `#[SubSuite(string $value)]` - `Allure::parentSuite(string $value)` - `Allure::suite(string $value)` - `Allure::subSuite(string $value)` Assign the name of the suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Attributes API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Attribute\ParentSuite; use Qameta\Allure\Attribute\SubSuite; use Qameta\Allure\Attribute\Suite; class TestMyWebsite extends Unit { #[ParentSuite('Web interface')] #[Suite('Essential features')] #[SubSuite('Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { Allure::parentSuite('Web interface'); Allure::suite('Essential features'); Allure::subSuite('Labels'); // ... } } ``` ## Test steps - `Allure::runStep(callable $callable, ?string $name = null)` - `Allure::addStep(string $name, ?Status $status = null)` Define [test steps](/docs/steps/). There are three ways of defining a step. - **Method-based steps** Write a test step in a public method and pass it to `runStep()` using the array syntax (i.e., `[$obj, 'method']`). The method should either accept no arguments or accept one argument of the type `StepContextInterface`. During the execution of a step, you can call `runStep()` again to create a sub-step. To change a step's title (defaults to just the word “step”), you can: - add the [`#[DisplayName]`](#title) attribute to the method, - pass the optional `name` argument to `runStep()`, - use the `name()` method on the `StepContextInterface` object. - **Lambda steps** Write a test step in a lambda function and pass it to `runStep()`. If the lambda function returns a value, `runStep()` will return it without modification, and it will not affect the report. During the execution of a step, you can call `runStep()` again to create a sub-step. To change a step's title (defaults to just the word “step”), you can: - add the [`#[DisplayName]`](#title) attribute to the lambda function, - pass the optional `name` argument to `runStep()`, - use the `name()` method on the `StepContextInterface` object. - **No-op steps** If you call `addStep()`, Allure will add to the report a no-op step. This allows for a log-style reporting within a test or within a larger step. A no-op step finishes immediately after it started and cannot have any sub-steps, or parameters. The optional second argument indicates the status that will be shown for the step in the report. Allowed values are: `Status::passed()` (the default), `Status::failed()`, `Status::broken()`, and `Status::skipped()`. During the execution of a method-based step or a lambda step, it is possible to describe the data the step works with, e.g., if you run it multiple times with different data. To do so, use the `parameter()` method of the `StepContextInterface` object. **Method-based steps:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; use Qameta\Allure\Attribute\DisplayName; use Qameta\Allure\StepContextInterface; class TestMyWebsite extends Unit { public function testLabels() { Allure::runStep([$this, 'logIn']); Allure::runStep([$this, 'createLabel']); Allure::runStep([$this, 'checkThatLabelExists']); } #[DisplayName('Log in')] function logIn(StepContextInterface $context) { $context->parameter('Email', 'johndoe@example.com'); $context->parameter('Password', 'qwerty'); // ... } #[DisplayName('Create label')] function createLabel(StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... } #[DisplayName('Check that label exists')] function checkThatLabelExists(StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... } } ``` **Lambda steps:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; use Qameta\Allure\StepContextInterface; class TestMyWebsite extends Unit { public function testLabels() { Allure::runStep(function (StepContextInterface $context) { $context->parameter('Email', 'johndoe@example.com'); $context->parameter('Password', 'qwerty'); // ... }, 'Log in'); Allure::runStep(function (StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... }, 'Create label'); Allure::runStep(function (StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... }, 'Check that label exists'); } } ``` **No-op steps:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { // ... Allure::addStep('Log in'); // ... Allure::addStep('Create label'); // ... Allure::addStep('Check that label exists'); } } ``` ## Parametrized tests - `Allure::parameter(string $name, ?string $value, bool $excluded = false, ?ParameterMode $mode = null)` Specify a `name` and `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. Note that Allure Codeception does not automatically extract the parameters from the Codeception's [`DataProvider`](https://codeception.com/docs/AdvancedUsage#dataprovider-attribute). In a test function that uses a data provider, pass the names and values to `Allure::parameter()` manually, as shown in the example below. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. The `mode` argument affects how the parameter will be displayed in the report. Available options are: - `null` — the parameter and its value will be shown in a table along with other parameters. - `ParameterMode::masked()` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `ParameterMode::hidden()` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. The parameters can be added not only to whole test results, but also to individual steps within them, see [Test steps](#test-steps) for more details. **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testAuthenticationWithUsername() { Allure::parameter('login', 'johndoe'); Allure::parameter('password', 'qwerty'); // ... } public function testAuthenticationWithEmail() { Allure::parameter('login', 'johndoe@example.com'); Allure::parameter('password', 'qwerty'); // ... } } ``` **DataProvider + Runtime API:** ```php use Codeception\Attribute\DataProvider; use Codeception\Test\Unit; use Qameta\Allure\Allure; class ParametersProviderTest extends Unit { protected function dataProvider() { return [ ['login' => 'johndoe', 'password' => 'qwerty'], ['login' => 'johndoe@example.com', 'password' => 'qwerty'], ]; } #[DataProvider('dataProvider')] public function testAuthenticationWithData($login, $password) { Allure::parameter('Login', $login); Allure::parameter('Password', $password); // ... } } ``` ## Attachments - `Allure::attachment(string $name, string $content, ?string $type = null, ?string $fileExtension = null)` - `Allure::attachmentFile(string $name, string $file, ?string $type = null, ?string $fileExtension = null)` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Tip: You can use data produced by any function, not necessarily read from an actual file. To create an attachment using the Runtime API, call `Allure::attachment()` or `Allure::attachmentFile()` at any point during your test. Pass either the `content` or the `path` from which the data will be read. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension as `fileExtension`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. **Runtime API:** ```php use Codeception\Test\Unit; use Qameta\Allure\Allure; class TestMyWebsite extends Unit { public function testLabels() { // ... Allure::attachment('data.txt', 'This is the file content.', 'text/plain'); Allure::attachmentFile('data.txt', '/path/to/image.png', 'image/png'); } } ``` --- ## CodeceptJS > URL: https://allurereport.org/docs/codeceptjs/ # Getting started with Allure CodeceptJS [![Allure CodeceptJS npm latest version](https://img.shields.io/npm/v/allure-codeceptjs?style=flat "Allure CodeceptJS npm latest version")](https://www.npmjs.com/package/allure-codeceptjs) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [CodeceptJS](https://codecept.io/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Acodeceptjs) to see Allure CodeceptJS in action. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure CodeceptJS is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure CodeceptJS integration. **npm:** ```bash npm install --save-dev codeceptjs allure-codeceptjs ``` **yarn:** ```bash yarn add --dev codeceptjs allure-codeceptjs allure-js-commons ``` **pnpm:** ```bash pnpm add -D codeceptjs allure-codeceptjs ``` 1. In your [`codecept.conf.js`](https://codecept.io/configuration/) file, enable the `allure` plugin. ```js exports.config = { tests: "tests/**.test.js", plugins: { allure: { enabled: true, require: "allure-codeceptjs", }, }, }; ``` ### 2. Run tests Run your CodeceptJS tests the same way as your would run them usually. For example: **npm:** ```bash npx codeceptjs run ``` **yarn:** ```bash yarn run codeceptjs run ``` **pnpm:** ```bash pnpx codeceptjs run ``` This will save necessary data into `allure-results` or other directory, according to the [configuration](/docs/codeceptjs-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure CodeceptJS integration extends the standard reporting features of CodeceptJS by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure CodeceptJS provides two different ways to use a feature: the Runtime API and the Tags API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Tags API**: use the [`tag()`](https://codecept.io/advanced/#tags) method to assign various data to a particular scenario. Most of the tags require values. You can use either a colon or an equal sign to separate the value from the name, e.g., `@allure.label.epic:WebInterface` is identical to `@allure.label.epic=WebInterface`. When using this approach, the data is guaranteed to be added to the test result regardless of how the test itself runs. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/codeceptjs-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.owner("John Doe"); await allure.severity("critical"); await allure.tags("Web interface", "Authentication"); // ... }); ``` **Tags API:** ```js Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... }) .tag("@allure.label.owner:JohnDoe") .tag("@allure.label.severity:critical"); .tag("Web interface") .tag("Authentication") ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Tags API:** ```js Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... }) .tag("@allure.label.epic:WebInterface") .tag("@allure.label.feature:EssentialFeatures") .tag("@allure.label.story:Authentication"); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Tags API:** ```js Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... }) .tag("@allure.label.parentSuite:TestsForWebInterface") .tag("@allure.label.suite:TestsForEssentialFeatures") .tag("@allure.label.subSuite:TestsForAuthentication"); ``` ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), you can use the `step()` function, see the [reference](/docs/codeceptjs-reference/#test-steps). ```js const allure = require("allure-js-commons"); const { Status } = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ### Describe parametrized tests If you use the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, call the `allure.parameter()` function to add the parameters to the test report, see the [reference](/docs/codeceptjs-reference/#parametrized-tests). ```js const allure = require("allure-js-commons"); Feature("Test My Website"); let accounts = new DataTable(["login", "password"]); accounts.add(["johndoe", "qwerty"]); accounts.add(["admin", "qwerty"]); Data(accounts).Scenario("Test Authentication", async ({ current }) => { await allure.parameter("Login", current.login); await allure.parameter("Password", current.password); // ... }); ``` ### Set labels globally Any [labels](/docs/codeceptjs-reference/#label), including custom ones, can be set via the environment variables in your operating system. Here's an example (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface npx codeceptjs run ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" npx codeceptjs run ``` ### Attach screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure CodeceptJS reference](/docs/codeceptjs-reference/#attachments). ```js const allure = require("allure-js-commons"); const { ContentType } = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` ### Select tests via a test plan file Danger: Test plan is currently not supported by the Allure CodeceptJS integration. ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, edit the [`environmentInfo`](/docs/codeceptjs-configuration/#environmentinfo) object in the configuration. For example, it is a good idea to use this to remember the OS version and Node.js version retrieved from the [`os`](https://nodejs.org/api/os.html) and [`process`](https://nodejs.org/api/process.html) objects. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/js/environment-allure3-light.png, /images/js/environment-allure3-dark.png, /images/js/environment-allure2-light.png, /images/js/environment-allure2-dark.png ```js const { setCommonPlugins } = require("@codeceptjs/configure"); const os = require("node:os"); setCommonPlugins(); /** @type {CodeceptJS.MainConfig} */ exports.config = { tests: "tests/**/*.js", plugins: { allure: { enabled: true, require: "allure-codeceptjs", environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }, }, }; ``` Note that if your launch includes multiple CodeceptJS runs (see [How it works](/docs/how-it-works/)), Allure CodeceptJS will only save the environment information from the latest run. --- ## Allure CodeceptJS configuration > URL: https://allurereport.org/docs/codeceptjs-configuration/ # Allure CodeceptJS configuration The [Allure CodeceptJS](/docs/codeceptjs/) integration behavior can be affected by some configuration options set in your [CodeceptJS configuration file](https://codecept.io/configuration/), e.g., `codecept.conf.js`. For example: ```js const { setCommonPlugins } = require("@codeceptjs/configure"); const { Status } = require("allure-js-commons"); const os = require("node:os"); setCommonPlugins(); /** @type {CodeceptJS.MainConfig} */ exports.config = { tests: "tests/**/*.js", plugins: { allure: { enabled: true, require: "allure-codeceptjs", resultsDir: "allure-results", links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: (v) => `https://jira.example.com/browse/${v}`, }, }, categories: [ { name: "foo", messageRegex: "bar", traceRegex: "baz", matchedStatuses: [Status.FAILED, Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }, }, }; ``` ## resultsDir Path to the directory where Allure CodeceptJS will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ## links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/codeceptjs-reference/#link)), you can specify the following: - `nameTemplate` — a template or a function for generating the link name when it is not provided. - `urlTemplate` — a template or a function for generating the link address when it is not provided. The templates can be strings (with `%s` where the identifier should be placed) or functions (accepting the identifier and returning the URL). For example, with the configuration above, `await allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/`. Allure will display the link with the appropriate icon for the `issue` type. ## categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). ## environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). --- ## Allure CodeceptJS reference > URL: https://allurereport.org/docs/codeceptjs-reference/ # Allure CodeceptJS reference These are the functions that you can use to integrate your CodeceptJS tests with Allure Report. In most cases, Allure CodeceptJS provides two different ways to use a feature: the Runtime API and the Tags API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Tags API**: use the [`tag()`](https://codecept.io/advanced/#tags) method to assign various data to a particular scenario. Most of the tags require values. You can use either a colon or an equal sign to separate the value from the name, e.g., `@allure.label.epic:WebInterface` is identical to `@allure.label.epic=WebInterface`. When using this approach, the data is guaranteed to be added to the test result regardless of how the test itself runs. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `allure.description(markdown: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.description("This test attempts to log into the website."); // ... }); ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.owner("John Doe"); // ... }); ``` **Tags API:** ```js Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... }).tag("@allure.label.owner:JohnDoe"); ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.tags("Web interface", "Authentication"); // ... }); ``` **Tags API:** ```js Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... }) .tag("Web interface") .tag("Authentication"); ``` ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Runtime API:** ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.severity("critical"); // ... }); ``` **Tags API:** ```js Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... }).tag("@allure.label.severity:critical"); ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` - `@allure.label.⟨NAME⟩:⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. **Runtime API:** ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.label("my custom label", "value"); // ... }); ``` **Tags API:** ```js Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... }).tag("@allure.label.MyCustomLabel:value"); ``` ### ID - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). ```js Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... }).tag("@allure.id:123"); ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string, defaults to “link”), Allure will try to load a corresponding **link template** to process the URL, as defined by the [`links`](/docs/codeceptjs-configuration/#links) configuration option. If no template is found for the given type or if the link already represents a proper URL, it's left unmodified. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.link("https://dev.example.com/", "Website"); await allure.issue("AUTH-123", "https://issues.example.com/AUTH-123"); await allure.tms("TMS-456", "https://tms.example.com/TMS-456"); // ... }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Tags API:** ```js Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... }) .tag("@allure.label.epic:WebInterface") .tag("@allure.label.feature:EssentialFeatures") .tag("@allure.label.story:Authentication"); ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the name of suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Runtime API:** ```js const allure = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Tags API:** ```js Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... }) .tag("@allure.label.parentSuite:TestsForWebInterface") .tag("@allure.label.suite:TestsForEssentialFeatures") .tag("@allure.label.subSuite:TestsForAuthentication"); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test step or sub-step](/docs/steps/) with the given `name`. The `step()` function is asynchronous, and it accepts an asynchronous anonymous function as its second argument. Notice the `async` and `await` keywords in the example below. The anonymous function can accept either no arguments or a single argument of class `StepContext`. This object provides the following methods: - `displayName()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are similar to the test-wide implementation, see [Parametrized tests](#parametrized-tests). To create a step without a body, call the `logStep()` function that accepts a name and, optionally, a step status and an error object. ```js const allure = require("allure-js-commons"); const { Status } = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` Specify a `name` and `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. The `options` argument, if given, must be an object with two optional properties `excluded` and `mode`. - If `excluded` is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. ```js const allure = require("allure-js-commons"); Feature("Test My Website"); let accounts = new DataTable(["login", "password"]); accounts.add(["johndoe", "qwerty"]); accounts.add(["admin", "qwerty"]); Data(accounts).Scenario("Test Authentication", async ({ current }) => { await allure.parameter("Login", current.login); await allure.parameter("Password", current.password); // ... }); ``` ## Attachments - `allure.attachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options in an object (as shown for the image attachment below) or just specify the media type and let Allure deduce the appropriate filename extension automatically (as shown for the text attachment below). In either case, the media type can be a value from the `ContentType` enumeration or any string. ```js const allure = require("allure-js-commons"); const { ContentType } = require("allure-js-commons"); Feature("Test My Website"); Scenario("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` --- ## Cucumber.js > URL: https://allurereport.org/docs/cucumberjs/ # Getting started with Allure Cucumber.js [![Allure Cucumber.js npm latest version](https://img.shields.io/npm/v/allure-cucumberjs?style=flat "Allure Cucumber.js npm latest version")](https://www.npmjs.com/package/allure-cucumberjs) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Cucumber.js](https://cucumber.io/docs/installation/javascript/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Acucumberjs) to see Allure Cucumber.js in action. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure Cucumber.js is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure Cucumber.js integration and make sure that all the packages it needs are installed. **npm:** ```bash npm install --save-dev @cucumber/cucumber @cucumber/messages allure-cucumberjs ``` **yarn:** ```bash yarn add --dev @cucumber/cucumber @cucumber/messages allure-cucumberjs allure-js-commons ``` **pnpm:** ```bash pnpm add -D @cucumber/cucumber @cucumber/messages allure-cucumberjs ``` 1. Enable the Allure formatter for Cucumber.js. See [Configuration](/docs/cucumberjs-configuration/) for more details and options. ```js export default { format: ["allure-cucumberjs/reporter"], formatOptions: { resultsDir: "allure-results", }, }; ``` Warning: In Yarn PnP, the configuration will not work with the default settings. See [a note for Yarn PnP users](/docs/cucumberjs-configuration/#a-note-for-yarn-pnp-users). 1. Write your tests, following the official Cucumber.js documentation and, optionally, using some extended features provided by Allure Cucumber.js, see [Writing tests](#writing-tests). ### 2. Run tests Run your Cucumber.js tests the same way as you would run them usually. For example: **npm:** ```bash npx cucumber-js ``` **yarn:** ```bash yarn run cucumber-js ``` **pnpm:** ```bash pnpx cucumber-js ``` This will save necessary data into `allure-results` or other directory, according to the [configuration](/docs/cucumberjs-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Cucumber.js integration extends the standard reporting features of Cucumber.js by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure Cucumber.js provides two different ways to use a feature: the Runtime API and the Gherkin tags. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Gherkin tags**: add a Gherkin tag of a specific format (may be configured via the [`labels`](/docs/cucumberjs-configuration/#labels) and [`links`](/docs/cucumberjs-configuration/#links) options). When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/cucumberjs-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.owner("John Doe"); await allure.severity("critical"); await allure.tags("Web interface", "Authentication"); // ... }); ``` **Gherkin tags:** ```gherkin Feature: MyFeature @allure.label.owner:JohnDoe @allure.label.severity:critical @WebInterface @Authentication Scenario: MyTest When something should be done Then do something ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Gherkin tags:** ```gherkin Feature: MyFeature @allure.label.epic:WebInterface @allure.label.feature:EssentialFeatures @allure.label.story:Authentication Scenario: MyTest When something should be done Then do something ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Gherkin tags:** ```gherkin Feature: MyFeature @allure.label.parentSuite:TestsForWebInterface @allure.label.suite:TestsForEssentialFeatures @allure.label.subSuite:TestsForAuthentication Scenario: MyTest When something should be done Then do something ``` ### Divide a test into steps Allure Cucumber.js allows you to [create sub-steps](/docs/steps/) within a Gherkin test step. To do so, use the [`allure.step()`](/docs/cucumberjs-reference/#step) function. Sub-steps are convenient for situations when you have a sequence of actions which is: - monolithic enough to be presented as a single step in a Gherkin file; - complex enough to have multiple potential points of failure. ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; Then("do something", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ### Describe parametrized tests With Cucumber.js, you can implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern using the [`Scenario Outline` and `Examples` keywords](https://cucumber.io/docs/gherkin/reference#scenario-outline). There are two approaches to indicate which parameters were used in a test: - Use the angle brackets syntax to insert the parameter values directly into a step title. - Use [`allure.parameter()`](/docs/cucumberjs-reference/#parametrized-tests) to display the values at the test level. In the example below: 1. The `Examples` block defines two values for the `login` variable: “johndoe” and “johndoe\@example.com”. 2. For each value, Cucumber.js uses it instead of the placeholder in the scenario title and step titles, e.g., `authorize as "johndoe"`. 3. Cucumber.js then uses the title to select and run the JavaScript function. The value (e.g., “johndoe”) is passed to it as an argument of type `string`. 4. The function calls [`allure.parameter()`](/docs/cucumberjs-reference/#parametrized-tests) to display the values at the test level in the report. ```gherkin Scenario Outline: test authorization as "" Then authorize as "" Examples: | login | | johndoe | | johndoe@example.com | ``` ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then(`authorize as {string}`, async (login) => { await allure.parameter("auth_method", "password"); await allure.parameter("login", login); // ... }); ``` ### Set labels globally Any [labels](/docs/cucumberjs-reference/#label), including custom ones, can be set via the environment variables in your operating system. Here's an example (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface npx cucumber-js ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" npx cucumber-js ``` ### Attach screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure Cucumber.js reference](/docs/cucumberjs-reference/#attachments). ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; Then("do something", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Cucumber.js will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json` (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json npx cucumber-js ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" npx cucumber-js ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, edit the [`environmentInfo`](/docs/cucumberjs-configuration/#environmentinfo) object in the configuration. For example, it is a good idea to use this to remember the OS version and Node.js version retrieved from the [`os`](https://nodejs.org/api/os.html) and [`process`](https://nodejs.org/api/process.html) objects. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/js/environment-allure3-light.png, /images/js/environment-allure3-dark.png, /images/js/environment-allure2-light.png, /images/js/environment-allure2-dark.png ```js import * as os from "node:os"; import * as process from "node:process"; export default { format: ["allure-cucumberjs/reporter"], formatOptions: { environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }, }; ``` Note that if your launch includes multiple Cucumber.js runs (see [How it works](/docs/how-it-works/)), Allure Cucumber.js will only save the environment information from the latest run. --- ## Cucumber.js configuration > URL: https://allurereport.org/docs/cucumberjs-configuration/ # Allure Cucumber.js configuration When setting up a project using Allure Cucumber.js (see [How to start](/docs/cucumberjs/#how-to-start)), you can pass some configuration options to Cucumber.js in the command line or in the configuration file, e.g., `cucumber.js`. Info: This section describes the configuration parameters for Allure Cucumber.js 3.0 and newer. For an older version, you may check this [readme](https://github.com/allure-framework/allure-js/tree/v2.15.1/packages/allure-cucumberjs). An example of a command-line call with additional options: **npm:** ```bash npx cucumber-js --format allure-cucumberjs/reporter \ --format-options '{ "resultsDir": "results" }' ``` **yarn:** ```bash yarn run cucumber-js --format allure-cucumberjs/reporter \ --format-options '{ "resultsDir": "results" }' ``` **pnpm:** ```bash pnpx cucumber-js --format allure-cucumberjs/reporter \ --format-options '{ "resultsDir": "results" }' ``` Warning: In Yarn PnP, the command will not work with the default settings. See [a note for Yarn PnP users](#a-note-for-yarn-pnp-users) below. An example of a larger configuration in a `cucumber.js` file: ```js import { Status } from "allure-js-commons"; import * as os from "node:os"; import * as process from "node:process"; export default { format: ["allure-cucumberjs/reporter"], formatOptions: { resultsDir: "allure-results", labels: [ { pattern: [/@epic:(.*)/], name: "epic", }, { pattern: [/@severity:(.*)/], name: "severity", }, ], links: { issue: { pattern: [/@issue:(.*)/], urlTemplate: "https://issues.example.com/%s", nameTemplate: "ISSUE %s", }, tms: { pattern: [/@tms:(.*)/], urlTemplate: "https://tms.example.com/%s", }, jira: { pattern: [/@jira:(.*)/], urlTemplate: (v) => `https://jira.example.com/browse/${v}`, }, }, categories: [ { name: "foo", messageRegex: "bar", traceRegex: "baz", matchedStatuses: [Status.FAILED, Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, globalLabels: [{ name: "layer", value: "api" }], }, }; ``` ## A note for Yarn PnP users By default, [Yarn PnP](https://yarnpkg.com/features/pnp) does not let Cucumber.js use third-party formatters (like Allure Cucumber.js). Using the command or the configuration file from the example above will lead to the “Failed to import formatter” error message. There are two options to fix this. - **Option 1: register Allure Cucumber.js as a peer dependency for Cucumber.js.** In the project's [`.yarnrc.yml`](https://yarnpkg.com/configuration/yarnrc) file, write the following section: ```yml packageExtensions: "@cucumber/cucumber@*": peerDependencies: allure-cucumberjs: "*" peerDependenciesMeta: allure-cucumberjs: optional: true ``` - **Option 2: re-export the Allure Cucumber.js module.** 1. Create a file called `reporter.js` with the following line: ```js export { default } from "allure-cucumberjs/reporter"; ``` 1. In the command line or in the configuration file, replace `allure-cucumberjs/reporter` with the path to your file. For example: ```bash yarn run cucumber-js --format ./reporter.js \ --format-options '{ "resultsDir": "results" }' ``` ## Supported options ### resultsDir Path to the directory where Allure Cucumber.js will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ### labels List of rules for automatically adding labels based on [Gherkin tags](https://cucumber.io/docs/cucumber/api/?lang=javascript#tags). To define a rule, add an object with two properties to the list: - `pattern`: a list of regular expressions to match Gherkin tags. Each regular expression must include exactly one capture group. - `name`: the name of the [labels](/docs/cucumberjs-reference/#label) that must be created when tags match the pattern. This label will be assigned the value captured by a regular expression. For example, with the configuration above, the `@severity:minor` tag will produce a [Severity](/docs/cucumberjs-reference/#severity) label with the value “minor”. ### links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/cucumberjs-reference/#link)), you can specify the following: - `nameTemplate` — a template or a function for generating the link name when it is not provided. - `urlTemplate` — a template or a function for generating the link address when it is not provided. The templates can be strings (with `%s` where the identifier should be placed) or functions (accepting the identifier and returning the URL). For example, with the configuration above, `await allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/`. Allure will display the link with the appropriate icon for the `issue` type. ### categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). ### environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ### globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Unlike [`labels`](#labels), which defines rules for mapping Gherkin tags to labels, `globalLabels` are applied statically to all test results regardless of tags. Can be provided as an array of `{ name, value }` label objects: ```js export default { format: ["allure-cucumberjs/reporter"], formatOptions: { globalLabels: [ { name: "layer", value: "api" }, { name: "team", value: "backend" }, ], }, }; ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```js export default { format: ["allure-cucumberjs/reporter"], formatOptions: { globalLabels: { layer: "api", team: "backend", }, }, }; ``` --- ## Cucumber.js reference > URL: https://allurereport.org/docs/cucumberjs-reference/ # Allure Cucumber.js reference These are the functions that you can use to integrate your Cucumber.js tests with Allure. In most cases, Allure Cucumber.js provides two different ways to use a feature: the Runtime API and the Gherkin tags. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Gherkin tags**: add a Gherkin tag of a specific format (may be configured via the [`labels`](/docs/cucumberjs-configuration/#labels) and [`links`](/docs/cucumberjs-configuration/#links) options). When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `allure.description(markdown: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. Alternatively, use the native Gherkin syntax for describing scenarios. **Runtime API:** ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.description("This test attempts to log into the website."); // ... }); ``` **Gherkin description:** ```gherkin Feature: MyFeature Scenario: MyTest This test attempts to log into the website. When something should be done Then do something ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.owner("John Doe"); // ... }); ``` **Gherkin tags:** ```gherkin Feature: MyFeature @allure.label.owner:JohnDoe Scenario: MyTest When something should be done Then do something ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` - `@⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). Alternatively, use the native Gherkin syntax for tags. **Runtime API:** ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.tag("New UI"); await allure.tags("Essentials", "Authentication"); // ... }); ``` **Gherkin tags:** ```gherkin Feature: MyFeature @NewUI @Essentials @Authentication Scenario: MyTest When something should be done Then do something ``` ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Runtime API:** ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; import { Severity } from "allure-js-commons"; Then("do something", async () => { await allure.severity(Severity.CRITICAL); // ... }); ``` **Gherkin tags:** ```gherkin Feature: MyFeature @allure.label.severity:critical Scenario: MyTest When something should be done Then do something ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.label("microservice", "UI"); // ... }); ``` ### ID - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). ```gherkin Feature: MyFeature @allure.id:123 Scenario: MyTest When something should be done Then do something ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string, defaults to “link”), Allure will try to load a corresponding **link template** to process the URL, as defined by the [`links`](/docs/cucumberjs-configuration/#links) configuration option. If no template is found for the given type or if the link already represents a proper URL, it's left unmodified. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.issue("AUTH-123", "Related issue"); await allure.tms("TMS-456", "Related TMS issue"); await allure.link("JIRA-777", "Related Jira issue", "jira"); await allure.link("https://example.com/", "Project website"); // ... }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features**, or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Gherkin tags:** ```gherkin Feature: MyFeature @allure.label.epic:WebInterface @allure.label.feature:EssentialFeatures @allure.label.story:Authentication Scenario: MyTest When something should be done Then do something ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the names of **parent suite**, **suite**, or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Runtime API:** ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Gherkin tags:** ```gherkin Feature: MyFeature @allure.label.parentSuite:TestsForWebInterface @allure.label.suite:TestsForEssentialFeatures @allure.label.subSuite:TestsForAuthentication Scenario: MyTest When something should be done Then do something ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test sub-step](/docs/steps/) with the given `name`. The `step()` function is asynchronous, and it accepts an asynchronous anonymous function as its second argument. Notice the `async` and `await` keywords in the example below. The anonymous function can accept either no arguments or a single argument of class `StepContext`. This object provides the following methods: - `displayName()` — override the sub-step name during its execution. - `parameter()` — indicate arbitrary parameters used for the sub-step. The available signatures of this method are similar to the test-wide implementation, see [Parametrized tests](#parametrized-tests). To create a step without a body, call the `logStep()` function that accepts a name and, optionally, a step status and an error object. ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; Then("do something", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` Specify a `name` and `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. The `options` argument, if given, must be an object with two optional properties `excluded` and `mode`. - If `excluded` is set to true, Allure will not use the parameter when comparing the current test result with the previous one in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. Note, that even when you use the `"masked"` or `"hidden"` mode, it is still possible to extract the value from the `allure_results` directory if you publish it. ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; Then("do something", async () => { await allure.parameter("login", "johndoe"); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); ``` ## Attachments - `allure.attachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` or the `path` from which the data will be read. Additional options may be passed in an object as the third parameter. - The `contentType` and `fileExtension` options control the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options or just specify the `contentType` type and let Allure deduce the appropriate `fileExtension` automatically. In either case, the media type can be a value from the `ContentType` enumeration or any string. - The `encoding` option specifies how the attached text should be saved. It only applies when providing `content` as a string, and it defaults to “utf-8”. If the third argument is not an object, it is interpreted as a single `contentType` option. ```js import { Then } from "@cucumber/cucumber"; import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; Then("do something", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` --- ## Cucumber-JVM > URL: https://allurereport.org/docs/cucumberjvm/ # Getting started with Allure Cucumber-JVM [![Allure Cucumber-JVM latest version](https://img.shields.io/maven-central/v/io.qameta.allure/allure-cucumber7-jvm?style=flat "allure-cucumber7-jvm")](https://mvnrepository.com/artifact/io.qameta.allure/allure-cucumber7-jvm) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Cucumber-JVM](https://cucumber.io/docs/installation/java/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Acucumberjvm%2Ccucumber-jvm) to see Allure Cucumber-JVM in action. ## Setting up Info: This guide is tailored for Cucumber-JVM 7, which is the only version supported by Allure Java 3.x. Allure Java 3.x requires Java 17 or newer. Integrations for Cucumber-JVM 4, 5 and 6 (`allure-cucumber4-jvm`, `allure-cucumber5-jvm` and `allure-cucumber6-jvm`) are only available in Allure Java 2.x. See [Migrating to Allure Java 3](/docs/guides/allure-java-3-migration/). To integrate [Allure](https://allurereport.org) into an existing **Cucumber-JVM** project, you need to: 1. Add Allure dependencies to your project. 2. Activate the Allure Cucumber-JVM plugin. 3. Set up AspectJ for `@Step` and `@Attachment` annotations support. 4. Designate a location for Allure results storage. The specific implementation of these steps varies based on how your Cucumber-JVM is set up. Cucumber-JVM can be used with three different types of runners: - JUnit Platform (recommended) - TestNG - JUnit 4 (deprecated) ### Using JUnit Platform JUnit Platform is the recommended runner for Cucumber-JVM tests. Full configuration demonstrated in [this example](https://github.com/allure-examples/allure-cucumber7-junit-platform-gradle-kts). #### Add Allure dependencies **Maven:** ```xml 3.0.0 io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-cucumber7-jvm test io.qameta.allure allure-junit-platform test ``` **Gradle (Kotlin):** ```kts // Define the version of Allure you want to use via the allureVersion property val allureVersion = "3.0.0" // ... dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) // Add necessary Allure dependencies to dependencies section testImplementation("io.qameta.allure:allure-cucumber7-jvm") testImplementation("io.qameta.allure:allure-junit-platform") } ``` **Gradle (Groovy):** ```groovy // Define the version of Allure you want to use via the allureVersion property def allureVersion = "3.0.0" dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") // Add necessary Allure dependencies to dependencies section testImplementation "io.qameta.allure:allure-cucumber7-jvm" testImplementation "io.qameta.allure:allure-junit-platform" } ``` #### Enable Allure Cucumber-JVM plugin **Using junit-platform.properties file:** ```properties # Create a `src/test/resources/junit-platform.properties` file and add property `cucumber.plugin` with value `io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm`: #junit-platform.properties cucumber.plugin=io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm ``` **Using @ConfigurationParameter:** ```java // Add `org.junit.platform.suite.api.ConfigurationParameter` annotation and specify property `cucumber.plugin` with value `io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm`: import org.junit.platform.suite.api.ConfigurationParameter; import static io.cucumber.core.options.Constants.PLUGIN_PROPERTY_NAME; @Suite @IncludeEngines("cucumber") @ConfigurationParameter( key = PLUGIN_PROPERTY_NAME, value = "io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm" ) class CucumberTest { } ``` ### Using TestNG Info: To ensure compatibility with Cucumber-JVM 7, you must use TestNG 7.8 or higher. If you also use [Allure TestNG](/docs/testng/) with Allure Java 3.x, use TestNG 7.10 or higher. #### Add Allure dependencies **Maven:** ```xml 3.0.0 io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-cucumber7-jvm test io.qameta.allure allure-testng test ``` **Gradle (Kotlin):** ```kts // Define the version of Allure you want to use via the allureVersion property val allureVersion = "3.0.0" // ... dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) // Add necessary Allure dependencies to dependencies section testImplementation("io.qameta.allure:allure-cucumber7-jvm") testImplementation("io.qameta.allure:allure-testng") } ``` **Gradle (Groovy):** ```groovy // Define the version of Allure you want to use via the allureVersion property def allureVersion = "3.0.0" dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") // Add necessary Allure dependencies to dependencies section testImplementation "io.qameta.allure:allure-cucumber7-jvm" testImplementation "io.qameta.allure:allure-testng" } ``` #### Enable Allure Cucumber-JVM plugin Add `io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm` plugin to your Cucumber Options: ```java @Test @CucumberOptions( plugin = { "io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm" } ) public class CucumberTest extends AbstractTestNGCucumberTests { } ``` ### Using JUnit 4 #### Add Allure dependencies **Maven:** ```xml 3.0.0 io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-cucumber7-jvm test io.qameta.allure allure-junit4 test ``` **Gradle (Kotlin):** ```kts // Define the version of Allure you want to use via the allureVersion property val allureVersion = "3.0.0" // ... dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) // Add necessary Allure dependencies to dependencies section testImplementation("io.qameta.allure:allure-cucumber7-jvm") testImplementation("io.qameta.allure:allure-junit4") testImplementation("io.qameta.allure:allure-junit4-aspect") } ``` **Gradle (Groovy):** ```groovy // Define the version of Allure you want to use via the allureVersion property def allureVersion = "3.0.0" dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") // Add necessary Allure dependencies to dependencies section testImplementation "io.qameta.allure:allure-cucumber7-jvm" testImplementation "io.qameta.allure:allure-junit4" testImplementation "io.qameta.allure:allure-junit4-aspect" } ``` #### Enable Allure Cucumber-JVM plugin Add `io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm` plugin to your Cucumber Options: ```java @RunWith(Cucumber.class) @CucumberOptions( plugin = { "io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm" } ) public class CucumberTest { } ``` ### Configure AspectJ Allure leverages AspectJ for the functionality of `@Step` and `@Attachment` annotations. Additionally, certain frameworks, such as JUnit 4 or AssertJ, rely on AspectJ integration to function correctly. These frameworks will require appropriate configuration to ensure they work as intended. **Maven:** ```xml 1.9.25 org.apache.maven.plugins maven-surefire-plugin 3.1.2 -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar" org.aspectj aspectjweaver ${aspectj.version} io.qameta.allure allure-junit4-aspect test ``` **Gradle (Kotlin):** ```kts // Define the version of AspectJ val aspectJVersion = "1.9.25" // Define configuration for AspectJ agent val agent: Configuration by configurations.creating { isCanBeConsumed = true isCanBeResolved = true } dependencies { // Add aspectjweaver dependency agent("org.aspectj:aspectjweaver:${aspectJVersion}") // Add AspectJ integration for Allure testImplementation("io.qameta.allure:allure-junit4-aspect") } // Configure javaagent for test execution tasks.test { jvmArgs = listOf( "-javaagent:${agent.singleFile}" ) } ``` **Gradle (Groovy):** ```groovy // Define the version of AspectJ def aspectJVersion = "1.9.25" // Define configuration for AspectJ agent configurations { agent { canBeResolved = true canBeConsumed = true } } dependencies { // Add aspectjweaver dependency agent "org.aspectj:aspectjweaver:$aspectJVersion" // Add AspectJ integration for Allure testImplementation "io.qameta.allure:allure-junit4-aspect" } // Configure javaagent for test execution test { jvmArgs = [ "-javaagent:${configurations.agent.singleFile}" ] } ``` ### Specifying Allure Results location Allure, by default, saves test results in the project's root directory. However, it is recommended to store your test results in the build output directory. To configure this, create an `allure.properties` file and place it in the test resources directory of your project, which is typically located at `src/test/resources`: **Maven:** ```properties #allure.properties allure.results.directory=target/allure-results ``` **Gradle (Kotlin):** ```properties #allure.properties allure.results.directory=build/allure-results ``` **Gradle (Groovy):** ```properties #allure.properties allure.results.directory=build/allure-results ``` ### Running tests Execute your Cucumber-JVM tests as you normally would. Below are the commands for Gradle and Maven users: For Gradle: **MacOS/Linux:** ```bash ./gradlew test ``` **Windows:** ```bash gradlew test ``` For Maven: **MacOS/Linux:** ```bash ./mvnw verify ``` **Windows:** ```bash mvnw verify ``` After running the tests, Allure will gather the test execution data and store it in the `allure-results` directory. You can then generate an HTML report from these results using Allure's reporting tools. ## Writing tests The Allure Cucumber-JVM integration extends the standard reporting features of Cucumber-JVM by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#adding-metadata). - **Test Organization**: Structure your tests into [clear hierarchies](#organize-tests) for better readability and organization. - **Step Division**: Break down tests into smaller [test steps](#creating-sub-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#support-for-scenario-outlines) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attaching-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#selective-test-run-allure-testops), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. ### Adding Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/cucumberjvm-reference/#metadata) for an exhaustive list of what can be added. You can assign metadata to a Cucumber-JVM features in two ways: 1. **Using Gherkin tags**: Use a naming convention to add metadata to test scenarios. 2. **Runtime API**: Use a method call within the body of a test step method to dynamically assign metadata during test runtime. This approach allows for the construction of strings and values on-the-fly. When assigning metadata to your tests using Allure's Runtime API, it's crucial to consider the balance between control and potential risks. While the Runtime API offers more flexibility in handling metadata dynamically, it also comes with certain drawbacks. For instance, if a test fails before all the dynamic metadata is applied, the resulting report may lack important information, leading to reduced clarity and completeness. Therefore, it's essential to ensure that metadata is set early in the test to mitigate this risk. **Gherkin Tags:** ```gherkin @allure.label.layer:web @allure.label.owner:eroshenkoam @allure.label.page:/{org}/{repo}/labels Feature: Labels @critical @allure.label.jira:AE-2 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java import io.cucumber.java.en.Given; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.label("layer", "web"); Allure.label("owner", "eroshenkoam"); Allure.label("page", "/{org}/{repo}/labels"); Allure.label("severity", "critical"); Allure.label("jira", "AE-2"); // ... } } ``` Warning: To use Gherkin tags for assigning data, be sure to upgrade to Allure Report 2.27 or higher or consider uploading your test results to [Allure TestOps](https://qameta.io/). ### Organize tests Allure facilitates enhanced navigation in test reports by allowing tests to be organized into hierarchical structures. As explained in the section on [Improving navigation in your test report](/docs/v3/navigation/), Allure Cucumber-JVM supports this feature. To define a test's position within the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Gherkin Tags:** ```gherkin @allure.label.epic:Web Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java import io.cucumber.java.en.Given; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.epic("Web"); // ... } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Gherkin Tags:** ```gherkin @allure.label.parentSuite:Cucumber @allure.label.suite:Labels @allure.label.subSuite:Authorized Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java import io.cucumber.java.en.Given; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.label("parentSuite", "Cucumber"); Allure.label("suite", "Labels"); Allure.label("subSuite", "Authorized"); // ... } } ``` A test's placement within the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy) is determined by the location of the feature files, relative to test resources directory. ### Creating Sub-Steps Allure Cucumber-JVM enhances test reports by allowing the creation of sub-steps within a single Gherkin test step, which can be particularly useful in cases where you have a set of actions that are: - Sufficiently cohesive to be described as one step in a Gherkin scenario. - Intricate enough that they involve several potential failure points. Allure provides three methods for defining sub-steps: 1. **Annotated Sub-Steps**: Defined using `@Step` annotation. 2. **Lambda Sub-Steps**: Utilizing lambda expressions for more concise and inline step definitions. 3. **No-Op Sub-Steps**: Employed as placeholders or for structuring steps without additional functionality. For detailed guidance on implementing each type of sub-step, consult the [test steps reference](/docs/cucumberjvm-reference/#test-steps). **Annotated sub-steps:** ```java import io.cucumber.java.en.Then; import io.qameta.allure.Step; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { subStep1(); subStep2(); } @Step("Sub-step 1") public void subStep1() { subSubStep1(); subSubStep2(); } @Step("Sub-sub-step 1") public void subSubStep1() { // ... } @Step("Sub-sub-step 2") public void subSubStep2() { // ... } @Step("Sub-step 2") public void subStep2() { // ... } } ``` **Lambda steps and no-op steps:** ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.step("Sub-step 1", (step) -> { // ... Allure.step("Sub-sub-step 1"); // ... Allure.step("Sub-sub-step 2"); }); Allure.step("Sub-step 2", (step) -> { // ... }); } } ``` ### Support for Scenario Outlines Allure Cucumber-JVM provides complete support for [Scenario Outlines](https://cucumber.io/docs/gherkin/reference/#scenario-outline), a feature of Cucumber-JVM that allows for parametrized tests. No special setup is needed to take advantage of this capability within Allure. ```gherkin Feature: Parameters Scenario Outline: test authorization as "" Then authorize as "" Examples: | login | | johndoe | | johndoe@example.com | ``` ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @Then("authorize as {string}") public void testAuthentication(String login) { // ... } } ``` Furthermore, Allure's Runtime API can be utilized to include extra parameters in the test report, offering enhanced documentation capabilities and greater detail for each test scenario: ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @Then("authorize as {string}") public void testAuthentication(String login) { Allure.parameter("Login", login); // ... } } ``` ### Attaching screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. Allure Cucumber-JVM offers multiple methods for creating attachments, whether from pre-existing files or from content generated on-the-fly. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure Cucumber-JVM reference](/docs/cucumberjvm-reference/#attachments). ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() throws IOException { Allure.attachment("data.txt", "This is the file content"); try (InputStream is = Files.newInputStream(Paths.get("/path/img.png"))) { Allure.attachment("image.png", is); } // ... } } ``` ### Selective test run (Allure TestOps) Warning: For this feature to work, the [AspectJ integration for Allure Report](#configure-aspectj) must be added to the project. In Allure TestOps, you have the option to rerun only a subset of tests by utilizing the same Continuous Integration (CI) job that was used for the full test run. The capability to perform selective test runs varies depending on the test runner you are using: - JUnit Platform Runner: This runner supports selective test runs natively, allowing you to easily specify which tests to include in the rerun without any additional setup. - JUnit 4 Runner: For Gradle users, JUnit 4 also supports selective test runs right out of the box. However, Maven users might encounter some issues with this functionality due to limitations within the Maven Surefire plugin. - TestNG Runner: To use selective test runs with TestNG, you must be using TestNG version 7.8 or later. Additionally, some extra configurations might be needed to enable this feature. #### TestNG For integrating selective test runs in a TestNG project with Allure TestOps, follow these steps: 1. Create `CucumberTestNgFilter`: Implement a filter class named `CucumberTestNgFilter`. This class will determine which tests to run based on your criteria. Place this class within your project's source directory, ensuring it's correctly packaged. 1. Register `CucumberTestNgFilter`: In your project, create a file named `META-INF/services/org.testng.ITestNGListener` in your test resources. Inside this file, write the fully qualified name of your `CucumberTestNgFilter` class. This step is crucial as it registers your filter as a TestNG listener, which TestNG will invoke during the test run. ```java // Step 1: Create CucumberTestNgFilter.java package com.yourproject.filters; import io.cucumber.testng.FeatureWrapper; import io.cucumber.testng.Pickle; import io.cucumber.testng.PickleWrapper; import org.testng.IDataProviderInterceptor; import org.testng.IDataProviderMethod; import org.testng.ITestContext; import org.testng.ITestNGMethod; import java.nio.file.Paths; import java.util.ArrayList; import java.util.Iterator; import java.util.List; public class CucumberTestNgFilter implements IDataProviderInterceptor { private static final String CUCUMBER_WORKING_DIR = Paths.get("").toUri().toString(); private static final String CLASSPATH_PREFIX = "classpath:"; private static final String CUCUMBER_RUN_METHOD = "io.cucumber.testng.AbstractTestNGCucumberTests.runScenario"; @Override public Iterator intercept( final Iterator original, final IDataProviderMethod dataProviderMethod, final ITestNGMethod method, final ITestContext iTestContext ) { if (!CUCUMBER_RUN_METHOD.equals(method.getQualifiedName())) { return original; } final List filtered = new ArrayList<>(); original.forEachRemaining(objects -> { if (objects.length != 2) { filtered.add(objects); } final PickleWrapper first = (PickleWrapper) objects[0]; final FeatureWrapper second = (FeatureWrapper) objects[1]; final Pickle pickle = first.getPickle(); final String fullName = String.format("%s:%d", getUri(pickle), pickle.getLine() ); System.out.println(fullName); filtered.add(new Object[]{first, second}); }); return filtered.iterator(); } private String getUri(final Pickle pickle) { final String testCaseUri = pickle.getUri().toString(); if (testCaseUri.startsWith(CUCUMBER_WORKING_DIR)) { return testCaseUri.substring(CUCUMBER_WORKING_DIR.length()); } if (testCaseUri.startsWith(CLASSPATH_PREFIX)) { return testCaseUri.substring(CLASSPATH_PREFIX.length()); } return testCaseUri; } } ``` For Step 2, the structure of the project should include: ```text src/ └── test/ └── resources/ └── META-INF/ └── services/ └── org.testng.ITestNGListener ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS and Java versions. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/java/environment-allure3-light.png, /images/java/environment-allure3-dark.png, /images/java/environment-allure2-light.png, /images/java/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](/docs/cucumberjvm-reference/#parametrized-tests). --- ## Cucumber-JVM configuration > URL: https://allurereport.org/docs/cucumberjvm-configuration/ # Allure Cucumber-JVM configuration This page describes the configuration options that affect the behavior of [Allure Cucumber-JVM](/docs/cucumberjvm/). To set the configuration options, create a `allure.properties` file in your project's `src/test/resources` directory. Allure Cucumber-JVM loads the configuration into the JVM system properties for the test runtime. ## allure.label.⟨NAME⟩ ```properties allure.label.epic = Authentication allure.label.severity = blocker ``` Assign an arbitrary [label](/docs/cucumberjvm-reference/#label) to all the tests in the current test run. ## allure.link.⟨TYPE⟩.pattern ```properties allure.link.issue.pattern = https://jira.example.org/browse/{} ``` Define a **link pattern** — a template that can be used to construct full URLs from short identifiers, see the [reference](/docs/cucumberjvm-reference/#link). The value must be a string pattern containing `{}` at the position where the identifier should be placed. The pattern will be applied when processing any link of the given `TYPE`. You can use the option multiple times, thus defining multiple link patterns for different link types. ## allure.results.directory ```properties allure.results.directory = build/allure-results ``` Path to the directory where Allure Cucumber-JVM will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. --- ## Cucumber-JVM reference > URL: https://allurereport.org/docs/cucumberjvm-reference/ # Allure Cucumber-JVM reference These are the functions that you can use to integrate your Cucumber-JVM tests with Allure. Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/cucumberjvm-reference/#metadata) for an exhaustive list of what can be added. You can assign metadata to a Cucumber-JVM features in two ways: 1. **Using Gherkin tags**: Use a naming convention to add metadata to test scenarios. 2. **Runtime API**: Use a method call within the body of a test step method to dynamically assign metadata during test runtime. This approach allows for the construction of strings and values on-the-fly. When assigning metadata to your tests using Allure's Runtime API, it's crucial to consider the balance between control and potential risks. While the Runtime API offers more flexibility in handling metadata dynamically, it also comes with certain drawbacks. For instance, if a test fails before all the dynamic metadata is applied, the resulting report may lack important information, leading to reduced clarity and completeness. Therefore, it's essential to ensure that metadata is set early in the test to mitigate this risk. Warning: To use Gherkin tags for assigning data, be sure to upgrade to Allure Report 2.27 or higher or consider uploading your test results to [Allure TestOps](https://qameta.io/). ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `Allure.getLifecycle().updateTest(Consumer update)` Set the test's [title](/docs/v3/readability/#title). Info: In Allure Java 2.x, the `AllureLifecycle` method for updating the test result is called `updateTestCase()`. ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.getLifecycle().updateTest(result -> result.setName("Some modified scenario name")); // ... } } ``` ### Description - `Allure.description(String description)` - `Allure.descriptionHtml(String description)` Set the test's [description](/docs/v3/readability/#description). You can either [begin a Feature with a description](https://cucumber.io/docs/gherkin/reference/#feature) in your Gherkin file or call the methods for updating the values dynamically. Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. **Gherkin description:** ```gherkin Feature: Labels This test attempts to create a label with specified title Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.description("This test attempts to create a label with specified title"); // ... } } ``` ### Owner Set the test's [owner](/docs/v3/readability/#owner). You can either use the `@allure.label.owner` tags in your Gherkin file or call the [`label()`](#label) method for updating the values dynamically. Note that when using tags in Gherkin, the value cannot contain spaces. **Gherkin tags:** ```gherkin @allure.label.owner=JohnDoe Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.label("owner", "JohnDoe"); // ... } } ``` ### Tag Set the test's [tags](/docs/v3/readability/#tags). You can either use the [tags in your Gherkin file](https://cucumber.io/docs/cucumber/api/?lang=java#tags) or call the [`label()`](#label) method for updating the values dynamically. **Gherkin tags:** ```gherkin @ui @labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.label("tag", "ui"); Allure.label("tag", "labels"); // ... } } ``` ### Severity Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Gherkin tags:** ```gherkin Feature: Labels @critical Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.label("severity", "critical"); // ... } } ``` ### Label - `Allure.label(String name, String value)` Allure labels are key-value pairs used to add additional meta-information to your tests. They can be considered as a way to categorize, organize, and search for test cases within your Allure reports. Labels can be used to specify things like the test case ID, the severity of the test, the feature being tested, or the story that relates to the test. This meta-information helps to create more detailed and structured reports. **Gherkin Tags:** ```gherkin @allure.label.layer:web @allure.label.owner:eroshenkoam @allure.label.page:/{org}/{repo}/labels Feature: Labels @critical @allure.label.jira:AE-2 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java import io.cucumber.java.en.Given; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.label("layer", "web"); Allure.label("owner", "eroshenkoam"); Allure.label("page", "/{org}/{repo}/labels"); Allure.label("severity", "critical"); Allure.label("jira", "AE-2"); // ... } } ``` ### Allure ID (Allure TestOps) You can specify test case ID for your Cucumber-JVM tests using a Gherkin tag or via the Runtime API: **Gherkin Tags:** ```gherkin Feature: Labels @allure.id:123 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java import io.cucumber.java.en.When; import io.qameta.allure.Allure; public class CucumberSteps { @When("I open labels page") public void openLabelsPage() { Allure.label("ALLURE_ID", "123"); // ... } } ``` ### Link - `Allure.link(String url)` - `Allure.link(String name, String url)` - `Allure.link(String name, String type, String url)` - `Allure.issue(String name, String url)` - `Allure.tms(String name, String url)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`allure.link.*.pattern`](/docs/cucumberjvm-configuration/#allure-link-⟨type⟩-pattern) configuration option. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.link("Website", "https://dev.example.com/"); Allure.issue("AUTH-123", "https://example.com/issues/AUTH-123"); Allure.tms("TMS-456", "https://example.com/tms/TMS-456"); // ... } } ``` ## Behavior-based hierarchy - `Allure.epic(String value)` - `Allure.feature(String value)` - `Allure.story(String value)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). You can either use the `@allure.label.*` tags in your Gherkin file or call the methods for updating the values dynamically. Note that when using tags in Gherkin, the values cannot contain spaces. **Gherkin tags:** ```gherkin @allure.label.epic:WebInterface @allure.label.feature:EssentialFeatures @allure.label.story:Labels Feature: My feature Scenario: My scenario Then my step ``` **Runtime API:** ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.epic("Web interface"); Allure.feature("Essential features"); Allure.story("Labels"); // ... } } ``` ## Suite-based hierarchy - `Allure.suite(String value)` Assign the name of suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). You can either use the `@allure.label.*` tags in your Gherkin file or call the `suite()` and [`label()`](#label) methods for updating the values dynamically. Note that when using tags in Gherkin, the values cannot contain spaces. **Gherkin tags:** ```gherkin @allure.label.parent_suite:WebInterface @allure.label.suite:EssentialFeatures @allure.label.sub_suite:Labels Feature: My feature Scenario: My scenario Then my step ``` **Runtime API:** ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.label("parentSuite", "Tests for web interface"); Allure.suite("Tests for essential features"); Allure.label("subSuite", "Tests for labels"); // ... } } ``` ## Test steps - `@Step(String value="")` - `Allure.step(String name)` - `Allure.step(String name, Status status)` - `Allure.step(String name, ThrowableRunnableVoid runnable)` - `Allure.step(String name, ThrowableRunnable runnable)` - `Allure.step(ThrowableContextRunnableVoid runnable)` - `Allure.step(String name, ThrowableContextRunnableVoid runnable)` - `Allure.step(String name, ThrowableContextRunnable runnable)` - `Allure.step(ThrowableContextRunnable runnable)` Define a [test step](/docs/steps/) with the given `name`. There are three ways of defining a step. - “An annotated step”: define a method containing a test step and decorate it with `@Step()`. - “A lambda step”: write a test step in a lambda function and pass it to `Allure.step()`. - “A no-op step”: just pass a `name` and an optional `status` to `Allure.step()` to immediately add a corresponding entry to the results as a sub-step of the current step. When you are implementing a function for a lambda step, it can accept either no arguments or a single argument of class `StepContext`. This object provides the following methods: - `name()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are the same as for the test-wide implementation, see [Parametrized tests](#parametrized-tests). **Annotated steps:** ```java import io.cucumber.java.en.Then; import io.qameta.allure.Step; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { step1(); step2(); } @Step("Step 1") public void step1() { subStep1(); subStep2(); } @Step("Sub-step 1") public void subStep1() { // ... } @Step("Sub-step 2") public void subStep2() { // ... } @Step("Step 2") public void step2() { // ... } } ``` **Lambda steps and no-op steps:** ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() { Allure.step("Step 1", (step) -> { // ... Allure.step("Sub-step 1"); // ... Allure.step("Sub-step 2"); }); Allure.step("Step 2", (step) -> { // ... }); } } ``` ## Parametrized tests - `Allure.parameter(String name, T value)` - `Allure.parameter(String name, T value, Boolean excluded)` - `Allure.parameter(String name, T value, Parameter.Mode mode)` - `Allure.parameter(String name, T value, Boolean excluded, Parameter.Mode mode)` Specify a `name` and `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. Warning: Parameters added via `Allure.parameter()` are part of the test's identity, together with the parameters provided by the test framework. If a parameter's value changes between runs (for example, a timestamp or a random ID), the test will be treated as a new test on every run for the purposes of [History and retries](/docs/history-and-retries/). To avoid this, mark such parameters as excluded, see below. In Allure Java 2.x, parameters added via `Allure.parameter()` did not affect the test's identity. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous ones in the history. In Allure Java 2.x, this argument is only used by Allure TestOps. The `mode` argument affects how the parameter will be displayed in the report. Available options are defined in the `Parameter.Mode` enumeration: - `Parameter.Mode.DEFAULT` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `Parameter.Mode.MASKED` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `Parameter.Mode.HIDDEN` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; public class CucumberSteps { @Then("authorize as {string}") public void testAuthentication(String login) { Allure.parameter("auth_method", "password"); Allure.parameter("login", login); // ... } @Then("authorize with empty login") public void testAuthenticationWithEmptyLogin() { Allure.parameter("login", ""); Allure.parameter("auth_method", "password"); // ... } } ``` ## Attachments Functions for [adding attachments](/docs/attachments/) to test results. ### Attaching content from variables - `Allure.attachment(String name, String content)` - `Allure.attachment(String name, String type, String content)` - `Allure.attachment(String name, String type, String content, AttachmentOptions options)` - `Allure.attachment(String name, InputStream content)` - `Allure.attachment(String name, String type, InputStream content, AttachmentOptions options)` Add `content` as an attachment to the test result under the given `name` (defaults to a unique pseudo-random string). Info: In Allure Java 2.x, the same functionality is provided by the `Allure.addAttachment()` methods, which accept the file extension as a string. They were removed in Allure Java 3.0. Tip: You can use data produced by any function, not necessarily read from an actual file. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension via `AttachmentOptions.withFileExtension(".txt")` as the `options` argument. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. ```java import io.cucumber.java.en.Then; import io.qameta.allure.Allure; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() throws IOException { // ... Allure.attachment("data.txt", "This is the file content."); Allure.attachment("img.png", Files.newInputStream(Paths.get("/path/img.png"))); } } ``` ### Generating attachments dynamically - `@Attachment(String value="", String type="", String fileExtension="")` Define a function that returns the data you need to attach and annotate it with `@Attachment`. Call the function at any point during your test to attach the data to the test result. The data will be processed as following: - If the data type is `byte[]`, Allure will use it without modification. - If the data type is `String`, Allure will convert it to `byte[]`. - If the data is of any other type, Allure will call the `toString()` method and then convert the string to `byte[]`. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension as `fileExtension`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. ```java import io.cucumber.java.en.Then; import io.qameta.allure.Attachment; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; public class CucumberSteps { @When("^I open labels page$") public void openLabelsPage() throws IOException { // ... attachDataTXT(); attachScreenshotPNG(); } @Attachment(value = "data", type = "text/plain", fileExtension = ".txt") public String attachDataTXT() { return "This is the file content."; } @Attachment(value = "screenshot", type = "image/png", fileExtension = ".png") public byte[] attachScreenshotPNG() throws IOException { return Files.readAllBytes(Paths.get("/path/to/image.png")); } } ``` Info: Additionally, Allure provides a way to spawn threads that will add attachments to the test results asynchronously without blocking the test runner. While not recommended for the majority of cases, this approach can improve tests that otherwise have to wait for processing very large files, such as videos larger than 1 GB. To do so, pass a `CompletionStage` that produces the attachment's content to `Allure.attachmentAsync(String name, String type, CompletionStage body)`. In Allure Java 2.x, use `Allure.addByteAttachmentAsync()` or `Allure.addStreamAttachmentAsync()` instead. ## Results history For Cucumber-JVM, the flaky/muted/known status flags are set via **Gherkin tags** on the scenario, not via Java annotations. ### Flaky Tag your scenario with `@FLAKY` to mark it as known-unstable. See [Flaky tests](/docs/test-stability/#flaky-tests). ```gherkin @FLAKY Scenario: Intermittently failing checkout When I place an order Then the order is confirmed ``` ### Muted Tag your scenario with `@MUTED` to exclude it from the report statistics. The test still runs and its result is recorded. ```gherkin @MUTED Scenario: Known broken payment flow When I pay with an expired card Then the payment is rejected ``` ### Known Tag your scenario with `@KNOWN` to flag it as a known issue that is currently failing. ```gherkin @KNOWN Scenario: Unresolved bug in password reset When I reset my password Then I receive a confirmation email ``` --- ## Cucumber.rb > URL: https://allurereport.org/docs/cucumberrb/ # Getting started with Allure Cucumber.rb [![Allure Cucumber.rb gem latest version](https://img.shields.io/gem/v/allure-cucumber?style=flat "Allure Cucumber.rb gem latest version")](https://rubygems.org/gems/allure-cucumber) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Cucumber.rb](https://cucumber.io/docs/installation/ruby/) tests. Info: Check out the example project at [github.com/allure-examples/allure-cucumber-example](https://github.com/allure-examples/allure-cucumber-example) to see Allure Cucumber.rb in action. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Add Allure Cucumber.rb to your project's `Gemfile`. For example: ```ruby source 'https://rubygems.org' gem 'allure-cucumber', '~> 2.23.0' gem 'cucumber', '~> 9.1' ``` 1. Open a terminal, go to the project directory and install the dependencies from the `Gemfile`. For example, if you use [Bundler](https://bundler.io/): ```bash cd /home/user/myproject bundle install ``` 1. In the project's [`cucumber.yml`](https://cucumber.io/docs/cucumber/configuration/?lang=ruby) file, specify the Allure's formatter for Cucumber.rb. ```yaml default: --format AllureCucumber::CucumberFormatter ``` ### 2. Run tests Run your Cucumber.rb tests same way as your would run them usually. For example: ```bash bundle exec cucumber ``` This will save necessary data into `allure-results` or other directory, according to the [Configuration](/docs/cucumberrb-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Cucumber.rb integration extends the standard reporting features of Cucumber.rb by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#adding-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. ### Adding Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/cucumberrb-reference/#metadata) for an exhaustive list of what can be added. ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.description_html 'This test attempts to log into the website using a login and a password.' Allure.label 'owner', 'John Doe' # ... end ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.epic 'Web interface' Allure.feature 'Essential features' Allure.story 'Authentication' # ... end ``` **Gherkin tags:** ```gherkin Feature: Labels @EPIC:WebInterface @FEATURE:EssentialFeatures @STORY:Authentication Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.label 'parentSuite' 'Web interface' Allure.suite 'Essential features' Allure.label 'subSuite', 'Authentication' # ... end ``` ### Divide a test into steps Allure Cucumber.rb provides three ways of [creating steps and sub-steps](/docs/steps/): “annotated steps”, “block-based steps” and “no-op steps”, see the [reference](/docs/cucumberrb-reference/#test-steps). **Annotated steps:** ```ruby require 'allure-cucumber' When 'I open labels page' do Steps.step_1 Steps.step_2 end class Steps extend AllureStepAnnotation step 'Step 1' def self.step_1 step_1_1 step_1_2 end step 'Step 1.1' def self.step_1_1 # ... end step 'Step 1.2' def self.step_1_2 # ... end step 'Step 2' def self.step_2 step_2_1 step_2_2 end step 'Step 2.1' def self.step_2_1 # ... end step 'Step 2.2' def self.step_2_2 # ... end end ``` **Block-based steps and no-op steps:** ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.run_step 'Step 1' do # ... Allure.step name: 'Step 1.1', status: :passed # ... Allure.step name: 'Step 1.2', status: :passed end Allure.run_step 'Step 2' do # ... Allure.step name: 'Step 2.1', status: :passed # ... Allure.step name: 'Step 2.2', status: :passed end end ``` ### Describe parametrized tests An Allure test report can reflect two ways in which you can pass data from your Gherkin file to your Ruby implementation code, namely: - [scenario outlines](/docs/cucumberrb-reference/#scenario-outlines), - [runtime parameters](/docs/cucumberrb-reference/#runtime-parameters). The example below shows a Gherkin file and a Ruby implementation file of a test. In this example, the four parameters for the “I enter my details...” step will be displayed in both instances of the scenario in the test report. ```gherkin Feature: User management Scenario Outline: Registration When I go to the registration form And I enter my details: , , , Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ```ruby require 'allure-cucumber' When 'I go to the registration form' do # ... end And /^I enter my details: (.*), (.*), (.*), (.*)$/ do |login, password, name, birthday| # ... end Then 'the profile should be created' do # ... end ``` ### Attach screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure Cucumber.rb reference](/docs/cucumberrb-reference/#attachments). ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.add_attachment name: 'Screenshot', source: File.new('/path/to/image.png'), type: Allure::ContentType::PNG Allure.add_attachment name: 'Data', source: 'This is the file content.', type: Allure::ContentType::TXT # ... end ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Cucumber.rb will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json bundle exec cucumber ``` **Windows:** ```plain setx ALLURE_TESTPLAN_PATH "testplan.json" bundle exec cucumber ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, specify the information in the [`environment_properties`](/docs/cucumberrb-configuration/#environment-properties) configuration parameter. For example, it is a good idea to use this to remember the OS version and Ruby version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/ruby/environment-allure3-light.png, /images/ruby/environment-allure3-dark.png, /images/ruby/environment-allure2-light.png, /images/ruby/environment-allure2-dark.png ```ruby require 'allure-cucumber' AllureCucumber.configure do |config| config.environment_properties = { os_platform: RbConfig::CONFIG['host_os'], ruby_version: RUBY_VERSION, } end ``` Note that if your launch includes multiple Cucumber.rb runs (see [How it works](/docs/how-it-works/)), Allure Cucumber.rb will only save the environment information from the latest run. --- ## Cucumber.rb configuration > URL: https://allurereport.org/docs/cucumberrb-configuration/ # Allure Cucumber.rb configuration Behavior of the [Allure Cucumber.rb](/docs/cucumberrb/) integration can be customized by calling the `AllureCucumber.configure()` function in your project's `features/support/env.rb` file. For example: ```ruby require 'allure-cucumber' AllureCucumber.configure do |config| config.results_directory = 'allure-results' config.clean_results_directory = true end ``` ## results_directory Path to the directory where Allure Cucumber.rb will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to “reports/allure-results”. ```ruby require 'allure-cucumber' AllureCucumber.configure do |config| config.results_directory = 'allure-results' end ``` ## clean_results_directory If set to `true`, the results directory will be cleaned before generating new test results. By default, the existing data is kept intact, which allows combining results of multiple test runs into a single test report. ```ruby require 'allure-cucumber' AllureCucumber.configure do |config| config.clean_results_directory = true end ``` ## logger, logging_level Configure the logger to which Allure Cucumber.rb will send messages about the beginning and end of each test or test step. ```ruby require 'allure-cucumber' AllureCucumber.configure do |config| config.logger = Logger.new($stdout, Logger::DEBUG) config.logging_level = Logger::INFO end ``` ## link_issue_pattern, link_tms_pattern Define templates that can be used to construct full URLs from short identifiers, see the [reference](/docs/cucumberrb-reference/#link). The pattern must contain `{}` at the position where the identifier should be placed. ```ruby require 'allure-cucumber' AllureCucumber.configure do |config| config.link_issue_pattern = 'http://www.jira.com/browse/{}' config.link_tms_pattern = 'http://www.jira.com/browse/{}' end ``` ## global_labels Labels that will be added to every test result, see the [reference](/docs/cucumberrb-reference/#label). Each label is a hash with the `:name` and `:value` keys. The labels don't replace the test's own labels, including the ones with the same name. By default, no global labels are added. ```ruby require 'allure-cucumber' AllureCucumber.configure do |config| config.global_labels = [ { name: 'layer', value: 'e2e' }, { name: 'owner', value: 'qa-team' }, ] end ``` ## environment_properties Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ```ruby require 'allure-cucumber' AllureCucumber.configure do |config| config.environment_properties = { os_platform: RbConfig::CONFIG['host_os'], ruby_version: RUBY_VERSION, } end ``` ## categories Path to the [categories file](/docs/how-it-works-categories-file/). Allure Cucumber.rb will copy this file to the result directory, so that it can be used for analyzing the test results in the report. ```ruby require 'allure-cucumber' AllureCucumber.configure do |config| config.categories = File.new('my_custom_categories.json') end ``` ## Gherkin tag prefixes Allure Cucumber.rb supports setting certain details of a test via special Gherkin tags on the feature level or the scenario level. You can configure which prefixes Allure Cucumber.rb will recognize as such special tags. - The tag prefix for setting the test's [severity](/docs/cucumberrb-reference/#severity) is specified by `severity_prefix`. - The tag prefixes for adding [issue and TMS links](/docs/cucumberrb-reference/#link) to the test are specified by `issue_prefix` and `tms_prefix`. - The tag prefixes for placing a test into the [behavior-based hierarchy](/docs/cucumberrb-reference/#behavior-based-hierarchy) are specified by `epic_prefix`, `feature_prefix`, `story_prefix`. By default, Allure Cucumber.rb uses the values shown in the example below. ```ruby require 'allure-cucumber' AllureCucumber.configure do |config| config.tms_prefix = 'TMS:' config.issue_prefix = 'ISSUE:' config.severity_prefix = 'SEVERITY:' config.epic_prefix = 'EPIC:' config.feature_prefix = 'FEATURE:' config.story_prefix = 'STORY:' end ``` --- ## Cucumber.rb reference > URL: https://allurereport.org/docs/cucumberrb-reference/ # Allure Cucumber.rb reference These are the functions that you can use to integrate your Cucumber.rb tests with Allure. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `Allure::description_html: (String html)` Set the test's [description](/docs/v3/readability/#description). You can either [begin a Scenario with a description](https://cucumber.io/docs/gherkin/reference/#feature) in your Gherkin file or call `Allure.description_html()` for updating it dynamically. **Runtime API:** ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.description_html 'This test attempts to create a label with specified title' # ... end ``` **Gherkin description:** ```gherkin Feature: Labels Scenario: Create new label for authorized user This test attempts to create a label with specified title When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Owner Set the test's [owner](/docs/v3/readability/#owner). ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.label 'owner', 'John Doe' # ... end ``` ### Tag - `Allure::tag: (String tag)` Set the test's [tags](/docs/v3/readability/#tags). You can either use the [tags in your Gherkin file](https://cucumber.io/docs/cucumber/api/?lang=java#tags) or call `Allure.tag()` for setting tags dynamically. **Gherkin tags:** ```gherkin @authentication @important Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.tag 'authentication' Allure.tag 'important' # ... end ``` ### Severity - `@SEVERITY:trivial` - `@SEVERITY:minor` - `@SEVERITY:normal` - `@SEVERITY:critical` - `@SEVERITY:blocker` Set the test's [severity](/docs/v3/readability/#severity). By default, Allure Cucumber.rb will look for a Gherkin tag with the `@SEVERITY:` prefix. The prefix [can be changed in the configuration](/docs/cucumberrb-configuration/#gherkin-tag-prefixes). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. ```gherkin Feature: Labels @SEVERITY:critical Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Label - `Allure::label: (String name, String value)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. The first argument of the function is the label name. It can be any string. You can call `Allure.label()` multiple times to create an array of values under the given name. ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.label 'my custom label', 'value' # ... end ``` ### Link - `Allure::add_link: (url: String, name: String?, type: String?)` - `Allure::issue: (String name, String url)` - `Allure::tms: (String name, String url)` - `@ISSUE:⟨name⟩` - `@TMS:⟨name⟩` Add a [link](/docs/v3/readability/#links) related to the test. In Runtime API, you can do it by calling the `Allure.add_link()` function. The `type` affects the icon that is displayed next to the link in the test report. For convenience, Allure provides two shorthand functions with pre-selected link types: `Allure.issue()` and `Allure.tms()`. ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.add_link url: 'https://dev.example.com/', name: 'Website' Allure.issue 'AUTH-123', 'https://issues.example.com/AUTH-123' Allure.tms 'TMS-456', 'https://tms.example.com/TMS-456' # ... end ``` Alternatively, you can add issue links and TMS links via short identifiers in the Gherkin tags. To enable this, you need to [specify link patterns](/docs/cucumberrb-configuration/#link-issue-pattern-link-tms-pattern) in the configuration. Allure Cucumber.rb will use the patterns to process Gherkin tags with specific prefixes (defaults to `@ISSUE:` and `@TMS:` respectively, but [can be changed in the configuration](/docs/cucumberrb-configuration/#gherkin-tag-prefixes)). Assuming the link patterns `https://issues.example.org/{}` and `https://tms.example.org/{}`, the test in the example below will have an issue link to `https://issues.example.org/AUTH-123` and a TMS link to `https://tms.example.org/TMS-456`. ```gherkin Feature: Labels @ISSUE:AUTH-123 @TMS:TMS-456 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ## Behavior-based hierarchy - `Allure::epic: (String epic)` - `Allure::feature: (String feature)` - `Allure::story: (String story)` - `@EPIC:⟨name⟩` - `@FEATURE:⟨name⟩` - `@STORY:⟨name⟩` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). You can do it via the functions in Runtime API or by specifying Gherkin tags with certain prefixes. The prefixes by default are `@EPIC:`, `@FEATURE:` and `@STORY:`, but they can be [changed in the configuration](/docs/cucumberrb-configuration/#gherkin-tag-prefixes). Note that due to a limitation in the Gherkin syntax, the value cannot contain spaces. **Runtime API:** ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.epic 'Web interface' Allure.feature 'Essential features' Allure.story 'Authentication' # ... end ``` **Gherkin tags:** ```gherkin Feature: Labels @EPIC:WebInterface @FEATURE:EssentialFeatures @STORY:Authentication Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ## Suite-based hierarchy - `Allure::suite: (String name)` Assign the names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.label 'parentSuite' 'Web interface' Allure.suite 'Essential features' Allure.label 'subSuite', 'Authentication' # ... end ``` ## Test sub-steps - `AllureStepAnnotation::step: (name: String, status: Status?)` - `Allure::run_step: (String name) { (*any) -> untyped }` - `Allure::step: (name: String, status: Status) { (*any) -> untyped }` Define a [test sub-step](/docs/steps/) with the given `name`. There are three ways of defining a sub-step. - **Annotated sub-steps** In a class that extends `AllureStepAnnotation`, define a method containing a test sub-step and add the `step` annotation to it, with an optional `name` argument (defaults to the method's name). Each time the method is called during the test execution, a new sub-step will be created within the current Gherkin step. If the method calls another function that also has the `step` attribute, Allure Report will create a sub-sub-step, and so on. - **Block-based sub-steps** Write a test sub-step inside a block for the `Allure.run_step()` function. If the block returns a value, `Allure.run_step()` will return it without modification, and it will not affect the report. - **No-op sub-steps** If you call the `Allure.step()` function, Allure will add to the report a no-op sub-step. This allows for a log-style reporting within a larger step. A no-op sub-step finishes immediately after it started and cannot have any sub-steps of its own. The optional second argument indicates the status that will be shown for the sub-step in the report. Allowed values are: `:passed` (the default), `:failed`, `:broken`, `:skipped`, and `:unknown`. **Annotated steps:** ```ruby require 'allure-cucumber' When 'I open labels page' do Steps.step_1 Steps.step_2 end class Steps extend AllureStepAnnotation step 'Step 1' def self.step_1 step_1_1 step_1_2 end step 'Step 1.1' def self.step_1_1 # ... end step 'Step 1.2' def self.step_1_2 # ... end step 'Step 2' def self.step_2 step_2_1 step_2_2 end step 'Step 2.1' def self.step_2_1 # ... end step 'Step 2.2' def self.step_2_2 # ... end end ``` **Block-based steps and no-op steps:** ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.run_step 'Step 1' do # ... Allure.step name: 'Step 1.1', status: :passed # ... Allure.step name: 'Step 1.2', status: :passed end Allure.run_step 'Step 2' do # ... Allure.step name: 'Step 2.1', status: :passed # ... Allure.step name: 'Step 2.2', status: :passed end end ``` ## Parametrized tests An Allure test report can reflect two ways in which you can pass data from your Gherkin file to your Ruby implementation code, namely: - [scenario outlines](#scenario-outlines), - [runtime parameters](#runtime-parameters). ### Scenario outlines In Gherkin, a [`Scenario Outline`](https://cucumber.io/docs/gherkin/reference#scenario-outline) (or a `Scenario Template`) implements the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. A scenario outline must contain an `Examples` table, from which Cucumber.rb loads sets of parameters, one row after another. Each set of parameters is being placed into the step declarations according to the placeholders, thus generating a new scenario based on the row. Cucumber.rb then runs each of them independently, as if it was a separate `Scenario`. The data can then be captured by a regular expression and passed as separate arguments to the Ruby code. Allure Cucumber.rb automatically recognizes this pattern. No additional configuration is required. The example below shows a Gherkin file and a Ruby implementation file of a test. In this example, the four parameters for the “I enter my details...” step will be displayed in both instances of the scenario in the test report. ```gherkin Feature: User management Scenario Outline: Registration When I go to the registration form And I enter my details: , , , Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ```ruby require 'allure-cucumber' When 'I go to the registration form' do # ... end And /^I enter my details: (.*), (.*), (.*), (.*)$/ do |login, password, name, birthday| # ... end Then 'the profile should be created' do # ... end ``` ### Runtime parameters - `Allure::parameter: (String name, String value)` The `Allure.parameter()` function can be used at any point, even if the Gherkin file does not specify actual parameters or data tables to any step. The example below shows a Gherkin file and a Ruby implementation file of a test. As far as Cucumber.rb is concerned, the “I enter my details” in the example does not have any data passed to it, however, Allure will have the four parameters displayed on the test level. ```gherkin Feature: User management Scenario: Registration When I go to the registration form And I enter my details Then the profile should be created ``` ```ruby require 'allure-cucumber' When 'I go to the registration form' do # ... end And 'I enter my details' do Allure.parameter 'login', 'johndoe' Allure.parameter 'password', 'qwerty' Allure.parameter 'name', 'John Doe' Allure.parameter 'birthday', '1970-01-01' # ... end Then 'the profile should be created' do # ... end ``` ## Attachments - `Allure::add_attachment: (name: String, source: File | String, type: String, test_case: bool?)` Add an attachment with a given `name`. The content is passed in the `source` argument and can be a text string or a [File](https://docs.ruby-lang.org/en/master/File.html). To ensure that the reader's web browser will display attachments correctly, pass the appropriate media type of the content as `type`. The `Allure::ContentType` class contains some popular media types, or you can use a custom string, as long as it will be understood by the web browser when viewing the test report. When inside a [test step](#test-sub-steps), the function adds the attachment to the current step by default. To add it to the whole test result instead, pass `true` to the optional `test_case` argument. If a Gherkin step has a [data table](https://cucumber.io/docs/gherkin/reference/#data-tables), Allure Report display it as one of the step's attachments. ```ruby require 'allure-cucumber' When 'I open labels page' do Allure.add_attachment name: 'Screenshot', source: File.new('/path/to/image.png'), type: Allure::ContentType::PNG Allure.add_attachment name: 'Data', source: 'This is the file content.', type: Allure::ContentType::TXT # ... end ``` ## Run-level attachments and errors - `Allure::add_global_attachment: (name: String, source: File | String, type: String)` - `Allure::add_global_error: (message: String?, trace: String?, known: bool?, muted: bool?, flaky: bool?)` Add an attachment or an error that belongs to the whole test run rather than to a single test, for example, a server log or an infrastructure failure. They are shown in the report as [global attachments and global errors](/docs/global-errors-and-attachments/) in Allure Report 3 and on the **Run** tab in Allure Report 2.46.0 and newer. The arguments of `add_global_attachment` work the same way as for [`add_attachment`](#attachments). If a `BeforeAll` or `AfterAll` hook fails, Allure Cucumber.rb 2.30.0 and newer records the failure as a global error automatically. ```ruby require 'allure-cucumber' AfterAll do Allure.add_global_attachment name: 'Server log', source: File.new('/path/to/server.log'), type: Allure::ContentType::TXT end ``` --- ## Cypress > URL: https://allurereport.org/docs/cypress/ # Getting started with Allure Cypress [![Allure Cypress npm latest version](https://img.shields.io/npm/v/allure-cypress?style=flat "Allure Cypress npm latest version")](https://www.npmjs.com/package/allure-cypress) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Cypress](https://cypress.io/) tests. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure Cypress is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure Cypress integration. **npm:** ```bash npm install --save-dev allure-cypress ``` **yarn:** ```bash yarn add --dev allure-cypress ``` **pnpm:** ```bash pnpm add -D allure-cypress ``` Info: Cypress 16 requires Allure Cypress 3.12.2 or later. Earlier versions fail on Cypress 16 and don't produce test results. Starting with 3.12.2, Allure Cypress also works when `Cypress.env()` is disabled with the `allowCypressEnv: false` option. 1. In the `e2e` section of your [Cypress configuration script](https://docs.cypress.io/guides/references/configuration), define a `setupNodeEvents()` function that calls `allureCypress()`, as shown in the example. Pass the configuration options if necessary, see [Allure Cypress configuration](/docs/cypress-configuration/). ```js import { allureCypress } from "allure-cypress/reporter"; export default { e2e: { setupNodeEvents(on, config) { allureCypress(on, config, { resultsDir: "allure-results", }); return config; }, }, }; ``` Tip: If you use custom event handlers or third-party Cypress plugins that may rely on event handling, follow the recommendations in [Using Allure Cypress with custom event handlers](/docs/cypress-configuration/#using-with-custom-handlers) or [Using Allure Cypress with other Cypress plugins](/docs/cypress-configuration/#using-with-other-plugins). 1. In your [E2E support file](https://docs.cypress.io/guides/core-concepts/writing-and-organizing-tests#Support-file), import the Allure Cypress library. ```js import "allure-cypress"; ``` ### 2. Run tests Run your Cypress tests the same way as your would run them usually. For example: **npm:** ```bash npx cypress run ``` **yarn:** ```bash yarn run cypress run ``` **pnpm:** ```bash pnpx cypress run ``` This will save necessary data into `allure-results` or other directory, according to the [configuration](/docs/cypress-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Cypress integration extends the standard reporting features of Cypress by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure Cypress provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Cypress will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/cypress-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```ts import * as allure from "allure-js-commons"; it("Authentication", () => { allure.description("This test attempts to log into the website."); allure.displayName("Test Authentication"); allure.owner("John Doe"); allure.tags("Web interface", "Authentication"); allure.severity("critical"); // ... }); ``` **Metadata API:** ```ts it( "Authentication" + " @allure.label.owner:JohnDoe" + " @allure.label.tag:WebInterface" + " @allure.label.tag:Authentication" + " @allure.label.severity:critical", () => { // ... }, ); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```ts import * as allure from "allure-js-commons"; it("Authentication", () => { allure.epic("Web interface"); allure.feature("Essential features"); allure.story("Authentication"); // ... }); ``` **Metadata API:** ```ts it( "Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", () => { // ... }, ); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```ts import * as allure from "allure-js-commons"; it("Authentication", () => { allure.parentSuite("Tests for web interface"); allure.suite("Tests for essential features"); allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```ts it( "Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", () => { // ... }, ); ``` ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), you can use the `step()` function, see the [reference](/docs/cypress-reference/#test-steps). ```ts import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; it("Authentication", () => { allure.step("Step 1", () => { allure.step("Sub-step 1", (ctx) => { ctx.parameter("foo", "1"); // ... }); allure.step("Sub-step 2", (ctx) => { ctx.parameter("foo", "2"); // ... }); }); allure.logStep("Step 2", Status.SKIPPED); }); ``` ### Describe parametrized tests A typical way to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern in Cypress is to define the test in a loop and use the loop variable in the test's title and body. To display a parameter value in the test report, pass it to the [`parameter()`](/docs/cypress-reference/#parametrized-tests) function. ```ts import * as allure from "allure-js-commons"; for (const login of ["johndoe", "johndoe@example.com"]) { it(`Authentication as ${login}`, () => { allure.parameter("login", login); allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } ``` ### Set labels globally Any [labels](/docs/cypress-reference/#label), including custom ones, can be set via the environment variables in your operating system. Here's an example (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface npx cypress run ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" npx cypress run ``` ### Attach screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. If your test calls [`cy.screenshot()`](https://docs.cypress.io/api/commands/screenshot) or if the [video recording](https://docs.cypress.io/guides/guides/screenshots-and-videos#Videos) is enabled in the Cypress configuration, the resulting images and video files will be attached to the test results automatically. Adding other file is possible by using the `allure.attachment()` and `allure.attachmentPath()` functions, see the [reference](/docs/cypress-reference/#attachments). ```ts import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; it("Authentication", () => { // ... allure.attachment("Text file", "This is the file content.", ContentType.TEXT); allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` ### Select tests via a test plan file Warning: For this feature to work, make sure that the `config` object is passed from Cypress to Allure Cypress, as shown in the [configuration example](/docs/cypress-configuration/). If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Cypress will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json` (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json npx cypress run ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" npx cypress run ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, edit the [`environmentInfo`](/docs/cypress-configuration/#environmentinfo) object in the configuration. For example, it is a good idea to use this to remember the OS version and Node.js version retrieved from the [`os`](https://nodejs.org/api/os.html) and [`process`](https://nodejs.org/api/process.html) objects. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/js/environment-allure3-light.png, /images/js/environment-allure3-dark.png, /images/js/environment-allure2-light.png, /images/js/environment-allure2-dark.png ```js import { allureCypress } from "allure-cypress/reporter"; import * as os from "node:os"; export default { e2e: { setupNodeEvents(on, config) { allureCypress(on, config, { environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }); return config; }, }, }; ``` Note that if your launch includes multiple Cypress runs (see [How it works](/docs/how-it-works/)), Allure Cypress will only save the environment information from the latest run. --- ## Cypress configuration > URL: https://allurereport.org/docs/cypress-configuration/ # Allure Cypress configuration The [Allure Cypress](/docs/cypress/) integration behavior can be affected by some configuration options set in your [Cypress configuration file](https://docs.cypress.io/guides/references/configuration), e.g., `cypress.config.ts`. The configuration options must be defined in the object passed to `allureCypress()` as the third argument. For example: ```js import { allureCypress } from "allure-cypress/reporter"; import { Status } from "allure-js-commons"; import * as os from "node:os"; export default { e2e: { setupNodeEvents(on, config) { allureCypress(on, config, { resultsDir: "allure-results", links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: (v) => `https://jira.example.com/browse/${v}`, }, }, categories: [ { name: "foo", messageRegex: "bar", traceRegex: "baz", matchedStatuses: [Status.FAILED, Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, globalLabels: [{ name: "layer", value: "api" }], }); return config; }, }, }; ``` Warning: Both the `config` object and the Allure Cypress configuration must be passed to `allureCypress()`. Skipping the `config` object may cause some of the integration features to not work. ## Using Allure Cypress with custom event handlers Some of the Allure Cypress functionality is implemented via the event handlers for the [`after:spec`](https://docs.cypress.io/api/plugins/after-spec-api) of [`after:run`](https://docs.cypress.io/api/plugins/after-run-api) events. If your project needs to define its own handlers to these events, it is important to make sure that Cypress executes all handlers, not the last defined ones. To do so, save the result of the `allureCypress()` call into a variable and call its `onAfterSpec()` or `onAfterRun()` at the beginning of your event handlers. For example: ```js import { allureCypress } from "allure-cypress/reporter"; import { defineConfig } from "cypress"; export default defineConfig({ e2e: { setupNodeEvents: (on, config) => { const allurePlugin = allureCypress(on, config); on("after:run", (results) => { allurePlugin.onAfterRun(results); // ... the rest of the code ... }); return config; }, }, }); ``` ## Using Allure Cypress with other Cypress plugins To avoid conflicts between Allure Cypress and other Cypress plugins that rely on handling the [`after:spec`](https://docs.cypress.io/api/plugins/after-spec-api) of [`after:run`](https://docs.cypress.io/api/plugins/after-run-api) events, we recommend using [`cypress-on-fix`](https://github.com/bahmutov/cypress-on-fix). It modifies the `on` object so that you can then pass it to multiple Cypress plugins (including Allure Cypress) without causing event handler conflicts. Here's an example that uses both Allure Cypress and [Cucumber preprocessor for Cypress](https://github.com/badeball/cypress-cucumber-preprocessor): ```js import { defineConfig } from "cypress"; import { allureCypress } from "allure-cypress/reporter"; import createBundler from "@bahmutov/cypress-esbuild-preprocessor"; import { addCucumberPreprocessorPlugin } from "@badeball/cypress-cucumber-preprocessor"; import { createEsbuildPlugin } from "@badeball/cypress-cucumber-preprocessor/esbuild"; import cypressOnFix from "cypress-on-fix"; export default defineConfig({ e2e: { setupNodeEvents = async (on, config) => { on = cypressOnFix(on); await addCucumberPreprocessorPlugin(on, config); on("file:preprocessor", createBundler({ plugins: [createEsbuildPlugin(config)], })); allureCypress(on, config); return config; }, // ... }, }); ``` ## Supported options ### resultsDir Path to the directory where Allure Cypress will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ### links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/cypress-reference/#link)), you can specify the following: - `nameTemplate` — a template or a function for generating the link name when it is not provided. - `urlTemplate` — a template or a function for generating the link address when it is not provided. The templates can be strings (with `%s` where the identifier should be placed) or functions (accepting the identifier and returning the URL). For example, with the configuration above, `allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/`. Allure will display the link with the appropriate icon for the `issue` type. ### categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). ### environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ### globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ name, value }` label objects: ```js allureCypress(on, config, { globalLabels: [ { name: "layer", value: "api" }, { name: "team", value: "backend" }, ], }); ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```js allureCypress(on, config, { globalLabels: { layer: "api", team: "backend", }, }); ``` --- ## Cypress reference > URL: https://allurereport.org/docs/cypress-reference/ # Allure Cypress reference These are the functions that you can use to integrate your Cypress tests with Allure. In most cases, Allure Cypress provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Cypress will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `allure.description(markdown: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ```ts import * as allure from "allure-js-commons"; it("Authentication", () => { allure.description("This test attempts to log into the website."); // ... }); ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```ts import * as allure from "allure-js-commons"; it("Authentication", () => { allure.owner("John Doe"); // ... }); ``` **Metadata API:** ```ts it("Authentication @allure.label.owner:JohnDoe", () => { // ... }); ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` - `@allure.label.tag:⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```ts import * as allure from "allure-js-commons"; it("Authentication", () => { allure.tag("Web interface"); allure.tag("Authentication"); // ... }); ``` **Metadata API:** ```ts it("Authentication @allure.label.tag:WebInterface @allure.label.tag:Authentication", () => { // ... }); ``` ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { Severity } from "allure-js-commons"; it("Authentication", () => { allure.severity(Severity.CRITICAL); // ... }); ``` **Metadata API:** ```ts it("Authentication @allure.label.severity:critical", () => { // ... }); ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` - `@allure.label.⟨NAME⟩:⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. **Runtime API:** ```ts import * as allure from "allure-js-commons"; it("Authentication", () => { allure.label("microservice", "UI"); // ... }); ``` **Metadata API:** ```ts it("Authentication @allure.label.microservice:UI", () => { // ... }); ``` ### ID - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). ```ts it("Authentication @allure.id:123", () => { // ... }); ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string, defaults to “link”), Allure will try to load a corresponding **link template** to process the URL, as defined by the [`links`](/docs/cypress-configuration/#links) configuration option. If no template is found for the given type or if the link already represents a proper URL, it's left unmodified. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```ts import * as allure from "allure-js-commons"; it("Authentication", () => { allure.issue("AUTH-123", "Related issue"); allure.tms("TMS-456", "Related TMS issue"); allure.link("JIRA-777", "Related Jira issue", "jira"); allure.link("https://example.com/", "Project website"); // ... }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features**, or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```ts import * as allure from "allure-js-commons"; it("Authentication", () => { allure.epic("Web interface"); allure.feature("Essential features"); allure.story("Authentication"); // ... }); ``` **Metadata API:** ```ts it( "Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", () => { // ... }, ); ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the names of **parent suite**, **suite**, or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Runtime API:** ```ts import * as allure from "allure-js-commons"; it("Authentication", () => { allure.parentSuite("Tests for web interface"); allure.suite("Tests for essential features"); allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```ts it( "Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", () => { // ... }, ); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test step or sub-step](/docs/steps/) with the given `name`. A step function accepts either no arguments or an object with the following methods: - `displayName()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are similar to the test-wide implementation, see [Parametrized tests](#parametrized-tests). To create a step without a body, call the `logStep()` function that accepts a name and, optionally, a step status and an error object. ```ts import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; it("Authentication", () => { allure.step("Step 1", () => { allure.step("Sub-step 1", (ctx) => { ctx.parameter("foo", "1"); // ... }); allure.step("Sub-step 2", (ctx) => { ctx.parameter("foo", "2"); // ... }); }); allure.logStep("Step 2", Status.SKIPPED); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` Specify a `name` and `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. The `options` argument, if given, must be an object with two optional properties `excluded` and `mode`. - If `excluded` is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. Note, that even when you use the `"masked"` or `"hidden"` mode, it is still possible to extract the value from the `allure_results` directory if you publish it. ```ts import * as allure from "allure-js-commons"; for (const login of ["johndoe", "johndoe@example.com"]) { it("Authentication", () => { allure.parameter("login", login); allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } ``` ## Attachments - `allure.attachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options in an object (as shown for the image attachment below) or just specify the media type and let Allure deduce the appropriate filename extension automatically (as shown for the text attachment below). In either case, the media type can be a value from the `ContentType` enumeration or any string. ```ts import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; it("Authentication", () => { // ... allure.attachment("Text file", "This is the file content.", ContentType.TEXT); allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` Tip: If your test calls [`cy.screenshot()`](https://docs.cypress.io/api/commands/screenshot) or if the [video recording](https://docs.cypress.io/guides/guides/screenshots-and-videos#Videos) is enabled in the Cypress configuration, the resulting images and video files will be attached to the test results automatically. --- ## Dart and Flutter > URL: https://allurereport.org/docs/dart/ # Getting started with Dart and Flutter [![allure_dart_test pub.dev latest version](https://img.shields.io/pub/v/allure_dart_test?style=flat&label=allure_dart_test "allure_dart_test pub.dev latest version")](https://pub.dev/packages/allure_dart_test) [![allure_flutter_test pub.dev latest version](https://img.shields.io/pub/v/allure_flutter_test?style=flat&label=allure_flutter_test "allure_flutter_test pub.dev latest version")](https://pub.dev/packages/allure_flutter_test) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your Dart and Flutter tests. Info: The main user-facing packages are [`allure_dart_test`](https://pub.dev/packages/allure_dart_test) for `package:test` suites and [`allure_flutter_test`](https://pub.dev/packages/allure_flutter_test) for Flutter widget tests and host-run `integration_test` suites. If you are building your own integration for a custom runner or framework, use [`allure_dart_commons`](https://pub.dev/packages/allure_dart_commons) instead. All three packages live in the official [`allure-dart`](https://github.com/allure-framework/allure-dart) repository. ## Setting up ### 1. Prepare your project 1. Make sure a recent Dart or Flutter toolchain is installed. The official `allure-dart` packages require Dart 3.8 or newer. The Flutter adapter additionally requires Flutter 3.24 or newer. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Install Allure Report, following the [installation guide](/docs/v3/install/). 1. Add the integration as a dev dependency: **Dart:** ```bash dart pub add --dev allure_dart_test ``` **Flutter:** ```bash flutter pub add --dev allure_flutter_test ``` 1. Enable the integration in your test files. The simplest way is the drop-in import: replace the test framework import with the matching Allure import, and keep the rest of the file unchanged. | Original import | Allure drop-in import | | ------------------------------------------------ | --------------------------------------------------- | | `package:test/test.dart` | `package:allure_dart_test/test.dart` | | `package:flutter_test/flutter_test.dart` | `package:allure_flutter_test/flutter_test.dart` | | `package:integration_test/integration_test.dart` | `package:allure_flutter_test/integration_test.dart` | The drop-in libraries re-export the original framework, so `expect`, matchers, and the other APIs keep working. Add a second, aliased import for Allure's runtime API: **Dart:** ```dart import 'package:allure_dart_test/test.dart'; import 'package:allure_dart_test/allure_dart_test.dart' as allure; void main() { group('authentication', () { test('login works', () async { await allure.feature('Authentication'); await allure.parameter('browser', 'chromium'); await allure.step('submit credentials', (step) async { await step.parameter('user', 'alice'); await allure.attachment( 'request', '{"user":"alice"}', contentType: 'application/json', fileExtension: 'json', ); }); expect(2 + 2, equals(4)); }); }); } ``` **Flutter:** ```dart import 'package:allure_flutter_test/flutter_test.dart'; import 'package:allure_flutter_test/allure_flutter_test.dart' as allure; void main() { testWidgets('renders empty state', (tester) async { await allure.feature('Home screen'); await allure.step('pump widget', (_) async { await tester.pumpWidget(const MyApp()); }); expect(find.text('No items'), findsOneWidget); }); } ``` **Flutter integration test:** ```dart import 'package:allure_flutter_test/integration_test.dart'; import 'package:allure_flutter_test/allure_flutter_test.dart' as allure; void main() { IntegrationTestWidgetsFlutterBinding.ensureInitialized(); testWidgets('signs in', (tester) async { await allure.story('Sign in'); expect(find.text('Sign in'), findsOneWidget); }); } ``` Alternatively, if you prefer to keep the original framework imports, install the runtime plugin once with `installAllure()`. In Dart tests, call it at the start of `main()`; in Flutter tests, call it from `test/flutter_test_config.dart` so it applies to the whole suite: **Dart:** ```dart import 'package:allure_dart_test/allure_dart_test.dart'; import 'package:test/test.dart'; void main() { installAllure(); test('login works', () async { await step('submit credentials', (_) async { expect(2 + 2, equals(4)); }); }); } ``` **Flutter — test/flutter_test_config.dart:** ```dart import 'dart:async'; import 'package:allure_flutter_test/allure_flutter_test.dart'; Future testExecutable(FutureOr Function() testMain) async { installAllure(); await testMain(); } ``` Both styles produce Allure results and support the same runtime API. The drop-in style additionally reports `setUp`/`tearDown` callbacks as Allure fixtures, converts a declared `skip:` into a runtime self-skip so it's still reported (see [Skip tests](#skip-tests)), and can skip tests excluded by a [test plan](#select-tests-via-a-test-plan-file). ### 2. Run tests Run your tests the same way as usual: **Dart:** ```bash dart test ``` **Flutter:** ```bash flutter test ``` By default, the integration writes the results to an `allure-results` directory in the current working directory. Warning: These adapters only work on the Dart VM — the default platform for `dart test` and `flutter test`. On browsers or Node.js (`dart test -p chrome`, `dart test -p node`, `flutter test --platform chrome`), every test fails with a message `Allure cannot write test results on this platform (no filesystem)`, even ones that never call the Allure API, because the adapters always try to write a result file with no filesystem to write it to. To use a different directory, set `ALLURE_RESULTS_DIR` before the test run: ```bash ALLURE_RESULTS_DIR=build/allure-results dart test ``` If the results directory already exists, the new files are added to the existing ones, so that a future report will be based on all of them. ### 3. Generate a report After the test run, generate and open the report with the Allure CLI: ```bash allure generate ./allure-results --output ./allure-report allure open ./allure-report ``` If you changed the results directory, use that path in the `allure generate` command. ## Writing tests The Allure Dart adapters extend plain `dart test` and `flutter test` output with richer reporting features. You can use them to: - add [descriptions, owners, links, and other metadata](#add-metadata), - organize tests into [behavior-based and suite-based hierarchies](#organize-tests), - split the execution into [nested steps](#divide-a-test-into-steps), - describe [parameters and attachments](#add-parameters-and-attachments), - report [setup and teardown fixtures](#report-fixtures), - [skip tests](#skip-tests) without losing them from the report, - run only selected tests through a [test plan file](#select-tests-via-a-test-plan-file). ### Add metadata Inside a test body, use the top-level runtime functions to enrich the test result: ```dart import 'package:allure_dart_test/test.dart'; import 'package:allure_dart_test/allure_dart_test.dart' as allure; void main() { test('login works', () async { await allure.displayName('Login works'); await allure.description('This test verifies login with a username and a password.'); await allure.owner('John Doe'); await allure.tag('smoke'); await allure.severity('critical'); await allure.allureId('AUTH-1'); await allure.issue('https://jira.example.com/browse/AUTH-123', name: 'AUTH-123'); await allure.tms('https://tms.example.com/cases/TMS-456', name: 'TMS-456'); }); } ``` You can also declare metadata statically, without touching the test body. Inline markers in the test name are parsed and removed from the displayed name: ```dart test('checkout works @allure.id:123 @allure.label.owner:payments', () async { // ... }); ``` Supported markers are `@allure.id:`, `@allure.label.:`, `@allure.link.:`, and `@allure.name:` (`=` works in place of `:`). Ordinary `package:test` tags are reported as Allure `tag` labels. Reserved `@allure.*` tags are treated as inline metadata instead and do not become `tag` labels: ```dart test( 'checkout works', () async { // ... }, tags: ['smoke'], ); ``` ### Organize tests Allure supports both behavior-based and suite-based hierarchies. For example: ```dart test('login works', () async { await allure.epic('Web interface'); await allure.feature('Authentication'); await allure.story('Login with username and password'); await allure.parentSuite('UI tests'); await allure.suite('Authentication'); await allure.subSuite('Positive scenarios'); }); ``` The adapters also derive suite labels from the `group()` hierarchy automatically. Explicit calls to `allure.parentSuite(...)`, `allure.suite(...)`, or `allure.subSuite(...)` override the automatically derived labels. See [Configuration](/docs/dart-configuration/#automatic-labels-and-identifiers) for the full list of automatic labels. ### Divide a test into steps Wrap parts of the test into named steps with `allure.step()`. Steps nest, record their own timing and status, and can carry parameters: ```dart test('login works', () async { await allure.step('open login page', (_) async { // ... }); await allure.step('submit credentials', (step) async { await step.parameter('user', 'alice'); await allure.step('fill the form', (_) async { // ... }); }); }); ``` If a step body throws, the step is reported as failed or broken, and the error is rethrown, so the test itself fails as usual. To record an already-completed action without wrapping code, use `allure.logStep()`: ```dart await allure.logStep('verify audit log'); ``` ### Add parameters and attachments Parameters and attachments are stored in the generated Allure results and rendered in the report: ```dart test('login works', () async { await allure.parameter('browser', 'firefox'); await allure.parameter('password', 'qwerty', mode: allure.AllureParameterMode.masked); await allure.attachment( 'request.json', '{"username":"demo","rememberMe":true}', contentType: 'application/json', fileExtension: 'json', ); await allure.attachmentPath( 'screenshot', 'screenshots/failure.png', contentType: 'image/png', ); }); ``` Attachment content can be a `String`, a list of bytes, or any JSON-encodable object. By default, attachments added from inside a test are wrapped in an attachment step, so they keep their logical position between the surrounding steps; pass `wrapInStep: false` to attach them directly to the test or current step instead. Info: `testWidgets` bodies run on a fake-async zone, so an attachment call that does real I/O (`attachmentPath`, or `attachment` with stream content) must run inside `tester.runAsync(...)`. Outside `runAsync`, the fake zone never pumps the real I/O callback and the test hangs. ### Capture Flutter failures automatically `allure_flutter_test`'s `installAllure()` accepts two Flutter-only opt-in flags that install a process-wide hook the first time either is passed as `true`: ```dart installAllure( autoScreenshotOnFailure: true, autoAttachGoldenDiff: true, ); ``` - `autoScreenshotOnFailure` captures a screenshot of the widget tree when a test fails and attaches it as `screenshot-on-failure`. - `autoAttachGoldenDiff` attaches the actual rendered image as `golden-actual` (plus, best-effort, the master and test images) whenever a `matchesGoldenFile` comparison fails. Both hooks are best-effort — a capture failure never masks the original test failure — and monotonic: once enabled by any `installAllure(...)` call in the process, a later bare `installAllure()` call does not disable them. Enable them once, for example in `test/flutter_test_config.dart`. See [Reference](/docs/dart-reference/#auto-screenshot-and-golden-diff-attachments) for details. ### Report fixtures When you use the drop-in imports, `setUp`, `tearDown`, `setUpAll`, and `tearDownAll` callbacks are reported as Allure fixtures, shown in the report alongside the tests they prepare: ```dart import 'package:allure_dart_test/test.dart'; import 'package:allure_dart_test/allure_dart_test.dart' as allure; void main() { group('authentication', () { setUp(() async { await allure.step('reset database', (_) async { // ... }); }); test('login works', () async { // ... }); }); } ``` Steps and attachments created inside a fixture are attached to the fixture result. Labels and links declared inside a `setUp` or `setUpAll` fixture apply to the tests in its scope. Starting with version 1.3.0, if a fixture fails, Allure Dart also records the failure as a [global error](/docs/global-errors-and-attachments/#custom-globals), with a message such as `setUpAll failed: `. This way, the failure is visible in the report even if no test in its scope gets a result, for example, when `setUpAll` fails before any test starts. ### Skip tests On the drop-in wrappers, a declaration-time `skip: true` (or a skip message) is converted into a runtime self-skip, so the test is still reported as **skipped**, not left out of the report: ```dart test('checkout applies a discount', () { // ... }, skip: true); ``` With `installAllure()` and the original framework imports, the same declaration produces **no Allure result at all**, because the framework never runs `setUp` for a declaration-skipped test, and that's where the adapter schedules the result. Call `markTestSkipped(...)` from inside the test body instead if you need a skipped result without switching to the drop-in import: ```dart test('checkout applies a discount', () { markTestSkipped('temporarily disabled'); }); ``` Warning: A drop-in `group(..., skip: true)` is not forwarded to the underlying framework's own group skip — each nested test still runs far enough to self-skip individually. This means `setUp`, `setUpAll`, and `tearDown` fixtures inside a skipped group **may still run**, unlike a stock declaration-time group skip, which never runs the group's fixtures at all. Skip individual tests instead of the group if fixtures must not run. ### Select tests via a test plan file The adapters support the standard Allure test plan mechanism via the `ALLURE_TESTPLAN_PATH` environment variable. Create a JSON file such as: ```json { "version": "1.0", "tests": [{ "id": "AUTH-1" }, { "selector": "test/auth_test.dart#authentication#login works" }] } ``` Then run the tests with: ```bash ALLURE_TESTPLAN_PATH=./testplan.json dart test ``` Entries with `id` match tests that declare an Allure ID, for example via the `@allure.id:AUTH-1` marker. Entries with `selector` match the test's full name: the test file path, the group names, and the test name, joined with `#`. When the tests are declared through the drop-in `test`, `group`, or `testWidgets` wrappers, tests not included in the plan are declaration-skipped: the body does not run, and no Allure result is written. In suites that use `installAllure()` with the original framework imports, excluded tests still run, but their results are left out of `allure-results`. ## Building a custom integration If you need to integrate Allure with a custom Dart test runner or framework, use `allure_dart_commons`: ```bash dart pub add allure_dart_commons ``` At that level, you create a lifecycle, start a test case, and stop and write it when execution ends: ```dart import 'package:allure_dart_commons/allure_dart_commons.dart'; Future main() async { final lifecycle = AllureLifecycle( writer: AllureResultsWriter(outputDirectory: 'allure-results'), ); final testUuid = lifecycle.startTest( name: 'login works', fullName: 'auth/login_test.dart#login works', ); // ... update metadata, add steps, and add attachments ... await lifecycle.stopTest(testUuid, status: AllureStatus.passed); await lifecycle.writeTest(testUuid); } ``` See [Reference](/docs/dart-reference/) for the runtime and lifecycle APIs, or [Configuration](/docs/dart-configuration/) for the supported environment variables and the `allure-dart.yaml` config file. --- ## Dart and Flutter configuration > URL: https://allurereport.org/docs/dart-configuration/ # Dart and Flutter configuration The [Allure Dart and Flutter integration](/docs/dart/) is configured through an optional `allure-dart.yaml` config file and a set of environment variables. Where both define the same setting, such as the results directory, the environment variable wins. ## allure-dart.yaml Sets checked-in project defaults: the results directory, labels applied to every test, and environment information for the report. The integration searches for `allure-dart.yaml` (or `allure-dart.yml`) from the current working directory upward, stopping after it checks the directory that contains `pubspec.yaml`. This way, each package in a repository can have its own config file. ```yaml resultsDir: build/allure-results labels: module: checkout layer: - api environment: target: local ``` The supported keys: - `resultsDir` (alias: `resultsDirectory`) — the directory where the result files are written. - `labels` — labels added to every test result in the run. Either a map of label names to values (a list value adds one label per item), or a list of `{name: ..., value: ...}` entries. - `environment` — a map of key-value pairs written to `environment.properties` in the results directory and shown on the report's Environment widget. ## ALLURE_CONFIG Points to an explicit config file, instead of searching for `allure-dart.yaml` in the directory tree. The path can be absolute or relative to the current working directory. ```bash ALLURE_CONFIG=./configs/allure-ci.yaml dart test ``` If the file does not exist, the integration throws an error. ## ALLURE_RESULTS_DIR Overrides the directory where the integration writes Allure results. This takes precedence over `resultsDir` from the config file. When neither is set, the integration uses `allure-results` in the current working directory. ```bash ALLURE_RESULTS_DIR=build/allure-results dart test ``` ## ALLURE_LABEL\_\* Adds global labels to every test result. Any environment variable whose name starts with `ALLURE_LABEL_` becomes an Allure label. For example: ```bash ALLURE_LABEL_epic="Web interface" \ ALLURE_LABEL_owner="QA Team" \ dart test ``` This applies the `epic` and `owner` labels to every test in the run. The name after the `ALLURE_LABEL_` prefix is used verbatim (case-sensitive) — prefer Allure’s usual names (`epic`, `owner`, `parentSuite`, …). `ALLURE_LABEL_host` and `ALLURE_LABEL_thread` are special-cased: instead of adding a second, conflicting label, they override the automatic `host` and `thread` values described below. ## ALLURE_TESTPLAN_PATH Points to a JSON file that defines which tests should run. The file uses the standard Allure test plan shape: ```json { "version": "1.0", "tests": [{ "id": "AUTH-1" }, { "selector": "test/auth_test.dart#authentication#login works" }] } ``` Run the tests with: ```bash ALLURE_TESTPLAN_PATH=./testplan.json dart test ``` Selection works as follows: - entries with `id` match tests that declare an Allure ID, for example via the `@allure.id:AUTH-1` inline marker, - entries with `selector` match the test's full name — the test file path relative to the package root, the group names, and the test name, joined with `#`. How excluded tests are handled depends on the integration style: - with the drop-in `test`, `group`, and `testWidgets` wrappers, excluded tests are declaration-skipped: the body does not run, and no Allure result is written (unlike a drop-in `skip: true`, which is reported as skipped), - with `installAllure()` and the original framework imports, excluded tests still run, but their results are not written to the results directory. If `ALLURE_TESTPLAN_PATH` is unset, the file does not exist, or the JSON is malformed, the integration prints a warning, skips filtering, and runs the tests normally. An empty `"tests": []` list is valid and selects no tests. ## Automatic labels and identifiers The integration adds a few labels to every test automatically: - `language` = `dart` - `framework` = `dart-test`, `flutter-test`, or `flutter-integration-test`, depending on the adapter and the active Flutter binding - `host` — the local host name, or the [`ALLURE_LABEL_host`](#allure-label) override when set - `thread` — the current process ID, or the [`ALLURE_LABEL_thread`](#allure-label) override when set - `package` — the test file path relative to the package root - `testClass` — the innermost group name, or the test file name when the test is not in a group - `testMethod` — the test name It also derives suite labels from the `group()` hierarchy: - a single group becomes `suite`, - two nested groups become `parentSuite` and `suite`, - three or more nested groups become `parentSuite`, `suite`, and `subSuite` (the remaining group names joined with `` ` > ` ``). Explicit calls to `allure.parentSuite(...)`, `allure.suite(...)`, or `allure.subSuite(...)` override the automatically derived labels. Finally, the integration derives stable identifiers used for history and retries: - `testCaseId` — a hash of the test's full name, - `historyId` — a hash of the test's full name and its non-excluded parameter values, so that the same test with different parameters is tracked as separate history entries. Both can be overridden from the test body via `allure.testCaseId(...)` and `allure.historyId(...)`. When `package:test` retries a test (`retry: N`), each attempt is written as its own result, and all attempts share the same `testCaseId`/`historyId` — the report groups them as retries of one test. Every attempt after the first carries an excluded `retry` parameter with the 1-based attempt number, so it doesn't affect the shared history ID. --- ## Dart and Flutter reference > URL: https://allurereport.org/docs/dart-reference/ # Dart and Flutter reference These are the main building blocks you can use to integrate Dart and Flutter tests with Allure by using `allure_dart_test` and `allure_flutter_test`. ## Drop-in test wrappers The drop-in libraries replace the declaration functions of the underlying framework and re-export everything else unchanged: - `package:allure_dart_test/test.dart` replaces `test`, `group`, `setUp`, `tearDown`, `setUpAll`, and `tearDownAll` from `package:test`, - `package:allure_flutter_test/flutter_test.dart` additionally replaces `testWidgets` from `flutter_test`, - `package:allure_flutter_test/integration_test.dart` provides the same wrappers plus the `integration_test` exports, including `IntegrationTestWidgetsFlutterBinding`. The wrappers accept the same arguments as the originals. On top of the original behavior, they: - report each test as an Allure result with [automatic labels](/docs/dart-configuration/#automatic-labels-and-identifiers) derived from the file path and group hierarchy, - parse [inline metadata markers](/docs/dart/#add-metadata) from test names, - report `setUp`, `tearDown`, `setUpAll`, and `tearDownAll` callbacks as Allure fixtures, - declaration-skip tests excluded by an [Allure test plan](/docs/dart-configuration/#allure-testplan-path) (no Allure result is written). ### `testWidgets` and variants When a widget test uses a `TestVariant` (for example, `ValueVariant`), the Allure wrapper reports each variant value as a separate test result: ```dart testWidgets( 'renders both layouts', (tester) async { // ... }, variant: ValueVariant({'compact', 'expanded'}), ); ``` Each result is named `renders both layouts (variant: compact)` and `renders both layouts (variant: expanded)`, carries a `variant` parameter with the value, and shares the `renders both layouts` test case name. Test plan selectors match the variant-specific full names. ### Auto screenshot and golden-diff attachments `allure_flutter_test`'s `installAllure()` — not the `allure_dart_test` one — accepts two Flutter-only flags: - `installAllure(autoScreenshotOnFailure: true)` — on a failing test, captures a screenshot of the widget tree and attaches it as `screenshot-on-failure` (PNG). - `installAllure(autoAttachGoldenDiff: true)` — on a `matchesGoldenFile` mismatch, attaches the actual rendered image as `golden-actual`, plus, best-effort (when the comparator is the default `LocalFileComparator`), whichever of `golden-masterImage`, `golden-testImage`, `golden-maskedDiff`, and `golden-isolatedDiff` it already wrote to disk. The comparator only computes a pixel diff when the actual and golden images are the same size, so a dimension mismatch attaches just the master and test images, while a same-size pixel mismatch also attaches the two diff visualizations. Both flags install a process-wide hook the first time either is passed as `true`, are best-effort (a capture failure never masks the original test failure), and are monotonic — once enabled, a later bare `installAllure()` call in the same process does not disable them: ```dart import 'dart:async'; import 'package:allure_flutter_test/allure_flutter_test.dart'; Future testExecutable(FutureOr Function() testMain) async { installAllure(autoScreenshotOnFailure: true, autoAttachGoldenDiff: true); await testMain(); } ``` You can still attach screenshots or golden files manually with the regular attachment APIs, whether or not these hooks are enabled. ## Runtime API The runtime functions are exported by `package:allure_dart_test/allure_dart_test.dart` and `package:allure_flutter_test/allure_flutter_test.dart`. They apply to the currently running test — or, when called inside a fixture, to the fixture and its scope. All of them are asynchronous and should be awaited. Import the library with a prefix to keep the short names readable: ```dart import 'package:allure_dart_test/allure_dart_test.dart' as allure; ``` ### Metadata and labels - `allure.displayName(name)` — overrides the test name shown in the report - `allure.description(markdown)` - `allure.descriptionHtml(html)` - `allure.label(name, value)` - `allure.labels([AllureLabel(...), ...])` - `allure.allureId(value)` - `allure.owner(value)` - `allure.severity(value)` - `allure.layer(value)` - `allure.tag(value)` - `allure.tags(['smoke', 'auth'])` - `allure.testCaseName(value)` - `allure.testCaseId(value)` - `allure.historyId(value)` Example: ```dart test('login works', () async { await allure.description('Checks that a valid user can sign in.'); await allure.owner('John Doe'); await allure.severity('critical'); await allure.label('microservice', 'ui'); await allure.tags(['smoke', 'auth']); }); ``` ### Hierarchies - `allure.epic(value)` - `allure.feature(value)` - `allure.story(value)` - `allure.parentSuite(value)` - `allure.suite(value)` - `allure.subSuite(value)` ### Links and parameters - `allure.link(url, name: ..., type: ...)` - `allure.links([AllureLink(...), ...])` - `allure.issue(url, name: ...)` - `allure.tms(url, name: ...)` - `allure.parameter(name, value, excluded: ..., mode: ...)` A parameter value can be any object; non-string values are JSON-encoded. Parameters with `excluded: true` don't affect the test's history ID. The `mode` argument controls rendering: `AllureParameterMode.masked` hides the value behind asterisks, and `AllureParameterMode.hidden` omits the parameter from the report. ```dart test('login works', () async { await allure.issue('https://jira.example.com/browse/AUTH-123', name: 'AUTH-123'); await allure.parameter('browser', 'firefox'); await allure.parameter('password', 'qwerty', mode: allure.AllureParameterMode.masked); }); ``` ### Statuses - `allure.statusDetails(message: ..., trace: ..., known: ..., muted: ..., flaky: ..., actual: ..., expected: ...)` - `allure.markKnown()` - `allure.markMuted()` - `allure.markFlaky()` A test's status is resolved automatically: assertion failures (such as failed `expect` matchers) are reported as **failed**, any other errors as **broken**, and skipped tests (for example, via `markTestSkipped`, or a declaration-time `skip:` on the drop-in wrappers, which is converted into a runtime self-skip) as **skipped**. The one exception is a declaration-time `skip:` with `installAllure()` and the original framework imports: the framework never runs `setUp` for a declaration-skipped test, which is where that style schedules the result, so no result is produced at all — see [Skip tests](/docs/dart/#skip-tests) for the full breakdown, including the group-skip fixture tradeoff. For matcher failures, the actual and expected values are extracted into the status details automatically. The `statusDetails` call lets you add classification flags on top: ```dart test('login works', () async { await allure.statusDetails(flaky: true); }); ``` ### Steps - `allure.step(name, (step) async { ... })` — runs the body inside a step and returns its value - `allure.logStep(name, status: ..., error: ...)` — records an instant, already-completed step The `step` context object passed to the body provides: - `step.displayName(name)` — renames the current step - `step.parameter(name, value, mode: ...)` — adds a parameter to the current step ```dart test('login works', () async { final session = await allure.step('create session', (step) async { await step.parameter('user', 'alice'); return Session('alice'); }); await allure.logStep('cleanup skipped', status: allure.AllureStatus.skipped); }); ``` Steps nest: a `step` call inside another step's body creates a child step. If the body throws, the step is marked failed or broken and the error is rethrown. ### Attachments - `allure.attachment(name, content, contentType: ..., fileExtension: ..., wrapInStep: ..., timestamp: ...)` - `allure.attachmentPath(name, path, contentType: ..., fileExtension: ..., wrapInStep: ..., timestamp: ...)` - `allure.attachTrace(name, path)` — attaches an existing [Playwright trace](/docs/attachments/#playwright-traces) archive so the report opens it in the Trace Viewer (does not generate traces or depend on Playwright) Attachment content can be a `String`, a list of bytes, or any JSON-encodable object. With `wrapInStep: true` (the default), the attachment is added as an attachment step, so it keeps its position between the surrounding steps. Warning: Inside a Flutter `testWidgets` body, real I/O attachment calls (`attachmentPath`, or `attachment` with stream content) must run inside `tester.runAsync(...)`, or the test hangs — `testWidgets` runs on a fake-async zone that never pumps real I/O callbacks otherwise. ```dart test('login works', () async { await allure.attachment( 'response.json', '{"status":"ok","user":"demo"}', contentType: 'application/json', fileExtension: 'json', ); await allure.attachmentPath( 'screenshot', 'screenshots/failure.png', contentType: 'image/png', ); }); ``` ### Run-level attachments and errors These write run-level data not tied to any single test — for example, a server log for the whole run or an infrastructure failure: - `allure.globalAttachment(name, content, contentType: ..., fileExtension: ...)` - `allure.globalAttachmentPath(name, path, contentType: ..., fileExtension: ...)` - `allure.globalError(message: ..., trace: ..., known: ..., muted: ..., flaky: ..., actual: ..., expected: ...)` ## `allureTest` `allureTest` from `package:allure_dart_test/allure_dart_test.dart` is an alternative to the drop-in wrappers for individual tests. It declares a `package:test` test whose body receives a context object with test-scoped helpers, including durable attachment APIs for large or late-produced artifacts: ```dart import 'package:allure_dart_test/allure_dart_test.dart'; void main() { allureTest('captures logs', (allure) async { await allure.step('start server', () async { // ... }); await allure.streamAttachment( name: 'server log', type: 'text/plain', extension: 'log', content: logFile.openRead(), ); }); } ``` The context object provides: - `step(name, body)` — like the top-level step API, without the step context argument - `label(name, value)`, `parameter(name, value, ...)`, `testCaseName(value)`, `statusDetails(...)` - `attachment(name: ..., type: ..., content: ..., extension: ..., wrapInStep: ...)` — binary content - `textAttachment(name: ..., content: ..., type: ..., extension: ..., wrapInStep: ...)` — text content, `text/plain` by default - `streamAttachment(name: ..., content: ..., type: ..., extension: ..., wrapInStep: ...)` — writes from a byte stream - `preparedAttachment(name: ..., type: ..., extension: ..., write: ..., wrapInStep: ...)` — reserves an attachment file and lets your callback write it before the result references it `allureTest` also accepts `timeout`, `skip`, and initial `labels`, `parameters`, and `links` lists. ## Customizing the lifecycle `installAllure()` accepts a custom `AllureLifecycle`, which is the place to set run-level report data: ```dart import 'package:allure_dart_test/allure_dart_test.dart'; import 'package:test/test.dart'; void main() { installAllure( lifecycle: AllureLifecycle( linkUrlTemplates: { 'issue': 'https://jira.example.com/browse/{}', 'tms': 'https://tms.example.com/cases/{}', }, executorInfo: const AllureExecutorInfo( name: 'GitHub Actions', type: 'github', buildName: 'Build #42', ), environmentInfo: const {'stand': 'staging'}, categories: [ AllureCategory(name: 'Infrastructure', messageRegex: RegExp('timeout')), ], ), ); // ... } ``` The most useful options: - `linkUrlTemplates` / `linkNameTemplates` — templates keyed by link type, applied to links whose URL is not absolute. `{}` (or `%s`) is replaced by the link value, so `allure.issue('AUTH-123')` resolves to the full tracker URL. - `executorInfo` — CI or launcher metadata written to `executor.json` (name, type, build name, build URL, report URL, build order). - `environmentInfo` — key-value pairs written to `environment.properties`, merged with the [config file](/docs/dart-configuration/#allure-dart-yaml) values. - `categories` — custom defect category definitions written to `categories.json`, matching results by message or trace regular expressions. - `globalLabels` — labels added to every test result, merged with the config file values. - `listeners` — a list of `AllureLifecycleListener` implementations. A listener observes lifecycle events (test start/stop/write, step stop, container write, attachments, global errors) and can mutate a result before it is written — for example, to add labels or scrub data. A throwing listener is reported to stderr without failing the run. Warning: `installAllure()` is safe to call more than once in the same process as long as `lifecycle` (and `frameworkLabelResolver`) are omitted or identical each time — a bare `installAllure()` never disturbs an already-installed configuration. Calling it again with a genuinely different `lifecycle` instance throws a `StateError` instead of silently keeping the first one. ## Building a custom integration with `allure_dart_commons` Use `allure_dart_commons` when you need low-level control over the lifecycle: ```dart import 'package:allure_dart_commons/allure_dart_commons.dart'; Future main() async { final lifecycle = AllureLifecycle( writer: AllureResultsWriter(outputDirectory: 'allure-results'), ); final testUuid = lifecycle.startTest( name: 'login works', fullName: 'auth/login_test.dart#login works', ); lifecycle.startStep(testUuid, 'submit credentials'); lifecycle.stopStep(testUuid, status: AllureStatus.passed); await lifecycle.stopTest(testUuid, status: AllureStatus.passed); await lifecycle.writeTest(testUuid); } ``` The main low-level types are: - `AllureLifecycle` — starts, updates, stops, and writes tests, fixtures, and steps; handles attachments and run-level data - `AllureResultsWriter` — writes result files, containers, and attachments to the results directory - `AllureConfig` — loads `allure-dart.yaml` - `AllureLifecycleListener` — lifecycle event hooks - `AllureStatus`, `AllureStatusDetails`, `AllureLabel`, `AllureLink`, `AllureParameter`, and the other model types For streaming and late artifacts, the lifecycle exposes `addAttachmentStreamToRoot` and `addPreparedAttachmentToRoot`, which write the payload durably before the result references it. To route the top-level runtime API to your own lifecycle, wire a `MessageTestRuntime` with `setGlobalTestRuntime` and carry the execution context across async boundaries with `runWithAllureContext`. --- ## Diesel > URL: https://allurereport.org/docs/diesel/ # Getting started with Allure Diesel [![allure-diesel crates.io latest version](https://img.shields.io/crates/v/allure-diesel?style=flat "allure-diesel crates.io latest version")](https://crates.io/crates/allure-diesel) Record executed [Diesel](https://diesel.rs) queries as steps in your [Allure Report](https://allurereport.org/docs/) test reports. It hooks into Diesel's `Instrumentation` API, so it's wired up once in test or setup code and never touches production query call sites. `allure-diesel` records against whichever Allure context is bound to the current thread when a query runs — the same thread-bound context [Rust Cargo Test](/docs/rust/) sets up automatically inside `#[allure_test]`. Outside an active Allure context (for example, in application code, or in tests that don't use `#[allure_test]`), every instrumentation event is a silent no-op — nothing is recorded and nothing fails. Info: The official `allure-rust` repository also publishes [`allure-reqwest`](/docs/reqwest/) for capturing `reqwest` HTTP calls as Allure attachments. ## Setting up 1. Make sure you have an Allure integration set up for your test runner. See [Rust Cargo Test](/docs/rust/). 1. Add the dependency: ```bash cargo add allure-diesel --dev ``` 1. Attach the instrumentation. Two ways to do it: **Per connection** — attach it right after establishing each connection you want captured: ```rust use diesel::prelude::*; use diesel::sqlite::SqliteConnection; fn connect() -> ConnectionResult { let mut conn = SqliteConnection::establish(":memory:")?; conn.set_instrumentation(allure_diesel::AllureInstrumentation::new()); Ok(conn) } ``` **Globally** — register it once during test setup, and every connection established afterward in the process is captured with no call-site changes: ```rust fn setup() -> diesel::QueryResult<()> { allure_diesel::install_default() } ``` 1. Run your tests and generate a report the same way as you would normally — see [Rust Cargo Test](/docs/rust/#_3-generate-a-report). ## What gets captured - Each executed statement becomes a step named after its rendered SQL (see [Reference](/docs/diesel-reference/#step-naming) for how long statements are shortened), and by default carries the full SQL as a nested `query.sql` attachment step. - `BEGIN`/`COMMIT`/`ROLLBACK` boundaries become a `transaction` step that nests the queries run inside that transaction, including the boundary statements themselves. See [Reference](/docs/diesel-reference/#transactions) for exactly how nesting and failure are represented. - A query that returns a database error marks its step (and, for a failing transaction boundary, the enclosing `transaction` step) failed, with the Diesel error message. ## What's not captured Diesel's instrumentation exposes the SQL of each statement but not the rows a query returns — no non-invasive Diesel hook does. This crate therefore records queries and transactions, not result sets. Warning: There is no redaction. The recorded SQL — in both the step name and the `query.sql` attachment — can include bind values exactly as Diesel renders them (for example, via a `-- binds: [...]` comment on parameterized queries), so a captured statement can contain the same sensitive data your query was parameterized to avoid logging in plain text. Keep this in mind before publishing reports built from tests that touch real data. --- ## Diesel configuration > URL: https://allurereport.org/docs/diesel-configuration/ # Allure Diesel configuration This page describes `CaptureOptions`, which controls what [`AllureInstrumentation`](/docs/diesel-reference/) records. Build it with the builder methods below and pass it to `AllureInstrumentation::with_options(...)`: ```rust use allure_diesel::{AllureInstrumentation, CaptureOptions}; let options = CaptureOptions::default() .without_transactions() .with_connection_events() .without_sql_attachment() .with_max_sql_preview(256); let instrumentation = AllureInstrumentation::with_options(options); ``` ## Disable transaction steps - `without_transactions()` By default, `BEGIN`/`COMMIT`/`ROLLBACK` boundaries are wrapped in a `transaction` step that nests the statements run inside that transaction — see [Reference](/docs/diesel-reference/#transactions). Call `without_transactions()` to turn this off; every statement (including the boundary `BEGIN`/`COMMIT`/`ROLLBACK` SQL itself) is then recorded as a flat, top-level step instead of being grouped. ## Capture connection events - `with_connection_events()` Off by default. When enabled, establishing a new connection is recorded as its own `connect` step (no SQL, no attachment), bracketing Diesel's `StartEstablishConnection`/ `FinishEstablishConnection` instrumentation events. A connection failure marks the step failed with the Diesel error. ## Control the SQL attachment - `without_sql_attachment()` By default, each query step gets a nested `query.sql` attachment step carrying the full, untruncated SQL. Call `without_sql_attachment()` to skip that nested step — the step name (see below) still shows the SQL, so this only removes the redundant full-text copy, not all visibility into the statement. - `with_max_sql_preview(max_sql_preview: usize)` (default: `1024`) Sets how many characters of the rendered SQL are used for the step **name**. Longer statements are cut to this length with a trailing `…`. This only affects the step name — when the `query.sql` attachment is enabled, it always carries the complete, untruncated SQL regardless of this setting. ```rust use allure_diesel::CaptureOptions; let options = CaptureOptions::default().with_max_sql_preview(80); ``` Warning: None of these options add redaction. Reducing `max_sql_preview` shortens the step _name_, but the `query.sql` attachment (when enabled) always contains the full statement, bind values included. See the warning in [Getting started](/docs/diesel/#what-s-not-captured). --- ## Diesel reference > URL: https://allurereport.org/docs/diesel-reference/ # Allure Diesel reference ## `AllureInstrumentation` - `AllureInstrumentation::new()` — instrumentation with the default `CaptureOptions` - `AllureInstrumentation::with_options(options: CaptureOptions)` — see [Configuration](/docs/diesel-configuration/) Implements Diesel's `Instrumentation` trait. Attach it to a connection with `conn.set_instrumentation(instrumentation)`. ## `install_default` - `allure_diesel::install_default() -> diesel::QueryResult<()>` Registers `AllureInstrumentation::new()` (with default `CaptureOptions`) as the default instrumentation for every Diesel connection established afterward in the current process, via Diesel's `set_default_instrumentation`. Call it once, before any connections you want captured are established. There's no built-in way to install default instrumentation with non-default `CaptureOptions` — use `set_instrumentation` per connection instead if you need that. ## Step naming A step's name is the query's rendered SQL — the exact text passed to `sql_query(...)`, or Diesel's own `Debug`-rendered output for typed query-builder calls (uppercase keywords, e.g. `INSERT INTO ...`), which for parameterized queries typically includes a trailing, Diesel-internal `-- binds: [...]` comment showing the bound values. When the rendered SQL is longer than `max_sql_preview` characters (1024 by default — see [Configuration](/docs/diesel-configuration/#control-the-sql-attachment)), the step name is cut to that many characters with a trailing `…`. This truncation only affects the step **name**; the `query.sql` attachment (when enabled) always holds the complete, untruncated text. ## Transactions With transaction capture enabled (the default), a `BEGIN`/`SAVEPOINT` boundary opens a `transaction (depth N)` step (`N` is Diesel's transaction nesting depth), and every statement run while it's open — including the closing `COMMIT`/`ROLLBACK` boundary itself — nests inside it as a sibling step, in execution order. The `transaction` step's own status and any failure message come from the **boundary statements**, not from your transaction closure's return value: - If the opening `BEGIN`/`SAVEPOINT` fails, the `transaction` step closes immediately as `failed` with that error, and no closing boundary follows (there's nothing to commit or roll back). Later queries are recorded at the top level again, not nested inside the failed transaction step. - If `BEGIN` succeeds and the transaction later **commits successfully**, the `transaction` step is `passed`. - If `COMMIT` itself fails (for example, a deferred foreign-key constraint that's only checked at commit time), the `transaction` step is `failed`, carrying the `COMMIT` error — even though every statement inside the transaction succeeded individually. - If your closure returns `Err` and Diesel issues a `ROLLBACK`, the `transaction` step is `passed` as long as the `ROLLBACK` statement itself succeeds — a rollback is a normal, successful database operation from the instrumentation's point of view. The `Err` your closure returned is never visible to `AllureInstrumentation`; only whether the boundary SQL succeeded is. If you want a report to visibly flag "this transaction rolled back," record that yourself (for example with a label or a log step) inside the closure. After a transaction step closes — however it closes — later queries are recorded as top-level siblings again; the internal step stack always rebalances, even when a boundary statement failed. ## `CaptureOptions` See [Configuration](/docs/diesel-configuration/) for the full list of builder methods (`without_transactions`, `with_connection_events`, `without_sql_attachment`, `with_max_sql_preview`) and their defaults. --- ## Frameworks > URL: https://allurereport.org/docs/frameworks/ # Frameworks Info: If you can't find test framework or tool you're searching for in our documentation, we invite you to request new articles or upvote existing requests in our [GitHub Discussions](https://github.com/orgs/allure-framework/discussions/categories/documentation). ## C# - [Allure NUnit](/docs/nunit/) - [Allure Reqnroll](/docs/reqnroll/) - [Allure SpecFlow](/docs/specflow/) - [Allure xUnit.net](/docs/xunit/) ## Java - [Allure Cucumber-JVM](/docs/cucumberjvm/) - [Allure JBehave](/docs/jbehave/) - [Allure JUnit 4](/docs/junit4/) - [Allure JUnit Jupiter](/docs/junit-jupiter/) - [Allure REST Assured](/docs/restassured/) - [Allure Selenide](/docs/selenide/) - [Allure Selenium BiDi](/docs/selenium-bidi/) - [Allure TestNG](/docs/testng/) ## Dart - [Allure Dart and Flutter](/docs/dart/) ## Go - [Allure Go](/docs/go/) ## Groovy - [Allure Spock Framework](/docs/spock/) ## JavaScript / TypeScript - [Allure AVA](/docs/ava/) - [Allure Axios](/docs/axios/) - [Allure Bun](/docs/bun/) - [Allure Chai](/docs/chai/) - [Allure CodeceptJS](/docs/codeceptjs/) - [Allure Cucumber.js](/docs/cucumberjs/) - [Allure Cypress](/docs/cypress/) - [Allure Fetch](/docs/fetch/) - [Allure Jasmine](/docs/jasmine/) - [Allure Jest](/docs/jest/) - [Allure Mocha](/docs/mocha/) - [Allure Newman](/docs/newman/) - [Allure Node.js Test Runner](/docs/node-test/) - [Allure Playwright](/docs/playwright/) - [Allure TestCafe](/docs/testcafe/) - [Allure Vitest](/docs/vitest/) - [Allure WebdriverIO](/docs/webdriverio/) ## PHP - [Allure Behat](/docs/behat/) - [Allure Codeception](/docs/codeception/) - [Allure PHPUnit](/docs/phpunit/) ## Python - [Allure Behave](/docs/behave/) - [Allure Pytest](/docs/pytest/) - [Allure Pytest-BDD](/docs/pytestbdd/) - [Allure Robot Framework](/docs/robotframework/) ## Ruby - [Allure Cucumber.rb](/docs/cucumberrb/) - [Allure RSpec](/docs/rspec/) ## Rust - [Allure Diesel](/docs/diesel/) - [Allure Reqwest](/docs/reqwest/) - [Rust Cargo Test](/docs/rust/) --- ## Fetch > URL: https://allurereport.org/docs/fetch/ # Getting started with Allure Fetch [![Allure Fetch npm latest version](https://img.shields.io/npm/v/allure-fetch?style=flat "Allure Fetch npm latest version")](https://www.npmjs.com/package/allure-fetch) Capture [`fetch`](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) HTTP requests and responses as attachments in your [Allure Report](https://allurereport.org/docs/) test reports. Allure Fetch is compatible with any Allure JS framework integration, including [Allure Vitest](/docs/vitest/), [Allure Jest](/docs/jest/), [Allure Playwright](/docs/playwright/), [Allure Mocha](/docs/mocha/), and others. To enable the integration in your project: 1. Make sure you have an Allure integration set up for the test framework you use. See the instructions in the integration's documentation in [Frameworks](/docs/frameworks/). 1. Install `allure-fetch`. **npm:** ```bash npm install --save-dev allure-fetch ``` **yarn:** ```bash yarn add --dev allure-fetch ``` **pnpm:** ```bash pnpm add -D allure-fetch ``` 1. Wrap your `fetch` implementation with `withAllure()` to start capturing requests. The example below shows usage in a Vitest test suite. ```ts import { withAllure } from "allure-fetch"; import { describe, expect, it } from "vitest"; const client = withAllure(fetch); describe("Orders API", () => { it("creates an order", async () => { const response = await client("https://api.example.com/orders", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ item: "book", qty: 1 }), }); expect(response.status).toBe(201); }); }); ``` Every HTTP request made through `client` will be added to the test result as an [HTTP exchange attachment](/docs/attachments/#http-exchanges), wrapped in a step of the same name. The attachment captures the full exchange in one place: the request method, URL, headers, query parameters, cookies, and body; the response status, headers, cookies (including attributes such as `Path`, `Domain`, `Expires`, `Secure`, `HttpOnly`, and `SameSite`), and body; the exchange start and stop timestamps; and, when the request fails, the error name and message. See the [attachment schema reference](/docs/fetch-reference/#attachment-schema) for the exact field list. If your tests call the global `fetch` directly, use `instrumentGlobalFetch()` instead. It replaces `globalThis.fetch` with an instrumented version and returns a restore function. Call the restore function when you no longer want requests captured — typically in `afterEach` or `afterAll`: ```ts import { instrumentGlobalFetch } from "allure-fetch"; import { afterEach, beforeEach, describe, expect, it } from "vitest"; describe("Orders API", () => { let restore: () => void; beforeEach(() => { restore = instrumentGlobalFetch(); }); afterEach(() => { restore(); }); it("creates an order", async () => { const response = await fetch("https://api.example.com/orders", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ item: "book", qty: 1 }), }); expect(response.status).toBe(201); }); }); ``` Info: To prevent secrets from leaking into reports, certain values are redacted by default — all cookies, any header or query parameter whose name suggests it contains a credential, and any URL-encoded form field whose name does the same. See [Allure Fetch configuration](/docs/fetch-configuration/) for the full list of defaults and how to customize the redaction rules. 1. Run your tests and generate a test report the same way as you would normally. --- ## Fetch configuration > URL: https://allurereport.org/docs/fetch-configuration/ # Allure Fetch configuration This page describes the options that affect the behavior of the [Allure Fetch](/docs/fetch/) integration. All options are passed as the second argument to `withAllure()` or `instrumentGlobalFetch()`: ```ts withAllure(fetch, { // options here }); instrumentGlobalFetch({ // options here }); ``` `withAllure()` validates its first argument and throws `"allure-fetch requires a fetch implementation"` immediately if it is `null`, `undefined`, or otherwise falsy. `instrumentGlobalFetch()` throws `"allure-fetch can't instrument global fetch because globalThis.fetch is not defined"` if `globalThis.fetch` is absent — for example, in Node.js environments without a `fetch` polyfill. ## Customize the attachment name - `attachmentName: string | ((exchange: HttpExchange) => string)` Specify the name under which Allure Fetch creates the attachment for each HTTP exchange. For the shape of the `HttpExchange` argument, see [Allure Fetch reference](/docs/fetch-reference/). By default, each attachment is named `"HTTP Exchange"`. To use a different static name for all attachments, pass a string: ```ts withAllure(fetch, { attachmentName: "API request", }); ``` To derive the name dynamically from the exchange data, pass a function. The URL available in `exchange.request.url` is the raw, unredacted URL. Redaction only applies to the structured fields inside the attachment (`query`, `headers`, `cookies`, `form`), not to the URL string used in `exchange.request.url`. ```ts withAllure(fetch, { attachmentName: (exchange) => `${exchange.request.method} ${new URL(exchange.request.url).pathname}`, }); ``` ## Control body capture - `captureRequestBody: boolean` (default: `true`) - `captureResponseBody: boolean` (default: `true`) - `maxBodySize: number` (default: `65536`) By default, Allure Fetch captures both the request and response body. Set `captureRequestBody` or `captureResponseBody` to `false` to skip one or both: ```ts withAllure(fetch, { captureRequestBody: false, captureResponseBody: false, }); ``` `maxBodySize` sets the maximum number of bytes to capture from any single body. Bodies that exceed this limit are captured up to the limit and flagged as truncated in the attachment. The default is 64 KiB (65 536 bytes). ```ts withAllure(fetch, { maxBodySize: 1024 * 1024, // 1 MiB }); ``` Any value less than `1` is treated as `0`, which captures body metadata but stores an empty `value`. For details on how captured bodies are encoded and what the attachment fields contain, see [Allure Fetch reference](/docs/fetch-reference/). ## Redaction Allure Fetch replaces sensitive values with the sentinel string `__ALLURE_REDACTED__` before writing the attachment. The defaults are: | Category | Redacted by default | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Headers | `authorization`, `cookie`, `set-cookie`, and any header whose name contains `token`, `password`, `passwd`, `secret`, `api-key`/`api_key`/`apikey` (one pattern), or `session` | | Query parameters | Any parameter whose name contains the same patterns listed above | | Cookies | All cookies | | URL-encoded form fields | Any field whose name contains the same patterns listed above | The following options let you override the defaults for each category. Each accepts an array of `RedactionMatcher` values — strings, regular expressions, or functions. Providing an array replaces the defaults for that category entirely. See [Allure Fetch reference](/docs/fetch-reference/) for the full `RedactionMatcher` and `RedactionContext` type documentation. - `redactHeaders: RedactionMatcher[]` - `redactQueryParams: RedactionMatcher[]` - `redactCookies: RedactionMatcher[]` - `redactFormFields: RedactionMatcher[]` ```ts withAllure(fetch, { // Only redact headers explicitly listed, nothing else redactHeaders: ["authorization", "x-api-key"], // Allow cookies through unredacted redactCookies: [], }); ``` ## Error options - `includeErrorStack: boolean` (default: `false`) When a request fails — whether due to a transport error, an abort, or a response body stream error — Allure Fetch records the error's name and message in the attachment. Set `includeErrorStack` to `true` to also capture the stack trace: ```ts withAllure(fetch, { includeErrorStack: true, }); ``` - `throwAttachmentErrors: boolean` (default: `false`) By default, if writing the attachment fails — for example, because no Allure runtime is active during the request — the error is silently ignored. Set `throwAttachmentErrors` to `true` to have such errors rethrow instead. This is useful when debugging why attachments are not appearing: ```ts withAllure(fetch, { throwAttachmentErrors: true, }); ``` Info: `throwAttachmentErrors` only takes effect when the HTTP request itself succeeds. If the request fails (due to a transport error, abort, or response body stream error) and writing the attachment also fails, the attachment error is still silently ignored so the original request error propagates cleanly. --- ## Fetch reference > URL: https://allurereport.org/docs/fetch-reference/ # Allure Fetch reference ## Attachment schema Each HTTP exchange is recorded as an [HTTP exchange attachment](/docs/attachments/#http-exchanges) with content type `application/vnd.allure.http+json` and file extension `.httpexchange`. The root object has the following shape: | Field | Type | Description | | --------------- | -------- | --------------------------------------------------------------------------------------------- | | `schemaVersion` | `1` | Schema version. Always `1`. | | `start` | `number` | Unix timestamp in milliseconds when the request started. | | `stop` | `number` | Unix timestamp in milliseconds when the response or error was received. | | `request` | object | Captured request data. See [Request](#request). | | `response` | object | Captured response data. Present only when a response was received. See [Response](#response). | | `error` | object | Error information. Present only when the request failed. See [Error](#error). | ### Request | Field | Type | Description | | --------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------- | | `method` | `string` | HTTP method in uppercase, e.g. `"GET"`. | | `url` | `string` | Request URL, including query parameters in their original unredacted form. | | `headers` | `{name, value}[]` | Request headers after redaction. Omitted when there are no headers. | | `query` | `{name, value}[]` | Query parameters parsed from the URL, after redaction. Omitted when there are no query parameters. | | `cookies` | `Cookie[]` | Cookies parsed from the `Cookie` request header, after redaction. Omitted when there are no cookies. See [Cookie](#cookie). | | `body` | object | Request body. Present only when `captureRequestBody` is `true` and a body was sent. See [Body](#body). | ### Response | Field | Type | Description | | ------------ | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `status` | `number` | HTTP status code. | | `statusText` | `string` | HTTP status text. | | `headers` | `{name, value}[]` | Response headers after redaction. Omitted when there are no headers. | | `cookies` | `Cookie[]` | Cookies parsed from `Set-Cookie` response headers, after redaction. Omitted when there are no `Set-Cookie` headers. See [Cookie](#cookie). | | `body` | object | Response body. Present only when `captureResponseBody` is `true` and a body was received. See [Body](#body). | Request and response bodies are captured from cloned `Request` and `Response` objects before they are consumed, so the `Response` returned to your test code remains fully readable. ### Body | Field | Type | Description | | ------------- | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | | `contentType` | `string` | Value of the `Content-Type` header. Absent when no `Content-Type` header is present. | | `encoding` | `"utf8" \| "base64"` | Encoding used for `value`. Present for all captured bodies except multipart. See [Body encoding](#body-encoding). | | `value` | `string` | The captured body content, encoded per `encoding`. Present for all captured bodies except multipart. | | `size` | `number` | Body size in bytes. See [Body size](#body-size). | | `truncated` | `boolean` | `true` when the body exceeded `maxBodySize` and was cut off. | | `form` | `{name, value}[]` | Structured fields parsed from a `application/x-www-form-urlencoded` body, after redaction. See [URL-encoded form bodies](#url-encoded-form-bodies). | | `stream` | object | Present alongside `value` when the body was truncated during streaming. See [Stream bodies](#stream-bodies). | | `parts` | object[] | Structured parts for `multipart/form-data` bodies. See [Multipart bodies](#multipart-bodies). Present instead of `value`, `encoding`, and `stream`. | #### Body encoding `encoding` is determined by the `Content-Type` header: - **Text content types** — `text/*`, `application/json`, `application/javascript`, `application/x-www-form-urlencoded`, `application/xml`, or any type ending in `+json` or `+xml` → `"utf8"`. - **No `Content-Type` header** → `"utf8"` (treated as text). - **All other content types** → `"base64"`. #### URL-encoded form bodies When the content type is `application/x-www-form-urlencoded`, the body contains two representations of the same data: - `value`: the re-serialized URL-encoded string, with sensitive fields replaced by the redaction placeholder. - `form`: a structured `{name, value}[]` array with the same redaction applied. #### Stream bodies When a body exceeds `maxBodySize`, reading stops and the `stream` field appears alongside `value` to indicate partial capture: | Field | Type | Description | | ---------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `complete` | `boolean` | Always `false` when `stream` is present. | | `type` | `string` | Stream type derived from the content type: `text/event-stream` → `"server-sent-events"`, `application/grpc` → `"grpc"`, all others → `"chunked"`. | When `stream` is present, `value` contains the captured bytes up to `maxBodySize` and `truncated` is `true`. #### Multipart bodies When the content type is `multipart/form-data`, the body is parsed via the Web API `formData()` method and stored as a `parts` array instead of `value`. Each part is one of: - **Text field**: `{ name, encoding: "utf8", value, size, truncated: false }`. Sensitive field values are redacted. - **File part**: `{ name, fileName?, contentType?, size, truncated: false }`. File contents are not captured; only the metadata is recorded. #### Body size The `size` field uses the `Content-Length` header when present. If `Content-Length` is absent and the body was fully captured within `maxBodySize`, `size` is the actual byte count. If `Content-Length` is absent and the body was truncated, `size` is omitted. ### Cookie Request cookies are parsed from the `Cookie` header. Response cookies are parsed from `Set-Cookie` headers. | Field | Type | Description | | ---------- | --------- | ----------------------------------------------------------------------- | | `name` | `string` | Cookie name. | | `value` | `string` | Cookie value, after redaction. | | `path` | `string` | `Path` attribute. Response cookies only. | | `domain` | `string` | `Domain` attribute. Response cookies only. | | `expires` | `string` | `Expires` attribute. Response cookies only. | | `httpOnly` | `boolean` | `true` when the `HttpOnly` attribute is present. Response cookies only. | | `secure` | `boolean` | `true` when the `Secure` attribute is present. Response cookies only. | | `sameSite` | `string` | `SameSite` attribute value. Response cookies only. | ### Error Present when the request failed. In `fetch`, this occurs for transport errors (e.g., network failure), aborts (via `AbortController`), and response body stream errors. HTTP error responses (4xx and 5xx) are not errors — they produce a normal `response` with no `error` field. | Field | Type | Description | | --------- | -------- | ------------------------------------------------------------- | | `name` | `string` | The error's `name` property. | | `message` | `string` | The error message. | | `stack` | `string` | Stack trace. Present only when `includeErrorStack` is `true`. | ## Redaction types ### `RedactionMatcher` A `RedactionMatcher` controls whether a specific field value should be redacted. It can be any of the following: - A **string**: redacts any field whose name equals the string, case-insensitively. - A **`RegExp`**: redacts any field whose name matches the pattern. - A **function** `(name: string, value: string, context: RedactionContext) => boolean`: redacts based on any logic, with access to the field name, value, and surrounding context. ```ts withAllure(fetch, { redactHeaders: [ "x-api-key", // exact name, case-insensitive /^x-internal-/i, // regex on the header name (name, value) => value.startsWith("sk-"), // match on the value itself ], }); ``` ### `RedactionContext` The `context` argument passed to a function `RedactionMatcher`: | Property | Type | Description | | ---------- | ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `kind` | `"header" \| "query" \| "cookie" \| "form"` | Which category is being evaluated. | | `name` | `string` | The field name. | | `value` | `string` | The field value. | | `url` | `string \| undefined` | The request URL. | | `request` | `Request \| undefined` | The Web API `Request` object. | | `response` | `Response \| undefined` | The Web API `Response` object. Present only when evaluating response-phase fields (response headers, response cookies). | --- ## Go Testing > URL: https://allurereport.org/docs/go/ # Getting started with Go Testing [![allure-go gotest package reference](https://pkg.go.dev/badge/github.com/allure-framework/allure-go/commons/gotest.svg "allure-go gotest package reference")](https://pkg.go.dev/github.com/allure-framework/allure-go/commons/gotest) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your Go tests. Info: The main user-facing package is [`commons/gotest`](https://github.com/allure-framework/allure-go/tree/main/commons/gotest), which works with Go's built-in `testing` package. If you are building your own integration for a custom runner or helper library, use [`commons`](https://github.com/allure-framework/allure-go/tree/main/commons) instead. Both packages live in the official `allure-go` repository, which also provides [testify-compatible assertion packages](#report-testify-assertions-as-steps) and [HTTP evidence helpers](#attach-http-traffic). ## Setting up ### 1. Prepare your project 1. Make sure a recent Go toolchain is installed. The official `allure-go` packages require Go 1.25 or newer. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Install Allure Report, following the [installation guide](/docs/v3/install/). 1. Add the `gotest` integration: ```bash go get github.com/allure-framework/allure-go/commons/gotest ``` 1. Wrap your tests with `allure.Test`. Inside the body, use the provided context to declare steps, parameters, and attachments. ```go package example_test import ( "testing" allure "github.com/allure-framework/allure-go/commons/gotest" ) func TestLogin(t *testing.T) { allure.Test(t, "logs in with valid credentials", func(a *allure.Context) { a.Step("submit credentials", func(a *allure.Context) { a.Parameter("user", "alice") a.Attachment("request", []byte(`{"user":"alice"}`), "application/json") }) session := allure.Step(a, "create session", func(a *allure.Context) string { return "session-1" }) a.Parameter("session", session) }, allure.WithOwner("qa-team"), allure.WithDescription("Checks that valid credentials create a session."), ) } ``` ### 2. Run tests Run your tests the same way as usual: ```bash go test ./... ``` By default, the integration writes the results to an `allure-results` directory next to each test package. Go runs every test binary in its package's source directory, so in a module with multiple test packages the relative default produces several `allure-results` directories. To collect all results in one place, set `ALLURE_RESULTS_DIR` to an absolute path before the test run: ```bash ALLURE_RESULTS_DIR=$PWD/allure-results go test ./... ``` If the results directory already exists, the new files are added to the existing ones, so that a future report will be based on all of them. ### 3. Generate a report After the test run, generate and open the report with the Allure CLI: ```bash allure generate ./allure-results --output ./allure-report allure open ./allure-report ``` ## Writing tests The Allure Go integration extends plain `go test` output with richer reporting features. You can use it to: - add [descriptions, owners, links, and other metadata](#add-metadata), - organize tests into [behavior-based and suite-based hierarchies](#organize-tests), - split the execution into [nested steps](#divide-a-test-into-steps), - describe [parameters and attachments](#add-parameters-and-attachments), - report [testify assertions as steps](#report-testify-assertions-as-steps), - attach [structured HTTP traffic evidence](#attach-http-traffic), - run only selected tests through a [test plan file](#select-tests-via-a-test-plan-file). ### Add metadata Metadata that is known before the test body runs is best passed as static `allure.With...` options, since they participate in [test plan filtering](#select-tests-via-a-test-plan-file). Metadata discovered during execution can be added at runtime with methods on the test's `*allure.Context`: ```go func TestLogin(t *testing.T) { allure.Test(t, "Login works", func(a *allure.Context) { a.Label("layer", "web") a.Link("https://example.com/docs", "Related docs", "link") }, allure.WithDescription("This test verifies login with a username and a password."), allure.WithOwner("John Doe"), allure.WithTag("smoke"), allure.WithSeverity("critical"), allure.WithAllureID("123"), allure.WithIssue("AUTH-123", "https://jira.example.com/browse/AUTH-123"), allure.WithTMS("TMS-456", "https://tms.example.com/cases/TMS-456"), ) } ``` ### Organize tests Allure supports both behavior-based and suite-based hierarchies. For example: ```go func TestLogin(t *testing.T) { allure.Test(t, "Login works", func(a *allure.Context) { // ... }, allure.WithEpic("Web interface"), allure.WithFeature("Authentication"), allure.WithStory("Login with username and password"), allure.WithParentSuite("UI tests"), allure.WithSuite("Authentication"), allure.WithSubSuite("Positive scenarios"), ) } ``` When no suite labels are set explicitly, the integration derives them automatically from the test file's path relative to the module root. Setting any of `WithParentSuite`, `WithSuite`, or `WithSubSuite` (or the equivalent runtime labels) replaces the entire derived hierarchy. ### Divide a test into steps Use `a.Step` to report a block of code as a step. Steps can be nested: ```go func TestLogin(t *testing.T) { allure.Test(t, "Login works", func(a *allure.Context) { a.Step("Open login page", func(a *allure.Context) { // ... }) a.Step("Submit credentials", func(a *allure.Context) { a.Step("Fill the form", func(a *allure.Context) { // ... }) a.Step("Click the button", func(a *allure.Context) { // ... }) }) }) } ``` When a step should produce a value, use the package-level generic `allure.Step` function: ```go token := allure.Step(a, "Authorize", func(a *allure.Context) string { return "token-123" }) ``` A step that panics is reported as broken with the panic message and stack trace; a step in which the test fails or is skipped is reported with the corresponding status. ### Add parameters and attachments Parameters and attachments are stored in the generated Allure results and rendered in the report: ```go func TestLogin(t *testing.T) { allure.Test(t, "Login works", func(a *allure.Context) { a.Parameter("browser", "firefox") a.Attachment( "request.json", []byte(`{"username":"demo","rememberMe":true}`), "application/json", ) a.AttachmentPath("server log", "./testdata/server.log", "text/plain") }, allure.WithParameter("environment", "staging")) } ``` Attachments added inside a step are attached to that step. The file extension of the stored attachment is derived from the content type automatically. ### Report testify assertions as steps If your tests use [testify](https://github.com/stretchr/testify), replace the assertion imports with the Allure proxy packages from the separate `github.com/allure-framework/allure-go/testify` module: ```diff import ( - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" + "github.com/allure-framework/allure-go/testify/assert" + "github.com/allure-framework/allure-go/testify/require" ) ``` The proxy packages keep the exact testify API. Pass the Allure context `a` instead of `t` to report each assertion as an Allure step: ```go func TestProfile(t *testing.T) { allure.Test(t, "loads profile", func(a *allure.Context) { assert.Equal(a, "alice", profile.Name) require.NoError(a, err) assert.New(a).Len(profile.Roles, 2) }) } ``` Assertions that still receive a plain `*testing.T` behave exactly like upstream testify, without step reporting. ### Attach HTTP traffic Use the `commons/httpexchange` package when tests need structured request and response evidence. To capture requests made by an HTTP client, wrap its transport: ```go import "github.com/allure-framework/allure-go/commons/httpexchange" client := &http.Client{ Transport: httpexchange.NewTransport(a.Context(), http.DefaultTransport), } ``` To capture requests received by an `httptest.Server` fake service, wrap its handler: ```go server := httptest.NewServer(httpexchange.NewHandler(a.Context(), handler)) defer server.Close() ``` Each captured exchange becomes an attachment in the [Allure HTTP Exchange format](/docs/attachments/#http-exchanges), showing the method, URL, headers, cookies, query parameters, status, and bodies. Common credentials such as `Authorization` headers, cookies, and token-like query parameters are redacted by default. See [Reference](/docs/go-reference/#http-exchange-attachments) for the available options. ### Select tests via a test plan file The integration supports the standard Allure test plan mechanism via the `ALLURE_TESTPLAN_PATH` environment variable. Create a JSON file such as: ```json { "version": "1.0", "tests": [ { "id": "123" }, { "selector": "example/login_test.go/TestLogin/logs_in_with_valid_credentials" } ] } ``` Then run the tests with: ```bash ALLURE_TESTPLAN_PATH=./testplan.json go test ./... ``` Entries with `id` match tests that declare an Allure ID, for example via `allure.WithAllureID("123")`. Entries with `selector` match the test's full name, which consists of the test file's path relative to the module root and the full Go test name. Tests not selected by the plan are skipped before their body runs. ## Building a custom integration If you need to integrate Allure with a custom Go test harness or helper library, use the `commons` module: ```bash go get github.com/allure-framework/allure-go/commons ``` Helper libraries that run inside an active Allure test only need a `context.Context` and the `commons` facade — the calls safely do nothing when no test is active: ```go import commons "github.com/allure-framework/allure-go/commons" func instrument(ctx context.Context) { _ = commons.Step(ctx, "call login endpoint", func(ctx context.Context) error { return commons.Attachment(ctx, "payload", []byte(`{"ok":true}`), commons.AttachmentOptions{ ContentType: "application/json", }) }) } ``` Adapters that manage the test lifecycle themselves additionally use the `runtime`, `writer`, `model`, and `testplan` packages. See [Reference](/docs/go-reference/) for the main functions and options, or [Configuration](/docs/go-configuration/) for the supported environment variables. --- ## Go Testing configuration > URL: https://allurereport.org/docs/go-configuration/ # Go Testing configuration The [`allure-go`](/docs/go/) integration is configured through environment variables. ## ALLURE_RESULTS_DIR Overrides the default directory where the integration writes Allure results. When unset, results are written to `allure-results`, resolved relative to the working directory of each test binary. Go runs each test binary in its package's source directory, so in a module with multiple test packages, set this variable to an absolute path to collect all results in one directory: ```bash ALLURE_RESULTS_DIR=$PWD/allure-results go test ./... ``` ## ALLURE_LABEL\_\* Adds global labels to every test result. Any environment variable whose name starts with `ALLURE_LABEL_` becomes an Allure label. For example: ```bash ALLURE_LABEL_EPIC="Web interface" \ ALLURE_LABEL_OWNER="QA Team" \ go test ./... ``` This applies the `epic` and `owner` labels to every test in the run. Label names are normalized to Allure-style lower camel case: the name after the prefix is lowercased, and underscores and hyphens start a new capitalized word. For example, `ALLURE_LABEL_MODULE=commons` becomes the label `module=commons`, and `ALLURE_LABEL_PARENT_SUITE=runtime` becomes `parentSuite=runtime`. Variables with empty values are ignored. ## ALLURE_TESTPLAN_PATH Points to a JSON file that defines which tests should run. The file uses the standard Allure test plan shape: ```json { "version": "1.0", "tests": [ { "id": "123" }, { "selector": "example/login_test.go/TestLogin/logs_in_with_valid_credentials" } ] } ``` Run the tests with: ```bash ALLURE_TESTPLAN_PATH=./testplan.json go test ./... ``` Selection works as follows: - entries with `id` match tests that declare an Allure ID, for example via `allure.WithAllureID("123")` or a `@allure.id=123` tag, - entries with `selector` match the test's full name: the test file's path relative to the module root, followed by the full Go test name, joined with `/`. For example, a test declared as `allure.Test(t, "logs in", ...)` inside `TestLogin` in `example/login_test.go` has the full name `example/login_test.go/TestLogin/logs_in`. Tests not selected by the plan are skipped with `t.Skip` before their body runs. Only metadata known before the test body runs participates in the selection: static `allure.With...` options and `ALLURE_LABEL_*` variables. An Allure ID set at runtime with `a.Label(...)` is too late to affect filtering. If `ALLURE_TESTPLAN_PATH` is unset or the file contains no test entries, all tests run normally. If the file cannot be read or contains invalid JSON, each reported test fails and is written to the results as broken, so a misconfigured test plan does not silently run or skip the whole suite. ## Automatic labels and identifiers The integration adds two labels to every test result automatically: - `language = go` - `framework = go-test` It also derives suite labels from the test file's path relative to the module root (the directory containing `go.mod`): - a single path segment becomes `suite`, - two segments become `parentSuite` and `suite`, - three or more segments become `parentSuite`, `suite`, and `subSuite` (the remaining segments joined with `>`). For example, a test in `commons/gotest/allure_test.go` gets `parentSuite = commons`, `suite = gotest`, and `subSuite = allure_test.go`. Setting any of the `parentSuite`, `suite`, or `subSuite` labels explicitly — via options such as `allure.WithSuite(...)`, runtime calls, or `ALLURE_LABEL_*` variables — disables the automatic derivation entirely. Finally, unless set explicitly, each result's `testCaseId` is computed from the test's full name, and its `historyId` is computed from the `testCaseId` and the test's parameters (parameters marked as excluded do not affect it). These identifiers keep retries, history, and trend charts stable across runs. --- ## Go Testing reference > URL: https://allurereport.org/docs/go-reference/ # Go Testing reference These are the main building blocks you can use to integrate Go tests with Allure by using the `github.com/allure-framework/allure-go/commons/gotest` package. Import it under the `allure` alias: ```go import allure "github.com/allure-framework/allure-go/commons/gotest" ``` ## Test functions ### `allure.Test` `allure.Test(t, name, body, options...)` reports one named Allure test result. It creates a Go subtest via `t.Run(name, ...)`, so failures, skips, logs, cleanups, steps, and attachments stay attached to the correct result, and standard `go test -run` filtering keeps working: ```go func TestLogin(t *testing.T) { allure.Test(t, "logs in with valid credentials", func(a *allure.Context) { // ... }, allure.WithOwner("qa-team")) } ``` One Go test function may contain multiple `allure.Test` calls; each produces a separate Allure result. ### `allure.Wrap` `allure.Wrap(t, body, options...)` reports the current Go test as exactly one Allure result. It uses `t.Name()` as the result name and does not create a child subtest: ```go func TestLogin(t *testing.T) { allure.Wrap(t, func(a *allure.Context) { // ... }) } ``` `allure.Wrap` may be called only once per test, and cannot be combined with `allure.Test` on the same `*testing.T` — use `allure.Test` when one Go test should produce multiple named Allure results. ### `allure.Step` The package-level generic `allure.Step(a, name, body)` reports a step whose body returns a typed value: ```go session := allure.Step(a, "create session", func(a *allure.Context) string { return "session-1" }) ``` For steps without a return value, use the [`a.Step` method](#steps) instead. ### Statuses The reported test status follows the outcome of the Go test: `passed`, `failed` (after `t.Error`, `t.Fatal`, or a failed assertion), or `skipped` (after `t.Skip`). A panic in the test body is reported as `broken` with the panic message and stack trace, and is then re-raised for `go test` to handle as usual. ## Static options Options are passed as trailing arguments to `allure.Test` and `allure.Wrap`. They apply before the test body runs, which makes them visible to [test plan filtering](/docs/go-configuration/#allure-testplan-path). Prefer options for metadata known up front; use the [runtime context API](#runtime-context-api) for metadata discovered during execution. ### Labels - `allure.WithLabel(name, value)` - `allure.WithLabels(labels...)` — takes `model.Label` values - `allure.WithTag(value)` - `allure.WithSeverity(value)` - `allure.WithOwner(value)` - `allure.WithAllureID(id)` ### Hierarchies - `allure.WithEpic(value)` - `allure.WithFeature(value)` - `allure.WithStory(value)` - `allure.WithParentSuite(value)` - `allure.WithSuite(value)` - `allure.WithSubSuite(value)` - `allure.WithPackage(value)` Setting any of the `parentSuite`, `suite`, or `subSuite` labels disables the [automatic suite hierarchy](/docs/go-configuration/#automatic-labels-and-identifiers) derived from the test file path. ### Links - `allure.WithLink(url, name, linkType)` - `allure.WithLinks(links...)` — takes `model.Link` values - `allure.WithIssue(name, url)` - `allure.WithTMS(name, url)` ### Parameters - `allure.WithParameter(name, value)` - `allure.WithParameterOptions(name, value, options)` `allure.ParameterOptions` controls display and history behavior: ```go allure.WithParameterOptions("password", "qwerty", allure.ParameterOptions{ Excluded: true, // do not affect the history ID Mode: model.ParameterModeMasked, // model.ParameterModeDefault, Masked, or Hidden }) ``` ### Descriptions and identity - `allure.WithDescription(markdown)` - `allure.WithDescriptionHTML(html)` - `allure.WithDisplayName(name)` — overrides the name shown in the report - `allure.WithTestCaseName(name)` — sets the logical test case name - `allure.WithTestCaseID(id)` — overrides the computed test case ID - `allure.WithHistoryID(id)` — overrides the computed history ID ### Test plan - `allure.WithTestPlan(plan)` — uses an already loaded `*testplan.Plan` instead of `ALLURE_TESTPLAN_PATH` - `allure.WithTestPlanPath(path)` — loads a test plan from a specific file ### Advanced - `allure.WithWriter(writer)` — routes this test's artifacts through a custom `writer.Writer`, for example an in-memory writer in adapter tests - `allure.WithClock(now)` — replaces the millisecond timestamp source - `allure.WithIDGenerator(newID)` — replaces the UUID generator ## Runtime context API The test body receives an `*allure.Context` (named `a` by convention) that combines access to the underlying `*testing.T` with Allure reporting methods. ### Bridge to `testing.T` - `a.T()` — the underlying `*testing.T` - `a.Name()` — the current Go test name - `a.Helper()` — marks the caller as a test helper - `a.Errorf(format, args...)` — fails the test and records the message as Allure status details - `a.Fatal(args...)`, `a.Fatalf(format, args...)` — fail the test with status details and stop execution - `a.FailNow()` — fails and stops without a message - `a.Context()` — the `context.Context` carrying the active Allure runtime state; pass it to helper libraries such as [`httpexchange`](#http-exchange-attachments) Failure messages reported through `a.Errorf`, `a.Fatal`, and `a.Fatalf` appear as the result's status details in the report. Calls made directly on `a.T()` still fail the test correctly but do not record status details. ### Steps - `a.Step(name, body)` — reports `body` as a step; steps nest naturally - `a.StepDisplayName(name)` — renames the active step - `a.StepParameter(name, value)` — adds a parameter to the active step - `a.StepDescription(markdown)`, `a.StepDescriptionHTML(html)` — set the active step description ```go a.Step("submit credentials", func(a *allure.Context) { a.StepParameter("endpoint", "/login") a.Step("wait for redirect", func(a *allure.Context) { // ... }) }) ``` ### Metadata - `a.Label(name, value)` - `a.Link(url, name, linkType)` - `a.Parameter(name, value)` - `a.Description(markdown)`, `a.DescriptionHTML(html)` - `a.DisplayName(name)` - `a.TestCaseName(name)`, `a.TestCaseID(id)`, `a.HistoryID(id)` These are the runtime counterparts of the static options. Note that runtime metadata is applied after test plan filtering, so an Allure ID needed for selection must be set with `allure.WithAllureID`. ### Attachments - `a.Attachment(name, content, contentType)` — attaches in-memory bytes - `a.AttachmentPath(name, path, contentType)` — attaches an existing file Attachments are added to the active step, or to the test itself when no step is active. For in-memory attachments, the stored file's extension is derived from the content type (for example, `application/json` produces a `.json` file); file attachments keep the extension of the source file. ### Run-level diagnostics - `a.GlobalAttachment(name, content, contentType)`, `a.GlobalAttachmentPath(name, path, contentType)` — attach evidence to the test run as a whole rather than to one test, for example a service log collected once per run - `a.GlobalError(details)` — records a run-level error, taking a `model.StatusDetails` value ## Testify assertions The separate module `github.com/allure-framework/allure-go/testify` provides drop-in replacements for testify's `assert` and `require` packages that report each assertion call as an Allure step: ```bash go get github.com/allure-framework/allure-go/testify/assert go get github.com/allure-framework/allure-go/testify/require ``` ```diff import ( - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" + "github.com/allure-framework/allure-go/testify/assert" + "github.com/allure-framework/allure-go/testify/require" ) ``` The proxy packages mirror the upstream testify API, including `assert.New(...)` / `require.New(...)` assertion objects. What is reported depends on the first argument of each call: - passing the Allure context `a` reports the assertion as an Allure step; failed assertions add status details, including the expected and actual values, - passing `t` or `a.T()` keeps plain testify behavior without step reporting. ```go allure.Test(t, "loads profile", func(a *allure.Context) { assert.Equal(a, "alice", profile.Name) // reported as a step require.NoError(a, err) // reported as a step assert.Equal(t, "alice", profile.Name) // plain testify, no step }) ``` Test context types from other integrations can opt into step reporting by satisfying testify's `TestingT` interface and the `commons.ContextProvider` interface (a `Context() context.Context` method). ## HTTP exchange attachments The `github.com/allure-framework/allure-go/commons/httpexchange` package produces attachments in the [Allure HTTP Exchange format](/docs/attachments/#http-exchanges) (`application/vnd.allure.http+json`, stored with the `.httpexchange` extension), which report viewers render as structured request/response evidence. ### Capture helpers - `httpexchange.NewTransport(ctx, base, options...)` — wraps an `http.RoundTripper`; every request sent through the client is attached to the active test or step - `httpexchange.NewHandler(ctx, next, options...)` — wraps an `http.Handler`; every request received by the server is attached, which suits `httptest.Server` fakes ```go client := &http.Client{ Transport: httpexchange.NewTransport(a.Context(), http.DefaultTransport), } server := httptest.NewServer(httpexchange.NewHandler(a.Context(), handler)) ``` Both take the Allure context from `a.Context()` and safely do nothing when no Allure test is active. ### Building exchanges manually When the body bytes are already captured by your own code: - `httpexchange.NewExchange(req, requestBody, resp, responseBody, options...)` — builds a complete exchange from `*http.Request` and `*http.Response` values - `httpexchange.FromRequest(req, body, options...)`, `httpexchange.FromResponse(resp, body, options...)` — build the request or response portion separately - `httpexchange.Attach(ctx, name, exchange)` — attaches an exchange to the active test or step - `httpexchange.Marshal(exchange)` — serializes an exchange to JSON without attaching it ### Options - `httpexchange.WithBodyLimit(limit)` — limits captured body values, 64 KiB by default; a negative limit captures everything, zero records body metadata without content - `httpexchange.WithRedactedHeaders(names...)` — adds header names to redact; `Authorization`, `Proxy-Authorization`, `Cookie`, `Set-Cookie`, `X-Api-Key`, and `X-Auth-Token` are redacted by default - `httpexchange.WithRedactedQueryParameters(names...)` — adds query parameters to redact; common token-like names such as `token`, `password`, `api_key`, and `client_secret` are redacted by default - `httpexchange.WithRedactedFormFields(names...)` — adds URL-encoded form fields to redact; the same token-like defaults apply - `httpexchange.WithRedactedCookies(names...)` — adds cookie names to redact; by default, all cookie values are redacted - `httpexchange.WithAttachmentName(name)` / `httpexchange.WithAttachmentNamer(func)` — set a fixed or computed attachment name; the default is `HTTP `, for example `HTTP GET /orders?page=2` - `httpexchange.WithStartStop(start, stop)` — sets explicit exchange timestamps in Unix epoch milliseconds - `httpexchange.WithError(err)` / `httpexchange.WithErrorDetails(name, message, stack)` — record a transport-level error on the exchange Redacted values are replaced with the `__ALLURE_REDACTED__` sentinel, which report viewers display as a redaction marker. ## Building a custom integration with `commons` Use the `github.com/allure-framework/allure-go/commons` module when you need to integrate Allure with a custom test harness, or to write a helper library that reports into whatever Allure test is currently active. Helper libraries only need the facade functions, which mirror the runtime context API but take a `context.Context` and return errors. All of them safely do nothing when no Allure runtime is bound to the context: ```go import commons "github.com/allure-framework/allure-go/commons" func instrument(ctx context.Context) error { if err := commons.Owner(ctx, "qa-team"); err != nil { return err } return commons.Step(ctx, "call login endpoint", func(ctx context.Context) error { return commons.Attachment(ctx, "payload", []byte(`{"ok":true}`), commons.AttachmentOptions{ ContentType: "application/json", }) }) } ``` The main facade groups are: - labels and links: `Label`, `Labels`, `Tag`, `Severity`, `Owner`, `Epic`, `Feature`, `Story`, `ParentSuite`, `Suite`, `SubSuite`, `Package`, `AllureID`, `Link`, `Links`, `Issue`, `TMS` - test metadata: `Parameter`, `ParameterWithOptions`, `Description`, `DescriptionHTML`, `DisplayName`, `TestCaseName`, `TestCaseID`, `HistoryID` - steps: `Step`, `StepValue` (returns a typed value), `StartStep`, `StopStep`, `LogStep` (records an already completed step), `StepDisplayName`, `StepParameter`, `StepParameterWithOptions`, `StepDescription`, `StepDescriptionHTML` - attachments and diagnostics: `Attachment`, `AttachmentPath`, `GlobalAttachment`, `GlobalAttachmentPath`, `GlobalError` Adapters that manage the test lifecycle themselves build on the lower-level subpackages: - `commons/runtime` — the context-bound message bridge; bind ownership with `runtime.WithRuntime`, `runtime.WithTest`, `runtime.WithFixture`, and `runtime.WithStep` - `commons/writer` — result persistence: `NewFileSystemWriter` for `allure-results` directories, `NewInMemoryWriter` for tests, `NewMultiWriter` to fan out - `commons/model` — the serializable Allure result types (`TestResult`, `StepResult`, `Label`, `Link`, `Parameter`, `Status`, `StatusDetails`, and others) - `commons/testplan` — test plan loading and matching: `LoadFromEnv`, `LoadFile`, `Parse`, `Includes` - `commons/ids` — stable identity helpers: `New`, `TestCaseID`, `HistoryID` - `commons/clock` — Unix epoch-millisecond timestamp helpers - `commons/lifecycle` — lifecycle interfaces and event hooks for adapter authors --- ## Jasmine > URL: https://allurereport.org/docs/jasmine/ # Getting started with Allure Jasmine [![Allure Jasmine npm latest version](https://img.shields.io/npm/v/allure-jasmine?style=flat "Allure Jasmine npm latest version")](https://www.npmjs.com/package/allure-jasmine) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Jasmine](https://jasmine.github.io/) tests. This integration also can be used with [Jest](https://jestjs.io/) if your project uses the Jasmine test runner, which is the default in Jest version 26 or older. (However, switching to Circus and using [Allure Jest](/docs/jest/) is recommended.) ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure Jasmine is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure Jasmine integration. **npm:** ```bash npm install --save-dev allure-jasmine ``` **yarn:** ```bash yarn add --dev allure-jasmine allure-js-commons ``` **pnpm:** ```bash pnpm add -D allure-jasmine ``` 1. Create a helper file, e.g., `helpers/setup.js` or `helpers/setup.ts`. You can choose any filename, as long as the test framework can find it according to its configuration. - If you use Jasmine, make sure that the filename matches one of the regular expressions in the `helpers` option from your [`spec/support/jasmine.json`](https://jasmine.github.io/setup/nodejs.html#configuration) file. - If you use Jest, make sure that the filename is listed in the [`setupFilesAfterEnv`](https://jestjs.io/docs/configuration#setupfilesafterenv-array) option from your `jest.config.js` file. ```ts import AllureJasmineReporter from "allure-jasmine"; jasmine.getEnv().addReporter( new AllureJasmineReporter({ resultsDir: "allure-results", }), ); ``` 1. Configure other options if necessary, see [Allure Jasmine configuration](/docs/jasmine-configuration/). ### 2. Run tests Run your Jasmine tests the same way as you would run them usually. For example: **npm:** ```bash npm test ``` **yarn:** ```bash yarn test ``` **pnpm:** ```bash pnpm test ``` This will save necessary data into `allure-results` or other directory, according to the [configuration](/docs/jasmine-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Jasmine integration extends the standard reporting features of Jasmine by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure Jasmine provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Jasmine will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/jasmine-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.owner("John Doe"); await allure.tags("Web interface", "Authentication"); await allure.severity("critical"); // ... }); }); ``` **Metadata API:** ```ts describe("Test My Website", function () { it( "Test Authentication" + " @allure.label.owner:JohnDoe" + " @allure.label.tag:WebInterface" + " @allure.label.tag:Authentication" + " @allure.label.severity:critical", async () => { // ... }, ); }); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); }); ``` **Metadata API:** ```ts describe("Test My Website", function () { it( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); }); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); }); ``` **Metadata API:** ```ts describe("Test My Website", function () { it( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); }); ``` ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), you can use the `step()` function, see the [reference](/docs/jasmine-reference/#test-steps). ```ts import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); }); ``` ### Describe parametrized tests A typical way to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern in Jasmine is to define the test in a loop and use the loop variable in the test's title and body. To display a parameter value in the test report, pass it to the [`parameter()`](/docs/jasmine-reference/#parametrized-tests) function. ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { for (const login of ["johndoe", "johndoe@example.com"]) { it(`Test Authentication as ${login}`, async () => { allure.parameter("login", login); allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } }); ``` ### Set labels globally Any [labels](/docs/jasmine-reference/#label), including custom ones, can be set via the environment variables in your operating system. Here's an example (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface npm test ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" npm test ``` ### Attach screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure Jasmine reference](/docs/jasmine-reference/#attachments). ```ts import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); }); ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Jasmine will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json` (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json npm test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" npm test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, edit the [`environmentInfo`](/docs/jasmine-configuration/#environmentinfo) object in the configuration. For example, it is a good idea to use this to remember the OS version and Node.js version retrieved from the [`os`](https://nodejs.org/api/os.html) and [`process`](https://nodejs.org/api/process.html) objects. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/js/environment-allure3-light.png, /images/js/environment-allure3-dark.png, /images/js/environment-allure2-light.png, /images/js/environment-allure2-dark.png ```ts import AllureJasmineReporter from "allure-jasmine"; import * as os from "node:os"; jasmine.getEnv().addReporter( new AllureJasmineReporter({ environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }), ); ``` Note that if your launch includes multiple Jasmine runs (see [How it works](/docs/how-it-works/)), Allure Jasmine will only save the environment information from the latest run. --- ## Jasmine configuration > URL: https://allurereport.org/docs/jasmine-configuration/ # Allure Jasmine configuration The [Allure Jasmine](/docs/jasmine/) integration behavior can be affected by some configuration options set in your project's `setup.js` or `setup.ts` file (see [Prepare your project](/docs/jasmine/#1-prepare-your-project)). The configuration options must be defined in the object passed to the `JasmineAllureReporter` constructor. For example: ```ts import AllureJasmineReporter from "allure-jasmine"; import { Status } from "allure-js-commons"; import * as os from "node:os"; jasmine.getEnv().addReporter( new AllureJasmineReporter({ resultsDir: "allure-results", links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: (v) => `https://jira.example.com/browse/${v}`, }, }, categories: [ { name: "foo", messageRegex: "bar", traceRegex: "baz", matchedStatuses: [Status.FAILED, Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, globalLabels: [{ name: "layer", value: "api" }], }), ); ``` Warning: This section describes the configuration options for Allure Jasmine 3.0 and newer. For older versions, you may check this [readme](https://github.com/allure-framework/allure-js/blob/v2.15.1/packages/allure-jasmine/README.md). ## resultsDir Path to the directory where Allure Jasmine will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ## links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/jasmine-reference/#link)), you can specify the following: - `nameTemplate` — a template or a function for generating the link name when it is not provided. - `urlTemplate` — a template or a function for generating the link address when it is not provided. The templates can be strings (with `%s` where the identifier should be placed) or functions (accepting the identifier and returning the URL). For example, with the configuration above, `await allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/`. Allure will display the link with the appropriate icon for the `issue` type. ## categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). ## environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ## globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ name, value }` label objects: ```ts jasmine.getEnv().addReporter( new AllureJasmineReporter({ globalLabels: [ { name: "layer", value: "api" }, { name: "team", value: "backend" }, ], }), ); ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```ts jasmine.getEnv().addReporter( new AllureJasmineReporter({ globalLabels: { layer: "api", team: "backend", }, }), ); ``` --- ## Jasmine reference > URL: https://allurereport.org/docs/jasmine-reference/ # Allure Jasmine reference These are the functions that you can use to integrate your Jasmine tests with Allure. In most cases, Allure Jasmine provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Jasmine will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `allure.description(markdown: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.description("This test attempts to log into the website."); // ... }); }); ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.owner("John Doe"); // ... }); }); ``` **Metadata API:** ```ts describe("Test My Website", function () { it("Test Authentication @allure.label.owner:JohnDoe", async () => { // ... }); }); ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` - `@allure.label.tag:⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.tag("New UI"); await allure.tag("Essentials"); await allure.tag("Authentication"); // ... }); }); ``` **Metadata API:** ```ts describe("Test My Website", function () { it( "Test Authentication" + " @allure.label.tag:NewUI" + " @allure.label.tag:Essentials" + " @allure.label.tag:Authentication", async () => { // ... }, ); }); ``` ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { Severity } from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.severity(Severity.CRITICAL); // ... }); }); ``` **Metadata API:** ```ts describe("Test My Website", function () { it("Test Authentication @allure.label.severity:critical", async () => { // ... }); }); ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` - `@allure.label.⟨NAME⟩:⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. **Runtime API:** ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.label("microservice", "UI"); // ... }); }); ``` **Metadata API:** ```ts describe("Test My Website", function () { it("Test Authentication @allure.label.microservice:UI", async () => { // ... }); }); ``` ### ID - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication @allure.id:123", async () => { // ... }); }); ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string, defaults to “link”), Allure will try to load a corresponding **link template** to process the URL, as defined by the [`links`](/docs/jasmine-configuration/#links) configuration option. If no template is found for the given type or if the link already represents a proper URL, it's left unmodified. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.issue("AUTH-123", "Related issue"); await allure.tms("TMS-456", "Related TMS issue"); await allure.link("JIRA-777", "Related Jira issue", "jira"); await allure.link("https://example.com/", "Project website"); // ... }); }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features**, or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); }); ``` **Metadata API:** ```ts describe("Test My Website", function () { it( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); }); ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the names of **parent suite**, **suite**, or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Runtime API:** ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); }); ``` **Metadata API:** ```ts describe("Test My Website", function () { it( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); }); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test step](/docs/steps/) with the given `name`. There are two ways of defining a step. - **Lambda steps** Write a test step in a function and pass it to `allure.step()`. If the function returns a value, `allure.step()` will return it without modification, and it will not affect the report. If it returns a promise, it will be awaited and resolved. The function can accept either no arguments or a single object of type `StepContext`. This object provides the following methods: - `displayName()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are similar to the test-wide implementation, see [Parametrized tests](#parametrized-tests). - **No-op steps** If you call `allure.logStep()`, Allure will add to the report a no-op step. This allows for a log-style reporting within a test or within a larger step. A no-op step finishes immediately after it started and cannot have any sub-steps, or parameters. The optional second argument indicates the status that will be shown for the step in the report. Allowed values are: `Status.PASSED` (the default), `Status.FAILED`, `Status.BROKEN`, and `Status.SKIPPED`. The optional third argument is an `Error` object. When provided, the data from this object will be displayed in a message inside the step. ```ts import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` Specify a `name` and `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. The `options` argument, if given, must be an object with two optional properties `excluded` and `mode`. - If `excluded` is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. Note, that even when you use the `"masked"` or `"hidden"` mode, it is still possible to extract the value from the `allure_results` directory if you publish it. ```ts import * as allure from "allure-js-commons"; describe("Test My Website", function () { for (const login of ["johndoe", "johndoe@example.com"]) { it(`Test Authentication as ${login}`, async () => { allure.parameter("login", login); allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } }); ``` ## Attachments - `allure.attachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options in an object (as shown for the image attachment below) or just specify the media type and let Allure deduce the appropriate filename extension automatically (as shown for the text attachment below). In either case, the media type can be a value from the `ContentType` enumeration or any string. ```ts import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; describe("Test My Website", function () { it("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); }); ``` --- ## JBehave > URL: https://allurereport.org/docs/jbehave/ # Getting started with Allure JBehave [![Allure JBehave latest version](https://img.shields.io/maven-central/v/io.qameta.allure/allure-jbehave5?style=flat "allure-jbehave5")](https://mvnrepository.com/artifact/io.qameta.allure/allure-jbehave5) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [JBehave](https://jbehave.org/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Ajbehave) to see Allure JBehave in action. ## Setting up To integrate [Allure](https://allurereport.org) into an existing **JBehave** project, you need to: 1. Add Allure dependencies to your project. 1. Activate the Allure JBehave plugin. 1. Designate a location for Allure results storage. Allure Java 3.x requires Java 17 or newer. ### Adding Allure dependencies **Maven:** ```xml 3.0.0 io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-jbehave5 test ``` **Gradle (Kotlin):** ```kts // Define the version of Allure you want to use via the allureVersion property val allureVersion = "3.0.0" // ... dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) // Add necessary Allure dependencies to dependencies section testImplementation("io.qameta.allure:allure-jbehave5") } ``` **Gradle (Groovy):** ```groovy // Define the version of Allure you want to use via the allureVersion property def allureVersion = "3.0.0" dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") // Add necessary Allure dependencies to dependencies section testImplementation "io.qameta.allure:allure-jbehave5" } ``` ### Enabling Allure JBehave plugin In your [Embeddable](https://jbehave.org/reference/stable/developing-stories.html#configuring) class (such as a `JUnitStories`-based class), add the `AllureJbehave5` reporter to the configuration object. ```java import io.qameta.allure.jbehave5.AllureJbehave5; import org.jbehave.core.configuration.Configuration; import org.jbehave.core.configuration.MostUsefulConfiguration; import org.jbehave.core.junit.JUnitStories; import org.jbehave.core.reporters.StoryReporterBuilder; public class Stories extends JUnitStories { @Override public Configuration configuration() { return new MostUsefulConfiguration() .useStoryReporterBuilder(new StoryReporterBuilder() .withReporters(new AllureJbehave5())); } // ... } ``` ### Specifying Allure Results location Allure, by default, saves test results in the project's root directory. However, it is recommended to store your test results in the build output directory. To configure this, create an `allure.properties` file and place it in the test resources directory of your project, which is typically located at `src/test/resources`: **Maven:** ```properties #allure.properties allure.results.directory=target/allure-results ``` **Gradle (Kotlin):** ```properties #allure.properties allure.results.directory=build/allure-results ``` **Gradle (Groovy):** ```properties #allure.properties allure.results.directory=build/allure-results ``` ### Running tests Execute your JBehave tests as you normally would. Below are the commands for Gradle and Maven users: For Gradle: **MacOS/Linux:** ```bash ./gradlew test ``` **Windows:** ```bash gradlew test ``` For Maven: **MacOS/Linux:** ```bash ./mvnw verify ``` **Windows:** ```bash mvnw verify ``` After running the tests, Allure will gather the test execution data and store it in the `allure-results` directory. You can then generate an HTML report from these results using Allure's reporting tools. ## Writing tests The Allure JBehave integration extends the standard reporting features of JBehave by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#adding-metadata). - **Test Organization**: Structure your tests into [clear hierarchies](#organize-tests) for better readability and organization. - **Step Division**: Break down tests into smaller [test steps](#creating-sub-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#support-for-scenario-outlines) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attaching-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#selective-test-run-allure-testops), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. ### Adding Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/jbehave-reference/#metadata) for an exhaustive list of what can be added. ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.description("This test attempts to create a label with specified title"); Allure.label("owner", "John Doe"); Allure.label("severity", "critical"); Allure.link("Website", "https://dev.example.com/"); Allure.issue("AUTH-123", "https://example.com/issues/AUTH-123"); Allure.tms("TMS-456", "https://example.com/tms/TMS-456"); // ... } } ``` ### Organize tests Allure facilitates enhanced navigation in test reports by allowing tests to be organized into hierarchical structures. As explained in the section on [Improving navigation in your test report](/docs/v3/navigation/), Allure JBehave supports this feature. To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.label("parentSuite", "Tests for web interface"); Allure.suite("Tests for essential features"); Allure.label("subSuite", "Tests for labels"); // ... } } ``` ### Creating Sub-Steps Allure JBehave enhances test reports by allowing the creation of sub-steps within a single JBehave step, which can be particularly useful in cases where you have a set of actions that are: - Sufficiently cohesive to be described as one step in a story file. - Intricate enough that they involve several potential failure points. Allure provides three methods for defining sub-steps: 1. **Annotated Sub-Steps**: Defined using `@Step` annotation. 2. **Lambda Sub-Steps**: Utilizing lambda expressions for more concise and inline step definitions. 3. **No-Op Sub-Steps**: Employed as placeholders or for structuring steps without additional functionality. Info: Annotated sub-steps (`@Step`) require AspectJ to be active at test runtime. Add the AspectJ weaver agent to your JVM arguments: **Gradle (Kotlin):** ```kts val agent: Configuration by configurations.creating { isCanBeConsumed = true isCanBeResolved = true } dependencies { agent("org.aspectj:aspectjweaver:1.9.25") } tasks.test { jvmArgs("-javaagent:${agent.singleFile}") } ``` Lambda sub-steps and no-op sub-steps do not require AspectJ. For detailed guidance on implementing each type of sub-step, consult the [test steps reference](/docs/jbehave-reference/#test-steps). **Annotated sub-steps:** ```java import io.qameta.allure.Step; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { step1(); step2(); } @Step("Step 1") public void step1() { subStep1(); subStep2(); } @Step("Sub-step 1") public void subStep1() { // ... } @Step("Sub-step 2") public void subStep2() { // ... } @Step("Step 2") public void step2() { // ... } } ``` **Lambda steps and no-op sub-steps:** ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.step("Step 1", (step) -> { // ... Allure.step("Sub-step 1"); // ... Allure.step("Sub-step 2"); }); Allure.step("Step 2", (step) -> { // ... }); } } ``` ### Support for Scenario Outlines Allure JBehave provides complete support for [Scenario Outlines](https://jbehave.org/reference/stable/parametrised-scenarios.html), a feature of JBehave that allows for parametrized tests. No special setup is needed to take advantage of this capability within Allure. ``` Scenario: Registration When I go to the registration form And I enter my details: , , , Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; import org.jbehave.core.annotations.When; public class JBehaveSteps { @When("I go to the registration form") public void goToRegistrationForm() { // ... } @When("I enter my details: $login, $password, $name, $birthday") public void enterMyDetails() { // ... } @Then("the profile should be created") public void profileShouldBeCreated() { // ... } } ``` Furthermore, Allure's Runtime API can be utilized to include extra parameters in the test report, offering enhanced documentation capabilities and greater detail for each test scenario: ``` Scenario: Registration When I go to the registration form And I enter my details Then the profile should be created ``` ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; import org.jbehave.core.annotations.When; public class JBehaveSteps { @When("I go to the registration form") public void goToRegistrationForm() { // ... } @When("I enter my details") public void enterMyDetails() { Allure.parameter("login", "johndoe"); Allure.parameter("password", "qwerty"); Allure.parameter("name", "John Doe"); Allure.parameter("birthday", "1970-01-01"); // ... } @Then("the profile should be created") public void profileShouldBeCreated() { // ... } } ``` ### Attaching screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. Allure JBehave offers multiple methods for creating attachments, whether from pre-existing files or from content generated on-the-fly. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure JBehave reference](/docs/jbehave-reference/#attachments). ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() throws IOException { // ... Allure.attachment("data.txt", "This is the file content."); Allure.attachment("img.png", Files.newInputStream(Paths.get("/path/img.png"))); } } ``` ### Selective test run (Allure TestOps) Danger: Test plan is currently not supported by the Allure JBehave integration. ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS and Java versions. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/java/environment-allure3-light.png, /images/java/environment-allure3-dark.png, /images/java/environment-allure2-light.png, /images/java/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](/docs/jbehave-reference/#parametrized-tests). --- ## JBehave configuration > URL: https://allurereport.org/docs/jbehave-configuration/ # Allure JBehave configuration To set the configuration options, create an `allure.properties` file in your project's `src/test/resources` directory. ## allure.label.⟨NAME⟩ ```properties allure.label.epic = Authentication allure.label.severity = blocker ``` Assign an arbitrary [label](/docs/jbehave-reference/#label) to all the tests in the current test run. ## allure.link.⟨TYPE⟩.pattern ```properties allure.link.issue.pattern = https://jira.example.org/browse/{} ``` Define a **link pattern** — a template that can be used to construct full URLs from short identifiers when using the Runtime API (e.g. `Allure.issue()` or `Allure.tms()`), see the [reference](/docs/jbehave-reference/#link). The value must be a string pattern containing `{}` at the position where the identifier should be placed. ## allure.results.directory ```properties allure.results.directory = build/allure-results ``` Path to the directory where Allure JBehave will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. --- ## JBehave reference > URL: https://allurereport.org/docs/jbehave-reference/ # Allure JBehave reference These are the functions that you can use to integrate your JBehave tests with Allure. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `Allure.getLifecycle().updateTest(Consumer update)` Set the test's [title](/docs/v3/readability/#title). Info: In Allure Java 2.x, the `AllureLifecycle` method for updating the test result is called `updateTestCase()`. ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.getLifecycle().updateTest(result -> result.setName("Some modified scenario name")); // ... } } ``` ### Description - `Allure.description(String description)` - `Allure.descriptionHtml(String description)` Set the test's [description](/docs/v3/readability/#description). You can either begin a story file with a description or call the methods for updating the values dynamically. Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. **Story file:** ```gherkin This test attempts to create a label with specified title Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.description("This test attempts to create a label with specified title"); // ... } } ``` ### Owner Set the test's [owner](/docs/v3/readability/#owner). ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.label("owner", "John Doe"); // ... } } ``` ### Tag Set the test's [tags](/docs/v3/readability/#tags). ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.label("tag", "ui"); Allure.label("tag", "labels"); // ... } } ``` ### Severity Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.label("severity", "critical"); // ... } } ``` ### Label - `Allure.label(String name, String value)` Allure labels are key-value pairs used to add additional meta-information to your tests. They can be considered as a way to categorize, organize, and search for test cases within your Allure reports. Labels can be used to specify things like the test case ID, the severity of the test, the feature being tested, or the story that relates to the test. This meta-information helps to create more detailed and structured reports. ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.label("layer", "web"); Allure.label("owner", "eroshenkoam"); Allure.label("page", "/{org}/{repo}/labels"); Allure.label("severity", "critical"); Allure.label("jira", "AE-2"); // ... } } ``` ### Allure ID (Allure TestOps) Set the test's [ID](/docs/v3/readability/#id). ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.label("ALLURE_ID", "123"); // ... } } ``` ### Link - `Allure.link(String url)` - `Allure.link(String name, String url)` - `Allure.link(String name, String type, String url)` - `Allure.issue(String name, String url)` - `Allure.tms(String name, String url)` Add a [link](/docs/v3/readability/#links) related to the test. The `name` will be used as the link's text. If it is omitted, the full URL will be used instead. The `type` affects the icon that is displayed next to the link in the test report. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.link("Website", "https://dev.example.com/"); Allure.issue("AUTH-123", "https://example.com/issues/AUTH-123"); Allure.tms("TMS-456", "https://example.com/tms/TMS-456"); // ... } } ``` ## Behavior-based hierarchy Being a BDD-oriented framework, JBehave organizes tests according to the story files. In the test report, Allure JBehave displays this hierarchy on the **Behaviors** tab. The names of the groups correspond to the names of the files. No special setup is needed. ## Suite-based hierarchy - `Allure.suite(String value)` Assign the name of suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.label("parentSuite", "Tests for web interface"); Allure.suite("Tests for essential features"); Allure.label("subSuite", "Tests for labels"); // ... } } ``` ## Test sub-steps - `Allure.step(String name)` - `Allure.step(String name, Status status)` - `Allure.step(String name, ThrowableRunnableVoid runnable)` - `Allure.step(String name, ThrowableRunnable runnable)` - `Allure.step(ThrowableContextRunnableVoid runnable)` - `Allure.step(String name, ThrowableContextRunnableVoid runnable)` - `Allure.step(String name, ThrowableContextRunnable runnable)` - `Allure.step(ThrowableContextRunnable runnable)` Define a [test sub-step](/docs/steps/) with the given `name`. There are three ways of defining a sub-step. - “An annotated sub-step”: define a method containing a test step and decorate it with `@Step()`. - “A lambda sub-step”: write a test sub-step in a lambda function and pass it to `Allure.step()`. - “A no-op sub-step”: just pass a `name` and an optional `status` to `Allure.step()` to immediately add a corresponding entry to the results as a sub-step of the current step. When you are implementing a function for a lambda sub-step, it can accept either no arguments or a single argument of class `StepContext`. This object provides the following methods: - `name()` — override the sub-step name during its execution. - `parameter()` — indicate arbitrary parameters used for the sub-step. The available signatures of this method are the same as for the test-wide implementation, see [Parametrized tests](#parametrized-tests). **Annotated sub-steps:** ```java import io.qameta.allure.Step; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { step1(); step2(); } @Step("Step 1") public void step1() { subStep1(); subStep2(); } @Step("Sub-step 1") public void subStep1() { // ... } @Step("Sub-step 2") public void subStep2() { // ... } @Step("Step 2") public void step2() { // ... } } ``` **Lambda steps and no-op sub-steps:** ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() { Allure.step("Step 1", (step) -> { // ... Allure.step("Sub-step 1"); // ... Allure.step("Sub-step 2"); }); Allure.step("Step 2", (step) -> { // ... }); } } ``` ## Parametrized tests Allure JBehave support two ways of implementing the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern: - [parametrized scenarios](#parametrized-scenarios), - [runtime parameters](#runtime-parameters). ### Parametrized scenarios In the JBehave story files, a [parametrized scenario](https://jbehave.org/reference/stable/parametrised-scenarios.html) (sometimes called a scenario outline or a scenario template) implements the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. A parametrized scenario must contain an `Examples` table, from which JBehave loads sets of parameters, one row after another. Each set of parameters is being placed into the step declarations according to the placeholders, thus generating a new iteration of the scenario based on the row. The data can then be captured using the [parameter injection](https://jbehave.org/reference/stable/parameter-injection.html) syntax and passed as separate arguments to the Java code. Allure JBehave automatically recognizes this pattern. No additional configuration is required. The example below shows a story file and a Java implementation file of a test. In this example, the four parameters for the “I enter my details...” step will be displayed in both instances of the scenario in the test report. ``` Scenario: Registration When I go to the registration form And I enter my details: , , , Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; import org.jbehave.core.annotations.When; public class JBehaveSteps { @When("I go to the registration form") public void goToRegistrationForm() { // ... } @When("I enter my details: $login, $password, $name, $birthday") public void enterMyDetails() { // ... } @Then("the profile should be created") public void profileShouldBeCreated() { // ... } } ``` ### Runtime parameters - `Allure.parameter(String name, T value)` - `Allure.parameter(String name, T value, Boolean excluded)` - `Allure.parameter(String name, T value, Parameter.Mode mode)` - `Allure.parameter(String name, T value, Boolean excluded, Parameter.Mode mode)` Specify a `name` and `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. Warning: Parameters added via `Allure.parameter()` are part of the test's identity, together with the parameters provided by the test framework. If a parameter's value changes between runs (for example, a timestamp or a random ID), the test will be treated as a new test on every run for the purposes of [History and retries](/docs/history-and-retries/). To avoid this, mark such parameters as excluded, see below. In Allure Java 2.x, parameters added via `Allure.parameter()` did not affect the test's identity. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous ones in the history. In Allure Java 2.x, this argument is only used by Allure TestOps. The `mode` argument affects how the parameter will be displayed in the report. Available options are defined in the `Parameter.Mode` enumeration: - `Parameter.Mode.DEFAULT` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `Parameter.Mode.MASKED` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `Parameter.Mode.HIDDEN` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. ``` Scenario: Registration When I go to the registration form And I enter my details Then the profile should be created ``` ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; import org.jbehave.core.annotations.When; public class JBehaveSteps { @When("I go to the registration form") public void goToRegistrationForm() { // ... } @When("I enter my details") public void enterMyDetails() { Allure.parameter("login", "johndoe"); Allure.parameter("password", "qwerty"); Allure.parameter("name", "John Doe"); Allure.parameter("birthday", "1970-01-01"); // ... } @Then("the profile should be created") public void profileShouldBeCreated() { // ... } } ``` ## Attachments - `Allure.attachment(String name, String content)` - `Allure.attachment(String name, String type, String content)` - `Allure.attachment(String name, String type, String content, AttachmentOptions options)` - `Allure.attachment(String name, InputStream content)` - `Allure.attachment(String name, String type, InputStream content, AttachmentOptions options)` Add `content` as an attachment to the test result under the given `name` (defaults to a unique pseudo-random string). Info: In Allure Java 2.x, the same functionality is provided by the `Allure.addAttachment()` methods, which accept the file extension as a string. They were removed in Allure Java 3.0. Tip: You can use data produced by any function, not necessarily read from an actual file. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension via `AttachmentOptions.withFileExtension(".txt")` as the `options` argument. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. ```java import io.qameta.allure.Allure; import org.jbehave.core.annotations.Then; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; public class JBehaveSteps { @When("I open labels page") public void openLabelsPage() throws IOException { // ... Allure.attachment("data.txt", "This is the file content."); Allure.attachment("img.png", Files.newInputStream(Paths.get("/path/img.png"))); } } ``` Info: Additionally, Allure provides a way to spawn threads that will add attachments to the test results asynchronously without blocking the test runner. While not recommended for the majority of cases, this approach can improve tests that otherwise have to wait for processing very large files, such as videos larger than 1 GB. To do so, pass a `CompletionStage` that produces the attachment's content to `Allure.attachmentAsync(String name, String type, CompletionStage body)`. In Allure Java 2.x, use `Allure.addByteAttachmentAsync()` or `Allure.addStreamAttachmentAsync()` instead. --- ## Jest > URL: https://allurereport.org/docs/jest/ # Getting started with Allure Jest [![Allure Jest npm latest version](https://img.shields.io/npm/v/allure-jest?style=flat "Allure Jest npm latest version")](https://www.npmjs.com/package/allure-jest) Generate beautiful HTML reports with [Allure Report](https://allurereport.org/docs/) and your [Jest](https://jestjs.io/) tests. Both the Node.js and jsdom environments are supported. The integration works with Jest 24.8 or higher. However, it relies on the [Circus](https://www.npmjs.com/package/jest-circus) runner (which is the default in Jest 27 or higher). If your project uses the Jasmine runner, and you can't switch to Circus, please use the [Allure Jasmine](/docs/jasmine/) integration instead. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Ajest) to see Allure Jest in action. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure Jest is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure Jest integration. **npm:** ```bash npm install --save-dev allure-jest ``` **yarn:** ```bash yarn add --dev allure-jest allure-js-commons ``` **pnpm:** ```bash pnpm add -D allure-jest ``` 1. Allure Jest supports two Jest environments: `jest-environment-node` and `jest-environment-jsdom`. Make sure to install the one you plan to use. For example: **npm:** ```bash npm install --save-dev jest-environment-node ``` **yarn:** ```bash yarn add --dev jest-environment-node ``` **pnpm:** ```bash pnpm add -D jest-environment-node ``` Warning: The version of the `jest-environment-node` or `jest-environment-jsdom` package **must** match the version of the `jest` package, otherwise the integration may not work. If you're a Yarn PnP user, don't skip this step even if you use `jest-environment-node` (which is installed automatically with Jest). Otherwise, you will observe the following error: > allure-jest tried to access jest-environment-node (a peer dependency) but it isn't provided by your application; this makes the require call ambiguous and unsound. 1. If the Jest version is lower than 27, install Circus and configure Jest to use it as the test runner. **npm:** ```bash npm install --save-dev jest-circus ``` **yarn:** ```bash yarn add --dev jest-circus ``` **pnpm:** ```bash pnpm add -D jest-circus ``` ```js const config = { testRunner: "jest-circus/runner", }; ``` Warning: The version of the `jest-circus` package **must** match the version of the `jest` package, otherwise the integration may not work. 1. In the [Jest configuration file](https://jestjs.io/docs/configuration), e.g., `jest.config.js`, set the appropriate test environment. **For client-side tests:** ```js const config = { testEnvironment: "allure-jest/jsdom", }; ``` **For server-side tests:** ```js const config = { testEnvironment: "allure-jest/node", }; ``` ### 2. Run tests Run your Jest tests the same way as you would run them usually. For example: **npm:** ```bash npm test ``` **yarn:** ```bash yarn test ``` **pnpm:** ```bash pnpm test ``` This will save necessary data into `allure-results` or other directory, according to the [configuration](/docs/jest-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Jest integration extends the standard reporting features of Jest by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure Jest provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Jest will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/jest-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.owner("John Doe"); await allure.tags("Web interface", "Authentication"); await allure.severity("critical"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.owner:JohnDoe" + " @allure.label.tag:WebInterface" + " @allure.label.tag:Authentication" + " @allure.label.severity:critical", async () => { // ... }, ); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); ``` ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), you can use the `step()` function, see the [reference](/docs/jest-reference/#test-steps). ```js import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; it("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ### Describe parametrized tests A typical way to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern in Jest is via the [`test.each()`](https://jestjs.io/docs/api#testeachtablename-fn-timeout) function. To display a parameter value in the test report, pass it to the [`parameter()`](/docs/jest-reference/#parametrized-tests) function. ```js import * as allure from "allure-js-commons"; test.each([["johndoe"], ["johndoe@example.com"]])("Test Authentication as %s", async (login) => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); ``` ### Set labels globally Any [labels](/docs/jest-reference/#label), including custom ones, can be set via the environment variables in your operating system. Here's an example (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface npm test ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" npm test ``` ### Attach screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure Jest reference](/docs/jest-reference/#attachments). ```js import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; it("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Jest will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json` (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json npm test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" npm test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, edit the [`environmentInfo`](/docs/vitest-configuration/#environmentinfo) object in the configuration. For example, it is a good idea to use this to remember the OS version and Node.js version retrieved from the [`os`](https://nodejs.org/api/os.html) and [`process`](https://nodejs.org/api/process.html) objects. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/js/environment-allure3-light.png, /images/js/environment-allure3-dark.png, /images/js/environment-allure2-light.png, /images/js/environment-allure2-dark.png ```js import { Status } from "allure-js-commons"; import os from "node:os"; export default { testEnvironment: "allure-jest/node", testEnvironmentOptions: { environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }, }; ``` Note that if your launch includes multiple Jest runs (see [How it works](/docs/how-it-works/)), Allure Jest will only save the environment information from the latest run. --- ## Jest configuration > URL: https://allurereport.org/docs/jest-configuration/ # Allure Jest configuration The [Allure Jest](/docs/jest/) integration behavior can be affected by some configuration options set in your project's `jest.config.js` file. To set the options, define an object under the `testEnvironmentOptions` key in `config`. For example: ```js import { Status } from "allure-js-commons"; import os from "node:os"; export default { testEnvironment: "allure-jest/node", testEnvironmentOptions: { resultsDir: "allure-results", links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: (v) => `https://jira.example.com/browse/${v}`, }, }, categories: [ { name: "foo", messageRegex: "bar", traceRegex: "baz", matchedStatuses: [Status.FAILED, Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, globalLabels: [{ name: "layer", value: "api" }], }, }; ``` Warning: This section describes the configuration options for Allure Jest 3.0 and newer. For older versions, you may check this [readme](https://github.com/allure-framework/allure-js/blob/v2.15.1/packages/allure-jest/README.md). ## resultsDir Path to the directory where Allure Jest will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ## links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/jest-reference/#link)), you can specify the following: - `nameTemplate` — a template or a function for generating the link name when it is not provided. - `urlTemplate` — a template or a function for generating the link address when it is not provided. The templates can be strings (with `%s` where the identifier should be placed) or functions (accepting the identifier and returning the URL). For example, with the configuration above, `await allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/`. Allure will display the link with the appropriate icon for the `issue` type. ## categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). ## environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ## globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ name, value }` label objects: ```js export default { testEnvironment: "allure-jest/node", testEnvironmentOptions: { globalLabels: [ { name: "layer", value: "api" }, { name: "team", value: "backend" }, ], }, }; ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```js export default { testEnvironment: "allure-jest/node", testEnvironmentOptions: { globalLabels: { layer: "api", team: "backend", }, }, }; ``` --- ## Jest reference > URL: https://allurereport.org/docs/jest-reference/ # Allure Jest reference These are the functions that you can use to integrate your Jest tests with Allure. In most cases, Allure Jest provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Jest will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `allure.displayName(name: string): PromiseLike` Set the test's [title](/docs/v3/readability/#title). ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.displayName("Test Authentication"); // ... }); ``` ### Description - `allure.description(markdown: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.description("This test attempts to log into the website."); // ... }); ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.owner("John Doe"); // ... }); ``` **Metadata API:** ```js it("Test Authentication @allure.label.owner:JohnDoe", async () => { // ... }); ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` - `@allure.label.tag:⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.tag("New UI"); await allure.tags("Essentials", "Authentication"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.tag:NewUI" + " @allure.label.tag:Essentials" + " @allure.label.tag:Authentication", async () => { // ... }, ); ``` ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Runtime API:** ```js import * as allure from "allure-js-commons"; import { Severity } from "allure-js-commons"; it("Test Authentication", async () => { await allure.severity(Severity.CRITICAL); // ... }); ``` **Metadata API:** ```js it("Test Authentication @allure.label.severity:critical", async () => { // ... }); ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` - `@allure.label.⟨NAME⟩:⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.label("microservice", "UI"); // ... }); ``` **Metadata API:** ```js it("Test Authentication @allure.label.microservice:UI", async () => { // ... }); ``` ### ID - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). ```js import * as allure from "allure-js-commons"; it("Test Authentication @allure.id:123", async () => { // ... }); ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string, defaults to “link”), Allure will try to load a corresponding **link template** to process the URL, as defined by the [`links`](/docs/jest-configuration/#links) configuration option. If no template is found for the given type or if the link already represents a proper URL, it's left unmodified. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.issue("AUTH-123", "Related issue"); await allure.tms("TMS-456", "Related TMS issue"); await allure.link("JIRA-777", "Related Jira issue", "jira"); await allure.link("https://example.com/", "Project website"); // ... }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features**, or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the names of **parent suite**, **suite**, or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test step](/docs/steps/) with the given `name`. There are two ways of defining a step. - **Lambda steps** Write a test step in a function and pass it to `allure.step()`. If the function returns a value, `allure.step()` will return it without modification, and it will not affect the report. If it returns a promise, it will be awaited and resolved. The function can accept either no arguments or a single object of type `StepContext`. This object provides the following methods: - `displayName()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are similar to the test-wide implementation, see [Parametrized tests](#parametrized-tests). - **No-op steps** If you call `allure.logStep()`, Allure will add to the report a no-op step. This allows for a log-style reporting within a test or within a larger step. A no-op step finishes immediately after it started and cannot have any sub-steps, or parameters. The optional second argument indicates the status that will be shown for the step in the report. Allowed values are: `Status.PASSED` (the default), `Status.FAILED`, `Status.BROKEN`, and `Status.SKIPPED`. The optional third argument is an `Error` object. When provided, the data from this object will be displayed in a message inside the step. ```js import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; it("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` Specify a `name` and `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. The `options` argument, if given, must be an object with two optional properties `excluded` and `mode`. - If `excluded` is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. Note, that even when you use the `"masked"` or `"hidden"` mode, it is still possible to extract the value from the `allure_results` directory if you publish it. ```js import * as allure from "allure-js-commons"; test.each([["johndoe"], ["johndoe@example.com"]])("Test Authentication as %s", async (login) => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); ``` ## Attachments - `allure.attachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options in an object (as shown for the image attachment below) or just specify the media type and let Allure deduce the appropriate filename extension automatically (as shown for the text attachment below). In either case, the media type can be a value from the `ContentType` enumeration or any string. ```js import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; it("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` --- ## JUnit Jupiter > URL: https://allurereport.org/docs/junit-jupiter/ # Getting started with Allure JUnit Jupiter Info: JUnit 5 is now JUnit Jupiter This page was previously called **JUnit 5**. JUnit 6 kept the same Jupiter programming model, so the Allure integration is now named after it: **Allure JUnit Jupiter** works with both JUnit 6 and JUnit 5. - The artifact is `io.qameta.allure:allure-jupiter`. It replaced `allure-junit5` in Allure Java 2.35.1. - Starting with Allure Java 3.0, the old `allure-junit5` and `allure-junit5-assert` coordinates are **no longer published**. Switch to `allure-jupiter` and `allure-jupiter-assert` before upgrading. - Allure Java 3.0 requires Java 17 or newer. See [Migrating to Allure Java 3](/docs/guides/allure-java-3-migration/) for all changes. [![Allure JUnit Jupiter latest version](https://img.shields.io/maven-central/v/io.qameta.allure/allure-jupiter?style=flat "allure-jupiter")](https://mvnrepository.com/artifact/io.qameta.allure/allure-jupiter) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/) and your [JUnit](https://junit.org/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Ajunit5) to see Allure JUnit Jupiter in action. ## Setting up To integrate [Allure](https://allurereport.org) into an existing **JUnit Jupiter** project (JUnit 6 or JUnit 5), you need to: 1. Add Allure dependencies to your project. 1. Set up AspectJ for `@Step` and `@Attachment` annotations support. 1. Designate a location for Allure results storage. Allure Java 3.x and JUnit 6 both require Java 17 or newer. ### Add Allure dependencies **Maven:** ```xml 3.0.0 io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-jupiter test ``` **Gradle (Kotlin):** ```kts // Define the version of Allure you want to use via the allureVersion property val allureVersion = "3.0.0" // ... dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) // Add necessary Allure dependencies to dependencies section testImplementation("io.qameta.allure:allure-jupiter") } ``` **Gradle (Groovy):** ```groovy // Define the version of Allure you want to use via the allureVersion property def allureVersion = "3.0.0" dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") // Add necessary Allure dependencies to dependencies section testImplementation "io.qameta.allure:allure-jupiter" } ``` ### Configure AspectJ Allure leverages AspectJ for the functionality of `@Step` and `@Attachment` annotations. Additionally, some framework integrations (such as [allure-assertj](https://mvnrepository.com/artifact/io.qameta.allure/allure-assertj)) rely on AspectJ integration to function correctly. **Maven:** ```xml 1.9.25 org.apache.maven.plugins maven-surefire-plugin 3.2.3 -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar" org.aspectj aspectjweaver ${aspectj.version} ``` **Gradle (Kotlin):** ```kts // Define the version of AspectJ val aspectJVersion = "1.9.25" // Define configuration for AspectJ agent val agent: Configuration by configurations.creating { isCanBeConsumed = true isCanBeResolved = true } dependencies { // Add aspectjweaver dependency agent("org.aspectj:aspectjweaver:${aspectJVersion}") } // Configure javaagent for test execution tasks.test { jvmArgs = listOf( "-javaagent:${agent.singleFile}" ) } ``` **Gradle (Groovy):** ```groovy // Define the version of AspectJ def aspectJVersion = '1.9.25' // Define configuration for AspectJ agent configurations { agent { canBeResolved = true canBeConsumed = true } } dependencies { // Add aspectjweaver dependency agent "org.aspectj:aspectjweaver:$aspectJVersion" } // Configure javaagent for test execution test { jvmArgs = [ "-javaagent:${configurations.agent.singleFile}" ] } ``` ### Specifying Allure Results location Allure, by default, saves test results in the project's root directory. However, it is recommended to store your test results in the build output directory. To configure this, create an `allure.properties` file and place it in the test resources directory of your project, which is typically located at `src/test/resources`: **Maven:** ```properties allure.results.directory=target/allure-results ``` **Gradle (Kotlin):** ```properties allure.results.directory=build/allure-results ``` **Gradle (Groovy):** ```properties allure.results.directory=build/allure-results ``` ### Run tests Run your JUnit tests the same way as you would run them usually. For example: For Gradle: **MacOS/Linux:** ```bash ./gradlew test ``` **Windows:** ```bash gradlew test ``` For Maven: **MacOS/Linux:** ```bash ./mvnw verify ``` **Windows:** ```bash mvnw verify ``` After running the tests, Allure will gather the test execution data and store it in the `allure-results` directory. You can then generate an HTML report from these results using Allure's reporting tools. ### Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Optional: report JUnit assertions as steps Add the `allure-jupiter-assert` dependency if you want each individual `Assertions.*` call to appear as a nested step in the Allure report. This makes assertion failures easier to pinpoint when multiple assertions run in one test. **Maven:** ```xml io.qameta.allure allure-jupiter-assert test ``` **Gradle (Kotlin):** ```kts testImplementation("io.qameta.allure:allure-jupiter-assert") ``` **Gradle (Groovy):** ```groovy testImplementation "io.qameta.allure:allure-jupiter-assert" ``` AspectJ must already be configured (see [Configure AspectJ](#configure-aspectj)) for `allure-jupiter-assert` to work. ## Writing tests The Allure JUnit Jupiter integration not only collects the data provided by JUnit's standard features, but also provides additional features for writing even better tests. This section lists the most notable ways to improve your tests, using both JUnit's and Allure JUnit Jupiter's features. With Allure JUnit Jupiter, you can: - provide [description, links and other metadata](#specify-description-links-and-other-metadata), - [organize tests](#organize-tests) into hierarchies, - divide the test into smaller, easier-to-read [test steps](#divide-a-test-into-steps), - describe parameters used when running [parametrized tests](#describe-parametrized-tests), - make the test save [screenshots and other files](#attach-screenshots-and-other-files) during execution, - [select which tests to run](#select-tests-via-a-test-plan-file) via a test plan file, - provide arbitrary [environment information](#environment-information) for the whole test report. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/junit-jupiter-reference/#metadata) for more details. For each of the metadata fields, there are two ways to assign it: via an annotation before a test method or via a method call inside a test method's body. The second way is called “dynamic”, because it allows you to construct strings and other values at runtime before passing to the methods. Note, however, that it is highly recommended to assign all metadata as early as possible. Otherwise, there is a risk of the test failing before having all metadata set, which is bad for the test report's readability. **Annotations API:** ```java import io.qameta.allure.Description; import io.qameta.allure.Issue; import io.qameta.allure.Link; import io.qameta.allure.Owner; import io.qameta.allure.Severity; import io.qameta.allure.TmsLink; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; import static io.qameta.allure.SeverityLevel.*; class TestMyWebsite { @Test @DisplayName("Test Authentication") @Description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") @Severity(CRITICAL) @Owner("John Doe") @Link(name = "Website", url = "https://dev.example.com/") @Issue("AUTH-123") @TmsLink("TMS-456") void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.getLifecycle().updateTest(result -> result.setName("Test Authentication")); Allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication."); Allure.label("severity", "critical"); Allure.label("owner", "John Doe"); Allure.link("Website", "https://dev.example.com/"); Allure.issue("AUTH-123", "https://example.com/issues/AUTH-123"); Allure.tms("TMS-456", "https://example.com/tms/TMS-456"); // ... } } ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure JUnit Jupiter provides functions to assign the relevant fields to tests either by adding annotations or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Annotations API:** ```java import io.qameta.allure.Epic; import io.qameta.allure.Feature; import io.qameta.allure.Story; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @Epic("Web interface") @Feature("Essential features") @Story("Authentication") void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.epic("Web interface"); Allure.feature("Essential features"); Allure.story("Authentication"); // ... } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.label("parentSuite", "Tests for web interface"); Allure.suite("Tests for essential features"); Allure.label("subSuite", "Tests for authentication"); // ... } } ``` A test's location in the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy) is defined by the fully qualified names of the classes they are declared in, with common prefixes shown as parent packages. ### Divide a test into steps Allure JUnit Jupiter provides three ways of [creating steps and sub-steps](/docs/steps/): “annotated steps”, “lambda steps” and “no-op steps”, see the [reference](/docs/junit-jupiter-reference/#test-steps). **Annotated steps:** ```java import io.qameta.allure.Step; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { step1(); step2(); } @Step("Step 1") void step1() { subStep1(); subStep2(); } @Step("Sub-step 1") void subStep1() { // ... } @Step("Sub-step 2") void subStep2() { // ... } @Step("Step 2") void step2() { // ... } } ``` **Lambda steps and no-op steps:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.step("Step 1", step -> { // ... Allure.step("Sub-step 1"); // ... Allure.step("Sub-step 2"); }); Allure.step("Step 2", step -> { // ... }); } } ``` ### Describe parametrized tests When using the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, use Allure JUnit Jupiter's `@Param` or `parameter()` to add the parameters to the report, see the [reference](/docs/junit-jupiter-reference/#parametrized-tests). **Annotations API:** ```java import org.junit.jupiter.params.ParameterizedTest; import org.junit.jupiter.params.provider.ValueSource; class TestMyWebsite { @ParameterizedTest(name = "{displayName} ({argumentsWithNames})") @ValueSource(strings = {"johndoe", "johndoe@example.com"}) void testAuthentication(String login) { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthenticationWithUsername() { Allure.parameter("login", "johndoe"); // ... } @Test void testAuthenticationWithEmail() { Allure.parameter("login", "johndoe@example.com"); // ... } } ``` Tip: Allure reads the arguments of `@ParameterizedTest` methods through a JUnit extension that is registered automatically. For it to be loaded, enable extension autodetection in `src/test/resources/junit-platform.properties`: ```properties junit.jupiter.extensions.autodetection.enabled=true ``` Without this property, the arguments are still visible in the test name, but they are not reported as parameters. ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure JUnit Jupiter provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/junit-jupiter-reference/#attachments). ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; import java.io.IOException; import java.io.InputStream; import java.nio.file.Files; import java.nio.file.Paths; class TestMyWebsite { @Test void testAuthentication() throws IOException { // ... Allure.attachment("data.txt", "This is the file content."); try (InputStream is = Files.newInputStream(Paths.get("/path/img.png"))) { Allure.attachment("image.png", is); } } } ``` Tip: Consult our guides for integrating screenshots in JUnit 5 with Allure Report: - [Integrating screenshots in JUnit 5 and Selenide](/docs/guides/junit5-selenide-screenshots/) - [Integrating screenshots in JUnit 5 and Selenium](/docs/guides/junit5-selenium-screenshots/) ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, JUnit will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json ./gradlew test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" gradlew test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS and Java versions. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/java/environment-allure3-light.png, /images/java/environment-allure3-dark.png, /images/java/environment-allure2-light.png, /images/java/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](/docs/junit-jupiter-reference/#parametrized-tests). --- ## JUnit Jupiter configuration > URL: https://allurereport.org/docs/junit-jupiter-configuration/ # Allure JUnit Jupiter configuration Info: JUnit 5 is now JUnit Jupiter This page was previously called **JUnit 5 configuration**. The same options apply to JUnit 6 and JUnit 5. See [Getting started](/docs/junit-jupiter/) for details about the rename. This page describes the configuration options that affect the behavior of [Allure JUnit Jupiter](/docs/junit-jupiter/). To set the configuration options, create a `allure.properties` file in your project's `src/test/resources` directory. Allure JUnit Jupiter loads the configuration into the JVM system properties for the test runtime. ## allure.label.⟨NAME⟩ ```properties allure.label.epic = Authentication allure.label.severity = blocker ``` Assign an arbitrary [label](/docs/junit-jupiter-reference/#label) to all the tests in the current test run. ## allure.link.⟨TYPE⟩.pattern ```properties allure.link.issue.pattern = https://jira.example.org/browse/{} ``` Define a **link pattern** — a template that can be used to construct full URLs from short identifiers, see the [reference](/docs/junit-jupiter-reference/#link). The value must be a string pattern containing `{}` at the position where the identifier should be placed. The pattern will be applied when processing any link of the given `TYPE`. You can use the option multiple times, thus defining multiple link patterns for different link types. ## allure.results.directory ```properties allure.results.directory = build/allure-results ``` Path to the directory where Allure JUnit Jupiter will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. --- ## JUnit Jupiter reference > URL: https://allurereport.org/docs/junit-jupiter-reference/ # Allure JUnit Jupiter reference Info: JUnit 5 is now JUnit Jupiter This page was previously called **JUnit 5 reference**. The same APIs apply to JUnit 6 and JUnit 5. See [Getting started](/docs/junit-jupiter/) for details about the rename. These are the functions that you can use to integrate your JUnit tests with Allure. Allure JUnit Jupiter provides more than one way to use some features. In most cases, one way involves using an annotation on the test method, while the other involves a method call inside the test method body. The latter style is called “dynamic” and allows to construct values dynamically. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `@DisplayName(String value)` - `Allure.getLifecycle().updateTest(Consumer update)` Set the test's [title](/docs/v3/readability/#title). Unlike most of the functions described here, the `@DisplayName` annotation is a part of JUnit Jupiter itself. If you need to construct a test's title dynamically, write a lambda function that does that and pass it to the `updateTest()` method of the `AllureLifecycle` object, see the example below. Info: In Allure Java 2.x, the `AllureLifecycle` method for updating the test result is called `updateTestCase()`. **Annotations API:** ```java import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @DisplayName("Test Authentication") void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.getLifecycle().updateTest(result -> { result.setName("Test Authentication"); }); // ... } } ``` ### Description - `@Description(String value="", boolean useJavaDoc=false)` - `Allure.description(String description)` - `Allure.descriptionHtml(String description)` Set the test's [description](/docs/v3/readability/#description). Use the `@Description()` annotation to set a description statically or use the `description()` method to set it dynamically in runtime. **Annotations API:** ```java import io.qameta.allure.Description; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @Description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication."); // ... } } ``` Alternatively, you can let Allure JUnit Jupiter parse the description from the test method's JavaDoc comment. To do so, first add a special annotation processor to your project's Maven or Gradle configuration. ```kotlin // In the project's `build.gradle.kts` val allureVersion = "3.0.0" dependencies { // ... testAnnotationProcessor("io.qameta.allure:allure-descriptions-javadoc:$allureVersion") } ``` Then, use the Annotations API with the `useJavaDoc=true` argument on the test methods for which you want to parse the description from JavaDoc. Note that on a large project, processing the annotations may significantly increase both the compilation time and the file size of the test executable files. Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ```java import io.qameta.allure.Description; import org.junit.jupiter.api.Test; class TestMyWebsite { /** * This test attempts to log into the website using a login and a password. Fails if any error happens. *

* Note that this test does not test 2-Factor Authentication. */ @Test @Description(useJavaDoc = true) void testAuthentication() { // ... } } ``` ### Owner - `@Owner(String value="")` Set the test's [owner](/docs/v3/readability/#owner). **Annotations API:** ```java import io.qameta.allure.Owner; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @Owner("John Doe") void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.label("owner", "John Doe"); // ... } } ``` ### Lead - `@Lead(String value="")` Set the test's lead — the person responsible for the test's maintenance. The lead is displayed alongside the owner in the report. **Annotations API:** ```java import io.qameta.allure.Lead; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @Lead("Jane Doe") void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.label("lead", "Jane Doe"); // ... } } ``` ### Tag - `@Tag(String value)` - `@Tags(Tag[] value)` Set the test's [tags](/docs/v3/readability/#tags). **Annotations API:** ```java import org.junit.jupiter.api.Tag; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @Tag("NewUI") @Tag("Essentials") @Tag("Authentication") void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.label("tag", "NewUI"); Allure.label("tag", "Essentials"); Allure.label("tag", "Authentication"); // ... } } ``` ### Severity - `@Severity(SeverityLevel value)` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Annotations API:** ```java import io.qameta.allure.Severity; import org.junit.jupiter.api.Test; import static io.qameta.allure.SeverityLevel.*; class TestMyWebsite { @Test @Severity(CRITICAL) void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.label("severity", "critical"); // ... } } ``` ### Label - `@LabelAnnotation(String name, String value=DEFAULT_VALUE)` - `Allure.label(String name, String value)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. To set a label dynamically, just call the `label()` function with a name and a value for a label. You can do it multiple times to create an array of values under that name. Another way of setting a label is to create a custom class using `@LabelAnnotation` and annotate the tests with the new annotation, see the example below. Please be careful to copy `@Retention`, `@Target` and the other necessary annotations as this is shown in the example, otherwise you custom annotation may not work properly. **Annotations API:** ```java import io.qameta.allure.LabelAnnotation; import org.junit.jupiter.api.Test; import java.lang.annotation.Documented; import java.lang.annotation.ElementType; import java.lang.annotation.Inherited; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; @Documented @Inherited @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.METHOD, ElementType.TYPE}) @LabelAnnotation(name = "CustomLabelName") @interface MyLabel { String value(); } class TestMyWebsite { @Test @MyLabel("custom label value") void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.label("language", "java"); Allure.label("framework", "junit5"); Allure.label("CustomLabelName", "custom label value"); // ... } } ``` ### Allure ID (Allure TestOps) - `@AllureId(String value)` Set the test's [ID](/docs/v3/readability/#id). ```java import io.qameta.allure.AllureId; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @AllureId("123") void testAuthentication() { // ... } } ``` ### Link - `@Link(String value="", name="", String url="", String type=CUSTOM_LINK_TYPE)` - `@Links(Link[] value)` - `@Issue(String value="")` - `@Issues(Issue[] value)` - `@TmsLink(String value="")` - `@TmsLinks(TmsLink[] value)` - `Allure.link(String url)` - `Allure.link(String name, String url)` - `Allure.link(String name, String type, String url)` - `Allure.issue(String name, String url)` - `Allure.tms(String name, String url)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`allure.link.*.pattern`](/docs/junit-jupiter-configuration/#allure-link-⟨type⟩-pattern) configuration option. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue()` and `tms()`. **Annotations API:** ```java import io.qameta.allure.Issue; import io.qameta.allure.Link; import io.qameta.allure.TmsLink; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @Link(name = "Website", url = "https://dev.example.com/") @Issue("AUTH-123") @TmsLink("TMS-456") void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.link("Website", "https://dev.example.com/"); Allure.issue("AUTH-123", "https://jira.example.org/browse/AUTH-123"); Allure.tms("TMS-456", "https://tms.example.org/TMS-456"); // ... } } ``` ## Behavior-based hierarchy - `@Epic(String value="")` - `@Epics(Epic[] value)` - `@Feature(String value="")` - `@Features(Feature[] value)` - `@Story(String value="")` - `@Stories(Story[] value)` - `Allure.epic(String value)` - `Allure.feature(String value)` - `Allure.story(String value)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Annotations API:** ```java import io.qameta.allure.Epic; import io.qameta.allure.Feature; import io.qameta.allure.Story; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @Epic("Web interface") @Feature("Essential features") @Story("Authentication") void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.epic("Web interface"); Allure.feature("Essential features"); Allure.story("Authentication"); // ... } } ``` ## Suite-based hierarchy - `Allure.suite(String value)` Assign the name of suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). To assign names of parent suite or sub-suite, use [`label()`](#label) with the corresponding label names. **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.label("parentSuite" "Tests for web interface"); Allure.suite("Tests for essential features"); Allure.label("subSuite", "Tests for authentication"); // ... } } ``` ## Test steps - `@Step(String value="")` - `Allure.step(String name)` - `Allure.step(String name, Status status)` - `Allure.step(String name, ThrowableRunnableVoid runnable)` - `Allure.step(String name, ThrowableRunnable runnable)` - `Allure.step(ThrowableContextRunnableVoid runnable)` - `Allure.step(String name, ThrowableContextRunnableVoid runnable)` - `Allure.step(String name, ThrowableContextRunnable runnable)` - `Allure.step(ThrowableContextRunnable runnable)` Define a [test step](/docs/steps/) with the given `name`. There are three ways of defining a step. - “An annotated step”: define a method containing a test step and decorate it with `@Step()`. - “A lambda step”: write a test step in a lambda function and pass it to `Allure.step()`. - “A no-op step”: just pass a `name` and an optional `status` to `Allure.step()` to immediately add a corresponding entry to the results as a sub-step of the current step. When you are implementing a function for a lambda step, it can accept either no arguments or a single argument of class `StepContext`. This object provides the following methods: - `name()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are the same as for the test-wide implementation, see [Parametrized tests](#parametrized-tests). **Annotated steps:** ```java import io.qameta.allure.Step; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { step1(); step2(); } @Step("Step 1") void step1() { subStep1(); subStep2(); } @Step("Sub-step 1") void subStep1() { // ... } @Step("Sub-step 2") void subStep2() { // ... } @Step("Step 2") void step2() { // ... } } ``` **Lambda steps and no-op steps:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.step("Step 1", step -> { // ... Allure.step("Sub-step 1"); // ... Allure.step("Sub-step 2"); }); Allure.step("Step 2", step -> { // ... }); } } ``` ## Parametrized tests - `@Param(String value="", String name="", Parameter.Mode mode=DEFAULT, boolean excluded=false)` - `Allure.parameter(String name, T value)` - `Allure.parameter(String name, T value, Boolean excluded)` - `Allure.parameter(String name, T value, Parameter.Mode mode)` - `Allure.parameter(String name, T value, Boolean excluded, Parameter.Mode mode)` Specify a `name` and `value` of a parameter that was used during this test. This can be used for adding the parameters even to those function that do not use the `@ParameterizedTest()` decorator. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. Warning: Parameters added via `Allure.parameter()` are part of the test's identity, together with the parameters provided by the test framework. If a parameter's value changes between runs (for example, a timestamp or a random ID), the test will be treated as a new test on every run for the purposes of [History and retries](/docs/history-and-retries/). To avoid this, mark such parameters as excluded, see below. In Allure Java 2.x, parameters added via `Allure.parameter()` did not affect the test's identity. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous ones in the history. In Allure Java 2.x, this argument is only used by Allure TestOps. The `mode` argument affects how the parameter will be displayed in the report. Available options are defined in the `Parameter.Mode` enumeration: - `Parameter.Mode.DEFAULT` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `Parameter.Mode.MASKED` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `Parameter.Mode.HIDDEN` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. **Annotations API:** ```java import org.junit.jupiter.params.ParameterizedTest; import org.junit.jupiter.params.provider.ValueSource; class TestMyWebsite { @ParameterizedTest(name = "{displayName} ({argumentsWithNames})") @ValueSource(strings = {"johndoe", "johndoe@example.com"}) void testAuthentication(String login) { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthenticationWithUsername() { Allure.parameter("login", "johndoe"); // ... } @Test void testAuthenticationWithEmail() { Allure.parameter("login", "johndoe@example.com"); // ... } } ``` Note that by default, the Java runtime does not have access to the names of method arguments. This means that the Annotations API can only display pseudo-names like “arg0”, “arg1”, etc. To let it display the original names in the report, add `-parameters` into your project's `compilerArgs`. For example (in the project's `build.gradle.kts`): ```kotlin tasks.withType(JavaCompile::class) { options.compilerArgs.add("-parameters") } ``` The Annotations API reads the arguments through a JUnit extension that is registered automatically. For it to be loaded, enable extension autodetection in `src/test/resources/junit-platform.properties`. Without this property, the arguments are not reported as parameters. ```properties junit.jupiter.extensions.autodetection.enabled=true ``` ## Attachments - `@Attachment(String value="", String type="", String fileExtension="")` - `Allure.attachment(String name, String content)` - `Allure.attachment(String name, String type, String content)` - `Allure.attachment(String name, String type, String content, AttachmentOptions options)` - `Allure.attachment(String name, InputStream content)` - `Allure.attachment(String name, String type, InputStream content, AttachmentOptions options)` Add `content` as an [attachment](/docs/attachments/) to the test result under the given `name` (defaults to a unique pseudo-random string). Info: In Allure Java 2.x, the same functionality is provided by the `Allure.addAttachment()` methods, which accept the file extension as a string. They were removed in Allure Java 3.0. Tip: You can use data produced by any function, not necessarily read from an actual file. To create an attachment using the Annotations API, define a method that returns some data and annotate it with `@Attachment`. Call the method at any point during your test. To create an attachment using the Runtime API, just call `attachment()` at any point during your test. Pass the data as the `content` argument. The data will be processed as following: - If the data type is `byte[]`, Allure will use it without modification. - If the data type is `String`, Allure will convert it to `byte[]`. - If the data is of any other type, Allure will call the `toString()` method and then convert the string to `byte[]`. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension: as `fileExtension` in the `@Attachment` annotation, or via `AttachmentOptions.withFileExtension(".txt")` as the `options` argument of `attachment()`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. **Annotations API:** ```java import io.qameta.allure.Attachment; import org.junit.jupiter.api.Test; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; class TestMyWebsite { @Test void testAuthentication() throws IOException { // ... attachDataTXT(); attachScreenshotPNG(); } @Attachment(value = "data", type = "text/plain", fileExtension = ".txt") public String attachDataTXT() { return "This is the file content."; } @Attachment(value = "screenshot", type = "image/png", fileExtension = ".png") public byte[] attachScreenshotPNG() throws IOException { return Files.readAllBytes(Paths.get("/path/to/image.png")); } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.jupiter.api.Test; import java.io.IOException; import java.io.InputStream; import java.nio.file.Files; import java.nio.file.Paths; class TestMyWebsite { @Test void testAuthentication() throws IOException { // ... Allure.attachment("data.txt", "This is the file content."); try (InputStream is = Files.newInputStream(Paths.get("/path/img.png"))) { Allure.attachment("image.png", is); } } } ``` Info: Additionally, Allure provides a way to spawn threads that will add attachments to the test results asynchronously without blocking the test runner. While not recommended for the majority of cases, this approach can improve tests that otherwise have to wait for processing very large files, such as videos larger than 1 GB. To do so, pass a `CompletionStage` that produces the attachment's content to `Allure.attachmentAsync(String name, String type, CompletionStage body)`. In Allure Java 2.x, use `Allure.addByteAttachmentAsync()` or `Allure.addStreamAttachmentAsync()` instead. Additionally, you can consult our guides for integrating screenshots in JUnit 5 with Allure Report: - [Integrating screenshots in JUnit 5 and Selenide](/docs/guides/junit5-selenide-screenshots/) - [Integrating screenshots in JUnit 5 and Selenium](/docs/guides/junit5-selenium-screenshots/) ## Results history ### Flaky - `@Flaky` - `Allure.getLifecycle().updateTest(Consumer update)` Indicate that the test is known to be unstable and may not succeed every time. See [Flaky tests](/docs/test-stability/#flaky-tests). **Annotations API:** ```java import io.qameta.allure.Flaky; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @Flaky void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import io.qameta.allure.model.StatusDetails; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.getLifecycle().updateTest(result -> { result.setStatusDetails(new StatusDetails().setFlaky(true)); }); // ... } } ``` ### Muted - `@Muted` - `Allure.getLifecycle().updateTest(Consumer update)` Indicate that the test result should not be counted in the overall pass/fail statistics. See [Muted tests](/docs/test-stability/#muted-tests). **Annotations API:** ```java import io.qameta.allure.Muted; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test @Muted void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import io.qameta.allure.model.StatusDetails; import org.junit.jupiter.api.Test; class TestMyWebsite { @Test void testAuthentication() { Allure.getLifecycle().updateTest(result -> { result.setStatusDetails(new StatusDetails().setMuted(true)); }); // ... } } ``` --- ## JUnit 4 > URL: https://allurereport.org/docs/junit4/ # Getting started with Allure JUnit 4 [![Allure JUnit 4 latest version](https://img.shields.io/maven-central/v/io.qameta.allure/allure-junit4?style=flat "allure-junit4")](https://mvnrepository.com/artifact/io.qameta.allure/allure-junit4) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/) and your [JUnit 4](https://junit.org/junit4/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Ajunit4+-topic%3Acucumber) to see Allure JUnit 4 in action. ## Setting up To integrate [Allure](https://allurereport.org) into an existing **JUnit 4** project, you need to: 1. Add Allure dependencies to your project. 1. Set up AspectJ. 1. Designate a location for Allure results storage. Allure Java 3.x requires Java 17 or newer. ### Add Allure dependencies **Maven:** ```xml 3.0.0 io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-junit4 test ``` **Gradle (Kotlin):** ```kts // Define the version of Allure you want to use via the allureVersion property val allureVersion = "3.0.0" // ... dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) // Add necessary Allure dependencies to dependencies section testImplementation("io.qameta.allure:allure-junit4") } ``` **Gradle (Groovy):** ```groovy // Define the version of Allure you want to use via the allureVersion property def allureVersion = "3.0.0" dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") // Add necessary Allure dependencies to dependencies section testImplementation "io.qameta.allure:allure-junit4" } ``` ### Configure AspectJ AspectJ is required for registering Allure JUnit 4 as a listener for events that occur during the test execution. Additionally, Allure and some other frameworks (such as [allure-assertj](https://mvnrepository.com/artifact/io.qameta.allure/allure-assertj)) leverage AspectJ for various integrations, including the `@Step` and `@Attachment` annotations. **Maven:** ```xml 1.9.25 org.apache.maven.plugins maven-surefire-plugin 3.2.3 false -Dfile.encoding=${project.build.sourceEncoding} -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar" listener io.qameta.allure.junit4.AllureJunit4 org.aspectj aspectjweaver ${aspectj.version} io.qameta.allure allure-junit4-aspect test ``` **Gradle (Kotlin):** ```kts // Define the version of AspectJ val aspectJVersion = "1.9.25" // Define configuration for AspectJ agent val agent: Configuration by configurations.creating { isCanBeConsumed = true isCanBeResolved = true } dependencies { // Add aspectjweaver dependency agent("org.aspectj:aspectjweaver:${aspectJVersion}") // Add AspectJ integration for Allure testImplementation("io.qameta.allure:allure-junit4-aspect") } // Configure javaagent for test execution tasks.test { useJUnit() jvmArgs = listOf( "-javaagent:${agent.singleFile}" ) } ``` **Gradle (Groovy):** ```groovy // Define the version of AspectJ def aspectJVersion = '1.9.25' // Define configuration for AspectJ agent configurations { agent { canBeResolved = true canBeConsumed = true } } dependencies { // Add aspectjweaver dependency agent "org.aspectj:aspectjweaver:$aspectJVersion" // Add AspectJ integration for Allure testImplementation "io.qameta.allure:allure-junit4-aspect" } // Configure javaagent for test execution test { useJUnit() jvmArgs = [ "-javaagent:${configurations.agent.singleFile}" ] } ``` ### Specifying Allure Results location Allure, by default, saves test results in the project's root directory. However, it is recommended to store your test results in the build output directory. To configure this, create an `allure.properties` file and place it in the test resources directory of your project, which is typically located at `src/test/resources`: **Maven:** ```properties allure.results.directory=target/allure-results ``` **Gradle (Kotlin):** ```properties allure.results.directory=build/allure-results ``` **Gradle (Groovy):** ```properties allure.results.directory=build/allure-results ``` ### Run tests Run your JUnit 4 tests the same way as you would run them usually. For example: For Gradle: **MacOS/Linux:** ```bash ./gradlew test ``` **Windows:** ```bash gradlew test ``` For Maven: **MacOS/Linux:** ```bash ./mvnw verify ``` **Windows:** ```bash mvnw verify ``` After running the tests, Allure will gather the test execution data and store it in the `allure-results` directory. You can then generate an HTML report from these results using Allure's reporting tools. ### Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure JUnit 4 integration not only collects the data provided by JUnit 4's standard features, but also provides additional features for writing even better tests. This section lists the most notable ways to improve your tests, using both JUnit 4's and Allure JUnit 4's features. With Allure JUnit 4, you can: - provide [description, links and other metadata](#specify-description-links-and-other-metadata), - [organize tests](#organize-tests) into hierarchies, - divide the test into smaller, easier-to-read [test steps](#divide-a-test-into-steps), - describe parameters used when running [parametrized tests](#describe-parametrized-tests), - make the test save [screenshots and other files](#attach-screenshots-and-other-files) during execution, - [select which tests to run](#select-tests-via-a-test-plan-file) via a test plan file, - provide arbitrary [environment information](#environment-information) for the whole test report. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/junit4-reference/#metadata) for more details. For each of the metadata fields, there are two ways to assign it: via an annotation before a test method or via a method call inside a test method's body. The second way is called “dynamic”, because it allows you to construct strings and other values at runtime before passing to the methods. Note, however, that it is highly recommended to assign all metadata as early as possible. Otherwise, there is a risk of the test failing before having all metadata set, which is bad for the test report's readability. **Annotations API:** ```java import io.qameta.allure.Description; import io.qameta.allure.Issue; import io.qameta.allure.Link; import io.qameta.allure.Owner; import io.qameta.allure.Severity; import io.qameta.allure.TmsLink; import io.qameta.allure.junit4.DisplayName; import org.junit.Test; import static io.qameta.allure.SeverityLevel.*; public class TestMyWebsite { @Test @DisplayName("Test Authentication") @Description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") @Severity(CRITICAL) @Owner("John Doe") @Link(name = "Website", url = "https://dev.example.com/") @Issue("AUTH-123") @TmsLink("TMS-456") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; import static io.qameta.allure.SeverityLevel.*; public class TestMyWebsite { @Test public void testAuthentication() { Allure.getLifecycle().updateTest(result -> result.setName("Test Authentication")); Allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication."); Allure.label("severity", "critical"); Allure.label("owner", "John Doe"); Allure.link("Website", "https://dev.example.com/"); Allure.issue("AUTH-123", "https://example.com/issues/AUTH-123"); Allure.tms("TMS-456", "https://example.com/tms/TMS-456"); // ... } } ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure JUnit 4 provides functions to assign the relevant fields to tests either by adding annotations or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Annotations API:** ```java import io.qameta.allure.Epic; import io.qameta.allure.Feature; import io.qameta.allure.Story; import org.junit.Test; public class TestMyWebsite { @Test @Epic("Web interface") @Feature("Essential features") @Story("Authentication") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.epic("Web interface"); Allure.feature("Essential features"); Allure.story("Authentication"); // ... } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("parentSuite", "Tests for web interface"); Allure.suite("Tests for essential features"); Allure.label("subSuite", "Tests for authentication"); // ... } } ``` A test's location in the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy) is defined by the fully qualified names of the classes they are declared in, with common prefixes shown as parent packages. ### Divide a test into steps Allure JUnit 4 provides three ways of [creating steps and sub-steps](/docs/steps/): “annotated steps”, “lambda steps” and “no-op steps”, see the [reference](/docs/junit4-reference/#test-steps). **Annotated steps:** ```java import io.qameta.allure.Param; import io.qameta.allure.Step; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { testAuthenticationWith("https://1.example.com/", "alice", "qwerty"); testAuthenticationWith("https://2.example.com/", "bob", "asdfgh"); testAuthenticationWith("https://3.example.com/", "charlie", "zxcvbn"); } @Step("Authenticate on {url}") void testAuthenticationWith(String url, String login, String password) { openWebPage(url); enterCredentials(login, password); checkIfLoggedInAs(login); } @Step("Visit {url}") void openWebPage(@Param("URL") String url) { // ... } @Step("Enter credentials") void enterCredentials( @Param("Login") String login, @Param("Password") String password ) { // ... } @Step("Check if I am logged in") void checkIfLoggedInAs( @Param("Login") String login ) { // ... } } ``` **Lambda steps:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { private static final String[][] authenticationData = new String[][]{ {"https://1.example.com/", "alice", "qwerty"}, {"https://2.example.com/", "bob", "asdfgh"}, {"https://3.example.com/", "charlie", "zxcvbn"}, }; @Test public void testAuthentication() { for (String[] data : authenticationData) { String url = data[0]; String login = data[1]; String password = data[2]; Allure.step("Authenticate on %s".formatted(url), () -> { Allure.step("Visit %s".formatted(url), stepContext -> { stepContext.parameter("URL", url); // ... }); Allure.step("Enter credentials", stepContext -> { stepContext.parameter("Login", login); stepContext.parameter("Password", password); // ... }); Allure.step("Check if I am logged in", stepContext -> { stepContext.parameter("Login", login); // ... }); }); } } } ``` **No-op steps:** ```java import io.qameta.allure.Allure; import io.qameta.allure.model.Status; import org.junit.Test; public class TestMyWebsite { private static final String[][] authenticationData = new String[][]{ {"https://1.example.com/", "alice", "qwerty"}, {"https://2.example.com/", "bob", "asdfgh"}, {"https://3.example.com/", "charlie", "zxcvbn"}, }; @Test public void testAuthentication() { for (String[] data : authenticationData) { String url = data[0]; String login = data[1]; String password = data[2]; Allure.step("Begin testing %s...".formatted(url)); try { // ... Allure.step("Visited %s.".formatted(url)); // ... Allure.step("Entered credentials."); // ... Allure.step("I am logged in!"); } catch (Exception e) { Allure.step("Failed!", Status.FAILED); } } } } ``` ### Describe parametrized tests When using the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, use the Allure JUnit 4's `parameter()` to add the parameters to the report, see the [reference](/docs/junit4-reference/#parametrized-tests). ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthenticationWithUsername() { Allure.parameter("login", "johndoe"); // ... } @Test public void testAuthenticationWithEmail() { Allure.parameter("login", "johndoe@example.com"); // ... } } ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure JUnit 4 provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/junit4-reference/#attachments). ```java import io.qameta.allure.Allure; import org.junit.Test; import java.io.IOException; import java.io.InputStream; import java.nio.file.Files; import java.nio.file.Paths; public class TestMyWebsite { @Test public void testAuthentication() throws IOException { // ... Allure.attachment("data.txt", "This is the file content."); try (InputStream is = Files.newInputStream(Paths.get("/path/img.png"))) { Allure.attachment("image.png", is); } } } ``` ### Select tests via a test plan file Warning: For this feature to work, the [AspectJ integration for Allure Report](#configure-aspectj) must be added to the project. If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, JUnit 4 will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json ./gradlew test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" gradlew test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS and Java versions. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/java/environment-allure3-light.png, /images/java/environment-allure3-dark.png, /images/java/environment-allure2-light.png, /images/java/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](/docs/junit4-reference/#parametrized-tests). --- ## JUnit 4 configuration > URL: https://allurereport.org/docs/junit4-configuration/ # Allure JUnit 4 configuration This page describes the configuration options that affect the behavior of [Allure JUnit 4](/docs/junit4/). To set the configuration options, create a `allure.properties` file in your project's `src/test/resources` directory. Allure JUnit 4 loads the configuration into the JVM system properties for the test runtime. ## allure.label.⟨NAME⟩ ```properties allure.label.epic = Authentication allure.label.severity = blocker ``` Assign an arbitrary [label](/docs/junit4-reference/#label) to all the tests in the current test run. ## allure.link.⟨TYPE⟩.pattern ```properties allure.link.issue.pattern = https://jira.example.org/browse/{} ``` Define a **link pattern** — a template that can be used to construct full URLs from short identifiers, see the [reference](/docs/junit4-reference/#link). The value must be a string pattern containing `{}` at the position where the identifier should be placed. The pattern will be applied when processing any link of the given `TYPE`. You can use the option multiple times, thus defining multiple link patterns for different link types. ## allure.results.directory ```properties allure.results.directory = build/allure-results ``` Path to the directory where Allure JUnit 4 will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. --- ## JUnit 4 reference > URL: https://allurereport.org/docs/junit4-reference/ # Allure JUnit 4 reference These are the functions that you can use to integrate your JUnit 4 tests with Allure. Allure JUnit 4 provides more than one way to use some features. In most cases, one way involves using an annotation on the test method, while the other involves a method call inside the test method body. The latter style is called “dynamic” and allows to construct values dynamically. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `@DisplayName(String value)` - `Allure.getLifecycle().updateTest(Consumer update)` Set the test's [title](/docs/v3/readability/#title). If you need to construct a test's title dynamically, write a lambda function that does that and pass it to the `updateTest()` method of the `AllureLifecycle` object, see the example below. Info: In Allure Java 2.x, the `AllureLifecycle` method for updating the test result is called `updateTestCase()`. **Annotations API:** ```java import io.qameta.allure.junit4.DisplayName; import org.junit.Test; public class TestMyWebsite { @Test @DisplayName("Test Authentication") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.getLifecycle().updateTest(result -> { result.setName("Test Authentication"); }); // ... } } ``` ### Description - `@Description(String value="")` - `Allure.description(String description)` - `Allure.descriptionHtml(String description)` Set the test's [description](/docs/v3/readability/#description). Use the `@Description()` annotation to set a description statically or use the `description()` method to set it dynamically in runtime. **Annotations API:** ```java import io.qameta.allure.Description; import org.junit.Test; public class TestMyWebsite { @Test @Description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication."); // ... } } ``` Alternatively, you can let Allure JUnit 4 parse the description from the test method's JavaDoc comment. To do so, first add a special annotation processor to your project's Maven or Gradle configuration. ```kotlin // In the project's `build.gradle.kts`: val allureVersion = "2.32.0" dependencies { // ... testAnnotationProcessor("io.qameta.allure:allure-descriptions-javadoc:$allureVersion") } ``` Then, use the Annotations API without arguments to parse the description from JavaDoc. Note that on a large project, processing the annotations may significantly increase both the compilation time and the file size of the test executable files. Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ```java import io.qameta.allure.Description; import org.junit.Test; public class TestMyWebsite { /** * This test attempts to log into the website using a login and a password. Fails if any error happens. *

* Note that this test does not test 2-Factor Authentication. */ @Test @Description public void testAuthentication() { // ... } } ``` ### Owner - `@Owner(String value="")` Set the test's [owner](/docs/v3/readability/#owner). **Annotations API:** ```java import io.qameta.allure.Owner; import org.junit.Test; public class TestMyWebsite { @Test @Owner("John Doe") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("owner", "John Doe"); // ... } } ``` ### Lead - `@Lead(String value="")` Set the test's lead — the person responsible for the test's maintenance. **Annotations API:** ```java import io.qameta.allure.Lead; import org.junit.Test; public class TestMyWebsite { @Test @Lead("Jane Doe") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("lead", "Jane Doe"); // ... } } ``` ### Tag - `@Tag(String value)` - `@Tags(Tag[] value)` Set the test's [tags](/docs/v3/readability/#tags). **Annotations API:** ```java import io.qameta.allure.junit4.Tag; import org.junit.Test; public class TestMyWebsite { @Test @Tag("NewUI") @Tag("Essentials") @Tag("Authentication") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("tag", "NewUI"); Allure.label("tag", "Essentials"); Allure.label("tag", "Authentication"); // ... } } ``` ### Severity - `@Severity(SeverityLevel value)` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Annotations API:** ```java import io.qameta.allure.Severity; import org.junit.Test; import static io.qameta.allure.SeverityLevel.*; public class TestMyWebsite { @Test @Severity(CRITICAL) public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("severity", "critical"); // ... } } ``` ### Label - `@LabelAnnotation(String name, String value=DEFAULT_VALUE)` - `Allure.label(String name, String value)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. To set a label dynamically, just call the `label()` function with a name and a value for a label. You can do it multiple times to create an array of values under that name. Another way of setting a label is to create a custom class using `@LabelAnnotation` (in a separate file) and annotate the tests with the new annotation, see the example below. Please be careful to copy `@Retention`, `@Target` and the other necessary annotations as this is shown in the example, otherwise you custom annotation may not work properly. **Annotations API:** ```java import io.qameta.allure.LabelAnnotation; import java.lang.annotation.Documented; import java.lang.annotation.ElementType; import java.lang.annotation.Inherited; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; @Documented @Inherited @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.METHOD, ElementType.TYPE}) @LabelAnnotation(name = "CustomLabelName") @interface MyLabel { String value(); } ``` ```java import org.junit.Test; public class TestMyWebsite { @Test @MyLabel("custom label value") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("language", "java"); Allure.label("framework", "junit4"); Allure.label("CustomLabelName", "custom label value"); // ... } } ``` ### Allure ID (Allure TestOps) - `@AllureId(String value)` Set the test's [ID](/docs/v3/readability/#id). ```java import io.qameta.allure.AllureId; import org.junit.Test; public class TestMyWebsite { @Test @AllureId("123") public void testAuthentication() { // ... } } ``` ### Link - `@Link(String value="", name="", String url="", String type=CUSTOM_LINK_TYPE)` - `@Links(Link[] value)` - `@Issue(String value="")` - `@Issues(Issue[] value)` - `@TmsLink(String value="")` - `@TmsLinks(TmsLink[] value)` - `Allure.link(String url)` - `Allure.link(String name, String url)` - `Allure.link(String name, String type, String url)` - `Allure.issue(String name, String url)` - `Allure.tms(String name, String url)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`allure.link.*.pattern`](/docs/junit4-configuration/#allure-link-⟨type⟩-pattern) configuration option. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue()` and `tms()`. **Annotations API:** ```java import io.qameta.allure.Issue; import io.qameta.allure.Link; import io.qameta.allure.TmsLink; import org.junit.Test; public class TestMyWebsite { @Test @Link(name = "Website", url = "https://dev.example.com/") @Issue("AUTH-123") @TmsLink("TMS-456") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.link("Website", "https://dev.example.com/"); Allure.issue("AUTH-123", "https://jira.example.org/browse/AUTH-123"); Allure.tms("TMS-456", "https://tms.example.org/TMS-456"); // ... } } ``` ## Behavior-based hierarchy - `@Epic(String value="")` - `@Epics(Epic[] value)` - `@Feature(String value="")` - `@Features(Feature[] value)` - `@Story(String value="")` - `@Stories(Story[] value)` - `Allure.epic(String value)` - `Allure.feature(String value)` - `Allure.story(String value)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Annotations API:** ```java import io.qameta.allure.Epic; import io.qameta.allure.Feature; import io.qameta.allure.Story; import org.junit.Test; public class TestMyWebsite { @Test @Epic("Web interface") @Feature("Essential features") @Story("Authentication") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.epic("Web interface"); Allure.feature("Essential features"); Allure.story("Authentication"); // ... } } ``` ## Suite-based hierarchy - `Allure.suite(String value)` Assign the name of suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("parentSuite" "Tests for web interface"); Allure.suite("Tests for essential features"); Allure.label("subSuite", "Tests for authentication"); // ... } } ``` ## Test steps - `@Step(String value="")` - `@Param(String value="", String name="", Parameter.Mode mode=DEFAULT, boolean excluded=false)` - `Allure.step(String name)` - `Allure.step(String name, Status status)` - `Allure.step(String name, ThrowableRunnableVoid runnable)` - `Allure.step(String name, ThrowableRunnable runnable)` - `Allure.step(ThrowableContextRunnableVoid runnable)` - `Allure.step(String name, ThrowableContextRunnableVoid runnable)` - `Allure.step(String name, ThrowableContextRunnable runnable)` - `Allure.step(ThrowableContextRunnable runnable)` Define a [test step](/docs/steps/) with the given `name`. There are three ways of defining a step. - “An annotated step”: define a method containing a test step and decorate it with `@Step()`. - “A lambda step”: write a test step in a lambda function and pass it to `Allure.step()`. - “A no-op step”: just pass a `name` and an optional `status` to `Allure.step()` to immediately add a corresponding entry to the results as a sub-step of the current step. When you are implementing an annotated step that takes arguments, you can specify human-readable names for them via the `@Param` annotation. The arguments will be displayed as step-level parameters under the provided names. For arguments without the `@Param` annotation, Allure JUnit 4 will use either names indicating the arguments' order (“arg1”, “arg2”, etc) or the argument names from the Java code. The latter requires the `-parameters` compiler argument, see [How to start](/docs/junit4/#how-to-start). When you are implementing a function for a lambda step, it can accept either no arguments or a single argument of class `StepContext`. This object provides the following methods: - `name()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. Both the `@Param` annotation and the step context's `parameter()` method allow for the same additional arguments as the test-wide implementation of the [Parametrized tests](#parametrized-tests). **Annotated steps:** ```java import io.qameta.allure.Param; import io.qameta.allure.Step; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { testAuthenticationWith("https://1.example.com/", "alice", "qwerty"); testAuthenticationWith("https://2.example.com/", "bob", "asdfgh"); testAuthenticationWith("https://3.example.com/", "charlie", "zxcvbn"); } @Step("Authenticate on {url}") void testAuthenticationWith(String url, String login, String password) { openWebPage(url); enterCredentials(login, password); checkIfLoggedInAs(login); } @Step("Visit {url}") void openWebPage(@Param("URL") String url) { // ... } @Step("Enter credentials") void enterCredentials( @Param("Login") String login, @Param("Password") String password ) { // ... } @Step("Check if I am logged in") void checkIfLoggedInAs( @Param("Login") String login ) { // ... } } ``` **Lambda steps:** ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { private static final String[][] authenticationData = new String[][]{ {"https://1.example.com/", "alice", "qwerty"}, {"https://2.example.com/", "bob", "asdfgh"}, {"https://3.example.com/", "charlie", "zxcvbn"}, }; @Test public void testAuthentication() { for (String[] data : authenticationData) { String url = data[0]; String login = data[1]; String password = data[2]; Allure.step("Authenticate on %s".formatted(url), () -> { Allure.step("Visit %s".formatted(url), stepContext -> { stepContext.parameter("URL", url); // ... }); Allure.step("Enter credentials", stepContext -> { stepContext.parameter("Login", login); stepContext.parameter("Password", password); // ... }); Allure.step("Check if I am logged in", stepContext -> { stepContext.parameter("Login", login); // ... }); }); } } } ``` **No-op steps:** ```java import io.qameta.allure.Allure; import io.qameta.allure.model.Status; import org.junit.Test; public class TestMyWebsite { private static final String[][] authenticationData = new String[][]{ {"https://1.example.com/", "alice", "qwerty"}, {"https://2.example.com/", "bob", "asdfgh"}, {"https://3.example.com/", "charlie", "zxcvbn"}, }; @Test public void testAuthentication() { for (String[] data : authenticationData) { String url = data[0]; String login = data[1]; String password = data[2]; Allure.step("Begin testing %s...".formatted(url)); try { // ... Allure.step("Visited %s.".formatted(url)); // ... Allure.step("Entered credentials."); // ... Allure.step("I am logged in!"); } catch (Exception e) { Allure.step("Failed!", Status.FAILED); } } } } ``` ## Parametrized tests - `Allure.parameter(String name, T value)` - `Allure.parameter(String name, T value, Boolean excluded)` - `Allure.parameter(String name, T value, Parameter.Mode mode)` - `Allure.parameter(String name, T value, Boolean excluded, Parameter.Mode mode)` Specify a `name` and `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. Warning: Parameters added via `Allure.parameter()` are part of the test's identity. Tests that only differ in parameters are treated as different tests, and if a parameter's value changes between runs (for example, a timestamp or a random ID), the test will be treated as a new test on every run for the purposes of [History and retries](/docs/history-and-retries/). To avoid this, mark such parameters as excluded, see below. In Allure Java 2.x, Allure JUnit 4 identifies tests based only on their names, so tests that only differ in parameters are treated as one test. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous ones in the history. In Allure Java 2.x, this argument is only used by Allure TestOps. The `mode` argument affects how the parameter will be displayed in the report. Available options are defined in the `Parameter.Mode` enumeration: - `Parameter.Mode.DEFAULT` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `Parameter.Mode.MASKED` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `Parameter.Mode.HIDDEN` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. ```java import io.qameta.allure.Allure; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthenticationWithUsername() { Allure.parameter("login", "johndoe"); // ... } @Test public void testAuthenticationWithEmail() { Allure.parameter("login", "johndoe@example.com"); // ... } } ``` ## Attachments - `@Attachment(String value="", String type="", String fileExtension="")` - `Allure.attachment(String name, String content)` - `Allure.attachment(String name, String type, String content)` - `Allure.attachment(String name, String type, String content, AttachmentOptions options)` - `Allure.attachment(String name, InputStream content)` - `Allure.attachment(String name, String type, InputStream content, AttachmentOptions options)` Add `content` as an [attachment](/docs/attachments/) to the test result under the given `name` (defaults to a unique pseudo-random string). Info: In Allure Java 2.x, the same functionality is provided by the `Allure.addAttachment()` methods, which accept the file extension as a string. They were removed in Allure Java 3.0. Tip: You can use data produced by any function, not necessarily read from an actual file. To create an attachment using the Annotations API, define a method that returns some data and annotate it with `@Attachment`. Call the method at any point during your test. To create an attachment using the Runtime API, just call `attachment()` at any point during your test. Pass the data as the `content` argument. The data will be processed as following: - If the data type is `byte[]`, Allure will use it without modification. - If the data type is `String`, Allure will convert it to `byte[]`. - If the data is of any other type, Allure will call the `toString()` method and then convert the string to `byte[]`. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension: as `fileExtension` in the `@Attachment` annotation, or via `AttachmentOptions.withFileExtension(".txt")` as the `options` argument of `attachment()`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. **Annotations API:** ```java import io.qameta.allure.Attachment; import org.junit.Test; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; public class TestMyWebsite { @Test public void testAuthentication() throws IOException { // ... attachDataTXT(); attachScreenshotPNG(); } @Attachment(value = "data", type = "text/plain", fileExtension = ".txt") public String attachDataTXT() { return "This is the file content."; } @Attachment(value = "screenshot", type = "image/png", fileExtension = ".png") public byte[] attachScreenshotPNG() throws IOException { return Files.readAllBytes(Paths.get("/path/to/image.png")); } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.junit.Test; import java.io.IOException; import java.io.InputStream; import java.nio.file.Files; import java.nio.file.Paths; public class TestMyWebsite { @Test public void testAuthentication() throws IOException { // ... Allure.attachment("data.txt", "This is the file content."); try (InputStream is = Files.newInputStream(Paths.get("/path/img.png"))) { Allure.attachment("image.png", is); } } } ``` Info: Additionally, Allure provides a way to spawn threads that will add attachments to the test results asynchronously without blocking the test runner. While not recommended for the majority of cases, this approach can improve tests that otherwise have to wait for processing very large files, such as videos larger than 1 GB. To do so, pass a `CompletionStage` that produces the attachment's content to `Allure.attachmentAsync(String name, String type, CompletionStage body)`. In Allure Java 2.x, use `Allure.addByteAttachmentAsync()` or `Allure.addStreamAttachmentAsync()` instead. ## Results history ### Flaky - `@Flaky` - `Allure.getLifecycle().updateTest(Consumer update)` Indicate that the test is known to be unstable and may not succeed every time. See [Flaky tests](/docs/test-stability/#flaky-tests). The `@Flaky` annotation is supported for JUnit 4 starting with Allure Java 3.0. In Allure Java 2.x, use the Runtime API. **Annotations API:** ```java import io.qameta.allure.Flaky; import org.junit.Test; public class TestMyWebsite { @Test @Flaky public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import io.qameta.allure.model.StatusDetails; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.getLifecycle().updateTest(result -> { result.setStatusDetails(new StatusDetails().setFlaky(true)); }); // ... } } ``` ### Muted - `@Muted` - `Allure.getLifecycle().updateTest(Consumer update)` Indicate that the test result should not be counted in the overall pass/fail statistics. See [Muted tests](/docs/test-stability/#muted-tests). The `@Muted` annotation is supported for JUnit 4 starting with Allure Java 3.0. In Allure Java 2.x, use the Runtime API. **Annotations API:** ```java import io.qameta.allure.Muted; import org.junit.Test; public class TestMyWebsite { @Test @Muted public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import io.qameta.allure.model.StatusDetails; import org.junit.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.getLifecycle().updateTest(result -> { result.setStatusDetails(new StatusDetails().setMuted(true)); }); // ... } } ``` --- ## Mocha > URL: https://allurereport.org/docs/mocha/ # Getting started with Allure Mocha [![Allure Mocha npm latest version](https://img.shields.io/npm/v/allure-mocha?style=flat "Allure Mocha npm latest version")](https://www.npmjs.com/package/allure-mocha) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Mocha](https://mochajs.org/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Amocha) to see Allure Mocha in action. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure Mocha is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure Mocha integration and make sure that the Mocha framework itself is also listed in the project's dependencies. **npm:** ```bash npm install --save-dev mocha allure-mocha ``` **yarn:** ```bash yarn add --dev mocha allure-mocha allure-js-commons ``` **pnpm:** ```bash pnpm add -D mocha allure-mocha ``` 1. Set `allure-mocha` as the Mocha reporter and specify the results directory. The exact way depends on how you configure and run Mocha. Here are some examples: **runner.mjs:** ```js import { glob } from "glob"; import Mocha from "mocha"; const mocha = new Mocha({ reporter: "allure-mocha", reporterOptions: { resultsDir: "allure-results", }, }); glob.sync("test/**/*.spec.{m,c,}js").forEach((file) => mocha.addFile(file)); await mocha.loadFilesAsync(); mocha.run((failures) => process.exit(failures)); ``` **.mocharc.json:** ```json { "reporter": "allure-mocha", "reporterOptions": ["resultsDir=allure-results"] } ``` **CLI:** ```shell npx mocha -R allure-mocha -O "resultsDir=allure-results" ``` Tip: If you want to keep the default console output, set the [`extraReporters`](/docs/mocha-configuration/#extrareporters) reporter option to `spec`. See also [Allure Mocha configuration](/docs/mocha-configuration/). ### 2. Run tests Run your Mocha tests the same way as you would run them usually. For example: **npm:** ```bash npx mocha ``` **yarn:** ```bash yarn run mocha ``` **pnpm:** ```bash pnpx mocha ``` Or, if you use a runner script (assuming it's named `runner.js`): **npm, pnpm:** ```bash node runner.js ``` **yarn:** ```bash yarn node runner.js ``` This will save necessary data into `allure-results` or other directory, according to the [configuration](/docs/mocha-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Mocha integration extends the standard reporting features of Mocha by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure Mocha provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Mocha will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/mocha-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.owner("John Doe"); await allure.tags("Web interface", "Authentication"); await allure.severity("critical"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.owner:JohnDoe" + " @allure.label.tag:WebInterface" + " @allure.label.tag:Authentication" + " @allure.label.severity:critical", async () => { // ... }, ); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); ``` ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), you can use the `step()` function, see the [reference](/docs/mocha-reference/#test-steps). ```js import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; it("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ### Describe parametrized tests A typical way to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern in Mocha is to define the test in a loop and use the loop variable in the test's title and body. To display a parameter value in the test report, pass it to the [`parameter()`](/docs/mocha-reference/#parametrized-tests) function. ```js import * as allure from "allure-js-commons"; for (const login of ["johndoe", "johndoe@example.com"]) { it(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } ``` ### Set labels globally Any [labels](/docs/mocha-reference/#label), including custom ones, can be set via the environment variables in your operating system. Here's an example (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface npx mocha ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" npx mocha ``` ### Attach screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure Mocha reference](/docs/mocha-reference/#attachments). ```js import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; it("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Mocha will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json` (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json npx mocha ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" npx mocha ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, edit the [`environmentInfo`](/docs/vitest-configuration/#environmentinfo) object in the configuration. For example, it is a good idea to use this to remember the OS version and Node.js version retrieved from the [`os`](https://nodejs.org/api/os.html) and [`process`](https://nodejs.org/api/process.html) objects. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/js/environment-allure3-light.png, /images/js/environment-allure3-dark.png, /images/js/environment-allure2-light.png, /images/js/environment-allure2-dark.png ```js import { glob } from "glob"; import Mocha from "mocha"; import * as os from "node:os"; const mocha = new Mocha({ reporter: "allure-mocha", reporterOptions: { environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }, }); glob.sync("test/**/*.spec.{m,c,}js").forEach((file) => mocha.addFile(file)); await mocha.loadFilesAsync(); mocha.run((failures) => process.exit(failures)); ``` Note that if your launch includes multiple Mocha runs (see [How it works](/docs/how-it-works/)), Allure Mocha will only save the environment information from the latest run. --- ## Allure Mocha configuration > URL: https://allurereport.org/docs/mocha-configuration/ # Allure Mocha configuration There are two ways of configuring [Allure Mocha](/docs/mocha/): [using a runner script](#using-a-runner-script) and via [Mocha's configuration](#using-mocha-s-configuration). The first one requires some extra steps but supports all the configuration options of Allure Mocha. The second one is easier to use, but only a limited set of options is supported. ## Using a runner script Create a runner script and pass the `reporter` and `reporterOptions` properties to an instance of `Mocha`. Then, use the configured instance to run the tests. Here are examples of ESM and CommonJS setups: **runner.mjs:** ```js import { Status } from "allure-js-commons"; import { glob } from "glob"; import Mocha from "mocha"; import * as os from "node:os"; const mocha = new Mocha({ reporter: "allure-mocha", reporterOptions: { resultsDir: "allure-results", extraReporters: "spec", links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: (v) => `https://jira.example.com/browse/${v}`, }, }, categories: [ { name: "foo", messageRegex: "bar", traceRegex: "baz", matchedStatuses: [Status.FAILED, Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, globalLabels: [{ name: "layer", value: "api" }], }, /* Other Mocha options... */ }); glob.sync("test/**/*.spec.{m,c,}js").forEach((file) => mocha.addFile(file)); await mocha.loadFilesAsync(); mocha.run((failures) => process.exit(failures)); ``` **runner.cjs:** ```js const { Status } = require("allure-js-commons"); const { glob } = require("glob"); const Mocha = require("mocha"); const os = require("node:os"); const mocha = new Mocha({ reporter: "allure-mocha", reporterOptions: { resultsDir: "allure-results", extraReporters: "spec", links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: (v) => `https://jira.example.com/browse/${v}`, }, }, categories: [ { name: "foo", messageRegex: "bar", traceRegex: "baz", matchedStatuses: [Status.FAILED, Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, globalLabels: [{ name: "layer", value: "api" }], }, /* Other Mocha options */ }); glob.sync("test/**/*.spec.{m,c,}js").forEach((file) => mocha.addFile(file)); mocha.loadFilesAsync().then(() => { mocha.run((failures) => process.exit(failures)); }); ``` Info: The examples use the [glob](https://www.npmjs.com/package/glob) package to locate and load the test files. You may need to adjust the pattern to your project's layout. Execute the script to run the tests: **npm, pnpm:** ```shell node ./runner.mjs ``` **yarn:** ```shell yarn node ./runner.mjs ``` ## Using Mocha's configuration You may use a [Mocha configuration file](https://mochajs.org/#configuring-mocha-nodejs) or [the CLI](https://mochajs.org/#command-line-usage) to configure Allure Mocha. Keep in mind, though, that only [`resultsDir`](#resultsdir) and [`extraReporters`](#extrareporters) are supported that way. **.mocharc.json:** ```json { "reporter": "allure-mocha", "reporterOptions": ["resultsDir=allure-results", "extraReporters=spec"] } ``` **package.json:** ```json { // ... "mocha": { "reporter": "allure-mocha", "reporterOptions": ["resultsDir=allure-results", "extraReporters=spec"] } } ``` **.mocharc.yaml:** ```yaml reporter: allure-mocha reporterOptions: - "resultsDir=allure-results" - "extraReporters=spec" ``` **.mocharc.cjs:** ```js module.exports = { reporter: "allure-mocha", reporterOptions: ["resultsDir=allure-results", "extraReporters=spec"], }; ``` **CLI:** ```shell npx mocha -R "allure-mocha" -O "resultsDir=allure-results,extraReporters=spec" ``` ## Configuration options Info: This section describes the configuration options for Allure Mocha 3.0 and newer. For older versions, you may check this [readme](https://github.com/allure-framework/allure-js/blob/v2.15.1/packages/allure-mocha/README.md). ### resultsDir Path to the directory where Allure Mocha will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. Warning: The path can't contain the `=` and `,` characters if provided via a Mocha configuration file or the CLI. Use a [runner script](#using-a-runner-script) if you need to overcome this limitation. ### extraReporters One or more Mocha reporters to run alongside Allure Mocha. In its most basic form, it takes a single string, which is a reporter's module name or path: **Built-in reporter:** ```js npx mocha -R allure-mocha -O extraReporters=spec ``` **Local reporter:** ```js npx mocha -R allure-mocha -O extraReporters=./reporter.cjs ``` If you need to pass options to the reporter, use the following syntax (it's only supported in a [runner script](#using-a-runner-script)): ```js const mocha = new Mocha({ reporter: "allure-mocha", reporterOptions: { extraReporters: ["json", { output: "results.json" }], }, }); ``` You may enable more than one extra reporter: ```js const mocha = new Mocha({ reporter: "allure-mocha", reporterOptions: { extraReporters: ["spec", ["json", { output: "results.json" }]], }, }); ``` ### links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/mocha-reference/#link)), you can specify the following: - `nameTemplate` — a template or a function for generating the link name when it is not provided. - `urlTemplate` — a template or a function for generating the link address when it is not provided. The templates can be strings (with `%s` where the identifier should be placed) or functions (accepting the identifier and returning the URL). For example, with the configuration above, `await allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/`. Allure will display the link with the appropriate icon for the `issue` type. Info: This option can only be provided via a [runner script](#using-a-runner-script). ### categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). Info: This option can only be provided via a [runner script](#using-a-runner-script). ### environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). Info: This option can only be provided via a [runner script](#using-a-runner-script). ### globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ name, value }` label objects: ```js const mocha = new Mocha({ reporter: "allure-mocha", reporterOptions: { globalLabels: [ { name: "layer", value: "api" }, { name: "team", value: "backend" }, ], }, }); ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```js const mocha = new Mocha({ reporter: "allure-mocha", reporterOptions: { globalLabels: { layer: "api", team: "backend", }, }, }); ``` Info: This option can only be provided via a [runner script](#using-a-runner-script). --- ## Allure Mocha reference > URL: https://allurereport.org/docs/mocha-reference/ # Allure Mocha reference These are the functions that you can use to integrate your Mocha tests with Allure. In most cases, Allure Mocha provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Mocha will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `allure.description(markdown: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.description("This test attempts to log into the website."); // ... }); ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.owner("John Doe"); // ... }); ``` **Metadata API:** ```js it("Test Authentication @allure.label.owner:JohnDoe", async () => { // ... }); ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` - `@allure.label.tag:⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.tag("New UI"); await allure.tags("Essentials", "Authentication"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.tag:NewUI" + " @allure.label.tag:Essentials" + " @allure.label.tag:Authentication", async () => { // ... }, ); ``` ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Runtime API:** ```js import * as allure from "allure-js-commons"; import { Severity } from "allure-js-commons"; it("Test Authentication", async () => { await allure.severity(Severity.CRITICAL); // ... }); ``` **Metadata API:** ```js it("Test Authentication @allure.label.severity:critical", async () => { // ... }); ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` - `@allure.label.⟨NAME⟩:⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.label("microservice", "UI"); // ... }); ``` **Metadata API:** ```js it("Test Authentication @allure.label.microservice:UI", async () => { // ... }); ``` ### ID - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). ```js import * as allure from "allure-js-commons"; it("Test Authentication @allure.id:123", async () => { // ... }); ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string, defaults to “link”), Allure will try to load a corresponding **link template** to process the URL, as defined by the [`links`](/docs/mocha-configuration/#links) configuration option. If no template is found for the given type or if the link already represents a proper URL, it's left unmodified. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.issue("AUTH-123", "Related issue"); await allure.tms("TMS-456", "Related TMS issue"); await allure.link("JIRA-777", "Related Jira issue", "jira"); await allure.link("https://example.com/", "Project website"); // ... }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features**, or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the names of **parent suite**, **suite**, or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Runtime API:** ```js import * as allure from "allure-js-commons"; it("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```js it( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test step](/docs/steps/) with the given `name`. There are two ways of defining a step. - **Lambda steps** Write a test step in a function and pass it to `allure.step()`. If the function returns a value, `allure.step()` will return it without modification, and it will not affect the report. If it returns a promise, it will be awaited and resolved. The function can accept either no arguments or a single object of type `StepContext`. This object provides the following methods: - `displayName()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are similar to the test-wide implementation, see [Parametrized tests](#parametrized-tests). - **No-op steps** If you call `allure.logStep()`, Allure will add to the report a no-op step. This allows for a log-style reporting within a test or within a larger step. A no-op step finishes immediately after it started and cannot have any sub-steps, or parameters. The optional second argument indicates the status that will be shown for the step in the report. Allowed values are: `Status.PASSED` (the default), `Status.FAILED`, `Status.BROKEN`, and `Status.SKIPPED`. The optional third argument is an `Error` object. When provided, the data from this object will be displayed in a message inside the step. ```js import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; it("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` Specify a `name` and a `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. The `options` argument, if given, must be an object with two optional properties `excluded` and `mode`. - If `excluded` is set to true, Allure will not use the parameter when comparing the current test result with the previous one in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. Note, that even when you use the `"masked"` or `"hidden"` mode, it is still possible to extract the value from the `allure_results` directory if you publish it. ```js import * as allure from "allure-js-commons"; for (const login of ["johndoe", "johndoe@example.com"]) { it(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } ``` ## Attachments - `allure.attachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options in an object (as shown for the image attachment below) or just specify the media type and let Allure deduce the appropriate filename extension automatically (as shown for the text attachment below). In either case, the media type can be a value from the `ContentType` enumeration or any string. ```js import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; it("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` --- ## Newman > URL: https://allurereport.org/docs/newman/ # Getting started with Allure Newman [![Allure Newman npm latest version](https://img.shields.io/npm/v/newman-reporter-allure?style=flat "Allure Newman npm latest version")](https://www.npmjs.com/package/newman-reporter-allure) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Newman](https://github.com/postmanlabs/newman) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Anewman) to see Allure Newman in action. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure Newman is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure Newman integration. **npm:** ```bash npm install --save-dev newman-reporter-allure ``` **yarn:** ```bash yarn add --dev newman-reporter-allure ``` **pnpm:** ```bash pnpm add -D newman-reporter-allure ``` ### 2. Run tests When running Newman: 1. Specify either a URL, an identifier, or a file path of your Postman collection. 1. Specify a list of [Newman reporters](https://learning.postman.com/docs/collections/using-newman-cli/newman-built-in-reporters/) that includes the `allure` reporter. You may want to include `cli` (the Newman's default reporter) as well to keep the terminal output informative. 1. Specify additional [Allure Newman configuration options](/docs/newman-configuration/) if necessary. The example below assumes that the tests are [public](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/manage-workspaces/#change-workspace-visibility). (Alternatively, [use the Postman API](https://github.com/postmanlabs/newman?tab=readme-ov-file#using-newman-with-the-postman-api) to run them.) **npm:** ```bash npx newman run https://www.postman.com/collections/8854915-454a2dc7-dcbe-41cf-9bfa-da544fcd93a2 \ --reporters cli,allure \ --reporter-allure-resultsDir output/allure-results ``` **yarn:** ```bash yarn run newman run https://www.postman.com/collections/8854915-454a2dc7-dcbe-41cf-9bfa-da544fcd93a2 \ --reporters cli,allure \ --reporter-allure-resultsDir output/allure-results ``` **pnpm:** ```bash pnpx newman run https://www.postman.com/collections/8854915-454a2dc7-dcbe-41cf-9bfa-da544fcd93a2 \ --reporters cli,allure \ --reporter-allure-resultsDir output/allure-results ``` This will save necessary data into `allure-results`. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Newman integration extends the standard reporting features of Newman by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/jest-reference/#metadata) for an exhaustive list of what can be added. ```js // @allure.label.owner=JohnDoe // @allure.label.severity=critical // @allure.label.tag=WebInterface // @allure.label.tag=Authentication pm.test("Test Authentication", function () { // ... }); ``` Danger: Adding links to issues and other webpages is currently not supported by the Allure Newman integration. ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): ```js // @allure.label.epic=WebInterface // @allure.label.feature=EssentialFeatures // @allure.label.story=Authentication pm.test("Test Authentication", function () { // ... }); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): ```js // @allure.label.parentSuite=WebInterface // @allure.label.suite=EssentialFeatures // @allure.label.subSuite=Authentication pm.test("Test Authentication", function () { // ... }); ``` ### Divide a test into steps If a Postman request has multiple `pm.test()` calls in its “Tests” section, Allure will display them as separate [test steps](/docs/steps/). ```js pm.test("Make sure the response is valid JSON", function () { // ... }); pm.test("Compare the response with the expected data", function () { // ... }); ``` ### Select tests via a test plan file Danger: Test plan is currently not supported by the Allure Newman integration. ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and Node.js version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/js/environment-allure3-light.png, /images/js/environment-allure3-dark.png, /images/js/environment-allure2-light.png, /images/js/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](/docs/jest-reference/#parametrized-tests). --- ## Newman configuration > URL: https://allurereport.org/docs/newman-configuration/ # Allure Newman configuration The [Allure Newman](/docs/newman/) integration behavior can be affected by some configuration options. To pass an option, you can just add it to the command when running Newman. The example below assumes that the tests are [public](https://learning.postman.com/docs/collaborating-in-postman/using-workspaces/manage-workspaces/#change-workspace-visibility). (Alternatively, [use the Postman API](https://github.com/postmanlabs/newman?tab=readme-ov-file#using-newman-with-the-postman-api) to run them.) **npm:** ```bash npx newman run https://www.postman.com/collections/8854915-454a2dc7-dcbe-41cf-9bfa-da544fcd93a2 \ --reporters cli,allure \ --reporter-allure-resultsDir output/allure-results ``` **yarn:** ```bash yarn run newman run https://www.postman.com/collections/8854915-454a2dc7-dcbe-41cf-9bfa-da544fcd93a2 \ --reporters cli,allure \ --reporter-allure-resultsDir allure-results ``` **pnpm:** ```bash pnpx newman run https://www.postman.com/collections/8854915-454a2dc7-dcbe-41cf-9bfa-da544fcd93a2 \ --reporters cli,allure \ --reporter-allure-resultsDir allure-results ``` ## --reporter-allure-resultsDir ⟨DIRECTORY⟩ Path to the directory where Allure Newman will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. --- ## Newman reference > URL: https://allurereport.org/docs/newman-reference/ # Allure Newman reference When collecting data for a test report, the Allure Newman integration analyzes the Newman's native metadata, as well as special commands found in the JavaScript comments. Note that the values in such commands cannot contain spaces. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description If a Postman request has [documentation](https://learning.postman.com/docs/publishing-your-api/authoring-your-documentation/), it will be used as the test's [description](/docs/v3/readability/#description) in Allure Report. ### Owner - `// @allure.label.owner=⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). ```js // @allure.label.owner=JohnDoe pm.test("Test Authentication", function () { // ... }); ``` ### Tag - `// @allure.label.tag=⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). ```js // @allure.label.tag=WebInterface // @allure.label.tag=Authentication pm.test("Test Authentication", function () { // ... }); ``` ### Severity - `// @allure.label.severity=⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. ```js // @allure.label.severity=critical pm.test("Test Authentication", function () { // ... }); ``` ### Label - `// @allure.label.⟨NAME⟩=⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other metadata. You can use this command multiple times to create an array of values under the given name. ```js // @allure.label.mylabel=value pm.test("Test Authentication", function () { // ... }); ``` ### ID - `// @allure.id=⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). ```js // @allure.id=123 pm.test("Test Authentication", function () { // ... }); ``` ## Behavior-based hierarchy - `// @allure.label.epic=⟨VALUE⟩` - `// @allure.label.feature=⟨VALUE⟩` - `// @allure.label.story=⟨VALUE⟩` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). ```js // @allure.label.epic=WebInterface // @allure.label.feature=EssentialFeatures // @allure.label.story=Authentication pm.test("Test Authentication", function () { // ... }); ``` ## Suite-based hierarchy - `// @allure.label.parentSuite=⟨VALUE⟩` - `// @allure.label.suite=⟨VALUE⟩` - `// @allure.label.subSuite=⟨VALUE⟩` By default, Allure Newman uses the folder hierarchy from the Postman collection for the Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). You can override the names of a test's **parent suite**, **suite** and **sub-suite** individually by providing the special commands in JavaScript comments. ```js // @allure.label.parentSuite=WebInterface // @allure.label.suite=EssentialFeatures // @allure.label.subSuite=Authentication pm.test("Test Authentication", function () { // ... }); ``` ## Test steps If a Postman request has multiple `pm.test()` calls in its “Tests” section, Allure will display them as separate [test steps](/docs/steps/). ```js pm.test("Make sure the response is valid JSON", function () { // ... }); pm.test("Compare the response with the expected data", function () { // ... }); ``` --- ## Node.js Test Runner > URL: https://allurereport.org/docs/node-test/ # Getting started with Allure Node.js Test Runner [![Allure Node.js Test Runner npm latest version](https://img.shields.io/npm/v/allure-node-test?style=flat "Allure Node.js Test Runner npm latest version")](https://www.npmjs.com/package/allure-node-test) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Node.js test runner](https://nodejs.org/api/test.html) (`node:test`) tests. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Full functionality, including the Runtime API with unchanged `node:test` imports, requires Node.js 26.1 or later. On older Node.js versions, starting with Node.js 20, the integration works in a reporter-only mode, see the [version note](#node-js-version-requirements) below. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure Node.js Test Runner integration: **npm:** ```bash npm install --save-dev allure-node-test allure-js-commons ``` **yarn:** ```bash yarn add --dev allure-node-test allure-js-commons ``` **pnpm:** ```bash pnpm add -D allure-node-test allure-js-commons ``` Keep `allure-node-test` and `allure-js-commons` on the same version. No configuration files are needed — the integration is activated entirely from the command line when you run the tests. Your test files themselves do not need any changes — keep using `node:test` as usual. ### 2. Run tests Run your Node.js tests with the Allure reporter and the setup preload: ```bash node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` The two command-line options play different roles: - `--test-reporter allure-node-test/reporter` collects test events and writes the Allure result files. This is the only required part. - `--import allure-node-test/setup` enables the Runtime API (`allure-js-commons` calls inside tests) and [test plan filtering](#select-tests-via-a-test-plan-file). Without it, Runtime API calls print a warning and have no effect on the report (the bodies of `step()` calls still run). For convenience, add the commands as scripts to your `package.json`: ```json { "scripts": { "test:allure": "node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter" } } ``` This will save the necessary data into `allure-results` or another directory, according to the [configuration](/docs/node-test-configuration/). If the directory already exists, new files will be added to the existing ones so that a future report will be based on them all. #### Node.js version requirements Warning: The `allure-node-test/setup` preload requires Node.js 26.1 or later and fails fast with a descriptive error on older versions. On older Node.js versions (20 or later), run the reporter without the preload: ```bash node --test --test-reporter allure-node-test/reporter ``` In this reporter-only mode, only the Runtime API and test plan filtering are unavailable. Everything else keeps working: test statuses, the [Metadata API](#writing-tests) (tags in test names), suite hierarchies, automatic output attachments, and all [configuration options](/docs/node-test-configuration/). There is one exception: when the `ALLURE_TESTPLAN_PATH` environment variable points to a [test plan file](#select-tests-via-a-test-plan-file), the preload can be used even on Node.js versions older than 26.1. In that case, it only filters the test execution; Runtime API calls still require Node.js 26.1 or later. #### Keeping the console output When a custom test reporter is specified, Node.js no longer prints its default console output. To keep a console reporter alongside Allure, list several reporters and give each of them a destination: ```bash node --test \ --test-reporter spec --test-reporter-destination stdout \ --test-reporter allure-node-test/reporter --test-reporter-destination stdout ``` The Allure reporter writes its data to the results directory and prints nothing, so its destination value has no visible effect — it is only required because Node.js expects one destination per reporter. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Node.js Test Runner integration extends the standard reporting features of the Node.js test runner by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization, see [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Capture [files and other content](#attach-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure Node.js Test Runner provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. The Runtime API requires running with `--import allure-node-test/setup` on Node.js 26.1 or later. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. The integration will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. The Metadata API works on any supported Node.js version (20 or later), including the reporter-only mode. Your test files keep their native imports — `allure-js-commons` is only used for the reporting functions: ```js import { label, step } from "allure-js-commons"; import assert from "node:assert/strict"; import test from "node:test"; test("signing in", async () => { await label("severity", "critical"); await step("submit form", async () => {}); assert.equal(1 + 1, 2); }); ``` All the usual ways of organizing `node:test` tests are supported: `describe()` suites are mapped to Allure suite labels, and native `t.test()` subtests are reported as separate test results. Other standard `node:test` features are reported as you would expect: - Tests marked with `skip` or `todo` appear in the report as Skipped. - If a native `before()`, `beforeEach()`, `afterEach()`, or `after()` hook fails, the failed hook is preserved in the report as a fixture attached to the affected tests, and the failure is also recorded as an error of the test run as a whole. Tests cancelled by a failed `before()` hook appear as Skipped, a failure in a `beforeEach()` or `afterEach()` hook is reported as the affected test's own status, and a failed `after()` hook changes the statuses of the tests in its suite only if the hook's failure is more severe than their own results. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/node-test-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.displayName("Test Authentication"); await allure.owner("John Doe"); await allure.tags("Web interface", "Authentication"); await allure.severity("critical"); // ... }); ``` **Metadata API:** ```js import test from "node:test"; test( "Test Authentication" + " @allure.label.owner:JohnDoe" + " @allure.label.tag:WebInterface" + " @allure.label.tag:Authentication" + " @allure.label.severity:critical", async () => { // ... }, ); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Metadata API:** ```js import test from "node:test"; test( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```js import test from "node:test"; test( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); ``` By default, the suite hierarchy is built from the `describe()` blocks the test is nested in. For native `t.test()` subtests, the parent test's name is also used as a suite label. Suite labels set via the Runtime API or the Metadata API override the generated ones. ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), you can use the `step()` function, see the [reference](/docs/node-test-reference/#test-steps). ```js import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ### Describe parametrized tests To implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern with the Node.js test runner, write the test inside a loop and use the variable parameters in both the title and the body. Each iteration is treated as a separate test run by Allure Report. To display a parameter value in the test report, pass it to the [`parameter()`](/docs/node-test-reference/#parametrized-tests) function. ```js import * as allure from "allure-js-commons"; import test from "node:test"; for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } ``` ### Set labels globally Any [labels](/docs/node-test-reference/#label), including custom ones, can be set via environment variables in your operating system. Here's an example: **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` ### Attach files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure Node.js Test Runner reference](/docs/node-test-reference/#attachments). ```js import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` Additionally, anything written to the standard output or the standard error stream during a test run is captured and added as text attachments named "stdout" and "stderr", and messages logged with the built-in `t.diagnostic()` are added as a text attachment named "diagnostics". These attachments are added to the test run as a whole rather than to individual tests, because Node.js does not currently include a test identifier in its output events. ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, only the tests listed in this file will be run. Test plan filtering requires the `--import allure-node-test/setup` preload: it skips the tests outside the plan before their bodies execute, and those skips do not appear in the report. In reporter-only mode (without the preload), the test plan is not applied — if tests run, they are reported. When `ALLURE_TESTPLAN_PATH` is set, the preload may be used even on Node.js versions older than 26.1: it then only filters test execution, while Runtime API calls still require Node.js 26.1 or later. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` Note that native `t.test()` subtests only exist while their parent test is running, so they cannot be filtered individually. If a test plan selector points to a subtest, the parent test is run in full, and every test and subtest that actually runs is reported. For this to work, a test plan entry that points to a subtest must include a selector, not just an Allure ID. ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, define the [`environmentInfo`](/docs/node-test-configuration/#environmentinfo) object in the configuration. For example, it is a good idea to record the OS version and the Node.js version. This may help the future reader investigate bugs that are reproducible only in some environments. **MacOS/Linux:** ```bash export ALLURE_NODE_TEST_CONFIG='{"environmentInfo":{"os_platform":"linux","node_version":"26.1.0"}}' node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` **Windows:** ```powershell $Env:ALLURE_NODE_TEST_CONFIG = '{"environmentInfo":{"os_platform":"windows","node_version":"26.1.0"}}' node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` --- ## Node.js Test Runner configuration > URL: https://allurereport.org/docs/node-test-configuration/ # Allure Node.js Test Runner configuration The [Allure Node.js Test Runner](/docs/node-test/) integration behavior can be configured through environment variables. ## Providing configuration Node.js does not pass an options object to custom test reporters, so the configuration is provided through environment variables. Most options are set via the `ALLURE_NODE_TEST_CONFIG` variable, which holds a JSON object: **MacOS/Linux:** ```bash export ALLURE_NODE_TEST_CONFIG='{"resultsDir":"allure-results","globalLabels":{"layer":"api"}}' node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` **Windows:** ```powershell $Env:ALLURE_NODE_TEST_CONFIG = '{"resultsDir":"allure-results","globalLabels":{"layer":"api"}}' node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` An example that combines the supported properties: ```json { "resultsDir": "allure-results", "links": { "issue": { "nameTemplate": "Issue #%s", "urlTemplate": "https://issues.example.com/%s" }, "tms": { "nameTemplate": "TMS #%s", "urlTemplate": "https://tms.example.com/%s" }, "jira": { "urlTemplate": "https://jira.example.com/browse/%s" } }, "categories": [ { "name": "foo", "messageRegex": "bar", "traceRegex": "baz", "matchedStatuses": ["failed", "broken"] } ], "environmentInfo": { "os_platform": "linux", "node_version": "26.1.0" }, "globalLabels": { "layer": "api" } } ``` Because the configuration is JSON, only values that can be represented as JSON are supported. Configuration options that require functions — listeners and link template functions — are not available. ## resultsDir Path to the directory where the test results will be saved, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. You can also set the output directory using the `ALLURE_RESULTS_DIR` environment variable. The `resultsDir` option takes priority over `ALLURE_RESULTS_DIR` if both are set. **MacOS/Linux:** ```bash export ALLURE_RESULTS_DIR=out/allure-results node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` **Windows:** ```powershell $Env:ALLURE_RESULTS_DIR = "out/allure-results" node --test --import allure-node-test/setup --test-reporter allure-node-test/reporter ``` ## links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/node-test-reference/#link)), you can specify the following: - `nameTemplate` — a template for generating the link name when it is not provided. - `urlTemplate` — a template for generating the full link address when the provided value is an identifier rather than a full URL. Both templates must contain `%s` at the position where the identifier should be placed. For example, with the configuration above, `await allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/123`. Allure will display the link with the appropriate icon for the `issue` type. ## categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `description` — a description of the category. - `descriptionHtml` — a description of the category, in HTML format. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. The valid statuses are `"failed"`, `"broken"`, `"passed"`, and `"skipped"`. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). ## environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ## globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ "name": ..., "value": ... }` label objects: ```json { "globalLabels": [ { "name": "layer", "value": "api" }, { "name": "team", "value": "backend" } ] } ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```json { "globalLabels": { "layer": "api", "team": "backend" } } ``` ## Other environment variables Besides `ALLURE_NODE_TEST_CONFIG` and `ALLURE_RESULTS_DIR`, the integration recognizes the following environment variables: - `ALLURE_TESTPLAN_PATH` — a path to a test plan file used to [select which tests to run](/docs/node-test/#select-tests-via-a-test-plan-file). - `ALLURE_LABEL_⟨NAME⟩` — [set a label globally](/docs/node-test/#set-labels-globally) for all tests in the run. --- ## Node.js Test Runner reference > URL: https://allurereport.org/docs/node-test-reference/ # Allure Node.js Test Runner reference These are the functions that you can use to integrate your Node.js test runner tests with Allure. In most cases, Allure Node.js Test Runner provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. The integration will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. Warning: Runtime API requirements The Runtime API only works when the tests are run with the `--import allure-node-test/setup` preload on Node.js 26.1 or later. Without the preload, Runtime API calls print a warning and have no effect on the report (the bodies of `step()` calls still run). The Metadata API works on any supported Node.js version (20 or later), including the reporter-only mode. See [Node.js version requirements](/docs/node-test/#node-js-version-requirements). ## Sync API All functions in this reference are asynchronous and return a `PromiseLike`. If your test uses synchronous helpers or matcher integrations that do not support async, import from `allure-js-commons/sync` instead: ```js import * as allure from "allure-js-commons/sync"; import test from "node:test"; test("Test Authentication", () => { allure.step("check result", () => { allure.parameter("mode", "sync"); }); }); ``` The sync facade is strict-sync only: `step()` must complete synchronously and must not return a `Promise`. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `allure.displayName(name: string): PromiseLike` Set the test's [title](/docs/v3/readability/#title). ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.displayName("Test Authentication!"); // ... }); ``` ### Description - `allure.description(markdown: string): PromiseLike` - `allure.descriptionHtml(html: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Use `description()` for Markdown content. Any HTML formatting, if present, will be stripped for security purposes. Use `descriptionHtml()` when you need to supply raw HTML directly instead of Markdown. ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.description("This test attempts to log into the website."); // ... }); ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.owner("John Doe"); // ... }); ``` **Metadata API:** ```js import test from "node:test"; test("Test Authentication @allure.label.owner:JohnDoe", async () => { // ... }); ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` - `@allure.label.tag:⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.tag("New UI"); await allure.tags("Essentials", "Authentication"); // ... }); ``` **Metadata API:** ```js import test from "node:test"; test("Test Authentication @allure.label.tag:WebInterface @allure.label.tag:Authentication", async () => { // ... }); ``` Additionally, if the running Node.js version supports native test tags, they are reported as Allure tags as well. ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: `"trivial"`, `"minor"`, `"normal"`, `"critical"`, and `"blocker"`. **Runtime API:** ```js import * as allure from "allure-js-commons"; import { Severity } from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.severity(Severity.CRITICAL); // ... }); ``` **Metadata API:** ```js import test from "node:test"; test("Test Authentication @allure.label.severity:critical", async () => { // ... }); ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` - `@allure.label.⟨NAME⟩:⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for many of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.label("microservice", "UI"); // ... }); ``` **Metadata API:** ```js import test from "node:test"; test("Test Authentication @allure.label.microservice:UI", async () => { // ... }); ``` ### ID - `allure.allureId(value: string): PromiseLike` - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.allureId("123"); // ... }); ``` **Metadata API:** ```js import test from "node:test"; test("Test Authentication @allure.id:123", async () => { // ... }); ``` ### History ID - `allure.historyId(value: string): PromiseLike` Override the value that Allure Report uses to find the same test in the [history](/docs/history-and-retries/). Use it if the value computed automatically from the test's full name and parameters does not suit your needs. ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.historyId("auth-test-1"); // ... }); ``` ### Test case ID - `allure.testCaseId(value: string): PromiseLike` Override the value that Allure Report uses to recognize results as belonging to the same test case, for example, when grouping [retries](/docs/history-and-retries/). ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.testCaseId("auth-test-case-1"); // ... }); ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` - `@allure.link.⟨TYPE⟩:⟨VALUE⟩` Add a [link](/docs/v3/readability/#links) related to the test. If the `url` is not a full URL (for example, it is just an issue identifier), Allure will try to construct the full address from a **link template** that corresponds to the `type` (which can be any string and defaults to `"link"`), as defined by the [`links`](/docs/node-test-configuration/#links) configuration option. If no template is found for the given type, or the value is already a full URL, it is left unmodified. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.issue("AUTH-123", "Related issue"); await allure.tms("TMS-456", "Related TMS issue"); await allure.link("JIRA-777", "Related Jira issue", "jira"); await allure.link("https://example.com/", "Project website"); // ... }); ``` **Metadata API:** ```js import test from "node:test"; test("Test Authentication @allure.link.issue:AUTH-123 @allure.link.tms:TMS-456", async () => { // ... }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```js import * as allure from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Metadata API:** ```js import test from "node:test"; test( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). This overrides the corresponding levels of the default suites hierarchy, which Allure Node.js Test Runner creates based on the `describe()` arguments. **Runtime API:** ```js import * as allure from "allure-js-commons"; import { describe, it } from "node:test"; describe("authentication", () => { it("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); }); ``` **Metadata API:** ```js import { describe, it } from "node:test"; describe("authentication", () => { it( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); }); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test step or sub-step](/docs/steps/) with the given `name`. The `step()` function accepts an anonymous function as its second argument, which can be synchronous or asynchronous. The anonymous function can accept either no arguments or a single argument of type `StepContext`. This object provides the following methods: - `displayName()` — override the step name during its execution. - `parameter(name: string, value: string, mode?: "default" | "masked" | "hidden")` — indicate arbitrary parameters used for the step. The optional third argument sets the parameter's display mode, see [Parametrized tests](#parametrized-tests). Step parameters do not support the `excluded` option. To create a step without a body, call the `logStep()` function that accepts a name and an optional step status. ```js import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` Note that Allure steps are different from native `t.test()` subtests: a subtest is reported as a separate test result, while a step is part of its test's result. ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` The [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern in the Node.js test runner can be implemented by running a `test()` inside a loop. Allure Report recognizes each iteration as a separate test run. The values that distinguish one iteration from another are called test parameters. To display a parameter value in the test report, pass it to the `parameter()` function. The `options` argument, if given, must be an object with two optional properties: `excluded` and `mode`. - If `excluded` is set to `true`, Allure will not use the parameter when comparing the current test result with previous ones in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. Note that even when you use the `"masked"` or `"hidden"` mode, it is still possible to extract the value from the `allure-results` directory if you publish it. ```js import * as allure from "allure-js-commons"; import test from "node:test"; for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } ``` ## Attachments - `allure.attachment(name: string, content: Buffer | Uint8Array | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` directly or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options in an object (as shown for the image attachment below) or just specify the media type and let Allure deduce the appropriate filename extension automatically (as shown for the text attachment below). In either case, the media type can be a value from the `ContentType` enumeration or any string. ```js import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import test from "node:test"; test("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` To add an attachment to the test run as a whole instead of the current test result, use the global variants of the functions. They accept the same arguments: - `allure.globalAttachment(name: string, content: Buffer | Uint8Array | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.globalAttachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Similarly, you can report an error that relates to the test run as a whole rather than to a particular test: - `allure.globalError(details: StatusDetails): PromiseLike` The `details` object may contain a `message` and a `trace` string. Additionally, the integration creates some attachments automatically: - everything a test writes to the standard output and the standard error stream is attached as "stdout" and "stderr"; - messages logged with the built-in `t.diagnostic()` are attached as "diagnostics"; - output that cannot be attributed to a particular test is attached to the test run as a whole. --- ## NUnit > URL: https://allurereport.org/docs/nunit/ # Getting started with Allure NUnit [![Allure NUnit latest version](https://img.shields.io/nuget/v/Allure.NUnit?style=flat "Allure NUnit latest version")](https://www.nuget.org/packages/Allure.NUnit) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [NUnit](https://nunit.org/) tests. Info: Check out the example project at [github.com/allure-examples/allure-nunit](https://github.com/allure-examples/allure-nunit) to see Allure NUnit in action. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Make sure that you have a compatible .NET version. Allure NUnit requires a framework that implements [.NET Standard version 2.0](https://learn.microsoft.com/en-us/dotnet/standard/net-standard?tabs=net-standard-2-0#select-net-standard-version). You can find the full list of compatible frameworks [on Allure.NUnit's page in NuGet](https://www.nuget.org/packages/Allure.NUnit#supportedframeworks-body-tab). 1. Add `Allure.NUnit` to your project's dependencies using your IDE or the command line. For example, here is how the dependency can be added to a project with the `dotnet` CLI: ```plain dotnet add ⟨PATH TO PROJECT⟩ package Allure.NUnit ``` 1. Add the `[AllureNUnit]` attribute to all test classes in your project. For example: ```csharp using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { // ... } } ``` Tip: If you have a common base class for all your tests, you can just add the `[AllureNUnit]` attribute to that class. ### 2. Run tests Run your NUnit tests the same way as you would run them usually. For example: ```plain dotnet test ``` This will save necessary data into `allure-results` or other directory, according to the [`allure.directory` setting](/docs/nunit-configuration/#allure-directory). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure NUnit integration extends the standard reporting features of NUnit by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#specify-description-links-and-other-metadata). - **Test Organization**: Structure your tests into [clear hierarchies](#organize-tests) for better readability and organization. - **Test statuses distinction**: Distinguish between [failed assertions and unhandled exceptions](#distinguish-between-test-statuses) with failed and broken statuses. - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Test Fixtures**: Include the [setup and teardown methods](#describe-test-fixtures) into the report. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure NUnit provides two different ways to use a feature: the Attributes API and the Runtime API. - **Attributes API**: add a C# attribute to a test method or a whole class to add certain data to the test result. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/nunit-reference/#metadata) for more details. **Attributes API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] [AllureSeverity(SeverityLevel.critical)] [AllureOwner("John Doe")] [AllureLink("Website", "https://dev.example.com/")] [AllureIssue("UI-123")] [AllureTms("TMS-456")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.SetSeverity(SeverityLevel.critical); AllureApi.SetOwner("John Doe"); AllureApi.AddLink("Website", "https://dev.example.com/"); AllureApi.AddIssue("UI-123"); AllureApi.AddTmsItem("TMS-456"); // ... } } ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure NUnit provides the API to assign the relevant fields to tests either by adding attributes or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] [AllureEpic("Web interface")] [AllureFeature("Essential features")] class TestLabels { [Test] [AllureStory("Labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.AddEpic("Web interface"); AllureApi.AddFeature("Essential features"); AllureApi.AddStory("Labels"); // ... } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] [AllureParentSuite("Web interface")] [AllureSuite("Essential features")] class TestLabels { [Test] [AllureSubSuite("Labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.AddParentSuite("Web interface"); AllureApi.AddSuite("Essential features"); AllureApi.AddSubSuite("Labels"); // ... } } ``` A test's location in the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy) is defined by the fully qualified names of the classes they are declared in, with common prefixes shown as parent packages. ### Distinguish between test statuses A test may be interrupted for various reasons. Allure NUnit reflects that by assigning an appropriate [test status](/docs/test-statuses/): - **Failed** for a failed assertion, e.g., [`Assert.That`](https://docs.nunit.org/articles/nunit/writing-tests/assertions/assertion-models/constraint.html), - **Skipped** for failed assumptions, e.g., [`Assume.That`](https://docs.nunit.org/articles/nunit/writing-tests/Assumptions.html), - **Broken** for unhandled exceptions. Note that if the test uses techniques like [`Assert.Multiple`](https://docs.nunit.org/articles/nunit/writing-tests/assertions/multiple-asserts.html), it will not necessarily stop after the first `Assert.That` failure. This means that it may end up with a list of multiple failed assertions. For such a test, Allure NUnit will still assign the **Failed** status if the failed assertions are the only reason the test did not succeed, but **Broken** if it eventually stopped due to an unhandled exception. If you want your test with unhandled exceptions to be **Failed** instead of **Broken**, there are multiple approaches to achieve that. For example, in an [`Assert.That`](https://docs.nunit.org/articles/nunit/writing-tests/assertions/assertion-models/constraint.html), make sure that evaluating the first argument (e.g., accessing an element of a list) cannot cause an exception. If necessary, move some logic to a [custom constraint](https://docs.nunit.org/articles/nunit/extending-nunit/Custom-Constraints.html) to avoid repetitive assertions in your code. Alternatively, consider wrapping the code that may throw in some construct that takes care of exceptions. It can be an assertion or assumption with the `Throws.Nothing` constraint or just a `try..catch` block. **Failed (with Throws.Nothing):** ```csharp using System; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] public class TestThrowsNothing { [Test] public void ThisTestWillBeFailed() { string value = GetValueOrFail(); Assert.That(value, Is.EqualTo("foo")); } static string GetValueOrFail() { string value = default; Assert.That(() => { value = GetValue(); }, Throws.Nothing); return value; } static string GetValue() { throw new Exception(); } } ``` **Failed (with try..catch):** ```csharp using System; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] public class TestTryCatch { [Test] public void ThisTestWillBeFailed() { string value = GetValueOrFail(); Assert.That(value, Is.EqualTo("foo")); } static string GetValueOrFail() { string value = default; try { value = GetValue(); } catch (Exception e) { Assert.Fail(e.Message); } return value; } static string GetValue() { throw new Exception(); } } ``` **Broken test:** ```csharp using System; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] public class TestBroken { [Test] public void ThisTestWillBeBroken() { string value = GetValue(); Assert.That(value, Is.EqualTo("foo")); } static string GetValue() { throw new Exception(); } } ``` Info: When an exception is thrown from within a [step](/docs/nunit-reference/#test-steps), the step status is selected according to the [`allure.failExceptions`](/docs/nunit-configuration/#allure-failexceptions) setting. ### Divide a test into steps Allure NUnit provides three ways of [creating steps and sub-steps](/docs/steps/): “attribute-based steps”, “lambda steps” and “no-op steps”, see the [reference](/docs/nunit-reference/#test-steps). **Attribute-based steps:** ```csharp using System; using Allure.Net.Commons; using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { TestDomain("https://domain1.example.com/"); TestDomain("https://domain2.example.com/"); } [AllureStep("Test {url}")] private void TestDomain([Name("Webpage URL")] string url) { AllureLifecycle.Instance.UpdateStep(stepResult => stepResult.parameters.Add( new Parameter { name = "Started at", value = DateTime.Now.ToString() } ) ); OpenBrowser(); GoToWebpage(url); CloseBrowser(); } [AllureStep("Open web browser")] private void OpenBrowser() { // ... } [AllureStep("Visit {url}")] private void GoToWebpage([Skip] string url) { // ... } [AllureStep("Close web browser")] private void CloseBrowser() { // ... } } ``` **Lambda steps and no-op steps:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestMyWebsite { [Test] public void TestVisitPages() { string[] urls = { "https://domain1.example.com/", "https://domain2.example.com/" }; foreach (string url in urls) { AllureApi.Step($"Test {url}", () => { AllureLifecycle.Instance.UpdateStep(stepResult => { stepResult.parameters.Add( new Parameter { name = "Webpage URL", value = url } ); }); AllureApi.Step("Opening web browser..."); // ... AllureApi.Step($"Visiting {url}..."); // ... AllureApi.Step("Closing web browser..."); // ... }); } } } ``` ### Describe test fixtures By default, a test report does not include information about [fixtures](https://docs.nunit.org/articles/nunit/writing-tests/setup-teardown/index.html) in NUnit test classes. To include it, add the `[AllureBefore]` and `[AllureAfter]` annotations to the fixture methods. The methods will be displayed as special kinds of [steps](#divide-a-test-into-steps). **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] public class TestLabels { [OneTimeSetUp] [AllureBefore("Start the test server")] public void StartServer() { // ... } [SetUp] [AllureBefore("Open the browser")] public void StartBrowser() { // ... } [Test] public void TestCreateLabel() { // ... } [TearDown] [AllureAfter("Close the browser")] public void CloseBrowser() { // ... } [OneTimeTearDown] [AllureAfter("Shutdown the test server")] public void StopServer() { // ... } } ``` **Attributes API + Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] public class TestFixtures { [OneTimeSetUp] [AllureBefore] public void StartServer() { AllureApi.SetFixtureName("Start the test server"); // ... } [SetUp] [AllureBefore] public void StartBrowser() { AllureApi.SetFixtureName("Open the browser"); // ... } [Test] public void Test_SetUp() { // ... } [TearDown] [AllureAfter] public void CloseBrowser() { AllureApi.SetFixtureName("Close the browser"); // ... } [OneTimeTearDown] [AllureAfter] public void StopServer() { AllureApi.SetFixtureName("Shutdown the test server"); // ... } } ``` ### Describe parametrized tests When using the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, use the NUnit's attributes or the `AddTestParameter()` function, see the [reference](/docs/nunit/#parametrized-tests). **Attributes API:** ```csharp using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [TestCase("johndoe", "qwerty")] [TestCase("johndoe@example.com", "qwerty")] public void TestAuthentication(string login, string password) { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestAuthenticaton { [Test] public void TestAuthenticationWithUsername() { AllureApi.AddTestParameter("login", "johndoe"); AllureApi.AddTestParameter("password", "qwerty", ParameterMode.Masked); // ... } [Test] public void TestAuthenticationWithEmail() { AllureApi.AddTestParameter("login", "johndoe@example.com"); AllureApi.AddTestParameter("password", "qwerty", ParameterMode.Masked); // ... } } ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure NUnit provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/nunit-reference/#attachments). ```csharp using System.IO; using System.Text; using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { // ... AllureApi.AddAttachment( "data.txt", "text/plain", Encoding.UTF8.GetBytes("This is the file content.") ); AllureApi.AddAttachment( "image1.png", "image/png", File.ReadAllBytes("/path/to/image1.png") ); AllureApi.AddAttachment( "image2.png", "image/png", "/path/to/image2.png" ); } } ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, NUnit will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json dotnet test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" dotnet test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and .NET version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/csharp/environment-allure3-light.png, /images/csharp/environment-allure3-dark.png, /images/csharp/environment-allure2-light.png, /images/csharp/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## NUnit configuration > URL: https://allurereport.org/docs/nunit-configuration/ # Allure NUnit configuration This page describes the configuration options that affect the behavior of [Allure NUnit](/docs/nunit/). By default, Allure NUnit tries to read the `allureConfig.json` file in the same directory where the test assembly is located. To make sure the file is present there, edit the file properties in your IDE or manually in your `*.csproj` file and set the corresponding item's `CopyToOutputDirectory` to either `Always` or `PreserveNewest`. For example: ```xml Always ``` Alternatively, use the `ALLURE_CONFIG` environment variable to specify a relative or absolute path of the configuration file. ## allure.directory Path to the directory where Allure NUnit will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. Note that the path is treated as relative to the build output directory of the project. When building and running tests from an IDE, this is typically different from the solution folder. Multiple `../` elements may be needed if you want to put the test results into a directory in the solution folder. ```json { "allure": { "directory": "../../../../../allure-results" } } ``` ## allure.links List of patterns that can be used to construct full URLs from short identifiers. Each pattern must be a string containing a placeholder surrounded by curly brackets, for example: `{issue}`. When processing a link, Allure NUnit selects the pattern whose placeholder matches the link type (as specified using the [`Link` or `[AllureLink]`](/docs/nunit-reference/#link) arguments). The placeholder is then replaced by the short identifier. For example, with the configuration below, the link `BUG-123` of type `issue` will be translated to `https://issues.example.com/BUG-123`. ```json { "allure": { "links": [ "https://issues.example.com/{issue}", "https://tms.example.com/{tms}", "https://jira.example.com/browse/{jira}" ] } } ``` ## allure.failExceptions A list of exception types which, when thrown from a [step](/docs/nunit-reference/#test-steps), should cause it to have the [Failed](/docs/test-statuses/#failed) status. In case of any other exception, the [Broken](/docs/test-statuses/#broken) status is assigned to the step. Info: Currently, the setting only affects the statuses of individual test steps. For the status of a test as a whole, see [Distinguish between test statuses](/docs/nunit/#distinguish-between-test-statuses). The default list includes the `NUnit.Framework.Internal.NUnitException` and `NUnit.Framework.AssertionException` classes. The setting, if not empty, overrides the default list. When looking for matches, Allure NUnit uses not only the exception class's own full name but also all its interfaces and base classes. With the example configuration below, a step will be considered failed if it throws `NUnit.Framework.AssertionException`, `MyNamespace.MyCustomException`, or anything that derives from them. At the same time, any other exception will make the step broken. ```json { "allure": { "failExceptions": ["NUnit.Framework.AssertionException", "MyNamespace.MyCustomException"] } } ``` --- ## NUnit reference > URL: https://allurereport.org/docs/nunit-reference/ # Allure NUnit reference These are the attributes and methods that you can use to integrate your NUnit tests with Allure Report. In most cases, Allure NUnit provides two different ways to use a feature: the Attributes API and the Runtime API. - **Attributes API**: add a C# attribute to a test method or a whole class to add certain data to the test result. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `[AllureName(string name)]` - `AllureApi.SetTestName(string name)` Set the test's [title](/docs/v3/readability/#title). **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] [AllureName("Create labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.SetTestName("Create labels"); // ... } } ``` ### Description - `[AllureDescription(string description)]` - `AllureApi.SetDescription(string description)` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] [AllureDescription("This test attempts to create a label with specified title")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.SetDescription("This test attempts to create a label with specified title"); // ... } } ``` ### Owner - `[AllureOwner(string value)]` - `AllureApi.SetOwner(string owner)` Set the test's [owner](/docs/v3/readability/#owner). **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] [AllureOwner("John Doe")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.SetOwner("John Doe"); // ... } } ``` ### Tag - `[AllureTag(params string[] tags)]` - `AllureApi.AddTags(params string[] tags)` Set the test's [tags](/docs/v3/readability/#tags). **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] [AllureTag("UI", "Labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.AddTags("UI", "Labels"); // ... } } ``` ### Severity - `[AllureSeverity(SeverityLevel value)]` - `AllureApi.SetSeverity(SeverityLevel severity)` Set the test's [severity](/docs/v3/readability/#severity). **Attributes API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] [AllureSeverity(SeverityLevel.critical)] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.SetSeverity(SeverityLevel.critical); // ... } } ``` ### Label - `[AllureLabel(string label, string value, bool overwrite=false)]` - `AllureApi.AddLabel(string name, string value)` - `AllureApi.AddLabel(Label label)` - `AllureApi.AddLabels(params Label[] labels)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other APIs. **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] [AllureLabel("MyCustomLabel", "value")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.AddLabel("MyCustomLabel", "value"); // ... } } ``` ### ID - `[AllureId(string value)]` Set the test's [ID](/docs/v3/readability/#id). ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] [AllureId(123)] public void TestCreateLabel() { // ... } } ``` ### Link - `[AllureLink(string name, string url)]` - `[AllureIssue(string name)]` - `[AllureIssue(string name, string url)]` - `[AllureTms(string name)]` - `[AllureTms(string name, string url)]` - `AllureApi.AddLink(string url)` - `AllureApi.AddLink(string name, string url)` - `AllureApi.AddLink(string name, string type, string url)` - `AllureApi.AddLinks(params Link[] links)` - `AllureApi.AddIssue(string url)` - `AllureApi.AddIssue(string name, string url)` - `AllureApi.AddTmsItem(string url)` - `AllureApi.AddTmsItem(string name, string url)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`links`](/docs/nunit-configuration/#links) configuration option. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. For convenience, Allure provides shorthand attributes and methods with pre-selected link types: `issue` and `tms`. **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] [AllureLink("Website", "https://dev.example.com/")] [AllureIssue("UI-123")] [AllureTms("TMS-456")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.AddLink("Website", "https://dev.example.com/"); AllureApi.AddIssue("UI-123"); AllureApi.AddTmsItem("TMS-456"); // ... } } ``` ## Behavior-based hierarchy - `[AllureEpic(string epic)]` - `[AllureFeature(params string[] feature)]` - `[AllureStory(params string[] story)]` - `AllureApi.AddEpic(string epic)` - `AllureApi.AddFeature(string feature)` - `AllureApi.AddStory(string story)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] [AllureEpic("Web interface")] [AllureFeature("Essential features")] class TestLabels { [Test] [AllureStory("Labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.AddEpic("Web interface"); AllureApi.AddFeature("Essential features"); AllureApi.AddStory("Labels"); // ... } } ``` ## Suite-based hierarchy - `[AllureParentSuite(string parentSuite)]` - `[AllureSuite(string suite)]` - `[AllureSubSuite(string subSuite)]` - `AllureApi.AddParentSuite(string parentSuite)` - `AllureApi.AddSuite(string suite)` - `AllureApi.AddSubSuite(string subSuite)` Assign the name of suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] [AllureParentSuite("Web interface")] [AllureSuite("Essential features")] class TestLabels { [Test] [AllureSubSuite("Labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { AllureApi.AddParentSuite("Web interface"); AllureApi.AddSuite("Essential features"); AllureApi.AddSubSuite("Labels"); // ... } } ``` ## Test steps - `[AllureStep]` - `[AllureStep(string name)]` - `AllureApi.Step(string name)` - `AllureApi.Step(string name, Action acti` - `T AllureApi.Step(string name, Func function)` - `async Task AllureApi.Step(string name, Func action)` - `async Task AllureApi.Step(string name, Func> function)` Define [test steps](/docs/steps/). There are three ways of defining a step. - **Attribute-based steps** Define a method containing a test step and add the `[AllureStep]` attribute to it, with an optional `name` argument (defaults to the function's name). Each time the function is called during the test execution, a new step will be created for the test report. If the function calls another function that also has the `[AllureStep]` attribute, Allure Report will create a sub-step inside the current step. If the function throws an exception, the step becomes **Failed** or **Broken** according to the [`failExceptions`](/docs/nunit-configuration/#allure-failexceptions) configuration parameter. If the step name contains placeholders (e.g., `{url}` or `{0}`), they will be replaced with the values of parameters passed to the function. Arguments passed to the function will be added to the list of **step parameters**. They are rendered the same way as described in [Parametrized tests](#parametrized-tests), but displayed in the current step's subtree in the report. Use the `[Name(string name)]` attribute to customize a parameter's name, as shown in the example below. Use the `[Skip]` attribute to not add a certain parameter to the report. Use `AllureLifecycle.Instance.UpdateStep()` to add custom parameters to the current step, as shown in the example below. - **Lambda steps** Write a test step in a lambda function and pass it to `Step()`. If the lambda function returns a value, `Step()` will return it without modification, and it will not affect the report. If the function throws an exception, the step becomes **Failed** or **Broken** according to the [`failExceptions`](/docs/nunit-configuration/#allure-failexceptions) configuration parameter. Use `AllureLifecycle.Instance.UpdateStep()` to add custom parameters to the current step, as shown in the example below. - **No-op steps** If you call `Step()` with a name but without a lambda function, Allure will add to the report a no-op step. This allows for a log-style reporting within a test or within a larger step. A no-op step finishes immediately after it started and cannot have any sub-steps, parameters or attachments. **Attribute-based steps:** ```csharp using System; using Allure.Net.Commons; using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestVisitPages() { TestDomain("https://domain1.example.com/"); TestDomain("https://domain2.example.com/"); } [AllureStep("Test {url}")] private void TestDomain([Name("Webpage URL")] string url) { AllureLifecycle.Instance.UpdateStep(stepResult => stepResult.parameters.Add( new Parameter { name = "Started at", value = DateTime.Now.ToString() } ) ); OpenBrowser(); GoToWebpage(url); CloseBrowser(); } [AllureStep("Open web browser")] private void OpenBrowser() { // ... } [AllureStep("Visit {url}")] private void GoToWebpage([Skip] string url) { // ... } [AllureStep("Close web browser")] private void CloseBrowser() { // ... } } ``` **Lambda steps and no-op steps:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestMyWebsite { [Test] public void TestVisitPages() { string[] urls = { "https://domain1.example.com/", "https://domain2.example.com/" }; foreach (string url in urls) { AllureApi.Step($"Test {url}", () => { AllureLifecycle.Instance.UpdateStep(stepResult => { stepResult.parameters.Add( new Parameter { name = "Webpage URL", value = url } ); }); AllureApi.Step("Opening web browser..."); // ... AllureApi.Step($"Visiting {url}..."); // ... AllureApi.Step("Closing web browser..."); // ... }); } } } ``` ## Test fixtures - `[AllureBefore(string name = null)]` - `[AllureAfter(string name = null)]` - `AllureApi.SetFixtureName(string newName)` To add information about setup and teardown methods (see the [NUnit's documentation](https://docs.nunit.org/articles/nunit/writing-tests/setup-teardown/index.html)), use the `[AllureBefore]` or `[AllureAfter]` attributes. The methods will be displayed in the test report as special kinds of [steps](#test-steps). By default, Allure will use the method names as the step names. To provide a more readable name for a ficture, pass a string to the attribute or call `AllureApi.SetFixtureName()` inside the methods. **Attributes API:** ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] public class TestLabels { [OneTimeSetUp] [AllureBefore("Start the test server")] public void StartServer() { // ... } [SetUp] [AllureBefore("Open the browser")] public void StartBrowser() { // ... } [Test] public void TestCreateLabel() { // ... } [TearDown] [AllureAfter("Close the browser")] public void CloseBrowser() { // ... } [OneTimeTearDown] [AllureAfter("Shutdown the test server")] public void StopServer() { // ... } } ``` **Attributes API + Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] public class TestFixtures { [OneTimeSetUp] [AllureBefore] public void StartServer() { AllureApi.SetFixtureName("Start the test server"); // ... } [SetUp] [AllureBefore] public void StartBrowser() { AllureApi.SetFixtureName("Open the browser"); // ... } [Test] public void Test_SetUp() { // ... } [TearDown] [AllureAfter] public void CloseBrowser() { AllureApi.SetFixtureName("Close the browser"); // ... } [OneTimeTearDown] [AllureAfter] public void StopServer() { AllureApi.SetFixtureName("Shutdown the test server"); // ... } } ``` ## Parametrized tests - `AllureApi.AddTestParameter(string name, object? value)` - `AllureApi.AddTestParameter(string name, object? value, ParameterMode mode)` - `AllureApi.AddTestParameter(string name, object? value, bool excluded)` - `AllureApi.AddTestParameter(string name, object? value, ParameterMode mode, bool excluded)` - `AllureApi.AddTestParameter(Parameter parameter)` - `AllureLifecycle.Instance.AddTypeFormatter(TypeFormatter typeFormatter)` To support the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, Allure Report automatically detects the parameters specified via NUnit's [`[TestCase()]`](https://docs.nunit.org/articles/nunit/technical-notes/usage/Parameterized-Tests.html) and similar attributes. In addition to this, the Runtime API can be used for adding the parameters even to functions without the NUnit's attributes. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. This argument is only used by Allure TestOps. The `mode` argument affects how the parameter will be displayed in the report. Available options are defined in the `ParameterMode` enumeration: - `ParameterMode.Default` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `ParameterMode.Masked` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `ParameterMode.Hidden` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. By default, Allure Report will render the parameter values via a basic JSON serialization. You can override this behavior for certain types by implementing a custom class that inherits from `TypeFormatter` and passing an object of this class to `AddTypeFormatter()`. The parameters can be added not only to whole test results, but also to individual steps within them, see [Test steps](#test-steps) for more details. **Attributes API:** ```csharp using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [TestCase("johndoe", "qwerty")] [TestCase("johndoe@example.com", "qwerty")] public void TestAuthentication(string login, string password) { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestAuthenticaton { [Test] public void TestAuthenticationWithUsername() { AllureApi.AddTestParameter("login", "johndoe"); AllureApi.AddTestParameter("password", "qwerty", ParameterMode.Masked); // ... } [Test] public void TestAuthenticationWithEmail() { AllureApi.AddTestParameter("login", "johndoe@example.com"); AllureApi.AddTestParameter("password", "qwerty", ParameterMode.Masked); // ... } } ``` ## Attachments - `AllureApi.AddAttachment(string name, string type, string path)` - `AllureApi.AddAttachment(string name, string type, byte[] content, string fileExtension = "")` - `AllureApi.AddAttachment(string path, string? name = null)` - `AllureApi.AddScreenDiff(string expectedPng, string actualPng, string diffPng)` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Tip: You can use data produced by any function, not necessarily read from an actual file. To create an attachment using the Runtime API, call `AddAttachment()` at any point during your test. Pass either the `content` or the `path` from which the data will be read. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension as `fileExtension`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. ```csharp using System.IO; using System.Text; using Allure.Net.Commons; using Allure.NUnit; using NUnit.Framework; [AllureNUnit] class TestLabels { [Test] public void TestCreateLabel() { // ... AllureApi.AddAttachment( "data.txt", "text/plain", Encoding.UTF8.GetBytes("This is the file content.") ); AllureApi.AddAttachment( "image1.png", "image/png", File.ReadAllBytes("/path/to/image1.png") ); AllureApi.AddAttachment( "image2.png", "image/png", "/path/to/image2.png" ); } } ``` Allure NUnit also provides a dedicated `AddScreenDiff()` function for attaching [visual comparisons](/docs/attachments/#attaching-visual-comparisons). ## Displaying ignored tests - `[AllureDisplayIgnored]` By default, the test report will not include the tests that were skipped due to NUnit's [`[Ignore]`](https://docs.nunit.org/articles/nunit/writing-tests/attributes/ignore.html) attribute. This behavior can be changed by adding the `[AllureDisplayIgnored]` attribute to the test class. If you have a common base class for all your tests, you can just add the `[AllureDisplayIgnored]` attribute to that class. The ignored tests will have the [Skipped](/docs/test-statuses/#skipped) status in the test report. ```csharp using Allure.NUnit; using Allure.NUnit.Attributes; using NUnit.Framework; [AllureNUnit] [AllureDisplayIgnored] class TestLabels { [Test] [Ignore("not ready yet")] public void TestCreateLabel() { // ... } } ``` --- ## PHPUnit > URL: https://allurereport.org/docs/phpunit/ # Getting started with Allure PHPUnit [![Allure PHPUnit latest version](https://img.shields.io/packagist/v/allure-framework/allure-phpunit?style=flat "Allure PHPUnit latest version")](https://packagist.org/packages/allure-framework/allure-phpunit) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [PHPUnit](https://phpunit.de/) tests. Info: Check out the example project at [github.com/allure-examples/phpunit](https://github.com/allure-examples/phpunit) to see Allure PHPUnit in action. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Make sure that your project uses: - PHP version 8.1, 8.2 or 8.3, - PHPUnit version 10. 1. Add `allure-framework/allure-phpunit` to your project's development dependencies: ```plain php composer.phar require allure-framework/allure-phpunit --dev ``` 1. Enable the Allure extension for PHPUnit via a `` tag in [`phpunit.xml`](https://docs.phpunit.de/en/10.5/configuration.html). In the `config` parameter, specify a path where you will put the configuration file for Allure PHPUnit. For example: ```xml ``` 1. Create an Allure PHPUnit configuration file at the specified location (`config/allure.config.php` in the example above). The file must return an associative array containing [configuration options](/docs/phpunit-configuration/). For example: ```php 'build/allure-results', ]; ``` ### 2. Run tests Run your PHPUnit tests the same way as you would run them usually. For example: ```plain vendor/bin/phpunit ``` This will save necessary data into `build/allure-results` or other directory, according to the [`outputDirectory`](/docs/phpunit-configuration/#outputdirectory) setting. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure PHPUnit integration extends the standard reporting features of PHPUnit by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#specify-description-links-and-other-metadata). - **Test Organization**: Structure your tests into [clear hierarchies](#organize-tests) for better readability and organization. - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure PHPUnit provides two different ways to use a feature: the Attributes API and the Runtime API. - **Attributes API**: add a PHP attribute to a test method or a whole class to add certain data to the test result. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/phpunit-reference/#metadata) for more details. **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\Description; use Qameta\Allure\Attribute\DisplayName; use Qameta\Allure\Attribute\Issue; use Qameta\Allure\Attribute\Link; use Qameta\Allure\Attribute\Owner; use Qameta\Allure\Attribute\Severity; use Qameta\Allure\Attribute\TmsLink; final class TestMyWebsite extends TestCase { #[DisplayName('Test Labels')] #[Description('This test attempts to create a label with specified title.')] #[Severity(Severity::CRITICAL)] #[Owner('John Doe')] #[Link('My Website', 'https://example.com/')] #[Issue('UI-123')] #[TmsLink('TMS-456')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; use Qameta\Allure\Model\Severity; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::displayName('Test Labels'); Allure::description('This test attempts to create a label with specified title.'); Allure::severity(Severity::critical()); Allure::owner('John Doe'); Allure::link('My Website', 'https://example.com/'); Allure::issue('UI-123'); Allure::tms('TMS-456'); // ... } } ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure PHPUnit provides the API to assign the relevant fields to tests either by adding attributes or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\Epic; use Qameta\Allure\Attribute\Feature; use Qameta\Allure\Attribute\Story; final class TestMyWebsite extends TestCase { #[Epic('Web interface')] #[Feature('Essential features')] #[Story('Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::epic('Web interface'); Allure::feature('Essential features'); Allure::story('Labels'); // ... } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\ParentSuite; use Qameta\Allure\Attribute\SubSuite; use Qameta\Allure\Attribute\Suite; final class TestMyWebsite extends TestCase { #[ParentSuite('Web interface')] #[Suite('Essential features')] #[SubSuite('Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::parentSuite('Web interface'); Allure::suite('Essential features'); Allure::subSuite('Labels'); // ... } } ``` ### Divide a test into steps Allure PHPUnit provides three ways of [creating steps and sub-steps](/docs/steps/): “method-based steps”, “lambda steps” and “no-op steps”, see the [reference](/docs/phpunit-reference/#test-steps). **Method-based steps:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; use Qameta\Allure\Attribute\DisplayName; use Qameta\Allure\StepContextInterface; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::runStep([$this, 'logIn']); Allure::runStep([$this, 'createLabel']); Allure::runStep([$this, 'checkThatLabelExists']); } #[DisplayName('Log in')] function logIn(StepContextInterface $context) { $context->parameter('Email', 'johndoe@example.com'); $context->parameter('Password', 'qwerty'); // ... } #[DisplayName('Create label')] function createLabel(StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... } #[DisplayName('Check that label exists')] function checkThatLabelExists(StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... } } ``` **Lambda steps:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; use Qameta\Allure\StepContextInterface; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::runStep(function (StepContextInterface $context) { $context->name('Log in'); $context->parameter('Email', 'johndoe@example.com'); $context->parameter('Password', 'qwerty'); // ... }); Allure::runStep(function (StepContextInterface $context) { $context->name('Create label'); $context->parameter('Label name', 'My Label'); // ... }); Allure::runStep(function (StepContextInterface $context) { $context->name('Check that label exists'); $context->parameter('Label name', 'My Label'); // ... }); } } ``` **No-op steps:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { // ... Allure::addStep('Log in'); // ... Allure::addStep('Create label'); // ... Allure::addStep('Check that label exists'); } } ``` ### Describe parametrized tests If you use the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, call the `Allure::parameter()` function to add the parameters to the test report, see the [reference](/docs/phpunit-reference/#parametrized-tests). **DataProvider:** ```php use PHPUnit\Framework\Attributes\DataProvider; use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public static function credentialsProvider() { return [ ['johndoe', 'qwerty'], ['johndoe@example.com', 'qwerty'], ]; } #[DataProvider('credentialsProvider')] public function testAuthentication(string $login, string $password) { Allure::parameter('login', $login); Allure::parameter('password', $password); // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testAuthenticationWithUsername() { Allure::parameter('login', 'johndoe'); Allure::parameter('password', 'qwerty'); // ... } public function testAuthenticationWithEmail() { Allure::parameter('login', 'johndoe@example.com'); Allure::parameter('password', 'qwerty'); // ... } } ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure PHPUnit provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/phpunit-reference/#attachments). **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { // ... Allure::attachment('data.txt', 'This is the file content.', 'text/plain'); Allure::attachmentFile('data.txt', '/path/to/image.png', 'image/png'); } } ``` ### Select tests via a test plan file Danger: Test plan is currently not supported by the Allure PHPUnit integration. ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and PHP version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/php/environment-allure3-light.png, /images/php/environment-allure3-dark.png, /images/php/environment-allure2-light.png, /images/php/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## PHPUnit configuration > URL: https://allurereport.org/docs/phpunit-configuration/ # Allure PHPUnit configuration This page describes the configuration options that can be set in the configuration file for [Allure PHPUnit](/docs/phpunit/). By default, the configuration file is looked up at `config/allure.config.php`, but this location can be overridden, see [Prepare your project](/docs/phpunit/#1-prepare-your-project). ## outputDirectory Path to the directory where Allure PHPUnit will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `build/allure-results`. ```php 'build/allure-results', ]; ``` ## linkTemplates A mapping of templates that can be used to construct full URLs from short identifiers, see the [reference](/docs/phpunit-reference/#link). Each key must be a link type identifier, and each value must be a class implementing the `LinkTemplateInterface` interface, or an object of such a class. When processing a link of the given type, Allure PHPUnit will call the `buildUrl()` method of the class. ```php [ 'issue' => new class implements LinkTemplateInterface { public function buildUrl(?string $name): ?string { return "https://issues.example.com/$name"; } }, 'tms' => new class implements LinkTemplateInterface { public function buildUrl(?string $name): ?string { return "https://tms.example.com/$name"; } }, 'jira' => new class implements LinkTemplateInterface { public function buildUrl(?string $name): ?string { return "https://jira.example.com/browse/$name"; } }, ], ]; ``` ## setupHook A function that will be called before initializing the Allure PHPUnit lifecycle. ```php function () { // ... }, ]; ``` ## lifecycleHooks An array of one or more classes, each implementing one or more [lifecycle hook interfaces](https://github.com/allure-framework/allure-php-commons2/tree/main/src/Hook) for Allure PHPUnit. For example, such a class can modify test results before or after Allure PHPUnit's standard processing. ```php [ new class implements BeforeTestStartHookInterface, AfterTestStopHookInterface { public function beforeTestStart(TestResult $test): void { // ... } public function afterTestStop(TestResult $test): void { // ... } }, ], ]; ``` ## threadDetector A class implementing the `ThreadDetectorInterface` interface, or an object of such a class. If set, Allure PHPUnit will use its `getHost()` and `getThread()` methods to determine the host and thread identifiers for a current test. The test results will be placed on the [Timeline tab](/docs/timeline/) accordingly. The example below assumes that you use [pthreads](https://www.php.net/manual/en/book.pthreads.php) for running your PHPUnit tests in parallel. ```php new class implements ThreadDetectorInterface { function getHost(): ?string { return gethostname(); } function getThread(): ?string { return strval(Thread::getCurrentThread() . getThreadId()); } }, ]; ``` --- ## PHPUnit reference > URL: https://allurereport.org/docs/phpunit-reference/ # Allure PHPUnit reference These are the attributes and methods that you can use to integrate your PHPUnit tests with Allure Report. In most cases, Allure PHPUnit provides two different ways to use a feature: the Attributes API and the Runtime API. - **Attributes API**: add a PHP attribute to a test method or a whole class to add certain data to the test result. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `#[DisplayName(string $value)]` - `Allure::displayName(string $name)` Set the test's [title](/docs/v3/readability/#title). **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\DisplayName; final class TestMyWebsite extends TestCase { #[DisplayName('Test Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::displayName('Test Labels'); // ... } } ``` ### Description - `#[Description(string $value)]` - `Allure::description(string $description)` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\Description; final class TestMyWebsite extends TestCase { #[Description('This test attempts to create a label with specified title.')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::description('This test attempts to create a label with specified title.'); // ... } } ``` ### Owner - `#[Owner(string $value)]` - `Allure::owner(string $value)` Set the test's [owner](/docs/v3/readability/#owner). **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\Owner; final class TestMyWebsite extends TestCase { #[Owner('John Doe')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::owner('John Doe'); // ... } } ``` ### Tag - `#[Tag(string $value)]` - `Allure::tag(string $value)` Set the test's [tags](/docs/v3/readability/#tags). **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\Tag; final class TestMyWebsite extends TestCase { #[Tag('UI'), Tag('Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::tag('UI'); Allure::tag('Labels'); // ... } } ``` ### Severity - `#[Severity(string $value)]` - `Allure::severity(Severity $value)` Set the test's [severity](/docs/v3/readability/#severity). For the Attributes API, use the constants from the `Severity` class. For the Runtime API, use the static methods on the `Severity` class to construct the values. **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\Severity; final class TestMyWebsite extends TestCase { #[Severity(Severity::CRITICAL)] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; use Qameta\Allure\Model\Severity; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::severity(Severity::critical()); // ... } } ``` ### Label - `#[Label(string $name, ?string $value = null)]` - `Allure::label(string $name, string $value)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other APIs. You may also extend the Attributes API by defining your own attribute that will work as a `#[Label]` with a predefined `$name`. **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\Label; final class TestMyWebsite extends TestCase { #[Label('MyCustomLabel', 'value')] public function testLabels() { // ... } } ``` **Attributes API with a custom attribute:** ```php use Attribute; use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\AbstractLabel; #[Attribute(Attribute::TARGET_CLASS | Attribute::TARGET_METHOD | Attribute::IS_REPEATABLE)] final class MyCustomLabel extends AbstractLabel { public function __construct(string $value) { parent::__construct('MyCustomLabel', $value); } } final class TestMyWebsite extends TestCase { #[MyCustomLabel('value')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::label('MyCustomLabel', 'value'); // ... } } ``` ### ID - `#[AllureId(string $value)]` Set the test's [ID](/docs/v3/readability/#id). ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\AllureId; final class TestMyWebsite extends TestCase { #[AllureId('123')] public function testLabels() { // ... } } ``` ### Link - `#[Link(?string $name = null, ?string $url = null, string $type = Link::CUSTOM)]` - `#[Issue(?string $name = null, ?string $url = null)]` - `#[TmsLink(?string $name = null, ?string $url = null)]` - `Allure::link(string $url, ?string $name = null, ?LinkType $type = null)` - `Allure::issue(string $name, ?string $url = null)` - `Allure::tms(string $name, ?string $url = null)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`linkTemplates`](/docs/phpunit-configuration/#linktemplates) configuration option. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. For convenience, Allure provides shorthand functions with pre-selected link types `Link::ISSUE` and `Link::TMS`. **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\Issue; use Qameta\Allure\Attribute\Link; use Qameta\Allure\Attribute\TmsLink; final class TestMyWebsite extends TestCase { #[Link('My Website', 'https://example.com/')] #[Issue('UI-123')] #[TmsLink('TMS-456')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::link('https://example.com/', 'My Website'); Allure::issue('UI-123'); Allure::tms('TMS-456'); // ... } } ``` ## Behavior-based hierarchy - `#[Epic(string $value)]` - `#[Feature(string $value)]` - `#[Story(string $value)]` - `Allure::epic(string $value)` - `Allure::feature(string $value)` - `Allure::story(string $value)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\Epic; use Qameta\Allure\Attribute\Feature; use Qameta\Allure\Attribute\Story; final class TestMyWebsite extends TestCase { #[Epic('Web interface')] #[Feature('Essential features')] #[Story('Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::epic('Web interface'); Allure::feature('Essential features'); Allure::story('Labels'); // ... } } ``` ## Suite-based hierarchy - `#[ParentSuite(string $value)]` - `#[Suite(string $value)]` - `#[SubSuite(string $value)]` - `Allure::parentSuite(string $value)` - `Allure::suite(string $value)` - `Allure::subSuite(string $value)` Assign the name of the suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Attributes API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Attribute\ParentSuite; use Qameta\Allure\Attribute\SubSuite; use Qameta\Allure\Attribute\Suite; final class TestMyWebsite extends TestCase { #[ParentSuite('Web interface')] #[Suite('Essential features')] #[SubSuite('Labels')] public function testLabels() { // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::parentSuite('Web interface'); Allure::suite('Essential features'); Allure::subSuite('Labels'); // ... } } ``` ## Test steps - `Allure::runStep(callable $callable, ?string $name = null)` - `Allure::addStep(string $name, ?Status $status = null)` Define [test steps](/docs/steps/). There are three ways of defining a step. - **Method-based steps** Write a test step in a public method and pass it to `runStep()` using the array syntax (i.e., `[$obj, 'method']`). The method should either accept no arguments or accept one argument of the type `StepContextInterface`. During the execution of a step, you can call `runStep()` again to create a sub-step. To change a step's title (defaults to just the word “step”), you can: - add the [`#[DisplayName]`](#title) attribute to the method, - pass the optional `name` argument to `runStep()`, - use the `name()` method on the `StepContextInterface` object. - **Lambda steps** Write a test step in a lambda function and pass it to `runStep()`. If the lambda function returns a value, `runStep()` will return it without modification, and it will not affect the report. During the execution of a step, you can call `runStep()` again to create a sub-step. To change a step's title (defaults to just the word “step”), you can: - add the [`#[DisplayName]`](#title) attribute to the lambda function, - pass the optional `name` argument to `runStep()`, - use the `name()` method on the `StepContextInterface` object. - **No-op steps** If you call `addStep()`, Allure will add to the report a no-op step. This allows for a log-style reporting within a test or within a larger step. A no-op step finishes immediately after it started and cannot have any sub-steps, or parameters. The optional second argument indicates the status that will be shown for the step in the report. Allowed values are: `Status::passed()` (the default), `Status::failed()`, `Status::broken()`, and `Status::skipped()`. During the execution of a method-based step or a lambda step, it is possible to describe the data the step works with, e.g., if you run it multiple times with different data. To do so, use the `parameter()` method of the `StepContextInterface` object. **Method-based steps:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; use Qameta\Allure\Attribute\DisplayName; use Qameta\Allure\StepContextInterface; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::runStep([$this, 'logIn']); Allure::runStep([$this, 'createLabel']); Allure::runStep([$this, 'checkThatLabelExists']); } #[DisplayName('Log in')] function logIn(StepContextInterface $context) { $context->parameter('Email', 'johndoe@example.com'); $context->parameter('Password', 'qwerty'); // ... } #[DisplayName('Create label')] function createLabel(StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... } #[DisplayName('Check that label exists')] function checkThatLabelExists(StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... } } ``` **Lambda steps:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; use Qameta\Allure\StepContextInterface; final class TestMyWebsite extends TestCase { public function testLabels() { Allure::runStep(function (StepContextInterface $context) { $context->parameter('Email', 'johndoe@example.com'); $context->parameter('Password', 'qwerty'); // ... }, 'Log in'); Allure::runStep(function (StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... }, 'Create label'); Allure::runStep(function (StepContextInterface $context) { $context->parameter('Label name', 'My Label'); // ... }, 'Check that label exists'); } } ``` **No-op steps:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { // ... Allure::addStep('Log in'); // ... Allure::addStep('Create label'); // ... Allure::addStep('Check that label exists'); } } ``` ## Parametrized tests - `Allure::parameter(string $name, ?string $value, bool $excluded = false, ?ParameterMode $mode = null)` Allure PHPUnit supports the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. If you pass parameters to your tests via PHPUnit's [data providers](https://docs.phpunit.de/en/10.5/writing-tests-for-phpunit.html#data-providers), the test report will add a numbered pseudo-parameter for each run iteration, e.g., “Data set: #0”, “Data set: #1”, etc. To make the report show the actual parameters and their values, pass them to `Allure::parameter()`. Note that `Allure::parameter()` to can be used for adding the parameters even to functions outside the PHPUnit's parametrized tests. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. The `mode` argument affects how the parameter will be displayed in the report. Available options are: - `null` — the parameter and its value will be shown in a table along with other parameters. - `ParameterMode::masked()` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `ParameterMode::hidden()` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. The parameters can be added not only to whole test results, but also to individual steps within them, see [Test steps](#test-steps) for more details. **DataProvider:** ```php use PHPUnit\Framework\Attributes\DataProvider; use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestDataProvider extends TestCase { public static function credentialsProvider() { return [ ['johndoe', 'qwerty'], ['johndoe@example.com', 'qwerty'], ]; } #[DataProvider('credentialsProvider')] public function testAuthentication(string $login, string $password) { Allure::parameter('login', $login); Allure::parameter('password', $password); // ... } } ``` **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testAuthenticationWithUsername() { Allure::parameter('login', 'johndoe'); Allure::parameter('password', 'qwerty'); // ... } public function testAuthenticationWithEmail() { Allure::parameter('login', 'johndoe@example.com'); Allure::parameter('password', 'qwerty'); // ... } } ``` ## Attachments - `Allure::attachment(string $name, string $content, ?string $type = null, ?string $fileExtension = null)` - `Allure::attachmentFile(string $name, string $file, ?string $type = null, ?string $fileExtension = null)` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Tip: You can use data produced by any function, not necessarily read from an actual file. To create an attachment using the Runtime API, call `Allure::attachment()` or `Allure::attachmentFile()` at any point during your test. Pass either the `content` or the `path` from which the data will be read. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension as `fileExtension`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. **Runtime API:** ```php use PHPUnit\Framework\TestCase; use Qameta\Allure\Allure; final class TestMyWebsite extends TestCase { public function testLabels() { // ... Allure::attachment('data.txt', 'This is the file content.', 'text/plain'); Allure::attachmentFile('data.txt', '/path/to/image.png', 'image/png'); } } ``` --- ## Playwright > URL: https://allurereport.org/docs/playwright/ # Getting started with Allure Playwright [![Allure Playwright npm latest version](https://img.shields.io/npm/v/allure-playwright?style=flat "Allure Playwright npm latest version")](https://www.npmjs.com/package/allure-playwright) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Playwright](https://playwright.dev/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Aplaywright) to see Allure Playwright in action. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure Playwright is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure Playwright integration and make sure that the Playwright framework itself is also listed in the project's dependencies. **npm:** ```bash npm install --save-dev @playwright/test allure-playwright ``` **yarn:** ```bash yarn add --dev @playwright/test allure-playwright allure-js-commons ``` **pnpm:** ```bash pnpm add -D @playwright/test allure-playwright ``` 1. In the `playwright.config.ts` file, add Allure Playwright as a reporter. **With default options:** ```ts export default defineConfig({ // ... reporter: [["line"], ["allure-playwright"]], }); ``` **With custom options:** ```ts export default defineConfig({ // ... reporter: [ ["line"], [ "allure-playwright", { resultsDir: "allure-results", }, ], ], }); ``` Some additional options can also be defined here, see [Configuration](/docs/playwright-configuration/). ### 2. Run tests Run your Playwright tests the same way as you would run them usually. For example: **npm:** ```bash npx playwright test ``` **yarn:** ```bash yarn run playwright test ``` **pnpm:** ```bash pnpx playwright test ``` This will save necessary data into `allure-results` or other directory, according to the [configuration](/docs/playwright-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Playwright integration extends the standard reporting features of Playwright by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Playwright Traces**: Open Playwright's own [trace files](#playwright-traces) directly from the test report. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure Playwright provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Playwright will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/playwright-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; test("Test Authentication", async () => { await allure.displayName("Test Authentication"); await allure.owner("John Doe"); await allure.tags("Web interface", "Authentication"); await allure.severity("critical"); // ... }); ``` **Metadata API:** ```ts import { test } from "@playwright/test"; test( "Test Authentication" + " @allure.label.owner:JohnDoe" + " @allure.label.tag:WebInterface" + " @allure.label.tag:Authentication" + " @allure.label.severity:critical", async () => { // ... }, ); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; test("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Metadata API:** ```ts import { test } from "@playwright/test"; test( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; test("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```ts import { test } from "@playwright/test"; test( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); ``` ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), you can use the `step()` function, see the [reference](/docs/playwright-reference/#test-steps). Playwright's [`test.step()`](https://playwright.dev/docs/api/class-test#test-step) is also supported. ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; test("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` Note that by default Allure displays automatically generated steps in the list, too, including those defined using Playwright's [`beforeAll()`](https://playwright.dev/docs/api/class-test#test-before-all), [`beforeEach()`](https://playwright.dev/docs/api/class-test#test-before-each), [`afterEach()`](https://playwright.dev/docs/api/class-test#test-after-each), and [`afterAll()`](https://playwright.dev/docs/api/class-test#test-after-all) functions. This can be disabled using the [`detail`](/docs/playwright-configuration/#detail) setting. Images: /images/js/playwright/hooks-allure3-light.png, /images/js/playwright/hooks-allure3-dark.png, /images/js/playwright/hooks-allure2-light.png, /images/js/playwright/hooks-allure2-dark.png ### Describe parametrized tests A typical way to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern in Playwright is to define the test in a loop and use the loop variable in the test's title and body. To display a parameter value in the test report, pass it to the [`parameter()`](/docs/playwright-reference/#parametrized-tests) function. ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } test("Test Authentication With Empty Login", async ({ page }) => { await allure.parameter("login", ""); await allure.parameter("auth_method", "password"); // ... }); ``` Tip: For more information, you can consult the guide on [Playwright parameterization](/docs/guides/playwright-parameterization/). ### Set labels globally Any [labels](/docs/playwright-reference/#label), including custom ones, can be set via the environment variables in your operating system. Here's an example (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface npx playwright test ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" npx playwright test ``` ### Attach screenshots and other files Reports generated by Allure can include any files attached to the test using `allure.attachment()` method. (Playwright's built-in [`TestInfo.attach()`](https://playwright.dev/docs/api/class-testinfo#test-info-attach) method is supported as well.) For example, a popular way to make a report easier to understand is to attach a screenshot of the opened web page. See [Attachments](/docs/attachments/). ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; test("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` ### Playwright traces If you enable Playwright's built-in [tracing](https://playwright.dev/docs/trace-viewer) in `playwright.config.ts`, Allure Playwright automatically recognizes the resulting trace file and attaches it as a [Playwright trace attachment](/docs/attachments/#playwright-traces), so it can be opened directly from the test report in Playwright's own Trace Viewer instead of being offered as a plain zip download. No extra Allure-specific setup is needed beyond enabling tracing itself. ```ts import { defineConfig } from "@playwright/test"; export default defineConfig({ use: { trace: "on", }, }); ``` `trace: "on"` records a trace for every test. To save space, you can instead use `"on-first-retry"` or `"retain-on-failure"` so traces are only kept for tests that fail — see the [Playwright tracing documentation](https://playwright.dev/docs/trace-viewer#recording-a-trace) for the full list of options. ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Playwright will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json` (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json npx playwright test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" npx playwright test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, specify the information in the `environmentInfo` configuration parameter. For example, it is a good idea to use this to remember the OS version and Node.js version retrieved from the [`os`](https://nodejs.org/api/os.html) and [`process`](https://nodejs.org/api/process.html) objects. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/js/environment-allure3-light.png, /images/js/environment-allure3-dark.png, /images/js/environment-allure2-light.png, /images/js/environment-allure2-dark.png ```ts import type { PlaywrightTestConfig } from "@playwright/test"; import * as os from "node:os"; const config: PlaywrightTestConfig = { reporter: [ ["line"], [ "allure-playwright", { environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }, ], ], }; export default config; ``` Note that if your launch includes multiple Playwright runs (see [How it works](/docs/how-it-works/)), Allure Playwright will only save the environment information from the latest run. --- ## Playwright configuration > URL: https://allurereport.org/docs/playwright-configuration/ # Allure Playwright configuration The [Allure Playwright](/docs/playwright/) integration behavior can be affected by some configuration options set in your project's `playwright.config.ts` file. Info: This section describes the configuration parameters for Allure Playwright 3.0 and newer. For an older version, you may check this [readme](https://github.com/allure-framework/allure-js/blob/v2.15.1/packages/allure-playwright/README.md). To set the options, define an object after the `"allure-playwright"` string in the `reporter` configuration section. For example: ```ts import type { PlaywrightTestConfig } from "@playwright/test"; import { Status } from "allure-js-commons"; import * as os from "node:os"; const config: PlaywrightTestConfig = { reporter: [ ["line"], [ "allure-playwright", { resultsDir: "allure-results", detail: true, suiteTitle: true, links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: (v) => `https://jira.example.com/browse/${v}`, }, }, categories: [ { name: "foo", messageRegex: "bar", traceRegex: "baz", matchedStatuses: [Status.FAILED, Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, globalLabels: [{ name: "layer", value: "api" }], }, ], ], }; export default config; ``` ## resultsDir Path to the directory where Allure Playwright will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ## detail If true (the default), Allure will automatically create [steps](/docs/playwright/#divide-a-test-into-steps) for: - Playwright API calls, such as [`browser.newPage()`](https://playwright.dev/docs/api/class-browser#browser-new-page), [`page.goto()`](https://playwright.dev/docs/api/class-page#page-goto). - Playwright test hooks, such as [`beforeEach()`](https://playwright.dev/docs/api/class-test#test-before-each). - assertions made using `except`. ## suiteTitle If true (the default), implicitly add each test into a [test suite](/docs/v3/navigation/#suite-based-hierarchy) named after its file name. ## links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/playwright-reference/#link)), you can specify the following: - `nameTemplate` — a template or a function for generating the link name when it is not provided. - `urlTemplate` — a template or a function for generating the link address when it is not provided. The templates can be strings (with `%s` where the identifier should be placed) or functions (accepting the identifier and returning the URL). For example, with the configuration above, `await allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/` . Allure will display the link with the appropriate icon for the `issue` type. ## categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). ## environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ## globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ name, value }` label objects: ```ts const config: PlaywrightTestConfig = { reporter: [ [ "allure-playwright", { globalLabels: [ { name: "layer", value: "api" }, { name: "team", value: "backend" }, ], }, ], ], }; ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```ts const config: PlaywrightTestConfig = { reporter: [ [ "allure-playwright", { globalLabels: { layer: "api", team: "backend", }, }, ], ], }; ``` --- ## Playwright reference > URL: https://allurereport.org/docs/playwright-reference/ # Allure Playwright reference These are the functions that you can use to integrate your Playwright tests with Allure. In most cases, Allure Playwright provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Playwright will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `allure.description(markdown: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; test("Test Authentication", async () => { await allure.description("This test attempts to log into the website."); // ... }); ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; test("Test Authentication", async () => { await allure.owner("John Doe"); // ... }); ``` **Metadata API:** ```ts import { test } from "@playwright/test"; test("Test Authentication @allure.label.owner:JohnDoe", async () => { // ... }); ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` - `@allure.label.tag:⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; test("Test Authentication", async () => { await allure.tag("New UI"); await allure.tags("Essentials", "Authentication"); // ... }); ``` **Metadata API:** ```ts import { test } from "@playwright/test"; test( "Test Authentication" + " @allure.label.tag:NewUI" + " @allure.label.tag:Essentials" + " @allure.label.tag:Authentication", async () => { // ... }, ); ``` ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Runtime API:** ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; import { Severity } from "allure-js-commons"; import { allure } from "allure-playwright"; test("Test Authentication", async () => { await allure.severity(Severity.CRITICAL); // ... }); ``` **Metadata API:** ```ts import { test } from "@playwright/test"; test("Test Authentication @allure.label.severity:critical", async () => { // ... }); ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` - `@allure.label.⟨NAME⟩:⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. **Runtime API:** ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; test("Test Authentication", async () => { await allure.label("microservice", "UI"); // ... }); ``` **Metadata API:** ```ts import { test } from "@playwright/test"; test("Test Authentication @allure.label.microservice:UI", async () => { // ... }); ``` ### ID - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). ```ts import { test } from "@playwright/test"; test("Test Authentication @allure.id:123", async () => { // ... }); ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string, defaults to “link”), Allure will try to load a corresponding **link template** to process the URL, as defined by the [`links`](/docs/playwright-configuration/#links) configuration option. If no template is found for the given type or if the link already represents a proper URL, it's left unmodified. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; test("Test Authentication", async () => { await allure.issue("AUTH-123", "Related issue"); await allure.tms("TMS-456", "Related TMS issue"); await allure.link("JIRA-777", "Related Jira issue", "jira"); await allure.link("https://example.com/", "Project website"); // ... }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features**, or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; test("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Metadata API:** ```ts import { test } from "@playwright/test"; test( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the names of **parent suite**, **suite**, or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Runtime API:** ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; test("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```ts import { test } from "@playwright/test"; test( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test step](/docs/steps/) with the given `name`. There are two ways of defining a step. - **Lambda steps** Write a test step in a function and pass it to `allure.step()`. If the function returns a value, `allure.step()` will return it without modification, and it will not affect the report. If it returns a promise, it will be awaited and resolved. The function can accept either no arguments or a single object of type `StepContext`. This object provides the following methods: - `displayName()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are similar to the test-wide implementation, see [Parametrized tests](#parametrized-tests). Info: Playwright's [`test.step()`](https://playwright.dev/docs/api/class-test#test-step) is also supported. - **No-op steps** If you call `allure.logStep()`, Allure will add to the report a no-op step. This allows for a log-style reporting within a test or within a larger step. A no-op step finishes immediately after it started and cannot have any sub-steps, or parameters. The optional second argument indicates the status that will be shown for the step in the report. Allowed values are: `Status.PASSED` (the default), `Status.FAILED`, `Status.BROKEN`, and `Status.SKIPPED`. The optional third argument is an `Error` object. When provided, the data from this object will be displayed in a message inside the step. ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; test("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` Specify a `name` and a `value` of a parameter that was used during this test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. The `options` argument, if given, must be an object with two optional properties `excluded` and `mode`. - If `excluded` is set to true, Allure will not use the parameter when comparing the current test result with the previous one in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. Note, that even when you use the `"masked"` or `"hidden"` mode, it is still possible to extract the value from the `allure_results` directory if you publish it. ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } ``` Tip: For more information, you can consult the guide on [Playwright parameterization](/docs/guides/playwright-parameterization/). ## Attachments - `allure.attachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options in an object (as shown for the image attachment below) or just specify the media type and let Allure deduce the appropriate filename extension automatically (as shown for the text attachment below). In either case, the media type can be a value from the `ContentType` enumeration or any string. ```ts import { test } from "@playwright/test"; import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; test("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` Playwright's own trace files (produced via the [`trace`](https://playwright.dev/docs/api/class-testoptions#test-options-trace) option, not through this API) are recognized automatically and attached as [Playwright trace attachments](/docs/attachments/#playwright-traces). See [Playwright traces](/docs/playwright/#playwright-traces). --- ## Playwright Java > URL: https://allurereport.org/docs/playwright-java/ # Getting started with Allure Playwright Java [![Allure Playwright Java latest version](https://img.shields.io/maven-central/v/io.qameta.allure/allure-playwright?style=flat "allure-playwright")](https://mvnrepository.com/artifact/io.qameta.allure/allure-playwright) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/) and your [Playwright Java](https://playwright.dev/java/) tests. ## Setting up Allure Playwright Java works alongside your test framework — set up [JUnit Jupiter](/docs/junit-jupiter/) or [TestNG](/docs/testng/) first, then add the steps below. To integrate Allure Playwright into your project, you need to: 1. Add Allure Playwright dependencies. 2. Set up the AspectJ weaving agent. 3. Specify a location for Allure results storage. ### Add Allure dependencies **Maven:** ```xml VERSION 1.9.25.1 io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-playwright test org.aspectj aspectjrt ${aspectj.version} test ``` **Gradle (Kotlin):** ```kts val allureVersion = "VERSION" val aspectjVersion = "1.9.25.1" val aspectjAgent: Configuration by configurations.creating dependencies { testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) testImplementation("io.qameta.allure:allure-playwright") testRuntimeOnly("org.aspectj:aspectjrt:$aspectjVersion") aspectjAgent("org.aspectj:aspectjweaver:$aspectjVersion") } ``` **Gradle (Groovy):** ```groovy def allureVersion = "VERSION" def aspectjVersion = "1.9.25.1" configurations { aspectjAgent } dependencies { testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") testImplementation "io.qameta.allure:allure-playwright" testRuntimeOnly "org.aspectj:aspectjrt:$aspectjVersion" aspectjAgent "org.aspectj:aspectjweaver:$aspectjVersion" } ``` ### Set up the AspectJ agent Allure Playwright uses AspectJ to intercept Playwright API calls at runtime and record them as Allure steps. The AspectJ weaver must be attached to the JVM when tests run. **Maven:** ```xml org.apache.maven.plugins maven-surefire-plugin 3.2.3 -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar" org.aspectj aspectjweaver ${aspectj.version} ``` **Gradle (Kotlin):** ```kts tasks.test { doFirst { jvmArgs("-javaagent:${configurations["aspectjAgent"].singleFile}") } } ``` **Gradle (Groovy):** ```groovy test { doFirst { jvmArgs "-javaagent:${configurations.aspectjAgent.singleFile}" } } ``` ### Specify Allure results location Allure saves test results in the project's root directory by default. It is recommended to store results in the build output directory instead. Create an `allure.properties` file in `src/test/resources`: **Maven:** ```properties allure.results.directory=target/allure-results ``` **Gradle (Kotlin):** ```properties allure.results.directory=build/allure-results ``` **Gradle (Groovy):** ```properties allure.results.directory=build/allure-results ``` ### Generate a report After running your tests, convert the results into an HTML report using one of these commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Browser actions as steps With the AspectJ agent active, Allure Playwright automatically intercepts calls to the Playwright Java API and records them as steps in the test report. No additional code is required. The following Playwright types are intercepted: - `Page`, `Frame`, `Locator`, `ElementHandle` — browser interaction methods - Playwright assertion classes — `assertThat(...)` assertions Step names are generated from the method name and its arguments. For example: ```java page.click("button"); // → "Click button" page.navigate("https://example.com"); // → "Navigate to https://example.com" page.hover(".menu-item"); // → "Hover .menu-item" page.fill("#search", "query"); // → "Fill #search with " page.type("#comment", "text"); // → "Type into #comment" page.screenshot(); // → "Take screenshot" step, with a "Screenshot" attachment inside it ``` Values passed to `fill()`, `type()`, and `press()` are replaced with `` in step names by default to avoid exposing passwords and other sensitive data. See [configuration](/docs/playwright-java-configuration/#allureplaywrightparameters) to change this behavior. Playwright action steps are nested under any enclosing Allure steps you define: ```java Allure.step("Checkout flow", () -> { page.click("button"); // appears as a sub-step of "Checkout flow" }); ``` By default, only recognized browser interaction methods (click, fill, navigate, hover, check, type, dragTo, waitForSelector, and more) and Playwright assertions are recorded as steps. To record every public Playwright API method call, set `allure.playwright.steps.mode=all` in `allure.properties`. ## Capturing browser artifacts Allure Playwright captures browser artifacts automatically and also provides helpers for manual capture. ### On test failure When a test fails or is broken, Allure Playwright automatically captures a screenshot and the current page source for each registered page, and attaches them to the test result. This requires pages or contexts to be registered, either explicitly or automatically (see [Registering pages and contexts](#registering-pages-and-contexts)). ### On close When a `Page`, `BrowserContext`, or `Browser` is closed while a test is active, Allure Playwright automatically collects: - **Console messages** — browser console output accumulated since the page was created (controlled by `allure.playwright.close.page-logs`). - **Page errors** — uncaught JavaScript exceptions (controlled by `allure.playwright.close.page-logs`). - **Video** — if the context was created with video recording enabled (controlled by `allure.playwright.close.video`). - **Trace** — if tracing was started with [`AllurePlaywright.startTracing()`](/docs/playwright-java-reference/#starttracing) (controlled by `allure.playwright.close.trace`). See [Trace capture](#trace-capture) for the full set of triggers; closing an individual `Page` does not trigger trace attachment. Console messages and page errors are also captured when the test ends for any registered pages whose context was not explicitly closed during the test. See [configuration](/docs/playwright-java-configuration/) for details on each setting. ### Manual capture You can capture and attach any of these artifacts at any point during a test using the static helpers on `AllurePlaywright`: ```java import io.qameta.allure.playwright.AllurePlaywright; // Attach a screenshot of the current page state AllurePlaywright.attachScreenshot("Before submit", page); // Attach the current page HTML AllurePlaywright.attachPageSource("Page source", page); // Attach console messages collected by the page AllurePlaywright.attachConsoleMessages("Browser console", page); // Attach uncaught JavaScript errors AllurePlaywright.attachPageErrors("Page errors", page); // Attach an existing trace archive AllurePlaywright.attachTrace("Playwright trace", Path.of("trace.zip")); // Attach an existing video file AllurePlaywright.attachVideo("Test recording", Path.of("video.webm")); ``` All helpers are safe to call when no Allure test is active — they do nothing in that case. ## Registering pages and contexts Allure Playwright tracks registered pages and contexts and uses them to capture screenshots, page source, console messages, and page errors when a test fails. When AspectJ weaving is active, pages and contexts are registered automatically in two ways: when created via `browser.newPage()`, `browser.newContext()`, or `context.newPage()`; and when any intercepted Playwright action (such as `click` or `fill`) is performed on a page while a test is active. Auto-registration only applies while a test is active; objects created and used exclusively before the test starts, such as in a `@BeforeAll` setup, must be registered explicitly. For pages or contexts created outside of Playwright's standard factory methods, or outside an active test, register them explicitly: ```java AllurePlaywright.register(page); AllurePlaywright.register(context); ``` Registering a `Page` also registers its parent `BrowserContext`. ## Trace capture Playwright traces record browser activity as a rich archive that can be opened in the [Playwright Trace Viewer](https://playwright.dev/java/docs/trace-viewer). Allure Playwright can start a trace and attach the resulting archive automatically. Use `AllurePlaywright.startTracing()` to begin tracing a browser context. The returned `TraceSession` stops tracing and attaches the archive when closed: ```java import io.qameta.allure.playwright.AllurePlaywright; import io.qameta.allure.playwright.TraceSession; BrowserContext context = browser.newContext(); try (TraceSession trace = AllurePlaywright.startTracing(context)) { // run browser actions } // trace archive is now attached to the test result ``` Tracing is also stopped and attached automatically when: - The `BrowserContext` or `Browser` is closed while a test is active (controlled by `allure.playwright.close.trace`). - The test fails while a trace is still open (always, regardless of `allure.playwright.close.trace`). - The test ends while a trace is still open (controlled by `allure.playwright.close.trace`). To assign a custom name to the trace attachment, use the overload that accepts a name: ```java TraceSession trace = AllurePlaywright.startTracing("Login flow trace", context); ``` ## Framework lifecycle integration When `allure-playwright` is on the classpath alongside any Allure framework integration, lifecycle integration is configured automatically — no additional setup is needed. For test frameworks without an Allure integration, call the following methods in your test setup and teardown. `beforeTest()` must run before the test body, and `afterTest()` must run after it — typically in a `@BeforeEach`/`@AfterEach` pair. Detecting failure requires intercepting the test body exception before it propagates: ```java import io.qameta.allure.playwright.AllurePlaywright; // Before the test body runs (@BeforeEach or equivalent) AllurePlaywright.beforeTest(); // Wrap the test body execution to detect failure: try { // test body } catch (Throwable t) { AllurePlaywright.afterTestFailure(t); throw t; } finally { AllurePlaywright.afterTest(); } ``` ## Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). ```properties os.name=Linux java.version=17.0.10 browser=chromium playwright.version=1.60.0 ``` --- ## Playwright Java configuration > URL: https://allurereport.org/docs/playwright-java-configuration/ # Allure Playwright Java configuration This page describes the configuration options that affect the behavior of [Allure Playwright Java](/docs/playwright-java/). To set the configuration options, create an `allure.properties` file in your project's `src/test/resources` directory. ```properties # src/test/resources/allure.properties allure.playwright.steps.enabled=true allure.playwright.steps.mode=actions allure.playwright.parameters=redacted allure.playwright.screenshots.attach=true allure.playwright.failure.screenshot=true allure.playwright.failure.page-source=true allure.playwright.close.trace=true allure.playwright.close.video=true allure.playwright.close.page-logs=true ``` ## allure.playwright.steps.enabled ```properties allure.playwright.steps.enabled=true ``` Whether to record Playwright API calls as Allure steps at all. Set to `false` to disable all automatic step recording while keeping artifact capture active. Defaults to `true`. ## allure.playwright.steps.mode ```properties allure.playwright.steps.mode=actions ``` Controls which Playwright API methods are recorded as steps. - `actions` (default) — records recognized browser interaction methods and Playwright assertions. This includes navigation (`navigate`, `reload`, `goBack`, `goForward`), interactions (`click`, `dblclick`, `fill`, `type`, `hover`, `check`, `uncheck`, `press`, `tap`, `focus`, `blur`, `clear`, `dragTo`, `dragAndDrop`, `selectOption`, `setInputFiles`, `dispatchEvent`, and more), wait methods (`waitForSelector`, `waitForNavigation`, `waitForURL`, `waitForTimeout`, and others), and all Playwright assertion methods, on `Page`, `Frame`, `Locator`, and `ElementHandle`. - `all` — records every public method call on the intercepted Playwright types, except `equals`, `hashCode`, `toString`, and `not`. Use `all` when you need full API call visibility for debugging. For most projects, `actions` produces a cleaner report. ## allure.playwright.parameters ```properties allure.playwright.parameters=redacted ``` Controls whether sensitive values passed to Playwright actions appear in step names. - `redacted` (default) — replaces values passed to `fill()`, `type()`, and `press()` with `` in the step name. For example: - `page.fill("#password", "secret")` → `Fill #password with ` - `page.type("#comment", "text")` → `Type into #comment` - Any other value (e.g., `plain`) — includes the actual argument value in the step name. Keep the default when your tests interact with password fields, tokens, or other sensitive inputs. ## allure.playwright.screenshots.attach ```properties allure.playwright.screenshots.attach=true ``` Whether to attach the image returned by `page.screenshot()`, `locator.screenshot()`, or `elementHandle.screenshot()` to the corresponding Allure step. Defaults to `true`. When enabled, the screenshot bytes are attached to the `Take screenshot` step that Allure records for each screenshot call. The screenshot is still returned to your test code unchanged. This setting only affects screenshots captured automatically when `page.screenshot()`, `locator.screenshot()`, or `elementHandle.screenshot()` is called as a step. It does not affect `AllurePlaywright.attachScreenshot()`, which always attaches the screenshot it captures. ## allure.playwright.failure.screenshot ```properties allure.playwright.failure.screenshot=true ``` Whether to capture a screenshot of each registered page and attach it to the test result when a test fails or is broken. Defaults to `true`. Requires at least one page to be registered, either explicitly via `AllurePlaywright.register(page)` or automatically through Playwright's `newPage()` factory when AspectJ weaving is active. ## allure.playwright.failure.page-source ```properties allure.playwright.failure.page-source=true ``` Whether to capture the current HTML of each registered page and attach it to the test result when a test fails or is broken. Defaults to `true`. Subject to the same registration requirement as `allure.playwright.failure.screenshot`. ## allure.playwright.close.trace ```properties allure.playwright.close.trace=true ``` Whether to stop and attach any open Playwright trace sessions when a `BrowserContext` or `Browser` is closed, or when the test ends. Defaults to `true`. Applies to trace sessions started via `AllurePlaywright.startTracing()`. When disabled, open trace sessions are discarded without producing an attachment. This setting does not affect trace attachment on test failure — open traces are always attached when a test fails, regardless of this value. ## allure.playwright.close.video ```properties allure.playwright.close.video=true ``` Whether to attach Playwright-recorded video files when a `Page`, `BrowserContext`, or `Browser` is closed. Defaults to `true`. Applies only to contexts created with `setRecordVideoDir()`. The video file is read after the close, when Playwright has finished writing it. ## allure.playwright.close.page-logs ```properties allure.playwright.close.page-logs=true ``` Whether to capture console messages and page errors from registered pages and attach them to the test result. Defaults to `true`. This fires when a `Page`, `BrowserContext`, or `Browser` is closed while a test is active, and also when the test ends normally (for any registered pages whose context was not explicitly closed). Two attachments are produced per page: one for console messages (`Console messages`) and one for uncaught JavaScript errors (`Page errors`). Empty attachments are not added. --- ## Playwright Java reference > URL: https://allurereport.org/docs/playwright-java-reference/ # Allure Playwright Java reference These are the classes and methods you can use to integrate your Playwright Java tests with Allure. ## AllurePlaywright `io.qameta.allure.playwright.AllurePlaywright` A utility class with static methods for registering Playwright objects, capturing browser artifacts, and managing the per-test Playwright state. All methods are thread-safe and safe to call when no Allure test is active. ### register - `AllurePlaywright.register(Page page)` - `AllurePlaywright.register(BrowserContext context)` Register a Playwright `Page` or `BrowserContext` for automatic failure diagnostics. Registered objects are used as sources for screenshots, page source, and page log attachments when a test fails or is broken. Registering a `Page` also registers its parent `BrowserContext`. When AspectJ weaving is active, pages and contexts are registered automatically in two ways: when created via `browser.newPage()`, `browser.newContext()`, or `context.newPage()`; and when any intercepted Playwright action is performed on a page while a test is active. Explicit registration is only needed for objects created and used exclusively outside an active test (for example, in a `@BeforeAll` setup). ```java import io.qameta.allure.playwright.AllurePlaywright; import com.microsoft.playwright.BrowserContext; import com.microsoft.playwright.Page; Page page = browser.newContext().newPage(); AllurePlaywright.register(page); BrowserContext context = browser.newContext(); AllurePlaywright.register(context); ``` ### attachScreenshot - `AllurePlaywright.attachScreenshot(String name, Page page)` Capture a screenshot of the given page and attach it to the current Allure test or step. This call does not produce an additional Allure step. Pass `null` or an empty string as `name` to use the default attachment name `Screenshot`. ```java AllurePlaywright.attachScreenshot("Before submit", page); AllurePlaywright.attachScreenshot(null, page); // attached as "Screenshot" ``` ### attachPageSource - `AllurePlaywright.attachPageSource(String name, Page page)` Capture the current HTML of the given page and attach it as a text/html attachment. Pass `null` or an empty string as `name` to use the default attachment name `Page source`. ```java AllurePlaywright.attachPageSource("Page source", page); ``` ### attachConsoleMessages - `AllurePlaywright.attachConsoleMessages(String name, Page page)` Attach the console messages collected by the given page since it was created. Each message is formatted as `[type] text (location)`. Empty output is not attached. Pass `null` or an empty string as `name` to use the default attachment name `Console messages`. ```java AllurePlaywright.attachConsoleMessages("Browser console", page); ``` ### attachPageErrors - `AllurePlaywright.attachPageErrors(String name, Page page)` Attach the uncaught JavaScript errors collected by the given page since it was created. Each error is written on its own line as plain text. Empty output is not attached. Pass `null` or an empty string as `name` to use the default attachment name `Page errors`. ```java AllurePlaywright.attachPageErrors("Page errors", page); ``` ### attachTrace - `AllurePlaywright.attachTrace(String name, Path traceZip)` Attach an existing Playwright trace archive (zip file) to the current Allure test or step. Pass `null` or an empty string as `name` to use the default attachment name `Playwright trace`. ```java AllurePlaywright.attachTrace("Login trace", Path.of("trace.zip")); ``` ### attachVideo - `AllurePlaywright.attachVideo(String name, Path videoFile)` Attach an existing video file to the current Allure test or step. WebM files are attached with the `video/webm` media type; other extensions are attached as `application/octet-stream`. Pass `null` or an empty string as `name` to use the default attachment name `Playwright video`. ```java AllurePlaywright.attachVideo("Test recording", Path.of("video.webm")); ``` ### startTracing - `AllurePlaywright.startTracing(BrowserContext context): TraceSession` - `AllurePlaywright.startTracing(String name, BrowserContext context): TraceSession` Start Playwright tracing on the given `BrowserContext` with screenshots and DOM snapshots enabled, and return a `TraceSession` that can stop and attach the trace. The context is registered for failure diagnostics automatically. Pass a `name` to control the attachment name. The default name is `Playwright trace`. Use `try-with-resources` to guarantee that tracing is stopped and the archive is attached even if the test throws: ```java try (TraceSession trace = AllurePlaywright.startTracing("Checkout trace", context)) { page.navigate("https://example.com"); page.click("#checkout"); } ``` If the context is closed before `TraceSession.close()` is called, the trace is still attached automatically. If the test ends while the session is still open, the trace is attached before the test result is written. ### beforeTest - `AllurePlaywright.beforeTest()` Clear the Playwright registry before a test starts. Call this at the start of each test when no Allure framework integration is managing the lifecycle automatically. Any open trace sessions present at this point are stopped without attaching. ### afterTest - `AllurePlaywright.afterTest()` Attach pending console messages, page errors, and open trace sessions for registered pages and contexts, then clear the Playwright registry. Call this at the end of each test when no Allure framework integration is managing the lifecycle automatically. Videos are attached when `page.close()`, `context.close()`, or `browser.close()` is called, not by this method. ### afterTestFailure - `AllurePlaywright.afterTestFailure()` - `AllurePlaywright.afterTestFailure(Throwable throwable)` Attach failure diagnostics — screenshots, page source, and open trace sessions — for all registered pages. This is called once per test; duplicate calls within the same test are ignored. Call this from your failure handler or `@AfterEach` equivalent when no Allure framework integration is managing the lifecycle automatically. The `Throwable` overload accepts the causing exception for debug logging. ```java // In an @AfterEach or equivalent try { testBody.run(); } catch (Throwable t) { AllurePlaywright.afterTestFailure(t); throw t; } finally { AllurePlaywright.afterTest(); } ``` --- ## TraceSession `io.qameta.allure.playwright.TraceSession` Represents an active Playwright trace session started by `AllurePlaywright.startTracing()`. Implements `AutoCloseable`. ### attach - `TraceSession.attach()` Stop the trace and attach the generated archive to the current Allure test or step. Equivalent to `close()`. ### close - `TraceSession.close()` Stop the trace and attach the generated archive to the current Allure test or step. Use `try-with-resources` to call this automatically: ```java try (TraceSession trace = AllurePlaywright.startTracing(context)) { // browser actions } ``` --- ## Pytest > URL: https://allurereport.org/docs/pytest/ # Getting started with Allure Pytest [![Allure Pytest pypi latest version](https://img.shields.io/pypi/v/allure-pytest?style=flat "Allure Pytest pypi latest version")](https://pypi.org/project/allure-pytest/) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [pytest](https://pytest.org/) tests. Info: Check out the example project at [github.com/allure-examples/pytest-pip](https://github.com/allure-examples/pytest-pip) to see Allure Pytest in action. ## Setting up ### 1. Prepare your project These instructions depend on whether [you have some tests already](#adding-allure-pytest-to-an-existing-project) or [you're just starting a project](#starting-a-new-project-with-allure-pytest). #### Adding Allure Pytest to an existing project If you have some pytest tests, and you want to be able to use Allure Report with them: 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. If necessary for your system configuration, activate the virtual Python environment for your project. For example, if the project uses a `venv` environment, the command to activate it may look like this: ```bash source .venv/bin/activate ``` This step is not necessary if you are using the system Python environment. 1. Install the Allure Pytest integration. ```bash pip install allure-pytest ``` #### Starting a new project with Allure Pytest If you are starting a new project which will contain pytest tests: 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Create a directory for the project. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. If necessary, create and activate virtual Python environment for your project. For example, the commands to create and activate a `venv` environment may look like this: ```bash python -m venv .venv source .venv/bin/activate ``` This step is not necessary if you are going to use the system Python environment in the project. 1. Write your tests, following the official pytest documentation and, optionally, using some extended features provided by Allure Pytest, see [Writing tests](#writing-tests). ### 2. Run tests When running your tests, specify a path for the test results directory in the `--alluredir` command-line argument. For example: ```bash python -m pytest --alluredir allure-results ``` This will save necessary data into the test results directory. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Pytest integration not only collects the data provided by pytest's standard features, but also provides additional features for writing even better tests. This section lists the most notable ways to improve your tests, using both pytest's and Allure Pytest's features. With Allure Pytest, you can: - provide [description, links and other metadata](#specify-description-links-and-other-metadata), - [organize tests](#organize-tests) into hierarchies, - divide the test into smaller, easier-to-read [test steps](#divide-a-test-into-steps), - describe parameters used when running [parametrized tests](#describe-parametrized-tests), - add readable titles to [fixtures](#describe-fixtures), - make the test save [screenshots and other files](#attach-screenshots-and-other-files) during execution, - [select which tests to run](#select-tests-via-a-test-plan-file) via a test plan file. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/pytest-reference/#metadata) for more details. For each of the metadata fields, there are multiple ways to assign it. - Call an Allure's function inside the test function's body. This way is called “dynamic” because it allows you to construct strings and other values at runtime before passing to the methods. - Use an Allure's function as a decorator on the test function or a test class. - Add an Allure's function as a pytest mark to a module or a package (see the [Marking whole classes or modules](https://docs.pytest.org/en/7.3.x/example/markers.html#marking-whole-classes-or-modules) section in pytest documentation). Tip: If you set the fields using the decorators or pytest marks, you can then select and run tests by some metadata fields, see [Options affecting test selection](/docs/pytest-configuration/#options-affecting-test-selection). **Decorators API:** ```python import allure @allure.title("Test Authentication") @allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") @allure.tag("NewUI", "Essentials", "Authentication") @allure.severity(allure.severity_level.CRITICAL) @allure.label("owner", "John Doe") @allure.link("https://dev.example.com/", name="Website") @allure.issue("AUTH-123") @allure.testcase("TMS-456") def test_authentication(): ... ``` **Runtime API:** ```python import allure def test_authentication(): allure.dynamic.title("Test Authentication") allure.dynamic.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") allure.dynamic.tag("NewUI", "Essentials", "Authentication") allure.dynamic.severity(allure.severity_level.CRITICAL) allure.dynamic.label("owner", "John Doe") allure.dynamic.link("https://dev.example.com/", name="Website") allure.dynamic.issue("AUTH-123") allure.dynamic.testcase("TMS-456") ... ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure Pytest provides functions to assign the relevant fields to tests either by adding decorators or marks or by calling the “dynamic” functions (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Decorators API:** ```python import allure @allure.epic("Web interface") @allure.feature("Essential features") @allure.story("Authentication") def test_authentication(): ... ``` **Runtime API:** ```python import allure def test_authentication(): allure.dynamic.epic("Web interface") allure.dynamic.feature("Essential features") allure.dynamic.story("Authentication") ... ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Decorators API:** ```python import allure @allure.parent_suite("Tests for web interface") @allure.suite("Tests for essential features") @allure.sub_suite("Tests for authentication") def test_authentication(): ... ``` **Runtime API:** ```python import allure def test_authentication(): allure.dynamic.parent_suite("Tests for web interface") allure.dynamic.suite("Tests for essential features") allure.dynamic.sub_suite("Tests for authentication") ... ``` A test's location in the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy) is defined by the fully qualified named of the classes and functions they are declared in, with common prefixes shown as parent packages. ### Divide a test into steps Allure Pytest provides two ways of [creating steps and sub-steps](/docs/steps/): “decorated steps” and “context steps”, both implemented by [`allure.step()`](/docs/pytest-reference/#test-steps). **Decorated steps:** ```python import allure def test_example(): steps = Steps() steps.step1() steps.step2() class Steps: @allure.step("Step 1") def step1(self): ... @allure.step("Step 2") def step2(self): ... ``` **Context steps:** ```python import allure def test_example(): with allure.step("Step 1"): ... with allure.step("Step 2"): ... ``` ### Describe parametrized tests When using the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, Allure Pytest automatically adds the parameter values to the test report. Allure supports all the methods of parametrizing tests supported by pytest, including [declaring parameters for fixtures](https://docs.pytest.org/en/latest/how-to/fixtures.html#fixture-parametrize). Additionally, you can: - incorporate a parameter's value into the test titles, see [`title()`](/docs/pytest-reference/#title); - manually add a pseudo-parameter, see [`parameter()`](/docs/pytest-reference/#parametrized-tests). - override a parameter's value with a more readable representation, see [`parameter()`](/docs/pytest-reference/#parametrized-tests); ```python from os.path import basename, expanduser import allure import pytest @pytest.mark.parametrize("login", [ "johndoe", "johndoe@example.com", ]) def test_authentication(login): ... def test_authentication_with_empty_login(): allure.dynamic.parameter("login", "(empty)") ... @pytest.mark.parametrize("ssh_key", [ expanduser("~/.ssh/id_rsa1"), expanduser("~/.ssh/id_rsa2"), expanduser("~/.ssh/id_rsa3"), ]) def test_authentication_with_ssh_key(ssh_key): allure.dynamic.parameter("ssh_key", basename(ssh_key)) ... ``` Tip: For more information, you can consult the guide on [Pytest parameterization](/docs/guides/pytest-parameterization/). ### Describe fixtures The pytest's concept of **fixtures** provides a way for each test to define features it requires by just specifying function arguments with their names. Some fixtures are provided by pytest itself, others can be provided by additional libraries or your own code. See the [pytest documentation](https://docs.pytest.org/en/latest/explanation/fixtures.html) for more details. When displaying a test in a test report, Allure displays operations related to fixtures similarly to how it displays [test steps](#divide-a-test-into-steps). To make it easier to understand a fixture's purpose, you can specify its title using the [`@allure.title()`](/docs/pytest-reference/#title) decorator. ```python import allure import pytest @pytest.fixture() @allure.title("Prepare for the test") def my_fixture(): ... # set up yield ... # tear down def test_with_my_fixture(my_fixture): ... ``` Images: /images/python/pytest/fixtures-allure3-light.png, /images/python/pytest/fixtures-allure3-dark.png, /images/python/pytest/fixtures-allure2-light.png, /images/python/pytest/fixtures-allure2-dark.png ### Attach screenshots and other files Tip: Check out our guides for screenshots and videos in Pytest using Selenium or Playwright plugin: - [Pytest & Selenium: screenshots](/docs/guides/pytest-selenium-screenshots/) - [Pytest & Playwright: screenshots](/docs/guides/playwright-pytest-screenshots/) - [Pytest & Playwright: videos](/docs/guides/pytest-playwright-video/) You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. To do so, use [`allure.attach()`](/docs/pytest-reference/#attach) or [`allure.attach.file()`](/docs/pytest-reference/#attachfile). ```python import allure from pathlib import Path def test_attach(page): page.goto("https://example.com/") png_bytes = page.screenshot() allure.attach( png_bytes, name="full-page", attachment_type=allure.attachment_type.PNG ) def test_attach_file(page): page.goto("https://example.com/") png_bytes = page.screenshot() Path("full-page.png").write_bytes(png_bytes) allure.attach.file( "full-page.png", name="full-page", attachment_type=allure.attachment_type.PNG ) ``` By default, Allure Pytest adds the following data as attached pseudo-files: | Attachment name | Contents | | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `stdout` | All data that was written to `sys.stdout`, e.g., via [`print(...)`](https://docs.python.org/3/library/functions.html#print). | | `stderr` | All data that was written to `sys.stderr`, e.g., via [`print(..., file=sys.stderr)`](https://docs.python.org/3/library/functions.html#print). | | `log` | All data that was logged to the standard logging mechanism, e.g., via [`logging.debug(...)`](https://docs.python.org/3/library/logging.html#logging.Logger.debug). | This behavior can be disabled by the [`--allure-no-capture`](/docs/pytest-configuration/#allure-no-capture) option. ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, pytest will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json python -m pytest ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" python -m pytest ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and Python version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/python/environment-allure3-light.png, /images/python/environment-allure3-dark.png, /images/python/environment-allure2-light.png, /images/python/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## Pytest configuration > URL: https://allurereport.org/docs/pytest-configuration/ # Allure Pytest configuration The [Allure Pytest](/docs/pytest/) integration provides some additional configuration options that you can pass to pytest. These options affect [how Allure Pytest collects the test results](#options-affecting-results-collection) and [how pytest itself selects which tests to run](#options-affecting-test-selection). To pass an option, you can just add it to the command when running pytest. For example: ```bash python -m pytest --alluredir allure-results --clean-alluredir ``` However, if you want some options to be applies for all pytest runs, add them to the pytest's [`addopts`](https://docs.pytest.org/en/latest/reference/reference.html#confval-addopts) configuration option. It must be declared in the project's configuration file. **pyproject.toml:** ```toml [tool.pytest.ini_options] addopts = ["--alluredir", "allure-results", "--clean-alluredir"] ``` **pytest.ini:** ```ini [pytest] addopts = --alluredir allure-results --clean-alluredir ``` ## Options affecting results collection ### --alluredir ⟨DIRECTORY⟩ Path to the directory where Allure Pytest will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. ### --clean-alluredir If set, the directory specified by [`--alluredir`](#--alluredir-directory) will be cleaned before generating new test results. By default, the existing data is kept intact, which allows combining results of multiple test runs into a single test report. ### --allure-no-capture If set, Allure Pytest will not automatically produce the `stdout`, `stderr` and `log` attachments, described in [Attach screenshots and other files](/docs/pytest/#attach-screenshots-and-other-files). ### --allure-link-pattern ⟨TYPE⟩:⟨PATTERN⟩ Define a **link pattern** — a template that can be used to construct full URLs from short identifiers, see the [reference](/docs/pytest-reference/#link). The option accepts two values, separated by a colon. - `TYPE` is an arbitrary string identifier. - `PATTERN` is a string containing `{}` at the position where the identifier should be placed. The given `PATTERN` will be applied when processing any link of the given `TYPE`. You can use the option multiple times, thus defining multiple link patterns for different link types. For example, here's a way to define a link pattern for issues in a GitHub repository: ```bash python -m pytest --allure-link-pattern issue:https://github.com/allure-framework/allure-python/issues/{} ``` ## Options affecting test selection Each one of these options, except for [`--inversion`](#--inversion), accepts a comma-separated list of values that narrow down the selection of tests pytest should run. When two or more of such options are used together, pytest will run tests matching any of the conditions. Warning: The options don't work with the metadata added dynamically inside the code of the tests, see [Specify description, links and other metadata](/docs/pytest/#specify-description-links-and-other-metadata). ### --allure-severities ⟨SEVERITY1⟩,⟨SEVERITY2⟩,... Only run tests that have one of the specified **severity** values, see [Specify description, links and other metadata](/docs/pytest/#specify-description-links-and-other-metadata). ### --allure-epics ⟨EPIC1⟩,⟨EPIC2⟩,... Only run tests that have one of the specified **epic** values, see [Behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). ### --allure-features ⟨FEATURE1⟩,⟨FEATURE2⟩,... Only run tests that have one of the specified **feature** values, see [Behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). ### --allure-stories ⟨STORY1⟩,⟨STORY2⟩,... Only run tests that have one of the specified **story** values, see [Behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). ### --allure-ids ⟨ID1⟩,⟨ID2⟩,... Only run tests that have one of the specified **IDs**, see [Specify description, links and other metadata](/docs/pytest/#specify-description-links-and-other-metadata). ### --allure-label ⟨LABEL1⟩=⟨VALUE1⟩,⟨LABEL2⟩=⟨VALUE2⟩,... Only run tests that have one of the specified **labels**, see [Specify description, links and other metadata](/docs/pytest/#specify-description-links-and-other-metadata). ### --inversion If set, pytest will only run the tests that **do not** match the conditions specified by other arguments or the test plan. --- ## Pytest reference > URL: https://allurereport.org/docs/pytest-reference/ # Allure Pytest reference These are the functions that you can use to integrate your pytest tests with Allure. Allure Pytest provides more than one way to use some features. In most cases, one way involves using a decorator on the test function, while the other involves a method call inside the test function body. The latter style is called “dynamic” and allows to construct values dynamically. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). Some metadata fields can be used for selecting which test to run, see [Specify description, links and other metadata](/docs/pytest/#specify-description-links-and-other-metadata) and [Options affecting test selection](/docs/pytest-configuration/#options-affecting-test-selection). ### Title - `@allure.title(test_title: str)` - `allure.dynamic.title(test_title: str)` Set the test's [title](/docs/v3/readability/#title). The decorator variant of the function can also be used for [describing pytest fixtures](/docs/pytest/#describe-fixtures). **Decorators API:** ```python import allure import pytest @allure.title("Test Authentication") def test_authentication(): ... ``` **Decorators API on a fixture:** ```python import allure import pytest @pytest.fixture() @allure.title("Prepare for the test") def my_fixture(): ... def test_with_my_fixture(my_fixture): ... ``` **Runtime API:** ```python import allure import pytest def test_authentication(): allure.dynamic.title("Test Authentication") ... ``` If the test function has parameters (see [Parametrized tests](#parametrized-tests) below), you can include the parameters' values in the title via **replacement fields**. A replacement field is a parameter name delimited by braces `{}`, as supported by the standard [`str.format()`](https://docs.python.org/3/library/stdtypes.html#str.format) method. In addition to the parameters themselves, a special field `{param_id}` is supported for inserting the parameter set ID (see the [pytest documentation](https://docs.pytest.org/en/latest/example/parametrize.html#different-options-for-test-ids)). Note that the replacement fields are only supported in the decorator variant of the `title()` function. **Using the parameter name:** ```python import allure import pytest @pytest.mark.parametrize("login", ["johndoe", "johndoe@example.com"]) @allure.title("Test Authentication (as {login})") def test_authentication(login): ... ``` **Using the parameter set ID:** ```python import allure import pytest @pytest.mark.parametrize("login", ["johndoe", "johndoe@example.com"], ids=["using-username", "using-email"]) @allure.title("Test Authentication ({param_id})") def test_authentication(login): ... ``` ### Description - `@allure.description(test_description: str)` - `allure.dynamic.description(test_description: str)` Set the test's [description](/docs/v3/readability/#description). If not provided, Allure will use the docstring of the test function. Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. **Decorators API:** ```python import allure @allure.description(""" This test attempts to log into the website using a login and a password. Fails if any error happens. Note that this test does not test 2-Factor Authentication. """) def test_authentication(): ... ``` **Runtime API:** ```python import allure def test_authentication(): allure.dynamic.description(""" This test attempts to log into the website using a login and a password. Fails if any error happens. Note that this test does not test 2-Factor Authentication. """) ... ``` **Docstring:** ```python def test_authentication(): """ This test attempts to log into the website using a login and a password. Fails if any error happens. Note that this test does not test 2-Factor Authentication. """ ... ``` ### Tag - `@allure.tag(*tags: str)` - `allure.dynamic.tag(*tags: str)` Set the test's [tags](/docs/v3/readability/#tags). Both variants of the function can accept as many tags at a time as needed, for example: **Decorators API:** ```python import allure @allure.tag("NewUI", "Essentials", "Authentication") def test_authentication(): ... ``` **Runtime API:** ```python import allure def test_authentication(): allure.dynamic.tag("NewUI", "Essentials", "Authentication") ... ``` ### Severity - `@allure.severity(severity_level: str)` - `allure.dynamic.severity(severity_level: str)` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Decorators API:** ```python import allure from allure_commons.types import Severity @allure.severity(Severity.CRITICAL) def test_authentication(): ... ``` **Runtime API:** ```python import allure from allure_commons.types import Severity def test_authentication(): allure.dynamic.severity(Severity.CRITICAL) ... ``` ### Label - `@allure.label(label_type: str, *labels: str)` - `allure.dynamic.label(label_type: str, *labels: str)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. The first argument of the function is the label name. It can be a constant from the `LabelType` class or any other string. The second and all remaining arguments are values to be added. For each of the values, the function will add a label with the given name and given value. An example of function that passes to `label()` multiple values at once is [`tag()`](#tag). **Decorators API:** ```python import allure from allure_commons.types import LabelType @allure.label(LabelType.LANGUAGE, "python") @allure.label(LabelType.FRAMEWORK, "pytest") def test_authentication(): ... ``` **Runtime API:** ```python import allure from allure_commons.types import LabelType def test_authentication(): allure.dynamic.label(LabelType.LANGUAGE, "python") allure.dynamic.label(LabelType.FRAMEWORK, "pytest") ... ``` ### Allure ID (Allure TestOps) - `@allure.id(id: str)` - `allure.dynamic.id(id: str)` Set the test's [ID](/docs/v3/readability/#id). **Decorators API:** ```python import allure @allure.id("123") def test_authentication(): ... ``` ### Manual (Allure TestOps) - `@allure.manual` - `allure.dynamic.manual()` Instructs Allure TestOps to create or update a manual test case from the test result. **Decorators API:** ```python import allure @allure.manual def test_authentication(): ... ``` ### Link - `@allure.link(url: str, link_type: str = LinkType.LINK, name: str = None)` - `@allure.issue(url: str, name: str = None)` - `@allure.testcase(url: str, name: str = None)` - `allure.dynamic.link(url: str, link_type: str = LinkType.LINK, name: str = None)` - `allure.dynamic.issue(url: str, name: str = None)` - `allure.dynamic.testcase(url: str, name: str = None)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `link_type` (which can be any string), Allure will try to load a corresponding **link pattern** to process the `url`, as defined by the [`--allure-link-pattern`](/docs/pytest-configuration/#allure-link-pattern-⟨type⟩-⟨pattern⟩) configuration option. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. For convenience, Allure provides shorthand functions and methods with pre-selected link types `issue` and `tms`. **Decorators API:** ```python import allure @allure.link("https://dev.example.com/", name="Website") @allure.issue("AUTH-123") @allure.testcase("TMS-456") def test_authentication(): ... ``` **Decorators API (custom decorator):** ```python import allure def pr(pr_id): return allure.link(pr_id, name="PR {}".format(pr_id), link_type="pr") @pr(123) def test_authentication(): ... ``` **Runtime API:** ```python import allure def test_authentication(): allure.dynamic.link("https://dev.example.com/", name="Website") allure.dynamic.issue("AUTH-123") allure.dynamic.testcase("TMS-456") ... ``` ## Behavior-based hierarchy - `@allure.epic(*epics: str)` - `@allure.feature(*features: str)` - `@allure.story(*stories: str)` - `allure.dynamic.epic(*epics: str)` - `allure.dynamic.feature(*features: str)` - `allure.dynamic.story(*stories: str)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). Each function support setting one or more values at once. **Decorators API:** ```python import allure @allure.epic("Web interface") @allure.feature("Essential features") @allure.story("Authentication") def test_authentication(): ... ``` **Runtime API:** ```python import allure def test_authentication(): allure.dynamic.epic("Web interface") allure.dynamic.feature("Essential features") allure.dynamic.story("Authentication") ... ``` ## Suite-based hierarchy - `@allure.parent_suite(parent_suite_name)` - `@allure.suite(suite_name)` - `@allure.sub_suite(sub_suite_name)` - `allure.dynamic.parent_suite(parent_suite_name)` - `allure.dynamic.suite(suite_name)` - `allure.dynamic.sub_suite(sub_suite_name)` Assign names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). By default, Allure Pytest sets the parent suite and the suite based on the module containing the test. For example, a test in a module named `tests.web.essentials.auth` will get assigned the `tests.web.essentials` parent suite and the `auth` suite. **Decorators API:** ```python import allure @allure.parent_suite("Tests for web interface") @allure.suite("Tests for essential features") @allure.sub_suite("Tests for authentication") def test_authentication(): ... ``` **Runtime API:** ```python import allure def test_authentication(): allure.dynamic.parent_suite("Tests for web interface") allure.dynamic.suite("Tests for essential features") allure.dynamic.sub_suite("Tests for authentication") ... ``` ## Test steps - `@allure.step(title: str)` - `with allure.step(title: str)` Define a [test step](/docs/steps/) with the given `title`. There are two ways of defining a step. - **Decorated steps** Define a function or a method containing a test step and decorate it with `@allure.step()`. If it accepts arguments, you can include their values in the step title via **replacement fields**. A replacement field is a parameter name delimited by braces `{}`, as supported by the standard [`str.format()`](https://docs.python.org/3/library/stdtypes.html#str.format) method. - **Context steps** Write a test step in-place but use the `with allure.step()` statement to create its context. **Decorated steps:** ```python import allure def test_example(): step1() for val in ["val1", "val2", "val3"]: step2(val) @allure.step("Step 1") def step1(): ... @allure.step("Step 2 (with value {val})") def step2(val): ... ``` **Context steps:** ```python import allure def test_example(): with allure.step("Step 1"): ... for val in ["val1", "val2", "val3"]: with allure.step(f"Step 2 (with value {val})"): ... ``` ## Parametrized tests - `allure.dynamic.parameter(name, value, excluded=None, mode=None)` Specify a `name` and `value` of a [parameter](/docs/v3/readability/#parametrized-tests) that was used during this test. This can be used for adding the parameters even to those function that do not use the [pytest's parameterization features](https://docs.pytest.org/en/latest/how-to/parametrize.html). In the second example below, there are no test parameters as far as pytest is concerned, however, the test report will display the “login” parameter. You can also override display values for the existing parameters. In the third example below, the test report will use the short value for the parameter instead of the full path. Note that changing the display value does not affect the [unique identifier](/docs/history-and-retries/#making-sure-tests-are-identified-correctly) for the tests history. The parameters can be used in the test's title, see [Title](#title). If the `excluded` argument is set to True, Allure will not use the parameter when comparing the current test result with previous one in the history. This argument is only used by Allure TestOps. The `mode` argument affects how the parameter will be displayed in the report. Available options are defined in the `allure.parameter_mode` enumeration: - `allure.parameter_mode.DEFAULT` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `allure.parameter_mode.MASKED` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `allure.parameter_mode.HIDDEN` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. ```python from os.path import basename, expanduser import allure import pytest @pytest.mark.parametrize("login", [ "johndoe", "johndoe@example.com", ]) def test_authentication(login): ... def test_authentication_with_empty_login(): allure.dynamic.parameter("login", "(empty)") ... @pytest.mark.parametrize("ssh_key", [ expanduser("~/.ssh/id_rsa1"), expanduser("~/.ssh/id_rsa2"), expanduser("~/.ssh/id_rsa3"), ]) def test_authentication_with_ssh_key(ssh_key): allure.dynamic.parameter("ssh_key", basename(ssh_key)) ... ``` Tip: For more information, you can consult the guide on [Pytest parameterization](/docs/guides/pytest-parameterization/). ## Attachments Functions for [adding attachments](/docs/attachments/) to test results. Tip: Check out our guides for screenshots and videos in Pytest using Selenium or Playwright plugin: - [Pytest & Selenium: screenshots](/docs/guides/pytest-selenium-screenshots/) - [Pytest & Playwright: screenshots](/docs/guides/playwright-pytest-screenshots/) - [Pytest & Playwright: videos](/docs/guides/pytest-playwright-video/) ### Attaching content from variables - `allure.attach(body, name=None, attachment_type="text/plain", extension="attach")` Add `body` as an attachment to the test result under the given `name` (defaults to a unique pseudo-random string). The `body` must be of type `bytes` or `str`. Info: You can use data produced by any function, not necessarily read from an actual file. For attaching contents of existing files, use [`attach.file()`](#reading-attachments-from-files) instead. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. There are two ways to do this: - Pass the media type of the content as `attachment_type` and, optionally, a filename extension as `extension`. For example: ```python allure.attach( '{"value":100}', attachment_type="application/json", extension="json" ) ``` Some popular media types are `image/png` and `image/jpeg` for screenshots and other images, `application/json` for JSON data, and `text/plain` for text files. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. - Pass a value from the `allure.attachment_type` class as `attachment_type`. For example: ```python allure.attach( '{"value":100}', attachment_type=allure.attachment_type.JSON ) ``` This will automatically set the media type and the appropriate filename extension. The example below is of a test that uses the playwright-pytest plugin to load a page and get its screenshot as bytes. The screenshot is then attached to the test result as `full-page.png`. ```python import allure def test_attach(page): page.goto("https://example.com/") png_bytes = page.screenshot() allure.attach( png_bytes, name="full-page", attachment_type=allure.attachment_type.PNG ) ``` ### Reading attachments from files - `allure.attach.file(source, name=None, attachment_type=None, extension=None)` Same as [`attach()`](#attach), but the content is loaded from the existing `source` file. ```python import allure from pathlib import Path def test_attach_file(page): page.goto("https://example.com/") png_bytes = page.screenshot() Path("full-page.png").write_bytes(png_bytes) allure.attach.file( "full-page.png", name="full-page", attachment_type=allure.attachment_type.PNG ) ``` --- ## Pytest-BDD > URL: https://allurereport.org/docs/pytestbdd/ # Getting started with Allure Pytest-BDD [![Allure Pytest-BDD pypi latest version](https://img.shields.io/pypi/v/allure-pytest-bdd?style=flat "Allure Pytest-BDD pypi latest version")](https://pypi.org/project/allure-pytest-bdd/) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Pytest-BDD](https://pytest-bdd.readthedocs.io/) tests. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. If necessary for your system configuration, activate the virtual Python environment for your project. For example, if the project uses a `venv` environment, the command to activate it may look like this: ```bash source .venv/bin/activate ``` This step is not necessary if you are using the system Python environment. 1. Install the Allure Pytest-BDD integration. ```bash pip install allure-pytest-bdd ``` ### 2. Run tests When running your tests, specify a path for the test results directory in the `--alluredir` command-line argument. For example: ```bash python -m pytest --alluredir allure-results ``` This will save necessary data into the test results directory. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ### Describe parametrized tests Allure Pytest-BDD can display parameters passed to the tests via `Examples` tables, see the [reference](/docs/pytestbdd-reference/#parametrized-tests). ```gherkin Feature: User management Scenario Outline: Registration When I go to the registration form And I enter my details: , , , Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ## Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure Pytest-BDD provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/pytestbdd-reference/#attachments). ```python import allure import requests from pytest_bdd import then @then("I open labels page") def test_labels(): ... png_bytes = requests.get('https://example.com/image.png').content allure.attach(png_bytes, name="my-image", attachment_type=allure.attachment_type.PNG) allure.attach.file('/path/to/image.png', name="my-image", attachment_type=allure.attachment_type.PNG) ``` ## Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and Python version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/python/environment-allure3-light.png, /images/python/environment-allure3-dark.png, /images/python/environment-allure2-light.png, /images/python/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## Pytest-BDD configuration > URL: https://allurereport.org/docs/pytestbdd-configuration/ # Allure Pytest-BDD configuration The [Allure Pytest-BDD](/docs/pytestbdd/) integration provides some additional configuration options that you can pass to pytest. To pass an option, you can just add it to the command when running pytest. For example: ```bash python -m pytest --alluredir allure-results --clean-alluredir ``` However, if you want some options to be applies for all pytest runs, add them to the pytest's [`addopts`](https://docs.pytest.org/en/latest/reference/reference.html#confval-addopts) configuration option. It must be declared in the project's configuration file. **pyproject.toml:** ```toml [tool.pytest.ini_options] addopts = [ "--alluredir", "allure-results", "--clean-alluredir" ] ``` **pytest.ini:** ```ini [pytest] addopts = --alluredir allure-results --clean-alluredir ``` ## --alluredir ⟨DIRECTORY⟩ Path to the directory where Allure Pytest-BDD will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. ## --clean-alluredir If set, the directory specified by [`--alluredir`](#--alluredir-directory) will be cleaned before generating new test results. By default, the existing data is kept intact, which allows combining results of multiple test runs into a single test report. --- ## Pytest-BDD reference > URL: https://allurereport.org/docs/pytestbdd-reference/ # Allure Pytest-BDD reference These are the attributes and methods that you can use to integrate your Pytest-BDD tests with Allure Report. ## Parametrized tests In Gherkin, a [`Scenario Outline`](https://pytest-bdd.readthedocs.io/en/stable/#scenario-outlines) (or a `Scenario Template`) implements the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. A scenario outline must contain an `Examples` table, from which Pytest-BDD loads sets of parameters, one row after another. Each set of parameters is being placed into the step declarations according to the placeholders, thus generating a new scenario based on the row. Pytest-BDD then runs each of them independently, as if it was a separate `Scenario`. The data can then be captured by Pytest-BDD and passed as separate arguments to the Python code. Allure Pytest-BDD automatically recognizes this pattern. No additional configuration is required. The example below shows a Gherkin file and a Python implementation file of a test. In this example, the four parameters for the “I enter my details...” step will be displayed in both instances of the scenario in the test report. ```gherkin Feature: User management Scenario Outline: Registration When I go to the registration form And I enter my details: , , , Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ```python from pytest_bdd import given, parsers, scenario, then, when @scenario('features/UserManagement.feature', 'Registration') def test_parameters(): pass @when('I go to the registration form') def i_go_to_registration_form(): ... @when(parsers.parse('I enter my details: {login}, {password}, {name}, {birthday}')) def i_enter_my_details(login, password, name, birthday): ... @then('the profile should be created') def profile_should_be_created(): ... ``` ## Attaching content from variables - `allure.attach(body, name=None, attachment_type="text/plain", extension="attach")` Add `body` as an attachment to the test result under the given `name` (defaults to a unique pseudo-random string). The `body` must be of type `bytes` or `str`. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. There are two ways to do this: - Pass the media type of the content as `attachment_type` and, optionally, a filename extension as `extension`. Some popular media types are `image/png` and `image/jpeg` for screenshots and other images, `application/json` for JSON data, and `text/plain` for text files. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. - Pass a value from the `allure.attachment_type` class as `attachment_type`. This will automatically set the media type and the appropriate filename extension. ```python import allure import requests from pytest_bdd import then @then("I open labels page") def test_labels(): ... png_bytes = requests.get('https://example.com/image.png').content allure.attach(png_bytes, name="my-image", attachment_type=allure.attachment_type.PNG) ``` ## Reading attachments from files - `allure.attach.file(source, name=None, attachment_type=None, extension=None)` Same as [`attach()`](#attaching-content-from-variables), but the content is loaded from the existing `source` file. ```python import allure from pytest_bdd import then @then("I open labels page") def test_labels(): ... allure.attach.file('/path/to/image.png', name="my-image", attachment_type=allure.attachment_type.PNG) ``` --- ## Reqnroll > URL: https://allurereport.org/docs/reqnroll/ # Getting started with Allure Reqnroll [![Allure Reqnroll latest release](https://img.shields.io/nuget/v/Allure.Reqnroll?style=flat "allure-reqnroll")](https://www.nuget.org/packages/Allure.Reqnroll) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Reqnroll](https://reqnroll.net/) tests. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Make sure that you have a compatible .NET version. Allure Reqnroll requires a framework that implements [.NET Standard version 2.0](https://learn.microsoft.com/en-us/dotnet/standard/net-standard?tabs=net-standard-2-0#select-net-standard-version). You can find the full list of compatible frameworks [on Allure.Reqnroll's page in NuGet](https://www.nuget.org/packages/Allure.Reqnroll#supportedframeworks-body-tab). 1. Add `Allure.Reqnroll` to your project's dependencies using your IDE or the command line. For example, here is how the dependency can be added to a project with the `dotnet` CLI: ```plain dotnet add ⟨PATH TO PROJECT⟩ package Allure.Reqnroll ``` ### 2. Run tests Run your Reqnroll tests the same way as you would run them usually. For example: ```plain dotnet test ``` This will save necessary data into `allure-results` or other directory, according to the [`allure.directory`](/docs/reqnroll-configuration/#allure-directory) configuration option. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Reqnroll integration extends the standard reporting features of Reqnroll by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#specify-description-links-and-other-metadata). - **Test Organization**: Structure your tests into [clear hierarchies](#organize-tests) for better readability and organization. - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, you need to indicate to Allure Reqnroll that a certain property needs to be assigned to the test result. Most properties can be assigned via Gherkin tags or via the Runtime API. - **Gherkin tags**: use [Gherkin tags](https://docs.reqnroll.net/latest/gherkin/gherkin-reference.html#tags) to assign various data to a particular `Scenario` or a whole `Feature`. You can configure the patterns for such conversion via regular expressions in the [configuration file](/docs/reqnroll-configuration/). Note that due to a limitation in the Gherkin syntax, a tag cannot contain spaces. When using this approach, the data is guaranteed to be added to the test result regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add data to the test result during the execution of its steps. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/reqnroll-reference/#metadata) for more details. **Gherkin tags:** ```gherkin Feature: Labels @critical @allure.owner:JohnDoe @allure.link:https://dev.example.com/ @allure.issue:AUTH-123 @allure.tms:TMS-456 Scenario: Create new label for authorized user This test attempts to create a label with specified title When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class TestAll { [When("I open labels page")] public void TestLabels() { AllureApi.SetDescription("This test attempts to create a label with specified title"); AllureApi.SetSeverity(SeverityLevel.critical); AllureApi.SetOwner("John Doe"); AllureApi.AddLink("Website", "https://dev.example.com/"); AllureApi.AddIssue("AUTH-123"); AllureApi.AddTmsItem("TMS-456"); // ... } } ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure Reqnroll provides the API to assign the relevant fields to tests either by adding attributes or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Gherkin tags:** ```gherkin @allure.epic:WebInterface Feature: Labels @allure.story:CreateLabels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddEpic("Web interface"); AllureApi.AddFeature("Essential features"); AllureApi.AddStory("Labels"); // ... } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Gherkin tags:** ```gherkin @allure.parentSuite:WebInterface @allure.suite:EssentialFeatures @allure.subSuite:Labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddParentSuite("Web interface"); AllureApi.AddSuite("Essential features"); AllureApi.AddSubSuite("Labels"); // ... } } ``` ### Divide a test into steps Allure Reqnroll provides two ways of [creating steps and sub-steps](/docs/steps/): “lambda steps” and “no-op steps”, see the [reference](/docs/reqnroll-reference/#test-steps). ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [Then("visit pages")] public void ThenVisitPages() { string[] urls = { "https://domain1.example.com/", "https://domain2.example.com/" }; foreach (string url in urls) { AllureApi.Step($"Test {url}", () => { AllureLifecycle.Instance.UpdateStep(stepResult => { stepResult.parameters.Add( new Parameter { name = "Webpage URL", value = url } ); }); AllureApi.Step("Opening web browser..."); // ... AllureApi.Step($"Visiting {url}..."); // ... AllureApi.Step("Closing web browser..."); // ... }); } } } ``` ### Describe parametrized tests Allure Reqnroll support multiple ways of implementing the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern: - [scenario outlines](/docs/reqnroll-reference/#scenario-outlines), - [vertical data tables](/docs/reqnroll-reference/#vertical-data-tables), - [horizontal data tables](/docs/reqnroll-reference/#horizontal-data-tables), - [runtime parameters](/docs/reqnroll-reference/#runtime-parameters). The example below shows a configuration file, a Gherkin file and a C# implementation file of a test. In this example, the four parameters for the “I enter my details” step will be displayed in the test report. ```json { "allure": { "gherkinPatterns": { "stepArguments": { "createFromDataTables": true, "nameColumn": "^Parameter$", "valueColumn": "^Value$" } } } } ``` ```gherkin Feature: User management Scenario: Registration When I go to the registration form And I enter my details | Parameter | Value | | login | johndoe | | password | qwerty | | name | John Doe | | birthday | 1970-01-01 | Then the profile should be created ``` ```csharp using Reqnroll; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When("I enter my details")] public void WhenIEnterMyDetails(Table table) { // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } } ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure Reqnroll provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/reqnroll-reference/#attachments). ```csharp using System.IO; using System.Text; using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddAttachment( "data.txt", "text/plain", Encoding.UTF8.GetBytes("This is the file content.") ); AllureApi.AddAttachment( "image1.png", "image/png", File.ReadAllBytes("/path/to/image1.png") ); AllureApi.AddAttachment( "image2.png", "image/png", "/path/to/image2.png" ); } } ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Reqnroll will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json dotnet test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" dotnet test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and .NET version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/csharp/environment-allure3-light.png, /images/csharp/environment-allure3-dark.png, /images/csharp/environment-allure2-light.png, /images/csharp/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## Reqnroll configuration > URL: https://allurereport.org/docs/reqnroll-configuration/ # Allure Reqnroll configuration This page describes the configuration options that affect the behavior of [Allure Reqnroll](/docs/reqnroll/). By default, Allure Reqnroll tries to read the `allureConfig.json` file in the same directory where the test assembly is located. To make sure the file is present there, edit the file properties in your IDE or manually in your `*.csproj` file and set the corresponding item's `CopyToOutputDirectory` to either `Always` or `PreserveNewest`. For example: ```xml Always ``` Alternatively, use the `ALLURE_CONFIG` environment variable to specify a relative or absolute path of the configuration file. Info: Many parameters for Allure Reqnroll are regular expressions. By default, such parameters will be automatically surrounded with `^` and `$` and interpreted with the default flags. For more control over the regular expression processing, wrap it with forward slashes and add [supported flags](https://github.com/JamesNK/Newtonsoft.Json/blob/release/13.0.3/Src/Newtonsoft.Json/Utilities/MiscellaneousUtils.cs#L148-L172), for example: `"/^allure.label.suite:(.+)$/i"`. ## allure.directory Path to the directory where Allure Reqnroll will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. Note that the path is treated as relative to the build output directory of the project. When building and running tests from an IDE, this is typically different from the solution folder. Multiple `../` elements may be needed if you want to put the test results into a directory in the solution folder. ```json { "allure": { "directory": "../../../../../allure-results" } } ``` ## allure.links List of patterns that can be used to construct full URLs from short identifiers. Each pattern must be a string containing a placeholder surrounded by curly brackets, for example: `{issue}`. When processing a link, Allure Reqnroll selects the pattern whose placeholder matches the link type (as specified using the [`Link`](/docs/reqnroll-reference/#link) arguments or the [`allure.gherkinPatterns.links`](#alluregherkinpatternslinks) patterns). The placeholder is then replaced by the short identifier. The code below represents the default values used by Allure Reqnroll. With this configuration, the link `BUG-123` of type `issue` will be translated to `https://issues.example.com/BUG-123`. ```json { "allure": { "links": [ "https://issues.example.com/{issue}", "https://tms.example.com/{tms}", "https://jira.example.com/browse/{jira}" ] } } ``` ## allure.failExceptions A list of exception types representing assertion failures. In case of an exception that matches one of the listed classes, the [Failed](/docs/test-statuses/#failed) status is assigned to the test, step, or fixture. In case of any other exception, the [Broken](/docs/test-statuses/#broken) status is assigned. When looking for matches, Allure Reqnroll uses not only the exception class's own full name but also all its interfaces and base classes. The default list includes the base classes and interfaces that could be thrown by various runners compatible with Reqnroll: `NUnit.Framework.AssertionException`, `Xunit.Sdk.IAssertionException`, `Xunit.Sdk.XunitException`, `Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException`. The setting, if not empty, overrides the default list. With the example configuration below, a test will be considered failed if it throws `Xunit.Sdk.XunitException`, `MyNamespace.MyCustomException`, or anything that derives from them. At the same time, any other exception will make the test broken. ```json { "allure": { "failExceptions": ["Xunit.Sdk.XunitException", "MyNamespace.MyCustomException"] } } ``` ## allure.gherkinPatterns.stepArguments Configuration options that affect how Allure Reqnroll processes [data tables](https://docs.reqnroll.net/latest/gherkin/gherkin-reference.html#data-tables) found in Gherkin files. - To enable the processing of [Horizontal data tables](/docs/reqnroll-reference/#horizontal-data-tables) only, set `createFromDataTables` to `true`. - To enable the processing of both [Horizontal data tables](/docs/reqnroll-reference/#horizontal-data-tables) and [Vertical data tables](/docs/reqnroll-reference/#vertical-data-tables), set `createFromDataTables` to `true` and specify `nameColumn` and `valueColumn` — the regular expressions for matching the vertical tables' headers. The code below represents the default values used by Allure Reqnroll. With this configuration, a two-columns table with the columns “Parameter” and “Value” will be processed, but a table with any other columns won't, even if there happens to be two of them. ```json { "allure": { "gherkinPatterns": { "stepArguments": { "createFromDataTables": true, "nameColumn": "/^parameter$/i", "valueColumn": "/^value$/i" } } } } ``` ## allure.gherkinPatterns.grouping.behaviors Regular expressions for Gherkin tags that can be used to set a test's epic and story, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). The code below represents the default values used by Allure Reqnroll. With this configuration, a feature with the Gherkin tag `@allure.epic:UI` will be added to the “UI” epic. ```json { "allure": { "gherkinPatterns": { "grouping": { "behaviors": { "epic": "allure\\.epic:(.+)", "story": "allure\\.story:(.+)" } } } } } ``` ## allure.gherkinPatterns.grouping.suites Regular expressions for Gherkin tags that can be used to set a test's suites, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). The code below represents the default values used by Allure Reqnroll. With this configuration, a feature with the Gherkin tag `@allure.suite:TestUI` will be added to the “TestUI” suite. ```json { "allure": { "gherkinPatterns": { "grouping": { "suites": { "parentSuite": "allure\\.parentSuite:(.+)", "suite": "allure\\.suite:(.+)", "subSuite": "allure\\.subSuite:(.+)" } } } } } ``` ## allure.gherkinPatterns.metadata Regular expressions for Gherkin tags that can be used to add an arbitrary [label](/docs/reqnroll-reference/#label) for the test, as well as its [owner](/docs/reqnroll-reference/#owner) and [severity](/docs/reqnroll-reference/#severity) specifically. The `label` regular expression must contain two capture groups. The two captured texts will be used as the name and the value for the label. The `owner` and `severity` regular expressions must contain one capture group. The captured text will be used as the value for the label. The code below represents the default values used by Allure Reqnroll. With this configuration: - the Gherkin tag `@allure.label.language:csharp` will add a label with the name “language” and the value “csharp”, - the Gherkin tag `@allure.owner:JohnDoe` will set the test's owner to “JohnDoe”, - the Gherkin tag `@critical` will set the test's severity to “critical”. ```json { "allure": { "gherkinPatterns": { "metadata": { "label": "allure\\.label\\.([^:]+):(.+)", "owner": "allure\\.owner:(.+)", "severity": "(normal|blocker|critical|minor|trivial)" } } } } ``` ## allure.gherkinPatterns.links Regular expressions for Gherkin tags that can be used to add a [link](/docs/reqnroll-reference/#link) of one of the standard types: “link”, “issue”, or “tms”. The code below represents the default values used by Allure Reqnroll. With this configuration, the Gherkin tag `@allure.issue:BUG-123` will add a link “BUG-123” of type “issue”. Optionally, it can then be processed according to the [`allure.links`](#allurelinks) patterns. ```json { "allure": { "gherkinPatterns": { "links": { "link": "allure\\.link:(.+)", "issue": "allure\\.issue:(.+)", "tms": "allure\\.tms:(.+)" } } } } ``` --- ## Reqnroll reference > URL: https://allurereport.org/docs/reqnroll-reference/ # Allure Reqnroll reference These are the attributes and methods that you can use to integrate your Reqnroll tests with Allure Report. In most cases, you need to indicate to Allure Reqnroll that a certain property needs to be assigned to the test result. Most properties can be assigned via Gherkin tags or via the Runtime API. - **Gherkin tags**: use [Gherkin tags](https://docs.reqnroll.net/latest/gherkin/gherkin-reference.html#tags) to assign various data to a particular `Scenario` or a whole `Feature`. You can configure the patterns for such conversion via regular expressions in the [configuration file](/docs/reqnroll-configuration/). Note that due to a limitation in the Gherkin syntax, a tag cannot contain spaces. When using this approach, the data is guaranteed to be added to the test result regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add data to the test result during the execution of its steps. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `AllureApi.SetTestName(string name)` Set the test's [title](/docs/v3/readability/#title). ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.SetTestName("Open labels page"); // ... } } ``` ### Description - `AllureApi.SetDescription(string description)` Set the test's [description](/docs/v3/readability/#description). Allure Reqnroll uses a scenario's description from the Gherkin file, if present. Alternatively, use Runtime API to set the description dynamically. Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. **Gherkin description:** ```gherkin Feature: Labels Scenario: Create new label for authorized user This test attempts to create a label with specified title When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```java [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.SetDescription("This test attempts to create a label with specified title"); // ... } } ``` ### Owner - `@allure.owner:⟨VALUE⟩` - `AllureApi.SetOwner(string owner)` Set the test's [owner](/docs/v3/readability/#owner). The format for the Gherkin tags is configured via the [`allure.gherkinPatterns.metadata.owner`](/docs/reqnroll-configuration/#allure-gherkinpatterns-metadata) option. **Gherkin tags:** ```gherkin @allure.owner:JohnDoe Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void ThenLabels() { AllureApi.SetOwner("John Doe"); // ... } } ``` ### Tag - `AllureApi.AddTags(params string[] tags)` Set the test's [tags](/docs/v3/readability/#tags). Any [Gherkin tag](https://docs.reqnroll.net/latest/gherkin/gherkin-reference.html#tags) is automatically added to the list of the test's tags, unless it matches one of the regular expressions in the [configuration](/docs/reqnroll-configuration/). **Gherkin tags:** ```gherkin @UI @Labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddTags("UI", "Labels"); // ... } } ``` ### Severity - `AllureApi.SetSeverity(SeverityLevel severity)` Set the test's [severity](/docs/v3/readability/#severity). The format for the Gherkin tags is configured via the [`allure.gherkinPatterns.metadata.severity`](/docs/reqnroll-configuration/#allure-gherkinpatterns-metadata) option. **Gherkin tags:** ```gherkin Feature: Labels @critical Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.SetSeverity(SeverityLevel.critical); // ... } } ``` ### Label - `@allure.label.⟨NAME⟩:⟨VALUE⟩` - `AllureApi.AddLabel(string name, string value)` - `AllureApi.AddLabel(Label label)` - `AllureApi.AddLabels(params Label[] labels)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other APIs. The format for the Gherkin tags is configured via the [`allure.gherkinPatterns.metadata.label`](/docs/reqnroll-configuration/#allure-gherkinpatterns-metadata) option. **Gherkin tags:** ```gherkin @allure.label.layer:web @allure.label.owner:eroshenkoam @allure.label.page:/{org}/{repo}/labels @allure.label.jira:AE-2 Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddLabel("layer", "web"); AllureApi.AddLabel("owner", "eroshenkoam"); AllureApi.AddLabel("page", "/{org}/{repo}/labels"); AllureApi.AddLabel("jira", "AE-2"); // ... } } ``` ### Allure ID (Allure TestOps) - `@allure.label.allure_id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). The format for the Gherkin tags is configured via the [`allure.gherkinPatterns.metadata.label`](/docs/reqnroll-configuration/#allure-gherkinpatterns-metadata) option. In the tag, use `allure_id` as the label name. **Gherkin tags:** ```gherkin Feature: Labels @allure.label.allure_id:123 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Link - `@allure.link:⟨URL⟩` - `@allure.issue:⟨URL⟩` - `@allure.tms:⟨URL⟩` - `AllureApi.AddLink(string url)` - `AllureApi.AddLink(string name, string url)` - `AllureApi.AddLink(string name, string type, string url)` - `AllureApi.AddLinks(params Link[] links)` - `AllureApi.AddIssue(string url)` - `AllureApi.AddIssue(string name, string url)` - `AllureApi.AddTmsItem(string url)` - `AllureApi.AddTmsItem(string name, string url)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string; defaults to “link”), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`allure.links`](/docs/reqnroll-configuration/#allure-links) section of the configuration file. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. Additionally, you can use the [`allure.gherkinPatterns.links`](/docs/reqnroll-configuration/#allure-gherkinpatterns-links) section of the configuration file to specify formats for adding “link”, “issue” and “tms” links via Gherkin tags. **Gherkin tags:** ```gherkin @allure.link:https://dev.example.com/ @allure.issue:UI-123 @allure.tms:TMS-456 Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddLink("Website", "https://dev.example.com/"); AllureApi.AddIssue("UI-123"); AllureApi.AddTmsItem("TMS-456"); // ... } } ``` ## Behavior-based hierarchy - `@allure.epic:⟨VALUE⟩` - `@allure.story:⟨VALUE⟩` - `AllureApi.AddEpic(string epic)` - `AllureApi.AddFeature(string feature)` - `AllureApi.AddStory(string story)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). The format for the Gherkin tags is configured via the [`allure.gherkinPatterns.grouping.behaviors`](/docs/reqnroll-configuration/#allure-gherkinpatterns-grouping-behaviors) options. **Gherkin tags:** ```gherkin @allure.epic:WebInterface Feature: Labels @allure.story:CreateLabels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddEpic("Web interface"); AllureApi.AddFeature("Essential features"); AllureApi.AddStory("Labels"); // ... } } ``` ## Suite-based hierarchy - `@allure.parentSuite:⟨VALUE⟩` - `@allure.suite:⟨VALUE⟩` - `@allure.subSuite:⟨VALUE⟩` - `AllureApi.AddParentSuite(string parentSuite)` - `AllureApi.AddSuite(string suite)` - `AllureApi.AddSubSuite(string subSuite)` Assign the names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). The format for the Gherkin tags is configured via the [`allure.gherkinPatterns.grouping.suites`](/docs/reqnroll-configuration/#allure-gherkinpatterns-grouping-suites) options. **Gherkin tags:** ```gherkin @allure.parentSuite:WebInterface @allure.suite:EssentialFeatures @allure.subSuite:Labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddParentSuite("Web interface"); AllureApi.AddSuite("Essential features"); AllureApi.AddSubSuite("Labels"); // ... } } ``` ## Test steps - `AllureApi.Step(string name)` - `AllureApi.Step(string name, Action acti` - `T AllureApi.Step(string name, Func function)` - `async Task AllureApi.Step(string name, Func action)` - `async Task AllureApi.Step(string name, Func> function)` Define a [test sub-steps](/docs/steps/) with the given `name`. There are two ways of defining a sub-step. - **Lambda sub-steps** Write a test sub-step in a lambda function and pass it to `Step()`. If the lambda function returns a value, `Step()` will return it without modification, and it will not affect the report. Use `AllureLifecycle.Instance.UpdateStep()` to add custom parameters to the current sub-step, as shown in the example below. - **No-op sub-steps** If you call `Step()` with a name but without a lambda function, Allure will add to the report a no-op sub-step. This allows for a log-style reporting within a larger step. A no-op sub-step finishes immediately after it started and cannot have any parameters, attachments or sub-steps of its own. In the example below, a new lambda sub-step is created at runtime for each URL from a list. The URL is added to the list of the sub-step's parameters. Within each of the sub-steps, a sequence of three no-op sub-steps occurs. ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [Then("visit pages")] public void ThenVisitPages() { string[] urls = { "https://domain1.example.com/", "https://domain2.example.com/" }; foreach (string url in urls) { AllureApi.Step($"Test {url}", () => { AllureLifecycle.Instance.UpdateStep(stepResult => { stepResult.parameters.Add( new Parameter { name = "Webpage URL", value = url } ); }); AllureApi.Step("Opening web browser..."); // ... AllureApi.Step($"Visiting {url}..."); // ... AllureApi.Step("Closing web browser..."); // ... }); } } } ``` ## Parametrized tests Allure Reqnroll support multiple ways of implementing the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern: - [scenario outlines](#scenario-outlines), - [vertical data tables](#vertical-data-tables), - [horizontal data tables](#horizontal-data-tables), - [runtime parameters](#runtime-parameters). ### Scenario outlines In Gherkin, a [`Scenario Outline`](https://docs.reqnroll.net/latest/gherkin/gherkin-reference.html#scenario-outline) (or a `Scenario Template`) implements the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. A scenario outline must contain an `Examples` table, from which Reqnroll loads sets of parameters, one row after another. Each set of parameters is being placed into the step declarations according to the placeholders, thus generating a new scenario based on the row. Reqnroll then runs each of them independently, as if it was a separate `Scenario`. The data can then be captured by a regular expression and passed as separate arguments to the C# code. Allure Reqnroll automatically recognizes this pattern. No additional configuration is required. The example below shows a Gherkin file and a C# implementation file of a test. In this example, the four parameters for the “I enter my details...” step will be displayed in both instances of the scenario in the test report. ```gherkin Feature: User management Scenario Outline: Registration When I go to the registration form And I enter my details: , , , Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ```csharp using System; using Reqnroll; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When(@"I enter my details: (.*), (.*), (.*), (.*)")] public void WhenIEnterMyDetails( string login, string password, string name, DateTime birthday) { // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } } ``` ### Vertical data tables If your Gherkin step declaration contains a [data table](https://docs.reqnroll.net/latest/gherkin/gherkin-reference.html#data-tables) with **two columns**, Allure Reqnroll can display its content as the step-level parameters. This behavior is only applied when the table columns are named in a certain way, e.g., “Parameter” and “Value”. The functionality is optional. To enable it, edit the [`allure.gherkinPatterns.stepArguments`](/docs/reqnroll-configuration/#allure-gherkinpatterns-steparguments) section of the configuration file, including the regular expressions for the headers. The table data itself is passed to the C# code without modifications. The example below shows a configuration file, a Gherkin file and a C# implementation file of a test. In this example, the four parameters for the “I enter my details” step will be displayed in the test report. ```json { "allure": { "gherkinPatterns": { "stepArguments": { "createFromDataTables": true, "nameColumn": "^Parameter$", "valueColumn": "^Value$" } } } } ``` ```gherkin Feature: User management Scenario: Registration When I go to the registration form And I enter my details | Parameter | Value | | login | johndoe | | password | qwerty | | name | John Doe | | birthday | 1970-01-01 | Then the profile should be created ``` ```csharp using Reqnroll; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When("I enter my details")] public void WhenIEnterMyDetails(Table table) { // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } } ``` ### Horizontal data tables If your Gherkin step declaration contains a [data table](https://docs.reqnroll.net/latest/gherkin/gherkin-reference.html#data-tables) with **one row**, Allure Reqnroll can display its content as the step-level parameters. The functionality is optional. To enable or disable it, edit the [`allure.gherkinPatterns.stepArguments.createFromDataTables`](/docs/reqnroll-configuration/#allure-gherkinpatterns-steparguments) configuration option. The table data itself is passed to the C# code without modifications. The example below shows a configuration file, a Gherkin file and a C# implementation file of a test. In this example, the four parameters for the “I enter my details” step will be displayed in the test report. ```json { "allure": { "gherkinPatterns": { "stepArguments": { "createFromDataTables": true, "nameColumn": "^Parameter$", "valueColumn": "^Value$" } } } } ``` ```gherkin Feature: User management Scenario: Registration When I go to the registration form And I enter my details | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | Then the profile should be created ``` ```csharp using Reqnroll; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When("I enter my details")] public void WhenIEnterMyDetails(Table table) { // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } } ``` ### Runtime parameters - `AllureApi.AddTestParameter(string name, object? value)` - `AllureApi.AddTestParameter(string name, object? value, ParameterMode mode)` - `AllureApi.AddTestParameter(string name, object? value, bool excluded)` - `AllureApi.AddTestParameter(string name, object? value, ParameterMode mode, bool excluded)` - `AllureApi.AddTestParameter(Parameter parameter)` The `AddTestParameter()` function can be used at any point, even if the Gherkin file does not specify actual parameters or data tables to any step. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. This argument is only used by Allure TestOps. The `mode` argument affects how the parameter will be displayed in the report. Available options are defined in the `ParameterMode` enumeration: - `ParameterMode.Default` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `ParameterMode.Masked` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `ParameterMode.Hidden` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. The example below shows a Gherkin file and a C# implementation file of a test. As far as Reqnroll is concerned, the “I enter my details” in the example does not have any data passed to it, however, Allure will have the four parameters displayed on the test level. ```gherkin Feature: User management Scenario: Registration When I go to the registration form And I enter my details Then the profile should be created ``` ```csharp using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When("I enter my details")] public void WhenIEnterMyDetails() { AllureApi.AddTestParameter("login", "johndoe"); AllureApi.AddTestParameter("password", "qwerty"); AllureApi.AddTestParameter("name", "John Doe"); AllureApi.AddTestParameter("birthday", "1970-01-01"); // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } } ``` ## Attachments - `AllureApi.AddAttachment(string name, string type, string path)` - `AllureApi.AddAttachment(string name, string type, byte[] content, string fileExtension = "")` - `AllureApi.AddAttachment(string path, string? name = null)` - `AllureApi.AddScreenDiff(string expectedPng, string actualPng, string diffPng)` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Tip: You can use data produced by any function, not necessarily read from an actual file. To create an attachment using the Runtime API, call `AddAttachment()` at any point during your test. Pass either the `content` or the `path` from which the data will be read. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension as `fileExtension`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. ```csharp using System.IO; using System.Text; using Allure.Net.Commons; using Reqnroll; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddAttachment( "data.txt", "text/plain", Encoding.UTF8.GetBytes("This is the file content.") ); AllureApi.AddAttachment( "image1.png", "image/png", File.ReadAllBytes("/path/to/image1.png") ); AllureApi.AddAttachment( "image2.png", "image/png", "/path/to/image2.png" ); } } ``` Allure Reqnroll also provides a dedicated `AddScreenDiff()` function for attaching [visual comparisons](/docs/attachments/#attaching-visual-comparisons). --- ## Reqwest > URL: https://allurereport.org/docs/reqwest/ # Getting started with Allure Reqwest [![allure-reqwest crates.io latest version](https://img.shields.io/crates/v/allure-reqwest?style=flat "allure-reqwest crates.io latest version")](https://crates.io/crates/allure-reqwest) Capture [reqwest](https://docs.rs/reqwest/) HTTP requests and responses as attachments in your [Allure Report](https://allurereport.org/docs/) test reports. `allure-reqwest` works with any Rust test integration that provides an Allure facade — currently [Rust Cargo Test](/docs/rust/) — or with `allure-rust-commons` directly. Info: The official `allure-rust` repository also publishes [`allure-diesel`](/docs/diesel/) for recording executed Diesel queries as Allure steps. ## Setting up 1. Make sure you have an Allure integration set up for your test runner. See [Rust Cargo Test](/docs/rust/). 1. Choose an integration style and add the dependency. **Middleware** — recommended for broad capture with minimal call-site changes. It wraps a `reqwest_middleware::ClientWithMiddleware`, so requests keep using a reqwest-style client while Allure records them automatically. Requires the `middleware` feature and the `reqwest-middleware` crate: ```bash cargo add allure-reqwest --features middleware cargo add reqwest-middleware ``` ```rust use allure_reqwest::AllureReqwestMiddleware; use reqwest_middleware::ClientBuilder; async fn create_order(allure: allure_rust_commons::AllureFacade) -> reqwest::Result<()> { let client = ClientBuilder::new(reqwest::Client::new()) .with(AllureReqwestMiddleware::new(allure)) .build(); let response = client .post("https://api.example.com/v1/orders") .header("content-type", "application/json") .body(r#"{"name":"demo"}"#) .send() .await?; response.error_for_status()?; Ok(()) } ``` **Wrapper client** — the smaller alternative when you don't want the `reqwest-middleware` dependency, or only need to instrument selected calls. Requests must go through `AllureReqwestClient::send`/`execute` instead of a plain `reqwest::Client`: ```bash cargo add allure-reqwest ``` ```rust use allure_reqwest::AllureReqwestClient; async fn create_order(allure: allure_rust_commons::AllureFacade) -> reqwest::Result<()> { let client = AllureReqwestClient::new(allure); let response = client .send( client .post("https://api.example.com/v1/orders") .header("content-type", "application/json") .body(r#"{"name":"demo"}"#), ) .await?; response.error_for_status()?; Ok(()) } ``` Both styles take an `allure_rust_commons::AllureFacade`. Inside a `#[allure_test]` function, that's the injected `allure` value; outside one, use `allure_rust_commons::current_allure()` or pass the facade through explicitly. 1. Run your tests and generate a report the same way as you would normally — see [Rust Cargo Test](/docs/rust/#_3-generate-a-report). Every request made through an instrumented client is recorded as an [HTTP exchange attachment](/docs/attachments/#http-exchanges), wrapped in a step (named `HTTP Exchange` by default). The attachment captures the request method, URL, query parameters, and headers; the response status, status text, and headers; the exchange start and stop timestamps; and, when the request fails, the error name and message. Request bodies are captured by default (up to a size limit); response bodies are opt-in. See the [attachment schema reference](/docs/reqwest-reference/#attachment-schema) for the exact field list. Info: To prevent secrets from leaking into reports, a default set of headers and query parameters is redacted before the attachment is written. See [Allure Reqwest configuration](/docs/reqwest-configuration/) for the full list of defaults and how to add to them. --- ## Reqwest configuration > URL: https://allurereport.org/docs/reqwest-configuration/ # Allure Reqwest configuration This page describes `CaptureOptions`, which controls what [`AllureReqwestMiddleware`](/docs/reqwest/#setting-up) and [`AllureReqwestClient`](/docs/reqwest/#setting-up) record. Build it with the builder methods below and pass it with `.with_options(...)`: ```rust use allure_reqwest::{AllureReqwestClient, CaptureOptions}; let client = AllureReqwestClient::new(allure).with_options( CaptureOptions::default() .with_attachment_name("Create order") .with_response_body_capture(64 * 1024), ); ``` ## Customize the attachment name - `with_attachment_name(name: impl Into)` Sets a fixed name for the step and attachment created for every captured exchange. By default, every exchange is attached under the name `HTTP Exchange`. Unlike some other Allure HTTP integrations, the name is always a static string — there is no callback that derives it from the request or response. ## Control body capture - `without_request_body_capture()` — request bodies are captured by default; call this to disable it - `with_response_body_capture(max_body_size: usize)` — response bodies are **not** captured by default; call this to enable it - `without_response_body_capture()` - `with_max_body_size(max_body_size: usize)` — sets the byte limit Warning: There is only **one** size limit, shared by both request and response body capture — not independent limits per side. `with_response_body_capture(n)` and `with_max_body_size(n)` set the same underlying value, so whichever one you call **last** wins for both sides: ```rust use allure_reqwest::CaptureOptions; // max_body_size ends up 1024 for BOTH request and response capture — the with_max_body_size(5) // call is overwritten by with_response_body_capture's own argument, which runs after it. let a = CaptureOptions::default() .with_max_body_size(5) .with_response_body_capture(1024); // max_body_size ends up 5 for BOTH sides — with_max_body_size runs last here. let b = CaptureOptions::default() .with_response_body_capture(1024) .with_max_body_size(5); ``` If you want a specific request-body limit together with response capture, call `with_max_body_size(...)` **after** `with_response_body_capture(...)`. The default limit is 64 KiB (65536 bytes). Bodies larger than the limit are captured up to the limit and flagged `truncated: true` in the attachment — `size` still reports the full, untruncated byte count. See the [reference](/docs/reqwest-reference/#body) for the exact fields. Response bodies are only captured when the body can be fully buffered: `AllureReqwestClient` and `AllureReqwestMiddleware` read the response into memory, record it, and reconstruct an equivalent `Response` so your test code can still read it normally afterward. ## Redaction Sensitive header and query-parameter values are replaced with the sentinel string `__ALLURE_REDACTED__` before the attachment is written. The defaults are: | Category | Redacted by default | | ---------------- | -------------------------------------------------------------------------------- | | Headers | `authorization`, `cookie`, `proxy-authorization`, `set-cookie` | | Query parameters | `access_token`, `api_key`, `key`, `password`, `refresh_token`, `secret`, `token` | Matching is by exact name, case-insensitive — there is no regex or pattern matching. - `redact_header(name: impl Into)` - `redact_query_param(name: impl Into)` Both **add** a name to the existing redaction list rather than replacing it — there is no way to remove a default redacted name, only to add more: ```rust use allure_reqwest::CaptureOptions; let options = CaptureOptions::default() .redact_header("x-internal-secret") .redact_query_param("session_id"); // authorization, cookie, etc. are still redacted, plus these two. ``` ## Errors When a request fails — a transport error, or a failure while reading the response body — the attachment's `error` field records the error's type name (`reqwest::Error` for `AllureReqwestClient`, `reqwest_middleware::Error` for `AllureReqwestMiddleware`) and its display message. There is no option to capture a stack trace. HTTP error status codes (4xx/5xx) are **not** treated as errors by `allure-reqwest` — reqwest itself does not treat them as errors either, so the attachment gets a normal `response` with no `error`, the same as any other status code. Call `.error_for_status()` yourself if you want reqwest to turn a 4xx/5xx response into an `Err`. --- ## Reqwest reference > URL: https://allurereport.org/docs/reqwest-reference/ # Allure Reqwest reference ## Attachment schema Each HTTP exchange is recorded as an [HTTP exchange attachment](/docs/attachments/#http-exchanges) with content type `application/vnd.allure.http+json` and file extension `.httpexchange`. The schema is shared across Allure's HTTP-capturing integrations (`allure_rust_commons::HttpExchange`); the tables below list every field the shared type supports, and note which ones `allure-reqwest` actually populates — several exist only for other, richer integrations and are always absent from a reqwest-captured exchange. The root object: | Field | Type | Populated by `allure-reqwest`? | | --------------- | -------- | -------------------------------------------------------------------------------- | | `schemaVersion` | `1` | Always `1`. | | `start` | `number` | Always — Unix timestamp in milliseconds when the request started. | | `stop` | `number` | Always — Unix timestamp in milliseconds when the response or error was received. | | `request` | object | Always. See [Request](#request). | | `response` | object | Present when a response was received. See [Response](#response). | | `error` | object | Present when the request failed. See [Error](#error). | Unlike some other Allure HTTP integrations, a non-2xx/3xx HTTP status is **not** an error by itself: reqwest doesn't treat 4xx/5xx as a failure, so `allure-reqwest` doesn't either — you get a normal `response` with no `error`. `error` is only present for transport failures (connection errors, timeouts) or a failure while reading the response body. ### Request | Field | Type | Populated by `allure-reqwest`? | | ------------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------- | | `method` | `string` | Always — HTTP method, e.g. `"POST"`. | | `url` | `string` | Always — the resolved request URL, including query parameters in unredacted form. | | `httpVersion` | `string` | Always — e.g. `"HTTP/1.1"`. | | `headers` | `{name, value}[]` | Present when the request has headers, after redaction. | | `query` | `{name, value}[]` | Present when the URL has query parameters, after redaction. | | `body` | object | Present when request body capture is enabled (default) and the body is available as in-memory bytes. See [Body](#body). | | `cookies` | `Cookie[]` | Never — `allure-reqwest` does not parse the `Cookie` header into structured cookies. | | `trailers` | `{name, value}[]` | Never. | Warning: A request body built from a stream (anything where `reqwest::Body::as_bytes()` returns `None` — for example a body constructed from a `futures::Stream`) is silently **not** captured: `body` is simply absent from the attachment, with no placeholder and no indication that a body existed. Only in-memory bodies (`&str`, `String`, `Vec`, `Bytes`, forms, and similar) are captured. ### Response | Field | Type | Populated by `allure-reqwest`? | | ------------------------ | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `status` | `number` | Always. | | `statusText` | `string` | Always, when the status has a canonical reason phrase (e.g. `"Created"` for 201). | | `httpVersion` | `string` | Always. | | `headers` | `{name, value}[]` | Present when the response has headers, after redaction. | | `body` | object | Present only when response body capture is explicitly enabled (see [Configuration](/docs/reqwest-configuration/#control-body-capture)) and the body was read successfully. See [Body](#body). | | `cookies` | `Cookie[]` | Never. | | `trailers` | `{name, value}[]` | Never. | | `informationalResponses` | object[] | Never. | ### Body | Field | Type | Populated by `allure-reqwest`? | | ------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | `contentType` | `string` | Present when a `Content-Type` header is present on that side of the exchange. | | `encoding` | `"utf8" \| "base64"` | Always present alongside `value`: `"utf8"` when the captured bytes are valid UTF-8, `"base64"` otherwise. | | `value` | `string` | Always present alongside `encoding` — the captured bytes, up to the size limit. | | `size` | `number` | Always — the **original, untruncated** byte length, even when `value` was cut short. | | `truncated` | `boolean` | Always — `true` when the original body was larger than the configured limit. | | `form` | `{name, value}[]` | Never — `allure-reqwest` does not parse `application/x-www-form-urlencoded` bodies into fields; they're only captured as the raw `value` string. | | `parts` | object[] | Never — multipart bodies are captured as a single raw `value`, not broken into parts. | | `stream` | object | Never — see the warning above; a streamed body has no `body` field at all, not a `stream` placeholder. | ### Error | Field | Type | Populated by `allure-reqwest`? | | --------- | -------- | ------------------------------------------------------------------------------------------------------------------- | | `name` | `string` | Always — `"reqwest::Error"` for `AllureReqwestClient`, `"reqwest_middleware::Error"` for `AllureReqwestMiddleware`. | | `message` | `string` | Always — the error's `Display` output. | | `stack` | `string` | Never — there is no option to capture one. | ## `AllureReqwestClient` - `AllureReqwestClient::new(allure: AllureFacade)` — wraps a new `reqwest::Client` - `AllureReqwestClient::with_client(client: reqwest::Client, allure: AllureFacade)` — wraps an existing client - `.with_options(options: CaptureOptions)` — see [Configuration](/docs/reqwest-configuration/) - `.inner() -> &reqwest::Client` — the wrapped client - `.request(method, url)`, `.get(url)`, `.post(url)`, `.put(url)`, `.patch(url)`, `.delete(url)`, `.head(url)` — return a plain `reqwest::RequestBuilder`, same as `reqwest::Client` - `.send(builder: RequestBuilder) -> reqwest::Result` — builds and executes the request builder while capturing the exchange - `.execute(request: Request) -> reqwest::Result` — executes an already-built `Request` while capturing the exchange Only requests sent through `.send()`/`.execute()` are captured. Building a request with `.get()`/ `.post()`/etc. and sending it through the plain `reqwest::RequestBuilder` API instead (bypassing `.send()`) is not captured. ## `AllureReqwestMiddleware` Requires the `middleware` feature. - `AllureReqwestMiddleware::new(allure: AllureFacade)` - `.with_options(options: CaptureOptions)` — see [Configuration](/docs/reqwest-configuration/) Implements `reqwest_middleware::Middleware`; attach it with `ClientBuilder::new(reqwest::Client::new()).with(middleware).build()`. Every request made through the resulting `ClientWithMiddleware` is captured — no per-call changes needed. --- ## REST Assured > URL: https://allurereport.org/docs/restassured/ # Getting started with Allure REST Assured [![Allure REST Assured latest version](https://img.shields.io/maven-central/v/io.qameta.allure/allure-rest-assured?style=flat "allure-rest-assured")](https://mvnrepository.com/artifact/io.qameta.allure/allure-rest-assured) Enrich your [Allure Report](https://allurereport.org/docs/) test reports with the detailed information about the HTTP requests performed via [REST Assured](https://rest-assured.io/). Info: Allure REST Assured 2.34.0 and later requires **REST Assured 6.x**. If you are using REST Assured 5.x, use Allure REST Assured 2.33.x or earlier. Allure REST Assured 3.x requires Java 17 or newer. Allure RestAssured is compatible with all JVM-based Allure integrations, including [Allure JUnit Jupiter](/docs/junit-jupiter/), [Allure Cucumber-JVM](/docs/cucumberjvm/), [Allure Spock](/docs/spock/), and more. To enable the integration in your project: 1. Make sure you have the Allure Report integration enabled for the test framework you use. See the instructions in the integration's documentation in [Frameworks](/docs/frameworks/). 1. Add the Allure REST Assured integration to your project's dependencies. **Maven:** ```xml 3.0.0 io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-rest-assured test ``` **Gradle (Kotlin):** ```kts // Define the version of Allure you want to use via the allureVersion property val allureVersion = "3.0.0" // ... dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) // Add necessary Allure dependencies to dependencies section testImplementation("io.qameta.allure:allure-rest-assured") } ``` **Gradle (Groovy):** ```groovy // Define the version of Allure you want to use via the allureVersion property def allureVersion = "3.0.0" dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") // Add necessary Allure dependencies to dependencies section testImplementation "io.qameta.allure:allure-rest-assured" } ``` 1. For every HTTP request you make in your tests, pass an instance of `AllureRestAssured` to the REST Assured's `filter()` method. The example below shows usage of Allure REST Assured in a JUnit Jupiter test. ```java import io.qameta.allure.restassured.AllureRestAssured; import org.junit.jupiter.api.Test; import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; class TestMyWebsite { @Test void testSomeRequest() { given() .filter(new AllureRestAssured()) .get("https://jsonplaceholder.typicode.com/todos/1") .then() .body("userId", equalTo(1)); } } ``` Each request and its response will be added to the test results as a single [HTTP exchange attachment](/docs/attachments/#http-exchanges). In Allure REST Assured 2.x, the request and the response are added as two separate HTML attachments. Some aspects of the `AllureRestAssured` behavior can be affected via its [configuration methods](/docs/restassured-configuration/). 1. Run your tests and generate a test report the same way as you would do usually. --- ## REST Assured configuration > URL: https://allurereport.org/docs/restassured-configuration/ # Allure REST Assured configuration This page describes the methods that affect the behavior of the [Allure REST Assured](/docs/restassured/) integration. All methods are chainable. The examples here assume that you use JUnit Jupiter for your tests. ## Configure the HTTP exchange attachment - `configureHttpExchange(Consumer exchangeCustomizer)` Starting with Allure Java 3.0, Allure REST Assured records each request and its response as a single [HTTP exchange attachment](/docs/attachments/#http-exchanges). Use `configureHttpExchange()` to control what this attachment contains. The following methods are available on the builder: - `redactHeader(String name)` — replace the value of the given request or response header with a placeholder. - `redactCookie(String name)` — replace the value of the given cookie with a placeholder. - `redactQueryParameter(String name)` — replace the value of the given query parameter with a placeholder. - `redactFormParameter(String name)` — replace the value of the given form parameter with a placeholder. - `setMaxBodySize(long maxBodySize)` — the maximum number of bytes of a request or response body to keep. Longer bodies are truncated and marked as truncated in the report. Defaults to 1 MiB. The `Authorization`, `Proxy-Authorization`, `Cookie` and `Set-Cookie` headers are always redacted, as well as the headers listed in REST Assured's own blacklist configuration. ```java import io.qameta.allure.restassured.AllureRestAssured; import org.junit.jupiter.api.Test; import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; class TestMyWebsite { static AllureRestAssured allureFilter = new AllureRestAssured() .configureHttpExchange(exchange -> exchange .redactHeader("X-Api-Key") .redactCookie("SESSION") .redactQueryParameter("token") .setMaxBodySize(64 * 1024)); @Test void testSomeRequest() { given() .filter(allureFilter) .queryParam("token", "my-secret-token") .get("https://jsonplaceholder.typicode.com/todos/1") .then() .body("userId", equalTo(1)); } } ``` ## Limit body prettify size - `setMaxAllowedPrettifyLength(int maxAllowedPrettifyLength)` Set the maximum number of characters of a response body that will be pretty-printed before attaching it to the report. Bodies longer than this threshold are attached as-is without any formatting. Defaults to `1_048_576` (1 MB). Use this to prevent out-of-memory errors when testing APIs that return very large JSON or XML payloads. ```java import io.qameta.allure.restassured.AllureRestAssured; static AllureRestAssured allureFilter = new AllureRestAssured() .setMaxAllowedPrettifyLength(512_000); // only prettify bodies up to 512 KB ``` ## Customize attachment names - `setAttachmentName(final String attachmentName)` Specify the name under which Allure REST Assured will create the HTTP exchange attachment for the test report. By default, the attachment is called “HTTP exchange”. ```java import io.qameta.allure.restassured.AllureRestAssured; import org.junit.jupiter.api.Test; import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; class TestMyWebsite { static AllureRestAssured allureFilter = new AllureRestAssured() .setAttachmentName("Get the first todo"); @Test void testSomeRequest() { given() .filter(allureFilter) .get("https://jsonplaceholder.typicode.com/todos/1") .then() .body("userId", equalTo(1)); } } ``` Info: Allure Java 2.x In Allure Java 2.x, the request and the response are separate attachments, named with `setRequestAttachmentName()` and `setResponseAttachmentName()`. By default, each request is called “Request”, while each response gets a name based on its HTTP code, e.g., “HTTP/1.1 200 OK”. In Allure Java 3.x, both methods are kept for compatibility, but they set the name of the same single attachment, so the last call wins. ## Customize templates (Allure Java 2.x only) - `setRequestTemplate(final String templatePath)` - `setResponseTemplate(final String templatePath)` Warning: These methods were removed in Allure Java 3.0, together with the FreeMarker-based HTML attachments. See [Migrating to Allure Java 3](/docs/guides/allure-java-3-migration/). Specify paths to custom templates that Allure REST Assured will use for formatting the HTTP requests and responses. The templates must be written in the [Apache Freemarker](https://freemarker.apache.org/) language. When processing a request or a response, an object with its full details is passed to the corresponding template as `data`. You can find [the default templates](https://github.com/allure-framework/allure-java/tree/2.35.3/allure-attachments/src/main/resources/tpl) in the Allure Java repository. The paths are interpreted as relative to the `tpl` directory in your project's resources. For example, the code below uses the templates located at `tpl/my-http-request.ftl` and `tpl/my-http-response.ftl`. ```java import io.qameta.allure.restassured.AllureRestAssured; import org.junit.jupiter.api.Test; import static io.restassured.RestAssured.*; import static org.hamcrest.Matchers.*; class TestMyWebsite { static AllureRestAssured allureFilter = new AllureRestAssured() .setRequestTemplate("my-http-request.ftl") .setResponseTemplate("my-http-response.ftl"); @Test void testSomeRequest() { given() .filter(allureFilter) .get("https://jsonplaceholder.typicode.com/todos/1") .then() .body("userId", equalTo(1)); } } ``` --- ## Robot Framework > URL: https://allurereport.org/docs/robotframework/ # Getting started with Allure Robot Framework [![Allure Robot Framework pypi latest version](https://img.shields.io/pypi/v/allure-robotframework?style=flat "Allure Robot Framework pypi latest version")](https://pypi.org/project/allure-robotframework/) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Robot Framework](https://robotframework.org/) tests. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. If necessary for your system configuration, activate the virtual Python environment for your project. For example, if the project uses a `venv` environment, the command to activate it may look like this: ```bash source .venv/bin/activate ``` This step is not necessary if you are using the system Python environment. 1. Install the Allure Robot Framework integration. ```bash pip install allure-robotframework ``` ### 2. Run tests When running your tests, add the `--listener allure_robotframework` command-line argument. This will allow Allure Robot Framework to collect test results into the `output/allure` directory. To specify another path for the test results directory, add it after the listener name, separated by a colon. If you are going to use the [test plans](#select-tests-via-a-test-plan-file), also add the `--prerunmodifier allure_robotframework.testplan` argument. **Use default test results directory:** ```bash python -m robot \ --listener allure_robotframework \ --prerunmodifier allure_robotframework.testplan \ ./my_tests ``` **Use custom test results directory:** ```bash python -m robot \ --listener allure_robotframework:allure-results \ --prerunmodifier allure_robotframework.testplan \ ./my_tests ``` ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Robot Framework integration extends the standard reporting features of Robot Framework by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#specify-description-links-and-other-metadata). - **Test Organization**: Structure your tests into [clear hierarchies](#organize-tests) for better readability and organization. - **Step Division**: Break down keywords into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, you need to indicate to Allure Robot Framework that a certain property needs to be assigned to the test result. Most properties can be assigned in the suite file, via the Decorators API or via the Runtime API. - **Suite file**: use the `[Tags]`, `Set Tags` or `Test Tags` keywords in the suite files (see [Tagging test cases](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#tagging-test-cases) in the Robot Framework documentation) to assign various data to tests. When using the `[Tags]` or `Test Tags` keyword, the data is guaranteed to be added to the test result regardless of how the test itself runs. When using the `Set Tags` keyword, it is recommended to put it as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. You can use either a colon or an equal sign to separate the value from the name, e.g., `allure.label.epic:Web interface` is identical to `allure.label.epic=Web interface`. - **Decorators API**: add a Python decorator to a method that implements a Robot Framework keyword. When using this approach, the data is guaranteed to be added regardless of how the keyword itself runs. - **Runtime API**: use Allure's functions to add data to the test result during the execution of a keyword. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the implementation as possible, e.g., in the first lines of the first keyword's implementation. This way, the data will be added even if the test fails early. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/robotframework-reference/#metadata) for more details. **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] ... allure.label.owner:John Doe ... allure.label.severity:critical # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.label('owner', 'John Doe') @allure.severity('critical') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.label('owner', 'John Doe') allure.dynamic.severity('critical') ... ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure Robot Framework provides functions to assign the relevant fields to tests either by adding decorators or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] ... allure.label.epic:Web interface ... allure.label.feature:Essential features ... allure.label.story:Authentication # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.epic('Web interface') @allure.feature('Essential features') @allure.story('Authentication') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.epic('Web interface') allure.dynamic.feature('Essential features') allure.dynamic.story('Authentication') ... ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] ... allure.label.parentSuite:Web interface ... allure.label.suite:Essential features ... allure.label.subSuite:Authentication # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.parent_suite('Web interface') @allure.suite('Essential features') @allure.sub_suite('Authentication') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.parent_suite('Web interface') allure.dynamic.suite('Essential features') allure.dynamic.sub_suite('Authentication') ... ``` To specify a test's location in the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy): **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] ... allure.label.package:org.example ... allure.label.testMethod:test_authentication() # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.label('package', 'org.example') @allure.label('testMethod', 'test_authentication()') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.label('package', 'org.example') allure.dynamic.label('testMethod', 'test_authentication()') ... ``` ### Divide keywords into steps Allure Robot Framework displays each keyword as a separate step in the test report. It also provides two ways of [creating sub-steps](/docs/steps/) from within the Python code: “decorated steps” and “context steps”, see the [reference](/docs/robotframework-reference/#test-steps). **Decorated steps:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): navigate_to_the_website() for login, password in [['johndoe', 'qwerty'], ['janedoe', 'pass123']]: try_to_authenticate(login, password) @allure.step('Navigate to the website') def navigate_to_the_website(): ... @allure.step('Try to authenticate as {login}') def try_to_authenticate(login, password): ... ``` **Context steps:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): with allure.step('Navigate to the website'): ... for login, password in [['johndoe', 'qwerty'], ['janedoe', 'pass123']]: with allure.step(f'Try to authenticate as {login}'): ... ``` ### Describe parametrized tests When you pass arguments to a keyword in the suite file, Allure Report displays them as parameters for the corresponding step, see the [reference](/docs/robotframework-reference/#parametrized-tests). ``` *** Keywords *** Try To Authenticate [Arguments] ${login} ${password} Log To Console Trying to authenticate with ${login} and ${password}... # ... *** Test Cases *** Test Authentication Try To Authenticate johndoe@example.com qwerty Try To Authenticate janedoe@example.com pass123 ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure Robot Framework provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/robotframework-reference/#attachments). **Suite file:** ``` *** Settings *** Library AllureLibrary Library RequestsLibrary *** Test Cases *** Test Authentication # ... ${response}= GET https://example.com/image.png Attach ${response.content} ... name=Image attachment_type=PNG Attach File /path/to/image.png ... name=Image attachment_type=PNG ``` **Runtime API:** ```python import allure import requests from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): ... png_bytes = requests.get('https://example.com/image.png').content allure.attach(png_bytes, name='img', attachment_type=allure.attachment_type.PNG) allure.attach.file('/path/to/image.png', name='img', attachment_type=allure.attachment_type.PNG) ``` ### Select tests via a test plan file If the `--prerunmodifier allure_robotframework.testplan` argument is used and the `ALLURE_TESTPLAN_PATH` environment variable points to an existing file, Robot Framework will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json python -m robot \ --listener allure_robotframework \ --prerunmodifier allure_robotframework.testplan \ ./my_tests ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" python -m robot \ --listener allure_robotframework \ --prerunmodifier allure_robotframework.testplan \ ./my_tests ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and Python version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/python/environment-allure3-light.png, /images/python/environment-allure3-dark.png, /images/python/environment-allure2-light.png, /images/python/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the test results directory (`output/allure` by default) after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## Robot Framework configuration > URL: https://allurereport.org/docs/robotframework-configuration/ # Allure Robot Framework configuration Allure Robot Framework's behavior in regard to log messages (such as [`logging.info()`](https://docs.python.org/3/library/logging.html#logging.Logger.info) calls) can be controlled by the `ALLURE_MAX_STEP_MESSAGE_COUNT` environment variable. The variable must be an integer number. When unset, the variable defaults to zero. - When the total number of log messages that occurred during a keyword execution is greater than `ALLURE_MAX_STEP_MESSAGE_COUNT`, Allure Robot Framework will present them as an [attachment](/docs/robotframework-reference/#attachments). This is usually the preferred behavior for tests that produce a lot of logs that you only occasionally need to read through. Since the default value of `ALLURE_MAX_STEP_MESSAGE_COUNT` is zero, this behavior is used for all keywords by default. - For keywords that produced fewer log messages than `ALLURE_MAX_STEP_MESSAGE_COUNT`, Allure Robot Framework will present each message as an individual [sub-step](/docs/robotframework-reference/#test-steps). This is usually the preferred behavior for tests that produce few messages that can be analyzed at a glance. Tip: You can use [test plans](/docs/robotframework/#select-tests-via-a-test-plan-file) to run different sets of tests at a time. This way, you will be able to set a different value of `ALLURE_MAX_STEP_MESSAGE_COUNT` before each run if necessary. --- ## Robot Framework reference > URL: https://allurereport.org/docs/robotframework-reference/ # Allure Robot Framework reference These are the attributes and methods that you can use to integrate your Robot Framework tests with Allure Report. In most cases, you need to indicate to Allure Robot Framework that a certain property needs to be assigned to the test result. Most properties can be assigned in the suite file, via the Decorators API or via the Runtime API. - **Suite file**: use the `[Tags]`, `Set Tags` or `Test Tags` keywords in the suite files (see [Tagging test cases](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#tagging-test-cases) in the Robot Framework documentation) to assign various data to tests. When using the `[Tags]` or `Test Tags` keyword, the data is guaranteed to be added to the test result regardless of how the test itself runs. When using the `Set Tags` keyword, it is recommended to put it as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. You can use either a colon or an equal sign to separate the value from the name, e.g., `allure.label.epic:Web interface` is identical to `allure.label.epic=Web interface`. - **Decorators API**: add a Python decorator to a method that implements a Robot Framework keyword. When using this approach, the data is guaranteed to be added regardless of how the keyword itself runs. - **Runtime API**: use Allure's functions to add data to the test result during the execution of a keyword. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the implementation as possible, e.g., in the first lines of the first keyword's implementation. This way, the data will be added even if the test fails early. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `[Documentation]` For the [description](/docs/v3/readability/#description), Allure Robot Framework uses the scenario's `[Documentation]` from the suite file. Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ``` *** Test Cases *** Test Authentication [Documentation] This test attempts to log into the website. # ... ``` ### Owner - `[Tags] allure.label.owner:⟨VALUE⟩` - `@allure.label('owner', value: str)` - `allure.dynamic.label('owner', value: str)` Set the test's [owner](/docs/v3/readability/#owner). **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] allure.label.owner:John Doe # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.label('owner', 'John Doe') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.label('owner', 'John Doe') ... ``` ### Tag - `[Tags]` - `@allure.tag(*tags: str)` - `allure.dynamic.tag(*tags: str)` Set the test's [tags](/docs/v3/readability/#tags). Any Robot Framework's native tag is automatically added to the list of the test's tags, unless it matches one of the known patterns for other labels. **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] UI Authentication # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.tag('UI') @allure.tag('Authentication') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.tag('UI') allure.dynamic.tag('Authentication') ... ``` ### Severity - `[Tags] allure.label.severity:⟨VALUE⟩` - `@allure.severity(severity_level: str)` - `allure.dynamic.severity(severity_level: str)` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] allure.label.severity:critical # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.severity('critical') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.severity('critical') ... ``` ### Label - `[Tags] allure.label.⟨NAME⟩:⟨VALUE⟩` - `@allure.label(label_type: str, *labels: str)` - `allure.dynamic.label(label_type: str, *labels: str)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] ... allure.label.layer:web ... allure.label.owner:eroshenkoam ... allure.label.page:/{org}/{repo}/authentication ... allure.label.jira:AE-2 # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.label('layer', 'web') @allure.label('owner', 'eroshenkoam') @allure.label('page', '/{org}/{repo}/authentication') @allure.label('jira', 'AE-2') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.label('layer', 'web') allure.dynamic.label('owner', 'eroshenkoam') allure.dynamic.label('page', '/{org}/{repo}/authentication') allure.dynamic.label('jira', 'AE-2') ... ``` ### Allure ID (Allure TestOps) - `[Tags] allure.as_id:⟨VALUE⟩` - `@allure.id(id: str)` Set the test's [ID](/docs/v3/readability/#id). **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] allure.as_id:123 # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.id('123') def test_authentication(): ... ``` ### Link - `[Tags] allure.link:⟨URL⟩` - `[Tags] allure.link.⟨NAME⟩:⟨URL⟩` - `[Tags] allure.issue:⟨URL⟩` - `[Tags] allure.issue.⟨NAME⟩:⟨URL⟩` - `[Tags] allure.tms:⟨URL⟩` - `[Tags] allure.tms.⟨NAME⟩:⟨URL⟩` - `allure.dynamic.link(url: str, link_type: str = LinkType.LINK, name: str = None)` - `allure.dynamic.issue(url: str, name: str = None)` - `allure.dynamic.testcase(url: str, name: str = None)` Add a [link](/docs/v3/readability/#links) related to the test. - In a suite file, add one of the tags listed above. Each tag indicates a different link type (`link`, `issue`, or `tms`) which affects the icon shown in the test report. - In your Python code, use the `allure.dynamic.link()` function with an optional `link_type` argument or its shorthand versions for the `issue` and `tms` link types. **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] ... allure.link.MyWebsite:https://dev.example.com/ ... allure.issue.UI-123:https://issues.example.com/UI-123 ... allure.tms.TMS-456:https://tms.example.com/TMS-456 # ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.link('https://dev.example.com/', name='Website') allure.dynamic.issue("'https://issues.example.com'/UI-123", name='UI-123') allure.dynamic.testcase('https://tms.example.com/TMS-456', name='TMS-456') ... ``` ## Behavior-based hierarchy - `[Tags] allure.label.epic:⟨VALUE⟩` - `[Tags] allure.label.feature:⟨VALUE⟩` - `[Tags] allure.label.story:⟨VALUE⟩` - `@allure.epic(*epics: str)` - `@allure.feature(*features: str)` - `@allure.story(*stories: str)` - `allure.dynamic.epic(*epics: str)` - `allure.dynamic.feature(*features: str)` - `allure.dynamic.story(*stories: str)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). Each function support setting one or more values at once. **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] ... allure.label.epic:Web interface ... allure.label.feature:Essential features ... allure.label.story:Authentication # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.epic('Web interface') @allure.feature('Essential features') @allure.story('Authentication') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.epic('Web interface') allure.dynamic.feature('Essential features') allure.dynamic.story('Authentication') ... ``` ## Suite-based hierarchy - `[Tags] allure.label.parentSuite:⟨VALUE⟩` - `[Tags] allure.label.suite:⟨VALUE⟩` - `[Tags] allure.label.subSuite:⟨VALUE⟩` - `@allure.parent_suite(parent_suite_name: str)` - `@allure.suite(suite_name: str)` - `@allure.sub_suite(sub_suite_name: str)` - `allure.dynamic.parent_suite(parent_suite_name: str)` - `allure.dynamic.suite(suite_name: str)` - `allure.dynamic.sub_suite(sub_suite_name: str)` Assign the names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] ... allure.label.parentSuite:Web interface ... allure.label.suite:Essential features ... allure.label.subSuite:Authentication # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.parent_suite('Web interface') @allure.suite('Essential features') @allure.sub_suite('Authentication') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.parent_suite('Web interface') allure.dynamic.suite('Essential features') allure.dynamic.sub_suite('Authentication') ... ``` ## Package-based hierarchy - `[Tags] allure.label.package:⟨VALUE⟩` - `[Tags] allure.label.testMethod:⟨VALUE⟩` Assign the names of the package and the test method, as part of Allure's [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy). **Suite file:** ``` *** Test Cases *** Test Authentication [Tags] ... allure.label.package:org.example ... allure.label.testMethod:test_authentication() # ... ``` **Decorators API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') @allure.label('package', 'org.example') @allure.label('testMethod', 'test_authentication()') def test_authentication(): ... ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): allure.dynamic.label('package', 'org.example') allure.dynamic.label('testMethod', 'test_authentication()') ... ``` ## Test steps - `@allure.step(title: str)` - `with allure.step(title: str)` Define a [test sub-step](/docs/steps/) within the current step. There are two ways of defining a sub-step. - **Decorated sub-steps** Define a function or a method containing a test sub-step and decorate it with `@allure.step()`. If the function accepts arguments, they will be displayed as step-level parameters in the test report. You can also include their values in the sub-step title via **replacement fields**. A replacement field is a parameter name delimited by braces `{}`, as supported by the standard [`str.format()`](https://docs.python.org/3/library/stdtypes.html#str.format) method. - **Context sub-steps** Write a test sub-step in-place but use the `with allure.step()` statement to create its context. **Decorated steps:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): navigate_to_the_website() for login, password in [['johndoe', 'qwerty'], ['janedoe', 'pass123']]: try_to_authenticate(login, password) @allure.step('Navigate to the website') def navigate_to_the_website(): ... @allure.step('Try to authenticate as {login}') def try_to_authenticate(login, password): ... ``` **Context steps:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): with allure.step('Navigate to the website'): ... for login, password in [['johndoe', 'qwerty'], ['janedoe', 'pass123']]: with allure.step(f'Try to authenticate as {login}'): ... ``` ## Parametrized tests When you pass arguments to a keyword in the suite file, Allure displays them as parameters for the corresponding step in the test report. No additional configuration is required. Note that due to an implementation detail, Allure Report is only able to show the keyword parameters in the same way as they are given in the corresponding line. This means that if a keyword's argument is a variable (e.g., a value yielded by the Robot Framework's `FOR` loop), Allure Report will display the variable's name, not the actual value it has at runtime. The example below avoid the issue by implementing a [user keyword with arguments](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#user-keyword-arguments) and then using it with literal values without loops. ``` *** Keywords *** Try To Authenticate [Arguments] ${login} ${password} Log To Console Trying to authenticate with ${login} and ${password}... # ... *** Test Cases *** Test Authentication Try To Authenticate johndoe@example.com qwerty Try To Authenticate janedoe@example.com pass123 ``` ## Attachments Allure Robot Framework provides multiple ways for [adding attachments](/docs/attachments/) to the current step. ### Attaching content from variables - `Attach ⟨CONTENT⟩` - `Attach ⟨CONTENT⟩ name=⟨NAME⟩ attachment_type=⟨TYPE⟩` - `allure.attach(body, name=None, attachment_type="text/plain", extension="attach")` Add `body` as an attachment to the test result under the given `name` (defaults to a unique pseudo-random string). The `body` must be of type `bytes` or `str`. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. There are two ways to do this: - Pass the media type of the content as `attachment_type` and, optionally, a filename extension as `extension`. Some popular media types are `image/png` and `image/jpeg` for screenshots and other images, `application/json` for JSON data, and `text/plain` for text files. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. - Pass a value from the `allure.attachment_type` class as `attachment_type`. This will automatically set the media type and the appropriate filename extension. **Suite file:** ``` *** Settings *** Library AllureLibrary Library RequestsLibrary *** Test Cases *** Test Authentication # ... ${response}= GET https://example.com/image.png Attach ${response.content} ... name=Image attachment_type=PNG ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): ... png_bytes = requests.get('https://example.com/image.png').content allure.attach(png_bytes, name='img', attachment_type=allure.attachment_type.PNG) ``` ### Reading attachments from files - `Attach File ⟨FILE⟩` - `Attach File ⟨FILE⟩ name=⟨NAME⟩ attachment_type=⟨TYPE⟩` - `allure.attach.file(source, name=None, attachment_type=None, extension=None)` Same as [Attaching content from variables](#attaching-content-from-variables), but the content is loaded from an existing file. **Suite file:** ``` *** Settings *** Library AllureLibrary *** Test Cases *** Test Authentication # ... Attach File /path/to/image.png ... name=Image attachment_type=PNG ``` **Runtime API:** ```python import allure from robot.api.deco import keyword @keyword('Test Authentication') def test_authentication(): ... allure.attach.file('/path/to/image.png', name='img', attachment_type=allure.attachment_type.PNG) ``` ### Attaching log messages By default, Allure Robot Framework automatically creates a text attachment containing the log messages that occurred during a given sub-step. This behavior can be changed, see the [`ALLURE_MAX_STEP_MESSAGE_COUNT`](/docs/robotframework-configuration/) environment variable. --- ## RSpec > URL: https://allurereport.org/docs/rspec/ # Getting started with Allure RSpec [![Allure RSpec gem latest version](https://img.shields.io/gem/v/allure-rspec?style=flat "Allure RSpec gem latest version")](https://rubygems.org/gems/allure-rspec) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [RSpec](https://rspec.info/) tests. Info: Check out the example project at [github.com/allure-examples/allure-rspec-example](https://github.com/allure-examples/allure-rspec-example) to see Allure RSpec in action. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Add Allure RSpec to your project's `Gemfile`. For example: ```ruby source 'https://rubygems.org' gem 'allure-rspec', '~> 2.23.0' gem 'rspec', '~> 3.12' ``` 1. Open a terminal, go to the project directory and install the dependencies from the `Gemfile`. For example, if you use [Bundler](https://bundler.io/): ```bash cd /home/user/myproject bundle install ``` 1. In the project's [`.rspec`](https://rspec.info/documentation/3.12/rspec-core/#store-command-line-options-rspec) file, specify `AllureRspecFormatter` as the formatter for RSpec. ```plain --format AllureRspecFormatter ``` ### 2. Run tests Run your RSpec tests same way as your would run them usually. For example: ```bash bundle exec rspec ``` This will save necessary data into `reports/allure-results` or other directory, according to the [Configuration](/docs/rspec-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure RSpec integration extends the standard reporting features of RSpec by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#specify-description-links-and-other-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. ### Adding Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/rspec-reference/#metadata) for an exhaustive list of what can be added. ```ruby describe 'Test my website' do it 'test authentication', issue: 'AUTH-123' do Allure.description_html 'This test attempts to log into the website using a login and a password.' Allure.label 'owner', 'John Doe' # ... end end ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```ruby describe 'Test my website' do it 'test authentication' do Allure.epic 'Web interface' Allure.feature 'Essential features' Allure.story 'Authentication' # ... end end ``` **RSpec metadata:** ```ruby describe 'Test my website' do it 'test authentication', epic: 'Web interface', feature: 'Essential features', story: 'Authentication' do # ... end end ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): ```ruby describe 'Test my website' do it 'test authentication' do Allure.label 'parentSuite' 'Web interface' Allure.suite 'Essential features' Allure.label 'subSuite', 'Authentication' # ... end end ``` ### Divide a test into steps Allure RSpec provides three ways of [creating steps and sub-steps](/docs/steps/): “annotated steps”, “block-based steps” and “no-op steps”, see the [reference](/docs/rspec-reference/#test-steps). **Annotated steps:** ```ruby describe 'Test my website' do it 'test authentication' do Steps.step_1 Steps.step_2 end end class Steps extend AllureStepAnnotation step 'Step 1' def self.step_1 step_1_1 step_1_2 end step 'Step 1.1' def self.step_1_1 # ... end step 'Step 1.2' def self.step_1_2 # ... end step 'Step 2' def self.step_2 step_2_1 step_2_2 end step 'Step 2.1' def self.step_2_1 # ... end step 'Step 2.2' def self.step_2_2 # ... end end ``` **Block-based steps and no-op steps:** ```ruby describe 'Test my website' do it 'test authentication' do Allure.run_step 'Step 1' do # ... Allure.step name: 'Step 1.1', status: :passed # ... Allure.step name: 'Step 1.2', status: :passed end Allure.run_step 'Step 2' do # ... Allure.step name: 'Step 2.1', status: :passed # ... Allure.step name: 'Step 2.2', status: :passed end end end ``` ### Describe parametrized tests With Allure RSpec, it is very easy to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, i.e. to run the same test logic with different test data. To do so, just write the test inside a loop and use the variable parameters in both its title and its body. To display a parameter value in the test report, pass it to the `Allure.parameter()` function. ```ruby describe 'Test my website' do auth_data = [ ['johndoe', 'qwerty'], ['johndoe@example.com', 'qwerty'], ] auth_data.each do |login, password| it "Test authentication as '#{login}'" do Allure.parameter 'Login', login Allure.parameter 'Password', password # ... end end end ``` ### Attach screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure RSpec reference](/docs/rspec-reference/#attachments). ```ruby describe 'Test my website' do it 'test authentication' do # ... Allure.add_attachment name: 'Screenshot', source: File.new('/path/to/image.png'), type: Allure::ContentType::PNG Allure.add_attachment name: 'Data', source: 'This is the file content.', type: Allure::ContentType::TXT end end ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, RSpec will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json bundle exec rspec ``` **Windows:** ```plain setx ALLURE_TESTPLAN_PATH "testplan.json" bundle exec rspec ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, specify the information in the [`environment_properties`](/docs/rspec-configuration/#environment-properties) configuration parameter. For example, it is a good idea to use this to remember the OS version and Ruby version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/ruby/environment-allure3-light.png, /images/ruby/environment-allure3-dark.png, /images/ruby/environment-allure2-light.png, /images/ruby/environment-allure2-dark.png ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.environment_properties = { os_platform: RbConfig::CONFIG['host_os'], ruby_version: RUBY_VERSION, } end ``` Note that if your launch includes multiple RSpec runs (see [How it works](/docs/how-it-works/)), Allure RSpec will only save the environment information from the latest run. --- ## RSpec configuration > URL: https://allurereport.org/docs/rspec-configuration/ # Allure RSpec configuration To customize behavior of the [Allure RSpec](/docs/rspec/) integration: 1. Create or edit a helper file to run at the beginning of each RSpec run, e.g., `spec/spec_helper.rb`. 1. In the helper file, require the `allure-rspec` module and run the `AllureRspec.configure()` function. The function yields a block with single argument `config`, in which you can modify various configuration parameters. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.results_directory = 'allure-results' config.clean_results_directory = true config.ignored_tags = [:retry, :retry_wait] end ``` 1. In the project's [`.rspec`](https://rspec.info/documentation/3.12/rspec-core/#store-command-line-options-rspec) file, add a `--require` directive for running the helper file, for example: ```plain --format AllureRspecFormatter --require spec_helper ``` ## results_directory Path to the directory where Allure RSpec will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to “reports/allure-results”. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.results_directory = 'allure-results' end ``` ## clean_results_directory If set to `true`, the results directory will be cleaned before generating new test results. By default, the existing data is kept intact, which allows combining results of multiple test runs into a single test report. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.clean_results_directory = true end ``` ## issue_tag, tms_tag The names of the RSpec metadata keys that will be used to add [issue links and TMS links](/docs/rspec-reference/#link) to the test result. Default to “issue” and “tms”. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.issue_tag = 'bug' config.tms_tag = 'test' end ``` See also: [`link_issue_pattern` and `link_tms_pattern`](#link_issue_pattern-link_tms_pattern). ## severity_tag The name of the RSpec metadata key that will be used to specify a test's [severity](/docs/rspec-reference/#severity). Defaults to “severity”. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.severity_tag = 'priority' end ``` ## epic_tag, feature_tag, story_tag The names of the RSpec metadata keys that will be used to specify a test's location in the [behavior-based hierarchy](/docs/rspec-reference/#behavior-based-hierarchy). Default to “epic”, “feature” and “story”. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.epic_tag = 'theme' config.feature_tag = 'functionality' config.story_tag = 'task' end ``` ## ignored_tags The names of tags that should not be processed when adding [tags](/docs/rspec-reference/#tag) via RSpec metadata. Use this to exclude tags that you set for purposes unrelated to Allure Report, e.g., for other RSpec plugins. Note that even when a tag is ignored, you still will be able to add it to the test result using the `Allure.tag()` function. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.ignored_tags = [:retry, :retry_wait] end ``` ## logger, logging_level Configure the logger to which Allure RSpec will send messages about the beginning and end of each test or test step. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.logger = Logger.new($stdout, Logger::DEBUG) config.logging_level = Logger::INFO end ``` ## link_issue_pattern, link_tms_pattern Define templates that can be used to construct full URLs from short identifiers, see the [reference](/docs/rspec-reference/#link). The pattern must contain `{}` at the position where the identifier should be placed. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.link_issue_pattern = 'https://issues.example.org/{}' config.link_tms_pattern = 'https://tms.example.org/{}' end ``` See also: [`issue_tag` and `tms_tag`](#issue_tag-tms_tag). ## global_labels Labels that will be added to every test result, see the [reference](/docs/rspec-reference/#label). Each label is a hash with the `:name` and `:value` keys. The labels don't replace the test's own labels, including the ones with the same name. By default, no global labels are added. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.global_labels = [ { name: 'layer', value: 'unit' }, { name: 'owner', value: 'qa-team' }, ] end ``` ## environment_properties Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.environment_properties = { os_platform: RbConfig::CONFIG['host_os'], ruby_version: RUBY_VERSION, } end ``` ## categories Path to the [categories file](/docs/how-it-works-categories-file/). Allure RSpec will copy this file to the result directory, so that it can be used for analyzing the test results in the report. ```ruby require 'allure-rspec' AllureRspec.configure do |config| config.categories = File.new("my_custom_categories.json") end ``` --- ## RSpec reference > URL: https://allurereport.org/docs/rspec-reference/ # Allure RSpec reference These are the functions that you can use to integrate your RSpec tests with Allure. Note that all the functions from the `Allure` module are also available via the object that RSpec passes to each test, like the `t` argument in the example below. ```ruby describe 'Test my website' do it 'test authentication' do |t| t.description_html 'This test attempts to log into the website using a login and a password.' # ... end end ``` ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `Allure::description_html: (String html)` Set the test's [description](/docs/v3/readability/#description). ```ruby describe 'Test my website' do it 'test authentication' do Allure.description_html 'This test attempts to log into the website using a login and a password.' # ... end end ``` ### Owner Set the test's [owner](/docs/v3/readability/#owner). ```ruby describe 'Test my website' do it 'test authentication' do Allure.label "owner", 'John Doe' # ... end end ``` ### Tag - `Allure::tag: (String tag)` Set the test's [tags](/docs/v3/readability/#tags). There are three ways to add a tag to your test. - From within the test body, call the `Allure.tag()` function. - Add a symbol to the RSpec metadata. Allure RSpec will add a tag using the symbol's name, unless it is one of the [ignored tags](/docs/rspec-configuration/#ignored-tags). - Add a key-value pair to the RSpec metadata. Allure RSpec will add a tag using the value, unless the key is one of the [ignored tags](/docs/rspec-configuration/#ignored-tags) or the value is false. The latter is convenient when you declare multiple tests in a loop and want to add or not add a tag based on a condition, as in the example below. **Runtime API:** ```ruby describe 'Test my website' do it 'test authentication' do Allure.tag 'Web interface' Allure.tag 'Authentication' # ... end end ``` **RSpec metadata:** ```ruby describe 'Test my website' do it 'test authentication', :'Web interface', :Authentication do # ... end end ``` **RSpec metadata (with a boolean value):** ```ruby describe 'Test my website' do auth_data = [ ['johndoe', false], ['johndoe@example.com', true], ] auth_data.each do |login, using_email| it "test authentication as #{login}", 'Using email': using_email do # ... end end end ``` ### Severity - `severity: String` Set the test's [severity](/docs/v3/readability/#severity). By default, Allure RSpec will use the value under the `severity` key in the RSpec metadata. The key [can be changed in the configuration](/docs/rspec-configuration/#severity-tag). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. ```ruby describe 'Test my website' do it 'test authentication', severity: 'critical' do # ... end end ``` ### Label - `Allure::label: (String name, String value)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. The first argument of the function is the label name. It can be any string. You can call `Allure.label()` multiple times to create an array of values under the given name. ```ruby describe 'Test my website' do it 'test authentication' do Allure.label 'my custom label', 'value' # ... end end ``` ### Allure ID (Allure TestOps) - `allure_id: String` Set the test's [ID](/docs/v3/readability/#id). This ID is used by the “test plan” feature in [Allure TestOps](https://qameta.io/) to select which tests to run. Allure RSpec does not export the ID to the test results. ```ruby describe 'Test my website' do it 'test authentication', allure_id: '123' do # ... end end ``` ### Link - `issue: String` - `tms: String` - `Allure::add_link: (url: String, name: String?, type: String?)` - `Allure::issue: (String name, String url)` - `Allure::tms: (String name, String url)` Add a [link](/docs/v3/readability/#links) related to the test. In Runtime API, you can do it by calling the `Allure.add_link()` function. The `type` affects the icon that is displayed next to the link in the test report. For convenience, Allure provides two shorthand functions with pre-selected link types: `Allure.issue()` and `Allure.tms()`. ```ruby describe 'Test my website' do it 'test authentication' do Allure.add_link url: 'https://dev.example.com/', name: 'Website' Allure.issue 'AUTH-123', 'https://issues.example.com/AUTH-123' Allure.tms 'TMS-456', 'https://tms.example.com/TMS-456' # ... end end ``` Alternatively, you can add issue links and TMS links via short identifiers in the RSpec metadata. To enable this, you need to [specify link patterns](/docs/rspec-configuration/#link-issue-pattern-link-tms-pattern) in the configuration. Allure RSpec will use the patterns to process values under specific keys (defaults to `issue` and `tms` respectively, but [can be changed in the configuration](/docs/rspec-configuration/#issue-tag-tms-tag)). Assuming the link patterns `https://issues.example.org/{}` and `https://tms.example.org/{}`, the test in the example below will have an issue link to `https://issues.example.org/AUTH-123` and a TMS link to `https://tms.example.org/TMS-456`. ```ruby describe 'Test my website' do it 'test authentication', issue: 'AUTH-123', tms: 'TMS-456' do # ... end end ``` If you need to specify more than one issue link or more than one TMS link to the same test, use keys with numeric postfixes after an underscore, e.g.: `issue_1`, `issue_2`, `tms_1`, `tms_2`, etc. Allure RSpec will process each of the values as usual, ignoring the keys' postfixes. ```ruby describe 'Test my website' do it 'test authentication', issue_1: 'AUTH-123', issue_2: 'AUTH-124' do # ... end end ``` ## Behavior-based hierarchy - `epic: String` - `feature: String` - `story: String` - `Allure::epic: (String epic)` - `Allure::feature: (String feature)` - `Allure::story: (String story)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). You can do it via the functions in Runtime API or by specifying RSpec metadata under certain keys. The keys by default are `epic`, `feature` and `story`, but they can be [changed in the configuration](/docs/rspec-configuration/#epic-tag-feature-tag-story-tag). **Runtime API:** ```ruby describe 'Test my website' do it 'test authentication' do Allure.epic 'Web interface' Allure.feature 'Essential features' Allure.story 'Authentication' # ... end end ``` **RSpec metadata:** ```ruby describe 'Test my website' do it 'test authentication', epic: 'Web interface', feature: 'Essential features', story: 'Authentication' do # ... end end ``` ## Suite-based hierarchy - `Allure::suite: (String name)` Assign the names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). ```ruby describe 'Test my website' do it 'test authentication' do Allure.label 'parentSuite' 'Web interface' Allure.suite 'Essential features' Allure.label 'subSuite', 'Authentication' # ... end end ``` ## Test steps - `AllureStepAnnotation::step: (name: String, status: Status?)` - `Allure::run_step: (String name) { (*any) -> untyped }` - `Allure::step: (name: String, status: Status) { (*any) -> untyped }` Define [test steps](/docs/steps/). There are three ways of defining a step. - **Annotated steps** In a class that extends `AllureStepAnnotation`, define a method containing a test step and add the `step` annotation to it, with an optional `name` argument (defaults to the method's name). Each time the method is called during the test execution, a new step will be created for the test report. If the method calls another function that also has the `step` attribute, Allure Report will create a sub-step inside the current step. - **Block-based steps** Write a test step inside a block for the `Allure.run_step()` function. If the block returns a value, `Allure.run_step()` will return it without modification, and it will not affect the report. - **No-op steps** If you call the `Allure.step()` function, Allure will add to the report a no-op step. This allows for a log-style reporting within a test or within a larger step. A no-op step finishes immediately after it started and cannot have any sub-steps. The optional second argument indicates the status that will be shown for the step in the report. Allowed values are: `:passed` (the default), `:failed`, `:broken`, `:skipped`, and `:unknown`. **Annotated steps:** ```ruby describe 'Test my website' do it 'test authentication' do Steps.step_1 Steps.step_2 end end class Steps extend AllureStepAnnotation step 'Step 1' def self.step_1 step_1_1 step_1_2 end step 'Step 1.1' def self.step_1_1 # ... end step 'Step 1.2' def self.step_1_2 # ... end step 'Step 2' def self.step_2 step_2_1 step_2_2 end step 'Step 2.1' def self.step_2_1 # ... end step 'Step 2.2' def self.step_2_2 # ... end end ``` **Block-based steps and no-op steps:** ```ruby describe 'Test my website' do it 'test authentication' do Allure.run_step 'Step 1' do # ... Allure.step name: 'Step 1.1', status: :passed # ... Allure.step name: 'Step 1.2', status: :passed end Allure.run_step 'Step 2' do # ... Allure.step name: 'Step 2.1', status: :passed # ... Allure.step name: 'Step 2.2', status: :passed end end end ``` ## Parametrized tests - `Allure::parameter: (String name, String value)` With Allure RSpec, it is very easy to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, i.e. to run the same test logic with different test data. To do so, just write the test inside a loop and use the variable parameters in both its title and its body. To display a parameter value in the test report, pass it to the `Allure.parameter()` function. ```ruby describe 'Test my website' do auth_data = [ ['johndoe', 'qwerty'], ['johndoe@example.com', 'qwerty'], ] auth_data.each do |login, password| it "Test authentication as '#{login}'" do Allure.parameter 'Login', login Allure.parameter 'Password', password # ... end end end ``` ## Attachments - `Allure::add_attachment: (name: String, source: File | String, type: String, test_case: bool?)` Add an attachment with a given `name`. The content is passed in the `source` argument and can be a text string or a [File](https://docs.ruby-lang.org/en/master/File.html). To ensure that the reader's web browser will display attachments correctly, pass the appropriate media type of the content as `type`. The `Allure::ContentType` class contains some popular media types, or you can use a custom string, as long as it will be understood by the web browser when viewing the test report. When inside a [test step](#test-steps), the function adds the attachment to the current step by default. To add it to the whole test result instead, pass `true` to the optional `test_case` argument. ```ruby describe 'Test my website' do it 'test authentication' do # ... Allure.add_attachment name: 'Screenshot', source: File.new('/path/to/image.png'), type: Allure::ContentType::PNG Allure.add_attachment name: 'Data', source: 'This is the file content.', type: Allure::ContentType::TXT end end ``` ## Run-level attachments and errors - `Allure::add_global_attachment: (name: String, source: File | String, type: String)` - `Allure::add_global_error: (message: String?, trace: String?, known: bool?, muted: bool?, flaky: bool?)` Add an attachment or an error that belongs to the whole test run rather than to a single test, for example, a server log or an infrastructure failure. They are shown in the report as [global attachments and global errors](/docs/global-errors-and-attachments/) in Allure Report 3 and on the **Run** tab in Allure Report 2.46.0 and newer. The arguments of `add_global_attachment` work the same way as for [`add_attachment`](#attachments). If a `before(:suite)` or `after(:suite)` hook fails, Allure RSpec 2.30.0 and newer records the failure as a global error automatically. ```ruby RSpec.configure do |config| config.after(:suite) do Allure.add_global_attachment name: 'Server log', source: File.new('/path/to/server.log'), type: Allure::ContentType::TXT end end ``` --- ## Rust Cargo Test > URL: https://allurereport.org/docs/rust/ # Getting started with Rust Cargo Test [![allure-cargotest crates.io latest version](https://img.shields.io/crates/v/allure-cargotest?style=flat "allure-cargotest crates.io latest version")](https://crates.io/crates/allure-cargotest) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your Rust tests. Info: The main user-facing crate is [`allure-cargotest`](https://github.com/allure-framework/allure-rust/tree/main/crates/allure-cargotest). If you are building your own integration for a custom runner or framework, use [`allure-rust-commons`](https://github.com/allure-framework/allure-rust/tree/main/crates/allure-rust-commons) instead. The official `allure-rust` repository also publishes [Allure Reqwest](/docs/reqwest/) (captures `reqwest` HTTP calls as Allure HTTP Exchange attachments) and [Allure Diesel](/docs/diesel/) (records executed Diesel queries as Allure steps). ## Setting up ### 1. Prepare your project 1. Make sure a recent stable Rust toolchain is installed. The official `allure-rust` workspace is published for Rust 1.74 and newer and uses the Rust 2021 edition. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Install Allure Report. The official `allure-rust` repository documents the workflow with the [Allure 3 CLI](/docs/v3/install/). 1. Add the `allure-cargotest` integration: ```bash cargo add allure-cargotest --dev ``` 1. Annotate your tests with `#[allure_test]`. You can also mark helper functions with `#[step]` so they appear as separate steps in the report. ```rust use allure_cargotest::{allure_test, step}; #[step] fn open_login_page() { // your step implementation } #[allure_test] #[test] fn login_works() { allure.epic("Web interface"); allure.feature("Authentication"); allure.story("Login with username and password"); allure.parameter("browser", "firefox"); open_login_page(); allure.attachment("page.html", "text/html", "..."); } ``` ### 2. Run tests Run your tests the same way as usual: ```bash cargo test ``` By default, `allure-cargotest` writes the results to `target/allure-results`. To use a different directory, set `ALLURE_RESULTS_DIR` before the test run: ```bash ALLURE_RESULTS_DIR=./allure-results cargo test ``` If the results directory already exists, the new files are added to the existing ones, so that a future report will be based on all of them. ### 3. Generate a report After the test run, generate and open the report with the Allure CLI: ```bash allure generate ./target/allure-results --output ./target/allure-report allure open ./target/allure-report ``` If you changed the results directory with `ALLURE_RESULTS_DIR`, use that path in the `allure generate` command. ## Writing tests Rust Cargo Test extends plain `cargo test` output with richer reporting features. You can use it to: - add [descriptions, owners, links, and other metadata](#add-metadata), - organize tests into [behavior-based and suite-based hierarchies](#organize-tests), - split the execution into [nested steps](#divide-a-test-into-steps), - describe [parameters and attachments](#add-parameters-and-attachments), - write [async tests with Tokio](#run-async-tests-with-tokio), - run only selected tests through a [test plan file](#select-tests-via-a-test-plan-file). ### Add metadata Inside a function marked with `#[allure_test]`, the macro injects an `allure` facade that you can use to enrich the test result: ```rust use allure_cargotest::allure_test; #[allure_test(name = "Login works", id = "AUTH-1")] #[test] fn login_works() { allure.description("This test verifies login with a username and a password."); allure.owner("John Doe"); allure.tag("smoke"); allure.severity("critical"); allure.issue("AUTH-123", "https://jira.example.com/browse/AUTH-123"); allure.tms("TMS-456", "https://tms.example.com/cases/TMS-456"); } ``` A Rust doc comment on the test function is used as the default markdown description, so you often don't need to call `allure.description(...)` at all: ```rust use allure_cargotest::allure_test; /// Verifies login with a username and a password. #[allure_test] #[test] fn login_works() { // ... } ``` Calling `allure.description(...)` in the test body overrides the doc-comment description for that run, and `allure.description_html(...)` sets an explicit HTML description. Use `#[allure_test(doc = false)]` to disable the doc-comment description for a single test. ### Organize tests Allure supports both behavior-based and suite-based hierarchies. For example: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() { allure.epic("Web interface"); allure.feature("Authentication"); allure.story("Login with username and password"); allure.parent_suite("UI tests"); allure.suite("Authentication"); allure.sub_suite("Positive scenarios"); } ``` When you use `#[allure_test]`, `allure-cargotest` also derives suite labels from the Rust module path automatically. Explicit calls to `allure.parent_suite(...)`, `allure.suite(...)`, or `allure.sub_suite(...)` override the synthetic labels with the same name. ### Divide a test into steps You can declare reusable step functions with `#[step]`: ```rust use allure_cargotest::{allure_test, step}; #[step(name = "Open login page")] fn open_login_page() { // ... } #[step(name = "Submit credentials")] fn submit_credentials() { // ... } #[allure_test] #[test] fn login_works() { open_login_page(); submit_credentials(); } ``` You can also create steps directly from the runtime API. `allure.step(name, || { ... })` wraps a closure as a step and returns its value; `allure.enter_step(name)` returns a guard that keeps the step open until it is dropped, which is useful when the step spans code that isn't a single closure: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() { allure.step("Open login page", || { // ... }); let _guard = allure.enter_step("Check profile page"); // ... } ``` ### Add parameters and attachments Parameters and attachments are stored in the generated Allure results and rendered in the report: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() { allure.parameter("browser", "firefox"); allure.parameter("environment", "staging"); allure.attachment( "request.json", "application/json", br#"{"username":"demo","rememberMe":true}"#, ); } ``` ### Run async tests with Tokio Compose `#[allure_test]` with a runtime-specific test macro such as `#[tokio::test]`. Add and configure Tokio in your own test crate; `allure-cargotest` itself does not depend on Tokio. ```rust use allure_cargotest::allure_test; #[allure_test] #[tokio::test] async fn login_works_async() { allure.feature("Authentication"); tokio::task::yield_now().await; allure.parameter("runtime", "tokio"); } ``` Allure context is available in the root async test body and in awaited helpers. Independently spawned Tokio tasks do not implicitly inherit the current Allure context. Both sync and async tests can also return `Result<(), E>`, `ExitCode`, or another type that implements `std::process::Termination`, instead of `()`. Returned `Err` values and unsuccessful termination values are reported to Allure before Cargo interprets the result: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() -> Result<(), String> { allure.feature("Authentication"); if !login("demo") { return Err("login failed".to_string()); } Ok(()) } ``` ### Select tests via a test plan file `allure-cargotest` supports the standard Allure test plan mechanism via the `ALLURE_TESTPLAN_PATH` environment variable. Create a JSON file such as: ```json { "version": "1.0", "tests": [{ "selector": "auth::tests::login_works" }] } ``` Then run the tests with: ```bash ALLURE_TESTPLAN_PATH=./testplan.json cargo test ``` Entries with `selector` match the full Rust test name, including its module path, and work with `#[allure_test]`. Warning: Entries with `id` are matched against an explicit Allure ID (or an `@allure.id=...` / `@allure.id:...` tag) that the caller passes to the test-plan lookup explicitly. `#[allure_test(id = "...")]` does not currently forward its `id` to that lookup, so `id` entries have no effect on tests written with the macro — use `selector` entries for them instead. Integrations that call `CargoTestReporter::run_test_with_metadata` directly can pass an `allure_id`, and `id` entries do match there. See [Reference](/docs/rust-reference/) for `CargoTestReporter`. ## Building a custom integration If you need to integrate Allure with a custom Rust test runner or framework, use `allure-rust-commons`: ```bash cargo add allure-rust-commons ``` At that level, you create a writer, initialize a runtime, start a test case, and stop it when execution ends: ```rust use allure_rust_commons::{AllureRuntime, FileSystemResultsWriter, StartTestCaseParams, Status}; fn main() -> Result<(), Box> { let writer = FileSystemResultsWriter::new("target/allure-results")?; let runtime = AllureRuntime::new(writer); let lifecycle = runtime.lifecycle(); lifecycle.start_test_case( StartTestCaseParams::new("login_works").with_full_name("auth::login_works"), ); // ... update metadata, add steps, and add attachments ... lifecycle.stop_test_case(Status::Passed, None); Ok(()) } ``` See [Reference](/docs/rust-reference/) for the main macros and runtime APIs, or [Configuration](/docs/rust-configuration/) for the supported environment variables. --- ## Rust Cargo Test configuration > URL: https://allurereport.org/docs/rust-configuration/ # Rust Cargo Test configuration The [`allure-cargotest`](/docs/rust/) integration is configured through environment variables. ## ALLURE_RESULTS_DIR Overrides the default directory where `#[allure_test]` writes Allure results. When unset, `allure-cargotest` uses `target/allure-results`. ```bash ALLURE_RESULTS_DIR=./allure-results cargo test ``` ## ALLURE_HOST_NAME Overrides the `host` label that `allure-cargotest` adds automatically. If this variable is not set, the integration tries to detect the host name from the current machine. ```bash ALLURE_HOST_NAME=ci-linux-01 cargo test ``` ## ALLURE_THREAD_NAME Overrides the `thread` label that `allure-cargotest` adds automatically. If this variable is not set, the integration uses the current thread name or thread ID. ```bash ALLURE_THREAD_NAME=worker-1 cargo test ``` ## ALLURE_LABEL\_\* Adds global labels to every test result. Any environment variable whose name starts with `ALLURE_LABEL_` becomes an Allure label. For example: ```bash ALLURE_LABEL_epic="Web interface" \ ALLURE_LABEL_owner="QA Team" \ cargo test ``` This applies the `epic` and `owner` labels to every test in the run. ## allure.label.\* Adds global labels using the alternative naming scheme that some CI tools already use. Any environment variable whose name starts with `allure.label.` is treated the same way as `ALLURE_LABEL_*`. ```bash allure.label.layer=e2e cargo test ``` ## ALLURE_TESTPLAN_PATH Points to a JSON file that defines which tests should run. The file uses the standard Allure test plan shape: ```json { "version": "1.0", "tests": [{ "selector": "auth::tests::login_works" }] } ``` Run the tests with: ```bash ALLURE_TESTPLAN_PATH=./testplan.json cargo test ``` Entries with `selector` match the full Rust test name, including its module path, and work with `#[allure_test]`. Warning: Entries with `id` are intended to match tests that expose an explicit Allure ID (for example via `#[allure_test(id = "AUTH-1")]`), but `#[allure_test]` does not currently forward its `id` into test-plan matching, so `id` entries have no effect on macro-based tests — use `selector` entries for them instead. `id` matching does work for integrations that call [`CargoTestReporter::run_test_with_metadata`](/docs/rust-reference/#manual-integration-with-cargotestreporter) directly with an explicit `allure_id`. If `ALLURE_TESTPLAN_PATH` is unset, the file does not exist, or the JSON is malformed, `allure-cargotest` skips filtering and runs the tests normally. ## ALLURE_LOG_ASSERTS Controls whether `assert!`, `assert_eq!`, `assert_ne!`, `debug_assert!`, `debug_assert_eq!`, and `debug_assert_ne!` calls inside `#[allure_test]` and `#[step]` function bodies are logged as Allure steps. Assertion logging is enabled by default. Set it to `false` to disable it for a run, overriding any package-level setting: ```bash ALLURE_LOG_ASSERTS=false cargo test ``` You can also disable assertion logging for a whole package with Cargo metadata instead of an environment variable — see [`log_asserts`](#add-labels-for-all-tests-in-a-package) below. ## Configure labels in Cargo.toml Use Cargo package metadata to add labels for local and CI runs without environment variables. A package is the Cargo package defined by the relevant `Cargo.toml`. ### Add labels for all tests in a package Add labels under `[package.metadata.allure.labels]`: ```toml [package.metadata.allure.labels] a = "a-value" b = ["b-value1", "b-value2"] ``` This adds `a=a-value`, `b=b-value1`, and `b=b-value2` to every `#[allure_test]` in the package. String array values add the same label multiple times. You can also disable assertion logging for the whole package here: ```toml [package.metadata.allure] log_asserts = false ``` ### Add labels for only some Rust modules Add one `[[package.metadata.allure.modules]]` entry per Rust module path. The `module` value matches the current Rust `module_path!()` exactly, or any module below it. ```toml [[package.metadata.allure.modules]] module = "org::example" labels = { a = "a-value", b = ["b-value1", "b-value2"] } ``` This applies to tests whose module path is `org::example`, `org::example::api`, `org::example::api::v1`, and so on. For integration tests in `tests/api.rs`, the test file is its own crate, so module paths usually start with the file stem: ```toml [[package.metadata.allure.modules]] module = "api::org::example" labels = { a = "a-value" } ``` ### Add labels for only test files Add one `[[package.metadata.allure.modules]]` entry per file and use `path`. The path is relative to the package root and uses the same element-wise file path that appears in `titlePath`. ```toml [[package.metadata.allure.modules]] path = "tests/payments.rs" labels = { a = "a-value", b = ["b-value1", "b-value2"] } ``` This applies to every `#[allure_test]` in `tests/payments.rs`. You can also match a source file or a directory: ```toml [[package.metadata.allure.modules]] path = "src/payments.rs" labels = { component = "payments" } [[package.metadata.allure.modules]] path = "tests/api/" labels = { layer = "api" } ``` The directory form (a path ending in `/`) matches every test file whose relative path starts with that directory. ## Automatic labels added by allure-cargotest When you use `#[allure_test]` or `CargoTestReporter`, `allure-cargotest` adds a few labels automatically: - `language = rust` - `framework = cargo-test` - `host` - `thread` It also derives suite labels from the Rust module path: - a single module segment becomes `suite`, - two segments become `parentSuite` and `suite`, - three or more segments become `parentSuite`, `suite`, and `subSuite`. Explicit calls to `allure.parent_suite(...)`, `allure.suite(...)`, or `allure.sub_suite(...)` override the automatically derived labels with the same name. --- ## Rust Cargo Test reference > URL: https://allurereport.org/docs/rust-reference/ # Rust Cargo Test reference These are the main building blocks you can use to integrate Rust tests with Allure by using `allure-cargotest`. ## Macros ### `#[allure_test]` Supported forms: - `#[allure_test]` - `#[allure_test(name = "Login works")]` - `#[allure_test(id = "AUTH-1")]` - `#[allure_test(doc = false)]` - `#[allure_test(name = "Login works", id = "AUTH-1")]` Use the macro together with `#[test]`: ```rust use allure_cargotest::allure_test; #[allure_test(name = "Login works", id = "AUTH-1")] #[test] fn login_works() { allure.feature("Authentication"); allure.story("Login with username and password"); } ``` What the macro does: - initializes the reporter by using `ALLURE_RESULTS_DIR` or `target/allure-results`, - injects an `allure` facade into the test body, - starts and stops the test lifecycle automatically, - applies the default labels described in [Configuration](/docs/rust-configuration/), - derives suite labels from `module_path!()`, - uses the function's Rust doc comment as the default markdown description, unless `doc = false` is set or `description(...)` is called in the test body. Behavior notes: - `#[allure_test]` supports both synchronous functions and `async fn`; compose it with a runtime-specific test macro such as `#[tokio::test]` placed below it (`allure-cargotest` does not depend on Tokio itself), - besides `()`, test functions may return `Result` (where `T` is itself a supported return type), `ExitCode`, or any other type implementing `std::process::Termination`; `Err` values and unsuccessful termination values are reported to Allure before Cargo interprets the result, - `#[should_panic]` is supported only for tests that return `()`, - `#[should_panic(expected = "...")]` marks the test as passed only when the panic message contains the expected substring, - a panic inside the test body is always reported `failed`, regardless of its message — `broken` is only produced by a _returned_ `Result::Err`, an unsuccessful `ExitCode`, or an unsuccessful custom `Termination` value (see the return-type bullet above), never by panicking. ### `#[step]` Supported forms: - `#[step]` - `#[step(name = "Open login page")]` Use `#[step]` on helper functions that you want to render as steps in the report: ```rust use allure_cargotest::{allure_test, step}; #[step(name = "Open login page")] fn open_login_page() { // ... } #[allure_test] #[test] fn login_works() { open_login_page(); } ``` When the function runs inside an active Allure test, the integration starts and stops a step automatically. Outside an active Allure context, the function behaves like a normal Rust function. ## Runtime facade API Inside `#[allure_test]`, the `allure` facade provides methods for the most common reporting tasks. ### Metadata and labels - `allure.description(text)` - `allure.description_html(html)` - `allure.label(name, value)` - `allure.labels([(name, value), ...])` - `allure.owner(value)` - `allure.severity(value)` - `allure.layer(value)` - `allure.tag(value)` - `allure.tags(["smoke", "auth"])` - `allure.id(value)` Example: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() { allure.description("Checks that a valid user can sign in."); allure.owner("John Doe"); allure.severity("critical"); allure.label("microservice", "ui"); allure.tags(["smoke", "auth"]); } ``` ### Identity and display - `allure.display_name(name)` — overrides the name shown in the report, independent of `#[allure_test(name = "...")]`, for when the display name has to be computed at runtime - `allure.history_id(value)` — overrides the identifier Allure uses to match this result against previous runs for retry/flaky/history tracking (by default derived from the full test name and non-excluded parameters) - `allure.test_case_id(value)` — overrides the identifier Allure uses to group results into one logical test case across environments (by default derived from the full test name) Example: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() { allure.display_name("Login works (computed at runtime)"); allure.history_id("login-works-stable-id"); allure.test_case_id("AUTH-LOGIN-001"); } ``` ### Hierarchies - `allure.epic(value)` - `allure.feature(value)` - `allure.story(value)` - `allure.parent_suite(value)` - `allure.suite(value)` - `allure.sub_suite(value)` Example: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() { allure.epic("Web interface"); allure.feature("Authentication"); allure.story("Login with username and password"); allure.parent_suite("UI tests"); allure.suite("Authentication"); allure.sub_suite("Positive scenarios"); } ``` ### Links and parameters - `allure.link(url, Some(name), Some(link_type))` - `allure.links([(url, Some(name), Some(link_type)), ...])` - `allure.issue(name, url)` - `allure.tms(name, url)` - `allure.parameter(name, value)` - `allure.parameter_excluded(name, value, excluded)` — `excluded: true` keeps the parameter visible in the report without letting it affect the history/retry identity computed from parameters - `allure.parameter_mode(name, value, mode)` — a `ParameterMode` from `allure_rust_commons`: `Masked` shows the name but hides the value (for secrets), `Hidden` removes the parameter from the report entirely, `Default` is the normal display - `allure.parameter_with_options(name, value, excluded, mode)` — combines both controls in one call Example: ```rust use allure_cargotest::allure_test; use allure_rust_commons::ParameterMode; #[allure_test] #[test] fn login_works() { allure.issue("AUTH-123", "https://jira.example.com/browse/AUTH-123"); allure.tms("TMS-456", "https://tms.example.com/cases/TMS-456"); allure.parameter("browser", "firefox"); allure.parameter_mode("password", "hunter2", ParameterMode::Masked); allure.parameter_excluded("sessionToken", "zzz-123", true); } ``` ### Attachments - `allure.attachment(name, content_type, body)` - `allure.attachment_path(name, content_type, path)` — reads the attachment body from a file - `allure.attach_trace(path)` / `allure.attach_trace_named(name, path)` — attaches an existing [Playwright trace](/docs/attachments/#playwright-traces) archive (a convenience wrapper around `attachment_path` using the `application/vnd.allure.playwright-trace` content type, so Allure's trace viewer opens it instead of offering a plain zip download; `attach_trace` defaults the attachment name to `trace.zip`) — does not generate traces or depend on Playwright itself Example: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() { allure.attachment( "response.json", "application/json", br#"{"status":"ok","user":"demo"}"#, ); allure .attachment_path("server.log", "text/plain", "assets/server.log") .expect("failed to read the log file"); } ``` ### Run-level (global) diagnostics These attach evidence to the whole test run (the report's launch/Environment level) instead of the current test. They can be called from inside `#[allure_test]`, or on their own even when no test is currently active: - `allure.global_attachment(name, content_type, body)` - `allure.global_attachment_path(name, content_type, path)` - `allure.global_error(message)` - `allure.global_error_with_trace(message, trace)` Example: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() { allure .global_attachment("run log", "text/plain", "shared setup output") .expect("failed to write the run-level attachment"); } ``` ### Steps - `allure.step(name, || { ... })` runs a closure as a step and returns its value - `allure.enter_step(name)` returns a `StepGuard` that keeps the step open until dropped - `allure.log_step(name)` - `allure.log_step_with(name, status, error)` Examples: ```rust use allure_cargotest::{allure_test, Status}; #[allure_test] #[test] fn login_works() { allure.step("Open login page", || { // ... }); let mut guard = allure.enter_step("Submit credentials"); // ... drop(guard); allure.log_step("Verify the page title"); allure.log_step_with("Check audit log", Some(Status::Failed), Some("entry not found")); } ``` `StepGuard` also lets you override the final step status before the guard is dropped, with `fail` or the more general `set_status`: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() { let mut guard = allure.enter_step("Validate response"); guard.fail("Unexpected status code"); } ``` ### Stages - `allure.stage(name)` opens a new named "stage" step. Unlike `step`/`enter_step`, you don't close a stage explicitly — starting the next stage (or ending the test) automatically closes the previous one as passed. Anything recorded in between (steps, attachments, logged assertions) nests under whichever stage is currently open. Example: ```rust use allure_cargotest::allure_test; #[allure_test] #[test] fn login_works() { allure.stage("open login page"); allure.log_step("login page opened"); allure.stage("collect evidence"); allure.attachment("page.html", "text/html", "..."); } ``` This produces two top-level steps — `open login page` (containing `login page opened`) and `collect evidence` (containing the `page.html` attachment) — without manually nesting closures. ## Manual integration with `CargoTestReporter` If macros are not enough for your test harness, you can use `CargoTestReporter` directly: ```rust use allure_cargotest::CargoTestReporter; fn main() -> Result<(), Box> { let reporter = CargoTestReporter::new("target/allure-results")?; reporter.run_test("login_works", |allure| { allure.feature("Authentication"); allure.parameter("browser", "firefox"); }); Ok(()) } ``` Useful methods: - `CargoTestReporter::new(results_dir)` - `run_test(name, |allure| { ... })` - `run_test_with_metadata(test_name, full_name, allure_id, tags, |allure| { ... })` - `run_test_with_result(name, |allure| { ... })` - `is_selected(test_name, full_name, allure_id, tags)` `run_test_with_metadata` and `is_selected` are the only entry points that forward an explicit `allure_id`/`tags` pair into test-plan matching, so `id` entries in an [`ALLURE_TESTPLAN_PATH`](/docs/rust-configuration/#allure-testplan-path) file only take effect for integrations built on `CargoTestReporter` directly — not for `#[allure_test(id = "...")]`, which currently only participates in `selector` matching. ## Building a custom integration with `allure-rust-commons` Use `allure-rust-commons` when you need low-level control over the lifecycle: ```rust use allure_rust_commons::{ AllureRuntime, FileSystemResultsWriter, StartTestCaseParams, Status, }; fn main() -> Result<(), Box> { let writer = FileSystemResultsWriter::new("target/allure-results")?; let runtime = AllureRuntime::new(writer); let lifecycle = runtime.lifecycle(); lifecycle.start_test_case( StartTestCaseParams::new("login_works").with_full_name("auth::login_works"), ); lifecycle.stop_test_case(Status::Passed, None); Ok(()) } ``` The main low-level types are: - `AllureRuntime` - `AllureLifecycle` - `StartTestCaseParams` - `FileSystemResultsWriter` - `Status` and `StatusDetails` - the model types exported from `allure_rust_commons::model` --- ## Selenide > URL: https://allurereport.org/docs/selenide/ # Getting started with Allure Selenide [![Allure Selenide latest version](https://img.shields.io/maven-central/v/io.qameta.allure/allure-selenide?style=flat "allure-selenide")](https://mvnrepository.com/artifact/io.qameta.allure/allure-selenide) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/) and your [Selenide](https://selenide.org/) tests. ## Setting up Allure Selenide works alongside your test framework — set up [JUnit Jupiter](/docs/junit-jupiter/) or [TestNG](/docs/testng/) first, then add the steps below. This integration targets Selenide 7.x and requires Java 17 or newer. To integrate Allure Selenide into your project, you need to: 1. Add Allure Selenide dependencies. 2. Register the listener. 3. Specify a location for Allure results storage. ### Add Allure dependencies **Maven:** ```xml VERSION io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-selenide test ``` **Gradle (Kotlin):** ```kts val allureVersion = "VERSION" dependencies { testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) testImplementation("io.qameta.allure:allure-selenide") } ``` **Gradle (Groovy):** ```groovy def allureVersion = "VERSION" dependencies { testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") testImplementation "io.qameta.allure:allure-selenide" } ``` ### Register the listener Register `AllureSelenide` as a Selenide listener before any browser actions run. The listener name (`"allure"`) is an identifier used by Selenide's listener registry and can be any string, as long as it is unique among registered listeners. ```java import com.codeborne.selenide.logevents.SelenideLogger; import io.qameta.allure.selenide.AllureSelenide; SelenideLogger.addListener("allure", new AllureSelenide()); ``` Register it once before your tests start — for example, in a JUnit 5 `@BeforeAll`, a TestNG `@BeforeSuite`, or a shared setup class: ```java @BeforeAll static void setUpAllure() { SelenideLogger.addListener("allure", new AllureSelenide()); } ``` Steps and artifacts are only recorded when there is an active Allure test context. Selenide actions that run outside of a test — for example, in a static initializer or before the test framework starts a test — are silently ignored. ### Specify Allure results location Allure saves test results in the project's root directory by default. It is recommended to store results in the build output directory instead. Create an `allure.properties` file in `src/test/resources`: **Maven:** ```properties allure.results.directory=target/allure-results ``` **Gradle (Kotlin):** ```properties allure.results.directory=build/allure-results ``` **Gradle (Groovy):** ```properties allure.results.directory=build/allure-results ``` ### Generate a report After running your tests, convert the results into an HTML report using one of these commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Selenide actions as steps Once the listener is registered, Allure Selenide records every Selenide log event as a step in the test report. No additional code is required. Step names are generated from Selenide's log events and include the element locator, action name, and any arguments. The exact format is determined by Selenide. A failed step gets `FAILED` status when the underlying exception is an assertion error, or `BROKEN` for any other exception. Steps are nested under any enclosing Allure steps you define: ```java Allure.step("Log in", () -> { $("#email").setValue("user@example.com"); // appears as a sub-step of "Log in" $("#password").setValue("secret"); $(".login-btn").click(); }); ``` By default, all Selenide log events are recorded as steps. To exclude Selenide's built-in element interaction steps (those of type `SelenideLog`) from the report, set `.includeSelenideSteps(false)` when registering the listener. Screenshots, page source, and browser logs are still captured on failure even when steps are excluded, and are attached to the enclosing Allure step or test. See [configuration](/docs/selenide-configuration/) for details. ## Capturing artifacts on failure By default, when a Selenide step fails, Allure Selenide automatically captures artifacts and attaches them to that step in the report. By default, the following are captured: - **Screenshot** — a PNG image of the current browser state (`image/png`), attached with the name `Screenshot`. - **Page source** — the current HTML of the page (`text/html`), attached with the name `Page source`. By default, these are attached to the failing step rather than to the overall test result. If the browser is not open at the time of failure — for example, when a step fails before the browser launches — no screenshot or page source is captured, and the step is still recorded with its failure status. To disable either artifact, configure the listener when registering it: ```java SelenideLogger.addListener("allure", new AllureSelenide() .screenshots(false) .savePageSource(false)); ``` See [configuration](/docs/selenide-configuration/) for all available options. ## Browser logs Allure Selenide can capture Selenium WebDriver browser logs and attach them to a failing step. Log capture is disabled by default and must be enabled explicitly for each log type you want to collect. ```java import io.qameta.allure.selenide.LogType; import java.util.logging.Level; SelenideLogger.addListener("allure", new AllureSelenide() .enableLogs(LogType.BROWSER, Level.SEVERE)); ``` Multiple log types can be enabled at once: ```java SelenideLogger.addListener("allure", new AllureSelenide() .enableLogs(LogType.BROWSER, Level.ALL) .enableLogs(LogType.PERFORMANCE, Level.ALL)); ``` Each enabled log type produces a separate attachment named `Logs from: ` (for example, `Logs from: browser`). When steps are included (the default), the attachment is added to the failing Selenide step. When `includeSelenideSteps(false)` is set, it is attached to the enclosing Allure step or test instead. Available `LogType` values correspond to Selenium's built-in log types: `BROWSER`, `CLIENT`, `DRIVER`, `PERFORMANCE`, `PROFILER`, and `SERVER`. Not all browsers support all log types — check your browser's WebDriver documentation for what is available. ## Environment information To add environment details (OS, Java version, browser, Selenide version) to the main report page, see [Environment file](/docs/how-it-works-environment-file/). --- ## Selenide configuration > URL: https://allurereport.org/docs/selenide-configuration/ # Allure Selenide configuration This page describes the configuration options that affect the behavior of [Allure Selenide](/docs/selenide/). All options are set via the fluent API on `AllureSelenide` when registering the listener: ```java SelenideLogger.addListener("allure", new AllureSelenide() .screenshots(true) .savePageSource(true) .includeSelenideSteps(true) .enableLogs(LogType.BROWSER, Level.SEVERE)); ``` ## screenshots ```java new AllureSelenide().screenshots(boolean) ``` Whether to capture a screenshot and attach it to the failing step when a Selenide step fails. Defaults to `true`. The screenshot is captured from the currently active WebDriver session. If no browser is open at the point of failure, no screenshot is taken and no attachment is added. ```java SelenideLogger.addListener("allure", new AllureSelenide() .screenshots(false)); ``` ## savePageSource ```java new AllureSelenide().savePageSource(boolean) ``` Whether to capture the current page HTML and attach it to the failing step when a Selenide step fails. Defaults to `true`. The page source is captured from the currently active WebDriver session as a `text/html` attachment. If no browser is open at the point of failure, no attachment is added. ```java SelenideLogger.addListener("allure", new AllureSelenide() .savePageSource(false)); ``` ## includeSelenideSteps ```java new AllureSelenide().includeSelenideSteps(boolean) ``` Whether to record Selenide's built-in element interaction events as Allure steps. Defaults to `true`. When set to `false`, events that are `SelenideLog` instances (Selenide's built-in element interaction events) are not recorded as steps. Events of other types — for example, from custom `LogEventListener` implementations — are still recorded regardless of this setting. When set to `false`, screenshots, page source, and browser logs are still captured when a Selenide step fails. They are attached to the enclosing Allure step or test rather than to a dedicated Selenide step. Use `false` when you want a cleaner report that shows only your high-level `Allure.step()` calls rather than every low-level Selenide action. ```java SelenideLogger.addListener("allure", new AllureSelenide() .includeSelenideSteps(false)); ``` ## enableLogs ```java new AllureSelenide().enableLogs(LogType logType, Level logLevel) ``` Enable capture of a Selenium WebDriver browser log type at the given level. When a Selenide step fails, logs of this type are retrieved and attached to the failing step. No log types are enabled by default. The attachment is named `Logs from: ` (for example, `Logs from: browser`). Multiple log types can be enabled by chaining calls: ```java SelenideLogger.addListener("allure", new AllureSelenide() .enableLogs(LogType.BROWSER, Level.ALL) .enableLogs(LogType.PERFORMANCE, Level.ALL)); ``` Available `LogType` values: `BROWSER`, `CLIENT`, `DRIVER`, `PERFORMANCE`, `PROFILER`, `SERVER`. Browser support varies — not all browsers expose all log types via WebDriver. ## disableLogs ```java new AllureSelenide().disableLogs(LogType logType) ``` Remove a previously enabled log type from capture. Has no effect if the log type was not enabled. --- ## Selenide reference > URL: https://allurereport.org/docs/selenide-reference/ # Allure Selenide reference These are the classes and methods you can use to integrate your Selenide tests with Allure. ## AllureSelenide `io.qameta.allure.selenide.AllureSelenide` A Selenide `LogEventListener` that records Selenide actions as Allure steps and captures screenshots, page source, and browser logs on failure. Register it with Selenide's listener system via `SelenideLogger.addListener()`. Selenide steps are nested under any enclosing `Allure.step()` calls. The reverse also holds: `Allure.step()` calls made while a Selenide event is in progress appear as children of that Selenide step. All configuration methods return `this` for method chaining. ### AllureSelenide() ```java new AllureSelenide() ``` Creates an instance with default configuration: screenshots enabled, page source enabled, Selenide steps included, no browser log types enabled. ### AllureSelenide(AllureLifecycle) ```java new AllureSelenide(AllureLifecycle lifecycle) ``` Creates an instance using the supplied `AllureLifecycle`. This is intended for framework authors and integration testing scenarios where you need to write to a specific lifecycle instance rather than the global default. Most tests should use `new AllureSelenide()` instead. ### screenshots ```java AllureSelenide screenshots(boolean saveScreenshots) ``` Whether to capture and attach a screenshot to the failing step when a Selenide step fails. Defaults to `true`. See [screenshots](/docs/selenide-configuration/#screenshots). ```java new AllureSelenide().screenshots(false) ``` ### savePageSource ```java AllureSelenide savePageSource(boolean savePageHtml) ``` Whether to capture and attach the current page HTML to the failing step when a Selenide step fails. Defaults to `true`. See [savePageSource](/docs/selenide-configuration/#savepagesource). ```java new AllureSelenide().savePageSource(false) ``` ### includeSelenideSteps ```java AllureSelenide includeSelenideSteps(boolean includeSelenideSteps) ``` Whether to record Selenide's built-in element interaction events as Allure steps. Defaults to `true`. See [includeSelenideSteps](/docs/selenide-configuration/#includeselenidesteps). ```java new AllureSelenide().includeSelenideSteps(false) ``` ### enableLogs ```java AllureSelenide enableLogs(LogType logType, Level logLevel) ``` Enable capture of a Selenium WebDriver log type at the specified level. When a Selenide step fails, logs of this type are retrieved and attached to the failing step as `Logs from: ` (for example, `Logs from: browser`). No log types are enabled by default. See [enableLogs](/docs/selenide-configuration/#enablelogs). ```java new AllureSelenide().enableLogs(LogType.BROWSER, Level.SEVERE) ``` ### disableLogs ```java AllureSelenide disableLogs(LogType logType) ``` Remove a previously enabled log type so it is no longer captured on failure. Has no effect if the log type was not enabled. See [disableLogs](/docs/selenide-configuration/#disablelogs). ```java new AllureSelenide().disableLogs(LogType.BROWSER) ``` --- ## LogType `io.qameta.allure.selenide.LogType` An enum wrapping Selenium's `org.openqa.selenium.logging.LogType` constants. Used with `AllureSelenide.enableLogs()` and `AllureSelenide.disableLogs()`. | Value | Selenium log type | Description | | ------------- | ----------------- | ----------------------------- | | `BROWSER` | `browser` | Browser console output | | `CLIENT` | `client` | WebDriver client library logs | | `DRIVER` | `driver` | WebDriver implementation logs | | `PERFORMANCE` | `performance` | Performance timing logs | | `PROFILER` | `profiler` | Profiler logs | | `SERVER` | `server` | Remote WebDriver server logs | Browser support for individual log types varies. Check your browser's WebDriver documentation for what is available. --- ## Selenium BiDi > URL: https://allurereport.org/docs/selenium-bidi/ # Getting started with Allure Selenium BiDi [![Allure Selenium BiDi latest version](https://img.shields.io/maven-central/v/io.qameta.allure/allure-selenium-bidi?style=flat "allure-selenium-bidi")](https://mvnrepository.com/artifact/io.qameta.allure/allure-selenium-bidi) Capture browser log events and network events from [Selenium WebDriver 4](https://www.selenium.dev/) tests using the BiDi protocol, and attach them as diagnostics to your [Allure Report](https://allurereport.org/) test results. The integration targets Selenium WebDriver 4 with BiDi support and is validated against Selenium Java 4.44. Allure Selenium BiDi is compatible with any Allure Java framework integration, including [Allure JUnit Jupiter](/docs/junit-jupiter/) and [Allure TestNG](/docs/testng/). Requires Java 17 or newer. To enable the integration in your project: 1. Make sure you have an Allure integration set up for the test framework you use. See the instructions in the integration's documentation in [Frameworks](/docs/frameworks/). 2. Add `allure-selenium-bidi` to your project dependencies. **Maven:** ```xml VERSION io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-selenium-bidi test ``` **Gradle (Kotlin):** ```kts val allureVersion = "VERSION" dependencies { testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) testImplementation("io.qameta.allure:allure-selenium-bidi") } ``` **Gradle (Groovy):** ```groovy def allureVersion = "VERSION" dependencies { testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") testImplementation "io.qameta.allure:allure-selenium-bidi" } ``` 3. Enable BiDi on your WebDriver options. ```java ChromeOptions options = new ChromeOptions().enableBiDi(); ``` BiDi must be enabled at the driver level — without it, the integration silently produces no attachments. Check your browser's Selenium documentation for BiDi support. 4. Decorate your WebDriver instance with `AllureWebDriverBiDi`. ```java import io.qameta.allure.seleniumbidi.AllureWebDriverBiDi; AllureWebDriverBiDi bidi = new AllureWebDriverBiDi(); WebDriver driver = bidi.decorate(new ChromeDriver(options)); ``` Use the decorated driver for all test interactions. When `driver.quit()` is called, the collected browser log and network events are flushed as JSON attachments to the active test result. Info: Events are only captured and attached while there is an active Allure test context. Browser activity that happens outside of a test — for example, in setup code that runs before any test starts — is not captured. Similarly, if `driver.quit()` is called after the test has ended (for example, in teardown code that runs after the test result is finalized), the collected events are not attached. 5. Run your tests and generate a test report the same way as you would normally. ## Report output Each test result receives up to two JSON attachments: - **WebDriver BiDi logs** — browser log events including console output (`console.log`, `console.warn`, `console.error`, etc.), uncaught JavaScript exceptions, and other generic browser log entries. - **WebDriver BiDi network** — network events: `beforeRequestSent`, `responseStarted`, `responseCompleted`, and `fetchError`. An attachment is omitted when no events of that type were received during the test and none were dropped. Sensitive header values are [redacted by default](/docs/selenium-bidi-configuration/#redacting-headers). See [configuration](/docs/selenium-bidi-configuration/) to control which event types are captured, set per-test entry limits, or customize header redaction. See [reference](/docs/selenium-bidi-reference/) for the full attachment schema. ## Closing the listener manually Attachments are flushed when `driver.quit()` is called on the decorated driver. To flush earlier — for example, before teardown logic runs — call `bidi.close()` explicitly: ```java bidi.close(); // flush buffered events as attachments now driver.quit(); // then quit the browser — no second flush occurs ``` Info: `close()` has the same context requirement as `driver.quit()`: events are only attached if an active Allure test context exists when `close()` is called. If you call it after the test result has been finalized, the collected events are discarded. After `close()`, the subsequent `driver.quit()` does not produce a second flush — the session has already been removed. --- ## Selenium BiDi configuration > URL: https://allurereport.org/docs/selenium-bidi-configuration/ # Allure Selenium BiDi configuration This page describes the configuration options that affect the behavior of [Allure Selenium BiDi](/docs/selenium-bidi/). All options are set via the fluent API on `AllureWebDriverBiDi` before decorating the driver: ```java AllureWebDriverBiDi bidi = new AllureWebDriverBiDi() .logs(true) .network(true) .maxLogEntries(500) .maxNetworkEvents(1000) .redactHeaders("X-My-Secret"); WebDriver driver = bidi.decorate(new ChromeDriver(options)); ``` ## logs ```java new AllureWebDriverBiDi().logs(boolean) ``` Whether to capture browser log events and produce a "WebDriver BiDi logs" attachment. Defaults to `true`. Log events include browser console output (`console.log`, `console.warn`, `console.error`, etc.), uncaught JavaScript exceptions, and other generic browser log entries. ```java new AllureWebDriverBiDi().logs(false) ``` ## network ```java new AllureWebDriverBiDi().network(boolean) ``` Whether to capture network events and produce a "WebDriver BiDi network" attachment. Defaults to `true`. Network events include `beforeRequestSent`, `responseStarted`, `responseCompleted`, and `fetchError`. ```java new AllureWebDriverBiDi().network(false) ``` ## maxLogEntries ```java new AllureWebDriverBiDi().maxLogEntries(int) ``` Maximum number of log entries to retain per test. Defaults to `1000`. When the limit is reached, additional entries are discarded. The `dropped` field in the attachment reports how many entries were dropped. Must be greater than or equal to `0`. ```java new AllureWebDriverBiDi().maxLogEntries(200) ``` ## maxNetworkEvents ```java new AllureWebDriverBiDi().maxNetworkEvents(int) ``` Maximum number of network events to retain per test. Defaults to `2000`. When the limit is reached, additional events are discarded. The `dropped` field in the attachment reports how many events were dropped. Must be greater than or equal to `0`. ```java new AllureWebDriverBiDi().maxNetworkEvents(500) ``` ## Redacting headers ```java new AllureWebDriverBiDi().redactHeaders(String...) ``` Adds header names to the redaction list. Redacted header values are replaced with `[REDACTED]` in the "WebDriver BiDi network" attachment. Redaction applies to both request and response headers. Header name matching is case-insensitive. The following headers are redacted by default: | Header | | --------------------- | | `authorization` | | `proxy-authorization` | | `cookie` | | `set-cookie` | | `x-api-key` | Calling `redactHeaders()` adds names to the default list. Each call replaces the previous call's additions, so pass all custom header names in a single call. ```java new AllureWebDriverBiDi() .redactHeaders("X-Auth-Token", "X-Internal-Secret") ``` --- ## Selenium BiDi reference > URL: https://allurereport.org/docs/selenium-bidi-reference/ # Allure Selenium BiDi reference ## AllureWebDriverBiDi `io.qameta.allure.seleniumbidi.AllureWebDriverBiDi` Captures browser log events and network events via the BiDi protocol and attaches them as JSON to the active Allure test result. All configuration methods return `this` for method chaining. Configure the instance before calling `decorate()`. ### AllureWebDriverBiDi() ```java new AllureWebDriverBiDi() ``` Creates an instance with default configuration: log capture enabled, network capture enabled, maximum 1000 log entries, maximum 2000 network events, default header redaction list. ### decorate ```java T decorate(T driver) ``` Decorates the supplied `WebDriver` with BiDi event capture and returns the decorated instance. Use the returned driver for all test interactions. Events are captured while an Allure test context is active. If the driver does not support BiDi — for example, because BiDi was not enabled on the driver options — the decoration still succeeds but no events are captured and no attachments are produced. When `driver.quit()` is called on the decorated driver, the collected events are flushed as attachments to the active test result. ### logs ```java AllureWebDriverBiDi logs(boolean enabled) ``` Whether to capture browser log events. Defaults to `true`. See [logs](/docs/selenium-bidi-configuration/#logs). ### network ```java AllureWebDriverBiDi network(boolean enabled) ``` Whether to capture network events. Defaults to `true`. See [network](/docs/selenium-bidi-configuration/#network). ### maxLogEntries ```java AllureWebDriverBiDi maxLogEntries(int maxLogEntries) ``` Maximum number of log entries to retain per test. Defaults to `1000`. Must be >= 0. See [maxLogEntries](/docs/selenium-bidi-configuration/#maxlogentries). ### maxNetworkEvents ```java AllureWebDriverBiDi maxNetworkEvents(int maxNetworkEvents) ``` Maximum number of network events to retain per test. Defaults to `2000`. Must be >= 0. See [maxNetworkEvents](/docs/selenium-bidi-configuration/#maxnetworkevents). ### redactHeaders ```java AllureWebDriverBiDi redactHeaders(String... headerNames) ``` Adds header names to the redaction list, extending the defaults. Each call replaces the previous call's additions — pass all custom header names in a single call. See [Redacting headers](/docs/selenium-bidi-configuration/#redacting-headers). ### close ```java void close() ``` Flushes all buffered events as attachments to the active test result and closes the BiDi listeners. Normally this happens automatically when `driver.quit()` is called. Call `close()` explicitly when you need to flush before quit. --- ## Attachment schemas Both attachments use content type `application/json`. ### Common envelope Both the "WebDriver BiDi logs" and "WebDriver BiDi network" attachments share the same top-level structure: | Field | Type | Description | | --------- | ------- | --------------------------------------------------------------- | | `entries` | array | Collected log entries or network events. | | `dropped` | integer | Number of events discarded because the entry limit was reached. | ### "WebDriver BiDi logs" entries Each element of `entries` in the log attachment represents one browser log event. Fields present on all log entries: | Field | Type | Description | | ------------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `event` | string | Event kind: `"console"`, `"javascript"`, or `"generic"`. | | `timestamp` | number | Event timestamp in milliseconds. | | `level` | string | Log level reported by the browser (e.g., `"info"`, `"warn"`, `"error"`). Omitted when not provided. | | `text` | string | Log message text. Omitted when not provided. | | `type` | string | The log entry type string as reported by the browser. For `"console"` and `"javascript"` entries this equals the `event` value. For `"generic"` entries it carries the browser's own label, which may differ from `"generic"`. Omitted when not provided. | | `source` | object | Browsing context source. Omitted when not provided. See [Source](#source). | | `stackTrace` | object | Call stack at the point the event was emitted. Omitted when not provided. See [StackTrace](#stacktrace). | Additional fields on `"console"` entries: | Field | Type | Description | | -------- | ------ | --------------------------------------------------------------------------------------------------------------------- | | `method` | string | Console method name (e.g., `"log"`, `"warn"`, `"error"`). Omitted when not provided. | | `args` | array | Arguments passed to the console call. Each element has the shape `{ type, value?, handle?, internalId?, sharedId? }`. | #### Source | Field | Type | Description | | ------------------- | ------ | ------------------------------------------------------- | | `realm` | string | Realm identifier. Omitted when not provided. | | `browsingContextId` | string | Browsing context identifier. Omitted when not provided. | #### StackTrace | Field | Type | Description | | ------------ | ----- | ----------------------------------------------------------------------------------------------- | | `callFrames` | array | List of stack frames. Each frame has `url?`, `functionName?`, `lineNumber`, and `columnNumber`. | ### "WebDriver BiDi network" entries Each element of `entries` in the network attachment represents one network event. Fields present on all network entries: | Field | Type | Description | | ------------------- | ---------------- | ------------------------------------------------------------------------------------------------- | | `event` | string | Event kind: `"beforeRequestSent"`, `"responseStarted"`, `"responseCompleted"`, or `"fetchError"`. | | `timestamp` | number | Event timestamp in milliseconds. | | `blocked` | boolean | Whether the request is currently blocked. | | `redirectCount` | number | Number of redirects that have occurred. | | `intercepts` | array of strings | Active intercept identifiers. | | `browsingContextId` | string | Browsing context identifier. Omitted when not provided. | | `navigationId` | string | Navigation identifier. Omitted when not provided. | | `request` | object | Request data. Omitted when not provided. See [Request](#request). | Additional fields by event type: | Event | Additional field | Description | | ------------------------------------------ | ---------------- | ----------------------------------------------------------- | | `"beforeRequestSent"` | `initiator` | How the request was initiated. See [Initiator](#initiator). | | `"responseStarted"`, `"responseCompleted"` | `response` | Response data. See [Response](#response). | | `"fetchError"` | `errorText` | Error description string. | #### Request | Field | Type | Description | | ------------- | ------ | --------------------------------------------------------------------------- | | `requestId` | string | Unique request identifier. Omitted when not provided. | | `url` | string | Request URL. Omitted when not provided. | | `method` | string | HTTP method (e.g., `"GET"`, `"POST"`). Omitted when not provided. | | `headers` | array | Request headers after redaction. See [Headers](#headers). | | `headersSize` | number | Size of the request headers in bytes. Omitted when not provided. | | `timings` | object | Fetch timing breakdown. Omitted when not provided. See [Timings](#timings). | #### Response | Field | Type | Description | | --------------- | ------- | ------------------------------------------------------------------------- | | `url` | string | Response URL. Omitted when not provided. | | `protocol` | string | Network protocol (e.g., `"http/1.1"`, `"h2"`). Omitted when not provided. | | `status` | number | HTTP status code. | | `statusText` | string | HTTP status text. Omitted when not provided. | | `fromCache` | boolean | Whether the response was served from cache. | | `headers` | array | Response headers after redaction. See [Headers](#headers). | | `mimeType` | string | Response MIME type. Omitted when not provided. | | `bytesReceived` | number | Total bytes received. | | `headersSize` | number | Size of the response headers in bytes. | | `bodySize` | number | Size of the response body in bytes. | | `contentLength` | number | Value of the `Content-Length` header. Omitted when not present. | #### Initiator | Field | Type | Description | | -------------- | ------ | ------------------------------------------------------------------------------------------------------------ | | `type` | string | Initiator type (e.g., `"script"`, `"parser"`). Omitted when not provided. | | `columnNumber` | number | Column number in the source. Omitted when not provided. | | `lineNumber` | number | Line number in the source. Omitted when not provided. | | `requestId` | string | Identifier of the initiating request. Omitted when not provided. | | `stackTrace` | object | Call stack at the point the request was initiated. See [StackTrace](#stacktrace). Omitted when not provided. | #### Headers Headers are represented as an array of objects: ```json [ { "name": "Content-Type", "value": { "type": "string", "value": "application/json" } }, { "name": "Authorization", "value": { "type": "string", "value": "[REDACTED]" } } ] ``` Each header object has `name` (string) and `value` (object with `type` and `value` fields). The `type` field is `"string"` for text values or `"base64"` for binary values. Redacted header values have their `value` field set to `"[REDACTED]"`. #### Timings Timing fields in milliseconds, mirroring the [Resource Timing](https://www.w3.org/TR/resource-timing/) model: `timeOrigin`, `requestTime`, `redirectStart`, `redirectEnd`, `fetchStart`, `dnsStart`, `dnsEnd`, `connectStart`, `connectEnd`, `tlsStart`, `requestStart`, `responseStart`, `responseEnd`. --- ## SpecFlow > URL: https://allurereport.org/docs/specflow/ # Getting started with Allure SpecFlow [![Allure SpecFlow latest release](https://img.shields.io/nuget/v/Allure.SpecFlow?style=flat "allure-specflow")](https://www.nuget.org/packages/Allure.SpecFlow) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [SpecFlow](https://www.nuget.org/packages/SpecFlow/) tests. Warning: SpecFlow has [reached its end of life](https://support-hub.tricentis.com/open?number=NEW0001432&id=post). We recommend you migrate to [Reqnroll](https://reqnroll.net/) and use [Allure Reqnroll](/docs/reqnroll/). Info: Check out the example project at [github.com/allure-examples/allure-specflow](https://github.com/allure-examples/allure-specflow) to see Allure SpecFlow in action. ## Setting up ### 1. Prepare your project 1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Node.js, see the [installation instructions](/docs/v3/install/). 1. Make sure that you have a compatible .NET version. Allure SpecFlow requires a framework that implements [.NET Standard version 2.0](https://learn.microsoft.com/en-us/dotnet/standard/net-standard?tabs=net-standard-2-0#select-net-standard-version). You can find the full list of compatible frameworks [on Allure.SpecFlow's page in NuGet](https://www.nuget.org/packages/Allure.SpecFlow#supportedframeworks-body-tab). 1. Add `Allure.SpecFlow` to your project's dependencies using your IDE or the command line. For example, here is how the dependency can be added to a project with the `dotnet` CLI: ```plain dotnet add ⟨PATH TO PROJECT⟩ package Allure.SpecFlow ``` 1. In the `specflow.json` file, add `Allure.SpecFlowPlugin` as a step assembly. ```json { "stepAssemblies": [{ "assembly": "Allure.SpecFlowPlugin" }] } ``` Make sure that `specflow.json` is copied to the output directory. You can do it in your IDE or manually in your `*.csproj` file by setting the item group's `CopyToOutputDirectory` to either `Always` or `PreserveNewest`. ```xml Always ``` ### 2. Run tests Run your SpecFlow tests the same way as you would run them usually. For example: ```plain dotnet test ``` This will save necessary data into `allure-results` or other directory, according to the [`allure.directory`](/docs/specflow-configuration/#allure-directory) configuration option. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure SpecFlow integration extends the standard reporting features of SpecFlow by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#specify-description-links-and-other-metadata). - **Test Organization**: Structure your tests into [clear hierarchies](#organize-tests) for better readability and organization. - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, you need to indicate to Allure SpecFlow that a certain property needs to be assigned to the test result. Most properties can be assigned via Gherkin tags or via the Runtime API. - **Gherkin tags**: use Gherkin tags to assign various data to a particular `Scenario` or a whole `Feature`. By default, all Gherkin tags will be converted to [test tags](/docs/specflow-reference/#tag), but you can configure conversion to various other metadata via regular expressions in the [configuration file](/docs/specflow-configuration/). Note that due to a limitation in the Gherkin syntax, a tag cannot contain spaces. - **Runtime API**: use Allure's functions to add data to the test result during the execution of its steps. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/specflow-reference/#metadata) for more details. **Gherkin tags:** ```gherkin Feature: Labels @critical @allure.label.owner:JohnDoe @link:https://dev.example.com/ @issue:AUTH-123 @tms:TMS-456 Scenario: Create new label for authorized user This test attempts to create a label with specified title When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.SetDescription("This test attempts to create a label with specified title"); AllureApi.SetSeverity(SeverityLevel.critical); AllureApi.SetOwner("John Doe"); AllureApi.AddLink("Website", "https://dev.example.com/"); AllureApi.AddIssue("AUTH-123"); AllureApi.AddTmsItem("TMS-456"); // ... } } ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure SpecFlow provides the API to assign the relevant fields to tests either by adding attributes or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Gherkin tags:** ```gherkin @allure.label.epic:WebInterface Feature: Labels @allure.label.story:CreateLabels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddEpic("Web interface"); AllureApi.AddFeature("Essential features"); AllureApi.AddStory("Labels"); // ... } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Gherkin tags:** ```gherkin @allure.label.parentSuite:WebInterface @allure.label.suite:EssentialFeatures @allure.label.subSuite:Labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddParentSuite("Web interface"); AllureApi.AddSuite("Essential features"); AllureApi.AddSubSuite("Labels"); // ... } } ``` To specify a test's location in the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy): **Gherkin tags:** ```gherkin @allure.label.package:org.example @allure.label.testClass:TestMyWebsite @allure.label.testMethod:TestLabels() Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddLabel(Label.Package("org.example")); AllureApi.AddLabel(Label.TestClass("TestMyWebsite")); AllureApi.AddLabel(Label.TestMethod("TestLabels()")); // ... } } ``` ### Divide a test into steps Allure SpecFlow provides two ways of [creating steps and sub-steps](/docs/steps/): “lambda steps” and “no-op steps”, see the [reference](/docs/specflow-reference/#test-steps). ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [Then("visit pages")] public void ThenVisitPages() { string[] urls = { "https://domain1.example.com/", "https://domain2.example.com/" }; foreach (string url in urls) { AllureApi.Step($"Test {url}", () => { AllureLifecycle.Instance.UpdateStep(stepResult => { stepResult.parameters.Add( new Parameter { name = "Webpage URL", value = url } ); }); AllureApi.Step("Opening web browser..."); // ... AllureApi.Step($"Visiting {url}..."); // ... AllureApi.Step("Closing web browser..."); // ... }); } } } ``` ### Describe parametrized tests An Allure test report can reflect multiple ways in which you can pass data from your Gherkin file to your C# implementation code, namely: - [scenario outlines](/docs/specflow-reference/#scenario-outlines), - [vertical data tables](/docs/specflow-reference/#vertical-data-tables), - [horizontal data tables](/docs/specflow-reference/#horizontal-data-tables), - [runtime parameters](/docs/specflow-reference/#runtime-parameters). The example below shows a configuration file, a Gherkin file and a C# implementation file of a test. In this example, the four parameters for the “I enter my details” step will be displayed in the test report. ```json { "specflow": { "stepArguments": { "convertToParameters": true, "paramNameRegex": "^Parameter$", "paramValueRegex": "^Value$" } } } ``` ```gherkin Feature: User management Scenario: Registration When I go to the registration form And I enter my details | Parameter | Value | | login | johndoe | | password | qwerty | | name | John Doe | | birthday | 1970-01-01 | Then the profile should be created ``` ```csharp using TechTalk.SpecFlow; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When("I enter my details")] public void WhenIEnterMyDetails(Table table) { // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } } ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure SpecFlow provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/specflow-reference/#attachments). ```csharp using System.IO; using System.Text; using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddAttachment( "data.txt", "text/plain", Encoding.UTF8.GetBytes("This is the file content.") ); AllureApi.AddAttachment( "image1.png", "image/png", File.ReadAllBytes("/path/to/image1.png") ); AllureApi.AddAttachment( "image2.png", "image/png", "/path/to/image2.png" ); } } ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, SpecFlow will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json dotnet test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" dotnet test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and .NET version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/csharp/environment-allure3-light.png, /images/csharp/environment-allure3-dark.png, /images/csharp/environment-allure2-light.png, /images/csharp/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## SpecFlow configuration > URL: https://allurereport.org/docs/specflow-configuration/ # Allure SpecFlow configuration This page describes the configuration options that affect the behavior of [Allure SpecFlow](/docs/specflow/). Warning: SpecFlow has [reached its end of life](https://support-hub.tricentis.com/open?number=NEW0001432&id=post). We recommend you migrate to [Reqnroll](https://reqnroll.net/) and use [Allure Reqnroll](/docs/reqnroll/). By default, Allure SpecFlow tries to read the `allureConfig.json` file in the same directory where the test assembly is located. To make sure the file is present there, edit the file properties in your IDE or manually in your `*.csproj` file and set the corresponding item's `CopyToOutputDirectory` to either `Always` or `PreserveNewest`. For example: ```xml Always ``` Alternatively, use the `ALLURE_CONFIG` environment variable to specify a relative or absolute path of the configuration file. ## allure.directory Path to the directory where Allure SpecFlow will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. Note that the path is treated as relative to the build output directory of the project. When building and running tests from an IDE, this is typically different from the solution folder. Multiple `../` elements may be needed if you want to put the test results into a directory in the solution folder. ```json { "allure": { "directory": "../../../../../allure-results" } } ``` ## allure.links List of patterns that can be used to construct full URLs from short identifiers. Each pattern must be a string containing a placeholder surrounded by curly brackets, for example: `{issue}`. When processing a link, Allure SpecFlow selects the pattern whose placeholder matches the link type (as specified using the [`Link`](/docs/specflow-reference/#link) arguments or the [`specflow.links`](#specflowlinks) patterns). The placeholder is then replaced by the short identifier. For example, with the configuration below, the link `BUG-123` of type `issue` will be translated to `https://issues.example.com/BUG-123`. ```json { "allure": { "links": [ "https://issues.example.com/{issue}", "https://tms.example.com/{tms}", "https://jira.example.com/browse/{jira}" ] } } ``` ## allure.failExceptions A list of exception types representing assertion failures. In case of an exception that matches one of the listed classes, the [Failed](/docs/test-statuses/#failed) status is assigned to the test, step, or fixture. In case of any other exception, the [Broken](/docs/test-statuses/#broken) status is assigned. When looking for matches, Allure SpecFlow uses not only the exception class's own full name but also all its interfaces and base classes. The default list includes the base classes and interfaces that could be thrown by various runners compatible with SpecFlow: `NUnit.Framework.AssertionException`, `Xunit.Sdk.IAssertionException`, `Xunit.Sdk.XunitException`, `Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException`. The setting, if not empty, overrides the default list. With the example configuration below, a test will be considered failed if it throws `Xunit.Sdk.XunitException`, `MyNamespace.MyCustomException`, or anything that derives from them. At the same time, any other exception will make the test broken. ```json { "allure": { "failExceptions": ["Xunit.Sdk.XunitException", "MyNamespace.MyCustomException"] } } ``` ## specflow.stepArguments Configuration options that affect how Allure SpecFlow processes data tables found in Gherkin files. The `convertToParameters` option defines whether Allure SpecFlow will support the [Vertical data tables](/docs/specflow-reference/#vertical-data-tables) and [Horizontal data tables](/docs/specflow-reference/#horizontal-data-tables) approaches. For the vertical data tables, the `paramNameRegex` and `paramValueRegex` regular expressions, if provided, are used to check whether a table should be considered a list of step parameters or not. For example, with the configuration below, a two-columns table with the columns “Parameter” and “Value” will be processed, but a table with any other columns won't, even if there happens to be two of them. ```json { "specflow": { "stepArguments": { "convertToParameters": true, "paramNameRegex": "^Parameter$", "paramValueRegex": "^Value$" } } } ``` ## specflow.grouping.behaviors Regular expressions for Gherkin tags that can be used to set a test's epic and story, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). For example, with the configuration below, a feature with the Gherkin tag `@allure.label.epic:UI` will be added to the “UI” epic. ```json { "specflow": { "grouping": { "behaviors": { "epic": "^allure.label.epic:(.+)$", "story": "^allure.label.story:(.+)$" } } } } ``` ## specflow.grouping.suites Regular expressions for Gherkin tags that can be used to set a test's suites, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). For example, with the configuration below, a feature with the Gherkin tag `@allure.label.suite:TestUI` will be added to the “TestUI” suite. ```json { "specflow": { "grouping": { "suites": { "parentSuite": "^allure.label.parentSuite:(.+)$", "suite": "^allure.label.suite:(.+)$", "subSuite": "^allure.label.subSuite:(.+)$" } } } } ``` ## specflow.grouping.packages Regular expressions for Gherkin tags that can be used to set a test's package, class and method, as part of Allure's [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy). For example, with the configuration below, a feature with the Gherkin tag `@allure.label.package:org.example.testui` will be added to the “org.example.testui” package. ```json { "specflow": { "grouping": { "packages": { "package": "^allure.label.package:(.+)$", "testClass": "^allure.label.class:(.+)$", "testMethod": "^allure.label.method:(.+)$" } } } } ``` ## specflow.labels Regular expressions for Gherkin tags that can be used to add an arbitrary [label](/docs/specflow-reference/#label) for the test, as well as its [owner](/docs/specflow-reference/#owner) and [severity](/docs/specflow-reference/#severity) specifically. The `label` regular expression must contain two capture groups. The two captured texts will be used as the name and the value for the label. The `owner` and `severity` regular expressions must contain one capture group. The captured text will be used as the value for the label. For example, with the configuration below: - the Gherkin tag `@allure.label.language=csharp` will add a label with the name “language” and the value “csharp”, - the Gherkin tag `@allure.label.owner:JohnDoe` will set the test's owner to “JohnDoe”, - the Gherkin tag `@critical` will set the test's severity to “critical”. ```json { "specflow": { "labels": { "label": "^allure.label.(.+):(.+)$" "owner": "^allure.label.owner:(.+)$", "severity": "^(normal|blocker|critical|minor|trivial)$", } } } ``` ## specflow.links Regular expressions for Gherkin tags that can be used to add a [link](/docs/specflow-reference/#link) of one of the standard types: “link”, “issue”, or “tms”. For example, with the configuration below, the Gherkin tag `@issue:BUG-123` will add a link “BUG-123” of type “issue”. Optionally, it can then be processed according to the [`allure.links`](#allurelinks) patterns. ```json { "specflow": { "links": { "link": "^link:(.+)", "issue": "^issue:(.+)", "tms": "^tms:(.+)" } } } ``` --- ## SpecFlow reference > URL: https://allurereport.org/docs/specflow-reference/ # Allure SpecFlow reference These are the attributes and methods that you can use to integrate your SpecFlow tests with Allure Report. Warning: SpecFlow has [reached its end of life](https://support-hub.tricentis.com/open?number=NEW0001432&id=post). We recommend you migrate to [Reqnroll](https://reqnroll.net/) and use [Allure Reqnroll](/docs/reqnroll/). In most cases, you need to indicate to Allure SpecFlow that a certain property needs to be assigned to the test result. Most properties can be assigned via Gherkin tags or via the Runtime API. - **Gherkin tags**: use Gherkin tags to assign various data to a particular `Scenario` or a whole `Feature`. By default, all Gherkin tags will be converted to [test tags](#tag), but you can configure conversion to various other metadata via regular expressions in the [configuration file](/docs/specflow-configuration/). Note that due to a limitation in the Gherkin syntax, a tag cannot contain spaces. When using this approach, the data is guaranteed to be added to the test result regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add data to the test result during the execution of its steps. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `AllureApi.SetTestName(string name)` Set the test's [title](/docs/v3/readability/#title). ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.SetTestName("Open labels page"); // ... } } ``` ### Description - `AllureApi.SetDescription(string description)` Set the test's [description](/docs/v3/readability/#description). Allure SpecFlow uses a scenario's description from the Gherkin file, if present. Alternatively, use Runtime API to set the description dynamically. **Gherkin description:** ```gherkin Feature: Labels Scenario: Create new label for authorized user This test attempts to create a label with specified title When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.SetDescription("This test attempts to create a label with specified title"); // ... } } ``` ### Owner - `AllureApi.SetOwner(string owner)` Set the test's [owner](/docs/v3/readability/#owner). The format for the Gherkin tags is configured via the [`specflow.labels.owner`](/docs/specflow-configuration/#specflow-labels) option. **Gherkin tags:** ```gherkin @allure.label.owner:JohnDoe Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void ThenLabels() { AllureApi.SetOwner("John Doe"); // ... } } ``` ### Tag - `AllureApi.AddTags(params string[] tags)` Set the test's [tags](/docs/v3/readability/#tags). Any Gherkin tag is automatically added to the list of the test's tags, unless it matches one of the regular expressions in the [configuration](/docs/specflow-configuration/). **Gherkin tags:** ```gherkin @UI @Labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddTags("UI", "Labels"); // ... } } ``` ### Severity - `AllureApi.SetSeverity(SeverityLevel severity)` Set the test's [severity](/docs/v3/readability/#severity). The format for the Gherkin tags is configured via the [`specflow.labels.severity`](/docs/specflow-configuration/#specflow-labels) option. **Gherkin tags:** ```gherkin Feature: Labels @critical Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.SetSeverity(SeverityLevel.critical); // ... } } ``` ### Label - `AllureApi.AddLabel(string name, string value)` - `AllureApi.AddLabel(Label label)` - `AllureApi.AddLabels(params Label[] labels)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other APIs. The format for the Gherkin tags is configured via the [`specflow.labels.label`](/docs/specflow-configuration/#specflow-labels) option. **Gherkin tags:** ```gherkin @allure.label.layer:web @allure.label.owner:eroshenkoam @allure.label.page:/{org}/{repo}/labels @allure.label.jira:AE-2 Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddLabel("layer", "web"); AllureApi.AddLabel("owner", "eroshenkoam"); AllureApi.AddLabel("page", "/{org}/{repo}/labels"); AllureApi.AddLabel("jira", "AE-2"); // ... } } ``` ### Allure ID (Allure TestOps) Set the test's [ID](/docs/v3/readability/#id). The format for the Gherkin tags is configured via the [`specflow.labels.label`](/docs/specflow-configuration/#specflow-labels) option. In the tag, use `allure_id` as the label name. **Gherkin tags:** ```gherkin Feature: Labels @allure.label.allure_id:123 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ### Link - `AllureApi.AddLink(string url)` - `AllureApi.AddLink(string name, string url)` - `AllureApi.AddLink(string name, string type, string url)` - `AllureApi.AddLinks(params Link[] links)` - `AllureApi.AddIssue(string url)` - `AllureApi.AddIssue(string name, string url)` - `AllureApi.AddTmsItem(string url)` - `AllureApi.AddTmsItem(string name, string url)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string; defaults to “link”), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`allure.links`](/docs/specflow-configuration/#allure-links) section of the configuration file. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. Additionally, you can use the [`specflow.links`](/docs/specflow-configuration/#specflow-links) section of the configuration file to specify formats for adding “link”, “issue” and “tms” links via Gherkin tags. **Gherkin tags:** ```gherkin @link:https://dev.example.com/ @issue:UI-123 @tms:TMS-456 Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddLink("Website", "https://dev.example.com/"); AllureApi.AddIssue("UI-123"); AllureApi.AddTmsItem("TMS-456"); // ... } } ``` ## Behavior-based hierarchy - `AllureApi.AddEpic(string epic)` - `AllureApi.AddFeature(string feature)` - `AllureApi.AddStory(string story)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). The format for the Gherkin tags is configured via the [`specflow.grouping.behaviors`](/docs/specflow-configuration/#specflow-grouping-behaviors) options. **Gherkin tags:** ```gherkin @allure.label.epic:WebInterface Feature: Labels @allure.label.story:CreateLabels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddEpic("Web interface"); AllureApi.AddFeature("Essential features"); AllureApi.AddStory("Labels"); // ... } } ``` ## Suite-based hierarchy - `AllureApi.AddParentSuite(string parentSuite)` - `AllureApi.AddSuite(string suite)` - `AllureApi.AddSubSuite(string subSuite)` Assign the names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). The format for the Gherkin tags is configured via the [`specflow.grouping.suites`](/docs/specflow-configuration/#specflow-grouping-suites) options. **Gherkin tags:** ```gherkin @allure.label.parentSuite:WebInterface @allure.label.suite:EssentialFeatures @allure.label.subSuite:Labels Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddParentSuite("Web interface"); AllureApi.AddSuite("Essential features"); AllureApi.AddSubSuite("Labels"); // ... } } ``` ## Package-based hierarchy Assign the names of package, test class and test method, as part of Allure's [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy). The format for the Gherkin tags is configured via the [`specflow.grouping.packages`](/docs/specflow-configuration/#specflow-grouping-packages) options. **Gherkin tags:** ```gherkin @allure.label.package:org.example @allure.label.testClass:TestMyWebsite @allure.label.testMethod:TestLabels() Feature: Labels Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` **Runtime API:** ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddLabel(Label.Package("org.example")); AllureApi.AddLabel(Label.TestClass("TestMyWebsite")); AllureApi.AddLabel(Label.TestMethod("TestLabels()")); // ... } } ``` ## Test steps - `AllureApi.Step(string name)` - `AllureApi.Step(string name, Action acti` - `T AllureApi.Step(string name, Func function)` - `async Task AllureApi.Step(string name, Func action)` - `async Task AllureApi.Step(string name, Func> function)` Define a [test sub-steps](/docs/steps/) with the given `name`. There are two ways of defining a sub-step. - **Lambda sub-steps** Write a test sub-step in a lambda function and pass it to `Step()`. If the lambda function returns a value, `Step()` will return it without modification, and it will not affect the report. Use `AllureLifecycle.Instance.UpdateStep()` to add custom parameters to the current sub-step, as shown in the example below. - **No-op sub-steps** If you call `Step()` with a name but without a lambda function, Allure will add to the report a no-op sub-step. This allows for a log-style reporting within a larger step. A no-op sub-step finishes immediately after it started and cannot have any parameters, attachments or sub-steps of its own. In the example below, a new lambda sub-step is created at runtime for each URL from a list. The URL is added to the list of the sub-step's parameters. Within each of the sub-steps, a sequence of three no-op sub-steps occurs. ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [Then("visit pages")] public void ThenVisitPages() { string[] urls = { "https://domain1.example.com/", "https://domain2.example.com/" }; foreach (string url in urls) { AllureApi.Step($"Test {url}", () => { AllureLifecycle.Instance.UpdateStep(stepResult => { stepResult.parameters.Add( new Parameter { name = "Webpage URL", value = url } ); }); AllureApi.Step("Opening web browser..."); // ... AllureApi.Step($"Visiting {url}..."); // ... AllureApi.Step("Closing web browser..."); // ... }); } } } ``` ## Parametrized tests An Allure test report can reflect multiple ways in which you can pass data from your Gherkin file to your C# implementation code, namely: - [scenario outlines](#scenario-outlines), - [vertical data tables](#vertical-data-tables), - [horizontal data tables](#horizontal-data-tables), - [runtime parameters](#runtime-parameters). ### Scenario outlines In Gherkin, a `Scenario Outline` (or a `Scenario Template`) implements the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. A scenario outline must contain an `Examples` table, from which SpecFlow loads sets of parameters, one row after another. Each set of parameters is being placed into the step declarations according to the placeholders, thus generating a new scenario based on the row. SpecFlow then runs each of them independently, as if it was a separate `Scenario`. The data can then be captured by a regular expression and passed as separate arguments to the C# code. Allure SpecFlow automatically recognizes this pattern. No additional configuration is required. The example below shows a Gherkin file and a C# implementation file of a test. In this example, the four parameters for the “I enter my details...” step will be displayed in both instances of the scenario in the test report. ```gherkin Feature: User management Scenario Outline: Registration When I go to the registration form And I enter my details: , , , Then the profile should be created Examples: | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | | janedoe | 123456 | Jane Doe | 1111-11-11 | ``` ```csharp using System; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When(@"I enter my details: (.*), (.*), (.*), (.*)")] public void WhenIEnterMyDetails( string login, string password, string name, DateTime birthday) { // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } } ``` ### Vertical data tables If your Gherkin step declaration contains a data table with **two columns**, Allure SpecFlow can display its content as the step-level parameters. Depending on the configuration, this behavior can be applied either to all tables with two columns or only to those whose headers match a specific regular expression. The functionality is optional. To enable or disable it or modify the regular expressions, edit the [`specflow.stepArguments`](/docs/specflow-configuration/#specflow-steparguments) section of the configuration file. The table data itself is passed to the C# code without modifications. The example below shows a configuration file, a Gherkin file and a C# implementation file of a test. In this example, the four parameters for the “I enter my details” step will be displayed in the test report. ```json { "specflow": { "stepArguments": { "convertToParameters": true, "paramNameRegex": "^Parameter$", "paramValueRegex": "^Value$" } } } ``` ```gherkin Feature: User management Scenario: Registration When I go to the registration form And I enter my details | Parameter | Value | | login | johndoe | | password | qwerty | | name | John Doe | | birthday | 1970-01-01 | Then the profile should be created ``` ```csharp using TechTalk.SpecFlow; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When("I enter my details")] public void WhenIEnterMyDetails(Table table) { // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } } ``` ### Horizontal data tables If your Gherkin step declaration contains a data table with **one row**, Allure SpecFlow can display its content as the step-level parameters. The functionality is optional. To enable or disable it, edit the [`specflow.stepArguments`](/docs/specflow-configuration/#specflow-steparguments) section of the configuration file. The table data itself is passed to the C# code without modifications. Warning: Due to an implementation detail, the Allure SpecFlow's horizontal data table processor only works correctly if you set all the configuration options for the [vertical data table](#vertical-data-tables) processor. Make sure to provide some regular expressions for both `paramNameRegex` and `paramValueRegex`, even if you do not plan to use vertical tables in your project. Also note that currently, Allure SpecFlow does not support horizontal tables with two columns only. Instead, such a table will be interpreted as a vertical data table. The example below shows a configuration file, a Gherkin file and a C# implementation file of a test. In this example, the four parameters for the “I enter my details” step will be displayed in the test report. ```json { "specflow": { "stepArguments": { "convertToParameters": true, "paramNameRegex": "^Parameter$", "paramValueRegex": "^Value$" } } } ``` ```gherkin Feature: User management Scenario: Registration When I go to the registration form And I enter my details | login | password | name | birthday | | johndoe | qwerty | John Doe | 1970-01-01 | Then the profile should be created ``` ```csharp using TechTalk.SpecFlow; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When("I enter my details")] public void WhenIEnterMyDetails(Table table) { // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } } ``` ### Runtime parameters - `AllureApi.AddTestParameter(string name, object? value)` - `AllureApi.AddTestParameter(string name, object? value, ParameterMode mode)` - `AllureApi.AddTestParameter(string name, object? value, bool excluded)` - `AllureApi.AddTestParameter(string name, object? value, ParameterMode mode, bool excluded)` - `AllureApi.AddTestParameter(Parameter parameter)` The `AddTestParameter()` function can be used at any point, even if the Gherkin file does not specify actual parameters or data tables to any step. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. This argument is only used by Allure TestOps. The `mode` argument affects how the parameter will be displayed in the report. Available options are defined in the `ParameterMode` enumeration: - `ParameterMode.Default` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `ParameterMode.Masked` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `ParameterMode.Hidden` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. The example below shows a Gherkin file and a C# implementation file of a test. As far as SpecFlow is concerned, the “I enter my details” in the example does not have any data passed to it, however, Allure will have the four parameters displayed on the test level. ```gherkin Feature: User management Scenario: Registration When I go to the registration form And I enter my details Then the profile should be created ``` ```csharp using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When("I enter my details")] public void WhenIEnterMyDetails() { AllureApi.AddTestParameter("login", "johndoe"); AllureApi.AddTestParameter("password", "qwerty"); AllureApi.AddTestParameter("name", "John Doe"); AllureApi.AddTestParameter("birthday", "1970-01-01"); // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } } ``` ## Attachments - `AllureApi.AddAttachment(string name, string type, string path)` - `AllureApi.AddAttachment(string name, string type, byte[] content, string fileExtension = "")` - `AllureApi.AddAttachment(string path, string? name = null)` - `AllureApi.AddScreenDiff(string expectedPng, string actualPng, string diffPng)` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Tip: You can use data produced by any function, not necessarily read from an actual file. To create an attachment using the Runtime API, call `AddAttachment()` at any point during your test. Pass either the `content` or the `path` from which the data will be read. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension as `fileExtension`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. ```csharp using System.IO; using System.Text; using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddAttachment( "data.txt", "text/plain", Encoding.UTF8.GetBytes("This is the file content.") ); AllureApi.AddAttachment( "image1.png", "image/png", File.ReadAllBytes("/path/to/image1.png") ); AllureApi.AddAttachment( "image2.png", "image/png", "/path/to/image2.png" ); } } ``` Allure SpecFlow also provides a dedicated `AddScreenDiff()` function for attaching [visual comparisons](/docs/attachments/#attaching-visual-comparisons). --- ## Spock > URL: https://allurereport.org/docs/spock/ # Getting started with Allure Spock [![Allure Spock latest version](https://img.shields.io/maven-central/v/io.qameta.allure/allure-spock2?style=flat "allure-spock2")](https://mvnrepository.com/artifact/io.qameta.allure/allure-spock2) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/) and your [Spock](https://spockframework.org/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Aspock) to see Allure Spock in action. ## Setting up ### 1. Prepare your project 1. Open a terminal and make sure that Java version 17 or higher is available in the environment. ```bash java --version ``` 1. In the `dependencies` section of your `build.gradle.kts` file, add the Allure Spock dependency specification **before** the Spock framework dependencies. Warning: Due to an implementation detail, some of the Allure Spock functionality **will not work** unless the dependencies are listed in the correct order. For technical details, see the [discussion on GitHub](https://github.com/allure-framework/allure-java/issues/928#issuecomment-1594596412). ```kts dependencies { // Allure Spock adapter testImplementation(platform("io.qameta.allure:allure-bom:2.35.3")) testImplementation("io.qameta.allure:allure-spock2") testImplementation("io.qameta.allure:allure-junit-platform") // Spock framework testImplementation(platform("org.spockframework:spock-bom:2.4-groovy-5.0")) testImplementation("org.spockframework:spock-core") // ... } ``` 1. To use `@Step` and `@Attachment` annotations, also configure the AspectJ weaver agent: ```kts val aspectJVersion = "1.9.25" val agent: Configuration by configurations.creating { isCanBeConsumed = true isCanBeResolved = true } dependencies { agent("org.aspectj:aspectjweaver:$aspectJVersion") } tasks.test { jvmArgs("-javaagent:${agent.singleFile}") } ``` Skip this step if you only use `Allure.step()` (no-op and lambda steps), which work without AspectJ. 1. Update the project and fetch all dependencies: **MacOS/Linux:** ```bash ./gradlew build ``` **Windows:** ```bash gradlew build ``` ### 2. Run tests Run your Spock tests the same way as you would run them usually. For example: **MacOS/Linux:** ```bash ./gradlew test ``` **Windows:** ```bash gradlew test ``` This will save necessary data into `build/allure-results` or other directory, according to the settings, see `AllureExtension`. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Spock integration not only collects the data provided by Spock's standard features, but also provides additional features for writing even better tests. This section lists the most notable ways to improve your tests, using both Spock's and Allure Spock's features. With Allure Spock, you can: - provide [description, links and other metadata](#specify-description-links-and-other-metadata), - [organize tests](#organize-tests) into hierarchies, - divide the test into smaller, easier-to-read [test steps](#divide-a-test-into-steps), - describe parameters used when running [parametrized tests](#describe-parametrized-tests), - make the test save [screenshots and other files](#attach-screenshots-and-other-files) during execution, - [select which tests to run](#select-tests-via-a-test-plan-file) via a test plan file, - provide arbitrary [environment information](#environment-information) for the whole test report. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/spock-reference/#metadata) for more details. For each of the metadata fields, there are two ways to assign it: via an annotation before a test method or via a method call inside a test method's body. The second way is called “dynamic”, because it allows you to construct strings and other values at runtime before passing to the methods. Note, however, that it is highly recommended to assign all metadata as early as possible. Otherwise, there is a risk of the test failing before having all metadata set, which is bad for the test report's readability. **Annotations API:** ```groovy import io.qameta.allure.Description import io.qameta.allure.Issue import io.qameta.allure.Link import io.qameta.allure.Owner import io.qameta.allure.Severity import io.qameta.allure.TmsLink import spock.lang.Specification import static io.qameta.allure.SeverityLevel.* class TestMyWebsite extends Specification { @Description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") @Severity(CRITICAL) @Owner("John Doe") @Link(name = "Website", url = "https://dev.example.com/") @Issue("AUTH-123") @TmsLink("TMS-456") def "Test Authentication"() { expect: true } } ``` **Runtime API:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") Allure.label("severity", "critical") Allure.label("owner", "John Doe") Allure.link("Website", "https://dev.example.com/") Allure.issue("AUTH-123", "https://example.com/issues/AUTH-123") Allure.tms("TMS-456", "https://example.com/tms/TMS-456") expect: true } } ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure Spock provides functions to assign the relevant fields to tests either by adding annotations or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Annotations API:** ```groovy import io.qameta.allure.Epic import io.qameta.allure.Feature import io.qameta.allure.Story import spock.lang.Specification class TestMyWebsite extends Specification { @Epic("Web interface") @Feature("Essential features") @Story("Authentication") def "Test Authentication"() { expect: true } } ``` **Runtime API:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.epic("Web interface") Allure.feature("Essential features") Allure.story("Authentication") expect: true } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.label("parentSuite" "Tests for web interface") Allure.suite("Tests for essential features") Allure.label("subSuite", "Tests for authentication") expect: true } } ``` A test's location in the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy) is defined by the fully qualified names of the classes they are declared in, with common prefixes shown as parent packages. ### Divide a test into steps Allure Spock provides two ways of [creating steps and sub-steps](/docs/steps/): “annotated steps” and “no-op steps”, see the [reference](/docs/spock-reference/#test-steps). **Annotated steps:** ```groovy import io.qameta.allure.Step import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { expect: true steps.step1() steps.step2() } private class steps { @Step("Step 1") static def step1() { expect: true } @Step("Step 2") static def step2() { expect: true } } } ``` **No-op steps:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.step("Step 1") expect: true Allure.step("Step 2") expect: true } } ``` ### Describe parametrized tests Allure Spock supports the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. The easiest way to write a parametrized test is to define a data table, see [Data Driven Testing](https://spockframework.org/spock/docs/2.3/data_driven_testing.html) in the Spock documentation. This will cause Spock to run the same test multiple times, and Allure will automatically detect each set of parameters and display them in the generated report. Additionally, you can call the Allure Spock's [`parameter()`](/docs/spock/#parametrized-tests) method manually to define pseudo-parameters in any function. ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { expect: true where: login | _ "johndoe" | _ "johndoe@example.com" | _ } def "Test Authentication With Empty Login"() { Allure.parameter("login", "") expect: true } } ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure Spock provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/spock-reference/#attachments). ```groovy import io.qameta.allure.Allure import spock.lang.Specification import java.nio.file.Files import java.nio.file.Paths class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.attachment("data.txt", "This is the file content.") Allure.attachment( "img.png", Files.newInputStream(Paths.get("/path/img.png")) ) expect: true } } ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Spock will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json ./gradlew test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" gradlew test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and Java version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/java/environment-allure3-light.png, /images/java/environment-allure3-dark.png, /images/java/environment-allure2-light.png, /images/java/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## Spock configuration > URL: https://allurereport.org/docs/spock-configuration/ # Allure Spock configuration This page describes the configuration options that affect the behavior of [Allure Spock](/docs/spock/). To set the configuration options, create a `allure.properties` file in your project's `src/test/resources` directory. Allure Spock loads the configuration into the JVM system properties for the test runtime. ## allure.label.⟨NAME⟩ ```properties allure.label.epic = Authentication allure.label.severity = blocker ``` Assign an arbitrary [label](/docs/spock-reference/#label) to all the tests in the current test run. ## allure.link.⟨TYPE⟩.pattern ```properties allure.link.issue.pattern = https://jira.example.org/browse/{} ``` Define a **link pattern** — a template that can be used to construct full URLs from short identifiers, see the [reference](/docs/spock-reference/#link). The value must be a string pattern containing `{}` at the position where the identifier should be placed. The pattern will be applied when processing any link of the given `TYPE`. You can use the option multiple times, thus defining multiple link patterns for different link types. ## allure.results.directory ```properties allure.results.directory = build/allure-results ``` Path to the directory where Allure Spock will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. --- ## Spock reference > URL: https://allurereport.org/docs/spock-reference/ # Allure Spock reference These are the functions that you can use to integrate your Spock tests with Allure. Allure Spock provides more than one way to use some features. In most cases, one way involves using an annotation on the test method, while the other involves a method call inside the test method body. The latter style is called “dynamic” and allows to construct values dynamically. ## Metadata ### Title - `Allure.getLifecycle().updateTest(Consumer update)` Set the test's [title](/docs/v3/readability/#title). Info: In Allure Java 2.x, the `AllureLifecycle` method for updating the test result is called `updateTestCase()`. ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.getLifecycle().updateTest { result -> result.setName("Test Authentication!") } // ... } } ``` ### Description - `@Description(String value)` - `Allure.description(String description)` - `Allure.descriptionHtml(String description)` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. **Annotations API:** ```groovy import io.qameta.allure.Description import spock.lang.Specification class TestMyWebsite extends Specification { @Description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") def "Test Authentication"() { // ... } } ``` **Runtime API:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") // ... } } ``` ### Owner - `@Owner(String value)` Set the test's [owner](/docs/v3/readability/#owner). Allure Spock does not provide a dedicated method for setting the owner dynamically, but you can use [`Allure.label()`](#label) with the appropriate label name instead. **Annotations API:** ```groovy import io.qameta.allure.Owner import spock.lang.Specification class TestMyWebsite extends Specification { @Owner("John Doe") def "Test Authentication"() { // ... } } ``` **Runtime API:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.label("owner", "John Doe") // ... } } ``` ### Lead - `@Lead(String value)` Set the test's lead — the person responsible for the test's maintenance. **Annotations API:** ```groovy import io.qameta.allure.Lead import spock.lang.Specification class TestMyWebsite extends Specification { @Lead("Jane Doe") def "Test Authentication"() { // ... } } ``` **Runtime API:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.label("lead", "Jane Doe") // ... } } ``` ### Tag Set the test's [tags](/docs/v3/readability/#tags). You can either use the Spock's [`@Tag`](https://spockframework.org/spock/docs/2.3/extensions.html#test-tag-extension) annotation or call the [`label()`](#label) method for updating the values dynamically. **Annotations API:** ```groovy import spock.lang.Specification import spock.lang.Tag class TestMyWebsite extends Specification { @Tag("auth") @Tag("security") def "Test Authentication"() { // ... } } ``` **Runtime API:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.label("tag", "auth") Allure.label("tag", "security") // ... } } ``` ### Severity Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. Allure Spock does not provide a dedicated method for setting the severity level dynamically, but you can use [`Allure.label()`](#label) with the appropriate label name instead. ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.label("severity", "critical") // ... } } ``` ### Label - `@LabelAnnotation(String name, String value=DEFAULT_VALUE)` - `Allure.label(String name, String value)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. To set a label dynamically, just call the `label()` function with a name and a value for a label. You can do it multiple times to create an array of values under that name. Another way of setting a label is to create a custom class using `@LabelAnnotation` and annotate the tests with the new annotation, see the example below. Please be careful to copy `@Retention`, `@Target` and the other necessary annotations as this is shown in the example, otherwise you custom annotation may not work properly. **Annotations API:** ```groovy import io.qameta.allure.LabelAnnotation import spock.lang.Specification import java.lang.annotation.Documented import java.lang.annotation.ElementType import java.lang.annotation.Inherited import java.lang.annotation.Retention import java.lang.annotation.RetentionPolicy import java.lang.annotation.Target @Documented @Inherited @Retention(RetentionPolicy.RUNTIME) @Target([ElementType.METHOD, ElementType.TYPE]) @LabelAnnotation(name = "CustomLabelName") @interface MyLabel { String value(); } class TestMyWebsite extends Specification { @MyLabel("custom label value") def "Test Authentication"() { // ... } } ``` **Runtime API:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.label("language", "groovy") Allure.label("framework", "spock") // ... } } ``` ### ID - `@AllureId(String value)` Set the test's [ID](/docs/v3/readability/#id). Allure Spock does not provide a dedicated method for setting the ID dynamically, but you can use [`Allure.label()`](#label) with the appropriate label name instead. **Annotations API:** ```groovy import io.qameta.allure.AllureId import spock.lang.Specification class TestMyWebsite extends Specification { @AllureId("123") def "Test Authentication"() { // ... } } ``` ### Link - `@Link(String value="", name="", String url="", String type=CUSTOM_LINK_TYPE)` - `@Links(Link[] value)` - `Allure.link(String url)` - `Allure.link(String name, String url)` - `Allure.link(String name, String type, String url)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`allure.link.*.pattern`](/docs/spock-configuration/#allure-link-⟨type⟩-pattern) configuration option. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: [`issue`](#issue) and [`tms`](#tms). **Annotations API:** ```groovy import io.qameta.allure.Issue import io.qameta.allure.Link import io.qameta.allure.TmsLink import spock.lang.Specification class TestMyWebsite extends Specification { @Link(name = "Website", url = "https://dev.example.com/") @Issue("AUTH-123") @TmsLink("TMS-456") def "Test Authentication"() { // ... } } ``` **Runtime API:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.link("Website", "https://dev.example.com/") Allure.issue("AUTH-123", "https://example.com/issues/AUTH-123") Allure.tms("TMS-456", "https://example.com/tms/TMS-456") // ... } } ``` ### Issue - `@Issue(String value="")` - `@Issues(Issue[] value)` - `Allure.issue(String name, String url)` Add a link to an issue in a bug tracker related to the test. This is a shorthand for [`@Link()`](#link) with argument `type="issue"`. ### TMS - `@TmsLink(String value="")` - `@TmsLinks(TmsLink[] value)` - `Allure.tms(String name, String url)` Add a link to an issue in a test management system (TMS) related to the test. This is a shorthand for [`@Link()`](#link) with argument `type="tms"`. ## Behavior-based hierarchy - `@Epic(String value="")` - `@Epics(Epic[] value)` - `@Feature(String value="")` - `@Features(Feature[] value)` - `@Story(String value="")` - `@Stories(Story[] value)` - `Allure.epic(String value)` - `Allure.feature(String value)` - `Allure.story(String value)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). ## Suite-based hierarchy - `Allure.suite(String value)` Assign the name of suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). To assign names of parent suite or sub-suite, use [`label()`](#label) with the corresponding label names. ## Test steps - `@Step(String value=””)` - `Allure.step(String name)` - `Allure.step(String name, Status status)` Define a [test step](/docs/steps/) with the given `name`. There are two ways of defining a step. - “An annotated step”: define a method containing a test step and decorate it with `@Step()`. - “A no-op step”: just pass a `name` and an optional `status` to `Allure.step()` to immediately add a corresponding entry to the results as a sub-step of the current step. Info: `@Step` requires the AspectJ weaver agent to be active at runtime. Add `agent(“org.aspectj:aspectjweaver:1.9.25”)` to your dependencies and pass `-javaagent:${agent.singleFile}` to `jvmArgs` in your test task (the same setup shown in the [overview](/docs/spock/)). No-op steps via `Allure.step()` do not require AspectJ. **Annotated steps:** ```groovy import io.qameta.allure.Step import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { // ... steps.step1() steps.step2() } private class steps { @Step("Step 1") static def step1() { // ... } @Step("Step 2") static def step2() { // ... } } } ``` **No-op steps:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.step("Step 1") // ... Allure.step("Step 2") // ... } } ``` ## Parametrized tests - `Allure.parameter(String name, T value)` - `Allure.parameter(String name, T value, Boolean excluded)` - `Allure.parameter(String name, T value, Parameter.Mode mode)` - `Allure.parameter(String name, T value, Boolean excluded, Parameter.Mode mode)` Specify a `name` and `value` of a parameter that was used during this test. This can be used for adding the parameters even to those function that do not use the `@ParameterizedTest()` decorator. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for more details. Note that using this method is not necessary in most cases, since Allure Spock automatically detects the parameters specified via Spock's [data tables](https://spockframework.org/spock/docs/2.3/data_driven_testing.html#data-tables). Warning: Parameters added via `Allure.parameter()` are part of the test's identity, together with the parameters provided by the test framework. If a parameter's value changes between runs (for example, a timestamp or a random ID), the test will be treated as a new test on every run for the purposes of [History and retries](/docs/history-and-retries/). To avoid this, mark such parameters as excluded, see below. In Allure Java 2.x, parameters added via `Allure.parameter()` did not affect the test's identity. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous ones in the history. In Allure Java 2.x, this argument is only used by Allure TestOps. The `mode` argument affects how the parameter will be displayed in the report. Available options are defined in the `Parameter.Mode` enumeration: - `Parameter.Mode.DEFAULT` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `Parameter.Mode.MASKED` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `Parameter.Mode.HIDDEN` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { // ... where: login | _ "johndoe" | _ "johndoe@example.com" | _ } def "Test Authentication With Empty Login"() { Allure.parameter("login", "") // ... } } ``` ## Attachments - `@Attachment(String value="", String type="", String fileExtension="")` - `Allure.attachment(String name, String content)` - `Allure.attachment(String name, String type, String content)` - `Allure.attachment(String name, String type, String content, AttachmentOptions options)` - `Allure.attachment(String name, InputStream content)` - `Allure.attachment(String name, String type, InputStream content, AttachmentOptions options)` Add `content` as an [attachment](/docs/attachments/) to the test result under the given `name` (defaults to a unique pseudo-random string). Info: In Allure Java 2.x, the same functionality is provided by the `Allure.addAttachment()` methods, which accept the file extension as a string. They were removed in Allure Java 3.0. Tip: You can use data produced by any function, not necessarily read from an actual file. To create an attachment using the Annotations API, define a method that returns some data and annotate it with `@Attachment`. Call the method at any point during your test. To create an attachment using the Runtime API, just call `attachment()` at any point during your test. Pass the data as the `content` argument. The data will be processed as following: - If the data type is `byte[]`, Allure will use it without modification. - If the data type is `String`, Allure will convert it to `byte[]`. - If the data is of any other type, Allure will call the `toString()` method and then convert the string to `byte[]`. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension: as `fileExtension` in the `@Attachment` annotation, or via `AttachmentOptions.withFileExtension(".txt")` as the `options` argument of `attachment()`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. **Annotations API:** ```groovy import io.qameta.allure.Attachment import spock.lang.Specification import java.nio.file.Files import java.nio.file.Paths class TestMyWebsite extends Specification { def "Test Authentication"() { // ... attachDataTXT() attachScreenshotPNG() } @Attachment(value = "data", type = "text/plain", fileExtension = ".txt") private String attachDataTXT() { return "This is the file content." } @Attachment(value = "screenshot", type = "image/png", fileExtension = ".png") private byte[] attachScreenshotPNG() throws IOException { return Files.readAllBytes(Paths.get("/path/to/image.png")) } } ``` **Runtime API:** ```groovy import io.qameta.allure.Allure import spock.lang.Specification class TestMyWebsite extends Specification { def "Test Authentication"() { Allure.attachment("data.txt", "This is the file content.") new File("/path/to/image.png").withInputStream { is -> Allure.attachment("image.png", is) } expect: true } } ``` Info: Additionally, Allure provides a way to spawn threads that will add attachments to the test results asynchronously without blocking the test runner. While not recommended for the majority of cases, this approach can improve tests that otherwise have to wait for processing very large files, such as videos larger than 1 GB. To do so, pass a `CompletionStage` that produces the attachment's content to `Allure.attachmentAsync(String name, String type, CompletionStage body)`. In Allure Java 2.x, use `Allure.addByteAttachmentAsync()` or `Allure.addStreamAttachmentAsync()` instead. ## Results history ### Flaky - `@Flaky` Indicate that the test is known to be unstable and can may not succeed every time. See [Flaky tests](/docs/test-stability/#flaky-tests). ### Muted - `@Muted` Exclude the test from the report statistics. A muted test is still executed and its result is recorded, but it does not count towards the pass/fail totals. --- ## TestCafe > URL: https://allurereport.org/docs/testcafe/ # Getting started with Allure TestCafe [![Allure TestCafe npm latest version](https://img.shields.io/npm/v/testcafe-reporter-allure?style=flat "Allure TestCafe npm latest version")](https://www.npmjs.com/package/testcafe-reporter-allure) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [TestCafe](https://testcafe.io/) tests. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. The integration is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure TestCafe reporter and make sure that TestCafe itself is also listed in the project's dependencies. **npm:** ```bash npm install --save-dev testcafe testcafe-reporter-allure ``` **yarn:** ```bash yarn add --dev testcafe testcafe-reporter-allure ``` **pnpm:** ```bash pnpm add -D testcafe testcafe-reporter-allure ``` Info: The package was renamed in 3.13.0 Up to version 3.12.2, the reporter was published as `testcafe-reporter-allure-official` and registered under the reporter name `allure-official`. To upgrade, replace the package with `testcafe-reporter-allure` and change the reporter name to `allure` in your TestCafe config or on the command line. The reporter options and the API are unchanged. Versions 1.x of `testcafe-reporter-allure` are an older, unrelated third-party reporter. Make sure you install version 3.13.0 or later. ### 2. Configure the reporter The recommended way to configure the reporter is with a JavaScript config file. This also enables [test plan filtering](#select-tests-via-a-test-plan-file), which is required for Allure's smart retry and agent-mode workflows. Create a `.testcaferc.cjs` file at the root of your project: ```js const { createAllureTestPlanFilter } = require("testcafe-reporter-allure/testplan"); module.exports = { src: ["tests/**/*.test.js"], browsers: ["chromium:headless"], reporter: ["spec", "allure"], filter: createAllureTestPlanFilter(), }; ``` The `createAllureTestPlanFilter()` call returns `undefined` when `ALLURE_TESTPLAN_PATH` is not set, so TestCafe runs normally in that case. It is safe to leave enabled permanently. Info: JSON config files (`.testcaferc.json`) cannot use function-based filters. For the recommended Allure setup, use `.testcaferc.js`, `.testcaferc.cjs`, or the [runner API](#runner-api). If you only need the reporter without test plan support, you can use a JSON config: ```json { "src": ["tests/**/*.test.js"], "browsers": ["chromium:headless"], "reporter": ["spec", "allure"] } ``` Or pass the reporter name on the command line: ```bash testcafe "chromium:headless" tests -r spec,allure ``` #### Runner API Use the [TestCafe runner API](https://testcafe.io/documentation/402641/reference/testcafe-api/runner) when you need to set reporter options such as a custom output directory or link templates: ```js const createTestCafe = require("testcafe"); const createAllureTestCafeReporter = require("testcafe-reporter-allure"); const { createAllureTestPlanFilter } = require("testcafe-reporter-allure/testplan"); (async () => { const testcafe = await createTestCafe(); try { const runner = testcafe.createRunner(); await runner .src(["tests/**/*.test.js"]) .browsers(["chromium:headless"]) .filter(createAllureTestPlanFilter()) .reporter( createAllureTestCafeReporter({ resultsDir: "./out/allure-results", }), ) .run(); } finally { await testcafe.close(); } })(); ``` See [Configuration](/docs/testcafe-configuration/) for all available reporter options. ### 3. Run tests Run your TestCafe tests the same way as you would run them usually. For example: **npm:** ```bash npx testcafe --config-file .testcaferc.cjs ``` **yarn:** ```bash yarn testcafe --config-file .testcaferc.cjs ``` **pnpm:** ```bash pnpm exec testcafe --config-file .testcaferc.cjs ``` This will save necessary data into `allure-results` or another directory, according to the [configuration](/docs/testcafe-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 4. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests This section covers what the integration adds beyond standard TestCafe reporting: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization, see [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Hooks**: Use the runtime API inside [fixture and test hooks](#hooks) to add labels, steps, and attachments from setup and teardown code. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Add [screenshots and other files](#attach-files) to the test report. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. Allure TestCafe provides three ways to assign metadata and annotations to tests: - **Runtime API**: call Allure's functions from `allure-js-commons` during test execution to add data dynamically. It is recommended to call these functions as close to the beginning of the test as possible so the data is recorded even if the test fails early. - **Meta API**: set TestCafe [fixture or test metadata](https://testcafe.io/documentation/403436/guides/intermediate-guides/metadata-and-filtering#define-test-metadata) using Allure-specific keys. This is the most robust approach: metadata is resolved before the test starts and is guaranteed to be present in the report. - **Title annotations**: embed annotations directly in the test name using `@allure.=` syntax. These are extracted at test start time, before the test body runs. ### Add metadata Add [descriptions, links, labels, and more](/docs/v3/readability/#description-links-and-other-metadata) to your test results. See the [reference](/docs/testcafe-reference/#metadata) for the complete list. **Runtime API:** ```js const { displayName, owner, tags, severity } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await displayName("Sign in"); await owner("alice"); await tags("Web interface", "Authentication"); await severity("critical"); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`.meta({ "allure.label.tag": ["Web interface", "Authentication"], }); test.meta({ "allure.label.owner": "alice", "allure.label.severity": "critical", })("sign in", async (t) => { // ... }); ``` **Title annotations:** ```js fixture`Authentication`.page`https://example.com/login`; test( "sign in" + " @allure.label.owner=alice" + " @allure.label.tag=WebInterface" + " @allure.label.tag=Authentication" + " @allure.label.severity=critical", async () => { // ... }, ); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```js const { epic, feature, story } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await epic("Web interface"); await feature("Essential features"); await story("Authentication"); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`.meta({ "allure.label.epic": "Web interface", "allure.label.feature": "Essential features", }); test.meta({ "allure.label.story": "Authentication", })("sign in", async (t) => { // ... }); ``` **Title annotations:** ```js fixture`Authentication`.page`https://example.com/login`; test( "sign in" + " @allure.label.epic=WebInterface" + " @allure.label.feature=EssentialFeatures" + " @allure.label.story=Authentication", async () => { // ... }, ); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```js const { parentSuite, suite, subSuite } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await parentSuite("Tests for web interface"); await suite("Tests for essential features"); await subSuite("Tests for authentication"); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`.meta({ "allure.label.parentSuite": "Tests for web interface", "allure.label.suite": "Tests for essential features", }); test.meta({ "allure.label.subSuite": "Tests for authentication", })("sign in", async (t) => { // ... }); ``` **Title annotations:** ```js fixture`Authentication`.page`https://example.com/login`; test( "sign in" + " @allure.label.parentSuite=TestsForWebInterface" + " @allure.label.suite=TestsForEssentialFeatures" + " @allure.label.subSuite=TestsForAuthentication", async () => { // ... }, ); ``` ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), use the `step()` function from `allure-js-commons`. See the [reference](/docs/testcafe-reference/#test-steps) for the full API. In addition, the reporter automatically captures TestCafe actions such as `t.click()`, `t.typeText()`, and `t.expect()` as steps. These automatic steps appear nested under any enclosing explicit step. To disable automatic action capture, set [`captureActionsAsSteps: false`](/docs/testcafe-configuration/#captureActionsAsSteps) in the reporter config. ```js const { step, logStep, Status } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async (t) => { await step("Fill credentials", async () => { await step("Enter login", async (ctx) => { await ctx.parameter("login", "demo-user"); await t.typeText("#login", "demo-user"); }); await t.typeText("#password", "secret").click("#submit"); }); await logStep("Verified", Status.PASSED); }); ``` ### Hooks The runtime API works in TestCafe fixture and test hooks. Labels, steps, and attachments added inside hooks are recorded as part of the owning test result. **Fixture hooks:** ```js const { owner, step, attachment } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login` .beforeEach(async (t) => { await owner("alice"); await step("Dismiss cookie banner", async () => { await t.click("#accept-cookies"); }); }) .afterEach(async () => { await step("Capture session log", async () => { await attachment("session.log", "...", "text/plain"); }); }); test("sign in", async (t) => { // ... }); ``` **Test hooks:** ```js const { severity, tag, step } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test .before(async (t) => { await tag("smoke"); await step("Seed credentials", async () => { await t.typeText("#username", "demo"); }); }) .after(async () => { await severity("critical"); })("sign in", async (t) => { // ... }); ``` ### Describe parametrized tests A typical way to implement [parametrized tests](/docs/v3/readability/#parametrized-tests) in TestCafe is to define a test in a loop. To display a parameter value in the test report, pass it to the [`parameter()`](/docs/testcafe-reference/#parametrized-tests) function. Additionally, any meta key that is not an Allure-specific key (`allure.*`) and whose value is a scalar string, number, boolean, or bigint is automatically added as a test parameter. This lets you annotate test variants with plain meta without any runtime API calls: **Runtime API:** ```js const { parameter } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; for (const login of ["johndoe", "johndoe@example.com"]) { test(`sign in as ${login}`, async () => { await parameter("login", login); await parameter("timestamp", new Date().toISOString(), { excluded: true }); // ... }); } ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`; for (const [login, role] of [ ["johndoe", "user"], ["admin", "admin"], ]) { test.meta({ login, role })(`sign in as ${login}`, async (t) => { // login and role appear as parameters in the report // ... }); } ``` ### Set labels globally Any [labels](/docs/testcafe-reference/#label), including custom ones, can be set via environment variables in your operating system. Here is an example (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface npx testcafe --config-file .testcaferc.cjs ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" npx testcafe --config-file .testcaferc.cjs ``` Alternatively, set `globalLabels` in the reporter config (runner API only) to apply labels to every test result in the run. See [Configuration](/docs/testcafe-configuration/#globalLabels). ### Attach files Reports generated by Allure can include any files attached to the test using `allure.attachment()` or `allure.attachmentPath()`. See [Attachments](/docs/attachments/). ```js const { attachment, attachmentPath } = require("allure-js-commons"); const { ContentType } = require("allure-js-commons"); const path = require("node:path"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { // ... await attachment("Server response", JSON.stringify({ status: "ok" }), "application/json"); await attachmentPath("Config file", path.join(__dirname, "config.txt"), ContentType.TEXT); }); ``` #### Automatic screenshots and videos When TestCafe is configured to capture screenshots or record videos, the reporter automatically attaches them to the corresponding test result. No additional configuration is needed beyond your TestCafe setup. Screenshots taken on failure are attached to the failed action step when TestCafe provides enough information to link them. Explicitly requested screenshots via `t.takeScreenshot()` appear as attachments on the `takeScreenshot` step. When running tests across multiple browsers, screenshots and videos are attributed to the appropriate per-browser result. A `Browser` parameter is also automatically added to each result, showing the browser name. #### Quarantine mode When TestCafe runs tests in [quarantine mode](https://testcafe.io/documentation/403841/guides/intermediate-guides/quarantine-mode), the reporter attaches a `Quarantine` JSON file summarizing the result of each attempt. If the test is marked unstable by TestCafe — meaning it passed after earlier failures in the same run — the reporter also adds an `"unstable"` tag to the result automatically. #### Warnings and errors If TestCafe produces warnings during a test, they are automatically attached as a `Warnings` text file on the affected result. When a test produces multiple errors, or when a single error lacks complete diagnostic information, a formatted `Errors` text attachment is added to the result. ### Select tests via a test plan file When the `ALLURE_TESTPLAN_PATH` environment variable is set and points to an existing file, the `createAllureTestPlanFilter()` helper filters tests to only those listed in the plan. Create an Allure test plan file: ```json { "version": "1.0", "tests": [ { "selector": "tests/auth.test.js#Authentication#sign in" }, { "id": "42" } ] } ``` The `selector` format is `##`, where `` is the clean test name with any title annotations stripped. Tests can also be matched by their Allure ID, set via the `"allure.id"` meta key or `@allure.id=` title annotation. Because filtering runs before the test body executes, the runtime `allureId()` function cannot be used to register a test for plan-based selection. Set `ALLURE_TESTPLAN_PATH`, then run normally. The filter only takes effect when the variable is set: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json npx testcafe --config-file .testcaferc.cjs ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" npx testcafe --config-file .testcaferc.cjs ``` Test plan filtering requires a JS/CJS config file or runner API so that `createAllureTestPlanFilter()` can be wired up as the TestCafe `filter` function. It is not possible to use this feature with a JSON config file. ### Environment information To include environment details on the report's main page, pass key-value pairs in the `environmentInfo` option when using the runner API. See [Configuration](/docs/testcafe-configuration/#environmentInfo). --- ## TestCafe configuration > URL: https://allurereport.org/docs/testcafe-configuration/ # Allure TestCafe configuration The [Allure TestCafe](/docs/testcafe/) reporter accepts configuration options that control where results are written, how steps are captured, how links are formatted, and what additional data is included in the report. Configuration is only available when using the [runner API](https://testcafe.io/documentation/402641/reference/testcafe-api/runner). When registering the reporter by name in a config file or on the CLI, the reporter runs with default settings. To set options, pass a configuration object to `createAllureTestCafeReporter()`: ```js const createTestCafe = require("testcafe"); const createAllureTestCafeReporter = require("testcafe-reporter-allure"); const { createAllureTestPlanFilter } = require("testcafe-reporter-allure/testplan"); const { Status } = require("allure-js-commons"); const os = require("node:os"); (async () => { const testcafe = await createTestCafe(); try { await testcafe .createRunner() .src(["tests/**/*.test.js"]) .browsers(["chromium:headless"]) .filter(createAllureTestPlanFilter()) .reporter( createAllureTestCafeReporter({ resultsDir: "allure-results", captureActionsAsSteps: true, links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: (v) => `https://jira.example.com/browse/${v}`, }, }, categories: [ { name: "Known flaky tests", messageRegex: /timeout/, matchedStatuses: [Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), node_version: process.version, }, globalLabels: [{ name: "layer", value: "e2e" }], }), ) .run(); } finally { await testcafe.close(); } })(); ``` ## resultsDir Path to the directory where the reporter will save test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ## captureActionsAsSteps If `true` (the default), the reporter automatically captures TestCafe actions and assertions as Allure steps. The following actions are captured: - Navigation: `t.navigateTo()` - Interactions: `t.click()`, `t.doubleClick()`, `t.rightClick()`, `t.hover()`, `t.drag()`, `t.dragToElement()` - Input: `t.typeText()`, `t.selectText()`, `t.selectTextAreaContent()`, `t.pressKey()`, `t.setFilesToUpload()`, `t.clearUpload()` - Scrolling: `t.scroll()`, `t.scrollBy()`, `t.scrollIntoView()` - Frames and roles: `t.switchToIframe()`, `t.useRole()` - Screenshots: `t.takeScreenshot()`, `t.takeElementScreenshot()` - HTTP: `t.request()` - Waits: `t.wait()` - Assertions: `t.expect(...).eql()`, `t.expect(...).ok()`, and other expect chains - Custom actions: `t.runCustomAction()` Automatic steps are nested under any enclosing explicit step created with `allure.step()`. When an assertion fails, the corresponding step is marked as failed with `actual` and `expected` values in the status details. Set to `false` to disable automatic step capture: ```js createAllureTestCafeReporter({ captureActionsAsSteps: false, }); ``` ## links A mapping of templates that can be used to construct full URLs from short identifiers. For each link type (see [`allure.link()`](/docs/testcafe-reference/#link)), you can specify the following: - `nameTemplate` — a template or a function for generating the link name when it is not provided. - `urlTemplate` — a template or a function for generating the link address when a short identifier is passed instead of a full URL. The templates can be strings (with `%s` where the identifier should be placed) or functions (accepting the identifier and returning the result). For example, with the configuration above, `await allure.issue("AUTH-123", "Related issue")` will produce a link with the name `"Related issue"` and the address `https://issues.example.com/AUTH-123`. If no name is passed, it defaults to `Issue #AUTH-123`. Link templates can also be set via meta keys. A value in `"allure.link."` meta is treated as a raw URL or identifier that is processed through the matching template, if one is configured. ## categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). - `description` — a description shown in the categories panel for this category. Markdown is supported. - `descriptionHtml` — an HTML description shown in the categories panel for this category. This format is based on Allure's legacy categories file and is supported by all Allure Report versions. Allure Report 3 also provides a richer native category configuration — with label-based matching, grouping controls, transitions, and more — that is set directly in the [Allure Report config file](/docs/categories/) rather than in the reporter. ## environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ## globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ name, value }` label objects: ```js createAllureTestCafeReporter({ globalLabels: [ { name: "layer", value: "e2e" }, { name: "team", value: "frontend" }, ], }); ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```js createAllureTestCafeReporter({ globalLabels: { layer: "e2e", team: "frontend", }, }); ``` ## createAllureTestPlanFilter `createAllureTestPlanFilter()` accepts one optional option: - `cwd` — the directory used to resolve a relative `ALLURE_TESTPLAN_PATH`. Use this in monorepos where TestCafe does not run from the package root: ```js const { createAllureTestPlanFilter } = require("testcafe-reporter-allure/testplan"); module.exports = { // ... filter: createAllureTestPlanFilter({ cwd: __dirname }), }; ``` --- ## TestCafe reference > URL: https://allurereport.org/docs/testcafe-reference/ # Allure TestCafe reference These are the functions and conventions you can use to integrate your TestCafe tests with Allure. For an introduction to the three API styles (Runtime API, Meta API, and title annotations), see [Getting started](/docs/testcafe/). For labels and links, **test-level meta overrides fixture-level meta** for the same key. This lets you set defaults on a fixture and override them on individual tests. ## Metadata Assign a test's [description, links, and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `allure.description(markdown: string): Promise` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. ```js const { description } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await description("This test verifies that a user can sign in with valid credentials."); // ... }); ``` To set an HTML description instead, use `allure.descriptionHtml(html: string)`. ### Display name - `allure.displayName(name: string): Promise` Override the test name as it appears in the report. This does not affect the test's identity or history tracking — only the displayed label. ```js const { displayName } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await displayName("Sign in with username and password"); // ... }); ``` ### Owner - `allure.owner(name: string): Promise` - Meta: `"allure.label.owner": ""` - Title: `@allure.label.owner=` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```js const { owner } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await owner("alice"); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`; test.meta({ "allure.label.owner": "alice" })("sign in", async (t) => { // ... }); ``` **Title annotation:** ```js fixture`Authentication`.page`https://example.com/login`; test("sign in @allure.label.owner=alice", async () => { // ... }); ``` ### Tag - `allure.tag(name: string): Promise` - `allure.tags(...names: string[]): Promise` - Meta: `"allure.label.tag": ""` or `"allure.label.tag": ["", ""]` - Title: `@allure.label.tag=` (repeat for multiple tags) Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```js const { tag, tags } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await tag("smoke"); await tags("authentication", "web"); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`; test.meta({ "allure.label.tag": ["smoke", "authentication", "web"] })("sign in", async (t) => { // ... }); ``` **Title annotation:** ```js fixture`Authentication`.page`https://example.com/login`; test( "sign in" + " @allure.label.tag=smoke" + " @allure.label.tag=authentication" + " @allure.label.tag=web", async () => { // ... }, ); ``` ### Severity - `allure.severity(level: string): Promise` - Meta: `"allure.label.severity": ""` - Title: `@allure.label.severity=` Set the test's [severity](/docs/v3/readability/#other-labels). Allowed values are: `"trivial"`, `"minor"`, `"normal"`, `"critical"`, and `"blocker"`. **Runtime API:** ```js const { severity, Severity } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await severity(Severity.CRITICAL); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`; test.meta({ "allure.label.severity": "critical" })("sign in", async (t) => { // ... }); ``` ### Label - `allure.label(name: string, value: string): Promise` - `allure.labels(...labels: Label[]): Promise` - Meta: `"allure.label.": ""` - Title: `@allure.label.=` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying function used by most other Allure label helpers. You can call `label()` multiple times with the same name to assign multiple values. **Runtime API:** ```js const { label } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await label("microservice", "auth-service"); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`; test.meta({ "allure.label.microservice": "auth-service" })("sign in", async (t) => { // ... }); ``` ### ID - `allure.allureId(id: string): Promise` - Meta: `"allure.id": ""` - Title: `@allure.id=` Set the test's [Allure ID](/docs/v3/readability/#id). Used by [test plan filtering](#test-plan) to select tests by ID. Test plan filtering runs before the test body executes, so only the meta key and title annotation forms are effective for plan-based selection — the runtime `allureId()` function runs too late to influence filtering. **Runtime API:** ```js const { allureId } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await allureId("42"); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`; test.meta({ "allure.id": "42" })("sign in", async (t) => { // ... }); ``` **Title annotation:** ```js fixture`Authentication`.page`https://example.com/login`; test("sign in @allure.id=42", async () => { // ... }); ``` ### Link - `allure.link(url: string, name?: string, type?: string): Promise` - `allure.links(...links: Link[]): Promise` - `allure.issue(url: string, name?: string): Promise` - `allure.tms(url: string, name?: string): Promise` - Meta: `"allure.link.": ""` or `"allure.link.": ["", ""]` - Title: `@allure.link.=` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string, defaults to `"link"`), Allure will try to load a corresponding **link template** as defined by the [`links`](/docs/testcafe-configuration/#links) configuration option. If no template is found or if the value is already a full URL, it is left unmodified. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. **Runtime API:** ```js const { issue, tms, link } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await issue("AUTH-123", "Related issue"); await tms("TMS-456", "Related TMS ticket"); await link("https://example.com/docs/auth", "Auth documentation"); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`.meta({ "allure.link.issue": "AUTH-123", }); test.meta({ "allure.link.tms": ["TMS-456", "TMS-789"], })("sign in", async (t) => { // ... }); ``` ### Test identity - `allure.historyId(id: string): Promise` - `allure.testCaseId(id: string): Promise` Override the identifiers Allure uses to match this test across runs when computing history and grouping retries. - `historyId` replaces the computed value used to look up previous results. Use this when the same logical test runs under different names or parameters across runs. - `testCaseId` replaces the stable identifier used to group retries. Results that share a `testCaseId` are treated as retries of the same test. ```js const { historyId, testCaseId } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await historyId("auth-signin-v2"); await testCaseId("TC-1001"); // ... }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): Promise` - `allure.feature(name: string): Promise` - `allure.story(name: string): Promise` - Meta: `"allure.label.epic"`, `"allure.label.feature"`, `"allure.label.story"` - Title: `@allure.label.epic=`, `@allure.label.feature=`, `@allure.label.story=` Assign names of **epics**, **features**, or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```js const { epic, feature, story } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await epic("Web interface"); await feature("Essential features"); await story("Authentication"); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`.meta({ "allure.label.epic": "Web interface", "allure.label.feature": "Essential features", }); test.meta({ "allure.label.story": "Authentication", })("sign in", async (t) => { // ... }); ``` A common pattern is to set epic and feature on the fixture so all tests in the file share them, and override story per test. ## Suite-based hierarchy - `allure.parentSuite(name: string): Promise` - `allure.suite(name: string): Promise` - `allure.subSuite(name: string): Promise` - Meta: `"allure.label.parentSuite"`, `"allure.label.suite"`, `"allure.label.subSuite"` - Title: `@allure.label.parentSuite=`, etc. Assign the names of **parent suite**, **suite**, or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Runtime API:** ```js const { parentSuite, suite, subSuite } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { await parentSuite("Tests for web interface"); await suite("Tests for essential features"); await subSuite("Tests for authentication"); // ... }); ``` **Meta API:** ```js fixture`Authentication`.page`https://example.com/login`.meta({ "allure.label.parentSuite": "Tests for web interface", "allure.label.suite": "Tests for essential features", "allure.label.subSuite": "Tests for authentication", }); test("sign in", async (t) => { // ... }); ``` **Title annotations:** ```js fixture`Authentication`.page`https://example.com/login`; test( "sign in" + " @allure.label.parentSuite=TestsForWebInterface" + " @allure.label.suite=TestsForEssentialFeatures" + " @allure.label.subSuite=TestsForAuthentication", async () => { // ... }, ); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | Promise): Promise` - `allure.logStep(name: string, status?: Status, error?: Error): Promise` Define a [test step](/docs/steps/) with the given `name`. There are two ways to define a step: - **Lambda steps** Pass a function to `allure.step()`. The function receives a `StepContext` object with two methods: - `displayName(name: string)` — override the step name while it executes. - `parameter(name, value, mode?)` — record a parameter for the step. `mode` can be `"default"`, `"masked"`, or `"hidden"`. TestCafe action steps created automatically are nested inside any enclosing lambda step. - **No-op steps** `allure.logStep()` adds an instant step with no body. Useful for log-style annotations inside a test. The optional `status` argument defaults to `Status.PASSED`. Pass an `Error` object as the third argument to include its message in the step. ```js const { step, logStep, Status } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async (t) => { await step("Fill in the form", async (ctx) => { await ctx.displayName("Fill in credentials form"); await ctx.parameter("environment", "staging"); await t.typeText("#login", "demo-user"); await t.typeText("#password", "secret"); }); await step("Submit the form", async () => { await t.click("#submit"); }); await logStep("Verified redirect", Status.PASSED); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): Promise` Record a `name`/`value` parameter pair for the test. See [Parametrized tests](/docs/v3/readability/#parametrized-tests) for details. The optional `options` argument is an object with two properties: - `excluded` — if `true`, Allure excludes this parameter when comparing the test with its history. Use this for timestamps or other values that change on every run. See [Common pitfall: retries displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - `mode` — controls how the parameter appears in the report: - `"default"` — shown in a table with other parameters. - `"masked"` — shown in the table, but the value is hidden. Use for passwords and tokens. - `"hidden"` — not shown in the report at all. ```js const { parameter } = require("allure-js-commons"); fixture`Authentication`.page`https://example.com/login`; for (const login of ["johndoe", "johndoe@example.com"]) { test(`sign in as ${login}`, async () => { await parameter("login", login); await parameter("timestamp", new Date().toISOString(), { excluded: true }); // ... }); } ``` #### Plain meta as parameters Any key in a fixture or test `meta` object that does not start with `"allure."` and whose value is a scalar (string, number, boolean, or bigint) is automatically added to the test result as a parameter. This means you can annotate test variants directly in meta without calling `parameter()` in the test body: ```js fixture`Authentication`.page`https://example.com/login`; test.meta({ env: "staging", shard: 1 })("sign in", async (t) => { // "env" and "shard" appear as parameters in the report // ... }); ``` ## Attachments - `allure.attachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): Promise` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): Promise` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) and the filename extension used if a user downloads the attachment. You can either specify both in an object or provide just the media type and let Allure infer the extension. ```js const { attachment, attachmentPath } = require("allure-js-commons"); const { ContentType } = require("allure-js-commons"); const path = require("node:path"); fixture`Authentication`.page`https://example.com/login`; test("sign in", async () => { // ... await attachment("Server response", JSON.stringify({ status: "ok" }), "application/json"); await attachmentPath("Config file", path.join(__dirname, "config.txt"), ContentType.TEXT); }); ``` #### Automatic screenshots When TestCafe captures screenshots (e.g., on test failure or via `t.takeScreenshot()`), the reporter attaches them automatically. Screenshots are attached to the action step they belong to when TestCafe provides the association; otherwise they are attached at the test level. #### Automatic videos When TestCafe records videos, they are attached to the corresponding test result automatically. ## Meta API reference The Meta API is a TestCafe-specific way to assign Allure metadata using fixture and test `.meta()` calls. It complements the Runtime API and title annotations. | Meta key | Effect | | ----------------------- | ------------------------------------------------------------------------------ | | `"allure.id"` | Sets the Allure ID for the test | | `"allure.label."` | Adds a label with the given name; value can be a string or an array of strings | | `"allure.link."` | Adds a link of the given type; value can be a URL/ID string or an array | | Any other scalar key | Becomes a test parameter visible in the report | Fixture-level meta is merged with test-level meta. For the same key, test-level meta takes precedence. Example combining fixture and test meta: ```js fixture`Checkout`.page`https://example.com/checkout`.meta({ "allure.label.epic": "Storefront", "allure.label.feature": "Checkout", "allure.link.issue": "SHOP-100", }); test.meta({ "allure.id": "42", "allure.label.story": "Pay with card", "allure.link.issue": "PAY-42", // overrides fixture-level issue link env: "staging", // becomes a parameter })("pay with card", async (t) => { await t.expect(true).ok(); }); ``` ## Title annotation reference Title annotations use `@allure.=` (or equivalently `@allure.:`) embedded in the test name. The clean test name displayed in the report has these annotations removed. Values that contain spaces must be wrapped in `"`, `'`, or backtick quotes: ```js test('sign in @allure.label.epic="Web interface" @allure.label.feature="Essential features"', async () => { // ... }); ``` | Annotation | Effect | | ------------------------------ | -------------------------------- | | `@allure.id=` | Sets the Allure ID | | `@allure.label.=` | Adds a label with the given name | | `@allure.link.=` | Adds a link of the given type | ```js fixture`Authentication`.page`https://example.com/login`; test( "sign in" + " @allure.id=42" + " @allure.label.severity=critical" + " @allure.label.tag=smoke" + " @allure.link.issue=AUTH-123", async () => { // test name shown in report: "sign in" // ... }, ); ``` --- ## TestNG > URL: https://allurereport.org/docs/testng/ # Getting started with Allure TestNG [![Allure TestNG latest version](https://img.shields.io/maven-central/v/io.qameta.allure/allure-testng?style=flat "allure-testng")](https://mvnrepository.com/artifact/io.qameta.allure/allure-testng) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [TestNG](https://testng.org/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Atestng) to see Allure TestNG in action. ## Setting up To integrate [Allure](https://allurereport.org) into an existing **TestNG** project, you need to: 1. Add Allure dependencies to your project. 1. Set up AspectJ for `@Step` and `@Attachment` annotations support. 1. Designate a location for Allure results storage. Allure Java 3.x requires Java 17 or newer and TestNG 7.10 or newer. ### Add Allure dependencies **Maven:** ```xml 3.0.0 io.qameta.allure allure-bom ${allure.version} pom import io.qameta.allure allure-testng test ``` **Gradle (Kotlin):** ```kts // Define the version of Allure you want to use via the allureVersion property val allureVersion = "3.0.0" // ... dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation(platform("io.qameta.allure:allure-bom:$allureVersion")) // Add necessary Allure dependencies to dependencies section testImplementation("io.qameta.allure:allure-testng") } ``` **Gradle (Groovy):** ```groovy // Define the version of Allure you want to use via the allureVersion property def allureVersion = "3.0.0" dependencies { // Import allure-bom to ensure correct versions of all the dependencies are used testImplementation platform("io.qameta.allure:allure-bom:$allureVersion") // Add necessary Allure dependencies to dependencies section testImplementation "io.qameta.allure:allure-testng" } ``` ### Configure AspectJ Allure leverages AspectJ for the functionality of `@Step` and `@Attachment` annotations. Additionally, some framework integrations (such as [allure-assertj](https://mvnrepository.com/artifact/io.qameta.allure/allure-assertj)) rely on AspectJ integration to function correctly. **Maven:** ```xml 1.9.25 org.apache.maven.plugins maven-surefire-plugin 3.1.2 -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar" org.aspectj aspectjweaver ${aspectj.version} ``` **Gradle (Kotlin):** ```kts // Define the version of AspectJ val aspectJVersion = "1.9.25" // Define configuration for AspectJ agent val agent: Configuration by configurations.creating { isCanBeConsumed = true isCanBeResolved = true } dependencies { // Add aspectjweaver dependency agent("org.aspectj:aspectjweaver:${aspectJVersion}") } // Configure javaagent for test execution tasks.test { jvmArgs = listOf( "-javaagent:${agent.singleFile}" ) } ``` **Gradle (Groovy):** ```groovy // Define the version of AspectJ def aspectJVersion = "1.9.25" // Define configuration for AspectJ agent configurations { agent { canBeResolved = true canBeConsumed = true } } dependencies { // Add aspectjweaver dependency agent "org.aspectj:aspectjweaver:$aspectJVersion" } // Configure javaagent for test execution test { jvmArgs = [ "-javaagent:${configurations.agent.singleFile}" ] } ``` ### Specifying Allure Results location Allure, by default, saves test results in the project's root directory. However, it is recommended to store your test results in the build output directory. To configure this, create an `allure.properties` file and place it in the test resources directory of your project, which is typically located at `src/test/resources`: **Maven:** ```properties allure.results.directory=target/allure-results ``` **Gradle (Kotlin):** ```properties allure.results.directory=build/allure-results ``` **Gradle (Groovy):** ```properties allure.results.directory=build/allure-results ``` ### Run tests Run your TestNG tests the same way as you would run them usually. For example: For Gradle: **MacOS/Linux:** ```bash ./gradlew test ``` **Windows:** ```bash gradlew test ``` For Maven: **MacOS/Linux:** ```bash ./mvnw verify ``` **Windows:** ```bash mvnw verify ``` After running the tests, Allure will gather the test execution data and store it in the `allure-results` directory. You can then generate an HTML report from these results using Allure's reporting tools. ### Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure TestNG integration not only collects the data provided by TestNG's standard features, but also provides additional features for writing even better tests. This section lists the most notable ways to improve your tests, using both TestNG's and Allure TestNG's features. With Allure TestNG, you can: - provide [description, links and other metadata](#specify-description-links-and-other-metadata), - [organize tests](#organize-tests) into hierarchies, - divide the test into smaller, easier-to-read [test steps](#divide-a-test-into-steps), - describe parameters used when running [parametrized tests](#describe-parametrized-tests), - make the test save [screenshots and other files](#attach-screenshots-and-other-files) during execution, - [select which tests to run](#select-tests-via-a-test-plan-file) via a test plan file, - provide arbitrary [environment information](#environment-information) for the whole test report, - track [DataProvider execution](#describe-parametrized-tests) as a setup fixture in the report. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/testng-reference/#metadata) for more details. For each of the metadata fields, there are two ways to assign it: via an annotation before a test method or via a method call inside a test method's body. The second way is called “dynamic”, because it allows you to construct strings and other values at runtime before passing to the methods. Note, however, that it is highly recommended to assign all metadata as early as possible. Otherwise, there is a risk of the test failing before having all metadata set, which is bad for the test report's readability. **Annotations API:** ```java import io.qameta.allure.Description; import io.qameta.allure.Issue; import io.qameta.allure.Link; import io.qameta.allure.Owner; import io.qameta.allure.Severity; import io.qameta.allure.TmsLink; import org.testng.annotations.Test; import static io.qameta.allure.SeverityLevel.*; public class TestMyWebsite { @Test @Description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") @Severity(CRITICAL) @Owner("John Doe") @Link(name = "Website", url = "https://dev.example.com/") @Issue("AUTH-123") @TmsLink("TMS-456") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication."); Allure.label("severity", "critical"); Allure.label("owner", "John Doe"); Allure.link("Website", "https://dev.example.com/"); Allure.issue("AUTH-123", "https://example.com/issues/AUTH-123"); Allure.tms("TMS-456", "https://example.com/tms/TMS-456"); // ... } } ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure TestNG provides functions to assign the relevant fields to tests either by adding annotations or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Annotations API:** ```java import io.qameta.allure.Epic; import io.qameta.allure.Feature; import io.qameta.allure.Story; import org.testng.annotations.Test; public class TestMyWebsite { @Test @Epic("Web interface") @Feature("Essential features") @Story("Authentication") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.epic("Web interface"); Allure.feature("Essential features"); Allure.story("Authentication"); // ... } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("parentSuite" "Tests for web interface"); Allure.suite("Tests for essential features"); Allure.label("subSuite", "Tests for authentication"); // ... } } ``` A test's location in the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy) is defined by the fully qualified names of the classes they are declared in, with common prefixes shown as parent packages. ### Divide a test into steps Allure TestNG provides three ways of [creating steps and sub-steps](/docs/steps/): “annotated steps”, “lambda steps” and “no-op steps”, see the [reference](/docs/testng-reference/#test-steps). **Annotated steps:** ```java import io.qameta.allure.Step; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { step1(); step2(); } @Step("Step 1") public void step1() { subStep1(); subStep2(); } @Step("Sub-step 1") public void subStep1() { // ... } @Step("Sub-step 2") public void subStep2() { // ... } @Step("Step 2") public void step2() { // ... } } ``` **Lambda steps and no-op steps:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.step("Step 1", step -> { // ... Allure.step("Sub-step 1"); // ... Allure.step("Sub-step 2"); }); Allure.step("Step 2", step -> { // ... }); } } ``` ### Describe parametrized tests Allure TestNG provides complete support for TestNG's [parameterized tests](https://testng.org/#_parameters), including DataProvider-based tests. When a `@DataProvider` method runs, Allure captures it as a setup fixture attached to the test container, so DataProvider failures and timings are visible in the report alongside the tests they feed. See the [reference](/docs/testng-reference/#parametrized-tests) for more examples. ```java import org.testng.annotations.Parameters; import org.testng.annotations.Test; public class TestMyWebsite { @Test @Parameters({"Login", "Password"}) public void testAuthentication(String login, String password) { // ... } } ``` ```xml ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure TestNG provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/testng-reference/#attachments). ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; import java.io.IOException; import java.io.InputStream; import java.nio.file.Files; import java.nio.file.Paths; public class TestMyWebsite { @Test public void testAuthentication() throws IOException { // ... Allure.attachment("data.txt", "This is the file content."); try (InputStream is = Files.newInputStream(Paths.get("/path/img.png"))) { Allure.attachment("image.png", is); } } } ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, TestNG will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json ./gradlew test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" gradlew test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS and Java versions. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/java/environment-allure3-light.png, /images/java/environment-allure3-dark.png, /images/java/environment-allure2-light.png, /images/java/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](/docs/testng-reference/#parametrized-tests). --- ## TestNG configuration > URL: https://allurereport.org/docs/testng-configuration/ # Allure TestNG configuration This page describes the configuration options that affect the behavior of [Allure TestNG](/docs/testng/). To set the configuration options, create a `allure.properties` file in your project's `src/test/resources` directory. Allure TestNG loads the configuration into the JVM system properties for the test runtime. ## allure.label.⟨NAME⟩ ```properties allure.label.epic = Authentication allure.label.severity = blocker ``` Assign an arbitrary [label](/docs/testng-reference/#label) to all the tests in the current test run. ## allure.link.⟨TYPE⟩.pattern ```properties allure.link.issue.pattern = https://jira.example.org/browse/{} ``` Define a **link pattern** — a template that can be used to construct full URLs from short identifiers, see the [reference](/docs/testng-reference/#link). The value must be a string pattern containing `{}` at the position where the identifier should be placed. The pattern will be applied when processing any link of the given `TYPE`. You can use the option multiple times, thus defining multiple link patterns for different link types. ## allure.results.directory ```properties allure.results.directory = build/allure-results ``` Path to the directory where Allure TestNG will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ## allure.testng.hide.disabled.tests ```properties allure.testng.hide.disabled.tests = true ``` In Allure, all tests from a run are reported by default. This includes disabled tests, which are marked as 'skipped' in the report. However, you can modify this behavior. To hide disabled tests from the report, set the `allure.testng.hide.disabled.tests` variable to `true`. This will exclude these tests from the report. ## allure.testng.hide.configuration.failures ```properties allure.testng.hide.configuration.failures = true ``` Allure, by default, treats configuration failures (e.g. errors in `@BeforeMethod` or `@AfterMethod` methods) uniquely. It generates a separate 'fake' test result for each configuration failure. This approach is adopted because configuration failures lead to the skipping of all dependent tests, which are then reported as `skipped`. The 'fake' test result explicitly shows the error in the configuration method, aiding in easier identification and troubleshooting of the issue. To prevent Allure from creating separate results for failed configuration methods, set the `allure.testng.hide.configuration.failures` variable to `true`. This will disable the reporting of failed configuration methods as separate test results. --- ## TestNG reference > URL: https://allurereport.org/docs/testng-reference/ # Allure TestNG reference These are the functions that you can use to integrate your TestNG tests with Allure. Allure TestNG provides more than one way to use some features. In most cases, one way involves using an annotation on the test method, while the other involves a method call inside the test method body. The latter style is called “dynamic” and allows to construct values dynamically. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `Allure.getLifecycle().updateTest(Consumer update)` Set the test's [title](/docs/v3/readability/#title). Allure recognizes the TestNG's standard `testName` argument as the title for the test. If you need to construct a test's title dynamically, write a lambda function that does that and pass it to the `updateTest()` method of the `AllureLifecycle` object, see the example below. Info: In Allure Java 2.x, the `AllureLifecycle` method for updating the test result is called `updateTestCase()`. **Annotations API:** ```java import org.testng.annotations.Test; public class TestMyWebsite { @Test(testName = "Test Authentication") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.getLifecycle().updateTest(result -> { result.setName("Test Authentication"); }); // ... } } ``` ### Description - `@Description(String value="", boolean useJavaDoc=false)` - `Allure.description(String description)` - `Allure.descriptionHtml(String description)` Set the test's [description](/docs/v3/readability/#description). Use the `@Description()` annotation to set a description statically or use the `description()` method to set it dynamically in runtime. **Annotations API:** ```java import io.qameta.allure.Description; import org.testng.annotations.Test; public class TestMyWebsite { @Test @Description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication."); // ... } } ``` Alternatively, you can let Allure TestNG parse the description from the test method's JavaDoc comment. To do so, first add a special annotation processor to your project's Maven or Gradle configuration: **JavaDoc:** ```kotlin // In the project's `build.gradle.kts`: val allureVersion = "2.32.0" dependencies { // ... testAnnotationProcessor("io.qameta.allure:allure-descriptions-javadoc:$allureVersion") } ``` Then, use the Annotations API with the `useJavaDoc=true` argument on the test methods for which you want to parse the description from JavaDoc. Note that on a large project, processing the annotations may significantly increase both the compilation time and the file size of the test executable files. Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ```java import io.qameta.allure.Description; import org.testng.annotations.Test; public class TestMyWebsite { /** * This test attempts to log into the website using a login and a password. Fails if any error happens. *

* Note that this test does not test 2-Factor Authentication. */ @Test @Description(useJavaDoc = true) public void testAuthentication() { // ... } } ``` ### Owner (Allure TestOps) - `@Owner(String value="")` Set the test's [owner](/docs/v3/readability/#owner). **Annotations API:** ```java import io.qameta.allure.Owner; import org.testng.annotations.Test; public class TestMyWebsite { @Test @Owner("John Doe") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("owner", "John Doe"); // ... } } ``` ### Lead - `@Lead(String value="")` Set the test's lead — the person responsible for the test's maintenance. **Annotations API:** ```java import io.qameta.allure.Lead; import org.testng.annotations.Test; public class TestMyWebsite { @Test @Lead("Jane Doe") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("lead", "Jane Doe"); // ... } } ``` ### Tag Set the test's [tags](/docs/v3/readability/#tags). ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("tag", "NewUI"); Allure.label("tag", "Essentials"); Allure.label("tag", "Authentication"); // ... } } ``` ### Severity - `@Severity(SeverityLevel value)` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Annotations API:** ```java import io.qameta.allure.Severity; import org.testng.annotations.Test; import static io.qameta.allure.SeverityLevel.*; public class TestMyWebsite { @Test @Severity(CRITICAL) public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("severity", "critical"); // ... } } ``` ### Label - `@LabelAnnotation(String name, String value=DEFAULT_VALUE)` - `Allure.label(String name, String value)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. To set a label dynamically, just call the `label()` function with a name and a value for a label. You can do it multiple times to create an array of values under that name. Another way of setting a label is to create a custom class using `@LabelAnnotation` and annotate the tests with the new annotation, see the example below. Please be careful to copy `@Retention`, `@Target` and the other necessary annotations as this is shown in the example, otherwise you custom annotation may not work properly. **Annotations API:** ```java import io.qameta.allure.LabelAnnotation; import org.testng.annotations.Test; import java.lang.annotation.Documented; import java.lang.annotation.ElementType; import java.lang.annotation.Inherited; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; @Documented @Inherited @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.METHOD, ElementType.TYPE}) @LabelAnnotation(name = "CustomLabelName") @interface MyLabel { String value(); } public class TestMyWebsite { @Test @MyLabel("custom label value") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("language", "java"); Allure.label("framework", "testng"); Allure.label("CustomLabelName", "custom label value"); // ... } } ``` ### Allure ID (Allure TestOps) - `@AllureId(String value)` Set the test's [ID](/docs/v3/readability/#id). ```java import io.qameta.allure.AllureId; import org.testng.annotations.Test; public class TestMyWebsite { @Test @AllureId("123") public void testAuthentication() { // ... } } ``` ### Link - `@Link(String value="", name="", String url="", String type=CUSTOM_LINK_TYPE)` - `@Links(Link[] value)` - `@Issue(String value="")` - `@Issues(Issue[] value)` - `@TmsLink(String value="")` - `@TmsLinks(TmsLink[] value)` - `Allure.link(String url)` - `Allure.link(String name, String url)` - `Allure.link(String name, String type, String url)` - `Allure.issue(String name, String url)` - `Allure.tms(String name, String url)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`allure.link.*.pattern`](/docs/testng-configuration/#allure-link-⟨type⟩-pattern) configuration option. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue()` and `tms()`. **Annotations API:** ```java import io.qameta.allure.Issue; import io.qameta.allure.Link; import io.qameta.allure.TmsLink; import org.testng.annotations.Test; public class TestMyWebsite { @Test @Link(name = "Website", url = "https://dev.example.com/") @Issue("AUTH-123") @TmsLink("TMS-456") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.link("Website", "https://dev.example.com/"); Allure.issue("AUTH-123", "https://jira.example.org/browse/AUTH-123"); Allure.tms("TMS-456", "https://tms.example.org/TMS-456"); // ... } } ``` ## Behavior-based hierarchy - `@Epic(String value="")` - `@Epics(Epic[] value)` - `@Feature(String value="")` - `@Features(Feature[] value)` - `@Story(String value="")` - `@Stories(Story[] value)` - `Allure.epic(String value)` - `Allure.feature(String value)` - `Allure.story(String value)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Annotations API:** ```java import io.qameta.allure.Epic; import io.qameta.allure.Feature; import io.qameta.allure.Story; import org.testng.annotations.Test; public class TestMyWebsite { @Test @Epic("Web interface") @Feature("Essential features") @Story("Authentication") public void testAuthentication() { // ... } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.epic("Web interface"); Allure.feature("Essential features"); Allure.story("Authentication"); // ... } } ``` ## Suite-based hierarchy - `Allure.suite(String value)` Assign the name of suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). To assign names of parent suite or sub-suite, use [`label()`](#label) with the corresponding label names. **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.label("parentSuite" "Tests for web interface"); Allure.suite("Tests for essential features"); Allure.label("subSuite", "Tests for authentication"); // ... } } ``` ## Test steps - `@Step(String value="")` - `Allure.step(String name)` - `Allure.step(String name, Status status)` - `Allure.step(String name, ThrowableRunnableVoid runnable)` - `Allure.step(String name, ThrowableRunnable runnable)` - `Allure.step(ThrowableContextRunnableVoid runnable)` - `Allure.step(String name, ThrowableContextRunnableVoid runnable)` - `Allure.step(String name, ThrowableContextRunnable runnable)` - `Allure.step(ThrowableContextRunnable runnable)` Define a [test step](/docs/steps/) with the given `name`. There are three ways of defining a step. - “An annotated step”: define a method containing a test step and decorate it with `@Step()`. - “A lambda step”: write a test step in a lambda function and pass it to `Allure.step()`. - “A no-op step”: just pass a `name` and an optional `status` to `Allure.step()` to immediately add a corresponding entry to the results as a sub-step of the current step. When you are implementing a function for a lambda step, it can accept either no arguments or a single argument of class `StepContext`. This object provides the following methods: - `name()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are the same as for the test-wide implementation, see [Parametrized tests](#parametrized-tests). **Annotated steps:** ```java import io.qameta.allure.Step; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { step1(); step2(); } @Step("Step 1") public void step1() { subStep1(); subStep2(); } @Step("Sub-step 1") public void subStep1() { // ... } @Step("Sub-step 2") public void subStep2() { // ... } @Step("Step 2") public void step2() { // ... } } ``` **Lambda steps and no-op steps:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthentication() { Allure.step("Step 1", step -> { // ... Allure.step("Sub-step 1"); // ... Allure.step("Sub-step 2"); }); Allure.step("Step 2", step -> { // ... }); } } ``` ## Parametrized tests - `@TestInstanceParameter(String value="")` - `Allure.parameter(String name, T value)` - `Allure.parameter(String name, T value, Boolean excluded)` - `Allure.parameter(String name, T value, Parameter.Mode mode)` - `Allure.parameter(String name, T value, Boolean excluded, Parameter.Mode mode)` TestNG provides multiple approaches for implementing the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern. Depending on the approach you use, you may or may not need to modify your code to add the list of parameters to the test report. ### Test method parameters via XML With this approach, TestNG uses the same test method for multiple tests, each one defined by a `` tag in the XML file. The names and values of parameters are specified in `` tags and mapped to the method argument using the `@Parameters` annotation. 1. In a test class, define a test method with one or more arguments. Above the method signature, add the `@Parameters` annotation and pass to it an array of names. The names passed to `@Parameters` will be used both for mapping the values from the XML file and for displaying the parameters in the test report. These names can differ from the actual names of the method arguments. 1. In the XML file that you use for running TestNG tests, add one or more `` tags with the same test implementation in ``, but with different sets of values in `` tags. ```java import org.testng.annotations.Parameters; import org.testng.annotations.Test; public class TestMyWebsite { @Test @Parameters({"Login", "Password"}) public void testAuthentication(String login, String password) { // ... } } ``` ```xml ``` ### Test instance parameters via XML With this approach, TestNG instantiates a test class multiple times, each one defined by a `` tag in the XML file. The names and values of parameters are specified in `` tags and mapped to the constructor using the `@Parameters` annotation. 1. In a test class, define a constructor with one or more arguments. Above the constructor signature, add the `@Parameters` annotation and pass to it an array of names. The names passed to `@Parameters` will be used both for mapping the values from the XML file and for displaying the parameters in the test report. These names can differ from the actual names of the constructor arguments. 1. Define one or more test methods in the class. In the test report, the parameters defined for the constructor will be displayed for each of the test methods. If a test method has its own `@Parameters` annotation (as with the “Test method parameters via XML” approach), its parameters will be appended to the list for this particular method. 1. In the XML file that you use for running TestNG tests, add one or more `` tags with the same test implementation in ``, but with different sets of values in `` tags. ```java import org.testng.annotations.Parameters; import org.testng.annotations.Test; public class TestMyWebsite { private String login; private String password; @Parameters({"Login", "Password"}) public TestMyWebsite(String login, String password) { this.login = login; this.password = password; } @Test public void testAuthentication() { // ... } } ``` ```xml ``` ### Test method parameters via DataProvider This approach does not involve any modifications in the XML files and instead gets both names and values of parameters from a special “data provider” method. Allure TestNG records each `@DataProvider` execution as a setup fixture attached to the test container, so DataProvider timing and failures appear in the report alongside the tests they supply. 1. In the test class, define a data provider — a static method that returns an array or arrays of values, i.e., an `Object[][]`. Annotate it with `@DataProvider`. 1. Define a test method that accept arguments. Specify the name of the data provider in the `dataProvider` argument for the `@Test` annotation. TestNG will run this test multiple times, each time with a new set of values from the data provider. Allure TestNG will recognize the parameters and display them to the test report for each test method of the class. 1. Optionally, customize the parameter names via a `@Parameter` annotation. ```java import org.testng.annotations.DataProvider; import org.testng.annotations.Parameters; import org.testng.annotations.Test; public class TestMyWebsite { @DataProvider public static Object[][] authenticationData() { return new Object[][]{ {"johndoe", "qwerty"}, {"johndoe@example.com", "qwerty"}, }; } @Test(dataProvider = "authenticationData") @Parameters({"Login", "Password"}) // optional public void testAuthentication(String login, String password) { // ... } } ``` ### Test instance parameters via DataProvider With this approach, TestNG instantiates a test class multiple times, each one with a new set of values provided by a special “data provider” method. 1. In the test class, define a data provider — a static method that returns an array or arrays of values, i.e., an `Object[][]`. Annotate it with `@DataProvider`. 1. Define the fields for storing the values for the parameters. Add the `@TestInstanceParameter` annotation to each of them, with an optional argument specifying the display name of the parameter for the test report. 1. Define a constructor that accepts arguments. Above the constructor signature, add the `@Factory` annotation and specify the name of the data provider in its `dataProvider` argument. 1. Define one or more test methods in the class. In the test report, the parameters defined via `@TestInstanceParameter` will be displayed for each of the test methods. If a test method has its own `@Parameters` annotation (as with the “Test method parameters via XML” approach), its parameters will be appended to the list for this particular method. ```java import io.qameta.allure.testng.TestInstanceParameter; import org.testng.annotations.DataProvider; import org.testng.annotations.Factory; import org.testng.annotations.Test; public class TestMyWebsite { @TestInstanceParameter("Login") private String login; @TestInstanceParameter("Password") private String passwordForTests; @DataProvider public static Object[][] authenticationData() { return new Object[][]{ {"johndoe", "qwerty"}, {"johndoe@example.com", "qwerty"}, }; } @Factory(dataProvider = "authenticationData") public TestMyWebsite(String login, String password) { this.login = login; this.passwordForTests = password; } @Test public void testAuthentication() { // ... } } ``` ### Runtime API Additionally, the `Allure.parameter()` method can be used at any point for adding the parameters even to those tests that do not use the TestNG's parametrized testing approaches. Warning: Parameters added via `Allure.parameter()` are part of the test's identity, together with the parameters provided by the test framework. If a parameter's value changes between runs (for example, a timestamp or a random ID), the test will be treated as a new test on every run for the purposes of [History and retries](/docs/history-and-retries/). To avoid this, mark such parameters as excluded by passing `true` as the `excluded` argument, for example: `Allure.parameter("startedAt", startedAt, true)`. In Allure Java 2.x, parameters added via `Allure.parameter()` did not affect the test's identity. ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; public class TestMyWebsite { @Test public void testAuthenticationWithUsername() { Allure.parameter("login", "johndoe"); Allure.parameter("password", "qwerty"); // ... } @Test public void testAuthenticationWithEmail() { Allure.parameter("login", "johndoe@example.com"); Allure.parameter("password", "qwerty"); // ... } } ``` ## Attachments - `@Attachment(String value="", String type="", String fileExtension="")` - `Allure.attachment(String name, String content)` - `Allure.attachment(String name, String type, String content)` - `Allure.attachment(String name, String type, String content, AttachmentOptions options)` - `Allure.attachment(String name, InputStream content)` - `Allure.attachment(String name, String type, InputStream content, AttachmentOptions options)` Add `content` as an [attachment](/docs/attachments/) to the test result under the given `name` (defaults to a unique pseudo-random string). Info: In Allure Java 2.x, the same functionality is provided by the `Allure.addAttachment()` methods, which accept the file extension as a string. They were removed in Allure Java 3.0. Tip: You can use data produced by any function, not necessarily read from an actual file. To create an attachment using the Annotations API, define a method that returns some data and annotate it with `@Attachment`. Call the method at any point during your test. To create an attachment using the Runtime API, just call `attachment()` at any point during your test. Pass the data as the `content` argument. The data will be processed as following: - If the data type is `byte[]`, Allure will use it without modification. - If the data type is `String`, Allure will convert it to `byte[]`. - If the data is of any other type, Allure will call the `toString()` method and then convert the string to `byte[]`. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension: as `fileExtension` in the `@Attachment` annotation, or via `AttachmentOptions.withFileExtension(".txt")` as the `options` argument of `attachment()`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. **Annotations API:** ```java import io.qameta.allure.Attachment; import org.testng.annotations.Test; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; public class TestMyWebsite { @Test public void testAuthentication() throws IOException { // ... attachDataTXT(); attachScreenshotPNG(); } @Attachment(value = "data", type = "text/plain", fileExtension = ".txt") public String attachDataTXT() { return "This is the file content."; } @Attachment(value = "screenshot", type = "image/png", fileExtension = ".png") public byte[] attachScreenshotPNG() throws IOException { return Files.readAllBytes(Paths.get("/path/to/image.png")); } } ``` **Runtime API:** ```java import io.qameta.allure.Allure; import org.testng.annotations.Test; import java.io.IOException; import java.io.InputStream; import java.nio.file.Files; import java.nio.file.Paths; public class TestMyWebsite { @Test public void testAuthentication() throws IOException { // ... Allure.attachment("data.txt", "This is the file content."); try (InputStream is = Files.newInputStream(Paths.get("/path/img.png"))) { Allure.attachment("image.png", is); } } } ``` Info: Additionally, Allure provides a way to spawn threads that will add attachments to the test results asynchronously without blocking the test runner. While not recommended for the majority of cases, this approach can improve tests that otherwise have to wait for processing very large files, such as videos larger than 1 GB. To do so, pass a `CompletionStage` that produces the attachment's content to `Allure.attachmentAsync(String name, String type, CompletionStage body)`. In Allure Java 2.x, use `Allure.addByteAttachmentAsync()` or `Allure.addStreamAttachmentAsync()` instead. ## Results history ### Flaky - `@Flaky` Indicate that the test is known to be unstable and can may not succeed every time. See [Flaky tests](/docs/test-stability/#flaky-tests). ### Muted - `@Muted` Exclude the test from the report statistics. A muted test is still executed and its result is recorded, but it does not count towards the pass/fail totals. --- ## Vitest > URL: https://allurereport.org/docs/vitest/ # Getting started with Allure Vitest [![Allure Vitest npm latest version](https://img.shields.io/npm/v/allure-vitest?style=flat "Allure Vitest npm latest version")](https://www.npmjs.com/package/allure-vitest) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [Vitest](https://vitest.dev/) tests. Info: Check out the example projects at [github.com/allure-examples](https://github.com/orgs/allure-examples/repositories?q=visibility%3Apublic+archived%3Afalse+topic%3Aexample+topic%3Avitest) to see Allure Vitest in action. ## Setting up ### 1. Prepare your project 1. Make sure [Node.js](https://nodejs.org/) is installed. Allure Vitest is tested against Node.js 18 and higher. Older versions may work, but we can't guarantee that. 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure Vitest integration and make sure that all the packages it needs are installed. **npm:** ```bash npm install --save-dev vitest @vitest/runner allure-vitest ``` **yarn:** ```bash yarn add --dev vitest @vitest/runner allure-vitest allure-js-commons ``` **pnpm:** ```bash pnpm add -D vitest @vitest/runner allure-vitest ``` Warning: If you run the tests and see the “no vitest context is detected” error, make sure the `vitest` and `@vitest/runner` packages are both listed as your project's `devDependencies` and have the same version. If you plan to run tests in a real browser using [Vitest Browser Mode](https://vitest.dev/guide/browser/), also install `@vitest/browser` and a browser provider. Allure Vitest supports the Playwright provider: **npm:** ```bash npm install --save-dev @vitest/browser @vitest/browser-playwright playwright npx playwright install chromium ``` **yarn:** ```bash yarn add --dev @vitest/browser @vitest/browser-playwright playwright yarn playwright install chromium ``` **pnpm:** ```bash pnpm add -D @vitest/browser @vitest/browser-playwright playwright pnpm playwright install chromium ``` 1. Modify your [Vitest configuration file](https://vitest.dev/config/), e.g., `vitest.config.ts`. - In the list of setup files, add `"allure-vitest/setup"` (for most package managers) or `require.resolve("allure-vitest/setup")` (for Yarn PnP). - In the list of reporters, add `"allure-vitest/reporter"`. - Optionally, specify `resultsDir` and other options for the reporter. See [Allure Vitest configuration](/docs/vitest-configuration/) for more details. **Normal configuration:** ```ts import AllureReporter from "allure-vitest/reporter"; import { defineConfig } from "vitest/config"; export default defineConfig({ test: { setupFiles: ["allure-vitest/setup"], reporters: [ "verbose", [ "allure-vitest/reporter", { resultsDir: "allure-results", }, ], ], }, }); ``` **Configuration for Yarn PnP:** ```ts import AllureReporter from "allure-vitest/reporter"; import { defineConfig } from "vitest/config"; const require = createRequire(import.meta.url); export default defineConfig({ test: { setupFiles: [require.resolve("allure-vitest/setup")], reporters: [ "verbose", [ "allure-vitest/reporter", { resultsDir: "allure-results", }, ], ], }, }); ``` ### 2. Run tests Run your Vitest tests same way as your would run them usually. For example: **npm:** ```bash npm test ``` **yarn:** ```bash yarn test ``` **pnpm:** ```bash pnpm test ``` This will save necessary data into `allure-results` or other directory, according to the [configuration](/docs/vitest-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure Vitest integration extends the standard reporting features of Vitest by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#add-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Set labels globally**: Use [environment variables](#set-labels-globally) to set metadata and other labels. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure Vitest provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Vitest will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ### Add Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/vitest-reference/#metadata) for an exhaustive list of what can be added. **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.displayName("Test Authentication"); await allure.owner("John Doe"); await allure.tags("Web interface", "Authentication"); await allure.severity("critical"); // ... }); ``` **Metadata API:** ```ts import { test } from "vitest"; test( "Test Authentication" + " @allure.label.owner:JohnDoe" + " @allure.label.tag:WebInterface" + " @allure.label.tag:Authentication" + " @allure.label.severity:critical", async () => { // ... }, ); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Metadata API:** ```ts import { test } from "vitest"; test( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```ts import { test } from "vitest"; test( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); ``` ### Divide a test into steps To [create steps and sub-steps](/docs/steps/), you can use the `step()` function, see the [reference](/docs/vitest-reference/#test-steps). ```ts import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ### Describe parametrized tests Since tests in Vitest, unlike in some other frameworks, are written as anonymous functions, it is very easy to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, i.e. to run the same test logic with different test data. To do so, just write the test inside a loop and use the variable parameters in both its title and its body. To display a parameter value in the test report, pass it to the [`parameter()`](/docs/vitest-reference/#parametrized-tests) function. **Using test():** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } ``` **Using test.each():** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test.each(["johndoe", "johndoe@example.com"])("Test Authentication as %s", async (login) => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); ``` ### Set labels globally Any [labels](/docs/vitest-reference/#label), including custom ones, can be set via the environment variables in your operating system. Here's an example (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_LABEL_epic=WebInterface npm test ``` **Windows:** ```powershell $Env:ALLURE_LABEL_epic = "WebInterface" npm test ``` ### Attach screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure Vitest reference](/docs/vitest-reference/#attachments). ```ts import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, Vitest will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json` (assuming you use the `npm` package manager): **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json npm test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" npm test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. To do so, edit the [`environmentInfo`](/docs/vitest-configuration/#environmentinfo) object in the configuration. For example, it is a good idea to use this to remember the OS version and Node.js version retrieved from the [`os`](https://nodejs.org/api/os.html) and [`process`](https://nodejs.org/api/process.html) objects. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/js/environment-allure3-light.png, /images/js/environment-allure3-dark.png, /images/js/environment-allure2-light.png, /images/js/environment-allure2-dark.png ```ts import { createRequire } from "node:module"; import * as os from "node:os"; import { defineConfig } from "vitest/config"; const require = createRequire(import.meta.url); export default defineConfig({ test: { setupFiles: [require.resolve("allure-vitest/setup")], reporters: [ "verbose", [ "allure-vitest/reporter", { environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }, ], ], }, }); ``` Note that if your launch includes multiple Vitest runs (see [How it works](/docs/how-it-works/)), Allure Vitest will only save the environment information from the latest run. ## Using Browser Mode [Vitest Browser Mode](https://vitest.dev/guide/browser/) runs your tests inside a real browser instead of a simulated DOM environment. Info: Browser Mode support requires Vitest 4 or later. To enable Browser Mode, add a `browser` block to your Vitest configuration. In Vitest 4, the browser provider is imported as a factory from its own package: ```ts import { playwright } from "@vitest/browser-playwright"; import { defineConfig } from "vitest/config"; export default defineConfig({ test: { setupFiles: ["allure-vitest/browser/setup"], reporters: [ "verbose", [ "allure-vitest/reporter", { resultsDir: "allure-results", }, ], ], browser: { enabled: true, provider: playwright(), instances: [{ browser: "chromium" }], }, }, }); ``` Warning: `allure-vitest/browser/setup` is not interchangeable with `allure-vitest/setup`. The browser setup file is designed to run in a browser context and registers additional commands required for test plan filtering. If your project has both unit tests and browser tests, use a separate Vitest config file for each, pointing to the appropriate setup file. All standard Allure Vitest features — metadata, steps, attachments, and so on — work the same way in Browser Mode. In addition, you can capture screenshots of the browser page and attach them to the test result using the `page` object from `vitest/browser`: ```ts import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import { test } from "vitest"; import { page } from "vitest/browser"; test("Renders the login form", async () => { // ... render and interact with the page ... const screenshotPath = await page.screenshot(); await allure.attachmentPath("Page screenshot", screenshotPath, { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` For more details on attaching screenshots and other files, see the [attachments section in the Allure Vitest reference](/docs/vitest-reference/#attachments). --- ## Vitest configuration > URL: https://allurereport.org/docs/vitest-configuration/ # Allure Vitest configuration The [Allure Vitest](/docs/vitest/) integration behavior can be affected by some configuration options set in your [Vitest configuration file](https://vitest.dev/config/), e.g., `vitest.config.ts`. For example: ```ts import { Status } from "allure-js-commons"; import { createRequire } from "node:module"; import * as os from "node:os"; import { defineConfig } from "vitest/config"; const require = createRequire(import.meta.url); export default defineConfig({ test: { setupFiles: [require.resolve("allure-vitest/setup")], reporters: [ "verbose", [ "allure-vitest/reporter", { resultsDir: "allure-results", links: { issue: { nameTemplate: "Issue #%s", urlTemplate: "https://issues.example.com/%s", }, tms: { nameTemplate: "TMS #%s", urlTemplate: "https://tms.example.com/%s", }, jira: { urlTemplate: "https://jira.example.com/browse/%s", }, }, categories: [ { name: "foo", messageRegex: "bar", traceRegex: "baz", matchedStatuses: [Status.FAILED, Status.BROKEN], }, ], environmentInfo: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, globalLabels: [{ name: "layer", value: "api" }], }, ], ], }, }); ``` ## resultsDir Path to the directory where Allure Vitest will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ## links A mapping of templates that can be used to construct full URLs from short identifiers. For each type of link (see [`allure.link()`](/docs/vitest-reference/#link)), you can specify the following: - `nameTemplate` — a template for generating the link name when it is not provided. - `urlTemplate` — a template for generating the link address when it is not provided. Both templates must contain `%s` at the position where the identifier should be placed. For example, with the configuration above, `await allure.issue("123")` will produce a link with the name `Issue #123` and the address `https://issues.example.com/`. Allure will display the link with the appropriate icon for the `issue` type. ## categories Define custom categories that will be used to distinguish test results by their errors; see [Categories](/docs/categories/). This setting is an array, each item being an object representing one custom category. The objects may have the following properties: - `name` — a category name. - `messageRegex` — a regular expression that the test result's message should match. - `traceRegex` — a regular expression that the test result's trace should match. - `matchedStatuses` — an array of statuses that the test result should be one of. - `flaky` — whether the test result should be marked as [flaky](/docs/test-stability/#flaky-tests). ## environmentInfo Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ## globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ name, value }` label objects: ```ts export default defineConfig({ test: { reporters: [ [ "allure-vitest/reporter", { globalLabels: [ { name: "layer", value: "api" }, { name: "team", value: "backend" }, ], }, ], ], }, }); ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```ts export default defineConfig({ test: { reporters: [ [ "allure-vitest/reporter", { globalLabels: { layer: "api", team: "backend", }, }, ], ], }, }); ``` --- ## Vitest reference > URL: https://allurereport.org/docs/vitest-reference/ # Allure Vitest reference These are the functions that you can use to integrate your Vitest tests with Allure. In most cases, Allure Vitest provides two different ways to use a feature: the Runtime API and the Metadata API. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. - **Metadata API**: add a metadata tag (beginning with `@`) into the test name. Allure Vitest will extract it and update the test result's data accordingly. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `allure.displayName(name: string): PromiseLike` Set the test's [title](/docs/v3/readability/#title). ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.displayName("Test Authentication!"); // ... }); ``` ### Description - `allure.description(markdown: string): PromiseLike` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.description("This test attempts to log into the website."); // ... }); ``` ### Owner - `allure.owner(name: string): PromiseLike` - `@allure.label.owner:⟨VALUE⟩` Set the test's [owner](/docs/v3/readability/#owner). **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.owner("John Doe"); // ... }); ``` **Metadata API:** ```ts import { test } from "vitest"; test("Test Authentication @allure.label.owner:JohnDoe", async () => { // ... }); ``` ### Tag - `allure.tag(name: string): PromiseLike` - `allure.tags(...tagsList: string[]): PromiseLike` - `@allure.label.tag:⟨VALUE⟩` Set the test's [tags](/docs/v3/readability/#tags). **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.tag("New UI"); await allure.tags("Essentials", "Authentication"); // ... }); ``` **Metadata API:** ```ts import { test } from "vitest"; test("Test Authentication @allure.label.tag:WebInterface @allure.label.tag:Authentication", async () => { // ... }); ``` ### Severity - `allure.severity(name: string): PromiseLike` - `@allure.label.severity:⟨VALUE⟩` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { Severity } from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.severity(Severity.CRITICAL); // ... }); ``` **Metadata API:** ```ts import { test } from "vitest"; test("Test Authentication @allure.label.severity:critical", async () => { // ... }); ``` ### Label - `allure.label(name: LabelName | string, value: string): PromiseLike` - `allure.labels(...labelsList: Label[]): PromiseLike` - `@allure.label.⟨NAME⟩:⟨VALUE⟩` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. You can call `label()` multiple times to create an array of values under the given name. **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.label("microservice", "UI"); // ... }); ``` **Metadata API:** ```ts import { test } from "vitest"; test("Test Authentication @allure.label.microservice:UI", async () => { // ... }); ``` ### ID - `@allure.id:⟨VALUE⟩` Set the test's [ID](/docs/v3/readability/#id). ```ts import { test } from "vitest"; test("Test Authentication @allure.id:123", async () => { // ... }); ``` ### Link - `allure.link(url: string, name?: string, type?: LinkType | string): PromiseLike` - `allure.links(...linksList: Link[]): PromiseLike` - `allure.issue(url: string, name?: string): PromiseLike` - `allure.tms(url: string, name?: string): PromiseLike` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string, defaults to “link”), Allure will try to load a corresponding **link template** to process the URL, as defined by the [`links`](/docs/vitest-configuration/#links) configuration option. If no template is found for the given type, the URL is left unmodified. If the URL does not start with “http” or “https”, it will be processed according to the [`links`](/docs/vitest-configuration/#links) configuration option. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `issue` and `tms`. ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.issue("AUTH-123", "Related issue"); await allure.tms("TMS-456", "Related TMS issue"); await allure.link("JIRA-777", "Related Jira issue", "jira"); await allure.link("https://example.com/", "Project website"); // ... }); ``` ## Behavior-based hierarchy - `allure.epic(name: string): PromiseLike` - `allure.feature(name: string): PromiseLike` - `allure.story(name: string): PromiseLike` - `@allure.label.epic:⟨VALUE⟩` - `@allure.label.feature:⟨VALUE⟩` - `@allure.label.story:⟨VALUE⟩` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.epic("Web interface"); await allure.feature("Essential features"); await allure.story("Authentication"); // ... }); ``` **Metadata API:** ```ts import { test } from "vitest"; test( "Test Authentication" + " @allure.label.epic:WebInterface" + " @allure.label.feature:EssentialFeatures" + " @allure.label.story:Authentication", async () => { // ... }, ); ``` ## Suite-based hierarchy - `allure.parentSuite(name: string): PromiseLike` - `allure.suite(name: string): PromiseLike` - `allure.subSuite(name: string): PromiseLike` - `@allure.label.parentSuite:⟨VALUE⟩` - `@allure.label.suite:⟨VALUE⟩` - `@allure.label.subSuite:⟨VALUE⟩` Assign the names of **parent suite**, **suite** or **sub-suite** for a test, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). This overrides the corresponding levels of the default suites hierarchy, which Allure Vitest creates based on the `describe()` arguments. **Runtime API:** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.parentSuite("Tests for web interface"); await allure.suite("Tests for essential features"); await allure.subSuite("Tests for authentication"); // ... }); ``` **Metadata API:** ```ts import { test } from "vitest"; test( "Test Authentication" + " @allure.label.parentSuite:TestsForWebInterface" + " @allure.label.suite:TestsForEssentialFeatures" + " @allure.label.subSuite:TestsForAuthentication", async () => { // ... }, ); ``` ## Test steps - `allure.step(name: string, body: (context: StepContext) => T | PromiseLike): PromiseLike` - `allure.logStep(name: string, status?: Status, error?: Error): PromiseLike` Define a [test step or sub-step](/docs/steps/) with the given `name`. The `step()` function is asynchronous, and it accepts an asynchronous anonymous function as its second argument. Notice the `async` and `await` keywords in the example below. The anonymous function can accept either no arguments or a single argument of class `StepContext`. This object provides the following methods: - `displayName()` — override the step name during its execution. - `parameter()` — indicate arbitrary parameters used for the step. The available signatures of this method are similar to the test-wide implementation, see [Parametrized tests](#parametrized-tests). To create a step without a body, call the `logStep()` function that accepts a name and an optional step status. ```ts import * as allure from "allure-js-commons"; import { Status } from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { await allure.step("Step 1", async () => { await allure.step("Sub-step 1", async (ctx) => { await ctx.parameter("foo", "1"); // ... }); await allure.step("Sub-step 2", async (ctx) => { await ctx.parameter("foo", "2"); // ... }); }); await allure.logStep("Step 2", Status.SKIPPED); }); ``` ## Parametrized tests - `allure.parameter(name: string, value: string, options?: ParameterOptions): PromiseLike` The [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern in Vitest can be implemented by calling the [`test.each()`](https://vitest.dev/api/#test-each) function or even just by running a simple [`test()`](https://vitest.dev/api/#test) inside a loop. In both cases, Allure Report recognizes each iteration as a separate test run. The values that distinguish one iteration from another are called test parameters. To display a parameter value in the test report, pass it to the `parameter()` function. The `options` argument, if given, must be an object with two optional properties `excluded` and `mode`. - If `excluded` is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. See [Common pitfall: a test's retries are displayed as separate tests](/docs/history-and-retries/#common-pitfall-a-tests-retries-are-displayed-as-separate-tests). - The `mode` affects how the parameter will be displayed in the report. Available options are: - `"default"` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `"masked"` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `"hidden"` — the parameter and its value will not be shown in the test report. Note, that even when you use the `"masked"` or `"hidden"` mode, it is still possible to extract the value from the `allure_results` directory if you publish it. **Using test():** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; for (const login of ["johndoe", "johndoe@example.com"]) { test(`Test Authentication as ${login}`, async () => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); } ``` **Using test.each():** ```ts import * as allure from "allure-js-commons"; import { test } from "vitest"; test.each(["johndoe", "johndoe@example.com"])("Test Authentication as %s", async (login) => { await allure.parameter("login", login); await allure.parameter("time", new Date().toUTCString(), { excluded: true }); // ... }); ``` ## Attachments - `allure.attachment(name: string, content: Buffer | string, options: ContentType | string | AttachmentOptions): PromiseLike` - `allure.attachmentPath(name: string, path: string, options: ContentType | string | Omit): PromiseLike` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Pass either the `content` or the `path` from which the data will be read. The `options` argument controls the [media type](https://en.wikipedia.org/wiki/Media_type) of the content and the filename extension that will be used if a user downloads the attachment from the test report. You can either specify both options in an object (as shown for the image attachment below) or just specify the media type and let Allure deduce the appropriate filename extension automatically (as shown for the text attachment below). In either case, the media type can be a value from the `ContentType` enumeration or any string. ```ts import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import { test } from "vitest"; test("Test Authentication", async () => { // ... await allure.attachment("Text file", "This is the file content.", ContentType.TEXT); await allure.attachmentPath("Screenshot", "/path/to/image.png", { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` When using [Vitest Browser Mode](/docs/vitest/#using-browser-mode), you can capture a screenshot of the browser page and attach it via `attachmentPath()`. Note that `page.screenshot()` from `@vitest/browser/context` saves the file to disk and returns the path — not the raw image bytes — so `attachmentPath()` is the correct function to use here: ```ts import * as allure from "allure-js-commons"; import { ContentType } from "allure-js-commons"; import { test } from "vitest"; import { page } from "vitest/browser"; test("Renders the login form", async () => { // ... const screenshotPath = await page.screenshot(); await allure.attachmentPath("Page screenshot", screenshotPath, { contentType: ContentType.PNG, fileExtension: "png", }); }); ``` --- ## WebdriverIO > URL: https://allurereport.org/docs/webdriverio/ # Getting started with Allure WebdriverIO [![Allure WebdriverIO npm latest version](https://img.shields.io/npm/v/@wdio/allure-reporter?style=flat "Allure WebdriverIO npm latest version")](https://www.npmjs.com/package/@wdio/allure-reporter) Generate beautiful HTML reports using [Allure Report](https://allurereport.org/docs/) and your [WebdriverIO](https://webdriver.io/) tests. ## Setting up ### 1. Prepare your project 1. Open a terminal and go to the project directory. For example: ```bash cd /home/user/myproject ``` 1. Make sure Allure Report is installed. If it's not, follow the [installation instructions](/docs/v3/install/). Alternatively, install Allure Report 3 as a local npm package: ```bash npm install --save-dev allure ``` 1. Install the Allure WebdriverIO integration. **npm:** ```bash npm install --save-dev @wdio/allure-reporter ``` **yarn:** ```bash yarn add --dev @wdio/allure-reporter ``` **pnpm:** ```bash pnpm add -D @wdio/allure-reporter ``` 1. In the `wdio.conf.ts` file, add `['allure', {}]` to the list of reporters. Inside `{}`, you can specify the [Allure WebdriverIO configuration options](/docs/webdriverio-configuration/), if necessary. For example: ```ts export const config = { // ... reporters: [ "spec", [ "allure", { outputDir: "allure-results", }, ], ], }; ``` ### 2. Run tests Run your WebdriverIO tests same way as your would run them usually. For example: **npm:** ```bash npm run wdio ``` **yarn:** ```bash yarn run wdio ``` **pnpm:** ```bash pnpm run wdio ``` This will save necessary data into `allure-results` or other directory, according to the [Configuration](/docs/webdriverio-configuration/). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 3. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure WebdriverIO integration extends the standard reporting features of WebdriverIO by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#adding-metadata). - **Test Organization**: Structure your tests into clear hierarchies for better readability and organization [organize tests](#organize-tests). - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. ### Adding Metadata Allure allows you to enrich your reports with a variety of [metadata](/docs/v3/readability/#description-links-and-other-metadata). This additional information provides context and details for each test, enhancing the report's usefulness. Refer to the [metadata reference section](/docs/webdriverio-reference/#metadata) for an exhaustive list of what can be added. To assign a metadata field, call a corresponding method at any point inside a test method's body. Note, however, that it is highly recommended to assign all metadata as early as possible. Otherwise, there is a risk of the test failing before having all metadata set, which is bad for the test report's readability. ```ts import { addDescription, addIssue, addLink, addOwner, addSeverity, addTestId, TYPE, } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addDescription("Attempt to log into the website.", TYPE.MARKDOWN); addSeverity("critical"); addOwner("John Doe"); addLink("https://dev.example.com/", "Website"); addIssue("AUTH-123"); addTestId("TMS-456"); // ... }); ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): ```ts import { addEpic, addFeature, addStory } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addEpic("Web interface"); addFeature("Essential features"); addStory("Authentication"); // ... }); ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): ```ts import { addParentSuite, addSubSuite, addSuite } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addParentSuite("Tests for web interface"); addSuite("Tests for essential features"); addSubSuite("Tests for authentication"); // ... }); ``` ### Divide a test into steps Allure WebdriverIO provides three ways of [creating steps and sub-steps](/docs/steps/): “lambda steps”, “noop steps”, and “low-level steps”, see the [reference](/docs/webdriverio-reference/#test-steps). **Lambda steps:** ```ts import { step } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { await step("Step 1", async (step) => { await step.step("Step 1.1", async () => { // ... }); await step.step("Step 1.2", async () => { // ... }); }); await step("Step 2", async (step) => { await step.step("Step 2.1", async () => { // ... }); await step.step("Step 2.2", async () => { // ... }); }); }); ``` **No-op steps:** ```ts import { addStep } from "@wdio/allure-reporter"; import { Status } from "allure-js-commons"; it("Test Authentication"), async () => { addStep("Successful step"); addStep("Skipped step", undefined, Status.SKIPPED); addStep( "Skipped step with attachment", { content: "This is attachment.", name: "file.txt", type: "text/plain" }, Status.SKIPPED, ); }); ``` **Low-level steps:** ```ts import { endStep, startStep } from "@wdio/allure-reporter"; import { Status } from "allure-js-commons"; it("Test Authentication"), async () => { startStep("Step 1"); try { // ... endStep(); } catch { endStep(Status.FAILED); } startStep("Step 2"); try { // ... endStep(); } catch { endStep(Status.FAILED); } }); ``` ### Describe parametrized tests Since tests in WebdriverIO, unlike in some other frameworks, are written as anonymous functions, it is very easy to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, i.e. to run the same test logic with different test data. To do so, just write the test inside a loop and use the variable parameters in both its title and its body. To display a parameter value in the test report, pass it to the `addArgument()` function. ```ts import { addArgument } from "@wdio/allure-reporter"; for (const login of ["johndoe", "johndoe@example.com"]) { it(`Test Authentication as ${login}`, async () => { addArgument("login", login); // ... }); } ``` ### Attach screenshots and other files In Allure reports, you have the ability to [attach various types of files](/docs/attachments/), which can greatly enhance the comprehensibility of the report. A common practice is to attach screenshots that capture the state of the user interface at specific moments during test execution. For detailed instructions on how to implement attachments, refer to the [attachments section in the Allure WebdriverIO reference](/docs/webdriverio-reference/#attachments). ```ts import { addAttachment } from "@wdio/allure-reporter"; import * as fs from "fs"; it("Test Authentication"), async () => { // ... addAttachment("Text", "This is the file content.", "text/plain"); addAttachment("Screenshot", fs.readFileSync("/path/to/image.png"), "image/png"); }); ``` ### Select tests via a test plan file Danger: Test plan is currently not supported by the Allure WebdriverIO integration. ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and Node.js version retrieved from the [`os`](https://nodejs.org/api/os.html) and [`process`](https://nodejs.org/api/process.html) objects. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/js/environment-allure3-light.png, /images/js/environment-allure3-dark.png, /images/js/environment-allure2-light.png, /images/js/environment-allure2-dark.png ```ts import * as os from "os"; export const config = { // ... reporters: [ "spec", [ "allure", { reportedEnvironmentVars: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, }, ], ], }; ``` Note that if your launch includes multiple WebdriverIO runs (see [How it works](/docs/how-it-works/)), Allure WebdriverIO will only save the environment information from the latest run. --- ## WebdriverIO configuration > URL: https://allurereport.org/docs/webdriverio-configuration/ # Allure WebdriverIO configuration The [Allure WebdriverIO](/docs/webdriverio/) integration behavior can be affected by some configuration options set in your project's [`wdio.conf.js`](https://webdriver.io/docs/configurationfile/) file. To set the options, define an object after the `"allure"` string in the `reporters` configuration section. For example: ```ts import * as os from "os"; export const config = { // ... reporters: [ "spec", [ "allure", { outputDir: "allure-results", issueLinkTemplate: "https://issues.example.com/{}", tmsLinkTemplate: "https://tms.example.com/{}", reportedEnvironmentVars: { os_platform: os.platform(), os_release: os.release(), os_version: os.version(), node_version: process.version, }, globalLabels: [{ name: "layer", value: "api" }], }, ], ], }; ``` ## Common parameters ### issueLinkTemplate, tmsLinkTemplate Define templates that can be used to construct full URLs from short identifiers, see the [reference](/docs/webdriverio-reference/#link). The pattern must contain `{}` at the position where the identifier should be placed. ### outputDir Path to the directory where Allure WebdriverIO will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. ### reportedEnvironmentVars Key-value pairs that will be displayed on the report's main page, see [Environment information](/docs/v3/readability/#environment-information). ### globalLabels Labels applied to every test result in the run. Useful for tagging all results with a layer, team, or environment identifier without repeating it in every test. Can be provided as an array of `{ name, value }` label objects: ```ts export const config = { reporters: [ [ "allure", { globalLabels: [ { name: "layer", value: "api" }, { name: "team", value: "backend" }, ], }, ], ], }; ``` Or as a plain object mapping label names to values. A name can map to a single string or an array of strings: ```ts export const config = { reporters: [ [ "allure", { globalLabels: { layer: "api", team: "backend", }, }, ], ], }; ``` ## Steps and attachments ### addConsoleLogs If `true`, Allure WebdriverIO will automatically create an [attachment](/docs/webdriverio-reference/#attachments) containing all the console output produced by the test, i.e., all the messages passed to `console.log()`. ### disableWebdriverScreenshotsReporting If `true`, Allure WebdriverIO will **not** automatically create an attachment with each screenshot taken via the WebdriverIO's [`saveScreenshot()`](https://webdriver.io/docs/api/element/saveScreenshot/) function. Note that you can always add screenshots manually by passing their file paths to Allure's [`attach()`](/docs/webdriverio-reference/#attachments). ### disableWebdriverStepsReporting If `true`, Allure WebdriverIO will **not** automatically create [steps](/docs/webdriverio-reference/#steps) for each WebDriver command. ## Runner-specific parameters ### disableMochaHooks This option is only relevant if you use the [Mocha test runner](https://webdriver.io/docs/frameworks#using-mocha). If `true`, Allure WebdriverIO will not add Mocha hooks to the test report. ### useCucumberStepReporter This option is only relevant if you use the [Cucumber.js test runner](https://webdriver.io/docs/frameworks#using-cucumber). - If `false` (or omitted), each Gherkin step will be displayed as a separate test. Allure will group together steps from the same Gherkin feature and scenario by default. You can move individual steps to other locations in the hierarchies by calling the corresponding functions, see [Behavior-based hierarchy](/docs/webdriverio-reference/#behavior-based-hierarchy) and [Suite-based hierarchy](/docs/webdriverio-reference/#suite-based-hierarchy). Note that if you use [`Scenario Outline`](https://cucumber.io/docs/gherkin/reference/#scenario-outline) to perform the same steps with different values, this may result in multiple steps with identical names in the test report. It is recommended to [sort tests by order of execution](/docs/sorting-and-filtering/#sort-tests) when viewing such a report. Also note that with this option, Allure WebdriverIO does not support setting [issue links and TMS links](/docs/webdriverio-reference/#link) via Gherkin tags. - If `true`, Allure will treat each Gherkin scenario as a test, with Gherkin steps displayed as test steps. If your step definition calls the [`allureReporter.step()`](/docs/webdriverio-reference/#test-steps) function, it will create a sub-step within the current step. Note that if you use [`Scenario Outline`](https://cucumber.io/docs/gherkin/reference/#scenario-outline) to perform the same steps with different values, Allure WebdriverIO will not be able to report it properly with this option turned on. Instead, the test report will only display the steps executed with the last set of values. --- ## WebdriverIO reference > URL: https://allurereport.org/docs/webdriverio-reference/ # Allure WebdriverIO reference These are the functions that you can use to integrate your WebdriverIO tests with Allure. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Description - `addDescription(name: string, descriptionType: string)` Set the test's [description](/docs/v3/readability/#description). The second argument determines how the text will be rendered. If it is equal to `TYPE.MARKDOWN`, the text will be interpreted using the Markdown syntax. Any HTML formatting, if present, will be stripped for security purposes. To keep the HTML intact, pass `TYPE.HTML` to the second argument. Info: Using HTML descriptions is unsafe and can potentially break the test report's layout or even create an XSS attack vector for your organization. ```ts import { addDescription, TYPE } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addDescription("Attempt to log into the website.", TYPE.MARKDOWN); // ... }); ``` ### Owner - `addOwner(owner: string)` Set the test's [owner](/docs/v3/readability/#owner). ```ts import { addOwner } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addOwner("John Doe"); // ... }); ``` ### Tag - `addTag(tag: string)` Set the test's [tags](/docs/v3/readability/#tags). ```ts import { addTag } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addTag("web interface"); addTag("authentication"); // ... }); ``` ### Severity - `addSeverity(severity: string)` Set the test's [severity](/docs/v3/readability/#severity). Allowed values are: “trivial”, “minor”, “normal”, “critical”, and “blocker”. ```ts import { addSeverity } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addSeverity("critical"); // ... }); ``` ### Label - `addLabel(name: string, value: string)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other functions. The first argument of the function is the label name. It can be any string. You can call `addLabel()` multiple times to create an array of values under the given name. ```ts import { addLabel } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addLabel("my custom label", "value"); // ... }); ``` ### ID - `addAllureId(allureId: string)` Set the test's [ID](/docs/v3/readability/#id). ```ts import { addAllureId } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addAllureId("123"); // ... }); ``` ### Link - `addLink(url: string, name?: string, type?: string)` - `addIssue(issue: string)` - `addTestId(testId: string)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`issueLinkTemplate` and `tmsLinkTemplate`](/docs/webdriverio-configuration/#issuelinktemplate-tmslinktemplate) configuration options. If no pattern found for the given type, the link will not be added. The `name` will be used as the link's text. If it is omitted, the URL will be used instead. For convenience, Allure provides two shorthand functions with pre-selected link types: `addIssue()` and `addTestId()`. ```ts import { addIssue, addLink, addTestId } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addLink("https://dev.example.com/", "Website"); addIssue("AUTH-123"); addTestId("TMS-456"); // ... }); ``` If you use the Cucumber.js runner and you have the [`useCucumberStepReporter`](/docs/webdriverio-configuration/#usecucumberstepreporter) option enabled, you can also add links via the `@issue` and `@testId` Gherkin tags. ```gherkin Feature: Labels @issue=AUTH-123 @testId=TMS-456 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello" ``` ## Behavior-based hierarchy - `addEpic(epicName: string)` - `addFeature(featureName: string)` - `addStory(storyName: string)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). ```ts import { addEpic, addFeature, addStory } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addEpic("Web interface"); addFeature("Essential features"); addStory("Authentication"); // ... }); ``` Tip: If you are using the Cucumber.js runner, Allure WebdriverIO uses the hierarchy from the Gherkin files by default. The exact behavior depends on the [`useCucumberStepReporter`](/docs/webdriverio-configuration/#usecucumberstepreporter) option. ## Suite-based hierarchy - `addParentSuite(suiteName: string)` - `addSuite(suiteName: string)` - `addSubSuite(suiteName: string)` Assign the name of suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). ```ts import { addParentSuite, addSubSuite, addSuite } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { addParentSuite("Tests for web interface"); addSuite("Tests for essential features"); addSubSuite("Tests for authentication"); // ... }); ``` Tip: If you are using the Cucumber.js runner, Allure WebdriverIO uses the hierarchy from the Gherkin files by default. The exact behavior depends on the [`useCucumberStepReporter`](/docs/webdriverio-configuration/#usecucumberstepreporter) option. ## Test steps - `step(name: string, body: StepBodyFunction): Promise` - `addStep(title: string, { content, name, type }?: any, status?: Status)` - `startStep(title: string)` - `endStep(status?: Status)` Define [test steps](/docs/steps/). There are three ways of defining a step. - **Lambda steps** Write a test step in an asynchronous lambda function and pass it to `allureReporter.step()`. The lambda function must accept either no arguments or a single argument of the `AllureCommandStepExecutable` type. You can create sub-steps within the current step by calling this argument's `step()` method. - **No-op steps** If you call `allureReporter.addStep()`, Allure will add to the report a no-op step. This allows for a log-style reporting within a test or within a larger step. A no-op step finishes immediately after it started and cannot have any sub-steps, or parameters. The optional second argument indicates the status that will be shown for the step in the report. Allowed values are: `Status.PASSED` (the default), `Status.FAILED`, `Status.BROKEN`, and `Status.SKIPPED`. - **Low-level steps** Call `allureReporter.startStep()` and `allureReporter.endStep()` to manually indicate to Allure WebdriverIO when a step starts or ends. It is recommended to use `try..catch` blocks to ensure ending the steps regardless of the exceptions thrown during their execution, see the example below. The optional argument for `endStep()` indicates the status that will be shown for the step in the report. Allowed values are: `Status.PASSED` (the default), `Status.FAILED`, `Status.BROKEN`, and `Status.SKIPPED`. **Lambda steps:** ```ts import { step } from "@wdio/allure-reporter"; it("Test Authentication"), async () => { await step("Step 1", async (step) => { await step.step("Step 1.1", async () => { // ... }); await step.step("Step 1.2", async () => { // ... }); }); await step("Step 2", async (step) => { await step.step("Step 2.1", async () => { // ... }); await step.step("Step 2.2", async () => { // ... }); }); }); ``` **No-op steps:** ```ts import { addStep } from "@wdio/allure-reporter"; import { Status } from "allure-js-commons"; it("Test Authentication"), async () => { addStep("Successful step"); addStep("Skipped step", undefined, Status.SKIPPED); addStep( "Skipped step with attachment", { content: "This is attachment.", name: "file.txt", type: "text/plain" }, Status.SKIPPED, ); }); ``` **Low-level steps:** ```ts import { endStep, startStep } from "@wdio/allure-reporter"; import { Status } from "allure-js-commons"; it("Test Authentication"), async () => { startStep("Step 1"); try { // ... endStep(); } catch { endStep(Status.FAILED); } startStep("Step 2"); try { // ... endStep(); } catch { endStep(Status.FAILED); } }); ``` Tip: By default, Allure WebdriverIO automatically adds a step for each WebDriver command, with both request details and response data added as step-level [attachments](#attachments). This can be disabled, see [`disableWebdriverStepsReporting`](/docs/webdriverio-configuration/#disablewebdriverstepsreporting). ## Parametrized tests - `allureReporter.addArgument(name: string, value: string)` Since tests in WebdriverIO, unlike in some other frameworks, are written as anonymous functions, it is very easy to implement the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, i.e. to run the same test logic with different test data. To do so, just write the test inside a loop and use the variable parameters in both its title and its body. To display a parameter value in the test report, pass it to the `addArgument()` function. ```ts import { addArgument } from "@wdio/allure-reporter"; for (const login of ["johndoe", "johndoe@example.com"]) { it(`Test Authentication as ${login}`, async () => { addArgument("login", login); // ... }); } ``` ## Attachments - `addAttachment(name: string, content: string | Buffer | object, type: string)` Add `content` as an attachment to the test result. The content can be a text string or a base64-encoded [`Buffer`](https://nodejs.org/api/buffer.html#buffer_class_buffer). As the `type` argument, pass the media type of content. You can use data produced by any function, not necessarily read from an actual file. ```ts import { addAttachment } from "@wdio/allure-reporter"; import * as fs from "fs"; it("Test Authentication"), async () => { // ... addAttachment("Text", "This is the file content.", "text/plain"); addAttachment("Screenshot", fs.readFileSync("/path/to/image.png"), "image/png"); }); ``` Tip: By default, Allure WebdriverIO automatically includes in the report all screenshots taken via the WebdriverIO's [`saveScreenshot()`](https://webdriver.io/docs/api/element/saveScreenshot/) function. This can be disabled, see the [`disableWebdriverScreenshotsReporting`](/docs/webdriverio-configuration/#disablewebdriverscreenshotsreporting) configuration option. --- ## xUnit.net > URL: https://allurereport.org/docs/xunit/ # Getting started with Allure xUnit.net [![Allure xUnit.net latest version](https://img.shields.io/nuget/v/Allure.Xunit?style=flat "Allure xUnit.net latest version")](https://www.nuget.org/packages/Allure.Xunit) Generate beautiful HTML reports using [Allure Report](/docs/) and your [xUnit.net](https://xunit.net/) tests. Info: Check out the example project at [github.com/allure-examples/allure-xunit](https://github.com/allure-examples/allure-xunit) to see Allure xUnit.net in action. ## Setting up ### 1. Check the prerequisites 1. Make sure the Allure Report command-line tool is installed. If it's not, see the [installation instructions](/docs/v3/install/). 1. Check the .NET SDK against which your test project is built. Allure xUnit.net supports the following SDKs: - .NET Core 3.1 - .NET 5.0 and later Visit the [official documentation](https://learn.microsoft.com/en-us/dotnet/core/install/) for more info about installing or upgrading .NET SDK. 1. Make sure your test project references a compatible version of xUnit.net. Currently, we support xUnit.net v2 starting from 2.4.1. The support for xUnit.net v3 will be added in the future (you may track this task in this [issue](https://github.com/allure-framework/allure-csharp/issues/554)). Info: Allure xUnit.net can't work with version 3 of [`xunit.runner.visualstudio`](https://www.nuget.org/packages/xunit.runner.visualstudio) due to changes in how the runner discovers custom reporters. You should downgrade to [version 2.8.2](https://www.nuget.org/packages/xunit.runner.visualstudio/2.8.2) or earlier to keep using it with Allure xUnit.net. ### 2. Prepare your project 1. Add `Allure.Xunit` to your project's dependencies using your IDE or the command line. For example, here is how the dependency can be added to a project with the `dotnet` CLI: ```plain dotnet add ⟨PATH TO PROJECT⟩ package Allure.Xunit ``` 1. Explicitly select `allure` as the xUnit.net runner reporter. This step is only necessary if you have two or more “environmentally enabled” xUnit.net reporters, which means that the test runner may or may not use the Allure xUnit.net reporter each time by default. This is often the case when running tests on a CI platform, such as Microsoft Azure or TeamCity, because a reporter that sends test results to the CI platform might be enabled in addition to Allure xUnit.net in such environment. The way to select the reporter depends on the test runner that you use in the project. For example, if you use xUnit.net's default `xunit.runner.visualstudio` runner, you need to modify the [`ReporterSwitch`](https://xunit.net/docs/runsettings#ReporterSwitch) setting: **In the .runsettings file:** ```xml allure ``` **In the “dotnet test” arguments:** ```plain dotnet test -- RunConfiguration.ReporterSwitch=allure ``` Tip: By default, Allure xUnit.net will also try to run another environmentally enabled reporter, if it is present. This behavior can be changed using the [`xunitRunnerReporter`](/docs/xunit-configuration/#allure-xunitrunnerreporter) configuration option. ### 3. Run tests Run your xUnit.net tests the same way as you would run them usually. For example: ```plain dotnet test ``` This will save necessary data into `allure-results` or other directory, according to the [`allure.directory` setting](/docs/xunit-configuration/#allure-directory). If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all. ### 4. Generate a report Finally, convert the test results into an HTML report. This can be done by one of two commands: - `allure generate` processes the test results and saves an HTML report into the `allure-report` directory. To view the report, use the `allure open` command. - `allure serve` creates the same report as `allure generate`, then automatically opens the main page of the report in a web browser. ## Writing tests The Allure xUnit.net integration extends the standard reporting features of xUnit.net by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized: - **Metadata Annotation**: Enhance test reports with [descriptions, links, and other metadata](#specify-description-links-and-other-metadata). - **Test Organization**: Structure your tests into [clear hierarchies](#organize-tests) for better readability and organization. - **Test statuses distinction**: Distinguish between [failed assertions and unhandled exceptions](#distinguish-between-test-statuses) with failed and broken statuses. - **Step Division**: Break down tests into smaller [test steps](#divide-a-test-into-steps) for easier understanding and maintenance. - **Constructors and Dispose methods**: Include the [initialization and finalization of shared contexts](#describe-constructors-and-dispose-methods) into the report. - **Parametrized Tests**: Clearly describe the parameters for [parametrized tests](#describe-parametrized-tests) to specify different scenarios. - **Attachments**: Automatically capture [screenshots and other files](#attach-screenshots-and-other-files) during test execution. - **Test Selection**: Use a test plan file to [select which tests to run](#select-tests-via-a-test-plan-file), allowing for flexible test execution. - **Environment Details**: Include comprehensive [environment information](#environment-information) to accompany the test report. In most cases, Allure xUnit.net provides two different ways to use a feature: the Attributes API and the Runtime API. - **Attributes API**: add a C# attribute to a test method or a whole class to add certain data to the test result. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ### Specify description, links and other metadata There is a lot of [metadata](/docs/v3/readability/#description-links-and-other-metadata) you can add to each test so that it would appear in the report. See the [reference](/docs/xunit-reference/#metadata) for more details. **Attributes API:** ```csharp using Allure.Net.Commons; using Allure.Xunit.Attributes; using Xunit; public class TestLabels { [Fact] [AllureSeverity(SeverityLevel.critical)] [AllureOwner("John Doe")] [AllureLink("Website", "https://dev.example.com/")] [AllureIssue("UI-123")] [AllureTms("TMS-456")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.SetSeverity(SeverityLevel.critical); AllureApi.SetOwner("John Doe"); AllureApi.AddLink("Website", "https://dev.example.com/"); AllureApi.AddIssue("UI-123"); AllureApi.AddTmsItem("TMS-456"); // ... } } ``` ### Organize tests As described in [Improving navigation in your test report](/docs/v3/navigation/), Allure supports multiple ways to organize tests into hierarchical structures. Allure xUnit.net provides the API to assign the relevant fields to tests either by adding attributes or “dynamically” (same as for the [metadata fields](#specify-description-links-and-other-metadata)). To specify a test's location in the [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy): **Attributes API:** ```csharp using Allure.Xunit.Attributes; using Xunit; [AllureEpic("Web interface")] [AllureFeature("Essential features")] public class TestLabels { [Fact] [AllureStory("Labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.AddEpic("Web interface"); AllureApi.AddFeature("Essential features"); AllureApi.AddStory("Labels"); // ... } } ``` To specify a test's location in the [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy): **Attributes API:** ```csharp using Allure.Xunit.Attributes; using Xunit; [AllureParentSuite("Web interface")] [AllureSuite("Essential features")] public class TestLabels { [Fact] [AllureSubSuite("Labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.AddParentSuite("Web interface"); AllureApi.AddSuite("Essential features"); AllureApi.AddSubSuite("Labels"); // ... } } ``` A test's location in the [package-based hierarchy](/docs/v3/navigation/#package-based-hierarchy) is defined by the fully qualified names of the classes they are declared in, with common prefixes shown as parent packages. ### Distinguish between test statuses When a test throws an exception, Allure xUnit.net reflects that by assigning an appropriate [test status](/docs/test-statuses/): - If the exception class belongs to the `Xunit.Sdk` namespace, the test is considered **Failed**. - With any other exception, the test is considered **Broken**. If you want your test with unhandled exceptions to be **Failed** instead of **Broken**, there are multiple approaches to achieve that. For example, in an assertion (e.g., in `Assert.Equal`), make sure that evaluating the arguments (e.g., accessing an element of a list) cannot cause an exception. Alternatively, consider wrapping the code that may throw in a `try..catch` block and fail the test explicitly. **Failed test:** ```csharp using Xunit; public class TestTryCatch { [Fact] public void ThisTestWillBeFailed() { string value = GetValueOrFail(); Assert.Equal("foo", value); } static string GetValueOrFail() { string value = default; try { value = GetValue(); } catch (Exception e) { Assert.Fail(e.Message); } return value; } static string GetValue() { throw new Exception(); } } ``` **Broken test:** ```csharp using Xunit; public class TestBroken { [Fact] public void ThisTestWillBeBroken() { string value = GetValue(); Assert.Equal("foo", value); } static string GetValue() { throw new Exception(); } } ``` Info: When an exception is thrown from within a [step](/docs/xunit-reference/#test-steps), the step status is selected according to the [`allure.failExceptions`](/docs/xunit-configuration/#allure-failexceptions) setting. ### Divide a test into steps Allure xUnit.net provides three ways of [creating steps and sub-steps](/docs/steps/): “attribute-based steps”, “lambda steps” and “no-op steps”, see the [reference](/docs/xunit-reference/#test-steps). **Attribute-based steps:** ```csharp using Allure.Net.Commons; using Allure.Xunit.Attributes.Steps; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { TestDomain("https://domain1.example.com/"); TestDomain("https://domain2.example.com/"); } [AllureStep("Test {url}")] private void TestDomain([Name("Webpage URL")] string url) { AllureLifecycle.Instance.UpdateStep(stepResult => stepResult.parameters.Add( new Parameter { name = "Started at", value = DateTime.Now.ToString() } ) ); OpenBrowser(); GoToWebpage(url); CloseBrowser(); } [AllureStep("Open web browser")] private void OpenBrowser() { // ... } [AllureStep("Visit {url}")] private void GoToWebpage([Skip] string url) { // ... } [AllureStep("Close web browser")] private void CloseBrowser() { // ... } } ``` **Lambda steps and no-op steps:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { string[] urls = { "https://domain1.example.com/", "https://domain2.example.com/" }; foreach (string url in urls) { AllureApi.Step($"Test {url}", () => { AllureLifecycle.Instance.UpdateStep(stepResult => { stepResult.parameters.Add( new Parameter { name = "Webpage URL", value = url } ); }); AllureApi.Step("Opening web browser..."); // ... AllureApi.Step($"Visiting {url}..."); // ... AllureApi.Step("Closing web browser..."); // ... }); } } } ``` ### Describe constructors and Dispose methods By default, a test report does not include information about [constructors and Dispose methods](https://xunit.net/docs/shared-context#constructor) in xUnit.net test classes. To include it, add the `[AllureBefore]` and `[AllureAfter]` annotations to the methods that setup and clean a test context. The methods will be displayed as special kinds of [steps](#divide-a-test-into-steps). ```csharp using Allure.Xunit.Attributes.Steps; using Xunit; public class TestLabels : IDisposable { [AllureBefore("Setup test context")] public TestLabels() { // ... } [Fact] public void TestCreateLabel() { // ... } [AllureAfter("Clean test context")] public void Dispose() { // ... } } ``` ### Describe parametrized tests When using the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, use `[Theory]` and other xUnit.net's attributes or or the `AddTestParameter()` function, see the [reference](/docs/nunit/#parametrized-tests). **Attributes API:** ```csharp using Xunit; public class TestLabels { [Theory] [InlineData("johndoe", "qwerty")] [InlineData("johndoe@example.com", "qwerty")] public void TestCreateLabel(string login, string password) { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestAuthenticationWithUsername() { AllureApi.AddTestParameter("login", "johndoe"); AllureApi.AddTestParameter("password", "qwerty", ParameterMode.Masked); // ... } [Fact] public void TestAuthenticationWithEmail() { AllureApi.AddTestParameter("login", "johndoe@example.com"); AllureApi.AddTestParameter("password", "qwerty", ParameterMode.Masked); // ... } } ``` ### Attach screenshots and other files You can [attach any sorts of files](/docs/attachments/) to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. Allure xUnit.net provides various ways to create an attachment, both from existing files or generated dynamically, see the [reference](/docs/xunit-reference/#attachments). **Runtime API:** ```csharp using System.IO; using System.Text; using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { // ... AllureApi.AddAttachment( "data.txt", "text/plain", Encoding.UTF8.GetBytes("This is the file content.") ); AllureApi.AddAttachment( "image1.png", "image/png", File.ReadAllBytes("/path/to/image1.png") ); AllureApi.AddAttachment( "image2.png", "image/png", "/path/to/image2.png" ); } } ``` ### Select tests via a test plan file If the `ALLURE_TESTPLAN_PATH` environment variable is defined and points to an existing file, xUnit.net will only run tests listed in this file. Here's an example of running tests according to a file named `testplan.json`: **MacOS/Linux:** ```bash export ALLURE_TESTPLAN_PATH=testplan.json dotnet test ``` **Windows:** ```powershell $Env:ALLURE_TESTPLAN_PATH = "testplan.json" dotnet test ``` ### Environment information For the main page of the report, you can collect various information about the environment in which the tests were executed. For example, it is a good idea to use this to remember the OS version and .NET version. This may help the future reader investigate bugs that are reproducible only in some environments. Images: /images/csharp/environment-allure3-light.png, /images/csharp/environment-allure3-dark.png, /images/csharp/environment-allure2-light.png, /images/csharp/environment-allure2-dark.png To provide environment information, put a file named `environment.properties` into the `allure-results` directory after running the tests. See the example in [Environment file](/docs/how-it-works-environment-file/). Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using [Parametrized tests](#describe-parametrized-tests). --- ## xUnit.net configuration > URL: https://allurereport.org/docs/xunit-configuration/ # Allure xUnit.net configuration This page describes the configuration options that affect the behavior of [Allure xUnit.net](/docs/xunit/). By default, Allure xUnit.net tries to read the `allureConfig.json` file in the same directory where the test assembly is located. To make sure the file is present there, edit the file properties in your IDE or manually in your `*.csproj` file and set the corresponding item's `CopyToOutputDirectory` to either `Always` or `PreserveNewest`. For example: ```xml Always ``` Alternatively, use the `ALLURE_CONFIG` environment variable to specify a relative or absolute path of the configuration file. ## allure.directory Path to the directory where Allure xUnit.net will save the test results, see [How it works](/docs/how-it-works/). If the directory does not exist, it will be created. Defaults to `allure-results`. Note that the path is treated as relative to the build output directory of the project. When building and running tests from an IDE, this is typically different from the solution folder. Multiple `../` elements may be needed if you want to put the test results into a directory in the solution folder. ```json { "allure": { "directory": "../../../../../allure-results" } } ``` ## allure.links List of patterns that can be used to construct full URLs from short identifiers. Each pattern must be a string containing a placeholder surrounded by curly brackets, for example: `{issue}`. When processing a link, Allure xUnit.net selects the pattern whose placeholder matches the link type (as specified using the [`Link` or `[AllureLink]`](/docs/xunit-reference/#link) arguments). The placeholder is then replaced by the short identifier. For example, with the configuration below, the link `BUG-123` of type `issue` will be translated to `https://issues.example.com/BUG-123`. ```json { "allure": { "links": [ "https://issues.example.com/{issue}", "https://tms.example.com/{tms}", "https://jira.example.com/browse/{jira}" ] } } ``` ## allure.xunitRunnerReporter The secondary reporter that should be used alongside Allure xUnit.net. Allowed values are: - `auto` (the default) — select a secondary reporter automatically, similar to how a test runner selects a reporter. This will only select among the xUnit.net reporters that are “environmentally enabled”. If there are no environmentally enabled reporters other than Allure xUnit.net, this value is identical to `none`. If there is only one environmentally enabled reporter other than Allure xUnit.net, this value is identical to specifying its reporter switch explicitly. In all other cases, Allure xUnit.net will select one of the environmentally enabled reporters, but the algorithm of this selection is not determined. - `none` — do not use a secondary reporter. - the [reporter switch](https://xunit.net/docs/runsettings#ReporterSwitch) of the reporter, e.g., `teamcity`. - the fully qualified name of the reporter class, e.g. `Xunit.Runner.Reporters.TeamCityReporter, xunit.runner.reporters.netcoreapp10, Version=2.5.2.0, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c`. Note that the fully qualified name is often framework-dependent and changes for each new version of the reporter. Because of that, using the fully qualified name is not recommended for most cases. However, it may be useful for large projects, where selection by the reporter switch would significantly slow down the startup. ```json { "allure": { "xunitRunnerReporter": "teamcity" } } ``` ## allure.failExceptions A list of exception types which, when thrown from a [step](/docs/xunit-reference/#test-steps), should cause it to have the [Failed](/docs/test-statuses/#failed) status. In case of any other exception, the [Broken](/docs/test-statuses/#broken) status is assigned to the step. Notes: Currently, the setting only affects the statuses of individual test steps. For the status of a test as a whole, see [Distinguish between test statuses](/docs/xunit/#distinguish-between-test-statuses). The default list includes the `Xunit.Sdk.XunitException` class and the `Xunit.Sdk.IAssertionException` interface. The setting, if not empty, overrides the default list. When looking for matches, Allure xUnit.net uses not only the exception class's own full name but also all its interfaces and base classes. With the example configuration below, a step will be considered failed if it throws `Xunit.Sdk.IAssertionException`, `MyNamespace.MyCustomException`, or anything that derives from them. At the same time, any other exception will make the step broken. ```json { "allure": { "failExceptions": ["Xunit.Sdk.IAssertionException", "MyNamespace.MyCustomException"] } } ``` --- ## xUnit.net reference > URL: https://allurereport.org/docs/xunit-reference/ # Allure xUnit.net reference These are the attributes and methods that you can use to integrate your xUnit.net tests with Allure Report. In most cases, Allure xUnit.net provides two different ways to use a feature: the Attributes API and the Runtime API. - **Attributes API**: add a C# attribute to a test method or a whole class to add certain data to the test result. When using this approach, the data is guaranteed to be added regardless of how the test itself runs. - **Runtime API**: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically. Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early. ## Metadata Assign a test's [description, links and other metadata](/docs/v3/readability/#description-links-and-other-metadata). ### Title - `[AllureName(string name)]` - `AllureApi.SetTestName(string name)` Set the test's [title](/docs/v3/readability/#title). Unlike most of the functions described here, the `DisplayName` argument is a part of xUnit.net itself. If you need to construct a test's title dynamically, write a lambda function that does that and pass it to the `updateTestCase()` method of the `AllureLifecycle` object, see the example below. **Attributes API:** ```csharp using Xunit; public class TestLabels { [Fact(DisplayName = "Create labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.SetTestName("Create labels"); // ... } } ``` ### Description - `[AllureDescription(string description)]` - `AllureApi.SetDescription(string description)` Set the test's [description](/docs/v3/readability/#description). Markdown formatting is allowed. Any HTML formatting, if present, will be stripped for security purposes. **Attributes API:** ```csharp using Allure.Xunit.Attributes; using Xunit; public class TestLabels { [Fact] [AllureDescription("This test attempts to create a label with specified title")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.SetDescription("This test attempts to create a label with specified title"); // ... } } ``` ### Owner - `[AllureOwner(string value)]` - `AllureApi.SetOwner(string owner)` Set the test's [owner](/docs/v3/readability/#owner). **Attributes API:** ```csharp using Allure.Xunit.Attributes; using Xunit; public class TestLabels { [Fact] [AllureOwner("John Doe")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.SetOwner("John Doe"); // ... } } ``` ### Tag - `[AllureTag(params string[] tags)]` - `AllureApi.AddTags(params string[] tags)` Set the test's [tags](/docs/v3/readability/#tags). **Attributes API:** ```csharp using Allure.Xunit.Attributes; using Xunit; public class TestLabels { [Fact] [AllureTag("UI", "Labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.AddTags("UI", "Labels"); // ... } } ``` ### Severity - `[AllureSeverity(SeverityLevel value)]` - `AllureApi.SetSeverity(SeverityLevel severity)` Set the test's [severity](/docs/v3/readability/#severity). **Attributes API:** ```csharp using Allure.Net.Commons; using Allure.Xunit.Attributes; using Xunit; public class TestLabels { [Fact] [AllureSeverity(SeverityLevel.critical)] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.SetSeverity(SeverityLevel.critical); // ... } } ``` ### Label - `[AllureLabel(string label, string value, bool overwrite=false)]` - `AllureApi.AddLabel(string name, string value)` - `AllureApi.AddLabel(Label label)` - `AllureApi.AddLabels(params Label[] labels)` Set an arbitrary [label](/docs/v3/readability/#other-labels) for the test. This is the underlying implementation for a lot of Allure's other APIs. **Attributes API:** ```csharp using Allure.Xunit.Attributes; using Xunit; public class TestLabels { [Fact] [AllureLabel("MyCustomLabel", "value")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.AddLabel("MyCustomLabel", "value"); // ... } } ``` ### ID - `[AllureId(string value)]` Set the test's [ID](/docs/v3/readability/#id). ```csharp using Allure.Xunit.Attributes; using Xunit; public class TestLabels { [Fact] [AllureId("123")] public void TestCreateLabel() { // ... } } ``` ### Link - `[AllureLink(string name, string url)]` - `[AllureIssue(string name)]` - `[AllureIssue(string name, string url)]` - `[AllureTms(string name)]` - `[AllureTms(string name, string url)]` - `AllureApi.AddLink(string url)` - `AllureApi.AddLink(string name, string url)` - `AllureApi.AddLink(string name, string type, string url)` - `AllureApi.AddLinks(params Link[] links)` - `AllureApi.AddIssue(string url)` - `AllureApi.AddIssue(string name, string url)` - `AllureApi.AddTmsItem(string url)` - `AllureApi.AddTmsItem(string name, string url)` Add a [link](/docs/v3/readability/#links) related to the test. Based on the `type` (which can be any string), Allure will try to load a corresponding **link pattern** to process the URL, as defined by the [`links`](/docs/xunit-configuration/#links) configuration option. If no pattern found for the given type, the URL is left unmodified. The `name` will be used as the link's text. If it is omitted, the unprocessed URL will be used instead. For convenience, Allure provides shorthand attributes and methods with pre-selected link types: `issue` and `tms`. **Attributes API:** ```csharp using Allure.Xunit.Attributes; using Xunit; public class TestLabels { [Fact] [AllureLink("Website", "https://dev.example.com/")] [AllureIssue("UI-123")] [AllureTms("TMS-456")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.AddLink("Website", "https://dev.example.com/"); AllureApi.AddIssue("UI-123"); AllureApi.AddTmsItem("TMS-456"); // ... } } ``` ## Behavior-based hierarchy - `[AllureEpic(string epic)]` - `[AllureFeature(params string[] feature)]` - `[AllureStory(params string[] story)]` - `AllureApi.AddEpic(string epic)` - `AllureApi.AddFeature(string feature)` - `AllureApi.AddStory(string story)` Assign names of **epics**, **features** or **user stories** for a test, as part of Allure's [behavior-based hierarchy](/docs/v3/navigation/#behavior-based-hierarchy). **Attributes API:** ```csharp using Allure.Xunit.Attributes; using Xunit; [AllureEpic("Web interface")] [AllureFeature("Essential features")] public class TestLabels { [Fact] [AllureStory("Labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.AddEpic("Web interface"); AllureApi.AddFeature("Essential features"); AllureApi.AddStory("Labels"); // ... } } ``` ## Suite-based hierarchy - `[AllureParentSuite(string parentSuite)]` - `[AllureSuite(string suite)]` - `[AllureSubSuite(string subSuite)]` - `AllureApi.AddParentSuite(string parentSuite)` - `AllureApi.AddSuite(string suite)` - `AllureApi.AddSubSuite(string subSuite)` Assign the name of suite, as part of Allure's [suite-based hierarchy](/docs/v3/navigation/#suite-based-hierarchy). **Attributes API:** ```csharp using Allure.Xunit.Attributes; using Xunit; [AllureParentSuite("Web interface")] [AllureSuite("Essential features")] public class TestLabels { [Fact] [AllureSubSuite("Labels")] public void TestCreateLabel() { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { AllureApi.AddParentSuite("Web interface"); AllureApi.AddSuite("Essential features"); AllureApi.AddSubSuite("Labels"); // ... } } ``` ## Test steps - `[AllureStep]` - `[AllureStep(string name)]` - `AllureApi.Step(string name)` - `AllureApi.Step(string name, Action acti` - `T AllureApi.Step(string name, Func function)` - `async Task AllureApi.Step(string name, Func action)` - `async Task AllureApi.Step(string name, Func> function)` Define [test steps](/docs/steps/). There are three ways of defining a step. - **Attribute-based steps** Define a method containing a test step and add the `[AllureStep]` attribute to it, with an optional `name` argument (defaults to the function's name). Each time the function is called during the test execution, a new step will be created for the test report. If the function calls another function that also has the `[AllureStep]` attribute, Allure Report will create a sub-step inside the current step. If the function throws an exception, the step becomes **Failed** or **Broken** according to the [`failExceptions`](/docs/xunit-configuration/#allure-failexceptions) configuration parameter. If the step name contains placeholders (e.g., `{url}` or `{0}`), they will be replaced with the values of parameters passed to the function. Arguments passed to the function will be added to the list of **step parameters**. They are rendered the same way as described in [Parametrized tests](#parametrized-tests), but displayed in the current step's subtree in the report. Use the `[Name(string name)]` attribute to customize a parameter's name, as shown in the example below. Use the `[Skip]` attribute to not add a certain parameter to the report. Use `AllureLifecycle.Instance.UpdateStep()` to add custom parameters to the current step, as shown in the example below. - **Lambda steps** Write a test step in a lambda function and pass it to `Step()`. If the lambda function returns a value, `Step()` will return it without modification, and it will not affect the report. If the function throws an exception, the step becomes **Failed** or **Broken** according to the [`failExceptions`](/docs/xunit-configuration/#allure-failexceptions) configuration parameter. Use `AllureLifecycle.Instance.UpdateStep()` to add custom parameters to the current step, as shown in the example below. - **No-op steps** If you call `Step()` with a name but without a lambda function, Allure will add to the report a no-op step. This allows for a log-style reporting within a test or within a larger step. A no-op step finishes immediately after it started and cannot have any sub-steps, parameters or attachments. **Attribute-based steps:** ```csharp using Allure.Net.Commons; using Allure.Xunit.Attributes.Steps; using Xunit; public class TestMyWebsite { [Fact] public void TestVisitPages() { TestDomain("https://domain1.example.com/"); TestDomain("https://domain2.example.com/"); } [AllureStep("Test {url}")] private void TestDomain([Name("Webpage URL")] string url) { AllureLifecycle.Instance.UpdateStep(stepResult => stepResult.parameters.Add( new Parameter { name = "Started at", value = DateTime.Now.ToString() } ) ); OpenBrowser(); GoToWebpage(url); CloseBrowser(); } [AllureStep("Open web browser")] private void OpenBrowser() { // ... } [AllureStep("Visit {url}")] private void GoToWebpage([Skip] string url) { // ... } [AllureStep("Close web browser")] private void CloseBrowser() { // ... } } ``` **Lambda steps and no-op steps:** ```csharp using Allure.Net.Commons; using Xunit; public class TestMyWebsite { [Fact] public void TestVisitPages() { string[] urls = { "https://domain1.example.com/", "https://domain2.example.com/" }; foreach (string url in urls) { AllureApi.Step($"Test {url}", () => { AllureLifecycle.Instance.UpdateStep(stepResult => { stepResult.parameters.Add( new Parameter { name = "Webpage URL", value = url } ); }); AllureApi.Step("Opening web browser..."); // ... AllureApi.Step($"Visiting {url}..."); // ... AllureApi.Step("Closing web browser..."); // ... }); } } } ``` ## Constructors and Dispose methods - `[AllureBefore(string name = null)]` - `[AllureAfter(string name = null)]` Add information about a constructor or a `Dispose()` method that manage a test context, see the [xUnit.net's documentation](https://xunit.net/docs/shared-context#constructor). The methods will be displayed in the test report as special kinds of [steps](#test-steps). ```csharp using Allure.Xunit.Attributes.Steps; using Xunit; public class TestLabels : IDisposable { [AllureBefore("Setup test context")] public TestLabels() { // ... } [Fact] public void TestCreateLabel() { // ... } [AllureAfter("Clean test context")] public void Dispose() { // ... } } ``` ## Parametrized tests - `AllureApi.AddTestParameter(string name, object? value)` - `AllureApi.AddTestParameter(string name, object? value, ParameterMode mode)` - `AllureApi.AddTestParameter(string name, object? value, bool excluded)` - `AllureApi.AddTestParameter(string name, object? value, ParameterMode mode, bool excluded)` - `AllureApi.AddTestParameter(Parameter parameter)` - `AllureLifecycle.Instance.AddTypeFormatter(TypeFormatter typeFormatter)` To support the [parametrized tests](/docs/v3/readability/#parametrized-tests) pattern, Allure Report automatically detects the parameters specified via `[Theory]` and other xUnit.net's attributes. In addition to this, the Runtime API can be used for adding the parameters even to functions without xUnit.net's attributes. If the `excluded` argument is set to true, Allure will not use the parameter when comparing the current test result with previous one in the history. This argument is only used by Allure TestOps. The `mode` argument affects how the parameter will be displayed in the report. Available options are defined in the `ParameterMode` enumeration: - `ParameterMode.Default` (same as not specifying any mode) — the parameter and its value will be shown in a table along with other parameters. - `ParameterMode.Masked` — the parameter will be shown in the table, but its value will be hidden. Use this mode for passwords, tokens and other sensitive parameters. - `ParameterMode.Hidden` — the parameter and its value will not be shown in the test report. Note, however, that it is still possible to extract the value from the `allure_results` directory if you publish it. By default, Allure Report will render the parameter values via a basic JSON serialization. You can override this behavior for certain types by implementing a custom class that inherits from `TypeFormatter` and passing an object of this class to `AddTypeFormatter()`. The parameters can be added not only to whole test results, but also to individual steps within them, see [Test steps](#test-steps) for more details. **Attributes API:** ```csharp using Xunit; public class TestLabels { [Theory] [InlineData("johndoe", "qwerty")] [InlineData("johndoe@example.com", "qwerty")] public void TestCreateLabel(string login, string password) { // ... } } ``` **Runtime API:** ```csharp using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestAuthenticationWithUsername() { AllureApi.AddTestParameter("login", "johndoe"); AllureApi.AddTestParameter("password", "qwerty", ParameterMode.Masked); // ... } [Fact] public void TestAuthenticationWithEmail() { AllureApi.AddTestParameter("login", "johndoe@example.com"); AllureApi.AddTestParameter("password", "qwerty", ParameterMode.Masked); // ... } } ``` ## Attachments - `AllureApi.AddAttachment(string name, string type, string path)` - `AllureApi.AddAttachment(string name, string type, byte[] content, string fileExtension = "")` - `AllureApi.AddAttachment(string path, string? name = null)` - `AllureApi.AddScreenDiff(string expectedPng, string actualPng, string diffPng)` Add an [attachment](/docs/attachments/) to the test result under the given `name`. Tip: You can use data produced by any function, not necessarily read from an actual file. To create an attachment using the Runtime API, call `AddAttachment()` at any point during your test. Pass either the `content` or the `path` from which the data will be read. To ensure that the reader's web browser will display attachments correctly, it is recommended to specify each attachment's type. To do so, pass the media type of the content as `type` and, optionally, a filename extension as `fileExtension`. The media type affects how the data will be displayed in the test report, while the filename extension is appended to the filename when user wants to save the file. ```csharp using System.IO; using System.Text; using Allure.Net.Commons; using Xunit; public class TestLabels { [Fact] public void TestCreateLabel() { // ... AllureApi.AddAttachment( "data.txt", "text/plain", Encoding.UTF8.GetBytes("This is the file content.") ); AllureApi.AddAttachment( "image1.png", "image/png", File.ReadAllBytes("/path/to/image1.png") ); AllureApi.AddAttachment( "image2.png", "image/png", "/path/to/image2.png" ); } } ``` Allure xUnit.net also provides a dedicated `AddScreenDiff()` function for attaching [visual comparisons](/docs/attachments/#attaching-visual-comparisons). ---