Export metrics

TIP

This page describes how to export test statistics, such as the number of tests with each status or the duration of the run. To collect your own measurements, such as page load times, see Performance Metrics. In Allure Report 3, you can export their averages together with the test statistics.

Allure Report can write aggregate test data to files that can be imported to InfluxDB or Prometheus for building custom graphs. By default, the files are placed in the export subdirectory of the report directory.

• Allure Report 2 writes the files for every report.
• Allure Report 3 writes the same files when the metrics plugin is enabled, with the same metric names except for the default categories. It can also push the data to Prometheus Pushgateway or InfluxDB.

Enable the export in Allure Report 3

The metrics plugin is included in the allure package. To enable it, add it to the plugins section of the configuration file. Only the plugins listed in this section are run, so keep the plugin of the report you want to generate, for example, awesome:

allurerc.mjs

import { defineConfig } from "allure";

export default defineConfig({
  plugins: {
    awesome: {},
    metrics: {},
  },
});

allurerc.json

{
  "plugins": {
    "awesome": {},
    "metrics": {}
  }
}

allurerc.yaml

plugins:
  awesome: {}
  metrics: {}

With this configuration, Allure writes both export/prometheusData.txt and export/influxDbData.txt to the report directory, next to the report itself.

To write only one of the files, enable it explicitly, for example, prometheus: true. If neither prometheus nor influxdb is set, both files are written.

Option	Description	Default
prometheus	true or an object with the Prometheus options	Enabled if neither is set
influxdb	true or an object with the InfluxDB options	Enabled if neither is set
performanceMetrics	If true, performance metrics are exported too	false
failOnPushError	If true, the report generation fails when pushing the data fails. Otherwise, Allure only prints a warning	false
pushTimeout	Timeout of a single push request, in milliseconds	10000

Prometheus options

Option	Description	Default
fileName	Path to the file, relative to the report directory, or an absolute path	export/prometheusData.txt
labels	Labels to add to every metric: an object, such as { team: "qa" }, or a ready string, such as team="qa",env="ci". If not set, Allure takes them from the ALLURE_PROMETHEUS_LABELS environment variable	No labels
pushgateway	Settings for pushing the metrics to Prometheus Pushgateway	No push

InfluxDB options

Option	Description	Default
fileName	Path to the file, relative to the report directory, or an absolute path	export/influxDbData.txt
push	Settings for pushing the data to the InfluxDB write API	No push

Push the data

After writing the files, Allure can send the same data to Prometheus Pushgateway and to InfluxDB:

allurerc.mjs

import { defineConfig } from "allure";

export default defineConfig({
  plugins: {
    awesome: {},
    metrics: {
      options: {
        prometheus: {
          labels: { team: "qa" },
          pushgateway: {
            url: "http://localhost:9091",
            job: "allure",
            grouping: { instance: "ci" },
          },
        },
        influxdb: {
          push: {
            url: "http://localhost:8086",
            org: "qa",
            bucket: "allure",
            tokenEnv: "INFLUXDB_TOKEN",
          },
        },
      },
    },
  },
});

allurerc.json

{
  "plugins": {
    "awesome": {},
    "metrics": {
      "options": {
        "prometheus": {
          "labels": { "team": "qa" },
          "pushgateway": {
            "url": "http://localhost:9091",
            "job": "allure",
            "grouping": { "instance": "ci" }
          }
        },
        "influxdb": {
          "push": {
            "url": "http://localhost:8086",
            "org": "qa",
            "bucket": "allure",
            "tokenEnv": "INFLUXDB_TOKEN"
          }
        }
      }
    }
  }
}

allurerc.yaml

plugins:
  awesome: {}
  metrics:
    options:
      prometheus:
        labels:
          team: qa
        pushgateway:
          url: http://localhost:9091
          job: allure
          grouping:
            instance: ci
      influxdb:
        push:
          url: http://localhost:8086
          org: qa
          bucket: allure
          tokenEnv: INFLUXDB_TOKEN

This example pushes to a Pushgateway that doesn't require authentication and to InfluxDB 2.x, with the API token taken from the INFLUXDB_TOKEN environment variable. The tables below list all the options.

We recommend keeping the secrets in environment variables rather than in the configuration file.

Pushgateway options

Option	Description	Default
url	Pushgateway address. Required
job	Value of the job grouping key	allure
grouping	Additional grouping keys, for example, { instance: "ci" }
authorizationEnv	Name of the environment variable with the value of the Authorization header, if the Pushgateway requires authentication. The value must include the scheme, for example, Basic dXNlcjpwYXNzd29yZA== for basic authentication	No header
authorization	Value of the Authorization header, set directly in the configuration file	No header

Each push replaces all metrics of the group in the Pushgateway, so metrics from previous runs don't remain there.

InfluxDB push options

Option	Description	InfluxDB version
url	InfluxDB address. Required	2.x and 1.x
org	Organization name or ID. Required with bucket	2.x
bucket	Bucket to write to	2.x
tokenEnv	Name of the environment variable with the API token	2.x
token	API token, set directly in the configuration file	2.x
db	Database to write to. Used if bucket is not set	1.x
username	User name for authentication	1.x
passwordEnv	Name of the environment variable with the password	1.x
password	Password, set directly in the configuration file	1.x

Export performance metrics

If you collect performance metrics, set performanceMetrics: true to export them too. For each metric key, Allure exports the average value of the run: in Prometheus, as launch_metric_<key>, with dots in the key replaced by underscores, and in InfluxDB, as a field of the launch_metric measurement named after the key.

InfluxDB measurements

The export/influxDbData.txt file contains aggregate test data in the line protocol format for InfluxDB. When imported, it creates the following measurements.

Measurement	Tags
launch_status	One tag per each possible test status: failed, broken, passed, skipped, unknown. Each value represents the number of tests with the status.
launch_time	duration - the time it took to run all tests. min_duration - the duration of the fastest test run. max_duration - the duration of the slowest test run. sum_duration - all durations, combined. start - the time when the first test started, as a timestamp in milliseconds. stop - the time when the last test finished, as a timestamp in milliseconds. The values for duration and sum_duration may differ significantly if your test framework runs tests in parallel or spends significant time preparing for each test.
launch_problems	One tag per each non-empty category. Each category's name is converted to lowercase, and the spaces are replaced with underscores, for example: test_defects, product_defects. Each value represents the number of tests in the category.
launch_retries	retries - the number of test runs that were retries of other test runs. run - the number of test runs that were not retries.
launch_metric Allure 3	One tag per each performance metric key, only if performanceMetrics is enabled. Each value represents the average value of the metric in the run.

Prometheus metrics

The export/prometheusData.txt file contains aggregate test data in the text-based format for Prometheus. When imported, it creates the following metrics.

Prefix	Metrics
launch_status_*	One metric per each possible test status: launch_status_failed, launch_status_broken, launch_status_passed, launch_status_skipped, launch_status_unknown. Each value represents the number of tests with the status.
launch_time_*	launch_time_duration - the time it took to run all tests. launch_time_min_duration - the duration of the fastest test run. launch_time_max_duration - the duration of the slowest test run. launch_time_sum_duration - all durations, combined. launch_time_start - the time when the first test started, as a timestamp in milliseconds. launch_time_stop - the time when the last test finished, as a timestamp in milliseconds. The values for launch_time_duration and launch_time_sum_duration may differ significantly if your test framework runs tests in parallel or spends significant time preparing for each test.
launch_problems_*	One metric per each non-empty category. Each category's name is converted to lowercase, the spaces are replaced with underscores, and the prefix is added, for example: launch_problems_test_defects, launch_problems_product_defects. Each value represents the number of tests in the category.
launch_retries_*	launch_retries_retries - the number of test runs that were retries of other test runs. launch_retries_run - the number of test runs that were not retries.
launch_metric_* Allure 3	One metric per each performance metric key, only if performanceMetrics is enabled, for example: launch_metric_page_load. Each value represents the average value of the metric in the run.

Category names in Allure Report 3

The default categories have different names in Allure Report 2 and Allure Report 3, and so do their metrics:

• Allure Report 2: launch_problems_product_defects and launch_problems_test_defects;
• Allure Report 3: launch_problems_product_errors and launch_problems_test_errors.

If your graphs use the Allure Report 2 names, define categories with these names in the Allure Report 3 configuration:

allurerc.mjs

import { defineConfig } from "allure";

export default defineConfig({
  categories: {
    rules: [
      { name: "Product defects", matchers: { statuses: ["failed"] } },
      { name: "Test defects", matchers: { statuses: ["broken"] } },
    ],
  },
  plugins: {
    awesome: {},
    metrics: {},
  },
});

allurerc.json

{
  "categories": {
    "rules": [
      { "name": "Product defects", "matchers": { "statuses": ["failed"] } },
      { "name": "Test defects", "matchers": { "statuses": ["broken"] } }
    ]
  },
  "plugins": {
    "awesome": {},
    "metrics": {}
  }
}

allurerc.yaml

categories:
  rules:
    - name: Product defects
      matchers:
        statuses: [failed]
    - name: Test defects
      matchers:
        statuses: [broken]
plugins:
  awesome: {}
  metrics: {}