Appearance
Allure Pytest
Generate beautiful HTML reports using Allure Report and your pytest tests.
Details
Check out the example project at github.com/allure-examples/allure-python-pytest to see Allure Pytest in action.
How to start
1. Prepare your project
These instructions depend on whether you have some tests already or you're just starting a project.
Adding Allure Pytest to an existing project
If you have some pytest tests, and you want to be able to use Allure Report with them:
Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Java, see the installation instructions.
Open a terminal and go to the project directory. For example:
bashcd /home/user/myproject
If necessary for your system configuration, activate the virtual Python environment for your project. For example, if the project uses a
venv
environment, the command to activate it may look like this:bashsource .venv/bin/activate
This step is not necessary if you are using the system Python environment.
Install the Allure Pytest adapter.
bashpip install allure-pytest
Starting a new project with Allure Pytest
If you are starting a new project which will contain pytest tests:
Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Java, see the installation instructions.
Create a directory for the project.
Open a terminal and go to the project directory. For example:
bashcd /home/user/myproject
If necessary, create and activate virtual Python environment for your project. For example, the commands to create and activate a
venv
environment may look like this:bashpython -m venv .venv source .venv/bin/activate
This step is not necessary if you are going to use the system Python environment in the project.
Write your tests, following the official pytest documentation and, optionally, using some extended features provided by Allure Pytest, see Writing tests.
2. Run tests
When running your tests, specify a path for the test results directory in the --alluredir
command-line argument. For example:
bash
python -m pytest --alluredir allure-results
This will save necessary data into the test results directory. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all.
3. Generate a report
Finally, run Allure to convert the test results into an HTML report. This will automatically open your browser to view the report.
bash
allure serve allure-results
Replace allure-results
with the path to the directory specified in the outputFolder
setting of the reporter, see Configuration.
There are some options that can affect how the report is generated. Run allure --help
for the full list of options.
Writing tests
The Allure Pytest adapter not only collects the data provided by pytest's standard features, but also provides additional features for writing even better tests. This section lists the most notable ways to improve your tests, using both pytest's and Allure Pytest's features.
With Allure Pytest, you can:
- provide description, links and other metadata,
- organize tests into hierarchies,
- divide the test into smaller, easier-to-read test steps,
- describe parameters used when running parametrized tests,
- add readable titles to fixtures,
- make the test save screenshots and other files during execution,
- select which tests to run via a test plan file.
Specify description, links and other metadata
There is a lot of metadata you can add to each test so that it would appear in the report. See the reference for more details.
For each of the metadata fields, there are multiple ways to assign it.
- Call an Allure's function inside the test function's body. This way is called “dynamic” because it allows you to construct strings and other values at runtime before passing to the methods.
- Use an Allure's function as a decorator on the test function or a test class.
- Add an Allure's function as a pytest mark to a module or a package (see the Marking whole classes or modules section in pytest documentation).
TIP
If you set the fields using the decorators or pytest marks, you can then select and run tests by some metadata fields, see Options affecting test selection.
python
import allure
@allure.title("Test Authentication")
@allure.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.")
@allure.tag("NewUI", "Essentials", "Authentication")
@allure.severity(allure.severity_level.CRITICAL)
@allure.label("owner", "John Doe")
@allure.link("https://dev.example.com/", name="Website")
@allure.issue("AUTH-123")
@allure.testcase("TMS-456")
def test_authentication():
...
python
import allure
def test_authentication():
allure.dynamic.title("Test Authentication")
allure.dynamic.description("This test attempts to log into the website using a login and a password. Fails if any error happens.\n\nNote that this test does not test 2-Factor Authentication.")
allure.dynamic.tag("NewUI", "Essentials", "Authentication")
allure.dynamic.severity(allure.severity_level.CRITICAL)
allure.dynamic.label("owner", "John Doe")
allure.dynamic.link("https://dev.example.com/", name="Website")
allure.dynamic.issue("AUTH-123")
allure.dynamic.testcase("TMS-456")
...
Organize tests
As described in Improving navigation in your test report, Allure supports multiple ways to organize tests into hierarchical structures. Allure Pytest provides functions to assign the relevant fields to tests either by adding decorators or marks or by calling the “dynamic” functions (same as for the metadata fields).
To specify a test's location in the behavior-based hierarchy:
python
import allure
@allure.epic("Web interface")
@allure.feature("Essential features")
@allure.story("Authentication")
def test_authentication():
...
python
import allure
def test_authentication():
allure.dynamic.epic("Web interface")
allure.dynamic.feature("Essential features")
allure.dynamic.story("Authentication")
...
To specify a test's location in the suite-based hierarchy:
python
import allure
@allure.parent_suite("Tests for web interface")
@allure.suite("Tests for essential features")
@allure.sub_suite("Tests for authentication")
def test_authentication():
...
python
import allure
def test_authentication():
allure.dynamic.parent_suite("Tests for web interface")
allure.dynamic.suite("Tests for essential features")
allure.dynamic.sub_suite("Tests for authentication")
...
A test's location in the package-based hierarchy is defined by the fully qualified named of the classes and functions they are declared in, with common prefixes shown as parent packages.
Divide a test into steps
Allure Pytest provides two ways of creating steps and sub-steps: “decorated steps” and “context steps”, both implemented by allure.step()
.
python
import allure
def test_example():
steps = Steps()
steps.step1()
steps.step2()
class Steps:
@allure.step("Step 1")
def step1(self):
...
@allure.step("Step 2")
def step2(self):
...
python
import allure
def test_example():
with allure.step("Step 1"):
...
with allure.step("Step 2"):
...
Describe parametrized tests
When using the parametrized tests pattern, Allure Pytest automatically adds the parameter values to the test report. Allure supports all the methods of parametrizing tests supported by pytest, including declaring parameters for fixtures.
Additionally, you can:
- incorporate a parameter's value into the test titles, see
title()
; - manually add a pseudo-parameter, see
parameter()
. - override a parameter's value with a more readable representation, see
parameter()
;
python
from os.path import basename, expanduser
import allure
import pytest
@pytest.mark.parametrize("login", [
"johndoe",
"[email protected]",
])
def test_authentication(login):
...
def test_authentication_with_empty_login():
allure.dynamic.parameter("login", "(empty)")
...
@pytest.mark.parametrize("ssh_key", [
expanduser("~/.ssh/id_rsa1"),
expanduser("~/.ssh/id_rsa2"),
expanduser("~/.ssh/id_rsa3"),
])
def test_authentication_with_ssh_key(ssh_key):
allure.dynamic.parameter("ssh_key", basename(ssh_key))
...
Describe fixtures
The pytest's concept of fixtures provides a way for each test to define features it requires by just specifying function arguments with their names. Some fixtures are provided by pytest itself, others can be provided by additional libraries or your own code. See the pytest documentation for more details.
When displaying a test in a test report, Allure displays operations related to fixtures similarly to how it displays test steps. To make it easier to understand a fixture's purpose, you can specify its title using the @allure.title()
decorator.
python
import allure
import pytest
@pytest.fixture()
@allure.title("Prepare for the test")
def my_fixture():
... # set up
yield
... # tear down
def test_with_my_fixture(my_fixture):
...
Attach screenshots and other files
You can attach any sorts of files to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point. To do so, use allure.attach()
or allure.attach.file()
.
python
import allure
from pathlib import Path
def test_attach(page):
page.goto("https://example.com/")
png_bytes = page.screenshot()
allure.attach(
png_bytes,
name="full-page",
attachment_type=allure.attachment_type.PNG
)
def test_attach_file(page):
page.goto("https://example.com/")
png_bytes = page.screenshot()
Path("full-page.png").write_bytes(png_bytes)
allure.attach.file(
"full-page.png",
name="full-page",
attachment_type=allure.attachment_type.PNG
)
By default, Allure Pytest adds the following data as attached pseudo-files:
Attachment name | Contents |
---|---|
stdout | All data that was written to sys.stdout , e.g., via print(...) . |
stderr | All data that was written to sys.stderr , e.g., via print(..., file=sys.stderr) . |
log | All data that was logged to the standard logging mechanism, e.g., via logging.debug(...) . |
This behavior can be disabled by the --allure-no-capture
option.
Select tests via a test plan file
If the ALLURE_TESTPLAN_PATH
environment variable is defined and points to an existing file, pytest will only run tests listed in this file.
Here's an example of running tests according to a file named testplan.json
:
bash
export ALLURE_TESTPLAN_PATH=testplan.json
python -m pytest
powershell
$Env:ALLURE_TESTPLAN_PATH = "testplan.json"
python -m pytest
Environment information
For the main page of the report, you can collect various information about the environment in which the tests were executed.
For example, it is a good idea to use this to remember the OS version and Python version. This may help the future reader investigate bugs that are reproducible only in some environments.
To provide environment information, put a file named environment.properties
into the allure-results
directory after running the tests. See the example in Environment file.
Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using Parametrized tests.