Getting started with Allure PHPUnit

Generate beautiful HTML reports using Allure Report and your PHPUnit tests.

INFO

Check out the example project at github.com/allure-examples/phpunit to see Allure PHPUnit in action.

Setting up

1. Prepare your project

1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Java, see the installation instructions.

2. Make sure that your project uses:

   • PHP version 8.1, 8.2 or 8.3,
   • PHPUnit version 10.

3. Add allure-framework/allure-phpunit to your project's development dependencies:

   php composer.phar require allure-framework/allure-phpunit --dev

4. Enable the Allure extension for PHPUnit via a <bootstrap> tag in phpunit.xml. In the config parameter, specify a path where you will put the configuration file for Allure PHPUnit. For example:

   <?xml version="1.0" encoding="UTF-8"?>
   <phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.0/phpunit.xsd">
       <extensions>
           <bootstrap class="Qameta\Allure\PHPUnit\AllureExtension">
               <parameter name="config" value="config/allure.config.php"/>
           </bootstrap>
       </extensions>
   </phpunit>

5. Create an Allure PHPUnit configuration file at the specified location (config/allure.config.php in the example above). The file must return an associative array containing configuration options. For example:

   <?php
   
   return [
       'outputDirectory' => 'build/allure-results',
   ];

2. Run tests

Run your PHPUnit tests the same way as you would run them usually. For example:

vendor/bin/phpunit

This will save necessary data into build/allure-results or other directory, according to the outputDirectory setting. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all.

3. Generate a report

Finally, convert the test results into an HTML report. This can be done by one of two commands:

• allure generate processes the test results and saves an HTML report into the allure-report directory. To view the report, use the allure open command.

  Use this command if you need to save the report for future reference or for sharing it with colleagues.

• allure serve creates the same report as allure generate but puts it into a temporary directory and starts a local web server configured to show this directory's contents. The command then automatically opens the main page of the report in a web browser.

  Use this command if you need to view the report for yourself and do not need to save it.

Writing tests

The Allure PHPUnit adapter extends the standard reporting features of PHPUnit by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized:

• Metadata Annotation: Enhance test reports with descriptions, links, and other metadata.
• Test Organization: Structure your tests into clear hierarchies for better readability and organization.
• Step Division: Break down tests into smaller test steps for easier understanding and maintenance.
• Parametrized Tests: Clearly describe the parameters for parametrized tests to specify different scenarios.
• Attachments: Automatically capture screenshots and other files during test execution.
• Environment Details: Include comprehensive environment information to accompany the test report.

In most cases, Allure PHPUnit provides two different ways to use a feature: the Attributes API and the Runtime API.

• Attributes API: add a PHP attribute to a test method or a whole class to add certain data to the test result. When using this approach, the data is guaranteed to be added regardless of how the test itself runs.

• Runtime API: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically.

  Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early.

Specify description, links and other metadata

There is a lot of metadata you can add to each test so that it would appear in the report. See the reference for more details.

Attributes API

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Attribute\Description;
use Qameta\Allure\Attribute\DisplayName;
use Qameta\Allure\Attribute\Issue;
use Qameta\Allure\Attribute\Link;
use Qameta\Allure\Attribute\Owner;
use Qameta\Allure\Attribute\Severity;
use Qameta\Allure\Attribute\TmsLink;

final class TestMyWebsite extends TestCase
{
    #[DisplayName('Test Labels')]
    #[Description('This test attempts to create a label with specified title.')]
    #[Severity(Severity::CRITICAL)]
    #[Owner('John Doe')]
    #[Link('My Website', 'https://example.com/')]
    #[Issue('UI-123')]
    #[TmsLink('TMS-456')]
    public function testLabels()
    {
        // ...
    }
}

Runtime API

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Allure;
use Qameta\Allure\Model\Severity;

final class TestMyWebsite extends TestCase
{
    public function testLabels()
    {
        Allure::displayName('Test Labels');
        Allure::description('This test attempts to create a label with specified title.');
        Allure::severity(Severity::critical());
        Allure::owner('John Doe');
        Allure::link('My Website', 'https://example.com/');
        Allure::issue('UI-123');
        Allure::tms('TMS-456');
        // ...
    }
}

Organize tests

As described in Improving navigation in your test report, Allure supports multiple ways to organize tests into hierarchical structures. Allure PHPUnit provides the API to assign the relevant fields to tests either by adding attributes or “dynamically” (same as for the metadata fields).

To specify a test's location in the behavior-based hierarchy:

Attributes API

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Attribute\Epic;
use Qameta\Allure\Attribute\Feature;
use Qameta\Allure\Attribute\Story;

final class TestMyWebsite extends TestCase
{
    #[Epic('Web interface')]
    #[Feature('Essential features')]
    #[Story('Labels')]
    public function testLabels()
    {
        // ...
    }
}

Runtime API

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Allure;

final class TestMyWebsite extends TestCase
{
    public function testLabels()
    {
        Allure::epic('Web interface');
        Allure::feature('Essential features');
        Allure::story('Labels');
        // ...
    }
}

To specify a test's location in the suite-based hierarchy:

Attributes API

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Attribute\ParentSuite;
use Qameta\Allure\Attribute\SubSuite;
use Qameta\Allure\Attribute\Suite;

final class TestMyWebsite extends TestCase
{
    #[ParentSuite('Web interface')]
    #[Suite('Essential features')]
    #[SubSuite('Labels')]
    public function testLabels()
    {
        // ...
    }
}

Runtime API

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Allure;

final class TestMyWebsite extends TestCase
{
    public function testLabels()
    {
        Allure::parentSuite('Web interface');
        Allure::suite('Essential features');
        Allure::subSuite('Labels');
        // ...
    }
}

Divide a test into steps

Allure PHPUnit provides three ways of creating steps and sub-steps: “method-based steps”, “lambda steps” and “no-op steps”, see the reference.

Method-based steps

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Allure;
use Qameta\Allure\Attribute\DisplayName;
use Qameta\Allure\StepContextInterface;

final class TestMyWebsite extends TestCase
{
    public function testLabels()
    {
        Allure::runStep([$this, 'logIn']);
        Allure::runStep([$this, 'createLabel']);
        Allure::runStep([$this, 'checkThatLabelExists']);
    }

    #[DisplayName('Log in')]
    function logIn(StepContextInterface $context)
    {
        $context->parameter('Email', '[email protected]');
        $context->parameter('Password', 'qwerty');
        // ...
    }

    #[DisplayName('Create label')]
    function createLabel(StepContextInterface $context)
    {
        $context->parameter('Label name', 'My Label');
        // ...
    }

    #[DisplayName('Check that label exists')]
    function checkThatLabelExists(StepContextInterface $context)
    {
        $context->parameter('Label name', 'My Label');
        // ...
    }
}

Lambda steps

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Allure;
use Qameta\Allure\StepContextInterface;

final class TestMyWebsite extends TestCase
{
    public function testLabels()
    {
        Allure::runStep(function (StepContextInterface $context) {
            $context->name('Log in');
            $context->parameter('Email', '[email protected]');
            $context->parameter('Password', 'qwerty');
            // ...
        });

        Allure::runStep(function (StepContextInterface $context) {
            $context->name('Create label');
            $context->parameter('Label name', 'My Label');
            // ...
        });

        Allure::runStep(function (StepContextInterface $context) {
            $context->name('Check that label exists');
            $context->parameter('Label name', 'My Label');
            // ...
        });
    }
}

No-op steps

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Allure;

final class TestMyWebsite extends TestCase
{
    public function testLabels()
    {
        // ...
        Allure::addStep('Log in');

        // ...
        Allure::addStep('Create label');

        // ...
        Allure::addStep('Check that label exists');
    }
}

Describe parametrized tests

If you use the parametrized tests pattern, call the Allure::parameter() function to add the parameters to the test report, see the reference.

DataProvider

use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\TestCase;
use Qameta\Allure\Allure;

final class TestMyWebsite extends TestCase
{
    public static function credentialsProvider()
    {
        return [
            ['johndoe', 'qwerty'],
            ['[email protected]', 'qwerty'],
        ];
    }

    #[DataProvider('credentialsProvider')]
    public function testAuthentication(string $login, string $password)
    {
        Allure::parameter('login', $login);
        Allure::parameter('password', $password);
        // ...
    }
}

Runtime API

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Allure;

final class TestMyWebsite extends TestCase
{
    public function testAuthenticationWithUsername()
    {
        Allure::parameter('login', 'johndoe');
        Allure::parameter('password', 'qwerty');
        // ...
    }

    public function testAuthenticationWithEmail()
    {
        Allure::parameter('login', '[email protected]');
        Allure::parameter('password', 'qwerty');
        // ...
    }
}

Attach screenshots and other files

You can attach any sorts of files to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point.

Allure PHPUnit provides various ways to create an attachment, both from existing files or generated dynamically, see the reference.

Runtime API

use PHPUnit\Framework\TestCase;
use Qameta\Allure\Allure;

final class TestMyWebsite extends TestCase
{
    public function testLabels()
    {
        // ...
        Allure::attachment('data.txt', 'This is the file content.', 'text/plain');
        Allure::attachmentFile('data.txt', '/path/to/image.png', 'image/png');
    }
}

Select tests via a test plan file

DANGER

Test plan is currently not supported by the Allure PHPUnit adapter.

Environment information

For the main page of the report, you can collect various information about the environment in which the tests were executed.

For example, it is a good idea to use this to remember the OS version and PHP version. This may help the future reader investigate bugs that are reproducible only in some environments.

To provide environment information, put a file named environment.properties into the allure-results directory after running the tests. See the example in Environment file.

Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using Parametrized tests.