Allure Codeception

Generate beautiful HTML reports using Allure Report and your Codeception tests.

Allure Report Codeception Example

How to start

1. Prepare your project

Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Java, see the installation instructions.
Make sure that your project uses PHP 8 and Codeception 5.
Add allure-framework/allure-codeception to your project's development dependencies:
```
php composer.phar require allure-framework/allure-codeception --dev
```

Enable the Allure extension for Codeception in codeception.yml.

YAML
namespace: Tests
support_namespace: Support
paths:
  tests: tests
  output: tests/_output
  data: tests/Support/Data
  support: tests/Support
  envs: tests/_envs
extensions:
  enabled:
    - Qameta\Allure\Codeception\AllureCodeception

Optionally, specify configuration options, as described in Configuration.

2. Run tests

Run your Codeception tests the same way as you would run them usually. For example:


vendor/bin/codecept run

This will save necessary data into the test results directory, according to the outputDirectory setting. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all.

3. Generate a report

Finally, convert the test results into an HTML report. This can be done by one of two commands:

allure generate processes the test results and saves an HTML report into the allure-report directory. To view the report, use the allure open command.

Use this command if you need to save the report for future reference or for sharing it with colleagues.
allure serve creates the same report as allure generate but puts it into a temporary directory and starts a local web server configured to show this directory's contents. The command then automatically opens the main page of the report in a web browser.

Use this command if you need to view the report for yourself and do not need to save it.

Writing tests

The Allure Codeception adapter extends the standard reporting features of Codeception by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized:

Metadata Annotation: Enhance test reports with descriptions, links, and other metadata.
Test Organization: Structure your tests into clear hierarchies for better readability and organization.
Step Division: Break down tests into smaller test steps for easier understanding and maintenance.
Parametrized Tests: Clearly describe the parameters for parametrized tests to specify different scenarios.
Attachments: Automatically capture screenshots and other files during test execution.
Environment Details: Include comprehensive environment information to accompany the test report.

In most cases, Allure Codeception provides two different ways to use a feature: the Attributes API and the Runtime API.

Attributes API: add a PHP attribute to a test method or a whole class to add certain data to the test result. When using this approach, the data is guaranteed to be added regardless of how the test itself runs.
Runtime API: use Allure's functions to add certain data to the test result during its execution. This approach allows for constructing the data dynamically.

Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early.

Specify description, links and other metadata

There is a lot of metadata you can add to each test so that it would appear in the report. See the reference for more details.

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Attribute\Description;
use Qameta\Allure\Attribute\DisplayName;
use Qameta\Allure\Attribute\Issue;
use Qameta\Allure\Attribute\Link;
use Qameta\Allure\Attribute\Owner;
use Qameta\Allure\Attribute\Severity;
use Qameta\Allure\Attribute\TmsLink;

class TestMyWebsite extends Unit
{
    #[DisplayName('Test Labels')]
    #[Description('This test attempts to create a label with specified title.')]
    #[Severity(Severity::CRITICAL)]
    #[Owner('John Doe')]
    #[Link('My Website', 'https://example.com/')]
    #[Issue('UI-123')]
    #[TmsLink('TMS-456')]
    public function testLabels()
    {
        // ...
    }
}

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Allure;
use Qameta\Allure\Model\Severity;

class TestMyWebsite extends Unit
{
    public function testLabels()
    {
        Allure::displayName('Test Metadata');
        Allure::description('This test attempts to create a label with specified title.');
        Allure::severity(Severity::critical());
        Allure::owner('John Doe');
        Allure::link('My Website', 'https://example.com/');
        Allure::issue('UI-123');
        Allure::tms('TMS-456');
        // ...
    }
}

Organize tests

As described in Improving navigation in your test report, Allure supports multiple ways to organize tests into hierarchical structures. Allure Codeception provides the API to assign the relevant fields to tests either by adding attributes or “dynamically” (same as for the metadata fields).

To specify a test's location in the behavior-based hierarchy:

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Attribute\Epic;
use Qameta\Allure\Attribute\Feature;
use Qameta\Allure\Attribute\Story;

class TestMyWebsite extends Unit
{
    #[Epic('Web interface')]
    #[Feature('Essential features')]
    #[Story('Labels')]
    public function testLabels()
    {
        // ...
    }
}

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Allure;

class TestMyWebsite extends Unit
{
    public function testLabels()
    {
        Allure::epic('Web interface');
        Allure::feature('Essential features');
        Allure::story('Labels');
        // ...
    }
}

To specify a test's location in the suite-based hierarchy:

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Attribute\ParentSuite;
use Qameta\Allure\Attribute\SubSuite;
use Qameta\Allure\Attribute\Suite;

class TestMyWebsite extends Unit
{
    #[ParentSuite('Web interface')]
    #[Suite('Essential features')]
    #[SubSuite('Labels')]
    public function testLabels()
    {
        // ...
    }
}

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Allure;

class TestMyWebsite extends Unit
{
    public function testLabels()
    {
        Allure::parentSuite('Web interface');
        Allure::suite('Essential features');
        Allure::subSuite('Labels');
        // ...
    }
}

Divide a test into steps

Allure Codeception provides three ways of creating steps and sub-steps: “method-based steps”, “lambda steps” and “no-op steps”, see the reference.

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Allure;
use Qameta\Allure\Attribute\DisplayName;
use Qameta\Allure\StepContextInterface;

class TestMyWebsite extends Unit
{
    public function testLabels()
    {
        Allure::runStep([$this, 'logIn']);
        Allure::runStep([$this, 'createLabel']);
        Allure::runStep([$this, 'checkThatLabelExists']);
    }

    #[DisplayName('Log in')]
    function logIn(StepContextInterface $context)
    {
        $context->parameter('Email', '[email protected]');
        $context->parameter('Password', 'qwerty');
        // ...
    }

    #[DisplayName('Create label')]
    function createLabel(StepContextInterface $context)
    {
        $context->parameter('Label name', 'My Label');
        // ...
    }

    #[DisplayName('Check that label exists')]
    function checkThatLabelExists(StepContextInterface $context)
    {
        $context->parameter('Label name', 'My Label');
        // ...
    }
}

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Allure;
use Qameta\Allure\StepContextInterface;

class TestMyWebsite extends Unit
{
    public function testLabels()
    {
        Allure::runStep(function (StepContextInterface $context) {
            $context->parameter('Email', '[email protected]');
            $context->parameter('Password', 'qwerty');
            // ...
        }, 'Log in');

        Allure::runStep(function (StepContextInterface $context) {
            $context->parameter('Label name', 'My Label');
            // ...
        }, 'Create label');

        Allure::runStep(function (StepContextInterface $context) {
            $context->parameter('Label name', 'My Label');
            // ...
        }, 'Check that label exists');
    }
}

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Allure;

class TestMyWebsite extends Unit
{
    public function testLabels()
    {
        // ...
        Allure::addStep('Log in');

        // ...
        Allure::addStep('Create label');

        // ...
        Allure::addStep('Check that label exists');
    }
}

Describe parametrized tests

If you use the parametrized tests pattern, call the Allure::parameter() function to add the parameters to the test report, see the reference.

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Allure;

class TestMyWebsite extends Unit
{
    public function testAuthenticationWithUsername()
    {
        Allure::parameter('login', 'johndoe');
        Allure::parameter('password', 'qwerty');
        // ...
    }

    public function testAuthenticationWithEmail()
    {
        Allure::parameter('login', '[email protected]');
        Allure::parameter('password', 'qwerty');
        // ...
    }
}

PHP
use Codeception\Attribute\DataProvider;
use Codeception\Test\Unit;
use Qameta\Allure\Allure;

class ParametersProviderTest extends Unit
{
    protected function dataProvider()
    {
        return [
            ['login' => 'johndoe', 'password' => 'qwerty'],
            ['login' => '[email protected]', 'password' => 'qwerty'],
        ];
    }

    #[DataProvider('dataProvider')]
    public function testAuthenticationWithData($login, $password)
    {
        Allure::parameter('Login', $login);
        Allure::parameter('Password', $password);
        // ...
    }
}

Attach screenshots and other files

You can attach any sorts of files to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point.

Allure Codeception provides various ways to create an attachment, both from existing files or generated dynamically, see the reference.

PHP
use Codeception\Test\Unit;
use Qameta\Allure\Allure;

class TestMyWebsite extends Unit
{
    public function testLabels()
    {
        // ...
        Allure::attachment('data.txt', 'This is the file content.', 'text/plain');
        Allure::attachmentFile('data.txt', '/path/to/image.png', 'image/png');
    }
}

Select tests via a test plan file

Test plan is currently not supported by the Allure Codeception adapter.

Environment information

For the main page of the report, you can collect various information about the environment in which the tests were executed.

For example, it is a good idea to use this to remember the OS version and PHP version. This may help the future reader investigate bugs that are reproducible only in some environments.

Allure Report Environments Widget

To provide environment information, put a file named environment.properties into the allure-results directory after running the tests. See the example in Environment file.

Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using Parametrized tests.