Allure SpecFlow

Allure SpecFlow latest release

Generate beautiful HTML reports using Allure Report and your SpecFlow tests.

Allure Report SpecFlow Example

How to start

1. Prepare your project

  1. Install the Allure Report command-line tool, if it is not yet installed in your operating system. Note that Allure Report requires Java, see Installation.

  2. Make sure that you have a compatible .NET version.

    Allure SpecFlow requires a framework that implements .NET Standard version 2.0. You can find the full list of compatible frameworks on Allure.SpecFlow's page in NuGet.

  3. Add Allure.SpecFlow to your project's dependencies using your IDE or the command line.

    For example, here is how the dependency can be added to a project that uses the default .NET implementation from Microsoft:

    dotnet add ⟨PATH TO PROJECT⟩ package Allure.SpecFlow
  4. In the specflow.json file, add Allure.SpecFlowPlugin as a step assembly.

    { "stepAssemblies": [{ "assembly": "Allure.SpecFlowPlugin" }] }

    Make sure that specflow.json is copied to the output directory. You can do it in your IDE or manually in your *.csproj file by setting the item group's CopyToOutputDirectory to either Always or PreserveNewest.

    <Project Sdk="Microsoft.NET.Sdk"> <!-- ... --> <ItemGroup> <None Update="specflow.json"> <CopyToOutputDirectory>Always</CopyToOutputDirectory> </None> </ItemGroup> </Project>

2. Run tests

Run your SpecFlow tests the same way as you would run them usually. For example:

dotnet test

This will save necessary data into allure-results or other directory, according to the configuration option. If the directory already exists, the new files will be added to the existing ones, so that a future report will be based on them all.

3. Generate a report

Finally, convert the test results into an HTML report. This can be done by one of two commands:

  • allure generate processes the test results and saves an HTML report into the allure-report directory. To view the report, use the allure open command.

    Use this command if you need to save the report for future reference or for sharing it with colleagues.

  • allure serve creates the same report as allure generate but puts it into a temporary directory and starts a local web server configured to show this directory's contents. The command then automatically opens the main page of the report in a web browser.

    Use this command if you need to view the report for yourself and do not need to save it.

Writing tests

The Allure SpecFlow adapter extends the standard reporting features of SpecFlow by providing additional capabilities for crafting more informative and structured tests. This section highlights key enhancements that can be utilized:

In most cases, you need to indicate to Allure SpecFlow that a certain property needs to be assigned to the test result. Most properties can be assigned via Gherkin tags or via the Runtime API.

  • Gherkin tags: use Gherkin tags to assign various data to a particular Scenario or a whole Feature. By default, all Gherkin tags will be converted to test tags, but you can configure conversion to various other metadata via regular expressions in the configuration file.

    Note that due to a limitation in the Gherkin syntax, a tag cannot contain spaces.

  • Runtime API: use Allure's functions to add data to the test result during the execution of its steps. This approach allows for constructing the data dynamically.

    Note that it is recommended to call the Allure's functions as close to the beginning of the test as possible. This way, the data will be added even if the test fails early.

There is a lot of metadata you can add to each test so that it would appear in the report. See reference for more details.

Feature: Labels @critical @allure.label.owner:JohnDoe @link: @issue:UI-123 @tms:TMS-456 Scenario: Create new label for authorized user When I open labels page And I create label with title "hello" Then I should see label with title "hello"
using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.SetSeverity(SeverityLevel.critical); AllureApi.SetOwner("John Doe"); AllureApi.AddLink("Website", ""); AllureApi.AddIssue("UI-123"); AllureApi.AddTmsItem("TMS-456"); // ... } }

Organize tests

As described in Improving navigation in your test report, Allure supports multiple ways to organize tests into hierarchical structures. Allure SpecFlow provides the API to assign the relevant fields to tests either by adding attributes or “dynamically” (same as for the metadata fields).

To specify a test's location in the behavior-based hierarchy:

@allure.label.epic:WebInterface @allure.label.story:Labels Feature: My feature Scenario: My scenario Then my step
using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddEpic("Web interface"); AllureApi.AddFeature("Essential features"); AllureApi.AddStory("Labels"); // ... } }

To specify a test's location in the suite-based hierarchy:

@allure.label.parentSuite:WebInterface @allure.label.suite:EssentialFeatures @allure.label.subSuite:Labels Feature: My feature Scenario: My scenario Then my step
using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddParentSuite("Web interface"); AllureApi.AddSuite("Essential features"); AllureApi.AddSubSuite("Labels"); // ... } }

To specify a test's location in the package-based hierarchy:

@allure.label.package:org.example @allure.label.class:TestMyWebsite @allure.label.method:TestLabels() Feature: My feature Scenario: My scenario Then my step
using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddLabel(Label.Package("org.example")); AllureApi.AddLabel(Label.TestClass("TestMyWebsite")); AllureApi.AddLabel(Label.TestMethod("TestLabels()")); // ... } }

Divide a test into steps

Allure SpecFlow provides two ways of creating steps and sub-steps: “lambda steps” and “no-op steps”, see reference.

using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [Then("visit pages")] public void ThenVisitPages() { string[] urls = { "", "" }; foreach (string url in urls) { AllureApi.Step($"Test {url}", () => { AllureLifecycle.Instance.UpdateStep(stepResult => { stepResult.parameters.Add( new Parameter { name = "Webpage URL", value = url } ); }); AllureApi.Step("Opening web browser..."); // ... AllureApi.Step($"Visiting {url}..."); // ... AllureApi.Step("Closing web browser..."); // ... }); } } }

Describe parametrized tests

An Allure test report can reflect multiple ways in which you can pass data from your Gherkin file to your C# implementation code, namely:

The example below shows a configuration file, a Gherkin file and a C# implementation file of a test. In this example, the four parameters for the “I enter my details” step will be displayed in the test report.

{ "specflow": { "stepArguments": { "convertToParameters": true, "paramNameRegex": "^Parameter$", "paramValueRegex": "^Value$" } } }
Feature: User management Scenario: Registration When I go to the registration form And I enter my details | Parameter | Value | | login | johndoe | | password | qwerty | | name | John Doe | | birthday | 1970-01-01 | Then the profile should be created
using TechTalk.SpecFlow; [Binding] public class Steps { [When("I go to the registration form")] public void WhenIGoToTheRegistrationForm() { // ... } [When("I enter my details")] public void WhenIEnterMyDetails(Table table) { // ... } [Then("the profile should be created")] public void ThenTheProfileShouldBeCreated() { // ... } }

Attach screenshots and other files

You can attach any sorts of files to your Allure report. For example, a popular way to make a report easier to understand is to attach a screenshot of the user interface at a certain point.

Allure SpecFlow provides various ways to create an attachment, both from existing files or generated dynamically, see reference.

using System.IO; using Allure.Net.Commons; using TechTalk.SpecFlow; [Binding] public class Steps { [When("I open labels page")] public void TestLabels() { AllureApi.AddAttachment( "data.txt", "text/plain", "This is the file content."u8.ToArray() ); AllureApi.AddAttachment( "image1.png", "image/png", File.ReadAllBytes("/path/to/image1.png") ); AllureApi.AddAttachment( "image2.png", "image/png", "/path/to/image2.png" ); } }

Select tests via a test plan file

If the ALLURE_TESTPLAN_PATH environment variable is defined and points to an existing file, SpecFlow will only run tests listed in this file.

Here's an example of running tests according to a file named testplan.json:

export ALLURE_TESTPLAN_PATH=testplan.json dotnet test
$Env:ALLURE_TESTPLAN_PATH = "testplan.json" dotnet test

Environment information

For the main page of the report, you can collect various information about the environment in which the tests were executed.

For example, it is a good idea to use this to remember the OS version and .NET version. This may help the future reader investigate bugs that are reproducible only in some environments.

Allure Report Environments Widget

To provide environment information, put a file named into the allure-results directory after running the tests. See the example in Environment file.

Note that this feature should be used for properties that do not change for all tests in the report. If you have properties that can be different for different tests, consider using Parametrized tests.

Powered by

Join our newsletter

Join our community

We aim to make Allure Report as reliable and user-friendly as possible, and together with the community, we're here to help when problems arise.

© 2024 Qameta Software Inc. All rights reserved.