Which Azure DevOps versions are supported?
SpecSync for Azure DevOps might work with other Azure DevOps installations as well, but has been tested with the following configurations:
- Azure DevOps Services (cloud service)
- Azure DevOps Server 2022 (Version 19.*)
- Azure DevOps Server 2020 (Version 18.*)
- Azure DevOps Server 2019 (Version 17.*, Dev17.*)
- Team Foundation Server 2018 (Version 16.*)
- Team Foundation Server 2017 (Version 15.*)
- Team Foundation Server 2015 Update 3 (Version 14.*) — up to SpecSync v2.1
- Team Foundation Server 2013 Update 1 (Version 12.*) — up to SpecSync v1.3
Can I synchronize scenarios that have been written for Cucumber, Behat or Behave?
Yes, SpecSync can synchronize any scenarios that are written in Gherkin format. Gherkin format is used by many tools in many platforms, like Cucumber, Cucumber JVM, Cucumber.js, Behat, Behave, SpecFlow and Reqnroll.
SpecSync runs on Windows, macOS and Linux and with the different installation options you can find the most convenient one for your environment.
See Using SpecSync with Cucumber documentation page for details.
For publishing test results of the particular tool, SpecSync needs to understand its test result file format. Many file formats are supported already, including SpecFlow, Reqnroll, Cucumber Java or Python Behave. If the test result format of your tool is not supported yet, please contact SpecSync support or write a small SpecSync plugin that can parse the results.
Can I synchronize scenarios that have been written for JBehave or Robot Framework?
SpecSync currently supports only scenarios written in Gherkin format. Please contact support to ask for options supporting the format of your tool as well. It is also possible to write a SpecSync plugin that makes other BDD frameworks usable for SpecSync.
Can I synchronize scenarios that are not automated?
Yes. You can synchronize any scenario that uses the Gherkin syntax, even if they are not automated. The synchronized non-automated test cases can be managed, linked and structured in Azure DevOps. You can also run them manually. You can even publish test results to non-automated test cases as well. See documentation page for details.
Do I have to use MsTest as a test runner for SpecFlow to be able to synchronize my scenarios?
No. All unit test providers (MsTest, xUnit, NUnit, SpecFlow+ Runner) are supported for synchronizing automated test cases and publishing test results.
Does SpecSync support SpecFlow projects using NUnit, xUnit or SpecFlow+ Runner?
Yes. All unit test providers (MsTest, xUnit, NUnit, SpecFlow+ Runner) are supported for synchronizing automated test cases and publishing test results.
Does SpecSync support SpecFlow projects using SpecFlow+ Runner (SpecRun)?
Yes. SpecFlow+ Runner is supported for synchronizing automated test cases and publishing test results. See documentation page for details.
Can I synchronize my SpecFlow scenarios to non-automated Azure DevOps test cases?
SpecSync synchronizes the scenarios to non-automated test cases by default but you can configure it to mark the synchronized test cases as Automated. In order to enable synchronizing automated test cases, please follow the steps described in the documentation.
Can I establish a link between the synchronized Azure DevOps Test Case and another work item, like a User Story, Feature, Product Backlog Item or a Bug?
Yes, you can configure SpecSync to synchronize scenario tags as Azure DevOps work item links, e.g. if the scenario is tagged with @us:123, you can configure SpecSync to link the synchronized test case to the Azure DevOps User Story #123. You can specify different tag prefixes and link types. See more at: Linking work items using tags documentation page.
Is it possible to synchronize SpecFlow or Reqnroll scenarios to existing Azure DevOps Test Cases?
Yes. If you synchronize a test case which does not have a test case tag yet, SpecSync will create a new one and tags the scenario. But if you add the test case tag manually (in the format of @tc:123), referring to an existing test case in Azure DevOps, SpecSync will update that test case at the next synchronization.
How to synchronize scenarios with a multi-team Azure DevOps project?
Multi-team Azure DevOps projects usually use different Azure DevOps Area Paths for each team. The team members might only have a read or a write access for the work items in the area of their own team.
SpecSync creates the test cases by default in the root area, which might be restricted for the user performing the synchronization. In these cases, you can change the area path to be used for creating the new test case . (Similarly you can specify the iteration as well.) See more at Add new test cases to an Area or an Iteration documentation page.
Can I use SpecSync without Visual Studio license?
Yes. SpecSync is a command line tool that can be executed from a Windows, macOS or Linux command line shell.
About using SpecSync on OSX/Linux-based systems, please check the documentation page: Using SpecSync on OSX/Linux.
Can I use SpecSync with Visual Studio Standard license that does not include Microsoft Azure DevOps?
Yes. Microsoft Azure DevOps is a desktop application to manage and execute test cases locally, but these tasks can be performed from the Azure DevOps web interface as well.
Can I use SpecSync from OSX (Mac) or Linux?
Yes. The command line tool works also on Linux and macOS. Please refer to the documentation page: Using SpecSync on macOS or Linux for details.