SpecFlow Reviews

In our development workflow, we use SpecFlow as a BDD framework to bridge the communication gap between business and development teams. We create Given-When-Then scenarios in plain language using Gherkin syntax, allowing easy collaboration with stakeholders. Under the hood, we implement these scenarios in C#, aligning with our .NET environment. Our use cases range from backend testing, making API calls, to mobile testing with APKs using APM Inspect or APM desktop. SpecFlow's ability to integrate seamlessly with our CI/CD pipeline ensures efficient testing and validation of both business and technical requirements.

SpecFlow enhances our organization by translating logical understanding into executable code in our C# environment. Similar to Cucumber in Java, it converts scenarios into callable methods, promoting collaboration and code interpretation. The framework's features, such as tagging and data segregation, provide flexibility and reusability without altering the core code. This allows us to efficiently run tests in different environments with varying data sets, streamlining our testing process.

One of the most valuable features of SpecFlow for us is its risk identification capabilities. This is particularly helpful for business development, managers, and product teams. We can tag risks in the requirement phase, even if user stories are distributed across different teams. This way, we can easily identify and concentrate on high-priority test cases, making it seamless for various teams to collaborate on risk assessment and validation.

In terms of improvement, SpecFlow's installation and configuration can be a bit challenging due to its flexibility as an open-source tool. While it offers a balanced flexibility, setting it up might require more effort compared to some licensed products that handle everything. However, SpecFlow is a framework, not an automation tool, so the actual automation is carried out through tools like Selenium. Despite setup complexities, the framework itself serves its purpose effectively. In future releases of SpecFlow, it would be beneficial to have some built-in methods for common actions like opening and closing browsers or implementing loops. Having predefined libraries for these functions would save automation engineers time and make the framework more user-friendly. While I'm not sure if such features already exist, they could be valuable, especially for those working on web applications. It would provide a generic and optimized solution that anyone can easily implement without worrying about performance issues.

I have been working with SpecFlow for over a year.

I would rate the stability of SpecFlow as a solid ten out of ten. 

SpecFlow is generally well-regarded for its scalability. Many users find it effective, although there is feedback about the time it takes to structure tests using the Given-When-Then format. Converting logical validations into SpecFlow scenarios can be time-consuming, especially for manual testers. I would rate SpecFlow's scalability as moderate. While it can increase capacity and functionalities based on demands, the effort required to structure tests using the Given-When-Then format can be significant, especially for complex projects. The process of converting requirements into human-readable Given-When-Then scenarios, while essential for understanding code, can pose challenges in maintaining traceability. Reviewing automated tests involves going through both the requirements and the breakdown into SpecFlow, adding a layer of complexity. In our company, three or four users, primarily developers, are currently using SpecFlow. SpecFlow is heavily used in our projects, running daily behind each release during the build phase.

Before SpecFlow, we primarily used Selenium. SpecFlow simplifies the communication between code and functionality, making it more accessible to a broader audience.

Unlike some licensed tools that provide a straightforward installation process, open-source tools often require more manual configuration and may lack a predefined folder structure, making it a bit more involved. If SpecFlow includes a well-defined folder structure in future releases, it could simplify the setup process.

We explored other options but SpecFlow was chosen because it helps translate complex requirements, especially those involving risks in invoices, into human-readable language. The decision was likely influenced by the need for clarity in understanding and validating requirements quickly, which SpecFlow facilitates by sitting on top of the automation code.

I would recommend SpecFlow to anyone who wants a clear understanding of the logical flow in their automation. It is beneficial for comprehending how things work. Overall, I would rate it as an eight out of ten.

In my company, we use SpecFlow as an automation solution.

The most valuable feature of the solution is unit testing.

SpecFlow has good integration capabilities with other products.

Regarding improvement, it would be good if SpecFlow could provide chain testing, which it currently doesn't allow. Rather than doing unit testing, we could use chain testing.

I cannot speak about the new features to be added or how to handle the product since I am not an expert. I know some features of the product. Some more features can be added, but I don't know if the product already has those features. It's a difficult process to speak about what can be improved in the product.

I have been using SpecFlow for a year. I use the solution's latest version.

It is a stable solution. Stability-wise, I rate the solution a seven out of ten.

Scalability-wise, I rate the solution a six out of ten.

Expanding the number of devices or users won't be a problem.

My company has around five to ten solution users who are engineers from the middle management cadre.

I haven't registered yet to be able to use the product's support. You need to go to the support since there isn't enough online documentation. I have not tried to contact the support, though I feel that the support is straightforward. So far, the product has been good because of which, we have not faced any issues in our company with the solution. SpecFlow is a clearly defined product.

I rate the technical support a nine out of ten.

Positive

Previously, I had used the chain method in Java. I have experience with Selenium and RadView. I shifted from Selenium and RadView to SpecFlow after I left my previous company.

The solution is deployed on-premises.

Initially, SpecFlow was integrated with .NET, which was like a plugin that you can use. I was not here when the product was installed or built from scratch in the company, though I currently use the product.

I believe that installation and the entire configuration process would require a few days to be completed.

SpecFlow is an open-source product. You just use SpecFlow's plugin capabilities or add it to your existing framework.

In our company, we need to maintain the scripts of the product daily, and in case of failures, we need to check out the scripts.

I would recommend the product to those planning to use it based on both they are planning to build. If you are going for unit testing or small scripts, then you can go with SpecFlow.

I don't have much experience with the solution. It is a good and straightforward product. It is also an easy-to-use and user-friendly product that can easily adapt to any framework.

I rate the overall solution a seven out of ten.

We are using SpecFlow to write down scenarios in Gherkin. And then, we put detailed coding in Selenium in C#. For the high-level sanity testing of the APIs, we use SoapUI for the SOAP APIs and Postman for the REST APIs.

With Cucumber and SpecFlow, the purpose of both is the same. We use a lot of Microsoft technology, so that's why we use SpecFlow. It helps us to write down the scenarios in Gherkin and write down the implementation of each step and scenario.

I don't see any other options. So that's why the only option for us to write down the Gherkin for the Selenium cases is the SpecFlow solution. 

The initial setup is easy.

It is stable. 

There should be more integration with the application life cycle management tools, including integration with Jira and Azure DevOps. We used to write our test cases in Microsoft Azure DevOps, which were deployed on-premises. We would write test cases in a classical way and the test data within the test steps of the test case. When we have to write down the problem, we write scenarios in Gherkin and then automate them in Selenium.

That said, there is no way to have traceability between the test feature files of this backflow with any work item in TFS, Azure DevOps, or in Jira. We cannot link them. While we can link the code, this feature file or this Selenium file cannot be properly linked.

I'd prefer in TFS if we could be writing test cases, not in the old classical version. We should be writing in Gherkin and then automatically have it convert that Gherkin test case into SpecFlow feature files. 

There is room for improvement in managing the test data related to the test cases.

I've been using the solution for about three years or so. 

The solution is stable and reliable. There are no bugs or glitches, and it doesn't crash or freeze. it is reliable. 

I'm basically supervising the whole quality team, and the whole team uses it. We have eight people on it currently. They are software test engineers. 

All versions are working for us. However, we don't always use the latest version as we work for a government agency, and it's hard to get permission to update. 

I haven't had issues with scaling, even on older versions. It seems to work well. That said, I haven't explored it really.

I've never reached out to technical support.

Before using this automated testing solution, we did POC, proof of concept, to find out which tool would be suitable for our application. We have a web application, a mobile app, and a hybrid as well. We did use different tools, including Katalon and Selenium. Coded UI expired, so we stopped that.

We also looked at Cypress. However, since we are a Microsoft partner company and we are using Microsoft technologies, Microsoft recommends using Selenium with C#. With that being the case, we only had one option - SpecFlow. 

Otherwise, if the company was a Java-based company, then we might not be using SpecFlow, we might be using Cucumber.

The solution is very straightforward to set up. It's pretty normal. We just have to use it in Visual Studio.

The deployment doesn't take too much time. 

The solution is open-source. We're not paying for anything. There are no hidden fees either. 

I don't have any other options, so I didn't compare them with any other options.

We're end-users.

I'd advise new users to just have a basic understanding of the writing scenarios in Gherkin and know how to manage the test data within the test cases. When the number of test cases increases, you've got to be ready. 

People who do not write well in Gherkin end up messing up the scenarios. They don't know how to make test cases more cohesive, or how to have test data that are more cohesive.

I'd rate the solution six out of ten.

I use SpecFlow in my framework to add a layer with the English statements to my programming.

SpecFlow's best feature is the ability to add additional layers to the programming.

SpecFlow would be improved with the addition of functionality reporting, which would be really helpful for automation testing.

I've been using SpecFlow for around four years.

SpecFlow is stable.

So long as your framework is stable, SpecFlow will scale with it.

The initial setup is easy, but you need some knowledge of programming. Deployment takes no more than three or four minutes.

SpecFlow is open-source and free of charge.

I'd advise anybody thinking of implementing SpecFlow to make sure they have knowledge of programming with automation or Microsoft technologies. I would rate SpecFlow eight out of ten.