Defensics Protocol Fuzzing Reviews

Presently we are using Codenomicon Defensics for a few suites, only for testing. we are doing few open floor tests. Nexus is the one which we will use for European testing. For US usage testing, we will use Codenomicon.

It tests for switches and router sections. We use it for product testing. We will get the license and then bring it back to the IT team.

The product is related to US usage with TLS contact fees, how more data center connections will help lower networking costs.

Codenomicon Defensics should be more advanced for the testing sector. It should be somewhat easy and flexible to install. 

What I see in the documentation isn't that. Even if something doesn't malfunction, sometimes it is hard to install and execute. The product needs video documentation. This would help a lot more.

The stability of this product is great. We tested it under multiple constraints. Even on cloud services, it is absolutely stable.

Our spread is scalable. Our internal team is using it. It is mandatory for us to check for every new release for Codenomicon updates. 

We are going to use it, but I don't see it increasing from the present levels of usage. Even for our internal releases, we will use it.

The technical support we didn't use much. We take it from our internal IT team. That is the initial source. They will take care of it for clients as well.

We used Nexus sometimes as an alternative to Codenomicon.

The initial setup is straightforward. It is like Fuzz Testing.

We approve and even suggest the product to the people who are doing security testing because this product is far more easy to use.

I would rate Codenomicon with an 8 to 9 out of ten. Unless these tests are passed, we will not go live internationally.

Codenomicon is a good tool. It is used for network protocol implementation testing, to find any Zero-day issues, vulnerabilities in XSL. It's an excellent tool that has its advantages and its limitations.

In our company, we have a lot of applications. A lot of protocols are used between embedded devices which are never tested for any abnormal behaviors. We have found multiple issues in our embedded system network protocols, related to buffer overflow. We have reduced some of these issues.

Whatever the test suit they give, it is intelligent. It will understand the protocol and it will generate the test cases based on the protocol: protocol, message sequence, protocol, message structure. That intelligence is very good. Because of that, we can eliminate a lot of unwanted test cases, so we can execute the tests and complete them very quickly.

Sometimes, when we are testing embedded devices, when we trigger the test cases, the target will crash immediately. It is very difficult for us to identify the root cause of the crash because they do not provide sophisticated tools on the target side. They cover only the client-side application, and from that we can generate automated test cases, but what happens on the target device, what is the reason for the crash, for that we have to do manual debugging. They do not have diagnostic tools for the target side. Rather, they have them but they are very minimal and not very helpful. They can improve a lot on that.

Technical support is good, they are very active, very supportive.

Licensing is a bit expensive.

It depends on your company's needs. Codenomicon will only help to identify protocol implementation issues. If a company is using a standard stack, which is already tested, then I don't think they will hit any issues while doing fuzzy. The tool is used only for the protocol implementation, done by the company itself.

I would rate Codenomicon at seven out of 10. It does not support the complete protocol stack. There are some IoT protocols that are not supported and new protocols that are not supported.

This collection of storage-related components were most valuable in extending a security assurance program into the area of black-box security testing for a NAS appliance.

A security assurance engineer was able to perform due diligence across all network-facing protocols.

My prior organization designed, developed and deployed a Network Attached Storage (NAS) appliance. A key part of the company wide security assurance program for all products, is to perform penetration testing against all network facing IP ports.

For the web, SSL and RESTful APIs, there are very good COTS and open source tools to perform Dynamic Application Security Testing (DAST) testing. Unfortunately for NAS protocols like SMB, NFS, CIFS, and iSCSI, I researched and found that Codenomicon Defensics was the only viable source to satisfy our DAST requirements.

Through the use of Selenium for automated web testing, it was easily found out that Codenomicon Defensics could be integrated into our Continuous Integration / Continuous Deployment (CI / CD) Agile processes, specific to automated testing.

Also, like many of the other application security testing products, Defensics incorporates automatic update support and works on Windows, MacOS and Linux desktops.

It requires understanding the Defensics protocol.

I have used it for five years.

I have not encountered any deployment issues. The product works as, or even better than, expected.

I have not encountered any stability issues.

I have not encountered any scalability issues.

Customer service is excellent.

As with most application security test suites, there are "false positives". On multiple occasions, Codenomicon technical support provided the details and protocol-specific documentation to prove that the positive was not false.

The step was very straightforward, error free, on multiple OS platforms.

An in-house team implemented it.

ROI was 100%. Since there are no product suites available that provide the level of testing available with Codenomicon, the development, quality and security assurance departments know that the investment was correct.

Start out with a single use per protocol, and expand to multiple units as needed.

No other COTS or open-source software fulfilled this testing requirement.

There are various protocol-specific testing suites, but most do not focus both the depth and breadth of each of the protocol's specific features.

• Test cases are not generated on the fly (which means that it isn't really fuzzing per se). They are organized in groups and defined according to the type of message and the tested part of the message. Compared to more random tools, fuzzing sessions can take less time and be more relevant.
• Simple and straightforward GUI.
• Context-sensitive helps in describing every configuration field along with their CLI equivalent.
• You can set a test sequence and thus test several protocols without any user interaction, and it can be sequential or in parallel.
• Interoperability feature which enables the user to ensure that the SUT supports the various types of tested protocols' messages. If a type of message failed the interoperability test, it won't be included in the fuzzing session, unless you want them to be included.
• Instrumentation capabilities (valid cases, ping, custom command) and actions (execution of a restart script of a device after a given number of failed instrumentation steps) upon instrumentation results.
• Reproduction of single test cases or along with the rest of the test case group.
• Network capture during fuzzing session as well as during the reproduction.
• Top 100 of the test cases which caused an important delay on the SUT response. Those cases are reproducible in order to check that the same test cases caused the unwanted behavior. This is useful for covering not well processed frames that don't necessarily make the SUT crash.
• Different sets to use depending on the available time and the coverage wanted (Full, Unlimited, Quick Run, Sample, etc.).
• We can create custom test cases by setting a value or a range of values on particular fields of a protocol.

• You can't implement proprietary ciphering algorithms, nor can you modify protocol models if you need to test customized public protocols.
• You can't use the program at all without the USB license dongle. This would be useful for instance to export results, prepare the wizard, and so on. It can be inconvenient if several teams use the license.
• Time estimation: order of magnitude is not always respected.
• To test ARP on the client side, you have to clear the MAC table of the SUT. A feature such as sending ping requests to the SUT with a different virtual IP/MAC address each time to force the client to send ARP request would be great.
• No automatic bug reproduction (as Peach has for example).
• You can't create a protocol model from scratch using the GUI. You can use the traffic capture fuzzer, import a PCAP file and generate tests cases from it. Known protocols are described according to a wireshark dissector, proprietary protocols have to be defined manually (by defining a label on a part of the data). It seems that we can go further with the Java SDK, but we didn't have enough time to test it.
• When using the GUI, you can't run fuzzing sessions both sequentially and in parallel at the same time, for instance for testing different protocols on different devices. One possible workaround is to use the CLI of Defensics and to use different configuration folders.
• When you choose the network interface to use, there is an «auto-configuration» box ticked by default. It means that Defensics will try to guess the interface you will use, but it often lead to mistakes.

I've been using, mainly evaluating, it for two weeks.

No issues encountered.

No issues encountered.

No issues encountered.

Very reactive and efficient.

I didn't use a previous solution.

There was nothing difficult, it was all typical Next>Next>Finish wizards.

We also met people from BreakingPoint (Ixia) and µSecurity (Spirent). I also tested an open-source solution, Peach.