OpenVINO Reviews

I use it in a project specifically with the models from Microsoft. These models, one was Florence-2, which is a model specialized in multiple computer vision tasks. The other was Microsoft Phi, which is a generative AI model for generative AI, also from Microsoft. Both of them were used for testing, not for this project. I also have a blog on YouTube where I do many experiments with different toolkits, and I tested it with other non-commercial models as well. For this specific project, I used it with two models from Microsoft. The idea was to run the models in Florence-2 and Microsoft Phi-3 in CPU for two specific use cases.

What I most appreciate about OpenVINO is the possibility to run models in CPU. That was obviously the reason to choose this solution. Another valuable feature is the possibility to run on non-NVIDIA GPUs. As this is a product developed by Intel, they developed it looking at the hardware, which is part of their house, to run inference or models, LLMs or deep learning models, in CPUs or from Intel, an Intel CPU or also Intel GPUs.

Before, it was not possible. The GPU market is completely dominated by NVIDIA. OpenVINO allowed me to run a model in an Intel CPU. Also, any framework PyTorch, TensorFlow, deep learning frameworks, they only have compatibility with NVIDIA GPUs, and OpenVINO opened the door to run LLMs in non-NVIDIA GPUs.

The benefit from using OpenVINO is that NVIDIA is dominating the market of GPUs and they set the price. There is no competitor. If I am able to run an LLM doing inference in commodity hardware, I am saving costs. To buy a GPU to run one of these models from NVIDIA costs a minimum of 5,000 pounds, and it's a very small GPU from NVIDIA. This cost is pretty high in comparison with a GPU from Intel or even a CPU with the same performance or similar performance on Intel. It is a cost saving solution and offers the possibility to run in cheaper hardware, which ultimately also saves cost.

What could be improved in OpenVINO is making the product more cross-platform. I know they are working with third-party plugins to extend the toolkit, and in this way, I can use it with NVIDIA GPUs or with other hardware because now it's primarily working in all Intel hardware. CPU, GPUs, TPUs, but only from Intel. If they make more cross-platform functionality, it would be great. It's difficult to make it work faster than the NVIDIA toolkit in their own GPUs. At least having the possibility and making it work faster than now in other hardware that is not from Intel provided would be beneficial.

I have been using OpenVINO for the last two years.

Regarding stability, it is good. Once I have the model running in OpenVINO, it is quite stable and performs on the machine. That aspect is good. Intel designed this product, OpenVINO, and they know their hardware, which is the same as NVIDIA with TensorRT. Their engineers know the hardware and they produce a product with very good quality. When I am running OpenVINO in Intel hardware, it's quite stable and works very efficiently.

In my opinion, OpenVINO is more designed for the end user or a solution because when I run this in CPUs, the reason to use it is for saving costs and then run this in cheap hardware. I think that it's not properly designed for scalability. It's designed for other purposes, specifically to be able to use Intel hardware and run inference using generative models or deep learning models in Intel hardware. I don't think that the product itself is designed to scale to multiple clusters of servers and this kind of implementation. It's probably not designed with that intention.

The support team has excellent documentation. It's complex and requires deep reading of the documentation. You need to have some skills in engineering products and ML products. It's complicated, but to be fair, with others such as TensorRT from NVIDIA or Onyx from Microsoft, they are quite similar. They are complex to set up and to convert models into their proprietary format.

I need to have experience, and there are many cases of compatibility problems. The product is not easy to use, and engineering skills and experience are necessary for these tasks. The documentation is comprehensive. I never was in touch with Intel directly. I used the library as it's open source, so I can use it for free. I didn't need to use the support.

When I had issues, I was able to find what I needed in the documentation available on the repository or on the website of OpenVINO. It's quite thoroughly documented. There are many examples of using OpenVINO with different models, for different purposes, and different use cases. I can take the examples and reuse them for my local or particular problem. Good skills in Python and machine learning are necessary, which is similar to other complex tools.

Positive

The initial setup and deployment of OpenVINO is complex to use, but this is similar to other solutions. These products are not easy to set up and require experience in machine learning. It's not for a junior developer. A senior engineer knowing what they're doing is necessary because they are complex to set up. This is especially true when working with containerization and needing to train or fine-tune models using the toolkit. It is a complex setup. The tool requires a specialist, a good specialist to work with.

Regarding their pricing, it's open source with no cost for using it. There is no licensing involved. It follows the line of other products in the same area. TensorRT from NVIDIA has no cost and is open source as well. Onyx from Microsoft, which is also doing something similar, is also open source. I don't need to pay a license to use it.

My company doesn't have a partnership with Intel. It was a product that I needed and used. It is a company that creates a product which is developing, introducing AI features on this product. It's a data governance and eDiscovery tool. I was helping with this project to introduce AI capabilities on the product. The company itself, Santax AI, didn't have a partnership with Intel itself. From Intel's perspective, they are just users.

Positive

I wanted to use OpenVINO for my Raspberry Pi to analyze my sleep with a night vision camera and to improve GPU performance on my Raspberry Pi.

I would have used OpenVINO's Model Optimizer feature to optimize the model for night vision to track body patterns, such as movement during sleep.

Unfortunately, I haven't implemented it yet, but I hope to utilize OpenVINO's capability to optimize neural network models from popular frameworks such as TensorFlow or PyTorch soon.

The only benefit I have seen from using OpenVINO is the GPU boost performance for the Raspberry Pi.

I have heard good things about OpenVINO. It doesn't consume much current for external GPU usage. However, it has some downsides because I couldn't get it to run on my Raspberry Pi 4.

While not specifically for my use case, I would be happy if the software packages were available for the Intel Neural Compute Stick 2. It is actually a really good product, but I couldn't get it to run on my Raspberry Pi 4 because the software packages to download were no longer available. They have been deleted. A wish would be to maintain the software packages so that even after five or ten years, they would still be available to download and install.

It would benefit me because it would run the model much faster than without using OpenVINO to boost the GPU performance.

I have been working with OpenVINO for about three to four months.

It is difficult to say because I couldn't apply OpenVINO extensively.

I would find OpenVINO stable.

I didn't try to scale OpenVINO up or down.

I haven't had any questions escalated to Intel technical support.

Positive

I hadn't used anything before, but afterward, I used the Google Coral TPU because the software packages were available, so I could run it on my Raspberry Pi 4.

I didn't actually get far enough to measure the cost-effectiveness of OpenVINO with any metrics.

I got OpenVINO through the vendor themselves.

There aren't any specific metrics that I use to evaluate the effectiveness of OpenVINO.

The pricing of OpenVINO is reasonable.

One counter argument would be that the Intel Neural Compute Stick 2 consumes a little more current than the Google Coral TPU. However, one positive aspect about OpenVINO is that it supports more frameworks than the Google Coral TPU.

The solution itself is excellent, and I would rate OpenVINO as a ten out of ten.

We currently make technology that uses the Intel VPU, the Movidius chipset. We run OpenVINO on it.

It's for Edge IoT. We make the hardware and we cater to customers who are looking for Edge IoT solutions, and the product is really for edge processing or video co-processing for machine vision. We distribute that data on the customer's network using our Edge solution, which is based on DDS, distributed data system. Basically, we use it for machine vision applications.

The solution's ability to stream data directly from camera inputs is the most valuable aspect for us. 

It's tailored to the Movidius chipset, which makes it a nice package. You don't have to run it on the Movidius. It runs on X86, however, we'd like to use it with our Movidius based co-processor. 

The ease of integration is fantastic. The option to run it just on X86 or X86 plus an Intel CPU is great.

It's an open-source solution.

The initial setup is quite simple.

Generally, when you deploy edge products, it's really about latency. It's about getting that camera input, being able to process it, extracting the information you need, and getting the solution back to the person who made the request. Although I'm not necessarily saying its latency or accuracy is bad, it's always something that can be improved upon. By focusing on improving these areas, they can make the overall solution even better.

At this point, the product could probably just use a greater integration with more machine learning model tools. However, that's not advice from experience per se. That's always just helpful in general. To be able to incorporate more models into the product makes it stronger. Therefore, to be clear, it's not coming from a point of a current deficiency. It's just a general comment.

We've used the solution for a while now. We've used it over the course of the last 12 months as well. My personal experience has been a bit more limited and I would say that I've been using it for the last eight or so months.

The solution is quite stable. There are no bugs or glitches. It doesn't crash or freeze. It's reliable.

The scalability seems to be pretty good. We've had good feedback on it. Our engineers seem to like it, and, for me as a kind of an end-user, it seems to be working fine.

As gold partners with Intel, we do get pretty good support. I do get feedback from them. I have key contacts that I can reach out to directly, and then they're fairly responsive. We've been quite satisfied so far.

The initial setup, for me at least, was pretty easy and straightforward. It is integrated with our binary image for our platform, so it already comes with it. The engineers already integrated it with our hardware solution and it comes to me as a binary that I install. It makes everything very simple. I wouldn't describe the setup as complex.

That's going to be really on our customers. We're not using it as a revenue generation tool. We use it as a development enhancement and then we sell our solutions to the end customer. The ROI is really a question for the end-user, on how they can realize their initial investment back.

This solution is made available as an open-source product.

I have done a cursory bit of research into the product against a few other options.

I'm not exactly sure which version of the solution we're using. I'm assuming it's fairly current as we're deploying our Edge IoT platform using it.

While the solution is deployed on-premise, but we have the availability to hook into Amazon web services and Microsoft Azure.

OpenVINO's part of Intel's framework and we're a gold partner with Intel.

I would say it's a very good vision processing unit. It's a very good VPU if you're using Intel type of architectures as a co-processor. I have only really had experience with Intel with any OpenVINO based on the Movidius and therefore I'd like to get more hands-on time with others. However, generally, it's a good platform. It's worth exploring and can handle multiple camera streams, and it's straightforward to use. A company would benefit from trying it out.

In general, I would rate the solution at an eight out of ten.

I created a retail recognition custom model — a model on RPX 2017. Afterward, I transferred it to OpenVINO for object detection and retail detection.

We have a team of three people who use OpenVINO.

It's a great solution for getting camera images, processing them, and extracting the reserves. It's better and more cost-effective than using Intel myriad X.

The inferencing and processing capabilities are quite beneficial for our requirements.

Compared to Jetson Nano or Jetson TX2, or Jetson Xavier, OpenVINO is a much more cost-effective solution. Processing-wise, they are comparable to Jetson and maybe Jetson Xavier NX.

The model optimization is a little bit slow — it could be improved. They should introduce some type of deep learning accelerator, like Jetson Xavier NX.

There is a lacking in vehicle recognition — types of vehicles. Differentiating between cars, SUVs and different types of light, heavy, and medium trucks can be tricky. We have to train such models ourselves and then transfer them onto OpenVINO.

I have been using this solution for roughly 18 months.

It works quite well, but regarding the frames per second — If we want to add on a few more cameras to the hardware, we can't do that.

By assigning different IPs to our own software, we can access different cameras on a real-time basis, to analyze the traffic from different lanes of vehicles and roads.

This solution has never crashed and we haven't experienced any bugs; however, there are improvements that need to be made.

I didn't scale it, but I want to build more than one board, on top of each other, so that we can improve the processing compute capabilities.

The technical support is very good.

I didn't experience any difficulties when setting up this solution.

We have a really small team of about two to three people. One person has to go to the site and set up the hardware so we can access the board remotely. We can work from our office or any location. It's a one-time installation process; from there, our team can work from the office.

We were planning on deploying this solution on different devices, so the planning aspect took a little bit of time, but overall, we were up and running within a week.

We didn't have to pay anything for Intel OpenVINO, everything was available on their site. All of their solutions, inference engines, and other model optimizations are all available for free. 

We didn't have to pay for any licensing with Intel OpenVINO. Everything is available on their site and easily downloadable for free.

We are currently trying Nvidia and Xavier NX because some of their models and capabilities look promising.

We are comparing the processing and usability of these two boards and we are planning to have a model GPU board with us also.

I would absolutely recommend OpenVINO. I think it's a good introduction to machine learning and inferencing.

On a scale from one to ten, I would give OpenVINO a rating of nine.

OpenVINO is good for budgets because you don't have a computer vision model for classification for object detection obligations. You can run it on a server with Azure but it can be costly. Sometimes the application has to be on heavy dedicated hardware, like a small computer. In this case, machine learning applications are not so good because they demand a lot of computer resources and a lot of CPU resources are not so fast. In terms of accuracy and speed trade-off performance, you have to sacrifice a bit of accuracy in your inference in order to get better speed. When you deploy models with the OpenVINO format into devices like PC boards, it's a great tool. The online testing platform that Intel has for OpenVINO is really nice.

It's sort of like a sandbox environment. You want to test what kind of hardware is available. You can test it and watch what works better in talking about the preference. Later you can decide based on the budget.

The features for model comparison, the feature for model testing, evaluation, and deployment are very nice. It can work almost with all the models. 

It has some disadvantages because when you're working with very complex models, neural networks, if OpenVINO cannot convert them automatically and you have to do a custom layer and later add it to the model. It is difficult. These are the main disadvantages that OpenVINO has that are a bit limited for some models.

I have been using OpenVINO for two years. 

Stability is good. It's growing fast and they implement fixes for old issues. I hope the installation and usage could be more clear and easier to implement for the deployment. 

Scalability is not so straightforward, it's not so easy. When working with some applications, using some of our scaling options to manage those multiple containers is not so easy. But it is a good tool, it's a good tool for working with Intel devices.

It took some time to install. Using the Docker container is not very straightforward. It raises many errors. The main installation has a lot of steps and there are many details that are easy to miss. You have to do it two or three times to get it right. It could take six to eight hours to set up. You have to test and build.

I have been using the open-source version, so it's free. 

Install the latest version that already has fixes for old problems. Work with some neural network, with a few layers to test it. If you use a neural network, like a fast R-CNN, it wouldn't work because it's too complex and there are some layers that are not recognized by OpenVINO. Start small and continue growing. Make an account in the Intel OpenVINO platform so you can start testing with Jupyter Notebook and send your inference jobs to different kinds of devices. Check the performance, know the differences between the different hardware, and how you can site the project. It is a good platform. The price I have seen is not so expensive. That's my advice.

OpenVINO is a good tool if you want to work not in the cloud, you have to work on the edge. If you want to adapt your models, it has a good pipeline to follow. You have to learn it well because it could take time to debug some of the errors. Some of them are not very explained and you have to go through your codes a bit blind.

I would rate it a nine out of ten because it's a very good tool. It's not complete but is the best in the market right now.