Amazon Kinesis Reviews

Our primary use case of this solution is for a streaming bus architecture, we get events and they come in through Kinesis like a Jason event. It's usually a change to a database, but it can be any event such as in our application, which feeds into the Kinesis and then we have a Lambda that consumes them and then finally it puts those into a data warehouse which is the ultimate goal. So it's a near real-time data warehouse. 

Instead of doing batch jobs of an ETL, like moving data into a data warehouse, we're now able to do it all through a continuous stream in Kinesis. It means that the data is more up-to-date in our data warehouse, it's more real time.

I've used Kafka in the past and Kinesis is a lot simpler. It's all hosted, it's nice it's really good. There aren't too many knobs and things to turn and ways to screw up. It's a pretty simple product and a lot easier to manage because it's hosted by AWS and it accomplishes what we need it to. The other nice thing is that we can make it available to external customers if they want to get a Kinesis feed of our things.  

I would say that the solution probably has the capability to do sharding so that you can do a lot of things in parallel. I think that the way the sharding works could be simplified and include features that make it easier to scale in a parallel way.

I've been using this solution for five years. 

It's much easier to maintain than a Kafka cluster, but there were some nuances with it. I'd say maybe once a month or once every couple of months we'd get some weird things like AWS getting overwhelmed or the service was degraded for a few hours in a day. I guess that's a drawback of going with the fully managed service, it's just that it depends on the company keeping everything up. To me it's pretty stable, we would just get kind of slow once every month or once every couple of months. Overall I would say it's not perfect, but it was totally great, acceptable.

You can do a bunch of shards, we only use one or a few shards and you can scale way up. It's way more scalable than we ever needed. And we were doing massive, millions of updates per minute. This is a backend service, so it's mostly used by developers and data engineers that were using Kinesis in our company, but our customers all benefited. Customers were sending send data through it, but they don't know that they are using Kinesis. We built the system and then they use it, so they don't know that under the covers it was consistent. We have five to 10 internal employees using it, but probably around 5,000 customers. Where I worked before, we used it basically everywhere. Probably 70% of all of our data was falling through Kinesis. Where I am now, we're just starting, so it's not yet being widely used. We plan to increase usage. 

When we had some of those slow downs, we used AWS support and I can't say that we had a great experience and they resolved the issues, but they looked into some of the flow downs and ultimately we just decided there was nothing we could do. It left me feeling there was something lacking on the technical support side. They didn't get to the bottom of all my issues, but the issues weren't bad enough to be unhappy about the product overall.

We previously used Apache Kafka. We switched because we were already using Amazon for everything else so it made sense, and it was a nice managed solution that would be a lot easier to deal with. It also integrated well with Lambdas. The whole AWS ecosystem is nice to work with because everything integrates with each other.

The initial setup is definitely straightforward because it's a managed service and you only get a few options when you set up a Kinesis stream. It's a lot less overwhelming than setting up a whole Kafka cluster or even if you've managed Kafka, there's still a lot of configuration required to get it up and running and all the choices you can make about topics and things. Kinesis is just much simpler. It only lets you configure what you need to configure. I'd say that kind of POC took about a week and then real production probably a month. We used Terraform for our implementation strategy, but I used CloudFormation in my past job to do that. The deployment was essentially running the CloudFormation template.

Compared to what we were doing with Kinesis or with Kafka, which was taking about 30% just to keep things together, with Kinesis I think we're probably saving tens of thousands, if not $100,000 per year. 

I would say pricing is really great. If pricing is an issue, I'd definitely recommend Kinesis because our Kinesis costs are under $1,000 a month. The product is super cost effective and it's the same with the licensing. Compared to Google Cloud and Azure, they're probably pretty similarly priced. I wouldn't say you're going to get a huge benefit going to Kinesis, but if you're considering using Kafka or another solution that's not hosted, it's not really worth all the effort when you could just go with a managed solution. It's a lot better cost-wise.

We took a look at Google cloud and Azure, they're Pub/Sub solutions, but not really in depth. Because we were already using Amazon, it just didn't make sense to use any other cloud provider.

It's nice to deploy this with the Amazon goodness of Cloud Formation and Terraform, to have it all deployed in a repeatable way. I know that it's easy to go into the console and do it manually, but it's best to do infrastructure as code, in particular with Kinesis.

I would rate this solution a nine out of 10. 

We work with our customers using Kinesis in conjunction with Amazon Connect because Amazon Connect relies on the Kinesis product for voice and screen recording. This is the primary use case for utilizing Kinesis.

If the agents do not receive real-time transcriptions, they will not know the actual words being spoken, impacting the overall experience. Kinesis aids our organization by providing these real-time transcriptions and sentiments processed by other services.

The Kinesis VideoStream and DataStream are the most important features. It provides a high rate of throughput, which is ideal for real-time streaming of voice or video.

I do not see any scope for improvement as it does what it is supposed to do. No changes are required. Since it's predominantly a back-end service, any end-user isn't going to interact with it directly. It's more for developers and architects.

I have been working with Amazon Kinesis for around four to five years.

There could be valid data in Kinesis that is not being processed, which affects stability. Although it rarely happens, this issue has been observed in many deployments, making it not completely stable.

The data stream and sharding are very important features. The sharding gets throttled, allowing scaling based on the number of video calls or voice calls a particular customer encounters. The platform scales up well, offering good configuration.

The customer service and support could be improved because it's a different team, and they take a bit more time analyzing the logs and returning with a response.

Neutral

The initial setup would be rated as seven out of ten. It is generally easy but has a few challenges.

It usually requires the developers and architects for deployment; it could be one to three people.

Amazon Kinesis is moderately priced. In comparison with other competitors, it is fairly priced, however, if they reduced the price a little, it could add more value to customers.

Ensure you consider the volume and the amount of data you want to process and provision them accordingly. Evaluate the prerequisites well as it's a back-end service.

I'd rate the solution eight out of ten.

The solution provides real-time event streaming. 

From my experience, one of the most valuable features is the ability to track silent events on endpoints. Previously, these events might have gone unnoticed, but now we can access them within the product range. For example, if a customer reports that their calls are not reaching the portal files, we can use this feature to troubleshoot and optimize the system.

I suggest integrating additional features, such as incorporating Amazon Pinpoint or Amazon Connect as bundled offerings, rather than deploying them as separate services.

I have been using the product for a year. 

My company has four to five customers. The scalability of Amazon Kinesis has improved various data processing capabilities.

I am happy with the tool's customer support. 

Amazon Kinesis' deployment is easy. 

The tool's pricing is cheap. 

If you're considering using Amazon Kinesis for the first time, I would advise exploring their services. It can be useful for capturing real-time events, greatly benefiting your solution. I rate it a seven out of ten. 

In terms of use cases, it depends of which component we're talking about, as we use three of the 4 components. The only one we don't use is the Video Streams.

Kinesis Data Stream is the module that we have been using the longest, essentially we use it to hold data which will be processed by multiple consumers. We have multiple data sources and we use Kinesis to funnel that data which is then consumed by multiple other consumers. We gather data coming from IoT devices, user phones, databases and a variety of other sources and then, as we have multiple consumers, we use Kinesis to actually gather the data and then we process it directly in Lambda, in Firehose, or in other applications.

Amazon Kinesis has absolutely improved our organization. Before Data Streams, we were using a couple of other solutions, including Talend and Pentaho, to move data around. Each of them were their own silos. So the ability to have one single flow of  data from multiple consumers simplified our architecture a lot because you didn't need to copy or read the data multiple times, you just pull that data and then use multiple consumers. It actually simplified our architecture. It will also help us in the future when we have to build additional applications based on the same input data. We already have that data available. It will just be a matter of building the application itself. So it saves us a lot of time.

For Firehose, we perceive time-savings as a result of its incorporation. It takes you a couple of minutes to configure and it saves quite a lot of time in trying to get our information into the data lake. 

Regarding Kinesis Analytics, we have real-time alarms and real-time data flows to populate other systems. For example, we populate Salesforce using a tumbling window implemented with Kinesis Data Analytics and Lambda. We also have alarms for things like knowing when someone is affecting our assets and we need to warn the operators in real-time. So Kinesis Analytics has actually given us the ability to track things in real-time that before we didn't have the ability to track.

Because we couldn't do that in the database we needed a component that had the ability to get the last window of data super quickly and if something was wrong, to notify and identify the failing record or the information that we wanted to trigger and with Lambda to notify the user. At certain points, when we had operational issues, we implemented alarms that have the key indicators to help us attack those issues before they grew and it was too late to attack them. So that has been essential for us.

I think that all Kinesis components have their own features and their own value. Starting from Data Streams, you have to have it as the data queue or else you would need to go to Kafka or another message broker (with higher implementation effort if your ecosystem is fully hosted in AWS already). I think that the solution they have put together in Kinesis is fairly easy to use. It is definitely a core component in any data architecture. 

On the other hand, I find Firehose super simple and super useful for certain use cases. I wouldn't say it is as essential as Data Streams, but it is very handy if you want to just dump data. The connection between Data Streams and Firehose allows you to do that without worrying too much about performance and configuration. I find Firehose super simple to use for a very specific use case, but that use case is very common.

Kinesis Analytics is definitely more cutting edge. Out of Kinesis this is the most innovative part. We have used it for some alarms and for some batch processing in time windows. If we are talking about massive amounts of data, then you need to move to other solutions such as EMR or Glue for big data. If the amount of data is manageable and you want something to analyze on the fly, Kinesis Analytics is very appropriate and it gives you the ability to interact via SQL. So it makes your life easier if you want to develop a relatively self-contained application to do analytics on the fly.

I would say that Data Streams, in a matter of weeks, created a massive time-saving. Something that we haven't factored in is cost savings because we don't need to repeat the same data flow multiple times since each of those data flows are actually cost associated. We're talking about a couple of $100's per month, which is significant. In terms of time-savings here, we are in the scale of weeks. 

In terms of what can be improved, I would say that within Data Streams, you have a variety of ways to interact with the data; you have the Kinesis client library, the KCL, and you have the Kinesis agent. When we were developing our architecture a couple years back, all the libraries to aggregate the data were very problematic. So the Kinesis Aggregator, which essentially improves the performance and cost by aggregating individual records into bigger one, is something that I found had a lot of room for improvement to make it a lot more refined. At the time I found a couple of limitations that I had to work around. So definitely on that side I found room for improvement.

Something else to mention is that we use Kinesis with Lambda a lot and the fact that you can only connect one Stream to one Lambda, I find is a limiting factor. I would definitely recommend to remove that constraint.

I have been using Amazon Kinesis for over 2 years.

Kinesis is super stable. This is one of the only few components in AWS for which we have never had any issues with the stability.

Regarding scalability, you wouldn't use Kinesis Analytics for huge, vast amounts of data or for complex processing. It's for relatively simple processing with not too much data. So I wouldn't say that it is infinitely scalable, it really depends on your application and the volume of data.

Right now I don't see us using more of Kinesis. It has a very clear role in our architecture and satisfies that perfectly well. This is one of the initial components that you build. In a roadmap that would be the first 10%. All our work is spent in different actions right now, but we don't have any plans to grow Kinesis further. We used to do some specific real-time analysis with Kinesis Analytics on a case by case basis.It's more on a per need basis.

In other companies we use Kafka, but we didn't replace it with Kinesis.

The initial setup is relatively straight forward. 

In terms of the initial setup of Kinesis Streams, is no big deal, you just choose the number of streams and assign a name to your application and that's pretty much it. The effort is in the applications that talk to Kinesis. I would say implementation took around six weeks. Deployment just took two people.

We have our own internal strategy which we started from scratch. So obviously we knew which components we would be deploying first. At the time we didn't use either CloudFormation or CodeBuild. So when we started, we didn't have these tools which we now use all the time for managing the architecture and CICD. But we didn't have it in the initial deployment. 

Amazon Kinesis has improved our ROI. We obviously pay monthly for Kinesis but for us it is an enabler. We wouldn't have an architecture, or we'd have a terrible architecture, if Kinesis wasn't there. 

For the data analytics component, we definitely saw that our ROI clearly improved. The alarms are something that we have actually implemented in very critical tasks when we had a company issue and that we have given visibility and a prompt response to the issues thanks to Kinesis Analytics. So that has definitely proven its ROI. 

In terms of the prices, I think it is a fair price. Kinesis Data Stream has a very fair price relative to the value that it provides. Same for Firehose. As for Kinesis Analytics, I find it on the more expensive side because it's a newer component, something fewer people use, and something more innovative, cutting edge, and more specific. I would say Analytics is more on the expensive side of the spectrum. I would say that Kinesis Analytics is the only one that I may complain about if you like low pricing.

Kafka is comparable to Data Streams, not to Kinesis Analytics. For Analytics on the fly, I can talk about doing Spark streaming, which is a lot more complex and you need to spend a lot more time setting it up, but it also has more capability in terms of the scaling, so I wouldn't say it's a one-to-one comparison.

I also used StreamSets in the past, where you can gather data and you can also do some transformations on the fly. But again it's not comparable one-to-one so I wouldn't use it for the same use cases.

My recommendation for Data Streams is to do a deep dive into the documentation before implementing to avoid what we did at the beginning. You try to process record by record or push record by record into Kinesis and then realize that it is not cost effective or even efficient. So you need to know that you  need to aggregate your data before you push it into Kinesis. So documenting yourself about the best practices in using Kinesis is definitely something I would recommend to anyone. For Kinesis Analytics, I was actually surprised at how easy it is to use an application with such power. I would say with a trial, users will realize that for for such a fairly complex application such as Kinesis Analytics, it is something that you can do very quickly with minimal resources and it gives you a lot of value for specific use cases. 

On a scale of one to ten, I would give Amazon Kinesis a nine. I don't have much to complain about Kinesis.

My company primarily uses Amazon Kinesis within our data processing workflow to push the data into our data lake.

AI processing or cleaning up data would be nice since I don't think it is a feature in Amazon Kinesis right now. Amazon Kinesis has a specific purpose, and it may not fit into use cases anymore if it is loaded with too many features.

I have been using Amazon Kinesis for a year. My company has a partnership with Amazon. My company is also a customer of Amazon.

Stability-wise, I rate the solution a nine out of ten. I have not encountered any problems with the product's stability.

I have not encountered any problems with the product's scalability. Scalability-wise, I rate the solution a nine out of ten.

My company mostly deals with enterprise-sized businesses.

Whether I recommend the product to others depends on the applications, but moving a lot of data in real-time is a very good solution.

The technical support a user receives from Amazon depends on the kind of relationship that they have with AWS. If you are not known by Amazon, support would be low from AWS. If you have a specific relationship, then the support is okay. I rate the technical support an eight out of ten.

Positive

I rate the setup phase a seven on a scale of one to ten, where one is difficult, and ten is easy.

The solution is deployed on a public cloud.

The time required to deploy the solution is something that depends. A quick deployment can be done in a few minutes. If it is a production deployment where all the scripts and other related areas must be managed, it takes a couple of weeks. It is not the deployment itself of the tool alone that is involved since there is a need to prepare everything around it.

I rate the product price a five on a scale of one to ten, where one is cheap, and ten is expensive.

In terms of the most useful part of the tool for our company's data analytics, I think that because we decided to use AWS, Amazon Kinesis is present in our organization. It is easy to configure, highly available and offers high throughput. Amazon Kinesis is one of our company's main resources.

The scalability hasn't impacted our real-time data processing capabilities. Actually, users can scale up, and it doesn't impact our company. There might be a positive impact on scalability, so there are no issues.

The product has had a positive impact on operational efficiency because it actually allows our company to filter out some data and save costs.

The integration capabilities of the product are good. My company has integrated Amazon Kinesis with some APIs, AWS Lambda, S3, and Amazon EKS.

The adoption of data processing and data lakes is the purpose for which Amazon Kinesis is designed.

I rate the tool a seven to eight out of ten.

It's a kind of data stream tool that actually does kind of external serverless data streaming to different services at Amazon. Especially for the usage I've done, it's basically for getting data to Redshift. 

Kinesis can be one of the sources, and at the same time, Amazon Kinesis can be used to get the data out to other systems through other APIs. Kinesis is one of the tools I have used for typical data streaming.

I've utilized Kinesis Data Streams along with Kinesis Video Streams for different use cases. Again, these are required for working with data as part of Amazon Connect configurations.

Kinesis is fairly easy to use; it's kind of a service that is there and then fairly easy to configure. And it gathers lots of data while having proper authentication mechanisms. It's easy to activate and easy to test and set up as well. 

Typically, all the contact tracing codes that are there as part of Amazon Connect and also some of the real-time data can be passed with different sources of streaming. Like, Genesys also has data streaming and video streaming, which actually passes the real-time data as well. 

So it's pretty straightforward to configure, easy to understand. It's a typical service that is required in Connect integrations, specifically.

For me, especially with video streams, there's sometimes a kind of delay when the data has to be pumped to other services. This delay could be improved in Kinesis, or especially the Kinesis Video Streams, which is being used for different use cases for Amazon Connect. With that improvement, a lot of other use cases of Amazon Connect integrating with third-party analytic tools would be easier. 

It's a stable product overall. So far, I haven't seen any issues wherever I have used it.

This is kind of more like a backend tool used by our developers and the engineering team, including myself, to improve configurations and connectivity. Typically, it depends on the use cases and company requirements, but for my company,  three people generally do this work.

The initial setup is pretty easy and straightforward. It can be done pretty quickly with the direct documentation we have from AWS for Amazon Connect and Amazon Kinesis. That helps things to be configured very quickly.

The pricing depends on the use cases and the level of usage. 

If you wanted to use Kinesis for different use cases, there's definitely a cheaper base cost involved. However, it's not entirely cheap, as different use cases might require different levels of Kinesis usage. 

But, from an Amazon Connect perspective, where I have used this, it's quite good. It's pretty much cheaper compared to alternative methods.

I would recommend using Kinesis. Again, it depends on the use cases. One specific advantage of Kinesis is how straightforward it is, and it offers a lot of integration possibilities. So, for those wanting to use it with Amazon Connect, this is a good solution.

Overall, I would rate it between an eight out of ten because of its ease of use. I like the ease of use and how we can quickly get the configurations done, making it pretty straightforward and stable.

I use the solution in my company for streaming purposes, considering that my company has an AI-based camera for streaming.

The most valuable features of the solution are data streaming and the real-time data of our screen, which helps provide the analytics our company needs.

There are certain shortcomings in the machine learning capacity offered by the product, making it an area where improvements are required. There is a need to introduce something more into the machine learning area because it helps users learn and get newer things in their day-to-day lives. I think Amazon Kinesis should update machine learning and be up to the mark.

I have been using Amazon Kinesis for a year. I am a customer of the product.

Stability-wise, I rate the solution a nine out of ten.

Scalability-wise, I rate the solution a ten out of ten.

There are around three users of the product in my company.

The solution's technical support is flawless. I rate the technical support a ten out of ten.

Positive

One of the plus points of the product that makes it so much better is that it is very easy to set up, especially since it is managed by AWS, making it a smooth process.

The product falls on a bit of an expensive side.

There is a separate team in my company that looks after the real-time data analytics in our organization, which I don't know much about. As a DevOps engineer, I take care of the cloud, and because of this, I know why my company uses the product.

The most valuable feature of the product for our company's data processing needs stems from the fact that it operates in real-time. The simplicity of the services offered by the product and the way I can use them is very smooth and easy to understand. The tool also provides good storage and an increase in on-demand capacity, which I think is the best for our company.

Integrating Amazon Kinesis with other AWS services has smoothly helped our analytics workflow. It is also very easy to integrate with other AWS services. The analytics part is also very good with Amazon Kinesis.

I recommend the product to those who plan to use it. Those who are new to data streaming and want to start with a new product can go for Amazon Kinesis, as it is very easy to set up, especially the installation part, which is very easy to handle. Amazon Kinesis is the first option others should consider since it is easy to set it up.

I rate the tool an eight out of ten.

Amazon Kinesis is a queuing or buffering system that we use as a central place to buffer the incoming data we receive from the source. The actual destination is open-faced. Amazon Kinesis is used as a buffer in between to decouple the workload.

Amazon Kinesis's main purpose is to provide near real-time data streaming at a consistent 2Mbps rate, which is really impressive.

The solution currently provides an option to retrieve data in the stream or the queue, but it's not that helpful. We have to write some custom scripts to fetch data from there. An option to search for data in the queue can really help us in our day-to-day operations.

Since the solution is a buffer system, you write to it and read from it. The readers are called consumers. If you want to run multiple consumers reading from the queue, you have to enable the enhanced fan-out feature on Amazon Kinesis. This enhanced fan-out feature is quite costly.

There was a point when we had a huge budget increase in one week just because of the enhanced fan-out feature. This feature does not provide any special out-of-the-box functionality. Hence, we struggle to optimize multiple consumers reading from a single queue. We were charged high costs for the solution’s enhanced fan-out feature.

I have been using Amazon Kinesis for more than two years.

The solution is pretty good in terms of scaling. Amazon Kinesis has shards, which are the instances or units that the solution spins up for you. Depending upon your account quota, you can spin up as many shards as you want. You can even raise a request to increase that quota, which will be done sooner. Overall, Amazon Kinesis is a really scalable solution.

Our team, consisting of four to five people, uses the solution extensively in our organization.

We really struggle to get better support for Amazon Kinesis.

Amazon Kinesis is an expensive solution.

Amazon Kinesis is an AWS-managed service, just like S3 or EC. We don't have to deploy it; it is just there, and we spin it up. You must go to AWS' service page and click on Kinesis. Then, you can create it by clicking on Create and entering the name.

I would not recommend Amazon Kinesis to other users. Users can choose a cheaper alternative. They can use any other queuing system or in-house Kafka if they have a Kafka team. Amazon Kinesis provides near real-time read-and-write, but its cost is too high. Users can choose another option that provides the same functionality at less cost.

With Amazon Kinesis, you have to run a consumer who sees from Amazon Kinesis. AWS provides the Kinesis Client Library (KCL), which reads from the Kinesis stream. That library is also used in DynamoDB for data checkpointing. For example, if you have one day of data in Amazon Kinesis and started reading from 12 AM yesterday. The Kinesis Client Library (KCL) will check on the data in the DynamoDB. You get charged for the DynamoDB table out-of-the-box, along with Amazon Kinesis.

The DynamoDB table also costs a lot, which should not be the case. It is just read-and-write and is downloaded from the Kinesis Client Library (KCL). The DynamoDB table's cost should be very minimal, but that's not the case. The consumer is not optimal for efficient read-and-write, which further increases the cost. Both Amazon Kinesis and DynamoDB come into the picture.

Overall, I rate the solution a five or six out of ten.