What is our primary use case?
My main use case for
Amazon Athena is serving a healthcare organization in the US that needed an interactive SQL editor to write complex queries and retrieve details instead of checking manually in the
S3 bucket. I proposed and implemented the solution end-to-end from scratch using
Amazon Athena and other
AWS services such as
AWS Glue services, including crawlers and modifiers, and CloudWatch for logging purposes. I implemented and provided a production-ready solution to the client, and they are very satisfied with it.
Ideally, there are no external or third-party applications installed, so the end user only needs access to AWS accounts with the Amazon Athena query editor. Through the query editor, they can write their query and receive results, and they can store the results if desired.
Regarding my main use case from the end user perspective, if they encounter issues with the data, it is not practical to check manually through each data file in the S3 bucket. Instead, I implemented Amazon Athena on top of the data using the AWS Glue service stack, and based on that implementation, whatever tables and databases are created allow users to write complex queries and retrieve the details they need. If they obtain the results they are looking for, their work speed increases significantly, and they are satisfied with writing SQL queries. I have also provided prerequisite queries based on known problem statements.
What is most valuable?
In my experience with Amazon Athena, the best feature is the cost model, which I appreciate greatly because it operates on a pay-per-use basis. Whatever queries I run on Amazon Athena, I am only charged for that usage on a monthly basis, which is very beneficial. Additionally, anyone with access to Amazon Athena on the AWS console can write queries and get results easily without requiring any prerequisite knowledge, so even new software engineering professionals can adapt quickly.
After implementing Amazon Athena in our project, we have observed significant savings in cost structure and effort. The end user is very happy and is conducting analytical work using Amazon Athena.
Regarding the features I mentioned, the one with the biggest impact for my team is the speed of results. When a user has more than eight terabytes of data in the S3 bucket, it is not practical to check each file manually. Instead, they write the specific database and complex SQL query to get results within three to four minutes maximum. This saves considerable time for end users.
What needs improvement?
While implementing Amazon Athena, I observed an issue where, even after writing an exclusion pattern on the AWS Glue side, whenever the end user queries, it skips the exclusion pattern, overriding it and directly fetching data from the S3 bucket. For instance, if we have an input file in CSV format along with another Python file or a Parquet file within the same folder, Amazon Athena reads that file during the query and could create a problem known as the bad hive data error. This is a struggle, and the Amazon Athena team is currently working on resolving this issue. I have checked with them, and their technical team is working on it.
My main concern is that whenever we have written an exclusion pattern during the ETL process on AWS Glue, Amazon Athena should use that pattern while querying and accessing data on S3. This is a crucial improvement area.
One aspect that could be improved in Amazon Athena is that many changes are happening in the background as the AWS team alters their algorithms and code. It would be better if there is proper communication from the Amazon Athena team to end developers about these changes so they could align their thought processes or solutions accordingly.
For how long have I used the solution?
I have been using Amazon Athena for more than four years across different clients.
What do I think about the stability of the solution?
Amazon Athena is stable and has been in production for more than a year now, functioning well with periodic cleanup activities to remove unnecessary data.
What do I think about the scalability of the solution?
Amazon Athena has great scalability. When the analytical team wants to onboard new data stacks, I implement Amazon Athena on top of that infrastructure. There are high chances for scalability, and the existing Amazon Athena data, which is live or dynamic, continues to grow gradually.
How are customer service and support?
Customer support for Amazon Athena from the support engineers is very good. Whenever we require general guidance from Amazon Athena, they join calls quickly, and we achieve solutions efficiently. It has been a positive experience with Amazon Athena engineers.
Which solution did I use previously and why did I switch?
We did not implement any other solution before Amazon Athena. We directly started using Amazon Athena because we gained knowledge about it from the Amazon team.
What was our ROI?
In terms of specific outcomes, let's consider a monthly scenario. Each end user has around two hundred plus hours per month, and by using Amazon Athena, they can save up to forty percent of their time needed for work. In terms of costs associated with analytical work, they save about sixty percent in billing costs, excluding AWS expenses.
I have not done a formal return on investment assessment; however, I have learned from the analytical team manager that they have saved sixty percent of effort compared to the manual methods previously used to find errors on the S3 bucket. Now, by directly writing queries, they save about sixty percent of effort and more than forty percent in cost after implementing Amazon Athena.
What's my experience with pricing, setup cost, and licensing?
My experience with Amazon Athena regarding pricing, setup costs, and licensing is quite clear after more than three years of usage. I am part of my client's FinOps cloud team, assisting in reducing costs associated with Amazon Athena. The costing model from Amazon Athena is quite transparent, so by deleting unnecessary S3 bucket details and results that are no longer required, overall it is a good approach.
Which other solutions did I evaluate?
We did not consider other options before choosing Amazon Athena because the client already had it installed, and we could not go beyond the client's limitations.
What other advice do I have?
My advice for others considering Amazon Athena is that if they are working in IT and have data stored on Amazon S3 and want quick analytical work done, they should think about implementing Amazon Athena in their projects.
I recommend Amazon Athena without hesitation if you want to conduct analytical work using existing S3 bucket data. I have provided a review rating of seven out of ten for Amazon Athena.