Portkey Reviews

Portkey serves as a centralized AI gateway for our organization. At Clydo, we are building multiple AI-powered features, including internal copilots, catalog intelligence, customer support assistance, and agentic workflows. This gives us a single layer for model routing, prompt management, and API key management.

A primary example of our main use case is our internal AI assistant used by our operations and customer support team. Every user query goes through Portkey before reaching the language model. Portkey handles authentication and routes requests to appropriate models. One practical scenario involves our engineering team wanting to evaluate a newer model for better reasoning at a lower cost. Instead of changing code across multiple services, we update the routing configuration in Portkey and compare the responses and performance. Another benefit is observability. When users report that AI responses are slower or inconsistent, we use Portkey's logs to see the request latency. Overall, Portkey has become the control layer for our LLM infrastructure. It allows our engineers to focus on building AI features while providing us with centralized visibility, governance, and flexibility.

The main use case is standardizing how AI is consumed across the organization. As more teams start building AI-powered features, we do not want every service to implement its own integration with retry logic or rate limiting, which becomes very difficult to manage. Portkey gives us a common layer that every AI application uses, whether it is an internal or customer-facing chatbot or any agentic flows. This makes the architecture much cleaner and easier to manage. It gives us the flexibility to experiment. We can compare different models and run A/B evaluations for specific workloads. From an engineering perspective, Portkey has evolved from being just an API gateway to becoming part of our platform. It is always a layer that provides consistency, observability, and governance across all our LLM-based applications.

The best feature that Portkey offers is a centralized AI gateway, which is the biggest advantage. Instead of integrating directly with different LLM providers, Portkey gives us a single, consistent interface. This results in a simpler architecture and has reduced maintenance. Another valuable feature is multi-model and multi-provider support, which allows us to work with different providers through the same integration. Observability and logging represent one of the most valuable features for production. We gain visibility into request latency, token consumption, errors, and success rates. Cost monitoring is critical since LLM costs can increase quickly, and we need centralized visibility into token usage and API consumption. Reliability features, including fallback mechanisms and routing to different LLMs when there is an issue with the provider, allow us to recover gracefully.

Without question, observability is followed by the AI gateway itself. Whenever we are developing a new AI feature or investigating an issue, the first place we look is the logs and dashboards. We use it for monitoring request latencies and response times, tracking token usage, and identifying timeout requests. We compare how different models perform for the same use cases. Debugging prompts becomes seamless and easy. For example, when our customer support copilots started responding more slowly, we could quickly determine the bottleneck, whether it was the LLM provider or the application itself. Without a centralized observability system, we would have to piece that information together from multiple systems. Portkey itself is something we benefit from continuously, but it is largely transparent once configured.

Some specific details that stand out include how it is vendor-agnostic. Since AI evolves very quickly and models are releasing every few weeks with frequent pricing changes and different performance characteristics, Portkey gives us the flexibility to evaluate and adapt to newer models without having to redesign our application architecture. Another valuable detail is how it encourages good engineering practices. Since every AI request goes through a centralized layer, it is much easier and standardized. If I had to summarize, Portkey gives engineering teams the flexibility to innovate with AI while maintaining the operational costs, control, visibility, and consistency needed to run AI applications at scale.

There are specific outcomes and changes I have noticed since implementing Portkey. The biggest impact is operational efficiency and maintainability. Before Portkey, each AI service managed its own model integration. After adopting Portkey, we centralized those responsibilities into a single gateway. This reduced duplicate engineering efforts and gave every team a consistent way of consuming LLM services. The outcomes we noticed include faster development, quicker troubleshooting, better cost awareness, greater flexibility, and improved reliability.

There are definitely some places where Portkey can be improved. Overall, the experience has been positive, but the first area is analytics and reporting. While the observability feature is excellent, we would like to have richer historical analytics and customizable dashboards. For example, it would be useful to see trends by applications and teams or features over long periods without exporting data to an external BI tool. Another area is governance for large organizations. As AI adoption grows, enterprises need more granular, role-based access control. We would like to see more advanced AI evaluation capability built into the platform itself, including features such as prompt versioning and automated quality scoring and regression testing. Finally, while the platform is supported with multiple providers, we would welcome even more intelligent routing capabilities, such as automatically selecting the best model based on latency, cost, and task complexity using configurable policies. These are not major pain points, but they are enhancements that would make an already strong platform even more valuable for an organization that scales their AI workloads.

Documentation and onboarding could be enhanced. Portkey is developer-friendly, but we need more end-to-end references, architecture, and implementation guides for common AI patterns. This would help teams adopt it even faster.

I did not give Portkey a perfect score because while it has become a foundational component in our AI stack and solves several operational challenges, I would still like to see deeper analytics, stronger enterprise governance features, and more built-in AI evaluation capabilities, especially for prompt testing, regression analysis, and model benchmarking. I would be comfortable recommending Portkey to any organization that is building multiple AI applications or wants to manage a scalable way to handle LLM providers and produce AI traffic.

I have been using Portkey for more than a year.

Regarding Portkey's accuracy and reliability of output, it has met my expectations. Portkey does not generate AI responses itself, but the quality of output primarily depends on the models. From a reliability standpoint, we have had positive experiences. The routing and retry mechanisms are centralized, and handling our AI applications is far better and easier. From an accuracy perspective, Portkey has helped us improve outcomes indirectly because it centralizes logging. To summarize, Portkey does not determine the intelligence of the model, but it gives the tooling to consistently deliver, evaluate, and improve the outputs.

There have not been any situations where I experienced outages. There have been occasional incidents, but they were only under LLMs. From an operations perspective, Portkey has been seamless, and I have not experienced any downtime. It is almost stable all the time.

Overall, Portkey's performance under heavy workloads or peak usage has been satisfying. AI traffic is not at the scale of a large enterprise, but I have experienced spikes during business hours and while running batch AI workflows. In those scenarios, I have not encountered any major scalability bottlenecks. Latency introduced by the gateway has been minimal. In most cases, the issue is the LLM provider rather than the gateway. During periods of high traffic, the observability features have been particularly valuable. There have been occasional cases where upstream LLM providers experienced slower response times or temporary service degradation. In those situations, the retry and routing capability helped reduce the impact. Overall, Portkey has met my expectations for scalability and reliability.

Portkey's scalability has kept up with my organization's growth and changing needs. When I first adopted Portkey, I was using a smaller number of AI providers, and one biggest advantage is that the gateway scales with the number of AI use cases rather than requiring each team to build its own integration. I have also benefited from Portkey's multi-provider capabilities. From an operational perspective, I have not had any major architectural changes. Overall, Portkey has kept pace with my organization's growth and has provided a scalable foundation for expanding AI adoption.

I have not reached out for customer support very often. Overall, the experience with customer support has been positive. Portkey's team has been responsive, technically knowledgeable, and understood the challenges I faced. Most of my questions were around configuring multiple provider routing and optimizing gateway settings. As the platform has matured, I have needed to contact them less frequently because of improved documentation. If I had one suggestion, it would be to expand the self-service knowledge base with more implementation guides. Overall, the customer experience rating is solid at nine out of ten.

Previously, I did not have any dedicated gateway. We were integrating directly through SDKs and APIs. While that worked initially, it became difficult to manage. The main challenge was that comparing different LLMs required a lot of code changes, and there was no visibility on token usage. Troubleshooting was also challenging. I evaluated a few alternatives, but I chose Portkey because it has all the capabilities that I needed. It allowed me to standardize my infrastructure. The switch was not from commercial gateways; it was from direct integration to a centralized integration.

I would describe the learning curve for new users or teams adopting Portkey as seamless and very easy to adapt. Knowledge transfer becomes increasingly easier as team members become familiar with the platform.

I have seen a return on investment with Portkey. I can share relevant metrics: thirty to forty percent reduction in development time, fifty percent faster evaluation cycles, twenty to twenty-five percent reduction in unnecessary LLM spending, and twenty to twenty-five percent improvement in engineering productivity.

Regarding pricing and licensing, the cost was relatively low because Portkey sits in front of our existing AI infrastructure. From a licensing perspective, I appreciate that the pricing model is predictable and aligned with how we use the platform, since it is an infrastructure component rather than a per-user productivity tool. One consideration was the engineering time saved. Even with a licensing cost and reduction in development efforts, a startup with a few thousand requests per day and a large enterprise processing millions of requests evaluate ROI very differently. Overall, costing and ownership has been favorable.

There are specific outcomes and changes I have noticed since implementing Portkey. The biggest impact is operational efficiency and maintainability. Before Portkey, each AI service managed its own model integration. After adopting Portkey, we centralized those responsibilities into a single gateway. This reduced duplicate engineering efforts and gave every team a consistent way of consuming LLM services. The outcomes we noticed include faster development, quicker troubleshooting, better cost awareness, greater flexibility, and improved reliability.

I have not evaluated much before choosing Portkey, but most of the capabilities were largely found on Portkey itself, and I went with this option.

Regarding governance and security, I would say security is quite high. In fact, one of the reasons we chose to implement Portkey is that it gives us a central control point for AI usage rather than every application managing it independently. From a security perspective, I appreciate that sensitive provider credentials are managed centrally instead of being distributed across multiple services. On the governance side, having a single layer for routing, usage tracking, and policy enforcement helps us standardize how AI is used across different teams. As more applications adopt LLMs, consistency becomes increasingly valuable. Overall, I would describe Portkey as a strong differentiator that provides visibility and centralized control.

Portkey is deployed in my organization as a hybrid cloud deployment. We use AWS as part of our hybrid cloud setup. Our applications and backend services run on AWS, and all AI requests are routed through Portkey before reaching the underlying models. We have backend services and API databases on AWS. Portkey centralizes routing, observability, and cost management. This approach gives us the flexibility of managing foundation models while keeping our application logic simpler. Operational controls are within our own cloud environment, allowing us to change model providers without impacting the applications.

In terms of customizing, I would say Portkey is very flexible because one of the reasons we adopted it is that it fits into the existing architecture. Most of my customization has been through configuring rather than custom development. For example, I customized different routing rules for different AI workloads. I have also integrated Portkey into my internal monitoring and engineering workflows. The best part is that our application code remains largely provider-agnostic. If we want to evaluate a new LLM or change routing for a specific use case, we typically do not need to make significant changes. I have not had to build major custom extensions to make Portkey work for us. Most of what I have needed has been supported through existing configurations. Overall, customizability is highly rated.

For my organization, compliance is important, though we are not heavily regulated. From my perspective, Portkey contributes to a centralized control layer. Since all LLM traffic passes through a single gateway, it is much easier. That said, we do not rely on Portkey alone to meet compliance requirements. We combine it with our own security controls.

I can share some approximate figures for faster development and better cost awareness. For development, I estimate that Portkey reduced engineering efforts by around thirty to forty percent. Previously, each team had to implement provider-specific authentication, retries, logging, and error handling. For model evaluation, we have seen about fifty percent reduction in time. From a cost management perspective, we have reduced unnecessary LLM consumption by around fifteen to twenty percent. I have also noticed thirty to forty percent reduction in troubleshooting time. Overall, Portkey has had a huge business impact in terms of cost savings and operational efficiency. I would rate this review overall at nine out of ten.

Portkey has been used as an AI gateway and observability layer for LLM applications. The main use case is to route requests across multiple LLM providers through a common API, monitor latency, cost tracking, prompts, and responses, and add reliability features such as fallback, retries, caching, and guardrails. It is especially useful when moving from experimentation to production because it provides a single control layer for model routing, prompt management, usage tracking, and operational visibility.

Whenever an API is called through Portkey, it displays the total time consumption that has been taken. Observability is provided there. When it fails to connect to an LLM model, it attempts to use a fallback mechanism, which is helpful.

The most valuable features are the unified AI gateway, observability, price calculations, routing controls, and guardrails. The universal API makes it easier to switch between and compare models without rewriting large parts of the application. For the organization, Portkey was providing two endpoints, one for the US and another for Europe, which was easily manageable. Some models were supported for the US, some not, some for the EU, and some for both. Switching between providers is very easy with Portkey. The fallback and retry capabilities are also useful because production LLM applications need resilience when a provider is slow, rate-limited, or temporarily unavailable. Cost and latency tracking are valuable because LLM usage can become difficult to manage across teams, but these can be easily managed through Portkey.

Observability is the best feature because at the production grid application, the focus is usually on observability, such as how the end user is using it, what the latency is, how many errors occurred within time frames, which models were used, and how many tokens were consumed. These aspects are easy to manage in Portkey.

Portkey has significantly improved the organization by streamlining the AI processes being followed. Portkey improves the workflow for centralizing LLM operations instead of every application team building its own logging, routing, fallback, and cost tracking logic. Portkey provides a shared layer for these capabilities so it can be synced between all teams. It helps reduce engineering efforts when experimenting with different providers and models. It also improves visibility into production behavior because request volumes, latency, cost, errors, and model usage can be seen in one place. For teams building multiple GenAI applications, the biggest improvement is standardization. Portkey makes it easier to enforce common practices across observability, reliability, and governance.

The main area for improvement is onboarding and trial experience.

Portkey is very useful for production teams, but it may feel more like infrastructure than needed for small prototypes. More guided setup templates for common use cases such as RAG chatbot or internal assistant, evaluation pipelines, and multi-provider failover could make adoption easier. If the onboarding process could be easier, that would be beneficial, as there have been issues faced during onboarding.

Portkey has been used for the last two years as it is an integral part of Rosh.

Portkey is reliable approximately 99 percent of the time. Some endpoints will occasionally be down. Over a six-month period, endpoints went down once or twice, but most of the time it works very well.

Portkey is designed for scale production uses. It is useful when multiple teams and applications need a consistent gateway for LLM applications, observability, prompt management, and governance. These aspects are good in terms of scalability. The scalability value becomes more obvious as the number of models, providers, applications, and users grows. For a single prototype, it may feel optional, but for production GenAI platforms with multiple applications, it becomes much more valuable.

For enterprise production, support quality is very important. Support was reached out to once on Discord. There is a support group on Discord where the team is very responsive. They provide a newsletter with updates and features. The support experience has been good.

Previous solutions were being used, but they were not considered optimal. AWS Marketplace is also being used for some LLM models, but AWS does not support all the models. Portkey is a better solution because it has everything in one place.

The best aspect in terms of cost saving is using Portkey cache, which was built-in to the packages. This literally saved token costs because cache tokens cost less to process. This was the best aspect in terms of productivity. The setup was also smooth because only a single Portkey library is needed, and then any models can be used at once. Whether using AWS Marketplace models, Google models, or OpenAI models, everything is available in one place. There is no need to worry about multiple platforms or multiple packages.

Time has been saved for multiple developers because of having a single package to manage multiple models and multiple providers. This was the main ease gained.

The pricing is great. The organization only pays for the model being used. For Portkey pricing, the organization is covering that cost. Roche is taking care of it. The pricing is very reasonable for an organization already running production LLM workloads that needs observability, routing, guardrails, or other features. Portkey pricing is separate from the underlying LLM provider cost. The model providers are still paid for token usage, and Portkey adds the gateway, observability, caching, guardrails, and management layers on top.

Nothing has been purchased from the AWS marketplace. Roche is a partner for Portkey and the organization is an enterprise for Portkey, utilizing Portkey features.

Portkey should be recommended for teams that are moving GenAI applications into production and need more control over reliability, cost, observability, and governance. It is especially useful if multiple LLM providers are being used and if changes to models are expected over time. The overall rating given for Portkey is nine out of ten.

Portkey has many features for deploying an application to production, including an AI gateway, observability, guardrails, governance, and prompt management. We primarily use the AI Gateway, observability, and the guardrails.

We have an application that uses LLM models such as Gemini, OpenAI models, or Claude models. We need to interact with these models, which we have deployed elsewhere, and we use Portkey as the AI/LLM gateway to check observability and monitor how our model is behaving. We also have custom fine-tuned trained models, and we want to check the observability or test these models using Portkey. In this scenario, when any person wants to call the LLM or use the AI, they do not need the API key for that model. They can directly call Portkey and it will provide the response. The API keys for all the models are not exposed to the developers or the engineers. They are stored in the vault and can only be accessed by Portkey.

We have an application that uses LLM models such as Gemini, OpenAI models, or Claude models. We need to interact with these models, which we have deployed elsewhere, and we use Portkey as the AI/LLM gateway to check observability and monitor how our model is behaving. We also have custom fine-tuned trained models, and we want to check the observability or test these models using Portkey.

We primarily use three features: the AI Gateway, observability, and the guardrails.

The AI gateway and observability are the two features that are very important for the team. The AI gateway acts as a single point of application for all the AI model requests. It also enhances security since we do not need to provide the API key to all the developers. Additionally, it supports load balancing and automatic retries. When a lot of load is coming in, it balances the load among different models. If one of the models is not available, it handles the failover by redirecting the request to the other model. Regarding observability, it is very useful to monitor and understand how our AI application is behaving or performing. It usually logs all the requests and the responses. We can easily get an idea of the token cost, token usage, API cost, and error rates. It makes debugging easier and tracks the AI spending to show how much we have spent. These are the valuable aspects of both features.

Another feature of the AI Gateway is that it lets you switch between providers without changing your application code. Suppose we are using three models for three different purposes and one model is not available, or I want to switch the model within the application without restarting my code or doing a redeployment. We can easily do this with Portkey. Regarding observability, we usually monitor and check how our AI applications are performing and behaving. Other things that we usually track in observability include token usage, API cost, error rates, success and failure metrics, and response latency. This is an important aspect that we usually prioritize.

Guardrails are safety checks and quality control that we apply to the input and output of AI. Suppose I do not want certain personally identifiable information or personal information data of a user to be sent to the AI. Portkey has guardrails that do not allow sending that data to the LLM gateway. It can also act as a middleware that checks all the data we are sending to the LLM before transmission, so we do not send any personally identifiable information or personal information of the user to the LLM gateway.

Portkey is a very good software for AI applications. It is a single place solution that you can use if you want your AI application to be in production. However, one thing I would like to say is that it could have better documentation. As of now, the documentation is very concise and short. If possible, they could provide more beginner-friendly tutorials with step-by-step instructions on how to set up and how to see the things in the dashboard. It would also be great if the documentation included sample projects or example projects so that users can easily see and learn from those.

Another thing I would like to add is that it has a steep learning curve. It takes time to learn Portkey because the dashboard is very complex. There are many things that a user can see in Portkey. If they can create good documentation, it could definitely increase their value in the market.

I have been using Portkey for around two years.

Portkey is totally stable. It is one of the best software solutions in the market for this purpose. The other software solutions I have read about do not provide all the solutions in a single place. The LLM Gateway is one of them, but Portkey provides all the solutions in a single place.

I have tried around fourteen models integrated with Portkey and it does not break under that load. I have seen millions of token consumption through Portkey, and it works well in that scenario. It is scalable.

Portkey's customer service is good. I remember when I was just setting up Portkey, I encountered a small issue. I tried to connect with customer support, raised a ticket, and they replied back within a few hours.

I do not believe we evaluated other options.

Everything is perfect except for the documentation and the steep learning curve that I already mentioned. That is the part for which I deducted one mark. Everything else is perfect. Portkey is a very useful software if you are going forward with your AI application that uses the LLM gateway. I would recommend you use it and you will love it within a few days.

Our client is completely using AWS products. For AI gateways, we are using Portkey, which is the product we have implemented, and we are leveraging all AWS services for internal application development.

Applications can interact with different multiple LLM models. We are restricting direct access from the application to LLMs. Instead, we are introducing Portkey as an AI gateway in between. We send the application's requests to the respective LLMs through Portkey, and we have enabled token usage, caching mechanisms, and guardrails at the AI gateway level. All applications must send requests via Portkey to leverage LLM capabilities.

Portkey's visualization is excellent. I particularly appreciate the multiple capabilities they have introduced, especially the MCP gateway, which is outstanding. The caching mechanism is very good, and the logs and traceability features are also excellent.

Portkey does not have an agent registry, which is a feature they should implement in a future release. We need to capture metadata of agents, and this capability is currently missing.

End-to-end traceability is not fully fulfilled. They are only fulfilling about 50 to 60 percent of our end-to-end agent traceability requirements, and the remaining functionality needs to be implemented.

The communication between MCP and AI Gateway is lacking. Additionally, pricing could be improved.

I have been using Portkey for the last two years.

Initially, we experienced some instability, but the product is now stable.

Portkey is completely cloud-based. We deployed the product into the cloud, and the cloud infrastructure handles scalability. Scalability is not a feature of Portkey itself but rather a feature of the deployment approach.

We have scheduled support calls with Portkey every week. The support quality and responsiveness are good overall.

We have received immediate responses when needed, and we have established a regular weekly call schedule. However, email communication has been somewhat lagging, and responses are not always provided within the proposed timeframes. Apart from email communication delays, the remaining support aspects are satisfactory.

We have evaluated several alternatives. Kong AI gateway is one alternative we have considered. Apigee from Google is another alternative we are aware of. Additionally, we are expecting another alternative from AWS called Gateway, which is expected to be released by Q4.

The initial deployment of Portkey was straightforward. For a SaaS product, you can directly use it. For the SaaS version, registration with an email ID and purchase can be completed in approximately one minute.

If deployed as a COTS product, the initial setup takes more time because the entire product needs to be installed into a Docker machine. Once that installation is available, the initial deployment takes time. After deployment, patching work takes approximately four hours.

I do not require any maintenance for Portkey on my end. On a scale from 1 to 10, I would give Portkey an overall rating of 8 for everything combined.

Our team is searching for an alternative to LiteLLM, which is another monitoring management platform, and we found that Portkey is an alternative solution that we decided to try out.

We want to conduct prompt template testing with Portkey by splitting our business logic along with the prompt, so we need to test that against different models and perform evaluation accordingly. This is one major thing we are trying. The product owner can separately conduct their test for the prompt template on the playground.

There could be some advanced usage with Portkey such as setting up the MCP and guard rail. We are thinking about having that part as well, but so far we haven't touched it yet.

Portkey provides the standard interfaces for the major features that interested the industry has. It has a very comprehensive feature support for the MCP agent and its playground, support for prompt templates, and allows the management of traceability analytics. It covers most agentic application usage very comprehensively.

Portkey definitely provides a solid alternative solution for the agent and large model hosting platforms, and it is very helpful for us to explore the possibilities across the industry rather than staying with a few mainstream options.

Portkey successfully provides a standard interface and large model based solutions, and we can see those features in many other platforms as well. This is solid, but so far I haven't found anything super exciting or very innovative on this platform. For our organization, it's a great alternative solution to consider.

One major problem I see with Portkey is that when I had not yet started any trial, it started to denote that I had exceeded the prompt limit. Users are usually expecting a trial stage with more tokens to at least work through a few days to understand the products inside out. That might be something that could be improved.

The onboarding process of Portkey is very good. However, it requests that there are no free tokens or trial tokens. That was the limitation.

I have been using Portkey for a few weeks.

When I was doing a test with Portkey, it was stable enough.

We have not reached the point for considering scalability with Portkey.

We do not have a customer relationship with Portkey.

We were using LiteLLM and sometimes Langfuse. Langfuse is also open source, so we can deploy that to cover many such features. We used them and felt that their price was a little bit higher than our budget. This was the reason why we were seeking alternative solutions and eventually found Portkey.

I did not want to quickly dive into too much advanced features. Portkey provides a very quick hands-on experience.

Portkey is requiring production to be $49, I guess that is US dollars per month for 100K logs. I would say it is manageable.

It is definitely a private cloud for Portkey from their provider. Using their online services with Portkey is good. If you are seeking an alternative solution for the product, Portkey is definitely worth a look. My overall review rating for this product is 7.