I first used Heroku to deploy and host Facebook apps, and I’m a big fan ever since.

Lately, I’ve been doing development with node.js and since Heroku supports it (wonder if they were first to offer it), it was a no-brainer:

• deployment via command line with git: nice way to enforce best dev practices
• package management with NPM – everything will be fetched and installed for you
• built-in SSL support on *.herokuapp.com subdomains
• easy monitoring: just type ‘heroku logs’
• easy scaling: just type ‘ps:scale web=x’ or ‘ps:scale worker=x’
• support of environment variables: one example – running multiple instances from the same git repo
• pretty good docs and tutorials
• tons of add-ons: you are free to do pretty much anything (I use Mongolab add-on for Mongo hosting)
• affordable!

Nodejitsu is another service focusing on node.js primarily, but I’ve been reluctant to switch because I just like Heroku so much.

It’s also super cool that they support multiple environments via buildpacks, I’d love to look under the hood and find out how they made it work (and here’s a great post describing their polyglot platform in high-level)

In contrast to the one-language-per-career programmer, today’s up-and-coming developers can often utilize many languages effectively. Borrowing a term from linguistics, we can call these versatile new developers “polyglot programmers.”

For the latest Show&Tell demo, I talked about benefits of using Heroku (as one of the PaaS options) for rapid deployment and easy hosting, so here’s the deck.

This week I’m working on a recommendation chart for cloud hosting, so I’ll share that as well soon.

Introduction

In my last post, I looked at some of the major IaaS vendors with a view on how they are being adopted. In this blog I want to look at the broad spectrum of the Platform as a Service (PaaS) models and the compelling reasons that make PaaS a strong option for developers and companies to speed up development and slash costs.  Current predictions estimate that globally the PaaS market is predicted to reach $22 billion in 2014. As a whole Europe’s cloud activities will gather pace and momentum, creating 3.8 million cloud professionals and jobs by 2020 mainly within the PaaS sector.

So why is PaaS becoming so popular?

To put it succinctly, PaaS allows developers to have the complete tools, operating systems, middleware and programming language to build their applications. Everything is then hosted and stored by the PaaS vendor. PaaS offers the developer a solution that is a complete software development, testing and deployment environment. In addition it has the benefit that the operating system, virtual machines, infrastructure and IDE are hidden and not a concern to the developer. PaaS service models have automatic scalability to allow for increased usage or spikes in activity – therefore making PaaS a really useful way to build high traffic web apps.

Some of the advantages of using PaaS platforms:

• Fast and speedy development environment: You can commission databases or VMs in seconds and quickly build and deploy code. It provides a single code repository and development environment, which is good for multi-location development teams.
• Huge cost savings on development: Due to faster build times which allow applications to get to market faster.
• Easily deployed databases: Which are configured and managed on the PaaS platform with many providers offering either traditional SQL or NoSQL databases
• Elastic scalability
• Easy support and connectivity: To various software or packages (wordpress, drupal, etc)
• Better security: You can leverage the security protocols from the PaaS vendor for your own application benefit

Who are some of the main PaaS Vendors?

There are also numerous other major PaaS Vendors with their own offerings such as OpenStack, Longjump, IBM Smartcloud, Redhat Openshift (based on Linux) Google App Engine, Cloudbees and Engine Yard.  

From a UK recruitment perspective: What skills do I need to hire to move my IT development unto a PaaS environment?

Architects: Software development specialists with a strong understanding of how to build on a specific PaaS platform, unlike IaaS architects who generally have come from Infrastructure background. Will have a strong coding background on a core programming language like Java or C# but will understand the build and deployment issues that are alligned to the PaaS platform. Currently as of November 2012, UK contract rates for PaaS Architects are £600-£700 per day and permanent salaries of £75-90k.

Database Admins: These will be specialist Database people with a strong understanding of how the database runs on the PaaS platform. For instance Windows Azure SQL DB is configured and set up different than on-premise SQL servers. It isnt necessarily a vast jump for an existing DBA to learn this, but for companies with large or complex databases, a DBA with specific PaaS product knowledge could be invaluable to move. Currently as of November 2012, UK contract rates for PaaS DBAs are £450-£550 per day and permanent salaries of £60-80k.

Developers: There will be a distinct advantage to hiring developers who have previously built and deployed applications using a PaaS platform such as AWS or Azure. Obviously, these PaaS platforms have been designed to be as easy and as quick to build on as possible, but having a few developers with prior platform development will assist large development teams get to grips with the specific idiosyncrasies of the PaaS platform and will enable the incumbent team get skilled up and productive as quickly as possible Currently as of November 2012, UK contract rates for Developers with PaaS experience are £450-£550 per day and permanent salaries of £60-80k

Lastly, in the UK, compared to the US, the skills pool for IT professionals at the current time with genuine commercial skills in cloud and PaaS technologies is very shallow. Companies looking to move to PaaS development environments are struggling to find external skills. As more companies use these technologies the skills market will expand, but the demand for these skills will be even greater and it will become even harder to recruit.

My advice to any CTO or CIO looking to move their IT to a PaaS development stream is: hire your team early before the rest of the industry wakes up and tries to hire the same person you want to!   As a candidate in 2012 and 2013 you will have a distinct competitive advantage and demand on your services if you have strong knowledge of these PaaS platforms and can bring this expertise to a new employer.

Next Time: I will look at the last Cloud service model – SaaS and see how the productized market vendors are getting on and how some of the new technologies are innovating the way we use IT.   PS – As an additional note since I started researching and writing this blog a few companies have started to offer a Database as a Service offering – DbaaS – which is a form of PaaS but focused predominantly on providing databases hosted in the Cloud. If this is of interest, check out: bit.ly/XdAnvI

We love Heroku. It makes deployment so easy and quick. However, it can start to get pricey when you add additional dynos at $35 each a month.

With a small amount of work, you can get a lot more out of your Heroku hosting whilst drastically improving the performance of your site. You might need to spend a little bit of cash on other services, but a lot less than if you simply moved the dyno slider up a few notches, and the result will be much better scalability.

So how do we max out the performance of our Heroku apps? First we stop using Heroku for things it’s bad at, then we let it do more of what it is good at, running your application code.

1. Offload Assets to S3 and CloudFront using asset_sync

By default a Heroku dyno is responsible for serving all the assets for your site, so every page load will involve multiple requests to the dyno.

The asset_sync gem modifies asset pre-compilation to sync all of your assets to an Amazon S3 bucket from where they are served directly and freeing up your dyno to handle more requests.

If you want to speed things up even more, you can slap Amazon’s Cloudfront CDN in front of your S3 bucket with multiple subdomains. Michel Sikkes has an excellent guide to serving you assets with S3 and cloudfront. Your assets will be served through Cloudfront from multiple subdomains (e.g. assets[0-3].myapp.com), all of which point to the same bucket. Not only will your assets be served through CloudFront’s speedy global CDN, but most will be downloaded in parallel. Browsers make a limited number of concurrent requests per host name (2 for IE, more for other browsers) so using multiple CNAMEs increases the number of concurrent connections, significantly reducing the page load time for users with good connections.

The cost of serving assets from S3, even with CloudFront, is very cheap and scales directly with the amount of data. Compared to adding another Heroku dyno this is great value, and has the added benefit of speeding up overall page loads.

2. Don’t Upload and Process Files with your Web Dynos

If you use something like CarrierWave or Paperclip, by default the uploading and processing of images is done by your dyno. While this is happening your dyno is completely tied up, unable to handle requests from any other user. If one person uploads a 2Mb image on a slow connection everyone else will be locked out for the duration.

To prevent this from happening you need to decouple the upload process from your dyno. The CarrierWave Direct gem does just this. With a bit of client-side magic it uploads files to S3 directly, rather than through the dyno. The images then get resized by background processes using DelayedJob or Resque. This obviously has the downside that you’ll need a worker running, but there are ways to manage these cost-effectively which I’ll talk about next.

Another option, which we’ve used recently, is the awesome Cloudinary service. They provide direct image uploading, on-demand image processing (including face detection, which even seems to work on cats) and a worldwide CDN all in one package. There is a free tier to get you started, and for $39 (slightly more than one Heroku dyno) their Basic plan will be more than enough for many sites. Obviously you could just spend money on another dyno, but that just scales your performance linearly, without really solving the fundamental performance bottleneck.

3. Background Processing the Smart Way with Delayed::Job and HireFire

Background processing with Delayed::Job is a great way of speeding up your web requests. Potentially slow tasks like image processing or sending signup emails can happen outside of the request-response cycle, making it much snappier and freeing up your dyno to handle more requests. The downside is that you need to run a worker dyno at $35/month.

Michael van Rooljen’s HireFire modifies Delayed::Job and Resque to automatically scale the number of worker dynos based on the jobs in the queue. Because Heroku charge by the dyno/second, spinning up 10 workers for one minute costs the same as one worker for ten minutes, so with HireFire you can potentially get things done quicker while paying less than you would if you ran a dedicated worker dyno.

HireFire does have one limitation, it only works for jobs scheduled for immediate execution. If that is an issue Michael has a HireFire service that will monitor your application for you, so jobs scheduled in the future will be run.

4. Offload Complex Search to a Dedicated Provider

If you have an application that needs to perform complex searches over large datasets don’t do it in your application directly. If searches regularly take a long time (a couple of seconds or more) consider using something like Solr (available as a Heroku plugin), Amazon CloudSearch, or one of the many Search as a Service providers. You’ll not only get faster search performance in many cases, but you’ll save vast amounts of development time trying to optimise your in-app search. Of course, if you have a simple application with straight-forward search then this probably won’t be worth it the cost, but it’s something worth considering.

5. Turbo-Charge your Application with Memcache backed View Caching

If you’ve not encountered caching in Rails, stop reading this article right now, go read the Rails Guide to Caching and then DHH’s short guide to key based cache expiry. Caching in Rails 4 will be even better, with improved support for “Russian Doll” caching.

View caching in Rails can have a profound effect on your application’s response time. In the past we have found that rendering pages, especially complex ones with lots of partials, can easily account for two-thirds of the total processing time, much more than you might expect.

Simply using caching will help speed up your application, but the default cache store is not shared between dynos so the benefits are limited. In contrast, a Memcache store is shared between your dynos so they all benefit from any cached item. Heroku has two add-ons that let you very easily add memcache to your project. The Memcachier addon gives you 25mb for free, and is pretty reasonably priced from there on up. Just adding a small cache store of 25mb can make a significant difference to the load time of your pages.

6. Finally: Slice and Dice your Dynos with Unicorn

So after spending a little bit of time, and a relatively small amount of cash, we’ve offloaded much of the work that was being done by our web dyno and onto services that are better suited to it; drastically speeding up our request-response cycle. Our single dyno can now handle significantly more users per minute, who are happier because they get a much faster response from the site.

However the default Heroku dyno configuration only handles one request at a time. If you wanted to increase your level of concurrency in the past you would have to increase the number of dynos. That’s all changed with the release of the Rack server Unicorn which can handle multiple concurrent connections. For most applications a single dyno should be able to handle between two and four connections at a time. The main constraint will be memory (limited to 512mb per dyno), so keep an eye on the gems you are loading in your production environment. Florian at Rails On Fire has done a great introduction on setting up Unicorn on Heroku. If you’ve followed the previous steps you should be using less memory on your web dynos, allowing you to use more threads.

Putting it all Together

At the end of all this we’ve freed up our Heroku dyno from doing things it’s not very good at like serving static files and uploads, and juiced up its performance when doing what it’s great at, serving Rails application requests with no sys-admin in sight.

Each technique can be easily applied to your existing applications, but if you develop with them in mind from the start you get all the benefits with almost no additional work. On their own each one will help the performance of your application, but combining them together will significantly extend the amount of time before you have to start forking out for lots more dynos, and when you do you’ll get much more bang for each of your thirty-five Heroku bucks.

If you’ve got any other tips for getting the most out of a Rails application, whether or not it’s on Heroku, we’d love to hear about it them!

Big spender.

There’s my heroku bill after hosting the voting for a local television contest. $2.37. Over 40,000 people used the app over a period of 2 weeks.

I spent the same amount on coffee this morning.

What?! How?!

I’ve always been really interested in scaling and getting the best possible performance out of limited resources.

I had the opportunity recently to build out the voting back-end for a local television contest. Projects like this are fun because I had the flexibility to try out something new and a lot of people would be using it.

The Stack

I wanted to build an API to handle the voting and get the best performance out of it. Usually I’d use Sinatra for this, but this time I chose to try out Goliath, which is a non-blocking Ruby web server for building APIs. I also used Grape since it would make writing the API even easier.

I hosted it on Heroku. My initial goal was to see if I could keep it to only 1 dyno (more fun with a challenge).

If you’re not familiar with Heroku, a dyno is the equivalent of a small virtual server with 512mb of memory. They also give you your first dyno for free. So all I ended up paying for was the postgres basic database add-on.

Stress Test

Before releasing it, I did some stress testing with Siege. I couldn’t have this app failing when it went live.

212.45 transactions per second.

On only 1 heroku dyno.

WOW GOLIATH IS FAST.

Each transaction was a single GET request, that hit the database (postgres) and returned the count of contest votes. Postgres was probably the slowest part of each request. I could optimize even further by caching with memcached. Maybe next time.

The 200+ transactions per second was way more speed than I needed for this app. And much more than I expected to get out of a single dyno.

I knew that the traffic for this app would be pretty sporadic. Highly dependent on when it was mentioned on TV. Without Goliath, I would have needed to use more dynos to account for the spikes in traffic. But with the performance I was seeing out of Goliath, I knew it could handle the peaks with just 1 dyno.

See the details of the Seige test here.

If you’d like to try it out yourself, the app I used to run this contest was a modified version of Mathy Poll. Which is open source and you can grab it on github.

A blog post went viral this week and uncovered an issue in PaaS provider, Heroku’s routing mesh, which has caused a significant degradation of Ruby on Rails app performance. Essentially, over the past three years Heroku moved from “smart” to “random” web request routing among an account’s “dynos”, Heroku’s processing units. Each dyno runs about $35/month.

The change in architecture, which is explained in detail here, is only a portion of why Heroku is catching heat. Here are the real reasons:

Same cost, less value

First and foremost, there have been no reductions in price for the net reduction in capacity provided by the service. This is especially frustrating to the community at a time when other PaaS providers like Amazon Web Services (AWS) continue to lower their prices. Heroku’s platform leverages AWS in its architecture. Over time, users were purchasing more dynos for their accounts with diminishing returns. Each additional $35 dyno does not provide $35 in value.

Appearance of deception

Second, Heroku boasts robust monitoring tools that allow you to see the performance of your applications across many layers of their architecture. However, the monitoring tools provided by Heroku and New Relic don’t reflect the area of latency caused by random request routing. For a highly sophisticated platform provider, this oversight comes across to the community as deceptive rather than as an oversight.

Lack of coherent communication

Finally, there was no communication to customers about the change, and no consistent and coherent documentation. In fact, documentation around Heroku’s site was confusing and contradictory. Again, the appearance of deception.

Response

Heroku is responding openly and humbly, which is the proper way to handle the situation, but there are many who don’t think the responses go far enough. They want monetary reparations as well. I’d guess Heroku is considering concessions quietly by necessity as they are part of a publicly traded company and any concessions could have an impact on current and prior period financials of the company.

The whole debacle reminds us that the lack of transparency over time can cause much larger PR problems for companies than the issues themselves. Individual customers have the platform to tell their story and initiate change.

For the next few months, I’ll be working with the team at LocalRuckus, building a new Node.js API and application.  As a small shop with no dedicated Sys Admin or Dev Ops, its essential that we find Node.js hosting that is flexible, fast, and cost-effective.  I’ve been considering three major players in the Node.js hosting scene, Heroku, Nodejitsu, and Appfog.  There are some good comparisons out there (I especially like Daniel Saewitz’s article), but I wanted to share my 2 cents.

Value for Development

Heroku provides a great feature for development/sandbox apps – your first dyno is FREE.  Combine this with the starter Postgres package, and you can have a development version of you app up and running for $0/month.

Nodejitsu does not offer a free tier, so you are on the hook for paying for pet projects, etc.  That said, their pricing starts at $9/mo for a micro package, and scales up pretty gently from there.

Appfog provides a pretty great package for trying out an app.  You can provision your database, caching server, queue server, and application servers in a few clicks, all managed from one central dashboard.

Winner:  Appfog

Value for Production

Heroku pricing scales linearly with your traffic.  Using a simple slider, you can add new dynos to your application.  Each new dyno runs $35/mo, however, there is no commitment – you can scale up for brief spikes, and scale down if traffic subsides.

Nodejitsu and Appfog, on the other hand have fixed monthly prices.

Nodejitsu prices based on drones, which seem to offer 256MB RAM and processing power roughly equivalent to half a heroku dyno.

Appfog prices based on RAM, which creates a bit of a problem.  While 2GB of memory can be had for $20/mo, moving up to 4GB is a rather steep $100/mo.

Winner: Heroku

Deployment

Heroku – Deploy to a git repository

Appfog – Use the downloadable af tool to push updates

NodeJitsu – Use the jitsu tool or git integration.

Winner: Nodejitsu

Database

Heroku leverages expertise in Postgres, providing plans that scale with your application, including free database levels for getting started.  Production databases support 1TB of data, starting at $50/month.  If you prefer another database platform, Add-Ons are available for Redis, Mongo, Couchdb, ClearDB, JustOneDB, and Amazon RDS.

Nodejitsu continues to take a fairly minimalist approach, with no built-in database.  They provide Add-Ons for Mongo, Redis, and CouchDb.

Appfog allows you to use your service instances to host Redis, Mongo, Postgres, or MySql databases.  They also provide add-ons for Mongo and ClearDB.  The main knock here is that your database shares the processing and memory quotas with your other services, and I’m skeptical that such an approach could support a high traffic app.

Winner: Heroku. Production quality Postgres, with great Add-Ons for a variety of other databases.

Other Languages

Tie: Appfog & Heroku. Appfog’s PHP support opens a lot of opportunities (such as hosting WordPress), however, Appfog seems to have trouble keeping their runtimes up to date. As of July 2013, Node.js was only up to version 8. Heroku provides good language options, and a serious commitment to keeping the runtimes up to date.

Other Considerations and Final Thoughts

An important consideration for many node apps is web socket support.  Nodejitsu has it, the others don’t.  If you need this feature, your choice is clear.  At this point, Heroku’s flexibility, large community, and great add-ons make it my go-to for applications, however, I think Appfog has put together a great offering, and I’m looking forward to using it more in the future.

When I wrote about Heroku and Amazon Web Services a few months ago it seemed both services were equally worthwhile in many respects. My rule of thumb was: small app–Heroku; large app–Amazon. Since then a few events have caused me to reevaluate my position.

Heroku’s customer non-service

Recently, Heroku revealed a new site interface. In the past if you needed something you could open up a free support ticket. Now the only options for free support are the docs or Stack Overflow. Yes, they actually link to Stack Overflow on their new help page. Ticket History is still visible, but there is no way to open up a new ticket.

Granted, AWS doesn’t provide much more than Heroku. But AWS does have a free forum where Amazon employees reply to users’ support questions.

I recently had an issue related to billing with Heroku. Heroku offered a beta period for their new database stack allowing developers to try out their new databases free of charge. I added a couple for testing believing they would remain free until their general release in August. Heroku’s blog post indicated they would notify users prior to billing them.

I assumed there would be an additional step needed to opt-in before being billed, but this was not the case. The only notice I found was a note in one of their email newsletters (which I never read). The notice was one sentence in the middle of the email (underlined in red).

Nearly two months later, I discovered we had accumulated several hundred dollars in charges for databases that we weren’t even using. I tried writing them back at their billing@heroku.com account, but it’s been three weeks and I have not received a response.

How AWS would have prevented this

Around the same time that I noticed the excessive Heroku charges, I logged into our AWS account. What a pleasant surprise to be greeted with a message telling me how to set up notifications for estimated cost triggers. (I already use Amazon’s Simple Notification Service for CPU usage alerts on our EC2 instances.) The alerts are easy to configure due to an intuitive web interface. After establishing a preferred trigger, you can review the past few days’ activity which shows when the trigger would have activated. I immediately set up cost alerts for my AWS account. AWS was nice enough to place the cost alert feature in a prominent place where everyone should see it. Heroku may have a similar feature, but I have not found it.

The billing issue is a small concern, but it illustrates how mature AWS has become while Heroku’s new interface is somewhat of a regression. In the future I will explore how to get up and running on AWS, and how to leverage Ruby gems to achieve similar functionality to that offered by Heroku.

Recently I have been playing PaaS for test thing out, and with recent General Available of AppFog, PaaS providers’ competition become more interesting. In this post, I will compare AppFog with a mature PaaS provider, Heroku.

So, there are many people think, what are their primary features that make you stick to particular PaaS provider? Lets do the comparison using case study.

Memory Management

Do you ever come across a senario where your app is consume 80MB most of the time, or well, 512MB is not enough for you when you deploy a huge Java Web Application (Yeah, Java)?

With AppFog, this is not an issue for you, since they allow developer to configure how much memory they want to allocate to their app. Wow, awesome right? No more wasting your money to invest on ten dynos on heroku, where a lot of memory is not used.

However, this can be a issue for you, if you choose to deploy your app to Heroku, you will need to buy a dyno with 512MB of RAM where either sometime is too larger, or too small. If your application consume 100MB RAM, and you deploy ten dynos, well, you wasted (512 - 100) * 10 = 4120MB of RAM!

Pricing Model

I have a web application on production, suddenly my application is getting huge traffic after a tweet by a famous person. I want quickly scale my application to handle such traffic.

According to AppFog pricing model, it is by subscription based model by select a plan. However, assume their able to upgrade plan within minutes, then it is fine (See Comment). But what if upgrade plan required more than 4 hours? Maybe your application will crash before upgrade to better plan. Futhermore, after 1 week your application traffic is back to normal, I believe you want to downgrade plan to save some money, which required you to submit a ticket to their support. However, AppFog offer free plan with 2GB of RAM, equivalent up to 4 dynos at heroku.

Heroku is pay as you go, scale on demand PaaS, so this kind of senerio will not affect heroku users. Futhermore, you can scale to 10 dynos for an hours then scale back to 2 dynos. Cool.

Data Centers

I have a application, with the target users are from Asia. I want able to serves my content from data center that with low latency to Asia visitors.

AppFog is the first PaaS provider provide cross regions deployment option to its customers. It allow me, a developer from Asia to deploy application that nearby here, such as Singapore. This enable my application serves visitors faster.

Heroku does not support cross regions deployment option to its customers. I have a test application writed in Ruby On Rails, with a feature to upload a image to AWS S3 bucket at Singapore. But its not stable because there is 30 seconds timeout on heroku platform, which is reasonable constraint. But I just cant accept the application always failed to upload a image file. The workflow petty simple:

Upload file to application at Heroku -> scale image into 3 deviations -> upload images to my AWS S3 bucket at Singaprore.

Maybe I need redesign my application to use dyno worker, Resque, Redis to do such thing? Wow, I give up.

Data Store

Most of the application required at least one datastore, let say I want a Relational Database for my application, I want able manual backup before deployment, daily automatic backup, and other features.

AppFog just GA not long ago, currently they have free shared DB, such as mongoDB, MySQL, Postgres. However, they still no plan for serious production database option, but only through add-on such as ClearDB. But their roadmap included dedicated database, hopefully will be feature rich database plan and release soon!

With heroku, it offer serious database production for RDBMS, Postgres. Futhermore, their offer also include other nice feature such as fork, follow, data clip and so on.

Deployment

Assume I use scripting language such as Ruby for my application, I want able to deploy to production with simplest way, such as git push.

Unforturenly AppFog does not have git integration into deployment, unless you need a custom application to deploy for you, such as node-pusher.

Heroku’s deployment is tightly integrated with git, it is great and most of the people use git anyway.

Conclusion

The table below show the sumarrized comparison.

My choose of PaaS is AppFog, even the comparison shows Heroku is better. There are some roadblock in heroku cause me to choose AppFog such as memory managemet and Data Centers.

Valuable Features:

Room for Improvement: