A problem (and a solution) with AWS: no segregation of instances and no budgets

by Michiel on January 4, 2010 · 2 comments

in Uncategorized

Amazon Web Services (AWS) is great and we use it as our #1 hosting infrastructure at Firmhouse. We still have some “old” traditional virtual servers laying around on some (also great) servers but we are slowly moving away from them to be totally scalable in a cloud environment.

There are however, one few problems with AWS if you’re hosting multiple websites or web applications on multiple instances for a variation of different clients and internal projects: you can’t keep track of costs and usage statistics in a categorized or budgetized manner.

Because we want to bill our clients what they are using from us (we do their managed hosting) and not give them some lame server package that is overpriced because of the overhead (100G they won’t be using) anyways, we want to give them an honest picture of what they are using and what virtual instances we have installed for them.

Amazon just gives you a credit card bill at the end of the month, which makes it  very hard to split all costs and usage into projects or budgets.

Also, Amazon bills you on one credit card but if you have multiple people working on one project with several AWS accounts, there is currently no way of getting some insights in what instances the members of a tream are launching and how much they cost all combined. All you get is $-signs on the credit card bills and the i-instance id’s in your management console(s).

And, budgets would really come in handy when you have a web application or piece of software that automatically scales itself on EC2 and launches instances by itself. Budgets shouldn’t mean terminating or stopping instances if the costs rise above a certain treshold but it would certainly come in handy if you would get a warning e-mail telling you the costs for this weeks where really of the charts so you could act accordingly and maybe re-thing your pricing strategy or make another business-wise decision.

BTW,  I should use another word for “budget”. I hate that word because usually involves guessing and the only thing you can guess is that your app shoud be scalable. But my usage of “budget” it should clarify my point. If you have another word, let me know ;-)

So, having that said:  *drumroll*

We’re Introducting Qloudwatch

We at Firmhouse have the need for solutions to these problems and we see these problems pop up in forums and mailing lists about AWS on other places we decided to get sweating and create a web app for this: Qloudwatch. We have a basic version up and running, so if you would like to try it please contact me at michiel@firmhouse.nl. The basic app currently has the following functionality:

  • Create projects in which you can add instances recognized by the API in your AWS account.
  • See the total cost of the project until “now” or view a history of the costs by month.
  • Add billable and non-billable instances so you can for example bill all production instances to your clients and not bill your test instances.
  • Invite other Qloudwatch users to a project so they can also add instances that can be set billable or non-billable so you can “share” statistics and costs on your instances.
  • Get an automatic e-mail notification if you have running instances in your AWS account that you haven’t added to a project yet so you will never forget to categorize that one test hour you ran at 4 AM in the morning when your caffeine withdrawal started to kick in.

We would LOVE to know what you think: wether you disagree, wether you agree, what features you would like to see, if you would like to use the app for free, if you would like to swipe your card for it or if you have any other questions about our work on AWS. Let us know!

Here are some sneak preview screenshots:

{ 2 comments }

How to Add Some Security with a Self-Signed SSL Certificate to Your Rails App Served by Nginx

by Michiel on January 2, 2010 · 13 comments

in General Posts

At Firmhouse, we are working on a new web service and we’re releasing an in-development demo to everyone who is interested. Normally, in demo apps security is less the case and there is always some disclaimer that says you shouldn’t use real production data like passwords and API keys. In my opinion, this sucks because you allways want to use your real-life data to test a demo and see if it’s useful for you specifically. That’s why, the best way to have some security in your demo app is by using a self-signed SSL certificate to secure passwords and other sensitive account information your demo users will be adding.

For our web app, we use Ruby on Rails running on Phusion Passenger, served by Nginx.

This blog post will guide you trough the process of adding a self-signed SSL certificate to your Rails app, running on Phusion Passenger and Nginx by following these steps:

  1. Generating the required SSL key and certificate files for use with nginx.
  2. Recompiling the nginx server through the passenger-install-nginx-module command.
  3. Configuring your web app in nginx to redirect non-secure connections to the secure address of the app with https:// and make sure www. will get redirected on both versions as well.

I use a few other blog posts in this article, so I would like to thank the authors for providing the information publicly and freely.

Ok, now let’s start:

[click to continue…]

{ 13 comments }

Back from the holidays, handy announcement in a few weeks

by Michiel on December 28, 2009 · 0 comments

in Uncategorized

I just got back from the holidays. Ski-ing was great btw. I actually know my parallels know. It was a fun week and I’m going back as soon as I can. But more important, this is wat’s on the schedule the following weeks (and maybe some 2010 resolutions):

  • Get all client projects up and running again and start working.
  • Write and launch a web app that let’s you monitor your cloud computing instances, organize them into projects and get you some financial insight in your costs and profits you get from cloud computing. This is especially handy if you have clients on your cloud servers, so you know what you can invoice them.
  • Get a lot of posts out on shapingclouds.com

{ 0 comments }

Rational emotions

by Michiel on December 9, 2009 · 0 comments

in General Posts

I know how you are, world
I know how you need to be.

That’s why I can’t get mad at you,
When you are sometimes so cruel to me.

I just liked this poem :)

{ 0 comments }

Use AWS to boot from EBS to start testing your web app

by Michiel on December 6, 2009 · 0 comments

in Uncategorized

Last week, Amazon Web Services released a great killer new feature in their virtual computing cloud EC2. You can now launch AWS instances (virtual servers) from an EBS (Elastic Block Storage). An Elastic Block Storage is like a USB hard drive you can plug in and out of your virtual server to expand space or back-up data. Now, you can boot your instances from an EBS which is great for testing your web apps and lowering costs. Here’s why this new feature rocks:

Previously when you would terminate/shut down your instance, all data would be lost and you would need to boot up a new instance from a default or self-configured AMI (instance images). The default AMI’s are quite great, Amazon and all kind of community people have set up a lot of AMI’s you can use with Windows, Fedora, Ubuntu, you name it. However, when launching an instance from a template, you will have to install your own software (like nginx, Apache or Ruby on Rails) over and over again. You can create your own AMI’s, but the process is technical and quite hard so I actually never had the chance to build my own one. With the new boot-from-EBS feature, in the AWS Console you can now create an AMI on the fly from a currently running instance and save it on an EBS to boot from. This saves you a lot of time on setting up your AMI and updating the software on it. The only downside is that you will pay for all the EBS storage, which is probably bigger than storing an AMI on S3.

But, this gives us a great opening for easily testing our web applications. You can now just create a bootable EBS with your self-configured AMI on it and run the instance when you would like to test your app or show it to a customer in a real-life server environment, and not locally on your laptop or development server you still have in a dusty corner of your office. Because you can now Stop the instance in stead of Terminating it. All data will be preserved without having to attach other EBSes with all your files and databases and such.

In the end you will have your testing instance up faster and maintaining the required software on it is a much simpler task, you just create new bootable EBS disks from your instance and launch them. Also, because you can now Stop the instance in stead of terminating, you only pay for storing the EBS image and not running your test server all the time.

You could provide a great service for your customers that they can push a button on your admin panel and the server with the app could go live when they requested it, without you even knowing they are testing or having to set up the instance of your web app manually. I see great time and cost-saving service opportunities here :)

{ 0 comments }

← Previous Entries

Next Entries →