Amazon Web Services (AWS) is great and we use it as our #1 hosting infrastructure at Firmhouse. We still have some “old” traditional virtual servers laying around on some (also great) servers but we are slowly moving away from them to be totally scalable in a cloud environment.
There are however, one few problems with AWS if you’re hosting multiple websites or web applications on multiple instances for a variation of different clients and internal projects: you can’t keep track of costs and usage statistics in a categorized or budgetized manner.
Because we want to bill our clients what they are using from us (we do their managed hosting) and not give them some lame server package that is overpriced because of the overhead (100G they won’t be using) anyways, we want to give them an honest picture of what they are using and what virtual instances we have installed for them.
Amazon just gives you a credit card bill at the end of the month, which makes it very hard to split all costs and usage into projects or budgets.
Also, Amazon bills you on one credit card but if you have multiple people working on one project with several AWS accounts, there is currently no way of getting some insights in what instances the members of a tream are launching and how much they cost all combined. All you get is $-signs on the credit card bills and the i-instance id’s in your management console(s).
And, budgets would really come in handy when you have a web application or piece of software that automatically scales itself on EC2 and launches instances by itself. Budgets shouldn’t mean terminating or stopping instances if the costs rise above a certain treshold but it would certainly come in handy if you would get a warning e-mail telling you the costs for this weeks where really of the charts so you could act accordingly and maybe re-thing your pricing strategy or make another business-wise decision.
BTW, I should use another word for “budget”. I hate that word because usually involves guessing and the only thing you can guess is that your app shoud be scalable. But my usage of “budget” it should clarify my point. If you have another word, let me know 
So, having that said: *drumroll*
We’re Introducting Qloudwatch
We at Firmhouse have the need for solutions to these problems and we see these problems pop up in forums and mailing lists about AWS on other places we decided to get sweating and create a web app for this: Qloudwatch. We have a basic version up and running, so if you would like to try it please contact me at michiel@firmhouse.nl. The basic app currently has the following functionality:
- Create projects in which you can add instances recognized by the API in your AWS account.
- See the total cost of the project until “now” or view a history of the costs by month.
- Add billable and non-billable instances so you can for example bill all production instances to your clients and not bill your test instances.
- Invite other Qloudwatch users to a project so they can also add instances that can be set billable or non-billable so you can “share” statistics and costs on your instances.
- Get an automatic e-mail notification if you have running instances in your AWS account that you haven’t added to a project yet so you will never forget to categorize that one test hour you ran at 4 AM in the morning when your caffeine withdrawal started to kick in.
We would LOVE to know what you think: wether you disagree, wether you agree, what features you would like to see, if you would like to use the app for free, if you would like to swipe your card for it or if you have any other questions about our work on AWS. Let us know!
Here are some sneak preview screenshots:
I just got back from the holidays. Ski-ing was great btw. I actually know my parallels know. It was a fun week and I’m going back as soon as I can. But more important, this is wat’s on the schedule the following weeks (and maybe some 2010 resolutions):
- Get all client projects up and running again and start working.
- Write and launch a web app that let’s you monitor your cloud computing instances, organize them into projects and get you some financial insight in your costs and profits you get from cloud computing. This is especially handy if you have clients on your cloud servers, so you know what you can invoice them.
- Get a lot of posts out on shapingclouds.com
Last week, Amazon Web Services released a great killer new feature in their virtual computing cloud EC2. You can now launch AWS instances (virtual servers) from an EBS (Elastic Block Storage). An Elastic Block Storage is like a USB hard drive you can plug in and out of your virtual server to expand space or back-up data. Now, you can boot your instances from an EBS which is great for testing your web apps and lowering costs. Here’s why this new feature rocks:
Previously when you would terminate/shut down your instance, all data would be lost and you would need to boot up a new instance from a default or self-configured AMI (instance images). The default AMI’s are quite great, Amazon and all kind of community people have set up a lot of AMI’s you can use with Windows, Fedora, Ubuntu, you name it. However, when launching an instance from a template, you will have to install your own software (like nginx, Apache or Ruby on Rails) over and over again. You can create your own AMI’s, but the process is technical and quite hard so I actually never had the chance to build my own one. With the new boot-from-EBS feature, in the AWS Console you can now create an AMI on the fly from a currently running instance and save it on an EBS to boot from. This saves you a lot of time on setting up your AMI and updating the software on it. The only downside is that you will pay for all the EBS storage, which is probably bigger than storing an AMI on S3.
But, this gives us a great opening for easily testing our web applications. You can now just create a bootable EBS with your self-configured AMI on it and run the instance when you would like to test your app or show it to a customer in a real-life server environment, and not locally on your laptop or development server you still have in a dusty corner of your office. Because you can now Stop the instance in stead of Terminating it. All data will be preserved without having to attach other EBSes with all your files and databases and such.
In the end you will have your testing instance up faster and maintaining the required software on it is a much simpler task, you just create new bootable EBS disks from your instance and launch them. Also, because you can now Stop the instance in stead of terminating, you only pay for storing the EBS image and not running your test server all the time.
You could provide a great service for your customers that they can push a button on your admin panel and the server with the app could go live when they requested it, without you even knowing they are testing or having to set up the instance of your web app manually. I see great time and cost-saving service opportunities here 
We are working on various projects that involve e-commerce. Having good content and a great website structure is important for this so it will show up nicely in all the search engine results and will be linked to by many people. I have some experience with building a web app so it displays nicely for Google but I decided that I needed to know more. Especially how users and search engines react to certain content and site changes.
If I need to learn something, I need to learn it the practical way. Oh of course I read books and tutorials for all the nice tips and best practices but actually experiencing the thing you’d like to research is the best way of learning something because you can tweak things fase and see how it comes out.
Regarding the e-commerce/SEO thing I decided to set up a “fake” website where the visitor can search for books and see where they can buy it online, hopefully for the lowest price. This however, is not a required thing. The goal is to see how people reach the website and if they actually use the referral links to order the books. And of course, how I can improve these sort of things.
To be certain: this is not a new web concept that is providing a great service for the user of finding and buying books. It’s a bad-ass affiliate site because I want to know if they work and how they work because I’ve heard of a lot of these sites and apparently they somethings work really well.
The website in question is Bestel het Bij. I’ve already had some conversations with people around and the next focus will be: adding dynamic content and quality content.
Today was another day of the first day of the first release of a new web application. After prototyping the new project for a few days, the time had come that we’d finally show the first part to the rest of the team members and the client. Currently, it’s the least thing I like about releasing new apps: deploying. It always involves setting up the Git repository on Github, adding domain configuration on the web server (or even worse, setting up a load balanced environment), launch ec2 instances and write or configure the (web|cap)istrano scripts. It takes hours and multiple things fail you’ve already done a thousand time on other deployments. Here’s a list of what the most awesome deployment app would have:
- It should be a web app, hosted, pre-installed, no fuzz, clean-interface, Basecamp-like subscription payed shizzle.
- It MUST support with deploying from various SCM’s like Git, SVN, Bzr and maybe even still CVS. (Git with submodules, svn with externals, you do the math)
- It MUST constrain the user to some broadly supported standard deployment strategies. For instance Ruby on Rails applications with Apache/NGINX and Phusion Passenger (aka Mod Rails). The user should not have to worry about the background of the deployment and should not have the power to break stuff.
- It MUST have a philosophy of projects/applications over servers. Your app runs “in the cloud”. The architecture behind it is secondary. Instead of installing an application clone on a new server and configure the load balancer, you add a server – or better an instance – to your application and the application makes sure the new instance of itself gets picked up.
- It SHOULD take over all the configuration of the application. From generating passwords to setting up databases and migrations to configuring new servers in a load balancer. The deployment app knows best.
- The user should only have to add their deployment keys or maybe a local deployment password everything else (like deployment locations and stuff) should be convention over configuration. Everything the user want’s to add should be available through nice gradient-like big buttons with smilies on them and the user MUST be greeted with “Yo, dude”.
- Everything should be as sexy as your girl or boyfriend.
And no worries guys, we are buiding this right now.
P.S.: If you don’t have a girl or boyfriend, pick your dog or cat. If you don’t have those either. You’re not allowed into the beta.
Michiel Sikkes is Technical Lead and partner at 
