I just wanted to share with you this presentation I created yesterday to communicate to other people and ourselves on how we use Scrum in Firmhouse. This is a first set of tools I’m creating for ourselves to get going with Scrum to create our products. But it can be a good piece of information for you too. The slides are great to keep as a reference on what kind of meetings you are going to have and what you need to discuss in them.

Expect more detailed Scrum and Agile Development stuff in the near future. We’re getting to an awesome workflow here. In the mean time, read 37signals excellent post about how they are going to approach development of new features in their products.

Here are the slides via SlideShare embed:

{ 0 comments }

So I thought I’d give you an update on Qloudwatch. Qloudwatch is going to be the easy-to-use web service that allows you to get insights on your Amazon Web Services cloud usage, group your instances into project and set budgets and warnings for AWS instance costs.

We’ve been working really hard the past few weeks to get ready for a first public release. Bob is busy thinking about an honest and functional business model and researching our terms of service and I’m crunching feature development, tweaking, adding even better security.

What we’re currently looking at releasing in our first launch is the following:

  • Create projects and give other people access so you can collaborate on a cloud project. Multiple users with different AWS accounts can be added to a project so you can share costs and get a good overview what your application or cloud team is using, spanning possibly multiple AWS account.
  • Adding instances to project by accessing the AWS API with your AWS credentials. Yes, we do ask you to enter them into your account for now. Because of this, we’ve implemented SSL security and encryption into our database. Even in our demo period you can test right now at http://qloudwatch.com
  • Give you total and monthly cost estimates based on the running hours of the instances in a project.
  • A personal dashboard where you can get insights in the instances your personal AWS account is running.

We’re going to rapidly develop more statistics and are going to build in ways to collect better usage data from your instances if you choose to. If you want to influence our priorities, please give us some feedback of what statistics and overviews you would like to see first.

Here are some more sneak preview screenshots:

{ 1 comment }

Amazon Web Services (AWS) is great and we use it as our #1 hosting infrastructure at Firmhouse. We still have some “old” traditional virtual servers laying around on some (also great) servers but we are slowly moving away from them to be totally scalable in a cloud environment.

There are however, one few problems with AWS if you’re hosting multiple websites or web applications on multiple instances for a variation of different clients and internal projects: you can’t keep track of costs and usage statistics in a categorized or budgetized manner.

Because we want to bill our clients what they are using from us (we do their managed hosting) and not give them some lame server package that is overpriced because of the overhead (100G they won’t be using) anyways, we want to give them an honest picture of what they are using and what virtual instances we have installed for them.

Amazon just gives you a credit card bill at the end of the month, which makes it  very hard to split all costs and usage into projects or budgets.

Also, Amazon bills you on one credit card but if you have multiple people working on one project with several AWS accounts, there is currently no way of getting some insights in what instances the members of a tream are launching and how much they cost all combined. All you get is $-signs on the credit card bills and the i-instance id’s in your management console(s).

And, budgets would really come in handy when you have a web application or piece of software that automatically scales itself on EC2 and launches instances by itself. Budgets shouldn’t mean terminating or stopping instances if the costs rise above a certain treshold but it would certainly come in handy if you would get a warning e-mail telling you the costs for this weeks where really of the charts so you could act accordingly and maybe re-thing your pricing strategy or make another business-wise decision.

BTW,  I should use another word for “budget”. I hate that word because usually involves guessing and the only thing you can guess is that your app shoud be scalable. But my usage of “budget” it should clarify my point. If you have another word, let me know ;-)

So, having that said:  *drumroll*

We’re Introducting Qloudwatch

We at Firmhouse have the need for solutions to these problems and we see these problems pop up in forums and mailing lists about AWS on other places we decided to get sweating and create a web app for this: Qloudwatch. We have a basic version up and running, so if you would like to try it please contact me at michiel@firmhouse.nl. The basic app currently has the following functionality:

  • Create projects in which you can add instances recognized by the API in your AWS account.
  • See the total cost of the project until “now” or view a history of the costs by month.
  • Add billable and non-billable instances so you can for example bill all production instances to your clients and not bill your test instances.
  • Invite other Qloudwatch users to a project so they can also add instances that can be set billable or non-billable so you can “share” statistics and costs on your instances.
  • Get an automatic e-mail notification if you have running instances in your AWS account that you haven’t added to a project yet so you will never forget to categorize that one test hour you ran at 4 AM in the morning when your caffeine withdrawal started to kick in.

We would LOVE to know what you think: wether you disagree, wether you agree, what features you would like to see, if you would like to use the app for free, if you would like to swipe your card for it or if you have any other questions about our work on AWS. Let us know!

Here are some sneak preview screenshots:

{ 2 comments }

At Firmhouse, we are working on a new web service and we’re releasing an in-development demo to everyone who is interested. Normally, in demo apps security is less the case and there is always some disclaimer that says you shouldn’t use real production data like passwords and API keys. In my opinion, this sucks because you allways want to use your real-life data to test a demo and see if it’s useful for you specifically. That’s why, the best way to have some security in your demo app is by using a self-signed SSL certificate to secure passwords and other sensitive account information your demo users will be adding.

For our web app, we use Ruby on Rails running on Phusion Passenger, served by Nginx.

This blog post will guide you trough the process of adding a self-signed SSL certificate to your Rails app, running on Phusion Passenger and Nginx by following these steps:

  1. Generating the required SSL key and certificate files for use with nginx.
  2. Recompiling the nginx server through the passenger-install-nginx-module command.
  3. Configuring your web app in nginx to redirect non-secure connections to the secure address of the app with https:// and make sure www. will get redirected on both versions as well.

    I use a few other blog posts in this article, so I would like to thank the authors for providing the information publicly and freely.

    Ok, now let’s start:

    [click to continue…]

    { 8 comments }

    I just got back from the holidays. Ski-ing was great btw. I actually know my parallels know. It was a fun week and I’m going back as soon as I can. But more important, this is wat’s on the schedule the following weeks (and maybe some 2010 resolutions):

    • Get all client projects up and running again and start working.
    • Write and launch a web app that let’s you monitor your cloud computing instances, organize them into projects and get you some financial insight in your costs and profits you get from cloud computing. This is especially handy if you have clients on your cloud servers, so you know what you can invoice them.
    • Get a lot of posts out on shapingclouds.com

    { 0 comments }

    Rational emotions

    December 9, 2009 · 0 comments

    in General Posts

    I know how you are, world
    I know how you need to be.

    That’s why I can’t get mad at you,
    When you are sometimes so cruel to me.

    I just liked this poem :)

    { 0 comments }

    Last week, Amazon Web Services released a great killer new feature in their virtual computing cloud EC2. You can now launch AWS instances (virtual servers) from an EBS (Elastic Block Storage). An Elastic Block Storage is like a USB hard drive you can plug in and out of your virtual server to expand space or back-up data. Now, you can boot your instances from an EBS which is great for testing your web apps and lowering costs. Here’s why this new feature rocks:

    Previously when you would terminate/shut down your instance, all data would be lost and you would need to boot up a new instance from a default or self-configured AMI (instance images). The default AMI’s are quite great, Amazon and all kind of community people have set up a lot of AMI’s you can use with Windows, Fedora, Ubuntu, you name it. However, when launching an instance from a template, you will have to install your own software (like nginx, Apache or Ruby on Rails) over and over again. You can create your own AMI’s, but the process is technical and quite hard so I actually never had the chance to build my own one. With the new boot-from-EBS feature, in the AWS Console you can now create an AMI on the fly from a currently running instance and save it on an EBS to boot from. This saves you a lot of time on setting up your AMI and updating the software on it. The only downside is that you will pay for all the EBS storage, which is probably bigger than storing an AMI on S3.

    But, this gives us a great opening for easily testing our web applications. You can now just create a bootable EBS with your self-configured AMI on it and run the instance when you would like to test your app or show it to a customer in a real-life server environment, and not locally on your laptop or development server you still have in a dusty corner of your office. Because you can now Stop the instance in stead of Terminating it. All data will be preserved without having to attach other EBSes with all your files and databases and such.

    In the end you will have your testing instance up faster and maintaining the required software on it is a much simpler task, you just create new bootable EBS disks from your instance and launch them. Also, because you can now Stop the instance in stead of terminating, you only pay for storing the EBS image and not running your test server all the time.

    You could provide a great service for your customers that they can push a button on your admin panel and the server with the app could go live when they requested it, without you even knowing they are testing or having to set up the instance of your web app manually. I see great time and cost-saving service opportunities here :)

    { 0 comments }

    Quick Search is a great thing on the Android platform. Especially since Android 1.6 will have a global suggestions list for you to add in. This means all apps that have global search enabled will show the search suggestions your app is providing. Quick Search is the little search UI that pops over your Activity when you press the looking glass search-key on your Android device. However, tapping into it with your application for the first time just sucks hard. Follow this blogpost and you’ll have search in your app in no time.

    Implementing basic Android search with SearchManager in your app involves these steps:

    1. Creating a search activity. This activity will display all the results for the keyword provided by the user via de Quick Search bar.
    2. Adding the required elements to your AndroidManifest.xml as well as configuring a searchable.xml meta-data file.
    3. Displaying results in your search activity.

    Creating the search activity

    I use Eclipse to develop my apps, so if you are too, create a ListActivity by creating a new Java Class in your application package. Called it something like MySearchActivity and make sure the Superclass is ListActivity.

    Press Command – Shift – O to load in the required imports and add the following onCreate function:

    	@Override
    	protected void onCreate(Bundle savedInstanceState) {
    		super.onCreate(savedInstanceState);
    
    		final Intent queryIntent = getIntent();
    		final String queryAction = queryIntent.getAction();
    		if (Intent.ACTION_SEARCH.equals(queryAction)) {
    			String searchKeywords = queryIntent.getStringExtra(SearchManager.QUERY);
    		}
    	}

    You’re basically all done here. This onCreate function gets the Intent which your Activity has been called with. It contains the name of the action (ACTION_SEARCH) and the actual search keywords. You can now use the searchKeywords string to actually search your application. For example, get a web search out using a GET request or search your application’s internal sqlite database.

    Adding the required elements to your AndroidManifest.xml

    This part was the most tricky for me. The Android SearchManager documentation just had a jungle of information on it but it’s just too much to handle and hard to distill the actual information from. So save yourself some time and let me tell you what you need to add.

    For the activity that actually displays the search results – the activity we created in the previous step – add the following:

    <activity android:name="MySearchActivity"
                      android:label="Search">
                <intent-filter>
                    <action android:name="android.intent.action.SEARCH" />
                    <category android:name="android.intent.category.DEFAULT" />
                </intent-filter>
                <meta-data android:name="android.app.searchable"
                           android:resource="@xml/searchable" />
            </activity>

    This activity element tells your application that the MySearchActivity is an Activity that can be used to invoke the search. It contains a meta-data element which is a pointer to an xml file that contains meta information for the search UI and behavior, such as labels and wether to include global search suggestions. This is the bit where I went wrong so pay close attention.

    Now, right-click your res/ folder and create a new Android XML file called searchable.xml. Select Searchable from the options of file types. Eclipse should now create the res/xml/searchable.xml file. It will create the xml/ directory if it didn’t exist yet. In this file, add the following:

     <searchable xmlns:android="http://schemas.android.com/apk/res/android"
         android:label="@string/search_label"
         android:hint="@string/search_hint" >
     </searchable>

    The contents of this file configures how the search field is displayed and you can set a lot of options for the behavior. Read about all the options in the SearchManager#SearchabilityMetadata section of the Android Developer documentation. Your own search bar won’t work if you don’t use @string resources for the configuration keys in this file. So don’t use static strings because they won’t work and you’ll loose precious hours of development time to debugging, like I did.

    So, add the search_label and search_hint strings to res/values/strings.xml and put in something nice.

    Displaying results in your search activity

    So we have the initial onCreate function with the search keyword, and we’ve configured the search UI to pop up when the user presses the search key. Now it’s time for displaying the actual results. I will not get into technical details here because you can implement this in a lot of ways. Maybe I’ll dedicate the next blog post to this part. For now, do the the following:

    • Based on the searchKeywords string you have, fire up your search in your local sqlite database or do some HTTP requests to fetch a list of search results from.
    • Add them into an ArrayAdapter<MyClass> with nice formatting.
    • Bind the ArrayAdapter<MyClass> with the SearchActivity by calling setListAdapter(myListAdapter) in the onCreate function.
    • You’re ready to go!

    Take a look at ListActivity on the Android Developer site.

    Conclusion

    If you know the right – and actually simple – steps, implementing search in your application is easy. It’s the actual researching of the docs that suck. Please feel free to leave a comment or reply me on Twitter at @michiels.

    { 3 comments }

    We are working on various projects that involve e-commerce. Having good content and a great website structure is important for this so it will show up nicely in all the search engine results and will be linked to by many people. I have some experience with building a web app so it displays nicely for Google but I decided that I needed to know more. Especially how users and search engines react to certain content and site changes.

    If I need to learn something, I need to learn it the practical way. Oh of course I read books and tutorials for all the nice tips and best practices but actually experiencing the thing you’d like to research is the best way of learning something because you can tweak things fase and see how it comes out.

    Regarding the e-commerce/SEO thing I decided to set up a “fake” website where the visitor can search for books and see where they can buy it online, hopefully for the lowest price. This however, is not a required thing. The goal is to see how people reach the website and if they actually use the referral links to order the books. And of course, how I can improve these sort of things.

    To be certain: this is not a new web concept that is providing a great service for the user of finding and buying books. It’s a bad-ass affiliate site because I want to know if they work and how they work because I’ve heard of a lot of these sites and apparently they somethings work really well.

    The website in question is Bestel het Bij. I’ve already had some conversations with people around and the next focus will be: adding dynamic content and quality content.

    { 0 comments }

    Today was another day of the first day of the first release of a new web application. After prototyping the new project for a few days, the time had come that we’d finally show the first part to the rest of the team members and the client. Currently, it’s the least thing I like about releasing new apps: deploying. It always involves setting up the Git repository on Github, adding domain configuration on the web server (or even worse, setting up a load balanced environment), launch ec2 instances and write or configure the (web|cap)istrano scripts. It takes hours and multiple things fail you’ve already done a thousand time on other deployments. Here’s a list of what the most awesome deployment app would have:

    • It should be a web app, hosted, pre-installed, no fuzz, clean-interface, Basecamp-like subscription payed shizzle.
    • It MUST support with deploying from various SCM’s like Git, SVN, Bzr and maybe even still CVS. (Git with submodules, svn with externals, you do the math)
    • It MUST constrain the user to some broadly supported standard deployment strategies. For instance Ruby on Rails applications with Apache/NGINX and Phusion Passenger (aka Mod Rails). The user should not have to worry about the background of the deployment and should not have the power to break stuff.
    • It MUST have a philosophy of projects/applications over servers. Your app runs “in the cloud”. The architecture behind it is secondary. Instead of installing an application clone on a new server and configure the load balancer, you add a server – or better an instance – to your application and the application makes sure the new instance of itself gets picked up.
    • It SHOULD take over all the configuration of the application. From generating passwords to setting up databases and migrations to configuring new servers in a load balancer. The deployment app knows best.
    • The user should only have to add their deployment keys or maybe a local deployment password everything else (like deployment locations and stuff) should be convention over configuration. Everything the user want’s to add should be available through nice gradient-like big buttons with smilies on them and the user MUST be greeted with “Yo, dude”.
    • Everything should be as sexy as your girl or boyfriend.

    And no worries guys, we are buiding this right now.

    P.S.: If you don’t have a girl or boyfriend, pick your dog or cat. If you don’t have those either. You’re not allowed into the beta.

    { 0 comments }