All posts in General Posts

This blog is about inspiring people of what’s available in the cloud and how they can make use of it.

Today I’m going to start experimenting with the concept of “cloud ideas”.

I have a ton of ideas on new web services, online social games or mobile apps that make use of the web and things in the cloud. Some of these ideas evolve into a personal prototype, some as input for Firmhousr projects but there’s still a large gap to fill because I don’t have infinite time, and some ideas just aren’t interesting for Firmhouse.

I encourage you to take the ideas, build apps or businesses on it, discuss it with personal friends or business partners. Hell, you can start the next Google if you want to. I will license the text of the ideas under a Creative Commons license (note: not the idea itself!) This way I’d like to bring people together, and make cool stuff, socially, businesswise or technical.

After reading a cloud idea and you want to bring it further, feel free to contact me and I’d be happy to discuss it personally, over e-mail, skype or other means. Perhaps I can even connect you to other people.

Moving to an open world where we can inspire each other to create such great stuff

Pole in the sea, originally uploaded by michiels.

I love this picture. I took it on a trip to the Dutch coast in Scheveningen. I really like the variations in the cloud’s colors and the movement in the sea.

I really like this paragraph from the following blogpost
 
http://productblog.37signals.com/products/2008/12/getting-real-is-all-about-being-lean-and-agile-and-keeping-things-simple.html
 
“From my own experience I know it’s really easy to overcomplicate things, as your company grows you start adding various control mechanisms, and before you know it your implementing controls mechanisms to control your control mechanisms. Getting Real reminds small businesses why they should stay lean and agile, and why they shouldn’t try to overcomplicate things.”

Posted via email from Shaping Clouds Notes

Currently, tweetburner.com has one table which is really big. We’re busy optimizing our database to make it more scalable and run faster on big calculations. For this, we need to set up multiple replication servers.
 
Normally you would just dump all tables into a sql or text file to copy them over to another database server. However, with a table of over 21 million (yes!) rows the SELECT query will lock the table for updates and therefore other reads so our service becomes unreachable.
 
By migrating your database into chunks you can make this process less painful.
 
We noticed that selecting 100,000 rows from a table it takes up to 0.8 second to run the query. If you select a million rows it can take up to 5 seconds. Apperantly the number grows exponentially. Keep in mind what selecting 21 million rows will take.
 
With a little scripting we can copy this table in about 22 times. 22 * 0.8 with 22 times a table lock of up to 1s this is less painful for your service and thus your users.
 
Conclusion: When migrating or backing up large tables, split them into acceptable chunks depending on row and table size so you won’t deadlock your service. Do this my selecting each 100,000 rows, put them in a dump file and copy them to your backup location. You can merge them after that.

Posted via email from Shaping Clouds Notes