What If Hosting Was a Library?

# Was thinking, "if hosting was a library, what might it look like..."

require 'ProviderX'

# We'll pretend we already have an account with fictitious ProviderX

host = ProviderX.new(:user => 'foo', :password => 'secret')

# Let's see what ProviderX will allow us to deploy to

host.provides
> ['Amazon', 'Google']

# Let's create a new environment to deploy to

production = host.cloud.new(:vendor => 'Amazon', :name => 'production', :min_cpu => 2, :min_mem => 1024)

# We also would need to define the application that we wish to deploy

application = host.application.new(:repository => 'http://foo.com/foo.git', :type => 'Rails', :name => 'foo', :domain => 'foo.com', :ssl_enabled => true)

# All set! Let's deploy.

application.deploy_to production

application.deployed?
> true

application.deployed_to
> ['production']

production.applications
> ['foo']

# Maybe we forgot that we want to redirect www.foo.com to foo.com

application.redirects.add :from => 'www.foo.com', :to => 'foo.com'

# We don't want to deploy the code again, we just want to sync configuration changes (the redirect).

application.sync_configs production

# Perhaps we could control our resources from within our application

if production.swapping?
  production.ram.add 128
end

# That's it for my day-dreaming.

HAProxy: Routing by domain name

Without too much effort I was able to find multiple search results that said HAProxy could direct traffic according to domain name. Unfortunately I was hard pressed to find any example configurations demonstrating that functionality. Turns out it is in section 7.7 of the documentation.

If we wanted to direct traffic for foo.com and bar.com to different servers we could do so as follows:

frontend http_proxy
  bind 192.168.0.1:80
  acl is_foo hdr_dom(host) -i foo
  acl is_bar hdr_dom(host) -i bar
  use_backend cluster1 if is_foo
  use backend cluster2 if is_bar

backend cluster1
  server server1 192.168.0.2:80
  server server2 192.168.0.3:80

backend cluster2
  server server3 192.168.0.4:80

So first we create a frontend called http_proxy that will listen on port 80 of 192.168.0.1. The next lines are the key parts.

We create an acl called is_foo which checks the Host portion of the HTTP header for the provided value, in this case 'foo' in one and 'bar' in the second acl. The '-i' flag in both examples sets the match to be case insensitive.

The last part of the frontend configuration is to tell it to use a backend named cluster1 or cluster2 depending on if the is_foo or is_bar acl matches.

Finally we define the backends cluster1 and cluster2. The backend cluster1 has 2 servers, backend cluster2 only has 1.

So in this case if we sent a request for http://foo.com to 192.168.0.1:80 the acl that checks for the value 'foo' in the host should pass, and our request should be proxied to port 80 of either 192.168.0.2 or 192.168.0.3.

The practical example shown in the HAProxy documention is that you could use this to direct traffic for assets.foo.com to a server for static content, and www.foo.com over to your application server. However, if you found this article through a search, I'm sure you already have in mind how you want to use this.

In addition to the hdr_dom shown in this example, there are many other hdr_* matchers, as well as many other configuration options with HAProxy. For full details and options see the official HAProxy documentation.

Tiny email sending service with Amazon SQS

I found the "Dynamo Paper" by Amazon to be interesting. In particular I liked the idea of small services. Nothing new, but I liked the concept and wanted to play with it.

The result was the 71 line SQS Mailer, available on Github.

For this experiment the code polls an Amazon SQS queue. If there are messages in that queue, it retrieves them, parses them, and sends them out as emails via SMTP. If there are no messages waiting, it just sleeps for 5 seconds and tries again. So if you ran multiple applications that all needed to send email, they could send it out through this single service by sending a message to the queue. Which SMTP server account the message would go out through is controlled by a combination of a potentially application-specific identifier in the message you send to the queue, and a service-side lookup in accounts.yml.

Will this be useful? Not sure. But it was fun to put together!

Basically I sketched this out onto some notepaper one night, and then later filled in the functionality for each method when I had time:



At the current Amazon SQS pricing of $0.01 per 10,000 requests, leaving this idling would incur a cost of about $6.50 per year from AWS.

As the disclaimer in the README file states, this doesn't handle failure scenarios with any grace. If I decide it is useful for anything, I will give that some attention. Or you can fork the project and let me know what you come up with.

Abort: couldn't find mercurial libraries

I've been setting up my fresh Snow Leopard installation today.

I followed the guide from Hivelogic: Compiling Mercurial on Snow Leopard. The installation completed without issue and 'which hg' showed everything was in place.

However, when I tried to issue an 'hg' command the following error popped up:

"abort: couldn't find mercurial libraries"

Google came to the rescue, and revealed these tweets:

https://wincent.com/twitter/155

https://wincent.com/twitter/156

For Snow Leopard, I just needed:

export PYTHONPATH=/usr/local/lib/python2.6/site-packages

If you hit the same, hope this works for you.

Typing? I'm a programmer not a secretary.

This last week has seen a few posts on the necessity, or lack thereof, for good touch typing skills.

We Are Typists First, Programmers Second - Jeff Atwood

Is Typing A Necessity For Programming? - Abhijit Nadgouda

I originally was going to say something along the lines of "I think it depends on how you view programming."  I was then going to differentiate between whether you just view programming as a job (where you are looking to scrape by with minimum investment), or a skill/craft/art to be cultivated.

But in the process of assembling my ideas, I thought back to how I learned to touch type.  I wanted a job that at the time seemed to pay pretty decent.  However, the job required a minimum typing speed of 40WPM.  I was a 2 finger typist up till that point, which wasn't going to cut it.  Since I wanted the job I put forth some effort.  In about 1 month, using some free typing tutor programs, I learned to touch type sufficiently to pass the test and get the job.  Once on the job further increases in speed and accuracy came.  

The glorious job that prompted this investment of effort was data entry.  It was boring as could be.  Really.  If you have never done data entry yourself it would be hard to imagine the level of boredom involved.  There was no craft or art to it, just glossy eyes and sore wrists.

Now programming is not "just typing".  It's not just entering characters into a buffer.  Believe me, I speak from experience, it is not just data entry.  So a 10x improvement in your typing speed will not equate to a 10x increase in your productivity.  Hopefully between all that typing, programming requires you to stop and think.

Nevertheless, whether you use some auto-complete super-fancy intellisensical snippet-fu brain scanning IDE or a bare bones text editor, you most likely use a keyboard as your central physical tool to transfer your thoughts from your brain into the computer.  If that is the case, even if programming is just your "job", you owe it to yourself to put forth a little bit of effort and learn to touch type. 

It really is worth the effort, and the initial awkwardness.  If you don't know how to touch type you won't understand until you do.  There is something, well, just "cool" about being able to see words appear on the screen as you think them without giving any conscious thought to directing the individual fingers and keystrokes.  

If you are going to use a tool on a daily basis, I can't imagine any reasonable argument against taking the time to learn to use it well.

Knowing Your Languages

You may only speak one language, but how well do you know your tools?  There are likely many similarities between learning a language and learning the tools you use daily.

Today I was thinking about the difference between someone who is just learning a spoken language and someone who knows it well, and how this may at times be seen by their knowledge of it's vocabulary.  For example, imagine someone explaining "I need a tool that you use to hit those little metal things that connect two pieces of wood together.  It has a wooden handle about 12" long and has a metal part on top.  You know?  The tool that people who build houses use."

Do you know what tool they are talking about?  Probably.  When I was learning Spanish, this is the sort of description I would find myself having to use at times. (To all the patient people who put up with me, Thank you.)

Of course, a person who knew the language well could have conveyed the same thought more concisely and clearly: "I need a hammer."

This same idea goes beyond spoken language.  

I use VIM on a daily basis.  Until recently my VIM vocabulary consisted of :wq, dd, i, /, :%s and esc.  Not a vary wide vocabulary at all for such a powerful tool.  I could get most anything done, but it was more like that first long-winded description of a hammer.  I am continuing to work on expanding my VIM vocabulary, and in doing so finding that what previously took many keystrokes can now be done nearly instantly with a single command.

However, the real value of a rich vocabulary is much more than simply the ability to be concise.  A rich vocabulary allows you multiple options to express your thoughts depending on the intended goal.

Instead of simply stating that it was "humid", a story-teller may choose to describe the air as "thick with moisture, soaking one's clothes and sapping one's strength."  Not as concise, but perhaps adding important details.

The same is true with our programming language vocabulary.  The goal of a rich vocabulary in Ruby, Erlang, or (insert your favorite language here)  is not always to write the least lines of code.  Our goal may be to increase maintainability, expose intentions, improve performance, etc.  By improving our vocabulary and grasp of the language, we are able to express our thoughts both efficiently and in the way that best fits the specific situation at hand.

How is your vocabulary with the tools that you use?  What are you doing to improve?

Remove Gems By Prefix

If you are working with Merb or DM you know that approximately a gazillion gems are involved.  When, for whatever reason, I want to remove them it is a pain to do by hand.

Here is a simple script to help:

#!/bin/bash

prefix=$1

gem list --local | grep $prefix  | awk '{ print $1 }' | xargs sudo gem uninstall

I saved mine as remove_gem_by_prefix, gave it executable permissions and placed it in my path.

Now I can issue the commands:

$ remove_gem_by_prefix merb

or 

$ remove_gem_by_prefix dm

And they all get blasted.  You do need to be careful to make sure the prefix you provide is unique to the gems you wish to remove.