I was sent an email from a colleague asking about our testing philosophy and how we would get a developer to take a test driven development approach to writing code.

Here was his question:

Do you guys approach development with TDD/BDD?

If so, do you ever have any issues with developers slipping back into the ‘traditional’ modes of development? i.e. “Who needs these tests.. I’ll fix it if its broken”. Obviously, TDD is a concept where all involved really need to buy in – but it takes discipline. I was wondering if you guys simply run rcov now and again to check for coverage, review code periodically etc..

My response:

Everyone at Less breathes tests. We love them. In my experience the easiest way to make a developer fall in love with testing is to have them write tests on their own code. Assuming they know how to write tests, they will find bugs in their own code. Some mentoring may be required to teach proper testing.

If someone said to me “Who needs these tests. I’ll fix it if its broken”. I would reply, “It is broken right now. You just haven’t found the bugs yet.” If I was asked that question again I would start thinking that maybe that person isn’t such a good fit for Less. If finding their own bugs wasn’t inspiring enough, I would start thinking that maybe that person isn’t such a good fit for Less.

From our perspective a buggy app is not done, so we will keep fixing the bugs until the app is right. If your developer has already used up the budget writing the buggy code, then you, as the owner, are paying for the bug fixes. Writing tests either before, or while the code is being written ends up being faster and cheaper because when you ship you are done. Otherwise you can’t have a proper budget for a project and you are likely loosing out.

We use rcov to see if there is a test we are missing. There usually is one or two so we fill in the blanks. But we rarely get 100% coverage. 100% coverage feels good, but it does not mean the app is well tested, just well covered. Also, rcov only provides C0 coverage, so it can be fooled into reporting something is covered when it is not. (Does anyone know a C1 or C2 tool for Rails?)

Code reviewing is actually very hard to do well. I admit that I am not the best at doing code review, so we don’t use it a lot. Code review is very good for making sure the code is good in general terms. Checking for obvious things, are finders being scoped or is security being applied properly. But reviewing the code diff between revision 737 and 738 may be tricky enough that you approve something that has a new bug in it.

It is easy to open a security hole in your Rails application. Fortunately, by scoping your finders, it is also easy to write your code without opening it.

Here is an example. Let’s say you have a expense tracking application and the url is /expenses/151. Obviously this calls the expenses controller with a params[:id] = 151.

#bad:
def show
  @expense = Expenses.find(params[:id])
end

#good:
#@user is the logged on user.
def show
  @expense = @user.expenses.find(params[:id])
end

The scoped finders actually add the proper where clause to the sql. It happens automatically. Without scoping the expense finder, anyone can see anyone else’s data. Generally you will want to set this up as a before filter.

This also works for nested routes. Let’s say the url is /invoices/25/line_items/87:

class LineItemsController < ApplicationController
  before_filter :setup
#snip many lines

  protected
  def setup
    @invoice = @user.invoices.find(params[:invoice_id]) unless params[:invoice_id].blank?
    @line_item = @invoice.blank? ? @user.line_items.find(params[:id]) : @invoices.line_items.find(params[:id])
  end
end

You don’t have to use the @user variable. In Less Accounting we use sub-domains for each business. Since each business may have several users, all the controllers are scoped around the @business variable, which is determined by the sub-domain of the url. The @business variable itself is scoped by the @user variable.

Let’s say you want to redirect users from the www sub-domain of your website to direct access via the non-sub-domain url. Nginx makes it really easy to do.

Just add this to your server{} block:

if ($host != 'your_domain.com' ) {
    rewrite  ^/(.*)$  http://your_domain.com/$1  permanent;
 }

This actually will redirect any sub-domain to the non-sub-domain url.

But what if, like Less Accounting, your site has user accounts for sub-domains or you have other valid sub-domains, but you still want to get users away from www?

Just add this to your server{} block:

if ($host = 'www.your_domain.com' ) {
    rewrite  ^/(.*)$  http://your_domain.com/$1  permanent;
 }

In my previous article we talked about how to use git and github. In this article we’ll talk about how to make a project that is based on another project.

The scenario: There is an existing open source project that you would like to use as the starting point for your new killer web 2.1 website. You are the bomb. I’m going to use one of my existing open source projects so you can all play along at home.

Note:

1. If you haven’t read the previous article, please go do so now.
2. This assumes that you are using github. (I still have some invites, so if you’d like one, just send me a note).
3. You are using git 1.5.4 or greater on your local machine. (git—version)

Here we go

1. Go to github and create a new repo.
   • Make sure you follow the instructions, doing the local git init through the git push origin master.
   • You now have a fresh repo for you new fancy almost funded web 2.2 website. You are the man.
2. git remote add stevenbristol git://github.com/stevenbristol/lovd-by-less.git
   • Now you have two remotes. One to your origin and one to the lovd-by-less master.
3. git pull stevenbristol master
   • this gets the lovd-by-less source and puts it into your local directory.
4. git push
   • This will push your changes to your master.
   • What!? I didn’t commit anything, what’s going on here? When you did the remote pull, the files are automatically comited.
5. git fetch stevenbristol
   • Git magic is happening right now.
6. git branch stevenbristol stevenbristol/master
   • This creates a tracking branch.
7. git branch
   • See all of your branches.
8. git remote
   • See all of your remotes.
9. git checkout stevenbristol
10. git pull
    • Nothing to pull because there have been no changes since the previous pull (step 3).
    • Notice that we didn’t have to specify which remote to pull from, as we would have if this had been a normal branch created with git branch -b.
11. git checkout master
    • Back to your master.
    • Now you can git (sic) to work building your ultimate about to be tech crunched web 2.3 website. Everyone is jealous of you. And you love it.

Thanks to Josh Owens who keeps answering my git questions and made this post possible.

You all know how to use git, and now there is a Textmate bundle to make git even easier to use. Go here and follow the instructions. That’s all. Enjoy,

Go to http://www.railsenvy.com/ to hear the latest episode of this great podcast. This week I was a guest host.

Passisizzle FTW!

Background:

Last week we released the open source social network Lovd By Less. Almost immediately we had requests for people wanting to add code back into Lovd. Lovd had been housed in a private svn repository and distributed via a zip file. This was really good for me, but not so for people that want to take the current version, build on it and later merge whatever newer version of Lovd is available into their app. Although I hadn’t yet tried git I knew that it made this sort of branching a lot easier than subversion, so I decided to try it.

Getting started with Github:

Github is the new cool git repository website (social network for geeks). I asked a friend for an invite and got started. Creating a github account and repository is really easy. The instructions are well laid out. The last step is to export your svn repo into the git directory and then commit and push. I’ve been told that if anyone needs a git invite to let me know and I’ll be given invites to send out. So let me know.

Now what?

So now I had my git repo setup on github and I started sending out the repo’s public address. And before I know it I had my first patch. Now what? I read Dr. Nic’s blog post about git when it came out, so I went back and read it again. After reading a few times I realized two things: (1) That git is very complicated and (2) Dr. Nic is a way bigger StarWars geek than I ever gave him credit for. I called my friend Josh Owens (of the Web 2.0 Show) and asked for some help. Josh talked me through my first merge and push, but I still didn’t quite get it. The next day I was trying to merge another patch (branch) and since Josh wasn’t around I went into #github where Tom Preston-Werner (mojombo) helped sort me out quite a bit.

Disclaimer:

I am still new at git so I may have things wrong, so if you find an error or omission, please say so in the comments.

Git explained (I think):

Understatement: Git is very different from Subversion. Git is a distributed source control system, which means you can work disconnected from the main repo (branch) and still commit. But you commit to your local repo (branch). The basic flow is (some crucial steps have been left out for now. I’ll fill those in later. Don’t use the following as a step by step guide, that comes later):

1. git clone. This basically tells your local git to go out an pull the files from the server.
2. Make your changes.
3. git commit. This commits your files to your LOCAL repository, not the one on the server.
4. git push. This “pushes” you committed changes up to the server.

OK, so far so good. This feels a lot like svn. So much that it’s time to get overconfident and think we get git. But we don’t. And our confidence is about to be shaken like a dry martini.

But I have my own repo and people are trying to commit to me. What do I do, how does that work?

You are the master:

In git everything revolves around branches (github calls them forks¹). When you create a git repo, that main branch is called “master.” Your master branch is kind of like what trunk is in svn. When someone wants to fork/branch your master, they go to your page in github and click the fork button. Now they have a fork/branch of your master branch. When they are ready for you to check out their changes and merge theirs back into your master they’ll send you a message via git hub. You’ll get this message in your inbox and have no idea what to do with it. Cool.

The first thing to notice is that the url for their branch looks a lot like the url to your master. My “public clone url” is git://github.com/stevenbristol/lovd-by-less.git. Dr. Nic’s looks like this: git://github.com/drnic/lovd-by-less.git. The similarities matter.

To get Dr. Nic’s branch and merge it into my master I do the following (let’s assume I have my master cloned to /lovd):

NOTE: Put the contents of this pastie in the top of your ~/.bash_profile file to see which branch you are currently working in. This is demonstrated in the examples below. The part in parens is the branch. (Another version of the same thing.) (After changing your ~/.bash_profile file you’ll either have to restart terminal or source the file to see the changes.)

1. /luvd(master) $ git remote add drnic git://github.com/drnic/lovd-by-less.git
2. /luvd(master) $ git pull
3. /luvd(master) $ git checkout -b drnic/master
4. /luvd(drnic/master) $ git pull drnic master
5. [look at the files, rake, test, etc]
6. /luvd(drnic/master) $ git status
7. /luvd(drnic/master) $ git checkout master
8. /luvd(master) $ git merge drnic/master
9. /luvd(master) $ git status
10. /luvd(master) $ git commit -a
11. /luvd(master) $ git push

• Step 1 tells my local git repo to add a new remote repository. This means that the same repo can pull and push to two different server². There is nothing analogous in svn. The syntax is “git remote add {name} {url}.” I am naming this remote “drnic,” because I am going to pull his branch.
• Step 2 simply pulls my local master. This is like doing an svn up. I do this because I gave commit rights to someone else and I want to make sure I have the latest changes in my local master. Step 4 also does a pull, but there we have to specify where we are pulling from. Here it defaults to the “origin,” which is my git master on github.
• Step 3 is a bit confusing because I am not doing anything like an svn checkout. A git checkout basically changes the branch I am working with. The -b switch means to create the new branch³. Notice that in steps 1 -3 I was in branch master (master) but after I “git checkout drnic/master” I am in branch drnic/master (drnic/master⁴). Get ready: Rather than have each branch in a different directory on the file system, all the branches live in the same directory. What? Exactly. Try this: Open Textmate and open a file that you know has changed between the two branches. Now go in to terminal and “git checkout {other branch}.” Go back to Textmate and notice that the file did actually change. What? Exactly. Isn’t this cool? But it means you must use the .bash_profile hack to know what branch you’re working in. Other wise you will go crazy. You can also do “git branch” which will tell you what branches are available locally and which one you are in.
• Step 4 just says pull all of the files from the remote repo named “drnic” (which we created in step 1) and the branch, in this case “master.” Syntax: “git pull {name of remote} {name of branch}.” This is kind of like an svn checkout.
• Step 6 shows which files have modifications. If you didn’t change anything then you can go on to step 7, otherwise you might have to git commit to your local branch of drnic’s branch of your master. (Don’t feel bad if you have to reread that a few times :)
• Step 7 puts us back in the context of our master branch.
• Step 8 merges the local drnic branch into our master branch.
• Step 10 commits all unadded files to the local master repo. Unadded? Yes, in git a file with local modifications needs to be added back into the repo for committing. You can do this with “git add {file}” or just git commit all the added and unadded files.
• Step 11 pushes your local changes back to your origin (master branch on github). Now other may fork/branch your master and start again. Easy right?

You just want to patch some else’s repo:

This is really simple. Here are the steps:

1. Go to github and click the “fork” button.
2. git clone git://github.com/stevenbristol/lovd-by-less.git
3. cd lovd-by-less
4. Make your cahnges
5. git status
6. git commit -a
7. git push
8. go back to git hub and click the “pull request” button.

1. Step 1 will create you own fork of the repo where you can make your changes. Click this on the person’s page you want to fork from.
2. Steps 2-7 you should understand by now. (I hope. :)
3. Step 8 will send a message to the person notifing them that you have something for them to see. Click this from your repo page.

References:

• cheat git
• http://drnicwilliams.com/2008/02/03/using-git-within-a-team/
• http://github.com/guides/
• http://wiki.sourcemage.org/Git_Guide
• http://git.or.cz/gitwiki/QuickStart

Notes from Dr. Nic:

¹ Technical not correct. “Within a repository/clone you can have branches. A fork is a clone of a repository. Its a conceptual thing.” He continues, “I could manually create a fork by cloning 1+ branches from a target repository, then push that repository to my own remote repository. At this stage, I’ve theoretically got a remote clone of your remote repo. “Forking” is a nice word for this.” This is how someone can take Lovd, put it in their own private repo, build their app on top of Lovd, and easily get the latest patches from Lovd into their new killer app.

² “A “remote” repo and a “local” repo contain the same information about the commits etc. The difference is that a “local” repo has a checkout of one branch plus a .git folder containing all the commit info. The remote repo just contains the commit etc information, but no checkout (see “git checkout—bare” I think creates a remote repo that you could host on from your laptop’s ~/Sites folder for example.”

³ ””git checkout -b drnic/master” – I tend to use “local/drnic/master” now – using local/ as a namespace to separate it out from any clashes with the remote repo. I think this is different from my original blog post.”

⁴ “The -b switch means to create the new branch, based on the current branch.”

Chris Wanstrath wrote a nice little post about a method he created called try(), I thought this was pretty cool, but I really want to be able to specify the return value if the object is nil. Plus, what if I want to use this sweetness on a method? So I wrote two methods to do just that:

class Object
  def if_nil out = nil
    return out if nil?
    self
  end

  def if_method_nil method, out = nil
    return out if nil?
    return send(method) if out.nil?
    return out if respond_to?(method) && send(method).nil?
    send method
  end
end

  def test_if_nil1
    n = nil
    assert_equal nil, n.if_nil
  end

  def test_if_nil2
    n = 1
    assert_equal 1, n.if_nil
  end

  def test_if_nil3
    n = :yo
    assert_equal :yo, n.if_nil
  end

  def test_if_nil4
    n = nil
    assert_equal 'blah', n.if_nil('blah')
  end

  def test_if_method_nil1
    n = nil
    assert_equal nil, n.if_method_nil(:to_s)
  end

  def test_if_method_nil2
    n = 1
    assert_raise NoMethodError do
      n.if_method_nil :yo
    end
  end

  def test_if_method_nil3
    n = 1
    assert_nothing_raised do
      assert_equal '1', n.if_method_nil( :to_s)
    end
  end

  def test_if_method_nil4
    n = 1
    assert_nothing_raised do
      assert_equal '1', n.if_method_nil( :to_s, 'blah')
    end
  end

  def test_if_method_nil5
    n = nil
    assert_nothing_raised do
      assert_equal 'blah', n.if_method_nil( :to_s, 'blah')
    end
  end

A good friend of mine recently asked me to look at his open source project and tell me what I think. While looking through the very nice code I discovered a security hole and promptly created a user account with administrative privileges using one command.

Here is the command:

curl -d “user[login]=hacked&user[is_admin]=true&user[password]=password&user[password_confirmation]=password&user[email]=hacked@by.me” http://url_not_shown/users

That’s right, that’s all it took. Try it. Take this line and point it at your favorite website and see if you can create an admin account.

There are a few easy things anyone can do to prevent this hack. In order of importance:

1. Use attr_protected.
2. Don’t use mass assignments for your users table.
3. Don’t have a users controller.
4. Split the users table into a users table and a profiles/people table.

At Less we do all four.

Use attr_protected

The attr_protected method in Rails will prevent the fields from being assigned via mass assignment. Here is an example:

class User < ActiveRecord::Base
#no attr_protected here
#this will allow the creation of your hacked admin user
end

class Users < ApplicationController
  def create
      @user = User.create params[:user]
  end
end

#this will not allow the creation of your hacked admin user
class User < ActiveRecord::Base
  attr_protected :is_admin
end

class Users < ApplicationController
  def create
      @user = User.create params[:user]
#user is created, but the is_admin flag is not set
  end
end

Wasn’t that easy? You should all do this.

Don’t use mass assignments for your users table

No where in your code should you allow the users table to use mass assignments. What this means is that when a user account is created, assign each field individually. Here is an example:

class Users < ApplicationController
  def create
      @user = User.create params[:user]
  end
end

class Users < ApplicationController
  def create
      @user = User.new
      @user.login = params[:user][:login]
      @user.password = params[:user][:password]
      @user.password_confirmation = params[:user][:password_confirmation]
      @user.save
#user is created, but the is_admin flag is not set
  end
end

I know what you’re thinking “Steve, that is too many lines of code, can’t we use the built in stuff that Rails gives us to make thing easy?” My wife’s cousin is an Assistant Warden at a prison in Texas. While taking a tour I noticed a sign that says “Security is not convenient.” This is also true for security code. It is more lines of code, it is harder to write, it should fail securely.

If you need to have an admin screen where admins set is_admin on users, then that action should also not use mass assignment and it should be protected so that only users who are already is_admin can access it.

Don’t have a users controller

This one is stupid. Obfuscation is the worst form of security, but combined with the others it is not too bad and worth doing. This will prevent all the script kiddies I just created from attacking your site because they will not know the name of the controller to hit. Of course if they just looked at the url that your signup form submits to they will know. That is why this is the weakest form of security. In fact it’s so weak it’s not really even security.

Split the users table into a users table and a profiles/people table

Keep user settings that the system needs in the untouchable users table and user configurable settings in a separate table called profiles or people or something. Doing this means the only place the users table gets written to is on account creation and if you have an admin screen. The fewer access points you have, the safer you are.

It also means that for profile data, you can use your profiles controller which allows mass assignment and keep the code nice and tight.

Why are you still reading? Go, fix your code right now. My spider is already loose, looking for your rails app.

Go fix it already!!

IE6 PNG Transparency Fix for Repeating Background Images

<!--[if IE 6]>
<style>
#IE_blows {
height: 100%; 
filter: progid:DXImageTransform.Microsoft.AlphaImageLoader(src='/images/transparent_bg.png', sizingMethod='scale');}
</style>
<![endif]-->

/* put in normal css sheet */
#IE_blows { 
  background: url(/images/transparent_bg.png) repeat-y;
}
/* put in normal css sheet */

IE6 PNG Transparency Fix for Repeating Background Images

<!--[if IE 6]>
<style>
#IE_blows {
height: 100%; 
filter: progid:DXImageTransform.Microsoft.AlphaImageLoader(src='/images/transparent_bg.png', sizingMethod='scale');}
</style>
<![endif]-->

/* put in normal css sheet */
#IE_blows { 
  background: url(/images/transparent_bg.png) repeat-y;
}
/* put in normal css sheet */

There are a lot of people who are really good programmers who never reach that “top .1%,” or whatever, level. Of late I have been thinking of one of the things that sets the best programmers apart: Discernment. I’m not talking about discerning a good technology from a bad one, but rather the ability to discern a good choice from a bad one. To be able to see the consequences of this path versus that one. In short, to know which rabbits to chase.

When writing software we face a million billion choices every day. Having the talent to make good choices is crucial to being productive. It’s the reason we can launch a product in seven hours.

Here is an example: Recently, one of my guys was refactoring some bad controller code we inherited. He was very excited to show me how he had abstracted and cleaned some of the bad controller code into a library so all of the controllers could call these methods instead of duplicating this bad code all over the place. At first glance it was clear to me that this abstraction was the beginnings of a new library that would be almost identical to Make Resourceful. Once I showed him where his path was leading he agreed that this was probably not the right time to write a Make Resourceful clone, especially when what was really called for was just making the bad code proper, not writing a support structure for it. This reminds me of what Malcom Gladwell was talking about in Blink: The ability to know instantly. (Perhaps to know truth instantly.)

I’m not talking about syntax here. Obie Fernandez and I have fairly different coding styles. He prefers to have the code be more explicit and readable while I prefer my code to be more terse and succinct. Neither is right or wrong, and neither is the reason we are both able to look at a problem and immediately prune it to its core and see clearly where the different paths take us.

This isn’t just true of code, I have seen Allan do the same thing with design. We were working with a designer who was very headstrong and really thought Allan was lacking in something (I’ll save the “why were you working with someone like that” story for another time). This person was doing the html/css for a website and using some popular grid-css-template-thing (I believe that is the technical term for it). When Allan first looked at the code, he just started ripping parts out, not because it was bad in and of itself, but because it was clear that was way too complicated. Using this framework did not buy the simplicity and maintainability of what has come to be known as “The Allan Way™.” (It is amazing to watch Allan take a 7000 line css file and without changing the way the site looks, trim the file to 300 lines.) The other designer made a fuss until Allan was done. Once he saw the difference, he was converted to “The Allan Way™.”

It’s easy to say, “that is just a product of experience.” And although experience certainly plays a part, I think it goes way beyond that. I think this discernment might also be called wisdom. Wisdom basically boils down to how well can you predict the future. When one approaches the wise old sage, the sage immediately knows the outcome of all the paths presented.

So how does one get this discernment/wisdom? I am not sure. I do believe one can learn it, and I think one of the requirements is to know that that is what one is trying to learn. When you play chess, do you only think about your strategy, or do you constantly think “why did my opponent make that move?” Acquiring wisdom might require applying that mental discipline to every part of life: Why did my wife/boss/client/adversary say that? Was it planned or just careless? How does my action affect others? And not just in dealing with others: If I do this, what will I do next? And then what? What paths am I forcing myself into later, if I make this choice now? How deep is this rabbit hole?

Maybe this is the difference between success and mediocrity, between satisfaction and melancholy. Or maybe not.

We have been going back and forth on Haml lately. There is no doubt that it is nice, it is short, tight and does a lot for you. At the root of haml is yaml, a format for storing data. In yaml (and consequently in haml) indenting is very important, and for this reason haml and javascript go together like oil and grape jelly.

To haml, javascript is just plain text, which means to make haml happy all of your lines of javascript code must allign vertically. Here is an example:

<script type="text/javascript" charset="utf-8">
//<![CDATA[
jQuery(function(){
  tog("#forgot_password_clicker", "#login_form");
  tog("#forgot_password_clicker", "#forgot_form", forgot_text);
});
function forgot_text(){
  if (jQuery("#forgot_text").html() == "forgot")
    jQuery("#forgot_text").html("remember");
  else
    jQuery("#forgot_text").html("forgot");
}
//]]>
</script>

%script{:type => "text/javascript", :charset => "utf-8"}
  //<![CDATA[
  jQuery(function(){
  tog("#forgot_password_clicker", "#login_form");
  tog("#forgot_password_clicker", "#forgot_form", forgot_text);
  });
  function forgot_text(){
  if (jQuery.trim(jQuery("#forgot_text").html()) == "forgot")
  jQuery("#forgot_text").html("remember");
  else
  jQuery("#forgot_text").html("forgot");
  }
  //]]>

1. Javascript becomes much harder to read and write.
2. Textmate does not do syntax highlighting on the javascript.
3. The haml requires the jQuery.trim() method in the first line of forgot_text() function because with haml you need to add a bunch of white space:
4. If you’re going to output ruby variables into your javascript, it only gets worse.

<span>I <span id="forgot_text">forgot</span> my 
  <a href="javascript:void(0)" id="forgot_password_clicker">password or username</a>
</span>

%span 
      I 
      %span#forgot_text
        forgot
      my 
      %a{:href => "javascript:void(0)", :id => "forgot_password_clicker"} password or username 

1. One could use :plain and then indent javascript.
   • The problem is doing that means one can’t render ruby variables to the javascript.
2. Write unobtrusive javascript.
   • Nice, but I really like to cheat.
3. Put all of your javascript in partials
   • Sorry, I just threw up in my mouth a little bit.
4. Patch haml.
   • Let me know when you do this.
5. Stop writing javascript.
   • I also think you should start using frames again.
6. If anyone knows of any other way around this, please let me know. We’d love to use haml, but not at this price.
7. Don’t use haml.
   • Our choice for today. Although we will review this regularly.

For those of you who do not use Cheat Sheets, you should.

For those of you that do, they just got better. There is now an auto complete for bash. You can find it here

sheets=`cheat sheets | grep '^  '`
function complete_cheat {
  COMPREPLY=()
  if [ $COMP_CWORD = 1 ]; then
    COMPREPLY=(`compgen -W "$sheets" -- $2`)
  fi
}
complete -F complete_cheat cheat

Here are the instructions:

1. Open terminal.
2. cd ~
3. Copy this text into your .bash_profile file.
4. Either restart your terminal or run source .bash_profile
5. Type cheat str(TAB)
6. Cool huh?