Archive for the ‘Ruby’ Category

Show Intent with Better Naming

March 8, 2011

I had an interesting experience at a code retreat with the creator, Corey Haines. I created some code that I felt was really perfect. I didn’t think there was room for much improvement, but it only took Corey a few seconds in passing to find a flaw. It starts with this list of rules for simple design:

  1. Passes tests – the code should be test-driven, and the tests should all pass.
  2. No duplication – often known as DRY – don’t repeat yourself. Every distinct piece of information in the system should have one (and only one) representation in the code.
  3. Expresses intent – the code should be self-explanatory.
  4. Small – methods, classes, indeed the entire application shouldn’t be any bigger than absolutely necessary.

My Original Version

I won’t explain what this code is supposed to do. That might defeat the point. See if you can figure out which principal I violated with this code. I’ll say it’s not Rule #1, but showing the tests would take up too much room.

def new_status current_status, neighbor_count
  return :alive if neighbor_count == 3
  return current_status if neighbor_count == 2
 
  :dead
end

The Problem

Corey asked me one question: what if one of the requirements changes? And there it was. In an attempt to do the most in the fewest lines, I'd over-refactored the method. Not only had I made the method brittle if business requirements should change in the future, I'd factored out the intent of the method itself.

As usual, one good software practice begets another. Test-driven development results in smaller, simpler methods for instance. And in this case, showing intent in your code reduces brittleness. So how do you accomplish this?

The Solution

Express the problem domain in the code itself. Here's my example, reworked:

def new_status current_status, neighbor_count
  return :dead if overpopulated?(neighbor_count)
      || underpopulated?(neighbor_count)

  return :alive if population_perfect?(neighbor_count)
 
  current_status
end
 
def overpopulated? neighbor_count
  neighbor_count > 3
end
 
def underpopulated? neighbor_count
  neighbor_count < 2
end
 
def population_perfect? neighbor_count
  neighbor_count == 3
end

This code is longer, but it shows intent much more clearly. You don't even need to know what "overpopulated" is to understand what the method is doing. But if you want to know, or need to change it, it's easy. In fact, we're passing neighbor_count around a lot, so it looks like it's time to abstract this into a class:

class Cell
  def initialize current_status, neighbor_count
    @current_status = current_status
    @neighbor_count = neighbor_count
  end
 
  def next_status
    return :dead if overpopulated? || underpopulated?
    return :alive if population_perfect?
 
    @current_status
  end
 
  private
 
  def overpopulated?
    @neighbor_count > 3  
  end
 
  def underpopulated?
    @neighbor_count < 2
  end
 
  def population_perfect?
    @neighbor_count == 3
  end
end

Now the code is much more readable, and understandable. We're now clearly showing intent. And now, just for fun, the tests:

class CellTest << Test::Unit::TestCase
  def test_should_die_when_alive_and_overpopulated
    cell = Cell.new :alive, 4
    assert_equal :dead, cell.next_status
  end
 
  def test_should_die_when_alive_and_underpopulated
    cell = Cell.new :alive, 1
    assert_equal :dead, cell.next_status
  end
 
  def test_should_live_when_alive_and_perfect_population
    cell = Cell.new :alive, 3
    assert_equal :alive, cell.next_status
  end
 
  def test_should_stay_alive_by_default_when_alive
    cell = Cell.new :alive, 2
    assert_equal :alive, cell.next_status
  end
 
  def test_should_die_when_dead_and_overpopulated
    cell = Cell.new :dead, 4
    assert_equal :dead, cell.next_status
  end
 
  def test_should_die_when_dead_and_underpopulated
    cell = Cell.new :dead, 1
    assert_equal :dead, cell.next_status
  end
 
  def test_should_live_when_dead_and_perfect_population
    cell = Cell.new :dead, 3
    assert_equal :alive, cell.next_status
  end
 
  def test_should_stay_dead_by_default
    cell = Cell.new :dead, 2
    assert_equal :dead, cell.next_status
  end
end

Code Retreat in Boulder, Colorado

February 26, 2011

I’m in beautiful downtown Boulder, getting ready to attend a code retreat with Ruby greats like Corey Haines, Chad Fowler, Dave Thomas*, Mike Clark, Michael Feathers and many more.

Last night I hung out with some KC friends and we setup our dev environments for the event. I got motivated, and created a base environment on GitHub you can download. It runs your tests automatically using Watchr every time you save your code file, and if you’re on a mac it even takes a screen shot at each save! Now you can go back and relive the magic. Maybe string them together into a video with a little commentary, and boom – easy post-retreat blog video.

Use the link above, and let me kno w if it was useful!

*not the Wendy’s guy, as my wife likes to ask. You’d think since the world is down to just one living, notable Dave Thomas that joke would get a little old. I think the people who grew up watching Wendy’s commercials will also have to die out first :)

Double-Blind Test-Driven Development in Rails 3: Part 3

February 2, 2011

  1. Simple Tests
  2. Double-Blind Tests
  3. Making it Practical with RSpec Matchers

This is the last article in this series describing the concept of double-blind test-driven development. This style of testing can add time to development, but this can be cut significantly using RSpec matchers.

If you’re not familiar with matchers, they’re the helpers that give RSpec its english-like syntax, and they can be a powerful tool speeding up all of your test-driven development – whether you follow the double-blind method or not.

If you’re using RSpec, you’re already using their built-in matchers. Say we have a Site model, and its url method takes the host attribute and appends the ‘http://&#8217; protocol. Here’s a likely test:

describe Site, 'url'
  it "should begin with http://" do
    site = Site.new :host => 'example.com'
    site.url.should equal('http://example.com')
  end
end

The equal() method in the code above is the matcher. You can pass it to any of RSpec's should or should_not methods, and it will magically work.

But the magic isn't that hard, and you can harness it yourself for custom matchers that conform to your application.

The Many Faces of Custom RSpec Matchers

While I don't want this article to turn into a primer on custom RSpec matchers (it's a little off-topic), I'll give you the three styles of defining them, and explain my recommendations. There are simple matchers, the Matcher DSL, and full RSpec matcher classes.

Let's start by writing a test we want to run:

it "should be at least 5" do
  6.should be_at_least(5)
end

This test should always pass, provided we've defined our matcher correctly. The first way to do this is the simple matcher:

def be_at_least(minimum)
  simple_matcher("at least #{minimum}"){|actual| actual >= minimum}
end

As you might guess, actual represents the object that ".should" whatever - in this case ".should be_at_least(5)". This version makes a lot of assumptions, including the auto-creation of generic pass and fail messages.

If you want a little more control, you can step up to RSpec's Matcher DSL. This is the middle-of-the-road option for creating custom matchers:

RSpec::Matchers.define :be_at_least do |minimum|
  match do |actual|
    actual >= minimum
  end

  failure_message_for_should do |actual|
    "expected #{actual} to be at least #{minimum}"
  end

  failure_message_for_should_not do |actual|
    "expected #{actual} to be less than #{minimum}"
  end

  description do
    "be at least #{minimum}"
  end
end

Now we're rocking custom failure messages, and test names. This is pretty cool, and honestly how I started out doing matchers. It's also how I started out doing the matchers for double-blind testing.

The problem is that by skipping the creation of actual matcher classes, we lose the ability to do things like inheritance. Not a big deal if our matchers stay simple, but they won't. Not if we use them as often as we should! I found myself re-defining the same helper methods in each matcher I defined this way.

So let's see just how daunting a full-fledged custom matcher class really is:

module CustomMatcher  
  class BeAtLeast
    def initialize(minimum)  
      @minimum = minimum
    end  
  
    def matches?(actual)  
      @actual = actual
      @actual >= @minimum
    end  
  
    def failure_message_for_should  
      "expected #{@actual} to be at least #{@minimum}"  
    end  
  
    def failure_message_for_should_not  
      "expected #{@actual} to be less than #{@minimum}"  
    end  
  end  
  
  def be_at_least(expected)  
    BeAtLeast.new(expected)  
  end  
end  

This isn't so bad! We're defining a new class, but you can see it doesn't have to inherit from anything, or use any unholy Ruby voodoo to work.

We just have to define four methods: initialize, match? (which returns true or false), and the two failure message methods. Along the way, we set some instance variables so we can access the data when we need it. Finally, we define a method that creates a new instance of this class, and that's what RSpec will rely on.

You can add as many other methods as these four will rely on. But you also get other benefits over the DSL. You can use inheritance, moving common methods up the chain so you only have to define them once, instead of in each matcher definition. You can also write setup/teardown code in your parent classes, make default arguments a breeze, and standardize any error handling. I do all of these in the matchers I created for the example app.

The bottom line is this: defining your own matcher classes directly really DRY's up your matchers, and that always makes life simpler. I think it's the only way to go for serious and heavy RSpec users. It allows the class for my validate_presence_of matcher to be this short and sweet:

module DoubleBlindMatchers
  class ValidatePresenceOf < ValidationMatcher
    def default_options
      {:message => "can't be blank", :with => 'x'}
    end

    def match
      set_to @options[:with]
      @object.valid?
      check !@object.errors[@attribute].include?(@options[:message]), shouldnt_exist
      
      set_to nil
      check !@object.valid?, valid_when('nil')
      check @object.errors[@attribute].include?(@options[:message])
      
      set_to ""
      check !@object.valid?, valid_when("blank")
      check @object.errors[@attribute].include?(@options[:message])
    end
  end
  
  def validate_presence_of expected, options = {}
    ValidatePresenceOf.new expected, options
  end
end

And the Teacher model, which grew considerably during our double-blind testing, now looks like this (in its entirety):

# spec/models/teacher_spec.rb

require 'spec_helper'

describe Teacher do
  it {should have_many :subjects}
  
  it {should validate_presence_of :name}
  it {should validate_length_of :name, :maximum => 50, :message => "must be 50 characters or less"}
  
  it {should validate_presence_of :salary}
  it {should validate_numericality_of :salary, :within => (20_000..100_000), :message => "must be between $20K and $100K"}
end

Summary

Now that you've seen my entire proposal for double-blind testing, let me know what you think. Be cruel if you must, it's the only way I'll learn. I'll do the best to explain (not defend) my reasoning, and keep an open mind to changes.

I'll also be publishing my double-blind matchers as a gem so you can add them to your project.

Double-Blind Test-Driven Development in Rails 3: Part 2

February 1, 2011

  1. Simple Tests
  2. Double-Blind Tests
  3. Making it Practical with RSpec Matchers

The last article in this series defined the concept of double-blind test-driven development, but didn’t get much into real-world examples. In this article, we’ll explore several such examples.

The Example Application

This article includes a sample app that you can download using the link above. Be sure to checkout tag “double_blind_tests” to see the code as it appears in this article. The next article will have a lot of refactoring. I limited my samples to the model layer, where 100% coverage is a very realistic goal, and this is likely to be the greatest benefit.

I chose a simple high school scheduling app with teachers, the subjects they teach, students, and courses. In this case, I’m defining a course as a student’s participation in a subject. Teachers teach (ie, have) many subjects. Students take (have) many subjects, via courses. The course record contains that student’s grade for the given subject.

The database constraints are intentionally strict, and most of the validations in the models ensure that these constraints are respected in the application layer. We don’t want the user seeing an error page because of bad data. Depending on the application, that can be worse than actually having bad data creep in.

Associations

Here’s an example of a has_many association:

# excerpt from spec/models/teacher_spec.rb

describe Teacher do
  it "has many subjects" do
    teacher = Factory.create :teacher
    teacher.subjects.should be_empty

    subject = teacher.subjects.create Factory.attributes_for(:subject)
    teacher.subjects.should include(subject)
  end
end

In order to factor out our own assumptions, we have to ask what they are. The assumption is that the subject we add to the teacher's subject list works because of the has_many relationship. So we'll first test that teacher.subjects is, in fact, empty when we assume it would be. Then we're free to test that adding a subject works as we expect.

Here's a belongs_to association:

# excerpt from spec/models/subject_spec.rb

describe Subject do
  it "belongs_to a teacher" do
    teacher = Factory.create :teacher

    subject = Subject.new
    subject.teacher.should be_nil
    
    subject.teacher = teacher
    subject.teacher.should == teacher
  end
end

Again, we're challenging the assumption that the association is nil by default, by testing against it before verifying that we can add a teacher. This tests that this is a true belongs_to association, and not simply an instance method. This is the kind of thing that can and will change over the life of an application.

Validations

Let's test validates_presence_of:

# excerpt from spec/models/teacher_spec.rb

describe Teacher do
  describe "name" do
    it "is present" do
      error_message = "can't be blank"
      
      teacher = Teacher.new :name => 'Joe Example'
      teacher.valid?
      teacher.errors[:name].should_not include(error_message)

      teacher.name = nil
      teacher.should_not be_valid
      teacher.errors[:name].should include(error_message)

      teacher.name = ''
      teacher.should_not be_valid
      teacher.errors[:name].should include(error_message)
    end
  end
end

This example was actually explained in detail in the last article. Validate that the error doesn't already exist before trying to trigger it. Don't just test the default value when you create a blank object, test the likely possibilities. Refactor the error message to DRY up the test and add readability. And finally, test by modifying the object you already created (as little as possible) rather than creating a new object from scratch for each part of the test.

A more complex version is needed to validate the presence of an association:

# excerpt from spec/models/subject_spec.rb

describe Subject do
  describe "teacher" do
    it "is present" do
      error_message = "can't be blank"

      teacher = Factory.create(:teacher)
      subject = Factory.create(:subject, :teacher => teacher)
      subject.valid?
      subject.errors[:teacher].should_not include(error_message)
    
      subject.teacher = nil
      subject.should_not be_valid
      subject.errors[:teacher].should include(error_message)
    end
  end
end

While the test is more complex, the code to satisfy it is not:

# excerpt from app/models/subject.rb

validates_presence_of :teacher

testing validates_length_of:

# excerpt from spec/models/teacher_spec.rb

describe Teacher do
  describe "name" do
    it "is at most 50 characters" do
      error_message = "must be 50 characters or less"
      
      teacher = Teacher.new :name => 'x' * 50
      teacher.valid?
      teacher.errors[:name].should_not include(error_message)
      
      teacher.name += 'x'
      teacher.should_not be_valid
      teacher.errors[:name].should include(error_message)
    end
  end
end

And here's the model code that satisfies the test:

# excerpt from app/models/teacher.rb

validates_length_of :name, :maximum => 50, :message => "must be 50 characters or less"

While you can definitely start to see a pattern in validation testing, this introduces a new element. Instead of freshly setting the name attribute to be 51 characters long, we test the valid edge case first and then add *just* enough to make it invalid - one more character.

This does two things: it verifies that our edge case was as "edgy" as it could be, and it makes our test less brittle. If we wanted to change the test to allow up to 100 characters, we'd only have to modify the test name and the initial set value.

validating a number's range using validates_numericality_of:

# excerpt from spec/models/teacher_spec.rb

describe Teacher do
  describe "salary" do
    it "is at or above $20K" do
      error_message = "must be between $20K and $100K"
      
      teacher = Teacher.new :salary => 20_000
      teacher.valid?
      teacher.errors[:salary].should_not include(error_message)

      teacher.salary -= 0.01
      teacher.should_not be_valid
      teacher.errors[:salary].should include(error_message)
    end

    it "is no more than $100K" do
      error_message = "must be between $20K and $100K"

      teacher = Teacher.new :salary => 100_000
      teacher.valid?
      teacher.errors[:salary].should_not include(error_message)
      
      teacher.salary += 0.01
      teacher.should_not be_valid
      teacher.errors[:salary].should include(error_message)
    end
  end
end

And here's the code that satisfies the test:

# excerpt from app/models/teacher.rb

validates_numericality_of :salary, :message => "must be between $20K and $100K",
  :greater_than_or_equal_to => 20_000, :less_than_or_equal_to => 100_000

We're doing the same here as in our testing of name's length. We're setting the edge value that's *just* within the allowed range, then adding or subtracting a penny to make it invalid. I split up the top and bottom edge tests, because it's better to test as atomically as possible - one limit per test.

Defaults

Another tricky database constraint to test for is a default value:

# excerpt from spec/models/course_spec.rb

describe Course do
  describe "grade_percentage" do
    it "defaults to 1.0" do
      course = Course.new :grade_percentage => nil
      course.grade_percentage.should be_nil
      
      course = Course.new :grade_percentage => ''
      course.grade_percentage.should be_blank
      
      course = Course.new :grade_percentage => 0.95
      course.grade_percentage.should == 0.95
      
      course = Course.new
      course.grade_percentage.should == 1.0
    end
  end
end

In this case, we can't avoid having to recreate the model from scratch, because the nature of the implementation. There's no actual code in the model that makes this happen, it's purely in the database schema. Why should we test it, then? Because we test any behavior we're going to rely on in the application. The fact that this model behavior is implemented at the database level (and therefore, not purely TDD) is a small inconvenience.

What's the assumption our double-blind test is verifying in this case? That the value is only set in the absence of other values being explicitly assigned. Testing with nil and blank values verifies that the default doesn't override them - it only works in the complete absence of any assignment. I also test an arbitrary (but valid) value as the anti-assumption test before finally verifying that the default is setting to the correct value.

Most default tests verify only that the correct default value is set - the double-blind version verifies that it's acting only as a default value in all cases.

Summary

The point of double-blind testing is bullet-proof tests, that can't be reasonably thwarted by antagonistic coding - whether that's your anti-social pairing partner, or yourself several months down the road. The bottom line is this: test all assumptions.

That being said, this is very time consuming, and we can see a ton of repetition even in this small test suite. What we need is a way to get back to speedy testing before our boss/client notices it now takes an hour to implement one validation.*

*Even if you work for a government owned/regulated institution that actually digs that kind of non-agile perversion, you WILL eventually go insane. Even in this small sample app, the voices in my head had to talk me off a building ledge twice.

The answer lies in RSpec matchers, which are easy to implement, and can grow with your application. The benefit is not just speedier development - it's also consistency across your application. We'll explore that in the last article of this series.

Double-Blind Test-Driven Development in Rails 3: Part 1

January 31, 2011

This is a three-part series introducing the concept of double-blind test-driven development in Rails. This post defines the concept itself, and lays the groundwork by showing the way tests are more commonly written. The next couple posts will show how to double-blind test various common rails elements, and how to make this added layer of protection automatic and quick.

  1. Simple Tests
  2. Double-Blind Tests
  3. Making it Practical with RSpec Matchers

Looking at a rails application that was built with test-driven development, you might expect to see something like this:

# spec/models/teacher_spec.rb

describe Teacher do
  it "has many subjects" do
    teacher = Factory.create :teacher
    subject = teacher.subjects.create Factory.attributes_for(:subject)

    teacher.subjects.should include(subject)
  end
  
  describe "name" do
    it "is present" do
      teacher = Teacher.new

      teacher.should_not be_valid
      teacher.errors[:name].should include("can't be blank")
    end
    
    it "is at most 50 characters" do
      teacher = Teacher.new :name => 'x' * 51
      
      teacher.should_not be_valid
      teacher.errors[:name].should include("must be 50 characters or less")
    end
  end
end

Truth be told, if you're seeing this in the wild the app is probably doing pretty good. This level of testing works great during the early stages of an app, when things are simple. But as things grow and/or multiple developers become involved, you need more.

Consider models where the associations and validations stretch into the dozens of lines. The more careful and specific you are about validations, the easier it is to get conflicting or overlapping validations. I actually came up with the concept of double-blind testing while retro-testing models in a client app that previously had no validation specs.

What is Double-Blind Testing?

In the world of scientific studies, you always need a control group. One set of participants gets the latest and greatest new diet pill, while the other gets a placebo. Researchers used to think this was good enough, and probably pretty funny to watch the placebo users rave about their shrinking waistlines. But it turns out studies like this still allowed some bias - as researchers observed the effects, their *own* preconceived notions tainted results. Enter the double-blind study.

In a double-blind study, the researchers themselves are unaware of which participants are in the control group, and which are being tested. Both sides are "blind". They may have lost funny patient anecdotes, but they gained research reliability.

Applying the Lessons of Double-Blind Studies to Test-Driven Development

As I said, in the early stages of an app the tests I showed above work great, as long as you're using TDD and the red-green-refactor cycle. This means you write the test, run it, and it fails. Then you write the simplest code that will make the test pass, run the test again, and confirm that it passes. Most testing tools will literally show red or green as you do this. Then, as you start to amass tests, you're free to refactor your code (abstracting common code into helper methods, changing for readability, etc) and run the tests again at any time. You will see failures if you broke anything. If not, you've more or less guaranteed your code refactoring works properly.

The problem comes in when you start changing old code, or adding tests to processes that didn't initially happen. What I'm calling double-blind testing is this:

each test needs to verify the object's behavior before testing what changes.

As an example, let's rewrite one of the tests from above:

# original test

describe "name" do
  it "is present" do
    teacher = Teacher.new

    teacher.should_not be_valid
    teacher.errors[:name].should include("can't be blank")
  end
end
# modified to be double-blind

describe "name" do
  it "is present" do
    error_message = "can't be blank"

    teacher = Teacher.new :name => 'Joe Example'
    teacher.valid?
    teacher.errors[:name].should_not include(error_message)

    teacher.name = nil
    teacher.should_not be_valid
    teacher.errors[:name].should include(error_message)

    teacher.name = ""
    teacher.should_not be_valid
    teacher.errors[:name].should include(error_message)
  end
end

This is the basic pattern for all double-blind testing. We're not leaving anything to chance. In the original version, we expected our object to be invalid, we treated it as such, and we got the result we expected. Do you see the problem with this?

Here's an exercise: can you make the original test pass, even though the object validation is not working correctly? There's actually a style of pair programming that routinely does exactly this. One developer writes the test, and the other writes just enough code to make it pass, with the good-natured intention of tripping up the first developer whenever possible. If you wrote the original test, I could satisfy it by just adding the error message to every record on validation, regardless of whether it's true! Your test would pass, but the app would fail.

The test is now "double-blind" in the sense that we as testers have factored out our own expectations from the test. In this case, we expect the error message to not be there until we initialize the object a certain way, and this can be bad. It may sound far-fetched or paranoid*, but in large codebases your original tests are often abused in this very way. The "you" that writes new code today is often at odds with the "you" from three months ago that wrote the older code with a different understanding of the problem at hand.

*Plus, everybody knows it's not paranoia when the world really is out to get you. I've discussed this at length with the voices in my head, and they all agree. Except Javier. That guy's a jerk.

Now that I've laid out the justification, let's take a closer look at how the test changed. The first thing I did was create a version of the object that I believe should NOT trigger the error message. Then I run through two cases that should. You can see right away, I was forced to be more *specific* about what should trigger an error. Instead of just a blank object with no values set, I've proactively set the attribute in question to both nil and blank. A key element here is to try to work with the *same* object, modifying between tests, rather than creating a new object each time. My test wouldn't have been as specific if I'd just recreated a blank Teacher object and run a single validation check.

Also, with the increased code comes the increased chance of typos. We don't want to DRY test code up too much, because a good rule is to keep your tests are readable (non-abstract) as possible. But I've specified the error message at the top of the test, and reused that string over and over. I did this in a way that DRY's the code and adds readability. You can see at a glance that all three tests are checking for the same error.

Finally, the first time I run the object's validation, notice I'm not asserting that it should be valid. If I had written teacher.should be_valid on line 8 of the double-blind test, I'd have to take the extra time to make sure every other part of the object was valid. Not only is this time-consuming, it's very brittle. Any future validations would break this test.

If you use factories often, you may suggest setting it up that way since a factory-generated object should always be valid. Then you could assert validity. However, this only slows down your test suite. it's enough just to run valid? on the object, which triggers all the validation checks to load up our errors hash.

Summary

I believe this is a new concept - I was already coding most of my tests this way, but it didn't dawn on me how valuable it was until I started retro-testing previously testless code. The value showed itself right away.

I would love to hear feedback on this - if you think it's unnecessary (I tend to be very rainman-ish about my testing code) or even detrimental. However, if you think it's too much work, I ask you to hold your criticism until you've read part 3 of this article, where I show how to use your own RSpec matchers to greatly speed this process.

Legacy Database Column Names in Rails 3

January 28, 2011

If you work with legacy databases, you don’t always have the option of changing column names when something conflicts with Ruby or Rails. A very common example is having a column named “class” in one of your tables. Rails *really* doesn’t like this, and like the wife or girlfriend who really hates your new haircut, it will complain at every possible opportunity:

# trying to set the poorly named attribute
ruby-1.9.2-p0 > u = User.new :class => '1995'
NoMethodError: undefined method `columns_hash' for nil:NilClass
# trying to set a different attribute that is only guilty by association
ruby-1.9.2-p0 > u = User.new :name
NoMethodError: undefined method `has_key?' for nil:NilClass
# trying to set the attribute later in the game
ruby-1.9.2-p0 > u = User.new
 => #<User id: nil, name: nil, class: nil, created_at: nil, updated_at: nil> 
ruby-1.9.2-p0 > u.class = '1995'
NoMethodError: undefined method `private_method_defined?' for nil:NilClass

Like the aforementioned wife/girlfriend, you're not going anywhere until this issue is resolved. Luckily, Brian Jones has solved this problem for us with his gem safe_attributes. Rails automatically creates accessors (getter and setter methods) for every attribute in an ActiveRecord model's table. Trying to override crucial methods like "class" is what gets us into trouble. The safe_attributes gem turns off the creation of any dangerously named attributes.

Just do this:

# app/models/user.rb
class User < ActiveRecord::Base
  bad_attribute_names :class
end

After including the gem in your bundler, pass bad_attribute_names the list of offending column names, and it will keep Rails from trying to generate accessor methods for it. Now, this does come with a caveat: you don't have those accessors. Let's try to get/set our :class attribute:

ruby-1.9.2-p0 > u = User.new
 => #<User id: nil, name: nil, class: nil, created_at: nil, updated_at: nil> 
ruby-1.9.2-p0 > u.class = '1995'
 => "1995" 
ruby-1.9.2-p0 > u
 => #<User id: nil, name: nil, class: "1995", created_at: nil, updated_at: nil> 
ruby-1.9.2-p0 > u.class
 => User(id: integer, name: string, class: string, created_at: datetime, updated_at: datetime) 

The setter still works (I'm guessing that it was still created because there wasn't a pre-existing "class=" method) and we can verify that the object's attribute has been properly set. But calling the getter defaults to...well, the default behavior.

The answer is to always use this attribute in the context of a hash. You can send the object a hash of attribute names/values, and that works. This means your controller creating and updating won't have to change. Methods like new, create, update_attribute, update_attributes, etc will work fine.

If you want to just set the single value (to prevent an immediate save, for example) do it like this:

ruby-1.9.2-p0 > u[:class] = '1996'
 => "1996" 
ruby-1.9.2-p0 > u
 => #<User id: nil, name: nil, class: "1996", created_at: nil, updated_at: nil> 

Basically, you can still set the attribute directly, instead of going through the rails-generated accessors. But we're still one step away from a complete solution. We want to be able to treat this attribute like any other, and that requires giving it a benign set of accessors (getter and setter methods). One reason to do this is so we can use standard validations on this attribute.

Adding accessors to our model is this simple:

# add to app/models/user.rb

def class_name= value
  self[:class] = value
end
  
def class_name
  self[:class]
end

We're calling the accessors "class_name", and now we can use that everywhere instead of the original attribute name. We can use it in forms:

# example, not found in code

<%= f.text_field :class_name %>

Or in validations:

# add to app/models/user.rb

validates_presence_of :class_name

Or when creating a new object:

# example, not found in code

User.create :class_name => 'class of 1995'

If you download the code, these additions are test-driven, meaning I wrote the tests for those methods before writing the methods themselves, to be sure they worked properly. I encourage you to do the same.

Good luck!

Ruby’s Splat Operator

December 22, 2010

Ruby is a dynamic language. One of the things it lets you do is define methods with an unknown (or variable) number of arguments. It does this using the splat operator. But the splat operator can actually be used for other things in your code, especially if you’re using Ruby 1.9. That’s because a small change to how splat operators work make them much more useful.

In the beginning

The humble splat operator was first used to slurp up unnamed arguments to a method:

def sum *numbers
  numbers.inject{|sum, number| sum += number}
end

Sometimes, this is just syntactic sugar, because passing a list of numbers like this:

sum(1, 2, 3)

Is more intuitive and prettier, and less error-prone, than passing them as an explicit array:

sum([1, 2, 3])

In many cases though, the splat operator is more than just a pretty face. Take Ruby's own method_missing instance method, that is available in every class. If defined, it will attempt to handle any calls to methods that don't explicitly exist:

class Person
  def method_missing sym, *args
    if sym.to_s =~ /^(\w+)=$/
      instance_variable_set "@#{$1}", args[0]
    else
      instance_variable_get "@#{sym}"
    end
  end
end

method_missing must be able to accept an unknown number of arguments, since just about any method call could be thrown at it. In this case, we're using it to get and set instance variables without having to define them first using attr_accessor:

ruby-1.8.7-p299 > p = Person.new
 => #<Person:0x10018f640 @turtles=nil> 
ruby-1.8.7-p299 > p.turtles = 'I like them'
 => "I like them" 
ruby-1.8.7-p299 > p.turtles
 => "I like them" 
ruby-1.8.7-p299 > p.squirrels
 => nil 
ruby-1.8.7-p299 > p.squirrels = 'are okay'
 => "are okay" 
ruby-1.8.7-p299 > p.squirrels
 => "are okay" 

We've just created a class that lets us define any attributes we want, and our method doesn't care whether *args contains zero arguments, or a hundred.

It gets unintentionally cooler

Up through Ruby 1.8 you could use this splat operator, in a limited fashion, for things other than method argument lists. You could use it to flatten an array, in contexts where it was the last element in the list:

ruby-1.8.7-p299 > last_numbers = [3, 4, 5]
 => [3, 4, 5] 
ruby-1.8.7-p299 > all_numbers = [1, 2, *last_numbers]
 => [1, 2, 3, 4, 5]

The splat operator took the last_numbers array and expanded it inline! Now our new array contains five numbers, instead of two numbers and a nested array. This comes in handy for meta-programming. So let's try putting the splat operator somewhere else in the array:

ruby-1.8.7-p299 > middle_numbers = [2, 3, 4]
 => [2, 3, 4] 
ruby-1.8.7-p299 > all_numbers = [1, *middle_numbers, 5]
SyntaxError: compile error
(irb):4: syntax error, unexpected ',', expecting ']'
all_numbers = [1, *middle_numbers, 5]
                                  ^
	from (irb):4
ruby-1.8.7-p299 > first_numbers = [1, 2, 3]
 => [1, 2, 3] 
ruby-1.8.7-p299 > all_numbers = [*first_numbers, 4, 5]
SyntaxError: compile error
(irb):6: syntax error, unexpected ',', expecting ']'
all_numbers = [*first_numbers, 4, 5]
                              ^
	from (irb):6

We can't use the splat operator anywhere else except the end of a list, just like in method calls. This really limits its value in Ruby 1.8.

Things get intentionally cooler

As of Ruby 1.9, however, the splat operator has been given a little more love, and now it can be used almost anywhere. Basically, any array that is given the splat operator will "flatten" itself, and return the list of elements NOT in an array. Now we can do cool stuff like this:

ruby-1.9.2-p0 > first_numbers = [1, 2, 3]
 => [1, 2, 3] 
ruby-1.9.2-p0 > middle_numbers = [4, 5, 6]
 => [4, 5, 6] 
ruby-1.9.2-p0 > last_numbers = [7, 8, 9]
 => [7, 8, 9] 
ruby-1.9.2-p0 > all_numbers = [*first_numbers, *middle_numbers, *last_numbers]
 => [1, 2, 3, 4, 5, 6, 7, 8, 9] 

This is handy for all sorts of meta-programming challenges - namely, handling dynamic argument lists. It's also great when you're building an array out of smaller pieces where some of the pieces are scalars (single values) and some are arrays. Let's say we have a Family class, that contains myself, my parents, and my siblings. I want a method that returns everybody in one large array. The usage would look like this:

ruby-1.9.2-p0 > family = Family.new
 => #<Family:0x000001018ed2d8> 
ruby-1.9.2-p0 > family.myself = 'Greg'
 => "Greg" 
ruby-1.9.2-p0 > family.parents = ['Mike', 'Carol']
 => ["Mike", "Carol"] 
ruby-1.9.2-p0 > family.siblings = ['Peter', 'Bobby', 'Marsha', 'Jan', 'Cindy']
 => ["Peter", "Bobby", "Marsha", "Jan", "Cindy"] 
ruby-1.9.2-p0 > family.everybody
 => ["Greg", "Mike", "Carol", "Peter", "Bobby", "Marsha", "Jan", "Cindy"] 

If I want to do this without the splat operator (as I'd have to in Ruby 1.8) it would look like this:

class Family
  attr_accessor :myself, :parents, :siblings
  
  def everybody
    members = []
    members << myself
    members += parents
    members += siblings
  end
end

As you can see, I have to go line-by-line, interacting with the members array differently depending on if I'm adding one value or an array. Here's how you'd do it without the splat operator:

class Family
  attr_accessor :myself, :parents, :siblings
  
  def everybody
    [myself, *parents, *siblings]
  end
end

Much cleaner! This version is shorter and easier to understand. This is a technique that might not seem useful, until you have a need for it. Then you'll start finding uses for it everywhere.

Dynamic Methods in Ruby with method_missing

December 21, 2010

Make it up as you go

One way Ruby is dynamic is that you can choose how to handle methods that are called, but don’t actually exist. If you have a lot of very similar methods, you can even use this to define them all at once! Ruby does this using the method_missing method, which you override in the classes where you need more dynamic method calling.

ActiveRecord's dynamic find_all_by methods

Ruby on Rails uses method_missing with ActiveRecord's find_all_by methods. There is no find_all_by_name method, but if your Person model has a name attribute, you can call Person.find_all_by_name('Bob') and it will return all the records that match that name.

Here's a very simplified version of how Rails handles find_all_by requests:

class Person < ActiveRecord::Base
  def self.method_missing method_name, *args
    if method_name =~ /^find_all_by_(\w+)$/
      self.all(:conditions => {$1 => args[0]})
    end
  end
end

Using regular expressions, method_missing sees if the method name matches something we expect. It parses out the interesting parts, and uses them to look up the objects we're searching for. This is a good use case, because the attributes of an ActiveRecord model aren't known until runtime.

Dynamic methods for dynamic objects outside Rails

We can apply this same technique outside of Rails. Let's create the world's most dynamic Ruby class:

# lib/widget.rb
class Widget
  def method_missing sym, *args
    if sym =~ /^(\w+)=$/
      instance_variable_set "@#{$1}", args[0]
    else
      instance_variable_get "@#{sym}"
    end
  end
end

We've just created a Widget object that can have any attributes you want to give it. method_missing checks if the called method ends with an equal sign - if so, it assigns the value you passed, to an instance variable with that name. If there's no equal sign, it tries to get the value of an instance variable by that name:

ruby-1.9.2-p0 > widget = Widget.new
 => #<Widget:0x0000010383f618> 
ruby-1.9.2-p0 > widget.name = 'Bob'
 => "Bob" 
ruby-1.9.2-p0 > widget.age = 30
 => 30 
ruby-1.9.2-p0 > widget.name
 => "Bob" 
ruby-1.9.2-p0 > widget.age
 => 30 

Use method_missing with methods that use blocks

You can also pass blocks to method_missing. Say we have an ActiveRecord model called Person, with name and age attributes. Let's create something similar to find_all_by that gets the list of matching people, and runs them through the map method. We'll call it map_by:

# app/models/person.rb
class Person < ActiveRecord::Base
  def self.method_missing method_name, *args, &block
    if method_name =~ /^map_by_(\w+)$/
      list = self.all(:conditions => {$1 => args[0]})
      list.map(&block)
    end
  end
end

If a method is called that can't be found, method_missing will check to see if it matches our map_by pattern, perform an ActiveRecord search, and push the results through map with the block we supplied.

Now let's see if it works, by grabbing the names of all people in our database age 30:

ruby-1.9.2-p0 > Person.create :name => 'Bob', :age => 30
 => #<Person id: 2, name: "Bob", age: 30, created_at: "2010-12-21 02:23:57", updated_at: "2010-12-21 02:23:57"> 
ruby-1.9.2-p0 > Person.create :name => 'John', :age => 29
 => #<Person id: 3, name: "John", age: 29, created_at: "2010-12-21 02:24:11", updated_at: "2010-12-21 02:24:11"> 
ruby-1.9.2-p0 > Person.create :name => 'Marsha', :age => 30
 => #<Person id: 4, name: "Marsha", age: 30, created_at: "2010-12-21 02:24:22", updated_at: "2010-12-21 02:24:22"> 
ruby-1.9.2-p0 > Person.map_by_age(30){|person| person.name}
 => ["Bob", "Marsha"] 

It works! Now I'm going to refactor the Person class to make it easier to add more dynamic methods in the future. I'll even add an each_by handler so we can see multiple dynamic methods in action:

# app/models/person.rb
class Person < ActiveRecord::Base
  class << self
    def method_missing method_name, *args, &block
      case method_name
      when /^map_by_(\w+)$/ then map_by $1, args[0], &block
      when /^each_by(\w+)$/ then each_by $1, args[0], &block
      else super method_name, *args, &block
      end
    end
    
    def map_by attribute, value, &block
      list = self.all(:conditions => {attribute => value})
      list.map(&block)
    end

    def each_by attribute, value, &block
      list = self.all(:conditions => {attribute => value})
      list.each(&block)
    end
  end
end

I've done a few things. First, I changed our "if" conditional to a case statement, so that we can add to it in the future, and it will be clean and readable. I also moved the actual map_by code into its own method, for the same reason. And now, method_missing calls its parent method if it doesn't find a match, to preserve inheritance.

You might also notice that instead of defining self.method_missing and self.map_by, I've wrapped these method definitions in a class << self block that essentially does the same thing. I think this is cleaner when you have several class methods.

method_missing can be used in any Ruby class, so long as you can anticipate dynamic methods that the users of your class might need, and preserve the chain of inheritance. This should be used sparingly, when you can cut down on method definitions by defining them dynamically. It's easy to abuse this, and there is extra overhead involved. But for the right situations, method_missing can create shorter, more readable code.

Ruby’s defined? Operator

December 20, 2010

Even if you’ve used Ruby’s defined? operator on a daily basis, you may not understand how it works. I sure didn't until recently, but it's worth a look.

A refresher in memoization etiquette

If you're acquainted with memoization, this might look familiar:

class Person
  attr_accessor :first_name, :last_name

  def full_name
    return @full_name if defined?(@full_name)
    @full_name = "#{first_name} #{last_name}"
  end
end

The full_name method above uses memoization - the return value of the method is calculated just once, on the first call. It's stored in the instance variable @full_name, and used for subsequent calls to the method. I first discovered this technique digging through the code base for Thoughtbot's shoulda gem. I've used it hundreds of times, and never really questioned how the defined? operator works until recently.

Question mark?

Ruby methods can contain some non-alphanumeric characters like "!" and "?", and rubyists take advantage of this to add readability to our code. Methods ending in "!" typically mean one of two things: the method is altering its receiver, or it's going to complain loudly if it fails (usually by raising an exception). By the same convention, methods ending in "?" are asking a question, and the answer is usually boolean (yes/no).

The defined? method follows this convention...sort of. I always assumed it returned true/false, but that's only half the story. If the object in question is defined, defined? gives you a string description of the object. This equates to "true" in any conditional arguments. If the object is not defined, it returns nil, which equates to "false".

Test anything. Almost.

So defined? works in any boolean context, but it also provides a little more info. And it works on just about anything. Classes:

ruby-1.9.2-p0 > defined? Person
 => nil 
ruby-1.9.2-p0 > class Person
ruby-1.9.2-p0 ?>  end
 => nil 
ruby-1.9.2-p0 > defined? Person
 => "constant" 

It works on methods:

ruby-1.9.2-p0 > def bark
ruby-1.9.2-p0 ?>  puts "woof"
ruby-1.9.2-p0 ?>  end
 => nil 
ruby-1.9.2-p0 > defined? bark
 => "method" 

And of course it works on variables of all kinds:

ruby-1.9.2-p0 > defined? @@a
 => nil 
ruby-1.9.2-p0 > @@a = 'a'
 => "a" 
ruby-1.9.2-p0 > defined? @@a
 => "class variable" 
ruby-1.9.2-p0 > defined? @b
 => nil 
ruby-1.9.2-p0 > @b = 'b'
 => "b" 
ruby-1.9.2-p0 > defined? @b
 => "instance-variable" 
ruby-1.9.2-p0 > defined? c
 => nil 
ruby-1.9.2-p0 > c = 'c'
 => "c" 
ruby-1.9.2-p0 > defined? c
 => "local-variable" 

I even tried other operators, just on a whim. But of course, this was too much to hope for :)

ruby-1.9.2-p0 > defined?(+)
SyntaxError: (irb):1: syntax error, unexpected ')'
	from /Users/bellmyer/.rvm/rubies/ruby-1.9.2-p0/bin/irb:17:in `<main>'

defined? is an operator, not a method

Because you can enclose the object you want to check in parentheses (as in, defined?(@full_name)), you might be tempted to think it's a method. It's not, it's a native operator. This is an important distinction, because it means defined? can't be overridden:

ruby-1.9.2-p0 > x = 'test variable'
 => "test variable" 
ruby-1.9.2-p0 > defined? x
 => "local-variable" 
ruby-1.9.2-p0 > def defined? object
ruby-1.9.2-p0 ?>  puts "defined? has been overridden!"
ruby-1.9.2-p0 ?>  end
 => nil 
ruby-1.9.2-p0 > defined? x
 => "local-variable" 

I don't get an error trying to override the operator with a method definition, but it doesn't work, either. Honestly, Ruby is so permissive I half expected the override to work anyway! Another clue that you're dealing with an operator is that it has no receiver. That's why you give it the object, instead of calling it from a receiver, like the nil? method:

ruby-1.9.2-p0 > @x.nil?
 => true 
ruby-1.9.2-p0 > @x.defined?
NoMethodError: undefined method `defined?' for nil:NilClass
	from (irb):8
	from /Users/bellmyer/.rvm/rubies/ruby-1.9.2-p0/bin/irb:17:in `<main>'

While defined? is most valuable (and most commonly used) in a boolean context, there may be meta-programming applications where you'd want to what type of "thing" you're dealing with. While you can always use the .class method, you have to already know that the object is defined. In the world of meta-programming, that's often a luxury you don't have.

Ruby Method Permissions: The Differences Between Public, Protected, and Private

December 15, 2010

It’s easy to code in Ruby for years, and never pay attention to method permissions. I know, because I did it. I came from a C/C++ background, so I understood the concepts of public vs private. And I vaguely understood that Ruby's protected fell somewhere in between. I just never really cared enough to look it up. Eventually I got curious. I'll run down the different types of method visibility here.

Examples

It's hard to think of a good, simple example to use for this. I'll go with hobbits. They're friendly to other species, but they have private lives, as well. Much of their privacy seems to be open to other hobbits, since they appear to be able to wander into each others' houses in the shire at will. Of course, they also have a few choice secrets kept just to themselves. They have all the qualities of a well-rounded Ruby method. Let's define their class here:

class Hobbit
  def initialize(name, rooms, has_ring)
    @name, @rooms, @has_ring = name, rooms, has_ring
  end

  def name
    @name
  end

  def name_of(hobbit)
    hobbit.name
  end

  def rooms_of(hobbit)
    hobbit.rooms
  end

  def hobbit_has_ring?(hobbit)
    hobbit.has_ring?
  end

  protected

  def rooms
    @rooms
  end

  private

  def has_ring?
    @has_ring
  end
end

Each hobbit has three attributes. Their name is public, the number of rooms in their house is protected, and whether or not they have The Ring is private. There are methods to read these attributes, and to try to read them from other hobbits. Now let's create a couple hobbits to work with:

irb(main):001:0> require 'hobbit.rb'
=> true
irb(main):002:0> frodo = Hobbit.new('Frodo', 3, true)
=> #<Hobbit:0x10038b5c0 @rooms=3, @name="Frodo", @has_ring=true>
irb(main):003:0> samwise = Hobbit.new('Samwise', 2, false)
=> #<Hobbit:0x1003769e0 @rooms=2, @name="samwise", @has_ring=false>

Public Methods

This is the default method type in Ruby (and most languages), and it means that it can be called from anywhere, by any hunk of code that knows what the object is. We can easily find out the names of our hobbits:

irb(main):004:0> frodo.name
=> "Frodo"
irb(main):005:0> samwise.name
=> "Samwise"

Protected Methods

Now, what about the rooms in their houses? This info is accessible only through protected methods:

irb(main):006:0> frodo.rooms
NoMethodError: protected method `rooms' called for #<Hobbit:0x10034a4a8 @rooms=3, @name="Frodo", @has_ring=true>
	from (irb):6
	from :0
irb(main):007:0> frodo.rooms_of(frodo)
=> 3
irb(main):008:0> frodo.rooms_of(samwise)
=> 2

As you can see, we can't call the hobbits' room methods directly, but one hobbit can call the protected rooms method for itself, or any other hobbit. In purely programming terms, public methods are globally accessible, where protected methods can only be called by other methods in the class.

Private Methods

What about ring status? This info is locked in private methods. How can we access this?

irb(main):013:0> frodo.has_ring?
NoMethodError: private method `has_ring?' called for #<Hobbit:0x10034a4a8 @rooms=3, @name="Frodo", @has_ring=true>
	from (irb):13
	from :0
irb(main):014:0> frodo.hobbit_has_ring?(frodo)
NoMethodError: private method `has_ring?' called for #<Hobbit:0x10034a4a8 @rooms=3, @name="Frodo", @has_ring=true>
	from ./hobbit.rb:19:in `hobbit_has_ring?'
	from (irb):14
	from :0
irb(main):015:0> frodo.hobbit_has_ring?(samwise)
NoMethodError: private method `has_ring?' called for #<Hobbit:0x100374c80 @rooms=2, @name="Samwise", @has_ring=false>
	from ./hobbit.rb:19:in `hobbit_has_ring?'
	from (irb):15
	from :0

We can't access the has_ring? method directly. We can't access it indirectly from another method, specifying the hobbit we're interested in. Not even if that hobbit is ourselves! Let's reopen our class and add the i_have_the_ring? method:

class Hobbit
  def i_have_the_ring?
     has_ring?
  end
end

Now frodo has a way to check if he has the ring:

irb(main):021:0> frodo.i_have_the_ring?
=> true

In Ruby, protected methods are still open to other objects of the same type. So as long as the "receiver" (hobbit in hobbit.rooms) is the same class as the calling object, method access is granted.

But private methods are not allowed to specify a receiver at all. I can't say frodo.has_ring? or even self.has_ring? inside the frodo object at all. I'm only allowed to say has_ring? and the object will assume it's calling its own method.

Summary

While public methods are fairly straightforward, the difference between protected and private can cause problems if you're not aware of it. Ruby's private implementation is even stricter than C's, not allowing receivers. But protected balances that out with the ability to "walk into sibling objects' houses".

I tend to use protected methods when I don't intend for them to be called directly, and I don't write tests for them. I write tests for the public methods that call them. And I generally avoid private methods, because I can't envision needing to block sibling-to-sibling access (although I probably should if I'm not testing them, right?) but I don't want to spend hours debugging a method call that breaks the no-receiver rule.


Follow

Get every new post delivered to your Inbox.