Test Multiple Rubies by Combining Bundler and RVM

I have read several opinions that are either pro-Bundler or pro-RVM, but despite the current debate they do not have to be mutually exclusive. RVM is terrific at managing multiple ruby versions, and Bundler is awesome at managing gem versions within a project. That was a eureka moment when I realized I wanted to support multiple ruby platforms with my library harvested.

While working on harvested I realized I didn’t want to be tied down to just ruby 1.9, so I decided I would maintain support for 1.8.7, JRuby, and the latest rubinius. Writing cross-platform ruby code really isn’t that difficult, since the language implementors have taken great pains to maintain compatibility. However manually switching among each ruby version and running the suite gets tedious. Following the theme of if it hurts, do it more often I had to believe there was a simpler way, so I write this mini-script called test_rubies.sh.

#!/usr/bin/env bash

set -o verbose
RBXOPT="-Xrbc.db" rvm ruby-1.8.7-p334,ruby-1.9.2-p180,rbx,jruby-1.6.2 exec bundle
RBXOPT="-Xrbc.db" rvm ruby-1.8.7-p334,ruby-1.9.2-p180,rbx,jruby-1.6.2 exec bundle exec rake spec

and here’s an example output from master:

ruby-1.9.2-p180 ~/projects/harvested/code (master) $ ./spec/test_rubies 
RBXOPT="-Xrbc.db" rvm ruby-1.8.7-p334,ruby-1.9.2-p180,rbx,jruby-1.6.2 exec bundle
Using rake (0.8.7) 
Using addressable (2.2.6) 
Using bundler (1.0.15) 
Using columnize (0.3.2) 
Using crack (0.1.8) 
Using diff-lcs (1.1.2) 
Using git (1.2.5) 
Using hashie (1.0.0) 
Using httparty (0.7.8) 
Using jeweler (1.6.0) 
Using json (1.5.2) 
Using linecache (0.43) 
Using rspec-core (2.6.3) 
Using rspec-expectations (2.6.0) 
Using rspec-mocks (2.6.0) 
Using rspec (2.6.0) 
Using ruby-debug-base (0.10.4) 
Using ruby-debug (0.10.4) 
Using vcr (1.10.0) 
Using webmock (1.6.4) 
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
Using rake (0.8.7) 
Using addressable (2.2.6) 
Using archive-tar-minitar (0.5.2) 
Using bundler (1.0.15) 
Using columnize (0.3.2) 
Using crack (0.1.8) 
Using diff-lcs (1.1.2) 
Using git (1.2.5) 
Using hashie (1.0.0) 
Using httparty (0.7.8) 
Using jeweler (1.6.0) 
Using json (1.5.2) 
Using ruby_core_source (0.1.5) 
Using linecache19 (0.5.12) 
Using rspec-core (2.6.3) 
Using rspec-expectations (2.6.0) 
Using rspec-mocks (2.6.0) 
Using rspec (2.6.0) 
Using ruby-debug-base19 (0.11.25) 
Using ruby-debug19 (0.11.6) 
Using vcr (1.10.0) 
Using webmock (1.6.4) 
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
Using rake (0.8.7) 
Using addressable (2.2.6) 
Using bundler (1.0.15) 
Using crack (0.1.8) 
Using diff-lcs (1.1.2) 
Using git (1.2.5) 
Using hashie (1.0.0) 
Using httparty (0.7.8) 
Using jeweler (1.6.0) 
Using json (1.5.2) 
Using rspec-core (2.6.3) 
Using rspec-expectations (2.6.0) 
Using rspec-mocks (2.6.0) 
Using rspec (2.6.0) 
Using vcr (1.10.0) 
Using webmock (1.6.4) 
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
Using rake (0.8.7) 
Using addressable (2.2.6) 
Using bouncy-castle-java (1.5.0146.1) 
Using bundler (1.0.15) 
Using columnize (0.3.2) 
Using crack (0.1.8) 
Using diff-lcs (1.1.2) 
Using git (1.2.5) 
Using hashie (1.0.0) 
Using httparty (0.7.8) 
Using jeweler (1.6.0) 
Using jruby-openssl (0.7.4) 
Using json (1.5.2) 
Using rspec-core (2.6.3) 
Using rspec-expectations (2.6.0) 
Using rspec-mocks (2.6.0) 
Using rspec (2.6.0) 
Using ruby-debug-base (0.10.4) 
Using ruby-debug (0.10.4) 
Using vcr (1.10.0) 
Using webmock (1.6.4) 
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
RBXOPT="-Xrbc.db" rvm ruby-1.8.7-p334,ruby-1.9.2-p180,rbx,jruby-1.6.2 exec bundle exec rake spec
(in /Users/zmoazeni/projects/harvested/code)
/Users/zmoazeni/.rvm/rubies/ruby-1.8.7-p334/bin/ruby -S bundle exec rspec ./spec/functional/account_spec.rb ./spec/functional/clients_spec.rb ./spec/functional/errors_spec.rb ./spec/functional/expenses_spec.rb ./spec/functional/hardy_client_spec.rb ./spec/functional/invoice_spec.rb ./spec/functional/project_spec.rb ./spec/functional/reporting_spec.rb ./spec/functional/tasks_spec.rb ./spec/functional/time_tracking_spec.rb ./spec/functional/users_spec.rb ./spec/harvest/base_spec.rb ./spec/harvest/credentials_spec.rb ./spec/harvest/expense_category_spec.rb ./spec/harvest/expense_spec.rb ./spec/harvest/invoice_spec.rb ./spec/harvest/project_spec.rb ./spec/harvest/task_assignment_spec.rb ./spec/harvest/task_spec.rb ./spec/harvest/time_entry_spec.rb ./spec/harvest/user_assignment_spec.rb ./spec/harvest/user_spec.rb
............*.........*........................*.........

Pending:
  harvest invoices allows adding, updating and removing invoices
    # having trouble following the API docs at http://www.getharvest.com/api/invoices
    # ./spec/functional/invoice_spec.rb:18
  harvest time tracking allows toggling of timers
    # Not Yet Implemented
    # ./spec/functional/time_tracking_spec.rb:52
  Harvest::Task casts default_hourly_rate to float
    # Not Yet Implemented
    # ./spec/harvest/task_spec.rb:6

Finished in 35.96 seconds
57 examples, 0 failures, 3 pending
(in /Users/zmoazeni/projects/harvested/code)
/Users/zmoazeni/.rvm/rubies/ruby-1.9.2-p180/bin/ruby -S bundle exec rspec ./spec/functional/account_spec.rb ./spec/functional/clients_spec.rb ./spec/functional/errors_spec.rb ./spec/functional/expenses_spec.rb ./spec/functional/hardy_client_spec.rb ./spec/functional/invoice_spec.rb ./spec/functional/project_spec.rb ./spec/functional/reporting_spec.rb ./spec/functional/tasks_spec.rb ./spec/functional/time_tracking_spec.rb ./spec/functional/users_spec.rb ./spec/harvest/base_spec.rb ./spec/harvest/credentials_spec.rb ./spec/harvest/expense_category_spec.rb ./spec/harvest/expense_spec.rb ./spec/harvest/invoice_spec.rb ./spec/harvest/project_spec.rb ./spec/harvest/task_assignment_spec.rb ./spec/harvest/task_spec.rb ./spec/harvest/time_entry_spec.rb ./spec/harvest/user_assignment_spec.rb ./spec/harvest/user_spec.rb
............*.........*........................*.........

Pending:
  harvest invoices allows adding, updating and removing invoices
    # having trouble following the API docs at http://www.getharvest.com/api/invoices
    # ./spec/functional/invoice_spec.rb:18
  harvest time tracking allows toggling of timers
    # Not Yet Implemented
    # ./spec/functional/time_tracking_spec.rb:52
  Harvest::Task casts default_hourly_rate to float
    # Not Yet Implemented
    # ./spec/harvest/task_spec.rb:6

Finished in 25.49 seconds
57 examples, 0 failures, 3 pending
(in /Users/zmoazeni/projects/harvested/code)
/Users/zmoazeni/.rvm/rubies/rbx-head/bin/rbx -S bundle exec rspec ./spec/functional/account_spec.rb ./spec/functional/clients_spec.rb ./spec/functional/errors_spec.rb ./spec/functional/expenses_spec.rb ./spec/functional/hardy_client_spec.rb ./spec/functional/invoice_spec.rb ./spec/functional/project_spec.rb ./spec/functional/reporting_spec.rb ./spec/functional/tasks_spec.rb ./spec/functional/time_tracking_spec.rb ./spec/functional/users_spec.rb ./spec/harvest/base_spec.rb ./spec/harvest/credentials_spec.rb ./spec/harvest/expense_category_spec.rb ./spec/harvest/expense_spec.rb ./spec/harvest/invoice_spec.rb ./spec/harvest/project_spec.rb ./spec/harvest/task_assignment_spec.rb ./spec/harvest/task_spec.rb ./spec/harvest/time_entry_spec.rb ./spec/harvest/user_assignment_spec.rb ./spec/harvest/user_spec.rb
............*.........*........................*.........

Pending:
  harvest invoices allows adding, updating and removing invoices
    # having trouble following the API docs at http://www.getharvest.com/api/invoices
    # ./spec/functional/invoice_spec.rb:18
  harvest time tracking allows toggling of timers
    # Not Yet Implemented
    # ./spec/functional/time_tracking_spec.rb:52
  Harvest::Task casts default_hourly_rate to float
    # Not Yet Implemented
    # ./spec/harvest/task_spec.rb:6

Finished in 40.09 seconds
57 examples, 0 failures, 3 pending
(in /Users/zmoazeni/projects/harvested/code)
/Users/zmoazeni/.rvm/rubies/jruby-1.6.2/bin/jruby -S bundle exec rspec ./spec/functional/account_spec.rb ./spec/functional/clients_spec.rb ./spec/functional/errors_spec.rb ./spec/functional/expenses_spec.rb ./spec/functional/hardy_client_spec.rb ./spec/functional/invoice_spec.rb ./spec/functional/project_spec.rb ./spec/functional/reporting_spec.rb ./spec/functional/tasks_spec.rb ./spec/functional/time_tracking_spec.rb ./spec/functional/users_spec.rb ./spec/harvest/base_spec.rb ./spec/harvest/credentials_spec.rb ./spec/harvest/expense_category_spec.rb ./spec/harvest/expense_spec.rb ./spec/harvest/invoice_spec.rb ./spec/harvest/project_spec.rb ./spec/harvest/task_assignment_spec.rb ./spec/harvest/task_spec.rb ./spec/harvest/time_entry_spec.rb ./spec/harvest/user_assignment_spec.rb ./spec/harvest/user_spec.rb
............*.........*........................*.........

Pending:
  harvest invoices allows adding, updating and removing invoices
    # having trouble following the API docs at http://www.getharvest.com/api/invoices
    # ./spec/functional/invoice_spec.rb:18
  harvest time tracking allows toggling of timers
    # Not Yet Implemented
    # ./spec/functional/time_tracking_spec.rb:52
  Harvest::Task casts default_hourly_rate to float
    # Not Yet Implemented
    # ./spec/harvest/task_spec.rb:6

Finished in 34.07 seconds
57 examples, 0 failures, 3 pending

So there are a few things going on in this script. But the main jist of it is: run “bundle” with 1.8, 1.9, rubinius, and JRuby to make sure the dependencies are installed across each platform. Then runs “rake spec” across the same ruby platforms and reports any test failures.

1) set -o verbose just sets the bash verbosity to echo the commands we execute

2) rvm platform1,platform2, .. exec ... is a way to execute a command across multiple ruby platforms

3) RBXOPT="-Xrbc.db" is a rubinius specific flag to store the .rbc files in a database folder instead of littering your source code.

4) rvm platform1,platform2 exec bundle exec ... seems repetitive, however it guards against some things like the recent rake versioning issue.

5) You need to make sure each RVM ruby platform is installed.

Fairly easy, and very useful. Now I can work on harvested with confidence that I support various ruby platforms.

Zach Moazeni

Comments

Chris Harper
August 22, 2011 at 13:24 PM

You might like to try a project I contribute to called Travis which will automate what your doing : http://travis-ci.org/.

Simply setup a github webhook and it will run through your test suite after every commit.

You can define Gemfiles and different ruby versions to make testing easier. Here is an example of a ‘complex’ test suite across multiple rubies : http://travis-ci.org/#!/thoughtbot/factory_girl/builds/92556
Daniel Morrison
August 22, 2011 at 14:43 PM

Chris: We definitely love Travis and hope to use it for more stuff soon.

This is more for running it locally, or if you have sensitive stuff you can’t send out.

By Zach Moazeni

August 22, 2011