Jan 07, 2015

Benchmarking Rubinius

blogpost author photo
Michał Gutowski
blogpost cover image
Rubinius is an implementation of Ruby designed for concurrency. Instead of running programs using one thread, as it is in CRuby, it uses native threads. What is more interesting most of Rubinius’ interpreter is written in ruby itself! I would like to show you how it compares to CRuby on my machine.

My setup

I run benchmarks on my mid-2010 MacBook Pro:

  • Processor: Intel Core 2 Duo, 2.66 Ghz

  • Memory: 6 GB DDR3 ram

  • Hard disc: Samsung SSD 840 PRO

During the tests I compared Rubinius 2.3.0.n327 with Ruby 2.1.5p273 and Ruby 2.2.0dev


Benchmark code was provided by ruby-benchmark-suite. I did not create plots for all tests. If you would like to take a look at results yourself, please feel free to download them


Lets start from multithreading benchmarks, as Rubinius is said to be well prepared for this.

def count_high
  100_000.times {}

Bench.run [1, 2, 4, 8, 16] do |n|
  threads = []
  n.times{ threads << Thread.new { count_high }}
  threads.each{|t| t.join}

This test just creates 1,2,4,8,16 threads and makes them run an empty loop. If the Rubinius VM supports threads, they should be run in concurrent manner so the test should be run much faster than in MRI.

It is clearly visible that Rubinius overcomes its rivals, it is over 2 times when 16 threads are created. Additionally time need to compute this grows much faster when using CRuby.


Eval is a function which evaluates expression given in string and returns result. It is widely used function in Ruby on Rails and the interpreter should be able to run it fast.

Bench.run [1000000] do |n|
  n.times { eval "a = 33" }

This benchmark creates simple expression and evaluates it 1000000 times. Lets see how the Rubinius did here.

Unfortunately for Rubinius this benchmark didn’t go well. CRuby is almost 6 times faster.


Every Rails application uses lists at some point and again interpreter that handles given RoR application should be capable of using lists in a fast way.

# from http://www.bagley.org/~doug/shootout/bench/lists/lists.ruby
# creates and does manipulation to arrays

def test_lists(size)
  # create a list of integers (Li1) from 1 to SIZE
  li1 = (1..size).to_a
  # copy the list to li2 (not by individual items)
  li2 = li1.dup
  # remove each individual item from left side of li2 and
  # append to right side of li3 (preserving order)
  li3 = Array.new
  while (not li2.empty?)
  # li2 must now be empty
  # remove each individual item from right side of li3 and
  # append to right side of li2 (reversing list)
  while (not li3.empty?)
  # li3 must now be empty
  # reverse li1 in place
  # check that first item is now SIZE
  if li1[0] != size then
    p "not size"
    # compare li1 and li2 for equality
    if li1 != li2 then
      # return the length of the list

Bench.run [1000] do |n|
  n.times { test_lists(10_000) }

Although this benchmark looks complicated it just does common operations with list like coping, reversing, taking it’s size and so on.

And again Rubinius’ performance isn’t the best here.

Garbage collector

Garbage collector is a thing which makes programmer’s day brighter. You don’t need to take care of managing memory management manually. But it comes with a price – it takes additional computing power and time to manage memory in automated way.

There are 3 benchmarks which measures how good given interpreter handles garbage collectionbm_gc_array.rbbm_gc_mb.rb and bm_gc_string.rb.

Bench.run [1] do |n|
  a = nil
  10.times do |i|
    a = Array.new(100_000)
    a.length.times do |x|
      a[x] = Array.new(100)
      10.times do |y|

This benchmark creates Arrays inside arrays and yet some more arrays, lets take a look at results.

Results show clearly that ruby-2.2.0 made a great improvement in collecting arrays, but numbers don’t lie – Rubinius is the fastest.

Bench.run [500_000, 1_000_000, 3_000_000] do |n|
  a = []
  n.times { a << []} # use up some RAM
  n.times {[]}

This benchmark just creates small objects which should be collected immediately.

Unfortunately for Rubinius, it doesn’t handle such situation well.

Bench.run [1] do |n|
  a = nil
  10.times do |i|
    a = Array.new(5000)
    a.length.times do |x|
      str = ""
      20.times{|j| str << "AaBbCcDdEeFfGgHhIiJjKkLlMmNnOoPpQqRrSsTtUuVvWwXxYy"}
      a[x] = str
      10.times do |y|
        junk = ""
        20.times{|k| str << "GaRbAgEGaRbAgEGaRbAgEGaRbAgEGaRbAgEGaRbAgEGaRbAgE_"}

This benchmark creates long string and concatenates it, many times. Lets take a look at Rubinius’ performance.

Yet again CRuby is faster.


Unfortunately Rubinius is left behind by CRuby in most of benchmarks, except ones that uses threads. In my opinion Rubinius shouldn’t be used for production yet, but I keep my fingers crossed for this project. Thanks for reading and as always, feel free to leave a comment!