# How to Benchmark my code in Ruby or Rails

Recently, I had a job to fix the slow code. Before diving into fixing the slow code, I used Scout APM to identify the performance bottlenecks, which helped me make sure where the slow code is.

To prove the new optimization solution is faster, in addition to checking the before-after response time in the log file, I think I need a more efficient tool to evaluate the solution.

### Benchmarking

Benchmarking is a way of measuring the performance of the code. It can answer the question like 'Which one is faster, A or B?'

Ruby has a benchmarking tool called `Benchmark` in its standard library. Benchmark module provides methods to measure and report the time used to execute Ruby code.

#### How to use Ruby Benchmark module?

You can add a Ruby file first.

```plaintext
$ touch test.rb
```

You should `require 'benchmark'` to make sure you can use it. Next, put the code that you would like to test into the `x.report { #put your code here }` block.

```ruby
require 'benchmark'

n = 100
Benchmark.bm do |x|
  x.report('A') { for i in 1..n; a = '1'; end }
  x.report('B') { n.times do   ; a = '1'; end }
  x.report('C') { 1.upto(n) do ; a = '1'; end }
end
```

Go back to the terminal and execute the file:

We can see the C plan is faster than the A or B plan.

```plaintext
$ ruby test.rb
       user     system      total        real
A  0.000026   0.000005   0.000031 (  0.000024)
B  0.000018   0.000001   0.000019 (  0.000018)
C  0.000016   0.000001   0.000017 (  0.000017)
```

> This report shows the user CPU time, system CPU time, the sum of the user and system CPU times, and the elapsed real time. The unit of time is seconds. ([Source](https://ruby-doc.org/stdlib-2.7.3/libdoc/benchmark/rdoc/Benchmark.html))

#### Another enhancement benchmarking gem - [benchmark-ips](https://github.com/evanphx/benchmark-ips)

When searching the benchmarking tool, I found out another interesting gem called benchmark-ips.

#### Feature

> An iterations per second enhancement to Benchmark.

#### What's the difference between the traditional Benchmark library and benchmark-ips?

> Benchmark/ips will report the number of iterations per second for a given block of code. **When analyzing the results, notice the percent of** [**standard deviation**](http://en.wikipedia.org/wiki/Standard_deviation) **which tells us how spread out our measurements are from the average.** A high standard deviation could indicate the results having too much variability.

benchmark-ips uses [standard deviation](http://en.wikipedia.org/wiki/Standard_deviation) which is the concept of Statistics to analyze the result. The benefit is that we can literally make sure the result is significant/extreme significant/no significant difference.

In other words, take the result above for example:

C plan seems to be faster than the A or B plan, but it might not be the fact.

```plaintext
# n = 100
$ ruby test.rb
       user     system      total        real
A  0.000026   0.000005   0.000031 (  0.000024)
B  0.000018   0.000001   0.000019 (  0.000018)
C  0.000016   0.000001   0.000017 (  0.000017)
```

Let's use benchmark-ips to analyze the result:

```ruby
require "benchmark/ips"

n = 100
Benchmark.ips do |x|
  x.report('A') { for i in 1..n; a = "1"; end }
  x.report('B') { n.times do  ; a = "1"; end }
  x.report('C') { 1.upto(n) do ; a = "1"; end }

  x.compare!
end
```

There is something interesting:

* B plan seems to iterate more times per second than the A or C plan.
    

```plaintext
Warming up --------------------------------------
                   A    11.038k i/100ms
                   B    11.868k i/100ms
                   C    12.166k i/100ms
Calculating -------------------------------------
                   A    125.949k (± 3.9%) i/s -    629.166k in   5.003441s
                   B    126.662k (± 2.7%) i/s -    640.872k in   5.063567s
                   C    125.172k (± 6.3%) i/s -    632.632k in   5.076965s

Comparison:
                   B:   126662.1 i/s
                   A:   125949.4 i/s - same-ish: difference falls within error
                   C:   125172.4 i/s - same-ish: difference falls within error
```

From the perspective of Statistics, there is not a meaningful difference between the three plans. When I change `n=100` to `n=10000`, the result is the same. I feel very surprised about the result. Very. (Please correct me if I misunderstood. 🙏)

### How to Benchmark the code in Rails

It's simple. Just add `require_relative '../../config/environment'` so that you can use the model class you have built in your Rails App.

```ruby
require 'benchmark/ips'
require_relative '../../config/environment'

transaction_ids = [1, 2, #....]
transactions = Transaction.where(id: transaction_ids)

Benchmark.ips do |x|
  x.report('new') do
    transactions.map(&:some_column).compact
  end

  x.report('old') do
    transaction_records = transactions.
      includes(:a_table, :b_table, :c_table).order(created_at: :desc)

    transaction_records.map(&:some_column).compact
  end

  x.compare!
end
```

---

### Reference

* [https://shopify.engineering/how-fix-slow-code-ruby](https://shopify.engineering/how-fix-slow-code-ruby)
    
* [https://github.com/evanphx/benchmark-ips](https://github.com/evanphx/benchmark-ips)
    
* [https://blog.saeloun.com/2020/02/11/rails-benchmark-generator](https://blog.saeloun.com/2020/02/11/rails-benchmark-generator)
    
* [https://guides.rubyonrails.org/v3.2.13/performance\_testing.html#helper-methods](https://guides.rubyonrails.org/v3.2.13/performance_testing.html#helper-methods)
    
* [https://ihower.tw/rails/performance.html](https://ihower.tw/rails/performance.html)
    
* [https://www.schneems.com/2016/11/14/statistical-lit.html](https://www.schneems.com/2016/11/14/statistical-lit.html)