To support the pgvector gem, I upgraded my local Postgres.app from v13 to v17 on macOS. Below is the full upgrade and restoration process for my project.

Environment

• OS: macOS

• PostgreSQL: Upgraded from v13 to v17 via Postgres.app

• Rails: Used for database management

• Databases: my_database_name_development, my_database_name_test

Steps

1. Back Up All Databases
   Export all databases to a single SQL file:

    pg_dumpall > ~/Desktop/db_backup.sql
   

2. Split the Backup File
   Extract specific databases from the full backup:

    # Development
    awk '/^\\connect my_database_name_development/{flag=1;next} /^\\connect/{flag=0} flag' ~/Desktop/db_backup.sql > ~/Desktop/my_database_name_development_only.sql
   
    # Test
    awk '/^\\connect my_database_name_test/{flag=1;next} /^\\connect/{flag=0} flag' ~/Desktop/db_backup.sql > ~/Desktop/my_database_name_test_only.sql
   

3. Create New Databases
   Recreate the databases using Rails:

    rails db:create
   

4. Restore Data into Databases
   Import the split SQL files into the respective databases:

    # Development
    psql -d my_database_name_development -f ~/Desktop/my_database_name_development_only.sql
   
    # Test
    psql -d my_database_name_test -f ~/Desktop/my_database_name_test_only.sql
   

This process ensures a clean PostgreSQL upgrade while preserving all development and test data.

In Ruby on Rails applications, handling database transactions efficiently is crucial to maintaining data integrity. One common issue developers face is dealing with runtime errors caused by unpersisted changes when using the with_lock method. This guide will help you understand and resolve these errors effectively.

Environment

• Ruby 3.0.7

• Rails 6.1.7.7

Error message

RunTimeError: Locking a record with unpersisted changes is not supported. 
Use save to persist the changes, or reload to discard them explicitly.

RunTimeError: Locking a record with unpersisted changes is not supported. Use save to persist the changes, or reload to discard them explicitly.

How to Reproduce

class Invoice < ApplicationRecord
  before_save :sync_monetary_columns

  # Existing code...

  private

  def sync_monetary_columns
    return unless amount_changed?

    with_lock do 
      self.amount_decimal = amount.to_f
    end
  end
end

invoice = Invoice.last
invoice.amount # 1000

invoice.amount = 500
invoice.save!
# RuntimeError: Locking a record with unpersisted changes is not supported. 
# Use `save` to persist the changes, or `reload` to discard them explicitly.

Understanding how with_lock Works

with_lock is a Rails method that provides pessimistic locking, which is a database-level lock that prevents multiple processes from modifying the same record simultaneously. Here's how it works:

1. When with_lock is called, Rails:

   • Starts a database transaction

   • Reloads the record with SELECT ... FOR UPDATE

   • Executes your block of code

   • Commits the transaction

2. The FOR UPDATE lock means:

   • Other transactions must wait to read or modify this record

   • Prevents race conditions where multiple processes might try to update the same data

   • Lock is held until the transaction commits or rolls back

def with_lock
  transaction do
    # Reloads the object and obtains lock:
    self.reload(lock: true)  # SELECT * FROM table WHERE id = 123 FOR UPDATE

    # Now yields to your block:
    yield

    # Transaction commits after your block completes
  end
end

How to Fix This Error

To fix the runtime error caused by unpersisted changes, here’s my key implementation details:

1. Cache Values Before with_lock:

   • with_lock internally calls reload, which would clear any unpersisted changes. (Source code from Rails)

   • Caching preserves the values we need to modify.

2. Reload Only Persisted Records:

   • For persisted records: with_lock executes reload(lock: true) using the record's ID. When a record has unpersisted changes, this causes the RuntimeError.

   • For new records: No ID exists yet, so reload cannot trigger the error.

This solution ensures proper handling of both new and persisted records while maintaining data integrity during the locking process.

class Invoice < ApplicationRecord
  before_save :sync_monetary_columns

  # Existing code...

  private

  def sync_monetary_columns
    return unless amount_changed?

    # Cache values 
    new_amount = amount.to_of

    reload if persisted?

    with_lock do 
      self.amount_decimal = new_amount
    end
  end
end

Conclusion

By understanding the mechanics of with_lock and implementing the suggested solutions, you can effectively manage runtime errors related to unpersisted changes in your Rails applications. This approach not only resolves the issue but also enhances the robustness of your data handling processes.

Reference

• Rails with_lock Documentation

• MacOS Sonoma 14.3

• Original Ruby version: 2.7.8

• Ruby Version Manager: RVM 1.29.12

Goal

Upgrade Ruby from 2.7.8 to 3.0.6.

Issues Description

After successfully installing Ruby 3.0.6, I encountered difficulties running bundle install in my Rails project due to two OpenSSL-related issues.

1. Could not load OpenSSL.

2. Bundler::Fetcher::CertificateFailureError

Issue 1: Could not load OpenSSL.

bundle install
# Bundler (2.2.33) is older than the version that created the lockfile (2.3.7). We suggest you to upgrade to the version that created the lockfile by running gem install bundler:2.3.7.

Could not load OpenSSL.
You must recompile Ruby with OpenSSL support or change the sources in your Gemfile from 'https' to 'http'. Instructions for compiling with OpenSSL using RVM are available at rvm.io/packages/openssl.

Solution

If you're using macOS and RVM, here's how you can recompile Ruby with OpenSSL support:

1. Install the required dependencies: This is a more general command for installing openssl library on your system, potentially for use by various applications beyond Ruby.

brew install openssl

2. Install OpenSSL using RVM: This is specific to managing OpenSSL for Ruby installations through rvm.

rvm pkg install openssl

3. Install Ruby with OpenSSL support: This is use for reinstalling the required Ruby version and specify the installation location of OpenSSL.

# Replace <your_version> with your Ruby version.
rvm reinstall ruby-<your_version> --with-openssl-dir=$(brew --prefix openssl)

4. After the installation is complete, set the new Ruby version as the default:

rvm use <your_version> --default

5. Replace <your_version> with the version you installed.

Check if OpenSSL is linked correctly:

ruby -r openssl -e "puts OpenSSL::OPENSSL_LIBRARY_VERSION"
#=> OpenSSL 1.0.1i 6 Aug 2014

This should output the OpenSSL version like above, confirming that Ruby is now compiled with OpenSSL support.

These steps should help you resolve the OpenSSL-related issue on macOS with RVM.

Issue 2: Bundler::Fetcher::CertificateFailureError

After resolving the OpenSSL loading issue for the Ruby process, I attempted to run bundle install again. However, a new error surfaced.

Fetching gem metadata from https://rubygems.org/.......
Fetching https://github.com/kaikhq/omniauth-line.git
Fetching https://github.com/banister/binding_of_caller.git
Fetching https://github.com/excid3/convertkit-ruby.git
Fetching source index from https://gems.graphql.pro/

Retrying fetcher due to error (2/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.graphql.pro/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see https://railsapps.github.io/openssl-certificate-verify-failed.html. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Retrying fetcher due to error (3/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.graphql.pro/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see https://railsapps.github.io/openssl-certificate-verify-failed.html. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Retrying fetcher due to error (4/4): Bundler::Fetcher::CertificateFailureError Could not verify the SSL certificate for https://gems.graphql.pro/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see https://railsapps.github.io/openssl-certificate-verify-failed.html. To connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Could not verify the SSL certificate for https://gems.graphql.pro/.
There is a chance you are experiencing a man-in-the-middle attack, but most likely your system doesn't have the CA certificates needed for verification. For information about OpenSSL certificates, see https://railsapps.github.io/openssl-certificate-verify-failed.html. To
connect without using SSL, edit your Gemfile sources and change 'https' to 'http'.

Solution

This was caused by "root certificate expiration" issue（Many thanks to santoshcop for providing this command.）. You can run the command below and run bundle install again. The issue will be possible to be solved.

bundle config ssl_verify_mode 0 && echo ":ssl_verify_mode: 0" > ~/.gemrc

This command is used to configure Bundler to set the SSL verification mode to 0, effectively turning off SSL certificate verification. Additionally, it writes a configuration file (~/.gemrc) that includes the SSL verification mode setting, making it persistent for future gem-related operations in the user's environment.

To lessen security risks, I turned on SSL certificate verification after resolving this issue.

bundle config ssl_verify_mode 1 && echo ":ssl_verify_mode: 1" > ~/.gemrc

Make sure the settings are configured.

bundle config ssl_verify_mode

Settings for `ssl_verify_mode` in order of priority. The top value will be used
Set for your local app (/Users/lynnbright/code/blog/.bundle/config): 1

What is require used for in Ruby?

require is used to load external libraries or modules into your program. For example,

Before you require the JSON module

# in irb

JSON
#=> Traceback (most recent call last):
        4: from /Users/lynnbright/.rvm/rubies/ruby-2.7.8/bin/irb:23:in `<main>'
        3: from /Users/lynnbright/.rvm/rubies/ruby-2.7.8/bin/irb:23:in `load'
        2: from /Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/gems/2.7.0/gems/irb-1.2.6/exe/irb:11:in `<top (required)>'
        1: from (irb):3
NameError (uninitialized constant JSON)

After you require the JSON module

# in irb

require 'JSON'
#=> true

JSON
#=> JSON

data = '{"name": "John", "age": 30}'
#=> => "{\"name\": \"John\", \"age\": 30}"

JSON.parse(data)
#=> {"name"=>"John", "age"=>30}

What’s the relationship between require, $LOAD_PATH and $LOADED_FEATURES in Ruby?

Before the file is loaded, there are some steps that Ruby will do.

First, when we use require, it means that we ask Ruby to load the file. However, it is needless to load the same file again and again, right? This is how the $LOAD_PATH and $LOADED_FEATURES play important roles.

• $LOAD_PATH: It is a global variable used to locate the file.

• $LOADED_FEATURES: It is a global variable, too. It’s used to keep tracking of which files have been loaded successfully using require.

Search for the file

When require is called, Ruby looks for a specific file from $LOAD_PATH which contains an array of directories.

If the file is found, Ruby loads and executes it.

If the file is not found listed in the $LOAD_PATH, LoadError is raised.

# in irb

require 'abc'

#=> Traceback (most recent call last):
        6: from /Users/lynnbright/.rvm/rubies/ruby-2.7.8/bin/irb:23:in `<main>'
        5: from /Users/lynnbright/.rvm/rubies/ruby-2.7.8/bin/irb:23:in `load'
        4: from /Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/gems/2.7.0/gems/irb-1.2.6/exe/irb:11:in `<top (required)>'
        3: from (irb):9
        2: from /Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
        1: from /Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
LoadError (cannot load such file -- abc)

Check if the file already loaded:

If the file is found, before actually loading it, Ruby uses $LOADED_FEATURES, which contains an array of files, to check if the file is already included in the array.

If it is included in the array, that means the file has been loaded. If not, it will be added to $LOADED_FEATURES to prevent it from loaded again.

Let’s see a real example.

# in irb

$LOAD_PATH
#=> ["/Users/lynnbright/.rvm/gems/ruby-2.7.8/gems/timeout-0.4.1/lib", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/site_ruby/2.7.0", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/site_ruby/2.7.0/x86_64-darwin21", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/site_ruby", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/vendor_ruby/2.7.0", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/vendor_ruby/2.7.0/x86_64-darwin21", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/vendor_ruby", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/2.7.0", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/2.7.0/x86_64-darwin21"]

$LOADED_FEATURES
#=> ["enumerator.so", "thread.rb", "rational.so", "complex.so",....]

$LOADED_FEATURES.length
#=> 171

require 'json'
#=> true

# You can see that the "json" directory has been included into the $LOAD_PATH.
$LOAD_PATH
#=> ["/Users/lynnbright/.rvm/gems/ruby-2.7.8/gems/timeout-0.4.1/lib", "/Users/lynnbright/.rvm/gems/ruby-2.7.8/gems/json-2.7.1/lib", "/Users/lynnbright/.rvm/gems/ruby-2.7.8/extensions/x86_64-darwin-21/2.7.0/json-2.7.1", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/site_ruby/2.7.0", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/site_ruby/2.7.0/x86_64-darwin21", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/site_ruby", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/vendor_ruby/2.7.0", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/vendor_ruby/2.7.0/x86_64-darwin21", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/vendor_ruby", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/2.7.0", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/2.7.0/x86_64-darwin21"]

# The array elements increased from 171 to 180, indicating that new elements have been added.
$LOADED_FEATURES.length
#=> 180

$LOADED_FEATURES
#=> [...., "/Users/lynnbright/.rvm/gems/ruby-2.7.8/gems/json-2.7.1/lib/json/version.rb", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/2.7.0/ostruct/version.rb", "/Users/lynnbright/.rvm/rubies/ruby-2.7.8/lib/ruby/2.7.0/ostruct.rb", "/Users/lynnbright/.rvm/gems/ruby-2.7.8/gems/json-2.7.1/lib/json/generic_object.rb", "/Users/lynnbright/.rvm/gems/ruby-2.7.8/gems/json-2.7.1/lib/json/common.rb", "/Users/lynnbright/.rvm/gems/ruby-2.7.8/gems/json-2.7.1/lib/json/ext/parser.bundle", "/Users/lynnbright/.rvm/gems/ruby-2.7.8/gems/json-2.7.1/lib/json/ext/generator.bundle", "/Users/lynnbright/.rvm/gems/ruby-2.7.8/gems/json-2.7.1/lib/json/ext.rb", "/Users/lynnbright/.rvm/gems/ruby-2.7.8/gems/json-2.7.1/lib/json.rb"]

# Attempting to require 'json' again returns false because it is already included in $LOADED_FEATURES.
require 'json'
#=> false

In summary:

• require is used to load the specific file.

• $LOAD_PATH is used to locate where the file is.

• $LOADED_FEATURES is used to keep tracking of which files have been loaded successfully using require.

School.joins(:students).where('students.age > ?', 7).limit(100)

Recently, I was assigned to query different operation metrics, product metrics, and marketing metrics. Due to the diversity of these metrics, I found out that if I know what the raw SQL of the metrics is, not only can I complete the query quickly, but also I can use ActiveRecord::QueryMethods more efficiently.

Here are the 4 steps that how I practice writing raw SQL.

1. Browse the Active Record association declaration

Each metric must derive from one or more models. In the huge codebase, we might not be familiar with every model, only when we can clearly understand the association declaration between the models do we write the correct query condition.

2. Imitation is the start of all learning

At first, I know how to query by Rails ActiveRecord::QueryMethods, but I don't know how to write the raw SQL.

It's OK. Rails console can help us.

When asking about the result below, we can realize that it'll return an ActiveRecord::Relation object.

# rails console

Student.where(name: 'Lynn').class
#=> Student::ActiveRecord_Relation

ActiveRecord::Relation class provides a #to_sql method, it will return sql statement for the relation after it is called.

# rails console

Student.where(name: 'Lynn').to_sql
#=> "SELECT \"students\".* FROM \"students\" WHERE \"students\".\"name\" = 'Lynn'"

When we use #to_sql frequently, we can be more familiar with raw SQL.

3. Practice by Blazer gem

Using #to_sql is like a learning process of input. Output is essential if we would like to be good at writing SQL.

[Blazer](Explore your data with SQL. Easily create charts and dashboards, and share them with your team.) is a Ruby gem.

Explore your data with SQL. Easily create charts and dashboards, and share them with your team.

After installing and configuring it to the application, we can write the raw SQL directly.

Blazer does help my SQL skills a lot.

Photo from: https://github.com/ankane/blazer

4. Take a quiz from SQLZOO

SQLZOO is a SQL tutorial, which provides many query samples and quizzes that we can try. After learning by imitating and practicing writing SQL, I can finish more and more examination questions in SQLZOO now.

I have a sense of accomplishment during this learning process.

Hope you find this information helpful, also.

Here are 7 steps I usually do before I involve in a complex feature implementation.

These steps bring many benefits to the development process. For example,

• Boost the accuracy of problem-solving.

• Keep the mind clear and not get lost in the codebase.

• Improve the efficiency of the implementation.

• Communicate effectively and efficiently with other engineers.

Let's take the software onboarding for example. (source: What is SaaS onboarding?)

1. Define the problem

When new users start to use the system, they might face different problems or say frustrations.

Here are some examples:

• New users don't know what they can do after registering an account.

• New users are overwhelmed by the complicated usage of the system.

• New users don't know how many steps they should do to complete all of the required settings.

In some companies, Define the problem might not be the engineers' job. However, as an engineer, we still can collect the information from PMs or related team members.

They define the problem and we can try to understand what the problem is. This helps us get clear on what we are going to do and why solving this problem is important to the users and the company.

This step is the key point to help us build the right things.

2. Write down the user story

When we fully understand the defined problem, it's time to brainstorm the solutions with your team members. If PMs have defined the user story, we still can discuss it with PMs, especially when we think the user story can be more specific or better.

Here are some user stories examples:

• As a user, I can watch a welcome video that can tell me what I can achieve with the system.

• As a user, I can see a dashboard immediately after registering.

• As a user, I can see the dashboard which lists all of the required tasks I need to complete before launching a website.

• As a user, I can receive a newsletter that includes 3 recommended features that match my needs.

3. Write the POC

After getting enough information, we finally can start to think about how to implement the feature.

Due to the feature being complex, I usually write down the POC (Proof of Concept) first.

POC is like a blueprint. When writing the POC, you will have many questions in your mind. In order to answer these questions, one of the approaches I will do is to explore the codebase.

4. Explore the codebase

Here are some questions usually in my mind:

• How many/Which models do I need? How about the association between models?

• Does any data need to migrate before/after the feature releases?

• Will the existed users be affected after the feature release? If the answer is yes, how many things need to be done first before I implement this feature?

• If there are modules/services I can reuse?

• How many modules/services that I need to re-design?

• ...more

Exploring the codebase is a process to answer your questions, brainstorm better practices, and collect the limitation.

5. Design the code flow

After exploring the codebase, I like to make a simple code flow with Figma. This is a good way to keep the mind clear, not get lost in the codebase, and take care of different situations.

Fake code flow example

6. Communicate with PMs and other engineers

To make sure your design makes sense, the code flow source is useful, especially when you have a topic and want to discuss it with your tech leader or team members.

What's more, this can help the PR reviewers quickly understand which scope they are reviewing.

7. Break down the tasks

The code flow is like a map, it helps us break down the tasks more easily.

If a task is smaller enough, it can be completed as soon as possible. Moreover, we can help reviewers to reduce the cognitive loading.

And the final step is: Just do it.

I do not always follow these 7 steps sequentially, but these steps do improve my delivery efficiency and quality.

I hope you find this information helpful.

That’ll be ok, if we only have 100 records in the database.

However, using klass_name.all might not be the best way to get records, especially when we need to query large numbers of records.

For example, when we have 1 million records and using the query below, ActiveRecord will instantiate all the objects at once. The memory consumption will increase quickly. The worst case is the application will be unable to load any additional program.

Project.all.map { |p| p.do_something_great }

Rails provides find_each, find_in_batches, and in_batches these three public methods to work with the records in batches, which helps reduce memory consumption.

What’s the difference between the three of them? Let’s see!

find_in_batches

Generally, if we do not specify the size of the batch, the default batch size is 1,000.

For example, there are 3,000 records, No.1~1,000 records will be the first batch, then No.1001~2000 will be the second batch, and so on.

If the block isn’t given to find_in_batches, it returns an Enumerator:

Project.find_in_batches.class
#=> Enumerator < Object

Project.find_in_batches.first.class
#=> Array < Object

If the block is given...

After records of each batch finish project.do_something_great!, the type of projects will be changed to an array.

Project.where(status: 'success').find_in_batches do |projects|
  projects.each { |project| project.do_something_great! }
end

find_each

The same as find_in_batches, the default batch size is 1,000.

If the block isn’t given to find_each, it returns an Enumerator:

Project.find_each.class
#=> Enumerator < Object

Project.find_each.first.class
#=> Project < ApplicationRecord

If the block is given, it will call find_in_batches.

# File activerecord/lib/active_record/relation/batches.rb, line 68

def find_each(start: nil, finish: nil, batch_size: 1000, error_on_ignore: nil, order: :asc)
  if block_given?
    find_in_batches(start: start, finish: finish, batch_size: batch_size, error_on_ignore: error_on_ignore, order: order) do |records|
      records.each { |record| yield record }
    end
  else
    #....
  end
end

According to the source code, we can get the same result from the two queries below. So, if we would like to iterate in batches, we can use find_each as a shortcut.

Project.where(status: 'success').find_in_batches do |projects|
  projects.each { |project| project.do_something_great! }
end

Project.where(status: 'success').find_each do |project|
  project.do_something_great!
end

in_batches

The default batch size is 1,000, too.

If the block isn’t given to in_batches, it returns a BatchEnumerator.

Different from find_each and find_in_batches return Enumerator, in_batches returns BatchEnumerator, and the type of each record is an ActiveRecord_Relation object.

Project.in_batches.class
#=> ActiveRecord::Batches::BatchEnumerator < Object

Project.in_batches.first.class
#=> Project::ActiveRecord_Relation < ActiveRecord::Relation

If the block is given…

Yields ActiveRecord::Relation objects to work with a batch of records.

Project.where(status: 'success')in_batches do |projects|
  projects.update_all(status: 'draft')
end

Reference Info:

• https://api.rubyonrails.org/v6.1.3.1/classes/ActiveRecord/Batches.html

• https://makandracards.com/makandra/1181-use-find_in_batches-or-find_each-to-deal-with-many-records-efficiently

• macOS Monterey (Version 12.1)

• Ruby 2.6.6

• Ruby version manager: RVM

• Editor: VS Code

To Reproduce:

I've installed rubocop gem in my app and add an extension ruby-rubocop in my VS Code.

Open VS Code and see the warning messages below. This caused Rubocop cannot execute correctly.

/Users/lynnbright/.rvm/gems/ruby-2.6.6/bin/rubocop returns empty output! please check configuration.

env: ruby_executable_hooks: No such file or directory

How to solve it:

1. Locate Rubocup:

$ which rubocup
# /Users/lynnbright/.rvm/gems/ruby-2.6.6/bin/rubocop

2. Replace bin with wrappers.

/Users/lynnbright/.rvm/gems/ruby-2.6.6/wrappers/rubocop

3. Go to your VS Code Settings page. You can press command + ,.

4. Search rubocup or Find the tab User > Extensions > Ruby-Rubocop > Ruby › Rubocop: Execute Path.

5. Paste Rubocop’s executable path:

/Users/lynnbright/.rvm/gems/ruby-2.6.6/wrappers/

6. Restart your VS Code.

7. The warning messages might disappear and your Rubocup can execute correctly now!

Reference

• https://stackoverflow.com/questions/26247926/how-to-solve-usr-bin-env-ruby-executable-hooks-no-such-file-or-directory

Keyword: Heredoc

What is heredoc?

Heredoc is used for a form of multiline strings and preserves the line breaks and whitespace (including indentation) in the text.

How to define a heredoc in Ruby?

string = <<~HEREDOC
  How are you?
HEREDOC

• Start with <<- or <<~.

• The word HEREDOC can be replaced with any text, ex. SQL, HTMLetc.

• End with the word you have defined, ex. HEREDOC.

How to use string interpolation with a heredoc?

It's simple.

string = <<~HEREDOC
  How are you, #{User.first.name}?
HEREDOC
#=> "How are you, Lynn\n"

If you would like to disable the interpolation, you can put the single quotes around the heredoc name.

string = <<~'HEREDOC'
  How are you, #{User.first.name}?
HEREDOC
#=> "How are you, \#{User.first.name}?\n"

What's the difference between <<- and <<~?

<<-

string = <<-HEREDOC
  Today is a nice day.
HEREDOC
#=> "  Today is a nice day.\n"

<<~

string = <<~HEREDOC
  Today is a nice day.
HEREDOC
#=> "Today is a nice day.\n"

<<- will maintain the original indentation. Ruby 2.3 introduced the squiggly heredoc <<~ which can removes extra indentation.

How to remove an extra newline?

You might notice that there is a newline character \n at the end of the return value. If you would like to remove an extra newline, strip will be useful.

string = <<~HEREDOC.strip
  Today is a nice day.
HEREDOC
#=> "Today is a nice day."

Reference

• https://en.wikipedia.org/wiki/Here_document

• https://www.rubyguides.com/2018/11/ruby-heredoc

• https://blog.saeloun.com/2020/04/08/heredoc-in-ruby-and-rails

Default setting: \x off

$ psql myblog_dev

myblog_dev=# SELECT "users".* FROM "users" WHERE "users"."role" = 10 LIMIT 1;

The output format would be:

It was difficult to read the result.

Turn on the expanded table formatting mode by \x on.

$ psql myblog_dev

myblog_dev=# \x on
Expanded display is on.

myblog_dev=# SELECT "users".* FROM "users" WHERE "users"."role" = 10 LIMIT 1;

The output format would be:

It became more pretty and readable!

Switching the mode manually might be inconvenient. For example,

myblog_dev=# \x on 

myblog_dev=# select * from json_each_text('{ "name": "Lynn", "size": "S", "gender": "Female"}');

I expected to got the format like this:

  key   | value
--------+--------
 name   | Lynn
 size   | S
 gender | Female
(3 rows)

However, the format I got looked like this:

-[ RECORD 1 ]-
key   | name
value | Lynn
-[ RECORD 2 ]-
key   | size
value | S
-[ RECORD 3 ]-
key   | gender
value | Female

It's simple to print the format I expect. All I need to do is switch \x on to \x off.

After PostgreSQL 9.2, we don't have to manually switch between the modes anymore. We can use \x auto and PostgreSQL fits records to the width of the screen automatically.

\x [ on | off | auto ] Sets or toggles expanded table formatting mode. As such it is equivalent to \pset expanded. (Source)

myblog_dev=# \x auto
Expanded display is used automatically.

myblog_dev=# select * from json_each_text('{ "name": "Lynn", "size": "S", "gender": "Female"}');
  key   | value
--------+--------
 name   | Lynn
 size   | S
 gender | Female
(3 rows)


myblog_dev=# SELECT "users".* FROM "users" WHERE "users"."role" = 10 LIMIT 1;
-[ RECORD 1 ]----------+-------------------------------------------------------------
id                     | 1
email                  | my_email
encrypted_password     | $secret
reset_password_token   |
reset_password_sent_at |
remember_created_at    |
sign_in_count          | 0
current_sign_in_at     |
last_sign_in_at        |
current_sign_in_ip     |
last_sign_in_ip        |
confirmation_token     |
confirmed_at           |
confirmation_sent_at   |
unconfirmed_email      |
failed_attempts        | 0
unlock_token           |
locked_at              |
created_at             | 2021-05-24 10:55:33.368015
updated_at             | 2021-05-24 12:02:59.605434
role                   | 10
name                   |
subscribe_edm          | {}

Reference

• https://www.postgresql.org/docs/9.2/app-psql.html

• https://www.postgresql.org/docs/9.5/functions-json.html

• https://stackoverflow.com/questions/9604723/alternate-output-format-for-psql-showing-one-column-per-line-with-column-name/16108898#16108898

You can create diverse tasks, put tasks in the Rakefile, and execute a command like rake :your_awesome_task. Your task will start running by Rake.

In Ruby, you can put task code inside a file named Rakefile, rakefile, Rakefile.rb or rakefile.rb.

# rakefile.rb

desc 'Say hello'
task :say_hello do
  puts 'Hello!'
end

# In your terminal
$ rake say_hello
#=> Hello!

In Rails, you can put task code under the lib/tasks folder.

# lib/tasks/say_hello.rake

desc 'Say hello'
task say_hello: :environment do
  user = User.first
  puts "Hello, #{user.name}!"
end

# In your terminal
$ rake say_hello
#=> Hello, Lynn!

Rake is used for common administration tasks. Some commands you might be probably familiar with, such as rake db:migrate, rake generate model, rake routes. (Take a look at more rake commands.)

Rake is useful when we need to update column values in the database. There are 6 tips I often use to ensure records can be updated smoothly when running a rake task.

Tips1: Ask yourself 4 questions

1. The assumption of the column value is as your expectation?

2. What will happen if the assumption isn't true?

3. How do you know when something went wrong?

4. How can you fix errors immediately when something went wrong?

These 4 questions can help us manage risks that might happen when running a task.

Tips2: Collect failed records list

• Use ! (ex. save! , update! etc) to raise an exception if a validation error occurs.

• Rescue the error, then you can collect the failed records and failed reason.

You can understand why, how many, and which records don't update successfully by collecting a failed records list. It's usable to fix failed cases.

namespace :user do
  desc 'update name'
  task update_name: :environment do
    failed_list = []

    User.find_each do |user|
      user.update!(name: "best-#{user.name}")

    rescue StandardError => e
      failed_list.push({ user_id: user.id, reason: user.inspect })

      Rails.logger.error "[task] user_id: #{user.id} failed to update user name."
      Rails.logger.error "[task] user_id: #{user.id} failed reason: #{e.inspect}"
    end

    Rails.logger.info "[task] Failed list: #{failed_list}"
    p "[task] Failed list: #{failed_list}"
  end
end

Tips3: Record a task running duration

• Record start_at and end_at timestamp.

Sometimes the system needs downtime when we run a task. However, downtime is expensive because it will impact the revenue of the company.

Before running a task on the production environment, running a task on the staging and recording a running duration can help us estimate how much time the task will spend. This will be a piece of important information to communicate with PMs or other departments.

namespace :user do
  desc 'update name'
  task update_name: :environment do

    start_at = Time.zone.now
    Rails.logger.info "[task] Start to update users name. Time: #{start_at}"

    User.find_each do |user|
      #...
    end

    end_at = Time.zone.now
    duration = ((end_at - start_at) / 60.seconds).to_i
    Rails.logger.info "[task] All of user records update completed. Time: #{end_at}"
    p "[task] Start to update users name. Time: #{start_at}"
    p "[task] All of user records update completed. Time: #{end_at}"
    p "[task] Task running duration: #{duration} minutes"
  end
end

Tips4: Make current progress visible

Print how many records you have updated now.

If you don't print anything when you run a task, your terminal will be very silent. Printing the log can help you keep track of the current progress. I believe this is an essential user experience for developers. 😂

namespace :user do
  desc 'update name'
  task update_name: :environment do

    user_updated_count = 0

    User.find_each do |user|
      user.update!(name: "best-#{user.name}")

      user_updated_count += 1
      p "Current updated count => #{user_updated_count}"
    end

    Rails.logger.info "[task] Final: Update #{user_updated_count} user records."
    p "[task] Final: Update #{user_updated_count} user records."
  end
end

Tips5: Add a confirmation step to your task

Some tasks are destructive, so avoiding the fat-finger problem is crucial.

namespace :user do
  desc 'replace_data'
  task replace_data: [:environment, :confirm_to_replace_data] do
    #...
  end
end

desc 'confirm to replace data'
task :confirm_to_replace_data do
  confirm_token = rand(36**6).to_s(36)
  $stdout.puts "[WARNING!!] Please enter confirmation code if you confirm to replace user data: #{confirm_token}"
  input = $stdin.gets.chomp

  raise "Aborted! Confirmation code #{input} is invalid." unless input == confirm_token.to_s

  Rails.logger.info "Confirm to replace. Time: #{Time.zone.now}"
end

Tips6: Write some test cases for your task

Last but not least, remember to write tests for your task. This can ensure your assumption is as you think.

Reference

• https://guides.rubyonrails.org/v4.2/command_line.html

• https://www.rubyguides.com/2019/02/ruby-rake

• https://coderwall.com/p/_ftisa/add-a-confirmation-step-to-your-rake-tasks

• https://5xcampus.com/posts/write-rake-to-export-report-easily

• https://dwye.dev/post/rake

To prove the new optimization solution is faster, in addition to checking the before-after response time in the log file, I think I need a more efficient tool to evaluate the solution.

Benchmarking

Benchmarking is a way of measuring the performance of the code. It can answer the question like 'Which one is faster, A or B?'

Ruby has a benchmarking tool called Benchmark in its standard library. Benchmark module provides methods to measure and report the time used to execute Ruby code.

How to use Ruby Benchmark module?

You can add a Ruby file first.

$ touch test.rb

You should require 'benchmark' to make sure you can use it. Next, put the code that you would like to test into the x.report { #put your code here } block.

require 'benchmark'

n = 100
Benchmark.bm do |x|
  x.report('A') { for i in 1..n; a = '1'; end }
  x.report('B') { n.times do   ; a = '1'; end }
  x.report('C') { 1.upto(n) do ; a = '1'; end }
end

Go back to the terminal and execute the file:

We can see the C plan is faster than the A or B plan.

$ ruby test.rb
       user     system      total        real
A  0.000026   0.000005   0.000031 (  0.000024)
B  0.000018   0.000001   0.000019 (  0.000018)
C  0.000016   0.000001   0.000017 (  0.000017)

This report shows the user CPU time, system CPU time, the sum of the user and system CPU times, and the elapsed real time. The unit of time is seconds. (Source)

Another enhancement benchmarking gem - benchmark-ips

When searching the benchmarking tool, I found out another interesting gem called benchmark-ips.

Feature

An iterations per second enhancement to Benchmark.

What's the difference between the traditional Benchmark library and benchmark-ips?

Benchmark/ips will report the number of iterations per second for a given block of code. When analyzing the results, notice the percent of standard deviation which tells us how spread out our measurements are from the average. A high standard deviation could indicate the results having too much variability.

benchmark-ips uses standard deviation which is the concept of Statistics to analyze the result. The benefit is that we can literally make sure the result is significant/extreme significant/no significant difference.

In other words, take the result above for example:

C plan seems to be faster than the A or B plan, but it might not be the fact.

# n = 100
$ ruby test.rb
       user     system      total        real
A  0.000026   0.000005   0.000031 (  0.000024)
B  0.000018   0.000001   0.000019 (  0.000018)
C  0.000016   0.000001   0.000017 (  0.000017)

Let's use benchmark-ips to analyze the result:

require "benchmark/ips"

n = 100
Benchmark.ips do |x|
  x.report('A') { for i in 1..n; a = "1"; end }
  x.report('B') { n.times do  ; a = "1"; end }
  x.report('C') { 1.upto(n) do ; a = "1"; end }

  x.compare!
end

There is something interesting:

• B plan seems to iterate more times per second than the A or C plan.

Warming up --------------------------------------
                   A    11.038k i/100ms
                   B    11.868k i/100ms
                   C    12.166k i/100ms
Calculating -------------------------------------
                   A    125.949k (± 3.9%) i/s -    629.166k in   5.003441s
                   B    126.662k (± 2.7%) i/s -    640.872k in   5.063567s
                   C    125.172k (± 6.3%) i/s -    632.632k in   5.076965s

Comparison:
                   B:   126662.1 i/s
                   A:   125949.4 i/s - same-ish: difference falls within error
                   C:   125172.4 i/s - same-ish: difference falls within error

From the perspective of Statistics, there is not a meaningful difference between the three plans. When I change n=100 to n=10000, the result is the same. I feel very surprised about the result. Very. (Please correct me if I misunderstood. 🙏)

How to Benchmark the code in Rails

It's simple. Just add require_relative '../../config/environment' so that you can use the model class you have built in your Rails App.

require 'benchmark/ips'
require_relative '../../config/environment'

transaction_ids = [1, 2, #....]
transactions = Transaction.where(id: transaction_ids)

Benchmark.ips do |x|
  x.report('new') do
    transactions.map(&:some_column).compact
  end

  x.report('old') do
    transaction_records = transactions.
      includes(:a_table, :b_table, :c_table).order(created_at: :desc)

    transaction_records.map(&:some_column).compact
  end

  x.compare!
end

Reference

• https://shopify.engineering/how-fix-slow-code-ruby

• https://github.com/evanphx/benchmark-ips

• https://blog.saeloun.com/2020/02/11/rails-benchmark-generator

• https://guides.rubyonrails.org/v3.2.13/performance_testing.html#helper-methods

• https://ihower.tw/rails/performance.html

• https://www.schneems.com/2016/11/14/statistical-lit.html

Situation

I tried to add an environment variable in the application.yml file.

# config/application.yml

default:
  #...
  API_TOKEN: ABC123456

And I expected when executing ENV['API_TOKEN'] in Rails console, I would get ABC123456. But I got nil.

$ rails console
$ ENV['API_TOKEN']
#=> nil

To see if I can figure out what's going on, I tried to do something below.

Problem-solving process

• If we edit any files which were used to start the application, like configs or initializers, the application needs to be restarted. So, I thought that reload! in Rails console might reload the new code and solve this problem. However, I got the nil result the same.

• Next, I google the keyword: 'rails console nil ymal env'. Not a good keyword, but I got another important keyword: spring stop.

What's Spring?

Spring is a Rails application preloader. It speeds up development by keeping your application running in the background so you don't need to boot it every time you run a test, rake task or migration. (Source)

Spring's Feature

• Totally automatic; no need to explicitly start and stop the background process

• Reloads your application code on each run

• Restarts your application when configs / initializers / gem dependencies are changed

What was the result I got in Rails console after executing spring stop ?

I succeeded in accessing the environment variable!

# In the terminal
$ spring stop
#=> Spring stopped.

$ rails console
$ ENV['API_TOKEN']
#=> ABC123456

So, why spring stop can solve this problem?

According to the Spring's README:

But if we edit any of the files which were used to start the application (configs, initializers, your gemfile), the application needs to be fully restarted. This happens automatically.

After I change the config file, Spring should re-run. That means Spring will reload the application code.

But the result seemed not as I expected. Even though I have changed the YAML file and restarted Rails console, but Spring didn't re-run automatically. I guessed that Spring saw that my YMAL file didn't change.

So, I stopped Spring running first and then re-ran it by restarting Rails console.

We can check out the following block:

$ spring stop
#=> Spring stopped.

$ bin/spring status
#=> Spring is not running.

$ rails console
#=> ...Running via Spring preloader in process 1182
#=> Loading development environment (Rails 5.2.6)
$ ENV['API_TOKEN']
#=> ABC123456
$ exit

$ bin/spring status
#=> Spring is running:

# 1171 spring server | backme | started 14 secs ago
# 1172 ruby -I /Users/yuchu/.rvm/rubies/ruby-2.6.6/lib/ruby/site_ruby/2.6.0 -I # /Users/yuchu/.rvm/gems/ruby-2.6.6/gems/spring-2.1.1/lib -e require 'spring/application/boot'

Rails console ran via Spring preloader and Spring started 14 secs ago. Spring has reloaded the application's newest code now.

I thought that was the exact reason why the environment variable ENV['API_TOKEN'] could be accessed successfully.

Reference

• https://stackoverflow.com/questions/8686215/failing-to-access-environment-variables-within-database-yml-file

• https://ithelp.ithome.com.tw/articles/10223709

• https://github.com/rails/spring/blob/main/README.md

Article.all.order(published_at: :desc)

Let's focus on the published_at attribute now.

What will happen if one of the article records' published_at value is nil and we are going to sort the result-set in descending?

$ article1 = Article.create(title: 'A1', published_at: nil)
$ article2 = Article.create(title: 'A2', published_at: Time.zone.now)
$ article3 = Article.create(title: 'A3', published_at: Time.zone.now - 2.days)

According to the query below, we could find out the result in order was article1, article2, article3. article1 whose published_at is nil was placed at the head of the result.

# Database: PostgreSQL
$ Article.where(id: [1,2,3]).order(published_at: :desc)
  Article Load (1.9ms)  SELECT "articles".* FROM "articles" WHERE "articles"."id" IN ($1, $2, $3) ORDER BY "articles"."published_at" DESC LIMIT $4  [["id", 1], ["id", 2], ["id", 3], ["LIMIT", 11]]

 => #<ActiveRecord::Relation [
 #<Article id: 1, title: "A1", content: "", status: "publish", created_at: "2021-02-16 13:40:58", updated_at: "2021-02-28 03:35:34", published_at: nil>, 
 #<Article id: 2, title: "A2", content: "", status: "publish", created_at: "2021-02-16 14:08:34", updated_at: "2021-09-01 16:19:30", published_at: "2021-09-01 16:19:27">, 
 #<Article id: 3, title: "A3", content: "", status: "publish", created_at: "2021-02-16 14:13:15", updated_at: "2021-09-01 16:19:48", published_at: "2021-08-30 16:19:45">
 ]>

I'm curious why NULL values come first when using ORDER BY.

If you apply the ORDER BY clause to a column with NULLs, the NULL values will be placed either first or last in the result set. The output depends on the database type. (Source)

In my project I use PostgreSQL, by default, PostgreSQL considers NULL values larger than any non-NULL value.

That's why the record whose published_at == nil came first when using ORDER BY in descending order.

How can I do if I would like to "Order by Column1 if Column1 is not null, otherwise order by Column2"?

For instance, when published_at == nil , I hope to replace published_at with created_at to be ordered.

PostgreSQL provides a wonderful function - COALESCE. It will return the first non-NULL argument. Let's jump into PostgreSQL to take a look at how it executes:

$ psql postgres
$ postgres=# \c myblog_dev
$ myblog_dev=# SELECT COALESCE (1, 2);
 coalesce
----------
        1
(1 row)
#=> `1` is the first non-NULL argument 

$ myblog_dev=# SELECT COALESCE (null, 2, 1);
 coalesce
----------
        2
(1 row)
#=> `2` is the first non-NULL argument

$ myblog_dev=# SELECT COALESCE (null, null, null);
 coalesce
----------

(1 row)
#=> There is no any non-NULL argument.

The SQL statement:

SELECT * FROM articles ORDER BY COALESCE(published_at, created_at) DESC;

COALESCE can be used in ActiveRecord Query Interface, too:

Article.where(id: [1,2,3]).order('COALESCE(published_at, created_At) DESC')
#=> Article Load (1.0ms)  SELECT "articles".* FROM "articles" WHERE "articles"."id" IN ($1, $2, $3) ORDER BY COALESCE(published_at, created_At) DESC LIMIT $4  [["id", 1], ["id", 2], ["id", 3], ["LIMIT", 11]]
 => #<ActiveRecord::Relation [
 #<Article id: 2, title: "A2", content: "", status: "publish", created_at: "2021-02-16 14:08:34", updated_at: "2021-09-01 16:19:30", published_at: "2021-09-01 16:19:27">,
 #<Article id: 3, title: "A3", content: "", status: "publish", created_at: "2021-02-16 14:13:15", updated_at: "2021-09-01 16:19:48", published_at: "2021-08-30 16:19:45">, 
 #<Article id: 1, title: "A1", content: "", status: "publish", created_at: "2021-02-16 13:40:58", updated_at: "2021-02-28 03:35:34", published_at: nil>]>

Whether we use COALESCE in SQL statement or by ActiveRecord Query Interface above, we can get the sorting result as we expected!

• article2 (published_at: 2021-09-01 16:19:27) came first

• article3(published_at: 2021-08-30 16:19:45)

• article1(created_at: 2021-02-16 14:08:34) used created_at to order by

Reference

• https://www.postgresqltutorial.com/postgresql-tutorial/postgresql-coalesce

• https://learnsql.com/blog/how-to-order-rows-with-nulls

• https://stackoverflow.com/questions/11003413/order-by-column1-if-column1-is-not-null-otherwise-order-by-column2

• https://kb.objectrocket.com/postgresql/how-to-run-an-sql-file-in-postgres-846

• https://www.postgresql.org/docs/13/app-psql.html

AJAX (Asynchronous Javascript and XML) is a kind of technique that can send HTTP requests and exchange data with a server by Javascript. It's an important and practical technique to enhance the user experience, especially when we want to achieve the effect that updating the partial content and wouldn't like to reload the whole page.

How to send an AJAX request in Rails

Rails has built-in Unobtrusive JavaScript that has required in the application.js file.

//app/assets/javascripts/application.js

//= require jquery
//= require jquery_ujs

Thanks to Unobtrusive JavaScript! We can just put something like data-* on the HTML DOM and some simple dynamic things can be implemented. (Read more about the list of data attributes)

The most common one is data-confirm. After adding data-confirm like the example below, when a user clicks the Delete, a pop-up dialog with 'Are you sure to delete this event' texts will show up.

<%= link_to(
  'Delete', 
  event_path(event), 
  method: :delete, 
  data: { confirm: 'Are you sure to delete this event?' } 
 )
%>

We can implement an AJAX request by the data attributes that Unobtrusive JavaScript has provided, too. Just like the example above, we just need to add data-remote on the target HTML DOM. After that, Rails will help us use AJAX to send the HTTP request.

For instance,

<%= link_to 'Show the article', article_path(article), remote: true %>

Rails helper will generate HTML tags like:

<a href="/artciles/1" data-remote="true">an article</a>

How to test an AJAX request with RSpec

For a general GET request, the test subject might look like this:

RSpec.describe ArticlesController, type: :controller do
  describe 'GET #show' do
    subject { get :show, params: params }

    let(:params) { { id: article.id } }

    it { is_expected.to have_http_status(:ok) }
    it { is_expected.to render_template(:show) }
  end
end

To make an AJAX request to the show action using GET,

we have to add xhr: true to the subject (Rails version 5.0+):

RSpec.describe ArticlesController, type: :controller do
  describe 'GET #show' do
    subject { get :show, params: params, xhr: true }

    #...
  end
end

According to Rails ActionController::TestCase,

# rails/actionpack/lib/action_controller/test_case.rb:569

if xhr
  @request.set_header "HTTP_X_REQUESTED_WITH", "XMLHttpRequest"
  @request.fetch_header("HTTP_ACCEPT") do |k|
  @request.set_header k, [Mime[:js], Mime[:html], Mime[:xml], "text/xml", "*/*"].join(", ")
  end
end

I'm curious if it is an AJAX request now, so I try to check it out by calling request.headers["X-Requested-With"] and request.xhr?.

# rails console

  def show
=>  binding.pry
    #...
  end

 $ request.headers["X-Requested-With"]
 #=> "XMLHttpRequest"

 $ request.xhr?
 #=> 0
 # means the value of headers["X-Requested-With"] is totally match "XMLHttpRequest" string

The answer is yes! It's an AJAX request now.

The process of AJAX interacting with the server is by an XMLHttpRequest object. After setting xhr: true means the “X-Requested-With” header has contained “XMLHttpRequest” and tells the client to send the request by AJAX.

Reference

• https://developer.mozilla.org/en-US/docs/Web/Guide/AJAX/Getting_Started

• https://guides.rubyonrails.org/working_with_javascript_in_rails.html

• https://github.com/rails/jquery-ujs/wiki/Unobtrusive-scripting-support-for-jQuery-%28list-of-data-attributes

• https://ihower.tw/rails/ajax.html

• https://newbedev.com/rails-testing-xhr-with-post-data

• https://github.com/rails/rails/blob/main/actionpack/lib/action_controller/test_case.rb#L569

nil is an instance of NilClass. It's a special Ruby object used to represent the absence of any value. And it also behaves like false when used in a conditional statement.

What's more, there is only one nil object, whose object_id is always 8 (in 64-bit Ruby).

# irb
$ nil.object_id
#=> 8

Let's play with nil now.

1. nil and nil?

In Ruby, NilClass inherits from Object, and both of NilClass and Object define nil? method.

• Only the object nil responds true to nil?

• Other objects, like '' , {}, [] etc, responds false to nil?

# irb
$ nil.class
#=> NilClass < Object

$ NilClass.instance_methods.include?(:nil?)
#=> true 

$ Object.instance_methods.include?(:nil?)
#=> true

For example,

# irb
$ nil.nil?
#=> true

$ ''.nil?
#=> false

$ [].nil?
#=> false

$ {}.nil?
#=> false

2. nil can only respond to blank? in Rails.

It's common to see blank? in the codebase. However, blank? is only defined in Rails library, not in Ruby.

We can see the different results when calling nil.blank? in irb and Rails console.

For example,

# irb
$ nil.blank?
#=> Traceback (most recent call last):
#        4: from /Users/yuchu/.rvm/rubies/ruby-2.6.6/bin/irb:23:in `<main>'
#        3: from /Users/yuchu/.rvm/rubies/ruby-2.6.6/bin/irb:23:in `load'
#        2: from /Users/yuchu/.rvm/rubies/ruby-2.6.6/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
#        1: from (irb):14
#NoMethodError (undefined method `blank?' for nil:NilClass)

# rails console
$ nil.blank?
#=> true

3. nil and ! , !! operator

! and !! are the operators to make sure the values return boolean.

$ !nil 
#=> true

$ !!nil
#=> false

4. nil and Ruby if modifier

The syntax of Ruby if modifier looks like:

code if condition

if expressions are used for conditional execution. The values false and nil are false, and everything else is true.

Executes code if the conditional is true.

For example,

$ result = 'Hello'
$ note = { nickname: 'Jenny' }
$ result = 'Hello, Jane.' if nil
$ result
#=> 'Hello'

$ result = 'Hello, Jane.' if false
$ result 
#=> 'Hello'

$ result = 'Hello, Jane.' if true
$ result
#=> 'Hello, Jane.'

$ result = 'Hello, Bob.' if note[:non_existent_key]
$ result
#=> 'Hello, Jane.'


$ result = 'Hello, Jenny.' if note[:nickname]
$ result
#=> 'Hello, Jenny.'

nil seems familiar, but there is something detailed worth learning.

Organizing the note helps me to correctly understand nil. It's helpful, especially when I need to read the code fastly.

Reference

• https://www.amazon.com/Practical-Object-Oriented-Design-Ruby-Addison-Wesley/dp/0321721330

• https://www.rubyguides.com/2018/01/ruby-nil/

• https://github.com/rubinius/rubinius/blob/master/core/nil.rb

• https://www.tutorialspoint.com/ruby/ruby_if_else.htm

• https://www.tutorialspoint.com/ruby/ruby_operators.htm

There are 3 tips to explore ideas through the Rails console I learned from Practicing Rails.

1. Use app object.

Who is app?

$ app.class
#=> ActionDispatch::Integration::Session

# That means
app = ActionDispatch::Integration::Session.new(Rails.application)

According to the [source code]($ app.class #=> ActionDispatch::Integration::Session # That means app = ActionDispatch::Integration::Session.new(Rails.application)),

An instance of this class represents a set of requests and responses performed sequentially by a test process.

And ActionDispatch::Integration::Session class includes the ActionDispatch::Integration::RequestHelpers. (Read more about this module)

So, we can use app to perform HTTP requests:

$ app.get('http://localhost:3000/articles/1')
# Started GET "/articles/1" for 127.0.0.1 at 2021-07-07 15:14:08 +0800
# Processing by ArticlesController#show as HTML
#  Parameters: {"id"=>"1"}
#  Article Load (0.6ms)  SELECT "articles".* FROM "articles" WHERE "articles"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
#  Rendering articles/show.html.erb within layouts/application
#  Rendered articles/show.html.erb within layouts/application (Duration: 2.7ms | Allocations: 344)
#  Rendered layouts/_header.html.erb (Duration: 0.4ms | Allocations: 145)
#  Rendered layouts/_footer.html.erb (Duration: 0.5ms | Allocations: 91)
# Completed 200 OK in 17ms (Views: 13.3ms | ActiveRecord: 0.6ms | Allocations: 4199)

#=> 200


$ puts app.response.body.first(200)
# <!DOCTYPE html>
# <html>
#   <head>
#    <title>Lynn's Blog</title>
#    <meta name="csrf-param" content="
# => nil

What's more, we can use app to get routes.

$ article = Article.create(title: 'beautiful_article')
$ app.article_path(article)
#=> '/articles/1'

2. Use helper object to preview HTML tags or other return values.

For example,

$ helper.link_to('Hello world!', 'https://www.example.com')
#=> "<a href=\"https://www.example.com\">Hello world!</a>"

If you find out that you get the undefined method error message when trying to call a custom helper. It might mean the module hasn't been included.

$ helper.your_custom_helper_method
#=> NoMethodError: undefined method `your_custom_helper_method' for #<ActionView::Base:0x00007fe53d1cf140>

It's good timing to check config/application.rb, if the setting below is false.

config.action_controller.include_all_helpers = false

If you just want to check the return value of this custom helper method and don't want to change the setting above, we can include this helper module first:

$ include Admin::ProjectsHelper
#=> Object < BasicObject

$ helper.your_custom_helper_method
#=> 'hi, how are you?'

3. Use a special variable _.

The result of the last line we ran in the console is automatically saved in a variable named _.

For example,

$ Article.new
#=> #<Article id: nil, title: "", content: "", cover: nil, status: "draft", created_at: nil, updated_at: nil>

$ _
#=> #<Article id: nil, title: "", content: "", cover: nil, status: "draft", created_at: nil, updated_at: nil>

# Assume `title` attribute cannot be blank
$ _.valid?
#=> false

$ _
#=> false

$ _.errors.messages
#=> Traceback (most recent call last):
#       2: from (irb):46
#       1: from (irb):47:in `rescue in irb_binding'
# NoMethodError (undefined method `errors' for #<NoMethodError: undefined method `error' for false:FalseClass>)

Next time, if we forget to assign a variable, we can just pick it up to use. :P

Reference

• https://www.justinweiss.com/practicing-rails/

• https://github.com/rails/rails/blob/89361447e1491c8a756e46e94df70d82595d1534/actionpack/lib/action_dispatch/testing/integration.rb

The code might look like this:

Though these test examples seem readable, there are some shortcomings:

1. When the rule of #setup_discount was changed, we need to revise when assigns @discount test example groups of FruitContrller#index and FruitController#show at the same time. Imagine when there are 10 controllers and all of them also need to execute #setup_discount which means we have to revise the same thing 10 times. horrible...

2. It seems not DRY.

Fortunately, RSpec provides shared_examples for us to solve the duplicate test examples problem.

How to use shared_examples

Take FruitContrller#index and FruitController#show RSpec tests for example.

1. Add a new shared_examples file.

Before adding this new file, we need to check where to put this file so that it will be required automatically before running tests.

In my project, I follow the convention and put this file at:

spec/support/controllers/shared_examples/expected_discount_examples.rb

# spec/spec_helper.rb

Dir["./spec/support/**/*.rb"].sort.each { |f| require f }

2. Pick up the duplicate part.

For example,

# index request test examples
context 'when assigns @discount' do
  let(:super_market) { create(:super_market) }
  let(:user) { create(:user, special_pass_available: special_pass_available? }

  #...

  context 'when current_user owns special pass' do     
    let(:special_pass_available?) { true }
    it { expect(assigns(:discount)).to eq(0.5) }
  end

  context 'when current_user does not own any special pass' do
    let(:special_pass_available?) { false }

    it { expect(assigns(:discount)).to eq(0) }
  end
end

# show request test examples
context 'when assigns @discount' do
  let(:super_market) { create(:super_market) }
  let(:user) { create(:user, special_pass_available: special_pass_available? }

  #...

  context 'when current_user owns special pass' do        
    let(:special_pass_available?) { true }

    it { expect(assigns(:discount)).to eq(0.5) }
  end

  context 'when current_user does not own any special pass' do
    let(:special_pass_available?) { false }

    it { expect(assigns(:discount)).to eq(0) }
  end
end

3. Pick up the different parts.

For example,

# index request test examples
context 'when assigns @discount' do
  #...

  before { index_request }

  #...
end

# show request test examples
context 'when assigns @discount' do
  #...

  before { show_request }

  #...
end

4. Put the duplicate part in the shared_examples file.

RSpec.shared_examples 'an expected @fruit' do
  let(:super_market) { create(:super_market) }
  let(:user) { create(:user, special_pass_available: special_pass_available? }

  #...

  context 'when current_user owns special pass' do  
    let(:special_pass_available?) { true }

    it { expect(assigns(:discount)).to eq(0.5) }
  end

  context 'when current_user does not own any special pass' do
    let(:special_pass_available?) { false }

    it { expect(assigns(:discount)).to eq(0) }
  end
end

5. Setup request variable for the different part

RSpec.shared_examples 'an expected @fruit' do
  #...

  before { request }

  #...
end

6. Include the shared_examples into the test file.

RSpec official documentation provides four ways to include:

include_examples "name"      # include the examples in the current context
it_behaves_like "name"       # include the examples in a nested context
it_should_behave_like "name" # include the examples in a nested context
matching metadata            # include the examples in the current context

I need to pass different request actions which are declared in subject or let into the shared group, so I use it_behaves_like to include it.

<!--kg-card-begin: html--> <script src="https://gist.github.com/lynnbright/aadb8c014ce5eca30a9a2a721e3c4cab.js"></script> <!--kg-card-end: html-->

After using shared_examples, we solve two problems:

• When the rule of #setup_discount changes, all we need to do is to revise the shared_examples file and make a minor adjustment to let declaration.

• It seems DRY now.

Although include_examples didn't come in handy this time, there is something worth noting.

According to the official documentation, there is a warning message:

WARNING: When you include parameterized examples in the current context multiple times, you may override previous method definitions and last declaration wins.

To prevent this kind of subtle error a warning is emitted if you declare multiple methods with the same name in the same context. Should you get this warning the simplest solution is to replace include_examples with it_behaves_like, in this way method overriding is avoided because of the nested context created by it_behaves_like

Read the detailed code sample: here

Reference

• https://relishapp.com/rspec/rspec-core/v/3-10/docs/example-groups/shared-examples

• https://github.com/rubocop/rspec-style-guide#shared-examples

• https://www.betterspecs.org/#shared

class Book < ApplicationRecord
  #...
  validates :barcode, presence: true
end

To increase the test coverage, adding the test example is necessary:

RSpec.describe Book, type: :model do
  #...
  it { is_expected.to validate_presence_of(:barcode) }
end

Everything seems to make sense. All the test examples have passed.

However, something wrong happened after deploying to the production. Rollbar showed warnings that were related to this validation. Some Book records failed to update.

At that time, I suddenly realized I need to think more about 'adding a new validation'.

How to avoid the same situation

The first thing is to ask:

After adding this new validation, can old records pass this new validation when they need updating something?

If the answer is no, that means there are two things we need to think about:

1. When does this validation need executing? Whenever a record is updated? Or, the validation can be executed only when the column value to an old record is different from before?

2. What if the new column has to be presence: true ? My opinion is that we only validate the new record. Because neither should we ask users to update the value themselves, nor inserting the value for them.

class Book < ApplicationRecord
  #...
  validates :barcode, presence: true, if: :new_record?
end

After adding the validation, we should add the test examples to make sure the timing of the validation execution is as we expect.

Thanks for this experience to remind me:

Even though adding a validation is a simple thing, it's important to estimate if it will cause side effects.

• git cm 'some_message'

• commit successfully, but git showed the message below:

error: The last gc run reported the following. Please correct the root
cause and remove gc.log.
Automatic cleanup will not be performed until the file is removed.
warning: There are too many unreachable loose objects; run 'git prune'
to remove them.

Why git shows this error message?

You are seeing this error because you have too many dangling commits for git to automatically clean up.

What is "dangling commit" & "dangling blob"?

Dangling objects exist but they are never directly used.

• Dangling blob

A change that made it to the staging area/index but never got committed.

• Dangling commit

A commit isn’t directly linked to any child commit, branch, tag, or other references.

(Information from: Stack overflow)

What will “dangling objects” have an influence on?

Actually, the worst situation is dangling objects take up too much space of the hard disk.

How to solve the "too many dangling objects" problem?

1. git fsck

This will verify the connectivity, validity of the object and list those dangling objects if they exist.

2. git gc --prune=now

• git prune

This will remove dangling objects.

• git gc

-prune=<date> Prune loose objects older than date (default is 2 weeks ago, overridable by the config variable gc.pruneExpire). — prune=now prunes loose objects regardless of their age and increases the risk of corruption if another process is writing to the repository concurrently; see “NOTES” below. — prune is on by default.

Reference:

• https://community.perforce.com/s/article/16023

• https://www.cnblogs.com/dylancao/p/6625431.html

• https://stackoverflow.com/questions/18514659/git-what-is-a-dangling-commit-blob-and-where-do-they-come-from

• https://git-scm.com/docs/git-fsck

• https://git-scm.com/docs/git-gc