rake arguments

Long ago I began to write some rake tasks, it's simple but doesn't have an instruction about how to add arguments to a rake task. What I did before is to use ruby environment variables.

task :try_argument do
  ENV['GLOBAL_ARGUMENT1'] or ENV['GLOBAL_ARGUMENT2']
end

GLOBAL_ARGUMENT1=xxx GLOBAL_ARGUMENT2=yyy rake try_argument

As you seen, I have to set the global environment variable to pass the arguement to a rake task.

But there is another way to pass the arguments to rake task via []

task :try_argument, [:key1, :key2] do |t, args|
  args.with_defaults(:key1 => value1, :key2 => value2)
  args[:key1] or args[:key2]
end

rake try_argument[xxx, yyy]

It looks like the difference between hash arguments and normal arguments.

Both of them have disadvantage:

ENV arguments also changes the system env variables normal arguments do not make sense when calling, difficult to remember the meanings of arguments.

Both work fine, it depends on you to use which one.

Posted in  rake


passenger with redis

Today I encountered an issue that passenger forks too many workers than what we set (6) on qa servers. I used strace, the passenger worker is blocked by failed to writing to a socket, like

select(15, [], [13], [], [58, 915000])

fd 13 is a socket.

I also tried netstat and found the status for some redis socket connections are CLOSE_WAIT.

So I judged this is the problem the ruby redis clients are not closed correctly. This reminds me that passenger fork() nature, I checked our source codes, unfortunately, we didn't do anything special for passenger fork.

This is the link tells you how to close the redis connection after passenger forks a worker. After deploy the new codes to qa servers, passenger never forks more workers than we expected. But the workers still hang up according strace result, that means some workers keep inactive status, they won't be able to handle any requests. Wooops...

I looked through the redis-rb source codes, we used redis 2.0.5, it didn't handle TIMEOUT error and always retry writing to redis. Fortunately, the latest redis version is 2.2.2 and it already fixed this issue, retry 3 times, if still failed, the release the connection.

Now it works fine, no unexpected additional passenger workers and no unexpected inactive workers.

Posted in  passenger redis


avoid committing git conflicts

I made a mistake when merging branch last week, I forgot to remove a conflict syntax "<<<<<< HEAD" and push it to remote repository. It breaks other one's development. So stupid to make such mistake.

To avoid making such mistake anymore, I write a git hook .git/hooks/pre-commit to check conflict syntax "<<<<<<" and ">>>>>>"

#!/usr/bin/env ruby

`git diff-index --name-status HEAD`.split("\n").each do
|status_with_filename|
  status, filename = status_with_filename.split(/\s+/)
  next if status == 'D'
  File.open(filename) do |file|
    while line = file.gets
      if line.include?("<<<<<<<") || line.include?(">>>>>>>")
        puts "ERROR: #{filename} is conflict"
        exit(1)
      end
    end
  end
end

It will prevent you from committing conflicts.

Posted in  git


after_commit

We are using RabbitMQ as our message queue system, ruby client is workling. This week we encountered a strange issue, we create a notification, and define an after_create callback to ask workling to find that notification and then push the notification to twitter or facebook, it works fine except that sometimes it will raise an error said "can't find the notification with the specified ID"

class Notification < ActiveRecord::Base
  after_create :asyns_send_notification
  ......
  def async_send_notification
    NotificationWorker.async_send_notification({:notification_id => id})
  end
end

class NotificationWorker < Workling::Base
  def send_notification(params)
    notification = Notification.find(params[:notification_id])
    ......
  end
end

It's wierd the notification_id is passed to NotificationWorker, that means the notification is already created, the notification is supposed to be existed.

After talking with MySQL DBA, we find the problem is the find sql is executed before insert transaction is committed.

Let me describe it step by step.

  1. Notification sends "Transaction Begin" command
  2. Notification sends "INSERT" command
  3. Notification gets "next sequence value" as new object id
  4. Notification sends "new object id" to NotificationWorker
  5. NotificationWorker sends "SELECT" command to find notification object
  6. Notification sends "Transaction Commit" command

As you seen, at step 5, the new notification is not existed in the mysql database yet, so the error "Not found" will be raised.

To solve this issue, we can use after_commit callback.

In rails 2.x, we should install after_commit gem, in rails 3.x, after_commit callback is supported by default.

class Notification < ActiveRecord::Base
  after_commit_on_create :asyns_send_notification
  ......
  def async_send_notification
    NotificationWorker.async_send_notification({:notification_id => id})
  end
end

So Notification asks NotificationWorker to run only after the transaction is committed.

Posted in  Rails


reset_counter in rails

I thought reset_counter method is to reset a counter_cache column to be 0, but it is not. After trying several times, I finally realize that reset_counter is to update the value of counter_cache column to the exact count of associations. The usecase of reset_counter is when you add the counter_cache in migration and update the counter_cache value, like

def self.up
  add_column :posts, :comments_count
  Post.all.each do |post|
    Post.reset_counter(post.id, :comments)
  end
end

it will add comments_count column to posts table, and calculate the comments count for each post, and set it to posts' comments_count column.

I didn't find a method to reset the counter_cache column to be 0, why? Because counter_cache is used to cache the association count, it will be incremented and decremeneted automatically, you should never reset it 0. If you find you need to reset counter_cache to 0, that means it's a wrong usage of counter_cache.

Posted in  Rails


Fork me on GitHub