everburning

We’ve been talking a bit about code reviews at work, I do a bit of this now by reading commit emails, and have been wondering if there was an easier solution. To that end, I started looking around for software that could help us out.

I ended up beating on Review Board for about six hours before getting it installed. Now that it’s running it looks pretty nice. I think we’ll end up doing more post-commits then pre-commits but hopefully it’ll fit into our workflow.

Anyway, I though I’d put up a quick post on what I needed to do it get it running. The docs do a good job of getting the base install done. So give that a gander but come back here before you start running rb-site install.

We need to setup our MySQL database before running the rb-site. I did this by initializing a reviewboard MySQL database with a reviewboard user. I then granted all priviledges by executing:

GRANT ALL PRIVILEGES ON reviewboard.* TO 'reviewboard'@'localhost'
            IDENTIFIED BY 'some_pass' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON reviewboard.* TO 'reviewboard'@'% '
            IDENTIFIED BY 'some_pass' WITH GRANT OPTION;

Once rb-site install is run you’ll need to configure your webserver. In my case I ended up using Apache with FastCGI. When I tried to use the mod_python option the webserver would get a Segmentation Violation and terminate. Not so good.

The provided Apache FastCGI configuration script didn’t work for me and I ended up using the following.

AddHandler fastcgi-script fcgi

FastCGIExternalServer "/var/www/reviews/htdocs/reviewboard.fcgi" -host 127.0.0.1:3033 -idle-timeout 60

<VirtualHost *:8888>
    ServerName reviews.local
    DocumentRoot /var/www/reviews/htdocs

    # Alias static media requests to filesystem
    Alias /media /var/www/reviews/htdocs/media
    Alias /errordocs /var/www/reviews/htdocs/errordocs

    # Error handlers
    ErrorDocument 500 /errordocs/500.html

    # Direct all other requests to the fastcgi server
    RewriteEngine on
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^/(.*)$ /reviewboard.fcgi/$1 [QSA,L]
</VirtualHost>

The first parameter to FastCGIExternalServer needs to be the full path to reviewboard.fcgi as if it existed in your DocumentRoot. Nothing else seemed to work for me.

Note, if you’re using VirtualHosts you’ll also need to make sure the NamedVirtualHost option is enabled in your httpd.conf file (or whatever your main Apache config file is named).

Now, this will run, but it won’t work. The reason it won’t work is that it needs an external server to send the FastCGI requests too. You can run this server by executing: rb-site manage /var/www/reviews/ runfcgi method=threaded port=3033 host=127.0.0.1 protocol=fcgi. You’ll notice the host and port match up to those specified in the Apache config file.

I ended up creating a simple shell script to handle starting and stopping the rb-site server.

#! /bin/sh

# chkconfig: 2345 90 90

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DESC="reviewboard daemon"
NAME=reviewboard

case "$1" in
  start)
    echo -n "Starting $DESC: $NAME"
    /usr/bin/rb-site manage /var/www/reviews/ runfcgi method=threaded port=3033 host=127.0.0.1 protocol=fcgi
    echo "."
    ;;
  stop)
    echo -n "Stopping $DESC: $NAME"
    pkill rb-site
    echo "."
    ;;
  *)
    echo "Usage: NAME {start|stop}" >&2
    exit 3
    ;;
esac

exit 0

With that I was able to login and start playing with Review Board. I ended up doing a bit of extra work to disable registration (which is kind of nasty as I had to edit the base HTML templates and urls.py) but everything seems to be working well now.

Owen has finally published some screenshots and copy for the game he’s been working on. If you’ve got an iPhone or iPod Touch you should head over and check out Dapple from Streaming Colour Studios.

You should also buy it when it comes out, else I’ll have to come find it. I know people who own big sticks….

(But then, the game stands on its own. Owen sent me a prototype that was a lot of fun to poke around. The new version looks like it’s miles ahead of what I played so I’m looking forward to the release.)

We’ve started using RabbitMQ at work for some of our messaging needs. We’ve been using the Ruby AMQP library for all of our communication with Rabbit so far. So, I thought I’d write up a quick article on how to use the Ruby AMQP library.

I’ll be showing some code from a fork of the Ruby AMQP code that adds a bit of reconnect logic. I’ll call out this difference when we get there.

The Ruby AMQP bindings are written using the EventMachine framework so we’ll be working under that for this article.

We’re going to look at a couple simple examples, adding to an exchange and draining messages off an exchange.

First, a quick note on AMQP terminology, this isn’t complete but should cover us for our purpose. When you publish a message to an AMQP server (I’m just going to use Rabbit from now on) it gets published onto an exchange. A client who wishes to listen to the exchange attaches a queue to the exchange. Queues and exchanges can be set as durable which means they’ll survive a restart of Rabbit. Messages can be marked as persistent which means they’ll sit on the queues attached to the exchange until the consumer picks them up. If a message isn’t persistent it will just disappear from the queue if there is no client listening.

The first step is setting up our Rabbit instance. I’m not going to explain this stuff, the Rabbit documentation can do a better job then I can.

[dj2@titania ~]# rabbitmqctl add_vhost /my_vhost
Creating vhost "/my_vhost" ...done.

[dj2@titania ~]# rabbitmqctl add_user dj2 secret_password
Creating user "dj2" ...done.
[dj2@titania ~]# rabbitmqctl add_user reader some_pass
Creating user "reader" ...done.

[dj2@titania ~]# rabbitmqctl map_user_vhost dj2 /my_vhost
Mapping user "dj2" to vhost "/my_vhost" ...done.
[dj2@titania ~]# rabbitmqctl map_user_vhost reader /my_vhost
Mapping user "reader" to vhost "/my_vhost" ...done.

[dj2@titania ~]# rabbitmqctl set_permissions dj2 /my_vhost /data all
Setting permissions for user "dj2", vhost "/my_vhost", realm "/data" ...done.
[dj2@titania ~]# rabbitmqctl set_permissions reader /my_vhost /data all
Setting permissions for user "reader", vhost "/my_vhost", realm "/data" ...done.

With that out of the way, lets move to something a bit more interesting.

#!/usr/local/bin/ruby

require 'rubygems'
require 'mq'

EM.run do
  AMQP.connect(:user => 'dj2', :pass => 'secret_password',
               :host => 'localhost', :vhost => '/my_vhost') do |conn|
    @connection = conn
    channel = MQ.new(@connection)
    @xchange = channel.fanout('my_exchange', :durable => true)
  end

  EM.add_timer(1) do
    @xchange.publish("data to publish")
    EM.add_timer(1) { EM.stop_event_loop }
  end
end

As I mentioned, everything runs under EventMachine, so our main code is wrapped inside an EM#run block. The first thing we’re doing is connecting to Rabbit. This is done with AMQP#connect. In the official Ruby AMQP code you don’t provide a block to AMQP#connect it returns the connection object. If you’re using the reconnect fork you pass a block to AMQP#connect that will be executed when we reconnect to the server (it also returns the connection object if you want).

Once we’ve got the connection we need to create the channel that we’re going to work with. This is done with MQ#new. With the channel in hand we can create our exchange. In this case we’re creating a fanout exchange. The exchange will be named my_exchange and will be durable, so the exchange will survive a Rabbit restart.

I’m setting up an EventMachine timer to publish my message. This gives everything a few seconds to startup and connect to Rabbit. We then use the publish method to send the message to the exchange.

We kill off EventMachine a second later so the application will exit.

#!/usr/local/bin/ruby

require 'rubygems'
require 'mq'

EM.run do
  trap("INT") { EM.stop }
  trap("TERM") { EM.stop }

  AMQP.connect(:user => 'reader', :pass => 'some_pass',
               :host => 'localhost', :vhost => '/my_vhost') do |conn|
    @connection = conn
    channel = MQ.new(@connection)

    xchange = channel.fanout('my_exchange', :durable => false)
    q = MQ::Queue.new(channel, 'my_queue', :durable => false)
    q.bind(xchange)

    q.subscribe do |header, msg|
      puts "GOT #{msg}"
    end
  end
end

You’ll notice the similarities between this code and the last block of code. Once we’ve created our exchange we create a queue on the exchange. This is done using MQ::Queue#new We give the queue a unique name and set it to a non-durable. We then bind the queue to the exchange. With the queue bound we use the subscribe method to listen on the queue. When messages arrive the code block will be executed with the header and message provided.

That’s it. It’s all pretty simple. Well, at least in these examples it is. This is just the tip of the AMQP iceberg but hopefully you’ve got enough information to start digging in.

Have fun.

The MountainWest Ruby Conf 2008 had an interesting talk on Ruby Internals by Patrick Farley . If you’ve worked with Ruby and are interested in learning more about classes, meta-classes and how all the method dispatch magic works give it a gander.

Well worth the 50 minutes of time.

So, I’ve been working at integrating some more of our test code into our automated build system at work and have spent a day trying to figure out why everything was failing with the tests. They ran fine in my lab, they’d run fine individually but when I used rake to run them everything would bomb.

Eventually, I found out that all the rspec tests were being run twice. The first set of tests all passed the second set all failed. After a while I also realized the second set of tests were being run in the background. Odd, very odd.

Fast forward another hour, a lot of digging in the rspec code and I realized it’s because we fork inside the spec files. We fork and execute the server we want to test. rSpec sets up an at_exit handler to run the specs at exit if they haven’t been run. That at_exit handler would run number of forks + 1 times for each spec file.

I finally found Don’t get forked by rSpec, setup the at_exit { exit! } code in my spec files and everything is happy again.

Wheeee…