Ewout blogt.

Web development, ruby, and rails

Translating shared attributes

ActiveRecord can translate the attribute names of a model when the translation is in a predefined path in the locale file. This is more powerful than I18n.translate, since functionality like form helpers can build on top of that. However, there are attribute names that are shared between multiple or all models. Translating them over and over again for every model is not DRY. One solution:

en:
  activerecord:
    attributes:
      shared:
        created_at: Date created
        updated_at: Date updated

Given this locale file, we can find the attribute name of a model, even if it is shared:

Person.human_attribute_name(:created_at, :default => :'shared.created_at')

This requires the caller to know about the shared attributes and violates encapsulation. Instead, ActiveRecord can be extended to look for shared attributes.

# Allow shared attributes to be defined in the locale file.
# active_record > attributes > shared
class ActiveRecord::Base
  class << self
    def human_attribute_name_with_default(attribute_key_name, options = {})
      (options[:default] ||= []) << :"shared.#{attribute_key_name}"
      human_attribute_name_without_default(attribute_key_name, options)
    end

    alias :human_attribute_name_without_default :human_attribute_name
    alias :human_attribute_name :human_attribute_name_with_default
  end
end
Fork me on GitHub

Localized error feedback in rails

Sometimes, something goes wrong in an application. Showing a general error message to the user, even if it is really pretty, is like showing your middle finger. The general goal is to avoid errors, and when they do occur tell the user what is going on and give him a way to resolve the problem. A classic example is validation errors. Feedback for this class of errors is straightforward: mark the fields that have problems and explain what is wrong. Another example is actions which require the user to have certain permissions. Permission errors can generally be avoided by not including actions in the view for which the user does not have permission.

However, there are errors that cannot be avoided or handled by the user. Sticking with the permissions example: suppose a user has permission to perform action A. The page is loaded and shows a link to action A. While the user is filling out some data on the page, an administrator revokes the user’s permission to perform action A. When the user presses the button for the action now, the controller will raise an error. The only sane feedback for this error is informing the user that he no longer has permission to perform the action.

This post is about the kind of errors that cannot be handled. Speaking HTTP, it’s about error 422.

The goal

  • Define exceptions that can be propagated to the user
  • Explain each of these exceptions in every language the application supports
  • Render the errors in a generic, DRY way, regardless of the request type (html or ajax)

There is a minimal example project on github that implements the idea.

Define the exception

DRY error handling means not having to rescue every possible exception in every action. Rails provides a controller class method rescue_from that avoids that repetition.

rescue_from(LockedError) { render_error :locked }

Exceptions that occur in multiple controllers can be defined in ApplicationController.

Explain the exception

In the locale files, the error can be explained in detail.

en:
  custom_errors:
    locked:
       title: Locked
       message: Could not complete the operation because the item is locked.

Render the exception

A single method becomes responsible for rendering errors to the user. For html requests, a template is rendered with local variables title and message. For ajax requests, the error message is returned in json format. When the site is also accessible as a web service, this method may need to return errors in different formats based on the requested mime type.

def render_error(scope, status=422)
  error = {
    :title => t(:title, :scope => [:custom_errors, *scope]),
    :message => t(:message, :scope => [:custom_errors, *scope])
  }

  if request.format.html?
    render :template => 'application/error', :locals => error, :status => status
  else
    render :json => {:error => error}, :status => status
  end
end

Growl and ajax errors

growl_error

Growl is a javascript library that displays notices and error messages way fancier then alert().

Growl.Bezel({
  title: 'Locked',
  text: 'Could not complete the operation because the item is locked',
  image: '/images/growl_warning.png'})

In this example, growl will be responsible for rendering errors that occur during an ajax request. It can be swapped out with any other notification method, the important thing here is there is one central piece of code on the client side responsible for showing errors to the user as they occur.

Ajax.Responders.register({
  onComplete: function(request, transport, json) {
    if(!request.success()) {

      var errorMessage = ['Unknown Error', 'An error occurred, but could not be determined correctly.'];

      if (transport.responseJSON && transport.responseJSON.error)
        errorMessage = [transport.responseJSON.error.title, transport.responseJSON.error.message]

      Growl.Bezel({
        title: errorMessage[0],
        text: errorMessage[1],
        image: '/images/growl_warning.png',
        duration: 2
      });
    }
  }
});

Conclusion

The idea presented here may seem dead simple, and it is. Yet, these few lines of code have saved me a lot of time worrying about errors that rarely occur.

Running a rails app with MRI and jRuby

jRuby has become a stable alternative for running rails applications. It gives the ruby programmer access to the vast amount of java libraries that are available. One of our applications uses jRuby for its import tasks, so it can use the power of JDBC to connect to a multitude of data sources, in our case Filemaker Pro.

One of the annoying things about switching between MRI and jRuby is the database configuration. jRuby uses the ‘jdbcmysql’ adapter while MRI uses the ‘mysql’ adapter. Turns out there is a simple fix for this:

adapter: <%= defined?(JRuby) ? 'jdbcmysql' : 'mysql' %>

The database.yml file is evaluated by erb before being parsed by yaml.

Foreign key migrations: hands on

In the previous article, I explained the benefits of backing a rails application with database constraints. Referential integrity was mentioned, along with some general gotchas. In this post, we’ll apply referential integrity to an existing rails application, using the foreign key migrations plugin. The sample project is on github.

The reason we use that plugin is it will automatically add the required foreign key constraints to associations named by the rails conventions. Add a column person_id to the comments table, and the plugin will assume you are referring to an id in the people table. Of course, the default behaviour can be overridden for foreign keys named differently. Convention over configuration.

Setup

Install foreign key migrations like any other plugin, it requires the redhillonrails_core plugin though.

./script/plugin install git://github.com/devwout/redhillonrails_core.git
./script/plugin install git://github.com/devwout/foreign_key_migrations.git

Create initial foreign key migration

./script/generate foreign_key_migration
rake db:migrate

When all foreign keys are named by the rails conventions, the migration should run without errors and result in a database with the proper foreign key constraints. Using sequel pro, it is easy to verify the presence of foreign key constraints: there should be an arrow next to each cell that refers to a related row.

Foreign keys named differently from the table they refer to may yield an error similar to this:

Mysql::Error: Can't create table './foreign_key_migrations_example/#sql-2ea7_163.frm' (errno: 150):
  ALTER TABLE comments ADD FOREIGN KEY (author_id) REFERENCES authors (id)

The error can be resolved by editing the generated migration, pointing it to the right table:

class CreateForeignKeys < ActiveRecord::Migration
  def self.up
    # next line was modified: the "authors" table does not exist
    add_foreign_key "comments", ["author_id"], "people", [:id]
    add_foreign_key "users", ["person_id"], "people", [:id]
  end
end

Disable foreign keys on older migrations

Now all foreign key constraints are created for migrations older then the foreign key migration. When the database is cleared and db:migrate is run again, the foreign key migration plugin will add foreign keys twice: once during the old migrations and once with the foreign key migration. To avoid that, we need to disable the plugin for all migrations older then the foreign key migration. Create a new file in config/initializers and add the following. The long number is the prefix of the foreign key migration itself.

class ActiveRecord::Base
  def self.foreign_key_migrations_enabled
    ActiveRecord::Migrator.current_version >= 20091220125415
  end
end

Test

Now would be a good time to run the test suite of the application and see if everything still passes. The redhill plugin overrides the schema_dumper, so the foreign key constraints are cloned to the test database when using ruby schema format, which is the default.

Migrate existing deployments

The addition of foreign key constraints to a database may yield errors: some of the constraints may already be violated. Therefore, it is a good idea to clone the production database on a test machine first and run the migration there. When errors are found, they can be fixed using SQL. Those SQL statements can then be applied to the production database, where after cap deploy:migrations should run without errors.

Forget about foreign key constraints

From now on, it is safe to forget about foreign key constraints during application development. When adding or modifying migrations, the plugin will add the foreign key constraints silently… until an error occurs.

Error in migration

Mysql::Error: Can't create table './foreign_key_migrations_example/posts.frm' (errno: 150):
  CREATE TABLE `posts` (
    `id` int(11) DEFAULT NULL auto_increment PRIMARY KEY,
    `body` text,
    `author_id` int(11),
    `created_at` datetime,
    `updated_at` datetime,
    FOREIGN KEY (author_id) REFERENCES authors (id))

This is a typical error that occurs when a new migration is added with nonstandard foreign key. It can be resolved by explicitly adding the referenced table to the migration.

create_table :posts do |t|
  t.text :body
  t.belongs_to :author, :references => :people
  t.timestamps
end

Error in application

ActiveRecord::StatementInvalid: Mysql::Error: Cannot delete or update a parent row:
a foreign key constraint fails
  (`foreign_key_migrations_example/posts`, CONSTRAINT `posts_ibfk_1`
   FOREIGN KEY (`author_id`) REFERENCES `people` (`id`)):
  DELETE FROM `people` WHERE `id` = 1

Here, the database is acting as a safety net. We just tried to delete a person, which still has an associated post. If the database would have let this happen, there would now be a post with nonexistent author. The solution is simple: when deleting a user, his posts should be deleted or orphaned. This behaviour can be specified in rails, something we should have done in the first place.

class Person < ActiveRecord::Base
  has_many :posts, :foreign_key => :author_id, :dependent => :destroy # or :nullify
end

Conclusion

Adding foreign key constraints to a rails application is quite easy using the foreign key migrations plugin. Once installed, it will mostly just work and provide an extra layer of protection for the data.

Rails models with teeth and database constraints

One of the things I do not like about ruby on rails is the arrogance it came with. Of course, creating a blog application from scratch in 15 minutes was cool at the time, but that does not mean the author  of the framework holds all truth. As a computer scientist I appreciate the ruby language and the elegant rails API. Writing tests for your code is a best practice that rails probably introduced to a lot of people. What I do not understand is that rails rejected a best practice that is quite old and proven: database integrity.

There has been quite some discussion in the rails community about enforcing integrity at the database level. To me, the discussion is simple: anything that helps me find or prevent bugs is gold. But even if your code is free of bugs, a ruby on rails database in production will become inconsistent given enough time. This is not rocket science: if there are 10′s or even 100′s of application servers using the database concurrently without knowing about each other, something will go wrong sometimes. At that point I’d rather have an ActiveRecord::StatementInvalid exception raised then letting the problem silently corrupt our customer’s data.

The rails argument against enforcing database integrity is that business logic should be DRY and not both in the ruby class and the database. With migrations, rails is violating its own fundamentals:

  • attributes are defined outside of the ruby class
  • associations are defined twice, once with belongs_to & co macros and once with a foreign key in the database

The attributes of a model and their data types are a core part of the model logic. One can also see these as constraints: you cannot store a string in a date field. Some constraints make sense at the database level, some make more sense at the application level. This article explains my as-dry-and-consistent-as-possible use of database constraints.

:null => false

One of the simplest constraints at the database level is the requirement for a value to be present. In the migrations, this can be specified with the :null => false parameter. One would think it is a replacement for validates_presence_of, which is wrong especially when using a mysql database not in strict mode. In general we want the user-facing validation in ActiveRecord, because then we get the nice validation errors. However, there are cases where the database constraint is more appropriate.

Required (parent) associations

Suppose we are building an invoicing application and we want to make sure a line item cannot exist without an invoice. LineItem.validates_presence_of :invoice will not do the trick. When saving a new invoice with a few new line items in it, the validation of these new line items will fail because the invoice is not present. Of course the invoice_id cannot be set because it is not saved yet.

Solution: t.belongs_to :invoice, :null => false

Boolean attributes

To stick with the invoice application, an invoice could have a boolean attribute “paid”. The boolean attribute can be 0, 1 or NULL. If NULL means unspecified that is fine, but in this case we want the invoice to be either paid or not paid. When creating an invoice it will generally not be paid, so we could interpret NULL as not paid. However, Invoice.all(:conditions => {:paid => false}) will not return the invoices that have a NULL value for paid, which can lead to subtle bugs.

Solution: t.boolean :paid, :null => false, :default => false

Uniqueness

Validates_uniqueness_of is not safe for concurrent updates. This is well documented in the rails API, where one of the suggestions is to use a unique index. No need to be redundant here. However, there is a hidden use case for unique indexes: one-to-one associations.

has_one

Suppose we have a Person model to store personal information and contact details. There is also a User model for authenticating users of the application. As good citizens of DRY-land, we use the Person model to keep the personal information about our users, where User.belongs_to :person and Person.has_one :user. Nothing is stopping the application now from creating two user accounts for the same person. One to one associations should be backed with unique indexes.

Solution: t.index :person_id, :unique => true

Referential integrity

While the data type of each attribute is defined in the migrations, foreign keys just get the integer data type. Rails made it look prettier: t.belongs_to :person, but in reality this is just syntactic sugar for t.integer :person_id. There is nothing that stops this integer from pointing to a row (object) that does not exist. Referential integrity makes the database aware of the associations and is just a natural extension of the data types. (As a bonus, you get these nice arrows next to a foreign key in sequel pro, which really ease table navigation while debugging.)

There are a number of rails plugins that help define foreign key constraints in the migrations. Foreigner and migration helpers both require you to explicitly state each foreign key constraint to be added. Sexy migrations looks nice but is a bit dated and I could not check out the repository. We use a patched version of the redhill foreign key migrations plugin. The advantage of this plugin is foreign key constraints are automagically added to the database when using the proper rails conventions. However, we encountered some issues when implementing foreign key constraints in our application.

:dependent => :destroy

Foreign key constraints will by default prevent a record from being removed when there are references to it. Deleting a record will then raise an error. It is therefore important to specify what should happen with the associated records when a record is destroyed. This can be done with the :dependent option for associations in ActiveRecord or in the database itself. We prefer to keep this logic in rails and use the database as a safety net: when rails fails to destroy a child record when destroying a parent record, the referential integrity will be violated and the database will complain.

parent_id

Suppose we are storing a tree of categories in a table. When removing a number of categories from the table in a single query, mysql determines the order in which they are removed. This may result in a violated foreign key constraint when a parent is deleted before one of its children. Postgresql has the option of adding foreign keys that are only checked at the end of a transaction, which would prevent this kind of problem. In mysql, the solution is to nullify the child foreign keys when deleting a parent.

Solution: t.belongs_to :parent, :on_delete => :set_null

Performance

There is overhead associated with checking the referential integrity of a database. However, we did not notice any performance decline in our application. One thing to watch though: mysql automatically creates indexes for all foreign keys. We use userstamp, which adds two foreign keys creator_id and updater_id to every record. Since we never delete users from our system (we merely deactivate them), it seemed overkill to add two indexes to every table just for the userstamps.

Conclusion

Data tends to outlive its applications. A tight data model is a good foundation for an application and can save you a lot of trouble when migrating the data to a different system (years later). Database constraints can make your models even tighter, and enforce integrity rules that are hard to enforce in a multi-process application environment. Don’t be afraid, use them!

Accumulated wealth

Every generation can build upon the wealth accumulated by previous generations. Using the tools of our forefathers, we create new things and invent the tools for the next generation. While “tool” refers to a material entity, it is  the knowledge to manipulate our environment that made us humans who we are. Knowledge has always been important and we are accumulating it faster then ever in the information age. The invention of the internet was not a coincidence, it was a necessity. We simply needed a tool to search and share more knowledge faster.

The internet has moved the right to publish from the happy few to the crowd. It spawned blogs, wikis, fora, open-source collaboration. I’ve been a ruby developer for 4 years now and couldn’t even begin to describe how much steeper the learning curve would have been without all the free information available on the internet. Now it is time to give something back, to contribute to the wisdom of the crowd. Let’s blog and hope at least a few people get some value out of it.