Ewout blogt.

Devise is becoming a popular gem for adding modular authentication to a rails application. It builds on top of warden, which provides a pluggable architecture for multiple authentication strategies at the rack level.

Since the cool kids are using it, we did not want to be left out and ported our main application from restful_authentication. Aside from being cool, the switch will make porting to rails 3 easier, the latest devise is compatible.

The application uses basic http authentication for private RSS feeds and ical subscriptions. This is pretty common at the service level of an application, machines do not like login forms. Devise works with basic authentication out of the box, but it will only work when the authentication headers are already present in the request. When they are not, devise will return a 302 redirect to the login form and the RSS reader gives up.

The solution is to create a new devise strategy  in config/initializers/devise.rb

class HttpAuthenticatableNonHtml < Devise::Strategies::HttpAuthenticatable
  def valid?
    not request_format.html? or super
  end
  def http_authentication
    super or ''
  end
end
Warden::Strategies.add(:http_auth_non_html, HttpAuthenticatableNonHtml)

Warden needs to be instructed to use the strategy, inside the Devise.config block.

config.warden do |manager|
  manager.default_strategies.unshift :http_auth_non_html
end

The strategy will return a 401 with authentication realm when accessing a protected resource that is not html.

How warden and devise work in rails

Figuring out this solution required diving into the warden an devise code, which is quite intimidating at first. I created a diagram that hopefully makes it easier to understand the basic working of the authentication stack.

1. The HTTP request enters the rack stack.
2. Warden gets the request and forwards it in the rack stack, adding an environment variable “warden” that points to an authentication proxy.
3. The request gets dispatched to the rails controller, which may call authenticate_user! from a filter. This is an alias for request.env['warden'].authenticate!(:scope => :user).
4. The warden proxy picks an authentication strategy. Any strategy for which valid? returns true is tried.
5. When authentication succeeds, a user object is returned to the controller. When it fails, the symbol :warden is thrown down the stack, and caught by the warden rack application. The latter will return a response, which is a redirect to the login page by default. This can be overridden by calling warden.custom_response!.

A while ago, a client asked us for a way to find and remove duplicate companies from his database. The mysql database behind the rails application contained over 10,000 companies, each having related contacts, phone numbers, email addresses, notes, … The data was imported from multiple sources and inevitably contained a lot of duplicates. We agreed on a two-fold solution

• Automatically merge companies that match certain conditions (same name, legal form, vat no). This would remove 90% of the duplicates without human intervention.
• Construct an interface to find companies with one or more fields in common (name, email, phone, www, …) and merge them set by set. The remaining 10% could be merged under supervision of the user this way.

The core implementation problem was merging together the company records. Even though the client talked about removing duplicates, he did not want to lose any related information on a duplicate. We had a feeling the client would soon ask for doing the same with other record types and therefore decided to implement it generically. The code is on github.

Merging attributes

Since a table row can only have 1 value for each column, some attributes of duplicate objects will need to be discarded. To control the attribute values on the merged object, the objects to merge need to be ordered. The first object gets priority, when it contains blank attributes, they can be looked up on the remaining objects in order.

Merging associations

belongs_to

Belongs to associations are backed with a foreign key attribute. When merging the attributes, belongs_to associations are already covered.

has_one

Suppose we are merging company A and company B and Company has_one :financial_info.

• Financial info present on company A and not on B => use the one from company A
• Financial info present on company B and not on A => use the one from company B
• Financial info present on both => merge the two

has_many, has_and_belongs_to_many

When merging company A with 2 phone numbers and company B with 1 phone number, the resulting company should have all 3 phone numbers. That is, if the phone number of company B is not already on company A as well. Associated objects should be compared and duplicates merged recursively. Comparison may differ for some models. Phone numbers are best compared by flattening them into a string with only numbers, this way the separators do not mess up the comparison.

has_many :through

These associations can be left alone, since they depend on another has_many association that can be merged.

The API

company.merge!(duplicate1, duplicate2, ...)

The object on which merge is called becomes the master object, duplicates are merged into this object and destroyed afterwards. The order in which the duplicates are passed to the merge function matters, since this will determine the priority for merging attributes. When the attribute alpha_code is nil on the master, it will get the value on duplicate1. When not present on duplicate1, it will get the value of duplicate2, and so on.

Hooks

merge_equal?(object)

Compares self with the object and returns true if they can be considered the same. When records with a has_many association are merged, associated objects are compared and duplicates are destroyed.

merge_attribute_names

The names of the attributes that should be merged. Defaults to all attributes minus id, timestamps and other meta data.

merge_exclude_associations

The names of associations that should not be merged. Can be used to exclude irrelevant or duplicate associations.

Known issues

Currently, the merge algorithm does not take cycles in the associations into account. Since the reverse belongs_to associations are never considered, this should not be a problem for most ActiveRecord models. An infinite loop may occur when Company has_many :companies and a company points to itself.

To make sure the merge does not leave invalid foreign keys, referential integrity can be used on the database.