importer

Importer is a tool to help you with importing objects to your database from various sources.

Links

Docs: rdoc.info/projects/szajbus/importer

Main features:

can import from XML and CSV formats, but it’s possible to add custom parsers
reports how many new objects got imported, how many objects was modified and how many objects were invalid
includes ActiveRecord, DataMapper and MongoMapper adapters

Installation

Install the gem

gem install importer

To use with Ruby < 1.9

gem install importer -v "~> 0.4.0"

Add to your model

# ActiveRecord
class Product < ActiveRecord::Base
  include Importer
end

# MongoMapper
class Product
  include MongoMapper::Document
  include Importer
end

# DataMapper
class Product
  include DataMapper::Resource
  include Importer
end

And start importing

Product.import(path_to_xml_or_csv_file)

This will parse the file and import all products there are defined in it. This will return import summary which will tell you how many products were created, modified or invalid. Exact information about each product (detected attributes and errors) will be available in summary too.

Customization

You can create your own parser to import from sources other than XML or CSV files. Check the implementation of one of existing parsers to find out how to write your own. Then just pass parser class to import method:

Product.import(path_to_file, :parser => CustomParserClass)

You can also create your custom versions of Import and ImportedObject classes. A possible alternative version could be ActiveRecord Import and ImportedObject models that would save import summary to database for later inspection. Check the rdocs for these classes for more information. You force the importer to use a custom Import class with:

Product.import(path_to_file, :import => CustomImportClass)

Updating objects

Importer is smart enough to figure out whether it has to create a new object or just update existing one during import. By default it’s tries to find existing object by detected id attribute. If the object is found Importer updates it, otherwise a new object is created. You can change the way how existing objects are searched for by overriding your model’s find_on_import class method.

class Product < ActiveRecord::Base
  include Importer

  def find_on_import(import, attributes)
    find_by_name(attributes["custom_attribute"])
  end
end

Building objects

The default way to build new objects or update existing ones is to merge their attributes with detected ones. To provide custom building logic override your model’s merge_attributes_on_import instance method.

def merge_attributes_on_import(import, attributes)
  self.attributes  = attributes
  self.imported_at = Time.now
end

Note on Patches/Pull Requests

Fork the project.
Make your feature addition or bug fix.
Add tests for it. This is important so I don’t break it in a future version unintentionally.
Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
Send me a pull request. Bonus points for topic branches.

Disclaimer

Importer gem/plugin was extracted from an actual Ruby on Rails application. It probably lacks some features or needs some polishing. Feel free to contribute.

The gem is still under development, backward compatibility can not be guaranteed (at least until it reaches 1.0 stable version).