CSV Importable
Intelligently parse CSVs and display errors to your users.
Installation
Add this line to your application’s Gemfile:
gem 'csv_importable'
And then execute:
$ bundle
Usage
High level steps:
- Create an
Import
model: this model stores the file, status, and results of the import for the user to see. - Create a
RowImporter
class: this class handles the logic surrounding how one row in the CSV should be imported. - Create route(s), controller, and view(s) to allow your users to upload a CSV and return comprehensive, easy to understand error messages for your users to correct the CSV and re-upload.
Please note, it is also possible to implement an Importer
class, which handles the logic surrounding how the entire file is imported. This is not usually needed though as our default Importer
will take care of the heavy lifting.
If any errors happen during the upload process, they are recorded, stored, and ultimately displayed to the user for an opportunity to correct the CSV file.
Create Import Model
This model handles and stores the file, status, and results of the import for the user to see. By storing the file and results, we can process the import in the background when the file is too large to process real-time, and then email the user when the import is finished.
Note: if you’re not using Paperclip, don’t worry about implementing the file
field. The important thing is that you implement a read_file
method on your Import
class so we know how to get the StringIO data for your CSV file.
$ rails g model Import status:string results:text type:string file:attachment should_replace:boolean
$ bundle exec rake db:migrate
Change the Import class to look something like below:
class Import < ActiveRecord::Base
include CSVImportable::Importable
def read_file
# needs to return StringIO of file
# for paperclip, use:
# Paperclip.io_adapters.for(file).read
end
def after_async_complete
# this is an optional hook for when an async import finishes
# e.g. SiteMailer.import_completed(import).deliver_later
end
def save_to_db
save
end
def big_file_threshold
# max number of rows before processing with a background job
# super returns the default of 10
super
end
end
And then create a subclass that should correspond to the specific importing task you are implementing. For example, if you are trying to import users from a CSV, you might implement a UserImport
class which inherits from Import
:
class UserImport < Import
def row_importer_class
UserRowImporter
end
end
The only method that you need to define here is the row_importer_class
, which tells csv_importable
how to import each row in the CSV. Let’s take a look.
Create RowImporter Class
The RowImporter
class handles the logic surrounding how one row in the CSV should be imported and added to the database. You need only (1) inherit from CSVImportable::CSVImporter
and (2) implement the import_row
method.
class UserRowImporter < CSVImportable::CSVImporter
def import_row
user = User.new
user.email = pull_string('email', required: true)
user.first_name = pull_string('first_name', required: true)
user.last_name = pull_string('last_name', required: true)
user.birthdate = pull_date('birthdate') # format: YYYYMMDD
user.salary = pull_float('salary')
end
end
See that pull_string
method? Let’s talk about that for a second…
Parsers
To assist you in getting data out of your CSV, we’ve implemented some basic parsers. These parsers will grab the raw data for the particular row/column and attempt to coerce it into the correct type (e.g. take string from CSV and convert to float).
If the parser fails to coerce the data properly, it will add an error message to the array of errors that your user receives after the import runs. These errors help the user fix the import file in order to try again.
- pull_string
- pull_boolean
- pull_date
- pull_float
- pull_integer
- pull_select
Basic syntax: pull_string(column_key, args)
where column_key
is the CSV header string for the column and args
is a hash with the following defaults: { required: false }
Custom Parsers
To implement a custom
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/launchpadlab/csv_importable