Module: TaliaCore::DataTypes::DataLoader::ClassMethods

Included in:
FileRecord
Defined in:
lib/talia_core/data_types/data_loader.rb

Overview

The create_from_url method will create a new FileRecord from a data source (file or web URL). The exact mechanism for creating the record will depend on the MIME type of the data.

How the MIME type is determined

  • If the :mime_type options is provided, the system will always use that MIME type for the new record

  • If the :location option is provided, the system will always attempt to determine the MIME type from the “file extension” of the location, unless the :mime_type option is set

  • If the uri is a file, the system will use the file extension to determine the MIME type automatically

  • If the uri is a web URL, the system will first check if the server provided a MIME type in the response. If not, it will use the “file extension” of the uri to determine the MIME type as above

If the loader is a FileRecord class (no loader method)

If no loader method is specified, the loader will simply create a new FileRecord object of the type specified in the loader. It will then use create_from_file (for files) or create_from_data (for a web uri) to load the data into the new record.

Example:

# Set a loader for png (usually done in the initializer, 
# this one is equal to the default)
TaliaCore::DataTypes::MimeMapping.add_mapping(:png, DataTypes::ImageData)

# Call the loader
FileRecord.create_from_url('test.png')
# This will result in the following:
# DataTypes::ImageData.new.create_from_file('test.png', 'test.png')

If the loader is a method

In case a loader method is specified, the system will simply call that method on the class provided by the loader. The loader method must take the following paramters: mime_type, location, source, is_file:

  • mime_type is the MIME type for the object being imported

  • location is the location string for the current record. This is either the location passed in as an option, or the base name of the uri

  • source is either the io object from which to read the data, or a file name

  • is_file is set to true in case the source is a file name

Example:

# Set the handler for tiff files, usually done in the initializer
TaliaCore::DataTypes::MimeMapping.add_mapping(:tiff, :image_data, :create_iip)

# Call the loader
FileRecord.create_from_url('test.tiff')
# This will result in the following:
# DataTypes::ImageData.create_iip(Mime::Type.lookup(:tiff), 'test.tif', 'test.tif', true)

Instance Method Summary collapse

Instance Method Details

#create_from_url(uri, options = {}) ⇒ Object

Load data from the given url and create FileRecord objects, as appropriate.

The way the FileRecord is created is determined by the MIME type for the data. Talia has a “loader” for each MIME type - see the MimeMapping class to see the default loaders and to find out how to configure them.

Each “loader” contains the FileRecord class that is used for new records, and (optionally) the name of a loader method which creates the new records. If no loader method is provided, a default mechanism is used.

Options

mime_type

Specify the MIME type to use for the import. The parameter can either be a MIME::TYPE object, a string like ‘text/html’ or a MIME type symbol like :jpeg

http_credentials

Credentials for http authentication, if the uri requires that. These are the same options that openuri accepts, so see the documentation for that library for more information. Example: :http_credentials => { :http_basic_authentication => [login, password] }

location

The location (e.g. filename) for the new FileRecord. If a location is given, it will always be used to determine the MIME type, unless a MIME type is passed explicitly as an option



93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
 
# File 'lib/talia_core/data_types/data_loader.rb', line 93

def create_from_url(uri, options = {})
  options.to_options!
  options.assert_valid_keys(:mime_type, :location, :http_credentials)
  
  mime_type = options[:mime_type] 
  location = options[:location]
  # If a Mime type is given, use that.
  if(mime_type)
    mime_type = Mime::Type.lookup(mime_type) if(mime_type.is_a?(String))
  end

  data_records = []

  # Remove file:// from URIs to allow standard file URIs
  uri = file_url(uri)
  
  # We have diffent code paths for local and remote files. This is mainly because
  # the system will try to not open local files at all and just copy them around -
  # which will greatly speed up the operation.
  is_file = File.exist?(uri)

  location ||= File.basename(uri) if(is_file)
  # If we have a "standard" uri, we cut off at the last slash (the
  # File.basename would use the system file separator)
  location ||= uri.rindex('/') ? uri[(uri.rindex('/') + 1)..-1] : uri
  
  assit(!location.blank?)
  
  if(is_file)
    mime_type ||= mime_by_location(location)
    open_and_create(mime_type, location, uri, true)
  else
    open_from_url(uri, options[:http_credentials]) do |io|
      mime_type ||= Mime::Type.lookup(io.content_type)
      # Just in case we didn't get any content type
      mime_type ||= mime_by_location(location)
      open_and_create(mime_type, location, io, false)
    end
  end

end