Module: TaliaCore::DataTypes::FileStore

Included in:
FileRecord
Defined in:
lib/talia_core/data_types/file_store.rb

Overview

The “hevy lifting” for FileRecords, handling the actual creation and manipulation of the files.

A record can be created with create_from_file (passing a filename) or with create_from_data, passing a byte array.

While a “location” can be passed to the record (which will be save in the database), the actual file name will be determined by the system. Also see the PathHelpers for info

The base directory for file storage can be configured in the talia_core.yml file as “data_directory_location”. Otherwise, RAILS_ROOT/data. Inside, each record will be stored at “<data class of the record>/<last three numbers of the id>/<record id>”

So, for example, a “XmlData” record with the id “123456” would be stored as “DATA_ROOT/XmlData/456/123456”

Creating new records

When creating a new record using create_from_data, the data will be cached inside the object and only be saved when the record itself is saved. In this case, the file data is simply written to file during the save operation.

When the record is created using create_from_file, the behaviour depens on the parameters passed and the system settings.

  • In case the delete_original flag is set, the system will try to move the file to the new location. If both are on the same file system, this will be quicker than a copy operation.

  • Currently, the move operation uses the “mv” command. This is does not work on windows, but is a workaround to stability problems with the file handling in JRuby

  • If the delete_original flag is not set, the system will attempt to copy the files:

    • If the “delay_file_copies” options is set in the environment, no copy operation will be done. Instead, the system will create a “delayed_copies.sh” script that can be executed from a UNIX shell to do the actual copying. This is extremely fast and stable, as no actual copying is done.

    • Otherwise Talia will attempt to copy the file by itself. If the “fast_copies” flag is set in environment, it will use the internal copy routine which will work on any system. Otherwise, it will call the system’s “cp” command, which can sometimes be more stable with jruby.

Also see the DataLoader module to see how the creation of records automatically selects the record type and loader, depending on the MIME type of the data.

Note: The above behaviour means that for files that are treated through “copy” or “move”, the original file must not be touched by external processes until the record is saved

Defined Under Namespace

Classes: DataPath

Instance Method Summary collapse

Instance Method Details

#all_text ⇒ Object

Returns the contents of the file as a text string.



97
98
99
100
101
102
 
# File 'lib/talia_core/data_types/file_store.rb', line 97

def all_text
  if(!is_file_open?)
    open_file
  end
  @file_handle.read(self.size)
end

#assign_type(content_type) ⇒ Object

Assign the STI subclass, perfoming a mime-type lookup.



137
138
139
 
# File 'lib/talia_core/data_types/file_store.rb', line 137

def assign_type(content_type)
  self.type = MimeMapping.class_type_from(content_type).name
end

#create_from_data(location, data, options = {}) ⇒ Object

Creates a new record from the given data (binary array). See the module documentation for the details; the data will be cached and written to disk once the new record is saved.



81
82
83
84
85
86
87
88
89
90
91
92
93
94
 
# File 'lib/talia_core/data_types/file_store.rb', line 81

def create_from_data(location, data, options = {})
  # close file if opened
  close_file
    
  # Set the location for the record
  self.location = location
    
  if(data.respond_to?(:read))
    @file_data_to_write = data.read
  else
    @file_data_to_write = data
  end
    
end

#create_from_file(location, file_path, delete_original = false) ⇒ Object

Creates a new record from an existing file on the file system. The file will be copied or moved when the new record is saved - see the module documentation to see how this works in detail.



71
72
73
74
75
76
 
# File 'lib/talia_core/data_types/file_store.rb', line 71

def create_from_file(location, file_path, delete_original = false)
  close_file
  self.location = location
  @file_data_to_write = DataPath.new(file_path)
  @delete_original_file = delete_original
end

#is_file_open? ⇒ Boolean

Return true if the data file is open

Returns:

  • (Boolean) 


131
132
133
 
# File 'lib/talia_core/data_types/file_store.rb', line 131

def is_file_open?
  (@file_handle != nil)
end

#write_file_after_save ⇒ Object

Callback for writing the data from create_from_data or create_from_file. If there is a problem saving this file, only an internal assertion is thrown so that it won’t crash production environments.



107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
 
# File 'lib/talia_core/data_types/file_store.rb', line 107

def write_file_after_save 
  # check if there are data to write
  return unless(@file_data_to_write)
    
  begin
    self.class.benchmark("\033[36m\033[1m\033[4mFileStore\033[0m Saving file for #{self.id}") do
      # create data directory path
      FileUtils.mkdir_p(data_directory)
    
      if(@file_data_to_write.is_a?(DataPath))
        copy_data_file
      else
        save_cached_data
      end
    
      @file_data_to_write = nil
    end
  rescue Exception => e
    assit_fail("Exception on writing file #{self.location}: #{e}")
  end

end