Class: Hyrax::Actors::CreateWithRemoteFilesOrderedMembersActor
- Inherits:
-
CreateWithRemoteFilesActor
- Object
- AbstractActor
- CreateWithRemoteFilesActor
- Hyrax::Actors::CreateWithRemoteFilesOrderedMembersActor
- Defined in:
- app/actors/hyrax/actors/create_with_remote_files_ordered_members_actor.rb
Overview
When adding member FileSets to a Work, Hyrax saves
and reloads the work for each new member FileSet.
This can significantly slow down ingest for Works
with many member FileSets. The saving and reloading
happens in FileSetActor#attach_to_work.
This is a ‘swappable’ alternative approach. It will
be of most value to Hyrax applications dealing with
works with many filesets. Anecdotally, a 600 fileset
can be processed in ~15 mins versus >3 hours
with the standard approach.
The tradeoff is that the ordered members are now added in a
single step after the creation of all the FileSets, thus
introducing a slight risk of orphan filesets if the upload
fails before the addition of the ordered members. This
has not been observed in practice.
Swapping out the actors can be achieved thus:
In ‘config/initializers/hyrax.rb`: “` Hyrax::CurationConcern.actor_factory.swap(Hyrax::Actors::CreateWithFilesActor,
Hyrax::Actors::CreateWithRemoteFilesOrderedMembersActor)
“‘ Alternatively, in `config/application.rb`: “` config.to_prepare
Hyrax::CurationConcern.actor_factory.swap(Hyrax::Actors::CreateWithFilesActor,
Hyrax::Actors::CreateWithRemoteFilesOrderedMembersActor)
end “‘
If there is a key ‘:remote_files’ in the attributes, it attaches the files at the specified URIs to the work. e.g.:
attributes[:remote_files] = filenames.map do |name|
{ url: "https://example.com/file/#{name}", file_name: name }
end
Browse everything may also return a local file. And although it’s in the url property, it may have spaces, and not be a valid URI.
Instance Attribute Summary collapse
-
#ordered_members ⇒ Object
readonly
Returns the value of attribute ordered_members.
Attributes inherited from AbstractActor
Instance Method Summary collapse
-
#add_ordered_members(user, work) ⇒ Object
Add all file_sets as ordered_members in a single action.
- #attach_files(env, remote_files) ⇒ TrueClass
-
#create_file_from_url(env, uri, file_name, auth_header = {}) ⇒ Object
Generic utility for creating FileSet from a URL Used in to import files using URLs from a file picker like browse_everything.
Methods inherited from CreateWithRemoteFilesActor
Methods inherited from AbstractActor
Constructor Details
This class inherits a constructor from Hyrax::Actors::AbstractActor
Instance Attribute Details
#ordered_members ⇒ Object (readonly)
Returns the value of attribute ordered_members.
45 46 47 |
# File 'app/actors/hyrax/actors/create_with_remote_files_ordered_members_actor.rb', line 45 def ordered_members @ordered_members end |
Instance Method Details
#add_ordered_members(user, work) ⇒ Object
Add all file_sets as ordered_members in a single action
87 88 89 90 |
# File 'app/actors/hyrax/actors/create_with_remote_files_ordered_members_actor.rb', line 87 def add_ordered_members(user, work) actor = Hyrax::Actors::OrderedMembersActor.new(ordered_members, user) actor.attach_ordered_members_to_work(work) end |
#attach_files(env, remote_files) ⇒ TrueClass
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
# File 'app/actors/hyrax/actors/create_with_remote_files_ordered_members_actor.rb', line 49 def attach_files(env, remote_files) return true unless remote_files @ordered_members = env.curation_concern.ordered_members.to_a remote_files.each do |file_info| next if file_info.blank? || file_info[:url].blank? # Escape any space characters, so that this is a legal URI uri = URI.parse(Addressable::URI.escape(file_info[:url])) unless validate_remote_url(uri) Rails.logger.error "User #{env.user.user_key} attempted to ingest file from url #{file_info[:url]}, which doesn't pass validation" return false end auth_header = file_info.fetch(:auth_header, {}) create_file_from_url(env, uri, file_info[:file_name], auth_header) end add_ordered_members(env.user, env.curation_concern) true end |
#create_file_from_url(env, uri, file_name, auth_header = {}) ⇒ Object
Generic utility for creating FileSet from a URL Used in to import files using URLs from a file picker like browse_everything
69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 |
# File 'app/actors/hyrax/actors/create_with_remote_files_ordered_members_actor.rb', line 69 def create_file_from_url(env, uri, file_name, auth_header = {}) ::FileSet.new(import_url: uri.to_s, label: file_name) do |fs| actor = file_set_actor_class.new(fs, env.user) actor.(visibility: env.curation_concern.visibility) actor.attach_to_work(env.curation_concern) fs.save! ordered_members << fs if uri.scheme == 'file' # Turn any %20 into spaces. file_path = CGI.unescape(uri.path) IngestLocalFileJob.perform_later(fs, file_path, env.user) else ImportUrlJob.perform_later(fs, operation_for(user: actor.user), auth_header) end end end |