Module: Sidekiq::Job::Iterable::Enumerators

Included in:
Sidekiq::Job::Iterable
Defined in:
lib/sidekiq/job/iterable/enumerators.rb

Instance Method Summary collapse

Instance Method Details

#active_record_batches_enumerator(relation, cursor:, **options) ⇒ Object

Builds Enumerator from ‘ActiveRecord::Relation` and enumerates on batches of records. Each Enumerator tick moves the cursor `:batch_size` rows forward.

Examples:

def build_enumerator(product_id, cursor:)
  active_record_batches_enumerator(
    Comment.where(product_id: product_id).select(:id),
    cursor: cursor,
    batch_size: 100
  )
end

def each_iteration(batch_of_comments, product_id)
  comment_ids = batch_of_comments.map(&:id)
  CommentService.call(comment_ids: comment_ids)
end

See Also:



68
69
70
 
# File 'lib/sidekiq/job/iterable/enumerators.rb', line 68

def active_record_batches_enumerator(relation, cursor:, **options)
  ActiveRecordEnumerator.new(relation, cursor: cursor, **options).batches
end

#active_record_records_enumerator(relation, cursor:, **options) ⇒ ActiveRecordEnumerator

Builds Enumerator from ‘ActiveRecord::Relation`. Each Enumerator tick moves the cursor one row forward.

Examples:

def build_enumerator(cursor:)
  active_record_records_enumerator(User.all, cursor: cursor)
end

def each_iteration(user)
  user.notify_about_something
end

Parameters:

  • relation (ActiveRecord::Relation) —

    relation to iterate

  • cursor (Object) —

    offset id to start iteration from

  • options (Hash) —

    additional options that will be passed to relevant ActiveRecord batching methods

Returns:



46
47
48
 
# File 'lib/sidekiq/job/iterable/enumerators.rb', line 46

def active_record_records_enumerator(relation, cursor:, **options)
  ActiveRecordEnumerator.new(relation, cursor: cursor, **options).records
end

#active_record_relations_enumerator(relation, cursor:, **options) ⇒ Object

Builds Enumerator from ‘ActiveRecord::Relation` and enumerates on batches, yielding `ActiveRecord::Relation`s.

Examples:

def build_enumerator(product_id, cursor:)
  active_record_relations_enumerator(
    Product.find(product_id).comments,
    cursor: cursor,
    batch_size: 100,
  )
end

def each_iteration(batch_of_comments, product_id)
  # batch_of_comments will be a Comment::ActiveRecord_Relation
  batch_of_comments.update_all(deleted: true)
end

See Also:



90
91
92
 
# File 'lib/sidekiq/job/iterable/enumerators.rb', line 90

def active_record_relations_enumerator(relation, cursor:, **options)
  ActiveRecordEnumerator.new(relation, cursor: cursor, **options).relations
end

#array_enumerator(array, cursor:) ⇒ Enumerator

Builds Enumerator object from a given array, using cursor as an offset.

Examples:

array_enumerator(['build', 'enumerator', 'from', 'any', 'array'], cursor: cursor)

Parameters:

  • array (Array)
  • cursor (Integer) —

    offset to start iteration from

Returns:

  • (Enumerator)

Raises:

  • (ArgumentError) 


20
21
22
23
24
25
 
# File 'lib/sidekiq/job/iterable/enumerators.rb', line 20

def array_enumerator(array, cursor:)
  raise ArgumentError, "array must be an Array" unless array.is_a?(Array)

  x = array.each_with_index.drop(cursor || 0)
  x.to_enum { x.size }
end

#csv_batches_enumerator(csv, cursor:, **options) ⇒ Object

Builds Enumerator from a CSV file and enumerates on batches of records.

Examples:

def build_enumerator(import_id, cursor:)
  import = Import.find(import_id)
  csv_batches_enumerator(import.csv, cursor: cursor)
end

def each_iteration(batch_of_csv_rows)
  # ...
end

Parameters:

  • csv (CSV) —

    an instance of CSV object

  • cursor (Integer) —

    offset to start iteration from

  • options (Hash) —

    a customizable set of options

Options Hash (**options):

  • :batch_size (Integer) — default: 100 —

    size of the batch



129
130
131
 
# File 'lib/sidekiq/job/iterable/enumerators.rb', line 129

def csv_batches_enumerator(csv, cursor:, **options)
  CsvEnumerator.new(csv).batches(cursor: cursor, **options)
end

#csv_enumerator(csv, cursor:) ⇒ Object

Builds Enumerator from a CSV file.

Examples:

def build_enumerator(import_id, cursor:)
  import = Import.find(import_id)
  csv_enumerator(import.csv, cursor: cursor)
end

def each_iteration(csv_row)
  # insert csv_row into database
end

Parameters:

  • csv (CSV) —

    an instance of CSV object

  • cursor (Integer) —

    offset to start iteration from



109
110
111
 
# File 'lib/sidekiq/job/iterable/enumerators.rb', line 109

def csv_enumerator(csv, cursor:)
  CsvEnumerator.new(csv).rows(cursor: cursor)
end