Class: MedPipe::BatchIdFetcher

Inherits:
Object
  • Object
show all
Defined in:
lib/med_pipe/batch_id_fetcher.rb

Overview

idを最大max_load_size件ずつ分割取得するためのクラス 使い時:

  • 10万件以上のidを取得したい場合

  • 速度を改善するために in_batches を使いたくない場合

Instance Method Summary collapse

Constructor Details

#initialize(relation, batch_size: 1_000, max_load_size: 100_000) ⇒ BatchIdFetcher

Returns a new instance of BatchIdFetcher.



8
9
10
11
12
13
 
# File 'lib/med_pipe/batch_id_fetcher.rb', line 8

def initialize(relation, batch_size: 1_000, max_load_size: 100_000)
  @relation = relation
  @batch_size = batch_size
  @max_load_size = max_load_size
  validate_parameters
end

Instance Method Details

#each ⇒ Object 



15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
 
# File 'lib/med_pipe/batch_id_fetcher.rb', line 15

def each
  return enum_for(:each) unless block_given?

  last_id = 0
  cached_ids = []

  loop do
    loaded_ids = load_ids(last_id)
    break if loaded_ids.blank?

    last_id = loaded_ids.last
    cached_ids.concat(loaded_ids)
    yield(cached_ids.shift(@batch_size)) while cached_ids.size >= @batch_size

    if loaded_ids.size < @max_load_size
      yield(cached_ids) if cached_ids.present?
      break
    end
  end
end