Module: Datadog::Core::Chunker

Defined in:
lib/datadog/core/chunker.rb

Overview

Chunks list of elements into batches

Class Method Summary collapse

Class Method Details

.chunk_by_size(list, max_chunk_size) ⇒ Enumerable

Chunks a list into batches of at most max_chunk_size elements each.

An exception can occur if a single element is too large. That single element will be returned in its own chunk. You have to verify by yourself when such elements are returned.

Parameters:

  • list (Enumerable) —

    list of elements

  • max_chunk_size (Numeric) —

    maximum acceptable chunk size

Returns:

  • (Enumerable) —

    lazy list of chunks



18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
 
# File 'lib/datadog/core/chunker.rb', line 18

def chunk_by_size(list, max_chunk_size)
  chunk_agg = 0
  list.slice_before do |elem|
    size = elem.size
    chunk_agg += size
    if chunk_agg > max_chunk_size
      # Can't fit element in current chunk, start a new one.
      chunk_agg = size
      true
    else
      # Add to current chunk
      false
    end
  end
end