Module: Audition::Static::WorkSplit

Defined in:
lib/audition/static/work_split.rb

Overview

How a scan is divided between Ractor workers.

Constant Summary collapse

RACTOR_CPU_DEFAULT =

Ractors run on a fixed pool of native threads sized by RUBY_MAX_CPU, which only the environment can set. Spawning past it buys no parallelism and costs a Ractor each.

8

Class Method Summary collapse

Class Method Details

.chunks(weighted, count) ⇒ Array<Array>

Longest-processing-time-first: deal the heaviest item onto the lightest worker. Consecutive files are neighbors in the tree and so alike in size, so contiguous slices come out lopsided: one worker can draw a slice weighing several times the mean and still be running once the rest have finished. Greedy is enough here: LPT finishes within 4/3 of an optimal split.

Parameters:

  • weighted (Array<Array>) —

    [item, weight] pairs

  • count (Integer) —

    worker count

Returns:

  • (Array<Array>) —

    one chunk of items per busy worker



41
42
43
44
45
46
47
48
49
50
51
 
# File 'lib/audition/static/work_split.rb', line 41

def chunks(weighted, count)
  chunks = Array.new(count) { [] }
  loads = Array.new(count, 0)
  weighted.sort_by { |item, weight| [-weight, item] }
    .each do |item, weight|
      lightest = loads.index(loads.min)
      chunks[lightest] << item
      loads[lightest] += weight
    end
  chunks.reject(&:empty?)
end

.ractor_cpu_limit ⇒ Object 



25
26
27
28
 
# File 'lib/audition/static/work_split.rb', line 25

def ractor_cpu_limit
  limit = ENV["RUBY_MAX_CPU"].to_i
  limit.positive? ? limit : RACTOR_CPU_DEFAULT
end

.workers ⇒ Integer

One worker per core the Ractor pool can actually run. The main Ractor only waits while workers scan, so no core is held back for it.

Returns:

  • (Integer) 


21
22
23
 
# File 'lib/audition/static/work_split.rb', line 21

def workers
  Etc.nprocessors.clamp(1, ractor_cpu_limit)
end