Class: Origen::Application::LSF
Overview
Responsible for handling all submissions to the LSF
Defined Under Namespace
Classes: Configuration
Class Method Summary collapse
-
.configuration {|@config| ... } ⇒ Object
Accessor for the global LSF configuration, use this to modify the default LSF configuration for a given setup.
Instance Method Summary collapse
-
#configuration ⇒ Object
(also: #config)
Returns the configuration for a given LSF instance, which always maps to the global configuration instance.
-
#limit_job_submissions ⇒ Object
Limits the number of jobs submitted to the LSF at one time, IT will start to warn if a single users current job count gets above 500.
- #queuing_job_ids ⇒ Object
- #remote_jobs_count ⇒ Object
- #running_job_ids ⇒ Object
-
#submit(command, options = {}) ⇒ Object
Submits the given command to the LSF, returns the LSF job ID.
Class Method Details
.configuration {|@config| ... } ⇒ Object
Accessor for the global LSF configuration, use this to modify the default LSF configuration for a given setup. Typically an alternate configuration would be added to the SoC class or the target file, but it can be set from anywhere. This method returns an instance of Origen::Application::LSF::Configuration and can be used as shown in the example.
Example
# soc/nevis.rb
Origen::Runner::LSF.configuration do |config|
# Use "msg.nevis" for the project string when running in Noida
if %x["domainname"] =~ /nidc/
config.lsf.project = "msg.nevis"
end
end
# Change the default group
Origen.config.lsf.group = "lam"
66 67 68 69 70 |
# File 'lib/origen/application/lsf.rb', line 66 def self.configuration @config ||= Configuration.new yield @config if block_given? @config end |
Instance Method Details
#configuration ⇒ Object Also known as: config
Returns the configuration for a given LSF instance, which always maps to the global configuration instance.
74 75 76 |
# File 'lib/origen/application/lsf.rb', line 74 def configuration self.class.configuration end |
#limit_job_submissions ⇒ Object
Limits the number of jobs submitted to the LSF at one time, IT will start to warn if a single users current job count gets above 500. This method prevents that stage from being reached.
152 153 154 155 156 157 158 159 160 161 162 163 164 165 |
# File 'lib/origen/application/lsf.rb', line 152 def limit_job_submissions @local_job_count ||= 0 if @local_job_count == 100 while remote_jobs_count > 400 puts 'Waiting for submitted jobs count to fall below limit...' sleep 5 end @local_job_count = 0 yield else @local_job_count += 1 yield end end |
#queuing_job_ids ⇒ Object
119 120 121 122 123 124 125 126 127 |
# File 'lib/origen/application/lsf.rb', line 119 def queuing_job_ids ids = [] `bjobs 2>&1`.split("\n").each do |line| if line =~ /^(\d+).*PEND/ ids << Regexp.last_match[1] end end ids end |
#remote_jobs_count ⇒ Object
139 140 141 142 143 144 145 146 147 |
# File 'lib/origen/application/lsf.rb', line 139 def remote_jobs_count i = 0 `bjobs 2>&1`.split("\n").each do |line| if line =~ /^(\d+).*(RUN|PEND)/ i += 1 end end i end |
#running_job_ids ⇒ Object
129 130 131 132 133 134 135 136 137 |
# File 'lib/origen/application/lsf.rb', line 129 def running_job_ids ids = [] `bjobs 2>&1`.split("\n").each do |line| if line =~ /^(\d+).*RUN/ ids << Regexp.last_match[1] end end ids end |
#submit(command, options = {}) ⇒ Object
Submits the given command to the LSF, returns the LSF job ID
80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 |
# File 'lib/origen/application/lsf.rb', line 80 def submit(command, = {}) = { dependents: [], rerunnable: true, # Will rerun automatically if the execution host fails }.merge() limit_job_submissions do group = [:group] || config.group group = group ? "-G #{group}" : '' project = [:project] || config.project project = project ? "-P #{project}" : '' resource = [:resource] || config.resource resource = resource ? "-R '#{resource}'" : '' queue = [:queue] || config.queue queue = queue ? "-q #{queue}" : '' cores = [:cores] || config.cores cores = cores ? "-n #{cores}" : '' rerunnable = [:rerunnable] ? '-r' : '' if [:dependents].empty? dependents = '' else dependents = [:dependents].map { |id| "ended(#{id})" }.join(' && ') dependents = "-w '#{dependents}'" end cmd = "bsub -oo /dev/null #{dependents} #{rerunnable} #{group} #{project} #{resource} #{queue} #{cores} '#{command}'" if config.debug puts cmd '496212' # Return a dummy ID to keep the caller happy else output = `#{cmd}` Origen.log.info output.strip if output.split("\n").last =~ /Job <(\d+)> is submitted/ Regexp.last_match[1] else :error end end end end |