Class: CodeRunner
- Extended by:
- const_get(SYSconst_get(SYS.variable_to_class_name)
- Includes:
- const_get(SYSconst_get(SYS.variable_to_class_name), Log
- Defined in:
- lib/coderunner.rb,
lib/coderunner.rb,
lib/cubecalccrmod.rb,
lib/coderunner/run.rb,
lib/cubecalccrmod/empty.rb,
lib/cubecalccrmod/sleep.rb,
lib/coderunner/class_methods.rb,
lib/coderunner/fortran_namelist.rb,
lib/coderunner/graphs_and_films.rb,
lib/coderunner/instance_methods.rb,
lib/coderunner/instance_methods.rb,
lib/coderunner/merged_code_runner.rb,
lib/coderunner/interactive_methods.rb,
lib/coderunner/system_modules/moab.rb,
lib/coderunner/system_modules/slurm.rb,
lib/coderunner/heuristic_run_methods.rb,
lib/coderunner/system_modules/hector.rb,
lib/coderunner/system_modules/helios.rb,
lib/coderunner/system_modules/iridis.rb,
lib/coderunner/system_modules/juropa.rb,
lib/coderunner/system_modules/macosx.rb,
lib/coderunner/system_modules/franklin.rb,
lib/coderunner/system_modules/new_hydra.rb,
lib/coderunner/system_modules/generic_linux.rb,
lib/coderunner/system_modules/genericlinux_testsystem.rb
Overview
CodeRunner Overview
CodeRunner is a class designed to make the running an analysis of large simulations and easy task. An instance of this class is instantiated for a given root folder. The runner, as it is known, knows about every simulation in this folder, each of which has a unique id and a unique subfolder.
The heart of the runner is the variable run_list. This is a hash of IDs and runs. A run is an instance of a class which inherits from CodeRunner::Run, and which is customised to be able to handle the input variables, and the output data, from the given simulation code. This is achieved by a module which contains a child class of CodeRunner::Run, which is provided independently of CodeRunner.
CodeRunner has methods to sort these runs, filter them according to quite complex conditions, print out the status of these runs, submit new runs, plot graphs using data from these runs, cancel running jobs, delete unwanted runs, and so on.
CodeRunner Interfaces
CodeRunner has two different interfaces:
-
Instance methods
-
Class methods
Instance Methods
The instance methods provide a classic Ruby scripting interface. A runner is instantiated from the CodeRunner class, and passed a root folder, and possibly some default options. Instance methods can then be called individually. This is what should be used for complex and non-standard tasks.
Class methods
The class methods are what are used by the command line interface. They define a standard set of tasks, each of which can be customised by a set of options known as command options, or copts for short.
There is a one-to-one correspondence between the long form of the commandline commands, and the class methods that handle those commands, and between the command line flags, and the options that are passed to the class methods. So for example:
$ coderunner submit -p ‘23.4, resolution: 256’ -n 32x4 -W 600
Becomes
CodeRunner.submit(p: ‘23.4, resolution: 256’, n: “32x4”, W: 600) # remembering that braces are not needed around a hash if it is the final parameter.
These methods are what should be used to automate a large number of standard tasks, which would be a pain to run from the command line.
Direct Known Subclasses
Defined Under Namespace
Modules: Franklin, GenericLinux, GenericlinuxTestsystem, Hector, Helios, InteractiveMethods, Iridis, Juropa, Macosx, Moab, NewHydra, Slurm Classes: CRError, CRFatal, CRMild, Cubecalc, Merged, Run
Constant Summary collapse
- COMMAND_FOLDER =
Dir.pwd
- SCRIPT_FOLDER =
i.e. where this script is
File.dirname(File.(__FILE__)) + '/coderunner'
- GLOBAL_OPTIONS =
global options are set by the environment but some can be changed.
{}
- SYS =
(GLOBAL_OPTIONS[:system] or ENV['CODE_RUNNER_SYSTEM'] or ENV['SYSTEM'] or "generic_linux")
- SYSTEM_MODULE =
const_get(SYS.variable_to_class_name)
- GraphKit =
Backwards compatibility
GraphKit
- CODE_RUNNER_VERSION =
Version.new(Gem.loaded_specs['coderunner'].version.to_s)
- GLOBAL_BINDING =
binding
- COMMAND_LINE_FLAGS_WITH_HELP =
[ ["--recalc-all", "-A", GetoptLong::NO_ARGUMENT, %[Causes each directory to be reprocessed, rather than reading the cache of data. Its exact effect depends on the code module being used. By convention it implies that ALL data analysis will be redone.]], ["--reprocess-all", "-a", GetoptLong::NO_ARGUMENT, %[Causes each directory to be reprocessed, rather than reading the cache of data. Its exact effect depends on the code module being used. By convention it implies that VERY LITTLE data analysis will be redone.]], ["--code", "-C", GetoptLong::REQUIRED_ARGUMENT, %[The code that is being used for simulations in this folder. This string must correspond to a code module supplied to CodeRunner. It usually only needs to be specified once as it will be stored as a default in the folder.]], ["--comment", "-c", GetoptLong::REQUIRED_ARGUMENT, %[A comment about the submitted run.]], ["--debug", "-d", GetoptLong::NO_ARGUMENT, %[Submit the simulation to the debug queue. This usually only has meaning on HPC systems. It does not mean debug CodeRunner!]], ["--defaults-file", "-D", GetoptLong::REQUIRED_ARGUMENT, %[Specify a defaults file to be used when submitting runs. The name should correspond to file named something like '<name>_defaults.rb' in the correct folders within the code module. Every time a different defaults file is specified, a local copy of that defaults file is stored in the root folder. This local copy can be edited and all runs will use the local copy to get defaults. CodeRunner will never overwrite the local copy.]], ["--film-options", "-F", GetoptLong::OPTIONAL_ARGUMENT, %[Specify a hash of options when making films. The most important one is fa (frame array). For example -F '{fa: [0, 200]}'. For all possible options see the CodeRunner method make_film_from_lists.]], ["--conditions", "-f", GetoptLong::REQUIRED_ARGUMENT, %[A string specifying conditions used to filter runs. This filter is used in a variety of circumstances, for example when printing out the status, plotting graphs etc. Example: '@height == 10 and @width = 2.2 and @status==:Complete'.]], ["--run-graph", "-g", GetoptLong::REQUIRED_ARGUMENT, %[Specify a run_graphkit to plot. A run_graphkit is one that is plotted for an individual run. The run graphkits available depend on the code module. The syntax is graphkit shorthand:\n -g '<graph_name>[ ; <graph_options> [ ; <conditions> [ ; <sort> ] ] ]'\n where conditions (i.e. filter) and sort will override the -f and -O flags respectively. The -g flag can be specified multiple times, which will plot multiple graphs on the same page.]], ["--graph", "-G", GetoptLong::REQUIRED_ARGUMENT, %[Specify a graphkit to plot. A graphkit combines data for every filtered run. The syntax is graphkit shorthand:\n -G '<axis1>[ : <axis2> [ : <axis3 [ : <axis4> ] ] ] [ ; <graph_options> [ ; <conditions> [ ; <sort> ] ] ]'\n where conditions (i.e. filter) and sort will override the -f and -O flags respectively. <axis1> etc are strings which can be evaluated by the runs. The -G flag can be specified multiple times, which will plot multiple graphs on the same page. For example\n -G 'width : 2*height ; {} ; depth == 2 ; width'\n will plot twice the height against the width for every run where the depth is equal to 2, and will order the data points by width.]], ["--heuristic-analysis", "-H", GetoptLong::NO_ARGUMENT, %[Should be specified whenever CodeRunner is being used to analyse simulations which did not originally submit (and which will therefore not have the usual CodeRunner meta data stored with them).] ], ["--use-phantom", "-h", GetoptLong::OPTIONAL_ARGUMENT, %[Specify whether to use real or phantom runs]], ["--just", "-j", GetoptLong::REQUIRED_ARGUMENT, %[Specify individual run ids. For example -j 45,63,128 is shorthand for -f 'id==45 or id==63 or id==128']], ["--job_chain", "-J", GetoptLong::NO_ARGUMENT, %[Chain multiple simulations into one batch/submission job. Most useful for HPC systems.]], ["--skip-similar-jobs-off", "-k", GetoptLong::NO_ARGUMENT, %[Normally CodeRunner will not submit a run whose input parameters identical to a previous run (to avoid wasting computer time). Specifying the flag will override that behaviour and force submission of an identical run.]], ["--loop", "-l", GetoptLong::NO_ARGUMENT, %[Used with the status command. Keep continually printing out live status information.]], ["--multiple-processes", "-M", GetoptLong::REQUIRED_ARGUMENT], ["--modlet", "-m", GetoptLong::REQUIRED_ARGUMENT, %[Specify the modlet to be used in the current folder. Only needs to be specified once as it will be stored as a default.]], ["--no-run", "-N", GetoptLong::NO_ARGUMENT, %[On some machines getting a list of currently running jobs takes a long time. Specifying this flag tells CodeRunner that you definitely know that no runs in the folder are still queueing or running. Do not specify it if there are still running jobs as it will cause their statuses to be updated incorrectly.]], ["--nprocs", "-n", GetoptLong::REQUIRED_ARGUMENT, %[A string specifying the processor layout for the simulation. For example -n 46x4 means use 46 nodes with four processors per node. In the case of a personal computer something like -n 2 is more likely. The default is 1]], ["--sort", "-O", GetoptLong::REQUIRED_ARGUMENT, %[Specify the sort order for the runs. Used for a variety of commands, for example status. It is a string of semicolon separated sort keys: for example -O height;width will sort the runs by height and then width.]], ["--project", "-P", GetoptLong::REQUIRED_ARGUMENT, %[Specify the project to be used for billing purposes. Only necessary on some systems.]], ["--parameters", "-p", GetoptLong::REQUIRED_ARGUMENT, %[A hash of parameters for the simulation. For example -p '{height: 20, width: 2.3}'. These parameters will override the defaults in the local defaults file.]], ["--no-auto-create-runner", "-q", GetoptLong::NO_ARGUMENT, %[Used for interactive mode when you don't want CodeRunner to analyse the current directory.]], ["--terminal-size", "-t", GetoptLong::REQUIRED_ARGUMENT, %[Specify the terminal size for situations where CodeRunner cannot work it out: -t '[rows, cols]' (square brackets are part of the syntax)]], ["--test-submission", "-T", GetoptLong::NO_ARGUMENT, %[Don't actually submit the run, but exit after printing out the run parameters and generating any input files necessary.]], ["--use-large-cache-but-recheck-incomplete", "-u", GetoptLong::NO_ARGUMENT, %[Use the large cache for speed, but check any runs whose status is not :Complete or :Failed.]], ["--use-large-cache", "-U", GetoptLong::NO_ARGUMENT, %[Use the large cache for speed. No run data will be updated.]], ["--version", "-v", GetoptLong::REQUIRED_ARGUMENT, %[Specify the version of the simulation code being used. Only has an effect for certain code modules.]], ["--wall-mins", "-W", GetoptLong::REQUIRED_ARGUMENT, %[Specify the wall clock limit in minutes.]], ["--write-options", "-w", GetoptLong::REQUIRED_ARGUMENT, %[Use when plotting graphs. A hash of custom options which are applied to the graphkit just before plotting it; for example: -w '{xlabel: 'X Axis Quantity, log_axis: 'y'}']], ["--executable", "-X", GetoptLong::REQUIRED_ARGUMENT, %[Specify the location of the executable of the simulation code. It only needs to be specified once in any folder, unless it needs to be changed.]], ["--other-folder", "-Y", GetoptLong::REQUIRED_ARGUMENT, %[Run CodeRunner in a different folder. On a local machine 'coderunner st -Y some/other/folder' is identical to 'cd some/other/folder; coderunner st -Y'. However, this flag can also be used for remote folders using RemoteCodeRunner (as long as CodeRunner is installed on the remote machine). e.g. -Y [email protected]:path/to/folder.]], ["--supplementary-options", "-y", GetoptLong::REQUIRED_ARGUMENT], ["--server", "-Z", GetoptLong::REQUIRED_ARGUMENT, %[Technical use only]], ["--log", "-z", GetoptLong::NO_ARGUMENT, %[Switch logging on (currently not working very well (05/2010)).]] # :nodoc: ]
- CLF =
COMMAND_LINE_FLAGS = COMMAND_LINE_FLAGS_WITH_HELP.map{|arr| arr.slice(0..2)}
- CODE_COMMAND_OPTIONS =
NEEDS FIXING!!!!
[]
- LONG_COMMAND_LINE_OPTIONS =
end
[ ["--replace-existing", "", GetoptLong::NO_ARGUMENT, %[Use with resubmit: causes each resubmitted run to replace the run being resubmitted.]], ["--smart-resubmit-name", "", GetoptLong::NO_ARGUMENT, %[Use with resubmit: causes each resubmitted run to only contain its original id and changed parameters in its run name.]], ] + CODE_COMMAND_OPTIONS
- LONG_COMMAND_LINE_FLAGS =
LONG_COMMAND_LINE_OPTIONS.map{|arr| [arr[0], arr[2]]}
- COMMANDS_WITH_HELP =
[ ["available_modlets", "av", 0, 'List the available modlets for the code module.', [], [:C]], ["available_defaults_files", "avd", 0, 'List the defaults files for the code module.', [], [:C]], ["cancel", "can", 1, 'Cancel the specified job.', ['id'], [:U]], ["code_command", "cc", 1, 'Call a class method of the run class. Effectively this will call run_class.class_eval(command). See documentation for whichever code module is in use.', ['command'], []], ["continue_in_new_folder", "cnf", 1, 'Make a new folder in the parent directory and copy all coderunner configuration files to that folder. If options j or f are specified, copy all matching runs to that new folder.', ['folder'], [:j, :f, :U, :N]], ["code_runner_execute", "crex", 1, 'Run (within the CodeRunner class) the fragment of Ruby code given.', ['Ruby fragment'], []], ["delete", "del", 0, 'Permanently erase all filtered runs.', [], [:j, :F, :U, :N]], ["directory", "dir", 1, 'Print out the directory for the given run.', ['id'], []], ["execute", "ex", 1, 'Run (at the top level) the fragment of Ruby code given.', ['Ruby fragment'], []], ['film', "fm", 0, 'Create a film of the specified graphkits.', [], [:F, :G, :g, :U, :N, :j, :f]], ["generate_modlet_from_input_file", "gm", 1, 'Deprecated', [], []], ["generate_documentation", "rdoc", 1, 'Create automatic documentation using the rdoc tool', [], []], ["interactive_mode", "im", 0, 'Launch an interactive terminal. Any command line flags specified set the defaults for the session.', [], [:U, :N, :j, :q]], ["load_file", "ld", 1, 'Load a Ruby script file using the CodeRunner framework.', ['script file'], []], ['manual', 'man', 0, 'Print out command line manual', [], []], ['netcdf_plot', 'ncplot', 3, 'Plot a comma separated list of variables, at a comma separated list of indices (nil for all) from the specified netcdf file against each other using gnuplot.', ['netcdf_file', 'vars', 'indices'], [:w]], ["plot_graph", "plot", 0, 'Plot the specified graphkits using Gnuplot', [], [:G, :g, :w, :O, :U, :N, :j, :f]], ["parameter_scan", "ps", 1, 'Read a parameter scan from file. For full details of how to write a parameter scan, see online documentation (coderunner.sourceforge.net).', ['scan file'], [:n, :W, :k, :v, :p, :T, :d]], ['print_queue_status', 'qstat', 0, 'Show the current status of the queue', [], [:U, :u]], ["readout", "ro", 0, 'Print a simple text readout of all data from the runs.', [], []], ["reference", "ri", 1, "Print out documentation for the given class or method. #{rihelp}", ['ruby_class_or_method'], []], ["resubmit", "resub", 0, 'Resubmit the filtered runs to be simulated. All parameters will be the same bar those altered by the p option.', [], [:p, :n, :W, :k, :v, :T, :d, :J, :f, :j]], ["run_command", "rc", 1, 'Cause all filtered runs to evaluate the given string.', ['command string'], [:U, :f, :j, :N]], ["runner_eval", "ev", 1, 'Cause the runner (the CodeRunner instance) to evaluate the given string.', ['command string'], [:U, :N, :j, :f]], ["scan", "scan", 1, 'Submit a simple scan. For full details of how to write a simple scan, see online documentation (coderunner.sourceforge.net).', ['scan string'], [:p, :n, :W, :k, :v, :T, :d]], ["show_values_of", "shvl", 1, 'Evaluate the expression for each run and print a unique sorted list of them.', ['expression'], [:U, :N, :j, :f]], ['start_launcher', 'launch', 2, 'Start a simple job launcher for non batch systems.', ['refresh_interval', 'max_queue_size'], []], ["status", "st", 0, 'Print out a summary of the status of the filtered runs.', [], [:U, :N, :j, :f, :O]], ["status_with_comments", "sc", 0, 'Print a list of ids with their status and any comments.', [], [:U, :N, :j, :f, :O]], ["status_loop", "sl", 0, 'Loop, updating the filtered runs, then printing out a summary of the status of the filtered runs. ', [], [:U, :N, :j, :f, :O]], ["submit", "sub", 0, 'Submit a run to be simulated.', [], [:p, :n, :W, :k, :v, :T, :d, :J]], ["submit_command", "subcom", 2, 'Submit an arbitrary shell command to the batch queue.', ['job name', 'command'], [:n, :W, :v, :T, :d]], ["write_graph", "wg", 1, 'Write a graph to disk.', ['filename'], [:G, :g, :w, :O, :U, :N, :j, :f]] ]
- COMMANDS =
This lists all the commands available on the command line. The first two items in each array indicate the long and short form of the command, and the third indicates the number of arguments the command takes. They are all implemented as Code Runner class methods (the method is named after the long form). The short form of the command is available as a global method in Code Runner interactive mode.
COMMANDS_WITH_HELP.map{|arr| arr.slice(0..2)}
- CLF_TO_SHORT_COPTS =
A lookup hash which gives the appropriate short command option (copt) key for a given long command flag
COMMAND_LINE_FLAGS.inject({}){ |hash, (long, short, req)| letter = short[1,1] hash[long] = letter.to_sym hash }
- CLF_BOOLS =
specifying flag sets a bool to be true
[:H, :U, :u, :A, :a, :T, :N, :q, :z, :d, :J, :replace_existing]
- CLF_INVERSE_BOOLS =
CLF_BOOLS = [:s, :r, :D, :H, :U, :u, :L, :l, :A, :a, :T, :N,:V, :q, :z, :d] #
[:k]
- LONG_TO_SHORT =
a look up hash that converts the long form of the command options to the short form (NB command options e.g. use_large_cache have a different form from command line flags e.g. –use-large-cache)
COMMAND_LINE_FLAGS.inject({}){ |hash, (long, short, req)| letter = short[1,1] hash[long[2, long.size].gsub(/\-/, '_').to_sym] = letter.to_sym hash }
- CLF_TO_LONG =
A look up table that converts long only command line options (in LONG_COMMAND_LINE_OPTIONS) to the equivalent CodeRunner command option
LONG_COMMAND_LINE_OPTIONS.inject({}) do |hash, (long, short, req, help)| option = long[2, long.size].gsub(/\-/, '_').to_sym hash[long] = option hash end
- DEFAULT_COMMAND_OPTIONS =
Default command options; they are usually determined by the command line flags, but can be set independently
{}
- CODE_OPTIONS =
{}
- SUBMIT_OPTIONS =
Parameters important to the submission of a run, which can be set by command line flags. The runner values provide the default values for the submit function, but can be overidden in that function. All the runner does with them is set them as properties of the run to be submitted. It is the run itself for which the options are relevant.
[:nprocs, :wall_mins, :sys, :project, :comment, :executable]
- DEFAULT_RUNNER_OPTIONS =
A hash containing the defaults for most runner options. They are overridden by any options provided during initialisation. They are mostly set at the command line (in practice, the command line flags are read into the command options, which set these defaults in the function CodeRunner.process_command_options which calls CodeRunner.set_runner_defaults). However, if Code Runner is being scripted, these defaults must be set manually or else the options they specify must be provided when initialising a runner.
([:conditions, :code, :executable, :sort, :debug, :script_folder, :recalc_all, :multiple_processes, :heuristic_analysis, :test_submission, :reprocess_all, :use_large_cache, :use_large_cache_but_recheck_incomplete, :use_phantom, :no_run, :server, :version, :parameters] + SUBMIT_OPTIONS).inject({}){|hash, option| hash[option] = nil; hash}
- CLASS_OPTIONS =
Options that apply across the CodeRunner class
[:multiple_processes].inject({}){|hash, option| class_accessor option set(option, nil) hash[option] = nil; hash }
- NECESSARY_RUN_CLASS_PROPERTIES =
These are properties of the run class that must be defined. For more details see CodeRunner::Run.
{ :code => [String], :variables => [Array], :naming_pars => [Array], :results => [Array], :run_info => [Array], :code_long => [String], :excluded_sub_folders => [Array], :modlet_required => [TrueClass, FalseClass], :uses_mpi => [TrueClass, FalseClass] }
- NECESSARY_RUN_CODE_METHODS =
These are methods that the run class must implement. They should be defined in a code module.
[ :process_directory_code_specific, :print_out_line, :parameter_string, :generate_input_file, :parameter_transition, :executable_location, :executable_name ]
- NECESSARY_RUN_SYSTEM_METHODS =
These are methods that the run class must implement. They should be defined in a system module.
[ :queue_status, :run_command, :execute, :error_file, :output_file, :cancel_job ]
- PERMITTED_STATI =
These are the only permitted values for the run instance variable
@status
. [:Unknown, :Complete, :Incomplete, :NotStarted, :Failed, :Queueing, :Running]
- FOLDER_DEFAULTS =
The defaults that are saved in the root folder
[:code, :modlet, :executable, :defaults_file, :project]
- SETUP_RUN_CLASSES =
[]
- @@sys =
end
SYS
- @@wait =
Do you wait for the previous run to have completed when using simple scan?
true
Instance Attribute Summary collapse
-
#cache ⇒ Object
Returns the value of attribute cache.
-
#cmaxes ⇒ Object
readonly
Returns the value of attribute cmaxes.
-
#cmins ⇒ Object
readonly
Returns the value of attribute cmins.
-
#code ⇒ Object
Returns the value of attribute code.
-
#combined_ids ⇒ Object
Returns the value of attribute combined_ids.
-
#combined_run_list ⇒ Object
Returns the value of attribute combined_run_list.
-
#current_request ⇒ Object
Returns the value of attribute current_request.
-
#current_status ⇒ Object
Returns the value of attribute current_status.
-
#defaults_file ⇒ Object
Returns the value of attribute defaults_file.
-
#executable ⇒ Object
Returns the value of attribute executable.
-
#ids ⇒ Object
Returns the value of attribute ids.
-
#max_id ⇒ Object
readonly
Returns the value of attribute max_id.
-
#maxes ⇒ Object
readonly
Returns the value of attribute maxes.
-
#mins ⇒ Object
readonly
Returns the value of attribute mins.
-
#modlet ⇒ Object
Returns the value of attribute modlet.
-
#phantom_ids ⇒ Object
Returns the value of attribute phantom_ids.
-
#phantom_run_list ⇒ Object
Returns the value of attribute phantom_run_list.
-
#print_out_size ⇒ Object
Returns the value of attribute print_out_size.
-
#requests ⇒ Object
Returns the value of attribute requests.
-
#root_folder ⇒ Object
Returns the value of attribute root_folder.
-
#run_class ⇒ Object
Returns the value of attribute run_class.
-
#run_list ⇒ Object
Returns the value of attribute run_list.
-
#start_id ⇒ Object
readonly
Returns the value of attribute start_id.
Class Method Summary collapse
-
.available_defaults_files(copts = {}) ⇒ Object
List the available defaults files for the given code (copts or -C on the command line).
-
.available_modlets(copts = {}) ⇒ Object
List the available modlets for the given code (copts or -C on the command line).
-
.cancel(id, copts = {}) ⇒ Object
Cancel the job with the given id.
- .code_command(string, copts = {}) ⇒ Object
- .code_runner_execute(ruby_fragment, copts = {}) ⇒ Object
- .continue_in_new_folder(folder, copts = {}) ⇒ Object
- .delete(copts = {}) ⇒ Object
- .directory(id, copts = {}) ⇒ Object
- .execute(ruby_fragment, copts = {}) ⇒ Object
-
.fetch_runner(copts = {}) ⇒ Object
Retrieve the runner with the folder (and possibly server) given in copts.
- .film(copts = {}) ⇒ Object
- .film_graphkit_frame_array(graphkit_frame_array, options) ⇒ Object
- .generate_documentation(username = nil, copts = {}) ⇒ Object
-
.get_run_class_name(code, modlet = nil) ⇒ Object
Return the name of the run class according to the standard CodeRunner naming scheme.
- .gets ⇒ Object
-
.graphkit_multiple_runners(list, options = {}) ⇒ Object
list is an array of [[runner, [graphs, run_graphs]], … ].
- .graphkit_multiple_runners_with_frame_array(frame_array, list, print_message = false) ⇒ Object
-
.interactive_mode(copts = {}) ⇒ Object
module InteractiveMethods.
-
.load_file(files, copts = {}) ⇒ Object
eval(ruby_fragment).
-
.make_film_multiple_runners(list, options) ⇒ Object
__END__.
- .manual(copts = {}) ⇒ Object
-
.netcdf_plot(netcdf_file, vars, indices, copts = {}) ⇒ Object
def self.executable_name # :nodoc: “” end def self.rcp # :nodoc: @rcp ||= KitHash.new end.
-
.old_get_run_class_name(code, modlet) ⇒ Object
:nodoc:.
- .parameter_scan(parameter_scan_array_file, copts = {}) ⇒ Object
- .plot_graph(copts = {}) ⇒ Object
- .print_queue_status(copts = {}) ⇒ Object
-
.process_command_line_option(opt, arg, copts) ⇒ Object
Converts a command line flag opt with value arg to a command option which is stored in copts.
- .process_command_options(copts) ⇒ Object
- .read_default_command_options(copts) ⇒ Object
- .readout(copts = {}) ⇒ Object
- .recheck(id, copts = {}) ⇒ Object
- .reference(class_or_method, copts = {}) ⇒ Object
- .repair_marshal_run_class_not_found_error(err) ⇒ Object
- .resubmit(copts = {}) ⇒ Object
-
.run_command(string, copts = {}) ⇒ Object
runner = fetch_runner(copts) runner.run_class.class_eval(string).
- .run_script ⇒ Object
- .runner ⇒ Object
- .runner_eval(string, copts = {}) ⇒ Object
- .scan(scan_string, copts = {}) ⇒ Object
- .server_dump(obj) ⇒ Object
- .set_class_defaults(copts = {}) ⇒ Object
- .set_default_command_options_from_command_line ⇒ Object
-
.set_runner_defaults(copts = {}) ⇒ Object
In the next section are the implementations of all the standard Code Runner commands and some helper functions.
- .setup_run_class(code, options = {}) ⇒ Object
- .show_values_of(expression, copts = {}) ⇒ Object
- .start_launcher(refresh, max_queue, copts = {}) ⇒ Object
- .status(copts = {}) ⇒ Object
- .status_loop(copts = {}) ⇒ Object
- .status_with_comments(copts = {}) ⇒ Object
- .submit(copts = {}) ⇒ Object
-
.submit_command(jid, comm, copts = {}) ⇒ Object
This method allows the straightforward submission of a single command using the batch queue on any system.
-
.update_runners ⇒ Object
@r.read_defaults.
- .write_graph(name, copts = {}) ⇒ Object
Instance Method Summary collapse
-
#add_phantom_run(run) ⇒ Object
def rename_variable(old, new) puts “Please confirm complete renaming of #old to #new” gets @run_list.each do |directory| Dir.chdir directory do begin @readme = File.read(“CODE_RUNNER_INPUTS”) rescue Errno::ENOENT => err # puts err, err.class @readme = File.read(“README”) end @readme.sub!(Regexp.new(“^s+#old:”), “^s+#new:”) File.open(“CODE_RUNNER_INPUTS_TEST”, ‘w’){|file| file.puts @readme} end end old_recalc, @recalc_all = @recalc_all, true update @recalc_all = old_recalc end.
-
#alter_ids(num, options = {}) ⇒ Object
Permanently change the id of every run in the folder by adding num to them.
-
#axiskit(string) ⇒ Object
def gnuplot(axes, options) a = graphkit(axes, options).gnuplot end.
-
#cancel_job(id, options = {}) ⇒ Object
Cancel the job with the given ID.
- #code_run_environment ⇒ Object
- #continue_in_new_folder(folder, options = {}) ⇒ Object
-
#create_archive(options = {}) ⇒ Object
Create a tar archive of the root folder and all the files in it.
-
#destroy(options = {}) ⇒ Object
Delete the folders of all runs for whom CodeRunner#filter(run) is true.
-
#dup ⇒ Object
Duplicate the runner, trying to be intelligent as far as possible in duplicating instance variables.
- #executable_location ⇒ Object
- #executable_name ⇒ Object
- #film_graphkit(axes, options, film_options = {}) ⇒ Object
- #film_run_graphkit(name, options, film_options = {}) ⇒ Object
-
#filter(run = @run, conditions = @conditions) ⇒ Object
Return true if run.instance_eval(conditions) == true and false if run.instance_eval(conditions) == false.
-
#filtered_ids(conditions = @conditions) ⇒ Object
Return a list of ids, filtered according to conditions.
- #generate_combined_ids(kind = nil) ⇒ Object
-
#get_all_root_folder_contents ⇒ Object
List every file in the root folder.
-
#get_max(run, variable, sweep, complete = nil) ⇒ Object
def max(variable,sweep=nil, complete=nil) logf(:max) return get_max(variable,sweep, complete) == @run.id end.
- #get_min(run, variable, sweep, complete = nil) ⇒ Object
-
#get_run_class_name(code, modlet) ⇒ Object
See CodeRunner.get_run_class_name.
-
#gets ⇒ Object
No reading from the command line thank you very much!.
-
#graphkit(*args) ⇒ Object
call-seq: graphkit(axes, options) Make a general graphkit, i.e.
- #graphkit_from_lists(graphs, run_graphs, extra_options = {}) ⇒ Object
- #graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, extra_options = {}) ⇒ Object
-
#graphkit_shorthand(*gr) ⇒ Object
Parse graphkit shorthand for a general graph and return it as [axes, options].
-
#increment_max_id ⇒ Object
Increase the value of
@max_id
by 1. -
#initialize(root_folder, options = {}) ⇒ CodeRunner
constructor
Instantiate a new runner.
- #job_identifier ⇒ Object
- #make_film_from_lists(graphs, run_graphs, film_options = {}) ⇒ Object
-
#marshalled_variables ⇒ Object
Dump all the instance variables of the runner to stdout as Marshalled binary data.
- #merge(*others) ⇒ Object
- #merge_method(meth, *args) ⇒ Object
-
#new_run ⇒ Object
Create a new instance of the class
@run_class
. -
#p(*args) ⇒ Object
Here we redefine the inspect function p to raise an error if anything is written to standard out while in server mode.
- #parameter_scan(parameter_scan, parameters, options = {}) ⇒ Object
-
#print(*args) ⇒ Object
Here we redefine the function print to raise an error if anything is written to standard out while in server mode.
-
#print_out(rewind = nil, options = {}) ⇒ Object
Print out a summary of all the runs in the root folder, formatted in nice pretty colours.
-
#puts(*args) ⇒ Object
Here we redefine the function puts to raise an error if anything is written to standard out while in server mode.
-
#rcp ⇒ Object
def submit.
- #read_defaults ⇒ Object
-
#read_folder_defaults ⇒ Object
Read any default options contained in the file
.code_runner_script_defaults.rb
in the root folder. -
#readout ⇒ Object
Similar to CodeRunner#write_data, except that the readout is written to stdout, and formatted a bit.
-
#readout_cols(*var_list) ⇒ Object
? Probably can get rid of this one.
-
#recheck_filtered_runs(write_status_dots = false) ⇒ Object
Self-explanatory! Call CodeRunner::Run#process_directory for every run for which CodeRunner#filter(run) is true.
-
#recheck_incomplete_runs ⇒ Object
Self-explanatory! Call CodeRunner::Run#process_directory for every run whose status is not either :Complete or :Failed.
-
#respond_to_requests ⇒ Object
One of the features of CodeRunner is two way communication between a runner and its runs.
- #run_graphkit(*args) ⇒ Object
- #run_graphkit_shorthand(*grs) ⇒ Object
-
#runs ⇒ Object
Return an array of runs (run_list.values).
- #save_all ⇒ Object
-
#save_large_cache ⇒ Object
Write the variable
@run_list
, which contains all information currently known about the simulations in the root folder, as Marshalled binary data in the file “.CODE_RUNNER_TEMP_RUN_LIST_CACHE”. - #server_dump(obj) ⇒ Object
- #set_start_id(id) ⇒ Object
-
#setup_run_class(code, options) ⇒ Object
See CodeRunner.setup_run_class.
-
#similar_runs(exclude_variables = [], run = @run) ⇒ Object
Find runs whose input parameters are all the same as those for
run
, with the exception of those listed inexclude_variables
. -
#simple_scan(scan_string, options = {}) ⇒ Object
Submit a series of runs according to scan_string.
-
#sort_runs(type = @use_phantom.to_s.sub(/real/, '')) ⇒ Object
Sort the runs according to the variable
@sort
which can be either whitespace separated string or an array of strings. -
#submit(runs, options = {}) ⇒ Object
Start a simulation: submit the run that is passed.
- #sweep_graphkits(sweeps, options, &block) ⇒ Object
-
#update(write_status_dots = true, use_large_cache = @use_large_cache, use_large_cache_but_recheck_incomplete = @use_large_cache_but_recheck_incomplete) ⇒ Object
Update the information about all the runs stored in the variable
@run_list
. -
#write_data(filename = "data.txt") ⇒ Object
Write out a simple datafile containing all the inputs and outputs listed in the run class property
readout_string
.
Constructor Details
#initialize(root_folder, options = {}) ⇒ CodeRunner
Instantiate a new runner. The root folder contains a set of simulations, each of which has a unique ID. There is a one-to-one correspondence between a runner and a root folder: no two runners should ever be given the same root folder in the same script (there are safeguards to prevent this causing much trouble, but it should still be avoided on philosophical grounds), and no runner should be given a folder which has more than one set of simulations within it (as these simulations will contain duplicate IDs).
Options is a hash whose keys may be any of the keys of the constant DEFAULT_RUNNER_OPTIONS
. I.e. to see what options can be passed:
p CodeRunner::DEFAULT_RUNNER_OPTIONS.keys
163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 |
# File 'lib/coderunner/instance_methods.rb', line 163 def initialize(root_folder, ={}) logf :initialize raise CRFatal.new("System not defined") unless SYS root_folder.sub!(/~/, ENV['HOME']) @root_folder = root_folder read_defaults .each do |key,value| key = LONG_TO_SHORT.key(key) if LONG_TO_SHORT.key(key) set(key, value) if value end # ep options log 'modlet in initialize', @modlet @version= [:version] # ep 'ex', @executable @run_class = setup_run_class(@code, modlet: @modlet, version: @version, executable: @executable) @cache = {} @n_checks = 0 @print_out_size = 0 @run_list = {}; @ids = [] set_max_id(0) @phantom_run_list = {}; @phantom_ids = [] @phantom_id = -1 @combined_run_list = {}; @combined_ids = [] @current_request = nil @requests = [] @pids= [] @maxes = {}; @mins = {} @cmaxes = {}; @cmins = {} end |
Instance Attribute Details
#cache ⇒ Object
Returns the value of attribute cache.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def cache @cache end |
#cmaxes ⇒ Object (readonly)
Returns the value of attribute cmaxes.
154 155 156 |
# File 'lib/coderunner/instance_methods.rb', line 154 def cmaxes @cmaxes end |
#cmins ⇒ Object (readonly)
Returns the value of attribute cmins.
154 155 156 |
# File 'lib/coderunner/instance_methods.rb', line 154 def cmins @cmins end |
#code ⇒ Object
Returns the value of attribute code.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def code @code end |
#combined_ids ⇒ Object
Returns the value of attribute combined_ids.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def combined_ids @combined_ids end |
#combined_run_list ⇒ Object
Returns the value of attribute combined_run_list.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def combined_run_list @combined_run_list end |
#current_request ⇒ Object
Returns the value of attribute current_request.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def current_request @current_request end |
#current_status ⇒ Object
Returns the value of attribute current_status.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def current_status @current_status end |
#defaults_file ⇒ Object
Returns the value of attribute defaults_file.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def defaults_file @defaults_file end |
#executable ⇒ Object
Returns the value of attribute executable.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def executable @executable end |
#ids ⇒ Object
Returns the value of attribute ids.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def ids @ids end |
#max_id ⇒ Object (readonly)
Returns the value of attribute max_id.
154 155 156 |
# File 'lib/coderunner/instance_methods.rb', line 154 def max_id @max_id end |
#maxes ⇒ Object (readonly)
Returns the value of attribute maxes.
154 155 156 |
# File 'lib/coderunner/instance_methods.rb', line 154 def maxes @maxes end |
#mins ⇒ Object (readonly)
Returns the value of attribute mins.
154 155 156 |
# File 'lib/coderunner/instance_methods.rb', line 154 def mins @mins end |
#modlet ⇒ Object
Returns the value of attribute modlet.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def modlet @modlet end |
#phantom_ids ⇒ Object
Returns the value of attribute phantom_ids.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def phantom_ids @phantom_ids end |
#phantom_run_list ⇒ Object
Returns the value of attribute phantom_run_list.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def phantom_run_list @phantom_run_list end |
#print_out_size ⇒ Object
Returns the value of attribute print_out_size.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def print_out_size @print_out_size end |
#requests ⇒ Object
Returns the value of attribute requests.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def requests @requests end |
#root_folder ⇒ Object
Returns the value of attribute root_folder.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def root_folder @root_folder end |
#run_class ⇒ Object
Returns the value of attribute run_class.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def run_class @run_class end |
#run_list ⇒ Object
Returns the value of attribute run_list.
153 154 155 |
# File 'lib/coderunner/instance_methods.rb', line 153 def run_list @run_list end |
#start_id ⇒ Object (readonly)
Returns the value of attribute start_id.
154 155 156 |
# File 'lib/coderunner/instance_methods.rb', line 154 def start_id @start_id end |
Class Method Details
.available_defaults_files(copts = {}) ⇒ Object
List the available defaults files for the given code (copts or -C on the command line).
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
# File 'lib/coderunner/class_methods.rb', line 41 def self.available_defaults_files(copts={}) (copts) puts "\nAvailable defaults files for #{copts[:C]}:" entries = [] begin entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/my_defaults_files") rescue end begin entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/defaults_files") rescue end entries.each do |defaults_file| puts "\t" + File.basename(defaults_file, '.rb').sub(/_defaults/, '') unless ['.', '..', '.svn', '.directory'].include? defaults_file end end |
.available_modlets(copts = {}) ⇒ Object
List the available modlets for the given code (copts or -C on the command line).
22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
# File 'lib/coderunner/class_methods.rb', line 22 def self.available_modlets(copts={}) (copts) puts "\nAvailable modlets for #{copts[:C]}:" entries = [] begin entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/my_modlets") rescue end begin entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/default_modlets") rescue end entries.each do |modlet| puts "\t" + File.basename(modlet, '.rb') unless ['.', '..', '.svn', '.directory'].include? modlet or modlet=~ /defaults/ end end |
.cancel(id, copts = {}) ⇒ Object
Cancel the job with the given id. The user is asked interactively for confirmation and whether they would like to delete the folder for that job as well.
60 61 62 63 |
# File 'lib/coderunner/class_methods.rb', line 60 def self.cancel(id, copts={}) runner = fetch_runner(copts) runner.cancel_job(id.to_i) end |
.code_command(string, copts = {}) ⇒ Object
328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 |
# File 'lib/coderunner/class_methods.rb', line 328 def self.code_command(string, copts = {}) (copts) copts[:no_update] = true unless copts[:C] if FileTest.exist? file=Dir.pwd + '/.code_runner_script_defaults.rb' copts[:C] = eval(File.read(file))[:code] copts[:m] = eval(File.read(file))[:modlet] elsif self.runner copts[:C] = self.runner.code copts[:m] = self.runner.modlet end end #ep ['code', 'modlet is', copts[:C], copts[:m]] run_class = setup_run_class(copts[:C], modlet: copts[:m]) run_class.class_eval(string) # runner = fetch_runner(copts) # runner.run_class.class_eval(string) end |
.code_runner_execute(ruby_fragment, copts = {}) ⇒ Object
265 266 267 268 |
# File 'lib/coderunner/class_methods.rb', line 265 def self.code_runner_execute(ruby_fragment, copts={}) #eval(ruby_fragment, GLOBAL_BINDING) eval(ruby_fragment) end |
.continue_in_new_folder(folder, copts = {}) ⇒ Object
65 66 67 68 69 70 71 72 73 |
# File 'lib/coderunner/class_methods.rb', line 65 def self.continue_in_new_folder(folder, copts={}) runner=fetch_runner(copts) = {} if copts[:f] or copts[:j] [:copy_ids] = runner.filtered_ids end runner.continue_in_new_folder(folder, ) end |
.delete(copts = {}) ⇒ Object
75 76 77 78 |
# File 'lib/coderunner/class_methods.rb', line 75 def self.delete(copts={}) runner = fetch_runner(copts) runner.destroy end |
.directory(id, copts = {}) ⇒ Object
151 152 153 154 |
# File 'lib/coderunner/class_methods.rb', line 151 def self.directory(id, copts={}) runner = fetch_runner(copts) puts runner.run_list[id.to_i].directory end |
.execute(ruby_fragment, copts = {}) ⇒ Object
269 270 271 272 |
# File 'lib/coderunner/class_methods.rb', line 269 def self.execute(ruby_fragment, copts={}) eval(ruby_fragment, GLOBAL_BINDING) #eval(ruby_fragment) end |
.fetch_runner(copts = {}) ⇒ Object
Retrieve the runner with the folder (and possibly server) given in copts. If no runner has been loaded for that folder, load one.
603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 |
# File 'lib/coderunner/class_methods.rb', line 603 def self.fetch_runner(copts={}) # ep copts # If copts(:Y) is an array of locations, return a merged runner of those locations if copts[:Y].kind_of? Array runners = copts[:Y].map do |location| new_copts = copts.dup.absorb(Y: location) fetch_runner(new_copts) end return Merged.new(*runners) end (copts) # ep copts @runners ||= {} runner = nil if copts[:Y] and copts[:Y] =~ /:/ copts_r = copts.dup host, folder = copts[:Y].split(':') copts_r[:Y] = nil copts[:Y] = nil unless @runners[[host, folder]] runner = @runners[[host, folder]] = RemoteCodeRunner.new(host, folder, copts) (eputs 'Updating remote...'; runner.update) unless (copts[:g] and (copts[:g].kind_of? String or copts[:g].size > 0)) or copts[:no_update] or copts[:cache] else runner = @runners[[host, folder]] end runner.process_copts(copts) else copts[:Y] ||= Dir.pwd Dir.chdir((copts[:Y] or Dir.pwd)) do unless @runners[copts[:Y]] runner = @runners[copts[:Y]] = CodeRunner.new(Dir.pwd, code: copts[:C], modlet: copts[:m], version: copts[:v], executable: copts[:X], defaults_file: copts[:D]) runner.update unless copts[:no_update] else runner = @runners[copts[:Y]] end # p 'reading defaults', @r.conditions, DEFAULT_RUNNER_OPTIONS runner.read_defaults # p 'read defaults', @r.conditions end #Dir.chdir end # ep copts return runner # @r.read_defaults end |
.film(copts = {}) ⇒ Object
155 156 157 158 159 |
# File 'lib/coderunner/class_methods.rb', line 155 def self.film(copts={}) runner = fetch_runner(copts) copts[:F][:graphkit_modify] = copts[:w] runner.make_film_from_lists(copts[:G], copts[:g], copts[:F]) end |
.film_graphkit_frame_array(graphkit_frame_array, options) ⇒ Object
648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 |
# File 'lib/coderunner/graphs_and_films.rb', line 648 def self.film_graphkit_frame_array(graphkit_frame_array, ) = [:frame_array, :fa, :skip_frames, :sf, :normalize, :n, :normalize_pieces, :np, :increment, :i, :skip_encoding, :se, :frame_rate, :fr, :size] # fa = (options[:frame_array] or options[:fa] or list[0][0].run_list[list[0][0].filtered_ids[0]].frame_array(options) ) fd = frame_digits = [:fd]||Math.log10(graphkit_frame_array.map{|f, g| f}.max).ceil extension = ([:extension] or [:ext] or '.png') extension = '.' + extension unless extension =~ /^\./ unless [:skip_frames] or [:sf] FileUtils.rm_r('film_frames') if FileTest.exist?('film_frames') FileUtils.makedirs('film_frames') # puts @@multiple_processes; gets no_forks = (@@multiple_processes or 1) ep @@multiple_processes, no_forks # end_graphkit = graphkit_multiple_runners(list, frame_index: fa[1]) # begin_graphkit = graphkit_multiple_runners(list, frame_index: fa[0]) # end_area = end_graphkit.plot_area_size # begin_area = begin_graphkit.plot_area_size # p end_area, begin_area, options # plot_size = {} # axes = [:x, :y, :z] # options[:normalize] ||= options[:nm] # options[:normalize_pieces] ||= options[:nmp] # for i in 0...end_area.size # next unless options[:normalize] and options[:normalize].include? axes[i] # min = [end_area[i][0], begin_area[i][0]].min # max = [end_area[i][1], begin_area[i][1]].max # key = axes[i] # plot_size[key + :range] = [min, max] # end # ep plot_size # exit # frames = [] # actual_frames = {} # i = fa[0] # j = 0 # while i <= fa[1] # frames.push i # actual_frames[i] = j # i += (options[:ic] or options[:increment] or 1) # j += 1 # end i = 0 actual_frames = graphkit_frame_array.map{|f, g| f}.inject({}){|hash, f| hash[f] = i; i+=1; hash} graphkit_frame_array.pieces(no_forks).each_with_index do |graphkit_frame_array_piece, myrank| fork do # if options[:normalize_pieces] # end_area = graphkit_multiple_runners(list, frame_index: piece.max).plot_area_size # begin_area = graphkit_multiple_runners(list, frame_index: piece.min).plot_area_size # axes = [:x, :y, :z] # for i in 0...end_area.size # next unless options[:normalize_pieces].include? axes[i] # min = [end_area[i][0], begin_area[i][0]].min # max = [end_area[i][1], begin_area[i][1]].max # key = axes[i] # # plot_size[key + :range] = [min, max] # end # end # eputs 'making graphs...'; sleep 1 # graph_array = graphkit_multiple_runners_with_frame_array(piece, list, myrank==0) # # ep graph_array # eputs graphkit_frame_array_piece.each_with_index do |(frame_index,g), pindex| if myrank == 0 eputs "\033[2A" # Terminal jargon - go back one line eputs sprintf("Plotting graphs: %2.2f", pindex.to_f/graphkit_frame_array_piece.size.to_f * 100.0) + "% Complete" end # g = graph_kit_multiple_runners(list, plot_size + {frame_index: frame_index}) # g.modify(plot_size) # g.modify(options) # p g; exit # g.title += sprintf(", frame %0#{fd}d", frame_index) unless options[:frame_title] == false folder = ("film_frames/"); file_name = sprintf("frame_%0#{fd}d", actual_frames[frame_index]) # g.gnuplot; gets; g.close # ep folder + file_name + '.png'; gets g.gnuplot_write(folder + file_name + extension, size: [:size]) end end end eputs "Waiting on subprocesses..." Process.waitall end unless [:skip_encoding] eputs "making film" frame_rate = ([:frame_rate] or [:fr] || 15) film_name = ([:film_name] or [:fn] or graphkit_frame_array[0][1].file_name + '_film').gsub(/\s/, '_') puts `ffmpeg -y #{[:bitrate] ? "-b #{[:bitrate]}" : ""} -r #{frame_rate} -threads #{(@multiple_processes or 1)} -i film_frames/frame_%0#{fd}d#{extension} -sameq #{film_name}.mp4` end end |
.generate_documentation(username = nil, copts = {}) ⇒ Object
161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 |
# File 'lib/coderunner/class_methods.rb', line 161 def self.generate_documentation(username = nil, copts = {}) ep 'username', username||=ENV['USER'] ####### Here we use the command line documentation to generate a fake ruby file that rdoc will understand. File.open("class_methods_rdoc.rb", 'w') do |file| file.puts <<EOF class CodeRunner #{COMMANDS_WITH_HELP.inject("") do |str, (long, short, nargs, comhelp, argnames, )| (puts "Please run this command in the coderunner trunk directory"; exit) unless Dir.pwd =~ /coderunner\/trunk$/ str << <<EOF2 # #{comhelp.gsub(/\n/, "\n # ")} # # Possible options: # #{.inject("") do |str, opt| longop, shortop, req, ophelp = COMMAND_LINE_FLAGS_WITH_HELP.find{|arr| arr[1] == "-" + opt.to_s} str << " # :#{opt} --- #{ophelp.gsub(/\n/, "\n # ")}\n #\n" end} def self.#{long}(#{(argnames+[""]).join(",")}command_options={}) end EOF2 end } end EOF end # exit system "rm -rf doc/" system "rm -rf ri/" raise 'Please set RDOC_COMMAND' unless ENV['RDOC_COMMAND'] system "#{ENV['RDOC_COMMAND']} --format=html -t 'CodeRunner Documentation' -m INDEX.rb INDEX.rb code_runner_version.rb gnuplot.rb graphkit_gnuplot.rb graphkit.rb gsl_tools.rb run_backwards_compatibility.rb feedback.rb run.rb fortran_namelist.rb graphs_and_films.rb class_methods_rdoc.rb instance_methods.rb" system "#{ENV['RDOC_COMMAND']} -r --op ri INDEX.rb code_runner_version.rb gnuplot.rb graphkit_gnuplot.rb graphkit.rb gsl_tools.rb run_backwards_compatibility.rb feedback.rb run.rb fortran_namelist.rb graphs_and_films.rb class_methods_rdoc.rb instance_methods.rb" exit if username == "" string = "rsync -av --delete doc/ #{username},[email protected]:htdocs/api_documentation/" puts string exec string end |
.get_run_class_name(code, modlet = nil) ⇒ Object
Return the name of the run class according to the standard CodeRunner naming scheme. If the code name is ‘a_code_name’, with no modlet, the run class name will be ACodeName
. If on the other hand there is a modlet called ‘modlet_name’, the class name will be ACodeName::ModletName
.
333 334 335 |
# File 'lib/coderunner/instance_methods.rb', line 333 def self.get_run_class_name(code, modlet=nil) return modlet ? "#{code.capitalize}::#{modlet.capitalize.sub(/\.rb$/, '').variable_to_class_name}" : "#{code.capitalize}" end |
.gets ⇒ Object
42 43 44 |
# File 'lib/coderunner.rb', line 42 def self.gets $stdin.gets end |
.graphkit_multiple_runners(list, options = {}) ⇒ Object
list is an array of [[runner, [graphs, run_graphs]], … ]
424 425 426 427 428 429 430 431 432 |
# File 'lib/coderunner/graphs_and_films.rb', line 424 def self.graphkit_multiple_runners(list, ={}) return list.inject(nil) do |kit, (runner, graph_lists)| graphs, run_graphs = graph_lists graphs.map!{|graph| runner.graphkit_shorthand(graph)} run_graphs.map!{|graph| runner.run_graphkit_shorthand(graph)} newkit = runner.graphkit_from_lists(graphs, run_graphs, ) kit ? kit + newkit : newkit end end |
.graphkit_multiple_runners_with_frame_array(frame_array, list, print_message = false) ⇒ Object
434 435 436 437 438 439 440 441 442 443 444 445 |
# File 'lib/coderunner/graphs_and_films.rb', line 434 def self.graphkit_multiple_runners_with_frame_array(frame_array, list, = false) i = 0 # return list.inject(nil) do |kit_array, (runner, graph_lists)| graphs, run_graphs = graph_lists graphs.map!{|graph| runner.graphkit_shorthand(graph)} run_graphs.map!{|graph| runner.run_graphkit_shorthand(graph)} newkit_array = runner.graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, {}) # eputs newkit_array.pretty_inspect kit_array ? (i=-1; kit.map{|(frame_index, kit)| i+= 1;[frame_index, new_kit[i][1]]}) : newkit_array end end |
.interactive_mode(copts = {}) ⇒ Object
module InteractiveMethods
53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 |
# File 'lib/coderunner/interactive_methods.rb', line 53 def CodeRunner.interactive_mode(copts={}) (copts) unless false and FileTest.exist? (ENV['HOME'] + '/code_runner_interactive_options.rb') File.open(ENV['HOME'] + '/.code_runner_interactive_options.rb', 'w') do |file| file.puts <<EOF $has_put_startup_message_for_code_runner = true #please leave! $code_runner_interactive_mode = true #please leave! require 'yaml' def reset Dispatcher.reset_application! end IRB.conf[:AUTO_INDENT] = true IRB.conf[:USE_READLINE] = true IRB.conf[:LOAD_MODULES] = [] unless IRB.conf.key?(:LOAD_MODULES) unless IRB.conf[:LOAD_MODULES].include?('irb/completion') IRB.conf[:LOAD_MODULES] << 'irb/completion' end require 'irb/completion' require 'irb/ext/save-history' require 'socket' unless ENV['CODE_RUNNER_SYSTEM'] == 'macosx' prompt_start = "\001#{Terminal::LIGHT_GREEN}\002CodeRunner\001#{Terminal::RESET}\002 v#{CODE_RUNNER_VERSION} - \001#{Terminal::CYAN}\002#\{ENV['USER']\}@#\{Socket::gethostname\}:#\{File.basename(Dir.pwd)}\001#{Terminal::RESET}\002 timemarker\n" prompt_start = "\001#{Terminal::LIGHT_GREEN}\002CodeRunner\001#{Terminal::RESET}\002 v#{CODE_RUNNER_VERSION} - \001#{Terminal::CYAN}\002#\{Socket::gethostname\}:#\{File.basename(Dir.pwd)}\001#{Terminal::RESET}\002 timemarker %02n,%i \n" else prompt_start = "CodeRunner #\{File.basename(Dir.pwd)}" end IRB.conf[:PROMPT][:CODE_RUNNER] = {:PROMPT_I=>"#\{prompt_start}>> %02n,%i >> ", :PROMPT_N=>"#\{prompt_start}>> %02n:%i > ", :PROMPT_S=>"#\{prompt_start}>> %02n:%i (%l)> ", :PROMPT_C=>"#\{prompt_start}>> %02n:%i >> ", :RETURN=>""} IRB.conf[:PROMPT][:CODE_RUNNER] = {:PROMPT_I=>"#\{prompt_start}>> ", :PROMPT_N=>"#\{prompt_start}> ", :PROMPT_S=>"#\{prompt_start}(%l)> ", :PROMPT_C=>"#\{prompt_start}>> ", :RETURN=>""} # IRB.conf[:PROMPT][:CODE_RUNNER] = {:PROMPT_I=>"#\{prompt_start} %02n,%i>> ", :PROMPT_N=>"#\{prompt_start} %02n:%i> ", :PROMPT_S=>"#\{prompt_start} %02n:%i (%l)> ", :PROMPT_C=>"#\{prompt_start} %02n:%i>> ", :RETURN=>""} IRB.conf[:PROMPT_MODE] = :CODE_RUNNER IRB.conf[:SAVE_HISTORY] = 400 IRB.conf[:HISTORY_FILE] = "\#\{Dir.pwd}/.code-runner-irb-save-history" IRB.conf[:INSPECT_MODE] = false EOF end # File.open end # unless File.open(".int.tmp.rb", 'w')do |file| file.puts "#{copts.inspect}.each do |key, val| CodeRunner::DEFAULT_COMMAND_OPTIONS[key] = val end" file.puts CodeRunner::InteractiveMethods::INTERACTIVE_METHODS end # asdfa exec %[#{RbConfig::CONFIG['bindir']}/irb#{RbConfig::CONFIG['ruby_install_name'].sub(/ruby/, '')} -f -I '#{Dir.pwd}' -I '#{SCRIPT_FOLDER}' -I '#{ENV['HOME']}' -r '.code_runner_interactive_options' -r 'coderunner' -r .int.tmp ] end |
.load_file(files, copts = {}) ⇒ Object
eval(ruby_fragment)
273 274 275 276 277 278 279 280 281 282 283 284 285 |
# File 'lib/coderunner/class_methods.rb', line 273 def self.load_file(files, copts={}) (copts) # begin files.split(/\s*,\s*/).each do |file| # p file raise ArgumentError.new("#{file} is not a file.") unless File.file? file load file end # rescue # eval(files) # end end |
.make_film_multiple_runners(list, options) ⇒ Object
__END__
543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 |
# File 'lib/coderunner/graphs_and_films.rb', line 543 def self.make_film_multiple_runners(list, ) = [:frame_array, :fa, :skip_frames, :sf, :normalize, :n, :normalize_pieces, :np, :increment, :i, :skip_encoding, :se] fa = ([:frame_array] or [:fa] or list[0][0].run_list[list[0][0].filtered_ids[0]].frame_array() ) fd = frame_digits = Math.log10(fa[1]).ceil unless [:skip_frames] or [:sf] `rm -rf film_frames` extension = ([:extension] or [:ext] or '.png') extension = '.' + extension unless extension =~ /^\./ FileUtils.makedirs('film_frames') # puts @@multiple_processes; gets no_forks = (@@multiple_processes or 1) ep @@multiple_processes, no_forks end_graphkit = graphkit_multiple_runners(list, frame_index: fa[1]) begin_graphkit = graphkit_multiple_runners(list, frame_index: fa[0]) end_area = end_graphkit.plot_area_size begin_area = begin_graphkit.plot_area_size # p end_area, begin_area, options plot_size = {} axes = [:x, :y, :z] [:normalize] ||= [:nm] [:normalize_pieces] ||= [:nmp] for i in 0...end_area.size next unless [:normalize] and [:normalize].include? axes[i] min = [end_area[i][0], begin_area[i][0]].min max = [end_area[i][1], begin_area[i][1]].max key = axes[i] plot_size[key + :range] = [min, max] end ep plot_size # exit frames = [] actual_frames = {} i = fa[0] j = 0 while i <= fa[1] frames.push i actual_frames[i] = j i += ([:ic] or [:increment] or 1) j += 1 end frames.pieces(no_forks).each_with_index do |piece, myrank| fork do if [:normalize_pieces] end_area = graphkit_multiple_runners(list, frame_index: piece.max).plot_area_size begin_area = graphkit_multiple_runners(list, frame_index: piece.min).plot_area_size axes = [:x, :y, :z] for i in 0...end_area.size next unless [:normalize_pieces].include? axes[i] min = [end_area[i][0], begin_area[i][0]].min max = [end_area[i][1], begin_area[i][1]].max key = axes[i] plot_size[key + :range] = [min, max] end end eputs 'making graphs...'; sleep 1 graph_array = graphkit_multiple_runners_with_frame_array(piece, list, myrank==0) # ep graph_array eputs graph_array.each_with_index do |(frame_index,g), pindex| if myrank == 0 eputs "\033[2A" # Terminal jargon - go back one line eputs sprintf("Plotting graphs: %2.2f", pindex.to_f/piece.size.to_f * 100.0) + "% Complete" end # g = graph_kit_multiple_runners(list, plot_size + {frame_index: frame_index}) g.modify(plot_size) g.modify() # p g; exit g.title += sprintf(", frame %0#{fd}d", frame_index) unless [:frame_title] == false folder = ("film_frames/"); file_name = sprintf("frame_%0#{fd}d", actual_frames[frame_index]) # g.gnuplot; gets; g.close # ep folder + file_name + '.png'; gets g.gnuplot_write(folder + file_name + extension) end end end eputs "Waiting on subprocesses..." Process.waitall end unless [:skip_encoding] eputs "making film" frame_rate = ([:frame_rate] or [:fr] || 15) film_name = ([:film_name] or [:fn] or end_graphkit.file_name + '_film').gsub(/\s/, '_') puts `ffmpeg -y #{[:bitrate] ? "-b #{[:bitrate]}" : ""} -r #{frame_rate} -threads #{(@multiple_processes or 1)} -i film_frames/frame_%0#{fd}d#{extension} -sameq #{film_name}.mp4` end end |
.manual(copts = {}) ⇒ Object
660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 |
# File 'lib/coderunner/class_methods.rb', line 660 def self.manual(copts={}) help = <<EOF -------------CodeRunner Manual--------------- Written by Edmund Highcock (2009) NAME coderunner SYNOPSIS coderunner <command> [arguments] [options] DESCRIPTION CodeRunner is a framework for the running and analysis of large simulations. It is a Ruby package and can be used to write Ruby scripts. However it also has a powerful command line interface. The aim is to be able to submit simulations, analyse data and plot graphs all using simple commands. This manual is a quick reference. For a more tutorial style introduction to CodeRunner go to http://coderunner.sourceforge.net This help page documents the commandline interface. For API documentation see http://coderunner.sourceforge.net/api_documentation As is standard, <> indicates a parameter to be supplied, and [] indicates an option, unless otherwise stated. EXAMPLES $ coderunner sub -p '{height: 34.2, width: 221}' -n 24x4 -W 300 $ coderunner can 34 -U $ coderunner plot -G 'height:width;{};depth==2.4 and status == :Completed;height' $ coderunner st -ul $ coderunner rc 'p status' -U COMMANDS Either the long or the short form of the command may be used, except in interactive mode, where only short form can be used. Long(Short) <Arguments> (Meaningful Options) --------------------------------------------- #{(COMMANDS_WITH_HELP.sort_by{|arr| arr[0]}.map do |arr| sprintf(" %s %s(%s) \n %s", "#{arr[0]}(#{arr[1]})", arr[4].map{|arg| "<#{arg}>"}.join(' ').sub(/(.)$/, '\1 '), arr[5].map{|op| op.to_s}.join(','), arr[3], ) end).join("\n\n")} OPTIONS #{((COMMAND_LINE_FLAGS_WITH_HELP + LONG_COMMAND_LINE_OPTIONS).map do |arr| sprintf(" %-15s %-2s\n %s", arr[0], arr[1], arr[3]) end).join("\n\n") } EOF puts help.gsub(/(.{63,73} |.{73})/){"#$1\\\n "} end |
.netcdf_plot(netcdf_file, vars, indices, copts = {}) ⇒ Object
def self.executable_name # :nodoc: “” end def self.rcp # :nodoc: @rcp ||= KitHash.new end
87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
# File 'lib/coderunner/class_methods.rb', line 87 def self.netcdf_plot(netcdf_file, vars, indices, copts={}) (copts) begin require "numru/netcdf" rescue LoadError eputs "Error: No Ruby NetCDF library (was it installed correctly?): data analysis for netcdf files not possible." return end start_indices = indices.split(',').map{|idx| idx = idx.split(':')[0] if idx =~ /:/ ; eval(idx) || 0} end_indices = indices.split(',').map{|idx| idx = idx.split(':')[1] if idx =~ /:/ ; eval(idx) || -1} ep 'start_indices', start_indices, 'end_indices', end_indices file = NumRu::NetCDF.open(netcdf_file) to_plot = vars.split(',').map do |var| ep 'var', var [file.var(var).get('start'=> start_indices, 'end'=> end_indices).to_a.flatten] end ep 'to_plot', to_plot kit = GraphKit.quick_create(*to_plot) ep 'copts', copts kit.instance_eval(copts[:w]) if copts[:w] kit.gnuplot STDIN.gets kit.close end |
.old_get_run_class_name(code, modlet) ⇒ Object
:nodoc:
327 328 329 |
# File 'lib/coderunner/instance_methods.rb', line 327 def self.old_get_run_class_name(code, modlet) # :nodoc: return modlet ? "#{code.capitalize}#{modlet.capitalize.sub(/\.rb$/, '').sub(/_(\w)/){"#$1".capitalize}}Run" : "#{code.capitalize}Run" end |
.parameter_scan(parameter_scan_array_file, copts = {}) ⇒ Object
287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 |
# File 'lib/coderunner/class_methods.rb', line 287 def self.parameter_scan(parameter_scan_array_file, copts={}) parameter_scan_array = eval(File.read(parameter_scan_array_file)) # process_copts(copts) runner = fetch_runner(copts) skip = true unless copts[:k] == false folder = Dir.pwd Log.logf("self.parameter_scan") # @@runners = {} @@mutex = Mutex.new # @runner = new(folder, code: copts[:C], modlet: copts[:m], version: copts[:v], executable: copts[:X]) @@psppipe = PPipe.new(parameter_scan_array.size + 2, true, controller_refresh: 0.5, redirect: false) parameter_scan_array.each do |parameter_scan| @@psppipe.fork do runner.parameter_scan(parameter_scan, copts[:p][0], skip: skip, nprocs: copts[:n]) end end @@psppipe.finish @@psppipe = nil end |
.plot_graph(copts = {}) ⇒ Object
306 307 308 309 310 311 312 313 314 315 316 |
# File 'lib/coderunner/class_methods.rb', line 306 def self.plot_graph(copts = {}) # process_copts(copts) runner = fetch_runner(copts) string_to_eval = copts[:w] #options = (options and options =~ /\S/) ? eval(options): {} eputs 'Starting Graph' kit = runner.graphkit_from_lists(copts[:G], copts[:g]) kit.gnuplot(eval: string_to_eval) gets kit.close end |
.print_queue_status(copts = {}) ⇒ Object
115 116 117 118 119 120 121 122 123 124 |
# File 'lib/coderunner/class_methods.rb', line 115 def self.print_queue_status(copts={}) begin eputs queue_status rescue => err eputs "General queue status doesn't work on this system; showing queue status for this folder" # p err runner = fetch_runner(copts) eputs runner.queue_status end end |
.process_command_line_option(opt, arg, copts) ⇒ Object
Converts a command line flag opt with value arg to a command option which is stored in copts
270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 |
# File 'lib/coderunner.rb', line 270 def self.process_command_line_option(opt, arg, copts) case opt when "--change-directory" copts[:c] = arg.to_i when "--delete" copts[:x] = arg.to_i when "--graph" copts[:G].push arg when "--run-graph" copts[:g].push arg # when "--cancel" # copts[:K] = arg.to_i when "--multiple-processes" copts[:M] = arg.to_i when "--film" copts[:F] = (arg or true) when "--recheck" copts[:R] = arg.to_i when "--use-large-cache-but-recheck-incomplete" copts[:U] = true copts[:u]=true when "--wall-mins" copts[:W] = arg.to_i when "--use-phantom" copts[:h] = (arg and arg =~ /\S/) ? arg.to_sym : :phantom when "--terminal-size" array = eval arg ENV['ROWS'], ENV['COLS'] = array[0].to_s, array[1].to_s when "--interactive-mode" @@interactive_mode = true when "--parameters" copts[:p].push arg else if CLF_BOOLS.include? CLF_TO_SHORT_COPTS[opt] copts[CLF_TO_SHORT_COPTS[opt]] = true elsif CLF_INVERSE_BOOLS.include? CLF_TO_SHORT_COPTS[opt] copts[CLF_TO_SHORT_COPTS[opt]] = false elsif CLF_TO_SHORT_COPTS[opt] # Applies to most options copts[CLF_TO_SHORT_COPTS[opt]] = arg elsif CLF_BOOLS.include? CLF_TO_LONG[opt] copts[CLF_TO_LONG[opt]] = true elsif CLF_INVERSE_BOOLS.include? CLF_TO_LONG[opt] copts[CLF_TO_LONG[opt]] = false elsif CODE_COMMAND_OPTIONS.map{|o| o[0]}.include? opt begin #copts[:code_copts] ||= {} copts[ #CLF_TO_LONG[opt].to_s.sub('_options','').to_sym CLF_TO_LONG[opt] ] = eval(arg) rescue SyntaxError => err eputs "\nOption #{opt} must be a hash\n\n" raise err end elsif CLF_TO_LONG[opt] copts[CLF_TO_LONG[opt]] = arg else raise "Unknown command line argument: #{opt}" end end copts end |
.process_command_options(copts) ⇒ Object
544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 |
# File 'lib/coderunner/class_methods.rb', line 544 def self.(copts) if copts[:true] copts[:true].to_s.split(//).each do |letter| copts[letter.to_sym] = true end end if copts[:false] copts[:false].to_s.split(//).each do |letter| copts[letter.to_sym] = false end end (copts) copts.each do |key, value| copts[LONG_TO_SHORT[key]] = value if LONG_TO_SHORT[key] end if copts[:j] # j can be a number '65' or list of numbers '65,43,382' copts[:f]= "#{eval("[#{copts[:j]}]").inspect}.include? id" end if copts[:z] Log.log_file = Dir.pwd + '/.cr_logfile.txt' Log.clean_up else Log.log_file = nil end copts[:F] = (copts[:F].class == Hash ? copts[:F] : (copts[:F].class == String and copts[:F] =~ /\A\{.*\}\Z/) ? eval(copts[:F]) : {}) copts[:G]= [copts[:G]] if copts[:G].kind_of? String copts[:g]= [copts[:g]] if copts[:g].kind_of? String # if copts[:p] and copts[:p].class == String # should be a hash or an inspected hash # copts[:p] = eval(copts[:p]) # end copts[:p] = [copts[:p]].compact unless copts[:p].class == Array #for i in 0...copts[:p].size copts[:Y] ||= DEFAULT_COMMAND_OPTIONS[:Y] if DEFAULT_COMMAND_OPTIONS[:Y] if copts[:Y] and copts[:Y] =~ /:/ set_class_defaults(copts) copts[:running_remotely] = true else copts[:Y].gsub!(/~/, ENV['HOME']) if copts[:Y] Dir.chdir((copts[:Y] or Dir.pwd)) do set_runner_defaults(copts) # ep DEFAULT_RUNNER_OPTIONS end end # ep Log.log_file #copts[:code_copts].each{|k,v| CODE_OPTIONS[k] = v} if copts[:code_copts] copts.keys.map{|k| k.to_s}.grep(/_options$/).map{|k| k.to_sym}.each do |k| CODE_OPTIONS[k.to_s.sub('_options','').to_sym] = copts[k] end end |
.read_default_command_options(copts) ⇒ Object
539 540 541 542 543 |
# File 'lib/coderunner/class_methods.rb', line 539 def self.(copts) DEFAULT_COMMAND_OPTIONS.each do |key, value| copts[key] ||= value end end |
.readout(copts = {}) ⇒ Object
317 318 319 320 321 |
# File 'lib/coderunner/class_methods.rb', line 317 def self.readout(copts={}) # process_copts(copts) runner = fetch_runner(copts) puts runner.readout end |
.recheck(id, copts = {}) ⇒ Object
322 323 324 325 326 327 |
# File 'lib/coderunner/class_methods.rb', line 322 def self.recheck(id, copts={}) # process_copts(copts) runner = fetch_runner(copts) runner.run_list[copts[:R]].recheck runner.respond_to_requests end |
.reference(class_or_method, copts = {}) ⇒ Object
130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 |
# File 'lib/coderunner/class_methods.rb', line 130 def self.reference(class_or_method, copts={}) code_folders = Dir.recursive_entries(SCRIPT_FOLDER + '/code_modules').grep(/\/ri$/).map{|fold| ['-d', fold]}.flatten # ep code_folders # require 'rdoc/ri/driver' # " # op = @ri_count ? [] : (@ri_count = true; ['--no-use-cache']) # trap(1){puts 'No help available'} # at_exit{raise ""} # p op begin eputs "Looking up #{class_or_method}" RDoc::RI::Driver.run ['-d', SCRIPT_FOLDER + '/ri', class_or_method.to_s] + code_folders rescue => err eputs "Unknown class or method or no help available: #{err}" end # trap(1){} end |
.repair_marshal_run_class_not_found_error(err) ⇒ Object
338 339 340 341 342 343 344 |
# File 'lib/coderunner/instance_methods.rb', line 338 def self.repair_marshal_run_class_not_found_error(err) code, modlet = err..scan(/CodeRunner\:\:([A-Z][a-z0-9_]+)(?:::([A-Z]\w+))?/)[0] #ep 'merror', err, code, modlet; gets code.gsub!(/([a-z0-9])([A-Z])/, '\1_\2') (modlet.gsub!(/([a-z0-9])([A-Z])/, '\1_\2'); modlet.downcase) if modlet setup_run_class(code.downcase, modlet: modlet) end |
.resubmit(copts = {}) ⇒ Object
429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 |
# File 'lib/coderunner/class_methods.rb', line 429 def self.resubmit(copts = {}) # process_copts(copts) runner = fetch_runner(copts) # raise "something is already submitting" if FileTest.exist? "submitting" runs = [] raise "Parameters must be an array of inspected hashes" unless copts[:p].kind_of? Array Dir.chdir(copts[:Y]) do runs = runner.filtered_ids.map do |id| eputs id run = runner.run_list[id].dup if copts[:smart_resubmit_name] eputs "Smart name" run.set(:naming_pars, [:resubmit_id]) run.resubmit_id = run.id end run.update_submission_parameters(copts[:p][0], false) run.run_name = nil unless copts[:rerun] run end end #throw(:here) runner.submit(runs, nprocs: copts[:n], version: copts[:v], skip: copts[:k], job_chain: copts[:J], no_update_before_submit: copts[:no_update_before_submit], replace_existing: copts[:replace_existing], smart_resubmit_name: copts[:smart_resubmit_name], rerun: copts[:rerun]) end |
.run_command(string, copts = {}) ⇒ Object
runner = fetch_runner(copts) runner.run_class.class_eval(string)
348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 |
# File 'lib/coderunner/class_methods.rb', line 348 def self.run_command(string, copts={}) # process_copts(copts) runner = fetch_runner(copts) eputs "Calling run_commmand..." # puts "Warning: Use large cache is on (-U or -u) -- no results will be saved" if runner.use_large_cache ppipe = PPipe.new(runner.filtered_ids.size + 1, false) if copts[:M] no_save = (runner.class == RemoteCodeRunner or copts[:y] =~ /no-save/) # runner.generate_combined_ids # ep runner.filtered_ids runner.filtered_ids.each do |id| run = runner.combined_run_list[id] if no_save or run.is_phantom if copts[:M] fork{run.instance_eval(string)} else run.instance_eval(string) end else if copts[:M] pn = ppipe.fork do Dir.chdir(run.directory) do run.instance_eval(string); run.save run.write_results end ppipe.i_send(id, Marshal.dump(run), tp: 0) end else Dir.chdir(run.directory){run.instance_eval(string); run.save; run.write_results} end end end unless no_save (runner.filtered_ids.each{|id| runner.run_list[id] = Marshal.load(ppipe.w_recv(id).contents)};ppipe.finish) if copts[:M] runner.save_large_cache end # Process.waitall runner.respond_to_requests end |
.run_script ⇒ Object
367 368 369 370 371 372 373 374 375 |
# File 'lib/coderunner.rb', line 367 def self.run_script if DEFAULT_COMMAND_OPTIONS[:Z] DEFAULT_COMMAND_OPTIONS.absorb(eval(DEFAULT_COMMAND_OPTIONS[:Z])) puts "Begin Output" end command = COMMANDS.find{|com| com.slice(0..1).include? ARGV[0]} raise "\n-------------------\nCommand #{ARGV[0].inspect} not found: try 'coderunner man' for help\n-------------------\n" unless command send(command[0].to_sym, *ARGV.values_at(*(1...(1+command[2])).to_a)) end |
.runner ⇒ Object
655 656 657 |
# File 'lib/coderunner/class_methods.rb', line 655 def self.runner @runners.values[0] end |
.runner_eval(string, copts = {}) ⇒ Object
391 392 393 394 395 396 397 398 399 400 401 402 403 |
# File 'lib/coderunner/class_methods.rb', line 391 def self.runner_eval(string, copts = {}) # process_copts(copts) runner = fetch_runner(copts) return_val = runner.instance_eval(string) if copts[:Z] Kernel.puts(server_dump(return_val)) else return return_val end end |
.scan(scan_string, copts = {}) ⇒ Object
404 405 406 407 408 |
# File 'lib/coderunner/class_methods.rb', line 404 def self.scan(scan_string, copts={}) # process_copts(copts) runner = fetch_runner(copts) runner.simple_scan(scan_string, nprocs: copts[:n], version: copts[:v], skip: copts[:k], parameters: copts[:p][0]) end |
.server_dump(obj) ⇒ Object
298 299 300 |
# File 'lib/coderunner/instance_methods.rb', line 298 def self.server_dump(obj) "code_runner_server_dump_start_E#{Marshal.dump(obj)}code_runner_server_dump_end_E" end |
.set_class_defaults(copts = {}) ⇒ Object
13 14 15 16 17 18 |
# File 'lib/coderunner/class_methods.rb', line 13 def self.set_class_defaults(copts={}) (CLASS_OPTIONS.keys - []).each do |var| CLASS_OPTIONS[var] = copts[LONG_TO_SHORT[var]] set(var, CLASS_OPTIONS[var]) end end |
.set_default_command_options_from_command_line ⇒ Object
338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 |
# File 'lib/coderunner.rb', line 338 def self. #some defaults # DEFAULT_COMMAND_OPTIONS[:p] ||= {} DEFAULT_COMMAND_OPTIONS[:v] ||= "" #DEFAULT_COMMAND_OPTIONS[:n] ||= "1" DEFAULT_COMMAND_OPTIONS[:G] = [] DEFAULT_COMMAND_OPTIONS[:g] = [] DEFAULT_COMMAND_OPTIONS[:k] = true DEFAULT_COMMAND_OPTIONS[:h] ||= :real DEFAULT_COMMAND_OPTIONS[:p] = [] # ep COMMAND_LINE_FLAGS opts = GetoptLong.new(*(COMMAND_LINE_FLAGS + LONG_COMMAND_LINE_FLAGS)) opts.each do |opt, arg| process_command_line_option(opt, arg, DEFAULT_COMMAND_OPTIONS) end #raise "\n\nCannot use large cache ('-U' or '-u' ) if submitting runs" if DEFAULT_COMMAND_OPTIONS[:U] and (DEFAULT_COMMAND_OPTIONS[:s] or DEFAULT_COMMAND_OPTIONS[:P]) if DEFAULT_COMMAND_OPTIONS[:z] Log.log_file = Dir.pwd + '/.cr_logfile.txt' Log.clean_up else Log.log_file = nil # puts Log.log_file end end |
.set_runner_defaults(copts = {}) ⇒ Object
In the next section are the implementations of all the standard Code Runner commands and some helper functions.
5 6 7 8 9 10 11 |
# File 'lib/coderunner/class_methods.rb', line 5 def self.set_runner_defaults(copts = {}) (DEFAULT_RUNNER_OPTIONS.keys - [:sys, :script_folder]).each do |var| DEFAULT_RUNNER_OPTIONS[var] = copts[LONG_TO_SHORT[var]] end set_class_defaults(copts) # ep DEFAULT_RUNNER_OPTIONS end |
.setup_run_class(code, options = {}) ⇒ Object
353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 |
# File 'lib/coderunner/instance_methods.rb', line 353 def self.setup_run_class(code, ={}) # logf(:setup_code) # log(:code, code) modlet = [:modlet] version = [:version] # log('modlet in setup_run_class', modlet) eputs "Loading modules for #{code}, #{modlet.inspect}..." # modlet = modlet.sub(/\.rb$/, '') if modlet raise CRFatal.new("Code must contain only lowercase letters, digits, and underscore, and must begin with a letter or underscore; it is '#{code}'") unless code =~ /\A[a-z_][a-z_\d]*\Z/ raise CRFatal.new("Input_file_module must contain only lowercase letters, digits, and underscore, and must begin with a letter or underscore; it is '#{modlet}'") if modlet and not modlet =~ /\A[a-z_][a-z_\d]*\Z/ run_class_name = get_run_class_name(code, modlet) # p run_class_name FileUtils.makedirs(ENV['HOME'] + "/.coderunner/#{code}crmod/") return const_get(run_class_name) if constants.include? (run_class_name).to_sym unless [:force] SETUP_RUN_CLASSES.push run_class_name.downcase #Create the run_class, a special dynamically created class which knows how to process runs of the given code on the current system. #run.rb contains the basic methods of the class # puts run_class_name; gets # run_class = add_a_child_class(run_class_name, "") # run_class.class_eval(%[ # @@code=#{code.inspect}; @@version=#{version.inspect} # @@modlet=#{modlet.inspect} #modlet should be nil if not @@modlet_required # SYS = #{SYS.inspect} # include Log # Log.logi(:code_just_after_runfile, @@code) # ] # ) code_module_name = SCRIPT_FOLDER+ "/code_modules/#{code}/#{code}.rb" code_module_name = "#{code}crmod" require code_module_name # ep get_run_class_name(code, nil) run_class = const_get(get_run_class_name(code, nil)) run_class.instance_variable_set(:@code, code) raise "#{run_class} must inherit from CodeRunner::Run: its ancestors are: #{run_class.ancestors}" unless run_class.ancestors.include? Run if [:runner] run_class.runner = [:runner] run_class.instance_variable_set(:@runner, [:runner]) end #Add methods appropriate to the current system system_module_name = SCRIPT_FOLDER+ "/system_modules/#{SYS}.rb" require system_module_name run_class.send(:include, const_get(SYS.variable_to_class_name)) # run_class.class_eval(File.read(system_module_name), system_module_name) #Some codes require an modlet; the flag modlet_required is specified in the code module if run_class.rcp.modlet_required raise CRFatal.new("Modlet necessary and none given") unless modlet end #If a modlet is specified (modlets can also be optional) if modlet # Log.logi(:modlet, modlet) #if ( modlet_location = "#{SCRIPT_FOLDER}/code_modules/#{code}/default_modlets"; #file_name = "#{modlet_location}/#{modlet}.rb"; #FileTest.exist? file_name #) #elsif ( modlet_location = "#{SCRIPT_FOLDER}/code_modules/#{code}/my_modlets"; #file_name = "#{modlet_location}/#{modlet}.rb"; #FileTest.exist? file_name #) #else #raise CRFatal.new("Could not find modlet file: #{modlet}.rb") #end #require file_name # ep run_class.constants # ep run_class_names modlet_file = "#{code}crmod/#{modlet}.rb" #ep ['requiring modlet file'] require modlet_file run_class = recursive_const_get(run_class_name) run_class.instance_variable_set(:@modlet, modlet) end run_class.check_and_update # log("random element of variables", run_class.variables.random) # logi("run_class.variables[0] in setup_code", run_class.variables[0]) # ep 'finished' return run_class end |
.show_values_of(expression, copts = {}) ⇒ Object
484 485 486 487 |
# File 'lib/coderunner/class_methods.rb', line 484 def self.show_values_of(expression, copts={}) runner = fetch_runner(copts) p runner.filtered_ids.map{|id| runner.run_list[id].instance_eval(expression)}.uniq.sort end |
.start_launcher(refresh, max_queue, copts = {}) ⇒ Object
208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 |
# File 'lib/coderunner/class_methods.rb', line 208 def self.start_launcher(refresh, max_queue, copts={}) raise "Raise refresh is #{refresh}: it must be >= 1" if refresh.to_i < 1 raise "Raise max_queue is #{max_queue}: it must be >= 5" if max_queue.to_i < 5 #raise "Launcher already running" if %x[ps -e -o cmd].split("\n").grep(/coderunner\s+launch/).size > 0 require 'thread' tl = ENV['HOME'] + "/.coderunner_to_launch_#{ENV['CODE_RUNNER_LAUNCHER']}" #SCRIPT_FOLDER + '/to_launch' exit unless Feedback.get_boolean( "Launch directory #{tl} already exists: it is suggested that you change the prefix by changing the environment variable CODE_RUNNER_LAUNCHER. Do you wish to continue (don't select yes unless you know what you are doing)?") if FileTest.exist? tl # FileUtils.rm_r tl if FileTest.exist? tl eputs "Starting launcher\n" at_exit{FileUtils.rm_r tl} FileUtils.makedirs tl Thread.new{loop{`cp #{tl}/queue_status.txt #{tl}/queue_status2.txt; ps > #{tl}/queue_status.txt`; sleep 1}} mutex = Mutex.new processes= [] Thread.new do loop do Dir.entries(tl).each do |file| next unless file =~ (/(^.*)\.stop/) pid = $1 mutex.synchronize{Process.kill(pid); processes.delete pid} end sleep refresh.to_i end end Dir.chdir(tl) do ppid = $$ loop do sleep refresh.to_i while processes.size >= max_queue.to_i # processes = [] Dir.entries(tl).grep(/(^.*)\.start/).each do |file| file =~ (/(^.*)\.start/) id = $1 command = File.read file pid = fork do processes.each do |wpid| sleep refresh.to_i while %x[ps -e -o pid,ppid].split("\n").grep(Regexp.new("^\\s*#{wpid}\\s+#{ppid}")).size > 0 end exec(command) end `cp #{tl}/queue_status.txt #{tl}/queue_status2.txt; ps > #{tl}/queue_status.txt` mutex.synchronize{processes.push pid} File.open(id + '.pid', 'w'){|file| file.puts pid} FileUtils.rm(file) Thread.new{Process.wait pid; mutex.synchronize{processes.delete pid}} end # processes.each{|pid| Process.wait pid} sleep refresh.to_i end end end |
.status(copts = {}) ⇒ Object
492 493 494 495 496 |
# File 'lib/coderunner/class_methods.rb', line 492 def self.status(copts={}) # process_copts(copts) runner = fetch_runner(copts) runner.print_out(0, with_comments: copts[:with_comments]) unless copts[:interactive_start] or copts[:Z] or copts[:no_print_out] end |
.status_loop(copts = {}) ⇒ Object
497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 |
# File 'lib/coderunner/class_methods.rb', line 497 def self.status_loop(copts={}) # process_copts(copts) runner = fetch_runner(copts) runner.print_out(0, with_comments: copts[:with_comments]) unless copts[:interactive_start] or copts[:Z] or copts[:no_print_out] break_out = false loop do old_trap = trap(2){eputs " Terminating loop, please wait..."; break_out = true} runner.use_large_cache = true runner.update(false) (trap(2, old_trap); break) if break_out runner.recheck_filtered_runs(false) runner.print_out(nil, with_comments: copts[:with_comments]) trap(2, old_trap) break if break_out break if not runner.run_list.values.find do |r| not [:Complete, :Failed].include? r.status end #ep "sleep" sleep 3 #ep "end sleep" end end |
.status_with_comments(copts = {}) ⇒ Object
488 489 490 491 |
# File 'lib/coderunner/class_methods.rb', line 488 def self.status_with_comments(copts={}) copts[:with_comments] = true status(copts) end |
.submit(copts = {}) ⇒ Object
409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 |
# File 'lib/coderunner/class_methods.rb', line 409 def self.submit(copts = {}) # process_copts(copts) runner = fetch_runner(copts) # raise "something is already submitting" if FileTest.exist? "submitting" runs = [] raise "Parameters must be an array of inspected hashes" unless copts[:p].kind_of? Array Dir.chdir(copts[:Y]) do copts[:p].push nil if copts[:p] == [] # ep copts[:p]; exit copts[:p].each do |pars| run = runner.run_class.new(runner) # p pars run.update_submission_parameters(pars) runs.push run end # exit end runner.submit(runs, nprocs: copts[:n], version: copts[:v], skip: copts[:k], job_chain: copts[:J], no_update_before_submit: copts[:no_update_before_submit]) end |
.submit_command(jid, comm, copts = {}) ⇒ Object
This method allows the straightforward submission of a single command using the batch queue on any system.
455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 |
# File 'lib/coderunner/class_methods.rb', line 455 def self.submit_command(jid, comm, copts={}) (copts) submitter = Object.new submitter.instance_variable_set(:@command, comm) submitter.instance_variable_set(:@jid, jid) submitter.instance_variable_set(:@nprocs, copts[:n]) submitter.instance_variable_set(:@wall_mins, copts[:W]) submitter.instance_variable_set(:@project, copts[:P]) class << submitter include CodeRunner::SYSTEM_MODULE def executable_name 'custom' end def job_identifier @jid end def run_command @command end end submitter.execute end |
.update_runners ⇒ Object
@r.read_defaults
649 650 651 652 |
# File 'lib/coderunner/class_methods.rb', line 649 def self.update_runners @runners ||= {} @runners.each{|runner| runner.update} end |
.write_graph(name, copts = {}) ⇒ Object
519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 |
# File 'lib/coderunner/class_methods.rb', line 519 def self.write_graph(name, copts={}) # process_copts(copts) runner = fetch_runner(copts) eputs 'Starting Graph' kit = runner.graphkit_from_lists(copts[:G], copts[:g]) = copts[:w] = ( and =~ /\S/) ? eval(): {} name = nil unless name =~ /\S/ max = 0 name.sub!(/^\~/, ENV['HOME']) if name if name and name =~ /%d\./ regex = Regexp.new(Regexp.escape(File.basename(name)).sub(/%d/, '(?<number>\d+)')) Dir.entries(File.dirname(name)).join("\n").scan(regex) do max = [max, $~[:number].to_i].max end name = name.sub(/%d/, (max + 1).to_s) end raise "kit doesn't have a file_name and no filename specified; can't write graph" unless name or (kit.file_name.class == String and kit.file_name =~ /\S/) Dir.chdir(COMMAND_FOLDER){kit.gnuplot_write((name or kit.file_name), )} end |
Instance Method Details
#add_phantom_run(run) ⇒ Object
def rename_variable(old, new) puts “Please confirm complete renaming of #old to #new” gets @run_list.each do |directory| Dir.chdir directory do begin @readme = File.read(“CODE_RUNNER_INPUTS”) rescue Errno::ENOENT => err # puts err, err.class @readme = File.read(“README”) end @readme.sub!(Regexp.new(“^s+#old:”), “^s+#new:”) File.open(“CODE_RUNNER_INPUTS_TEST”, ‘w’){|file| file.puts @readme} end end old_recalc, @recalc_all = @recalc_all, true update @recalc_all = old_recalc end
1554 1555 1556 1557 1558 1559 1560 |
# File 'lib/coderunner/instance_methods.rb', line 1554 def add_phantom_run(run) @phantom_run_list[@phantom_id] = run @phantom_ids.push @phantom_id #run.real_id = run.id run.id = @phantom_id @phantom_id += -1 end |
#alter_ids(num, options = {}) ⇒ Object
Permanently change the id of every run in the folder by adding num to them. Num can be negative unless it makes any of the ids negative. Use if you want to combine these runs with the runs in another folder, either by creating an instance of CodeRunner::Merge, or by directly copying and pasting the run folders.
1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 |
# File 'lib/coderunner/instance_methods.rb', line 1613 def alter_ids(num, ={}) Dir.chdir(@root_folder) do return unless [:no_confirm] or Feedback.get_boolean("This will permanently alter all the ids in the folder #@root_folder. Scripts that use those ids may be affected. Do you wish to continue?") raise ArgumentError.new("Cannot have negative ids") if @run_list.keys.min + num < 0 runs = @run_list.values fids = filtered_ids @run_list = {} runs.each do |run| old_id = run.id if fids.include? old_id run.id += num old_dir = run.relative_directory new_dir = old_dir.sub(Regexp.new("id_#{old_id}(\\D|$)")){"id_#{run.id}#$1"} # ep old_dir, new_dir FileUtils.mv(old_dir, new_dir) run.relative_directory = new_dir run.directory = File.(new_dir) end @run_list[run.id] = run end @ids = @run_list.keys set_max_id(@ids.max || 0) save_all_and_overwrite_info end end |
#axiskit(string) ⇒ Object
def gnuplot(axes, options) a = graphkit(axes, options).gnuplot end
315 316 317 318 319 320 321 322 323 324 325 326 327 |
# File 'lib/coderunner/graphs_and_films.rb', line 315 def axiskit(string) generate_combined_ids data = filtered_ids.inject([]) do |arr, id| begin arr.push @combined_run_list[id].instance_eval(string) rescue => err eputs "Evaluation of #{string} failed for run #{id}" raise err end end # p data return GraphKit::AxisKit.new(title: string, data: data) end |
#cancel_job(id, options = {}) ⇒ Object
Cancel the job with the given ID. Options are: :no_confirm —> true or false, cancel without asking for confirmation if true :delete —> if (no_confirm and delete), delete cancelled run
1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 |
# File 'lib/coderunner/instance_methods.rb', line 1513 def cancel_job(id, ={}) @run=@run_list[id] raise "Run with id #{id} does not exist" unless @run unless [:no_confirm] eputs "Cancelling job: #{@run.job_no}: #{@run.run_name}. \n Press enter to confirm" # puts 'asfda' gets end @run.cancel_job if [:no_confirm] delete = [:delete] else delete = Feedback.get_boolean("Do you want to delete the folder as well?") end FileUtils.rm_r(@run.directory) if delete update print_out end |
#code_run_environment ⇒ Object
1181 1182 1183 |
# File 'lib/coderunner/instance_methods.rb', line 1181 def code_run_environment run_class.new(self).code_run_environment end |
#continue_in_new_folder(folder, options = {}) ⇒ Object
1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 |
# File 'lib/coderunner/instance_methods.rb', line 1639 def continue_in_new_folder(folder, ={}) Dir.chdir(@root_folder) do raise "Folder already exists" if FileTest.exist?(folder) FileUtils.makedirs("#{folder}/v") #FileUtils.makedirs(folder) FileUtils.cp(".code_runner_script_defaults.rb", "#{folder}/.code_runner_script_defaults.rb") FileUtils.cp(".code-runner-irb-save-history", "#{folder}/.code-runner-irb-save-history") FileUtils.cp("#{@defaults_file}_defaults.rb", "#{folder}/#{@defaults_file}_defaults.rb") if [:copy_ids] [:copy_ids].each do |id| FileUtils.cp_r(@run_list[id].directory, "#{folder}/v/id_#{id}") end end end end |
#create_archive(options = {}) ⇒ Object
Create a tar archive of the root folder and all the files in it. Options are :compression => true or false :folder => folder in which to place the archive. :verbose => true or false :group => group of new files
1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 |
# File 'lib/coderunner/instance_methods.rb', line 1662 def create_archive(={}) verbose = [:verbose] ? 'v' : '' very_verbose = [:very_verbose] ? 'v' : '' comp = [:compression] Dir.chdir(@root_folder) do temp_root = ".tmparch/#{File.basename(@root_folder)}" FileUtils.makedirs(temp_root) system "chgrp #{[:group]} #{temp_root}" if [:group] size=@run_list.size @run_list.values.each_with_index do |run, index| archive_name = "#{File.basename(run.directory)}.tar#{comp ? '.gz' : ''}" tar_name = archive_name.delsubstr('.gz') relative = run.directory.delete_substrings(@root_folder, File.basename(run.directory)) FileUtils.makedirs(temp_root + relative) unless FileTest.exist? temp_root + relative + archive_name eputs "Archiving #{index} out of #{size}" if [:verbose] Dir.chdir(run.directory + '/..') do command = "tar -cW#{very_verbose}f #{tar_name} #{File.basename(run.directory)}" eputs command if [:verbose] unless system command raise "Archiving failed" end break unless comp command = "gzip -4 -vf #{tar_name}" eputs command if [:verbose] unless system command raise "Compression failed" end command = "gzip -t #{archive_name}" eputs command if [:verbose] unless system command raise "Compression failed" end #exit end FileUtils.mv(relative.delsubstr('/') + archive_name, temp_root + '/' + relative + archive_name) end system "chgrp -R #{[:group]} #{temp_root}" if [:group] end Dir.entries.each do |file| case file when '.', '..', '.tmparch' next when /^v/ next unless File.file? file else FileUtils.cp_r(file, "#{temp_root}/#{file}") end end Dir.chdir('.tmparch') do command = "tar -cWv --remove-files -f #{File.basename(@root_folder)}.tar #{File.basename(@root_folder)}" command = "tar -cWv -f #{File.basename(@root_folder)}.tar #{File.basename(@root_folder)}" eputs command if [:verbose] raise "Archiving Failed" unless system command end FileUtils.mv(".tmparch/#{File.basename(@root_folder)}.tar", "#{File.basename(@root_folder)}.tar") #FileUtils.rm_r(".tmparch") end end |
#destroy(options = {}) ⇒ Object
Delete the folders of all runs for whom CodeRunner#filter(run) is true. This will permanently erase the runs. This is an interactive method which asks for confirmation.
1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 |
# File 'lib/coderunner/instance_methods.rb', line 1492 def destroy(={}) ids = @ids.find_all{|id| filter @run_list[id]} unless [:no_confirm] logf(:destroy) puts "About to delete:" ids.each{|id| eputs @run_list[id].run_name} return unless Feedback.get_boolean("You are about to DESTROY #{ids.size} jobs. There is no way back. All data will be eliminated. Please confirm the delete.") #gets eputs "Please confirm again. Press Enter to confirm, Ctrl + C to cancel" gets end ids.each{|id| FileUtils.rm_r @run_list[id].directory; @run_list.delete(id); @ids.delete(id); generate_combined_ids} set_max_id(@ids.max || 0) save_large_cache generate_combined_ids end |
#dup ⇒ Object
Duplicate the runner, trying to be intelligent as far as possible in duplicating instance variables. Not fully correct yet. Avoid using at the moment.
1481 1482 1483 1484 1485 1486 1487 1488 |
# File 'lib/coderunner/instance_methods.rb', line 1481 def dup logf(:dup) new_one = self.class.new(@code, @root_folder, modlet: @modlet, version: @version) new_one.ids = @ids.dup; new_one.phantom_ids = @phantom_ids.dup; new_one.run_list = @run_list.dup; new_one.phantom_run_list = @phantom_run_list.dup new_one.run_class = @run_class return new_one end |
#executable_location ⇒ Object
1177 1178 1179 |
# File 'lib/coderunner/instance_methods.rb', line 1177 def executable_location File.dirname(@executable) end |
#executable_name ⇒ Object
1173 1174 1175 |
# File 'lib/coderunner/instance_methods.rb', line 1173 def executable_name File.basename(@executable) end |
#film_graphkit(axes, options, film_options = {}) ⇒ Object
408 409 410 411 |
# File 'lib/coderunner/graphs_and_films.rb', line 408 def film_graphkit(axes, , = {}) # self.class.make_film_multiple_runners([[self,[[[axes, options]],[]]]], film_options) film_from_lists([[axes, ]], [], ) end |
#film_run_graphkit(name, options, film_options = {}) ⇒ Object
413 414 415 416 |
# File 'lib/coderunner/graphs_and_films.rb', line 413 def film_run_graphkit(name, , = {}) # self.class.make_film_multiple_runners([[self,[[],[[name, options]]]]], film_options) film_from_lists([], [[name, ]], ) end |
#filter(run = @run, conditions = @conditions) ⇒ Object
Return true if run.instance_eval(conditions) == true and false if run.instance_eval(conditions) == false. performing some checks on the validity of the variable conditions
. For people who are new to Ruby instance_eval means ‘evaluate in the context of the run’. Generally conditions
will be something like ‘status == :Complete and height == 4.5 and not width == 20’, where height and width might be some input parameters or results from the diminishing.
707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 |
# File 'lib/coderunner/instance_methods.rb', line 707 def filter(run=@run, conditions=@conditions) logf(:filter) @run = run # to_instance_variables(directory) return true unless conditions # p conditions, @run.id, @run.is_phantom conditions = conditions.dup raise CRFatal.new(" ----------------------------- Conditions contain a single = sign: #{conditions} -----------------------------") if conditions =~ /[^!=<>]=[^=~<>]/ log conditions begin fil = @run.instance_eval(conditions) rescue => err eputs run.directory eputs conditions raise err end return fil end |
#filtered_ids(conditions = @conditions) ⇒ Object
Return a list of ids, filtered according to conditions. See CodeRunner#filter
694 695 696 697 698 699 |
# File 'lib/coderunner/instance_methods.rb', line 694 def filtered_ids(conditions=@conditions) generate_combined_ids # ep @combined_run_list.keys # sort_runs return @combined_ids.find_all{|id| filter(@combined_run_list[id], conditions)} end |
#generate_combined_ids(kind = nil) ⇒ Object
1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 |
# File 'lib/coderunner/instance_methods.rb', line 1562 def generate_combined_ids(kind= nil) logf(:generate_combined_ids) # case purpose # when :print_out # @combined_ids = [] # @combined_ids += @phantom_ids if @run_class.print_out_phantom_run_list # @combined_ids += @ids if @run_class.print_out_real_run_list # when :readout # @combined_ids = [] # @combined_ids += @phantom_ids if @run_class.readout_phantom_run_list # @combined_ids += @ids if @run_class.readout_real_run_list # when :submitting # @combined_ids = @ids kind ||= @use_phantom case kind when :real @combined_ids = @ids when :phantom @combined_ids = @phantom_ids when :both @combined_ids = @ids + @phantom_ids else raise CRFatal.new("Bad use phantom variable: #{kind.inspect}") end log('@phantom_run_list.class', @phantom_run_list.class) #puts 'crlist', @phantom_run_list.keys, @run_list.keys @combined_run_list = @phantom_run_list + @run_list log(:kind, kind) # log(:combined_ids, @combined_ids) sort_runs(:combined) end |
#get_all_root_folder_contents ⇒ Object
List every file in the root folder.
773 774 775 776 |
# File 'lib/coderunner/instance_methods.rb', line 773 def get_all_root_folder_contents # :nodoc: @root_folder_contents =[] Find.find(@root_folder){|file| @root_folder_contents.push file} end |
#get_max(run, variable, sweep, complete = nil) ⇒ Object
def max(variable,sweep=nil, complete=nil) logf(:max) return get_max(variable,sweep, complete) == @run.id end
def get_max(variable,sweep=nil, complete=nil) logf :get_max sweep ||= variable logi @run.maxes @run.maxes ||= {} similar = similar_runs() similar = similar.find_all{|id| @combined_run_list.status == :Complete} if complete logi(:similar, similar, @combined_run_list[similar].send(variable)) @run.maxes[sweep] ||= similar ? similar.max{|id1,id2| @combined_run_list.send(variable) <=> @combined_run_list.send(variable)} : @run.id # puts “got_here” logi(“@run.maxes[#sweep]”, @run.maxes[sweep]) return @run.maxes[sweep] end
1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 |
# File 'lib/coderunner/instance_methods.rb', line 1441 def get_max(run, variable,sweep, complete=nil) logf :get_max generate_combined_ids sweep = [sweep] unless sweep.class == Array similar = similar_runs(sweep, run) similar = similar.find_all{|id| @combined_run_list[id].status == :Complete} if complete logi(:similar, similar, @combined_run_list[similar[0]].send(variable)) max = similar[1] ? similar.max{|id1,id2| @combined_run_list[id1].send(variable) <=> @combined_run_list[id2].send(variable)} : similar[0] # puts "got_here" return max end |
#get_min(run, variable, sweep, complete = nil) ⇒ Object
1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 |
# File 'lib/coderunner/instance_methods.rb', line 1453 def get_min(run, variable,sweep, complete=nil) logf :get_min generate_combined_ids sweep = [sweep] unless sweep.class == Array similar = similar_runs(sweep, run) similar = similar.find_all{|id| @combined_run_list[id].status == :Complete} if complete logi(:similar, similar, @combined_run_list[similar[0]].send(variable)) min = similar[1] ? similar.min{|id1,id2| @combined_run_list[id1].send(variable) <=> @combined_run_list[id2].send(variable)} : similar[0] # puts "got_here" return min end |
#get_run_class_name(code, modlet) ⇒ Object
See CodeRunner.get_run_class_name
316 317 318 |
# File 'lib/coderunner/instance_methods.rb', line 316 def get_run_class_name(code, modlet) self.class.get_run_class_name(code, modlet) end |
#gets ⇒ Object
No reading from the command line thank you very much!
39 40 41 |
# File 'lib/coderunner.rb', line 39 def gets #No reading from the command line thank you very much! $stdin.gets end |
#graphkit(*args) ⇒ Object
call-seq: graphkit(axes, options) Make a general graphkit, i.e. a graph that combines output from lots of runs. The axes should be an array of strings, each string defining what should be plotted on each axis. Each string is actually a fragment of Ruby code which will be evaluated by each run for whom option or @conditions evaulates to true. The most important other option is options. This is a string or an array of strings, each of which will be evaluated by each run; the data will then be grouped for the values of these strings. E.g. graphkit([‘a’, ‘b’], [“c”], conditions: ‘d==1’
254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 |
# File 'lib/coderunner/graphs_and_films.rb', line 254 def graphkit(*args) logf(:graphkit) if args.size == 1 axes, = graphkit_shorthand(args[0]) elsif args.size == 2 axes, = args else raise ArgumentError end # p axes # return named_graphkit(axes, options) if axes.class == String or axes.class == Symbol # ep @sort (return sweep_graphkits([:sweep], ) do || graphkit(axes, ) end) if [:sweep] intial_conditions = @conditions initial_sort = @sort @conditions = ([:conditions] or @conditions) @sort = ([:sort] or @sort) sort_runs case axes.size when 1 kit = GraphKit.autocreate({x: axiskit(axes[0])}) kit.data[0].gp.with = "linespoints" when 2 kit = GraphKit.autocreate({x: axiskit(axes[0]), y: axiskit(axes[1])}) kit.data[0].gp.with = "linespoints" when 3 kit = GraphKit.autocreate({x: axiskit(axes[0]), y: axiskit(axes[1]), z: axiskit(axes[2])}) kit.data[0].gp.with = "linespoints" when 4 kit = GraphKit.autocreate({x: axiskit(axes[0]), y: axiskit(axes[1]), z: axiskit(axes[2]), f: axiskit(axes[3])}) kit.data[0].gp.with = "linespoints palette" else raise CRError.new("Bad graph string") end kit.title += %| for #{@conditions.gsub(/==/, "=").gsub(/\&\&/, " and ").gsub(/\|\|/, " or ").gsub(/\A(.{,40}).*\s/m, '\1')}| if @conditions kit.file_name = kit.title.gsub(/=/, ' eq ').gsub(/\&\&/, " and ").gsub(/\|\|/, " or ").gsub(/ /, "_").gsub(/\//, "over").gsub(/\*/, '.') kit.modify() # p kit kit.data[0].gp.title = "#@code:#{kit.data[0].title}" #Kernel.puts server_dump(kit) if @server return kit end |
#graphkit_from_lists(graphs, run_graphs, extra_options = {}) ⇒ Object
376 377 378 379 380 381 382 383 384 385 386 387 388 389 |
# File 'lib/coderunner/graphs_and_films.rb', line 376 def graphkit_from_lists(graphs, run_graphs, = {}) run_kits = run_graphs.map do |gr| name, = run_graphkit_shorthand(gr) += run_graphkit(name, .dup) end kits = graphs.map do |gr| axes, = graphkit_shorthand(gr) += graphkit(axes, .dup) end (run_kits + kits).inject{|kit, k| kit+k} end |
#graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, extra_options = {}) ⇒ Object
390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 |
# File 'lib/coderunner/graphs_and_films.rb', line 390 def graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, = {}) server = @server; @server = false # ep "Frame array to calculate is", frame_array, "\n\n" i=0 array = frame_array.map do |frame_index| # if print_message eputs "\033[2A" # Terminal jargon - go back one line eputs sprintf("Fetching graphs: %2.2f", i.to_f/frame_array.size.to_f * 100.0) + "% Complete" # end i+=1 [frame_index, graphkit_from_lists(graphs, run_graphs, .absorb({frame_index: frame_index}))] end # eputs Marshal.load(Marshal.dump(array)).pretty_inspect #Kernel.puts server_dump(array) if server return array end |
#graphkit_shorthand(*gr) ⇒ Object
Parse graphkit shorthand for a general graph and return it as [axes, options]. GraphKit shorthand is: ‘<axis1>[ : <axis2> [ : <axis3 [ : <axis4> ] ] ] [ ; <graph_options> [ ; <conditions> [ ; <sort> ] ] ]’ i.e. all commands for the graph in one string
13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# File 'lib/coderunner/graphs_and_films.rb', line 13 def graphkit_shorthand(*gr) return gr if gr.size > 1 gr = gr[0] return gr if gr.class == Array axes, , cons, srt = gr.split(/;/) axes = eval("['#{axes.split(/(?<!\\|:):(?!:)/).map{|s| s.gsub(/\\:/, ':')}.join("','")}']") = ( and =~ /\S/) ? eval(): {} # p srt cons = nil unless cons and cons=~ /\S/ srt = nil unless srt and srt=~ /\S/ [:conditions] ||= cons [:sort] ||= srt return [axes, ] end |
#increment_max_id ⇒ Object
Increase the value of @max_id
by 1.
216 217 218 |
# File 'lib/coderunner/instance_methods.rb', line 216 def increment_max_id @max_id +=1 end |
#job_identifier ⇒ Object
1192 1193 1194 |
# File 'lib/coderunner/instance_methods.rb', line 1192 def job_identifier "#{@id_list[0]}-#{@id_list[-1]}" end |
#make_film_from_lists(graphs, run_graphs, film_options = {}) ⇒ Object
418 419 420 |
# File 'lib/coderunner/graphs_and_films.rb', line 418 def make_film_from_lists(graphs, run_graphs, = {}) self.class.make_film_multiple_runners([[self,[graphs, run_graphs]]], ) end |
#marshalled_variables ⇒ Object
Dump all the instance variables of the runner to stdout as Marshalled binary data. This is used for RemoteCodeRunner server functions.
899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 |
# File 'lib/coderunner/instance_methods.rb', line 899 def marshalled_variables #ep 'marsh1' instance_vars = {} instance_variables.each do |var| instance_vars[var] = instance_variable_get(var) end instance_vars[:@run_list].values.each{|run| run.runner=nil} #Kernel.puts server_dump(instance_vars) instance_vars[:@cache]={} instance_vars[:@phantom_run_list].values.each{|run| run.runner = nil} #ep 'marsh2' #eputs instance_vars.pretty_inspect #instance_vars.each do |var, val| #ep var #eputs server_dump(val) #end instance_vars end |
#merge(*others) ⇒ Object
3 4 5 |
# File 'lib/coderunner/merged_code_runner.rb', line 3 def merge(*others) Merged.new(*others.unshift(self)) end |
#merge_method(meth, *args) ⇒ Object
7 8 9 |
# File 'lib/coderunner/merged_code_runner.rb', line 7 def merge_method(meth, *args) send(meth, *args) end |
#new_run ⇒ Object
Create a new instance of the class @run_class
1241 1242 1243 1244 |
# File 'lib/coderunner/instance_methods.rb', line 1241 def new_run logf(:new_run) @run_class.new(self) end |
#p(*args) ⇒ Object
Here we redefine the inspect function p to raise an error if anything is written to standard out while in server mode. This is because the server mode uses standard out to communicate and writing to standard out can break it. All messages, debug information and so on, should always be written to standard error.
264 265 266 267 268 269 270 |
# File 'lib/coderunner/instance_methods.rb', line 264 def p(*args) if @server raise "Writing to stdout in server mode will break things!" else super(*args) end end |
#parameter_scan(parameter_scan, parameters, options = {}) ⇒ Object
1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 |
# File 'lib/coderunner/instance_methods.rb', line 1349 def parameter_scan(parameter_scan, parameters, ={}) skip = true unless [:skip] == false # version= (options[:version] or "") nprocs = [:nprocs] logf(:parameter_scan) raise CRFatal.new("Wrong directory: parameter scan being conducted in #{Dir.pwd} instead of my root folder: #{@root_folder}") unless Dir.pwd == File.(@root_folder) @skip = skip puts parameter_scan.inspect @nprocs = nprocs; @skip=skip; log '@run_class', @run_class @run = @run_class.new(self) @run.update_submission_parameters(parameters) # @running_scans = {}; @gammas = {} beginning = "catch(:finished) do \n" end_string = "\nend" parameter_scan.each do |variable_scan| beginning += %[ #{variable_scan[1].inspect}.each do |value|\n\t@run.#{variable_scan[0]} = value\n] @run_class.rcp.naming_pars.push variable_scan[0].to_sym; @run_class.rcp.naming_pars.uniq! end_string += %[\nend] end middle = <<EOF @@psppipe.synchronize(:pssubmit){@running = submit(@run, nprocs: @nprocs, version: @version, skip: @skip); update}; loop do # @@mutex.synchronize{} @run_class.update_status(self) # puts run_class.current_status Dir.chdir(@run_list[@running].directory){@run_list.delete(@running); traverse_directories} # ep @run_list[@running].status, @run_list[@running].id, @run_list[@running].job_no, queue_status unless @run_list[@running].status == :Incomplete or @run_list[@running].status == :NotStarted @run.parameter_transition(@run_list[@running]) break end sleep 3 # Thread.pass # puts Thread.current.to_s + " is looping, @run_list[@running].status = " + @run_list[@running].status.to_s + " @running = " + @running.to_s + " @run_list[@running].job_no = " + @run_list[@running].job_no.to_s end EOF command = beginning + middle + end_string puts command instance_eval(command, 'parameter_scan_code') # puts Thread.current.object_id # puts Thread.current.to_s; print " is finished" # @@psppipe.i_send(:finished, true, tp: 0) end |
#print(*args) ⇒ Object
Here we redefine the function print to raise an error if anything is written to standard out while in server mode. This is because the server mode uses standard out to communicate and writing to standard out can break it. All messages, debug information and so on, should always be written to standard error.
290 291 292 293 294 295 296 |
# File 'lib/coderunner/instance_methods.rb', line 290 def print(*args) if @server raise "Writing to stdout in server mode will break things!" else super(*args) end end |
#print_out(rewind = nil, options = {}) ⇒ Object
Print out a summary of all the runs in the root folder, formatted in nice pretty colours. Since this is often called in a loop, if called twice without any arguments it will erase the first printout. To stop this happening set rewind to 0. If the command is being issued not in a terminal, so that CodeRunner cannot determine the size of the terminal, the second argument must be passed as an array of [rows, columns].
607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 |
# File 'lib/coderunner/instance_methods.rb', line 607 def print_out(rewind = nil, ={}) # terminal_size = [rows, cols] rewind ||= @print_out_size terminal_size = [:terminal_size] logf(:print_out) # raise CRFatal.new("terminal size must be given if this is called any where except inside a terminal (for example if yo u've called this as a subprocess)") unless terminal_size or $stdout.tty? terminal_size ||= Terminal.terminal_size #lots of gritty terminal jargon in here. Edit at your peril! unless ENV['CODE_RUNNER_NO_COLOUR']=='true' or ENV['CODE_RUNNER_NO_COLOR']=='true' dc= Terminal::WHITE # .default_colour green= Terminal::LIGHT_GREEN cyan = Terminal::CYAN bblck = Terminal::BACKGROUND_BLACK else dc= ""# Terminal::WHITE # .default_colour green= "" # Terminal::LIGHT_GREEN cyan = "" #Terminal::CYAN bblck = "" #Terminal::BACKGROUND_BLACK end cln = Terminal::CLEAR_LINE # print "\033[#{rewind}A" deco = '-'*terminal_size[1] Terminal.erewind(rewind) eputs "\n#{cln}\n#{cln}\n#{bblck}#{dc}#{deco}\n#{@run_class.rcp.code_long} Status:#{cln}\n#{deco}" i = 0; j=0 # i is no. of actual terminal lines; j is number of results lines # Group the lines by major sort key #@split = @sort && @sort.split(/\s+/).size > 1 ? @sort.split(/\s+/)[0].to_sym : nil @split_point = nil generate_combined_ids @combined_ids.each do |id| begin # puts id, :hello; gets @run = @combined_run_list[id] if filter # puts @run[:id], @id; gets #@new_split_point = @split ? @run.send(@split) : nil #if @split_point && @split_point != @new_split_point then eputs sprintf(" #{cln}", ""); i+=1 end #@split_point = @new_split_point eprint j%2==0 ? j%4==0 ? cyan : green : dc line = [:with_comments] ? @run.comment_line : @run.print_out_line.chomp eprint line eputs cln # puts (line.size / Terminal.terminal_size[1]).class # puts (line.size / Terminal.terminal_size[1]) i+=((line.sub(/\w*$/, '').size-1) / terminal_size[1]).ceiling j+=1 end # raise "monkeys" rescue => err eputs err eputs err.backtrace eputs "---------------------\nUnable to print out line for this job:" eputs "run_name: #{@run.run_name}" eputs "status: #{@run.status}\n-----------------------" Terminal.reset return # raise CRFatal.new end end @print_out_size = i+7# + (@run_list.keys.find{|id| not [:Complete, :Failed].include? @run_list[id].status } ? 0 : 1) eprint dc, deco; Terminal.reset; eputs # puts # puts # puts dc # "\033[1;37m" # print 'rewind size is', rewind end |
#puts(*args) ⇒ Object
Here we redefine the function puts to raise an error if anything is written to standard out while in server mode. This is because the server mode uses standard out to communicate and writing to standard out can break it. All messages, debug information and so on, should always be written to standard error.
277 278 279 280 281 282 283 |
# File 'lib/coderunner/instance_methods.rb', line 277 def puts(*args) if @server raise "Writing to stdout in server mode will break things!" else super(*args) end end |
#rcp ⇒ Object
def submit
1169 1170 1171 |
# File 'lib/coderunner/instance_methods.rb', line 1169 def rcp @run_class.rcp end |
#read_defaults ⇒ Object
207 208 209 210 211 212 |
# File 'lib/coderunner/instance_methods.rb', line 207 def read_defaults DEFAULT_RUNNER_OPTIONS.each{|key,value| set(key, value)} #ep DEFAULT_RUNNER_OPTIONS, @multiple_processes read_folder_defaults end |
#read_folder_defaults ⇒ Object
Read any default options contained in the file .code_runner_script_defaults.rb
in the root folder.
241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 |
# File 'lib/coderunner/instance_methods.rb', line 241 def read_folder_defaults # p @root_folder + '/.code_runner_script_defaults.rb' Hash.phoenix(@root_folder + '/.code_runner_script_defaults.rb') do |hash| # ep hash FOLDER_DEFAULTS.each do |var| # p send(var), hash[var] hash[var] = (send(var) or hash[var]) hash[:code_runner_version] ||= CodeRunner::CODE_RUNNER_VERSION.to_s set(var, hash[var]) end @start_id = hash[:start_id] if hash[:start_id] # ep "start_id: #@start_id" hash end raise "No default information exists for this folder. If you are running CodeRunner from the commmand line please run again with the -C <code> and -X <executable> (and -m <modlet>, if required) flag (you only need to specify these flags once in each folder). Else, please specify :code and :executable (and :modlet if required) as options in CodeRunner.new(folder, options)" unless @code and @executable end |
#readout ⇒ Object
Similar to CodeRunner#write_data, except that the readout is written to stdout, and formatted a bit. Will probably be rarely used.
731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 |
# File 'lib/coderunner/instance_methods.rb', line 731 def readout logf(:readout) generate_combined_ids @split = @sort && @sort.split(/\s+/).size > 1 ? @sort.split(/\s+/)[0].to_sym : nil @split_point = nil string = @combined_ids.inject("") do |s, id| run = @combined_run_list[id] if run.status =~ /Complete/ and filter(run) @new_split_point = @split ? run.send(@split) : nil splitter = (@split_point and @split_point != @new_split_point) ? "\n\n" : "" @split_point = @new_split_point splitter = "" # comment to put split points back in # puts s.class, splitter.class, data_string(directory) data_string = run.data_string raise CRFatal.new("data_string did not return a string") unless data_string.class == String s + splitter + data_string else s end end # puts @max.inspect return string end |
#readout_cols(*var_list) ⇒ Object
? Probably can get rid of this one
758 759 760 761 762 763 764 765 766 767 768 769 |
# File 'lib/coderunner/instance_methods.rb', line 758 def readout_cols(*var_list) # :nodoc: logf :readout_cols ans = [[]] * var_list.size generate_combined_ids filtered_ids.each do |id| run = combined_run_list[id] var_list.each_with_index do |var, index| ans[index].push run.send(var) end end ans end |
#recheck_filtered_runs(write_status_dots = false) ⇒ Object
Self-explanatory! Call CodeRunner::Run#process_directory for every run for which CodeRunner#filter(run) is true. (Note, for historical reasons traverse_directories is called rather than CodeRunner::Run#process_directory directly but the effect is nearly the same).
962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 |
# File 'lib/coderunner/instance_methods.rb', line 962 def recheck_filtered_runs(write_status_dots=false) @write_status_dots = write_status_dots logf :recheck_filtered_runs @run_class.update_status(self) @run_list.each do |id, run| if filter(run) and not (run.status == :Complete or run.status == :Failed) # eputs run.directory Dir.chdir(run.directory){run.process_directory} # eputs run.status # ep run end end save_large_cache # run_list, @run_list = @run_list, {}; @ids = []; @max_id = 0 # run_list.each do |id, run| # if not filter(run) or run.status == :Complete # @run_list[id] = run # @ids.push id # else # Dir.chdir(run.directory){traverse_directories} # end # end # @ids.sort! # @ids = @run_list.keys set_max_id(@ids.max || 0) sort_runs respond_to_requests save_large_cache end |
#recheck_incomplete_runs ⇒ Object
Self-explanatory! Call CodeRunner::Run#process_directory for every run whose status is not either :Complete or :Failed. (Note, for historical reasons traverse_directories is called rather than CodeRunner::Run#process_directory directly but the effect is nearly the same).
935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 |
# File 'lib/coderunner/instance_methods.rb', line 935 def recheck_incomplete_runs logf :recheck_incomplete_runs @run_class.update_status(self) redone_count = 0 run_list, @run_list = @run_list, {}; @ids = []; run_list.each do |id, run| # print id if run.status == :Complete or run.status == :Failed # print ".", id, "___" @run_list[id] = run @ids.push id else # print id, "%%%"; redone_count+=1 Dir.chdir(run.directory){traverse_directories} end end @ids.sort! # @ids = @run_list.keys set_max_id(@ids.max || 0) sort_runs respond_to_requests save_large_cache end |
#respond_to_requests ⇒ Object
One of the features of CodeRunner is two way communication between a runner and its runs. The runs can request actions from the runner directly by calling its instance methods, but often the runs want something to happen after the runner has processed every run in the directory. For example if a run wanted to check if it was resolved, it would need to know about all the other runs so it could compare itself with them. In this case it would place an instance method that it wanted to call in the variable @requests
in the runner. The runner would then call that instance method on every run after it had finished processing all the runs.
In summary, this method is called after every time the runner has checked through the directory. When it is called, it looks in the variable @requests
which contains symbols representing methods. It calls each symbol as an instance method of every run in turn. So if @requests
was [:check_converged]
it would call run.check_converged
for every run.
996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 |
# File 'lib/coderunner/instance_methods.rb', line 996 def respond_to_requests logf(:respond_to_requests) logi(:@requests, @requests) log('@phantom_run_list.class', @phantom_run_list.class) while @requests[0] old_requests = @requests.uniq @requests = [] old_requests.each do |request| @current_request = request if request == :traverse_directories # a special case @ids = [] @run_list = {} @phantom_run_list = {} @phantom_ids = [] Dir.chdir(@root_folder){traverse_directories} else filtered_ids.each do |id| run = @run_list[id] Dir.chdir(run.directory){run.instance_eval(request.to_s); run.save; run.write_results} end end end end end |
#run_graphkit(*args) ⇒ Object
345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 |
# File 'lib/coderunner/graphs_and_films.rb', line 345 def run_graphkit(*args) if args.size == 1 name, = run_graphkit_shorthand(args[0]) elsif args.size == 2 name, = args else raise ArgumentError end (return sweep_graphkits([:sweep], ) do || run_graphkit(name, ) end) if [:sweep] old_sort, old_conditions = @sort, @conditions @sort = [:sort] if [:sort] @conditions = [:conditions] if [:conditions] generate_combined_ids fids = filtered_ids raise CRError.new("No ids match these conditions: #@conditions") unless fids.size > 0 kit = (fids.map do |id| run = @combined_run_list[id]; # p run; STDIN.gets kit = run.graphkit(name, .dup); kit.data[0].title ||= run.run_name; kit end).inject{|kit, k| kit+k} @sort, @conditions = old_sort, old_conditions #Kernel.puts server_dump(kit) if @server kit end |
#run_graphkit_shorthand(*grs) ⇒ Object
329 330 331 332 333 334 335 336 337 338 339 340 341 342 |
# File 'lib/coderunner/graphs_and_films.rb', line 329 def run_graphkit_shorthand(*grs) # p grs return grs if grs.size > 1 gr = grs[0] # p gr return gr if gr.class == Array name, , cons, srt = gr.split(/;/) = ( and =~ /\S/) ? eval(): {} cons = nil unless cons and cons=~ /\S/ srt = nil unless srt and srt=~ /\S/ [:conditions] ||= cons if cons [:sort] ||= srt if srt [name, ] end |
#runs ⇒ Object
Return an array of runs (run_list.values)
222 223 224 |
# File 'lib/coderunner/instance_methods.rb', line 222 def runs run_list.values end |
#save_all ⇒ Object
1594 1595 1596 1597 1598 1599 1600 |
# File 'lib/coderunner/instance_methods.rb', line 1594 def save_all save_large_cache @run_list.values.each do |run| run.save run.write_results end end |
#save_large_cache ⇒ Object
Write the variable @run_list
, which contains all information currently known about the simulations in the root folder, as Marshalled binary data in the file “.CODE_RUNNER_TEMP_RUN_LIST_CACHE”. This cache will be used later by CodeRunner#update.
922 923 924 925 926 927 928 929 930 931 |
# File 'lib/coderunner/instance_methods.rb', line 922 def save_large_cache # pp self # ep @run_list # pp @run_list.values.map{|run| run.instance_variables.map{|var| [var, run.instance_variable_get(var).class]}} generate_combined_ids @combined_run_list.each{|id, run| run.runner = nil} File.open(@root_folder + "/.CODE_RUNNER_TEMP_RUN_LIST_CACHE", 'w'){|file| file.puts Marshal.dump @run_list} @combined_run_list.each{|id, run| run.runner = self} end |
#server_dump(obj) ⇒ Object
302 303 304 |
# File 'lib/coderunner/instance_methods.rb', line 302 def server_dump(obj) self.class.server_dump(obj) end |
#set_start_id(id) ⇒ Object
306 307 308 309 310 311 312 |
# File 'lib/coderunner/instance_methods.rb', line 306 def set_start_id(id) raise "start_id #{id} lower than max_id #@max_id" if @max_id and id < @max_id Hash.phoenix(@root_folder + '/.code_runner_script_defaults.rb') do |hash| @start_id = hash[:start_id] = id hash end end |
#setup_run_class(code, options) ⇒ Object
See CodeRunner.setup_run_class
322 323 324 325 |
# File 'lib/coderunner/instance_methods.rb', line 322 def setup_run_class(code, ) [:runner] ||= self self.class.setup_run_class(code, ) end |
#similar_runs(exclude_variables = [], run = @run) ⇒ Object
Find runs whose input parameters are all the same as those for run
, with the exception of those listed in exclude_variables
1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 |
# File 'lib/coderunner/instance_methods.rb', line 1399 def similar_runs(exclude_variables=[], run=@run) #all runs for which variables are the same as 'run', with the exception of exclude_variables logf(:similar_runs) raise CRFatal.new("generate_combined_ids must be called before this function is called") unless (@combined_run_list.size > 0 and @combined_ids.size > 0) or @ids.size ==0 command = (run.class.rcp.variables+run.class.rcp.run_info-exclude_variables - [:output_file, :error_file, :runner, :phantom_runs]).inject("@combined_ids.find_all{|id| @combined_run_list[id].class == run.class}"){ |s,v| s + %<.find_all{|id| ep '#{v}'; @combined_run_list[id].#{v}.class == #{run.send(v).inspect}.class and @combined_run_list[id].#{v} == #{run.send(v).inspect}}>} #the second send call retrieves the type conversion # log command #puts command begin similar = instance_eval(command) rescue => err log command raise err end return similar end |
#simple_scan(scan_string, options = {}) ⇒ Object
Submit a series of runs according to scan_string. scan_string specifies a number of scans separated by hashes. Each scan has a number which is the ID of the run to start from followed by a colon. After the colon the user can write some code which modifies the original run. For example:
simple_scan(‘23: @height *= 2; @weight *=1.1’)
will submit a series of runs, where each successive run has height twice the last one, and weight 1.1 times the last one, where we assume that height and weight are input parameters.
Options are the same as CodeRunner#submit
1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 |
# File 'lib/coderunner/instance_methods.rb', line 1256 def simple_scan(scan_string, ={}) scans = scan_string.split('#') ppipe = PPipe.new(scans.size + 1, true, controller_refresh: 0.5, redirect: false) pipe_numbers = ppipe.fork(scans.size - 1) #p @run_class.naming_pars instructions = (scans[(ppipe.mpn > 0 ? ppipe.mpn - 1 : 0)]) id = instructions.scan(/^\d+\:/)[0].to_i instructions = instructions.sub(/^\d+\:/, '') @run= id > 0 ? @run_list[id] : (run = @run_class.new(self); run.update_submission_parameters(([:parameters] or '{}')); run) @run_class.rcp.variables.each do |par| #p par, @run_class.naming_pars @run.naming_pars.push par if scan_string =~ Regexp.new(Regexp.escape(par.to_s)) end @run.naming_pars.uniq! # @run.naming_pars +== @run_class.naming_pars catch(:finished) do loop do #submit loop ppipe.synchronize(:submit) do @running = submit(@run, nprocs: [:nprocs], version: @run_class.rcp.version) @conditions = "id == #@running" #print_out #ep 'Running run', @running end loop do # check finished loop dir = @run_list[@running].directory @run_list.delete(@running) @run_class.update_status(self) Dir.chdir(dir) do traverse_directories end unless @@wait and (@run_list[@running].status == :Incomplete or @run_list[@running].status == :NotStarted or @run_list[@running].status == :Queueing) @run.parameter_transition(@run_list[@running]) @old_run = @run_list[@running] break end #p @running ppipe.i_send(:running, @running, tp: 0) if ppipe.is_root arr = (pipe_numbers + [0]).inject([]) do |arr, pn| arr.push ppipe.w_recv(:running, fp: pn) end #p arr @conditions = "#{arr.inspect}.include? id" print_out end sleep 3 end @run.instance_eval(instructions) end end (ppipe.die; exit) unless ppipe.is_root ppipe.finish end |
#sort_runs(type = @use_phantom.to_s.sub(/real/, '')) ⇒ Object
Sort the runs according to the variable @sort
which can be either whitespace separated string or an array of strings. In the former case, the string is split to form an array of strings, using the separator /\s+/
. For example, if
Then the runs will be sorted according to first height, then (descending) weight, then colour. What actually happens is that the variable @ids
is sorted, rather than @run_list
. For this to work, height, weight and colour must be either variables or results or instance methods of the run class.
Type can be either ” or ‘phantom’, in which case the variable sorted will be either @ids
or @phantom_ids
respectively.
587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 |
# File 'lib/coderunner/instance_methods.rb', line 587 def sort_runs(type = @use_phantom.to_s.sub(/real/, '')) logf(:sort_runs) log(:type, type) #ep 'sort', @sort #sort_list = @sort ? (@sort.class == String ? eval(@sort) : @sort) : [] run_list_name = [type.to_s, 'run_list'].join('_').gsub(/^_/, '') ids_name = [type.to_s, 'ids'].join('_').gsub(/^_/, '') log run_list_name set(ids_name, send(ids_name).sort_by do |id| run = send(run_list_name)[id] sortkey = run.instance_eval((@sort or '[]')) #ep 'sortkey', sortkey sortkey #sort_list.map{|str| run.instance_eval(str)} end) end |
#submit(runs, options = {}) ⇒ Object
Start a simulation: submit the run that is passed. What happens is as follows:
-
Modify the run according to options.
-
Check if any other processes are submitting runs in the same root folder. In this case there will be a file called ‘submitting’ in the folder. If there is such a file wait until it is gone.
-
Check if a run with identical parameters has been submitted before. In which case skip submitting the run unless options == false.
-
Call
run.submit
Options can be any one of CodeRunner::SUBMIT_OPTIONS
. The options passed here will override values stored as instance variables of the runner with the same name, which will override these values if they are set in the runs itself. For example if run.nprocs == ‘32x4’ runner.nprocs == nil options == nil the number of processes will be 32x4. On the other hand if run.nprocs == ‘32x4’ runner.nprocs == ‘24x4’ options == ‘48x4’ the number of processes will be 48x4.
1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 |
# File 'lib/coderunner/instance_methods.rb', line 1039 def submit(runs, ={}) eputs "System " + SYS eputs "No. Procs " + @nprocs.inspect runs = [runs] unless runs.class == Array #can pass a single run to submit outruns = runs.dup skip = true unless [:skip] == false SUBMIT_OPTIONS.each do |option| set(option, [option]) if .keys.include? option end logf(:submit) Dir.chdir(@root_folder) do @skip=skip mess = false while FileTest.exist?("submitting") (eputs " Waiting for another process to finish submitting. If you know that no other CodeRunner processes are submitting in this folder (#@root_folder) then delete the file 'submitting' and try again"; mess = true) unless mess sleep rand end # old_trap = trap(0) old_trap0 = trap(0){eputs "Aborted Submit!"; File.delete("#@root_folder/submitting"); exit!} old_trap2 = trap(2){eputs "Aborted Submit!"; File.delete("#@root_folder/submitting") if FileTest.exist? "#@root_folder/submitting"; trap(2, "DEFAULT"); trap(0, "DEFAULT"); Process.kill(2, 0)} # File.open("submitting", "w"){|file| file.puts ""} FileUtils.touch("submitting") unless [:no_update_before_submit] @use_large_cache, ulc = false, @use_large_cache; update; @use_large_cache = ulc end generate_combined_ids(:real) # old_job_nos = queue_status.scan(/^\s*(\d+)/).map{|match| match[0].to_i} script = "" if [:job_chain] runs.each_with_index do |run, index| similar = similar_runs([], run) if @skip and similar[0] and not ([:replace_existing] or [:rerun]) eputs "Found similar run: #{@run_list[similar[0]].run_name}" eputs "Skipping submission..." runs[index] = nil next end unless [:replace_existing] or [:rerun] @max_id+=1 run.id = @max_id else if [:replace_existing] FileUtils.rm_r run.directory elsif [:rerun] ################# For backwards compatibility SUBMIT_OPTIONS.each do |opt| run.set(opt, send(opt)) unless run.send(opt) end ########################################### FileUtils.rm "#{run.directory}/code_runner_results.rb" FileUtils.rm "#{run.directory}/.code_runner_run_data" end @run_list.delete(run.id) @ids.delete run.id generate_combined_ids end begin unless [:job_chain] run.prepare_submission unless [:rerun] next if @test_submission Dir.chdir(run.directory) do old_job_nos = queue_status.scan(/^\s*(\d+)/).map{|match| match[0].to_i} ######################### The big tomale! run.job_no = run.execute # Start the simulation and get the job_number ######################### run.job_no = get_new_job_no(old_job_nos) unless run.job_no.kind_of? Integer # (if the execute command does not return the job number, look for it manually) # eputs 'run.job_no', run.job_no run.output_file # Sets the output_file on first call run.error_file # Sets the error_file on first call run.write_info eputs "Submitted run: #{run.run_name}" end else run.prepare_submission unless [:rerun] script << "cd #{run.directory}\n" script << "#{run.run_command}\n" next if @test_submission end rescue => err File.delete("submitting") raise(err) end end # runs.each runs.compact! if [:job_chain] and not @test_submission and runs.size > 0 FileUtils.makedirs('job_chain_files') @id_list = runs.map{|run| run.id} @submission_script = script # A hook... default is to do nothing @submission_script = @run_class.modify_job_script(self, runs, @submission_script) # To get out of job_chain_files folder @submission_script = "cd .. \n" + @submission_script old_job_nos = queue_status.scan(/^\s*(\d+)/).map{|match| match[0].to_i} ################ Submit the run Dir.chdir('job_chain_files'){job_no = execute} ################ job_no = get_new_job_no(old_job_nos) unless job_no.kind_of? Integer # (if the execute command does not return the job number, look for it manually) # eputs 'jobchain no', job_no #runs.each{|run| run.job_no = job_no} runs.each do |run| run.job_no = @job_no = job_no run.output_file = run.relative_directory.split("/").map{|d| ".."}.join("/") + "/job_chain_files/" + output_file run.error_file = run.relative_directory.split("/").map{|d| ".."}.join("/") + "/job_chain_files/" + error_file run.write_info eputs "Submitted run: #{run.run_name}" end end @write_status_dots, wsd = false, @write_status_dots @run_class.update_status(self) runs.each do |run| # ep run.id, run_list.keys Dir.chdir(run.directory){traverse_directories} end @write_status_dots = wsd save_large_cache File.delete("submitting") trap(0, old_trap0) trap(2, old_trap2) end # Dir.chdir(@root_folder) # eputs #ep 'runs submitted', outruns return outruns[0].id if outruns.size == 1 #used in parameter scans end |
#sweep_graphkits(sweeps, options, &block) ⇒ Object
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
# File 'lib/coderunner/graphs_and_films.rb', line 29 def sweep_graphkits(sweeps, , &block) server = @server; @server = false intial_conditions = @conditions initial_sort = @sort @conditions = ([:conditions] or @conditions) @sort = ([:sort] or @sort) sweeps = sweeps.class == Array ? sweeps : [sweeps] generate_combined_ids # ep sweeps sweep_values = filtered_ids.inject([]) do |arr, id| arr.push(sweeps.inject([]) do |sweep_arr, var| sweep_arr.push @combined_run_list[id].send(var) # ep sweep_arr, var sweep_arr end) arr.uniq end old_conditions = @conditions = .dup .delete(:sweep) if [:sweep] # p new_options; gets kits = sweep_values.sort.map do |values| new_cons = (sweeps.zip(values).map do |name, value| "#{name}==#{value}" end).join(" and ") [:conditions] = old_conditions ? old_conditions + " and #{new_cons}" : new_cons # ep new_options[:conditions] kit = yield() kit.data[0].gp.title = new_cons.gsub(/==/, '=') # p kit kit end # @conditions = old_conditions @conditions = intial_conditions @sort = initial_sort if [:sweep_multid] kitdata = kits.map do |kit| kit.data[0].axes.values.map{|axiskit| axiskit.data} end final_axes = (kitdata[0].size + sweeps.size - 1).times.inject([]){|arr, i| arr.push []} data_hash = {} sweep_values.sort.each_with_index do |vals, i| # ep 'v', vals data = kitdata[i] length = data[0].size vals.each_with_index do |val, j| final_axes[j].push val # final_axes[j].uniq! end function = data.pop for j in 0...length data.each_with_index do |axisdata, k| # ep 'vk', vals.size + k # break if k == data.size - 1 final_axes[vals.size + k].push axisdata[j] final_axes[vals.size + k].uniq! end # ep 'h', final_axes, function data_hash[vals + data.map{|axisdata| axisdata[j]}] = function[j] end end final_axes.map!{|vals| vals.uniq.sort} final_data = SparseTensor.new(final_axes.size) data_hash.each do |coordinates, function| indices = [] coordinates.each_with_index do |val, i| indices[i] = final_axes[i].index(val) end final_data[*indices] = function end hash = {} # pp final_axes, final_data titles = sweeps + kits[0].data[0].axes.values.map{|axiskit| axiskit[:title]} (final_axes + [final_data]).each_with_index do |d, i| hash[GraphKit::AXES[i]] = {data: d, title: titles[i].to_s} end kit = GraphKit.autocreate(hash) # length = data[i].size # array = vals.map{|val| [val] * length} + data # p array # end # final_data = [[]] * (kitdata[0].size + sweeps.size) # sweep_values.sort.each_with_index do |vals, i| # data = kitdata[0] # length = data[0].size # array = vals.map{|val| [val] * length} + data # p array # end else kit = kits.inject{|ans, kit| ans + kit} end #Kernel.puts server_dump(kit) if server return kit end |
#update(write_status_dots = true, use_large_cache = @use_large_cache, use_large_cache_but_recheck_incomplete = @use_large_cache_but_recheck_incomplete) ⇒ Object
Update the information about all the runs stored in the variable @run_list
. By default, this is done by calling CodeRunner#traverse_directories. If, on the other hand, @use_large_cache
is set to true, this is done by reading the temporary cache maintained in “.CODE_RUNNER_TEMP_RUN_LIST_CACHE” in the root folder. This is much quicker. If in addition @use_large_cache_but_recheck_incomplete
is set to true, all runs whose status is not either :Complete or :Failed will be rechecked.
780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 |
# File 'lib/coderunner/instance_methods.rb', line 780 def update(write_status_dots=true, use_large_cache=@use_large_cache, use_large_cache_but_recheck_incomplete=@use_large_cache_but_recheck_incomplete) @use_large_cache = use_large_cache logf(:update) @write_status_dots = write_status_dots @run_list={} @ids=[] @phantom_run_list = {} @phantom_ids = [] @run_store =[] # @multiple_processes_directories = [] set_max_id 0 @run_class.update_status(self) if @use_large_cache and not @recalc_all and not @reprocess_all log("using large cache") begin begin eputs 'Loading large cache...' if @write_status_dots Dir.chdir(@root_folder) do @run_list = Marshal.load(File.read(".CODE_RUNNER_TEMP_RUN_LIST_CACHE")) end @run_list.values.each{|run| run.runner = self} rescue ArgumentError => err eputs err raise err unless err. =~ /undefined class/ #NB all code_names have to contain only lowercase letters: # modlet, code = err.message.scan(/CodeRunner\:\:([A-Z][\w+])?([A-Z]\w+)Run/)[0] code, modlet = err..scan(/CodeRunner\:\:([A-Z][a-z0-9_]+)(?:::([A-Z]\w+))?/)[0] # ep code.downcase, modlet modlet.downcase! if modlet setup_run_class(code.downcase, modlet: modlet) retry end # ep @run_list @ids = @run_list.keys @run_list.each{|id, run| run.runner = self } #eputs "Setting max id..." set_max_id(@ids.max || 0) #eputs "max_id = #@max_id" # puts @max_id; gets # @use_large_cache = false @ids.sort! redone_count = 0 # puts "hello" # ep @use_large_cache_but_recheck_incomplete; exit recheck_incomplete_runs if use_large_cache_but_recheck_incomplete # sort_runs eprint "Updating runs..." if @write_status_dots # get_all_root_folder_contents # puts @run_list.values[0].directory, File.expand_path(@root_folder).esc_regex fix_directories = (run_list.size > 0 and not @run_list.values[0].directory =~ File.(@root_folder).esc_regex) # eputs 'fdirectories', fix_directories # exit eputs "Fixing Directories..." if fix_directories @run_list.each do |id, run| eprint '.' if @write_status_dots run.directory = File.join(@root_folder, run.relative_directory) if fix_directories # run.directory = "#@root_folder/#{run.relative_directory}" # unless @root_folder_contents.include? run.directory# File.directory? run.directory and run.directory =~ File.expand_path(@root_folder).esc_regex # if @root_folder_contents.include?(rel = File.join(@root_folder, run.relative_directory)) # run.directory = rel # else # raise CRFatal.new("Directory #{run.directory} not found") # end # end # eputs @use_phantom #run.generate_phantom_runs #if @use_phantom.to_s =~ /phantom/i run.phantom_runs.each{|r| add_phantom_run(r)} if run.phantom_runs end eputs if @write_status_dots save_large_cache if fix_directories # puts redone_count return self rescue Errno::ENOENT, ArgumentError, TypeError => err if err.class == ArgumentError and not err. =~ /marshal data too short/ or err.class == TypeError and not err. =~ /incompatible marshal file format/ eputs err eputs err.backtrace raise CRFatal.new end eputs err, "Rereading run data" # @use_large_cache = false end end log("not using large cache") @run_list={} @ids=[] @phantom_run_list = {} @phantom_ids = [] @run_store =[] # @multiple_processes_directories = [] set_max_id 0 log("traversing directories") eprint 'Analysing runs..' if @write_status_dots #interrupted = false; #old_trap2 = trap(2){} #trap(2){eputs "Interrupt acknowledged... reloading saved cache. Interrupt again to override (may cause file corruption)."; trap(2, old_trap2); update(true, true, false); Process.kill(2,0)} Dir.chdir(@root_folder){traverse_directories} @max_id ||= 0 eputs # puts 'before request', @ids, @requests; respond_to_requests # @n_checks += 1 # exit if ($nruns > 0 && @n_checks > $nruns) sort_runs @recalc_all = false # pp @run_list save_large_cache #@run_list.each{|id, run| run.generate_phantom_runs} #trap(2, old_trap2) #Process.kill(2, 0) if interrupted return self end |
#write_data(filename = "data.txt") ⇒ Object
Write out a simple datafile containing all the inputs and outputs listed in the run class property readout_string
. What is actually written out can be customised by redefining the method data_string
in the run class, or changing readout_string
or both.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 |
# File 'lib/coderunner/instance_methods.rb', line 564 def write_data(filename = "data.txt") logf(:write_data) generate_combined_ids File.open(filename, "w") do |file| @combined_ids.each do |id| run = @combined_run_list[id] if run.status =~ /Complete/ data_string = run.data_string raise CRFatal.new("data_string did not return a string") unless data_string.class == String file.puts data_string end end end end |