Class: CodeRunner
- Extended by:
- const_get(SYSconst_get(SYS.variable_to_class_name)
- Includes:
- const_get(SYSconst_get(SYS.variable_to_class_name), Log
- Defined in:
- lib/coderunner/instance_methods.rb,
lib/coderunner.rb,
lib/coderunner.rb,
lib/coderunner/run.rb,
lib/cubecalccrmod/empty.rb,
lib/cubecalccrmod/sleep.rb,
lib/cubecalccrmod/cubecalc.rb,
lib/coderunner/class_methods.rb,
lib/coderunner/fortran_namelist.rb,
lib/coderunner/graphs_and_films.rb,
lib/coderunner/instance_methods.rb,
lib/cubecalccrmod/with_namelist.rb,
lib/coderunner/merged_code_runner.rb,
lib/coderunner/interactive_methods.rb,
lib/coderunner/system_modules/loki.rb,
lib/coderunner/system_modules/moab.rb,
lib/coderunner/system_modules/dirac.rb,
lib/coderunner/system_modules/slurm.rb,
lib/coderunner/heuristic_run_methods.rb,
lib/coderunner/system_modules/archer.rb,
lib/coderunner/system_modules/edison.rb,
lib/coderunner/system_modules/hector.rb,
lib/coderunner/system_modules/helios.rb,
lib/coderunner/system_modules/hopper.rb,
lib/coderunner/system_modules/iridis.rb,
lib/coderunner/system_modules/juropa.rb,
lib/coderunner/system_modules/macosx.rb,
lib/coderunner/system_modules/saturne.rb,
lib/coderunner/system_modules/franklin.rb,
lib/coderunner/system_modules/stampede.rb,
lib/coderunner/system_modules/new_hydra.rb,
lib/coderunner/system_modules/blue_joule.rb,
lib/coderunner/system_modules/load_leveler.rb,
lib/coderunner/system_modules/generic_linux.rb,
lib/coderunner/system_modules/genericlinux_testsystem.rb
Overview
CodeRunner Overview
CodeRunner is a class designed to make the running an analysis of large simulations and easy task. An instance of this class is instantiated for a given root folder. The runner, as it is known, knows about every simulation in this folder, each of which has a unique id and a unique subfolder.
The heart of the runner is the variable run_list. This is a hash of IDs and runs. A run is an instance of a class which inherits from CodeRunner::Run, and which is customised to be able to handle the input variables, and the output data, from the given simulation code. This is achieved by a module which contains a child class of CodeRunner::Run, which is provided independently of CodeRunner.
CodeRunner has methods to sort these runs, filter them according to quite complex conditions, print out the status of these runs, submit new runs, plot graphs using data from these runs, cancel running jobs, delete unwanted runs, and so on.
CodeRunner Interfaces
CodeRunner has two different interfaces:
-
Instance methods
-
Class methods
Instance Methods
The instance methods provide a classic Ruby scripting interface. A runner is instantiated from the CodeRunner class, and passed a root folder, and possibly some default options. Instance methods can then be called individually. This is what should be used for complex and non-standard tasks.
Class methods
The class methods are what are used by the command line interface. They define a standard set of tasks, each of which can be customised by a set of options known as command options, or copts for short.
There is a one-to-one correspondence between the long form of the commandline commands, and the class methods that handle those commands, and between the command line flags, and the options that are passed to the class methods. So for example:
$ coderunner submit -p ‘23.4, resolution: 256’ -n 32x4 -W 600
Becomes
CodeRunner.submit(p: ‘23.4, resolution: 256’, n: “32x4”, W: 600) # remembering that braces are not needed around a hash if it is the final parameter.
These methods are what should be used to automate a large number of standard tasks, which would be a pain to run from the command line.
Direct Known Subclasses
Defined Under Namespace
Modules: Archer, BlueJoule, Dirac, Edison, Franklin, GenericLinux, GenericlinuxTestsystem, Hector, Helios, Hopper, InteractiveMethods, Iridis, Juropa, LoadLeveler, Loki, Macosx, Moab, NewHydra, Saturne, Slurm, Stampede Classes: CRError, CRFatal, CRMild, Cubecalc, Merged, Run
Constant Summary collapse
- COMMAND_FOLDER =
Dir.pwd
- SCRIPT_FOLDER =
i.e. where this script is
File.dirname(File.(__FILE__)) + '/coderunner'
- GLOBAL_OPTIONS =
global options are set by the environment but some can be changed.
{}
- SYS =
(GLOBAL_OPTIONS[:system] or ENV['CODE_RUNNER_SYSTEM'] or ENV['SYSTEM'] or "generic_linux")
- SYSTEM_MODULE =
const_get(SYS.variable_to_class_name)
- GraphKit =
Backwards compatibility
GraphKit
- CODE_RUNNER_VERSION =
Version.new(Gem.loaded_specs['coderunner'].version.to_s) rescue "test"
- GLOBAL_BINDING =
binding
- COMMAND_LINE_FLAGS_WITH_HELP =
Here are all the methods that map the command line invocation into the correct class method call
[ ["--recalc-all", "-A", GetoptLong::NO_ARGUMENT, %[Causes each directory to be reprocessed, rather than reading the cache of data. Its exact effect depends on the code module being used. By convention it implies that ALL data analysis will be redone.]], ["--reprocess-all", "-a", GetoptLong::NO_ARGUMENT, %[Causes each directory to be reprocessed, rather than reading the cache of data. Its exact effect depends on the code module being used. By convention it implies that VERY LITTLE data analysis will be redone.]], ["--code", "-C", GetoptLong::REQUIRED_ARGUMENT, %[The code that is being used for simulations in this folder. This string must correspond to a code module supplied to CodeRunner. It usually only needs to be specified once as it will be stored as a default in the folder.]], ["--comment", "-c", GetoptLong::REQUIRED_ARGUMENT, %[A comment about the submitted run.]], ["--debug", "-d", GetoptLong::NO_ARGUMENT, %[Submit the simulation to the debug queue. This usually only has meaning on HPC systems. It does not mean debug CodeRunner!]], ["--defaults-file", "-D", GetoptLong::REQUIRED_ARGUMENT, %[Specify a defaults file to be used when submitting runs. The name should correspond to file named something like '<name>_defaults.rb' in the correct folders within the code module. Every time a different defaults file is specified, a local copy of that defaults file is stored in the root folder. This local copy can be edited and all runs will use the local copy to get defaults. CodeRunner will never overwrite the local copy.]], ["--film-options", "-F", GetoptLong::OPTIONAL_ARGUMENT, %[Specify a hash of options when making films. The most important one is fa (frame array). For example -F '{fa: [0, 200]}'. For all possible options see the CodeRunner method make_film_from_lists.]], ["--conditions", "-f", GetoptLong::REQUIRED_ARGUMENT, %[A string specifying conditions used to filter runs. This filter is used in a variety of circumstances, for example when printing out the status, plotting graphs etc. Example: '@height == 10 and @width = 2.2 and @status==:Complete'.]], ["--run-graph", "-g", GetoptLong::REQUIRED_ARGUMENT, %[Specify a run_graphkit to plot. A run_graphkit is one that is plotted for an individual run. The run graphkits available depend on the code module. The syntax is graphkit shorthand:\n -g '<graph_name>[ ; <graph_options> [ ; <conditions> [ ; <sort> ] ] ]'\n where conditions (i.e. filter) and sort will override the -f and -O flags respectively. The -g flag can be specified multiple times, which will plot multiple graphs on the same page.]], ["--graph", "-G", GetoptLong::REQUIRED_ARGUMENT, %[Specify a graphkit to plot. A graphkit combines data for every filtered run. The syntax is graphkit shorthand:\n -G '<axis1>[ : <axis2> [ : <axis3 [ : <axis4> ] ] ] [ ; <graph_options> [ ; <conditions> [ ; <sort> ] ] ]'\n where conditions (i.e. filter) and sort will override the -f and -O flags respectively. <axis1> etc are strings which can be evaluated by the runs. The -G flag can be specified multiple times, which will plot multiple graphs on the same page. For example\n -G 'width : 2*height ; {} ; depth == 2 ; width'\n will plot twice the height against the width for every run where the depth is equal to 2, and will order the data points by width.]], ["--heuristic-analysis", "-H", GetoptLong::NO_ARGUMENT, %[Should be specified whenever CodeRunner is being used to analyse simulations which did not originally submit (and which will therefore not have the usual CodeRunner meta data stored with them).] ], ["--use-phantom", "-h", GetoptLong::OPTIONAL_ARGUMENT, %[Specify whether to use real or phantom runs]], ["--just", "-j", GetoptLong::REQUIRED_ARGUMENT, %[Specify individual run ids. For example -j 45,63,128 is shorthand for -f 'id==45 or id==63 or id==128']], ["--job_chain", "-J", GetoptLong::NO_ARGUMENT, %[Chain multiple simulations into one batch/submission job. Most useful for HPC systems.]], ["--skip-similar-jobs-off", "-k", GetoptLong::NO_ARGUMENT, %[Normally CodeRunner will not submit a run whose input parameters identical to a previous run (to avoid wasting computer time). Specifying the flag will override that behaviour and force submission of an identical run.]], ["--loop", "-l", GetoptLong::NO_ARGUMENT, %[Used with the status command. Keep continually printing out live status information.]], ["--multiple-processes", "-M", GetoptLong::REQUIRED_ARGUMENT], ["--modlet", "-m", GetoptLong::REQUIRED_ARGUMENT, %[Specify the modlet to be used in the current folder. Only needs to be specified once as it will be stored as a default.]], ["--no-run", "-N", GetoptLong::NO_ARGUMENT, %[On some machines getting a list of currently running jobs takes a long time. Specifying this flag tells CodeRunner that you definitely know that no runs in the folder are still queueing or running. Do not specify it if there are still running jobs as it will cause their statuses to be updated incorrectly.]], ["--nprocs", "-n", GetoptLong::REQUIRED_ARGUMENT, %[A string specifying the processor layout for the simulation. For example -n 46x4 means use 46 nodes with four processors per node. In the case of a personal computer something like -n 2 is more likely. The default is 1]], ["--sort", "-O", GetoptLong::REQUIRED_ARGUMENT, %[Specify the sort order for the runs. Used for a variety of commands, for example status. It is a string of semicolon separated sort keys: for example -O height;width will sort the runs by height and then width.]], ["--project", "-P", GetoptLong::REQUIRED_ARGUMENT, %[Specify the project to be used for billing purposes. Only necessary on some systems.]], ["--parameters", "-p", GetoptLong::REQUIRED_ARGUMENT, %[A hash of parameters for the simulation. For example -p '{height: 20, width: 2.3}'. These parameters will override the defaults in the local defaults file.]], ["--queue", "-Q", GetoptLong::REQUIRED_ARGUMENT, %[The name of the queue to submit to on HPC systems. Not yet implemented for all systems. Please submit a feature request if it is not working on your system.]], ["--no-auto-create-runner", "-q", GetoptLong::NO_ARGUMENT, %[Used for interactive mode when you don't want CodeRunner to analyse the current directory.]], ["--terminal-size", "-t", GetoptLong::REQUIRED_ARGUMENT, %[Specify the terminal size for situations where CodeRunner cannot work it out: -t '[rows, cols]' (square brackets are part of the syntax)]], ["--test-submission", "-T", GetoptLong::NO_ARGUMENT, %[Don't actually submit the run, but exit after printing out the run parameters and generating any input files necessary.]], ["--use-large-cache-but-recheck-incomplete", "-u", GetoptLong::NO_ARGUMENT, %[Use the large cache for speed, but check any runs whose status is not :Complete or :Failed.]], ["--use-large-cache", "-U", GetoptLong::NO_ARGUMENT, %[Use the large cache for speed. No run data will be updated.]], ["--version", "-v", GetoptLong::REQUIRED_ARGUMENT, %[Specify the version of the simulation code being used. Only has an effect for certain code modules.]], ["--wall-mins", "-W", GetoptLong::REQUIRED_ARGUMENT, %[Specify the wall clock limit in minutes.]], ["--write-options", "-w", GetoptLong::REQUIRED_ARGUMENT, %[Use when plotting graphs. A hash of custom options which are applied to the graphkit just before plotting it; for example: -w '{xlabel: 'X Axis Quantity, log_axis: 'y'}']], ["--executable", "-X", GetoptLong::REQUIRED_ARGUMENT, %[Specify the location of the executable of the simulation code. It only needs to be specified once in any folder, unless it needs to be changed.]], ["--other-folder", "-Y", GetoptLong::REQUIRED_ARGUMENT, %[Run CodeRunner in a different folder. On a local machine 'coderunner st -Y some/other/folder' is identical to 'cd some/other/folder; coderunner st -Y'. However, this flag can also be used for remote folders using RemoteCodeRunner (as long as CodeRunner is installed on the remote machine). e.g. -Y [email protected]:path/to/folder.]], ["--supplementary-options", "-y", GetoptLong::REQUIRED_ARGUMENT], ["--server", "-Z", GetoptLong::REQUIRED_ARGUMENT, %[Technical use only]], ["--log", "-z", GetoptLong::NO_ARGUMENT, %[Switch logging on (currently not working very well (05/2010)).]] # :nodoc: ]
- CLF =
COMMAND_LINE_FLAGS = COMMAND_LINE_FLAGS_WITH_HELP.map{|arr| arr.slice(0..2)}
- CODE_COMMAND_OPTIONS =
NEEDS FIXING!!!!
[]
- LONG_COMMAND_LINE_OPTIONS =
end
[ ["--replace-existing", "", GetoptLong::NO_ARGUMENT, %[Use with resubmit: causes each resubmitted run to replace the run being resubmitted.]], ["--smart-resubmit-name", "", GetoptLong::NO_ARGUMENT, %[Use with resubmit: causes each resubmitted run to only contain its original id and changed parameters in its run name.]], ] + CODE_COMMAND_OPTIONS
- LONG_COMMAND_LINE_FLAGS =
LONG_COMMAND_LINE_OPTIONS.map{|arr| [arr[0], arr[2]]}
- COMMANDS_WITH_HELP =
[ ["available_modlets", "av", 0, 'List the available modlets for the code module.', [], [:C]], ["available_defaults_files", "avd", 0, 'List the defaults files for the code module.', [], [:C]], ["cancel", "can", 1, 'Cancel the specified job.', ['id'], [:U]], ["code_command", "cc", 1, 'Call a class method of the run class. Effectively this will call run_class.class_eval(command). See documentation for whichever code module is in use.', ['command'], []], ["continue_in_new_folder", "cnf", 1, 'Make a new folder in the parent directory and copy all coderunner configuration files to that folder. If options j or f are specified, copy all matching runs to that new folder.', ['folder'], [:j, :f, :U, :N]], ["code_runner_execute", "crex", 1, 'Run (within the CodeRunner class) the fragment of Ruby code given.', ['Ruby fragment'], []], ["delete", "del", 0, 'Permanently erase all filtered runs.', [], [:j, :F, :U, :N]], ["differences_between", "diff", 0, 'Print a table of all the differences between the input parameters of the filtered ids.', [], [:j, :F, :U, :N]], ["directory", "dir", 1, 'Print out the directory for the given run.', ['id'], []], ['dumb_film', "dfm", 0, 'Create a film of the specified graphkits using gnuplot "dumb" ASCII terminal.', [], [:F, :G, :g, :U, :N, :j, :f]], ["execute", "ex", 1, 'Run (at the top level) the fragment of Ruby code given.', ['Ruby fragment'], []], ['film', "fm", 0, 'Create a film of the specified graphkits.', [], [:F, :G, :g, :U, :N, :j, :f]], ["generate_modlet_from_input_file", "gm", 1, 'Deprecated', [], []], ["generate_cubecalc", "gencc", 0, 'Generate the file cubecalc.cc, the source code for the coderunner test program.', [], []], ["generate_documentation", "rdoc", 1, 'Create automatic documentation using the rdoc tool (deprecated, use the command line tool ri for getting help, or see rubygems.org/gems/coderunner).', [], []], ["interactive_mode", "im", 0, 'Launch an interactive terminal. Any command line flags specified set the defaults for the session.', [], [:U, :N, :j, :q]], ["load_file", "ld", 1, 'Load a Ruby script file using the CodeRunner framework.', ['script file'], []], ['manual', 'man', 0, 'Print out command line manual', [], []], ['netcdf_plot', 'ncplot', 3, 'Plot a comma separated list of variables, at a comma separated list of indices (nil for all) from the specified netcdf file against each other using gnuplot.', ['netcdf_file', 'vars', 'indices'], [:w]], ["plot_graph", "plot", 0, 'Plot the specified graphkits using Gnuplot', [], [:G, :g, :w, :O, :U, :N, :j, :f]], ["parameter_scan", "ps", 1, 'Read a parameter scan from file. For full details of how to write a parameter scan, see online documentation (coderunner.sourceforge.net).', ['scan file'], [:n, :W, :k, :v, :p, :T, :d]], ['print_queue_status', 'qstat', 0, 'Show the current status of the queue', [], [:U, :u]], ["readout", "ro", 0, 'Print a simple text readout of all data from the runs.', [], []], ["reference", "ri", 1, "Print out documentation for the given class or method. #{rihelp}", ['ruby_class_or_method'], []], ["resubmit", "resub", 0, 'Resubmit the filtered runs to be simulated. All parameters will be the same bar those altered by the p option.', [], [:p, :n, :W, :k, :v, :T, :d, :J, :f, :j]], ["run_command", "rc", 1, 'Cause all filtered runs to evaluate the given string.', ['command string'], [:U, :f, :j, :N]], ["runner_eval", "ev", 1, 'Cause the runner (the CodeRunner instance) to evaluate the given string.', ['command string'], [:U, :N, :j, :f]], ["scan", "scan", 1, 'Submit a simple scan. For full details of how to write a simple scan, see online documentation (coderunner.sourceforge.net).', ['scan string'], [:p, :n, :W, :k, :v, :T, :d]], ["show_values_of", "shvl", 1, 'Evaluate the expression for each run and print a unique sorted list of them.', ['expression'], [:U, :N, :j, :f]], ['start_launcher', 'launch', 2, 'Start a simple job launcher for non batch systems.', ['refresh_interval', 'max_queue_size'], []], ["status", "st", 0, 'Print out a summary of the status of the filtered runs.', [], [:U, :N, :j, :f, :O]], ["status_with_comments", "sc", 0, 'Print a list of ids with their status and any comments.', [], [:U, :N, :j, :f, :O]], ["status_loop", "sl", 0, 'Loop, updating the filtered runs, then printing out a summary of the status of the filtered runs. ', [], [:U, :N, :j, :f, :O]], ["submit", "sub", 0, 'Submit a run to be simulated.', [], [:p, :n, :W, :k, :v, :T, :d, :J]], ["submit_command", "subcom", 2, 'Submit an arbitrary shell command to the batch queue.', ['job name', 'command'], [:n, :W, :v, :T, :d]], ["write_graph", "wg", 1, 'Write a graph to disk.', ['filename'], [:G, :g, :w, :O, :U, :N, :j, :f]] ]
- COMMANDS =
This lists all the commands available on the command line. The first two items in each array indicate the long and short form of the command, and the third indicates the number of arguments the command takes. They are all implemented as Code Runner class methods (the method is named after the long form). The short form of the command is available as a global method in Code Runner interactive mode.
COMMANDS_WITH_HELP.map{|arr| arr.slice(0..2)}
- CLF_TO_SHORT_COPTS =
A lookup hash which gives the appropriate short command option (copt) key for a given long command flag
COMMAND_LINE_FLAGS.inject({}){ |hash, (long, short, req)| letter = short[1,1] hash[long] = letter.to_sym hash }
- CLF_BOOLS =
specifying flag sets a bool to be true
[:H, :U, :u, :A, :a, :T, :N, :q, :z, :d, :J, :replace_existing]
- CLF_INVERSE_BOOLS =
CLF_BOOLS = [:s, :r, :D, :H, :U, :u, :L, :l, :A, :a, :T, :N,:V, :q, :z, :d] #
[:k]
- LONG_TO_SHORT =
a look up hash that converts the long form of the command options to the short form (NB command options e.g. use_large_cache have a different form from command line flags e.g. –use-large-cache)
COMMAND_LINE_FLAGS.inject({}){ |hash, (long, short, req)| letter = short[1,1] hash[long[2, long.size].gsub(/\-/, '_').to_sym] = letter.to_sym hash }
- CLF_TO_LONG =
A look up table that converts long only command line options (in LONG_COMMAND_LINE_OPTIONS) to the equivalent CodeRunner command option
LONG_COMMAND_LINE_OPTIONS.inject({}) do |hash, (long, short, req, help)| option = long[2, long.size].gsub(/\-/, '_').to_sym hash[long] = option hash end
- DEFAULT_COMMAND_OPTIONS =
Default command options; they are usually determined by the command line flags, but can be set independently
{}
- CODE_OPTIONS =
{}
- FOLDER_DEFAULTS =
The defaults that are saved in the root folder
[:code, :modlet, :executable, :defaults_file, :project]
- SUBMIT_OPTIONS =
Parameters important to the submission of a run, which can be set by command line flags. The runner values provide the default values for the submit function, but can be overidden in that function. All the runner does with them is set them as properties of the run to be submitted. It is the run itself for which the options are relevant.
[:nprocs, :wall_mins, :sys, :project, :queue, :comment, :executable]
- DEFAULT_RUNNER_OPTIONS =
A hash containing the defaults for most runner options. They are overridden by any options provided during initialisation. They are mostly set at the command line (in practice, the command line flags are read into the command options, which set these defaults in the function CodeRunner.process_command_options which calls CodeRunner.set_runner_defaults). However, if Code Runner is being scripted, these defaults must be set manually or else the options they specify must be provided when initialising a runner.
([:conditions, :sort, :debug, :script_folder, :recalc_all, :multiple_processes, :heuristic_analysis, :test_submission, :reprocess_all, :use_large_cache, :use_large_cache_but_recheck_incomplete, :use_phantom, :no_run, :server, :version, :parameters] + SUBMIT_OPTIONS + FOLDER_DEFAULTS).inject({}){|hash, option| hash[option] = nil; hash}
- CLASS_OPTIONS =
Options that apply across the CodeRunner class
[:multiple_processes].inject({}){|hash, option| class_accessor option set(option, nil) hash[option] = nil; hash }
- NECESSARY_RUN_CLASS_PROPERTIES =
These are properties of the run class that must be defined. For more details see CodeRunner::Run.
{ :code => [String], :variables => [Array], :naming_pars => [Array], :results => [Array], :run_info => [Array], :code_long => [String], :excluded_sub_folders => [Array], :modlet_required => [TrueClass, FalseClass], :uses_mpi => [TrueClass, FalseClass] }
- NECESSARY_RUN_CODE_METHODS =
These are methods that the run class must implement. They should be defined in a code module.
[ :process_directory_code_specific, :print_out_line, :parameter_string, :generate_input_file, :parameter_transition, :executable_location, :executable_name ]
- NECESSARY_RUN_SYSTEM_METHODS =
These are methods that the run class must implement. They should be defined in a system module.
[ :queue_status, :run_command, :execute, :error_file, :output_file, :cancel_job ]
- PERMITTED_STATI =
These are the only permitted values for the run instance variable
@status
. [:Unknown, :Complete, :Incomplete, :NotStarted, :Failed, :Queueing, :Running, :Held]
- SETUP_RUN_CLASSES =
[]
- @@sys =
end
SYS
- @@wait =
Do you wait for the previous run to have completed when using simple scan?
true
Instance Attribute Summary collapse
-
#cache ⇒ Object
Returns the value of attribute cache.
-
#cmaxes ⇒ Object
readonly
Returns the value of attribute cmaxes.
-
#cmins ⇒ Object
readonly
Returns the value of attribute cmins.
-
#code ⇒ Object
Returns the value of attribute code.
-
#combined_ids ⇒ Object
Returns the value of attribute combined_ids.
-
#combined_run_list ⇒ Object
Returns the value of attribute combined_run_list.
-
#current_request ⇒ Object
Returns the value of attribute current_request.
-
#current_status ⇒ Object
Returns the value of attribute current_status.
-
#defaults_file ⇒ Object
Returns the value of attribute defaults_file.
-
#executable ⇒ Object
Returns the value of attribute executable.
-
#ids ⇒ Object
Returns the value of attribute ids.
-
#max_id ⇒ Object
readonly
Returns the value of attribute max_id.
-
#maxes ⇒ Object
readonly
Returns the value of attribute maxes.
-
#mins ⇒ Object
readonly
Returns the value of attribute mins.
-
#modlet ⇒ Object
Returns the value of attribute modlet.
-
#phantom_ids ⇒ Object
Returns the value of attribute phantom_ids.
-
#phantom_run_list ⇒ Object
Returns the value of attribute phantom_run_list.
-
#print_out_size ⇒ Object
Returns the value of attribute print_out_size.
-
#requests ⇒ Object
Returns the value of attribute requests.
-
#root_folder ⇒ Object
Returns the value of attribute root_folder.
-
#run_class ⇒ Object
Returns the value of attribute run_class.
-
#run_list ⇒ Object
Returns the value of attribute run_list.
-
#start_id ⇒ Object
readonly
Returns the value of attribute start_id.
Class Method Summary collapse
-
.available_defaults_files(copts = {}) ⇒ Object
List the available defaults files for the given code (copts or -C on the command line).
-
.available_modlets(copts = {}) ⇒ Object
List the available modlets for the given code (copts or -C on the command line).
-
.cancel(id, copts = {}) ⇒ Object
Cancel the job with the given id.
- .code_command(string, copts = {}) ⇒ Object
- .code_runner_execute(ruby_fragment, copts = {}) ⇒ Object
- .continue_in_new_folder(folder, copts = {}) ⇒ Object
- .delete(copts = {}) ⇒ Object
- .differences_between(copts = {}) ⇒ Object
- .directory(id, copts = {}) ⇒ Object
- .dumb_film(copts = {}) ⇒ Object
- .execute(ruby_fragment, copts = {}) ⇒ Object
-
.fetch_runner(copts = {}) ⇒ Object
Retrieve the runner with the folder (and possibly server) given in copts.
- .film(copts = {}) ⇒ Object
- .film_graphkit_frame_array(graphkit_frame_array, options) ⇒ Object
- .generate_cubecalc(copts = {}) ⇒ Object
- .generate_documentation(username = nil, copts = {}) ⇒ Object
-
.get_run_class_name(code, modlet = nil) ⇒ Object
Return the name of the run class according to the standard CodeRunner naming scheme.
- .gets ⇒ Object
-
.graphkit_multiple_runners(list, options = {}) ⇒ Object
list is an array of [[runner, [graphs, run_graphs]], … ].
- .graphkit_multiple_runners_with_frame_array(frame_array, list, extra_options, print_message = false) ⇒ Object
-
.interactive_mode(copts = {}) ⇒ Object
module InteractiveMethods.
- .launcher_directory ⇒ Object
-
.load_file(files, copts = {}) ⇒ Object
eval(ruby_fragment).
-
.make_film_multiple_runners(list, options) ⇒ Object
__END__.
- .make_film_multiple_runners_old(list, options) ⇒ Object
- .manual(copts = {}) ⇒ Object
-
.netcdf_plot(netcdf_file, vars, indices, copts = {}) ⇒ Object
end.
-
.old_get_run_class_name(code, modlet) ⇒ Object
:nodoc:.
- .parameter_scan(parameter_scan_array_file, copts = {}) ⇒ Object
- .plot_graph(copts = {}) ⇒ Object
- .print_queue_status(copts = {}) ⇒ Object
-
.process_command_line_option(opt, arg, copts) ⇒ Object
Converts a command line flag opt with value arg to a command option which is stored in copts.
- .process_command_options(copts) ⇒ Object
- .read_default_command_options(copts) ⇒ Object
- .readout(copts = {}) ⇒ Object
- .recheck(id, copts = {}) ⇒ Object
- .reference(class_or_method, copts = {}) ⇒ Object
- .repair_marshal_run_class_not_found_error(err) ⇒ Object
-
.resubmit(copts = {}) ⇒ Object
puts “Got here” exit(0).
- .run_class(copts = {}) ⇒ Object
-
.run_command(string, copts = {}) ⇒ Object
runner = fetch_runner(copts) runner.run_class.class_eval(string).
- .run_script ⇒ Object
- .runner ⇒ Object
- .runner_eval(string, copts = {}) ⇒ Object
- .scan(scan_string, copts = {}) ⇒ Object
- .server_dump(obj) ⇒ Object
- .set_class_defaults(copts = {}) ⇒ Object
- .set_default_command_options_from_command_line ⇒ Object
-
.set_runner_defaults(copts = {}) ⇒ Object
In the next section are the implementations of all the standard Code Runner commands and some helper functions.
- .setup_run_class(code, options = {}) ⇒ Object
- .show_values_of(expression, copts = {}) ⇒ Object
- .start_launcher(refresh, max_queue, copts = {}) ⇒ Object
- .status(copts = {}) ⇒ Object
- .status_loop(copts = {}) ⇒ Object
- .status_with_comments(copts = {}) ⇒ Object
- .submit(copts = {}) ⇒ Object
-
.submit_command(jid, comm, copts = {}) ⇒ Object
This method allows the straightforward submission of a single command using the batch queue on any system.
-
.update_runners ⇒ Object
@r.read_defaults.
- .write_graph(name, copts = {}) ⇒ Object
Instance Method Summary collapse
-
#add_phantom_run(run) ⇒ Object
def rename_variable(old, new) puts “Please confirm complete renaming of #old to #new” gets @run_list.each do |directory| Dir.chdir directory do begin @readme = File.read(“CODE_RUNNER_INPUTS”) rescue Errno::ENOENT => err # puts err, err.class @readme = File.read(“README”) end @readme.sub!(Regexp.new(“^s+#old:”), “^s+#new:”) File.open(“CODE_RUNNER_INPUTS_TEST”, ‘w’){|file| file.puts @readme} end end old_recalc, @recalc_all = @recalc_all, true update @recalc_all = old_recalc end.
-
#alter_ids(num, options = {}) ⇒ Object
Permanently change the id of every run in the folder by adding num to them.
-
#axiskit(string) ⇒ Object
def gnuplot(axes, options) a = graphkit(axes, options).gnuplot end.
-
#cancel_job(id, options = {}) ⇒ Object
Cancel the job with the given ID.
- #code_run_environment ⇒ Object
- #continue_in_new_folder(folder, options = {}) ⇒ Object
-
#create_archive(options = {}) ⇒ Object
Create a tar archive of the root folder and all the files in it.
-
#destroy(options = {}) ⇒ Object
Delete the folders of all runs for whom CodeRunner#filter(run) is true.
-
#dup ⇒ Object
Duplicate the runner, trying to be intelligent as far as possible in duplicating instance variables.
- #executable_location ⇒ Object
- #executable_name ⇒ Object
- #film_graphkit(axes, options, film_options = {}) ⇒ Object
- #film_run_graphkit(name, options, film_options = {}) ⇒ Object
-
#filter(run = @run, conditions = @conditions) ⇒ Object
Return true if run.instance_eval(conditions) == true and false if run.instance_eval(conditions) == false.
-
#filtered_ids(conditions = @conditions) ⇒ Object
Return a list of ids, filtered according to conditions.
- #generate_combined_ids(kind = nil) ⇒ Object
-
#get_all_root_folder_contents ⇒ Object
List every file in the root folder.
-
#get_max(run, variable, sweep, complete = nil) ⇒ Object
def max(variable,sweep=nil, complete=nil) logf(:max) return get_max(variable,sweep, complete) == @run.id end.
- #get_min(run, variable, sweep, complete = nil) ⇒ Object
- #get_run_class ⇒ Object
-
#get_run_class_name(code, modlet) ⇒ Object
See CodeRunner.get_run_class_name.
-
#gets ⇒ Object
No reading from the command line thank you very much!.
-
#graphkit(*args) ⇒ Object
call-seq: graphkit(axes, options) Make a general graphkit, i.e.
- #graphkit_from_lists(graphs, run_graphs, extra_options = {}) ⇒ Object
- #graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, extra_options = {}) ⇒ Object
-
#graphkit_shorthand(*gr) ⇒ Object
Parse graphkit shorthand for a general graph and return it as [axes, options].
-
#increment_max_id ⇒ Object
Increase the value of
@max_id
by 1. -
#initialize(root_folder, options = {}) ⇒ CodeRunner
constructor
Instantiate a new runner.
- #job_identifier ⇒ Object
- #make_film_from_lists(graphs, run_graphs, film_options = {}) ⇒ Object
-
#marshalled_variables ⇒ Object
Dump all the instance variables of the runner to stdout as Marshalled binary data.
- #merge(*others) ⇒ Object
- #merge_method(meth, *args) ⇒ Object
-
#new_run ⇒ Object
Create a new instance of the class
@run_class
. -
#p(*args) ⇒ Object
Here we redefine the inspect function p to raise an error if anything is written to standard out while in server mode.
- #parameter_scan(parameter_scan, parameters, options = {}) ⇒ Object
-
#print(*args) ⇒ Object
Here we redefine the function print to raise an error if anything is written to standard out while in server mode.
-
#print_out(rewind = nil, options = {}) ⇒ Object
Print out a summary of all the runs in the root folder, formatted in nice pretty colours.
-
#puts(*args) ⇒ Object
Here we redefine the function puts to raise an error if anything is written to standard out while in server mode.
-
#rcp ⇒ Object
def submit.
- #read_defaults ⇒ Object
-
#read_folder_defaults ⇒ Object
Read any default options contained in the file
.code_runner_script_defaults.rb
in the root folder. -
#readout ⇒ Object
Similar to CodeRunner#write_data, except that the readout is written to stdout, and formatted a bit.
-
#readout_cols(*var_list) ⇒ Object
? Probably can get rid of this one.
-
#recheck_filtered_runs(write_status_dots = false) ⇒ Object
Self-explanatory! Call CodeRunner::Run#process_directory for every run for which CodeRunner#filter(run) is true.
-
#recheck_incomplete_runs ⇒ Object
Self-explanatory! Call CodeRunner::Run#process_directory for every run whose status is not either :Complete or :Failed.
-
#respond_to_requests ⇒ Object
One of the features of CodeRunner is two way communication between a runner and its runs.
- #run_graphkit(*args) ⇒ Object
- #run_graphkit_shorthand(*grs) ⇒ Object
-
#runs ⇒ Object
Return an array of runs (run_list.values).
- #save_all ⇒ Object
-
#save_large_cache ⇒ Object
Write the variable
@run_list
, which contains all information currently known about the simulations in the root folder, as Marshalled binary data in the file “.CODE_RUNNER_TEMP_RUN_LIST_CACHE”. - #server_dump(obj) ⇒ Object
- #set_start_id(id) ⇒ Object
-
#setup_run_class(code, options) ⇒ Object
See CodeRunner.setup_run_class.
-
#similar_runs(exclude_variables = [], run = @run) ⇒ Object
Find runs whose input parameters are all the same as those for
run
, with the exception of those listed inexclude_variables
. -
#simple_scan(scan_string, options = {}) ⇒ Object
Submit a series of runs according to scan_string.
-
#sort_runs(type = @use_phantom.to_s.sub(/real/, '')) ⇒ Object
Sort the runs according to the variable
@sort
which can be either whitespace separated string or an array of strings. -
#submit(runs, options = {}) ⇒ Object
Start a simulation: submit the run that is passed.
- #sweep_graphkits(sweeps, options, &block) ⇒ Object
-
#update(write_status_dots = true, use_large_cache = @use_large_cache, use_large_cache_but_recheck_incomplete = @use_large_cache_but_recheck_incomplete) ⇒ Object
Update the information about all the runs stored in the variable
@run_list
. -
#write_data(filename = "data.txt") ⇒ Object
Write out a simple datafile containing all the inputs and outputs listed in the run class property
readout_string
.
Constructor Details
#initialize(root_folder, options = {}) ⇒ CodeRunner
Instantiate a new runner. The root folder contains a set of simulations, each of which has a unique ID. There is a one-to-one correspondence between a runner and a root folder: no two runners should ever be given the same root folder in the same script (there are safeguards to prevent this causing much trouble, but it should still be avoided on philosophical grounds), and no runner should be given a folder which has more than one set of simulations within it (as these simulations will contain duplicate IDs).
Options is a hash whose keys may be any of the keys of the constant DEFAULT_RUNNER_OPTIONS
. I.e. to see what options can be passed:
p CodeRunner::DEFAULT_RUNNER_OPTIONS.keys
166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 |
# File 'lib/coderunner/instance_methods.rb', line 166 def initialize(root_folder, ={}) logf :initialize raise CRFatal.new("System not defined") unless SYS root_folder.sub!(/~/, ENV['HOME']) @root_folder = root_folder read_defaults .each do |key,value| key = LONG_TO_SHORT.key(key) if LONG_TO_SHORT.key(key) set(key, value) if value end # ep options log 'modlet in initialize', @modlet @version= [:version] # ep 'ex', @executable #ep 'modlet is ', @modlet get_run_class @cache = {} @n_checks = 0 @print_out_size = 0 @run_list = {}; @ids = [] set_max_id(0) @phantom_run_list = {}; @phantom_ids = [] @phantom_id = -1 @combined_run_list = {}; @combined_ids = [] @current_request = nil @requests = [] @pids= [] @maxes = {}; @mins = {} @cmaxes = {}; @cmins = {} end |
Instance Attribute Details
#cache ⇒ Object
Returns the value of attribute cache.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def cache @cache end |
#cmaxes ⇒ Object (readonly)
Returns the value of attribute cmaxes.
157 158 159 |
# File 'lib/coderunner/instance_methods.rb', line 157 def cmaxes @cmaxes end |
#cmins ⇒ Object (readonly)
Returns the value of attribute cmins.
157 158 159 |
# File 'lib/coderunner/instance_methods.rb', line 157 def cmins @cmins end |
#code ⇒ Object
Returns the value of attribute code.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def code @code end |
#combined_ids ⇒ Object
Returns the value of attribute combined_ids.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def combined_ids @combined_ids end |
#combined_run_list ⇒ Object
Returns the value of attribute combined_run_list.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def combined_run_list @combined_run_list end |
#current_request ⇒ Object
Returns the value of attribute current_request.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def current_request @current_request end |
#current_status ⇒ Object
Returns the value of attribute current_status.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def current_status @current_status end |
#defaults_file ⇒ Object
Returns the value of attribute defaults_file.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def defaults_file @defaults_file end |
#executable ⇒ Object
Returns the value of attribute executable.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def executable @executable end |
#ids ⇒ Object
Returns the value of attribute ids.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def ids @ids end |
#max_id ⇒ Object (readonly)
Returns the value of attribute max_id.
157 158 159 |
# File 'lib/coderunner/instance_methods.rb', line 157 def max_id @max_id end |
#maxes ⇒ Object (readonly)
Returns the value of attribute maxes.
157 158 159 |
# File 'lib/coderunner/instance_methods.rb', line 157 def maxes @maxes end |
#mins ⇒ Object (readonly)
Returns the value of attribute mins.
157 158 159 |
# File 'lib/coderunner/instance_methods.rb', line 157 def mins @mins end |
#modlet ⇒ Object
Returns the value of attribute modlet.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def modlet @modlet end |
#phantom_ids ⇒ Object
Returns the value of attribute phantom_ids.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def phantom_ids @phantom_ids end |
#phantom_run_list ⇒ Object
Returns the value of attribute phantom_run_list.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def phantom_run_list @phantom_run_list end |
#print_out_size ⇒ Object
Returns the value of attribute print_out_size.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def print_out_size @print_out_size end |
#requests ⇒ Object
Returns the value of attribute requests.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def requests @requests end |
#root_folder ⇒ Object
Returns the value of attribute root_folder.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def root_folder @root_folder end |
#run_class ⇒ Object
Returns the value of attribute run_class.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def run_class @run_class end |
#run_list ⇒ Object
Returns the value of attribute run_list.
156 157 158 |
# File 'lib/coderunner/instance_methods.rb', line 156 def run_list @run_list end |
#start_id ⇒ Object (readonly)
Returns the value of attribute start_id.
157 158 159 |
# File 'lib/coderunner/instance_methods.rb', line 157 def start_id @start_id end |
Class Method Details
.available_defaults_files(copts = {}) ⇒ Object
List the available defaults files for the given code (copts or -C on the command line).
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
# File 'lib/coderunner/class_methods.rb', line 41 def self.available_defaults_files(copts={}) (copts) entries = [] #begin ##entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/my_defaults_files") #entries += #rescue #end #begin #run_class = setup_run_class(copts[:C], modlet: copts[:m]) rc = run_class(copts) entries = [rc.rcp.user_defaults_location, rc.rcp.code_module_folder + "/defaults_files"].map{|folder| Dir.entries(folder).grep(/_defaults\.rb$/) rescue []}.sum #entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/defaults_files") #entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/defaults_files") #rescue #end puts "\nAvailable defaults files for #{copts[:C]}:" entries.each do |defaults_file| #puts "\t" + File.basename(defaults_file, '.rb').sub(/_defaults/, '') unless ['.', '..', '.svn', '.directory'].include? defaults_file puts "\t" + File.basename(defaults_file, '.rb').sub(/_defaults/, '') end end |
.available_modlets(copts = {}) ⇒ Object
List the available modlets for the given code (copts or -C on the command line).
22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
# File 'lib/coderunner/class_methods.rb', line 22 def self.available_modlets(copts={}) (copts) puts "\nAvailable modlets for #{copts[:C]}:" entries = [] begin entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/my_modlets") rescue end begin entries += Dir.entries(SCRIPT_FOLDER + "/code_modules/#{copts[:C]}/default_modlets") rescue end entries.each do |modlet| puts "\t" + File.basename(modlet, '.rb') unless ['.', '..', '.svn', '.directory'].include? modlet or modlet=~ /defaults/ end end |
.cancel(id, copts = {}) ⇒ Object
Cancel the job with the given id. The user is asked interactively for confirmation and whether they would like to delete the folder for that job as well.
66 67 68 69 |
# File 'lib/coderunner/class_methods.rb', line 66 def self.cancel(id, copts={}) runner = fetch_runner(copts) runner.cancel_job(id.to_i) end |
.code_command(string, copts = {}) ⇒ Object
465 466 467 468 469 470 |
# File 'lib/coderunner/class_methods.rb', line 465 def self.code_command(string, copts = {}) run_class(copts).class_eval(string) # runner = fetch_runner(copts) # runner.run_class.class_eval(string) end |
.code_runner_execute(ruby_fragment, copts = {}) ⇒ Object
386 387 388 389 |
# File 'lib/coderunner/class_methods.rb', line 386 def self.code_runner_execute(ruby_fragment, copts={}) #eval(ruby_fragment, GLOBAL_BINDING) eval(ruby_fragment) end |
.continue_in_new_folder(folder, copts = {}) ⇒ Object
71 72 73 74 75 76 77 78 79 |
# File 'lib/coderunner/class_methods.rb', line 71 def self.continue_in_new_folder(folder, copts={}) runner=fetch_runner(copts) = {} if copts[:f] or copts[:j] [:copy_ids] = runner.filtered_ids end runner.continue_in_new_folder(folder, ) end |
.delete(copts = {}) ⇒ Object
81 82 83 84 |
# File 'lib/coderunner/class_methods.rb', line 81 def self.delete(copts={}) runner = fetch_runner(copts) runner.destroy end |
.differences_between(copts = {}) ⇒ Object
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
# File 'lib/coderunner/class_methods.rb', line 85 def self.differences_between(copts = {}) runner = fetch_runner(copts) runs = runner.filtered_ids.map{|id| runner.run_list[id]} vars = runner.run_class.rcp.variables.dup + runner.run_class.rcp.run_info.dup vars.delete_if{|var| runs.map{|r| r.send(var)}.uniq.size == 1} vars.delete :id vars.delete :run_name vars.delete :output_file vars.delete :error_file vars.delete :executable vars.delete :comment vars.delete :naming_pars vars.delete :parameter_hash vars.unshift :id #vars.push 'File.basename(executable)' table = vars.map{|var| [var] + runs.map{|r| str = r.instance_eval(var.to_s).to_s; str.size>10?str[0..9]:str} } #vars[-1] = 'exec' col_widths = table.map{|row| row.map{|v| v.to_s.size}}.inject{|o,n| o.zip(n).map{|a| a.max}} eputs table.each{|row| i=0; eputs row.map{|v| str = sprintf(" %#{col_widths[i]}s ", v.to_s); i+=1; str}.join('|'); eputs '-' * (col_widths.sum + col_widths.size*3 - 1) } #p table, col_widths end |
.directory(id, copts = {}) ⇒ Object
201 202 203 204 |
# File 'lib/coderunner/class_methods.rb', line 201 def self.directory(id, copts={}) runner = fetch_runner(copts) puts runner.run_list[id.to_i].directory end |
.dumb_film(copts = {}) ⇒ Object
108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 |
# File 'lib/coderunner/class_methods.rb', line 108 def self.dumb_film(copts = {}) # process_copts(copts) #old_term = GraphKit::GNUPLOT_DEFAULT_TERM size = Terminal.terminal_size size[0] -= 2 term = "dumb #{size.reverse.join(' ')}" string = "\n" * size[0] runner = fetch_runner(copts) string_to_eval = copts[:w] frame_array = copts[:F][:frame_array] || copts[:F][:fa] index_name = copts[:F][:index_name] || copts[:F][:in] #options = (options and options =~ /\S/) ? eval(options): {} puts string for index in frame_array[0]..frame_array[1] string.true_lines.times{print "\033[A"} kit = runner.graphkit_from_lists(copts[:G], copts[:g], index_name => index) kit.gp.term = term kit.gnuplot(eval: string_to_eval) sleep(copts[:F][:fr] ? 1.0/copts[:F][:fr] : 0.1) end end |
.execute(ruby_fragment, copts = {}) ⇒ Object
390 391 392 393 |
# File 'lib/coderunner/class_methods.rb', line 390 def self.execute(ruby_fragment, copts={}) eval(ruby_fragment, GLOBAL_BINDING) #eval(ruby_fragment) end |
.fetch_runner(copts = {}) ⇒ Object
Retrieve the runner with the folder (and possibly server) given in copts. If no runner has been loaded for that folder, load one.
731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 |
# File 'lib/coderunner/class_methods.rb', line 731 def self.fetch_runner(copts={}) # ep copts # If copts(:Y) is an array of locations, return a merged runner of those locations if copts[:Y].kind_of? Array runners = copts[:Y].map do |location| new_copts = copts.dup.absorb(Y: location) fetch_runner(new_copts) end return Merged.new(*runners) end (copts) # ep copts @runners ||= {} runner = nil if copts[:Y] and copts[:Y] =~ /:/ copts_r = copts.dup host, folder = copts[:Y].split(':') copts_r[:Y] = nil copts[:Y] = nil unless @runners[[host, folder]] copts[:cache] ||= :auto runner = @runners[[host, folder]] = RemoteCodeRunner.new(host, folder, copts) #(eputs 'Updating remote...'; runner.update) unless (copts[:g] and (copts[:g].kind_of? String or copts[:g].size > 0)) or copts[:no_update] or copts[:cache] else runner = @runners[[host, folder]] end runner.process_copts(copts) else copts[:Y] ||= Dir.pwd Dir.chdir((copts[:Y] or Dir.pwd)) do unless @runners[copts[:Y]] runner = @runners[copts[:Y]] = CodeRunner.new(Dir.pwd, code: copts[:C], modlet: copts[:m], version: copts[:v], executable: copts[:X], defaults_file: copts[:D]) runner.update unless copts[:no_update] else runner = @runners[copts[:Y]] end #p 'reading defaults', runner.recalc_all, DEFAULT_RUNNER_OPTIONS runner.read_defaults #p 'read defaults', runner.recalc_all end #Dir.chdir end # ep copts return runner # @r.read_defaults end |
.film(copts = {}) ⇒ Object
205 206 207 208 209 |
# File 'lib/coderunner/class_methods.rb', line 205 def self.film(copts={}) runner = fetch_runner(copts) copts[:F][:graphkit_modify] = copts[:w] runner.make_film_from_lists(copts[:G], copts[:g], copts[:F]) end |
.film_graphkit_frame_array(graphkit_frame_array, options) ⇒ Object
650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 |
# File 'lib/coderunner/graphs_and_films.rb', line 650 def self.film_graphkit_frame_array(graphkit_frame_array, ) = [:frame_array, :fa, :skip_frames, :sf, :normalize, :n, :normalize_pieces, :np, :increment, :i, :skip_encoding, :se, :frame_rate, :fr, :size] # fa = (options[:frame_array] or options[:fa] or list[0][0].run_list[list[0][0].filtered_ids[0]].frame_array(options) ) fd = frame_digits = [:fd]||Math.log10(graphkit_frame_array.map{|f, g| f}.max).ceil extension = ([:extension] or [:ext] or '.png') extension = '.' + extension unless extension =~ /^\./ unless [:skip_frames] or [:sf] FileUtils.rm_r('film_frames') if FileTest.exist?('film_frames') FileUtils.makedirs('film_frames') # puts @@multiple_processes; gets no_forks = (@@multiple_processes or 1) ep @@multiple_processes, no_forks # end_graphkit = graphkit_multiple_runners(list, frame_index: fa[1]) # begin_graphkit = graphkit_multiple_runners(list, frame_index: fa[0]) # end_area = end_graphkit.plot_area_size # begin_area = begin_graphkit.plot_area_size # p end_area, begin_area, options # plot_size = {} # axes = [:x, :y, :z] # options[:normalize] ||= options[:nm] # options[:normalize_pieces] ||= options[:nmp] # for i in 0...end_area.size # next unless options[:normalize] and options[:normalize].include? axes[i] # min = [end_area[i][0], begin_area[i][0]].min # max = [end_area[i][1], begin_area[i][1]].max # key = axes[i] # plot_size[key + :range] = [min, max] # end # ep plot_size # exit # frames = [] # actual_frames = {} # i = fa[0] # j = 0 # while i <= fa[1] # frames.push i # actual_frames[i] = j # i += (options[:ic] or options[:increment] or 1) # j += 1 # end i = 0 actual_frames = graphkit_frame_array.map{|f, g| f}.inject({}){|hash, f| hash[f] = i; i+=1; hash} graphkit_frame_array.pieces(no_forks).each_with_index do |graphkit_frame_array_piece, myrank| fork do # if options[:normalize_pieces] # end_area = graphkit_multiple_runners(list, frame_index: piece.max).plot_area_size # begin_area = graphkit_multiple_runners(list, frame_index: piece.min).plot_area_size # axes = [:x, :y, :z] # for i in 0...end_area.size # next unless options[:normalize_pieces].include? axes[i] # min = [end_area[i][0], begin_area[i][0]].min # max = [end_area[i][1], begin_area[i][1]].max # key = axes[i] # # plot_size[key + :range] = [min, max] # end # end # eputs 'making graphs...'; sleep 1 # graph_array = graphkit_multiple_runners_with_frame_array(piece, list, myrank==0) # # ep graph_array # eputs graphkit_frame_array_piece.each_with_index do |(frame_index,g), pindex| if myrank == 0 eputs "\033[2A" # Terminal jargon - go back one line eputs sprintf("Plotting graphs: %2.2f", pindex.to_f/graphkit_frame_array_piece.size.to_f * 100.0) + "% Complete" end # g = graph_kit_multiple_runners(list, plot_size + {frame_index: frame_index}) # g.modify(plot_size) # g.modify(options) # p g; exit # g.title += sprintf(", frame %0#{fd}d", frame_index) unless options[:frame_title] == false folder = ("film_frames/"); file_name = sprintf("frame_%0#{fd}d", actual_frames[frame_index]) # g.gnuplot; gets; g.close # ep folder + file_name + '.png'; gets g.gnuplot_write(folder + file_name + extension, size: [:size]) end end end eputs "Waiting on subprocesses..." Process.waitall end unless [:skip_encoding] eputs "making film" frame_rate = ([:frame_rate] or [:fr] || 15) film_name = ([:film_name] or [:fn] or graphkit_frame_array[0][1].file_name + '_film').gsub(/\s/, '_') puts `ffmpeg -y #{[:bitrate] ? "-b #{[:bitrate]}" : ""} -r #{frame_rate} -threads #{(@multiple_processes or 1)} -i film_frames/frame_%0#{fd}d#{extension} #{film_name}.mp4` end end |
.generate_cubecalc(copts = {}) ⇒ Object
259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 |
# File 'lib/coderunner/class_methods.rb', line 259 def self.generate_cubecalc(copts={}) #return File.open('cubecalc.cc', 'w') do |file| file.puts <<EOF #include <stdio.h> #include <iostream> #include <cstdlib> #include <fstream> #include <cstring> #include <ctime> using namespace std; int main(int argc, char* argv[]){ string line; cout << "Starting..." << endl; int calculate_sides = atoi(argv[1]); //Should the program calculate the area of the sides of the cube? cout << calculate_sides << endl; char* input_file_name = argv[2]; //Get the input file name from the command line cout << input_file_name << endl; if (argc > 3){ //It has been told to sleep for a time bool cont = true; time_t start_t; time(&start_t); while (cont){ time_t new_t; time(&new_t); cont = (new_t < (start_t + atoi(argv[3]) * 1.0)); } } ifstream edges_file(input_file_name); //Read the edges from the input file float* edges = new float[3]; int j = 0; while (edges_file >> edges[j++]){ cout << edges[j-1] << endl; } FILE* output = fopen("results.txt", "w"); //Write the volume to the output file fprintf(output, "Volume was %f", edges[0] * edges[1] * edges[2]); fclose(output); if (calculate_sides == 1){ //If it has been told to calculate the sides cout << "calculating sides" << endl; FILE* sides = fopen("sides.txt", "w"); for(int i=0; i<3; i++){ cout << "Side " << i << ": " << edges[(i%3)] * edges[((i+1)%3)] << endl; fprintf(sides, "The area of side %d is %f\\n", i, edges[i%3] * edges[(i+1)%3]); } fclose(sides); } } EOF end end |
.generate_documentation(username = nil, copts = {}) ⇒ Object
211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 |
# File 'lib/coderunner/class_methods.rb', line 211 def self.generate_documentation(username = nil, copts = {}) ep 'username', username||=ENV['USER'] ####### Here we use the command line documentation to generate a fake ruby file that rdoc will understand. File.open("class_methods_rdoc.rb", 'w') do |file| file.puts <<EOF class CodeRunner #{COMMANDS_WITH_HELP.inject("") do |str, (long, short, nargs, comhelp, argnames, )| (puts "Please run this command in the coderunner trunk directory"; exit) unless Dir.pwd =~ /coderunner\/trunk$/ str << <<EOF2 # #{comhelp.gsub(/\n/, "\n # ")} # # Possible options: # #{.inject("") do |str, opt| longop, shortop, req, ophelp = COMMAND_LINE_FLAGS_WITH_HELP.find{|arr| arr[1] == "-" + opt.to_s} str << " # :#{opt} --- #{ophelp.gsub(/\n/, "\n # ")}\n #\n" end} def self.#{long}(#{(argnames+[""]).join(",")}command_options={}) end EOF2 end } end EOF end # exit system "rm -rf doc/" system "rm -rf ri/" raise 'Please set RDOC_COMMAND' unless ENV['RDOC_COMMAND'] system "#{ENV['RDOC_COMMAND']} --format=html -t 'CodeRunner Documentation' -m INDEX.rb INDEX.rb code_runner_version.rb gnuplot.rb graphkit_gnuplot.rb graphkit.rb gsl_tools.rb run_backwards_compatibility.rb feedback.rb run.rb fortran_namelist.rb graphs_and_films.rb class_methods_rdoc.rb instance_methods.rb" system "#{ENV['RDOC_COMMAND']} -r --op ri INDEX.rb code_runner_version.rb gnuplot.rb graphkit_gnuplot.rb graphkit.rb gsl_tools.rb run_backwards_compatibility.rb feedback.rb run.rb fortran_namelist.rb graphs_and_films.rb class_methods_rdoc.rb instance_methods.rb" exit if username == "" string = "rsync -av --delete doc/ #{username},[email protected]:htdocs/api_documentation/" puts string exec string end |
.get_run_class_name(code, modlet = nil) ⇒ Object
Return the name of the run class according to the standard CodeRunner naming scheme. If the code name is ‘a_code_name’, with no modlet, the run class name will be ACodeName
. If on the other hand there is a modlet called ‘modlet_name’, the class name will be ACodeName::ModletName
.
347 348 349 |
# File 'lib/coderunner/instance_methods.rb', line 347 def self.get_run_class_name(code, modlet=nil) return modlet ? "#{code.capitalize}::#{modlet.capitalize.sub(/\.rb$/, '').variable_to_class_name}" : "#{code.capitalize}" end |
.gets ⇒ Object
42 43 44 |
# File 'lib/coderunner.rb', line 42 def self.gets $stdin.gets end |
.graphkit_multiple_runners(list, options = {}) ⇒ Object
list is an array of [[runner, [graphs, run_graphs]], … ]
424 425 426 427 428 429 430 431 432 |
# File 'lib/coderunner/graphs_and_films.rb', line 424 def self.graphkit_multiple_runners(list, ={}) return list.inject(nil) do |kit, (runner, graph_lists)| graphs, run_graphs = graph_lists graphs.map!{|graph| runner.graphkit_shorthand(graph)} run_graphs.map!{|graph| runner.run_graphkit_shorthand(graph)} newkit = runner.graphkit_from_lists(graphs, run_graphs, ) kit ? kit + newkit : newkit end end |
.graphkit_multiple_runners_with_frame_array(frame_array, list, extra_options, print_message = false) ⇒ Object
434 435 436 437 438 439 440 441 442 443 444 445 |
# File 'lib/coderunner/graphs_and_films.rb', line 434 def self.graphkit_multiple_runners_with_frame_array(frame_array, list, , = false) i = 0 # return list.inject(nil) do |kit_array, (runner, graph_lists)| graphs, run_graphs = graph_lists graphs.map!{|graph| runner.graphkit_shorthand(graph)} run_graphs.map!{|graph| runner.run_graphkit_shorthand(graph)} newkit_array = runner.graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, ) # eputs newkit_array.pretty_inspect kit_array ? (i=-1; kit.map{|(frame_index, kit)| i+= 1;[frame_index, new_kit[i][1]]}) : newkit_array end end |
.interactive_mode(copts = {}) ⇒ Object
module InteractiveMethods
53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 |
# File 'lib/coderunner/interactive_methods.rb', line 53 def CodeRunner.interactive_mode(copts={}) (copts) unless false and FileTest.exist? (ENV['HOME'] + '/code_runner_interactive_options.rb') File.open(ENV['HOME'] + '/.code_runner_interactive_options.rb', 'w') do |file| file.puts <<EOF $has_put_startup_message_for_code_runner = true #please leave! $code_runner_interactive_mode = true #please leave! require 'yaml' def reset Dispatcher.reset_application! end IRB.conf[:AUTO_INDENT] = true IRB.conf[:USE_READLINE] = true IRB.conf[:LOAD_MODULES] = [] unless IRB.conf.key?(:LOAD_MODULES) unless IRB.conf[:LOAD_MODULES].include?('irb/completion') IRB.conf[:LOAD_MODULES] << 'irb/completion' end require 'irb/completion' require 'irb/ext/save-history' require 'socket' unless ENV['CODE_RUNNER_SYSTEM'] == 'macosx' prompt_start = "\001#{Terminal::LIGHT_GREEN}\002CodeRunner\001#{Terminal::RESET}\002 v#{CODE_RUNNER_VERSION} - \001#{Terminal::CYAN}\002#\{ENV['USER']\}@#\{Socket::gethostname\}:#\{File.basename(Dir.pwd)}\001#{Terminal::RESET}\002 timemarker\n" prompt_start = "\001#{Terminal::LIGHT_GREEN}\002CodeRunner\001#{Terminal::RESET}\002 v#{CODE_RUNNER_VERSION} - \001#{Terminal::CYAN}\002#\{Socket::gethostname\}:#\{File.basename(Dir.pwd)}\001#{Terminal::RESET}\002 timemarker %02n,%i \n" else prompt_start = "CodeRunner #\{File.basename(Dir.pwd)}" end IRB.conf[:PROMPT][:CODE_RUNNER] = {:PROMPT_I=>"#\{prompt_start}>> %02n,%i >> ", :PROMPT_N=>"#\{prompt_start}>> %02n:%i > ", :PROMPT_S=>"#\{prompt_start}>> %02n:%i (%l)> ", :PROMPT_C=>"#\{prompt_start}>> %02n:%i >> ", :RETURN=>""} IRB.conf[:PROMPT][:CODE_RUNNER] = {:PROMPT_I=>"#\{prompt_start}>> ", :PROMPT_N=>"#\{prompt_start}> ", :PROMPT_S=>"#\{prompt_start}(%l)> ", :PROMPT_C=>"#\{prompt_start}>> ", :RETURN=>""} # IRB.conf[:PROMPT][:CODE_RUNNER] = {:PROMPT_I=>"#\{prompt_start} %02n,%i>> ", :PROMPT_N=>"#\{prompt_start} %02n:%i> ", :PROMPT_S=>"#\{prompt_start} %02n:%i (%l)> ", :PROMPT_C=>"#\{prompt_start} %02n:%i>> ", :RETURN=>""} IRB.conf[:PROMPT_MODE] = :CODE_RUNNER IRB.conf[:SAVE_HISTORY] = 400 IRB.conf[:HISTORY_FILE] = "\#\{Dir.pwd}/.code-runner-irb-save-history" IRB.conf[:INSPECT_MODE] = false EOF end # File.open end # unless File.open(".int.tmp.rb", 'w')do |file| file.puts "#{copts.inspect}.each do |key, val| CodeRunner::DEFAULT_COMMAND_OPTIONS[key] = val end" file.puts CodeRunner::InteractiveMethods::INTERACTIVE_METHODS end # asdfa exec %[#{RbConfig::CONFIG['bindir']}/irb#{RbConfig::CONFIG['ruby_install_name'].sub(/ruby/, '')} -f -I '#{Dir.pwd}' -I '#{SCRIPT_FOLDER}' -I '#{ENV['HOME']}' -r '.code_runner_interactive_options' -r 'coderunner' -r .int.tmp ] end |
.launcher_directory ⇒ Object
321 322 323 |
# File 'lib/coderunner/class_methods.rb', line 321 def self.launcher_directory ENV['HOME'] + "/.coderunner/to_launch/#{ENV['CODE_RUNNER_LAUNCHER']}" end |
.load_file(files, copts = {}) ⇒ Object
eval(ruby_fragment)
394 395 396 397 398 399 400 401 402 403 404 405 406 |
# File 'lib/coderunner/class_methods.rb', line 394 def self.load_file(files, copts={}) (copts) # begin files.split(/\s*,\s*/).each do |file| # p file raise ArgumentError.new("#{file} is not a file.") unless File.file? file load file end # rescue # eval(files) # end end |
.make_film_multiple_runners(list, options) ⇒ Object
__END__
544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 |
# File 'lib/coderunner/graphs_and_films.rb', line 544 def self.make_film_multiple_runners(list, ) = [:frame_array, :fa, :skip_frames, :sf, :normalize, :n, :normalize_pieces, :np, :increment, :i, :skip_encoding, :se] fa = ([:frame_array] or [:fa] or list[0][0].run_list[list[0][0].filtered_ids[0]].frame_array() ) iname = [:in]||[:index_name]||:frame_index fd = frame_digits = Math.log10(fa[1]).ceil unless [:skip_frames] or [:sf] # `rm -rf film_frames` # extension = (options[:extension] or options[:ext] or '.png') # extension = '.' + extension unless extension =~ /^\./ # FileUtils.makedirs('film_frames') # puts @@multiple_processes; gets no_forks = (@@multiple_processes or 1) ep @@multiple_processes, no_forks end_graphkit = graphkit_multiple_runners(list, iname => fa[1]) begin_graphkit = graphkit_multiple_runners(list, iname => fa[0]) end_area = end_graphkit.plot_area_size begin_area = begin_graphkit.plot_area_size # p end_area, begin_area, options plot_size = {} axes = [:x, :y, :z] [:normalize] ||= [:nm] [:normalize_pieces] ||= [:nmp] for i in 0...end_area.size next unless [:normalize] and [:normalize].include? axes[i] min = [end_area[i][0], begin_area[i][0]].min max = [end_area[i][1], begin_area[i][1]].max key = axes[i] plot_size[key + :range] = [min, max] end ep plot_size # exit frames = [] actual_frames = {} i = fa[0] j = 0 while i <= fa[1] frames.push i actual_frames[i] = j i += ([:ic] or [:increment] or 1) j += 1 end # graphkit_frame_array = [] myrank = -1 # graphkit_frame_array = (frames.pieces(no_forks).parallel_map(n_procs: no_forks, with_rank: true) do |piece, myrank| graphkit_frame_array = (frames.pieces(no_forks).map do |piece| myrank +=1 # ep 'myrank is', myrank # unless myrank==0 # # $stdout = $stderr = StringIO.new # end # fork do if [:normalize_pieces] end_area = graphkit_multiple_runners(list, iname => piece.max).plot_area_size begin_area = graphkit_multiple_runners(list, iname => piece.min).plot_area_size axes = [:x, :y, :z] for i in 0...end_area.size next unless [:normalize_pieces].include? axes[i] min = [end_area[i][0], begin_area[i][0]].min max = [end_area[i][1], begin_area[i][1]].max key = axes[i] plot_size[key + :range] = [min, max] end end eputs 'making graphs...'; sleep 1 if myrank==0 graph_array = graphkit_multiple_runners_with_frame_array(piece, list, {:in => iname}, myrank==0) # ep graph_array eputs graph_array.each_with_index do |(frame_index,g), pindex| # if myrank == 0 # eputs "\033[2A" # Terminal jargon - go back one line # eputs sprintf("Plotting graphs: %2.2f", pindex.to_f/piece.size.to_f * 100.0) + "% Complete" # end # g = graph_kit_multiple_runners(list, plot_size + {frame_index: frame_index}) g.modify(plot_size) g.modify() g.instance_eval [:graphkit_modify] if [:graphkit_modify] # p g; exit g.title += sprintf(", frame %0#{fd}d", frame_index) unless [:frame_title] == false # folder = ("film_frames/"); # file_name = sprintf("frame_%0#{fd}d", actual_frames[frame_index]) # g.gnuplot; gets; g.close # ep folder + file_name + '.png'; gets # g.gnuplot_write(folder + file_name + extension) end graph_array # end end).sum # eputs "Waiting on subprocesses..." # Process.waitall end film_graphkit_frame_array(graphkit_frame_array, ) # unless options[:skip_encoding] # eputs "making film" # frame_rate = (options[:frame_rate] or options[:fr] || 15) # film_name = (options[:film_name] or options [:fn] or end_graphkit.file_name + '_film').gsub(/\s/, '_') # puts `ffmpeg -y #{options[:bitrate] ? "-b #{options[:bitrate]}" : ""} -r #{frame_rate} -threads #{(@multiple_processes or 1)} -i film_frames/frame_%0#{fd}d#{extension} -sameq #{film_name}.mp4` # end end |
.make_film_multiple_runners_old(list, options) ⇒ Object
447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 |
# File 'lib/coderunner/graphs_and_films.rb', line 447 def self.make_film_multiple_runners_old(list, ) = [:frame_array, :fa, :skip_frames, :sf, :normalize, :n, :normalize_pieces, :np, :increment, :i, :skip_encoding, :se, :index_name, :in] fa = ([:frame_array] or [:fa] or list[0][0].run_list[list[0][0].filtered_ids[0]].frame_array() ) iname = [:in]||[:index_name]||:frame_index fd = frame_digits = Math.log10(fa[1]).ceil unless [:skip_frames] or [:sf] `rm -rf film_frames` extension = ([:extension] or [:ext] or '.png') extension = '.' + extension unless extension =~ /^\./ FileUtils.makedirs('film_frames') # puts @@multiple_processes; gets no_forks = (@@multiple_processes or 1) ep @@multiple_processes, no_forks end_graphkit = graphkit_multiple_runners(list, iname => fa[1]) begin_graphkit = graphkit_multiple_runners(list, iname => fa[0]) end_area = end_graphkit.plot_area_size begin_area = begin_graphkit.plot_area_size # p end_area, begin_area, options plot_size = {} axes = [:x, :y, :z] [:normalize] ||= [:nm] [:normalize_pieces] ||= [:nmp] for i in 0...end_area.size next unless [:normalize] and [:normalize].include? axes[i] min = [end_area[i][0], begin_area[i][0]].min max = [end_area[i][1], begin_area[i][1]].max key = axes[i] plot_size[key + :range] = [min, max] end ep plot_size # exit frames = [] actual_frames = {} i = fa[0] j = 0 while i <= fa[1] frames.push i actual_frames[i] = j i += ([:ic] or [:increment] or 1) j += 1 end frames.pieces(no_forks).each_with_index do |piece, myrank| fork do if [:normalize_pieces] end_area = graphkit_multiple_runners(list, iname => piece.max).plot_area_size begin_area = graphkit_multiple_runners(list, iname => piece.min).plot_area_size axes = [:x, :y, :z] for i in 0...end_area.size next unless [:normalize_pieces].include? axes[i] min = [end_area[i][0], begin_area[i][0]].min max = [end_area[i][1], begin_area[i][1]].max key = axes[i] plot_size[key + :range] = [min, max] end end eputs 'making graphs...'; sleep 1 graph_array = graphkit_multiple_runners_with_frame_array(piece, list, {:in => iname}, myrank==0) # ep graph_array eputs graph_array.each_with_index do |(frame_index,g), pindex| if myrank == 0 eputs "\033[2A" # Terminal jargon - go back one line eputs sprintf("Plotting graphs: %2.2f", pindex.to_f/piece.size.to_f * 100.0) + "% Complete" end # g = graph_kit_multiple_runners(list, plot_size + {frame_index: frame_index}) g.modify(plot_size) g.modify() # p g; exit g.title += sprintf(", frame %0#{fd}d", frame_index) unless [:frame_title] == false folder = ("film_frames/"); file_name = sprintf("frame_%0#{fd}d", actual_frames[frame_index]) # g.gnuplot; gets; g.close # ep folder + file_name + '.png'; gets g.gnuplot_write(folder + file_name + extension) end end end eputs "Waiting on subprocesses..." Process.waitall end unless [:skip_encoding] eputs "making film" frame_rate = ([:frame_rate] or [:fr] || 15) film_name = ([:film_name] or [:fn] or end_graphkit.file_name + '_film').gsub(/\s/, '_') puts `ffmpeg -y #{[:bitrate] ? "-b #{[:bitrate]}" : ""} -r #{frame_rate} -threads #{(@multiple_processes or 1)} -i film_frames/frame_%0#{fd}d#{extension} -qscale 0 #{film_name}.mp4` end end |
.manual(copts = {}) ⇒ Object
790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 |
# File 'lib/coderunner/class_methods.rb', line 790 def self.manual(copts={}) help = <<EOF -------------CodeRunner Manual--------------- Written by Edmund Highcock (2009) NAME coderunner SYNOPSIS coderunner <command> [arguments] [options] DESCRIPTION CodeRunner is a framework for the running and analysis of large simulations. It is a Ruby package and can be used to write Ruby scripts. However it also has a powerful command line interface. The aim is to be able to submit simulations, analyse data and plot graphs all using simple commands. This manual is a quick reference. For a more tutorial style introduction to CodeRunner go to http://coderunner.sourceforge.net This help page documents the commandline interface. For API documentation see http://coderunner.sourceforge.net/api_documentation As is standard, <> indicates a parameter to be supplied, and [] indicates an option, unless otherwise stated. EXAMPLES $ coderunner sub -p '{height: 34.2, width: 221}' -n 24x4 -W 300 $ coderunner can 34 -U $ coderunner plot -G 'height:width;{};depth==2.4 and status == :Completed;height' $ coderunner st -ul $ coderunner rc 'p status' -U COMMANDS Either the long or the short form of the command may be used, except in interactive mode, where only short form can be used. Long(Short) <Arguments> (Meaningful Options) --------------------------------------------- #{(COMMANDS_WITH_HELP.sort_by{|arr| arr[0]}.map do |arr| sprintf(" %s %s(%s) \n\t%s", "#{arr[0]}(#{arr[1]})", arr[4].map{|arg| "<#{arg}>"}.join(' ').sub(/(.)$/, '\1 '), arr[5].map{|op| op.to_s}.join(','), arr[3], ) end).join("\n\n")} OPTIONS #{((COMMAND_LINE_FLAGS_WITH_HELP + LONG_COMMAND_LINE_OPTIONS).map do |arr| sprintf("%-15s %-2s\n\t%s", arr[0], arr[1], arr[3]) end).join("\n\n") } EOF #help.gsub(/(.{63,73} |.{73})/){"#$1\n\t"}.paginate help.paginate end |
.netcdf_plot(netcdf_file, vars, indices, copts = {}) ⇒ Object
end
137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 |
# File 'lib/coderunner/class_methods.rb', line 137 def self.netcdf_plot(netcdf_file, vars, indices, copts={}) (copts) begin require "numru/netcdf" rescue LoadError eputs "Error: No Ruby NetCDF library (was it installed correctly?): data analysis for netcdf files not possible." return end start_indices = indices.split(',').map{|idx| idx = idx.split(':')[0] if idx =~ /:/ ; eval(idx) || 0} end_indices = indices.split(',').map{|idx| idx = idx.split(':')[1] if idx =~ /:/ ; eval(idx) || -1} ep 'start_indices', start_indices, 'end_indices', end_indices file = NumRu::NetCDF.open(netcdf_file) to_plot = vars.split(',').map do |var| ep 'var', var [file.var(var).get('start'=> start_indices, 'end'=> end_indices).to_a.flatten] end ep 'to_plot', to_plot kit = GraphKit.quick_create(*to_plot) ep 'copts', copts kit.instance_eval(copts[:w]) if copts[:w] kit.gnuplot STDIN.gets kit.close end |
.old_get_run_class_name(code, modlet) ⇒ Object
:nodoc:
341 342 343 |
# File 'lib/coderunner/instance_methods.rb', line 341 def self.old_get_run_class_name(code, modlet) # :nodoc: return modlet ? "#{code.capitalize}#{modlet.capitalize.sub(/\.rb$/, '').sub(/_(\w)/){"#$1".capitalize}}Run" : "#{code.capitalize}Run" end |
.parameter_scan(parameter_scan_array_file, copts = {}) ⇒ Object
408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 |
# File 'lib/coderunner/class_methods.rb', line 408 def self.parameter_scan(parameter_scan_array_file, copts={}) parameter_scan_array = eval(File.read(parameter_scan_array_file)) # process_copts(copts) runner = fetch_runner(copts) skip = true unless copts[:k] == false folder = Dir.pwd Log.logf("self.parameter_scan") # @@runners = {} @@mutex = Mutex.new # @runner = new(folder, code: copts[:C], modlet: copts[:m], version: copts[:v], executable: copts[:X]) @@psppipe = PPipe.new(parameter_scan_array.size + 2, true, controller_refresh: 0.5, redirect: false) parameter_scan_array.each do |parameter_scan| @@psppipe.fork do runner.parameter_scan(parameter_scan, copts[:p][0], skip: skip, nprocs: copts[:n]) end end @@psppipe.finish @@psppipe = nil end |
.plot_graph(copts = {}) ⇒ Object
427 428 429 430 431 432 433 434 435 436 437 |
# File 'lib/coderunner/class_methods.rb', line 427 def self.plot_graph(copts = {}) # process_copts(copts) runner = fetch_runner(copts) string_to_eval = copts[:w] #options = (options and options =~ /\S/) ? eval(options): {} eputs 'Starting Graph' kit = runner.graphkit_from_lists(copts[:G], copts[:g]) kit.gnuplot(eval: string_to_eval) gets kit.close end |
.print_queue_status(copts = {}) ⇒ Object
165 166 167 168 169 170 171 172 173 174 |
# File 'lib/coderunner/class_methods.rb', line 165 def self.print_queue_status(copts={}) begin eputs queue_status rescue => err eputs "General queue status doesn't work on this system; showing queue status for this folder" # p err runner = fetch_runner(copts) eputs runner.queue_status end end |
.process_command_line_option(opt, arg, copts) ⇒ Object
Converts a command line flag opt with value arg to a command option which is stored in copts
274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 |
# File 'lib/coderunner.rb', line 274 def self.process_command_line_option(opt, arg, copts) case opt when "--change-directory" copts[:c] = arg.to_i when "--delete" copts[:x] = arg.to_i when "--graph" copts[:G].push arg when "--run-graph" copts[:g].push arg # when "--cancel" # copts[:K] = arg.to_i when "--multiple-processes" copts[:M] = arg.to_i when "--film" copts[:F] = (arg or true) when "--recheck" copts[:R] = arg.to_i when "--use-large-cache-but-recheck-incomplete" copts[:U] = true copts[:u]=true when "--wall-mins" copts[:W] = arg.to_i when "--use-phantom" copts[:h] = (arg and arg =~ /\S/) ? arg.to_sym : :phantom when "--terminal-size" array = eval arg ENV['ROWS'], ENV['COLS'] = array[0].to_s, array[1].to_s when "--interactive-mode" @@interactive_mode = true when "--parameters" copts[:p].push arg else if CLF_BOOLS.include? CLF_TO_SHORT_COPTS[opt] copts[CLF_TO_SHORT_COPTS[opt]] = true elsif CLF_INVERSE_BOOLS.include? CLF_TO_SHORT_COPTS[opt] copts[CLF_TO_SHORT_COPTS[opt]] = false elsif CLF_TO_SHORT_COPTS[opt] # Applies to most options copts[CLF_TO_SHORT_COPTS[opt]] = arg elsif CLF_BOOLS.include? CLF_TO_LONG[opt] copts[CLF_TO_LONG[opt]] = true elsif CLF_INVERSE_BOOLS.include? CLF_TO_LONG[opt] copts[CLF_TO_LONG[opt]] = false elsif CODE_COMMAND_OPTIONS.map{|o| o[0]}.include? opt begin #copts[:code_copts] ||= {} copts[ #CLF_TO_LONG[opt].to_s.sub('_options','').to_sym CLF_TO_LONG[opt] ] = eval(arg) rescue SyntaxError => err eputs "\nOption #{opt} must be a hash\n\n" raise err end elsif CLF_TO_LONG[opt] copts[CLF_TO_LONG[opt]] = arg else raise "Unknown command line argument: #{opt}" end end copts end |
.process_command_options(copts) ⇒ Object
669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 |
# File 'lib/coderunner/class_methods.rb', line 669 def self.(copts) if copts[:true] copts[:true].to_s.split(//).each do |letter| copts[letter.to_sym] = true end end if copts[:false] copts[:false].to_s.split(//).each do |letter| copts[letter.to_sym] = false end end (copts) copts.each do |key, value| copts[LONG_TO_SHORT[key]] = value if LONG_TO_SHORT[key] end if copts[:j] # j can be a number '65' or list of numbers '65,43,382' copts[:f]= "#{eval("[#{copts[:j]}]").inspect}.include? id" end if copts[:X] and FileTest.exist? copts[:X] copts[:X] = File.(copts[:X]) end if copts[:z] Log.log_file = Dir.pwd + '/.cr_logfile.txt' Log.clean_up else Log.log_file = nil end copts[:F] = (copts[:F].class == Hash ? copts[:F] : (copts[:F].class == String and copts[:F] =~ /\A\{.*\}\Z/) ? eval(copts[:F]) : {}) copts[:G]= [copts[:G]] if copts[:G].kind_of? String copts[:g]= [copts[:g]] if copts[:g].kind_of? String # if copts[:p] and copts[:p].class == String # should be a hash or an inspected hash # copts[:p] = eval(copts[:p]) # end copts[:p] = [copts[:p]].compact unless copts[:p].class == Array #for i in 0...copts[:p].size copts[:Y] ||= DEFAULT_COMMAND_OPTIONS[:Y] if DEFAULT_COMMAND_OPTIONS[:Y] if copts[:Y] and copts[:Y] =~ /:/ set_class_defaults(copts) copts[:running_remotely] = true else copts[:Y] = copts[:Y].gsub(/~/, ENV['HOME']) if copts[:Y] Dir.chdir((copts[:Y] or Dir.pwd)) do set_runner_defaults(copts) # ep DEFAULT_RUNNER_OPTIONS end end # ep Log.log_file #copts[:code_copts].each{|k,v| CODE_OPTIONS[k] = v} if copts[:code_copts] copts.keys.map{|k| k.to_s}.grep(/_options$/).map{|k| k.to_sym}.each do |k| CODE_OPTIONS[k.to_s.sub('_options','').to_sym] = copts[k] end end |
.read_default_command_options(copts) ⇒ Object
664 665 666 667 668 |
# File 'lib/coderunner/class_methods.rb', line 664 def self.(copts) DEFAULT_COMMAND_OPTIONS.each do |key, value| copts[key] ||= value end end |
.readout(copts = {}) ⇒ Object
438 439 440 441 442 |
# File 'lib/coderunner/class_methods.rb', line 438 def self.readout(copts={}) # process_copts(copts) runner = fetch_runner(copts) puts runner.readout end |
.recheck(id, copts = {}) ⇒ Object
443 444 445 446 447 448 |
# File 'lib/coderunner/class_methods.rb', line 443 def self.recheck(id, copts={}) # process_copts(copts) runner = fetch_runner(copts) runner.run_list[copts[:R]].recheck runner.respond_to_requests end |
.reference(class_or_method, copts = {}) ⇒ Object
180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 |
# File 'lib/coderunner/class_methods.rb', line 180 def self.reference(class_or_method, copts={}) code_folders = Dir.recursive_entries(SCRIPT_FOLDER + '/code_modules').grep(/\/ri$/).map{|fold| ['-d', fold]}.flatten # ep code_folders # require 'rdoc/ri/driver' # " # op = @ri_count ? [] : (@ri_count = true; ['--no-use-cache']) # trap(1){puts 'No help available'} # at_exit{raise ""} # p op begin eputs "Looking up #{class_or_method}" RDoc::RI::Driver.run ['-d', SCRIPT_FOLDER + '/ri', class_or_method.to_s] + code_folders rescue => err eputs "Unknown class or method or no help available: #{err}" end # trap(1){} end |
.repair_marshal_run_class_not_found_error(err) ⇒ Object
352 353 354 355 356 357 358 359 |
# File 'lib/coderunner/instance_methods.rb', line 352 def self.repair_marshal_run_class_not_found_error(err) #ep 'error', err, err.message code, modlet = err..scan(/CodeRunner\:\:([A-Z][a-z0-9_]+)(?:::([A-Z]\w+))?/)[0] #ep 'merror', err, code, modlet; gets code.gsub!(/([a-z0-9])([A-Z])/, '\1_\2') (modlet.gsub!(/([a-z0-9])([A-Z])/, '\1_\2'); modlet.downcase!) if modlet setup_run_class(code.downcase, modlet: modlet) end |
.resubmit(copts = {}) ⇒ Object
puts “Got here” exit(0)
554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 |
# File 'lib/coderunner/class_methods.rb', line 554 def self.resubmit(copts = {}) # process_copts(copts) runner = fetch_runner(copts) # raise "something is already submitting" if FileTest.exist? "submitting" runs = [] raise "Parameters must be an array of inspected hashes" unless copts[:p].kind_of? Array Dir.chdir(copts[:Y]) do runs = runner.filtered_ids.map do |id| eputs id run = runner.run_list[id].dup if copts[:smart_resubmit_name] eputs "Smart name" run.set(:naming_pars, [:resubmit_id]) run.resubmit_id = run.id end run.update_submission_parameters(copts[:p][0], false) run.run_name = nil unless copts[:rerun] run end end #throw(:here) runner.submit(runs, nprocs: copts[:n], version: copts[:v], skip: copts[:k], job_chain: copts[:J], no_update_before_submit: copts[:no_update_before_submit], replace_existing: copts[:replace_existing], smart_resubmit_name: copts[:smart_resubmit_name], rerun: copts[:rerun]) end |
.run_class(copts = {}) ⇒ Object
449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 |
# File 'lib/coderunner/class_methods.rb', line 449 def self.run_class(copts={}) (copts) copts[:no_update] = true unless copts[:C] if FileTest.exist? file=Dir.pwd + '/.code_runner_script_defaults.rb' copts[:C] = eval(File.read(file))[:code] copts[:m] = eval(File.read(file))[:modlet] elsif self.runner copts[:C] = self.runner.code copts[:m] = self.runner.modlet end end #ep ['code', 'modlet is', copts[:C], copts[:m]] return setup_run_class(copts[:C], modlet: copts[:m]) end |
.run_command(string, copts = {}) ⇒ Object
runner = fetch_runner(copts) runner.run_class.class_eval(string)
471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 |
# File 'lib/coderunner/class_methods.rb', line 471 def self.run_command(string, copts={}) # process_copts(copts) runner = fetch_runner(copts) eputs "Calling run_commmand..." # puts "Warning: Use large cache is on (-U or -u) -- no results will be saved" if runner.use_large_cache ppipe = PPipe.new(runner.filtered_ids.size + 1, false) if copts[:M] no_save = (runner.class == RemoteCodeRunner or copts[:y] =~ /no-save/) # runner.generate_combined_ids # ep runner.filtered_ids runner.filtered_ids.each do |id| run = runner.combined_run_list[id] if no_save or run.is_phantom if copts[:M] fork{run.instance_eval(string)} else run.instance_eval(string) end else if copts[:M] pn = ppipe.fork do Dir.chdir(run.directory) do run.instance_eval(string); run.save run.write_results end ppipe.i_send(id, Marshal.dump(run), tp: 0) end else Dir.chdir(run.directory){run.instance_eval(string); run.save; run.write_results} end end end unless no_save (runner.filtered_ids.each{|id| runner.run_list[id] = Marshal.load(ppipe.w_recv(id).contents)};ppipe.finish) if copts[:M] runner.save_large_cache end # Process.waitall runner.respond_to_requests end |
.run_script ⇒ Object
371 372 373 374 375 376 377 378 379 |
# File 'lib/coderunner.rb', line 371 def self.run_script if DEFAULT_COMMAND_OPTIONS[:Z] DEFAULT_COMMAND_OPTIONS.absorb(eval(DEFAULT_COMMAND_OPTIONS[:Z])) puts "Begin Output" end command = COMMANDS.find{|com| com.slice(0..1).include? ARGV[0]} raise "\n-------------------\nCommand #{ARGV[0].inspect} not found: try 'coderunner man' for help\n-------------------\n" unless command send(command[0].to_sym, *ARGV.values_at(*(1...(1+command[2])).to_a)) end |
.runner ⇒ Object
784 785 786 787 |
# File 'lib/coderunner/class_methods.rb', line 784 def self.runner @runners ||={} @runners.values[0] end |
.runner_eval(string, copts = {}) ⇒ Object
514 515 516 517 518 519 520 521 522 523 524 525 526 |
# File 'lib/coderunner/class_methods.rb', line 514 def self.runner_eval(string, copts = {}) # process_copts(copts) runner = fetch_runner(copts) return_val = runner.instance_eval(string) if copts[:Z] Kernel.puts(server_dump(return_val)) else return return_val end end |
.scan(scan_string, copts = {}) ⇒ Object
527 528 529 530 531 |
# File 'lib/coderunner/class_methods.rb', line 527 def self.scan(scan_string, copts={}) # process_copts(copts) runner = fetch_runner(copts) runner.simple_scan(scan_string, nprocs: copts[:n], version: copts[:v], skip: copts[:k], parameters: copts[:p][0]) end |
.server_dump(obj) ⇒ Object
312 313 314 |
# File 'lib/coderunner/instance_methods.rb', line 312 def self.server_dump(obj) "code_runner_server_dump_start_E#{Marshal.dump(obj)}code_runner_server_dump_end_E" end |
.set_class_defaults(copts = {}) ⇒ Object
13 14 15 16 17 18 |
# File 'lib/coderunner/class_methods.rb', line 13 def self.set_class_defaults(copts={}) (CLASS_OPTIONS.keys - []).each do |var| CLASS_OPTIONS[var] = copts[LONG_TO_SHORT[var]] set(var, CLASS_OPTIONS[var]) end end |
.set_default_command_options_from_command_line ⇒ Object
342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 |
# File 'lib/coderunner.rb', line 342 def self. #some defaults # DEFAULT_COMMAND_OPTIONS[:p] ||= {} DEFAULT_COMMAND_OPTIONS[:v] ||= "" #DEFAULT_COMMAND_OPTIONS[:n] ||= "1" DEFAULT_COMMAND_OPTIONS[:G] = [] DEFAULT_COMMAND_OPTIONS[:g] = [] DEFAULT_COMMAND_OPTIONS[:k] = true DEFAULT_COMMAND_OPTIONS[:h] ||= :real DEFAULT_COMMAND_OPTIONS[:p] = [] # ep COMMAND_LINE_FLAGS opts = GetoptLong.new(*(COMMAND_LINE_FLAGS + LONG_COMMAND_LINE_FLAGS)) opts.each do |opt, arg| process_command_line_option(opt, arg, DEFAULT_COMMAND_OPTIONS) end #raise "\n\nCannot use large cache ('-U' or '-u' ) if submitting runs" if DEFAULT_COMMAND_OPTIONS[:U] and (DEFAULT_COMMAND_OPTIONS[:s] or DEFAULT_COMMAND_OPTIONS[:P]) if DEFAULT_COMMAND_OPTIONS[:z] Log.log_file = Dir.pwd + '/.cr_logfile.txt' Log.clean_up else Log.log_file = nil # puts Log.log_file end end |
.set_runner_defaults(copts = {}) ⇒ Object
In the next section are the implementations of all the standard Code Runner commands and some helper functions.
5 6 7 8 9 10 11 |
# File 'lib/coderunner/class_methods.rb', line 5 def self.set_runner_defaults(copts = {}) (DEFAULT_RUNNER_OPTIONS.keys - [:sys, :script_folder]).each do |var| DEFAULT_RUNNER_OPTIONS[var] = copts[LONG_TO_SHORT[var]] end set_class_defaults(copts) # ep DEFAULT_RUNNER_OPTIONS end |
.setup_run_class(code, options = {}) ⇒ Object
368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 |
# File 'lib/coderunner/instance_methods.rb', line 368 def self.setup_run_class(code, ={}) # logf(:setup_code) # log(:code, code) modlet = [:modlet] version = [:version] # log('modlet in setup_run_class', modlet) eputs "Loading modules for #{code}, #{modlet.inspect}..." # modlet = modlet.sub(/\.rb$/, '') if modlet raise CRFatal.new("Code must contain only lowercase letters, digits, and underscore, and must begin with a letter or underscore; it is '#{code}'") unless code =~ /\A[a-z_][a-z_\d]*\Z/ raise CRFatal.new("Input_file_module must contain only lowercase letters, digits, and underscore, and must begin with a letter or underscore; it is '#{modlet}'") if modlet and not modlet =~ /\A[a-z_][a-z_\d]*\Z/ run_class_name = get_run_class_name(code, modlet) # p run_class_name return recursive_const_get(run_class_name) if SETUP_RUN_CLASSES.include?(run_class_name) #constants.include? (run_class_name).to_sym unless options[:force] #and const_get(run_class_name).rcp.code? #return const_get(run_class_name) if constants.include? (run_class_name).to_sym unless options[:force] #and const_get(run_class_name).rcp.code? SETUP_RUN_CLASSES.push run_class_name #.downcase FileUtils.makedirs(ENV['HOME'] + "/.coderunner/#{code}crmod/") FileUtils.makedirs(ENV['HOME'] + "/.coderunner/#{code}crmod/defaults_files") #Create the run_class, a special dynamically created class which knows how to process runs of the given code on the current system. #run.rb contains the basic methods of the class # puts run_class_name; gets # run_class = add_a_child_class(run_class_name, "") # run_class.class_eval(%[ # @@code=#{code.inspect}; @@version=#{version.inspect} # @@modlet=#{modlet.inspect} #modlet should be nil if not @@modlet_required # SYS = #{SYS.inspect} # include Log # Log.logi(:code_just_after_runfile, @@code) # ] # ) code_module_name = SCRIPT_FOLDER+ "/code_modules/#{code}/#{code}.rb" code_module_name = "#{code}crmod" require code_module_name # ep get_run_class_name(code, nil) run_class = const_get(get_run_class_name(code, nil)) run_class.instance_variable_set(:@code, code) run_class.instance_variable_set(:@sys, SYS) raise "#{run_class} must inherit from CodeRunner::Run: its ancestors are: #{run_class.ancestors}" unless run_class.ancestors.include? Run if [:runner] run_class.runner = [:runner] run_class.instance_variable_set(:@runner, [:runner]) end #Add methods appropriate to the current system system_module_name = SCRIPT_FOLDER+ "/system_modules/#{SYS}.rb" require system_module_name run_class.send(:include, const_get(SYS.variable_to_class_name)) # run_class.class_eval(File.read(system_module_name), system_module_name) #Some codes require an modlet; the flag modlet_required is specified in the code module if run_class.rcp.modlet_required raise CRFatal.new("Modlet necessary and none given") unless modlet end #If a modlet is specified (modlets can also be optional) if modlet # Log.logi(:modlet, modlet) #if ( modlet_location = "#{SCRIPT_FOLDER}/code_modules/#{code}/default_modlets"; #file_name = "#{modlet_location}/#{modlet}.rb"; #FileTest.exist? file_name #) #elsif ( modlet_location = "#{SCRIPT_FOLDER}/code_modules/#{code}/my_modlets"; #file_name = "#{modlet_location}/#{modlet}.rb"; #FileTest.exist? file_name #) #else #raise CRFatal.new("Could not find modlet file: #{modlet}.rb") #end #require file_name # ep run_class.constants # ep run_class_names modlet_file = "#{code}crmod/#{modlet}.rb" #ep ['requiring modlet file'] require modlet_file run_class = recursive_const_get(run_class_name) run_class.instance_variable_set(:@modlet, modlet) end run_class.check_and_update # log("random element of variables", run_class.variables.random) # logi("run_class.variables[0] in setup_code", run_class.variables[0]) # ep 'finished' return run_class end |
.show_values_of(expression, copts = {}) ⇒ Object
609 610 611 612 |
# File 'lib/coderunner/class_methods.rb', line 609 def self.show_values_of(expression, copts={}) runner = fetch_runner(copts) p runner.filtered_ids.map{|id| runner.run_list[id].instance_eval(expression)}.uniq.sort end |
.start_launcher(refresh, max_queue, copts = {}) ⇒ Object
324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 |
# File 'lib/coderunner/class_methods.rb', line 324 def self.start_launcher(refresh, max_queue, copts={}) #eputs "Starting launcher!" raise "Raise refresh is #{refresh}: it must be >= 0.1" if refresh.to_f < 0.1 raise "Raise max_queue is #{max_queue}: it must be >= 5" if max_queue.to_i < 5 #raise "Launcher already running" if %x[ps -e -o cmd].split("\n").grep(/coderunner\s+launch/).size > 0 require 'thread' tl = launcher_directory #SCRIPT_FOLDER + '/to_launch' #exit unless Feedback.get_boolean( "Launch directory #{tl} already exists: it is suggested that you change the prefix by changing the environment variable CODE_RUNNER_LAUNCHER. Do you wish to continue (don't select yes unless you know what you are doing)?") if FileTest.exist? tl raise "Launch directory #{tl} already exists: it is suggested that you change the prefix by changing the environment variable CODE_RUNNER_LAUNCHER. Do you wish to continue (don't select yes unless you know what you are doing)?" if FileTest.exist? tl # FileUtils.rm_r tl if FileTest.exist? tl eputs "Starting launcher\n" at_exit{FileUtils.rm_r tl} FileUtils.makedirs tl Thread.new{loop{`cp #{tl}/queue_status.txt #{tl}/queue_status2.txt; ps > #{tl}/queue_status.txt`; sleep 1}} mutex = Mutex.new processes= [] Thread.new do loop do Dir.entries(tl).each do |file| next unless file =~ (/(^.*)\.stop/) pid = $1 mutex.synchronize{Process.kill(pid); processes.delete pid} end sleep refresh.to_i end end #Dir.chdir(tl) do ppid = $$ loop do sleep refresh.to_i while processes.size >= max_queue.to_i # processes = [] Dir.entries(tl).grep(/(^.*)\.start/).each do |file| file =~ (/(^.*)\.start/) id = $1 command = "" command = File.read tl + '/' + file while command == "" pid = fork do processes.each do |wpid| # Make sure all previously submitted jobs have finished. sleep refresh.to_i while %x[ps -e -o pid,ppid].split("\n").grep(Regexp.new("^\\s*#{wpid}\\s+#{ppid}")).size > 0 end #p ["command", command] exec(command) end `cp #{tl}/queue_status.txt #{tl}/queue_status2.txt; ps > #{tl}/queue_status.txt` mutex.synchronize{processes.push pid} File.open(tl + '/' + id + '.pid', 'w'){|file| file.puts pid} FileUtils.rm(tl + '/' + file) Thread.new{Process.wait pid; mutex.synchronize{processes.delete pid}} end # processes.each{|pid| Process.wait pid} sleep refresh.to_i end #end end |
.status(copts = {}) ⇒ Object
617 618 619 620 621 |
# File 'lib/coderunner/class_methods.rb', line 617 def self.status(copts={}) # process_copts(copts) runner = fetch_runner(copts) runner.print_out(0, with_comments: copts[:with_comments]) unless copts[:interactive_start] or copts[:Z] or copts[:no_print_out] end |
.status_loop(copts = {}) ⇒ Object
622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 |
# File 'lib/coderunner/class_methods.rb', line 622 def self.status_loop(copts={}) # process_copts(copts) runner = fetch_runner(copts) runner.print_out(0, with_comments: copts[:with_comments]) unless copts[:interactive_start] or copts[:Z] or copts[:no_print_out] break_out = false loop do old_trap = trap(2){eputs " Terminating loop, please wait..."; break_out = true} runner.use_large_cache = true runner.update(false) (trap(2, old_trap); break) if break_out runner.recheck_filtered_runs(false) runner.print_out(nil, with_comments: copts[:with_comments]) trap(2, old_trap) break if break_out break if not runner.run_list.values.find do |r| not [:Complete, :Failed].include? r.status end #ep "sleep" sleep 3 #ep "end sleep" end end |
.status_with_comments(copts = {}) ⇒ Object
613 614 615 616 |
# File 'lib/coderunner/class_methods.rb', line 613 def self.status_with_comments(copts={}) copts[:with_comments] = true status(copts) end |
.submit(copts = {}) ⇒ Object
532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 |
# File 'lib/coderunner/class_methods.rb', line 532 def self.submit(copts = {}) # process_copts(copts) runner = fetch_runner(copts) # raise "something is already submitting" if FileTest.exist? "submitting" runs = [] raise "Parameters must be an array of inspected hashes" unless copts[:p].kind_of? Array Dir.chdir(copts[:Y]) do copts[:p].push nil if copts[:p] == [] # ep copts[:p]; exit copts[:p].each do |pars| run = runner.run_class.new(runner) # p pars run.update_submission_parameters(pars) runs.push run end # exit end runner.submit(runs, nprocs: copts[:n], version: copts[:v], skip: copts[:k], job_chain: copts[:J], no_update_before_submit: copts[:no_update_before_submit]) #puts "Got here" #exit(0) end |
.submit_command(jid, comm, copts = {}) ⇒ Object
This method allows the straightforward submission of a single command using the batch queue on any system.
580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 |
# File 'lib/coderunner/class_methods.rb', line 580 def self.submit_command(jid, comm, copts={}) (copts) submitter = Object.new submitter.instance_variable_set(:@command, comm) submitter.instance_variable_set(:@jid, jid) submitter.instance_variable_set(:@nprocs, copts[:n]) submitter.instance_variable_set(:@wall_mins, copts[:W]) submitter.instance_variable_set(:@project, copts[:P]) class << submitter include CodeRunner::SYSTEM_MODULE def executable_name 'custom' end def job_identifier @jid end def run_command @command end end submitter.execute end |
.update_runners ⇒ Object
@r.read_defaults
778 779 780 781 |
# File 'lib/coderunner/class_methods.rb', line 778 def self.update_runners @runners ||= {} @runners.each{|runner| runner.update} end |
.write_graph(name, copts = {}) ⇒ Object
644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 |
# File 'lib/coderunner/class_methods.rb', line 644 def self.write_graph(name, copts={}) # process_copts(copts) runner = fetch_runner(copts) eputs 'Starting Graph' kit = runner.graphkit_from_lists(copts[:G], copts[:g]) #options = copts[:w] #options = (options and options =~ /\S/) ? eval(options): {} name = nil unless name =~ /\S/ max = 0 name.sub!(/^\~/, ENV['HOME']) if name if name and name =~ /%d\./ regex = Regexp.new(Regexp.escape(File.basename(name)).sub(/%d/, '(?<number>\d+)')) Dir.entries(File.dirname(name)).join("\n").scan(regex) do max = [max, $~[:number].to_i].max end name = name.sub(/%d/, (max + 1).to_s) end raise "kit doesn't have a file_name and no filename specified; can't write graph" unless name or (kit.file_name.class == String and kit.file_name =~ /\S/) Dir.chdir(COMMAND_FOLDER){kit.gnuplot_write((name or kit.file_name), {eval: copts[:w]})} end |
Instance Method Details
#add_phantom_run(run) ⇒ Object
def rename_variable(old, new) puts “Please confirm complete renaming of #old to #new” gets @run_list.each do |directory| Dir.chdir directory do begin @readme = File.read(“CODE_RUNNER_INPUTS”) rescue Errno::ENOENT => err # puts err, err.class @readme = File.read(“README”) end @readme.sub!(Regexp.new(“^s+#old:”), “^s+#new:”) File.open(“CODE_RUNNER_INPUTS_TEST”, ‘w’){|file| file.puts @readme} end end old_recalc, @recalc_all = @recalc_all, true update @recalc_all = old_recalc end
1603 1604 1605 1606 1607 1608 1609 |
# File 'lib/coderunner/instance_methods.rb', line 1603 def add_phantom_run(run) @phantom_run_list[@phantom_id] = run @phantom_ids.push @phantom_id #run.real_id = run.id run.id = @phantom_id @phantom_id += -1 end |
#alter_ids(num, options = {}) ⇒ Object
Permanently change the id of every run in the folder by adding num to them. Num can be negative unless it makes any of the ids negative. Use if you want to combine these runs with the runs in another folder, either by creating an instance of CodeRunner::Merge, or by directly copying and pasting the run folders.
1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 |
# File 'lib/coderunner/instance_methods.rb', line 1662 def alter_ids(num, ={}) Dir.chdir(@root_folder) do return unless [:no_confirm] or Feedback.get_boolean("This will permanently alter all the ids in the folder #@root_folder. Scripts that use those ids may be affected. Do you wish to continue?") raise ArgumentError.new("Cannot have negative ids") if @run_list.keys.min + num < 0 runs = @run_list.values fids = filtered_ids @run_list = {} runs.each do |run| old_id = run.id if fids.include? old_id run.id += num old_dir = run.relative_directory new_dir = old_dir.sub(Regexp.new("id_#{old_id}(\\D|$)")){"id_#{run.id}#$1"} # ep old_dir, new_dir FileUtils.mv(old_dir, new_dir) run.relative_directory = new_dir run.directory = File.(new_dir) end @run_list[run.id] = run end @ids = @run_list.keys set_max_id(@ids.max || 0) save_all_and_overwrite_info end end |
#axiskit(string) ⇒ Object
def gnuplot(axes, options) a = graphkit(axes, options).gnuplot end
315 316 317 318 319 320 321 322 323 324 325 326 327 |
# File 'lib/coderunner/graphs_and_films.rb', line 315 def axiskit(string) generate_combined_ids data = filtered_ids.inject([]) do |arr, id| begin arr.push @combined_run_list[id].instance_eval(string) rescue => err eputs "Evaluation of #{string} failed for run #{id}" raise err end end # p data return GraphKit::AxisKit.new(title: string, data: data) end |
#cancel_job(id, options = {}) ⇒ Object
Cancel the job with the given ID. Options are: :no_confirm —> true or false, cancel without asking for confirmation if true :delete —> if (no_confirm and delete), delete cancelled run
1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 |
# File 'lib/coderunner/instance_methods.rb', line 1562 def cancel_job(id, ={}) @run=@run_list[id] raise "Run with id #{id} does not exist" unless @run unless [:no_confirm] eputs "Cancelling job: #{@run.job_no}: #{@run.run_name}. \n Press enter to confirm" # puts 'asfda' gets end @run.cancel_job if [:no_confirm] delete = [:delete] else delete = Feedback.get_boolean("Do you want to delete the folder (#{@run.directory}) as well?") end FileUtils.rm_r(@run.directory) if delete and @run.directory and not @run.directory == "" update print_out end |
#code_run_environment ⇒ Object
1219 1220 1221 |
# File 'lib/coderunner/instance_methods.rb', line 1219 def code_run_environment run_class.new(self).code_run_environment end |
#continue_in_new_folder(folder, options = {}) ⇒ Object
1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 |
# File 'lib/coderunner/instance_methods.rb', line 1688 def continue_in_new_folder(folder, ={}) Dir.chdir(@root_folder) do raise "Folder already exists" if FileTest.exist?(folder) FileUtils.makedirs("#{folder}/v") #FileUtils.makedirs(folder) FileUtils.cp(".code_runner_script_defaults.rb", "#{folder}/.code_runner_script_defaults.rb") FileUtils.cp(".code-runner-irb-save-history", "#{folder}/.code-runner-irb-save-history") FileUtils.cp("#{@defaults_file}_defaults.rb", "#{folder}/#{@defaults_file}_defaults.rb") if [:copy_ids] [:copy_ids].each do |id| FileUtils.cp_r(@run_list[id].directory, "#{folder}/v/id_#{id}") end end end end |
#create_archive(options = {}) ⇒ Object
Create a tar archive of the root folder and all the files in it. Options are :compression => true or false :folder => folder in which to place the archive. :verbose => true or false :group => group of new files
1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 |
# File 'lib/coderunner/instance_methods.rb', line 1711 def create_archive(={}) verbose = [:verbose] ? 'v' : '' very_verbose = [:very_verbose] ? 'v' : '' comp = [:compression] Dir.chdir(@root_folder) do temp_root = ".tmparch/#{File.basename(@root_folder)}" FileUtils.makedirs(temp_root) system "chgrp #{[:group]} #{temp_root}" if [:group] size=@run_list.size @run_list.values.each_with_index do |run, index| archive_name = "#{File.basename(run.directory)}.tar#{comp ? '.gz' : ''}" tar_name = archive_name.delsubstr('.gz') relative = run.directory.delete_substrings(@root_folder, File.basename(run.directory)) FileUtils.makedirs(temp_root + relative) unless FileTest.exist? temp_root + relative + archive_name eputs "Archiving #{index} out of #{size}" if [:verbose] Dir.chdir(run.directory + '/..') do command = "tar -cW#{very_verbose}f #{tar_name} #{File.basename(run.directory)}" eputs command if [:verbose] unless system command raise "Archiving failed" end break unless comp command = "gzip -4 -vf #{tar_name}" eputs command if [:verbose] unless system command raise "Compression failed" end command = "gzip -t #{archive_name}" eputs command if [:verbose] unless system command raise "Compression failed" end #exit end FileUtils.mv(relative.delsubstr('/') + archive_name, temp_root + '/' + relative + archive_name) end system "chgrp -R #{[:group]} #{temp_root}" if [:group] end Dir.entries.each do |file| case file when '.', '..', '.tmparch' next when /^v/ next unless File.file? file else FileUtils.cp_r(file, "#{temp_root}/#{file}") end end Dir.chdir('.tmparch') do command = "tar -cWv --remove-files -f #{File.basename(@root_folder)}.tar #{File.basename(@root_folder)}" command = "tar -cWv -f #{File.basename(@root_folder)}.tar #{File.basename(@root_folder)}" eputs command if [:verbose] raise "Archiving Failed" unless system command end FileUtils.mv(".tmparch/#{File.basename(@root_folder)}.tar", "#{File.basename(@root_folder)}.tar") #FileUtils.rm_r(".tmparch") end end |
#destroy(options = {}) ⇒ Object
Delete the folders of all runs for whom CodeRunner#filter(run) is true. This will permanently erase the runs. This is an interactive method which asks for confirmation.
1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 |
# File 'lib/coderunner/instance_methods.rb', line 1539 def destroy(={}) ids = @ids.find_all{|id| filter @run_list[id]} unless [:no_confirm] logf(:destroy) puts "About to delete:" ids.each{|id| eputs @run_list[id].run_name} return unless Feedback.get_boolean("You are about to DESTROY #{ids.size} jobs. There is no way back. All data will be eliminated. Please confirm the delete.") #gets eputs "Please confirm again. Press Enter to confirm, Ctrl + C to cancel" gets end ids.each{|id| FileUtils.rm_r @run_list[id].directory if @run_list[id].directory and not ["", ".", ".."].include? @run_list[id].directory @run_list.delete(id); @ids.delete(id); generate_combined_ids} set_max_id(@ids.max || 0) save_large_cache generate_combined_ids end |
#dup ⇒ Object
Duplicate the runner, trying to be intelligent as far as possible in duplicating instance variables. Not fully correct yet. Avoid using at the moment.
1528 1529 1530 1531 1532 1533 1534 1535 |
# File 'lib/coderunner/instance_methods.rb', line 1528 def dup logf(:dup) new_one = self.class.new(@code, @root_folder, modlet: @modlet, version: @version) new_one.ids = @ids.dup; new_one.phantom_ids = @phantom_ids.dup; new_one.run_list = @run_list.dup; new_one.phantom_run_list = @phantom_run_list.dup new_one.run_class = @run_class return new_one end |
#executable_location ⇒ Object
1214 1215 1216 1217 |
# File 'lib/coderunner/instance_methods.rb', line 1214 def executable_location return '' unless @executable File.dirname(@executable) end |
#executable_name ⇒ Object
1209 1210 1211 1212 |
# File 'lib/coderunner/instance_methods.rb', line 1209 def executable_name return 'job_chain' unless @executable File.basename(@executable) end |
#film_graphkit(axes, options, film_options = {}) ⇒ Object
408 409 410 411 |
# File 'lib/coderunner/graphs_and_films.rb', line 408 def film_graphkit(axes, , = {}) # self.class.make_film_multiple_runners([[self,[[[axes, options]],[]]]], film_options) film_from_lists([[axes, ]], [], ) end |
#film_run_graphkit(name, options, film_options = {}) ⇒ Object
413 414 415 416 |
# File 'lib/coderunner/graphs_and_films.rb', line 413 def film_run_graphkit(name, , = {}) # self.class.make_film_multiple_runners([[self,[[],[[name, options]]]]], film_options) film_from_lists([], [[name, ]], ) end |
#filter(run = @run, conditions = @conditions) ⇒ Object
Return true if run.instance_eval(conditions) == true and false if run.instance_eval(conditions) == false. performing some checks on the validity of the variable conditions
. For people who are new to Ruby instance_eval means ‘evaluate in the context of the run’. Generally conditions
will be something like ‘status == :Complete and height == 4.5 and not width == 20’, where height and width might be some input parameters or results from the diminishing.
740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 |
# File 'lib/coderunner/instance_methods.rb', line 740 def filter(run=@run, conditions=@conditions) logf(:filter) @run = run # to_instance_variables(directory) return true unless conditions # p conditions, @run.id, @run.is_phantom conditions = conditions.dup raise CRFatal.new(" ----------------------------- Conditions contain a single = sign: #{conditions} -----------------------------") if conditions =~ /[^!=<>]=[^=~<>]/ log conditions begin fil = @run.instance_eval(conditions) rescue => err eputs run.directory eputs conditions raise err end return fil end |
#filtered_ids(conditions = @conditions) ⇒ Object
Return a list of ids, filtered according to conditions. See CodeRunner#filter
727 728 729 730 731 732 |
# File 'lib/coderunner/instance_methods.rb', line 727 def filtered_ids(conditions=@conditions) generate_combined_ids # ep @combined_run_list.keys # sort_runs return @combined_ids.find_all{|id| filter(@combined_run_list[id], conditions)} end |
#generate_combined_ids(kind = nil) ⇒ Object
1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 |
# File 'lib/coderunner/instance_methods.rb', line 1611 def generate_combined_ids(kind= nil) logf(:generate_combined_ids) # case purpose # when :print_out # @combined_ids = [] # @combined_ids += @phantom_ids if @run_class.print_out_phantom_run_list # @combined_ids += @ids if @run_class.print_out_real_run_list # when :readout # @combined_ids = [] # @combined_ids += @phantom_ids if @run_class.readout_phantom_run_list # @combined_ids += @ids if @run_class.readout_real_run_list # when :submitting # @combined_ids = @ids kind ||= @use_phantom case kind when :real @combined_ids = @ids when :phantom @combined_ids = @phantom_ids when :both @combined_ids = @ids + @phantom_ids else raise CRFatal.new("Bad use phantom variable: #{kind.inspect}") end log('@phantom_run_list.class', @phantom_run_list.class) #puts 'crlist', @phantom_run_list.keys, @run_list.keys @combined_run_list = @phantom_run_list + @run_list log(:kind, kind) # log(:combined_ids, @combined_ids) sort_runs(:combined) end |
#get_all_root_folder_contents ⇒ Object
List every file in the root folder.
806 807 808 809 |
# File 'lib/coderunner/instance_methods.rb', line 806 def get_all_root_folder_contents # :nodoc: @root_folder_contents =[] Find.find(@root_folder){|file| @root_folder_contents.push file} end |
#get_max(run, variable, sweep, complete = nil) ⇒ Object
def max(variable,sweep=nil, complete=nil) logf(:max) return get_max(variable,sweep, complete) == @run.id end
def get_max(variable,sweep=nil, complete=nil) logf :get_max sweep ||= variable logi @run.maxes @run.maxes ||= {} similar = similar_runs() similar = similar.find_all{|id| @combined_run_list.status == :Complete} if complete logi(:similar, similar, @combined_run_list[similar].send(variable)) @run.maxes[sweep] ||= similar ? similar.max{|id1,id2| @combined_run_list.send(variable) <=> @combined_run_list.send(variable)} : @run.id # puts “got_here” logi(“@run.maxes[#sweep]”, @run.maxes[sweep]) return @run.maxes[sweep] end
1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 |
# File 'lib/coderunner/instance_methods.rb', line 1488 def get_max(run, variable,sweep, complete=nil) logf :get_max generate_combined_ids sweep = [sweep] unless sweep.class == Array similar = similar_runs(sweep, run) similar = similar.find_all{|id| @combined_run_list[id].status == :Complete} if complete logi(:similar, similar, @combined_run_list[similar[0]].send(variable)) max = similar[1] ? similar.max{|id1,id2| @combined_run_list[id1].send(variable) <=> @combined_run_list[id2].send(variable)} : similar[0] # puts "got_here" return max end |
#get_min(run, variable, sweep, complete = nil) ⇒ Object
1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 |
# File 'lib/coderunner/instance_methods.rb', line 1500 def get_min(run, variable,sweep, complete=nil) logf :get_min generate_combined_ids sweep = [sweep] unless sweep.class == Array similar = similar_runs(sweep, run) similar = similar.find_all{|id| @combined_run_list[id].status == :Complete} if complete logi(:similar, similar, @combined_run_list[similar[0]].send(variable)) min = similar[1] ? similar.min{|id1,id2| @combined_run_list[id1].send(variable) <=> @combined_run_list[id2].send(variable)} : similar[0] # puts "got_here" return min end |
#get_run_class ⇒ Object
206 207 208 |
# File 'lib/coderunner/instance_methods.rb', line 206 def get_run_class @run_class = setup_run_class(@code, modlet: @modlet, version: @version, executable: @executable) end |
#get_run_class_name(code, modlet) ⇒ Object
See CodeRunner.get_run_class_name
330 331 332 |
# File 'lib/coderunner/instance_methods.rb', line 330 def get_run_class_name(code, modlet) self.class.get_run_class_name(code, modlet) end |
#gets ⇒ Object
No reading from the command line thank you very much!
39 40 41 |
# File 'lib/coderunner.rb', line 39 def gets #No reading from the command line thank you very much! $stdin.gets end |
#graphkit(*args) ⇒ Object
call-seq: graphkit(axes, options) Make a general graphkit, i.e. a graph that combines output from lots of runs. The axes should be an array of strings, each string defining what should be plotted on each axis. Each string is actually a fragment of Ruby code which will be evaluated by each run for whom option or @conditions evaulates to true. The most important other option is options. This is a string or an array of strings, each of which will be evaluated by each run; the data will then be grouped for the values of these strings. E.g. graphkit([‘a’, ‘b’], [“c”], conditions: ‘d==1’
254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 |
# File 'lib/coderunner/graphs_and_films.rb', line 254 def graphkit(*args) logf(:graphkit) if args.size == 1 axes, = graphkit_shorthand(args[0]) elsif args.size == 2 axes, = args else raise ArgumentError end # p axes # return named_graphkit(axes, options) if axes.class == String or axes.class == Symbol # ep @sort (return sweep_graphkits([:sweep], ) do || graphkit(axes, ) end) if [:sweep] intial_conditions = @conditions initial_sort = @sort @conditions = ([:conditions] or @conditions) @sort = ([:sort] or @sort) sort_runs case axes.size when 1 kit = GraphKit.autocreate({x: axiskit(axes[0])}) kit.data[0].gp.with = "linespoints" when 2 kit = GraphKit.autocreate({x: axiskit(axes[0]), y: axiskit(axes[1])}) kit.data[0].gp.with = "linespoints" when 3 kit = GraphKit.autocreate({x: axiskit(axes[0]), y: axiskit(axes[1]), z: axiskit(axes[2])}) kit.data[0].gp.with = "linespoints" when 4 kit = GraphKit.autocreate({x: axiskit(axes[0]), y: axiskit(axes[1]), z: axiskit(axes[2]), f: axiskit(axes[3])}) kit.data[0].gp.with = "linespoints palette" else raise CRError.new("Bad graph string") end kit.title += %| for #{@conditions.gsub(/==/, "=").gsub(/\&\&/, " and ").gsub(/\|\|/, " or ").gsub(/\A(.{,40}).*\s/m, '\1')}| if @conditions kit.file_name = kit.title.gsub(/=/, ' eq ').gsub(/\&\&/, " and ").gsub(/\|\|/, " or ").gsub(/ /, "_").gsub(/\//, "over").gsub(/\*/, '.') kit.modify() # p kit kit.data[0].gp.title = "#@code:#{kit.data[0].title}" #Kernel.puts server_dump(kit) if @server return kit end |
#graphkit_from_lists(graphs, run_graphs, extra_options = {}) ⇒ Object
376 377 378 379 380 381 382 383 384 385 386 387 388 389 |
# File 'lib/coderunner/graphs_and_films.rb', line 376 def graphkit_from_lists(graphs, run_graphs, = {}) run_kits = run_graphs.map do |gr| name, = run_graphkit_shorthand(gr) += run_graphkit(name, .dup) end kits = graphs.map do |gr| axes, = graphkit_shorthand(gr) += graphkit(axes, .dup) end (run_kits + kits).inject{|kit, k| kit+k} end |
#graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, extra_options = {}) ⇒ Object
390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 |
# File 'lib/coderunner/graphs_and_films.rb', line 390 def graphkit_from_lists_with_frame_array(frame_array, graphs, run_graphs, = {}) server = @server; @server = false # ep "Frame array to calculate is", frame_array, "\n\n" i=0 array = frame_array.map do |frame_index| # if print_message eputs "\033[2A" # Terminal jargon - go back one line eputs sprintf("Fetching graphs: %2.2f", i.to_f/frame_array.size.to_f * 100.0) + "% Complete" # end i+=1 [frame_index, graphkit_from_lists(graphs, run_graphs, .absorb({([:in]||[:index_name]||:frame_index) => frame_index}))] end # eputs Marshal.load(Marshal.dump(array)).pretty_inspect #Kernel.puts server_dump(array) if server return array end |
#graphkit_shorthand(*gr) ⇒ Object
Parse graphkit shorthand for a general graph and return it as [axes, options]. GraphKit shorthand is: ‘<axis1>[ : <axis2> [ : <axis3 [ : <axis4> ] ] ] [ ; <graph_options> [ ; <conditions> [ ; <sort> ] ] ]’ i.e. all commands for the graph in one string
13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# File 'lib/coderunner/graphs_and_films.rb', line 13 def graphkit_shorthand(*gr) return gr if gr.size > 1 gr = gr[0] return gr if gr.class == Array axes, , cons, srt = gr.split(/;/) axes = eval("['#{axes.split(/(?<!\\|:):(?!:)/).map{|s| s.gsub(/\\:/, ':')}.join("','")}']") = ( and =~ /\S/) ? eval(): {} # p srt cons = nil unless cons and cons=~ /\S/ srt = nil unless srt and srt=~ /\S/ [:conditions] ||= cons [:sort] ||= srt return [axes, ] end |
#increment_max_id ⇒ Object
Increase the value of @max_id
by 1.
225 226 227 |
# File 'lib/coderunner/instance_methods.rb', line 225 def increment_max_id @max_id +=1 end |
#job_identifier ⇒ Object
1230 1231 1232 |
# File 'lib/coderunner/instance_methods.rb', line 1230 def job_identifier "#{@id_list[0]}-#{@id_list[-1]}" end |
#make_film_from_lists(graphs, run_graphs, film_options = {}) ⇒ Object
418 419 420 |
# File 'lib/coderunner/graphs_and_films.rb', line 418 def make_film_from_lists(graphs, run_graphs, = {}) self.class.make_film_multiple_runners([[self,[graphs, run_graphs]]], ) end |
#marshalled_variables ⇒ Object
Dump all the instance variables of the runner to stdout as Marshalled binary data. This is used for RemoteCodeRunner server functions.
932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 |
# File 'lib/coderunner/instance_methods.rb', line 932 def marshalled_variables #ep 'marsh1' instance_vars = {} instance_variables.each do |var| instance_vars[var] = instance_variable_get(var) end instance_vars[:@run_list].values.each{|run| run.runner=nil} #Kernel.puts server_dump(instance_vars) instance_vars[:@cache]={} instance_vars[:@phantom_run_list].values.each{|run| run.runner = nil} #ep 'marsh2' #eputs instance_vars.pretty_inspect #instance_vars.each do |var, val| #ep var #eputs server_dump(val) #end instance_vars end |
#merge(*others) ⇒ Object
3 4 5 |
# File 'lib/coderunner/merged_code_runner.rb', line 3 def merge(*others) Merged.new(*others.unshift(self)) end |
#merge_method(meth, *args) ⇒ Object
7 8 9 |
# File 'lib/coderunner/merged_code_runner.rb', line 7 def merge_method(meth, *args) send(meth, *args) end |
#new_run ⇒ Object
Create a new instance of the class @run_class
1288 1289 1290 1291 |
# File 'lib/coderunner/instance_methods.rb', line 1288 def new_run logf(:new_run) @run_class.new(self) end |
#p(*args) ⇒ Object
Here we redefine the inspect function p to raise an error if anything is written to standard out while in server mode. This is because the server mode uses standard out to communicate and writing to standard out can break it. All messages, debug information and so on, should always be written to standard error.
278 279 280 281 282 283 284 |
# File 'lib/coderunner/instance_methods.rb', line 278 def p(*args) if @server raise "Writing to stdout in server mode will break things!" else super(*args) end end |
#parameter_scan(parameter_scan, parameters, options = {}) ⇒ Object
1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 |
# File 'lib/coderunner/instance_methods.rb', line 1396 def parameter_scan(parameter_scan, parameters, ={}) skip = true unless [:skip] == false # version= (options[:version] or "") nprocs = [:nprocs] logf(:parameter_scan) raise CRFatal.new("Wrong directory: parameter scan being conducted in #{Dir.pwd} instead of my root folder: #{@root_folder}") unless Dir.pwd == File.(@root_folder) @skip = skip puts parameter_scan.inspect @nprocs = nprocs; @skip=skip; log '@run_class', @run_class @run = @run_class.new(self) @run.update_submission_parameters(parameters) # @running_scans = {}; @gammas = {} beginning = "catch(:finished) do \n" end_string = "\nend" parameter_scan.each do |variable_scan| beginning += %[ #{variable_scan[1].inspect}.each do |value|\n\t@run.#{variable_scan[0]} = value\n] @run_class.rcp.naming_pars.push variable_scan[0].to_sym; @run_class.rcp.naming_pars.uniq! end_string += %[\nend] end middle = <<EOF @@psppipe.synchronize(:pssubmit){@running = submit(@run, nprocs: @nprocs, version: @version, skip: @skip); update}; loop do # @@mutex.synchronize{} @run_class.update_status(self) # puts run_class.current_status Dir.chdir(@run_list[@running].directory){@run_list.delete(@running); traverse_directories} # ep @run_list[@running].status, @run_list[@running].id, @run_list[@running].job_no, queue_status unless @run_list[@running].status == :Incomplete or @run_list[@running].status == :NotStarted @run.parameter_transition(@run_list[@running]) break end sleep 3 # Thread.pass # puts Thread.current.to_s + " is looping, @run_list[@running].status = " + @run_list[@running].status.to_s + " @running = " + @running.to_s + " @run_list[@running].job_no = " + @run_list[@running].job_no.to_s end EOF command = beginning + middle + end_string puts command instance_eval(command, 'parameter_scan_code') # puts Thread.current.object_id # puts Thread.current.to_s; print " is finished" # @@psppipe.i_send(:finished, true, tp: 0) end |
#print(*args) ⇒ Object
Here we redefine the function print to raise an error if anything is written to standard out while in server mode. This is because the server mode uses standard out to communicate and writing to standard out can break it. All messages, debug information and so on, should always be written to standard error.
304 305 306 307 308 309 310 |
# File 'lib/coderunner/instance_methods.rb', line 304 def print(*args) if @server raise "Writing to stdout in server mode will break things!" else super(*args) end end |
#print_out(rewind = nil, options = {}) ⇒ Object
Print out a summary of all the runs in the root folder, formatted in nice pretty colours. Since this is often called in a loop, if called twice without any arguments it will erase the first printout. To stop this happening set rewind to 0. If the command is being issued not in a terminal, so that CodeRunner cannot determine the size of the terminal, the second argument must be passed as an array of [rows, columns].
640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 |
# File 'lib/coderunner/instance_methods.rb', line 640 def print_out(rewind = nil, ={}) # terminal_size = [rows, cols] rewind ||= @print_out_size terminal_size = [:terminal_size] logf(:print_out) # raise CRFatal.new("terminal size must be given if this is called any where except inside a terminal (for example if yo u've called this as a subprocess)") unless terminal_size or $stdout.tty? terminal_size ||= Terminal.terminal_size #lots of gritty terminal jargon in here. Edit at your peril! unless ENV['CODE_RUNNER_NO_COLOUR']=='true' or ENV['CODE_RUNNER_NO_COLOR']=='true' dc= Terminal::WHITE # .default_colour green= Terminal::LIGHT_GREEN cyan = Terminal::CYAN bblck = Terminal::BACKGROUND_BLACK else dc= ""# Terminal::WHITE # .default_colour green= "" # Terminal::LIGHT_GREEN cyan = "" #Terminal::CYAN bblck = "" #Terminal::BACKGROUND_BLACK end cln = Terminal::CLEAR_LINE # print "\033[#{rewind}A" deco = '-'*terminal_size[1] Terminal.erewind(rewind) eputs "\n#{cln}\n#{cln}\n#{bblck}#{dc}#{deco}\n#{@run_class.rcp.code_long} Status:#{cln}\n#{deco}" i = 0; j=0 # i is no. of actual terminal lines; j is number of results lines # Group the lines by major sort key #@split = @sort && @sort.split(/\s+/).size > 1 ? @sort.split(/\s+/)[0].to_sym : nil @split_point = nil generate_combined_ids @combined_ids.each do |id| begin # puts id, :hello; gets @run = @combined_run_list[id] if filter # puts @run[:id], @id; gets #@new_split_point = @split ? @run.send(@split) : nil #if @split_point && @split_point != @new_split_point then eputs sprintf(" #{cln}", ""); i+=1 end #@split_point = @new_split_point eprint j%2==0 ? j%4==0 ? cyan : green : dc line = [:with_comments] ? @run.comment_line : @run.print_out_line.chomp eprint line eputs cln # puts (line.size / Terminal.terminal_size[1]).class # puts (line.size / Terminal.terminal_size[1]) i+=((line.sub(/\w*$/, '').size-1) / terminal_size[1]).ceiling j+=1 end # raise "monkeys" rescue => err eputs err eputs err.backtrace eputs "---------------------\nUnable to print out line for this job:" eputs "run_name: #{@run.run_name}" eputs "status: #{@run.status}\n-----------------------" Terminal.reset return # raise CRFatal.new end end @print_out_size = i+7# + (@run_list.keys.find{|id| not [:Complete, :Failed].include? @run_list[id].status } ? 0 : 1) eprint dc, deco; Terminal.reset; eputs # puts # puts # puts dc # "\033[1;37m" # print 'rewind size is', rewind end |
#puts(*args) ⇒ Object
Here we redefine the function puts to raise an error if anything is written to standard out while in server mode. This is because the server mode uses standard out to communicate and writing to standard out can break it. All messages, debug information and so on, should always be written to standard error.
291 292 293 294 295 296 297 |
# File 'lib/coderunner/instance_methods.rb', line 291 def puts(*args) if @server raise "Writing to stdout in server mode will break things!" else super(*args) end end |
#rcp ⇒ Object
def submit
1205 1206 1207 |
# File 'lib/coderunner/instance_methods.rb', line 1205 def rcp @run_class.rcp end |
#read_defaults ⇒ Object
215 216 217 218 219 220 221 |
# File 'lib/coderunner/instance_methods.rb', line 215 def read_defaults DEFAULT_RUNNER_OPTIONS.each{|key,value| set(key, value)} #ep DEFAULT_RUNNER_OPTIONS, @multiple_processes read_folder_defaults get_run_class end |
#read_folder_defaults ⇒ Object
Read any default options contained in the file .code_runner_script_defaults.rb
in the root folder.
247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 |
# File 'lib/coderunner/instance_methods.rb', line 247 def read_folder_defaults # p @root_folder + '/.code_runner_script_defaults.rb' if ENV['CODE_RUNNER_READONLY_DEFAULTS'] hash = eval(File.read(@root_folder + '/.code_runner_script_defaults.rb')) FOLDER_DEFAULTS.each do |var| set(var, hash[var]) if hash[var] end else Hash.phoenix(@root_folder + '/.code_runner_script_defaults.rb') do |hash| # ep hash FOLDER_DEFAULTS.each do |var| # p send(var), hash[var] hash[var] = (send(var) or hash[var]) hash[:code_runner_version] ||= CodeRunner::CODE_RUNNER_VERSION.to_s set(var, hash[var]) end @start_id = hash[:start_id] if hash[:start_id] # ep "start_id: #@start_id" hash end end raise "No default information exists for this folder. If you are running CodeRunner from the commmand line please run again with the -C <code> and -X <executable> (and -m <modlet>, if required) flag (you only need to specify these flags once in each folder). Else, please specify :code and :executable (and :modlet if required) as options in CodeRunner.new(folder, options)" unless @code and @executable end |
#readout ⇒ Object
Similar to CodeRunner#write_data, except that the readout is written to stdout, and formatted a bit. Will probably be rarely used.
764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 |
# File 'lib/coderunner/instance_methods.rb', line 764 def readout logf(:readout) generate_combined_ids @split = @sort && @sort.split(/\s+/).size > 1 ? @sort.split(/\s+/)[0].to_sym : nil @split_point = nil string = @combined_ids.inject("") do |s, id| run = @combined_run_list[id] if run.status =~ /Complete/ and filter(run) @new_split_point = @split ? run.send(@split) : nil splitter = (@split_point and @split_point != @new_split_point) ? "\n\n" : "" @split_point = @new_split_point splitter = "" # comment to put split points back in # puts s.class, splitter.class, data_string(directory) data_string = run.data_string raise CRFatal.new("data_string did not return a string") unless data_string.class == String s + splitter + data_string else s end end # puts @max.inspect return string end |
#readout_cols(*var_list) ⇒ Object
? Probably can get rid of this one
791 792 793 794 795 796 797 798 799 800 801 802 |
# File 'lib/coderunner/instance_methods.rb', line 791 def readout_cols(*var_list) # :nodoc: logf :readout_cols ans = [[]] * var_list.size generate_combined_ids filtered_ids.each do |id| run = combined_run_list[id] var_list.each_with_index do |var, index| ans[index].push run.send(var) end end ans end |
#recheck_filtered_runs(write_status_dots = false) ⇒ Object
Self-explanatory! Call CodeRunner::Run#process_directory for every run for which CodeRunner#filter(run) is true. (Note, for historical reasons traverse_directories is called rather than CodeRunner::Run#process_directory directly but the effect is nearly the same).
995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 |
# File 'lib/coderunner/instance_methods.rb', line 995 def recheck_filtered_runs(write_status_dots=false) @write_status_dots = write_status_dots logf :recheck_filtered_runs @run_class.update_status(self) @run_list.each do |id, run| if filter(run) and not (run.status == :Complete or run.status == :Failed) # eputs run.directory Dir.chdir(run.directory){run.process_directory} # eputs run.status # ep run end end save_large_cache # run_list, @run_list = @run_list, {}; @ids = []; @max_id = 0 # run_list.each do |id, run| # if not filter(run) or run.status == :Complete # @run_list[id] = run # @ids.push id # else # Dir.chdir(run.directory){traverse_directories} # end # end # @ids.sort! # @ids = @run_list.keys set_max_id(@ids.max || 0) sort_runs respond_to_requests save_large_cache end |
#recheck_incomplete_runs ⇒ Object
Self-explanatory! Call CodeRunner::Run#process_directory for every run whose status is not either :Complete or :Failed. (Note, for historical reasons traverse_directories is called rather than CodeRunner::Run#process_directory directly but the effect is nearly the same).
968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 |
# File 'lib/coderunner/instance_methods.rb', line 968 def recheck_incomplete_runs logf :recheck_incomplete_runs @run_class.update_status(self) redone_count = 0 run_list, @run_list = @run_list, {}; @ids = []; run_list.each do |id, run| # print id if run.status == :Complete or run.status == :Failed # print ".", id, "___" @run_list[id] = run @ids.push id else # print id, "%%%"; redone_count+=1 Dir.chdir(run.directory){traverse_directories} end end @ids.sort! # @ids = @run_list.keys set_max_id(@ids.max || 0) sort_runs respond_to_requests save_large_cache end |
#respond_to_requests ⇒ Object
One of the features of CodeRunner is two way communication between a runner and its runs. The runs can request actions from the runner directly by calling its instance methods, but often the runs want something to happen after the runner has processed every run in the directory. For example if a run wanted to check if it was resolved, it would need to know about all the other runs so it could compare itself with them. In this case it would place an instance method that it wanted to call in the variable @requests
in the runner. The runner would then call that instance method on every run after it had finished processing all the runs.
In summary, this method is called after every time the runner has checked through the directory. When it is called, it looks in the variable @requests
which contains symbols representing methods. It calls each symbol as an instance method of every run in turn. So if @requests
was [:check_converged]
it would call run.check_converged
for every run.
1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 |
# File 'lib/coderunner/instance_methods.rb', line 1029 def respond_to_requests logf(:respond_to_requests) logi(:@requests, @requests) log('@phantom_run_list.class', @phantom_run_list.class) while @requests[0] old_requests = @requests.uniq @requests = [] old_requests.each do |request| @current_request = request if request == :traverse_directories # a special case @ids = [] @run_list = {} @phantom_run_list = {} @phantom_ids = [] Dir.chdir(@root_folder){traverse_directories} else filtered_ids.each do |id| run = @run_list[id] Dir.chdir(run.directory){run.instance_eval(request.to_s); run.save; run.write_results} end end end end end |
#run_graphkit(*args) ⇒ Object
345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 |
# File 'lib/coderunner/graphs_and_films.rb', line 345 def run_graphkit(*args) if args.size == 1 name, = run_graphkit_shorthand(args[0]) elsif args.size == 2 name, = args else raise ArgumentError end (return sweep_graphkits([:sweep], ) do || run_graphkit(name, ) end) if [:sweep] old_sort, old_conditions = @sort, @conditions @sort = [:sort] if [:sort] @conditions = [:conditions] if [:conditions] generate_combined_ids fids = filtered_ids raise CRError.new("No ids match these conditions: #@conditions") unless fids.size > 0 kit = (fids.map do |id| run = @combined_run_list[id]; # p run; STDIN.gets kit = run.graphkit(name, .dup); kit.data[0].title ||= run.run_name; kit end).inject{|kit, k| kit+k} @sort, @conditions = old_sort, old_conditions #Kernel.puts server_dump(kit) if @server kit end |
#run_graphkit_shorthand(*grs) ⇒ Object
329 330 331 332 333 334 335 336 337 338 339 340 341 342 |
# File 'lib/coderunner/graphs_and_films.rb', line 329 def run_graphkit_shorthand(*grs) # p grs return grs if grs.size > 1 gr = grs[0] # p gr return gr if gr.class == Array name, , cons, srt = gr.split(/;/) = ( and =~ /\S/) ? eval(): {} cons = nil unless cons and cons=~ /\S/ srt = nil unless srt and srt=~ /\S/ [:conditions] ||= cons if cons [:sort] ||= srt if srt [name, ] end |
#runs ⇒ Object
Return an array of runs (run_list.values)
231 232 233 |
# File 'lib/coderunner/instance_methods.rb', line 231 def runs run_list.values end |
#save_all ⇒ Object
1643 1644 1645 1646 1647 1648 1649 |
# File 'lib/coderunner/instance_methods.rb', line 1643 def save_all save_large_cache @run_list.values.each do |run| run.save run.write_results end end |
#save_large_cache ⇒ Object
Write the variable @run_list
, which contains all information currently known about the simulations in the root folder, as Marshalled binary data in the file “.CODE_RUNNER_TEMP_RUN_LIST_CACHE”. This cache will be used later by CodeRunner#update.
955 956 957 958 959 960 961 962 963 964 |
# File 'lib/coderunner/instance_methods.rb', line 955 def save_large_cache # pp self # ep @run_list # pp @run_list.values.map{|run| run.instance_variables.map{|var| [var, run.instance_variable_get(var).class]}} generate_combined_ids @combined_run_list.each{|id, run| run.runner = nil; run.phantom_runs.each{|pr| pr.runner=nil} if run.phantom_runs} File.open(@root_folder + "/.CODE_RUNNER_TEMP_RUN_LIST_CACHE", 'w'){|file| file.puts Marshal.dump @run_list} @combined_run_list.each{|id, run| run.runner = self; run.phantom_runs.each{|pr| pr.runner=self} if run.phantom_runs} end |
#server_dump(obj) ⇒ Object
316 317 318 |
# File 'lib/coderunner/instance_methods.rb', line 316 def server_dump(obj) self.class.server_dump(obj) end |
#set_start_id(id) ⇒ Object
320 321 322 323 324 325 326 |
# File 'lib/coderunner/instance_methods.rb', line 320 def set_start_id(id) raise "start_id #{id} lower than max_id #@max_id" if @max_id and id < @max_id Hash.phoenix(@root_folder + '/.code_runner_script_defaults.rb') do |hash| @start_id = hash[:start_id] = id hash end end |
#setup_run_class(code, options) ⇒ Object
See CodeRunner.setup_run_class
336 337 338 339 |
# File 'lib/coderunner/instance_methods.rb', line 336 def setup_run_class(code, ) [:runner] ||= self self.class.setup_run_class(code, ) end |
#similar_runs(exclude_variables = [], run = @run) ⇒ Object
Find runs whose input parameters are all the same as those for run
, with the exception of those listed in exclude_variables
1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 |
# File 'lib/coderunner/instance_methods.rb', line 1446 def similar_runs(exclude_variables=[], run=@run) #all runs for which variables are the same as 'run', with the exception of exclude_variables logf(:similar_runs) raise CRFatal.new("generate_combined_ids must be called before this function is called") unless (@combined_run_list.size > 0 and @combined_ids.size > 0) or @ids.size ==0 command = (run.class.rcp.variables+run.class.rcp.run_info-exclude_variables - [:output_file, :error_file, :runner, :phantom_runs]).inject("@combined_ids.find_all{|id| @combined_run_list[id].class == run.class}"){ |s,v| s + %<.find_all{|id| @combined_run_list[id].#{v}.class == #{run.send(v).inspect}.class and @combined_run_list[id].#{v} == #{run.send(v).inspect}}>} #the second send call retrieves the type conversion # log command #puts command begin similar = instance_eval(command) rescue => err log command raise err end return similar end |
#simple_scan(scan_string, options = {}) ⇒ Object
Submit a series of runs according to scan_string. scan_string specifies a number of scans separated by hashes. Each scan has a number which is the ID of the run to start from followed by a colon. After the colon the user can write some code which modifies the original run. For example:
simple_scan(‘23: @height *= 2; @weight *=1.1’)
will submit a series of runs, where each successive run has height twice the last one, and weight 1.1 times the last one, where we assume that height and weight are input parameters.
Options are the same as CodeRunner#submit
1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 |
# File 'lib/coderunner/instance_methods.rb', line 1303 def simple_scan(scan_string, ={}) scans = scan_string.split('#') ppipe = PPipe.new(scans.size + 1, true, controller_refresh: 0.5, redirect: false) pipe_numbers = ppipe.fork(scans.size - 1) #p @run_class.naming_pars instructions = (scans[(ppipe.mpn > 0 ? ppipe.mpn - 1 : 0)]) id = instructions.scan(/^\d+\:/)[0].to_i instructions = instructions.sub(/^\d+\:/, '') @run= id > 0 ? @run_list[id] : (run = @run_class.new(self); run.update_submission_parameters(([:parameters] or '{}')); run) @run_class.rcp.variables.each do |par| #p par, @run_class.naming_pars @run.naming_pars.push par if scan_string =~ Regexp.new(Regexp.escape(par.to_s)) end @run.naming_pars.uniq! # @run.naming_pars +== @run_class.naming_pars catch(:finished) do loop do #submit loop ppipe.synchronize(:submit) do @running = submit(@run, nprocs: [:nprocs], version: @run_class.rcp.version) @conditions = "id == #@running" #print_out #ep 'Running run', @running end loop do # check finished loop dir = @run_list[@running].directory @run_list.delete(@running) @run_class.update_status(self) Dir.chdir(dir) do traverse_directories end unless @@wait and (@run_list[@running].status == :Incomplete or @run_list[@running].status == :NotStarted or @run_list[@running].status == :Queueing) @run.parameter_transition(@run_list[@running]) @old_run = @run_list[@running] break end #p @running ppipe.i_send(:running, @running, tp: 0) if ppipe.is_root arr = (pipe_numbers + [0]).inject([]) do |arr, pn| arr.push ppipe.w_recv(:running, fp: pn) end #p arr @conditions = "#{arr.inspect}.include? id" print_out end sleep 3 end @run.instance_eval(instructions) end end (ppipe.die; exit) unless ppipe.is_root ppipe.finish end |
#sort_runs(type = @use_phantom.to_s.sub(/real/, '')) ⇒ Object
Sort the runs according to the variable @sort
which can be either whitespace separated string or an array of strings. In the former case, the string is split to form an array of strings, using the separator /\s+/
. For example, if
Then the runs will be sorted according to first height, then (descending) weight, then colour. What actually happens is that the variable @ids
is sorted, rather than @run_list
. For this to work, height, weight and colour must be either variables or results or instance methods of the run class.
Type can be either ” or ‘phantom’, in which case the variable sorted will be either @ids
or @phantom_ids
respectively.
620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 |
# File 'lib/coderunner/instance_methods.rb', line 620 def sort_runs(type = @use_phantom.to_s.sub(/real/, '')) logf(:sort_runs) log(:type, type) #ep 'sort', @sort #sort_list = @sort ? (@sort.class == String ? eval(@sort) : @sort) : [] run_list_name = [type.to_s, 'run_list'].join('_').gsub(/^_/, '') ids_name = [type.to_s, 'ids'].join('_').gsub(/^_/, '') log run_list_name set(ids_name, send(ids_name).sort_by do |id| run = send(run_list_name)[id] sortkey = run.instance_eval((@sort or '[]')) #ep 'sortkey', sortkey sortkey #sort_list.map{|str| run.instance_eval(str)} end) end |
#submit(runs, options = {}) ⇒ Object
Start a simulation: submit the run that is passed. What happens is as follows:
-
Modify the run according to options.
-
Check if any other processes are submitting runs in the same root folder. In this case there will be a file called ‘submitting’ in the folder. If there is such a file wait until it is gone.
-
Check if a run with identical parameters has been submitted before. In which case skip submitting the run unless options == false.
-
Call
run.submit
Options can be any one of CodeRunner::SUBMIT_OPTIONS
. The options passed here will override values stored as instance variables of the runner with the same name, which will override these values if they are set in the runs itself. For example if run.nprocs == ‘32x4’ runner.nprocs == nil options == nil the number of processes will be 32x4. On the other hand if run.nprocs == ‘32x4’ runner.nprocs == ‘24x4’ options == ‘48x4’ the number of processes will be 48x4.
1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 |
# File 'lib/coderunner/instance_methods.rb', line 1072 def submit(runs, ={}) eputs "System " + SYS eputs "No. Procs " + @nprocs.inspect runs = [runs] unless runs.class == Array #can pass a single run to submit outruns = runs.dup skip = true unless [:skip] == false SUBMIT_OPTIONS.each do |option| set(option, [option]) if .keys.include? option end logf(:submit) Dir.chdir(@root_folder) do @skip=skip mess = false while FileTest.exist?("submitting") (eputs " Waiting for another process to finish submitting. If you know that no other CodeRunner processes are submitting in this folder (#@root_folder) then delete the file 'submitting' and try again"; mess = true) unless mess sleep rand end # old_trap = trap(0) #old_trap0 = trap(0){eputs "Aborted Submit (0)!"; File.delete("#@root_folder/submitting"); exit!} old_trap2 = trap(2){eputs "Aborted Submit (2)!"; File.delete("#@root_folder/submitting") if FileTest.exist? "#@root_folder/submitting"; trap(2, "DEFAULT"); trap(0, "DEFAULT"); Process.kill(2, 0)} # File.open("submitting", "w"){|file| file.puts ""} FileUtils.touch("submitting") unless [:no_update_before_submit] @use_large_cache, ulc = false, @use_large_cache; update; @use_large_cache = ulc end generate_combined_ids(:real) # old_job_nos = queue_status.scan(/^\s*(\d+)/).map{|match| match[0].to_i} script = "" if [:job_chain] runs.each_with_index do |run, index| similar = similar_runs([], run) if @skip and similar[0] and not ([:replace_existing] or [:rerun]) eputs "Found similar run: #{@run_list[similar[0]].run_name}" eputs "Skipping submission..." runs[index] = nil next end unless [:replace_existing] or [:rerun] @max_id+=1 run.id = @max_id else if [:replace_existing] FileUtils.rm_r run.directory elsif [:rerun] ################# For backwards compatibility SUBMIT_OPTIONS.each do |opt| run.set(opt, send(opt)) unless run.send(opt) end ########################################### FileUtils.rm "#{run.directory}/code_runner_results.rb" FileUtils.rm "#{run.directory}/.code_runner_run_data" end @run_list.delete(run.id) @ids.delete run.id generate_combined_ids end begin unless [:job_chain] run.prepare_submission unless [:rerun] next if @test_submission Dir.chdir(run.directory) do old_job_nos = queue_status.scan(/^\s*(\d+)/).map{|match| match[0].to_i} ######################### The big tomale! run.job_no = run.execute # Start the simulation and get the job_number ######################### run.job_no = get_new_job_no(old_job_nos) unless run.job_no.kind_of? Integer # (if the execute command does not return the job number, look for it manually) # eputs 'run.job_no', run.job_no run.output_file = nil # reset the output file run.output_file # Sets the output_file on first call run.error_file = nil # reset the output file run.error_file # Sets the error_file on first call run.write_info eputs "Submitted run: #{run.run_name}" end else run.prepare_submission unless [:rerun] script << "cd #{run.directory}\n" script << "#{run.run_command}\n" next if @test_submission end rescue => err File.delete("submitting") raise(err) end end # runs.each runs.compact! if [:job_chain] and not @test_submission and runs.size > 0 FileUtils.makedirs('job_chain_files') @id_list = runs.map{|run| run.id} #@executable ||= runs[0].executable @submission_script = script # A hook... default is to do nothing @submission_script = @run_class.modify_job_script(self, runs, @submission_script) # To get out of job_chain_files folder @submission_script = "cd .. \n" + @submission_script old_job_nos = queue_status.scan(/^\s*(\d+)/).map{|match| match[0].to_i} ################ Submit the run Dir.chdir('job_chain_files'){job_no = execute} ################ job_no = get_new_job_no(old_job_nos) unless job_no.kind_of? Integer # (if the execute command does not return the job number, look for it manually) # eputs 'jobchain no', job_no #runs.each{|run| run.job_no = job_no} runs.each do |run| run.job_no = @job_no = job_no run.output_file = run.relative_directory.split("/").map{|d| ".."}.join("/") + "/job_chain_files/" + output_file run.error_file = run.relative_directory.split("/").map{|d| ".."}.join("/") + "/job_chain_files/" + error_file run.write_info eputs "Submitted run: #{run.run_name}" end end @write_status_dots, wsd = false, @write_status_dots @run_class.update_status(self) runs.each do |run| # ep run.id, run_list.keys Dir.chdir(run.directory){traverse_directories} end @write_status_dots = wsd save_large_cache File.delete("submitting") #trap(0, old_trap0) trap(2, old_trap2) end # Dir.chdir(@root_folder) # eputs #ep 'runs submitted', outruns return outruns[0].id if outruns.size == 1 #used in parameter scans end |
#sweep_graphkits(sweeps, options, &block) ⇒ Object
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
# File 'lib/coderunner/graphs_and_films.rb', line 29 def sweep_graphkits(sweeps, , &block) server = @server; @server = false intial_conditions = @conditions initial_sort = @sort @conditions = ([:conditions] or @conditions) @sort = ([:sort] or @sort) sweeps = sweeps.class == Array ? sweeps : [sweeps] generate_combined_ids # ep sweeps sweep_values = filtered_ids.inject([]) do |arr, id| arr.push(sweeps.inject([]) do |sweep_arr, var| sweep_arr.push @combined_run_list[id].send(var) # ep sweep_arr, var sweep_arr end) arr.uniq end old_conditions = @conditions = .dup .delete(:sweep) if [:sweep] # p new_options; gets kits = sweep_values.sort.map do |values| new_cons = (sweeps.zip(values).map do |name, value| "#{name}==#{value}" end).join(" and ") [:conditions] = old_conditions ? old_conditions + " and #{new_cons}" : new_cons # ep new_options[:conditions] kit = yield() kit.data[0].gp.title = new_cons.gsub(/==/, '=') # p kit kit end # @conditions = old_conditions @conditions = intial_conditions @sort = initial_sort if [:sweep_multid] kitdata = kits.map do |kit| kit.data[0].axes.values.map{|axiskit| axiskit.data} end final_axes = (kitdata[0].size + sweeps.size - 1).times.inject([]){|arr, i| arr.push []} data_hash = {} sweep_values.sort.each_with_index do |vals, i| # ep 'v', vals data = kitdata[i] length = data[0].size vals.each_with_index do |val, j| final_axes[j].push val # final_axes[j].uniq! end function = data.pop for j in 0...length data.each_with_index do |axisdata, k| # ep 'vk', vals.size + k # break if k == data.size - 1 final_axes[vals.size + k].push axisdata[j] final_axes[vals.size + k].uniq! end # ep 'h', final_axes, function data_hash[vals + data.map{|axisdata| axisdata[j]}] = function[j] end end final_axes.map!{|vals| vals.uniq.sort} final_data = SparseTensor.new(final_axes.size) data_hash.each do |coordinates, function| indices = [] coordinates.each_with_index do |val, i| indices[i] = final_axes[i].index(val) end final_data[*indices] = function end hash = {} # pp final_axes, final_data titles = sweeps + kits[0].data[0].axes.values.map{|axiskit| axiskit[:title]} (final_axes + [final_data]).each_with_index do |d, i| hash[GraphKit::AXES[i]] = {data: d, title: titles[i].to_s} end kit = GraphKit.autocreate(hash) # length = data[i].size # array = vals.map{|val| [val] * length} + data # p array # end # final_data = [[]] * (kitdata[0].size + sweeps.size) # sweep_values.sort.each_with_index do |vals, i| # data = kitdata[0] # length = data[0].size # array = vals.map{|val| [val] * length} + data # p array # end else kit = kits.inject{|ans, kit| ans + kit} end #Kernel.puts server_dump(kit) if server return kit end |
#update(write_status_dots = true, use_large_cache = @use_large_cache, use_large_cache_but_recheck_incomplete = @use_large_cache_but_recheck_incomplete) ⇒ Object
Update the information about all the runs stored in the variable @run_list
. By default, this is done by calling CodeRunner#traverse_directories. If, on the other hand, @use_large_cache
is set to true, this is done by reading the temporary cache maintained in “.CODE_RUNNER_TEMP_RUN_LIST_CACHE” in the root folder. This is much quicker. If in addition @use_large_cache_but_recheck_incomplete
is set to true, all runs whose status is not either :Complete or :Failed will be rechecked.
813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 |
# File 'lib/coderunner/instance_methods.rb', line 813 def update(write_status_dots=true, use_large_cache=@use_large_cache, use_large_cache_but_recheck_incomplete=@use_large_cache_but_recheck_incomplete) @use_large_cache = use_large_cache logf(:update) @write_status_dots = write_status_dots @run_list={} @ids=[] @phantom_run_list = {} @phantom_ids = [] @run_store =[] # @multiple_processes_directories = [] set_max_id 0 @run_class.update_status(self) if @use_large_cache and not @recalc_all and not @reprocess_all log("using large cache") begin begin eputs 'Loading large cache...' if @write_status_dots Dir.chdir(@root_folder) do @run_list = Marshal.load(File.read(".CODE_RUNNER_TEMP_RUN_LIST_CACHE")) end @run_list.values.each{|run| run.runner = self} rescue ArgumentError => err eputs err raise err unless err. =~ /undefined class/ #NB all code_names have to contain only lowercase letters: # modlet, code = err.message.scan(/CodeRunner\:\:([A-Z][\w+])?([A-Z]\w+)Run/)[0] code, modlet = err..scan(/CodeRunner\:\:([A-Z][a-z0-9_]+)(?:::([A-Z]\w+))?/)[0] # ep code.downcase, modlet modlet.downcase! if modlet setup_run_class(code.downcase, modlet: modlet) retry end # ep @run_list @ids = @run_list.keys @run_list.each{|id, run| run.runner = self } #eputs "Setting max id..." set_max_id(@ids.max || 0) #eputs "max_id = #@max_id" # puts @max_id; gets # @use_large_cache = false @ids.sort! redone_count = 0 # puts "hello" # ep @use_large_cache_but_recheck_incomplete; exit recheck_incomplete_runs if use_large_cache_but_recheck_incomplete # sort_runs eprint "Updating runs..." if @write_status_dots # get_all_root_folder_contents # puts @run_list.values[0].directory, File.expand_path(@root_folder).esc_regex fix_directories = (run_list.size > 0 and not @run_list.values[0].directory =~ File.(@root_folder).esc_regex) # eputs 'fdirectories', fix_directories # exit eputs "Fixing Directories..." if fix_directories @run_list.each do |id, run| eprint '.' if @write_status_dots run.directory = File.join(@root_folder, run.relative_directory) if fix_directories # run.directory = "#@root_folder/#{run.relative_directory}" # unless @root_folder_contents.include? run.directory# File.directory? run.directory and run.directory =~ File.expand_path(@root_folder).esc_regex # if @root_folder_contents.include?(rel = File.join(@root_folder, run.relative_directory)) # run.directory = rel # else # raise CRFatal.new("Directory #{run.directory} not found") # end # end # eputs @use_phantom #run.generate_phantom_runs #if @use_phantom.to_s =~ /phantom/i run.phantom_runs.each{|r| add_phantom_run(r)} if run.phantom_runs end eputs if @write_status_dots save_large_cache if fix_directories # puts redone_count return self rescue Errno::ENOENT, ArgumentError, TypeError => err if err.class == ArgumentError and not err. =~ /marshal data too short/ or err.class == TypeError and not err. =~ /incompatible marshal file format/ eputs err eputs err.backtrace raise CRFatal.new end eputs err, "Rereading run data" # @use_large_cache = false end end log("not using large cache") @run_list={} @ids=[] @phantom_run_list = {} @phantom_ids = [] @run_store =[] # @multiple_processes_directories = [] set_max_id 0 log("traversing directories") eprint 'Analysing runs..' if @write_status_dots #interrupted = false; #old_trap2 = trap(2){} #trap(2){eputs "Interrupt acknowledged... reloading saved cache. Interrupt again to override (may cause file corruption)."; trap(2, old_trap2); update(true, true, false); Process.kill(2,0)} Dir.chdir(@root_folder){traverse_directories} @max_id ||= 0 eputs # puts 'before request', @ids, @requests; respond_to_requests # @n_checks += 1 # exit if ($nruns > 0 && @n_checks > $nruns) sort_runs @recalc_all = false # pp @run_list save_large_cache #@run_list.each{|id, run| run.generate_phantom_runs} #trap(2, old_trap2) #Process.kill(2, 0) if interrupted return self end |
#write_data(filename = "data.txt") ⇒ Object
Write out a simple datafile containing all the inputs and outputs listed in the run class property readout_string
. What is actually written out can be customised by redefining the method data_string
in the run class, or changing readout_string
or both.
597 598 599 600 601 602 603 604 605 606 607 608 609 610 |
# File 'lib/coderunner/instance_methods.rb', line 597 def write_data(filename = "data.txt") logf(:write_data) generate_combined_ids File.open(filename, "w") do |file| @combined_ids.each do |id| run = @combined_run_list[id] if run.status =~ /Complete/ data_string = run.data_string raise CRFatal.new("data_string did not return a string") unless data_string.class == String file.puts data_string end end end end |