Module: Bio::Command

Defined in:
lib/bio/command.rb

Overview

Bio::Command

Bio::Command is a collection of useful methods for execution of external commands or web applications. Any wrapper class for applications shall use this class.

Library internal use only. Users should not directly use it.

Constant Summary collapse

UNSAFE_CHARS_UNIX = 
/[^A-Za-z0-9\_\-\.\:\,\/\@\x1b\x80-\xfe]/n
QUOTE_CHARS_WINDOWS = 
/[^A-Za-z0-9\_\-\.\:\,\/\@\\]/n
UNESCAPABLE_CHARS = 
/[\x00-\x08\x10-\x1a\x1c-\x1f\x7f\xff]/n

Class Method Summary collapse

Class Method Details

.call_command(cmd, options = {}, &block) ⇒ Object

Executes the program. Automatically select popen for Windows environment and fork for the others. A block must be given. An IO object is passed to the block.

Available options:

:chdir => "path" : changes working directory to the specified path.

Arguments:

  • (required) cmd: Array containing String objects

  • (optional) options: Hash

Returns

(undefined)



145
146
147
148
149
150
151
152
 
# File 'lib/bio/command.rb', line 145

def call_command(cmd, options = {}, &block) #:yields: io
  case RUBY_PLATFORM
  when /mswin32|bccwin32/
    call_command_popen(cmd, options, &block)
  else
    call_command_fork(cmd, options, &block)
  end
end

.call_command_fork(cmd, options = {}) ⇒ Object

Executes the program via fork (by using IO.popen(“-”)) and exec. A block must be given. An IO object is passed to the block.

From the view point of security, this method is recommended rather than call_command_popen.

Arguments:

  • (required) cmd: Array containing String objects

  • (optional) options: Hash

Returns

(undefined)



196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
 
# File 'lib/bio/command.rb', line 196

def call_command_fork(cmd, options = {})
  dir = options[:chdir]
  cmd = safe_command_line_array(cmd)
  IO.popen("-", "r+") do |io|
    if io then
      # parent
      yield io
    else
      # child
      # chdir to options[:chdir] if available
      begin
        Dir.chdir(dir) if dir
      rescue Exception
        Process.exit!(1)
      end
      # executing the command
      begin
        Kernel.exec(*cmd)
      rescue Errno::ENOENT, Errno::EACCES
        Process.exit!(127)
      rescue Exception
      end
      Process.exit!(1)
    end
  end
end

.call_command_open3(cmd) ⇒ Object

Executes the program via Open3.popen3 A block must be given. IO objects are passed to the block.

You would use this method only when you really need to get stderr.

Arguments:

  • (required) cmd: Array containing String objects

Returns

(undefined)



232
233
234
235
236
237
 
# File 'lib/bio/command.rb', line 232

def call_command_open3(cmd)
  cmd = safe_command_line_array(cmd)
  Open3.popen3(*cmd) do |pin, pout, perr|
    yield pin, pout, perr
  end
end

.call_command_popen(cmd, options = {}) ⇒ Object

Executes the program via IO.popen for OS which doesn’t support fork. A block must be given. An IO object is passed to the block.

Arguments:

  • (required) cmd: Array containing String objects

  • (optional) options: Hash

Returns

(undefined)



161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
 
# File 'lib/bio/command.rb', line 161

def call_command_popen(cmd, options = {})
  str = make_command_line(cmd)
  # processing options
  if dir = options[:chdir] then
    case RUBY_PLATFORM
    when /mswin32|bccwin32/
      # Unix-like dir separator is changed to Windows dir separator
      # by using String#gsub.
      dirstr = dir.gsub(/\//, "\\")
      chdirstr = make_command_line([ 'cd', '/D', dirstr ])
      str = chdirstr + ' && ' + str
    else
      # UNIX shell
      chdirstr = make_command_line([ 'cd', dir ])
      str = chdirstr + ' && ' + str
    end
  end
  # call command by using IO.popen
  IO.popen(str, "w+") do |io|
    io.sync = true
    yield io
  end
end

.escape_shell(str) ⇒ Object

Escape special characters in command line string.

Arguments:

  • (required) str: String

Returns

String object



68
69
70
71
72
73
74
75
 
# File 'lib/bio/command.rb', line 68

def escape_shell(str)
  case RUBY_PLATFORM
  when /mswin32|bccwin32/
    escape_shell_windows(str)
  else
    escape_shell_unix(str)
  end
end

.escape_shell_unix(str) ⇒ Object

Escape special characters in command line string for UNIX shells.

Arguments:

  • (required) str: String

Returns

String object



57
58
59
60
61
 
# File 'lib/bio/command.rb', line 57

def escape_shell_unix(str)
  str = str.to_s
  raise 'cannot escape control characters' if UNESCAPABLE_CHARS =~ str
  str.gsub(UNSAFE_CHARS_UNIX) { |x| "\\#{x}" }
end

.escape_shell_windows(str) ⇒ Object

Escape special characters in command line string for cmd.exe on Windows.

Arguments:

  • (required) str: String

Returns

String object



42
43
44
45
46
47
48
49
50
 
# File 'lib/bio/command.rb', line 42

def escape_shell_windows(str)
  str = str.to_s
  raise 'cannot escape control characters' if UNESCAPABLE_CHARS =~ str
  if QUOTE_CHARS_WINDOWS =~ str then
    '"' + str.gsub(/\"/, '""') + '"'
  else
    String.new(str)
  end
end

.http_post_form(http, path, params = nil, header = {}) ⇒ Object

Same as:

http = Net::HTTP.new(...); http.post_form(path, params)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set. In addition, header can be set. (Note that Content-Type and Content-Length are automatically set by default.) uri must be a URI object, params must be a hash, and header must be a hash.

Arguments:

  • (required) http: Net::HTTP object or compatible object

  • (required) path: String

  • (optional) params: Hash containing parameters

  • (optional) header: Hash containing header strings

Returns

(same as Net::HTTP::post_form)



481
482
483
484
485
486
487
488
489
490
491
 
# File 'lib/bio/command.rb', line 481

def http_post_form(http, path, params = nil, header = {})
  data = make_cgi_params(params)

  hash = {
    'Content-Type'   => 'application/x-www-form-urlencoded',
    'Content-Length' => data.length.to_s
  }
  hash.update(header)

  http.post(path, data, hash)
end

.make_cgi_params(params) ⇒ Object

Builds parameter string for from Hash of parameters for application/x-www-form-urlencoded.

Arguments:

  • (required) params: Hash containing parameters

Returns

String



535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
 
# File 'lib/bio/command.rb', line 535

def make_cgi_params(params)
  data = ""
  case params
  when Hash
    data = params.map do |key, val|
      make_cgi_params_key_value(key, val)
    end.join('&')
  when Array
    case params.first
    when Hash
      data = params.map do |hash|
        hash.map do |key, val|
          make_cgi_params_key_value(key, val)
        end
      end.join('&')
    when Array
      data = params.map do |key, val|
        make_cgi_params_key_value(key, val)
      end.join('&')
    when String
      data = params.map do |str|
        key, val = str.split(/\=/, 2)
        if val then
          make_cgi_params_key_value(key, val)
        else
          CGI.escape(str)
        end
      end.join('&')
    end
  when String
    data = URI.escape(params.strip)
  end
  return data
end

.make_cgi_params_key_value(key, value) ⇒ Object

Builds parameter string for from a key string and a value (or values) for application/x-www-form-urlencoded.

Arguments:

  • (required) key: String

  • (required) value: String or Array containing String

Returns

String



578
579
580
581
582
583
584
585
586
587
588
589
 
# File 'lib/bio/command.rb', line 578

def make_cgi_params_key_value(key, value)
  result = []
  case value
  when Array
    value.each do |val|
      result << [key, val].map {|x| CGI.escape(x.to_s) }.join('=')
    end
  else
    result << [key, value].map {|x| CGI.escape(x.to_s) }.join('=')
  end
  return result
end

.make_command_line(ary) ⇒ Object

Generate command line string with special characters escaped.

Arguments:

  • (required) ary: Array containing String objects

Returns

String object



82
83
84
85
86
87
88
89
 
# File 'lib/bio/command.rb', line 82

def make_command_line(ary)
  case RUBY_PLATFORM
  when /mswin32|bccwin32/
    make_command_line_windows(ary)
  else
    make_command_line_unix(ary)
  end
end

.make_command_line_unix(ary) ⇒ Object

Generate command line string with special characters escaped for UNIX shells.

Arguments:

  • (required) ary: Array containing String objects

Returns

String object



107
108
109
 
# File 'lib/bio/command.rb', line 107

def make_command_line_unix(ary)
  ary.collect { |str| escape_shell_unix(str) }.join(" ")
end

.make_command_line_windows(ary) ⇒ Object

Generate command line string with special characters escaped for cmd.exe on Windows.

Arguments:

  • (required) ary: Array containing String objects

Returns

String object



97
98
99
 
# File 'lib/bio/command.rb', line 97

def make_command_line_windows(ary)
  ary.collect { |str| escape_shell_windows(str) }.join(" ")
end

.mktmpdir(prefix = 'd', tmpdir = nil, &block) ⇒ Object

Backport of Dir.mktmpdir in Ruby 1.9.

Same as Dir.mktmpdir(prefix_suffix) in Ruby 1.9 except that prefix must be a String, nil, or omitted.

Arguments:

  • (optional) prefix: String



367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
 
# File 'lib/bio/command.rb', line 367

def mktmpdir(prefix = 'd', tmpdir = nil, &block)
  prefix = prefix.to_str
  begin
    Dir.mktmpdir(prefix, tmpdir, &block)
  rescue NoMethodError
    suffix = ''
    # backported from Ruby 1.9.0.
    # ***** Below is excerpted from Ruby 1.9.0's lib/tmpdir.rb ****
    # ***** Be careful about copyright. ****
    tmpdir ||= Dir.tmpdir
    t = Time.now.strftime("%Y%m%d")
    n = nil
    begin
      path = "#{tmpdir}/#{prefix}#{t}-#{$$}-#{rand(0x100000000).to_s(36)}"
      path << "-#{n}" if n
      path << suffix
      Dir.mkdir(path, 0700)
    rescue Errno::EEXIST
      n ||= 0
      n += 1
      retry
    end

    if block_given?
      begin
        yield path
      ensure
        remove_entry_secure path
      end
    else
      path
    end
    # ***** Above is excerpted from Ruby 1.9.0's lib/tmpdir.rb ****
  end
end

.new_http(address, port = 80) ⇒ Object

Same as:

Net::HTTP.new(address, port)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set.

Arguments:

  • (required) address: String containing host name or IP address

  • (optional) port: port (sanme as Net::HTTP::start)

Returns

(same as Net::HTTP.new except for proxy support)



451
452
453
454
455
456
457
458
459
460
461
 
# File 'lib/bio/command.rb', line 451

def new_http(address, port = 80)
  uri = URI.parse("http://#{address}:#{port}")
  # Note: URI#find_proxy is an unofficial method defined in open-uri.rb.
  # If the spec of open-uri.rb would be changed, we should change below.
  if proxyuri = uri.find_proxy then
    raise 'Non-HTTP proxy' if proxyuri.class != URI::HTTP
    Net::HTTP.new(address, port, proxyuri.host, proxyuri.port)
  else
    Net::HTTP.new(address, port)
  end
end

.post_form(uri, params = nil, header = {}) ⇒ Object

Same as: Net::HTTP.post_form(uri, params) and it uses proxy if an environment variable (same as OpenURI.open_uri) is set. In addition, header can be set. (Note that Content-Type and Content-Length are automatically set by default.) uri must be a URI object, params must be a hash, and header must be a hash.

Arguments:

  • (required) uri: URI object or String

  • (optional) params: Hash containing parameters

  • (optional) header: Hash containing header strings

Returns

(same as Net::HTTP::post_form)



510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
 
# File 'lib/bio/command.rb', line 510

def post_form(uri, params = nil, header = {})
  unless uri.is_a?(URI)
    uri = URI.parse(uri)
  end

  data = make_cgi_params(params)

  hash = {
    'Content-Type'   => 'application/x-www-form-urlencoded',
    'Content-Length' => data.length.to_s
  }
  hash.update(header)

  start_http(uri.host, uri.port) do |http|
    http.post(uri.path, data, hash)
  end
end

.query_command(cmd, query = nil, options = {}) ⇒ Object

Executes the program with the query (String) given to the standard input, waits the program termination, and returns the output data printed to the standard output as a string.

Automatically select popen for Windows environment and fork for the others.

Available options:

:chdir => "path" : changes working directory to the specified path.

Arguments:

  • (required) cmd: Array containing String objects

  • (optional) query: String

  • (optional) options: Hash

Returns

String or nil



254
255
256
257
258
259
260
261
 
# File 'lib/bio/command.rb', line 254

def query_command(cmd, query = nil, options = {})
  case RUBY_PLATFORM
  when /mswin32|bccwin32/
    query_command_popen(cmd, query, options)
  else
    query_command_fork(cmd, query, options)
  end
end

.query_command_fork(cmd, query = nil, options = {}) ⇒ Object

Executes the program with the query (String) given to the standard input, waits the program termination, and returns the output data printed to the standard output as a string.

Fork (by using IO.popen(“-”)) and exec is used to execute the program.

From the view point of security, this method is recommended rather than query_command_popen.

Arguments:

  • (required) cmd: Array containing String objects

  • (optional) query: String

  • (optional) options: Hash

Returns

String or nil



301
302
303
304
305
306
307
308
309
310
 
# File 'lib/bio/command.rb', line 301

def query_command_fork(cmd, query = nil, options = {})
  ret = nil
  call_command_fork(cmd, options) do |io|
    io.sync = true
    io.print query if query
    io.close_write
    ret = io.read
  end
  ret
end

.query_command_open3(cmd, query = nil) ⇒ Object

Executes the program via Open3.popen3 with the query (String) given to the stain, waits the program termination, and returns the data from stdout and stderr as an array of the strings.

You would use this method only when you really need to get stderr.

Arguments:

  • (required) cmd: Array containing String objects

  • (optional) query: String

Returns

Array containing 2 objects: output string (or nil) and stderr string (or nil)



323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
 
# File 'lib/bio/command.rb', line 323

def query_command_open3(cmd, query = nil)
  errorlog = nil
  cmd = safe_command_line_array(cmd)
  Open3.popen3(*cmd) do |pin, pout, perr|
    perr.sync = true
    t = Thread.start { errorlog = perr.read }
    begin
      pin.print query if query
      pin.close
      output = pout.read
    ensure
      t.join
    end
    [ output, errorlog ]
  end
end

.query_command_popen(cmd, query = nil, options = {}) ⇒ Object

Executes the program with the query (String) given to the standard input, waits the program termination, and returns the output data printed to the standard output as a string.

IO.popen is used for OS which doesn’t support fork.

Arguments:

  • (required) cmd: Array containing String objects

  • (optional) query: String

  • (optional) options: Hash

Returns

String or nil



275
276
277
278
279
280
281
282
283
284
 
# File 'lib/bio/command.rb', line 275

def query_command_popen(cmd, query = nil, options = {})
  ret = nil
  call_command_popen(cmd, options) do |io|
    io.sync = true
    io.print query if query
    io.close_write
    ret = io.read
  end
  ret
end

.read_uri(uri) ⇒ Object

Same as OpenURI.open_uri(uri).read and it uses proxy if an environment variable (same as OpenURI.open_uri) is set.

Arguments:

  • (required) uri: URI object or String

Returns

String



412
413
414
 
# File 'lib/bio/command.rb', line 412

def read_uri(uri)
  OpenURI.open_uri(uri).read
end

.remove_entry_secure(path, force = false) ⇒ Object

Same as FileUtils.remove_entry_secure after Ruby 1.8.3. In Ruby 1.8.2 or previous version, it only shows warning message and does nothing.

It is strongly recommended using Ruby 1.8.5 or later.

Arguments:

  • (required) path: String

  • (optional) force: boolean



349
350
351
352
353
354
355
356
 
# File 'lib/bio/command.rb', line 349

def remove_entry_secure(path, force = false)
  begin
    FileUtils.remove_entry_secure(path, force)
  rescue NoMethodError
    warn "The temporary file or directory is not removed because of the lack of FileUtils.remove_entry_secure. Use Ruby 1.8.3 or later (1.8.5 or later is strongly recommended): #{path}"
    nil
  end
end

.safe_command_line_array(ary) ⇒ Object

Returns an Array of command-line command and arguments that can be safely passed to Kernel.exec etc. If the given array is already safe (or empty), returns the given array.

Arguments:

  • (required) ary: Array

Returns

Array



118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'lib/bio/command.rb', line 118

def safe_command_line_array(ary)
  ary = ary.to_ary
  return ary if ary.size >= 2 or ary.empty?
  if ary.size != 1 then
    raise 'Bug: assersion of ary.size == 1 failed'
  end
  arg0 = ary[0]
  begin
    arg0 = arg0.to_ary
  rescue NoMethodError
    arg0 = [ arg0, arg0 ]
  end
  [ arg0 ]
end

.start_http(address, port = 80, &block) ⇒ Object

Same as:

Net::HTTP.start(address, port)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set.

Arguments:

  • (required) address: String containing host name or IP address

  • (optional) port: port (sanme as Net::HTTP::start)

Returns

(same as Net::HTTP::start except for proxy support)



427
428
429
430
431
432
433
434
435
436
437
438
 
# File 'lib/bio/command.rb', line 427

def start_http(address, port = 80, &block)
  uri = URI.parse("http://#{address}:#{port}")
  # Note: URI#find_proxy is an unofficial method defined in open-uri.rb.
  # If the spec of open-uri.rb would be changed, we should change below.
  if proxyuri = uri.find_proxy then
    raise 'Non-HTTP proxy' if proxyuri.class != URI::HTTP
    http = Net::HTTP.Proxy(proxyuri.host, proxyuri.port)
  else
    http = Net::HTTP
  end
  http.start(address, port, &block)
end