Class: HybridPlatformsConductor::HpcPlugins::Connector::Ssh

Inherits:
Connector
  • Object
show all
Defined in:
lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb

Overview

Connect to node using SSH

Defined Under Namespace

Modules: PlatformsDslSsh Classes: NotConnectableError

Constant Summary collapse

TMP_SSH_SUB_DIR =

String: Sub-path of the system’s temporary directory where temporary SSH config are generated

'hpc_ssh'
MAX_CMD_ARG_LENGTH =

Integer: Max size for an argument that can be executed without getting through an intermediary file

131_055

Constants included from LoggerHelpers

LoggerHelpers::LEVELS_MODIFIERS, LoggerHelpers::LEVELS_TO_STDERR

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods inherited from Connector

#initialize, #prepare_for

Methods inherited from Plugin

extend_config_dsl_with, #initialize, valid?

Methods included from LoggerHelpers

#err, #init_loggers, #log_component=, #log_debug?, #log_level=, #out, #section, #set_loggers_format, #stderr_device, #stderr_device=, #stderr_displayed?, #stdout_device, #stdout_device=, #stdout_displayed?, #stdouts_to_s, #with_progress_bar

Constructor Details

This class inherits a constructor from HybridPlatformsConductor::Connector

Instance Attribute Details

#auth_password ⇒ Object

Do we expect some connections to require password authentication? [default: false] Boolean



120
121
122
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 120

def auth_password
  @auth_password
end

#passwords ⇒ Object

Passwords to be used, per node [default: {}] Hash<String, String>



116
117
118
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 116

def passwords
  @passwords
end

#ssh_gateway_user ⇒ Object

Name of the gateway user to be used. [default: ENV or ubradm]

String



96
97
98
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 96

def ssh_gateway_user
  @ssh_gateway_user
end

#ssh_gateways_conf ⇒ Object

Name of the gateways configuration, or nil if no gateway. [default: ENV or nil]

Symbol or nil



100
101
102
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 100

def ssh_gateways_conf
  @ssh_gateways_conf
end

#ssh_strict_host_key_checking ⇒ Object

Do we use strict host key checking in our SSH commands? [default: true] Boolean



108
109
110
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 108

def ssh_strict_host_key_checking
  @ssh_strict_host_key_checking
end

#ssh_use_control_master ⇒ Object

Do we use the control master? [default: true] Boolean



112
113
114
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 112

def ssh_use_control_master
  @ssh_use_control_master
end

#ssh_user ⇒ Object

User name used in SSH connections. [default: ENV or ENV]

String



104
105
106
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 104

def ssh_user
  @ssh_user
end

Instance Method Details

#connectable_nodes_from(nodes) ⇒ Object

Select nodes where this connector can connect.

API

  • This method is mandatory

API

  • @cmd_runner can be used

API

  • @nodes_handler can be used

Parameters

  • nodes (Array<String>): List of candidate nodes

Result

  • Array<String>: List of nodes we can connect to from the candidates



201
202
203
204
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 201

def connectable_nodes_from(nodes)
  @nodes_handler.prefetch_metadata_of nodes, :host_ip
  nodes.select { |node| @nodes_handler.get_host_ip_of(node) }
end

#init ⇒ Object

Initialize the connector. This can be used to initialize global variables that are used for this connector

API

  • This method is optional

API

  • @cmd_runner can be used

API

  • @nodes_handler can be used



130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 130

def init
  # Default values
  @ssh_user = ENV['hpc_ssh_user']
  @ssh_user = ENV['USER'] if @ssh_user.nil? || @ssh_user.empty?
  if @ssh_user.nil? || @ssh_user.empty?
    _exit_status, stdout = @cmd_runner.run_cmd 'whoami', log_to_stdout: log_debug?
    @ssh_user = stdout.strip
  end
  @ssh_use_control_master = true
  @ssh_strict_host_key_checking = true
  @passwords = {}
  @auth_password = false
  @ssh_gateways_conf = ENV['hpc_ssh_gateways_conf'].nil? ? nil : ENV['hpc_ssh_gateways_conf'].to_sym
  @ssh_gateway_user = ENV['hpc_ssh_gateway_user'].nil? ? 'ubradm' : ENV['hpc_ssh_gateway_user']
  # The map of existing ssh directories that have been created, per node that can access them
  # Array< String, Array<String> >
  @ssh_dirs = {}
  # Mutex protecting the map to make sure it's thread-safe
  @ssh_dirs_mutex = Mutex.new
  # Temporary directory used by all ActionsExecutors, even from different processes
  @tmp_dir = "#{Dir.tmpdir}/#{TMP_SSH_SUB_DIR}"
  FileUtils.mkdir_p @tmp_dir
end

#options_parse(options_parser) ⇒ Object

Complete an option parser with options meant to control this connector

API

  • This method is optional

API

  • @cmd_runner can be used

API

  • @nodes_handler can be used

Parameters

  • options_parser (OptionParser): The option parser to complete



161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 161

def options_parse(options_parser)
  options_parser.on('-g', '--ssh-gateway-user USER', "Name of the gateway user to be used by the gateways. Can also be set from environment variable hpc_ssh_gateway_user. Defaults to #{@ssh_gateway_user}.") do |user|
    @ssh_gateway_user = user
  end
  options_parser.on('-j', '--ssh-no-control-master', 'If used, don\'t create SSH control masters for connections.') do
    @ssh_use_control_master = false
  end
  options_parser.on('-q', '--ssh-no-host-key-checking', 'If used, don\'t check for SSH host keys.') do
    @ssh_strict_host_key_checking = false
  end
  options_parser.on('-u', '--ssh-user USER', 'Name of user to be used in SSH connections (defaults to hpc_ssh_user or USER environment variables)') do |user|
    @ssh_user = user
  end
  options_parser.on('-w', '--password', 'If used, then expect SSH connections to ask for a password.') do
    @auth_password = true
  end
  options_parser.on('-y', '--ssh-gateways-conf GATEWAYS_CONF', "Name of the gateways configuration to be used. Can also be set from environment variable hpc_ssh_gateways_conf.") do |gateway|
    @ssh_gateways_conf = gateway.to_sym
  end
end

#remote_bash(bash_cmds) ⇒ Object

Run bash commands on a given node.

API

  • This method is mandatory

API

  • If defined, then with_connection_to has been called before this method.

API

  • @cmd_runner can be used

API

  • @nodes_handler can be used

API

  • @node can be used to access the node on which we execute the remote bash

API

  • @timeout can be used to know when the action should fail

API

  • @stdout_io can be used to send stdout output

API

  • @stderr_io can be used to send stderr output

Parameters

  • bash_cmds (String): Bash commands to execute



239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 239

def remote_bash(bash_cmds)
  ssh_cmd =
    if @nodes_handler.get_ssh_session_exec_of(@node) == false
      # When ExecSession is disabled we need to use stdin directly
      "{ cat | #{ssh_exec} #{ssh_url} -T; } <<'HPC_EOF'\n#{bash_cmds}\nHPC_EOF"
    else
      "#{ssh_exec} #{ssh_url} /bin/bash <<'HPC_EOF'\n#{bash_cmds}\nHPC_EOF"
    end
  # Due to a limitation of Process.spawn, each individual argument is limited to 128KB of size.
  # Therefore we need to make sure that if bash_cmds exceeds MAX_CMD_ARG_LENGTH bytes (considering EOF chars) then we use an intermediary shell script to store the commands.
  if bash_cmds.size > MAX_CMD_ARG_LENGTH
    # Write the commands in a file
    temp_file = "#{Dir.tmpdir}/hpc_temp_cmds_#{Digest::MD5.hexdigest(bash_cmds)}.sh"
    File.open(temp_file, 'w+') do |file|
      file.write ssh_cmd
      file.chmod 0700
    end
    begin
      run_cmd(temp_file)
    ensure
      File.unlink(temp_file)
    end
  else
    run_cmd ssh_cmd
  end
end

#remote_copy(from, to, sudo: false, owner: nil, group: nil) ⇒ Object

Copy a file to the remote node in a directory

API

  • This method is mandatory

API

  • If defined, then with_connection_to has been called before this method.

API

  • @cmd_runner can be used

API

  • @nodes_handler can be used

API

  • @node can be used to access the node on which we execute the remote bash

API

  • @timeout can be used to know when the action should fail

API

  • @stdout_io can be used to send stdout output

API

  • @stderr_io can be used to send stderr output

Parameters

  • from (String): Local file to copy

  • to (String): Remote directory to copy to

  • sudo (Boolean): Do we use sudo on the remote to copy? [default: false]

  • owner (String or nil): Owner to be used when copying the files, or nil for current one [default: nil]

  • group (String or nil): Group to be used when copying the files, or nil for current one [default: nil]



302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 302

def remote_copy(from, to, sudo: false, owner: nil, group: nil)
  if @nodes_handler.get_ssh_session_exec_of(@node) == false
    # We don't have ExecSession, so don't use ssh, but scp instead.
    if sudo
      # We need to first copy the file in an accessible directory, and then sudo mv
      remote_bash('mkdir -p hpc_tmp_scp')
      run_cmd "scp -S #{ssh_exec} #{from} #{ssh_url}:./hpc_tmp_scp"
      remote_bash("#{@nodes_handler.sudo_on(@node)} mv ./hpc_tmp_scp/#{File.basename(from)} #{to}")
    else
      run_cmd "scp -S #{ssh_exec} #{from} #{ssh_url}:#{to}"
    end
  else
    run_cmd <<~EOS
      cd #{File.dirname(from)} && \
      tar \
        --create \
        --gzip \
        --file - \
        #{owner.nil? ? '' : "--owner #{owner}"} \
        #{group.nil? ? '' : "--group #{group}"} \
        #{File.basename(from)} | \
      #{ssh_exec} \
        #{ssh_url} \
        \"#{sudo ? "#{@nodes_handler.sudo_on(@node)} " : ''}tar \
          --extract \
          --gunzip \
          --file - \
          --directory #{to} \
          --owner root \
        \"
    EOS
  end
end

#remote_interactive ⇒ Object

Execute an interactive shell on the remote node

API

  • This method is mandatory

API

  • If defined, then with_connection_to has been called before this method.

API

  • @cmd_runner can be used

API

  • @nodes_handler can be used

API

  • @node can be used to access the node on which we execute the remote bash

API

  • @timeout can be used to know when the action should fail

API

  • @stdout_io can be used to send stdout output

API

  • @stderr_io can be used to send stderr output



275
276
277
278
279
280
281
282
283
284
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 275

def remote_interactive
  interactive_cmd = "#{ssh_exec} #{ssh_url}"
  out interactive_cmd
  # As we're not using run_cmd here, make sure we handle the dry_run switch ourselves
  if @cmd_runner.dry_run
    out 'Won\'t execute interactive shell in dry_run mode'
  else
    system interactive_cmd
  end
end

#ssh_config(ssh_exec: 'ssh', known_hosts_file: nil, nodes: @nodes_handler.known_nodes) ⇒ Object

Get an SSH configuration content giving access to nodes of the platforms with the current configuration

Parameters

  • ssh_exec (String): SSH command to be used [default: ‘ssh’]

  • known_hosts_file (String or nil): Path to the known hosts file, or nil for default [default: nil]

  • nodes (Array<String>): List of nodes to generate the config for [default: @nodes_handler.known_nodes]

Result

  • String: The SSH config



360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 360

def ssh_config(ssh_exec: 'ssh', known_hosts_file: nil, nodes: @nodes_handler.known_nodes)
  config_content = <<~EOS
    ############
    # GATEWAYS #
    ############

    #{@ssh_gateways_conf.nil? || !@config.known_gateways.include?(@ssh_gateways_conf) ? '' : @config.ssh_for_gateway(@ssh_gateways_conf, ssh_exec: ssh_exec, user: @ssh_user)}

    #############
    # ENDPOINTS #
    #############

  EOS

  # Add each node
  # Query for the metadata of all nodes at once
  @nodes_handler.prefetch_metadata_of nodes, %i[private_ips hostname host_ip description]
  nodes.sort.each do |node|
    # Generate the conf for the node
    connection, connection_user, gateway, gateway_user = connection_info_for(node, no_exception: true)
    if connection.nil?
      config_content << "# #{node} - Not connectable using SSH - #{@nodes_handler.get_description_of(node) || ''}\n"
    else
      config_content << "# #{node} - #{connection} - #{@nodes_handler.get_description_of(node) || ''}\n"
      config_content << "Host #{ssh_aliases_for(node).join(' ')}\n"
      config_content << "  Hostname #{connection}\n"
      config_content << "  User \"#{connection_user}\"\n" if connection_user != @ssh_user
      config_content << "  ProxyCommand #{ssh_exec} -q -W %h:%p #{gateway_user}@#{gateway}\n" unless gateway.nil?
      if @passwords.key?(node)
        config_content << "  PreferredAuthentications password\n"
        config_content << "  PubkeyAuthentication no\n"
      end
    end
    config_content << "\n"
  end
  # Add global definitions at the end of the SSH config, as they might be overriden by previous ones, and first match wins.
  config_content << <<~EOS
    ###########
    # GLOBALS #
    ###########

    Host *
      User #{@ssh_user}
      # Default control socket path to be used when multiplexing SSH connections
      ControlPath #{control_master_file('%h', '%p', '%r')}
      #{open_ssh_major_version >= 7 ? 'PubkeyAcceptedKeyTypes +ssh-dss' : ''}
      #{known_hosts_file.nil? ? '' : "UserKnownHostsFile #{known_hosts_file}"}
      #{@ssh_strict_host_key_checking ? '' : 'StrictHostKeyChecking no'}

  EOS
  config_content
end

#ssh_exec ⇒ Object

Get the ssh executable to be used when connecting to the current node

Result

  • String: The ssh executable



340
341
342
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 340

def ssh_exec
  ssh_exec_for @node
end

#ssh_url ⇒ Object

Get the ssh URL to be used to connect to the current node

Result

  • String: The ssh URL connecting to the current node



348
349
350
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 348

def ssh_url
  "hpc.#{@node}"
end

#validate_params ⇒ Object

Validate that parsed parameters are valid

API

  • This method is optional

API

  • @cmd_runner can be used

API

  • @nodes_handler can be used



186
187
188
189
190
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 186

def validate_params
  raise 'No SSH user name specified. Please use --ssh-user option or hpc_ssh_user environment variable to set it.' if @ssh_user.nil? || @ssh_user.empty?
  known_gateways = @config.known_gateways
  raise "Unknown gateway configuration provided: #{@ssh_gateways_conf}. Possible values are: #{known_gateways.join(', ')}." if !@ssh_gateways_conf.nil? && !known_gateways.include?(@ssh_gateways_conf)
end

#with_connection_to(nodes, no_exception: false) ⇒ Object

Prepare connections to a given set of nodes. Useful to prefetch metadata or open bulk connections.

API

  • This method is optional

API

  • @cmd_runner can be used

API

  • @nodes_handler can be used

Parameters

  • nodes (Array<String>): Nodes to prepare the connection to

  • no_exception (Boolean): Should we still continue if some nodes have connection errors? [default: false]

  • Proc: Code called with the connections prepared.

    • Parameters

      • connected_nodes (Array<String>): The list of connected nodes (should be equal to nodes unless no_exception == true and some nodes failed to connect)



218
219
220
221
222
 
# File 'lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb', line 218

def with_connection_to(nodes, no_exception: false)
  with_ssh_master_to(nodes, no_exception: no_exception) do |connected_nodes|
    yield connected_nodes
  end
end