Class: Net::SFTP::Operations::Upload

Inherits:
Object
  • Object
show all
Includes:
Net::SSH::Loggable
Defined in:
lib/net/sftp/operations/upload.rb

Overview

A general purpose uploader module for Net::SFTP. It can upload IO objects, files, and even entire directory trees via SFTP, and provides a flexible progress reporting mechanism.

To upload a single file to the remote server, simply specify both the local and remote paths:

uploader = sftp.upload("/path/to/local.txt", "/path/to/remote.txt")

By default, this operates asynchronously, so if you want to block until the upload finishes, you can use the ‘bang’ variant:

sftp.upload!("/path/to/local.txt", "/path/to/remote.txt")

Or, if you have multiple uploads that you want to run in parallel, you can employ the #wait method of the returned object:

uploads = %w(file1 file2 file3).map { |f| sftp.upload(f, "remote/#{f}") }
uploads.each { |u| u.wait }

To upload an entire directory tree, recursively, simply pass the directory path as the first parameter:

sftp.upload!("/path/to/directory", "/path/to/remote")

This will upload “/path/to/directory”, its contents, its subdirectories, and their contents, recursively, to “/path/to/remote” on the remote server.

For uploading a directory without creating it, do sftp.upload!(“/path/to/directory”, “/path/to/remote”, :mkdir => false)

If you want to send data to a file on the remote server, but the data is in memory, you can pass an IO object and upload its contents:

require 'stringio'
io = StringIO.new(data)
sftp.upload!(io, "/path/to/remote")

The following options are supported:

  • :progress - either a block or an object to act as a progress callback. See the discussion of “progress monitoring” below.

  • :requests - the number of pending SFTP requests to allow at any given time. When uploading an entire directory tree recursively, this will default to 16, otherwise it will default to 2. Setting this higher might improve throughput. Reducing it will reduce throughput.

  • :read_size - the maximum number of bytes to read at a time from the source. Increasing this value might improve throughput. It defaults to 32,000 bytes.

  • :name - the filename to report to the progress monitor when an IO object is given as local. This defaults to “<memory>”.

Progress Monitoring

Sometimes it is desirable to track the progress of an upload. There are two ways to do this: either using a callback block, or a special custom object.

Using a block it’s pretty straightforward:

sftp.upload!("local", "remote") do |event, uploader, *args|
  case event
  when :open then
    # args[0] : file metadata
    puts "starting upload: #{args[0].local} -> #{args[0].remote} (#{args[0].size} bytes}"
  when :put then
    # args[0] : file metadata
    # args[1] : byte offset in remote file
    # args[2] : data being written (as string)
    puts "writing #{args[2].length} bytes to #{args[0].remote} starting at #{args[1]}"
  when :close then
    # args[0] : file metadata
    puts "finished with #{args[0].remote}"
  when :mkdir then
    # args[0] : remote path name
    puts "creating directory #{args[0]}"
  when :finish then
    puts "all done!"
end

However, for more complex implementations (e.g., GUI interfaces and such) a block can become cumbersome. In those cases, you can create custom handler objects that respond to certain methods, and then pass your handler to the uploader:

class CustomHandler
  def on_open(uploader, file)
    puts "starting upload: #{file.local} -> #{file.remote} (#{file.size} bytes)"
  end

  def on_put(uploader, file, offset, data)
    puts "writing #{data.length} bytes to #{file.remote} starting at #{offset}"
  end

  def on_close(uploader, file)
    puts "finished with #{file.remote}"
  end

  def on_mkdir(uploader, path)
    puts "creating directory #{path}"
  end

  def on_finish(uploader)
    puts "all done!"
  end
end

sftp.upload!("local", "remote", :progress => CustomHandler.new)

If you omit any of those methods, the progress updates for those missing events will be ignored. You can create a catchall method named “call” for those, instead.

Defined Under Namespace

Classes: LiveFile

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(sftp, local, remote, options = {}, &progress) ⇒ Upload

Instantiates a new uploader process on top of the given SFTP session. local is either an IO object containing data to upload, or a string identifying a file or directory on the local host. remote is a string identifying the location on the remote host that the upload should target.

This will return immediately, and requires that the SSH event loop be run in order to effect the upload. (See #wait.)



143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
 
# File 'lib/net/sftp/operations/upload.rb', line 143

def initialize(sftp, local, remote, options={}, &progress) #:nodoc:
  @sftp = sftp
  @local = local
  @remote = remote
  @progress = progress || options[:progress]
  @options = options
  @properties = options[:properties] || {}
  @active = 0

  self.logger = sftp.logger

  @uploads = []
  @recursive = local.respond_to?(:read) ? false : ::File.directory?(local)

  if recursive?
    @stack = [entries_for(local)]
    @local_cwd = local
    @remote_cwd = remote

    @active += 1
    if @options[:mkdir]
      sftp.mkdir(remote) do |response|
        @active -= 1
        raise StatusException.new(response, "mkdir `#{remote}'") unless response.ok?
        (options[:requests] || RECURSIVE_READERS).to_i.times do
          break unless process_next_entry
        end
      end
    else
      @active -= 1
      process_next_entry
    end
  else
    raise ArgumentError, "expected a file to upload" unless local.respond_to?(:read) || ::File.exist?(local)
    @stack = [[local]]
    process_next_entry
  end
end

Instance Attribute Details

#local ⇒ Object (readonly)

The source of the upload (on the local server)



121
122
123
 
# File 'lib/net/sftp/operations/upload.rb', line 121

def local
  @local
end

#options ⇒ Object (readonly)

The hash of options that were given when the object was instantiated



127
128
129
 
# File 'lib/net/sftp/operations/upload.rb', line 127

def options
  @options
end

#properties ⇒ Object (readonly)

The properties hash for this object



133
134
135
 
# File 'lib/net/sftp/operations/upload.rb', line 133

def properties
  @properties
end

#remote ⇒ Object (readonly)

The destination of the upload (on the remote server)



124
125
126
 
# File 'lib/net/sftp/operations/upload.rb', line 124

def remote
  @remote
end

#sftp ⇒ Object (readonly)

The SFTP session object used by this upload instance



130
131
132
 
# File 'lib/net/sftp/operations/upload.rb', line 130

def sftp
  @sftp
end

Instance Method Details

#[](name) ⇒ Object

Returns the property with the given name. This allows Upload instances to store their own state when used as part of a state machine.



209
210
211
 
# File 'lib/net/sftp/operations/upload.rb', line 209

def [](name)
  @properties[name.to_sym]
end

#[]=(name, value) ⇒ Object

Sets the given property to the given name. This allows Upload instances to store their own state when used as part of a state machine.



215
216
217
 
# File 'lib/net/sftp/operations/upload.rb', line 215

def []=(name, value)
  @properties[name.to_sym] = value
end

#abort! ⇒ Object

Forces the transfer to stop.



195
196
197
198
199
 
# File 'lib/net/sftp/operations/upload.rb', line 195

def abort!
  @active = 0
  @stack.clear
  @uploads.clear
end

#active? ⇒ Boolean

Returns true if the uploader is currently running. When this is false, the uploader has finished processing.

Returns:

  • (Boolean) 


190
191
192
 
# File 'lib/net/sftp/operations/upload.rb', line 190

def active?
  @active > 0 || @stack.any?
end

#recursive? ⇒ Boolean

Returns true if a directory tree is being uploaded, and false if only a single file is being uploaded.

Returns:

  • (Boolean) 


184
185
186
 
# File 'lib/net/sftp/operations/upload.rb', line 184

def recursive?
  @recursive
end

#wait ⇒ Object

Blocks until the upload has completed.



202
203
204
205
 
# File 'lib/net/sftp/operations/upload.rb', line 202

def wait
  sftp.loop { active? }
  self
end