Class: Net::SFTP::Operations::Download

Inherits:
Object
  • Object
show all
Includes:
Net::SSH::Loggable
Defined in:
lib/net/sftp/operations/download.rb

Overview

A general purpose downloader module for Net::SFTP. It can download files into IO objects, or directly to files on the local file system. It can even download entire directory trees via SFTP, and provides a flexible progress reporting mechanism.

To download a single file from the remote server, simply specify both the remote and local paths:

downloader = sftp.download("/path/to/remote.txt", "/path/to/local.txt")

By default, this operates asynchronously, so if you want to block until the download finishes, you can use the ‘bang’ variant:

sftp.download!("/path/to/remote.txt", "/path/to/local.txt")

Or, if you have multiple downloads that you want to run in parallel, you can employ the #wait method of the returned object:

dls = %w(file1 file2 file3).map { |f| sftp.download("remote/#{f}", f) }
dls.each { |d| d.wait }

To download an entire directory tree, recursively, simply specify :recursive => true:

sftp.download!("/path/to/remotedir", "/path/to/local", :recursive => true)

This will download “/path/to/remotedir”, its contents, its subdirectories, and their contents, recursively, to “/path/to/local” on the local host. (If you specify :recursive => true and the source is not a directory, you’ll get an error!)

If you want to pull the contents of a file on the remote server, and store the data in memory rather than immediately to disk, you can pass an IO object as the destination:

require 'stringio'
io = StringIO.new
sftp.download!("/path/to/remote", io)

This will only work for single-file downloads. Trying to do so with :recursive => true will cause an error.

The following options are supported:

  • :progress - either a block or an object to act as a progress callback. See the discussion of “progress monitoring” below.

  • :requests - the number of pending SFTP requests to allow at any given time. When downloading an entire directory tree recursively, this will default to 16. Setting this higher might improve throughput. Reducing it will reduce throughput.

  • :read_size - the maximum number of bytes to read at a time from the source. Increasing this value might improve throughput. It defaults to 32,000 bytes.

Progress Monitoring

Sometimes it is desirable to track the progress of a download. There are two ways to do this: either using a callback block, or a special custom object.

Using a block it’s pretty straightforward:

sftp.download!("remote", "local") do |event, downloader, *args|
  case event
  when :open then
    # args[0] : file metadata
    puts "starting download: #{args[0].remote} -> #{args[0].local} (#{args[0].size} bytes}"
  when :get then
    # args[0] : file metadata
    # args[1] : byte offset in remote file
    # args[2] : data that was received
    puts "writing #{args[2].length} bytes to #{args[0].local} starting at #{args[1]}"
  when :close then
    # args[0] : file metadata
    puts "finished with #{args[0].remote}"
  when :mkdir then
    # args[0] : local path name
    puts "creating directory #{args[0]}"
  when :finish then
    puts "all done!"
  end
end

However, for more complex implementations (e.g., GUI interfaces and such) a block can become cumbersome. In those cases, you can create custom handler objects that respond to certain methods, and then pass your handler to the downloader:

class CustomHandler
  def on_open(downloader, file)
    puts "starting download: #{file.remote} -> #{file.local} (#{file.size} bytes)"
  end

  def on_get(downloader, file, offset, data)
    puts "writing #{data.length} bytes to #{file.local} starting at #{offset}"
  end

  def on_close(downloader, file)
    puts "finished with #{file.remote}"
  end

  def on_mkdir(downloader, path)
    puts "creating directory #{path}"
  end

  def on_finish(downloader)
    puts "all done!"
  end
end

sftp.download!("remote", "local", :progress => CustomHandler.new)

If you omit any of those methods, the progress updates for those missing events will be ignored. You can create a catchall method named “call” for those, instead.

Defined Under Namespace

Classes: Entry

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(sftp, local, remote, options = {}, &progress) ⇒ Download

Instantiates a new downloader process on top of the given SFTP session. local is either an IO object that should receive the data, or a string identifying the target file or directory on the local host. remote is a string identifying the location on the remote host that the download should source.

This will return immediately, and requires that the SSH event loop be run in order to effect the download. (See #wait.)



147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
 
# File 'lib/net/sftp/operations/download.rb', line 147

def initialize(sftp, local, remote, options={}, &progress)
  @sftp = sftp
  @local = local
  @remote = remote
  @progress = progress || options[:progress]
  @options = options
  @active = 0
  @properties = options[:properties] || {}

  self.logger = sftp.logger

  if recursive? && local.respond_to?(:write)
    raise ArgumentError, "cannot download a directory tree in-memory"
  end

  @stack = [Entry.new(remote, local, recursive?)]
  process_next_entry
end

Instance Attribute Details

#local ⇒ Object (readonly)

The destination of the download (the name of a file or directory on the local server, or an IO object)



124
125
126
 
# File 'lib/net/sftp/operations/download.rb', line 124

def local
  @local
end

#options ⇒ Object (readonly)

The hash of options that was given to this Download instance.



131
132
133
 
# File 'lib/net/sftp/operations/download.rb', line 131

def options
  @options
end

#properties ⇒ Object (readonly)

The properties hash for this object



137
138
139
 
# File 'lib/net/sftp/operations/download.rb', line 137

def properties
  @properties
end

#remote ⇒ Object (readonly)

The source of the download (the name of a file or directory on the remote server)



128
129
130
 
# File 'lib/net/sftp/operations/download.rb', line 128

def remote
  @remote
end

#sftp ⇒ Object (readonly)

The SFTP session instance that drives this download.



134
135
136
 
# File 'lib/net/sftp/operations/download.rb', line 134

def sftp
  @sftp
end

Instance Method Details

#[](name) ⇒ Object

Returns the property with the given name. This allows Download instances to store their own state when used as part of a state machine.



193
194
195
 
# File 'lib/net/sftp/operations/download.rb', line 193

def [](name)
  @properties[name.to_sym]
end

#[]=(name, value) ⇒ Object

Sets the given property to the given name. This allows Download instances to store their own state when used as part of a state machine.



199
200
201
 
# File 'lib/net/sftp/operations/download.rb', line 199

def []=(name, value)
  @properties[name.to_sym] = value
end

#abort! ⇒ Object

Forces the transfer to stop.



179
180
181
182
 
# File 'lib/net/sftp/operations/download.rb', line 179

def abort!
  @active = 0
  @stack.clear
end

#active? ⇒ Boolean

Returns true if there are any active requests or pending files or directories.

Returns:

  • (Boolean) 


174
175
176
 
# File 'lib/net/sftp/operations/download.rb', line 174

def active?
  @active > 0 || stack.any?
end

#recursive? ⇒ Boolean

Returns the value of the :recursive key in the options hash that was given when the object was instantiated.

Returns:

  • (Boolean) 


168
169
170
 
# File 'lib/net/sftp/operations/download.rb', line 168

def recursive?
  options[:recursive]
end

#wait ⇒ Object

Runs the SSH event loop for as long as the downloader is active (see #active?). This can be used to block until the download completes.



186
187
188
189
 
# File 'lib/net/sftp/operations/download.rb', line 186

def wait
  sftp.loop { active? }
  self
end