Class: IMAPProcessor

Inherits:
Object
  • Object
show all
Defined in:
lib/imap_processor.rb

Overview

IMAPProcessor is a client for processing messages on an IMAP server.

Subclasses need to provide:

  • A process_args class method that adds any extra options to the default IMAPProcessor options.

  • An initialize method that connects to an IMAP server and sets the @imap instance variable

  • A run method that uses the IMAP connection to process messages.

Reference:

email: http://www.ietf.org/rfc/rfc0822.txt
 imap: http://www.ietf.org/rfc/rfc3501.txt

Direct Known Subclasses

Archive, Client, IDLE, Keywords, Mkdir, Tidy

Defined Under Namespace

Classes: Archive, Cleanse, Client, Connection, Error, Flag, IDLE, Keywords, Learn, Mkdir, Tidy

Constant Summary collapse

VERSION =

The version of IMAPProcessor you are using

"1.8.1"
@@options = 
{}
@@extra_options = 
[]

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options) ⇒ IMAPProcessor

Handles the basic settings from options including verbosity, mailboxes to process, and Net::IMAP::debug



339
340
341
342
343
344
 
# File 'lib/imap_processor.rb', line 339

def initialize(options)
  @options = options
  @verbose = options[:Verbose]
  @boxes = options[:Boxes]
  Net::IMAP.debug = options[:Debug]
end

Instance Attribute Details

#imap ⇒ Object (readonly)

Net::IMAP connection, set this via #initialize



63
64
65
 
# File 'lib/imap_processor.rb', line 63

def imap
  @imap
end

#options ⇒ Object (readonly)

Options Hash from process_args



68
69
70
 
# File 'lib/imap_processor.rb', line 68

def options
  @options
end

Class Method Details

.add_move ⇒ Object

Adds a –move option to the option parser which stores the destination mailbox in the MoveTo option. Call this from a subclass’ process_args method.



78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'lib/imap_processor.rb', line 78

def self.add_move
  @@options[:MoveTo] = nil

  @@extra_options << proc do |opts, options|
    opts.on(      "--move=MAILBOX",
            "Mailbox to move message to",
            "Default: #{options[:MoveTo].inspect}",
            "Options file name: :MoveTo") do |mailbox|
      options[:MoveTo] = mailbox
    end
  end
end

.process_args(processor_file, args, required_options = {}) ⇒ Object

Handles processing of args loading defaults from a file in ~ based on processor_file. Extra option defaults can be specified by required_options. Yields an option parser instance to add new OptionParser options to:

class MyProcessor < IMAPProcessor
  def self.process_args(args)
    required_options = {
      :MoveTo => [nil, "MoveTo not set"],
    }

    super __FILE__, args, required_options do |opts, options|
      opts.banner << "Explain my_processor's executable"

      opts.on(      "--move=MAILBOX",
              "Mailbox to move message to",
              "Default: #{options[:MoveTo].inspect}",
              "Options file name: :MoveTo") do |mailbox|
        options[:MoveTo] = mailbox
      end
    end
  end
end

NOTE: You can add a –move option using ::add_move



118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
 
# File 'lib/imap_processor.rb', line 118

def self.process_args(processor_file, args,
                      required_options = {}) # :yield: OptionParser
  @@opts_file_name = File.basename processor_file, '.rb'
  @@opts_file_name = "imap_#{@@opts_file_name}" unless
    @@opts_file_name =~ /^imap_/
  opts_file = File.expand_path "~/.#{@@opts_file_name}"

  if required_options then
    required_options.each do |option, (default, message)|
      raise ArgumentError,
            "required_options message is missing for #{option}" if
        default.nil? and message.nil?
    end
  end

  defaults = [{}]

  if File.exist? opts_file then
    unless File.stat(opts_file).mode & 077 == 0 then
      $stderr.puts "WARNING! #{opts_file} is group/other readable or writable!"
      $stderr.puts "WARNING! I'm not doing a thing until you fix it!"
      exit 1
    end

    defaults = Array(YAML.load_file(opts_file))
  end

  defaults.map { |default|
    options = default.merge @@options.dup

    options[:SSL]        = true unless options.key? :SSL
    options[:Username] ||= ENV['USER']
    options[:Root]     ||= nil
    options[:Verbose]  ||= false
    options[:Debug]    ||= false

    required_options.each do |k,(v,_)|
      options[k]       ||= v
    end

    op = OptionParser.new do |opts|
      opts.program_name = File.basename $0
      opts.banner = "Usage: #{opts.program_name} [options]\n\n"

      opts.separator ''
      opts.separator 'Connection options:'

      opts.on_tail("-h", "--help", "Show this message") do
        puts opts
        exit
      end

      opts.on("-H", "--host HOST",
              "IMAP server host",
              "Default: #{options[:Host].inspect}",
              "Options file name: :Host") do |host|
        options[:Host] = host
      end

      opts.on("-P", "--port PORT",
              "IMAP server port",
              "Default: The correct port SSL/non-SSL mode",
              "Options file name: :Port") do |port|
        options[:Port] = port
      end

      opts.on("-s", "--[no-]ssl",
              "Use SSL for IMAP connection",
              "Default: #{options[:SSL].inspect}",
              "Options file name: :SSL") do |ssl|
        options[:SSL] = ssl
      end

      opts.on(      "--[no-]debug",
              "Display Net::IMAP debugging info",
              "Default: #{options[:Debug].inspect}",
              "Options file name: :Debug") do |debug|
        options[:Debug] = debug
      end

      opts.separator ''
      opts.separator 'Login options:'

      opts.on("-u", "--username USERNAME",
              "IMAP username",
              "Default: #{options[:Username].inspect}",
              "Options file name: :Username") do |username|
        options[:Username] = username
      end

      opts.on("-p", "--password PASSWORD",
              "IMAP password",
              "Default: Read from ~/.#{@@opts_file_name}",
              "Options file name: :Password") do |password|
        options[:Password] = password
      end

      authenticators = Net::IMAP.authenticators
      auth_types = authenticators.keys.sort.join ', '
      opts.on("-a", "--auth AUTH", auth_types,
              "IMAP authentication type override",
              "Authentication type will be auto-",
              "discovered",
              "Default: #{options[:Auth].inspect}",
              "Options file name: :Auth") do |auth|
        options[:Auth] = auth
      end

      opts.separator ''
      opts.separator "IMAP options:"

      opts.on("-r", "--root ROOT",
              "Root of mailbox hierarchy",
              "Default: #{options[:Root].inspect}",
              "Options file name: :Root") do |root|
        options[:Root] = root
      end

      opts.on("-b", "--boxes BOXES", Array,
              "Comma-separated list of mailbox names",
              "to search",
              "Default: #{options[:Boxes].inspect}",
              "Options file name: :Boxes") do |boxes|
        options[:Boxes] = boxes
      end

      opts.on("-v", "--[no-]verbose",
              "Be verbose",
              "Default: #{options[:Verbose].inspect}",
              "Options file name: :Verbose") do |verbose|
        options[:Verbose] = verbose
      end

      opts.on("-n", "--noop",
              "Perform no destructive operations",
              "Best used with the verbose option",
              "Default: #{options[:Noop].inspect}",
              "Options file name: Noop") do |noop|
        options[:Noop] = noop
      end

      opts.on("-q", "--quiet",
              "Be quiet") do
        options[:Verbose] = false
      end

      if block_given? then
        opts.separator ''
        opts.separator "#{self} options:"

        yield opts, options if block_given?
      end

      @@extra_options.each do |block|
        block.call opts, options
      end

      opts.separator ''

      opts.banner << <<-EOF

Options may also be set in the options file ~/.#{@@opts_file_name}

Example ~/.#{@@opts_file_name}:
\tHost=mail.example.com
\tPassword=my password

      EOF

    end # OptionParser.new do

    op.parse! args.dup

    options[:Port] ||= options[:SSL] ? 993 : 143

    # HACK: removed :Boxes -- push down
    required_keys = [:Host, :Password] + required_options.keys
    if required_keys.any? { |k| options[k].nil? } then
      $stderr.puts op
      $stderr.puts
      $stderr.puts "Host name not set" if options[:Host].nil?
      $stderr.puts "Password not set"  if options[:Password].nil?
      $stderr.puts "Boxes not set"     if options[:Boxes].nil?
      required_options.each do |option_name, (_, missing_message)|
        $stderr.puts missing_message if options[option_name].nil?
      end
      exit 1
    end

    options
  } # defaults.map
end

.run(args = ARGV, &block) ⇒ Object

Sets up an IMAP processor’s options then calls its #run method.



314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
 
# File 'lib/imap_processor.rb', line 314

def self.run(args = ARGV, &block)
  client = nil
  multi_options = process_args args

  multi_options.each do |options|
    client = new(options, &block)
    client.run
  end
rescue Interrupt
  exit
rescue SystemExit
  raise
rescue Exception => e
  $stderr.puts "Failed to finish with exception: #{e.class}:#{e.message}"
  $stderr.puts "\t#{e.backtrace.join "\n\t"}"

  exit 1
ensure
  client.imap.logout if client and client.imap
end

Instance Method Details

#capability(imap, res = nil) ⇒ Object

Extracts capability information for imap from res or by contacting the server.



350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
 
# File 'lib/imap_processor.rb', line 350

def capability imap, res = nil
  return imap.capability unless res

  data = res.data

  if data.code and data.code.name == 'CAPABILITY' then
    case data.code.data
    when Array then
      data.code.data
    when String then
      data.code.data.split ' '
    else
      raise ArgumentError, "unknown type: #{data.code.data.class}"
    end
  else
    imap.capability
  end
end

#connect(host = , port = , ssl = , username = , password = , auth = ) ⇒ Object

Connects to IMAP server host at port using ssl if ssl is true then authenticates with username and password. IMAPProcessor is only known to work with PLAIN auth on SSL sockets. IMAPProcessor does not support LOGIN.

Returns a Connection object.



377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
 
# File 'lib/imap_processor.rb', line 377

def connect(host = @options[:Host],
            port = @options[:Port],
            ssl = @options[:SSL],
            username = @options[:Username],
            password = @options[:Password],
            auth = @options[:Auth]) # :yields: Connection
  imap = Net::IMAP.new host, port, ssl, nil, false
  log "Connected to imap://#{host}:#{port}/"

  capabilities = capability imap, imap.greeting

  log "Capabilities: #{capabilities.join ', '}"

  auth_caps = capabilities.select { |c| c =~ /^AUTH/ }

  if auth.nil? then
    raise "Couldn't find a supported auth type" if auth_caps.empty?
    auth = auth_caps.first.sub(/AUTH=/, '')
  end

  # Net::IMAP supports using AUTHENTICATE with LOGIN, PLAIN, and
  # CRAM-MD5... if the server reports a different AUTH method, then we
  # should fall back to using LOGIN
  if %w( LOGIN PLAIN CRAM-MD5 XOAUTH2 ).include?( auth.upcase )
    auth = auth.upcase
    log "Trying #{auth} authentication"
    res = imap.authenticate auth, username, password
    log "Logged in as #{username} using AUTHENTICATE"
  else
    log "Trying to authenticate via LOGIN"
    res = imap. username, password
    log "Logged in as #{username} using LOGIN"
  end

  # CAPABILITY may have changed
  capabilities = capability imap, res

  connection = Connection.new imap, capabilities

  if block_given? then
    begin
      yield connection
    ensure
      connection.imap.logout
    end
  else
    return connection
  end
end

#create_mailbox(name) ⇒ Object

Create the mailbox name if it doesn’t exist. Note that this will SELECT the mailbox if it exists.



431
432
433
434
435
436
437
 
# File 'lib/imap_processor.rb', line 431

def create_mailbox name
  log "LIST #{name}"
  list = imap.list '', name
  return if list
  log "CREATE #{name}"
  imap.create name unless noop?
end

#delete_messages(uids, expunge = true) ⇒ Object

Delete and expunge the specified uids.



442
443
444
445
446
447
448
449
 
# File 'lib/imap_processor.rb', line 442

def delete_messages uids, expunge = true
  log "DELETING [...#{uids.size} uids]"
  imap.store uids, '+FLAGS.SILENT', [:Deleted] unless noop?
  if expunge then
    log "EXPUNGE"
    imap.expunge unless noop?
  end
end

#each_message(uids, type) ⇒ Object

Yields each uid and message as a TMail::Message for uids of MIME type type.

If there’s an exception raised during handling a message the subject, message-id and inspected body are logged.

If the block returns nil or false, the message is considered skipped and its uid is not returned in the uid list. (Hint: next false unless …)

Returns the uids of successfully handled messages.



463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
 
# File 'lib/imap_processor.rb', line 463

def each_message(uids, type) # :yields: TMail::Mail
  parts = mime_parts uids, type

  uids = []

  each_part parts, true do |uid, message|
    mail = TMail::Mail.parse message

    begin
      success = yield uid, mail

      uids << uid if success
    rescue => e
      log e.message
      puts "\t#{e.backtrace.join "\n\t"}" unless $DEBUG # backtrace at bottom
      log "Subject: #{mail.subject}"
      log "Message-Id: #{mail.message_id}"
      p mail.body if verbose?

      raise if $DEBUG
    end
  end

  uids
end

#each_part(parts, header = false) ⇒ Object

Yields each message part from parts. If header is true, a complete message is yielded, appropriately joined for use with TMail::Mail.



493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
 
# File 'lib/imap_processor.rb', line 493

def each_part(parts, header = false) # :yields: uid, message
  parts.each do |uid, section|
    sequence = ["BODY[#{section}]"]
    sequence.unshift "BODY[#{section}.MIME]" unless section == 'TEXT'
    sequence.unshift 'BODY[HEADER]' if header

    body = imap.fetch(uid, sequence).first

    sequence = sequence.map { |item| body.attr[item] }

    unless section == 'TEXT' and header then
      sequence[0].sub!(/\r\n\z/, '')
    end

    yield uid, sequence.join
  end
end

#log(message) ⇒ Object

Logs message to $stderr if verbose



514
515
516
517
 
# File 'lib/imap_processor.rb', line 514

def log(message)
  return unless @verbose
  $stderr.puts "# #{message}"
end

#mime_parts(uids, mime_type) ⇒ Object

Retrieves the BODY data item name for the mime_type part from messages uids. Returns an array of uid/part pairs. If no matching part with mime_type is found the uid is omitted.

Returns an Array of uid, section pairs.

Use a subsequent Net::IMAP#fetch to retrieve the selected part.



528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
 
# File 'lib/imap_processor.rb', line 528

def mime_parts(uids, mime_type)
  media_type, subtype = mime_type.upcase.split('/', 2)

  structures = imap.fetch uids, 'BODYSTRUCTURE'

  structures.zip(uids).map do |body, uid|
    section = nil
    structure = body.attr['BODYSTRUCTURE']

    case structure
    when Net::IMAP::BodyTypeMultipart then
      parts = structure.parts

      section = parts.each_with_index do |part, index|
        break index if part.media_type == media_type and
                       part.subtype == subtype
      end

      next unless Integer === section
    when Net::IMAP::BodyTypeText, Net::IMAP::BodyTypeBasic then
      section = 'TEXT' if structure.media_type == media_type and
                          structure.subtype == subtype
    end

    [uid, section]
  end.compact
end

#move_messages(uids, destination, expunge = true) ⇒ Object

Move the specified uids to a new destination then delete and expunge them. Creates the destination mailbox if it doesn’t exist.



560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
 
# File 'lib/imap_processor.rb', line 560

def move_messages uids, destination, expunge = true
  return if uids.empty?
  verb = expunge ? "MOVE" : "COPY"
  log "%s %d uids to %s:" % [verb, uids.size, destination]

  begin
    imap.copy uids, destination unless noop?
  rescue Net::IMAP::NoResponseError
    unless noop? then
      create_mailbox destination
      imap.copy uids, destination
    end
  end

  delete_messages uids, expunge
end

#noop? ⇒ Boolean

Did the user set –noop?

Returns:

  • (Boolean) 


604
605
606
 
# File 'lib/imap_processor.rb', line 604

def noop?
  options[:Noop]
end

#show_messages(uids) ⇒ Object

Displays Date, Subject and Message-Id from messages in uids



580
581
582
583
584
585
586
587
588
589
590
591
592
 
# File 'lib/imap_processor.rb', line 580

def show_messages(uids)
  return if uids.nil? or (Array === uids and uids.empty?)

  fetch_data = 'BODY.PEEK[HEADER.FIELDS (DATE FROM TO SUBJECT)]'
  messages = imap.fetch uids, fetch_data
  fetch_data.sub! '.PEEK', '' # stripped by server

  messages ||= []

  messages.each do |res|
    puts res.attr[fetch_data].delete("\r").gsub(/^/, "  ")
  end
end

#verbose? ⇒ Boolean

Did the user set –verbose?

Returns:

  • (Boolean) 


597
598
599
 
# File 'lib/imap_processor.rb', line 597

def verbose?
  @verbose
end