Module: Maid::Tools

Includes:
Deprecated
Defined in:
lib/maid/tools.rb

Overview

These "tools" are methods available in the Maid DSL.

In general, methods are expected to:

  • Automatically expand paths (that is, '~/Downloads/foo.zip' becomes '/home/username/Downloads/foo.zip')
  • Respect the noop (dry-run) option if it is set

Some methods are not available on all platforms. An ArgumentError is raised when a command is not available. See tags such as: [Mac OS X]

Instance Method Summary (collapse)

Instance Method Details

- (Object) accessed_at(path)

Get the time that a file was last accessed.

In Unix speak, atime.

Examples

accessed_at('foo.zip') # => Sat Apr 09 10:50:01 -0400 2011


459
460
461
# File 'lib/maid/tools.rb', line 459

def accessed_at(path)
  File.atime(expand(path))
end

- (Object) checksum_of(path)

Get a checksum for a file.

Examples

checksum_of('foo.zip') # => "ae8dbb203dfd560158083e5de90969c2"


497
498
499
# File 'lib/maid/tools.rb', line 497

def checksum_of(path)
  Digest::MD5.hexdigest(File.read(path))
end

- (Object) content_types(path)

Get the content types of a path.

Content types can be MIME types, Internet media types or Spotlight content types (OS X only).

Examples

content_types('foo.zip') # => ["public.zip-archive", "com.pkware.zip-archive", "public.archive", "application/zip", "application"]
content_types('bar.jpg') # => ["public.jpeg", "public.image", "image/jpeg", "image"]


587
588
589
# File 'lib/maid/tools.rb', line 587

def content_types(path)
  [spotlight_content_types(path), mime_type(path), media_type(path)].flatten
end

- (Object) created_at(path)

Get the creation time of a file.

In Unix speak, ctime.

Examples

created_at('foo.zip') # => Sat Apr 09 10:50:01 -0400 2011


448
449
450
# File 'lib/maid/tools.rb', line 448

def created_at(path)
  File.ctime(expand(path))
end

- (Object) dir(globs)

Give all files matching the given glob.

Note that the globs are not regexps (they're closer to shell globs). However, some regexp-like notation can be used, e.g. ?, [a-z], {tgz,zip}. For more details, see Ruby's documentation on Dir.glob.

The matches are sorted lexically to aid in readability when using --dry-run.

Examples

Single glob:

dir('~/Downloads/*.zip')

Specifying multiple extensions succinctly:

dir('~/Downloads/*.{exe,deb,dmg,pkg,rpm}')

Multiple glob (all are equivalent):

dir(['~/Downloads/*.zip', '~/Dropbox/*.zip'])
dir(%w(~/Downloads/*.zip ~/Dropbox/*.zip))
dir('~/{Downloads,Dropbox}/*.zip')

Recursing into subdirectories (see also: find):

dir('~/Music/**/*.m4a')


208
209
210
211
212
213
# File 'lib/maid/tools.rb', line 208

def dir(globs)
  expand_all(globs).
    map { |glob| Dir.glob(glob) }.
    flatten.
    sort
end

- (Object) dir_safe(globs)

Same as dir, but excludes files that are (possibly) being downloaded.

Example

Move Debian/Ubuntu packages that are finished downloading into a software directory.

move dir_safe('~/Downloads/*.deb'), '~/Archive/Software'


224
225
226
227
# File 'lib/maid/tools.rb', line 224

def dir_safe(globs)
  dir(globs).
    reject { |path| downloading?(path) }
end

- (Object) disk_usage(path)

Calculate disk usage of a given path in kilobytes.

See also: Maid::NumericExtensions::SizeToKb.

Examples

disk_usage('foo.zip') # => 136


429
430
431
432
433
434
435
436
437
438
439
# File 'lib/maid/tools.rb', line 429

def disk_usage(path)
  raw = cmd("du -s #{ sh_escape(path) }")
  # FIXME: This reports in kilobytes, but should probably report in bytes.
  usage_kb = raw.split(/\s+/).first.to_i
 
  if usage_kb.zero?
    raise "Stopping pessimistically because of unexpected value from du (#{ raw.inspect })"
  else
    usage_kb
  end
end

- (Object) downloaded_from(path)

[Mac OS X] Use Spotlight metadata to determine the site from which a file was downloaded.

Examples

downloaded_from('foo.zip') # => ['http://www.site.com/foo.zip', 'http://www.site.com/']


323
324
325
# File 'lib/maid/tools.rb', line 323

def downloaded_from(path)
  mdls_to_array(path, 'kMDItemWhereFroms')
end

- (Boolean) downloading?(path)

Detect whether the path is currently being downloaded in Chrome or Firefox.

See also: dir_safe

Returns:

  • (Boolean)


330
331
332
# File 'lib/maid/tools.rb', line 330

def downloading?(path)
  chrome_downloading?(path) || firefox_downloading?(path)
end

- (Object) dupes_in(globs)

Find all duplicate files in the given globs.

More often than not, you'll want to use newest_dupes_in or verbose_dupes_in instead of using this method directly.

Globs are expanded as in dir, then all non-files are filtered out. The remaining files are compared by size, and non-dupes are filtered out. The remaining candidates are then compared by checksum. Dupes are returned as an array of arrays.

Examples

dupes_in('~/{Downloads,Desktop}/*') # => [
                                           ['~/Downloads/foo.zip', '~/Downloads/foo (1).zip'],
                                           ['~/Desktop/bar.txt', '~/Desktop/bar copy.txt']
                                         ]

Keep the newest dupe:

dupes_in('~/Desktop/*', '~/Downloads/*').each do |dupes|
  trash dupes.sort_by { |p| File.mtime(p) }[0..-2]
end


357
358
359
360
361
362
363
364
365
366
367
368
369
# File 'lib/maid/tools.rb', line 357

def dupes_in(globs)
  dupes = []
  files(globs).                           # Start by filtering out non-files
    group_by { |f| size_of(f) }.          # ... then grouping by size, since that's fast
    reject { |s, p| p.length < 2 }.       # ... and filter out any non-dupes
    map do |size, candidates|
      dupes += candidates.
        group_by { |p| checksum_of(p) }.  # Now group our candidates by a slower checksum calculation
        reject { |c, p| p.length < 2 }.   # ... and filter out any non-dupes
        values
    end
  dupes
end

- (Object) duration_s(path)

[Mac OS X] Use Spotlight metadata to determine audio length.

Examples

duration_s('foo.mp3') # => 235.705


406
407
408
# File 'lib/maid/tools.rb', line 406

def duration_s(path)
  cmd("mdls -raw -name kMDItemDurationSeconds #{ sh_escape(path) }").to_f
end

- (Object) escape_glob(glob)

Escape characters that have special meaning as a part of path global patterns.

Useful when using dir with file names that may contain { } [ ] characters.

Example

escape_glob('test [tmp]') # => 'test \\[tmp\\]'


245
246
247
# File 'lib/maid/tools.rb', line 245

def escape_glob(glob)
  glob.gsub(/[\{\}\[\]]/) { |s| '\\' + s }
end

- (Object) files(globs)

Give only files matching the given glob.

This is the same as dir but only includes actual files (no directories or symlinks).



233
234
235
236
# File 'lib/maid/tools.rb', line 233

def files(globs)
  dir(globs).
    select { |f| File.file?(f) }
end

- (Object) find(path, &block)

Find matching files, akin to the Unix utility find.

If no block is given, it will return an array. Otherwise, it acts like Find.find.

Examples

Without a block:

find('~/Downloads/') # => [...]

Recursing and filtering using a regular expression:

find('~/Downloads/').grep(/\.pdf$/)

(Note: It's just Ruby, so any methods in Array and Enumerable can be used.)

Recursing with a block:

find('~/Downloads/') do |path|
  # ...
end


297
298
299
300
301
302
303
304
305
# File 'lib/maid/tools.rb', line 297

def find(path, &block)
  expanded_path = expand(path)

  if block.nil?
    Find.find(expanded_path).to_a
  else
    Find.find(expanded_path, &block)
  end
end

- (Object) git_piston(path)

Deprecated.

Pull and push the git repository at the given path.

Since this is deprecated, you might also be interested in SparkleShare, a great git-based file syncronization project.

Examples

git_piston('~/code/projectname')


511
512
513
514
515
# File 'lib/maid/tools.rb', line 511

def git_piston(path)
  full_path = expand(path)
  stdout = cmd("cd #{ sh_escape(full_path) } && git pull && git push 2>&1")
  log("Fired git piston on #{ sh_escape(full_path) }.  STDOUT:\n\n#{ stdout }")
end

- (Object) last_accessed(path)

Deprecated.

Alias of accessed_at.



466
467
468
469
# File 'lib/maid/tools.rb', line 466

def last_accessed(path)
  # Not a normal `alias` so the deprecation notice shows in the docs.
  accessed_at(path)
end

- (Object) locate(name)

[Mac OS X] Use Spotlight to locate all files matching the given filename.

[Ubuntu] Use locate to locate all files matching the given filename.

Examples

locate('foo.zip') # => ['/a/foo.zip', '/b/foo.zip']


314
315
316
# File 'lib/maid/tools.rb', line 314

def locate(name)
  cmd("#{Maid::Platform::Commands.locate} #{ sh_escape(name) }").split("\n")
end

- (Object) media_type(path)

Get the Internet media type of the file.

In other words, the first part of mime_type.

Examples

media_type('bar.jpg') # => "image"


611
612
613
614
615
616
617
# File 'lib/maid/tools.rb', line 611

def media_type(path)
  type = MIME::Types.type_for(path)[0]
  
  if type
    type.media_type
  end
end

- (Object) mime_type(path)

Get the MIME type of the file.

Examples

mime_type('bar.jpg') # => "image/jpeg"


596
597
598
599
600
601
602
# File 'lib/maid/tools.rb', line 596

def mime_type(path)
  type = MIME::Types.type_for(path)[0]

  if type
    [type.media_type, type.sub_type].join('/')
  end
end

- (Object) mkdir(path, options = {})

Create a directory and all of its parent directories.

The path of the created directory is returned, which allows for chaining (see examples).

Options

:mode

The symbolic and absolute mode can both be used, for example: 0700, 'u=wr,go=rr'

Examples

Creating a directory with a specific mode:

mkdir('~/Music/Pink Floyd/', :mode => 0644)

Ensuring a directory exists when moving:

move('~/Downloads/Pink Floyd*.mp3', mkdir('~/Music/Pink Floyd/'))


268
269
270
271
272
273
# File 'lib/maid/tools.rb', line 268

def mkdir(path, options = {})
  path = expand(path)
  log("mkdir -p #{ sh_escape(path) }")
  FileUtils.mkdir_p(path, @file_options.merge(options))
  path
end

- (Object) modified_at(path)

Get the modification time of a file.

In Unix speak, mtime.

Examples

modified_at('foo.zip') # => Sat Apr 09 10:50:01 -0400 2011


479
480
481
# File 'lib/maid/tools.rb', line 479

def modified_at(path)
  File.mtime(expand(path))
end

- (Object) move(sources, destination)

Move sources to a destination directory.

Movement is only allowed to directories that already exist. If your intention is to rename, see the rename method.

Examples

Single path:

move('~/Downloads/foo.zip', '~/Archive/Software/Mac OS X/')

Multiple paths:

move(['~/Downloads/foo.zip', '~/Downloads/bar.zip'], '~/Archive/Software/Mac OS X/')
move(dir('~/Downloads/*.zip'), '~/Archive/Software/Mac OS X/')


36
37
38
39
40
41
42
43
44
45
46
47
48
# File 'lib/maid/tools.rb', line 36

def move(sources, destination)
  destination = expand(destination)

  if File.directory?(destination)
    expand_all(sources).each do |source|
      log("move #{ sh_escape(source) } #{ sh_escape(destination) }")
      FileUtils.mv(source, destination, @file_options)
    end
  else
    # Unix `mv` warns about the target not being a directory with multiple sources.  Maid checks the same.
    warn("skipping move because #{ sh_escape(destination) } is not a directory (use 'mkdir' to create first, or use 'rename')")
  end
end

- (Object) newest_dupes_in(globs)

Convenience method that is like dupes_in but excludes the oldest dupe.

Example

Keep the oldest dupe (trash the others):

trash newest_dupes_in('~/Downloads/*')


379
380
381
382
383
# File 'lib/maid/tools.rb', line 379

def newest_dupes_in(globs)
  dupes_in(globs).
    map { |dupes| dupes.sort_by { |p| File.mtime(p) }[1..-1] }.
    flatten
end

- (Object) remove(paths, options = {})

Delete the files at the given path recursively.

NOTE: In most cases, trash is a safer choice, since the files will be recoverable by retreiving them from the trash. Once you delete a file using remove, it's gone! Please use trash whenever possible and only use remove when necessary.

Options

:force => boolean

Force deletion (no error is raised if the file does not exist).

:secure => boolean

Infrequently needed. See FileUtils.remove_entry_secure

Examples

Single path:

remove('~/Downloads/foo.zip')

Multiple path:

remove(['~/Downloads/foo.zip', '~/Downloads/bar.zip'])
remove(dir('~/Downloads/*.zip'))


172
173
174
175
176
177
178
179
# File 'lib/maid/tools.rb', line 172

def remove(paths, options = {})
  expand_all(paths).each do |path|
    options = @file_options.merge(options)

    log("Removing #{ sh_escape(path) }")
    FileUtils.rm_r(path, options)
  end
end

- (Object) rename(source, destination)

Rename a single file.

Any directories needed in order to complete the rename are made automatically.

Overwriting is not allowed; it logs a warning. If overwriting is desired, use remove to delete the file first, then use rename.

Examples

Simple rename:

rename('foo.zip', 'baz.zip') # "foo.zip" becomes "baz.zip"

Rename needing directories:

rename('foo.zip', 'bar/baz.zip') # "bar" is created, "foo.zip" becomes "baz.zip" within "bar"

Attempting to overwrite:

rename('foo.zip', 'existing.zip') # "skipping move of..."


69
70
71
72
73
74
75
76
77
78
79
80
81
# File 'lib/maid/tools.rb', line 69

def rename(source, destination)
  source = expand(source)
  destination = expand(destination)

  mkdir(File.dirname(destination))

  if File.exist?(destination)
    warn("skipping rename of #{ sh_escape(source) } to #{ sh_escape(destination) } because it would overwrite")
  else
    log("rename #{ sh_escape(source) } #{ sh_escape(destination) }")
    FileUtils.mv(source, destination, @file_options)
  end
end

- (Object) size_of(path)

Get the size of a file.

Examples

size_of('foo.zip') # => 2193


488
489
490
# File 'lib/maid/tools.rb', line 488

def size_of(path)
  File.size(path)
end

- (Object) spotlight_content_types(path)

[Mac OS X] Use Spotlight metadata to determine which content types a file has.

Examples

spotlight_content_types('foo.zip') # => ['public.zip-archive', 'public.archive']


575
576
577
# File 'lib/maid/tools.rb', line 575

def spotlight_content_types(path)
  mdls_to_array(path, 'kMDItemContentTypeTree')
end

- (Object) sync(from, to, options = {})

Simple sync two files/folders using rsync.

The host OS must provide rsync. See the rsync man page for a detailed description.

man rsync

Options

:delete => boolean :verbose => boolean :archive => boolean (default true) :update => boolean (default true) :exclude => string :prune_empty => boolean

Examples

Syncing a directory to a backup:

sync('~/music', '/backup/music')

Excluding a path:

sync('~/code', '/backup/code', :exclude => '.git')

Excluding multiple paths:

sync('~/code', '/backup/code', :exclude => ['.git', '.rvmrc'])


547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
# File 'lib/maid/tools.rb', line 547

def sync(from, to, options = {})
  # expand removes trailing slash
  # cannot use str[-1] due to ruby 1.8.7 restriction
  from = expand(from) + (from.end_with?('/') ? '/' : '')
  to = expand(to) + (to.end_with?('/') ? '/' : '')
  # default options
  options = { :archive => true, :update => true }.merge(options)
  ops = []
  ops << '-a' if options[:archive]
  ops << '-v' if options[:verbose]
  ops << '-u' if options[:update]
  ops << '-m' if options[:prune_empty]
  ops << '-n' if @file_options[:noop]

  Array(options[:exclude]).each do |path|
    ops << "--exclude=#{ sh_escape(path) }"
  end

  ops << '--delete' if options[:delete]
  stdout = cmd("rsync #{ ops.join(' ') } #{ sh_escape(from) } #{ sh_escape(to) } 2>&1")
  log("Fired sync from #{ sh_escape(from) } to #{ sh_escape(to) }.  STDOUT:\n\n#{ stdout }")
end

- (Object) trash(paths, options = {})

Move the given paths to the user's trash.

The path is still moved if a file already exists in the trash with the same name. However, the current date and time is appended to the filename.

Note: the OS-native "restore" or "put back" functionality for trashed files is not currently supported. (See issue #63.) However, they can be restored manually, and the Maid log can help assist with this.

Options

:remove_over => Fixnum (e.g. 1.gigabyte, 1024.megabytes)

Delete files over the given size rather than moving to the trash.

See also Maid::NumericExtensions::SizeToKb

Examples

Single path:

trash('~/Downloads/foo.zip')

Multiple paths:

trash(['~/Downloads/foo.zip', '~/Downloads/bar.zip'])
trash(dir('~/Downloads/*.zip'))


110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
# File 'lib/maid/tools.rb', line 110

def trash(paths, options = {})
  # ## Implementation Notes
  #
  # Trashing files correctly is surprisingly hard.  What Maid ends up doing is one the easiest, most foolproof
  # solutions:  moving the file.
  #
  # Unfortunately, that means it's not possile to restore files automatically in OSX or Ubuntu.  The previous location
  # of the file is lost.
  #
  # OSX support depends on AppleScript or would require a not-yet-written C extension to interface with the OS.  The
  # AppleScript solution is less than ideal: the user has to be logged in, Finder has to be running, and it makes the
  # "trash can sound" every time a file is moved.
  #
  # Ubuntu makes it easy to implement, and there's a Python library for doing so (see `trash-cli`).  However, there's
  # not a Ruby equivalent yet.

  expand_all(paths).each do |path|
    target = File.join(@trash_path, File.basename(path))
    safe_trash_path = File.join(@trash_path, "#{ File.basename(path) } #{ Time.now.strftime('%Y-%m-%d-%H-%M-%S') }")

    if options[:remove_over] &&
        File.exist?(path) &&
        disk_usage(path) > options[:remove_over]
      remove(path)
    end

    if File.exist?(path)
      if File.exist?(target)
        rename(path, safe_trash_path)
      else
        move(path, @trash_path)
      end
    end
  end
end

- (Object) verbose_dupes_in(globs)

Convenience method for dupes_in that excludes the dupe with the shortest name.

This is ideal for dupes like foo.zip, foo (1).zip, foo copy.zip.

Example

Keep the dupe with the shortest name (trash the others):

trash verbose_dupes_in('~/Downloads/*')


395
396
397
398
399
# File 'lib/maid/tools.rb', line 395

def verbose_dupes_in(globs)
  dupes_in(globs).
    map { |dupes| dupes.sort_by { |p| File.basename(p).length }[1..-1] }.
    flatten
end

- (Object) where_content_type(paths, filter_types)

Filter an array by content types.

Content types can be MIME types, internet media types or Spotlight content types (OS X only).

If you need your rules to work on multiple platforms, it's recommended to avoid using Spotlight content types.

Examples

Using media types

where_content_type(dir('~/Downloads/*'), 'video')
where_content_type(dir('~/Downloads/*'), ['image', 'audio'])

Using MIME types

where_content_type(dir('~/Downloads/*'), 'image/jpeg')

Using Spotlight content types

Less portable, but richer data in some cases.

where_content_type(dir('~/Downloads/*'), 'public.image')


641
642
643
644
# File 'lib/maid/tools.rb', line 641

def where_content_type(paths, filter_types)
  filter_types = Array(filter_types)
  Array(paths).select { |p| !(filter_types & content_types(p)).empty? }
end

- (Object) zipfile_contents(path)

List the contents of a zip file.

Examples

zipfile_contents('foo.zip') # => ['foo.exe', 'README.txt', 'subdir/anything.txt']


415
416
417
418
419
420
# File 'lib/maid/tools.rb', line 415

def zipfile_contents(path)
  # It might be nice to use `glob` from `Zip::FileSystem`, but it seems buggy.  (Subdirectories aren't included.)
  Zip::File.open(path) do |zip_file|
    zip_file.entries.map { |entry| entry.name }.sort
  end
end