Class: Himawari::Download

Inherits:
Base
  • Object
show all
Extended by:
OsUtils
Includes:
NetUtils, Process
Defined in:
lib/himawari/download.rb

Overview

Let’s expand our basic Himawari class with the ability to actually download stuff in addition to the downloading, we will also need to:

- parse CLI arguments / set crontab (OsUtils)
- check network status (NetUtils)
- check and try to fix downloaded picture Tiles (Process)

Instance Attribute Summary

Attributes inherited from Base

#app_root, #blacklist_wifi, #by_schedule, #cron_action, #data_path, #destination_path, #focus, #latest_local, #latest_remote, #mode, #now, #resolution, #verbose, #work_path

Instance Method Summary collapse

Methods included from OsUtils

crontab, os, parse_cli_args, scriptify_sys, tenmin

Methods included from NetUtils

#internet_connection?

Methods inherited from Base

#background, #crontab, #initialize, #params_valid?, #up_to_date?, #update_backgrnd

Constructor Details

This class inherits a constructor from Himawari::Base

Instance Method Details

#pic(timestamp, tenmin = '{0..5}') ⇒ true, false

Sometimes is called from the ‘pics` method. Downloads one pic from himawari (in many tiles) and then reassembles the pieces back into one image `parallel –header : ’montage -mode concatenate -tile 2x 00,10,01,11/year-mo-dyThrOsUtils.tenmin000-*.png

full/{year}-{mo}-{dy}T{hr}{tenmin}000.png' ::: year 2015 ::: mo 11 ::: dy {27..28} ::: hr {00..23} ::: tenmin {0..5}`

Parameters:

  • timestamp (DateTime)
  • tenmin (String) (defaults to: '{0..5}') —

    this is the crucial ending bit for himawari website that is used by parallel. 10 minutes gets converted into 1; 40mins -> 4, etc.

Returns:

  • (true, false) —

    depending on the success/failure of the download



67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/himawari/download.rb', line 67

def pic(timestamp, tenmin = '{0..5}')
  return false unless everything_ok

  if timestamp > latest_remote
    puts "Can't download #{timestamp} because it is newer than the most recent available (#{latest_remote})"
    return false
  end

  yr = timestamp.year.to_s.rjust(2, '0')
  mo = timestamp.month.to_s.rjust(2, '0')
  dy = timestamp.day.to_s.rjust(2, '0')
  hr = timestamp.hour.to_s.rjust(2, '0')
  x, y = download_region

  command1 = 'parallel -j5 --delay 0.1 --header : '\
             "'curl -sC - \"#{HIMAWARI_URL}/"\
             "#{resolution}d/550/{year}/{mo}/{dy}/{hr}{tenmin}000_{x}_{y}.png\" > "\
             "#{data_path}/t_{year}-{mo}-{dy}T{hr}{tenmin}0-{y}_{x}.png'"\
             " ::: year #{yr} ::: mo #{mo} ::: dy #{dy} ::: hr #{hr} ::: tenmin #{tenmin} ::: x #{x} ::: y #{y}"
  command2 = 'parallel --header : '\
             "'montage -mode concatenate -tile #{resolution}x #{data_path}/t_{year}-{mo}-{dy}T{hr}{tenmin}0-*.png "\
             "#{data_path}/h_{year}-{mo}-{dy}T{hr}{tenmin}0.png'"\
             " ::: year #{yr} ::: mo #{mo} ::: dy #{dy} ::: hr #{hr} ::: tenmin #{tenmin}"
  # command3 = 'parallel convert -channel R -gamma 1.2 -channel G -gamma 1.1 +channel -sigmoidal-contrast 3,50% {} '\
  #            "pretty/{/} ::: #{data_path}/*.png"

  # system(command1) # works totally fine on mac...
  # OOOK>> I have NO idea why linux is not parsing {..} from parallel above and instead sticks it as a literal,
  # but we need to make it work >> hack it this way for the time being...
  script = "#{data_path}/script_#{yr}_#{yr}_#{mo}_#{dy}_#{hr}.sh"
  OsUtils.scriptify_sys(script, command1)

  check_tiles

  # system(command2) # works totally fine on mac...
  OsUtils.scriptify_sys(script, command2)
  `rm #{data_path}/t_*`
  true
end

#pics(from_datetime, to_datetime) ⇒ true, false

the public method to download pics in w/in the datetime range

Parameters:

  • from_datetime (DateTime)
  • to_datetime (DateTime)

Returns:

  • (true, false) —

    depending on the success/failure of the download operation



28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
 
# File 'lib/himawari/download.rb', line 28

def pics(from_datetime, to_datetime)
  # returns true if any new pictures were downloaded from the web, false otherwise
  pic_dwnlded = false
  from_datetime += 10 * 60 # +10.minutes from our latest pic on local drive

  while from_datetime <= to_datetime
    tenmin = if last_iter?(from_datetime, to_datetime)
               OsUtils.tenmin(from_datetime.min, to_datetime.min)
             elsif !pic_dwnlded # i.e. it's while's first iteration. don't update the whole hour, only the remaining minutes
               OsUtils.tenmin(from_datetime.min)
             else
               OsUtils.tenmin
             end

    pic_dwnlded = pic(from_datetime, tenmin)

    break if last_iter?(from_datetime, to_datetime) || !pic_dwnlded

    from_datetime = if to_datetime - from_datetime > 3600
                      from_datetime + 3600
                    else
                      Time.new(to_datetime.year, to_datetime.month, to_datetime.day, to_datetime.hour, 0, 0, '+00:00')
                    end
  end

  if verbose
    puts pic_dwnlded ? "Latest Fetched:  #{find_latest_local[:timestamp]}" : 'Nothing downloaded.'
  end
  pic_dwnlded
end

#start ⇒ Object

go go go

Returns:

  • nothing useful



16
17
18
19
20
21
22
 
# File 'lib/himawari/download.rb', line 16

def start
  return false unless everything_ok && pics_updated?

  set_background(latest_local[:filename], destination_path) if mode == :live
  # clean up: remove any files that are more than 2 days old
  `find #{data_path} -name \"*.png\" -type f -mtime +2 -exec rm -f {} \\;`
end