Module: RUtilAnts::URLCache::URLCacheInterface

Included in:
URLCache
Defined in:
lib/rUtilAnts/URLCache.rb

Instance Method Summary collapse

Instance Method Details

#get_url_content(iURL, iParameters = {}) ⇒ Object

Get a content from a URL. Here are the different formats the URL can have:

  • Local file name

  • http/https/ftp/ftps:// protocols

  • data:image URI

  • file:// protocol

It also handles redirections or zipped files

Parameters

  • iURL (String): The URL

  • iParameters (map<Symbol,Object>): Additional parameters:

    • :force_load (Boolean): Do we force to refresh the cache ? [optional = false]

    • :follow_redirections (Boolean): Do we follow redirections ? [optional = true]

    • :nbr_redirections_allowed (Integer): Number of redirections allowed [optional = 10]

    • :local_file_access (Boolean): Do we need a local file to read the content from ? If not, the content itslef will be given the code block. [optional = false]

  • CodeBlock: The code returning the object corresponding to the content:

    • iContent (String): File content, or file name if :local_file_access was true

    • Return

    • Object: Object read from the content, or nil in case of error

    • Exception: The error encountered, or nil in case of success

Return

  • Object: The corresponding URL content, or nil in case of failure

  • Exception: The error, or nil in case of success



45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
 
# File 'lib/rUtilAnts/URLCache.rb', line 45

def get_url_content(iURL, iParameters = {})
  rObject = nil
  rError = nil

  # Parse parameters
  lForceLoad = iParameters[:force_load]
  if (lForceLoad == nil)
    lForceLoad = false
  end
  # Get the URL handler corresponding to this URL
  lURLHandler = get_url_handler(iURL)
  lServerID = lURLHandler.get_server_id
  if (@HostsDown.has_key?(lServerID))
    rError = ServerDownError.new("Server #{iURL} is currently down.")
  else
    lURLHash = iURL.hash
    # Check if it is in the cache, or if we force refresh, or if the URL was invalidated
    lCurrentCRC = lURLHandler.get_crc
    if ((@URLs[lURLHash] == nil) or
        (lForceLoad) or
        (@URLs[lURLHash][0] != lCurrentCRC))
      # Load it for real
      # Reset previous value if it was set
      @URLs[lURLHash] = nil
      # Get the object
      lObject = nil
      lAccessError = access_file(iURL, iParameters.merge(:url_handler => lURLHandler)) do |iContent, iBaseName|
        lObject, rError = yield(iContent)
      end
      if (lAccessError != nil)
        rError = lAccessError
      end
      # Put lObject in the cache if no error was found
      if (rError == nil)
        # OK, register it
        @URLs[lURLHash] = [ lCurrentCRC, lObject ]
      else
        if ((defined?(SocketError) != nil) and
            (rError.is_a?(SocketError)))
          # We have a server down
          @HostsDown[lServerID] = nil
        end
      end
    end
    # If no error was found (errors can only happen if it was not already in the cache), take it from the cache
    if (rError == nil)
      rObject = @URLs[lURLHash][1]
    end
  end

  return rObject, rError
end

#init_url_cache ⇒ Object

Constructor



12
13
14
15
16
17
18
19
20
 
# File 'lib/rUtilAnts/URLCache.rb', line 12

def init_url_cache
  # Map of known contents, interpreted in many flavors
  # map< Integer, [ Integer, Object ] >
  # map< URL's hash, [ CRC, Content ] >
  @URLs = {}
  # Map of hosts down (no need to try again such a host)
  # map< String >
  @HostsDown = {}
end