Method: HTMLProofer::UrlValidator::External#run_external_link_checker

Defined in:
lib/html_proofer/url_validator/external.rb

Proofer runs faster if we pull out all the external URLs and run the checks at the end. Otherwise, we’re halting the consuming process for every file during process_files.

In addition, sorting the list lets libcurl keep connections to the same hosts alive.

Finally, we’ll first make a HEAD request, rather than GETing all the contents. If the HEAD fails, we’ll fall back to GET, as some servers are not configured for HEAD. If we’ve decided to check for hashes, we must do a GET–HEAD is not available as an option.



46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
 
# File 'lib/html_proofer/url_validator/external.rb', line 46

def run_external_link_checker(external_urls)
  # Route log from Typhoeus/Ethon to our own logger
  Ethon.logger = @logger

  external_urls.each_pair do |external_url, metadata|
    url = Attribute::Url.new(@runner, external_url, base_url: nil)

    unless url.valid?
      add_failure(metadata, "#{url} is an invalid URL", 0)
      next
    end

    next unless new_url_query_values?(url)

    method = if @runner.options[:check_external_hash] && url.hash?
      :get
    else
      :head
    end

    queue_request(method, url, metadata)
  end

  @hydra.run
end