Class: Request
- Inherits:
-
Object
- Object
- Request
- Defined in:
- lib/collamine/request.rb
Overview
Makes the request to CollaMine servers
Class Method Summary collapse
-
.setup_collamine(options) ⇒ Object
COLLAMINE_DOWNLOAD_URL = ‘172.31.22.135:9001/download/html/’ COLLAMINE_UPLOAD_URL = ‘172.31.22.135:9001/upload/html/multipart/’.
-
.try_collamine(url) ⇒ Object
Try downloading the content from CollaMine servers.
-
.upload_to_collamine(url, content, filename, crawltime) ⇒ Object
Upload the content to Collamine servers.
Class Method Details
.setup_collamine(options) ⇒ Object
COLLAMINE_DOWNLOAD_URL = ‘172.31.22.135:9001/download/html/’ COLLAMINE_UPLOAD_URL = ‘172.31.22.135:9001/upload/html/multipart/’
9 10 11 12 |
# File 'lib/collamine/request.rb', line 9 def self.setup_collamine() @collamine_download_url = [:download] @collamine_upload_url = [:upload] end |
.try_collamine(url) ⇒ Object
Try downloading the content from CollaMine servers
16 17 18 19 20 21 22 23 24 25 26 27 |
# File 'lib/collamine/request.rb', line 16 def self.try_collamine(url) uri = URI.parse(@collamine_download_url+CGI::escape(url.to_s)) Net::HTTP.start(uri.host, uri.port) do |http| response = http.get(uri) case response when Net::HTTPSuccess then return nil if response.body == 'not found' rescue nil return response else nil end end end |
.upload_to_collamine(url, content, filename, crawltime) ⇒ Object
Upload the content to Collamine servers
31 32 33 34 35 36 37 38 39 40 |
# File 'lib/collamine/request.rb', line 31 def self.upload_to_collamine(url, content, filename, crawltime) post_request = Net::HTTP::Post::Multipart.new @collamine_upload_url, 'domain' => Domainatrix.parse(url).domain, 'url' => url, 'crawltime' => crawltime, 'contributor' => 'belson', 'document' => UploadIO.new(StringIO.new(content.encode('UTF-8', 'ISO-8859-15')), 'text/html', filename) response = Net::HTTP.start(URI.parse(@collamine_upload_url).host, URI.parse(@collamine_upload_url).port) { |http| http.request(post_request) } puts response.body end |