Class: GetAllClearedJobs
- Inherits:
-
Object
- Object
- GetAllClearedJobs
- Defined in:
- lib/clearedjobsnet/get_all_cleared_jobs.rb
Overview
Get as many jobs as possible
Instance Method Summary collapse
-
#crawl ⇒ Object
Crawl through many options.
-
#crawl_each(term_list, filter_name = nil) ⇒ Object
Crawl each item.
-
#gen_json ⇒ Object
Generates output JSON.
-
#get_by_clearance ⇒ Object
Crawl by security clearance.
-
#get_by_company ⇒ Object
Crawl company pages.
-
#get_by_country ⇒ Object
Crawl each country.
-
#get_by_searchterm ⇒ Object
Crawl search term list.
-
#get_first_1000 ⇒ Object
Get the most recent jobs from blank search.
-
#initialize(requests, cm_hash) ⇒ GetAllClearedJobs
constructor
A new instance of GetAllClearedJobs.
-
#save_listings(listings) ⇒ Object
Save unique listings in output.
-
#start_crawler(search_term, filter = nil) ⇒ Object
Start the crawler.
Constructor Details
#initialize(requests, cm_hash) ⇒ GetAllClearedJobs
Returns a new instance of GetAllClearedJobs.
8 9 10 11 12 13 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 8 def initialize(requests, cm_hash) @output = Array.new @requests = requests @cm_hash = cm_hash @current_path = File.dirname(File.(__FILE__)) end |
Instance Method Details
#crawl ⇒ Object
Crawl through many options
16 17 18 19 20 21 22 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 16 def crawl get_by_clearance get_first_1000 get_by_country get_by_company get_by_searchterm end |
#crawl_each(term_list, filter_name = nil) ⇒ Object
Crawl each item
55 56 57 58 59 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 55 def crawl_each(term_list, filter_name=nil) term_list.each do |term| start_crawler(term, filter_name) end end |
#gen_json ⇒ Object
Generates output JSON
74 75 76 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 74 def gen_json return JSON.pretty_generate(@output) end |
#get_by_clearance ⇒ Object
Crawl by security clearance
30 31 32 33 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 30 def get_by_clearance clearance_levels = JSON.parse(File.read(@current_path+"/terms/clearance_levels.json")) crawl_each(clearance_levels, "security_clearance") end |
#get_by_company ⇒ Object
Crawl company pages
42 43 44 45 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 42 def get_by_company company_names = JSON.parse(File.read(@current_path+"/terms/company_names.json")) crawl_each(company_names, "company_page") end |
#get_by_country ⇒ Object
Crawl each country
36 37 38 39 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 36 def get_by_country country_names = JSON.parse(File.read(@current_path+"/terms/country_names.json")) crawl_each(country_names, "country") end |
#get_by_searchterm ⇒ Object
Crawl search term list
48 49 50 51 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 48 def get_by_searchterm search_terms = JSON.parse(File.read(@current_path+"/terms/search_terms.json")) crawl_each(search_terms) end |
#get_first_1000 ⇒ Object
Get the most recent jobs from blank search
25 26 27 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 25 def get_first_1000 start_crawler("all") end |
#save_listings(listings) ⇒ Object
Save unique listings in output
69 70 71 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 69 def save_listings(listings) @output = @output | JSON.parse(listings) end |
#start_crawler(search_term, filter = nil) ⇒ Object
Start the crawler
62 63 64 65 66 |
# File 'lib/clearedjobsnet/get_all_cleared_jobs.rb', line 62 def start_crawler(search_term, filter=nil) c = ClearedJobsNetCrawler.new(search_term, filter, @requests, @cm_hash) c.crawl_listings save_listings(c.gen_json) end |