Class: ThreatDetector::Search

Inherits:
Object
  • Object
show all
Includes:
Utility
Defined in:
lib/threat_detector/search.rb

Overview

Note:

This loads our threat cache to memory, and therefore, we should re-use this instance, where possible.

Class used to search threats in our Cache

ThreatDetector tries to be smart in understanding that the database may not necessarily contain the exact search item, and often, employs logic in deciding whether a given IP, network, host or URL is a threat. For example, in addition to directly matching threats in this database, it knows that:

  • An IP is a threat if it belongs to an identified network.

  • A network is a threat if it belongs to a wider identified network.

  • An IP is a threat if the hostname it resolves to is marked as a threat.

  • A host can be a threat if the IP it resolves to is marked as a threat.

  • A url can be a threat if its hostname or IP is marked as a threat.

  • etc.

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Utility

#categorize_ip_or_uri, #categorize_uri, #feeds_config_path, #fetch_page, #raise_error, #refresh=, #refresh?, #sanitize_options, #working_directory

Constructor Details

#initialize(options = {}) ⇒ Search

Note:

This loads our threat cache to memory, and therefore, we should re-use this instance, where possible.

Instantiate a new ThreatDetector::Search

The default options are:

working_directory: ~/.threat_detector
feeds_config_path: <gem_path>/threat_detector/feeds.yaml

Parameters:

  • opts (Hash)

    a customizable set of options

  • opts (Hash)

    options received from the user


37
38
39
40
# File 'lib/threat_detector/search.rb', line 37

def initialize(options = {})
  @options = sanitize_options(options)
  @cache = ThreatDetector::Cache.load(@options)
end

Instance Attribute Details

#cacheObject (readonly)

Returns the value of attribute cache


23
24
25
# File 'lib/threat_detector/search.rb', line 23

def cache
  @cache
end

#optionsObject (readonly)

Returns the value of attribute options


23
24
25
# File 'lib/threat_detector/search.rb', line 23

def options
  @options
end

Instance Method Details

#find(item, options = {}) ⇒ Hash

Search for the given item in threats database.

By default, options are:

smarter: true, resolve: true

Passing `smarter: false` will turn off:

  • An IP is a threat if it belongs to an identified network.

  • A network is a threat if it belongs to a wider identified network.

  • A url can be a threat if its hostname or IP is marked as a threat.

Passing `resolve: false` will turn off:

  • A host can be a threat if the IP it resolves to is marked as a threat.

  • An IP is a threat if the hostname it resolves to is marked as a threat.

Parameters:

  • item (String, URI, #to_s)

    Item to search in database

  • options (Hash) (defaults to: {})

    search options for this search

Options Hash (options):

  • :smarter (Bool)

    Apply heuristics to identify threat smartly?

  • :resolve (Bool)

    Whether to resolve IP/hosts to host/IPs for identifying threats?

Returns:

  • (Hash)

    hash containing keys that correspond to:

    • :safe - whether item was identified as safe or not

    • :type - category for this item from IP, URL, Host, Network or Unknown

    • :reason - if unsafe, the reason why this item was marked as unsafe

    • :identified - relevant threat item found in the database


66
67
68
69
70
71
72
73
74
75
# File 'lib/threat_detector/search.rb', line 66

def find(item, options = {})
  return { safe: true, type: :unknown } if item.to_s.strip.empty?

  category = categorize_ip_or_uri(item)
  reason, found = send "unsafe_#{category}?", item, options
  data = options.merge(safe: !found, type: category)
  return data unless found

  data.merge(reason: reason, identified: found)
end

#process(*keys, options = {}) {|item, data| ... } ⇒ Array<Hash>

Identify multiple items or files against threat database. Optionally, append the results of this search in a CSV file.

If for a given search term, a file exists on disk with the same name, and is readable, then, preference will be given to that file. If you are only specifying search terms, please use #process_items instead.

Returns array of search results.

Parameters:

  • - (Array<String, URI, File, #to_s, #read>)

    search items or files containing search items

  • options (Hash) (defaults to: {})

    search options for this search

Options Hash (options):

  • :smarter (Bool)

    use heuristics to check threat smartly?

  • :resolve (Bool)

    Whether to resolve IP/hosts to host/IPs for identifying threats?

  • :save (String) — default: optional

    Path to CSV file where search results will be appended

Yields:

  • (item, data)

    iterator for search results

Yield Parameters:

  • item (String)

    current search item

  • data (Hash)

    search results for current search item

Returns:

  • (Array<Hash>)

    array of search results

See Also:


129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
# File 'lib/threat_detector/search.rb', line 129

def process(*args, &block)
  items, options = extract_options(args)

  items = items.map do |item|
    if File.exist?(item) && File.readable?(item)
      item = File.readlines(item).map(&:strip)
    elsif File.exist?(item)
      raise_error "Found unreadable item file: #{item}"
    end

    item
  end.flatten

  process_items(*items, options, &block)
end

#process_items(*keys, options = {}) {|item, data| ... } ⇒ Array<Hash>

Search multiple given items in the database. Optionally, append the results of this search in a CSV file.

Returns array of search results.

Parameters:

  • - (Array<String, URI, #to_s>)

    search items

  • options (Hash) (defaults to: {})

    search options for this search

Options Hash (options):

  • :smarter (Bool)

    use heuristics to check threats smartly?

  • :resolve (Bool)

    Whether to resolve IP/hosts to host/IPs for identifying threats?

  • :save (String) — default: optional

    Path to CSV file where search results will be appended

Yields:

  • (item, data)

    iterator for search results

Yield Parameters:

  • item (String)

    current search item

  • data (Hash)

    search results for current search item

Returns:

  • (Array<Hash>)

    array of search results

See Also:


94
95
96
97
98
99
100
101
102
103
104
# File 'lib/threat_detector/search.rb', line 94

def process_items(*args, &block)
  items, options = extract_options(args)

  results = items.map do |item|
    data = find(item, options.except(:save))
    block.call(item, data) if block
    [item, data]
  end.to_h

  options[:save] ? append_to_csv(results, options[:save]) : results
end