Class: WebMiner::Util

Inherits:
Object
  • Object
show all
Defined in:
lib/webminer/util.rb

Class Method Summary collapse

Class Method Details

.displace(arr, d) ⇒ Object

Given 1d array X, put X(t),X(t-1)…X(t-d) into one row, resulting in a 2d array of size (N-1)x(d+1) n>=1



31
32
33
34
35
 
# File 'lib/webminer/util.rb', line 31

def self.displace(arr, d)
  (0...arr.length-d).map {|i|
    (i..(i+d)).map {|j|
      arr[j]}}
end

.strip_js(raw_text) ⇒ Object 



5
6
7
 
# File 'lib/webminer/util.rb', line 5

def self.strip_js(raw_text)
  return raw_text.gsub(/<script.*?<\/script>/im,'')
end

.strip_specialchars(raw_text) ⇒ Object 



17
18
19
 
# File 'lib/webminer/util.rb', line 17

def self.strip_specialchars(raw_text)
  return raw_text.gsub(/[^A-Za-z0-9-]+/,' ')
end

.strip_story(raw_text) ⇒ Object 



21
22
23
24
25
26
27
 
# File 'lib/webminer/util.rb', line 21

def self.strip_story(raw_text)
  tmp1=self.strip_js(raw_text)
  tmp2=self.strip_stylesheet(tmp1)
  tmp3=self.strip_tags(tmp2)
  tmp4=self.strip_specialchars(tmp3)
  return tmp4
end

.strip_stylesheet(raw_text) ⇒ Object 



9
10
11
 
# File 'lib/webminer/util.rb', line 9

def self.strip_stylesheet(raw_text)
  return raw_text.gsub(/<style.*?<\/style>/im,'')
end

.strip_tags(raw_text) ⇒ Object 



13
14
15
 
# File 'lib/webminer/util.rb', line 13

def self.strip_tags(raw_text)
  return raw_text.gsub(/<.*?>/im,'')
end