Module: Beanpicker

Extended by:
Beanpicker, MsgLogger
Included in:
Beanpicker
Defined in:
lib/beanpicker.rb,
lib/beanpicker/version.rb,
lib/beanpicker/job_server.rb

Overview

The fucking master job DSL to beanstalkd

Just use it and go to beach ;)

Defined Under Namespace

Modules: MsgLogger Classes: Server, Worker

Constant Summary collapse

VERSION =

A array with the version [MAJOR, MINOR, PATCH]

[0, 1, 0]
VERSION_STRING =

A string of the array joined by “.”

VERSION.join(".")

Instance Method Summary collapse

Methods included from MsgLogger

debug, error, fatal, info, log_handler, log_handler=, msg, warn

Instance Method Details

#add_worker(worker) ⇒ Object

Add a worker to the list of workers



254
255
256
 
# File 'lib/beanpicker.rb', line 254

def add_worker(worker)
  workers << worker
end

#beanstalk ⇒ Object

Return the default beanstalk connection os create a new one with new_beanstalk



113
114
115
 
# File 'lib/beanpicker.rb', line 113

def beanstalk
  @@beanstalk ||= new_beanstalk
end

#beanstalk_urls ⇒ Object

Look in ENV and ENV for beanstalk urls and process returning a array.

If don’t find a good url it will return a array with just “localhost:11300”(default beanstalk port)



125
126
127
128
129
130
131
132
133
134
135
136
 
# File 'lib/beanpicker.rb', line 125

def beanstalk_urls
  urls = [ENV['BEANSTALK_URL'], ENV['BEANSTALK_URLS']].compact.join(",").split(",").map do |url|
    if url =~ /^beanstalk:\/\//
      uri = URI.parse(url)
      url = "#{uri.host}:#{uri.port}"
    else
      url = url.gsub(/^([^:\/]+)(:(\d+)).*$/) { "#{$1}:#{$3 || 11300}" }
    end
    url =~ /^[^:\/]+:\d+$/ ? url : nil
  end.compact
  urls.empty? ? ["localhost:11300"] : urls
end

#default_childs_number ⇒ Object

Return the default number of childs that a Worker should create(1 by default)

This is used by Worker::Child::process



186
187
188
 
# File 'lib/beanpicker.rb', line 186

def default_childs_number
  @@default_childs_number ||= 1
end

#default_childs_number=(v) ⇒ Object

Set the default childs number



191
192
193
 
# File 'lib/beanpicker.rb', line 191

def default_childs_number=(v)
  @@default_childs_number = v
end

#default_delay ⇒ Object

Return the default delay(0 by default)

This is used by enqueue



162
163
164
 
# File 'lib/beanpicker.rb', line 162

def default_delay
  @@default_delay ||= 0
end

#default_delay=(v) ⇒ Object

Set the default delay



167
168
169
 
# File 'lib/beanpicker.rb', line 167

def default_delay=(v)
  @@default_delay = v
end

#default_fork_every ⇒ Object

Return if a child should fork every time that a job will process. This option is overwrited by job options and fork_every

This is used by Worker::Child



199
200
201
 
# File 'lib/beanpicker.rb', line 199

def default_fork_every
  defined?(@@default_fork_every) ? @@default_fork_every : true
end

#default_fork_every=(v) ⇒ Object

Set the default_fork_every



204
205
206
 
# File 'lib/beanpicker.rb', line 204

def default_fork_every=(v)
  @@default_fork_every = !!v
end

#default_fork_master ⇒ Object

Return if a child should fork itself on intialize. This should be used when default_fork_every is false. This option is overwrited by job options and fork_master

Use it only if the jobs need high speed and are “memory leak”-safe

This is used by Worker::Child



215
216
217
 
# File 'lib/beanpicker.rb', line 215

def default_fork_master
  defined?(@@default_fork_master) ? @@default_fork_master : false
end

#default_fork_master=(v) ⇒ Object

Set the default_fork_master



220
221
222
 
# File 'lib/beanpicker.rb', line 220

def default_fork_master=(v)
  @@default_fork_master = !!v
end

#default_pri ⇒ Object

Return the default priority(65536 by default).

This is used by enqueue



150
151
152
 
# File 'lib/beanpicker.rb', line 150

def default_pri
  @@default_pri ||= 65536
end

#default_pri=(v) ⇒ Object

Set the default priority



155
156
157
 
# File 'lib/beanpicker.rb', line 155

def default_pri=(v)
  @@default_pri = v
end

#default_ttr ⇒ Object

Set the default time-to-work(120 by default)

This is used by enqueue



174
175
176
 
# File 'lib/beanpicker.rb', line 174

def default_ttr
  @@default_ttr ||= 120
end

#default_ttr=(v) ⇒ Object

Set the default time-to-work



179
180
181
 
# File 'lib/beanpicker.rb', line 179

def default_ttr=(v)
  @@default_ttr = v
end

#enqueue(jobs, args = {}, o = {}) ⇒ Object

Send a new queue to beanstalkd

The first argument is a String with the name of job or a Array of Strings to do job chains

The second argument should be any object that will be passed in a YAML format to the job

The third argument should be a hash containing :pri(priority) => Integer, :delay => Integer and :ttr(time-to-work) => Integer

If beanstalk raise a Beanstalk::NotConnected, enqueue will create a new instance of beanstalk connection and retry. If it raise again, enqueue will raise the error



90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
 
# File 'lib/beanpicker.rb', line 90

def enqueue(jobs, args={}, o={})
  opts = [
    o[:pri]   || default_pri,
    o[:delay] || default_delay,
    o[:ttr]   || default_ttr
  ]

  jobs = [jobs.to_s] unless jobs.is_a?(Array)
  jobs.compact!
  raise ArgumentError, "you need at least 1 job" if jobs.empty?
  job = jobs.first

  beanstalk.use(job)
  beanstalk.yput({ :args => args, :next_jobs => jobs[1..-1]}, *opts)
rescue Beanstalk::NotConnected => e
  raise e if defined?(r)
  r = true
  error exception_message(e, "You have a problem with beanstalkd.\nIs it running?")
  @@beanstalk = new_beanstalk
  retry
end

#exception_message(e, msg = nil) ⇒ Object

Helper to should a exception message



139
140
141
142
143
144
145
 
# File 'lib/beanpicker.rb', line 139

def exception_message(e, msg=nil)
  m = []
  m << msg if msg
  m << e.message
  m += e.backtrace
  m.join("\n")
end

#fork_every ⇒ Object

See default_fork_every

This option overwrite all others



227
228
229
 
# File 'lib/beanpicker.rb', line 227

def fork_every
  defined?(@@fork_every) ? @@fork_every : nil
end

#fork_every=(v) ⇒ Object

Set the fork_every



232
233
234
 
# File 'lib/beanpicker.rb', line 232

def fork_every=(v)
  @@fork_every = v.nil? ? nil : !!v
end

#fork_master ⇒ Object

See default_fork_master

This option overwrite all others



239
240
241
 
# File 'lib/beanpicker.rb', line 239

def fork_master
  defined?(@@fork_master) ? @@fork_master : nil
end

#fork_master=(v) ⇒ Object

Set the fork_master



244
245
246
 
# File 'lib/beanpicker.rb', line 244

def fork_master=(v)
  @@fork_master = v.nil? ? nil : !!v
end

#new_beanstalk ⇒ Object

Create a new beanstalk connection using the urls from beanstalk_urls



118
119
120
 
# File 'lib/beanpicker.rb', line 118

def new_beanstalk
  Beanstalk::Pool.new(beanstalk_urls)
end

#stop_workers ⇒ Object

Call die! for all childs of every worker and clear the list See workers



260
261
262
263
264
265
266
267
 
# File 'lib/beanpicker.rb', line 260

def stop_workers
  for worker in workers
    for child in worker.childs
      child.die!
    end
  end
  workers.clear
end

#workers ⇒ Object

Return a Array with the workers registered



249
250
251
 
# File 'lib/beanpicker.rb', line 249

def workers
  @@workers ||= []
end