= jobmanager
* Project Page: http://rubyforge.org/projects/jobmanager/
== DESCRIPTION:
This package simplifies monitoring / reporting cron jobs on unix systems.
It is composed of a single program, +jobmanager+, which is intended to be
invoked by cron, or a cron-like application. +jobmanager+, in turn,
invokes the specified cron job, in addition to providing the following
added value:
* Logs the output of the cron jobs.
* Rotates the cron job logs.
* Manages a central log which records all cron events (when each job
starts, finishes, the job's success, etc).
* Emails the user the results of the cron job on a configurable
condition (on failure only, on success only, etc.) via sendmail or
smtp.
* Times out a cron job that runs past the configurable timeout.
* Easily scales to handle multiple users' cron jobs.
* Both the default behavior and the per-cron job behavior are highly
configurable.
== SYNOPSIS:
Lets walk through an example invocation of +jobmanager+ via cron.
Here is an example +crontab+ entry that invokes a simplified database
backup script via +jobmanager+ every day at 17:16.
16 17 * * * jobmanager --job_name nightly_backup "mysql_backup test"
Here is the example +mysql_backup+ script referenced above:
======<tt>examples/mysql_backup</tt>:
#!/usr/bin/env ruby
print "=================================================\n"
print "Backing up Database\n"
print "=================================================\n"
database = ARGV[0]
command = "mysqldump #database > /tmp/#database.mysql"
print command, "\n"
if (system("#command"))
print"\nSuccess!\n"
else
print "\nFailure!\n"
exit(1)
end
Here is the example configuration file to be read by +jobmanager+.
======<tt>/etc/jobmanager.yaml</tt>:
################################################################################
# An example jobmanager configuration file.
#
# For a full description of each of the fields, and their default values,
# see the CONFIGURATION FILE section in README.txt.
################################################################################
# The number of log files to be kept per job.
number_of_job_logs: 5
# The directory in which the job logs will be kept, to be interpreted
# by ERB. Allowed variables: user_name.
job_logs_directory: /tmp/logs
# jobmanager logs operational information (when jobs are launched,
# exit, etc). This can be done via syslog or directly to a log file.
# Possible values are: syslog, file.
central_log_mode: file
# If the central_log_mode is set to "file", the log file must be
# specified. This parameter will be interpreted by ERB. Allowed
# variables: user_name.
central_log_file: /tmp/logs/jobmanager.log
# The path to search for the command that jobmanager is invoked with.
#command_path: /usr/bin:/usr/designingpatterns/bin
# Whether jobmanager should print debug trace to the central log file.
#debug: true
# The condition upon which condition results should be emailed.
# Possible values are: always, never, on_failure, on_job_output_or_failure.
email_condition: always
# The configuration file for the simpleemail gem. This parameter will
# be interpreted by ERB. Allowed variables: user_name. Note: If a
# relative path is specified, it will be considered to be relative to
# the location of this configuration file.
email_settings_file: email_settings.rb
# The email subject, to be interpreted by ERB.
# Allowed variables: job_name, command, host_name, results, user_name.
email_subject: "jobmanager results for <%=job_name%> on <%=host_name%> : <%=result%>"
# Whether the job log file should be rotated.
#rotate_job_log: false
# The extension of the rotated job log file.
# True -> The extension is a date_time.
# False -> It is a simple count (.1, .2, etc.).
#date_time_extension: true
# If date_time_extension is true, this field will be used as the
# format for the date/time extension of the rotated job log file
# name.
#date_time_extension_format: '%F_%T'
# Whether the rotated job log file should be zipped.
#zip_rotated_log_file: true
# The timeout (in seconds). If not present, no timeout will be used.
timeout: 1800
# Whether each line in the job log file will be prepended with the date/time.
#job_log_prepend_date_time: true
# The date/time format of the prepended date/time.
#job_log_prepend_date_time_format: '%F_%T'
# The sender of emails sent from jobmanager. This can also be
# specified in the email_settings_file (attribute default_from).
#email_from_address: "Sender <[email protected]>"
# The receiver of emails sent from jobmanager. This can also be
# specified in the email_settings_file (attributes default_to).
#email_to_address: "Receiver <[email protected]>"
Here is the email configuration file referenced in +jobmanager.yaml+
above. In this example, we choose to email via smtp, although
sendmail is also an option. This file is documented further in the
+simpleemail+[http://simpleemail.rubyforge.org] gem.
======<tt>/etc/email_settings.rb</tt>:
ActionMailer::Base.delivery_method = :smtp
ActionMailer::Base.smtp_settings = {
:address => 'smtp.gmail.com',
:port => 587,
:domain => "gmail.com",
:authentication => "login",
:user_name => "[email protected]",
:password => "password",
:tls => :auto
}
SimpleEmail::Email.default_from = "Sender Address <[email protected]>"
SimpleEmail::Email.default_to = "Receiver Address <[email protected]>"
Note: In order to run this example oneself, either
* +mysql_backup+ script must live in a directory that is in one's path, or
* the +command_path+ field in +jobmanager.yaml+ must be set to include the path
where +mysql_backup+ lives.
Here is the central log file that +jobmanager+ updated.
======<tt>/tmp/logs/jobmanager.log</tt>:
I, [2009-06-01T17:16:32 #32235] INFO -- : [janet, nightly_backup] Command (/usr/bin/mysql_backup test) launched, pid = 32237.
I, [2009-06-01T17:16:33 #32235] INFO -- : [janet, nightly_backup] Job exited successfully
I, [2009-06-01T17:16:33 #32235] INFO -- : [janet, nightly_backup] Job log rotated to /tmp/logs/nightly_backup.log.2009-06-01_17:16:33.
I, [2009-06-01T17:16:33 #32235] INFO -- : [janet, nightly_backup] Sent mail to Receiver Address <[email protected]>
Here is the rotated job log file.
======<tt>/tmp/logs/nightly_backup.log.2009-06-01_17:16:33</tt>:
=================================================
Backing up Database
=================================================
mysqldump test > /tmp/test.mysql
Success!
Here is the subject and body of the email that was sent upon job completion.
======<tt>Email Subject</tt>:
jobmanager results for nightly_backup on jackfruit : Success
======<tt>Email Body</tt>:
---------------------------------------------------------
jobmanager Output:
---------------------------------------------------------
I, [2009-06-01T17:16:23 #32219] INFO -- : [janet, nightly_backup] Command (/usr/bin/mysql_backup test) launched, pid = 32222.
I, [2009-06-01T17:16:24 #32219] INFO -- : [janet, nightly_backup] Job exited successfully
I, [2009-06-01T17:16:24 #32219] INFO -- : [janet, nightly_backup] Job log rotated to /tmp/logs/nightly_backup.log.2009-06-01_17:16:24.
---------------------------------------------------------
Job Output:
---------------------------------------------------------
=================================================
Backing up Database
=================================================
mysqldump test > /tmp/test.mysql
Success!
== CONFIGURATION FILE:
A configuration file is required for +jobmanager+ to run. By default,
+jobmanager+ will expect the configuration file to be located at
+/etc/jobmanager.yaml+. If the environment variable
+JOBMANAGER_CONFIG_FILE+ is set, +jobmanager+ will instead read the
configuration file specified by this variable.
The configuration file provides the following parameters:
[+job_logs_directory+: required]
The directory in which the job log files will be kept. The
+job_logs_directory+ parameter will be interpreted by ERB. The
allowed variables are: +user_name+. Specifying the +user_name+
variable in the +job_logs_directory+ value can be useful if you
want multiple users to share the same configuration file, but have
different job logs directories.
[+central_log_mode+: optional, default: +syslog+]
+jobmanager+ logs operational information (when jobs are
launched, exit, etc). This can be done via +syslog+ or directly to a
log file. Possible values are: <tt>syslog, file</tt>.
[+central_log_file+: optional]
If +central_log_mode+ is set to +file+, this parameter must be set.
The +central_log_file+ parameter will be interpreted by ERB. The
allowed variables are: +user_name+. Specifying the +user_name+
variable in the +central_log_file+ value can be useful if you want
multiple users to share the same configuration file, but have
different central log files.
[+number_of_job_logs+: optional, default: +3+]
The number of logs to be kept per job.
[+job_log_prepend_date_time+: optional, default: +false+]
Whether the current date/time should be prepended to each line of the
job log file.
[+job_log_prepend_date_time_format+: optional, default: <tt>'%F_%T'</tt> ]
if +job_log_prepend_date_time+ is +true+, this parameter will be used as
the format for the date/time that is prepended to each line of the job
log file.
[+command_path+: optional, default: <tt>ENV['PATH']</tt> ]
The path to be searched for the command that +jobmanager+ is
invoked with. If unspecified, the environment path will be used.
[+debug+: optional, default: +false+]
Whether +jobmanager+ should print debug trace to the central log file.
[+email_condition+: optional, default: +on_failure+]
The condition upon which results should be emailed. Possible
values are: <tt>always, never, on_failure, on_job_output_or_failure</tt>.
The condition +on_job_output_or_failure+ denotes that an email will be
sent if the job failed and/or the job output was non-empty.
[+email_settings_file+: optional, default: +email_settings.rb+]
The configuration file for the
+simpleemail+[http://simpleemail.rubyforge.org] gem. This
specifies the delivery method (+smtp+, +sendmail+), as well as
delivery method specific settings. Default to/from emails can be
specified in here. If the path specified is relative, it is assumed
that it is relative to the directory in which the +jobmanager+
configuration file is located. If +email_condition+ is set to a
value other than +never+, this parameter must specified. The
+email_settings_file+ parameter will be interpreted by ERB. The
allowed variables are: +user_name+. Specifying the +user_name+
variable in the +email_settings_file+ value can be useful if you
want multiple users to share the same configuration file, but have
separate email settings files.
[+email_subject+: optional, default: <tt>"jobmanager results for <%=job_name%> on <%=host_name%> : <%=result%>"</tt>]
The email subject, to be interpreted by ERB. Allowed variables
inside the string are: <tt>job_name, command, host_name, results, user_name</tt>.
[+rotate_job_log+: optional, default: +true+]
Whether the job log file should be rotated. If the job log file is not
rotated, it will simply be appended to during each run.
[+date_time_extension+: optional, default: +true+]
This refers to the extension of the rotated job log file. If
+date_time_extension+ is present and set to +true+, the extension
represents a date and time. Otherwise, it is a simple count (.1, .2, etc.).
[+date_time_extension_format+: optional, default: <tt>'%F_%T'</tt>]
If +date_time_extension+ is +true+, this parameter will be used as the
format for the date and time extension of the rotated job log file.
[+zip_rotated_log_file+: optional, default: +false+]
Whether the rotated job log file should be gzip'ed.
[+timeout+: optional]
The timeout (in seconds). If the job spawned by +jobmanager+ does not
exit after +timeout+ seconds, +jobmanager+ will kill the process, and
report failure. If +timeout+ is unspecifed, +jobmanager+ will let the
process run for an unlimited amount of time (+jobmanager+ will simply
wait til it returns).
[+email_from_address+: optional]
The sender of emails sent from +jobmanager+. This can also be
specified in the +email_settings_file+ (attribute +default_from+).
[+email_to_address+: optional]
The receiver of emails sent from +jobmanager+. This can also be
specified in the +email_settings_file+ (attribute +default_to+).
== COMMAND LINE ARGUMENTS:
Usage:
Usage: jobmanager.rb [options] <command>
Notable command line options:
[<tt>-j, --job_name</tt>]
The name of the job to be run. If this is unspecified, the
job name will be set to the basename of the COMMAND specified.
The job name is used when logging to the central log file, and
in the name of the job log file.
The remaining command line options are duplicates of the configuration
file parameters. In fact, all of the configuration parameters
specified in the configuration file can be overriden via +jobmanager+
command line options.
== STARTING JOBMANAGER FROM THE COMMAND LINE VS. CRON:
The most common use case for +jobmanager+ is to be run from cron, or a
cron-like application. However, invoking +jobmanager+ from the
command line can be useful for testing, debugging, and one-off runs.
As +jobmanager+ is commonly invoked via the cron, +jobmanager+ logs
all output to either +syslog+ or a central log file (specified by the
configuration parameters +central_log_mode+, +central_log_file+).
When +jobmanager+ is run from the command line (or any script whose
STDIN, STDOUT, or STDERR is connected to a terminal), these two
configuration parameters will be ignored and +jobmanager+'s output
will be sent to the terminal.
== REQUIREMENTS:
The +jobmanager+ application requires a number of gems: hoe,
SyslogLogger, configtoolkit, simpleemail, logrotate,
assertions, and relative.
Hoe and assertions are required only for running the tests. Relative is
required for running the tests and the examples. The rest of the gems
listed above are required for the proper functioning of the
+jobmanager+ application.
== INSTALL:
1. <tt>sudo gem install jobmanager</tt>
2. Set up the +jobmanager+ configuration file (+jobmanager.yaml+).
For more detailed instructions, see the CONFIGURATION FILE section
above. There is an example file provided in this distribution:
+examples/jobmanager.yaml+.
3. Set up the simpleemail configuration file (+email_settings.rb+).
Alternately, if you want to disable email, set the configuration
parameter +email_condition+ to +never+ in +jobmanager.yaml+. See
the +simpleemail+[http://simpleemail.rubyforge.org] gem rdoc for
more details.
4. Confirm that the configuration files are valid by invoking +jobmanager+
from the command line with one of your cron jobs or a test command. If
+jobmanager+ is invoked via the command line, the central log will be
redirected to screen (instead of the central log file).
5. Edit your +crontab+ to run new and/or existing cron jobs through
+jobmanager+.
For possible reasons why running +jobmanager+ via the command line may
be successful, but running +jobmanager+ with the same configuration
via the cron may fail, consult FAQ.txt.
== PROBLEMS:
None (known).
== AUTHORS:
=== Designing Patterns
* Homepage: http://www.designingpatterns.com
* Blogs: http://blogs.designingpatterns.com
== SUPPORT:
Please post questions, concerns, or requests for enhancement to the forums on
the project page. Alternatively, direct contact information for
Designing Patterns can be found on the project page for this gem.
== ENHANCEMENTS:
Please feel free to contact us with any ideas; we will try our best to
enhance the software and respond to user requests. Of course, we are more
likely to work on a particular enhancement if we know that there are users
who want it. Designing Patterns provides contracting and consulting services,
so if there is an enhancement that *must* get done (and in a specified time
frame), please inquire about retaining our services!
== LICENSE:
The license text can be found in the +LICENSE+ file at the root of the
distribution.
This package is licensed with an MIT license:
Copyright (c) 2008-2009 Designing Patterns
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
== SHARE AND ENJOY!