log_twuncator
by Adam Meehan ([email protected])
http://logtwuncator.rubyforge.org/
== DESCRIPTION:
A log truncator for win32 originally to truncate rails log files but
generalised to any log files needing truncation and archiving. There
didn't seem be something easy, free and open out there so I thought
I would hack up a gem.
It comes with a windows service to install and leave running to monitor
the directories you specify in the config file. The config file allows the
specification of many directories to monitor for log file age and size.
File age is monitored by using the file creation date. In *nix programs like
logrotate the log file can be moved and recreated using POSIX signalling to
restart the process, and hence starting with a new file and creation date. On
windows this is not possible without getting really intimate with the running
process, so to get around it this program uses win32 API call to change the
created date after arching and truncating the file, leaving it in plave.
This seems to work fine with Rails and Apache but may cause problems
depending on how the running application has opened the log file to append
to it.
Please report any problems/suggestions you have.
Credits:
Herryanto Siatono (http://www.pluitsolutions.com/) for his
win32 Ferret service which I based some of the service structure on.
pdumpfs project (http://0xcc.net/pdumpfs/index.html.en) where I found
the win32 API stuff in Ruby to do the file creation date changes.
== FEATURES/PROBLEMS:
* Multiple log truncation and archiving
* Nominate specific file or directory to monitor for truncation
* Use name formatting for archived file name
* Set defaults for settings to use in multiple monitored locations
* Set file or folder specific size and/or age limit
Future:
* Optional formats for the timestamp
* More archive name options
* Compress archived logs
* Email archived logs
== REQUIREMENTS:
win32-service gem required to run as service
== INSTALL:
I messed up the naming of the gem when creating on rubyforge forgive me. So the gem is call logtwuncator but executables are log_twuncator_*
gem install logtwuncator
== USAGE:
log_truncator_service <command> -N service_name <options>
For help
log_truncator_service -h
To install service:
log_twuncator_service install -N log_twuncator -c c:/twuncator_config.yml -d 1 -l c:/log_twuncator.log
where:
-N : is the name of the service to install
-c : is the config file with the settings for various log monitoring paths
-l : is the log file for the truncator.
-d : is the number of minutes between checking the log files for truncation. Default is 5.
To remove service
log_twuncator_service remove -N log_twuncator
To start/stop service
log_twuncator_service start -N log_twuncator
log_twuncator_service stop -N log_twuncator
And remember kids the service creation does not set the startup-type to 'Automatic',
so if you it want it to start with every windows startup then change this setting in
the Services management console ('services.msc' on the command prompt).
--- Config File
The config file allows for multiple monitored source directories for log files. The
setting options are as follows:
source_dir:
This is the root folder to monitor
filter:
This is used filter the files in the folder and takes the allowed settings used in
the ruby Dir.glob method. You could specify '*.log' for all files with log extension.
Or a full filename like 'production.log' to limit to just one file. See Dir.glob
method for more fancy possibilites including files in subdirectories.
age:
The age in days of the file before it should be archived and truncated. If set to
zero then age is never used as the trigger to truncate.
size:
The size of the log file in kilobytes above which the file should be truncated.
If set to 0 then size is never used as the trigger to truncate
archive_dir:
The full path directory to store the archived log file
archive_name:
The format of the archived log file name. You can use three components mixed with
any other characters you wish to insert into the file name when its archived. The
components are
[set_name] - the value of key in the config file for the set of options
[basename] - the original log filename without its extension
[ext] - the file extension of the log file
[timestamp] - the timestamp in the format yyyymmddHHMMSS
These can be combined to make the archived file name eg.
[basename]_[timestamp].[ext] - a log file called production.log archived at
2007-01-01 12:12:35 would become production_20070101121235.log
The config file allows for a 'defaults' section where you can set the defaults for
all subsequent monitored directories. Any setting specified in a monitoring set
will override the defaults for the same setting.
Best shown with an example:
Below is config settings in yaml format
---------------------------------------------------------------------------
defaults:
age: 7
size: 5000
filter: '*.log'
archive_dir: 'c:/logs'
archive_name: '[set_name]_[basename]_[date].[ext]'
app_name:
source_dir: 'c:/web/app/logs'
filter: 'production.log'
age: 0
archive_dir: 'c:/web/app/logs/archive'
---------------------------------------------------------------------------
This config defaults section defines all the settings, but keep in mind that a
defaults section alone does not constitute a monitored set. If you only have a
defaults section then nothing will be monitored. In the example the 'app_name'
monitored set will override the default source directory, filter, age and
archive_dir. The age is set to 0 so files will not be truncated based on age.
It will inherit the default size of 5000 (5000 KB or 5 MB) as the threshold for
truncation above which the file will be archived in c:/web/app/logs/archive
and truncated.
== LICENSE:
(The MIT License)
Copyright (c) 2007 Adam Meehan
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
by Adam Meehan ([email protected])
http://logtwuncator.rubyforge.org/
== DESCRIPTION:
A log truncator for win32 originally to truncate rails log files but
generalised to any log files needing truncation and archiving. There
didn't seem be something easy, free and open out there so I thought
I would hack up a gem.
It comes with a windows service to install and leave running to monitor
the directories you specify in the config file. The config file allows the
specification of many directories to monitor for log file age and size.
File age is monitored by using the file creation date. In *nix programs like
logrotate the log file can be moved and recreated using POSIX signalling to
restart the process, and hence starting with a new file and creation date. On
windows this is not possible without getting really intimate with the running
process, so to get around it this program uses win32 API call to change the
created date after arching and truncating the file, leaving it in plave.
This seems to work fine with Rails and Apache but may cause problems
depending on how the running application has opened the log file to append
to it.
Please report any problems/suggestions you have.
Credits:
Herryanto Siatono (http://www.pluitsolutions.com/) for his
win32 Ferret service which I based some of the service structure on.
pdumpfs project (http://0xcc.net/pdumpfs/index.html.en) where I found
the win32 API stuff in Ruby to do the file creation date changes.
== FEATURES/PROBLEMS:
* Multiple log truncation and archiving
* Nominate specific file or directory to monitor for truncation
* Use name formatting for archived file name
* Set defaults for settings to use in multiple monitored locations
* Set file or folder specific size and/or age limit
Future:
* Optional formats for the timestamp
* More archive name options
* Compress archived logs
* Email archived logs
== REQUIREMENTS:
win32-service gem required to run as service
== INSTALL:
I messed up the naming of the gem when creating on rubyforge forgive me. So the gem is call logtwuncator but executables are log_twuncator_*
gem install logtwuncator
== USAGE:
log_truncator_service <command> -N service_name <options>
For help
log_truncator_service -h
To install service:
log_twuncator_service install -N log_twuncator -c c:/twuncator_config.yml -d 1 -l c:/log_twuncator.log
where:
-N : is the name of the service to install
-c : is the config file with the settings for various log monitoring paths
-l : is the log file for the truncator.
-d : is the number of minutes between checking the log files for truncation. Default is 5.
To remove service
log_twuncator_service remove -N log_twuncator
To start/stop service
log_twuncator_service start -N log_twuncator
log_twuncator_service stop -N log_twuncator
And remember kids the service creation does not set the startup-type to 'Automatic',
so if you it want it to start with every windows startup then change this setting in
the Services management console ('services.msc' on the command prompt).
--- Config File
The config file allows for multiple monitored source directories for log files. The
setting options are as follows:
source_dir:
This is the root folder to monitor
filter:
This is used filter the files in the folder and takes the allowed settings used in
the ruby Dir.glob method. You could specify '*.log' for all files with log extension.
Or a full filename like 'production.log' to limit to just one file. See Dir.glob
method for more fancy possibilites including files in subdirectories.
age:
The age in days of the file before it should be archived and truncated. If set to
zero then age is never used as the trigger to truncate.
size:
The size of the log file in kilobytes above which the file should be truncated.
If set to 0 then size is never used as the trigger to truncate
archive_dir:
The full path directory to store the archived log file
archive_name:
The format of the archived log file name. You can use three components mixed with
any other characters you wish to insert into the file name when its archived. The
components are
[set_name] - the value of key in the config file for the set of options
[basename] - the original log filename without its extension
[ext] - the file extension of the log file
[timestamp] - the timestamp in the format yyyymmddHHMMSS
These can be combined to make the archived file name eg.
[basename]_[timestamp].[ext] - a log file called production.log archived at
2007-01-01 12:12:35 would become production_20070101121235.log
The config file allows for a 'defaults' section where you can set the defaults for
all subsequent monitored directories. Any setting specified in a monitoring set
will override the defaults for the same setting.
Best shown with an example:
Below is config settings in yaml format
---------------------------------------------------------------------------
defaults:
age: 7
size: 5000
filter: '*.log'
archive_dir: 'c:/logs'
archive_name: '[set_name]_[basename]_[date].[ext]'
app_name:
source_dir: 'c:/web/app/logs'
filter: 'production.log'
age: 0
archive_dir: 'c:/web/app/logs/archive'
---------------------------------------------------------------------------
This config defaults section defines all the settings, but keep in mind that a
defaults section alone does not constitute a monitored set. If you only have a
defaults section then nothing will be monitored. In the example the 'app_name'
monitored set will override the default source directory, filter, age and
archive_dir. The age is set to 0 so files will not be truncated based on age.
It will inherit the default size of 5000 (5000 KB or 5 MB) as the threshold for
truncation above which the file will be archived in c:/web/app/logs/archive
and truncated.
== LICENSE:
(The MIT License)
Copyright (c) 2007 Adam Meehan
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.