About
bwkfanboy is a HTML to Atom feed converter that you can use to watch sites that do not provide its own feed.
The converter is not a magick tool: you’ll need to write a plugin (in Ruby) for each site you want to watch. bwkfanboy provides guidelines and general assistance.
Plugins from version 1.1.4 are incompatible with the 0.1.x series. Please reread in doc/plugin.rdoc the example of the skeleton plugin.
Included Plugins
NAME VER DESCRIPTION
bwk 2 Brian Kernighan's articles from Daily Princetonian
freebsd-ports-update 3 News from FreeBSD ports
quora 11 Last n answers (per-user) from Quora; requires nodejs 0.3.7+
econlib 1 Latest articles from econlib.org
Installation
The easiest way is to type:
# gem install bwkfanboy
Beware that plugins above are usually updated more frequently than the whole gem on rubygems.org, so grab the source before struggling with a particular plugin.
If the setup with gem install ...
fails, go to rubygems.org/gems/bwkfanboy and click ‘Download’ link, save the file and run:
# gem install -l /where/is/your/bwkfanboy-x.y.z.gem
If even this doesn’t help, drop me a email.
Architecture
Plugins
The plugin is a Ruby class Page
that inherits Bwkfanboy::Parse parent, overriding 1 method.
Plugins can be in system
`gem env gemdir`/gems/bwkfanboy-x.y.z/lib/bwkfanboy/plugins
or user’s home
~/.bwkfanboy/plugins
directories.
Pipeline
The program consists of 4 parts:
-
bwkfanboy script that takes 1 parameter: the name of a file in plugins directories (without the .rb suffix). So, for example to get an atom feed from dailyprincetonian.com you type:
% bwkfanboy bwk
and it will load
/usr/local/lib/ruby/gems/1.9/gems/bwkfanboy-0.0.1/lib/bwkfanboy/plugins/bwk.rb
file on my FreeBSD machine, fetch and parse html from dailyprincetonian.com and generate the required feed, dumping it to stdout.The script is just a convinient wrapper for 3 separate utils.
-
bwkfanboy_fetch
It reads 1 line from stdin for the URL to fetch from. The result will be dumped to stdout.
-
bwkfanboy_parse
It takes 1 parameter: a full path to a plugin file.
This util reads stdin expecting it to be a xhtml, parses it and dumps the result to stdout in JSON-formatted object.
-
bwkfanboy_generate
Reads stdin expecting it to be a proper JSON-formatted object.
The result will be an Atom feed dumped to stdout in UTF-8.
So, without the wrapper all this together looks like:
% echo http://example.org | bwkfanboy_fetch |
bwkfanboy_parse /path/to/my/plugin.rb | bwkfanboy_generate
Log
All utils write to /tmp/bwkfanboy/USER/log/general.log
file if permissions allows it.
HTTP
There are 2 method to get an Atom feed via HTTP:
-
web/bwkfanboy.cgi
(from the program tarball), which you may copy to your Apache cgi directory and run it. This prohibits you from using HOME directory for your own plugins. Also the cgi script requires some manual editing (setting 1 variable in it) before even you can start utilizing it. -
Small bwkfanboy_server HTTP server. It can run from any user and thus is able to inherit env variables for discovering your HOME directory. Read doc/bwkfanboy_server.rdoc to know how to operate it.
License
(The MIT License)
Copyright © 2010 Alexander Gromnitsky.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ‘Software’), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.