About

A live example: bwkfanboy.herokuapp.com

bwkfanboy is a converter from a raw HTML to an Atom feed. You can use it to watch sites that do not provide its own feed.

The converter is not a magick tool: you’ll need to write a plugin (in Ruby) for an each site you want to watch. bwkfanboy gives guidelines and a general assistance.

Version 2.0.0 was rewritten form scratch. Plugins from previous versions will not work. See github.com/gromnitsky/bwkfanboy_old repo for old versions.

Included Plugins

NAME                 VER DESCRIPTION

bwk                    2 Brian Kernighan's articles from Daily Princetonian
freebsd-ports-update   3 News from FreeBSD ports
inc                    1 Articles (per-user) from inc.com
econlib                1 Latest articles from econlib.org

Installation

Install bundler:

# gem install bundler

Install all dependencies. You have 2 choices:

  1. Usual via

    # bundle install

  2. Or only for this program

    % bundle install --path vendor/bundle

    You’ll have to prefix other commands with ‘bundle exec’ though.

Run application:

% rackup -E production

Point a browser to 127.0.0.1:9292

Architecture

Plugins

Plugins can be in system

`gem env gemdir`/gems/bwkfanboy-x.y.z/plugins

or user’s home

~/.bwkfanboy/plugins

directory.

Pipeline

The program consists of 3 parts:

  1. bwkfanboy script that takes 1 parameter: the name of a file in plugin directories (without the .rb suffix). So, for example to get an atom feed from dailyprincetonian.com, you type:

    % bwkfanboy bwk

    and it will load /usr/local/lib/ruby/gems/1.9/gems/bwkfanboy-2.0.0/plugins/bwk.rb file on my FreeBSD machine, fetch and parse the html from dailyprincetonian.com and generate the required feed, dumping it to stdout.

    The script is just a convenient wrapper for 2 separate utils below. (Although it doesn’t executes them, but uses their internal api.)

  2. bwkfanboy_parse

    Takes 1 parameter: a plugin name.

    This util reads stdin, expecting it to be a html, parses it and dumps the result to stdout as MessagePack object in UTF-8.

  3. bwkfanboy_generate

    Reads stdin expecting it to be a proper MessagePack object.

    The result will be an Atom feed dumped to stdout in UTF-8.

So, without the wrapper all this together looks like:

% curl http://example.org | bwkfanboy_parse myplugin | bwkfanboy_generate

HTTP

bwkfanboy includes a small sinatra app. See the installation phase.