0. Introduction
Xmltv consists of a single base grabber and a bunch of site specific grabbers.
The base grabber provides:
- a driver
- option parsing
- start/stop time fixing
- xml production
- helper functions that site specific grabbers might want to use
Currently grabbers are included for radiotimes.com, tvtoday.de, tvgids.nl,
upc.nl and film1.nl.
Grabbers for other sites are (hopefully) easy to add.
1. Installation.
First get Ruby and Hpricot.
Then do 'gem install xmltv'
You could now run a site specific executable, e.g.:
xmltv upc
This will notice that there isn't yet any configuration for upc, and will fetch
and display all available channels and advice you to rerun the same command
with the --add switch
You the can do something like:
xmltv upc --add 211,212,216,215,214
(if you're a sports fan), and get back
Configured channels for upc
211 Sport1
212 Sport1+Extra
216 Sport1+Extra+1
215 Sport1+Extra+3
214 Sport1+Extra+5
With --add and --del you can configure this grabber further.
Other grabbers can be added in the same way
Then you can either run the individual grabbers, one at a time, or run:
xmltv
which has the effect of running _all_ configured grabbers.
2. What it does.
The grabbers produce xmltv files in spool directories. The locations of these are
configurable and overridable, but in the end that's all that the grabbers do at
the moment.
If you want to use these xmltv files for mythtv or something alike, you'll have
to take care of that yourself. In the sample directory are some scripts I use
for mythtv's mythfilldatabase but you'll have to edit these to make them useful.
At the moment you cannot configure mythtv to call these grabbers: I'm quite
willing to accept patches that add that ability, but I've never used any grabber
in this way, and I'm too lazy to study the mythtv documentation.
3. What it does not do.
It doesn't add any <channel/> elements to the xmltv output files. It assumes you
have those channels configured somehow.
It also doesn't anything with icons. You don't need all this stuff for daily
program updates.
4. Reasons why this is better than the Perl XMLTV suite
- Thanks to Hpricot, much more robust data gathering
- DTD driven xml production
- Shared code:
Option parsing
Xml generation
Start/stop time corrections
Configuration handling
Cacheing services
5. Todo
- Write documentation (rdoc)
- Have it run under Windows and MacOS. I don't have any Windows machines or Macs
myself and I didn't write anything with portability in mind (after all I wrote
it for my own program listings) but it shouldn't be too hard.
I understand Ruby converts forward slashes to whatever is needed anyway.
Problem areas could be validation (which is a pipe through xmllint and tv_sort),
and sending exceptions via mail.
6. What you need.
- Ruby obviously. I only tested under 1.8.6, but see no reason why it shouldn't
run on older versions. Didn't try 1.9.0 either.
- Hpricot. Just install the gem.
- If you want to validate you need xmllint (part of libxml2) and tv_sort (part
of the Perl XMLTV package).
7. Adding other grabbers.
There no documentation yet, but there are 5 example implementations that should
help. The implemented sites are quite different as well:
- radiotimes doesn't have html (but a lot of encoding errors)
- film1.nl is so simple and small, there's no need for a cache. The structure of
the site is quite different from the others, and shown is how to fake the normal
pattern.
- tvgids.nl and tvtoday.de resemble each other in structure a lot. Probably some
extra code could be shared between these two, by deriving a subclass from
grabber, called DetailGrabber, and then deriving these two from that.
- gids.vpro.nl is dependent on cookies. Nice.
In general three thing have to be implemented:
- Getting all available channels (almost always there is a useful <SELECT>
element somewhere.)
- Grabbing program information and sometimes extra details (Firebug is very much
your friend here)
- Transforming data from site-specific, error-prone raw data to a fixed
structure the xml producer understands.
8. Warning.
There are people on the Perl XMLTV mailing list who think that grabbing from
tvtoday.de might be illegal (in Germany). For that reason they withdrew the
tvtoday grabber.
If you think these people are right then don't use this implementation. I only
wrote it to check if I had the underlying code ready for creating new grabbers.
Writing and distributing this grabber isn't illegal in any way.
Han Holl
Xmltv consists of a single base grabber and a bunch of site specific grabbers.
The base grabber provides:
- a driver
- option parsing
- start/stop time fixing
- xml production
- helper functions that site specific grabbers might want to use
Currently grabbers are included for radiotimes.com, tvtoday.de, tvgids.nl,
upc.nl and film1.nl.
Grabbers for other sites are (hopefully) easy to add.
1. Installation.
First get Ruby and Hpricot.
Then do 'gem install xmltv'
You could now run a site specific executable, e.g.:
xmltv upc
This will notice that there isn't yet any configuration for upc, and will fetch
and display all available channels and advice you to rerun the same command
with the --add switch
You the can do something like:
xmltv upc --add 211,212,216,215,214
(if you're a sports fan), and get back
Configured channels for upc
211 Sport1
212 Sport1+Extra
216 Sport1+Extra+1
215 Sport1+Extra+3
214 Sport1+Extra+5
With --add and --del you can configure this grabber further.
Other grabbers can be added in the same way
Then you can either run the individual grabbers, one at a time, or run:
xmltv
which has the effect of running _all_ configured grabbers.
2. What it does.
The grabbers produce xmltv files in spool directories. The locations of these are
configurable and overridable, but in the end that's all that the grabbers do at
the moment.
If you want to use these xmltv files for mythtv or something alike, you'll have
to take care of that yourself. In the sample directory are some scripts I use
for mythtv's mythfilldatabase but you'll have to edit these to make them useful.
At the moment you cannot configure mythtv to call these grabbers: I'm quite
willing to accept patches that add that ability, but I've never used any grabber
in this way, and I'm too lazy to study the mythtv documentation.
3. What it does not do.
It doesn't add any <channel/> elements to the xmltv output files. It assumes you
have those channels configured somehow.
It also doesn't anything with icons. You don't need all this stuff for daily
program updates.
4. Reasons why this is better than the Perl XMLTV suite
- Thanks to Hpricot, much more robust data gathering
- DTD driven xml production
- Shared code:
Option parsing
Xml generation
Start/stop time corrections
Configuration handling
Cacheing services
5. Todo
- Write documentation (rdoc)
- Have it run under Windows and MacOS. I don't have any Windows machines or Macs
myself and I didn't write anything with portability in mind (after all I wrote
it for my own program listings) but it shouldn't be too hard.
I understand Ruby converts forward slashes to whatever is needed anyway.
Problem areas could be validation (which is a pipe through xmllint and tv_sort),
and sending exceptions via mail.
6. What you need.
- Ruby obviously. I only tested under 1.8.6, but see no reason why it shouldn't
run on older versions. Didn't try 1.9.0 either.
- Hpricot. Just install the gem.
- If you want to validate you need xmllint (part of libxml2) and tv_sort (part
of the Perl XMLTV package).
7. Adding other grabbers.
There no documentation yet, but there are 5 example implementations that should
help. The implemented sites are quite different as well:
- radiotimes doesn't have html (but a lot of encoding errors)
- film1.nl is so simple and small, there's no need for a cache. The structure of
the site is quite different from the others, and shown is how to fake the normal
pattern.
- tvgids.nl and tvtoday.de resemble each other in structure a lot. Probably some
extra code could be shared between these two, by deriving a subclass from
grabber, called DetailGrabber, and then deriving these two from that.
- gids.vpro.nl is dependent on cookies. Nice.
In general three thing have to be implemented:
- Getting all available channels (almost always there is a useful <SELECT>
element somewhere.)
- Grabbing program information and sometimes extra details (Firebug is very much
your friend here)
- Transforming data from site-specific, error-prone raw data to a fixed
structure the xml producer understands.
8. Warning.
There are people on the Perl XMLTV mailing list who think that grabbing from
tvtoday.de might be illegal (in Germany). For that reason they withdrew the
tvtoday grabber.
If you think these people are right then don't use this implementation. I only
wrote it to check if I had the underlying code ready for creating new grabbers.
Writing and distributing this grabber isn't illegal in any way.
Han Holl