SiteTap
SiteTap takes a home page URL and turns into into a packaged directory of:
- html
- plain text
- markdown
Installation
To install this to a ruby project, add the following to your Gemfile
:
gem 'sitetap'
And then execute:
$ bundle install
Or install it so you can run it globally:
$ gem install sitetap
Usage
Using SiteTap is quite simple. You just run the executable and give it a URL.
$ sitetap [URL]
So, if I wanted to scrape Sapwood's website, I could do this:
$ sitetap "http://sapwood.org/"
Within your current directory, this will create the following directory structure:
- sapwood.org
- html
- markdown
- txt
- tmp
Within each are the converted files from the website.
Bugs
Please create an issue if you encounter a bug.
Contributing
Missing a feature? Add it!
Found a bug? Fix it!
- Fork it ( https://github.com/[my-github-username]/sitetap/fork )
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create a new Pull Request