Alephant::Publisher::Queue
Static publishing to S3 based on SQS messages.
Dependencies
- JRuby 1.7.8+
- An AWS account, with:
- S3 bucket.
- SQS Queue.
- Dynamo DB table.
- Elasticache (if using the "revalidate" pattern)
Migrating from Alephant::Publisher
Add the new gem in your Gemfile:
gem 'alephant-publisher-queue'
Run:
bundle install
Require the new gem in your app:
require 'alephant/publisher/queue'
Important - note that the namespace has changed from Alephant::Publisher
to Alephant::Publisher::Queue
.
Installation
Add this line to your application's Gemfile:
gem 'alephant-publisher-queue'
And then execute:
bundle
Or install it yourself as:
gem install alephant-publisher-queue
Setup
Ensure you have a config/aws.yml
in the format:
access_key_id: ACCESS_KEY_ID
secret_access_key: SECRET_ACCESS_KEY
Structure
Provide a view and template:
└── views
├── models
│ └── foo.rb
└── templates
└── foo.mustache
foo.rb
class Foo < Alephant::Views::Base
def content
@data['content']
end
end
foo.mustache
{{ content }}
Usage (standard setup - non-revalidate)
require "alephant/logger"
require "alephant/publisher/queue"
module MyApp
def self.run!
loop do
Alephant::Publisher::Queue.create().run!
end
rescue => e
Alephant::Logger.get_logger.error "Error: #{e.}"
end
private
def self.
Alephant::Publisher::Queue::Options.new.tap do |opts|
opts.add_queue(
:aws_account_id => 'example',
:sqs_queue_name => 'test_queue'
)
opts.add_writer(
:keep_all_messages => 'false',
:lookup_table_name => 'lookup-dynamo-table',
:renderer_id => 'renderer-id',
:s3_bucket_id => 'bucket-id',
:s3_object_path => 'example-s3-path',
:sequence_id_path => '$.sequential_id',
:sequencer_table_name => 'sequence-dynamo-table',
:view_path => 'path/to/views'
)
end
end
end
Add a message to your SQS queue, with the following format:
{
"content": "Hello World!",
"sequential_id": 1
}
Output:
Hello World!
S3 Path:
S3 / bucket-id / example-s3-path / renderer-id / foo / 7e0c33c476b1089500d5f172102ec03e / 1
Usage (revalidate pattern)
require "addressable/uri"
require "alephant/logger"
require "alephant/publisher/queue"
module MyApp
class UrlGenerator
class << self
# This function is called to generate the URL to be requested as
# part of the rendering process. The return must be a URL as a string.
def generate(opts)
"http://example.com/?#{url_params(opts)}"
end
private
def url_params(params_hash)
uri = Addressable::URI.new
uri.query_values = params_hash
uri.query
end
end
end
class HttpResponseProcessor
class << self
# This function is called upon a successful HTTP response.
#
# Use it to modify or process the response of your HTTP request
# as you please, but there is one rule - the return value MUST
# be a JSON object.
def process(opts, status, body)
# our response is already JSON, pass it through
body
end
# If you wish to vary your revalidate TTL on a per-endpoint (or
# other logic) basis, you can do it here - simply return an Integer
# value.
#
# If nil is returned the 'revalidate_cache_ttl' config setting on the
# broker will be used as the default TTL, otherwise the default in
# 'alephant-broker' will be used.
def self.ttl(opts)
# 30s revalidate time for all
30
end
end
end
def self.run!
loop do
Alephant::Publisher::Queue.create(, processor).run!
end
rescue => e
Alephant::Logger.get_logger.error "Error: #{e.}"
end
private
def self.processor
Alephant::Publisher::Queue::RevalidateProcessor.new(, UrlGenerator, HttpResponseProcessor)
end
def self.
Alephant::Publisher::Queue::Options.new.tap do |opts|
opts.add_queue(
:aws_account_id => 'example',
:sqs_queue_name => 'test_queue'
)
opts.add_writer(
:lookup_table_name => 'lookup-dynamo-table',
:s3_bucket_id => 'bucket-id',
:s3_object_path => 'example-s3-path',
:view_path => 'path/to/views'
)
opts.add_cache(
:elasticache_config_endpoint => 'example',
:elasticache_cache_version => '100',
:revalidate_cache_ttl => '30'
)
end
end
end
Add a message to your SQS queue, with the following format (id
, batch_id
, options
):
{
"id": "renderer_id",
"batch_id": null,
"options": {
"id": "foo",
"type": "chart"
}
}
This will then make a HTTP GET request to the configured endpoint (via UrlGenerator
), process the response (via HttpResponseProcessor
), render and store your content.
You will not ordinarily need to push messages onto SQS manually, this will be handled via the broker in real use.
Preview Server
Alephant Preview allows you to see the HTML generated by your templates, both standalone and in the context of a page.
Contributing
- Fork it!
- Create your feature branch:
git checkout -b my-new-feature
- Commit your changes:
git commit -am 'Add some feature'
- Push to the branch:
git push origin my-new-feature
- Create new Pull Request.
Feel free to create an issue if you find a bug.