Class: SearchFlip::Bulk Private

Inherits:
Object
  • Object
show all
Defined in:
lib/search_flip/bulk.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

The SearchFlip::Bulk class implements the bulk support, ie it collects single requests and emits batches of requests.

Examples:

SearchFlip::Bulk.new "http://127.0.0.1:9200/index/type/_bulk" do |bulk|
  bulk.create record.id, MyIndex.serialize(record)
  bulk.index record.id, MyIndex.serialize(record), version: record.version, version_type: "external"
  bulk.delete record.id, routing: record.user_id
  bulk.update record.id, doc: MyIndex.serialize(record)
end

Defined Under Namespace

Classes: Error

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(url, options = {}) {|_self| ... } ⇒ Bulk

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Builds and yields a new Bulk object, ie initiates the buffer, yields, sends batches of records each time the buffer is full, and sends a final batch after the yielded code returns and there are still documents present within the buffer.

Examples:

Basic use

SearchFlip::Bulk.new "http://127.0.0.1:9200/index/type/_bulk" do |bulk|
  # ...
end

Ignore certain errors

SearchFlip::Bulk.new "http://127.0.0.1:9200/index/type/_bulk", 1_000, ignore_errors: [409] do |bulk|
  # ...
end

Parameters:

  • url (String) —

    The endpoint to send bulk requests to

  • options (Hash) (defaults to: {}) —

    Options for the bulk requests

  • bulk_max_mb (Hash) —

    a customizable set of options

Options Hash (options):

  • bulk_limit (Fixnum) —

    The maximum number of documents per bulk request

  • ignore_errors (Array, Fixnum) —

    Errors that should be ignored. If you eg want to ignore errors resulting from conflicts, you can specify to ignore 409 here.

  • raise (Boolean) —

    If you want the bulk requests to never raise any exceptions (fire and forget), you can pass false here. Default is true.

  • http_client (SearchFlip::HTTPClient) —

    An optional http client instance

Yields:

  • (_self)

Yield Parameters:



52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/search_flip/bulk.rb', line 52

def initialize(url, options = {})
  @url = url
  @options = options
  @http_client = options[:http_client] || SearchFlip::HTTPClient.new
  @ignore_errors = Array(options[:ignore_errors] || []).to_set

  @bulk_limit = options[:bulk_limit] || SearchFlip::Config[:bulk_limit]
  @bulk_max_mb = options[:bulk_max_mb] || SearchFlip::Config[:bulk_max_mb]

  @bulk_max_bytes = @bulk_max_mb * 1024 * 1024

  init

  yield self

  upload if @num > 0
end

Instance Attribute Details

#ignore_errors ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



18
19
20
 
# File 'lib/search_flip/bulk.rb', line 18

def ignore_errors
  @ignore_errors
end

#options ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



18
19
20
 
# File 'lib/search_flip/bulk.rb', line 18

def options
  @options
end

#url ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



18
19
20
 
# File 'lib/search_flip/bulk.rb', line 18

def url
  @url
end

Instance Method Details

#create(id, object, options = {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Adds a create request to the bulk batch.

Parameters:

  • id (Fixnum, String) —

    The document/record id

  • json (String) —

    The json document

  • options (options) (defaults to: {}) —

    Options for the index request, like eg routing and versioning



102
103
104
 
# File 'lib/search_flip/bulk.rb', line 102

def create(id, object, options = {})
  perform(:create, id, SearchFlip::JSON.generate(object), options)
end

#delete(id, options = {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Adds a delete request to the bulk batch.

Parameters:

  • id (Fixnum, String) —

    The document/record id

  • options (options) (defaults to: {}) —

    Options for the index request, like eg routing and versioning



127
128
129
 
# File 'lib/search_flip/bulk.rb', line 127

def delete(id, options = {})
  perform(:delete, id, nil, options)
end

#index(id, object, options = {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Adds an index request to the bulk batch.

Parameters:

  • id (Fixnum, String) —

    The document/record id

  • json (String) —

    The json document

  • options (options) (defaults to: {}) —

    Options for the index request, like eg routing and versioning



79
80
81
 
# File 'lib/search_flip/bulk.rb', line 79

def index(id, object, options = {})
  perform(:index, id, SearchFlip::JSON.generate(object), options)
end

#update(id, object, options = {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Adds a update request to the bulk batch.

Parameters:

  • id (Fixnum, String) —

    The document/record id

  • json (String) —

    The json document

  • options (options) (defaults to: {}) —

    Options for the index request, like eg routing and versioning



115
116
117
 
# File 'lib/search_flip/bulk.rb', line 115

def update(id, object, options = {})
  perform(:update, id, SearchFlip::JSON.generate(object), options)
end