Class: Riddle::Client
- Inherits:
-
Object
- Object
- Riddle::Client
- Defined in:
- lib/riddle/client.rb,
lib/riddle/1.10/client.rb,
lib/riddle/0.9.9/client.rb,
lib/riddle/client/filter.rb,
lib/riddle/client/message.rb,
lib/riddle/client/response.rb
Overview
This class was heavily based on the existing Client API by Dmytro Shteflyuk and Alexy Kovyrin. Their code worked fine, I just wanted something a bit more Ruby-ish (ie. lowercase and underscored method names). I also have used a few helper classes, just to neaten things up.
Feel free to use it wherever. Send bug reports, patches, comments and suggestions to pat at freelancing-gods dot com.
Most properties of the client are accessible through attribute accessors, and where relevant use symboles instead of the long constants common in other clients. Some examples:
client.sort_mode = :extended
client.sort_by = "birthday DESC"
client.match_mode = :extended
To add a filter, you will need to create a Filter object:
client.filters << Riddle::Client::Filter.new("birthday",
Time.at(1975, 1, 1).to_i..Time.at(1985, 1, 1).to_i, false)
Defined Under Namespace
Classes: Filter, Message, Response
Constant Summary collapse
- Commands =
{ :search => 0, # SEARCHD_COMMAND_SEARCH :excerpt => 1, # SEARCHD_COMMAND_EXCERPT :update => 2, # SEARCHD_COMMAND_UPDATE :keywords => 3, # SEARCHD_COMMAND_KEYWORDS :persist => 4, # SEARCHD_COMMAND_PERSIST :status => 5, # SEARCHD_COMMAND_STATUS :query => 6, # SEARCHD_COMMAND_QUERY :flushattrs => 7 # SEARCHD_COMMAND_FLUSHATTRS }
- Versions =
{ :search => 0x113, # VER_COMMAND_SEARCH :excerpt => 0x100, # VER_COMMAND_EXCERPT :update => 0x101, # VER_COMMAND_UPDATE :keywords => 0x100, # VER_COMMAND_KEYWORDS :status => 0x100, # VER_COMMAND_STATUS :query => 0x100, # VER_COMMAND_QUERY :flushattrs => 0x100 # VER_COMMAND_FLUSHATTRS }
- Statuses =
{ :ok => 0, # SEARCHD_OK :error => 1, # SEARCHD_ERROR :retry => 2, # SEARCHD_RETRY :warning => 3 # SEARCHD_WARNING }
- MatchModes =
{ :all => 0, # SPH_MATCH_ALL :any => 1, # SPH_MATCH_ANY :phrase => 2, # SPH_MATCH_PHRASE :boolean => 3, # SPH_MATCH_BOOLEAN :extended => 4, # SPH_MATCH_EXTENDED :fullscan => 5, # SPH_MATCH_FULLSCAN :extended2 => 6 # SPH_MATCH_EXTENDED2 }
- RankModes =
{ :proximity_bm25 => 0, # SPH_RANK_PROXIMITY_BM25 :bm25 => 1, # SPH_RANK_BM25 :none => 2, # SPH_RANK_NONE :wordcount => 3, # SPH_RANK_WORDCOUNT :proximity => 4, # SPH_RANK_PROXIMITY :match_any => 5, # SPH_RANK_MATCHANY :fieldmask => 6, # SPH_RANK_FIELDMASK :sph04 => 7, # SPH_RANK_SPH04 :total => 8 # SPH_RANK_TOTAL }
- SortModes =
{ :relevance => 0, # SPH_SORT_RELEVANCE :attr_desc => 1, # SPH_SORT_ATTR_DESC :attr_asc => 2, # SPH_SORT_ATTR_ASC :time_segments => 3, # SPH_SORT_TIME_SEGMENTS :extended => 4, # SPH_SORT_EXTENDED :expr => 5 # SPH_SORT_EXPR }
- AttributeTypes =
{ :integer => 1, # SPH_ATTR_INTEGER :timestamp => 2, # SPH_ATTR_TIMESTAMP :ordinal => 3, # SPH_ATTR_ORDINAL :bool => 4, # SPH_ATTR_BOOL :float => 5, # SPH_ATTR_FLOAT :bigint => 6, # SPH_ATTR_BIGINT :string => 7, # SPH_ATTR_STRING :multi => 0x40000000 # SPH_ATTR_MULTI }
- GroupFunctions =
{ :day => 0, # SPH_GROUPBY_DAY :week => 1, # SPH_GROUPBY_WEEK :month => 2, # SPH_GROUPBY_MONTH :year => 3, # SPH_GROUPBY_YEAR :attr => 4, # SPH_GROUPBY_ATTR :attrpair => 5 # SPH_GROUPBY_ATTRPAIR }
- FilterTypes =
{ :values => 0, # SPH_FILTER_VALUES :range => 1, # SPH_FILTER_RANGE :float_range => 2 # SPH_FILTER_FLOATRANGE }
- @@connection =
nil
Instance Attribute Summary collapse
-
#anchor ⇒ Object
Returns the value of attribute anchor.
-
#connection ⇒ Object
Returns the value of attribute connection.
-
#cut_off ⇒ Object
Returns the value of attribute cut_off.
-
#field_weights ⇒ Object
Returns the value of attribute field_weights.
-
#filters ⇒ Object
Returns the value of attribute filters.
-
#group_by ⇒ Object
Returns the value of attribute group_by.
-
#group_clause ⇒ Object
Returns the value of attribute group_clause.
-
#group_distinct ⇒ Object
Returns the value of attribute group_distinct.
-
#group_function ⇒ Object
Returns the value of attribute group_function.
-
#id_range ⇒ Object
Returns the value of attribute id_range.
-
#index_weights ⇒ Object
Returns the value of attribute index_weights.
-
#key ⇒ Object
Returns the value of attribute key.
-
#limit ⇒ Object
Returns the value of attribute limit.
-
#match_mode ⇒ Object
Returns the value of attribute match_mode.
-
#max_matches ⇒ Object
Returns the value of attribute max_matches.
-
#max_query_time ⇒ Object
Returns the value of attribute max_query_time.
-
#offset ⇒ Object
Returns the value of attribute offset.
-
#overrides ⇒ Object
Returns the value of attribute overrides.
-
#port ⇒ Object
Returns the value of attribute port.
-
#queue ⇒ Object
readonly
Returns the value of attribute queue.
-
#rank_expr ⇒ Object
Returns the value of attribute rank_expr.
-
#rank_mode ⇒ Object
Returns the value of attribute rank_mode.
-
#retry_count ⇒ Object
Returns the value of attribute retry_count.
-
#retry_delay ⇒ Object
Returns the value of attribute retry_delay.
-
#select ⇒ Object
Returns the value of attribute select.
-
#servers ⇒ Object
Returns the value of attribute servers.
-
#sort_by ⇒ Object
Returns the value of attribute sort_by.
-
#sort_mode ⇒ Object
Returns the value of attribute sort_mode.
-
#timeout ⇒ Object
Returns the value of attribute timeout.
-
#weights ⇒ Object
Returns the value of attribute weights.
Class Method Summary collapse
Instance Method Summary collapse
- #add_override(attribute, type, values) ⇒ Object
-
#append_query(search, index = '*', comments = '') ⇒ Object
Append a query to the queue.
- #close ⇒ Object
-
#excerpts(options = {}) ⇒ Object
Build excerpts from search terms (the
words
) and the text of documents. - #flush_attributes ⇒ Object
-
#initialize(servers = nil, port = nil, key = nil) ⇒ Client
constructor
Can instantiate with a specific server and port - otherwise it assumes defaults of localhost and 9312 respectively.
-
#keywords(query, index, return_hits = false) ⇒ Object
Generates a keyword list for a given query.
- #open ⇒ Object
-
#query(search, index = '*', comments = '') ⇒ Object
Query the Sphinx daemon - defaulting to all indices, but you can specify a specific one if you wish.
-
#reset ⇒ Object
Reset attributes and settings to defaults.
-
#run ⇒ Object
Run all the queries currently in the queue.
-
#server ⇒ Object
The searchd server to query.
-
#server=(server) ⇒ Object
Backwards compatible writer to the @servers array.
-
#set_anchor(lat_attr, lat, long_attr, long) ⇒ Object
Set the geo-anchor point - with the names of the attributes that contain the latitude and longitude (in radians), and the reference position.
- #status ⇒ Object
-
#update(index, attributes, values_by_doc) ⇒ Object
Update attributes - first parameter is the relevant index, second is an array of attributes to be updated, and the third is a hash, where the keys are the document ids, and the values are arrays with the attribute values - in the same order as the second parameter.
Constructor Details
#initialize(servers = nil, port = nil, key = nil) ⇒ Client
Can instantiate with a specific server and port - otherwise it assumes defaults of localhost and 9312 respectively. All other settings can be accessed and changed via the attribute accessors.
143 144 145 146 147 148 149 150 151 152 153 154 |
# File 'lib/riddle/client.rb', line 143 def initialize(servers = nil, port = nil, key = nil) Riddle.version_warning @servers = Array(servers || "localhost") @port = port || 9312 @socket = nil @key = key reset @queue = [] end |
Instance Attribute Details
#anchor ⇒ Object
Returns the value of attribute anchor.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def anchor @anchor end |
#connection ⇒ Object
Returns the value of attribute connection.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def connection @connection end |
#cut_off ⇒ Object
Returns the value of attribute cut_off.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def cut_off @cut_off end |
#field_weights ⇒ Object
Returns the value of attribute field_weights.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def field_weights @field_weights end |
#filters ⇒ Object
Returns the value of attribute filters.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def filters @filters end |
#group_by ⇒ Object
Returns the value of attribute group_by.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def group_by @group_by end |
#group_clause ⇒ Object
Returns the value of attribute group_clause.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def group_clause @group_clause end |
#group_distinct ⇒ Object
Returns the value of attribute group_distinct.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def group_distinct @group_distinct end |
#group_function ⇒ Object
Returns the value of attribute group_function.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def group_function @group_function end |
#id_range ⇒ Object
Returns the value of attribute id_range.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def id_range @id_range end |
#index_weights ⇒ Object
Returns the value of attribute index_weights.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def index_weights @index_weights end |
#key ⇒ Object
Returns the value of attribute key.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def key @key end |
#limit ⇒ Object
Returns the value of attribute limit.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def limit @limit end |
#match_mode ⇒ Object
Returns the value of attribute match_mode.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def match_mode @match_mode end |
#max_matches ⇒ Object
Returns the value of attribute max_matches.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def max_matches @max_matches end |
#max_query_time ⇒ Object
Returns the value of attribute max_query_time.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def max_query_time @max_query_time end |
#offset ⇒ Object
Returns the value of attribute offset.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def offset @offset end |
#overrides ⇒ Object
Returns the value of attribute overrides.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def overrides @overrides end |
#port ⇒ Object
Returns the value of attribute port.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def port @port end |
#queue ⇒ Object (readonly)
Returns the value of attribute queue.
126 127 128 |
# File 'lib/riddle/client.rb', line 126 def queue @queue end |
#rank_expr ⇒ Object
Returns the value of attribute rank_expr.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def rank_expr @rank_expr end |
#rank_mode ⇒ Object
Returns the value of attribute rank_mode.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def rank_mode @rank_mode end |
#retry_count ⇒ Object
Returns the value of attribute retry_count.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def retry_count @retry_count end |
#retry_delay ⇒ Object
Returns the value of attribute retry_delay.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def retry_delay @retry_delay end |
#select ⇒ Object
Returns the value of attribute select.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def select @select end |
#servers ⇒ Object
Returns the value of attribute servers.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def servers @servers end |
#sort_by ⇒ Object
Returns the value of attribute sort_by.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def sort_by @sort_by end |
#sort_mode ⇒ Object
Returns the value of attribute sort_mode.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def sort_mode @sort_mode end |
#timeout ⇒ Object
Returns the value of attribute timeout.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def timeout @timeout end |
#weights ⇒ Object
Returns the value of attribute weights.
120 121 122 |
# File 'lib/riddle/client.rb', line 120 def weights @weights end |
Class Method Details
.connection ⇒ Object
136 137 138 |
# File 'lib/riddle/client.rb', line 136 def self.connection @@connection end |
Instance Method Details
#add_override(attribute, type, values) ⇒ Object
484 485 486 |
# File 'lib/riddle/client.rb', line 484 def add_override(attribute, type, values) @overrides[attribute] = {:type => type, :values => values} end |
#append_query(search, index = '*', comments = '') ⇒ Object
Append a query to the queue. This uses the same parameters as the query method.
221 222 223 |
# File 'lib/riddle/client.rb', line 221 def append_query(search, index = '*', comments = '') @queue << (search, index, comments) end |
#close ⇒ Object
496 497 498 |
# File 'lib/riddle/client.rb', line 496 def close close_socket end |
#excerpts(options = {}) ⇒ Object
Build excerpts from search terms (the words
) and the text of documents. Excerpts are bodies of text that have the words
highlighted. They may also be abbreviated to fit within a word limit.
As part of the options hash, you will need to define:
-
:docs
-
:words
-
:index
Optional settings include:
-
:before_match (defaults to <span class=“match”>)
-
:after_match (defaults to </span>)
-
:chunk_separator (defaults to ‘ … ’ - which is an HTML ellipsis)
-
:limit (defaults to 256)
-
:around (defaults to 5)
-
:exact_phrase (defaults to false)
-
:single_passage (defaults to false)
The defaults differ from the official PHP client, as I’ve opted for semantic HTML markup.
Example:
client.excerpts(:docs => ["Pat Allan, Pat Cash"], :words => 'Pat', :index => 'pats')
#=> ["<span class=\"match\">Pat</span> Allan, <span class=\"match\">Pat</span> Cash"]
lorem_lipsum = "Lorem ipsum dolor..."
client.excerpts(:docs => ["Pat Allan, #{lorem_lipsum} Pat Cash"], :words => 'Pat', :index => 'pats')
#=> ["<span class=\"match\">Pat</span> Allan, Lorem ipsum dolor sit amet, consectetur adipisicing
elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua … . Excepteur
sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est
laborum. <span class=\"match\">Pat</span> Cash"]
Workflow:
Excerpt creation is completely isolated from searching the index. The nominated index is only used to discover encoding and charset information.
Therefore, the workflow goes:
-
Do the sphinx query.
-
Fetch the documents found by sphinx from their repositories.
-
Pass the documents’ text to
excerpts
for marking up of matched terms.
395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 |
# File 'lib/riddle/client.rb', line 395 def excerpts( = {}) [:index] ||= '*' [:before_match] ||= '<span class="match">' [:after_match] ||= '</span>' [:chunk_separator] ||= ' … ' # ellipsis [:limit] ||= 256 [:limit_passages] ||= 0 [:limit_words] ||= 0 [:around] ||= 5 [:exact_phrase] ||= false [:single_passage] ||= false [:query_mode] ||= false [:force_all_words] ||= false [:start_passage_id] ||= 1 [:load_files] ||= false [:html_strip_mode] ||= 'index' [:allow_empty] ||= false [:passage_boundary] ||= 'none' [:emit_zones] ||= false [:load_files_scattered] ||= false response = Response.new request(:excerpt, ()) [:docs].collect { response.next } end |
#flush_attributes ⇒ Object
476 477 478 479 480 481 482 |
# File 'lib/riddle/client.rb', line 476 def flush_attributes response = Response.new request( :flushattrs, Message.new ) response.next_int end |
#keywords(query, index, return_hits = false) ⇒ Object
Generates a keyword list for a given query. Each keyword is represented by a hash, with keys :tokenised and :normalised. If return_hits is set to true it will also report on the number of hits and documents for each keyword (see :hits and :docs keys respectively).
443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 |
# File 'lib/riddle/client.rb', line 443 def keywords(query, index, return_hits = false) response = Response.new request( :keywords, (query, index, return_hits) ) (0...response.next_int).collect do hash = {} hash[:tokenised] = response.next hash[:normalised] = response.next if return_hits hash[:docs] = response.next_int hash[:hits] = response.next_int end hash end end |
#open ⇒ Object
488 489 490 491 492 493 494 |
# File 'lib/riddle/client.rb', line 488 def open open_socket return if Versions[:search] < 0x116 @socket.send request_header(:persist) + [1].pack('N'), 0 end |
#query(search, index = '*', comments = '') ⇒ Object
Query the Sphinx daemon - defaulting to all indices, but you can specify a specific one if you wish. The search parameter should be a string following Sphinx’s expectations.
The object returned from this method is a hash with the following keys:
-
:matches
-
:fields
-
:attributes
-
:attribute_names
-
:words
-
:total
-
:total_found
-
:time
-
:status
-
:warning (if appropriate)
-
:error (if appropriate)
The key :matches
returns an array of hashes - the actual search results. Each hash has the document id (:doc
), the result weighting (:weight
), and a hash of the attributes for the document (:attributes
).
The :fields
and :attribute_names
keys return list of fields and attributes for the documents. The key :attributes
will return a hash of attribute name and type pairs, and :words
returns a hash of hashes representing the words from the search, with the number of documents and hits for each, along the lines of:
results[:words]["Pat"] #=> {:docs => 12, :hits => 15}
:total
, :total_found
and :time
return the number of matches available, the total number of matches (which may be greater than the maximum available, depending on the number of matches and your sphinx configuration), and the time in milliseconds that the query took to run.
:status
is the error code for the query - and if there was a related warning, it will be under the :warning
key. Fatal errors will be described under :error
.
344 345 346 347 348 |
# File 'lib/riddle/client.rb', line 344 def query(search, index = '*', comments = '') @queue.clear @queue << (search, index, comments) self.run.first end |
#reset ⇒ Object
Reset attributes and settings to defaults.
157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 |
# File 'lib/riddle/client.rb', line 157 def reset # defaults @offset = 0 @limit = 20 @max_matches = 1000 @match_mode = :all @sort_mode = :relevance @sort_by = '' @weights = [] @id_range = 0..0 @filters = [] @group_by = '' @group_function = :day @group_clause = '@weight DESC' @group_distinct = '' @cut_off = 0 @retry_count = 0 @retry_delay = 0 @anchor = {} # string keys are index names, integer values are weightings @index_weights = {} @rank_mode = :proximity_bm25 @rank_expr = '' @max_query_time = 0 # string keys are field names, integer values are weightings @field_weights = {} @timeout = 0 @overrides = {} @select = "*" end |
#run ⇒ Object
Run all the queries currently in the queue. This will return an array of results hashes.
227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 |
# File 'lib/riddle/client.rb', line 227 def run response = Response.new request(:search, @queue) results = @queue.collect do result = { :matches => [], :fields => [], :attributes => {}, :attribute_names => [], :words => {} } result[:status] = response.next_int case result[:status] when Statuses[:warning] result[:warning] = response.next when Statuses[:error] result[:error] = response.next next result end result[:fields] = response.next_array attributes = response.next_int attributes.times do attribute_name = response.next type = response.next_int result[:attributes][attribute_name] = type result[:attribute_names] << attribute_name end result_attribute_names_and_types = result[:attribute_names]. inject([]) { |array, attr| array.push([ attr, result[:attributes][attr] ]) } matches = response.next_int is_64_bit = response.next_int result[:matches] = (0...matches).map do |i| doc = is_64_bit > 0 ? response.next_64bit_int : response.next_int weight = response.next_int current_match_attributes = {} result_attribute_names_and_types.each do |attr, type| current_match_attributes[attr] = attribute_from_type(type, response) end {:doc => doc, :weight => weight, :index => i, :attributes => current_match_attributes} end result[:total] = response.next_int.to_i || 0 result[:total_found] = response.next_int.to_i || 0 result[:time] = ('%.3f' % (response.next_int / 1000.0)).to_f || 0.0 words = response.next_int words.times do word = response.next docs = response.next_int hits = response.next_int result[:words][word] = {:docs => docs, :hits => hits} end result end @queue.clear results rescue Riddle::OutOfBoundsError => error raise error rescue => original error = ResponseError.new original. error.original = original raise error end |
#server ⇒ Object
The searchd server to query. Servers are removed from @server after a Timeout::Error is hit to allow for fail-over.
190 191 192 |
# File 'lib/riddle/client.rb', line 190 def server @servers.first end |
#server=(server) ⇒ Object
Backwards compatible writer to the @servers array.
195 196 197 |
# File 'lib/riddle/client.rb', line 195 def server=(server) @servers = server.to_a end |
#set_anchor(lat_attr, lat, long_attr, long) ⇒ Object
Set the geo-anchor point - with the names of the attributes that contain the latitude and longitude (in radians), and the reference position. Note that for geocoding to work properly, you must also set match_mode to :extended. To sort results by distance, you will need to set sort_by to ‘@geodist asc’, and sort_mode to extended (as an example). Sphinx expects latitude and longitude to be returned from you SQL source in radians.
Example:
client.set_anchor('lat', -0.6591741, 'long', 2.530770)
210 211 212 213 214 215 216 217 |
# File 'lib/riddle/client.rb', line 210 def set_anchor(lat_attr, lat, long_attr, long) @anchor = { :latitude_attribute => lat_attr, :latitude => lat, :longitude_attribute => long_attr, :longitude => long } end |
#status ⇒ Object
463 464 465 466 467 468 469 470 471 472 473 474 |
# File 'lib/riddle/client.rb', line 463 def status response = Response.new request( :status, Message.new ) rows, cols = response.next_int, response.next_int (0...rows).inject({}) do |hash, row| hash[response.next.to_sym] = response.next hash end end |
#update(index, attributes, values_by_doc) ⇒ Object
Update attributes - first parameter is the relevant index, second is an array of attributes to be updated, and the third is a hash, where the keys are the document ids, and the values are arrays with the attribute values - in the same order as the second parameter.
Example:
client.update('people', ['birthday'], {1 => [Time.at(1982, 20, 8).to_i]})
430 431 432 433 434 435 436 437 |
# File 'lib/riddle/client.rb', line 430 def update(index, attributes, values_by_doc) response = Response.new request( :update, (index, attributes, values_by_doc) ) response.next_int end |