Class: Ultrasphinx::Search

Inherits:
Object show all
Includes:
Internals, Parser
Defined in:
lib/ultrasphinx/search.rb,
lib/ultrasphinx/search/parser.rb,
lib/ultrasphinx/search/internals.rb

Overview

Command-interface Search object.

Basic usage

To set up a search, instantiate an Ultrasphinx::Search object with a hash of parameters. Only the :query key is mandatory.

@search = Ultrasphinx::Search.new(
  :query => @query, 
  :sort_mode => 'descending', 
  :sort_by => 'created_at'
)

Now, to run the query, call its run method. Your results will be available as ActiveRecord instances via the results method. Example:

@search.run
@search.results

Options

Query format

The query string supports boolean operation, parentheses, phrases, and field-specific search. Query words are stemmed and joined by an implicit AND by default.

  • Valid boolean operators are AND, OR, and NOT.

  • Field-specific searches should be formatted as fieldname:contents. (This will only work for text fields. For numeric and date fields, see the :filters parameter, below.)

  • Phrases must be enclosed in double quotes.

A Sphinx::SphinxInternalError will be raised on invalid queries. In general, queries can only be nested to one level. 

@query = 'dog OR cat OR "white tigers" NOT (lions OR bears) AND title:animals'

Hash parameters

The hash lets you customize internal aspects of the search.

:per_page

An integer. How many results per page.

:page

An integer. Which page of the results to return.

:class_names

An array or string. The class name of the model you want to search, an array of model names to search, or nil for all available models.

:sort_mode

'relevance' or 'ascending' or 'descending'. How to order the result set. Note that 'time' and 'extended' modes are available, but not tested.

:sort_by

A field name. What field to order by for 'ascending' or 'descending' mode. Has no effect for 'relevance'.

:weights

A hash. Text-field names and associated query weighting. The default weight for every field is 1.0. Example: :weights => {'title' => 2.0}

:filters

A hash. Names of numeric or date fields and associated values. You can use a single value, an array of values, or a range. (See the bottom of the ActiveRecord::Base page for an example.)

:facets

An array of fields for grouping/faceting. You can access the returned facet values and their result counts with the facets method.

:location

A hash. Specify the names of your latititude and longitude attributes as declared in your is_indexed calls. To sort the results by distance, set :sort_mode => 'extended' and :sort_by => 'distance asc'.

:indexes

An array of indexes to search. Currently only Ultrasphinx::MAIN_INDEX and Ultrasphinx::DELTA_INDEX are available. Defaults to both; changing this is rarely needed.

Query Defaults

Note that you can set up your own query defaults in environment.rb: 

self.class.query_defaults = HashWithIndifferentAccess.new({
  :per_page => 10,
  :sort_mode => 'relevance',
  :weights => {'title' => 2.0}
})

Advanced features

Geographic distance

If you pass a :location Hash, distance from the location in meters will be available in your result records via the distance accessor:

@search = Ultrasphinx::Search.new(:class_names => 'Point', 
          :query => 'pizza',
          :sort_mode => 'extended',
          :sort_by => 'distance',
          :location => {
            :lat => 40.3,
            :long => -73.6
          })

 @search.run.first.distance #=> 1402.4

Note that Sphinx expects lat/long to be indexed as radians. If you have degrees in your database, do the conversion in the is_indexed as so:

is_indexed 'fields' => [
    'name', 
    'description',
    {:field => 'lat', :function_sql => "RADIANS(?)"}, 
    {:field => 'lng', :function_sql => "RADIANS(?)"}
  ]

Then, set Ultrasphinx::Search.client_options[:location][:units] = 'degrees'.

The MySQL :double column type is recommended for storing location data. For Postgres, use <tt>:float</tt.

Interlock integration

Ultrasphinx uses the find_all_by_id method to instantiate records. If you set with_finders: true in Interlock’s config/memcached.yml, Interlock overrides find_all_by_id with a caching version.

Will_paginate integration

The Search instance responds to the same methods as a WillPaginate::Collection object, so once you have called run or excerpt you can use it directly in your views:

will_paginate(@search)

Excerpt mode

You can have Sphinx excerpt and highlight the matched sections in the associated fields. Instead of calling run, call excerpt. 

@search.excerpt

The returned models will be frozen and have their field contents temporarily changed to the excerpted and highlighted results.

You need to set the content_methods key on Ultrasphinx::Search.excerpting_options to whatever groups of methods you need the excerpter to try to excerpt. The first responding method in each group for each record will be excerpted. This way Ruby-only methods are supported (for example, a metadata method which combines various model fields, or an aliased field so that the original record contents are still available).

There are some other keys you can set, such as excerpt size, HTML tags to highlight with, and number of words on either side of each excerpt chunk. Example (in environment.rb):

Ultrasphinx::Search.excerpting_options = HashWithIndifferentAccess.new({
  :before_match => '<strong>', 
  :after_match => '</strong>',
  :chunk_separator => "...",
  :limit => 256,
  :around => 3,
  :content_methods => [['title'], ['body', 'description', 'content'], ['metadata']] 
})

Note that your database is never changed by anything Ultrasphinx does.

Defined Under Namespace

Modules: Internals, Parser

Constant Summary collapse

SPHINX_CLIENT_PARAMS =

Friendly sort mode mappings

{ 
  'sort_mode' => {
    'relevance' => :relevance,
    'descending' => :attr_desc, 
    'ascending' => :attr_asc, 
    'time' => :time_segments,
    'extended' => :extended,
  }
}
INTERNAL_KEYS =

:nodoc:

['parsed_query']
MODELS_TO_IDS = 
Ultrasphinx.get_models_to_class_ids || {}
IDS_TO_MODELS =

:nodoc:

MODELS_TO_IDS.invert
MAX_MATCHES = 
DAEMON_SETTINGS["max_matches"].to_i
FACET_CACHE =

:nodoc:

{}

Constants included from Parser

Parser::OPERATORS

Constants included from Internals

Internals::INFINITY

Instance Method Summary collapse

Methods included from Associations

#get_association, #get_association_model

Constructor Details

#initialize(opts = {}) ⇒ Search

Builds a new command-interface Search object.

Raises:



295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
 
# File 'lib/ultrasphinx/search.rb', line 295

def initialize opts = {} 

  # Change to normal hashes with String keys for speed
  opts = Hash[HashWithIndifferentAccess.new(opts._deep_dup._coerce_basic_types)]
  unless self.class.query_defaults.instance_of? Hash
    self.class.query_defaults = Hash[self.class.query_defaults]
    self.class.query_defaults['location'] = Hash[self.class.query_defaults['location']]
    
    self.class.client_options = Hash[self.class.client_options]
    self.class.excerpting_options = Hash[self.class.excerpting_options]
    self.class.excerpting_options['content_methods'].map! {|ary| ary.map {|m| m.to_s}}
  end    

  # We need an annoying deep merge on the :location parameter
  opts['location'].reverse_merge!(self.class.query_defaults['location']) if opts['location']

  # Merge the rest of the defaults      
  @options = self.class.query_defaults.merge(opts)
  
  @options['query'] = @options['query'].to_s
  @options['class_names'] = Array(@options['class_names'])
  @options['facets'] = Array(@options['facets'])
  @options['indexes'] = Array(@options['indexes']).join(" ")
        
  raise UsageError, "Weights must be a Hash" unless @options['weights'].is_a? Hash
  raise UsageError, "Filters must be a Hash" unless @options['filters'].is_a? Hash
  
  @options['parsed_query'] = parse(query)
  
  @results, @subtotals, @facets, @response = [], {}, {}, {}
    
  extra_keys = @options.keys - (self.class.query_defaults.keys + INTERNAL_KEYS)
  log "discarded invalid keys: #{extra_keys * ', '}" if extra_keys.any? and RAILS_ENV != "test" 
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(*args, &block) ⇒ Object

Delegates enumerable methods to @results, if possible. This allows us to behave directly like a WillPaginate::Collection. Failing that, we delegate to the options hash if a key is set. This lets us use self directly in view helpers.



431
432
433
434
435
436
437
438
439
 
# File 'lib/ultrasphinx/search.rb', line 431

def method_missing(*args, &block)
  if @results.respond_to? args.first
    @results.send(*args, &block)
  elsif options.has_key? args.first.to_s
    @options[args.first.to_s]
  else
    super
  end
end

Instance Method Details

#current_page ⇒ Object

Returns the current page number of the result set. (Page indexes begin at 1.)



264
265
266
 
# File 'lib/ultrasphinx/search.rb', line 264

def current_page
  @options['page']
end

#excerpt ⇒ Object

Overwrite the configured content attributes with excerpted and highlighted versions of themselves. Runs run if it hasn’t already been done.



369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
 
# File 'lib/ultrasphinx/search.rb', line 369

def excerpt

  require_run         
  return if results.empty?

  # See what fields in each result might respond to our excerptable methods
  results_with_content_methods = results.map do |result|
    [result, 
      self.class.excerpting_options['content_methods'].map do |methods|
        methods.detect do |this| 
          result.respond_to? this
        end
      end
    ]
  end
  
  # Fetch the actual field contents
  docs = results_with_content_methods.map do |result, methods|
    methods.map do |method| 
      method and strip_bogus_characters(result.send(method)) or ""
    end
  end.flatten
  
  excerpting_options = {
    :docs => docs,         
    :index => MAIN_INDEX, # http://www.sphinxsearch.com/forum/view.html?id=100
    :words => strip_query_commands(parsed_query)
  }
  self.class.excerpting_options.except('content_methods').each do |key, value|
    # Riddle only wants symbols
    excerpting_options[key.to_sym] ||= value
  end
  
  responses = perform_action_with_retries do 
    # Ship to Sphinx to highlight and excerpt
    @request.excerpts(excerpting_options)
  end
  
  responses = responses.in_groups_of(self.class.excerpting_options['content_methods'].size)
  
  results_with_content_methods.each_with_index do |result_and_methods, i|
    # Override the individual model accessors with the excerpted data
    result, methods = result_and_methods
    methods.each_with_index do |method, j|
      data = responses[i][j]
      if method
        result._metaclass.send('define_method', method) { data }
        attributes = result.instance_variable_get('@attributes')
        attributes[method] = data if attributes[method]
      end
    end
  end
  
  @results = results_with_content_methods.map do |result_and_content_method| 
    result_and_content_method.first.freeze
  end
  
  self
end

#facets ⇒ Object

Returns the facet map for this query, if facets were used.

Raises:



225
226
227
228
229
 
# File 'lib/ultrasphinx/search.rb', line 225

def facets
  raise UsageError, "No facet field was configured" unless @options['facets']
  require_run
  @facets
end

#log(msg) ⇒ Object

:nodoc:



441
442
443
 
# File 'lib/ultrasphinx/search.rb', line 441

def log msg #:nodoc:
  Ultrasphinx.log msg
end

#next_page ⇒ Object

Returns the next page number.



285
286
287
 
# File 'lib/ultrasphinx/search.rb', line 285

def next_page
  current_page < page_count ? (current_page + 1) : nil
end

#offset ⇒ Object

Returns the global index position of the first result on this page.



290
291
292
 
# File 'lib/ultrasphinx/search.rb', line 290

def offset 
  (current_page - 1) * per_page
end

#options ⇒ Object

Returns the options hash.



203
204
205
 
# File 'lib/ultrasphinx/search.rb', line 203

def options
  @options
end

#page_count ⇒ Object

Returns the last available page number in the result set.



274
275
276
277
 
# File 'lib/ultrasphinx/search.rb', line 274

def page_count
  require_run    
  (total_entries / per_page.to_f).ceil
end

#parsed_query ⇒ Object

:nodoc:



213
214
215
216
 
# File 'lib/ultrasphinx/search.rb', line 213

def parsed_query #:nodoc:
  # Redundant with method_missing
  @options['parsed_query']
end

#per_page ⇒ Object

Returns the number of records per page.



269
270
271
 
# File 'lib/ultrasphinx/search.rb', line 269

def per_page
  @options['per_page']
end

#previous_page ⇒ Object

Returns the previous page number.



280
281
282
 
# File 'lib/ultrasphinx/search.rb', line 280

def previous_page 
  current_page > 1 ? (current_page - 1) : nil
end

#query ⇒ Object

Returns the query string used.



208
209
210
211
 
# File 'lib/ultrasphinx/search.rb', line 208

def query
  # Redundant with method_missing
  @options['query']
end

#response ⇒ Object

Returns the raw response from the Sphinx client.



232
233
234
235
 
# File 'lib/ultrasphinx/search.rb', line 232

def response
  require_run
  @response
end

#results ⇒ Object

Returns an array of result objects.



219
220
221
222
 
# File 'lib/ultrasphinx/search.rb', line 219

def results
  require_run
  @results
end

#run(reify = true) ⇒ Object

Run the search, filling results with an array of ActiveRecord objects. Set the parameter to false if you only want the ids returned.



332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
 
# File 'lib/ultrasphinx/search.rb', line 332

def run(reify = true)
  @request = build_request_with_options(@options)

  log "searching for #{@options.inspect}"

  perform_action_with_retries do
    @response = @request.query(parsed_query, @options['indexes'])
    log "search returned #{total_entries}/#{response[:total_found].to_i} in #{time.to_f} seconds."
      
    if self.class.client_options['with_subtotals']        
      @subtotals = get_subtotals(@request, parsed_query) 
      
      # If the original query has a filter on this class, we will use its more accurate total rather the facet's 
      # less accurate total.
      if @options['class_names'].size == 1
        @subtotals[@options['class_names'].first] = response[:total_found]
      end
      
    end
    
    Array(@options['facets']).each do |facet|
      @facets[facet] = get_facets(@request, parsed_query, facet)
    end        
    
    @results = convert_sphinx_ids(response[:matches])
    @results = reify_results(@results) if reify
    
    say "warning; #{response[:warning]}" if response[:warning]
    raise UsageError, response[:error] if response[:error]
    
  end      
  self
end

#run? ⇒ Boolean

Returns whether the query has been run.

Returns:

  • (Boolean) 


259
260
261
 
# File 'lib/ultrasphinx/search.rb', line 259

def run?
  !@response.blank?
end

#say(msg) ⇒ Object

:nodoc:



445
446
447
 
# File 'lib/ultrasphinx/search.rb', line 445

def say msg #:nodoc:
  Ultrasphinx.say msg
end

#subtotals ⇒ Object

Returns a hash of total result counts, scoped to each available model. Set Ultrasphinx::Search.client_options[:with_subtotals] = true to enable.

The subtotals are implemented as a special type of facet.

Raises:



240
241
242
243
244
 
# File 'lib/ultrasphinx/search.rb', line 240

def subtotals
  raise UsageError, "Subtotals are not enabled" unless self.class.client_options['with_subtotals']
  require_run
  @subtotals
end

#time ⇒ Object

Returns the response time of the query, in milliseconds.



253
254
255
256
 
# File 'lib/ultrasphinx/search.rb', line 253

def time
  require_run
  response[:time]
end

#total_entries ⇒ Object

Returns the total result count.



247
248
249
250
 
# File 'lib/ultrasphinx/search.rb', line 247

def total_entries
  require_run
  [response[:total_found] || 0, MAX_MATCHES].min
end