Class: ActiveRecord::Base

Inherits:
Object show all
Defined in:
lib/ultrasphinx/is_indexed.rb

Class Method Summary collapse

Class Method Details

.is_indexed(opts = {}) ⇒ Object

The is_indexed method configures a model for indexing. Its parameters help generate SQL queries for Sphinx.

Options

Including regular fields

Use the :fields key.

Accepts an array of field names or field hashes. 

:fields => [
  'created_at', 
  'title', 
  {:field => 'body', :as => 'description'},
  {:field => 'user_category', :facet => true, :as => 'category' }
]

To alias a field, pass a hash instead of a string and set the :as key.

To allow faceting support on a text field, also pass a hash and set the :facet key to true. Faceting is off by default for text fields because there is some indexing overhead associated with it. Faceting is always on for numeric or date fields.

To allow sorting by a text field, also pass a hash and set the :sortable key to true. This is turned off by default for the same reason as above. Sorting is always on for numeric or date fields.

To apply an SQL function to a field before it is indexed, use the key :function_sql. Pass a string such as "REPLACE(?, '_', ' ')". The table and column name for your field will be interpolated into the first ? in the string.

Note that float fields are supported, but require Sphinx 0.98.

Requiring conditions

Use the :conditions key.

SQL conditions, to scope which records are selected for indexing. Accepts a string. 

:conditions => "created_at < NOW() AND deleted IS NOT NULL"

The :conditions key is especially useful if you delete records by marking them deleted rather than removing them from the database.

Ordering subgroups

Use the :order key.

An SQL order string.

:order => 'posts.id ASC'

Including a field from an association

Use the :include key.

Accepts an array of hashes. 

:include => [{:association_name => 'category', :field => 'name', :as => 'category_name'}]

Each should contain an :association_name key (the association name for the included model), a :field key (the name of the field to include), and an optional :as key (what to name the field in the parent).

:include hashes also accept their own :conditions key. You can use this if you need custom WHERE conditions for this particular association (e.g, this JOIN).

The keys :facet, :sortable, :class_name, :association_sql, and :function_sql are also recognized.

Concatenating several fields within one record

Use the :concatenate key.

Accepts an array of option hashes.

To concatenate several fields within one record as a combined field, use a regular (or lateral) concatenation. Regular concatenations contain a :fields key (again, an array of field names), and a mandatory :as key (the name of the result of the concatenation). For example, to concatenate the title and body into one field called text: 

:concatenate => [{:fields => ['title', 'body'], :as => 'text'}]

The keys :facet, :sortable, :conditions, :function_sql, :class_name, and :association_sql, are also recognized.

Lateral concatenations are implemented with CONCAT_WS on MySQL and with a stored procedure on PostgreSQL.

Concatenating the same field from a set of associated records

Also use the :concatenate key.

To concatenate one field from a set of associated records as a combined field in the parent record, use a group (or vertical) concatenation. A group concatenation should contain an :association_name key (the association name for the included model), a :field key (the field on the included model to concatenate), and an optional :as key (also the name of the result of the concatenation). For example, to concatenate all Post#body contents into the parent’s responses field:

:concatenate => [{:association_name => 'posts', :field => 'body', :as => 'responses'}]

The keys :facet, :sortable, :order, :conditions, :function_sql, :class_name, and :association_sql, are also recognized.

Vertical concatenations are implemented with GROUP_CONCAT on MySQL and with an aggregate and a stored procedure on PostgreSQL. Note that :order is useful if you need to order the grouping so that proximity search works correctly, and :conditions are currently ignored if you have :association_sql defined.

Custom joins

:include and :concatenate accept an :association_sql key. You can use this if you need to pass a custom JOIN string, for example, a double JOIN for a has_many :through). If :association_sql is present, the default JOIN for belongs_to will not be generated.

Also, If you want to include a model that you don’t have an actual ActiveRecord association for, you can use :association_sql combined with :class_name instead of :association_name. :class_name should be camelcase.

Ultrasphinx is not an object-relational mapper, and the association generation is intended to stay minimal–don’t be afraid of :association_sql.

Enabling delta indexing

Use the :delta key.

Accepts either true, or a hash with a :field key.

If you pass true, the updated_at column will be used for choosing the delta records, if it exists. If it doesn’t exist, the entire table will be reindexed at every delta. Example:

:delta => true

If you need to use a non-default column name, use a hash:

:delta => {:field => 'created_at'}

Note that the column type must be time-comparable in the DB. Also note that faceting may return higher counts than actually exist on delta-indexed tables, and that sorting by string columns will not work well. These are both limitations of Sphinx’s index merge scheme. You can perhaps mitigate the issues by only searching the main index for facets or sorts:

Ultrasphinx::Search.new(:query => "query", :indexes => Ultrasphinx::MAIN_INDEX)

The date range of the delta include is set in the .base file.

Examples

Complex configuration

Here’s an example configuration using most of the options, taken from production code:

class Story < ActiveRecord::Base  
  is_indexed :fields => [
      'title', 
      'published_at',
      {:field => 'author', :facet => true}
    ],
    :include => [
      {:association_name => 'category', :field => 'name', :as => 'category_name'}
    ],      
    :concatenate => [
      {:fields => ['title', 'long_description', 'short_description'], 
        :as => 'editorial'},
      {:association_name => 'pages', :field => 'body', :as => 'body'},
      {:association_name => 'comments', :field => 'body', :as => 'comments', 
        :conditions => "comments.item_type = '#{base_class}'"}
    ],
    :delta => {:field => 'published_at'},
    :conditions => self.live_condition_string
end

Note how setting the :conditions on Comment is enough to configure a polymorphic has_many.

Association scoping

A common use case is to only search records that belong to a particular parent model. Ultrasphinx configures Sphinx to support a :filters element on any date or numeric field, so any *_id fields you have will be filterable.

For example, say a Company has_many :users and each User has_many :articles. If you want to to filter Articles by Company, add company_id to the Article’s is_indexed method. The best way is to grab it from the User association:

class Article < ActiveRecord::Base 
   is_indexed :include => [{:association_name => 'users', :field => 'company_id'}]
end

Now you can run:

@search = Ultrasphinx::Search.new('something', 
  :filters => {'company_id' => 493})

If the associations weren’t just has_many and belongs_to, you would need to use the :association_sql key to set up a custom JOIN.



168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
 
# File 'lib/ultrasphinx/is_indexed.rb', line 168

def self.is_indexed opts = {}    
  opts.stringify_keys!          
  opts.assert_valid_keys ['fields', 'concatenate', 'conditions', 'include', 'delta', 'order']

  # Single options
  
  if opts['conditions']
    # Do nothing
  end
  
  if opts['delta']
    if opts['delta'] == true
      opts['delta'] = {'field' => 'updated_at'} 
    elsif opts['delta'].is_a? String
      opts['delta'] = {'field' => opts['delta']} 
    end
    
    opts['delta']._stringify_all!
    opts['delta'].assert_valid_keys ['field']
  end
  
  # Enumerable options
  
  opts['fields'] = Array(opts['fields'])
  opts['concatenate'] = Array(opts['concatenate'])
  opts['include'] = Array(opts['include'])
              
  opts['fields'].map! do |entry|
    if entry.is_a? Hash
      entry._stringify_all!('sortable', 'facet')
      entry.assert_valid_keys ['field', 'as', 'facet', 'function_sql', 'sortable', 'table_alias']
      entry
    else
      # Single strings
      {'field' => entry.to_s}
    end
  end
  
  opts['concatenate'].each do |entry|
    entry._stringify_all!('fields', 'sortable', 'facet')
  
    entry.assert_valid_keys ['class_name', 'association_name', 'conditions', 'field', 'as', 'fields', 'association_sql', 'facet', 'function_sql', 'sortable', 'order', 'table_alias']
    raise Ultrasphinx::ConfigurationError, "You can't mix regular concat and group concats" if entry['fields'] and (entry['field'] or entry['class_name'] or entry['association_name'])
    raise Ultrasphinx::ConfigurationError, "Concatenations must specify an :as key" unless entry['as']
    raise Ultrasphinx::ConfigurationError, "Group concatenations must not have multiple fields" if entry['field'].is_a? Array
    raise Ultrasphinx::ConfigurationError, "Regular concatenations should have multiple fields" if entry['fields'] and !entry['fields'].is_a?(Array)
    raise Ultrasphinx::ConfigurationError, "Regular concatenations can't specify an order" if entry['fields'] and entry['order']

    entry['fields'].map!(&:to_s) if entry['fields'] # Stringify fields array
  end
  
  opts['include'].each do |entry|
    entry._stringify_all!('sortable', 'facet')
    entry.assert_valid_keys ['class_name', 'association_name', 'field', 'as', 'association_sql', 'facet', 'function_sql', 'sortable', 'table_alias', 'dont_group_by']
  end
              
  Ultrasphinx::MODEL_CONFIGURATION[self.name] = opts
end