Class: Scrivito::ObjSearchEnumerator

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
app/cms/scrivito/obj_search_enumerator.rb,
app/cms/scrivito/obj_search_enumerator/batch.rb,
app/cms/scrivito/obj_search_enumerator/facet_query.rb,
app/cms/scrivito/obj_search_enumerator/batch_iterator.rb,
app/cms/scrivito/obj_search_enumerator/query_executor.rb

Overview

Provides an enumerator for iterating over the results of searches for CMS objects to retrieve instances of these objects. This is achieved through the Enumerable mixin, which provides methods such as map, select or take.

This enumerator is lazy. If, for example, you are looking for Objs whose object class is Publication, and there are 93 objects in total, then enum.take(10) fetches the first 10 objects only, ignoring the other 83. This implies that repeatedly iterating over this enumerator causes the search results and the objects to be fetched again and again. If you want to get all objects at once, use enum.to_a.

To start searching, use one of the Obj methods that return an ObjSearchEnumerator. The preferred way is to start with Obj.where or Obj.all.

Currently available fields and their values

:*

Searches all fields. This is only possible with the contains, contains_prefix, refers_to and links_to operators.

:id

Id of an Obj. This is a string field.

:_path

Path of an Obj. This is a string field.

:_name

Name of an Obj. This is a string field.

:_obj_class

Object class of an Obj. This is a string field.

:_permalink

Permalink of an Obj. This is a string field.

:_created_at

The creation date of an Obj.

:_last_changed

Date of last change to an Obj.

every :custom_attribute

Custom attribute of an Obj. Note that depending on the attribute type (e.g. an html field), some operators cannot be applied.

Meta Data

If an Obj has a binary attribute named blob, the meta data of this attribute is searchable. For a full list of the available meta data attributes, see the documentation of the MetaDataCollection. The meta data attribute name needs to be prefixed with blob: when searching for it. So, for example, when searching for the width, you need to specify the attribute name using blob:width. Binary attributes other than blob are not searchable.

Currently available operators

contains and contains_prefix

These operators are intended for full text search of natural language texts. They are applicable to string, stringlist, enum, multienum and html fields.

For contains and contains_prefix, the examples are based on the following field value: “Behind every cloud is another cloud.”

:contains

Searches for one or more whole words. Each word needs to be present.

Example subquery values:

✔ “behind cloud” (case insensitive)

✘ “behi clo” (not whole words)

✘ “behind everything” (second word does not match)

:contains_prefix

Searches for a word prefix.

Example subquery values:

✔ “Clou” (case insensitive)

✔ “Every” (case insensitive)

equals

The equals operator is intended for programmatic comparisons of string and date values.

The operator has some limits with regard to string length. String values are only guaranteed to be considered if they are at most 1000 characters in length. String values of more than 1000 characters may be ignored by these operators.

For equals, the examples are based on the following field value: “Some content.”

:equals

The field value needs to be identical to the value of this subquery.

Applicable to string, stringlist, enum, multienum, float, integer and date fields.

Example subquery values:

✔ “Some content.” (exact value)

✘ “Some” (not exact value)

starts_with

The starts_with is intended for programmatic comparisons of string values.

The starts_with operator has a precision limit: Only prefixes of up to 20 characters are guaranteed to be matched. If you supply a prefix of more than 20 characters, the additional characters may be ignored.

When combined with the system attribute _path, the operator starts_with has some special functionality: There is not precision limit, i.e. a prefix of arbitrary length may be used to match on _path. Also, prefix matching on _path automatically matches entire path components, i.e. the prefix matching is delimited by slashes (the character ‘/’).

For starts_with, the examples are based on the following field value: “Some content.”

:starts_with

The field value needs to start exactly with the value of this subquery.

Applicable to string, stringlist, enum and multienum fields.

Example subquery values:

✔ “Som” (prefix of the value)

✘ “som” (incorrect case of prefix)

✘ “content” (not prefix of the whole value)

is_less_than and is_greater_than

These operators are intended for comparing date, integer, or float values. It only considers attributes of Objs and not of Widgets. Therefore, Widget attributes are not searchable using the is_less_than and is_greater_than operators.

For is_less_than and is_greater_than, the examples are based on the following date value: Time.new(2000,01,01,00,00,00)

:is_less_than

Matches if the field value is less than the subquery string value.

Example subquery values:

✔ Time.new(1999,12,31,23,59,59) (is less than)

✘ Time.new(2000,01,01,00,00,00) (equal, not less than)

:is_greater_than

Matches if the field value is greater than the subquery string value.

Example subquery values:

✔ Time.new(2000,01,01,00,00,01) (is greater than)

✘ Time.new(2000,01,01,00,00,00) (equal, not greater than)

For is_less_than and is_greater_than, the examples are based on the following float value: 23.42

:is_less_than

Matches if the field value is less than the subquery numeric value.

Example subquery values:

✔ 23.41 (is less than)

✔ 5 (is less than)

✘ 23.42 (equal, not less than)

:is_greater_than

Matches if the field value is greater than the subquery numeric value.

Example subquery values:

✔ 23.43 (is greater than)

✔ 42 (is greater than)

✘ 23.42 (equal, not greater than)

The links_to operator searches for CMS objects containing one or more attributes linking to specific CMS objects. So the operator returns the CMS objects in which at least one html, link, linklist, reference or referencelist attribute links to specific CMS objects.

The operator can only be applied to all attributes, so the “*” wildcard must be specified for the attributes to search. If you want to search specific reference or referencelist attributes, please use the refers_to operator.

Using nil instead of an instance of Obj raises an error.

Note that, in contrast to the refers_to operator, the links_to operator searches the attributes directly part of the CMS objects as well as the attributes contained in widgets.

:links_to

Searches for CMS objects linking to a specific CMS object.

Example subquery values:

✔ my_obj (an instance of Obj)

✔ [my_obj1, my_obj2] (an Array of instances of Obj)

✘ nil (not an instance of Obj)

✘ “some_string” (not an instance of Obj)

refers_to

The refers_to operator searches for CMS objects in which at least one of the specified reference or referencelist attributes refers to specific CMS objects.

Using the “*” wildcard for the attributes to search causes all reference and referencelist attributes of the searched CMS objects to be taken into account.

Using nil instead of Objs searches for all CMS objects in which none of the specified attributes refer to a CMS object.

Note that, in contrast to the links_to operator, the refers_to operator only searches attributes directly part of the CMS objects. Currently, attributes contained in widgets are not searched.

:refers_to

Searches for CMS objects in which specific reference or referencelist attributes refer to specific CMS objects.

Example subquery values:

✔ my_obj (an instance of Obj)

✔ [my_obj1, my_obj2] (an Array of instances of Obj)

✔ nil

✘ “some_string” (not an instance of Obj)

Matching multienum and stringlist

Attributes of type multienum and stringlist contain an array of strings. Each of these strings is searched individually. A search query matches a multienum or stringlist, if at least one string in the list matches. Example: A query using the operator :equals and the value “Eggs” matches an Obj containing [“Spam”,“Eggs”] in a stringlist or multienum attribute.

Limits

The number of chainable subqueries is limited. The limit depends on the number of values, fields, and boost parameters requested, as well as the number of words in a free text search.

Concurrent changes

Please be aware that concurrent changes can change the search result and can yield incomplete results. This is due to the fact, that search results are lazily loaded in batches. If you want to modify the result of a search, please call to_a first.

Examples:

Concurrent changes

# bad example
books = Book.where(:price, :equals, 10.99)
books.map { |book| book.update(price: 9.99) }

# good example
books = Book.where(:price, :equals, 10.99).to_a
books.map { |book| book.update(price: 9.99) }

Chainable methods collapse

Instance Method Summary collapse

Instance Method Details

#and(field, operator, value, boost = nil) ⇒ Scrivito::ObjSearchEnumerator

Adds the given AND subquery to this Scrivito::ObjSearchEnumerator.

Compares the field(s) with the value(s) using the operator of this subquery. All CMS objects to which this criterion applies remain in the result set.

Parameters:

  • field (Symbol, String, Array<Symbol, String>) —

    Name(s) of the field(s) to be searched. For arrays, the subquery matches if one or more of these fields meet this criterion.

  • operator (Symbol, String) —

    See “Currently available operators” above.

  • value (String, Integer, Float, Date, Time, Array<String, Date, Time, Float, Integer>) —

    The value(s) to compare with the field value(s) using the operator of this subquery. For arrays, the subquery matches if the condition is met for one or more of the array elements.

  • boost (Hash) (defaults to: nil) —

    A hash where the keys are field names and their values are boosting factors. Boosting factors must be in the range from 1 to 10. Boosting can only be applied to subqueries in which the contains or contains_prefix operator is used.

Returns:



293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 293

def and(field, operator, value, boost = nil)
  symbolized_operator = ensure_symbol_and_validate_operator(operator)

  subquery = { field: field, operator: symbolized_operator, value: convert_value(value) }
  if boost.present?
    valid_boost_operators = [:contains, :contains_prefix]
    if valid_boost_operators.include?(symbolized_operator)
      subquery[:boost] = boost
    else
      raise "Boost is not allowed with operator '#{operator}'. " +
          "Valid operators are: #{valid_boost_operators.join(', ')}"
    end
  end
  reset_for_changed_query
  @query = (query || []) + [subquery]

  self
end

#and_not(field, operator, value) ⇒ Scrivito::ObjSearchEnumerator

Adds the given negated AND subquery to this Scrivito::ObjSearchEnumerator.

Compares the field(s) with the value(s) using the negated operator of this subquery. All CMS objects to which this criterion applies are removed from the result set.

Parameters:

  • field (Symbol, String, Array<Symbol, String>) —

    Name(s) of the field(s) to be searched. For arrays, the subquery matches if one or more of these fields meet this criterion.

  • operator (Symbol, String) —

    Must be one of: equals, starts_with, is_greater_than, is_less_than, links_to, refers_to. (See “Currently available operators” above).

  • value (String, Date, Time, Integer, Float Array<String, Date, Time, Integer, Float>) —

    The value(s) to compare with the field value(s) using the operator of this subquery. For arrays, the subquery matches if the condition is met for one or more of the array elements.

Returns:



327
328
329
330
331
332
333
334
335
336
337
338
339
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 327

def and_not(field, operator, value)
  if INVALID_NEGATED_OPERATORS.include?(operator.to_sym)
    raise "Negating operator '#{operator}' is not valid."
  end
  symbolized_operator = ensure_symbol_and_validate_operator(operator)

  subquery = { field: field, operator: symbolized_operator, value: convert_value(value),
      negate: true }
  reset_for_changed_query
  @query = (query || []) + [subquery]

  self
end

#batch_size(size) ⇒ Scrivito::ObjSearchEnumerator

Number of search results to be returned by each of the internal search requests.

The default is 10.

Scrivito makes a best effort to return the given number of search results, but may under certain circumstances return larger or smaller batches due to technical reasons.

Parameters:

  • size (Integer) —

    number of search results to be returned by each of the internal search requests. Scrivito tries to honor the requested size as much as possible, but there is no guarantee. At the time of writing, size is capped at 100, for example.

Returns:



405
406
407
408
409
410
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 405

def batch_size(size)
  @batch_size = size
  @preload_batch = true

  self
end

#each {|Obj| ... }

This method returns an undefined value.

Iterates over the search result, yielding Obj.

Yields:



435
436
437
438
439
440
441
442
443
444
445
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 435

def each
  iterator = BatchIterator.new(workspace, search_dsl_params, @preloaded_batch)

  iterator.each do |batch|
    batch.objs.each do |obj|
      yield obj
    end
  end

  @size = iterator.total
end

#facet(attribute, options = {}) ⇒ Array<Scrivito::ObjFacetValue> #facet(facets) ⇒ Hash

Perform a faceted search over up to ten attributes to retrieve structured results for individual values of these attributes.

Applicable to attributes of the following types: string, stringlist, enum, multienum.

Please note that there is a precision limit for faceting: Only the first 50 characters of a string are guaranteed to be considered for faceting. If two string values have the same first 50 characters, they may be grouped into the same facet value.

Please note that by default #facet does not preload the first batch of the search results. In order to reduce the number of search requests, batch_size can be explicitly set using the #batch_size method. This causes Scrivito to preload the first batch of the search results.

Examples:

Faceted request: colors of big balloons:

facets = Balloon.where(:size, :equals, "big").facet("color")

# Big balloons come in 3 colors:
facets.count #=> 3

# There are 3 big red balloons:
red_balloons = facets.first
red_balloons.name #=> "red"
red_balloons.count #=> 3

# There are 2 big green balloons:
green_balloons = facets.second
green_balloons.name #=> "green"
green_balloons.count #=> 2

# There is 1 big blue balloon:
blue_balloons = facets.third
blue_balloons.name #=> "blue"
blue_balloons.count #=> 1

Faceted request with limit: at most 2 colors of big balloons:

facets = Balloon.where(:size, :equals, "big").facet("color", limit: 2)

# Although there are 3 different colors of big balloons,
# only the first 2 colors will be taken into account.
facets.count # => 2

Faceted request with included Objs:

facets = Balloon.where(:size, :equals, "big").facet("color", include_objs: 2)

facets.each do |facet|
  facet.included_objs.each do |obj|
    puts "#{obj.size} #{obj.color} #{obj.class}"
  end
end

# If there are 2 big red balloons, 2 big green balloons and 1 big blue balloon,
# then this will produce:

"big red Balloon"
"big red Balloon"
"big green Balloon"
"big green Balloon"
"big blue Balloon"

Multiple faceting request:

facets = Balloon.where(:size, :equals, "big").facet(
  color: {limit: 3, include_objs: 5},
  motif: {limit: 3, include_objs: 5}
)

color_facet_obj_values = facets[:color]
motif_facet_obj_values = facets[:motif]

color_facet_obj_values.each do |facet|
  facet.included_objs.each do |obj|
    puts "#{obj.size} #{obj.color} #{obj.class}"
  end
end

motif_facet_obj_values.each do |facet|
  facet.included_objs.each do |obj|
    puts "#{obj.size} #{obj.motif} #{obj.class}"
  end
end

# If there are 2 big red balloons, 2 big green balloons and 1 big blue balloon,
# this will produce:

"big red Balloon"
"big red Balloon"
"big green Balloon"
"big green Balloon"
"big blue Balloon"

# If there are 1 big birthday balloon and 1 big wedding balloon,
# this will produce:

"big birthday Balloon"
"big wedding Balloon"

Faceted where query with batch_size:

big_balloons = Balloon.where(:size, :equals, "big")

# Without preloading
balloon_colors = big_balloons.facet("color")
first_ten_balloons = big_balloons.take(10) # This will cause a search request.

# With preloading
big_balloons.batch_size(10) # Make Scrivito preload the first ten balloons.
balloon_colors = big_balloons.facet("color")
first_ten_balloons = big_balloons.take(10) # This will cause _no_ search request.

Overloads:

  • #facet(attribute, options = {}) ⇒ Array<Scrivito::ObjFacetValue>

    Single-attribute faceting request.

    Parameters:

    • attribute (String) —

      the name of an attribute.

    • options (Hash) (defaults to: {}) —

      the options to facet a request with.

    Options Hash (options):

    • :limit (Integer) —

      maximum number of unique values to return. Defaults to 10.

    • :include_objs (Integer) —

      maximum number of Objs to fetch for each unique value. Defaults to 0.

    Returns:

    • (Array<Scrivito::ObjFacetValue>) —

      A list of unique values that were found for the given attribute name. The list is ordered by frequency, i.e. values occurring more frequently come first.

  • #facet(facets) ⇒ Hash

    Multi-attribute faceting request. The maximum number of attributes that may be specified is 10.

    Parameters:

    • facets (Hash) —

      a hash where the keys are attribute names and the values are options. The available options are identical to the options for single faceting requests.

    Returns:

    • (Hash) —

      a hash where the keys are identical to the keys given in the facets parameter. The values of the hash are lists of Scrivito::ObjFacetValue.

    Raises:

    • (Scrivito::ClientError) —

      If the number of attributes exceeds 10.

Raises:

  • (Scrivito::ClientError) —

    If the maximum number of results has been exceeded. The number of results is limited to 100 with respect to the facets themselves and the included Objs.



621
622
623
624
625
626
627
628
629
630
631
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 621

def facet(*facet_params)
  search_params = search_dsl_params
  search_params[:size] = 0 unless @preload_batch

  facet_query = FacetQuery.new(facet_params, search_params, workspace)
  facet_query.execute!

  @preloaded_batch = facet_query.batch if @preload_batch

  facet_query.result
end

#load_batch ⇒ Array

Loads a single batch of search results from the backend. Usually returns batch_size results if available, but may occasionally return more or fewer than batch_size results (due to technical reasons). If you need an exact number of hits, use methods from Enumerable, for example take.

Returns:

  • (Array) —

    of Obj.



478
479
480
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 478

def load_batch
  fetch_batch.objs
end

#offset(amount) ⇒ Scrivito::ObjSearchEnumerator

Omits the first amount of Objs from the results. The default is 0.

Parameters:

  • amount (Integer)

Returns:



417
418
419
420
421
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 417

def offset(amount)
  options[:offset] += amount

  self
end

#order(field_name) ⇒ Scrivito::ObjSearchEnumerator #order(field_and_direction) ⇒ Scrivito::ObjSearchEnumerator

Orders the results by field_name.

Applicable to the attribute types string, enum, integer, float and date.

There is a precision limit when sorting string values: Only the first 50 characters of a string are guaranteed to be considered when sorting search results.

Examples:

Sorting descending

Obj.all.order(_last_changed: :desc)

Overloads:

  • #order(field_name) ⇒ Scrivito::ObjSearchEnumerator

    Parameters:

    • field_name (Symbol, String) —

      This parameter specifies the field by which the hits are sorted (e.g. :_path).

  • #order(field_and_direction) ⇒ Scrivito::ObjSearchEnumerator

    Parameters:

    • field_and_direction (Hash) —

      The field name and sort direction can be specfied as the key and value of a hash. Valid directions are :asc and :desc. The default is :asc.

Returns:



362
363
364
365
366
367
368
369
370
371
372
373
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 362

def order(field_name)
  field_name, direction = if field_name.is_a?(Hash)
    field_name.to_a.first
  else
    [field_name, :asc]
  end

  options[:sort_by] = field_name
  options[:sort_order] = direction.to_sym

  self
end

#reverse_order ⇒ Scrivito::ObjSearchEnumerator

Deprecated.

This method is deprecated and will be removed in the next major version. Please specify the direction using #order.

Reverses the order of the results. Requires #order to be applied before.

Returns:



380
381
382
383
384
385
386
387
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 380

def reverse_order
  Scrivito::Deprecation.warn_method("reverse_order", "order")
  options[:sort_by].present? or raise "A search order has to be specified"\
      " before reverse_order can be applied."
  options[:sort_order] = options[:sort_order] == :asc ? :desc : :asc

  self
end

#size ⇒ Integer Also known as: length, count

The total number of hits.

This number is an approximation. Scrivito makes a best effort to deliver the exact number of hits. But due to technical reasons, the returned number may differ from the actual number under certain circumstances.

Returns:

  • (Integer) 


455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 455

def size
  return @size if @size

  size_query = {
    query: query,
    size: 0
  }
  if @include_deleted
    size_query[:options] = {
      include_deleted: true
    }
  end

  @size ||= CmsBackend.search_objs(workspace, size_query)['total'].to_i
end

#suggest(prefix, options = {}) ⇒ Array<String>

Suggests search terms that start with the provided prefix.

#suggest works case insensitive, and all results are converted to lower case.

Preceding calls to #and or #and_not may limit the amount of documents that are searched for suggestions.

Examples:

List suggestions that start with sc

Obj.all.suggest('sc')
# ['scrivito', 'science', 'screen']

Limit suggestions to specific attributes

Obj.all.suggest('ham', { attributes: ['title'] })
# ['hammer']

Limit the number of suggestions to 2

Obj.all.suggest('sc', { limit: 2 })
# ['scrivito', 'science']

Limit suggestions for the prefix Kin to objects for which the “de” language flag is set

Obj.where(:language, :equals, 'de').suggest('Kin')
# ['kindergarten', 'kind', 'kinder']

Parameters:

  • prefix (String) —

    The prefix string for which suggestions should be determined.

  • options (Hash) (defaults to: {})

Options Hash (options):

  • :limit (Integer) —

    The maximum number of suggestions to return. Defaults to 5. Maximum: 100.

  • :attributes (Array<String>) —

    The list of attributes that serve as input for suggestions. The special value [“*”], which is the default, causes all html, string and stringlist attributes to be taken account of.

Returns:

  • (Array<String>) —

    A list of words the user could search for.



667
668
669
670
671
672
673
674
675
676
677
 
# File 'app/cms/scrivito/obj_search_enumerator.rb', line 667

def suggest(prefix, options = {})
  o = options.with_indifferent_access
  params = {
    prefix: prefix,
    fields: o[:attributes] || ['*'],
    limit: o[:limit] || 5,
    from_search: search_dsl_params,
  }

  CmsRestApi.put("workspaces/#{workspace.id}/objs/search/suggest", params)['results']
end