Module: ActiveRecord::ConnectionAdapters::Elasticsearch::SchemaStatements

Extended by:
ActiveSupport::Concern
Included in:
ActiveRecord::ConnectionAdapters::ElasticsearchAdapter
Defined in:
lib/active_record/connection_adapters/elasticsearch/schema_statements.rb

Overview

extend adapter with schema-related statements

ORIGINAL methods untouched:

  • internal_string_options_for_primary_key
  • options_include_default?
  • fetch_type_metadata
  • column_exists?

SUPPORTED but not used:

  • strip_table_name_prefix_and_suffix

UNSUPPORTED methods that will be ignored:

  • native_database_types
  • table_options
  • table_comment
  • table_alias_for
  • columns_for_distinct
  • extract_new_default_value
  • insert_versions_sql
  • data_source_sql
  • quoted_scope
  • add_column_for_alter
  • rename_column_sql
  • remove_column_for_alter
  • remove_columns_for_alter
  • add_timestamps_for_alter
  • remove_timestamps_for_alter
  • foreign_key_name
  • foreign_key_for
  • foreign_key_for!
  • extract_foreign_key_action
  • check_constraint_name
  • check_constraint_for
  • check_constraint_for!
  • validate_index_length!
  • can_remove_index_by_name?
  • index_column_names
  • index_name_options
  • add_index_sort_order
  • options_for_index_columns
  • add_options_for_index_columns
  • index_name_for_remove
  • add_index_options
  • index_algorithm
  • quoted_columns_for_index
  • check_constraint_options
  • check_constraints
  • foreign_key_exists?
  • foreign_key_column_for
  • foreign_key_options
  • foreign_keys
  • index_name_exists?
  • indexes
  • index_name
  • index_exists?

UNSUPPORTED methods that will fail:

  • views
  • view_exists?
  • add_index
  • remove_index
  • rename_index
  • add_reference
  • remove_reference
  • add_foreign_key
  • remove_foreign_key
  • add_check_constraint
  • remove_check_constraint
  • rename_table_indexes
  • rename_column_indexes
  • create_alter_table
  • insert_fixture
  • insert_fixtures_set
  • bulk_change_table
  • dump_schema_information

OVERWRITTEN methods for Elasticsearch: ...

Instance Method Summary collapse

Instance Method Details

#access_id_fielddata? ⇒ Boolean

returns true if the cluster option 'id_field_data' is enabled or not configured. This is required to check if a general sorting on the +_id+-field is possible or not.

Returns:

  • (Boolean) 


374
375
376
377
378
379
380
381
382
383
384
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 374

def access_id_fielddata?
  @access_id_fielddata = begin
                           status = self.cluster_settings['indices.id_field_data.enabled']
                           # for cluster version lower 7.6 this might not configured.
                           status = (cluster_info[:version] < "7.6") if status.nil?

                           status
                         end if @access_id_fielddata.nil?

  @access_id_fielddata
end

#access_shard_doc? ⇒ Boolean

returns true if _shard_doc field can be accessed through PIT-search.

Returns:

  • (Boolean) 


388
389
390
391
392
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 388

def access_shard_doc?
  @access_shard_doc = cluster_info[:version] >= "7.12" if @access_shard_doc.nil?

  @access_shard_doc
end

#alias_exists?(table_name, alias_name) ⇒ Boolean

Checks to see if a alias alias_name within a table table_name exists on the database.

alias_exists?(:developers, 'my-alias')

Parameters:

  • table_name (String)
  • alias_name (String, Symbol)

Returns:

  • (Boolean) 


300
301
302
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 300

def alias_exists?(table_name, alias_name)
  table_aliases(table_name).keys.include?(alias_name.to_s)
end

#assume_migrated_upto_version(version) ⇒ Object 



94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 94

def assume_migrated_upto_version(version)
  version = version.to_i
  migrated = migration_context.get_all_versions
  versions = migration_context.migrations.map(&:version)

  unless migrated.include?(version)
    # use Arel syntax to create a new version
    schema_migration.create_version(version)
  end

  inserting = (versions - migrated).select { |v| v < version }
  if inserting.any?
    if (duplicate = inserting.detect { |v| inserting.count(v) > 1 })
      raise "Duplicate migration #{duplicate}. Please renumber your migrations to resolve the conflict."
    end

    # use Arel syntax to create new versions
    inserting.each { |iversion| schema_migration.create_version(iversion) }
  end

  true
end

#clone_table_definition(name, target, **options) ⇒ Object 



353
354
355
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 353

def clone_table_definition(name, target, **options)
  ::ActiveRecord::ConnectionAdapters::Elasticsearch::CloneTableDefinition.new(self, name, target, **options)
end

#cluster_health(**options) ⇒ Hash

returns the cluster health

Returns:

  • (Hash) 


419
420
421
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 419

def cluster_health(**options)
  api('cluster.health', options, 'CLUSTER HEALTH').to_h
end

#cluster_info ⇒ Hash{Symbol->Unknown

Returns basic information about the cluster.

Returns:

  • (Hash{Symbol->Unknown) —

    ]



396
397
398
399
400
401
402
403
404
405
406
407
408
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 396

def cluster_info
  @cluster_info ||= begin
                      response = api(:info, {}, 'CLUSTER INFO')

                      {
                        name: response.dig('name'),
                        cluster_name: response.dig('cluster_name'),
                        cluster_uuid: response.dig('cluster_uuid'),
                        version: Gem::Version.new(response.dig('version', 'number')),
                        lucene_version: response.dig('version', 'lucene_version')
                      }
                    end
end

#cluster_settings ⇒ Hash

returns a hash of current set, none-default settings in flat

Returns:

  • (Hash) 


412
413
414
415
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 412

def cluster_settings
  settings = api('cluster.get_settings', { flat_settings: true }, 'CLUSTER SETTINGS')
  settings['persistent'].merge(settings['transient'])
end

#column_definitions(table_name) ⇒ Array<Hash>

Returns the list of a table's column names, data types, and default values.

Parameters:

  • table_name (String)

Returns:

  • (Array<Hash>)

See Also:

  • SchemaStatements#columns
  • AbstractMysqlAdapter#column_definitions


198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 198

def column_definitions(table_name)
  mappings = table_mappings(table_name)

  # prevent exceptions on missing mappings, to provide the possibility to create them
  # otherwise loading the table (index) will always fail!
  mappings = { 'properties' => {} } if mappings.blank? || mappings['properties'].blank?
  # raise(ActiveRecord::StatementInvalid, "Could not find valid mappings for '#{table_name}'") if mappings.blank? || mappings['properties'].blank?

  # since the received mappings do not have the "primary" +_id+-column we manually need to add this here
  # The METADATA_FIELDS will also include some meta keys like '_score', '_type', ...
  ActiveRecord::ConnectionAdapters::ElasticsearchAdapter::METADATA_FIELDS + mappings['properties'].map { |key, prop|
    # resolve (nested) fields and properties
    fields, properties = resolve_fields_and_properties(key, prop, true)

    # fallback for possible empty type
    type = prop['type'].presence || (properties.present? ? 'object' : 'nested')

    # return a new hash
    prop.merge('name' => key, 'type' => type, 'fields' => fields, 'properties' => properties)
  }
end

#create_schema_dumper(options) ⇒ Object

overwrite original methods to provide a elasticsearch version



339
340
341
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 339

def create_schema_dumper(options)
  ActiveRecord::ConnectionAdapters::Elasticsearch::SchemaDumper.create(self, options)
end

#create_table_definition(name, **options) ⇒ Object

overwrite original methods to provide a elasticsearch version



344
345
346
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 344

def create_table_definition(name, **options)
  ::ActiveRecord::ConnectionAdapters::Elasticsearch::CreateTableDefinition.new(self, name, **options)
end

#data_source_exists?(name) ⇒ Boolean

Checks to see if the data source name exists on the database.

data_source_exists?(:ebooks)

Parameters:

  • name (String, Symbol)

Returns:

  • (Boolean)

See Also:

  • SchemaStatements#data_source_exists?


276
277
278
279
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 276

def data_source_exists?(name)
  # response returns boolean
  api('indices.exists?', { index: name, expand_wildcards: [:open, :closed] }, 'SCHEMA')
end

#data_sources ⇒ Array<String>

Returns the relation names usable to back Active Record models. For Elasticsearch this means all indices - which also includes system dot '.' indices.

Returns:

  • (Array<String>)

See Also:

  • SchemaStatements#data_sources


121
122
123
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 121

def data_sources
  api('indices.get', { index: :_all, expand_wildcards: [:open, :closed] }, 'SCHEMA').keys
end

#lookup_multicast_cast_type(sql_type) ⇒ ActiveRecord::ConnectionAdapters::Elasticsearch::Type::MulticastValue

resolves the cast type of the provided sql_type and wraps it into a MulticastValue. since Elasticsearch has the "feature" to provide multicast values on any type, we need to fetch them ... you know, ES can return an integer or an array of integers for any column ...

Parameters:

  • sql_type (String)

Returns:



246
247
248
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 246

def lookup_multicast_cast_type(sql_type)
  type_map.lookup(:multicast_value, lookup_cast_type(sql_type))
end

#mapping_exists?(table_name, mapping_name, type = nil) ⇒ Boolean

Checks to see if a mapping mapping_name within a table table_name exists on the database.

mapping_exists?(:developers, :status, :integer)

Parameters:

  • table_name (String, Symbol)
  • mapping_name (String, Symbol)

Returns:

  • (Boolean) 


323
324
325
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 323

def mapping_exists?(table_name, mapping_name, type = nil)
  column_exists?(table_name, mapping_name, type)
end

#max_result_window(table_name) ⇒ Integer

returns the maximum allowed size for queries for the provided table_name. The query will raise an ActiveRecord::StatementInvalid if the requested limit is above this value.

Returns:

  • (Integer) 


365
366
367
368
369
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 365

def max_result_window(table_name)
  # IMPORTANT: the settings API returns every value as a String - without the cast the
  # callers would compare a String against their (Integer) batch sizes.
  table_settings(table_name).dig('index.max_result_window').presence&.to_i || 10_000
end

#meta_exists?(table_name, meta_name) ⇒ Boolean

Checks to see if a meta meta_name within a table table_name exists on the database.

meta_exists?(:developers, 'class')

Parameters:

  • table_name (String)
  • meta_name (String, Symbol)

Returns:

  • (Boolean) 


334
335
336
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 334

def meta_exists?(table_name, meta_name)
  table_metas(table_name).keys.include?(meta_name.to_s)
end

#new_column_from_field(_table_name, field, _definitions) ⇒ ActiveRecord::ConnectionAdapters::Column

creates a new column object from provided field Hash

Parameters:

  • _table_name (String)
  • field (Hash)

Returns:

  • (ActiveRecord::ConnectionAdapters::Column)

See Also:

  • SchemaStatements#columns
  • MySQL::SchemaStatements#new_column_from_field


226
227
228
229
230
231
232
233
234
235
236
237
238
239
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 226

def new_column_from_field(_table_name, field, _definitions)
  ActiveRecord::ConnectionAdapters::Elasticsearch::Column.new(
    field["name"],
    # the cast type is resolved upfront and stored on the column
    lookup_multicast_cast_type(field["type"]),
    field["null_value"],
    fetch_type_metadata(field["type"]),
    meta: field['meta'],
    virtual: field['virtual'],
    fields: field['fields'],
    properties: field['properties'],
    enabled: field['enabled']
  )
end

#primary_keys(table_name) ⇒ Array<String>

Parameters:

  • table_name (String)

Returns:

  • (Array<String>)

See Also:

  • AbstractMysqlAdapter#primary_keys


261
262
263
264
265
266
267
268
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 261

def primary_keys(table_name)
  # IMPORTANT: the +_meta+ node stores the primary_key as a plain String - it must be
  # wrapped, so every branch of this method returns an Array (as the API demands).
  Array.wrap(table_metas(table_name).dig('primary_key')).presence || column_definitions(table_name).
    select { |f| f['meta'] && f['meta']['primary_key'] == 'true' }.
    # only take the last found primary key (if no custom primary_key was provided this will return +_id+ )
    map { |f| f["name"] }[-1..-1]
end

#schema_creation ⇒ Object

overwrite original methods to provide a elasticsearch version



358
359
360
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 358

def schema_creation
  ::ActiveRecord::ConnectionAdapters::Elasticsearch::SchemaCreation.new(self)
end

#setting_exists?(table_name, setting_name) ⇒ Boolean

Checks to see if a setting setting_name within a table table_name exists on the database. The provided setting_name must be flat!

setting_exists?(:developers, 'index.number_of_replicas')

Parameters:

  • table_name (String)
  • setting_name (String, Symbol)

Returns:

  • (Boolean) 


312
313
314
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 312

def setting_exists?(table_name, setting_name)
  table_settings(table_name).keys.include?(setting_name.to_s)
end

#table_aliases(table_name) ⇒ Hash

returns a hash of all aliases by provided table_name (index).

Parameters:

  • table_name (String)

Returns:

  • (Hash) 


159
160
161
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 159

def table_aliases(table_name)
  api('indices.get_alias', { index: table_name, expand_wildcards: [:open, :closed] }, 'SCHEMA').dig(table_name, 'aliases')
end

#table_exists?(table_name) ⇒ Boolean

Checks to see if the table table_name exists on the database.

table_exists?(:developers)

Parameters:

  • table_name (String, Symbol)

Returns:

  • (Boolean)

See Also:

  • SchemaStatements#table_exists?


288
289
290
291
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 288

def table_exists?(table_name)
  # just reference to the data sources
  data_source_exists?(table_name)
end

#table_mappings(table_name) ⇒ Hash

returns a hash of all mappings by provided table_name (index)

Parameters:

  • table_name (String)

Returns:

  • (Hash) 


136
137
138
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 136

def table_mappings(table_name)
  api('indices.get_mapping', { index: table_name, expand_wildcards: [:open, :closed] }, 'SCHEMA').dig(table_name, 'mappings')
end

#table_metas(table_name) ⇒ Hash

returns a hash of all meta data by provided table_name (index). HINT: _meta is resolved from the table mappings

Parameters:

  • table_name (String)

Returns:

  • (Hash) 


144
145
146
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 144

def table_metas(table_name)
  table_mappings(table_name).dig('_meta').presence || {}
end

#table_schema(table_name, features = [:aliases, :mappings, :settings]) ⇒ Hash

returns a hash of the full definition of the provided table_name (index). (includes settings, mappings & aliases)

Parameters:

  • table_name (String)
  • features (Array, Symbol) (defaults to: [:aliases, :mappings, :settings])

Returns:

  • (Hash) 


179
180
181
182
183
184
185
186
187
188
189
190
191
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 179

def table_schema(table_name, features = [:aliases, :mappings, :settings])
  if cluster_info[:version] >= '8.5.0'
    response = api('indices.get', { index: table_name, expand_wildcards: [:open, :closed], features: features, flat_settings: true }, 'SCHEMA')
  else
    response = api('indices.get', { index: table_name, expand_wildcards: [:open, :closed], flat_settings: true }, 'SCHEMA')
  end

  {
    settings: response.dig(table_name, 'settings'),
    mappings: response.dig(table_name, 'mappings'),
    aliases: response.dig(table_name, 'aliases')
  }
end

#table_settings(table_name, flat_settings = true) ⇒ Hash

returns a hash of all settings by provided table_name

Parameters:

  • table_name (String)
  • flat_settings (Boolean) (defaults to: true) —

    (default: true)

Returns:

  • (Hash) 


152
153
154
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 152

def table_settings(table_name, flat_settings = true)
  api('indices.get_settings', { index: table_name, expand_wildcards: [:open, :closed], flat_settings: flat_settings }, 'SCHEMA').dig(table_name, 'settings')
end

#table_state(table_name) ⇒ Hash

returns information about number of primaries and replicas, document counts, disk size, ... by provided table_name (index).

Parameters:

  • table_name (String)

Returns:

  • (Hash) 


166
167
168
169
170
171
172
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 166

def table_state(table_name)
  response = api('cat.indices', { index: table_name, expand_wildcards: [:open, :closed] }, 'SCHEMA')

  [:health, :status, :name, :uuid, :pri, :rep, :docs_count, :docs_deleted, :store_size, :pri_store_size].zip(
    response.body.split(' ')
  ).to_h
end

#tables ⇒ Array<String>

Returns an array of table names defined in the database. For Elasticsearch this means all normal indices (no system dot '.' indices)

Returns:

  • (Array<String>)

See Also:

  • SchemaStatements#tables


129
130
131
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 129

def tables
  data_sources.reject { |key| key[0] == '.' }
end

#type_to_sql(type) ⇒ Object

transforms provided schema-type to a sql-type overwrite original methods to provide a elasticsearch version

Parameters:

  • type (String, Symbol) 


426
427
428
429
430
431
432
433
434
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 426

def type_to_sql(type, **)
  return '' if type.blank?

  if (native = native_database_types[type.to_sym])
    (native.is_a?(Hash) ? native[:name] : native).dup
  else
    type.to_s
  end
end

#update_table_definition(name, base = self, **options) ⇒ Object

overwrite original methods to provide a elasticsearch version



349
350
351
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 349

def update_table_definition(name, base = self, **options)
  ::ActiveRecord::ConnectionAdapters::Elasticsearch::UpdateTableDefinition.new(base, name, **options)
end