Module: ActiveRecord::ConnectionAdapters::Elasticsearch::SchemaStatements

Extended by:
ActiveSupport::Concern
Included in:
ActiveRecord::ConnectionAdapters::ElasticsearchAdapter
Defined in:
lib/active_record/connection_adapters/elasticsearch/schema_statements.rb

Overview

extend adapter with schema-related statements

ORIGINAL methods untouched:

  • internal_string_options_for_primary_key
  • options_include_default?
  • fetch_type_metadata
  • column_exists?

SUPPORTED but not used:

  • strip_table_name_prefix_and_suffix

UNSUPPORTED methods that will be +ignored+:

  • native_database_types
  • table_options
  • table_comment
  • table_alias_for
  • columns_for_distinct
  • extract_new_default_value
  • insert_versions_sql
  • data_source_sql
  • quoted_scope
  • add_column_for_alter
  • rename_column_sql
  • remove_column_for_alter
  • remove_columns_for_alter
  • add_timestamps_for_alter
  • remove_timestamps_for_alter
  • foreign_key_name
  • foreign_key_for
  • foreign_key_for!
  • extract_foreign_key_action
  • check_constraint_name
  • check_constraint_for
  • check_constraint_for!
  • validate_index_length!
  • can_remove_index_by_name?
  • index_column_names
  • index_name_options
  • add_index_sort_order
  • options_for_index_columns
  • add_options_for_index_columns
  • index_name_for_remove
  • add_index_options
  • index_algorithm
  • quoted_columns_for_index
  • check_constraint_options
  • check_constraints
  • foreign_key_exists?
  • foreign_key_column_for
  • foreign_key_options
  • foreign_keys
  • index_name_exists?
  • indexes
  • index_name
  • index_exists?

UNSUPPORTED methods that will +fail+:

  • views
  • view_exists?
  • add_index
  • remove_index
  • rename_index
  • add_reference
  • remove_reference
  • add_foreign_key
  • remove_foreign_key
  • add_check_constraint
  • remove_check_constraint
  • rename_table_indexes
  • rename_column_indexes
  • create_alter_table
  • insert_fixture
  • insert_fixtures_set
  • bulk_change_table
  • dump_schema_information

OVERWRITTEN methods for Elasticsearch: ...

Instance Method Summary collapse

Instance Method Details

#access_id_fielddata? ⇒ Boolean

returns true if the cluster option 'id_field_data' is enabled or not configured. This is required to check if a general sorting on the +_id+-field is possible or not.

Returns:

  • (Boolean) 


367
368
369
370
371
372
373
374
375
376
377
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 367

def access_id_fielddata?
  @access_id_fielddata = begin
                           status = self.cluster_settings['indices.id_field_data.enabled']
                           # for cluster version lower 7.6 this might not configured.
                           status = (cluster_info[:version] < "7.6") if status.nil?

                           status
                         end if @access_id_fielddata.nil?

  @access_id_fielddata
end

#access_shard_doc? ⇒ Boolean

returns true if +_shard_doc+ field can be accessed through PIT-search.

Returns:

  • (Boolean) 


381
382
383
384
385
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 381

def access_shard_doc?
  @access_shard_doc = cluster_info[:version] >= "7.12" if @access_shard_doc.nil?

  @access_shard_doc
end

#alias_exists?(table_name, alias_name) ⇒ Boolean

Checks to see if a alias +alias_name+ within a table +table_name+ exists on the database.

alias_exists?(:developers, 'my-alias')

Parameters:

  • table_name (String)
  • alias_name (String, Symbol)

Returns:

  • (Boolean) 


295
296
297
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 295

def alias_exists?(table_name, alias_name)
  table_aliases(table_name).keys.include?(alias_name.to_s)
end

#assume_migrated_upto_version(version) ⇒ Object 



94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 94

def assume_migrated_upto_version(version)
  version  = version.to_i
  migrated = migration_context.get_all_versions
  versions = migration_context.migrations.map(&:version)

  unless migrated.include?(version)
    # use a ActiveRecord syntax to create a new version
    schema_migration.create(version: version)
  end

  inserting = (versions - migrated).select { |v| v < version }
  if inserting.any?
    if (duplicate = inserting.detect { |v| inserting.count(v) > 1 })
      raise "Duplicate migration #{duplicate}. Please renumber your migrations to resolve the conflict."
    end

    # use a ActiveRecord syntax to create new versions
    inserting.each { |iversion| schema_migration.create(version: iversion) }
  end

  true
end

#clone_table_definition(name, target, **options) ⇒ Object 



348
349
350
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 348

def clone_table_definition(name, target, **options)
  ::ActiveRecord::ConnectionAdapters::Elasticsearch::CloneTableDefinition.new(self, name, target, **options)
end

#cluster_health(**options) ⇒ Hash

returns the cluster health

Returns:

  • (Hash) 


412
413
414
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 412

def cluster_health(**options)
  api(:cluster, :health, options, 'CLUSTER HEALTH').to_h
end

#cluster_info ⇒ Hash{Symbol->Unknown

Returns basic information about the cluster.

Returns:

  • (Hash{Symbol->Unknown) —

    ]



389
390
391
392
393
394
395
396
397
398
399
400
401
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 389

def cluster_info
  @cluster_info ||= begin
                      response = api(:core, :info, {}, 'CLUSTER INFO')

                      {
                        name:           response.dig('name'),
                        cluster_name:   response.dig('cluster_name'),
                        cluster_uuid:   response.dig('cluster_uuid'),
                        version:        Gem::Version.new(response.dig('version', 'number')),
                        lucene_version: response.dig('version', 'lucene_version')
                      }
                    end
end

#cluster_settings ⇒ Hash

returns a hash of current set, none-default settings in flat

Returns:

  • (Hash) 


405
406
407
408
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 405

def cluster_settings
  settings = api(:cluster, :get_settings, { flat_settings: true }, 'CLUSTER SETTINGS')
  settings['persistent'].merge(settings['transient'])
end

#column_definitions(table_name) ⇒ Array<Hash>

Returns the list of a table's column names, data types, and default values.

Parameters:

  • table_name (String)

Returns:

  • (Array<Hash>)

See Also:

  • SchemaStatements#columns
  • AbstractMysqlAdapter#column_definitions


198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 198

def column_definitions(table_name)
  mappings = table_mappings(table_name)

  # prevent exceptions on missing mappings, to provide the possibility to create them
  # otherwise loading the table (index) will always fail!
  mappings = { 'properties' => {} } if mappings.blank? || mappings['properties'].blank?
  # raise(ActiveRecord::StatementInvalid, "Could not find valid mappings for '#{table_name}'") if mappings.blank? || mappings['properties'].blank?

  # since the received mappings do not have the "primary" +_id+-column we manually need to add this here
  # The BASE_STRUCTURE will also include some meta keys like '_score', '_type', ...
  ActiveRecord::ConnectionAdapters::ElasticsearchAdapter::BASE_STRUCTURE + mappings['properties'].map { |key, prop|
    # resolve (nested) fields and properties
    fields, properties = resolve_fields_and_properties(key, prop, true)

    # fallback for possible empty type
    type = prop['type'].presence || (properties.present? ? 'object' : 'nested')

    # return a new hash
    prop.merge('name' => key, 'type' => type, 'fields' => fields, 'properties' => properties)
  }
end

#create_schema_dumper(options) ⇒ Object

overwrite original methods to provide a elasticsearch version



334
335
336
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 334

def create_schema_dumper(options)
  ActiveRecord::ConnectionAdapters::Elasticsearch::SchemaDumper.create(self, options)
end

#create_table_definition(name, **options) ⇒ Object

overwrite original methods to provide a elasticsearch version



339
340
341
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 339

def create_table_definition(name, **options)
  ::ActiveRecord::ConnectionAdapters::Elasticsearch::CreateTableDefinition.new(self, name, **options)
end

#data_source_exists?(name) ⇒ Boolean

Checks to see if the data source +name+ exists on the database.

data_source_exists?(:ebooks)

Parameters:

  • name (String, Symbol)

Returns:

  • (Boolean)

See Also:

  • SchemaStatements#data_source_exists?


271
272
273
274
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 271

def data_source_exists?(name)
  # response returns boolean
  api(:indices, :exists?, { index: name, expand_wildcards: [:open, :closed] }, 'SCHEMA')
end

#data_sources ⇒ Array<String>

Returns the relation names usable to back Active Record models. For Elasticsearch this means all indices - which also includes system +dot+ '.' indices.

Returns:

  • (Array<String>)

See Also:

  • SchemaStatements#data_sources


121
122
123
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 121

def data_sources
  api(:indices, :get, { index: :_all, expand_wildcards: [:open, :closed] }, 'SCHEMA').keys
end

#lookup_cast_type_from_column(column) ⇒ ActiveRecord::ConnectionAdapters::Elasticsearch::Type::MulticastValue

lookups from building the @columns_hash. since Elasticsearch has the "feature" to provide multicast values on any type, we need to fetch them ... you know, ES can return an integer or an array of integers for any column ...

Parameters:

Returns:



244
245
246
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 244

def lookup_cast_type_from_column(column)
  type_map.lookup(:multicast_value, super)
end

#mapping_exists?(table_name, mapping_name, type = nil) ⇒ Boolean

Checks to see if a mapping +mapping_name+ within a table +table_name+ exists on the database.

mapping_exists?(:developers, :status, :integer)

Parameters:

  • table_name (String, Symbol)
  • mapping_name (String, Symbol)

Returns:

  • (Boolean) 


318
319
320
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 318

def mapping_exists?(table_name, mapping_name, type = nil)
  column_exists?(table_name, mapping_name, type)
end

#max_result_window(table_name) ⇒ Integer

returns the maximum allowed size for queries for the provided +table_name+. The query will raise an ActiveRecord::StatementInvalid if the requested limit is above this value.

Returns:

  • (Integer) 


360
361
362
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 360

def max_result_window(table_name)
  table_settings(table_name).dig('index', 'max_result_window').presence || 10000
end

#meta_exists?(table_name, meta_name) ⇒ Boolean

Checks to see if a meta +meta_name+ within a table +table_name+ exists on the database.

meta_exists?(:developers, 'class')

Parameters:

  • table_name (String)
  • meta_name (String, Symbol)

Returns:

  • (Boolean) 


329
330
331
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 329

def meta_exists?(table_name, meta_name)
  table_metas(table_name).keys.include?(meta_name.to_s)
end

#new_column_from_field(_table_name, field) ⇒ ActiveRecord::ConnectionAdapters::Column

creates a new column object from provided field Hash

Parameters:

  • _table_name (String)
  • field (Hash)

Returns:

  • (ActiveRecord::ConnectionAdapters::Column)

See Also:

  • SchemaStatements#columns
  • MySQL::SchemaStatements#new_column_from_field


226
227
228
229
230
231
232
233
234
235
236
237
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 226

def new_column_from_field(_table_name, field)
  ActiveRecord::ConnectionAdapters::Elasticsearch::Column.new(
    field["name"],
    field["null_value"],
    fetch_type_metadata(field["type"]),
    meta:       field['meta'],
    virtual:    field['virtual'],
    fields:     field['fields'],
    properties: field['properties'],
    enabled:    field['enabled']
  )
end

#primary_keys(table_name) ⇒ Object

Returns a array of tables primary keys. PLEASE NOTE: Elasticsearch does not have a concept of primary key. The only thing that uniquely identifies a document is the index together with the +_id+. To support this concept we simulate this through the +_meta+ field (from the index).

As a alternative, the primary_key can also be provided through the mappings +meta+ field.

see @ https://www.elastic.co/guide/en/elasticsearch/reference/8.5/mapping-meta-field.html

Parameters:

  • table_name (String)

See Also:

  • AbstractMysqlAdapter#primary_keys


258
259
260
261
262
263
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 258

def primary_keys(table_name)
  table_metas(table_name).dig('primary_key').presence || column_definitions(table_name).
    select { |f| f['meta'] && f['meta']['primary_key'] == 'true' }.
    # only take the last found primary key (if no custom primary_key was provided this will return +_id+ )
    map { |f| f["name"] }[-1..-1]
end

#schema_creation ⇒ Object

overwrite original methods to provide a elasticsearch version



353
354
355
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 353

def schema_creation
  ::ActiveRecord::ConnectionAdapters::Elasticsearch::SchemaCreation.new(self)
end

#setting_exists?(table_name, setting_name) ⇒ Boolean

Checks to see if a setting +setting_name+ within a table +table_name+ exists on the database. The provided +setting_name+ must be flat!

setting_exists?(:developers, 'index.number_of_replicas')

Parameters:

  • table_name (String)
  • setting_name (String, Symbol)

Returns:

  • (Boolean) 


307
308
309
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 307

def setting_exists?(table_name, setting_name)
  table_settings(table_name).keys.include?(setting_name.to_s)
end

#table_aliases(table_name) ⇒ Hash

returns a hash of all aliases by provided table_name (index).

Parameters:

  • table_name (String)

Returns:

  • (Hash) 


159
160
161
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 159

def table_aliases(table_name)
  api(:indices, :get_alias, { index: table_name, expand_wildcards: [:open, :closed] }, 'SCHEMA').dig(table_name, 'aliases')
end

#table_exists?(table_name) ⇒ Boolean

Checks to see if the table +table_name+ exists on the database.

table_exists?(:developers)

Parameters:

  • table_name (String, Symbol)

Returns:

  • (Boolean)

See Also:

  • SchemaStatements#table_exists?


283
284
285
286
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 283

def table_exists?(table_name)
  # just reference to the data sources
  data_source_exists?(table_name)
end

#table_mappings(table_name) ⇒ Hash

returns a hash of all mappings by provided table_name (index)

Parameters:

  • table_name (String)

Returns:

  • (Hash) 


136
137
138
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 136

def table_mappings(table_name)
  api(:indices, :get_mapping, { index: table_name, expand_wildcards: [:open, :closed] }, 'SCHEMA').dig(table_name, 'mappings')
end

#table_metas(table_name) ⇒ Hash

returns a hash of all meta data by provided table_name (index). HINT: +_meta+ is resolved from the table mappings

Parameters:

  • table_name (String)

Returns:

  • (Hash) 


144
145
146
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 144

def table_metas(table_name)
  table_mappings(table_name).dig('_meta').presence || {}
end

#table_schema(table_name, features = [:aliases, :mappings, :settings]) ⇒ Hash

returns a hash of the full definition of the provided table_name (index). (includes settings, mappings & aliases)

Parameters:

  • table_name (String)
  • features (Array, Symbol) (defaults to: [:aliases, :mappings, :settings])

Returns:

  • (Hash) 


179
180
181
182
183
184
185
186
187
188
189
190
191
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 179

def table_schema(table_name, features = [:aliases, :mappings, :settings])
  if cluster_info[:version] >= '8.5.0'
    response = api(:indices, :get, { index: table_name, expand_wildcards: [:open, :closed], features: features, flat_settings: true }, 'SCHEMA')
  else
    response = api(:indices, :get, { index: table_name, expand_wildcards: [:open, :closed], flat_settings: true }, 'SCHEMA')
  end

  {
    settings: response.dig(table_name, 'settings'),
    mappings: response.dig(table_name, 'mappings'),
    aliases:  response.dig(table_name, 'aliases')
  }
end

#table_settings(table_name, flat_settings = true) ⇒ Hash

returns a hash of all settings by provided table_name

Parameters:

  • table_name (String)
  • flat_settings (Boolean) (defaults to: true) —

    (default: true)

Returns:

  • (Hash) 


152
153
154
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 152

def table_settings(table_name, flat_settings = true)
  api(:indices, :get_settings, { index: table_name, expand_wildcards: [:open, :closed], flat_settings: flat_settings }, 'SCHEMA').dig(table_name, 'settings')
end

#table_state(table_name) ⇒ Hash

returns information about number of primaries and replicas, document counts, disk size, ... by provided table_name (index).

Parameters:

  • table_name (String)

Returns:

  • (Hash) 


166
167
168
169
170
171
172
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 166

def table_state(table_name)
  response = api(:cat, :indices, { index: table_name, expand_wildcards: [:open, :closed] }, 'SCHEMA')

  [:health, :status, :name, :uuid, :pri, :rep, :docs_count, :docs_deleted, :store_size, :pri_store_size].zip(
    response.body.split(' ')
  ).to_h
end

#tables ⇒ Array<String>

Returns an array of table names defined in the database. For Elasticsearch this means all normal indices (no system +dot+ '.' indices)

Returns:

  • (Array<String>)

See Also:

  • SchemaStatements#tables


129
130
131
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 129

def tables
  data_sources.reject { |key| key[0] == '.' }
end

#type_to_sql(type) ⇒ Object

transforms provided schema-type to a sql-type overwrite original methods to provide a elasticsearch version

Parameters:

  • type (String, Symbol) 


419
420
421
422
423
424
425
426
427
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 419

def type_to_sql(type, **)
  return '' if type.blank?

  if (native = native_database_types[type.to_sym])
    (native.is_a?(Hash) ? native[:name] : native).dup
  else
    type.to_s
  end
end

#update_table_definition(name, base = self, **options) ⇒ Object

overwrite original methods to provide a elasticsearch version



344
345
346
 
# File 'lib/active_record/connection_adapters/elasticsearch/schema_statements.rb', line 344

def update_table_definition(name, base = self, **options)
  ::ActiveRecord::ConnectionAdapters::Elasticsearch::UpdateTableDefinition.new(base, name, **options)
end