Class: ElasticGraph::Local::RakeTasks

Inherits:
Rake::TaskLib
  • Object
show all
Defined in:
lib/elastic_graph/local/rake_tasks.rb

Overview

Note:

All tasks (besides the ‘schema_artifacts` tasks) require `docker` and `docker-compose` to be available on your machine.

Defines tasks for local development. These tasks include:

  • Running OpenSearch and/or Elasticsearch locally (‘(elasticsearch|opensearch)::(boot|daemon|halt)`)

  • Managing schema artifacts (‘schema_artifacts:(check|dump)`)

  • Configuring OpenSearch/Elasticsearch locally (‘clusters:configure:(dry_run|perform)`)

  • Indexing fake data (‘index_fake_data:`)

  • Booting an ElasticGraph application locally (‘boot_locally`)

Constant Summary collapse

UI_PORT_OFFSET =

Offset we add to a port number for the UI (e.g. Kibana or OpenSearch Dashboards).

Example: if Elasticsearch/OpenSearch is running on port 9876, the UI for it will run on port 19876.

10_000
VALID_PORT_RANGE =

As per en.wikipedia.org/wiki/Registered_port, valid user port numbers are 1024 to 49151, but with our UI offset we need to truncate the range further.

1024..(49151 - UI_PORT_OFFSET)

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(local_config_yaml:, path_to_schema:) {|RakeTasks| ... } ⇒ RakeTasks

Note:

This method uses keyword args for all required arguments. Optional task settings are instead specified using the block.

Returns a new instance of RakeTasks.

Parameters:

  • local_config_yaml (String, Pathname) —

    path to the settings YAML file for the local/development environment

  • path_to_schema (String, Pathname) —

    path to the Ruby schema definition file–either the only file that defines the schema (using ‘ElasticGraph.define_schema`) or the “main” schema definition file, which loads other files which further define parts of the schema.

Yields:

  • (RakeTasks) —

    instance for further configuration

Yield Returns:

  • (void) 


352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 352

def initialize(local_config_yaml:, path_to_schema:)
  @local_config_yaml = local_config_yaml.to_s

  self.index_document_sizes = false
  self.schema_element_name_form = :camelCase
  self.schema_element_name_overrides = {}
  self.derived_type_name_formats = {}
  self.type_name_overrides = {}
  self.enum_value_overrides_by_type = {}
  self.schema_definition_extension_modules = []
  self.enforce_json_schema_version = true
  self.env_port_mapping = {}
  self.output = $stdout
  self.daemon_timeout = 120

  datastore_versions = ::YAML.load_file("#{__dir__}/tested_datastore_versions.yaml")
  self.elasticsearch_versions = datastore_versions.fetch("elasticsearch")
  self.opensearch_versions = datastore_versions.fetch("opensearch")

  @fake_data_batch_generator_by_type = {}

  yield self if block_given?

  # Default the local port from the local_config_yaml file.
  self.env_port_mapping = {"local" => local_datastore_port}.merge(env_port_mapping || {})
  if (invalid_port_mapping = env_port_mapping.reject { |env, port| VALID_PORT_RANGE.cover?(port) }).any?
    raise "`env_port_mapping` has invalid ports: #{invalid_port_mapping.inspect}. Valid ports must be in the #{VALID_PORT_RANGE} range."
  end

  # Load admin and schema def rake tasks...
  Admin::RakeTasks.from_yaml_file(local_config_yaml, output: output)
  SchemaDefinition::RakeTasks.new(
    index_document_sizes: index_document_sizes,
    path_to_schema: path_to_schema,
    schema_artifacts_directory: local_config.fetch("schema_artifacts").fetch("directory"),
    schema_element_name_form: schema_element_name_form,
    schema_element_name_overrides: schema_element_name_overrides,
    derived_type_name_formats: derived_type_name_formats,
    type_name_overrides: type_name_overrides,
    enum_value_overrides_by_type: enum_value_overrides_by_type,
    extension_modules: schema_definition_extension_modules,
    enforce_json_schema_version: enforce_json_schema_version,
    output: output
  )

  # ...then define a bunch of our own.
  define_docker_tasks("Elasticsearch", "Kibana", elasticsearch_versions, /license \[[^\]]+\] mode \[[^\]]+\] - valid/)
  define_docker_tasks("OpenSearch", "OpenSearch Dashboards", opensearch_versions, /o\.o\.n\.Node.+started/)
  define_other_tasks
end

Instance Attribute Details

#daemon_timeout ⇒ Integer

Maximum time (in seconds) to wait for the datastore to boot when booting it as a daemon. Defaults to 120.

Returns:

  • (Integer) —

    maximum time in seconds to wait when booting Elasticsearch/OpenSearch as a daemon



312
313
314
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 312

def daemon_timeout
  @daemon_timeout
end

#derived_type_name_formats ⇒ Object

Overrides for the naming formats used by ElasticGraph for derived GraphQL type names. For example, to use ‘Metrics` instead of `AggregatedValues` as the suffix for the generated types supporting getting aggregated metrid values, set to `“%{baseMetrics”}`. See SchemaDefinition::SchemaElements::TypeNamer::DEFAULT_FORMATS for the available formats.

Defaults to an empty hash.

Examples:

Change the ‘AggregatedValues` type suffix to `Metrics`

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.derived_type_name_formats = {AggregatedValues: "Metrics"}
end


109
110
111
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 109

def derived_type_name_formats
  @derived_type_name_formats
end

#elasticsearch_versions ⇒ Array<String>

List of Elasticsearch versions you want to be able to boot. Rake tasks will be defined for each version to support booting and halting Elasticsearch locally. Defaults to the versions of Elasticsearch that are exercised by the ElasticGraph test suite, as defined by ‘lib/elastic_graph/local/tested_datastore_versions.yaml`:

Examples:

Disable Elasticsearch tasks for a project that uses OpenSearch

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.elasticsearch_versions = []
end

Returns:

  • (Array<String>) —

    list of Elasticsearch versions

See Also:



259
260
261
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 259

def elasticsearch_versions
  @elasticsearch_versions
end

#enforce_json_schema_version ⇒ Boolean

Note:

Generally speaking, you will want this to be ‘true` for any ElasticGraph application that is in production as the versioning of JSON schemas is what supports safe schema evolution as it allows ElasticGraph to identify which version of the JSON schema the publishing system was operating on when it published an event.

It can be useful to set it to ‘false` before your application is in production, as you do not want to be forced to bump the version after every single schema change while you are building an initial prototype.

Whether or not to enforce the requirement that the JSON schema version is incremented every time dumping the JSON schemas results in a changed artifact. Defaults to ‘true`.

Examples:

Disable enforcement during initial prototyping

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  # TODO: remove this once we're past the prototyping stage
  tasks.enforce_json_schema_version = false
end

Returns:

  • (Boolean) —

    whether to require ‘json_schema_version` to be incremented on changes that impact `json_schemas.yaml`

See Also:

  • SchemaDefinition::API#json_schema_version


239
240
241
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 239

def enforce_json_schema_version
  @enforce_json_schema_version
end

#enum_value_overrides_by_type ⇒ Hash<Symbol, Hash<Symbol, String>>

Overrides for the names of specific GraphQL enum values for specific enum types. For example, to rename the ‘DayOfWeek.MONDAY` enum to `DayOfWeek.MON`, set to `{MONDAY: “MON”}`.

Defaults to an empty hash.

Examples:

Shorten the names of the ‘DayOfWeek` enum values

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.enum_value_overrides_by_type = {
    DayOfWeek: {
      MONDAY: "MON",
      TUESDAY: "TUE",
      WEDNESDAY: "WED",
      THURSDAY: "THU",
      FRIDAY: "FRI",
      SATURDAY: "SAT",
      SUNDAY: "SUN"
    }
  }
end

Returns:

  • (Hash<Symbol, Hash<Symbol, String>>) —

    overrides for the names of specific enum values for specific enum types



155
156
157
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 155

def enum_value_overrides_by_type
  @enum_value_overrides_by_type
end

#env_port_mapping ⇒ Hash<Symbol, Integer>

Note:

When booting Elasticsearch/OpenSearch, Kibana (or its OpenSearch equivalent, “OpenSearch Dashboards”) will also get booted, selecting the port by adding ‘10000` to the configured port.

Hash mapping environments (e.g. ‘:test`, `:dev`, etc) to port numbers for use when booting Elasticsearch or OpenSearch. The hash automatically includes an entry for the `:local` environment, using a port number extracted from `local_config_yaml`.

Examples:

Define what port to use to boot the datastore for the ‘:test` environment

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.env_port_mapping = {test: 9999}
end

Returns:

  • (Hash<Symbol, Integer>) —

    mapping from environment name to port number



298
299
300
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 298

def env_port_mapping
  @env_port_mapping
end

#index_document_sizes ⇒ Boolean

Note:

Enabling this requires the [mapper-size plugin](www.elastic.co/guide/en/elasticsearch/plugins/8.15/mapper-size.html) to be installed on your datastore cluster. You are responsible for ensuring that is installed if you enable this feature. If you enable this and the plugin is not installed, you will get errors!

When enabled, ElasticGraph will configure the index mappings so that the datastore indexes a ‘_size` field in each index document. ElasticGraph itself does not do anything with this field, but it will be available for your use in any direct queries (e.g. via Kibana).

Defaults to ‘false` since it requires a plugin.

Examples:

Enable indexing document sizes

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.index_document_sizes = true
end

Returns:

  • (Boolean) —

    whether or not the ‘_size` field should be indexed on each indexed type



49
50
51
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 49

def index_document_sizes
  @index_document_sizes
end

#opensearch_versions ⇒ Array<String>

List of OpenSearch versions you want to be able to boot. Rake tasks will be defined for each version to support booting and halting OpenSearch locally. Defaults to the versions of OpenSearch that are exercised by the ElasticGraph test suite, as defined by ‘lib/elastic_graph/local/tested_datastore_versions.yaml`:

Examples:

Disable OpenSearch tasks for a project that uses Elasticsearch

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.opensearch_versions = []
end

Returns:

  • (Array<String>) —

    list of OpenSearch versions

See Also:



279
280
281
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 279

def opensearch_versions
  @opensearch_versions
end

#output ⇒ IO

IO for printing output (defaults to stdout).

Returns:

  • (IO) —

    IO object used for printing output.



305
306
307
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 305

def output
  @output
end

#schema_definition_extension_modules ⇒ Array<Module>

List of Ruby modules to extend onto the SchemaDefinition::API instance. Designed to support ElasticGraph extensions (such as Apollo::SchemaDefinition::APIExtension). Defaults to an empty list.

Examples:

Use ‘elasticgraph-apollo`

require "elastic_graph/apollo/schema_definition/api_extension"

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.schema_definition_extension_modules = [ElasticGraph::Apollo::SchemaDefinition::APIExtension]
end

Extension that defines a ‘@since` directive and offers a `since` API on fields

module SinceExtension
  # `self.extended` is a standard Ruby hook that gets called when a module is extended onto an object.
  # The argument is the object the module was extended onto (a `SchemaDefinition::API` instance in this case).
  def self.extended(api)
    # Define our `@since` directive
    api.raw_sdl "directive @since(date: Date!) on FIELD_DEFINITION"

    # In order to hook into fields, extend the `SchemaDefinition::Factory` with a module. The factory is used
    # for creation of all schema definition objects.
    api.factory.extend FactoryExtension
  end

  module FactoryExtension
    # Hook into the creation of all `SchemaDefinition::Field` objects so that we can extend each field
    # instance with our `FieldExtension` module.
    def new_field(*args, **options)
      super(*args, **options) do |field|
        field.extend FieldExtension
        yield field if block_given?
      end
    end
  end

  # Offer a `f.since date` API on fields.
  module FieldExtension
    def since(date)
      directive "since", date: date
    end
  end
end

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.schema_definition_extension_modules = [SinceExtension]
end

Returns:

  • (Array<Module>) —

    list of extension modules



212
213
214
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 212

def schema_definition_extension_modules
  @schema_definition_extension_modules
end

#schema_element_name_form ⇒ :camelCase, :snake_case

The form of names for schema elements (fields, arguments, directives) generated by ElasticGraph, either ‘:snake_case` or `:camelCase`. For example, if set to `:camelCase`, ElasticGraph will generate a `groupedBy` field, but if set to `:snake_case`, ElasticGraph will generate a `grouped_by` field.

Defaults to ‘:camelCase` since most GraphQL schemas use that casing.

Examples:

Use ‘snake_case` names instead of `camelCase`

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.schema_element_name_form = :snake_case
end

Returns:

  • (:camelCase, :snake_case) —

    which form to use



68
69
70
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 68

def schema_element_name_form
  @schema_element_name_form
end

#schema_element_name_overrides ⇒ Hash<Symbol, String>

Overrides for specific names of schema elements (fields, arguments, directives) generated by ElasticGraph. For example, to rename the ‘gt` filter field to `greaterThan`, set to `“greaterThan”`.

Defaults to an empty hash.

Examples:

Spell out comparison operators instead of using shortened forms

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.schema_element_name_overrides = {
    gt: "greaterThan",
    gte: "greaterThanOrEqualTo",
    lt: "lessThan",
    lte: "lessThanOrEqualTo"
  }
end

Returns:

  • (Hash<Symbol, String>) —

    overrides for specific field, argument, or directive names



91
92
93
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 91

def schema_element_name_overrides
  @schema_element_name_overrides
end

#type_name_overrides ⇒ Hash<Symbol, String>

Overrides for the names of specific GraphQL types. For example, to rename the ‘JsonSafeLong` scalar to `BigInt`, set to `“BigInt`.

Defaults to an empty hash.

Examples:

Rename ‘JsonSafeLong` to `BigInt`

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.type_name_overrides = {JsonSafeLong: "BigInt"}
end

Returns:

  • (Hash<Symbol, String>) —

    overrides for specific type names



127
128
129
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 127

def type_name_overrides
  @type_name_overrides
end

Instance Method Details

#define_fake_data_batch_for(type) {|Array<Hash<String, Object>>, Array<Hash<Symbol, Object>>| ... } ⇒ Object

Register a callback for use when indexing a batch fake data. An ‘index_fake_data:` rake task will be generated for each registered callback.

Examples:

Register a callback to generate fake ‘campaigns` data

ElasticGraph::Local::RakeTasks.new(
  local_config_yaml: "config/settings/local.yaml",
  path_to_schema: "config/schema.rb"
) do |tasks|
  tasks.define_fake_data_batch_for :campaigns do |batch|
    batch.concat(FactoryBot.build_list(:campaigns))
  end
end

Parameters:

  • type (Symbol) —

    type of data batch. Can be the name of a GraphQL type or any other name you want to give a batch of fake data

Yields:

  • (Array<Hash<String, Object>>, Array<Hash<Symbol, Object>>) —

    list the block should append to when generating data

Yield Returns:

  • (void) 


341
342
343
 
# File 'lib/elastic_graph/local/rake_tasks.rb', line 341

def define_fake_data_batch_for(type, &block)
  @fake_data_batch_generator_by_type[type] = block
end