Class: Sequent::Migrations::ViewSchema

Inherits:
Object
  • Object
show all
Includes:
Util::Printer, Util::Timer
Defined in:
lib/sequent/migrations/view_schema.rb

Overview

Responsible for migration of Projectors between view schema versions.

A Projector needs migration when for instance:

  • New columns are added

  • Structure is changed

To maintain your migrations you need to:

  1. Create a class that extends ‘Sequent::Migrations::Projectors` and specify in `Sequent.configuration.migrations_class_name`

  2. Define per version which Projectors you want to migrate See the definition of ‘Sequent::Migrations::Projectors.versions` and `Sequent::Migrations::Projectors.version`

  3. Specify in Sequent where your sql files reside (Sequent.configuration.migration_sql_files_directory)

  4. Ensure that you add %SUFFIX% to each name that needs to be unique in postgres (like TABLE names, INDEX names, PRIMARY KEYS) E.g. ‘create table foo%SUFFIX% (id serial NOT NULL, CONSTRAINT foo_pkey%SUFFIX% PRIMARY KEY (id))`

Defined Under Namespace

Classes: ReplayedIds, Versions

Constant Summary collapse

LENGTH_OF_SUBSTRING_INDEX_ON_AGGREGATE_ID_IN_EVENT_STORE =

Corresponds with the index on aggregate_id column in the event_records table

Since we replay in batches of the first 3 chars of the uuid we created an index on these 3 characters. Hence the name ;-)

This also means that the online replay is divided up into 16**3 groups This might seem a lot for starting event store, but when you will get more events, you will see that this is pretty good partitioned.

3

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Util::Printer

#recursively_print

Methods included from Util::Timer

#time

Constructor Details

#initialize(db_config:) ⇒ ViewSchema

Returns a new instance of ViewSchema.



51
52
53
54
55
 
# File 'lib/sequent/migrations/view_schema.rb', line 51

def initialize(db_config:)
  @db_config = db_config
  @view_schema = Sequent.configuration.view_schema_name
  @logger = Sequent.logger
end

Instance Attribute Details

#db_config ⇒ Object (readonly)

Returns the value of attribute db_config.



49
50
51
 
# File 'lib/sequent/migrations/view_schema.rb', line 49

def db_config
  @db_config
end

#logger ⇒ Object (readonly)

Returns the value of attribute logger.



49
50
51
 
# File 'lib/sequent/migrations/view_schema.rb', line 49

def logger
  @logger
end

#view_schema ⇒ Object (readonly)

Returns the value of attribute view_schema.



49
50
51
 
# File 'lib/sequent/migrations/view_schema.rb', line 49

def view_schema
  @view_schema
end

Instance Method Details

#create_view_schema_if_not_exists ⇒ Object

Utility method that creates the view_schema and the meta data tables

This method is mainly useful during an initial setup of the view schema



90
91
92
93
94
95
96
 
# File 'lib/sequent/migrations/view_schema.rb', line 90

def create_view_schema_if_not_exists
  exec_sql(%Q{CREATE SCHEMA IF NOT EXISTS #{view_schema}})
  in_view_schema do
    exec_sql(%Q{CREATE TABLE IF NOT EXISTS #{Versions.table_name} (version integer NOT NULL, CONSTRAINT version_pk PRIMARY KEY(version))})
    exec_sql(%Q{CREATE TABLE IF NOT EXISTS #{ReplayedIds.table_name} (event_id bigint NOT NULL, CONSTRAINT event_id_pk PRIMARY KEY(event_id))})
  end
end

#create_view_tables ⇒ Object

Utility method that creates all tables in the view schema

This method is mainly useful in test scenario to just create the entire view schema without replaying the events



68
69
70
71
72
73
74
75
76
 
# File 'lib/sequent/migrations/view_schema.rb', line 68

def create_view_tables
  create_view_schema_if_not_exists
  in_view_schema do
    Sequent::Core::Migratable.all.flat_map(&:managed_tables).each do |table|
      statements = sql_file_to_statements("#{Sequent.configuration.migration_sql_files_directory}/#{table.table_name}.sql") { |raw_sql| raw_sql.remove('%SUFFIX%') }
      statements.each { |statement| exec_sql(statement) }
    end
  end
end

#current_version ⇒ Object

Returns the current version from the database



59
60
61
 
# File 'lib/sequent/migrations/view_schema.rb', line 59

def current_version
  Versions.order('version desc').limit(1).first&.version || 0
end

#migrate_offline ⇒ Object

Last part of a view schema migration

You have to ensure no events are being added to the event store while this method is running. For instance put your application in maintenance mode.

The offline part consists of:

  1. Replay all events not yet replayed since #migration_online

  2. Within a single transaction do:

2.1 Rename current tables with the current version as SUFFIX 2.2 Rename the new tables and remove the new version suffix 2.3 Add the new version in the Versions table

  1. Performs cleanup of replayed event ids

If anything fails an exception is raised and everything is rolled back

When this method succeeds you can safely start the application from Sequent’s point of view.



153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
 
# File 'lib/sequent/migrations/view_schema.rb', line 153

def migrate_offline
  return if Sequent.new_version == current_version

  ensure_version_correct!

  set_table_names_to_new_version

  # 1 replay events not yet replayed
  replay!(projectors_to_migrate, Sequent.configuration.offline_replay_persistor_class.new, exclude_ids: true, group_exponent: 1)

  in_view_schema do
    ActiveRecord::Base.transaction do
      for_each_table_to_migrate do |table|
        current_table_name = table.table_name.gsub("_#{Sequent.new_version}", "")
        # 2 Rename old table
        exec_sql("ALTER TABLE IF EXISTS #{current_table_name} RENAME TO #{current_table_name}_#{current_version}")
        # 3 Rename new table
        exec_sql("ALTER TABLE #{table.table_name} RENAME TO #{current_table_name}")
        # Use new table from now on
        table.table_name = current_table_name
        table.reset_column_information
      end
      # 4. Create migration record
      Versions.create!(version: Sequent.new_version)
    end

    # 5. Truncate replayed ids
    truncate_replay_ids_table!
  end
  logger.info "Migrated to version #{Sequent.new_version}"
rescue Exception => e
  rollback_migration
  raise e
end

#migrate_online ⇒ Object

First part of a view schema migration

Call this method while your application is running. The online part consists of:

  1. Ensure any previous migrations are cleaned up

  2. Create new tables for the Projectors which need to be migrated to the new version

These tables will be called `table_name_VERSION`.

  1. Replay all events to populate the tables

It keeps track of all events that are already replayed.

If anything fails an exception is raised and everything is rolled back



112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
 
# File 'lib/sequent/migrations/view_schema.rb', line 112

def migrate_online
  return if Sequent.new_version == current_version

  ensure_version_correct!

  in_view_schema do
    truncate_replay_ids_table!

    drop_old_tables(Sequent.new_version)
    for_each_table_to_migrate do |table|
      statements = sql_file_to_statements("#{Sequent.configuration.migration_sql_files_directory}/#{table.table_name}.sql") { |raw_sql| raw_sql.gsub('%SUFFIX%', "_#{Sequent.new_version}") }
      statements.each { |statement| exec_sql(statement) }
      table.table_name = "#{table.table_name}_#{Sequent.new_version}"
      table.reset_column_information
    end
  end
  replay!(projectors_to_migrate, Sequent.configuration.online_replay_persistor_class.new)
rescue Exception => e
  rollback_migration
  raise e
end

#replay_all! ⇒ Object

Utility method that replays events for all managed_tables from all Sequent::Core::Projector’s

This method is mainly useful in test scenario’s or development tasks



82
83
84
 
# File 'lib/sequent/migrations/view_schema.rb', line 82

def replay_all!
  replay!(Sequent::Core::Migratable.all, Sequent.configuration.online_replay_persistor_class.new)
end