Class: Gitlab::Redis::MultiStore

Inherits:
Object
  • Object
show all
Includes:
Utils::StrongMemoize
Defined in:
lib/gitlab/redis/multi_store.rb

Defined Under Namespace

Classes: MethodMissingError, NestedReadonlyPipelineError, PipelinedDiffError

Constant Summary collapse

FAILED_TO_READ_ERROR_MESSAGE = 
'Failed to read from the redis default_store.'
FAILED_TO_WRITE_ERROR_MESSAGE = 
'Failed to write to the redis non_default_store.'
FAILED_TO_RUN_PIPELINE = 
'Failed to execute pipeline on the redis non_default_store.'
SKIP_LOG_METHOD_MISSING_FOR_COMMANDS = 
%i[info].freeze
REDIS_CLIENT_COMMANDS =

_client and without_reconnect are Redis::Client methods which may be called through multistore

%i[
  _client
  without_reconnect
].freeze
PUBSUB_SUBSCRIBE_COMMANDS = 
%i[
  subscribe
  unsubscribe
].freeze
READ_COMMANDS = 
%i[
  exists
  exists?
  get
  hexists
  hget
  hgetall
  hlen
  hmget
  hscan_each
  llen
  lrange
  mapped_hmget
  mget
  pfcount
  pttl
  scan
  scan_each
  scard
  sismember
  smembers
  sscan
  sscan_each
  strlen
  ttl
  type
  zcard
  zcount
  zrange
  zrangebyscore
  zrevrange
  zscan_each
  zscore
].freeze
WRITE_COMMANDS = 
%i[
  decr
  del
  eval
  expire
  flushdb
  hdel
  hincrby
  hset
  incr
  incrby
  incrbyfloat
  ltrim
  mapped_hmset
  pfadd
  pfmerge
  publish
  rpush
  sadd
  sadd?
  set
  setex
  setnx
  spop
  srem
  srem?
  unlink
  zadd
  zpopmin
  zrem
  zremrangebyrank
  zremrangebyscore

  memory
].freeze
PIPELINED_COMMANDS = 
%i[
  pipelined
  multi
].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(primary_pool:, secondary_pool:, primary_store:, secondary_store:, instance_name:) ⇒ MultiStore

To transition between two Redis store, primary_pool should be the connection pool for the target store, and secondary_pool should be the connection pool for the current store.

Transition is controlled with feature flags:

  • At the default state, all read and write operations are executed in the secondary instance.

  • Turning use_primary_and_secondary_stores_for_<instance_name> on: The store writes to both instances. The read commands are executed in the default store with no fallbacks. Other commands are executed in the the default instance (Secondary).

  • Turning use_primary_store_as_default_for_<instance_name> on: The behavior is the same as above, but other commands are executed in the primary now.

  • Turning use_primary_and_secondary_stores_for_<instance_name> off: commands are executed in the primary store.



173
174
175
176
177
178
179
180
181
 
# File 'lib/gitlab/redis/multi_store.rb', line 173

def initialize(primary_pool:, secondary_pool:, primary_store:, secondary_store:, instance_name:)
  @instance_name = instance_name
  @primary_pool = primary_pool
  @secondary_pool = secondary_pool
  @primary_store = primary_store
  @secondary_store = secondary_store

  validate_attributes!
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing ⇒ Object 



234
235
236
237
238
239
240
 
# File 'lib/gitlab/redis/multi_store.rb', line 234

def method_missing(...)
  return @instance.send(...) if @instance

  log_method_missing(...)

  default_store.send(...)
end

Instance Attribute Details

#instance_name ⇒ Object (readonly)

Returns the value of attribute instance_name.



37
38
39
 
# File 'lib/gitlab/redis/multi_store.rb', line 37

def instance_name
  @instance_name
end

#primary_pool ⇒ Object (readonly)

Returns the value of attribute primary_pool.



37
38
39
 
# File 'lib/gitlab/redis/multi_store.rb', line 37

def primary_pool
  @primary_pool
end

#primary_store ⇒ Object (readonly)

Returns the value of attribute primary_store.



37
38
39
 
# File 'lib/gitlab/redis/multi_store.rb', line 37

def primary_store
  @primary_store
end

#secondary_pool ⇒ Object (readonly)

Returns the value of attribute secondary_pool.



37
38
39
 
# File 'lib/gitlab/redis/multi_store.rb', line 37

def secondary_pool
  @secondary_pool
end

#secondary_store ⇒ Object (readonly)

Returns the value of attribute secondary_store.



37
38
39
 
# File 'lib/gitlab/redis/multi_store.rb', line 37

def secondary_store
  @secondary_store
end

Class Method Details

.create_using_client(primary_store, secondary_store, instance_name) ⇒ Object

Initialises a MultiStore with 2 client connections. This is used in situations where the caller does not use connection pools and holds long-running connections,e.g. ActionCable pubsub. This would create additional connections to the primary and secondary store on per MultiStore instance.



151
152
153
154
155
156
157
158
159
 
# File 'lib/gitlab/redis/multi_store.rb', line 151

def create_using_client(primary_store, secondary_store, instance_name)
  new(
    primary_store: primary_store,
    secondary_store: secondary_store,
    instance_name: instance_name,
    primary_pool: nil,
    secondary_pool: nil
  )
end

.create_using_pool(primary_pool, secondary_pool, instance_name) ⇒ Object

This initialises a MultiStore instance with references to 2 pools. All method calls must be wrapped in a .with_borrowed_connection scope.

This is the preferred way to use MultiStore as re-using connections from connection pools is more efficient than spinning up new ones.



138
139
140
141
142
143
144
145
146
 
# File 'lib/gitlab/redis/multi_store.rb', line 138

def create_using_pool(primary_pool, secondary_pool, instance_name)
  new(
    primary_pool: primary_pool,
    secondary_pool: secondary_pool,
    instance_name: instance_name,
    primary_store: nil,
    secondary_store: nil
  )
end

Instance Method Details

#blpop(*args) ⇒ Object

blpop blocks until an element to be popped exist in the list or after a timeout.



368
369
370
371
372
373
374
375
376
377
378
 
# File 'lib/gitlab/redis/multi_store.rb', line 368

def blpop(*args)
  result = default_store.blpop(*args)
  if !!result && use_primary_and_secondary_stores?
    # special case to accommodate Gitlab::JobWaiter as blpop is only used in JobWaiter
    # 1s should be sufficient wait time to account for delays between 1st and 2nd lpush
    # https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/2520#note_1630893702
    non_default_store.blpop(args.first, timeout: 1)
  end

  result
end

#close ⇒ Object

connection_pool gem calls #close method:

github.com/mperham/connection_pool/blob/v2.4.1/lib/connection_pool.rb#L63

Let’s define it explicitly instead of propagating it to method_missing



354
355
356
357
358
359
360
361
362
363
364
365
 
# File 'lib/gitlab/redis/multi_store.rb', line 354

def close
  return if primary_store.nil? || secondary_store.nil?

  if same_redis_store?
    # if same_redis_store?, `use_primary_store_as_default?` returns false
    # but we should avoid a feature-flag check in `.close` to avoid checking out
    # an ActiveRecord connection during clean up.
    secondary_store.close
  else
    [primary_store, secondary_store].map(&:close).first
  end
end

#default_store ⇒ Object 



330
331
332
 
# File 'lib/gitlab/redis/multi_store.rb', line 330

def default_store
  use_primary_store_as_default? ? primary_store : secondary_store
end

#feature_flag_type(feature_flag) ⇒ Object 



305
306
307
308
309
310
 
# File 'lib/gitlab/redis/multi_store.rb', line 305

def feature_flag_type(feature_flag)
  feature_definition = Feature::Definition.get(feature_flag)
  return if feature_definition

  :undefined
end

#increment_method_missing_count(command_name) ⇒ Object 



318
319
320
321
322
 
# File 'lib/gitlab/redis/multi_store.rb', line 318

def increment_method_missing_count(command_name)
  @method_missing_counter ||= Gitlab::Metrics.counter(:gitlab_redis_multi_store_method_missing_total,
    'Client side Redis MultiStore method missing')
  @method_missing_counter.increment(command: command_name, instance_name: instance_name)
end

#increment_pipelined_command_error_count(command_name) ⇒ Object 



312
313
314
315
316
 
# File 'lib/gitlab/redis/multi_store.rb', line 312

def increment_pipelined_command_error_count(command_name)
  @pipelined_command_error ||= Gitlab::Metrics.counter(:gitlab_redis_multi_store_pipelined_diff_error_total,
    'Redis MultiStore pipelined command diff between stores')
  @pipelined_command_error.increment(command: command_name, instance_name: instance_name)
end

#is_a?(klass) ⇒ Boolean Also known as: kind_of?

This is needed because of Redis::Rack::Connection is requiring Redis::Store github.com/redis-store/redis-rack/blob/a833086ba494083b6a384a1a4e58b36573a9165d/lib/redis/rack/connection.rb#L15 Done similarly in github.com/lsegal/yard/blob/main/lib/yard/templates/template.rb#L122

Returns:



272
273
274
275
276
 
# File 'lib/gitlab/redis/multi_store.rb', line 272

def is_a?(klass)
  return true if klass == default_store.class

  super(klass)
end

#log_error(exception, command_name, extra = {}) ⇒ Object 



324
325
326
327
328
 
# File 'lib/gitlab/redis/multi_store.rb', line 324

def log_error(exception, command_name, extra = {})
  Gitlab::ErrorTracking.log_exception(
    exception,
    extra.merge(command_name: command_name, instance_name: instance_name))
end

#non_default_store ⇒ Object 



334
335
336
 
# File 'lib/gitlab/redis/multi_store.rb', line 334

def non_default_store
  use_primary_store_as_default? ? secondary_store : primary_store
end

#ping(message = nil) ⇒ Object 



338
339
340
341
342
343
344
345
346
347
 
# File 'lib/gitlab/redis/multi_store.rb', line 338

def ping(message = nil)
  if use_primary_and_secondary_stores?
    # Both stores have to response success for the ping to be considered success.
    # We assume both stores cannot return different responses (only both "PONG" or both echo the message).
    # If either store is not reachable, an Error will be raised anyway thus taking any response works.
    [primary_store, secondary_store].map { |store| store.ping(message) }.first
  else
    default_store.ping(message)
  end
end

#readonly_pipeline? ⇒ Boolean

Returns:



199
200
201
 
# File 'lib/gitlab/redis/multi_store.rb', line 199

def readonly_pipeline?
  Thread.current[:readonly_pipeline].present?
end

#respond_to_missing?(command_name, include_private = false) ⇒ Boolean

rubocop:enable GitlabSecurity/PublicSend

Returns:



243
244
245
 
# File 'lib/gitlab/redis/multi_store.rb', line 243

def respond_to_missing?(command_name, include_private = false)
  true
end

#to_s ⇒ Object 



279
280
281
 
# File 'lib/gitlab/redis/multi_store.rb', line 279

def to_s
  use_primary_and_secondary_stores? ? primary_store.to_s : default_store.to_s
end

#use_primary_and_secondary_stores? ⇒ Boolean

Returns:



283
284
285
286
287
288
289
290
291
292
 
# File 'lib/gitlab/redis/multi_store.rb', line 283

def use_primary_and_secondary_stores?
  # We interpolate the feature flag name within `Feature.enabled?` instead of defining a variable to allow
  # `RuboCop::Cop::Gitlab::MarkUsedFeatureFlags`'s optimistic matching to work.
  feature_table_exists? &&
    Feature.enabled?( # rubocop:disable Cop/FeatureFlagUsage -- The flags are dynamic
      "use_primary_and_secondary_stores_for_#{instance_name.underscore}",
      type: feature_flag_type("use_primary_and_secondary_stores_for_#{instance_name.underscore}")
    ) &&
    !same_redis_store?
end

#use_primary_store_as_default? ⇒ Boolean

Returns:



294
295
296
297
298
299
300
301
302
303
 
# File 'lib/gitlab/redis/multi_store.rb', line 294

def use_primary_store_as_default?
  # We interpolate the feature flag name within `Feature.enabled?` instead of defining a variable to allow
  # `RuboCop::Cop::Gitlab::MarkUsedFeatureFlags`'s optimistic matching to work.
  feature_table_exists? &&
    Feature.enabled?( # rubocop:disable Cop/FeatureFlagUsage -- The flags are dynamic
      "use_primary_store_as_default_for_#{instance_name.underscore}",
      type: feature_flag_type("use_primary_store_as_default_for_#{instance_name.underscore}")
    ) &&
    !same_redis_store?
end

#with_borrowed_connection ⇒ Object 



247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
 
# File 'lib/gitlab/redis/multi_store.rb', line 247

def with_borrowed_connection
  return yield if @primary_pool.nil? && @secondary_pool.nil? && @primary_store && @secondary_store

  primary_pool.with do |ps|
    secondary_pool.with do |ss|
      original_primary_store = @primary_store
      original_secondary_store = @secondary_store

      # borrow from both pool as feature-flag could change during the period where connections are borrowed
      # this guarantees that we avoids a NilClass error
      @primary_store = ps
      @secondary_store = ss

      yield
    ensure
      # resets value
      @primary_store = original_primary_store
      @secondary_store = original_secondary_store
    end
  end
end

#with_readonly_pipeline ⇒ Object

Pipelines are sent to both instances by default since they could execute both read and write commands.

But for pipelines that only consists of read commands, this method can be used to scope the pipeline and send it only to the default store.



189
190
191
192
193
194
195
196
197
 
# File 'lib/gitlab/redis/multi_store.rb', line 189

def with_readonly_pipeline
  raise NestedReadonlyPipelineError if readonly_pipeline?

  Thread.current[:readonly_pipeline] = true

  yield
ensure
  Thread.current[:readonly_pipeline] = false
end