Class: Gitlab::HashedStorage::Migrator

Inherits:
Object
  • Object
show all
Defined in:
lib/gitlab/hashed_storage/migrator.rb

Overview

Hashed Storage Migrator

This is responsible for scheduling and flagging projects to be migrated from Legacy to Hashed storage, either one by one or in bulk.

Constant Summary collapse

BATCH_SIZE = 
100

Instance Method Summary collapse

Instance Method Details

#abort_rollback! ⇒ Object

Remove all remaining scheduled rollback operations



98
99
100
101
102
 
# File 'lib/gitlab/hashed_storage/migrator.rb', line 98

def abort_rollback!
  [::HashedStorage::RollbackerWorker, ::HashedStorage::ProjectRollbackWorker].each do |worker|
    Sidekiq::Queue.new(worker.queue).clear
  end
end

#bulk_migrate(start:, finish:) ⇒ Object

Start migration of projects from specified range

Flagging a project to be migrated is a synchronous action but the migration runs through async jobs

rubocop: disable CodeReuse/ActiveRecord

Parameters:

  • start (Integer) —

    first project id for the range

  • finish (Integer) —

    last project id for the range



36
37
38
39
40
41
42
 
# File 'lib/gitlab/hashed_storage/migrator.rb', line 36

def bulk_migrate(start:, finish:)
  projects = build_relation(start, finish)

  projects.with_route.find_each(batch_size: BATCH_SIZE) do |project|
    migrate(project)
  end
end

#bulk_rollback(start:, finish:) ⇒ Object

Start rollback of projects from specified range

Flagging a project to be rolled back is a synchronous action but the rollback runs through async jobs

rubocop: disable CodeReuse/ActiveRecord

Parameters:

  • start (Integer) —

    first project id for the range

  • finish (Integer) —

    last project id for the range



53
54
55
56
57
58
59
 
# File 'lib/gitlab/hashed_storage/migrator.rb', line 53

def bulk_rollback(start:, finish:)
  projects = build_relation(start, finish)

  projects.with_route.find_each(batch_size: BATCH_SIZE) do |project|
    rollback(project)
  end
end

#bulk_schedule_migration(start:, finish:) ⇒ Object

Schedule a range of projects to be bulk migrated with #bulk_migrate asynchronously

Parameters:

  • start (Integer) —

    first project id for the range

  • finish (Integer) —

    last project id for the range



16
17
18
 
# File 'lib/gitlab/hashed_storage/migrator.rb', line 16

def bulk_schedule_migration(start:, finish:)
  ::HashedStorage::MigratorWorker.perform_async(start, finish)
end

#bulk_schedule_rollback(start:, finish:) ⇒ Object

Schedule a range of projects to be bulk rolledback with #bulk_rollback asynchronously

Parameters:

  • start (Integer) —

    first project id for the range

  • finish (Integer) —

    last project id for the range



24
25
26
 
# File 'lib/gitlab/hashed_storage/migrator.rb', line 24

def bulk_schedule_rollback(start:, finish:)
  ::HashedStorage::RollbackerWorker.perform_async(start, finish)
end

#migrate(project) ⇒ Object

Flag a project to be migrated to Hashed Storage

Parameters:

  • project (Project) —

    that will be migrated



65
66
67
68
69
70
71
 
# File 'lib/gitlab/hashed_storage/migrator.rb', line 65

def migrate(project)
  Gitlab::AppLogger.info "Starting storage migration of #{project.full_path} (ID=#{project.id})..."

  project.migrate_to_hashed_storage!
rescue StandardError => err
  Gitlab::AppLogger.error("#{err.message} migrating storage of #{project.full_path} (ID=#{project.id}), trace - #{err.backtrace}")
end

#migration_pending? ⇒ Boolean

Returns whether we have any pending storage migration

Returns:

  • (Boolean) 


86
87
88
 
# File 'lib/gitlab/hashed_storage/migrator.rb', line 86

def migration_pending?
  any_non_empty_queue?(::HashedStorage::MigratorWorker, ::HashedStorage::ProjectMigrateWorker)
end

#rollback(project) ⇒ Object

Flag a project to be rolled-back to Legacy Storage

Parameters:

  • project (Project) —

    that will be rolled-back



76
77
78
79
80
81
82
 
# File 'lib/gitlab/hashed_storage/migrator.rb', line 76

def rollback(project)
  Gitlab::AppLogger.info "Starting storage rollback of #{project.full_path} (ID=#{project.id})..."

  project.rollback_to_legacy_storage!
rescue StandardError => err
  Gitlab::AppLogger.error("#{err.message} rolling-back storage of #{project.full_path} (ID=#{project.id}), trace - #{err.backtrace}")
end

#rollback_pending? ⇒ Boolean

Returns whether we have any pending storage rollback

Returns:

  • (Boolean) 


92
93
94
 
# File 'lib/gitlab/hashed_storage/migrator.rb', line 92

def rollback_pending?
  any_non_empty_queue?(::HashedStorage::RollbackerWorker, ::HashedStorage::ProjectRollbackWorker)
end