Class: Gitlab::BackgroundMigration::BatchedMigrationJob

Inherits:
Object
  • Object
show all
Includes:
ClassAttributes, Database::DynamicModelHelpers
Defined in:
lib/gitlab/background_migration/batched_migration_job.rb

Overview

Base class for batched background migrations. Subclasses should implement the ‘#perform` method as the entry point for the job’s execution.

Job arguments needed must be defined explicitly, see docs.gitlab.com/ee/development/database/batched_background_migrations.html#job-arguments. rubocop:disable Metrics/ClassLength rubocop:disable Metrics/ParameterLists

Direct Known Subclasses

AddNamespacesEmailsEnabledColumnData, AddProjectsEmailsEnabledColumnData, BackfillAdminModeScopeForPersonalAccessTokens, BackfillClusterAgentsHasVulnerabilities, BackfillCodeSuggestionsNamespaceSettings, BackfillComplianceViolations, BackfillDefaultBranchProtectionNamespaceSetting, BackfillDesignManagementRepositories, BackfillDismissalReasonInVulnerabilityReads, BackfillEnvironmentTiers, BackfillEpicCacheCounts, BackfillFindingIdInVulnerabilities, BackfillHasMergeRequestOfVulnerabilityReads, BackfillImportedIssueSearchData, BackfillInternalOnNotes, BackfillMissingCiCdSettings, BackfillMissingVulnerabilityDismissalDetails, BackfillNamespaceDetails, BackfillNamespaceIdOfVulnerabilityReads, BackfillNamespaceLdapSettings, BackfillNugetNormalizedVersion, BackfillPartitionedTable, BackfillPreparedAtMergeRequests, BackfillProjectFeaturePackageRegistryAccessLevel, BackfillProjectImportLevel, BackfillProjectMemberNamespaceId, BackfillProjectNamespaceDetails, BackfillProjectNamespaceOnIssues, BackfillProjectStatisticsContainerRepositorySize, BackfillProjectStatisticsStorageSizeWithRecentSize, BackfillProjectStatisticsStorageSizeWithoutPipelineArtifactsSizeJob, BackfillProjectStatisticsStorageSizeWithoutUploadsSize, BackfillProjectWikiRepositories, BackfillReleasesAuthorId, BackfillResourceLinkEvents, BackfillRootStorageStatisticsForkStorageSizes, BackfillUserDetailsFields, BackfillUserPreferencesWithDefaults, BackfillUsersWithDefaults, BackfillUuidConversionColumnInVulnerabilityOccurrences, BackfillVulnerabilityReadsClusterAgent, BackfillWorkItemTypeIdForIssues, BackfillWorkspacePersonalAccessToken, CleanupOrphanedRoutes, CleanupPersonalAccessTokensWithNilExpiresAt, ConvertCreditCardValidationDataToHashes, CopyColumnUsingBackgroundMigrationJob, CreateComplianceStandardsAdherence, CreateVulnerabilityLinks, DeleteApprovalRulesWithVulnerability, DeleteInvalidEpicIssues, DeleteOrphanedOperationalVulnerabilities, DeleteOrphanedPackagesDependencies, DeleteOrphanedTransferredProjectApprovalRules, DeleteOrphansApprovalMergeRequestRules, DeleteOrphansApprovalProjectRules, DestroyInvalidGroupMembers, DestroyInvalidMembers, DestroyInvalidProjectMembers, DisableLegacyOpenSourceLicenceForRecentPublicProjects, DisableLegacyOpenSourceLicenseForInactivePublicProjects, DisableLegacyOpenSourceLicenseForNoIssuesNoRepoProjects, DisableLegacyOpenSourceLicenseForOneMemberNoRepoProjects, DisableLegacyOpenSourceLicenseForProjectsLessThanFiveMb, DisableLegacyOpenSourceLicenseForProjectsLessThanOneMb, EncryptCiTriggerToken, ExpireOAuthTokens, FixAllowDescendantsOverrideDisabledSharedRunners, FixApprovalProjectRulesWithoutProtectedBranches, FixIncoherentPackagesSizeOnProjectStatistics, FixNamespaceIdsOfVulnerabilityReads, FixSecurityScanStatuses, FixVulnerabilityReadsHasIssues, IssuesInternalIdScopeUpdater, MarkDuplicateNpmPackagesForDestruction, MigrateEvidencesForVulnerabilityFindings, MigrateHumanUserType, MigrateLinksForVulnerabilityFindings, MigrateRemediationsForVulnerabilityFindings, MigrateSharedVulnerabilityIdentifiers, MigrateSharedVulnerabilityScanners, MigrateVulnerabilitiesFeedbackToVulnerabilitiesStateTransition, NullifyCreatorIdColumnOfOrphanedProjects, NullifyLastErrorFromProjectMirrorData, PopulateApprovalMergeRequestRulesWithSecurityOrchestration, PopulateApprovalProjectRulesWithSecurityOrchestration, PopulateDenormalizedColumnsForSbomOccurrences, PopulateOperationVisibilityPermissionsFromOperations, PopulateProjectsStarCount, PopulateVulnerabilityDismissalFields, PruneStaleProjectExportJobs, RecountEpicCacheCounts, RemoveBackfilledJobArtifactsExpireAt, RemoveInvalidDeployAccessLevelGroups, RemoveProjectGroupLinkWithMissingGroups, RemoveSelfManagedWikiNotes, RenameTaskSystemNoteToChecklistItem, ResetDuplicateCiRunnersTokenEncryptedValues, ResetDuplicateCiRunnersTokenValues, ResetStatusOnContainerRepositories, SanitizeConfidentialTodos, SecondRecountEpicCacheCounts, SetCorrectVulnerabilityState, SetLegacyOpenSourceLicenseAvailableForNonPublicProjects, SyncScanResultPolicies, ThirdRecountEpicCacheCounts, TruncateOverlongVulnerabilityHtmlTitles, UpdateCiPipelineArtifactsUnknownLockedStatus, UpdateDelayedProjectRemovalToNullForUserNamespaces, UpdateJiraTrackerDataDeploymentTypeBasedOnUrl, UpdateUsersSetExternalIfServiceAccount

Constant Summary collapse

DEFAULT_FEATURE_CATEGORY = 
:database

Constants included from Database::DynamicModelHelpers

Database::DynamicModelHelpers::BATCH_SIZE

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Database::DynamicModelHelpers

#define_batchable_model, #each_batch, #each_batch_range

Constructor Details

#initialize(start_id:, end_id:, batch_table:, batch_column:, sub_batch_size:, pause_ms:, job_arguments: [], connection:, sub_batch_exception: nil) ⇒ BatchedMigrationJob

Returns a new instance of BatchedMigrationJob.



64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/gitlab/background_migration/batched_migration_job.rb', line 64

def initialize(
  start_id:, end_id:, batch_table:, batch_column:, sub_batch_size:, pause_ms:, job_arguments: [], connection:,
  sub_batch_exception: nil
)

  @start_id = start_id
  @end_id = end_id
  @batch_table = batch_table
  @batch_column = batch_column
  @sub_batch_size = sub_batch_size
  @pause_ms = pause_ms
  @job_arguments = job_arguments
  @connection = connection
  @sub_batch_exception = sub_batch_exception
end

Class Method Details

.feature_category(feature_category_name = nil) ⇒ Object 



55
56
57
58
59
60
61
 
# File 'lib/gitlab/background_migration/batched_migration_job.rb', line 55

def feature_category(feature_category_name = nil)
  if feature_category_name.present?
    set_class_attribute(:feature_category, feature_category_name)
  else
    get_class_attribute(:feature_category) || DEFAULT_FEATURE_CATEGORY
  end
end

.generic_instance(batch_table:, batch_column:, job_arguments: [], connection:) ⇒ Object 



19
20
21
22
23
24
25
 
# File 'lib/gitlab/background_migration/batched_migration_job.rb', line 19

def generic_instance(batch_table:, batch_column:, job_arguments: [], connection:)
  new(
    batch_table: batch_table, batch_column: batch_column,
    job_arguments: job_arguments, connection: connection,
    start_id: 0, end_id: 0, sub_batch_size: 0, pause_ms: 0
  )
end

.job_arguments(*args) ⇒ Object 



37
38
39
40
41
42
43
44
45
46
47
 
# File 'lib/gitlab/background_migration/batched_migration_job.rb', line 37

def job_arguments(*args)
  args.each.with_index do |arg, index|
    define_method(arg) do
      @job_arguments[index]
    end
  end

  define_singleton_method(:job_arguments_count) do
    args.count
  end
end

.job_arguments_count ⇒ Object 



27
28
29
 
# File 'lib/gitlab/background_migration/batched_migration_job.rb', line 27

def job_arguments_count
  0
end

.operation_name(operation) ⇒ Object 



31
32
33
34
35
 
# File 'lib/gitlab/background_migration/batched_migration_job.rb', line 31

def operation_name(operation)
  define_method(:operation_name) do
    operation
  end
end

.scope_to(scope) ⇒ Object 



49
50
51
52
53
 
# File 'lib/gitlab/background_migration/batched_migration_job.rb', line 49

def scope_to(scope)
  define_method(:filter_batch) do |relation|
    instance_exec(relation, &scope)
  end
end

Instance Method Details

#batch_metrics ⇒ Object 



88
89
90
 
# File 'lib/gitlab/background_migration/batched_migration_job.rb', line 88

def batch_metrics
  @batch_metrics ||= Gitlab::Database::BackgroundMigration::BatchMetrics.new
end

#filter_batch(relation) ⇒ Object 



80
81
82
 
# File 'lib/gitlab/background_migration/batched_migration_job.rb', line 80

def filter_batch(relation)
  relation
end

#perform ⇒ Object

Raises:

  • (NotImplementedError) 


84
85
86
 
# File 'lib/gitlab/background_migration/batched_migration_job.rb', line 84

def perform
  raise NotImplementedError, "subclasses of #{self.class.name} must implement #{__method__}"
end