Class: Webhookdb::BackfillJob

Inherits:
Object
  • Object
show all
Defined in:
lib/webhookdb/backfill_job.rb

Overview

Represents the boundaries around a single execution of backfilling an integration. Each instance points to a single run of a single integration. There may be child jobs pointing to dependent integrations, or a parent job for a dependency backfill.

When creating jobs, you can create a single job (a ‘shallow’ backfill) with create, or use create_recursive to create jobs for all dependencies (a ‘job group’).

Each job tracks when the backfill starts and ends. Iterating the full job graph can determine if a group is fully finished, or still in-progress.

Defined Under Namespace

Classes: ServiceIntegrationLock

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#_fixture_cascade ⇒ Object

Returns the value of attribute _fixture_cascade.



25
26
27
 
# File 'lib/webhookdb/backfill_job.rb', line 25

def _fixture_cascade
  @_fixture_cascade
end

Class Method Details

.create_recursive(service_integration:, incremental:, created_by: nil, parent_job: nil, criteria: nil) ⇒ Webhookdb::BackfillJob

Returns:



28
29
30
31
32
33
34
 
# File 'lib/webhookdb/backfill_job.rb', line 28

def self.create_recursive(service_integration:, incremental:, created_by: nil, parent_job: nil, criteria: nil)
  self.db.transaction do
    root = self.create(service_integration:, parent_job:, incremental:, created_by:, criteria: criteria || {})
    root.setup_recursive
    root
  end
end

Instance Method Details

#before_create ⇒ Object

:section: Sequel Hooks



85
86
87
 
# File 'lib/webhookdb/backfill_job.rb', line 85

def before_create
  self[:opaque_id] ||= Webhookdb::Id.new_opaque_id("bfj")
end

#enqueue ⇒ Object 



67
68
69
 
# File 'lib/webhookdb/backfill_job.rb', line 67

def enqueue
  self.publish_deferred("run", self.id)
end

#enqueue_children ⇒ Object 



71
72
73
 
# File 'lib/webhookdb/backfill_job.rb', line 71

def enqueue_children
  self.child_jobs.each(&:enqueue)
end

#ensure_service_integration_lock ⇒ Object 



75
76
77
78
79
 
# File 'lib/webhookdb/backfill_job.rb', line 75

def ensure_service_integration_lock
  return Webhookdb::BackfillJob::ServiceIntegrationLock.find_or_create_or_find(
    service_integration_id: self.service_integration_id,
  )
end

#finished? ⇒ Boolean

Returns:

  • (Boolean) 


48
 
# File 'lib/webhookdb/backfill_job.rb', line 48

def finished? = !!self.finished_at

#fully_finished? ⇒ Boolean

Returns:

  • (Boolean) 


59
 
# File 'lib/webhookdb/backfill_job.rb', line 59

def fully_finished? = !!self.fully_finished_at

#fully_finished_at ⇒ Object 



50
51
52
53
54
55
56
57
 
# File 'lib/webhookdb/backfill_job.rb', line 50

def fully_finished_at
  parent_finished = self.finished_at
  return nil if parent_finished.nil?
  children_finished = self.child_jobs.map(&:fully_finished_at)
  return nil if children_finished.any?(&:nil?)
  children_finished << parent_finished
  return children_finished.max
end

#incremental? ⇒ Boolean

Returns:

  • (Boolean) 


45
 
# File 'lib/webhookdb/backfill_job.rb', line 45

def incremental? = self.incremental

#setup_recursive ⇒ Object

You should use ::create_recursive instead. This is mostly here for use in tests/fixtures.

Raises:



38
39
40
41
42
43
 
# File 'lib/webhookdb/backfill_job.rb', line 38

def setup_recursive
  raise Webhookdb::InvalidPrecondition, "already has children" if self.child_jobs.present?
  self.service_integration.dependents.map do |dep|
    self.class.create_recursive(service_integration: dep, parent_job: self, incremental:, criteria:)
  end
end

#started? ⇒ Boolean

Returns:

  • (Boolean) 


47
 
# File 'lib/webhookdb/backfill_job.rb', line 47

def started? = !!self.started_at

#status ⇒ Object 



61
62
63
64
65
 
# File 'lib/webhookdb/backfill_job.rb', line 61

def status
  return "enqueued" unless self.started?
  return "finished" if self.fully_finished?
  return "inprogress"
end