Module: CounterAttribute

Extended by:
ActiveSupport::Concern
Includes:
AfterCommitQueue, Gitlab::ExclusiveLeaseHelpers, Gitlab::Utils::StrongMemoize
Included in:
ProjectDailyStatistic, ProjectStatistics, Projects::DataTransfer
Defined in:
app/models/concerns/counter_attribute.rb

Overview

Add capabilities to increment a numeric model attribute efficiently by using Redis and flushing the increments asynchronously to the database after a period of time (10 minutes).

The ActiveRecord model is required to either have a project_id or a group_id foreign key.

When an attribute is incremented by a value, the increment is added to a Redis key. Then, FlushCounterIncrementsWorker will execute commit_increment! which removes increments from Redis for a given model attribute and updates the values in the database.

@example:

class ProjectStatistics
  include CounterAttribute

  counter_attribute :commit_count
  counter_attribute :storage_size
end

It’s possible to define a conditional counter attribute. You need to pass a proc that must accept a single argument, the object instance on which this concern is included.

@example:

class ProjectStatistics
  include CounterAttribute

  counter_attribute :conditional_one, if: -> { |object| object.use_counter_attribute? }
end

The counter_attribute by default will return last persisted value. It’s possible to always return accurate (real) value instead by using ‘returns_current: true`. While doing this the counter_attribute will overwrite attribute accessor to fetch the buffered information added to the last persisted value. This will incur cost a Redis call per attribute fetched.

@example:

class ProjectStatistics
  include CounterAttribute

  counter_attribute :commit_count, returns_current: true
end

in that case

model.commit_count => persisted value + buffered amount to be added

To increment the counter we can use the method:

increment_amount(:commit_count, 3)

Bumping counters relies on the Rails .update_counters class method. As such, we can pass a :touch option that can accept true, timestamp columns are updated, or attribute names, which will be updated along with updated_at/on

@example:

class ProjectStatistics

include CounterAttribute

counter_attribute :my_counter, touch: :my_counter_updated_at

end

This method would determine whether it would increment the counter using Redis, or fallback to legacy increment on ActiveRecord counters.

It is possible to register callbacks to be executed after increments have been flushed to the database. Callbacks are not executed if there are no increments to flush.

counter_attribute_after_commit do |statistic|
  Namespaces::ScheduleAggregationWorker.perform_async(statistic.namespace_id)
end

Constant Summary

Constants included from Gitlab::ExclusiveLeaseHelpers

Gitlab::ExclusiveLeaseHelpers::FailedToObtainLockError

Instance Method Summary collapse

Methods included from Gitlab::ExclusiveLeaseHelpers

#in_lock

Methods included from AfterCommitQueue

#run_after_commit, #run_after_commit_or_now

Instance Method Details

#bulk_increment_counter(attribute, increments) ⇒ Object 



152
153
154
155
156
157
158
 
# File 'app/models/concerns/counter_attribute.rb', line 152

def bulk_increment_counter(attribute, increments)
  run_after_commit_or_now do
    new_value = counter(attribute).bulk_increment(increments)

    log_bulk_increment_counter(attribute, increments, new_value)
  end
end

#counter(attribute) ⇒ Object 



125
126
127
128
129
130
131
 
# File 'app/models/concerns/counter_attribute.rb', line 125

def counter(attribute)
  strong_memoize_with(:counter, attribute) do
    # This needs #to_sym because attribute could come from a Sidekiq param,
    # which would be a string.
    build_counter_for(attribute.to_sym)
  end
end

#counter_attribute_enabled?(attribute) ⇒ Boolean

Returns:

  • (Boolean) 


117
118
119
120
121
122
123
 
# File 'app/models/concerns/counter_attribute.rb', line 117

def counter_attribute_enabled?(attribute)
  counter_attribute = self.class.counter_attributes[attribute]
  return false unless counter_attribute
  return true unless counter_attribute[:if_proc]

  counter_attribute[:if_proc].call(self)
end

#counters_key_prefix ⇒ Object 



194
195
196
 
# File 'app/models/concerns/counter_attribute.rb', line 194

def counters_key_prefix
  with_parent { |type, id| "#{type}:{#{id}}" }
end

#current_counter(attribute) ⇒ Object 



138
139
140
 
# File 'app/models/concerns/counter_attribute.rb', line 138

def current_counter(attribute)
  read_attribute(attribute) + counter(attribute).get
end

#execute_after_commit_callbacks ⇒ Object 



188
189
190
191
192
 
# File 'app/models/concerns/counter_attribute.rb', line 188

def execute_after_commit_callbacks
  self.class.after_commit_callbacks.each do |callback|
    callback.call(self.reset)
  end
end

#finalize_refresh(attribute) ⇒ Object

Raises:

  • (ArgumentError) 


182
183
184
185
186
 
# File 'app/models/concerns/counter_attribute.rb', line 182

def finalize_refresh(attribute)
  raise ArgumentError, %(attribute "#{attribute}" cannot be refreshed) unless counter_attribute_enabled?(attribute)

  counter(attribute).finalize_refresh
end

#increment_amount(attribute, amount) ⇒ Object 



133
134
135
136
 
# File 'app/models/concerns/counter_attribute.rb', line 133

def increment_amount(attribute, amount)
  counter = Gitlab::Counters::Increment.new(amount: amount)
  increment_counter(attribute, counter)
end

#increment_counter(attribute, increment) ⇒ Object 



142
143
144
145
146
147
148
149
150
 
# File 'app/models/concerns/counter_attribute.rb', line 142

def increment_counter(attribute, increment)
  return if increment.amount == 0

  run_after_commit_or_now do
    new_value = counter(attribute).increment(increment)

    log_increment_counter(attribute, increment, new_value)
  end
end

#initiate_refresh!(attribute) ⇒ Object

Raises:

  • (ArgumentError) 


175
176
177
178
179
180
 
# File 'app/models/concerns/counter_attribute.rb', line 175

def initiate_refresh!(attribute)
  raise ArgumentError, %(attribute "#{attribute}" cannot be refreshed) unless counter_attribute_enabled?(attribute)

  counter(attribute).initiate_refresh!
  log_clear_counter(attribute)
end

#update_counters(increments) ⇒ Object 



160
161
162
163
164
165
166
167
 
# File 'app/models/concerns/counter_attribute.rb', line 160

def update_counters(increments)
  touch = increments.each_key.flat_map do |attribute|
    self.class.counter_attributes.dig(attribute, :touch)
  end

  increments[:touch] = touch if touch.any?
  self.class.update_counters(id, increments)
end

#update_counters_with_lease(increments) ⇒ Object 



169
170
171
172
173
 
# File 'app/models/concerns/counter_attribute.rb', line 169

def update_counters_with_lease(increments)
  detect_race_on_record(log_fields: { caller: __method__, attributes: increments.keys }) do
    update_counters(increments)
  end
end