Class: Cloudtasker::Config

Inherits:
Object
  • Object
show all
Defined in:
lib/cloudtasker/config.rb

Overview

Holds cloudtasker configuration. See Cloudtasker#configure

Constant Summary collapse

MAX_TASK_SIZE =

Max Cloud Task size in bytes

100 * 1024
RETRY_HEADER =

Retry header in Cloud Task responses

TODO: use ‘X-CloudTasks-TaskExecutionCount’ instead of ‘X-CloudTasks-TaskRetryCount’

'X-CloudTasks-TaskExecutionCount' is currently bugged and remains at 0 even on retries.

See bug: issuetracker.google.com/issues/154532072

Definitions:

X-CloudTasks-TaskRetryCount: total number of retries (including 504 "instance unreachable")
X-CloudTasks-TaskExecutionCount: number of non-503 retries (= actual number of job failures)

'X-CloudTasks-TaskRetryCount'
TASK_ID_HEADER =

Cloud Task ID header

'X-CloudTasks-TaskName'
ENCODING_HEADER =

Content-Transfer-Encoding header in Cloud Task responses

'Content-Transfer-Encoding'
CONTENT_TYPE_HEADER =

Content Type

'Content-Type'
AUTHORIZATION_HEADER =

Authorization header

'Authorization'
DEFAULT_LOCATION_ID =

Default values

'us-east1'
DEFAULT_PROCESSOR_PATH = 
'/cloudtasker/run'
DEFAULT_JOB_QUEUE =

Default queue values

'default'
DEFAULT_QUEUE_CONCURRENCY = 
10
DEFAULT_QUEUE_RETRIES =

unlimited

-1 # unlimited
DEFAULT_DISPATCH_DEADLINE =

Job timeout configuration for Cloud Tasks

10 * 60
MIN_DISPATCH_DEADLINE =

10 minutes

15
MAX_DISPATCH_DEADLINE =

seconds

30 * 60
DEFAULT_ON_ERROR =

Default on_error Proc

->(error, worker) {}
DEFAULT_MAX_RETRY_ATTEMPTS =

The number of times jobs will be attempted before declaring them dead.

With the default retry configuration (maxDoublings = 16 and minBackoff = 0.100s) it means that jobs will be declared dead after 20h of consecutive failing.

Note that this configuration parameter is internal to Cloudtasker and does not affect the Cloud Task queue configuration. The number of retries configured on the Cloud Task queue should be higher than the number below to also cover failures due to the instance being unreachable.

25
PROCESSOR_HOST_MISSING = 
<<~DOC
  Missing host for processing.
  Please specify a processor hostname in form of `https://some-public-dns.example.com`'
DOC
PROJECT_ID_MISSING_ERROR = 
<<~DOC
  Missing GCP project ID.
  Please specify a project ID in the cloudtasker configurator.
DOC
SECRET_MISSING_ERROR = 
<<~DOC
  Missing cloudtasker secret.
  Please specify a secret in the cloudtasker initializer or add Rails secret_key_base in your credentials
DOC

Instance Attribute Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#dispatch_deadline ⇒ Integer

Return the Dispatch deadline duration. Cloud Tasks will timeout the job after this duration is elapsed.

Returns:

  • (Integer) —

    The value in seconds.



211
212
213
 
# File 'lib/cloudtasker/config.rb', line 211

def dispatch_deadline
  @dispatch_deadline || DEFAULT_DISPATCH_DEADLINE
end

#gcp_location_id ⇒ String

Return the GCP location ID. Default to ‘us-east1’

Returns:

  • (String) —

    The location ID where tasks will be processed.



201
202
203
 
# File 'lib/cloudtasker/config.rb', line 201

def gcp_location_id
  @gcp_location_id || DEFAULT_LOCATION_ID
end

#gcp_project_id ⇒ String

Return the GCP project ID.

Returns:

  • (String) —

    The ID of the project for which tasks will be processed.



192
193
194
 
# File 'lib/cloudtasker/config.rb', line 192

def gcp_project_id
  @gcp_project_id || raise(StandardError, PROJECT_ID_MISSING_ERROR)
end

#gcp_queue_prefix ⇒ Object

Returns the value of attribute gcp_queue_prefix.



8
9
10
 
# File 'lib/cloudtasker/config.rb', line 8

def gcp_queue_prefix
  @gcp_queue_prefix
end

#logger ⇒ Logger, any

Return the Cloudtasker logger.

Returns:

  • (Logger, any) —

    The cloudtasker logger.



133
134
135
 
# File 'lib/cloudtasker/config.rb', line 133

def logger
  @logger ||= defined?(Rails) ? Rails.logger : ::Logger.new(STDOUT)
end

#max_retries ⇒ Integer

The number of times jobs will be retried. This number of retries does not include failures due to the application being unreachable.

Returns:

  • (Integer) —

    The number of retries



104
105
106
 
# File 'lib/cloudtasker/config.rb', line 104

def max_retries
  @max_retries ||= DEFAULT_MAX_RETRY_ATTEMPTS
end

#mode ⇒ <Type>

The operating mode.

- :production => process tasks via GCP Cloud Task.
- :development => process tasks locally via Redis.

Returns:

  • (<Type>) —

    <description>



115
116
117
 
# File 'lib/cloudtasker/config.rb', line 115

def mode
  @mode ||= environment == 'development' ? :development : :production
end

#on_dead ⇒ Proc

Return a Proc invoked whenever a worker DeadWorkerError is raised. See Cloudtasker::WorkerHandler.with_worker_handling

Returns:

  • (Proc) —

    A Proc handler



243
244
245
 
# File 'lib/cloudtasker/config.rb', line 243

def on_dead
  @on_dead || DEFAULT_ON_ERROR
end

#on_error ⇒ Proc

Return a Proc invoked whenever a worker runtime error is raised. See Cloudtasker::WorkerHandler.with_worker_handling

Returns:

  • (Proc) —

    A Proc handler



233
234
235
 
# File 'lib/cloudtasker/config.rb', line 233

def on_error
  @on_error || DEFAULT_ON_ERROR
end

#processor_path ⇒ String

The path on the host when worker payloads will be sent. Default to ‘/cloudtasker/run`

Returns:

  • (String) —

    The processor path



183
184
185
 
# File 'lib/cloudtasker/config.rb', line 183

def processor_path
  @processor_path || DEFAULT_PROCESSOR_PATH
end

#redis ⇒ Object

Returns the value of attribute redis.



8
9
10
 
# File 'lib/cloudtasker/config.rb', line 8

def redis
  @redis
end

#secret ⇒ String

Return the secret to use to sign the verification tokens attached to tasks.

Returns:

  • (String) —

    The cloudtasker secret



221
222
223
224
225
 
# File 'lib/cloudtasker/config.rb', line 221

def secret
  @secret ||= (
    defined?(Rails) && Rails.application.credentials&.dig(:secret_key_base)
  ) || raise(StandardError, SECRET_MISSING_ERROR)
end

#store_payloads_in_redis ⇒ Object

Returns the value of attribute store_payloads_in_redis.



8
9
10
 
# File 'lib/cloudtasker/config.rb', line 8

def store_payloads_in_redis
  @store_payloads_in_redis
end

Instance Method Details

#client_middleware {|@client_middleware| ... } ⇒ Cloudtasker::Middleware::Chain

Return the chain of client middlewares.

Yields:

Returns:



252
253
254
255
256
 
# File 'lib/cloudtasker/config.rb', line 252

def client_middleware
  @client_middleware ||= Middleware::Chain.new
  yield @client_middleware if block_given?
  @client_middleware
end

#environment ⇒ String

Return the current environment.

Returns:

  • (String) —

    The environment name.



124
125
126
 
# File 'lib/cloudtasker/config.rb', line 124

def environment
  ENV['CLOUDTASKER_ENV'] || ENV['RAILS_ENV'] || ENV['RACK_ENV'] || 'development'
end

#processor_host ⇒ String

The hostname of the application processing the workers. The hostname must be reachable from Cloud Task.

Returns:

  • (String) —

    The processor host.



172
173
174
 
# File 'lib/cloudtasker/config.rb', line 172

def processor_host
  @processor_host || raise(StandardError, PROCESSOR_HOST_MISSING)
end

#processor_host=(val) ⇒ Object

Set the processor host. In the context of Rails the host will also be added to the list of authorized Rails hosts.

Parameters:

  • val (String) —

    The processor host to set.



153
154
155
156
157
158
159
160
161
162
163
164
 
# File 'lib/cloudtasker/config.rb', line 153

def processor_host=(val)
  @processor_host = val

  # Check if Rails supports host filtering
  return unless val &&
                defined?(Rails) &&
                Rails.application.config.respond_to?(:hosts) &&
                Rails.application.config.hosts&.any?

  # Add processor host to the list of authorized hosts
  Rails.application.config.hosts << val.gsub(%r{https?://}, '')
end

#processor_url ⇒ String

Return the full URL of the processor. Worker payloads will be sent to this URL.

Returns:

  • (String) —

    The processor URL.



143
144
145
 
# File 'lib/cloudtasker/config.rb', line 143

def processor_url
  File.join(processor_host, processor_path)
end

#redis_payload_storage_threshold ⇒ Integer?

Return the threshold above which job arguments must be stored in Redis instead of being sent to the backend as part of the job payload.

Return nil if redis payload storage is disabled.

Returns:

  • (Integer, nil) —

    The threshold above which payloads will be stored in Redis.



91
92
93
94
95
 
# File 'lib/cloudtasker/config.rb', line 91

def redis_payload_storage_threshold
  return nil unless store_payloads_in_redis

  store_payloads_in_redis.respond_to?(:to_i) ? store_payloads_in_redis.to_i : 0
end

#server_middleware {|@server_middleware| ... } ⇒ Cloudtasker::Middleware::Chain

Return the chain of server middlewares.

Yields:

Returns:



263
264
265
266
267
 
# File 'lib/cloudtasker/config.rb', line 263

def server_middleware
  @server_middleware ||= Middleware::Chain.new
  yield @server_middleware if block_given?
  @server_middleware
end