Module: ActiveModel::Datastore::ClassMethods

Defined in:
lib/active_model/datastore.rb

Overview

Methods defined here will be class methods when ‘include ActiveModel::Datastore’.

Instance Method Summary collapse

Instance Method Details

#all(options = {}) ⇒ Array<Model>, String

Queries entities from Cloud Datastore by named kind and using the provided options. When a limit option is provided queries up to the limit and returns results with a cursor.

This method may make several API calls until all query results are retrieved. The ‘run` method returns a QueryResults object, which is a special case Array with additional values. QueryResults are returned in batches, and the batch size is determined by the Datastore API. Batch size is not guaranteed. It will be affected by the size of the data being returned, and by other forces such as how distributed and/or consistent the data in Datastore is. Calling `all` on the QueryResults retrieves all results by repeatedly loading #next until #next? returns false. The `all` method returns an enumerator which from_entities iterates on.

Be sure to use as narrow a search criteria as possible. Please use with caution.

or if options was provided:

Parameters:

  • options (Hash) (defaults to: {}) —

    The options to construct the query with.

Options Hash (options):

  • :ancestor (Google::Cloud::Datastore::Key) —

    Filter for inherited results.

  • :cursor (String) —

    Sets the cursor to start the results at.

  • :limit (Integer) —

    Sets a limit to the number of results to be returned.

  • :order (String) —

    Sort the results by property name.

  • :desc_order (String) —

    Sort the results by descending property name.

  • :select (Array) —

    Retrieve only select properties from the matched entities.

  • :distinct_on (Array) —

    Group results by a list of properties.

  • :where (Array) —

    Adds a property filter of arrays in the format [name, operator, value].

Returns:

  • (Array<Model>, String) —

    An array of ActiveModel results

  • (Array<Model>, String) —

    An array of ActiveModel results and a cursor that can be used to query for additional results.



290
291
292
293
294
295
296
297
298
299
 
# File 'lib/active_model/datastore.rb', line 290

def all(options = {})
  next_cursor = nil
  query = build_query(options)
  query_results = retry_on_exception { CloudDatastore.dataset.run query }
  if options[:limit]
    next_cursor = query_results.cursor if query_results.size == options[:limit]
    return from_entities(query_results.all), next_cursor
  end
  from_entities(query_results.all)
end

#build_model(entity) ⇒ Object 



497
498
499
500
501
502
503
 
# File 'lib/active_model/datastore.rb', line 497

def build_model(entity)
  model_entity = new
  model_entity.id = entity.key.id unless entity.key.id.nil?
  model_entity.id = entity.key.name unless entity.key.name.nil?
  model_entity.parent_key_id = entity.key.parent.id if entity.key.parent.present?
  model_entity
end

#build_query(options = {}) ⇒ Query

Constructs a Google::Cloud::Datastore::Query.

Parameters:

  • options (Hash) (defaults to: {}) —

    The options to construct the query with.

Options Hash (options):

  • :ancestor (Google::Cloud::Datastore::Key) —

    Filter for inherited results.

  • :cursor (String) —

    Sets the cursor to start the results at.

  • :limit (Integer) —

    Sets a limit to the number of results to be returned.

  • :order (String) —

    Sort the results by property name.

  • :desc_order (String) —

    Sort the results by descending property name.

  • :select (Array) —

    Retrieve only select properties from the matched entities.

  • :distinct_on (Array) —

    Group results by a list of properties.

  • :where (Array) —

    Adds a property filter of arrays in the format [name, operator, value].

Returns:

  • (Query) —

    A datastore query.



398
399
400
401
 
# File 'lib/active_model/datastore.rb', line 398

def build_query(options = {})
  query = CloudDatastore.dataset.query name
  query_options(query, options)
end

#find(*ids, parent: nil) ⇒ Model, ...

Find entity by id - this can either be a specific id (1), a list of ids (1, 5, 6), or an array of ids ([5, 6, 10]). The parent key is optional.

Parameters:

  • ids (Integer) —

    One or more ids to retrieve.

  • parent (Google::Cloud::Datastore::Key) (defaults to: nil) —

    The parent key of the entity.

Returns:

  • (Model, nil) —

    An ActiveModel object or nil for a single id.

  • (Array<Model>) —

    An array of ActiveModel objects for more than one id.



311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
 
# File 'lib/active_model/datastore.rb', line 311

def find(*ids, parent: nil)
  expects_array = ids.first.is_a?(Array)
  ids = ids.flatten.compact.uniq.map(&:to_i)

  case ids.size
  when 0
    raise EntityError, "Couldn't find #{name} without an ID"
  when 1
    entity = find_entity(ids.first, parent)
    model_entity = from_entity(entity)
    expects_array ? [model_entity].compact : model_entity
  else
    lookup_results = find_all_entities(ids, parent)
    from_entities(lookup_results.all)
  end
end

#find_all_entities(ids_or_names, parent) ⇒ Object

Finds entities by keys using the provided array items. Results provided by the dataset ‘find_all` is a Dataset::LookupResults object.

Parameters:

  • ids_or_names (Array<Integer>, Array<String>) —

    An array of ids or names.



491
492
493
494
495
 
# File 'lib/active_model/datastore.rb', line 491

def find_all_entities(ids_or_names, parent)
  keys = ids_or_names.map { |id| CloudDatastore.dataset.key name, id }
  keys.map { |key| key.parent = parent } if parent.present?
  retry_on_exception { CloudDatastore.dataset.find_all keys }
end

#find_by(args) ⇒ Model?

Finds the first entity matching the specified condition.

Examples:

User.find_by(name: 'Joe')
User.find_by(name: 'Bryce', ancestor: parent_key)

Parameters:

  • args (Hash) —

    In which the key is the property and the value is the value to look for.

Options Hash (args):

  • :ancestor (Google::Cloud::Datastore::Key) —

    filter for inherited results

Returns:

  • (Model, nil) —

    An ActiveModel object or nil.



340
341
342
343
344
345
346
347
 
# File 'lib/active_model/datastore.rb', line 340

def find_by(args)
  query = CloudDatastore.dataset.query name
  query.ancestor(args[:ancestor]) if args[:ancestor]
  query.limit(1)
  query.where(args.keys[0].to_s, '=', args.values[0])
  query_results = retry_on_exception { CloudDatastore.dataset.run query }
  from_entity(query_results.first)
end

#find_entities(*ids_or_names, parent: nil) ⇒ Array<Entity>

Retrieves the entities for the provided ids by key and by an optional parent. The find_all method returns LookupResults, which is a special case Array with additional values. LookupResults are returned in batches, and the batch size is determined by the Datastore API. Batch size is not guaranteed. It will be affected by the size of the data being returned, and by other forces such as how distributed and/or consistent the data in Datastore is. Calling ‘all` on the LookupResults retrieves all results by repeatedly loading #next until #next? returns false. The `all` method returns an enumerator unless passed a block. We iterate on the enumerator to return the model entity objects.

Parameters:

  • ids_or_names (Integer, String) —

    One or more ids to retrieve.

  • parent (Google::Cloud::Datastore::Key) (defaults to: nil) —

    The parent Key of the entity.

Returns:

  • (Array<Entity>) —

    an array of Google::Cloud::Datastore::Entity objects.



251
252
253
254
255
 
# File 'lib/active_model/datastore.rb', line 251

def find_entities(*ids_or_names, parent: nil)
  ids_or_names = ids_or_names.flatten.compact.uniq
  lookup_results = find_all_entities(ids_or_names, parent)
  lookup_results.all.collect { |x| x }
end

#find_entity(id_or_name, parent = nil) ⇒ Entity?

Retrieves an entity by id or name and by an optional parent.

Parameters:

  • id_or_name (Integer or String) —

    The id or name value of the entity Key.

  • parent (Google::Cloud::Datastore::Key) (defaults to: nil) —

    The parent Key of the entity.

Returns:

  • (Entity, nil) —

    a Google::Cloud::Datastore::Entity object or nil.



229
230
231
232
233
 
# File 'lib/active_model/datastore.rb', line 229

def find_entity(id_or_name, parent = nil)
  key = CloudDatastore.dataset.key name, id_or_name
  key.parent = parent if parent.present?
  retry_on_exception { CloudDatastore.dataset.find key }
end

#from_entities(entities) ⇒ Object

Translates an Enumerator of Datastore::Entity objects to ActiveModel::Model objects.

Results provided by the dataset ‘find_all` or `run query` will be a Dataset::LookupResults or Dataset::QueryResults object. Invoking `all` on those objects returns an enumerator.

Parameters:

  • entities (Enumerator) —

    An enumerator representing the datastore entities.

Raises:

  • (ArgumentError) 


357
358
359
360
361
 
# File 'lib/active_model/datastore.rb', line 357

def from_entities(entities)
  raise ArgumentError, 'Entities param must be an Enumerator' unless entities.is_a? Enumerator

  entities.map { |entity| from_entity(entity) }
end

#from_entity(entity) ⇒ Model

Translates between Datastore::Entity objects and ActiveModel::Model objects.

Parameters:

  • entity (Entity) —

    Entity from Cloud Datastore.

Returns:

  • (Model) —

    The translated ActiveModel object.



369
370
371
372
373
374
375
376
377
378
379
 
# File 'lib/active_model/datastore.rb', line 369

def from_entity(entity)
  return if entity.nil?

  model_entity = build_model(entity)
  model_entity.entity_property_values = entity.properties.to_h
  entity.properties.to_h.each do |name, value|
    model_entity.send "#{name}=", value if model_entity.respond_to? "#{name}="
  end
  model_entity.reload!
  model_entity
end

#log_google_cloud_error ⇒ Object 



437
438
439
440
441
442
 
# File 'lib/active_model/datastore.rb', line 437

def log_google_cloud_error
  yield
rescue Google::Cloud::Error => e
  puts "\e[33m[#{e.message.inspect}]\e[0m"
  raise e
end

#parent_key(parent_id) ⇒ Object

A default parent key for specifying an ancestor path and creating an entity group.



217
218
219
 
# File 'lib/active_model/datastore.rb', line 217

def parent_key(parent_id)
  CloudDatastore.dataset.key('Parent' + name, parent_id.to_i)
end

#query_options(query, options) ⇒ Object

**************** private ****************



446
447
448
449
450
451
452
453
454
 
# File 'lib/active_model/datastore.rb', line 446

def query_options(query, options)
  query.ancestor(options[:ancestor]) if options[:ancestor]
  query.cursor(options[:cursor]) if options[:cursor]
  query.limit(options[:limit]) if options[:limit]
  query_sort(query, options)
  query.select(*options[:select]) if options[:select]
  query.distinct_on(*options[:distinct_on]) if options[:distinct_on]
  query_property_filter(query, options)
end

#query_property_filter(query, options) ⇒ Object

Adds property filters to the query if included in the options. Accepts individual or nested Arrays:

[['superseded', '=', false], ['email', '=', 'something']]



470
471
472
473
474
475
476
477
478
479
480
481
482
 
# File 'lib/active_model/datastore.rb', line 470

def query_property_filter(query, options)
  if options[:where]
    opts = options[:where]
    if opts[0].is_a?(Array)
      opts.each do |opt|
        query.where(opt[0], opt[1], opt[2]) unless opt.nil?
      end
    else
      query.where(opts[0], opts[1], opts[2])
    end
  end
  query
end

#query_sort(query, options) ⇒ Object

Adds sorting to the results by a property name if included in the options.



459
460
461
462
463
 
# File 'lib/active_model/datastore.rb', line 459

def query_sort(query, options)
  query.order(options[:order]) if options[:order]
  query.order(options[:desc_order], :desc) if options[:desc_order]
  query
end

#retry_on_exception(max_retry_count = 5) ⇒ Object 



420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
 
# File 'lib/active_model/datastore.rb', line 420

def retry_on_exception(max_retry_count = 5)
  retries = 0
  sleep_time = 0.25
  begin
    yield
  rescue Google::Cloud::Error => e
    raise e if retries >= max_retry_count

    puts "\e[33mRescued exception #{e.message.inspect}, retrying in #{sleep_time}\e[0m"
    # 0.25, 0.5, 1, 2, and 4 second between retries.
    sleep sleep_time
    retries += 1
    sleep_time *= 2
    retry
  end
end

#retry_on_exception?(max_retry_count = 5) ⇒ Boolean

Returns:

  • (Boolean) 


403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
 
# File 'lib/active_model/datastore.rb', line 403

def retry_on_exception?(max_retry_count = 5)
  retries = 0
  sleep_time = 0.25
  begin
    yield
  rescue Google::Cloud::Error => e
    return false if retries >= max_retry_count

    puts "\e[33mRescued exception #{e.message.inspect}, retrying in #{sleep_time}\e[0m"
    # 0.25, 0.5, 1, 2, and 4 second between retries.
    sleep sleep_time
    retries += 1
    sleep_time *= 2
    retry
  end
end