Class: Google::Cloud::Bigquery::External::AvroSource

Inherits:
DataSource
  • Object
show all
Defined in:
lib/google/cloud/bigquery/external/avro_source.rb

Overview

AvroSource

AvroSource is a subclass of DataSource and represents a Avro external data source that can be queried from directly, even though the data is not stored in BigQuery. Instead of loading or streaming the data, this object references the external data source.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

avro_url = "gs://bucket/path/to/*.avro"
avro_table = bigquery.external avro_url do |avro|
  avro.use_avro_logical_types = 1
end

data = bigquery.query "SELECT * FROM my_ext_table",
                      external: { my_ext_table: avro_table }

# Iterate over the first page of results
data.each do |row|
  puts row[:name]
end
# Retrieve the next page of results
data = data.next if data.next?

Instance Method Summary collapse

Methods inherited from DataSource

#autodetect, #autodetect=, #avro?, #backup?, #bigtable?, #compression, #compression=, #csv?, #format, #hive_partitioning?, #hive_partitioning_mode, #hive_partitioning_mode=, #hive_partitioning_require_partition_filter=, #hive_partitioning_require_partition_filter?, #hive_partitioning_source_uri_prefix, #hive_partitioning_source_uri_prefix=, #ignore_unknown, #ignore_unknown=, #json?, #max_bad_records, #max_bad_records=, #orc?, #parquet?, #sheets?, #urls

Instance Method Details

#use_avro_logical_types ⇒ Boolean

Indicates whether to interpret logical types as the corresponding BigQuery data type (for example, TIMESTAMP), instead of using the raw type (for example, INTEGER).

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

avro_url = "gs://bucket/path/to/*.avro"
avro_table = bigquery.external avro_url do |avro|
  avro.use_avro_logical_types = true
end

avro_table.use_avro_logical_types #=> true

Returns:

  • (Boolean) 


77
78
79
 
# File 'lib/google/cloud/bigquery/external/avro_source.rb', line 77

def use_avro_logical_types
  @gapi.avro_options.use_avro_logical_types
end

#use_avro_logical_types=(new_use_avro_logical_types) ⇒ Object

Sets whether to interpret logical types as the corresponding BigQuery data type (for example, TIMESTAMP), instead of using the raw type (for example, INTEGER).

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

avro_url = "gs://bucket/path/to/*.avro"
avro_table = bigquery.external avro_url do |avro|
  avro.use_avro_logical_types = true
end

avro_table.use_avro_logical_types #=> true

Parameters:

  • new_use_avro_logical_types (Boolean) —

    The new use_avro_logical_types value.



99
100
101
102
 
# File 'lib/google/cloud/bigquery/external/avro_source.rb', line 99

def use_avro_logical_types= new_use_avro_logical_types
  frozen_check!
  @gapi.avro_options.use_avro_logical_types = new_use_avro_logical_types
end