Class: Google::Cloud::Bigquery::External::JsonSource

Inherits:
DataSource
  • Object
show all
Defined in:
lib/google/cloud/bigquery/external.rb

Overview

JsonSource

JsonSource is a subclass of DataSource and represents a JSON external data source that can be queried from directly, such as Google Cloud Storage or Google Drive, even though the data is not stored in BigQuery. Instead of loading or streaming the data, this object references the external data source.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

json_url = "gs://bucket/path/to/data.json"
json_table = bigquery.external json_url do |json|
  json.schema do |schema|
    schema.string "name", mode: :required
    schema.string "email", mode: :required
    schema.integer "age", mode: :required
    schema.boolean "active", mode: :required
  end
end

data = bigquery.query "SELECT * FROM my_ext_table",
                      external: { my_ext_table: json_table }

data.each do |row|
  puts row[:name]
end

Instance Method Summary collapse

Methods inherited from DataSource

#autodetect, #autodetect=, #avro?, #backup?, #bigtable?, #compression, #compression=, #csv?, #format, #ignore_unknown, #ignore_unknown=, #json?, #max_bad_records, #max_bad_records=, #sheets?, #urls

Instance Method Details

#fields ⇒ Object

The fields of the schema.



1107
1108
1109
 
# File 'lib/google/cloud/bigquery/external.rb', line 1107

def fields
  schema.fields
end

#headers ⇒ Object

The names of the columns in the schema.



1114
1115
1116
 
# File 'lib/google/cloud/bigquery/external.rb', line 1114

def headers
  schema.headers
end

#schema(replace: false) {|schema| ... } ⇒ Google::Cloud::Bigquery::Schema

The schema for the data.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

json_url = "gs://bucket/path/to/data.json"
json_table = bigquery.external json_url do |json|
  json.schema do |schema|
    schema.string "name", mode: :required
    schema.string "email", mode: :required
    schema.integer "age", mode: :required
    schema.boolean "active", mode: :required
  end
end

Parameters:

  • replace (Boolean) (defaults to: false) —

    Whether to replace the existing schema with the new schema. If true, the fields will replace the existing schema. If false, the fields will be added to the existing schema. The default value is false.

Yields:

  • (schema) —

    a block for setting the schema

Yield Parameters:

  • schema (Schema) —

    the object accepting the schema

Returns:



1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
 
# File 'lib/google/cloud/bigquery/external.rb', line 1067

def schema replace: false
  @schema ||= Schema.from_gapi @gapi.schema
  if replace
    frozen_check!
    @schema = Schema.from_gapi
  end
  @schema.freeze if frozen?
  yield @schema if block_given?
  @schema
end

#schema=(new_schema) ⇒ Object

Set the schema for the data.

Examples:

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new

json_shema = bigquery.schema do |schema|
  schema.string "name", mode: :required
  schema.string "email", mode: :required
  schema.integer "age", mode: :required
  schema.boolean "active", mode: :required
end

json_url = "gs://bucket/path/to/data.json"
json_table = bigquery.external json_url
json_table.schema = json_shema

Parameters:

  • new_schema (Schema) —

    The schema object.



1099
1100
1101
1102
 
# File 'lib/google/cloud/bigquery/external.rb', line 1099

def schema= new_schema
  frozen_check!
  @schema = new_schema
end