Class: Gcloud::Bigquery::Table
- Inherits:
-
Object
- Object
- Gcloud::Bigquery::Table
- Defined in:
- lib/gcloud/bigquery/table.rb,
lib/gcloud/bigquery/table/list.rb
Overview
Table
A named resource representing a BigQuery table that holds zero or more records. Every table is defined by a schema that may contain nested and repeated fields. (For more information about nested and repeated fields, see Preparing Data for BigQuery.)
require "gcloud"
gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table"
schema = {
"fields" => [
{
"name" => "first_name",
"type" => "STRING",
"mode" => "REQUIRED"
},
{
"name" => "cities_lived",
"type" => "RECORD",
"mode" => "REPEATED",
"fields" => [
{
"name" => "place",
"type" => "STRING",
"mode" => "REQUIRED"
},
{
"name" => "number_of_years",
"type" => "INTEGER",
"mode" => "REQUIRED"
}
]
}
]
}
table.schema = schema
row = {
"first_name" => "Alice",
"cities_lived" => [
{
"place": "Seattle",
"number_of_years": 5
},
{
"place": "Stockholm",
"number_of_years": 6
}
]
}
table.insert row
Defined Under Namespace
Classes: List
Instance Attribute Summary collapse
-
#connection ⇒ Object
The Connection object.
-
#gapi ⇒ Object
The Google API Client object.
Class Method Summary collapse
-
.from_gapi(gapi, conn) ⇒ Object
New Table from a Google API Client object.
Instance Method Summary collapse
-
#bytes_count ⇒ Object
The number of bytes in the table.
-
#copy(destination_table, options = {}) ⇒ Object
Copies the data from the table to another table.
-
#created_at ⇒ Object
The time when this table was created.
-
#data(options = {}) ⇒ Object
Retrieves data from the table.
-
#dataset_id ⇒ Object
The ID of the
Dataset
containing this table. -
#delete ⇒ Object
Permanently deletes the table.
-
#description ⇒ Object
The description of the table.
-
#description=(new_description) ⇒ Object
Updates the description of the table.
-
#etag ⇒ Object
A string hash of the dataset.
-
#expires_at ⇒ Object
The time when this table expires.
-
#extract(extract_url, options = {}) ⇒ Object
Extract the data from the table to a Google Cloud Storage file.
-
#fields ⇒ Object
The fields of the table.
-
#headers ⇒ Object
The names of the columns in the table.
-
#initialize ⇒ Table
constructor
Create an empty Table object.
-
#insert(rows, options = {}) ⇒ Object
Inserts data into the table for near-immediate querying, without the need to complete a #load operation before the data can appear in query results.
-
#link(source_url, options = {}) ⇒ Object
Links the table to a source table identified by a URI.
-
#load(file, options = {}) ⇒ Object
Loads data into the table.
-
#location ⇒ Object
The geographic location where the table should reside.
-
#modified_at ⇒ Object
The date when this table was last modified.
-
#name ⇒ Object
The name of the table.
-
#name=(new_name) ⇒ Object
Updates the name of the table.
-
#project_id ⇒ Object
The ID of the
Project
containing this table. -
#rows_count ⇒ Object
The number of rows in the table.
-
#schema ⇒ Object
The schema of the table.
-
#schema=(new_schema) ⇒ Object
Updates the schema of the table.
-
#table? ⇒ Boolean
Checks if the table’s type is “TABLE”.
-
#table_id ⇒ Object
A unique ID for this table.
-
#url ⇒ Object
A URL that can be used to access the dataset using the REST API.
-
#view? ⇒ Boolean
Checks if the table’s type is “VIEW”.
Constructor Details
#initialize ⇒ Table
Create an empty Table object.
95 96 97 98 |
# File 'lib/gcloud/bigquery/table.rb', line 95 def initialize #:nodoc: @connection = nil @gapi = {} end |
Instance Attribute Details
#connection ⇒ Object
The Connection object.
87 88 89 |
# File 'lib/gcloud/bigquery/table.rb', line 87 def connection @connection end |
#gapi ⇒ Object
The Google API Client object.
91 92 93 |
# File 'lib/gcloud/bigquery/table.rb', line 91 def gapi @gapi end |
Class Method Details
.from_gapi(gapi, conn) ⇒ Object
New Table from a Google API Client object.
705 706 707 708 709 710 711 |
# File 'lib/gcloud/bigquery/table.rb', line 705 def self.from_gapi gapi, conn #:nodoc: klass = class_for gapi klass.new.tap do |f| f.gapi = gapi f.connection = conn end end |
Instance Method Details
#bytes_count ⇒ Object
The number of bytes in the table.
:category: Data
191 192 193 194 |
# File 'lib/gcloud/bigquery/table.rb', line 191 def bytes_count ensure_full_data! @gapi["numBytes"] end |
#copy(destination_table, options = {}) ⇒ Object
Copies the data from the table to another table.
Parameters
destination_table
-
The destination for the copied data. (
Table
) options
-
An optional Hash for controlling additional behavior. (
Hash
) options[:create]
-
Specifies whether the job is allowed to create new tables. (
String
)The following values are supported:
-
needed
- Create the table if it does not exist. -
never
- The table must already exist. A ‘notFound’ error is raised if the table does not exist.
-
options[:write]
-
Specifies how to handle data already present in the destination table. The default value is
empty
. (String
)The following values are supported:
-
truncate
- BigQuery overwrites the table data. -
append
- BigQuery appends the data to the table. -
empty
- An error will be returned if the destination table already contains data.
-
Returns
Gcloud::Bigquery::CopyJob
Example
require "gcloud"
gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
destination_table = dataset.table "my_destination_table"
copy_job = table.copy destination_table
:category: Data
426 427 428 429 430 431 432 433 434 |
# File 'lib/gcloud/bigquery/table.rb', line 426 def copy destination_table, = {} ensure_connection! resp = connection.copy_table gapi, destination_table.gapi, if resp.success? Job.from_gapi resp.data, connection else fail ApiError.from_response(resp) end end |
#created_at ⇒ Object
The time when this table was created.
:category: Attributes
211 212 213 214 |
# File 'lib/gcloud/bigquery/table.rb', line 211 def created_at ensure_full_data! Time.at(@gapi["creationTime"] / 1000.0) end |
#data(options = {}) ⇒ Object
Retrieves data from the table.
Parameters
options
-
An optional Hash for controlling additional behavior. (
Hash
) options[:token]
-
Page token, returned by a previous call, identifying the result set. (
String
) options[:max]
-
Maximum number of results to return. (
Integer
) options[:start]
-
Zero-based index of the starting row to read. (
Integer
)
Returns
Gcloud::Bigquery::Data
Example
require "gcloud"
gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
data = table.data
data.each do |row|
puts row["first_name"]
end
more_data = table.data token: data.token
:category: Data
372 373 374 375 376 377 378 379 380 |
# File 'lib/gcloud/bigquery/table.rb', line 372 def data = {} ensure_connection! resp = connection.list_tabledata dataset_id, table_id, if resp.success? Data.from_response resp, self else fail ApiError.from_response(resp) end end |
#dataset_id ⇒ Object
The ID of the Dataset
containing this table.
:category: Attributes
116 117 118 |
# File 'lib/gcloud/bigquery/table.rb', line 116 def dataset_id @gapi["tableReference"]["datasetId"] end |
#delete ⇒ Object
693 694 695 696 697 698 699 700 701 |
# File 'lib/gcloud/bigquery/table.rb', line 693 def delete ensure_connection! resp = connection.delete_table dataset_id, table_id if resp.success? true else fail ApiError.from_response(resp) end end |
#description ⇒ Object
The description of the table.
:category: Attributes
172 173 174 175 |
# File 'lib/gcloud/bigquery/table.rb', line 172 def description ensure_full_data! @gapi["description"] end |
#description=(new_description) ⇒ Object
Updates the description of the table.
:category: Attributes
182 183 184 |
# File 'lib/gcloud/bigquery/table.rb', line 182 def description= new_description patch_gapi! description: new_description end |
#etag ⇒ Object
A string hash of the dataset.
:category: Attributes
152 153 154 155 |
# File 'lib/gcloud/bigquery/table.rb', line 152 def etag ensure_full_data! @gapi["etag"] end |
#expires_at ⇒ Object
The time when this table expires. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.
:category: Attributes
223 224 225 226 227 |
# File 'lib/gcloud/bigquery/table.rb', line 223 def expires_at ensure_full_data! return nil if @gapi["expirationTime"].nil? Time.at(@gapi["expirationTime"] / 1000.0) end |
#extract(extract_url, options = {}) ⇒ Object
Extract the data from the table to a Google Cloud Storage file. For more information, see Exporting Data From BigQuery .
Parameters
extract_url
-
The Google Storage file or file URI pattern(s) to which BigQuery should extract the table data. (
Gcloud::Storage::File
orString
orArray
) options
-
An optional Hash for controlling additional behavior. (
Hash
) options[:format]
-
The exported file format. The default value is
csv
. (String
)The following values are supported:
-
csv
- CSV -
json
- Newline-delimited JSON -
avro
- Avro
-
Returns
Gcloud::Bigquery::ExtractJob
Example
require "gcloud"
gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
extract_job = table.extract "gs://my-bucket/file-name.json",
format: "json"
:category: Data
517 518 519 520 521 522 523 524 525 |
# File 'lib/gcloud/bigquery/table.rb', line 517 def extract extract_url, = {} ensure_connection! resp = connection.extract_table gapi, extract_url, if resp.success? Job.from_gapi resp.data, connection else fail ApiError.from_response(resp) end end |
#fields ⇒ Object
The fields of the table.
:category: Attributes
320 321 322 323 324 325 |
# File 'lib/gcloud/bigquery/table.rb', line 320 def fields f = schema["fields"] f = f.to_hash if f.respond_to? :to_hash f = [] if f.nil? f end |
#headers ⇒ Object
The names of the columns in the table.
:category: Attributes
332 333 334 |
# File 'lib/gcloud/bigquery/table.rb', line 332 def headers fields.map { |f| f["name"] } end |
#insert(rows, options = {}) ⇒ Object
Inserts data into the table for near-immediate querying, without the need to complete a #load operation before the data can appear in query results. See Streaming Data Into BigQuery .
Parameters
rows
-
A hash object or array of hash objects containing the data. (
Array
orHash
) options
-
An optional Hash for controlling additional behavior. (
Hash
) options[:skip_invalid]
-
Insert all valid rows of a request, even if invalid rows exist. The default value is
false
, which causes the entire request to fail if any invalid rows exist. (Boolean
) options[:ignore_unknown]
-
Accept rows that contain values that do not match the schema. The unknown values are ignored. Default is false, which treats unknown values as errors. (
Boolean
)
Returns
Gcloud::Bigquery::InsertResponse
Example
require "gcloud"
gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
rows = [
{ "first_name" => "Alice", "age" => 21 },
{ "first_name" => "Bob", "age" => 22 }
]
table.insert rows
:category: Data
662 663 664 665 666 667 668 669 670 671 |
# File 'lib/gcloud/bigquery/table.rb', line 662 def insert rows, = {} rows = [rows] if rows.is_a? Hash ensure_connection! resp = connection.insert_tabledata dataset_id, table_id, rows, if resp.success? InsertResponse.from_gapi rows, resp.data else fail ApiError.from_response(resp) end end |
#link(source_url, options = {}) ⇒ Object
Links the table to a source table identified by a URI.
Parameters
source_url
-
The URI of source table to link. (
String
) options
-
An optional Hash for controlling additional behavior. (
Hash
) options[:create]
-
Specifies whether the job is allowed to create new tables. (
String
)The following values are supported:
-
needed
- Create the table if it does not exist. -
never
- The table must already exist. A ‘notFound’ error is raised if the table does not exist.
-
options[:write]
-
Specifies how to handle data already present in the table. The default value is
empty
. (String
)The following values are supported:
-
truncate
- BigQuery overwrites the table data. -
append
- BigQuery appends the data to the table. -
empty
- An error will be returned if the table already contains data.
-
Returns
Gcloud::Bigquery::Job
:category: Data
468 469 470 471 472 473 474 475 476 |
# File 'lib/gcloud/bigquery/table.rb', line 468 def link source_url, = {} #:nodoc: ensure_connection! resp = connection.link_table gapi, source_url, if resp.success? Job.from_gapi resp.data, connection else fail ApiError.from_response(resp) end end |
#load(file, options = {}) ⇒ Object
Loads data into the table.
Parameters
file
-
A file or the URI of a Google Cloud Storage file containing data to load into the table. (
File
orGcloud::Storage::File
orString
) options
-
An optional Hash for controlling additional behavior. (
Hash
) options[:format]
-
The exported file format. The default value is
csv
. (String
)The following values are supported:
-
csv
- CSV -
json
- Newline-delimited JSON -
avro
- Avro
-
options[:create]
-
Specifies whether the job is allowed to create new tables. (
String
)The following values are supported:
-
needed
- Create the table if it does not exist. -
never
- The table must already exist. A ‘notFound’ error is raised if the table does not exist.
-
options[:write]
-
Specifies how to handle data already present in the table. The default value is
empty
. (String
)The following values are supported:
-
truncate
- BigQuery overwrites the table data. -
append
- BigQuery appends the data to the table. -
empty
- An error will be returned if the table already contains data.
-
Returns
Gcloud::Bigquery::LoadJob
Examples
require "gcloud"
gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
load_job = table.load "gs://my-bucket/file-name.csv"
You can also pass a gcloud storage file instance.
require "gcloud"
require "gcloud/storage"
gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
storage = gcloud.storage
bucket = storage.bucket "my-bucket"
file = bucket.file "file-name.csv"
load_job = table.load file
Or, you can upload a smaller file directly. See Data with a POST Request[ cloud.google.com/bigquery/loading-data-post-request#multipart].
require "gcloud"
gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
file = File.open "my_data.csv"
load_job = table.load file
:category: Data
608 609 610 611 612 613 614 615 616 617 |
# File 'lib/gcloud/bigquery/table.rb', line 608 def load file, = {} ensure_connection! if storage_url? file load_storage file, elsif local_file? file load_local file, else fail Gcloud::Bigquery::Error, "Don't know how to load #{file}" end end |
#location ⇒ Object
The geographic location where the table should reside. Possible values include EU and US. The default value is US.
:category: Attributes
263 264 265 266 |
# File 'lib/gcloud/bigquery/table.rb', line 263 def location ensure_full_data! @gapi["location"] end |
#modified_at ⇒ Object
The date when this table was last modified.
:category: Attributes
234 235 236 237 |
# File 'lib/gcloud/bigquery/table.rb', line 234 def modified_at ensure_full_data! Time.at(@gapi["lastModifiedTime"] / 1000.0) end |
#name ⇒ Object
The name of the table.
:category: Attributes
134 135 136 |
# File 'lib/gcloud/bigquery/table.rb', line 134 def name @gapi["friendlyName"] end |
#name=(new_name) ⇒ Object
Updates the name of the table.
:category: Attributes
143 144 145 |
# File 'lib/gcloud/bigquery/table.rb', line 143 def name= new_name patch_gapi! name: new_name end |
#project_id ⇒ Object
The ID of the Project
containing this table.
:category: Attributes
125 126 127 |
# File 'lib/gcloud/bigquery/table.rb', line 125 def project_id @gapi["tableReference"]["projectId"] end |
#rows_count ⇒ Object
The number of rows in the table.
:category: Data
201 202 203 204 |
# File 'lib/gcloud/bigquery/table.rb', line 201 def rows_count ensure_full_data! @gapi["numRows"] end |
#schema ⇒ Object
The schema of the table.
:category: Attributes
273 274 275 276 277 278 279 |
# File 'lib/gcloud/bigquery/table.rb', line 273 def schema ensure_full_data! s = @gapi["schema"] s = s.to_hash if s.respond_to? :to_hash s = {} if s.nil? s end |
#schema=(new_schema) ⇒ Object
Updates the schema of the table.
Example
require "gcloud"
gcloud = Gcloud.new
bigquery = gcloud.bigquery
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table"
schema = {
"fields" => [
{
"name" => "first_name",
"type" => "STRING",
"mode" => "REQUIRED"
},
{
"name" => "age",
"type" => "INTEGER",
"mode" => "REQUIRED"
}
]
}
table.schema = schema
:category: Attributes
311 312 313 |
# File 'lib/gcloud/bigquery/table.rb', line 311 def schema= new_schema patch_gapi! schema: new_schema end |
#table? ⇒ Boolean
Checks if the table’s type is “TABLE”.
:category: Attributes
244 245 246 |
# File 'lib/gcloud/bigquery/table.rb', line 244 def table? @gapi["type"] == "TABLE" end |
#table_id ⇒ Object
A unique ID for this table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.
:category: Attributes
107 108 109 |
# File 'lib/gcloud/bigquery/table.rb', line 107 def table_id @gapi["tableReference"]["tableId"] end |
#url ⇒ Object
A URL that can be used to access the dataset using the REST API.
:category: Attributes
162 163 164 165 |
# File 'lib/gcloud/bigquery/table.rb', line 162 def url ensure_full_data! @gapi["selfLink"] end |
#view? ⇒ Boolean
Checks if the table’s type is “VIEW”.
:category: Attributes
253 254 255 |
# File 'lib/gcloud/bigquery/table.rb', line 253 def view? @gapi["type"] == "VIEW" end |