Class: Gcloud::Bigquery::LoadJob
- Defined in:
- lib/gcloud/bigquery/load_job.rb
Overview
LoadJob
A Job subclass representing a load operation that may be performed on a Table. A LoadJob instance is created when you call Table#load.
See Loading Data Into BigQuery and the Jobs API reference for details.
Instance Attribute Summary
Attributes inherited from Job
Instance Method Summary collapse
-
#allow_jagged_rows? ⇒ Boolean
Checks if the load operation accepts rows that are missing trailing optional columns.
-
#backup? ⇒ Boolean
Checks if the source data is a Google Cloud Datastore backup.
-
#csv? ⇒ Boolean
Checks if the format of the source data is CSV.
-
#delimiter ⇒ Object
The delimiter used between fields in the source data.
-
#destination ⇒ Object
The table into which the operation loads data.
-
#ignore_unknown_values? ⇒ Boolean
Checks if the load operation allows extra values that are not represented in the table schema.
-
#input_file_bytes ⇒ Object
The number of bytes of source data.
-
#input_files ⇒ Object
The number of source files.
-
#iso8859_1? ⇒ Boolean
Checks if the character encoding of the data is ISO-8859-1.
-
#json? ⇒ Boolean
Checks if the format of the source data is newline-delimited JSON.
-
#max_bad_records ⇒ Object
The maximum number of bad records that the load operation can ignore.
-
#output_bytes ⇒ Object
The number of bytes that have been loaded into the table.
-
#output_rows ⇒ Object
The number of rows that have been loaded into the table.
-
#quote ⇒ Object
The value that is used to quote data sections in a CSV file.
-
#quoted_newlines? ⇒ Boolean
Checks if quoted data sections may contain newline characters in a CSV file.
-
#schema ⇒ Object
The schema for the data.
-
#skip_leading_rows ⇒ Object
The number of header rows at the top of a CSV file to skip.
-
#sources ⇒ Object
The URI or URIs representing the Google Cloud Storage files from which the operation loads data.
-
#utf8? ⇒ Boolean
Checks if the character encoding of the data is UTF-8.
Methods inherited from Job
#configuration, #created_at, #done?, #ended_at, #error, #errors, #failed?, from_gapi, #initialize, #job_id, #pending?, #project_id, #refresh!, #rerun!, #running?, #started_at, #state, #statistics, #status
Constructor Details
This class inherits a constructor from Gcloud::Bigquery::Job
Instance Method Details
#allow_jagged_rows? ⇒ Boolean
Checks if the load operation accepts rows that are missing trailing optional columns. The missing values are treated as nulls. If false
, records with missing trailing columns are treated as bad records, and if there are too many bad records, an error is returned. The default value is false
. Only applicable to CSV, ignored for other formats.
144 145 146 147 148 |
# File 'lib/gcloud/bigquery/load_job.rb', line 144 def allow_jagged_rows? val = config["load"]["allowJaggedRows"] val = false if val.nil? val end |
#backup? ⇒ Boolean
Checks if the source data is a Google Cloud Datastore backup.
133 134 135 136 |
# File 'lib/gcloud/bigquery/load_job.rb', line 133 def backup? val = config["load"]["sourceFormat"] val == "DATASTORE_BACKUP" end |
#csv? ⇒ Boolean
Checks if the format of the source data is CSV. The default is true
.
125 126 127 128 129 |
# File 'lib/gcloud/bigquery/load_job.rb', line 125 def csv? val = config["load"]["sourceFormat"] return true if val.nil? val == "CSV" end |
#delimiter ⇒ Object
The delimiter used between fields in the source data. The default is a comma (,).
52 53 54 55 56 |
# File 'lib/gcloud/bigquery/load_job.rb', line 52 def delimiter val = config["load"]["fieldDelimiter"] val = "," if val.nil? val end |
#destination ⇒ Object
The table into which the operation loads data. This is the table on which Table#load was invoked. Returns a Table instance.
41 42 43 44 45 46 47 |
# File 'lib/gcloud/bigquery/load_job.rb', line 41 def destination table = config["load"]["destinationTable"] return nil unless table retrieve_table table["projectId"], table["datasetId"], table["tableId"] end |
#ignore_unknown_values? ⇒ Boolean
Checks if the load operation allows extra values that are not represented in the table schema. If true
, the extra values are ignored. If false
, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned. The default is false
.
156 157 158 159 160 |
# File 'lib/gcloud/bigquery/load_job.rb', line 156 def ignore_unknown_values? val = config["load"]["ignoreUnknownValues"] val = false if val.nil? val end |
#input_file_bytes ⇒ Object
The number of bytes of source data.
180 181 182 |
# File 'lib/gcloud/bigquery/load_job.rb', line 180 def input_file_bytes stats["load"]["inputFileBytes"] end |
#input_files ⇒ Object
The number of source files.
174 175 176 |
# File 'lib/gcloud/bigquery/load_job.rb', line 174 def input_files stats["load"]["inputFiles"] end |
#iso8859_1? ⇒ Boolean
Checks if the character encoding of the data is ISO-8859-1.
78 79 80 81 |
# File 'lib/gcloud/bigquery/load_job.rb', line 78 def iso8859_1? val = config["load"]["encoding"] val == "ISO-8859-1" end |
#json? ⇒ Boolean
Checks if the format of the source data is newline-delimited JSON. The default is false
.
118 119 120 121 |
# File 'lib/gcloud/bigquery/load_job.rb', line 118 def json? val = config["load"]["sourceFormat"] val == "NEWLINE_DELIMITED_JSON" end |
#max_bad_records ⇒ Object
The maximum number of bad records that the load operation can ignore. If the number of bad records exceeds this value, an error is returned. The default value is 0
, which requires that all records be valid.
100 101 102 103 104 |
# File 'lib/gcloud/bigquery/load_job.rb', line 100 def max_bad_records val = config["load"]["maxBadRecords"] val = 0 if val.nil? val end |
#output_bytes ⇒ Object
The number of bytes that have been loaded into the table. While an import job is in the running state, this value may change.
194 195 196 |
# File 'lib/gcloud/bigquery/load_job.rb', line 194 def output_bytes stats["load"]["outputBytes"] end |
#output_rows ⇒ Object
The number of rows that have been loaded into the table. While an import job is in the running state, this value may change.
187 188 189 |
# File 'lib/gcloud/bigquery/load_job.rb', line 187 def output_rows stats["load"]["outputRows"] end |
#quote ⇒ Object
The value that is used to quote data sections in a CSV file. The default value is a double-quote (+“+). If your data does not contain quoted sections, the value should be an empty string. If your data contains quoted newline characters, #quoted_newlines? should return true
.
89 90 91 92 93 |
# File 'lib/gcloud/bigquery/load_job.rb', line 89 def quote val = config["load"]["quote"] val = "\"" if val.nil? val end |
#quoted_newlines? ⇒ Boolean
Checks if quoted data sections may contain newline characters in a CSV file. The default is false
.
109 110 111 112 113 |
# File 'lib/gcloud/bigquery/load_job.rb', line 109 def quoted_newlines? val = config["load"]["allowQuotedNewlines"] val = true if val.nil? val end |
#schema ⇒ Object
The schema for the data. Returns a hash. Can be empty if the table has already has the correct schema (see Table#schema= and Table#schema), or if the schema can be inferred from the loaded data.
166 167 168 169 170 |
# File 'lib/gcloud/bigquery/load_job.rb', line 166 def schema val = config["load"]["schema"] val = {} if val.nil? val end |
#skip_leading_rows ⇒ Object
The number of header rows at the top of a CSV file to skip. The default value is 0
.
61 62 63 64 65 |
# File 'lib/gcloud/bigquery/load_job.rb', line 61 def skip_leading_rows val = config["load"]["skipLeadingRows"] val = 0 if val.nil? val end |
#sources ⇒ Object
The URI or URIs representing the Google Cloud Storage files from which the operation loads data.
34 35 36 |
# File 'lib/gcloud/bigquery/load_job.rb', line 34 def sources Array config["load"]["sourceUris"] end |
#utf8? ⇒ Boolean
Checks if the character encoding of the data is UTF-8. This is the default.
70 71 72 73 74 |
# File 'lib/gcloud/bigquery/load_job.rb', line 70 def utf8? val = config["load"]["encoding"] return true if val.nil? val == "UTF-8" end |