Class: Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest
- Inherits:
-
Object
- Object
- Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest
- Extended by:
- Protobuf::MessageExts::ClassMethods
- Includes:
- Protobuf::MessageExts
- Defined in:
- proto_docs/google/cloud/bigquery/storage/v1/storage.rb
Overview
Request message for AppendRows
.
Because AppendRows is a bidirectional streaming RPC, certain parts of the AppendRowsRequest need only be specified for the first request before switching table destinations. You can also switch table destinations within the same connection for the default stream.
The size of a single AppendRowsRequest must be less than 10 MB in size.
Requests larger than this return an error, typically INVALID_ARGUMENT
.
Defined Under Namespace
Modules: MissingValueInterpretation Classes: ArrowData, MissingValueInterpretationsEntry, ProtoData
Instance Attribute Summary collapse
-
#arrow_rows ⇒ ::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::ArrowData
Rows in arrow format.
-
#default_missing_value_interpretation ⇒ ::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation
Optional.
-
#missing_value_interpretations ⇒ ::Google::Protobuf::Map{::String => ::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation}
A map to indicate how to interpret missing value for some fields.
-
#offset ⇒ ::Google::Protobuf::Int64Value
If present, the write is only performed if the next append offset is same as the provided value.
-
#proto_rows ⇒ ::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::ProtoData
Rows in proto format.
-
#trace_id ⇒ ::String
Id set by client to annotate its identity.
-
#write_stream ⇒ ::String
Required.
Instance Attribute Details
#arrow_rows ⇒ ::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::ArrowData
Returns Rows in arrow format. This is an experimental feature only selected for allowlisted customers.
298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 |
# File 'proto_docs/google/cloud/bigquery/storage/v1/storage.rb', line 298 class AppendRowsRequest include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Arrow schema and data. # Arrow format is an experimental feature only selected for allowlisted # customers. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowSchema] # Optional. Arrow Schema used to serialize the data. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowRecordBatch] # Required. Serialized row data in Arrow format. class ArrowData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # ProtoData contains the data rows and schema when constructing append # requests. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoSchema] # The protocol buffer schema used to serialize the data. Provide this value # whenever: # # * You send the first request of an RPC connection. # # * You change the input schema. # # * You specify a new destination table. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoRows] # Serialized row data in protobuf message format. # Currently, the backend expects the serialized rows to adhere to # proto2 semantics when appending rows, particularly with respect to # how default values are encoded. class ProtoData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # @!attribute [rw] key # @return [::String] # @!attribute [rw] value # @return [::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation] class MissingValueInterpretationsEntry include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # An enum to indicate how to interpret missing values of fields that are # present in user schema but missing in rows. A missing value can represent a # NULL or a column default value defined in BigQuery table schema. module MissingValueInterpretation # Invalid missing value interpretation. Requests with this value will be # rejected. MISSING_VALUE_INTERPRETATION_UNSPECIFIED = 0 # Missing value is interpreted as NULL. NULL_VALUE = 1 # Missing value is interpreted as column default value if declared in the # table schema, NULL otherwise. DEFAULT_VALUE = 2 end end |
#default_missing_value_interpretation ⇒ ::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation
Returns Optional. Default missing value interpretation for all columns in the
table. When a value is specified on an AppendRowsRequest
, it is applied
to all requests on the connection from that point forward, until a
subsequent AppendRowsRequest
sets it to a different value.
missing_value_interpretation
can override
default_missing_value_interpretation
. For example, if you want to write
NULL
instead of using default values for some columns, you can set
default_missing_value_interpretation
to DEFAULT_VALUE
and at the same
time, set missing_value_interpretations
to NULL_VALUE
on those columns.
298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 |
# File 'proto_docs/google/cloud/bigquery/storage/v1/storage.rb', line 298 class AppendRowsRequest include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Arrow schema and data. # Arrow format is an experimental feature only selected for allowlisted # customers. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowSchema] # Optional. Arrow Schema used to serialize the data. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowRecordBatch] # Required. Serialized row data in Arrow format. class ArrowData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # ProtoData contains the data rows and schema when constructing append # requests. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoSchema] # The protocol buffer schema used to serialize the data. Provide this value # whenever: # # * You send the first request of an RPC connection. # # * You change the input schema. # # * You specify a new destination table. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoRows] # Serialized row data in protobuf message format. # Currently, the backend expects the serialized rows to adhere to # proto2 semantics when appending rows, particularly with respect to # how default values are encoded. class ProtoData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # @!attribute [rw] key # @return [::String] # @!attribute [rw] value # @return [::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation] class MissingValueInterpretationsEntry include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # An enum to indicate how to interpret missing values of fields that are # present in user schema but missing in rows. A missing value can represent a # NULL or a column default value defined in BigQuery table schema. module MissingValueInterpretation # Invalid missing value interpretation. Requests with this value will be # rejected. MISSING_VALUE_INTERPRETATION_UNSPECIFIED = 0 # Missing value is interpreted as NULL. NULL_VALUE = 1 # Missing value is interpreted as column default value if declared in the # table schema, NULL otherwise. DEFAULT_VALUE = 2 end end |
#missing_value_interpretations ⇒ ::Google::Protobuf::Map{::String => ::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation}
Returns A map to indicate how to interpret missing value for some fields. Missing values are fields present in user schema but missing in rows. The key is the field name. The value is the interpretation of missing values for the field.
For example, a map {'foo': NULL_VALUE, 'bar': DEFAULT_VALUE} means all missing values in field foo are interpreted as NULL, all missing values in field bar are interpreted as the default value of field bar in table schema.
If a field is not in this map and has missing values, the missing values in this field are interpreted as NULL.
This field only applies to the current request, it won't affect other requests on the connection.
Currently, field name can only be top-level column name, can't be a struct field path like 'foo.bar'.
298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 |
# File 'proto_docs/google/cloud/bigquery/storage/v1/storage.rb', line 298 class AppendRowsRequest include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Arrow schema and data. # Arrow format is an experimental feature only selected for allowlisted # customers. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowSchema] # Optional. Arrow Schema used to serialize the data. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowRecordBatch] # Required. Serialized row data in Arrow format. class ArrowData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # ProtoData contains the data rows and schema when constructing append # requests. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoSchema] # The protocol buffer schema used to serialize the data. Provide this value # whenever: # # * You send the first request of an RPC connection. # # * You change the input schema. # # * You specify a new destination table. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoRows] # Serialized row data in protobuf message format. # Currently, the backend expects the serialized rows to adhere to # proto2 semantics when appending rows, particularly with respect to # how default values are encoded. class ProtoData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # @!attribute [rw] key # @return [::String] # @!attribute [rw] value # @return [::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation] class MissingValueInterpretationsEntry include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # An enum to indicate how to interpret missing values of fields that are # present in user schema but missing in rows. A missing value can represent a # NULL or a column default value defined in BigQuery table schema. module MissingValueInterpretation # Invalid missing value interpretation. Requests with this value will be # rejected. MISSING_VALUE_INTERPRETATION_UNSPECIFIED = 0 # Missing value is interpreted as NULL. NULL_VALUE = 1 # Missing value is interpreted as column default value if declared in the # table schema, NULL otherwise. DEFAULT_VALUE = 2 end end |
#offset ⇒ ::Google::Protobuf::Int64Value
Returns If present, the write is only performed if the next append offset is same as the provided value. If not present, the write is performed at the current end of stream. Specifying a value for this field is not allowed when calling AppendRows for the '_default' stream.
298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 |
# File 'proto_docs/google/cloud/bigquery/storage/v1/storage.rb', line 298 class AppendRowsRequest include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Arrow schema and data. # Arrow format is an experimental feature only selected for allowlisted # customers. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowSchema] # Optional. Arrow Schema used to serialize the data. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowRecordBatch] # Required. Serialized row data in Arrow format. class ArrowData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # ProtoData contains the data rows and schema when constructing append # requests. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoSchema] # The protocol buffer schema used to serialize the data. Provide this value # whenever: # # * You send the first request of an RPC connection. # # * You change the input schema. # # * You specify a new destination table. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoRows] # Serialized row data in protobuf message format. # Currently, the backend expects the serialized rows to adhere to # proto2 semantics when appending rows, particularly with respect to # how default values are encoded. class ProtoData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # @!attribute [rw] key # @return [::String] # @!attribute [rw] value # @return [::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation] class MissingValueInterpretationsEntry include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # An enum to indicate how to interpret missing values of fields that are # present in user schema but missing in rows. A missing value can represent a # NULL or a column default value defined in BigQuery table schema. module MissingValueInterpretation # Invalid missing value interpretation. Requests with this value will be # rejected. MISSING_VALUE_INTERPRETATION_UNSPECIFIED = 0 # Missing value is interpreted as NULL. NULL_VALUE = 1 # Missing value is interpreted as column default value if declared in the # table schema, NULL otherwise. DEFAULT_VALUE = 2 end end |
#proto_rows ⇒ ::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::ProtoData
Returns Rows in proto format.
298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 |
# File 'proto_docs/google/cloud/bigquery/storage/v1/storage.rb', line 298 class AppendRowsRequest include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Arrow schema and data. # Arrow format is an experimental feature only selected for allowlisted # customers. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowSchema] # Optional. Arrow Schema used to serialize the data. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowRecordBatch] # Required. Serialized row data in Arrow format. class ArrowData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # ProtoData contains the data rows and schema when constructing append # requests. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoSchema] # The protocol buffer schema used to serialize the data. Provide this value # whenever: # # * You send the first request of an RPC connection. # # * You change the input schema. # # * You specify a new destination table. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoRows] # Serialized row data in protobuf message format. # Currently, the backend expects the serialized rows to adhere to # proto2 semantics when appending rows, particularly with respect to # how default values are encoded. class ProtoData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # @!attribute [rw] key # @return [::String] # @!attribute [rw] value # @return [::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation] class MissingValueInterpretationsEntry include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # An enum to indicate how to interpret missing values of fields that are # present in user schema but missing in rows. A missing value can represent a # NULL or a column default value defined in BigQuery table schema. module MissingValueInterpretation # Invalid missing value interpretation. Requests with this value will be # rejected. MISSING_VALUE_INTERPRETATION_UNSPECIFIED = 0 # Missing value is interpreted as NULL. NULL_VALUE = 1 # Missing value is interpreted as column default value if declared in the # table schema, NULL otherwise. DEFAULT_VALUE = 2 end end |
#trace_id ⇒ ::String
Returns Id set by client to annotate its identity. Only initial request setting is respected.
298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 |
# File 'proto_docs/google/cloud/bigquery/storage/v1/storage.rb', line 298 class AppendRowsRequest include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Arrow schema and data. # Arrow format is an experimental feature only selected for allowlisted # customers. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowSchema] # Optional. Arrow Schema used to serialize the data. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowRecordBatch] # Required. Serialized row data in Arrow format. class ArrowData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # ProtoData contains the data rows and schema when constructing append # requests. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoSchema] # The protocol buffer schema used to serialize the data. Provide this value # whenever: # # * You send the first request of an RPC connection. # # * You change the input schema. # # * You specify a new destination table. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoRows] # Serialized row data in protobuf message format. # Currently, the backend expects the serialized rows to adhere to # proto2 semantics when appending rows, particularly with respect to # how default values are encoded. class ProtoData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # @!attribute [rw] key # @return [::String] # @!attribute [rw] value # @return [::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation] class MissingValueInterpretationsEntry include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # An enum to indicate how to interpret missing values of fields that are # present in user schema but missing in rows. A missing value can represent a # NULL or a column default value defined in BigQuery table schema. module MissingValueInterpretation # Invalid missing value interpretation. Requests with this value will be # rejected. MISSING_VALUE_INTERPRETATION_UNSPECIFIED = 0 # Missing value is interpreted as NULL. NULL_VALUE = 1 # Missing value is interpreted as column default value if declared in the # table schema, NULL otherwise. DEFAULT_VALUE = 2 end end |
#write_stream ⇒ ::String
Returns Required. The write_stream identifies the append operation. It must be provided in the following scenarios:
In the first request to an AppendRows connection.
In all subsequent requests to an AppendRows connection, if you use the same connection to write to multiple tables or change the input schema for default streams.
For explicitly created write streams, the format is:
projects/{project}/datasets/{dataset}/tables/{table}/streams/{id}
For the special default stream, the format is:
projects/{project}/datasets/{dataset}/tables/{table}/streams/_default
.
An example of a possible sequence of requests with write_stream fields within a single connection:
r1: {write_stream: stream_name_1}
r2: {write_stream: /omit/}
r3: {write_stream: /omit/}
r4: {write_stream: stream_name_2}
r5: {write_stream: stream_name_2}
The destination changed in request_4, so the write_stream field must be populated in all subsequent requests in this stream.
298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 |
# File 'proto_docs/google/cloud/bigquery/storage/v1/storage.rb', line 298 class AppendRowsRequest include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Arrow schema and data. # Arrow format is an experimental feature only selected for allowlisted # customers. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowSchema] # Optional. Arrow Schema used to serialize the data. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ArrowRecordBatch] # Required. Serialized row data in Arrow format. class ArrowData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # ProtoData contains the data rows and schema when constructing append # requests. # @!attribute [rw] writer_schema # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoSchema] # The protocol buffer schema used to serialize the data. Provide this value # whenever: # # * You send the first request of an RPC connection. # # * You change the input schema. # # * You specify a new destination table. # @!attribute [rw] rows # @return [::Google::Cloud::Bigquery::Storage::V1::ProtoRows] # Serialized row data in protobuf message format. # Currently, the backend expects the serialized rows to adhere to # proto2 semantics when appending rows, particularly with respect to # how default values are encoded. class ProtoData include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # @!attribute [rw] key # @return [::String] # @!attribute [rw] value # @return [::Google::Cloud::Bigquery::Storage::V1::AppendRowsRequest::MissingValueInterpretation] class MissingValueInterpretationsEntry include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods end # An enum to indicate how to interpret missing values of fields that are # present in user schema but missing in rows. A missing value can represent a # NULL or a column default value defined in BigQuery table schema. module MissingValueInterpretation # Invalid missing value interpretation. Requests with this value will be # rejected. MISSING_VALUE_INTERPRETATION_UNSPECIFIED = 0 # Missing value is interpreted as NULL. NULL_VALUE = 1 # Missing value is interpreted as column default value if declared in the # table schema, NULL otherwise. DEFAULT_VALUE = 2 end end |