Class: Google::Cloud::Dataplex::V1::DataDiscoverySpec::StorageConfig

Inherits:
Object
  • Object
show all
Extended by:
Protobuf::MessageExts::ClassMethods
Includes:
Protobuf::MessageExts
Defined in:
proto_docs/google/cloud/dataplex/v1/data_discovery.rb

Overview

Configurations related to Cloud Storage as the data source.

Defined Under Namespace

Classes: CsvOptions, JsonOptions

Instance Attribute Summary collapse

Instance Attribute Details

#csv_options ⇒ ::Google::Cloud::Dataplex::V1::DataDiscoverySpec::StorageConfig::CsvOptions

Returns Optional. Configuration for CSV data.

Returns:



88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'proto_docs/google/cloud/dataplex/v1/data_discovery.rb', line 88

class StorageConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # Describes CSV and similar semi-structured data formats.
  # @!attribute [rw] header_rows
  #   @return [::Integer]
  #     Optional. The number of rows to interpret as header rows that should be
  #     skipped when reading data rows.
  # @!attribute [rw] delimiter
  #   @return [::String]
  #     Optional. The delimiter that is used to separate values. The default is
  #     `,` (comma).
  # @!attribute [rw] encoding
  #   @return [::String]
  #     Optional. The character encoding of the data. The default is UTF-8.
  # @!attribute [rw] type_inference_disabled
  #   @return [::Boolean]
  #     Optional. Whether to disable the inference of data types for CSV data.
  #     If true, all columns are registered as strings.
  # @!attribute [rw] quote
  #   @return [::String]
  #     Optional. The character used to quote column values. Accepts `"`
  #     (double quotation mark) or `'` (single quotation mark). If unspecified,
  #     defaults to `"` (double quotation mark).
  class CsvOptions
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Describes JSON data format.
  # @!attribute [rw] encoding
  #   @return [::String]
  #     Optional. The character encoding of the data. The default is UTF-8.
  # @!attribute [rw] type_inference_disabled
  #   @return [::Boolean]
  #     Optional. Whether to disable the inference of data types for JSON data.
  #     If true, all columns are registered as their primitive types
  #     (strings, number, or boolean).
  class JsonOptions
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#exclude_patterns ⇒ ::Array<::String>

Returns Optional. Defines the data to exclude during discovery. Provide a list of patterns that identify the data to exclude. For Cloud Storage bucket assets, these patterns are interpreted as glob patterns used to match object names. For BigQuery dataset assets, these patterns are interpreted as patterns to match table names.

Returns:

  • (::Array<::String>) —

    Optional. Defines the data to exclude during discovery. Provide a list of patterns that identify the data to exclude. For Cloud Storage bucket assets, these patterns are interpreted as glob patterns used to match object names. For BigQuery dataset assets, these patterns are interpreted as patterns to match table names.



88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'proto_docs/google/cloud/dataplex/v1/data_discovery.rb', line 88

class StorageConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # Describes CSV and similar semi-structured data formats.
  # @!attribute [rw] header_rows
  #   @return [::Integer]
  #     Optional. The number of rows to interpret as header rows that should be
  #     skipped when reading data rows.
  # @!attribute [rw] delimiter
  #   @return [::String]
  #     Optional. The delimiter that is used to separate values. The default is
  #     `,` (comma).
  # @!attribute [rw] encoding
  #   @return [::String]
  #     Optional. The character encoding of the data. The default is UTF-8.
  # @!attribute [rw] type_inference_disabled
  #   @return [::Boolean]
  #     Optional. Whether to disable the inference of data types for CSV data.
  #     If true, all columns are registered as strings.
  # @!attribute [rw] quote
  #   @return [::String]
  #     Optional. The character used to quote column values. Accepts `"`
  #     (double quotation mark) or `'` (single quotation mark). If unspecified,
  #     defaults to `"` (double quotation mark).
  class CsvOptions
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Describes JSON data format.
  # @!attribute [rw] encoding
  #   @return [::String]
  #     Optional. The character encoding of the data. The default is UTF-8.
  # @!attribute [rw] type_inference_disabled
  #   @return [::Boolean]
  #     Optional. Whether to disable the inference of data types for JSON data.
  #     If true, all columns are registered as their primitive types
  #     (strings, number, or boolean).
  class JsonOptions
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#include_patterns ⇒ ::Array<::String>

Returns Optional. Defines the data to include during discovery when only a subset of the data should be considered. Provide a list of patterns that identify the data to include. For Cloud Storage bucket assets, these patterns are interpreted as glob patterns used to match object names. For BigQuery dataset assets, these patterns are interpreted as patterns to match table names.

Returns:

  • (::Array<::String>) —

    Optional. Defines the data to include during discovery when only a subset of the data should be considered. Provide a list of patterns that identify the data to include. For Cloud Storage bucket assets, these patterns are interpreted as glob patterns used to match object names. For BigQuery dataset assets, these patterns are interpreted as patterns to match table names.



88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'proto_docs/google/cloud/dataplex/v1/data_discovery.rb', line 88

class StorageConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # Describes CSV and similar semi-structured data formats.
  # @!attribute [rw] header_rows
  #   @return [::Integer]
  #     Optional. The number of rows to interpret as header rows that should be
  #     skipped when reading data rows.
  # @!attribute [rw] delimiter
  #   @return [::String]
  #     Optional. The delimiter that is used to separate values. The default is
  #     `,` (comma).
  # @!attribute [rw] encoding
  #   @return [::String]
  #     Optional. The character encoding of the data. The default is UTF-8.
  # @!attribute [rw] type_inference_disabled
  #   @return [::Boolean]
  #     Optional. Whether to disable the inference of data types for CSV data.
  #     If true, all columns are registered as strings.
  # @!attribute [rw] quote
  #   @return [::String]
  #     Optional. The character used to quote column values. Accepts `"`
  #     (double quotation mark) or `'` (single quotation mark). If unspecified,
  #     defaults to `"` (double quotation mark).
  class CsvOptions
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Describes JSON data format.
  # @!attribute [rw] encoding
  #   @return [::String]
  #     Optional. The character encoding of the data. The default is UTF-8.
  # @!attribute [rw] type_inference_disabled
  #   @return [::Boolean]
  #     Optional. Whether to disable the inference of data types for JSON data.
  #     If true, all columns are registered as their primitive types
  #     (strings, number, or boolean).
  class JsonOptions
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#json_options ⇒ ::Google::Cloud::Dataplex::V1::DataDiscoverySpec::StorageConfig::JsonOptions

Returns Optional. Configuration for JSON data.

Returns:



88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'proto_docs/google/cloud/dataplex/v1/data_discovery.rb', line 88

class StorageConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # Describes CSV and similar semi-structured data formats.
  # @!attribute [rw] header_rows
  #   @return [::Integer]
  #     Optional. The number of rows to interpret as header rows that should be
  #     skipped when reading data rows.
  # @!attribute [rw] delimiter
  #   @return [::String]
  #     Optional. The delimiter that is used to separate values. The default is
  #     `,` (comma).
  # @!attribute [rw] encoding
  #   @return [::String]
  #     Optional. The character encoding of the data. The default is UTF-8.
  # @!attribute [rw] type_inference_disabled
  #   @return [::Boolean]
  #     Optional. Whether to disable the inference of data types for CSV data.
  #     If true, all columns are registered as strings.
  # @!attribute [rw] quote
  #   @return [::String]
  #     Optional. The character used to quote column values. Accepts `"`
  #     (double quotation mark) or `'` (single quotation mark). If unspecified,
  #     defaults to `"` (double quotation mark).
  class CsvOptions
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Describes JSON data format.
  # @!attribute [rw] encoding
  #   @return [::String]
  #     Optional. The character encoding of the data. The default is UTF-8.
  # @!attribute [rw] type_inference_disabled
  #   @return [::Boolean]
  #     Optional. Whether to disable the inference of data types for JSON data.
  #     If true, all columns are registered as their primitive types
  #     (strings, number, or boolean).
  class JsonOptions
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end