Class: OCI::DataFlow::Models::CreateRunDetails

Inherits:
Object
  • Object
show all
Defined in:
lib/oci/data_flow/models/create_run_details.rb

Overview

The create run details. The following properties are optional and override the default values set in the associated application:

- applicationId
- archiveUri
- arguments
- configuration
- definedTags
- displayName
- driverShape
- execute
- executorShape
- freeformTags
- logsBucketUri
- metastoreId
- numExecutors
- parameters
- sparkVersion
- warehouseBucketUri

It is expected that either the applicationId or the execute parameter is specified; but not both. If both or none are set, a Bad Request (HTTP 400) status will be sent as the response. If an appicationId is not specified, then a value for the execute parameter is expected. Using data parsed from the value, a new application will be created and assicated with the new run. See information on the execute parameter for details on the format of this parameter.

The optional parameter spark version can only be specified when using the execute parameter. If it is not specified when using the execute parameter, the latest version will be used as default. If the execute parameter is not used, the spark version will be taken from the associated application.

If displayName is not specified, it will be derived from the displayName of associated application or set by API using fileUri’s application file name. Once a run is created, its properties (except for definedTags and freeformTags) cannot be changed. If the parent application’s properties (including definedTags and freeformTags) are updated, the corresponding properties of the run will not update.

Constant Summary collapse

TYPE_ENUM = 
[
  TYPE_BATCH = 'BATCH'.freeze,
  TYPE_STREAMING = 'STREAMING'.freeze
].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(attributes = {}) ⇒ CreateRunDetails

Initializes the object

Parameters:

  • attributes (Hash) (defaults to: {}) —

    Model attributes in the form of hash

Options Hash (attributes):

  • :application_id (String) —

    The value to assign to the #application_id property

  • :archive_uri (String) —

    The value to assign to the #archive_uri property

  • :arguments (Array<String>) —

    The value to assign to the #arguments property

  • :compartment_id (String) —

    The value to assign to the #compartment_id property

  • :configuration (Hash<String, String>) —

    The value to assign to the #configuration property

  • :defined_tags (Hash<String, Hash<String, Object>>) —

    The value to assign to the #defined_tags property

  • :display_name (String) —

    The value to assign to the #display_name property

  • :driver_shape (String) —

    The value to assign to the #driver_shape property

  • :execute (String) —

    The value to assign to the #execute property

  • :executor_shape (String) —

    The value to assign to the #executor_shape property

  • :freeform_tags (Hash<String, String>) —

    The value to assign to the #freeform_tags property

  • :logs_bucket_uri (String) —

    The value to assign to the #logs_bucket_uri property

  • :metastore_id (String) —

    The value to assign to the #metastore_id property

  • :num_executors (Integer) —

    The value to assign to the #num_executors property

  • :parameters (Array<OCI::DataFlow::Models::ApplicationParameter>) —

    The value to assign to the #parameters property

  • :spark_version (String) —

    The value to assign to the #spark_version property

  • :type (String) —

    The value to assign to the #type property

  • :warehouse_bucket_uri (String) —

    The value to assign to the #warehouse_bucket_uri property



239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 239

def initialize(attributes = {})
  return unless attributes.is_a?(Hash)

  # convert string to symbol for hash key
  attributes = attributes.each_with_object({}) { |(k, v), h| h[k.to_sym] = v }

  self.application_id = attributes[:'applicationId'] if attributes[:'applicationId']

  raise 'You cannot provide both :applicationId and :application_id' if attributes.key?(:'applicationId') && attributes.key?(:'application_id')

  self.application_id = attributes[:'application_id'] if attributes[:'application_id']

  self.archive_uri = attributes[:'archiveUri'] if attributes[:'archiveUri']

  raise 'You cannot provide both :archiveUri and :archive_uri' if attributes.key?(:'archiveUri') && attributes.key?(:'archive_uri')

  self.archive_uri = attributes[:'archive_uri'] if attributes[:'archive_uri']

  self.arguments = attributes[:'arguments'] if attributes[:'arguments']

  self.compartment_id = attributes[:'compartmentId'] if attributes[:'compartmentId']

  raise 'You cannot provide both :compartmentId and :compartment_id' if attributes.key?(:'compartmentId') && attributes.key?(:'compartment_id')

  self.compartment_id = attributes[:'compartment_id'] if attributes[:'compartment_id']

  self.configuration = attributes[:'configuration'] if attributes[:'configuration']

  self.defined_tags = attributes[:'definedTags'] if attributes[:'definedTags']

  raise 'You cannot provide both :definedTags and :defined_tags' if attributes.key?(:'definedTags') && attributes.key?(:'defined_tags')

  self.defined_tags = attributes[:'defined_tags'] if attributes[:'defined_tags']

  self.display_name = attributes[:'displayName'] if attributes[:'displayName']

  raise 'You cannot provide both :displayName and :display_name' if attributes.key?(:'displayName') && attributes.key?(:'display_name')

  self.display_name = attributes[:'display_name'] if attributes[:'display_name']

  self.driver_shape = attributes[:'driverShape'] if attributes[:'driverShape']

  raise 'You cannot provide both :driverShape and :driver_shape' if attributes.key?(:'driverShape') && attributes.key?(:'driver_shape')

  self.driver_shape = attributes[:'driver_shape'] if attributes[:'driver_shape']

  self.execute = attributes[:'execute'] if attributes[:'execute']

  self.executor_shape = attributes[:'executorShape'] if attributes[:'executorShape']

  raise 'You cannot provide both :executorShape and :executor_shape' if attributes.key?(:'executorShape') && attributes.key?(:'executor_shape')

  self.executor_shape = attributes[:'executor_shape'] if attributes[:'executor_shape']

  self.freeform_tags = attributes[:'freeformTags'] if attributes[:'freeformTags']

  raise 'You cannot provide both :freeformTags and :freeform_tags' if attributes.key?(:'freeformTags') && attributes.key?(:'freeform_tags')

  self.freeform_tags = attributes[:'freeform_tags'] if attributes[:'freeform_tags']

  self.logs_bucket_uri = attributes[:'logsBucketUri'] if attributes[:'logsBucketUri']

  raise 'You cannot provide both :logsBucketUri and :logs_bucket_uri' if attributes.key?(:'logsBucketUri') && attributes.key?(:'logs_bucket_uri')

  self.logs_bucket_uri = attributes[:'logs_bucket_uri'] if attributes[:'logs_bucket_uri']

  self.metastore_id = attributes[:'metastoreId'] if attributes[:'metastoreId']

  raise 'You cannot provide both :metastoreId and :metastore_id' if attributes.key?(:'metastoreId') && attributes.key?(:'metastore_id')

  self.metastore_id = attributes[:'metastore_id'] if attributes[:'metastore_id']

  self.num_executors = attributes[:'numExecutors'] if attributes[:'numExecutors']

  raise 'You cannot provide both :numExecutors and :num_executors' if attributes.key?(:'numExecutors') && attributes.key?(:'num_executors')

  self.num_executors = attributes[:'num_executors'] if attributes[:'num_executors']

  self.parameters = attributes[:'parameters'] if attributes[:'parameters']

  self.spark_version = attributes[:'sparkVersion'] if attributes[:'sparkVersion']

  raise 'You cannot provide both :sparkVersion and :spark_version' if attributes.key?(:'sparkVersion') && attributes.key?(:'spark_version')

  self.spark_version = attributes[:'spark_version'] if attributes[:'spark_version']

  self.type = attributes[:'type'] if attributes[:'type']
  self.type = "BATCH" if type.nil? && !attributes.key?(:'type') # rubocop:disable Style/StringLiterals

  self.warehouse_bucket_uri = attributes[:'warehouseBucketUri'] if attributes[:'warehouseBucketUri']

  raise 'You cannot provide both :warehouseBucketUri and :warehouse_bucket_uri' if attributes.key?(:'warehouseBucketUri') && attributes.key?(:'warehouse_bucket_uri')

  self.warehouse_bucket_uri = attributes[:'warehouse_bucket_uri'] if attributes[:'warehouse_bucket_uri']
end

Instance Attribute Details

#application_id ⇒ String

The OCID of the associated application. If this value is set, then no value for the execute parameter is required. If this value is not set, then a value for the execute parameter is required, and a new application is created and associated with the new run.

Returns:

  • (String) 


51
52
53
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 51

def application_id
  @application_id
end

#archive_uri ⇒ String

An Oracle Cloud Infrastructure URI of an archive.zip file containing custom dependencies that may be used to support the execution a Python, Java, or Scala application. See docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

Returns:

  • (String) 


57
58
59
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 57

def archive_uri
  @archive_uri
end

#arguments ⇒ Array<String>

The arguments passed to the running application as command line arguments. An argument is either a plain text or a placeholder. Placeholders are replaced using values from the parameters map. Each placeholder specified must be represented in the parameters map else the request (POST or PUT) will fail with a HTTP 400 status code. Placeholders are specified as ‘Service Api Spec`, where name is the name of the parameter. Example: `[ "–input", "$input_file", "–name", "John Doe" ]` If "input_file" has a value of "mydata.xml", then the value above will be translated to `–input mydata.xml –name "John Doe"`

Returns:

  • (Array<String>) 


69
70
71
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 69

def arguments
  @arguments
end

#compartment_id ⇒ String

[Required] The OCID of a compartment.

Returns:

  • (String) 


74
75
76
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 74

def compartment_id
  @compartment_id
end

#configuration ⇒ Hash<String, String>

The Spark configuration passed to the running process. See spark.apache.org/docs/latest/configuration.html#available-properties. Example: { "spark.app.name" : "My App Name", "spark.shuffle.io.maxRetries" : "4" } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

Returns:

  • (Hash<String, String>) 


83
84
85
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 83

def configuration
  @configuration
end

#defined_tags ⇒ Hash<String, Hash<String, Object>>

Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see [Resource Tags](docs.cloud.oracle.com/iaas/Content/General/Concepts/resourcetags.htm). Example: ‘{"CostCenter": "42"}`

Returns:

  • (Hash<String, Hash<String, Object>>) 


89
90
91
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 89

def defined_tags
  @defined_tags
end

#display_name ⇒ String

A user-friendly name that does not have to be unique. Avoid entering confidential information. If this value is not specified, it will be derived from the associated application’s displayName or set by API using fileUri’s application file name.

Returns:

  • (String) 


94
95
96
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 94

def display_name
  @display_name
end

#driver_shape ⇒ String

The VM shape for the driver. Sets the driver cores and memory.

Returns:

  • (String) 


99
100
101
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 99

def driver_shape
  @driver_shape
end

#execute ⇒ String

The input used for spark-submit command. For more details see spark.apache.org/docs/latest/submitting-applications.html#launching-applications-with-spark-submit. Supported options include --class, --file, --jars, --conf, --py-files, and main application file with arguments. Example: “–jars oci://path/to/a.jar,oci://path/to/b.jar –files oci://path/to/a.json,oci://path/to/b.csv –py-files oci://path/to/a.py,oci://path/to/b.py –conf spark.sql.crossJoin.enabled=true –class org.apache.spark.examples.SparkPi oci://path/to/main.jar 10“ Note: If execute is specified together with applicationId, className, configuration, fileUri, language, arguments, parameters during application create/update, or run create/submit, Data Flow service will use derived information from execute input only.

Returns:

  • (String) 


108
109
110
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 108

def execute
  @execute
end

#executor_shape ⇒ String

The VM shape for the executors. Sets the executor cores and memory.

Returns:

  • (String) 


113
114
115
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 113

def executor_shape
  @executor_shape
end

#freeform_tags ⇒ Hash<String, String>

Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see [Resource Tags](docs.cloud.oracle.com/iaas/Content/General/Concepts/resourcetags.htm). Example: ‘"Finance"`

Returns:

  • (Hash<String, String>) 


120
121
122
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 120

def freeform_tags
  @freeform_tags
end

#logs_bucket_uri ⇒ String

An Oracle Cloud Infrastructure URI of the bucket where the Spark job logs are to be uploaded. See docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

Returns:

  • (String) 


126
127
128
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 126

def logs_bucket_uri
  @logs_bucket_uri
end

#metastore_id ⇒ String

The OCID of OCI Hive Metastore.

Returns:

  • (String) 


131
132
133
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 131

def metastore_id
  @metastore_id
end

#num_executors ⇒ Integer

The number of executor VMs requested.

Returns:

  • (Integer) 


136
137
138
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 136

def num_executors
  @num_executors
end

#parameters ⇒ Array<OCI::DataFlow::Models::ApplicationParameter>

An array of name/value pairs used to fill placeholders found in properties like Application.arguments. The name must be a string of one or more word characters (a-z, A-Z, 0-9, _). The value can be a string of 0 or more characters of any kind. Example: [ { name: "iterations", value: "10"}, { name: "input_file", value: "mydata.xml" }, { name: "variable_x", value: "$x"} ]

Returns:



144
145
146
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 144

def parameters
  @parameters
end

#spark_version ⇒ String

The Spark version utilized to run the application. This value may be set if applicationId is not since the Spark version will be taken from the associated application.

Returns:

  • (String) 


149
150
151
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 149

def spark_version
  @spark_version
end

#type ⇒ String

The Spark application processing type.

Returns:

  • (String) 


154
155
156
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 154

def type
  @type
end

#warehouse_bucket_uri ⇒ String

An Oracle Cloud Infrastructure URI of the bucket to be used as default warehouse directory for BATCH SQL runs. See docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

Returns:

  • (String) 


161
162
163
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 161

def warehouse_bucket_uri
  @warehouse_bucket_uri
end

Class Method Details

.attribute_map ⇒ Object

Attribute mapping from ruby-style variable name to JSON key.



164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 164

def self.attribute_map
  {
    # rubocop:disable Style/SymbolLiteral
    'application_id': :'applicationId',
    'archive_uri': :'archiveUri',
    'arguments': :'arguments',
    'compartment_id': :'compartmentId',
    'configuration': :'configuration',
    'defined_tags': :'definedTags',
    'display_name': :'displayName',
    'driver_shape': :'driverShape',
    'execute': :'execute',
    'executor_shape': :'executorShape',
    'freeform_tags': :'freeformTags',
    'logs_bucket_uri': :'logsBucketUri',
    'metastore_id': :'metastoreId',
    'num_executors': :'numExecutors',
    'parameters': :'parameters',
    'spark_version': :'sparkVersion',
    'type': :'type',
    'warehouse_bucket_uri': :'warehouseBucketUri'
    # rubocop:enable Style/SymbolLiteral
  }
end

.swagger_types ⇒ Object

Attribute type mapping.



190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 190

def self.swagger_types
  {
    # rubocop:disable Style/SymbolLiteral
    'application_id': :'String',
    'archive_uri': :'String',
    'arguments': :'Array<String>',
    'compartment_id': :'String',
    'configuration': :'Hash<String, String>',
    'defined_tags': :'Hash<String, Hash<String, Object>>',
    'display_name': :'String',
    'driver_shape': :'String',
    'execute': :'String',
    'executor_shape': :'String',
    'freeform_tags': :'Hash<String, String>',
    'logs_bucket_uri': :'String',
    'metastore_id': :'String',
    'num_executors': :'Integer',
    'parameters': :'Array<OCI::DataFlow::Models::ApplicationParameter>',
    'spark_version': :'String',
    'type': :'String',
    'warehouse_bucket_uri': :'String'
    # rubocop:enable Style/SymbolLiteral
  }
end

Instance Method Details

#==(other) ⇒ Object

Checks equality by comparing each attribute.

Parameters:

  • other (Object) —

    the other object to be compared



350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 350

def ==(other)
  return true if equal?(other)

  self.class == other.class &&
    application_id == other.application_id &&
    archive_uri == other.archive_uri &&
    arguments == other.arguments &&
    compartment_id == other.compartment_id &&
    configuration == other.configuration &&
    defined_tags == other.defined_tags &&
    display_name == other.display_name &&
    driver_shape == other.driver_shape &&
    execute == other.execute &&
    executor_shape == other.executor_shape &&
    freeform_tags == other.freeform_tags &&
    logs_bucket_uri == other.logs_bucket_uri &&
    metastore_id == other.metastore_id &&
    num_executors == other.num_executors &&
    parameters == other.parameters &&
    spark_version == other.spark_version &&
    type == other.type &&
    warehouse_bucket_uri == other.warehouse_bucket_uri
end

#build_from_hash(attributes) ⇒ Object

Builds the object from hash

Parameters:

  • attributes (Hash) —

    Model attributes in the form of hash

Returns:

  • (Object) —

    Returns the model itself



397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 397

def build_from_hash(attributes)
  return nil unless attributes.is_a?(Hash)

  self.class.swagger_types.each_pair do |key, type|
    if type =~ /^Array<(.*)>/i
      # check to ensure the input is an array given that the the attribute
      # is documented as an array but the input is not
      if attributes[self.class.attribute_map[key]].is_a?(Array)
        public_method("#{key}=").call(
          attributes[self.class.attribute_map[key]]
            .map { |v| OCI::Internal::Util.convert_to_type(Regexp.last_match(1), v) }
        )
      end
    elsif !attributes[self.class.attribute_map[key]].nil?
      public_method("#{key}=").call(
        OCI::Internal::Util.convert_to_type(type, attributes[self.class.attribute_map[key]])
      )
    end
    # or else data not found in attributes(hash), not an issue as the data can be optional
  end

  self
end

#eql?(other) ⇒ Boolean

Parameters:

  • other (Object) —

    the other object to be compared

Returns:

  • (Boolean)

See Also:

  • `==` method


377
378
379
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 377

def eql?(other)
  self == other
end

#hash ⇒ Fixnum

Calculates hash code according to all attributes.

Returns:

  • (Fixnum) —

    Hash code



386
387
388
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 386

def hash
  [application_id, archive_uri, arguments, compartment_id, configuration, defined_tags, display_name, driver_shape, execute, executor_shape, freeform_tags, logs_bucket_uri, metastore_id, num_executors, parameters, spark_version, type, warehouse_bucket_uri].hash
end

#to_hash ⇒ Hash

Returns the object in the form of hash

Returns:

  • (Hash) —

    Returns the object in the form of hash



430
431
432
433
434
435
436
437
438
439
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 430

def to_hash
  hash = {}
  self.class.attribute_map.each_pair do |attr, param|
    value = public_method(attr).call
    next if value.nil? && !instance_variable_defined?("@#{attr}")

    hash[param] = _to_hash(value)
  end
  hash
end

#to_s ⇒ String

Returns the string representation of the object

Returns:

  • (String) —

    String presentation of the object



424
425
426
 
# File 'lib/oci/data_flow/models/create_run_details.rb', line 424

def to_s
  to_hash.to_s
end