ClusterAttributes

Properties

Name

Type

Description

Notes

cluster_name

Option<String>

Cluster name requested by the user. This doesn’t have to be unique. If not specified at creation, the cluster name is an empty string.

[optional]

spark_version

Option<String>

The runtime version of the cluster, for example “5.0.x-scala2.11”. You can retrieve a list of available runtime versions by using the Runtime versions API call.

[optional]

spark_conf

Option<::std::collections::HashMap<String, serde_json::Value>>

An arbitrary object where the object key is a configuration propery name and the value is a configuration property value.

[optional]

aws_attributes

Option<crate::models::AwsAttributes>

[optional]

node_type_id

Option<String>

This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads A list of available node types can be retrieved by using the List node types API call.

[optional]

driver_node_type_id

Option<String>

The node type of the Spark driver. This field is optional; if unset, the driver node type is set as the same value as node_type_id defined above.

[optional]

ssh_public_keys

Option<Vec>

SSH public key contents that is added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name ubuntu on port 2200. Up to 10 keys can be specified.

[optional]

custom_tags

Option<::std::collections::HashMap<String, serde_json::Value>>

An object with key value pairs. The key length must be between 1 and 127 UTF-8 characters, inclusive. The value length must be less than or equal to 255 UTF-8 characters. For a list of all restrictions, see AWS Tag Restrictions: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html#tag-restrictions

[optional]

cluster_log_conf

Option<crate::models::ClusterLogConf>

[optional]

init_scripts

Option<Veccrate::models::InitScriptInfo>

The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If cluster_log_conf is specified, init script logs are sent to <destination>/<cluster-ID>/init_scripts.

[optional]

docker_image

Option<crate::models::DockerImage>

[optional]

runtime_engine

Option<String>

The type of runtime engine to use. If not specified, the runtime engine type is inferred based on the spark_version value. Allowed values include * PHOTON: Use the Photon runtime engine type. * STANDARD: Use the standard runtime engine type. This field is optional.

[optional]

spark_env_vars

Option<::std::collections::HashMap<String, serde_json::Value>>

An arbitrary object where the object key is an environment variable name and the value is an environment variable value.

[optional]

autotermination_minutes

Option<i32>

Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster is not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. You can also set this value to 0 to explicitly disable automatic termination.

[optional]

enable_elastic_disk

Option<bool>

Autoscaling Local Storage: when enabled, this cluster dynamically acquires additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly. Refer to Autoscaling local storage for details.

[optional]

instance_pool_id

Option<String>

The optional ID of the instance pool to which the cluster belongs. Refer to Pools for details.

[optional]

cluster_source

Option<crate::models::ClusterSource>

[optional]

policy_id

Option<String>

A cluster policy ID.

[optional]