RunSubmitTaskSettings
Last updated
Last updated
task_key
String
A unique name for the task. This field is used to refer to this task from other tasks. This field is required and must be unique within its parent job. On Update or Reset, this field is used to reference the tasks to be updated or reset. The maximum length is 100 characters.
depends_on
Option<>
An optional array of objects specifying the dependency graph of the task. All tasks specified in this field must complete successfully before executing this task. The key is task_key
, and the value is the name assigned to the dependent task. This field is required when a job consists of more than one task.
[optional]
existing_cluster_id
Option<String>
If existing_cluster_id, the ID of an existing cluster that is used for all runs of this task. When running tasks on an existing cluster, you may need to manually restart the cluster if it stops responding. We suggest running jobs on new clusters for greater reliability.
[optional]
new_cluster
Option<>
[optional]
notebook_task
Option<>
[optional]
spark_jar_task
Option<>
[optional]
spark_python_task
Option<>
[optional]
spark_submit_task
Option<>
[optional]
pipeline_task
Option<>
[optional]
python_wheel_task
Option<>
[optional]
sql_task
Option<>
[optional]
dbt_task
Option<>
[optional]
libraries
Option<>
An optional list of libraries to be installed on the cluster that executes the task. The default value is an empty list.
[optional]
timeout_seconds
Option<i32>
An optional timeout applied to each run of this job task. The default behavior is to have no timeout.
[optional]