oci_datascience_pipeline
This resource provides the Pipeline resource in Oracle Cloud Infrastructure Data Science service.
Creates a new Pipeline.
Example Usage
resource "oci_datascience_pipeline" "test_pipeline" {
#Required
compartment_id = var.compartment_id
project_id = oci_datascience_project.test_project.id
step_details {
#Required
step_name = var.pipeline_step_details_step_name
step_type = var.pipeline_step_details_step_type
#Optional
depends_on = var.pipeline_step_details_depends_on
description = var.pipeline_step_details_description
is_artifact_uploaded = var.pipeline_step_details_is_artifact_uploaded
job_id = oci_datascience_job.test_job.id
step_configuration_details {
#Optional
command_line_arguments = var.pipeline_step_details_step_configuration_details_command_line_arguments
environment_variables = var.pipeline_step_details_step_configuration_details_environment_variables
maximum_runtime_in_minutes = var.pipeline_step_details_step_configuration_details_maximum_runtime_in_minutes
}
step_container_configuration_details {
#Required
container_type = var.pipeline_step_details_step_container_configuration_details_container_type
image = var.pipeline_step_details_step_container_configuration_details_image
#Optional
cmd = var.pipeline_step_details_step_container_configuration_details_cmd
entrypoint = var.pipeline_step_details_step_container_configuration_details_entrypoint
image_digest = var.pipeline_step_details_step_container_configuration_details_image_digest
image_signature_id = oci_datascience_image_signature.test_image_signature.id
}
step_infrastructure_configuration_details {
#Optional
block_storage_size_in_gbs = var.pipeline_step_details_step_infrastructure_configuration_details_block_storage_size_in_gbs
shape_config_details {
#Optional
memory_in_gbs = var.pipeline_step_details_step_infrastructure_configuration_details_shape_config_details_memory_in_gbs
ocpus = var.pipeline_step_details_step_infrastructure_configuration_details_shape_config_details_ocpus
}
shape_name = oci_core_shape.test_shape.name
subnet_id = oci_core_subnet.test_subnet.id
}
}
#Optional
configuration_details {
#Required
type = var.pipeline_configuration_details_type
#Optional
command_line_arguments = var.pipeline_configuration_details_command_line_arguments
environment_variables = var.pipeline_configuration_details_environment_variables
maximum_runtime_in_minutes = var.pipeline_configuration_details_maximum_runtime_in_minutes
}
defined_tags = {"Operations.CostCenter"= "42"}
description = var.pipeline_description
display_name = var.pipeline_display_name
freeform_tags = {"Department"= "Finance"}
infrastructure_configuration_details {
#Required
block_storage_size_in_gbs = var.pipeline_infrastructure_configuration_details_block_storage_size_in_gbs
shape_name = oci_core_shape.test_shape.name
#Optional
shape_config_details {
#Optional
memory_in_gbs = var.pipeline_infrastructure_configuration_details_shape_config_details_memory_in_gbs
ocpus = var.pipeline_infrastructure_configuration_details_shape_config_details_ocpus
}
subnet_id = oci_core_subnet.test_subnet.id
}
log_configuration_details {
#Optional
enable_auto_log_creation = var.pipeline_log_configuration_details_enable_auto_log_creation
enable_logging = var.pipeline_log_configuration_details_enable_logging
log_group_id = oci_logging_log_group.test_log_group.id
log_id = oci_logging_log.test_log.id
}
}
Argument Reference
The following arguments are supported:
compartment_id
- (Required) (Updatable) The OCID of the compartment where you want to create the pipeline.configuration_details
- (Optional) (Updatable) The configuration details of a pipeline.command_line_arguments
- (Optional) (Updatable) The command line arguments to set for steps in the pipeline.environment_variables
- (Optional) (Updatable) Environment variables to set for steps in the pipeline.maximum_runtime_in_minutes
- (Optional) (Updatable) A time bound for the execution of the entire Pipeline. Timer starts when the Pipeline Run is in progress.type
- (Required) (Updatable) The type of pipeline.
defined_tags
- (Optional) (Updatable) Defined tags for this resource. Each key is predefined and scoped to a namespace. See Resource Tags. Example:{"Operations.CostCenter": "42"}
description
- (Optional) (Updatable) A short description of the pipeline.display_name
- (Optional) (Updatable) A user-friendly display name for the resource.freeform_tags
- (Optional) (Updatable) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. See Resource Tags. Example:{"Department": "Finance"}
infrastructure_configuration_details
- (Optional) (Updatable) The infrastructure configuration details of a pipeline or a step.block_storage_size_in_gbs
- (Required) (Updatable) The size of the block storage volume to attach to the instance.shape_config_details
- (Optional) (Updatable) Details for the pipeline step run shape configuration. Specify only when a flex shape is selected.memory_in_gbs
- (Optional) (Updatable) A pipeline step run instance of type VM.Standard.E3.Flex allows memory to be specified. This specifies the size of the memory in GBs.ocpus
- (Optional) (Updatable) A pipeline step run instance of type VM.Standard.E3.Flex allows the ocpu count to be specified.
shape_name
- (Required) (Updatable) The shape used to launch the instance for all step runs in the pipeline.subnet_id
- (Optional) (Updatable) The subnet to create a secondary vnic in to attach to the instance running the pipeline step.
log_configuration_details
- (Optional) (Updatable) The pipeline log configuration details.enable_auto_log_creation
- (Optional) (Updatable) If automatic on-behalf-of log object creation is enabled for pipeline runs.enable_logging
- (Optional) (Updatable) If customer logging is enabled for pipeline.log_group_id
- (Optional) (Updatable) The OCID of the log group.log_id
- (Optional) (Updatable) The OCID of the log.
project_id
- (Required) The OCID of the project to associate the pipeline with.step_details
- (Required) (Updatable) Array of step details for each step.depends_on
- (Optional) The list of step names this current step depends on for execution.description
- (Optional) (Updatable) A short description of the step.is_artifact_uploaded
- (Applicable when step_type=CONTAINER | CUSTOM_SCRIPT) A flag to indicate whether the artifact has been uploaded for this step or not.job_id
- (Required when step_type=ML_JOB) The OCID of the job to be used as a step.step_configuration_details
- (Optional) (Updatable) The configuration details of a step.command_line_arguments
- (Optional) (Updatable) The command line arguments to set for step.environment_variables
- (Optional) (Updatable) Environment variables to set for step.maximum_runtime_in_minutes
- (Optional) (Updatable) A time bound for the execution of the step.
step_container_configuration_details
- (Required when step_type=CONTAINER) Container Details for a step in pipeline.cmd
- (Optional) The container image run CMD as a list of strings. UseCMD
as arguments to theENTRYPOINT
or the only command to run in the absence of anENTRYPOINT
. The combined size ofCMD
andENTRYPOINT
must be less than 2048 bytes.container_type
- (Required) The type of container.entrypoint
- (Optional) The container image run ENTRYPOINT as a list of strings. Accept theCMD
as extra arguments. The combined size ofCMD
andENTRYPOINT
must be less than 2048 bytes. More information on howCMD
andENTRYPOINT
interact are here.image
- (Required) The full path to the Oracle Container Repository (OCIR) registry, image, and tag in a canonical format.image_digest
- (Optional) The digest of the container image. For example,sha256:881303a6b2738834d795a32b4a98eb0e5e3d1cad590a712d1e04f9b2fa90a030
image_signature_id
- (Optional) OCID of the container image signature
step_infrastructure_configuration_details
- (Applicable when step_type=CONTAINER | CUSTOM_SCRIPT) (Updatable) The infrastructure configuration details of a pipeline or a step.block_storage_size_in_gbs
- (Required when step_type=CONTAINER | CUSTOM_SCRIPT) (Updatable) The size of the block storage volume to attach to the instance.shape_config_details
- (Applicable when step_type=CONTAINER | CUSTOM_SCRIPT) (Updatable) Details for the pipeline step run shape configuration. Specify only when a flex shape is selected.memory_in_gbs
- (Applicable when step_type=CONTAINER | CUSTOM_SCRIPT) (Updatable) A pipeline step run instance of type VM.Standard.E3.Flex allows memory to be specified. This specifies the size of the memory in GBs.ocpus
- (Applicable when step_type=CONTAINER | CUSTOM_SCRIPT) (Updatable) A pipeline step run instance of type VM.Standard.E3.Flex allows the ocpu count to be specified.
shape_name
- (Required when step_type=CONTAINER | CUSTOM_SCRIPT) (Updatable) The shape used to launch the instance for all step runs in the pipeline.subnet_id
- (Applicable when step_type=CONTAINER | CUSTOM_SCRIPT) (Updatable) The subnet to create a secondary vnic in to attach to the instance running the pipeline step.
step_name
- (Required) (Updatable) The name of the step. It must be unique within the pipeline. This is used to create the pipeline DAG.step_type
- (Required) (Updatable) The type of step.
** IMPORTANT ** Any change to a property that does not support update will force the destruction and recreation of the resource with the new property values
Attributes Reference
The following attributes are exported:
compartment_id
- The OCID of the compartment where you want to create the pipeline.configuration_details
- The configuration details of a pipeline.command_line_arguments
- The command line arguments to set for steps in the pipeline.environment_variables
- Environment variables to set for steps in the pipeline.maximum_runtime_in_minutes
- A time bound for the execution of the entire Pipeline. Timer starts when the Pipeline Run is in progress.type
- The type of pipeline.
created_by
- The OCID of the user who created the pipeline.defined_tags
- Defined tags for this resource. Each key is predefined and scoped to a namespace. See Resource Tags. Example:{"Operations.CostCenter": "42"}
description
- A short description of the pipeline.display_name
- A user-friendly display name for the resource.freeform_tags
- Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. See Resource Tags. Example:{"Department": "Finance"}
id
- The OCID of the pipeline.infrastructure_configuration_details
- The infrastructure configuration details of a pipeline or a step.block_storage_size_in_gbs
- The size of the block storage volume to attach to the instance.shape_config_details
- Details for the pipeline step run shape configuration. Specify only when a flex shape is selected.memory_in_gbs
- A pipeline step run instance of type VM.Standard.E3.Flex allows memory to be specified. This specifies the size of the memory in GBs.ocpus
- A pipeline step run instance of type VM.Standard.E3.Flex allows the ocpu count to be specified.
shape_name
- The shape used to launch the instance for all step runs in the pipeline.subnet_id
- The subnet to create a secondary vnic in to attach to the instance running the pipeline step.
lifecycle_details
- A message describing the current state in more detail. For example, can be used to provide actionable information for a resource in ‘Failed’ state.log_configuration_details
- The pipeline log configuration details.project_id
- The OCID of the project to associate the pipeline with.state
- The current state of the pipeline.step_details
- Array of step details for each step.depends_on
- The list of step names this current step depends on for execution.description
- A short description of the step.is_artifact_uploaded
- A flag to indicate whether the artifact has been uploaded for this step or not.job_id
- The OCID of the job to be used as a step.step_configuration_details
- The configuration details of a step.command_line_arguments
- The command line arguments to set for step.environment_variables
- Environment variables to set for step.maximum_runtime_in_minutes
- A time bound for the execution of the step.
step_container_configuration_details
- Container Details for a step in pipeline.cmd
- The container image run CMD as a list of strings. UseCMD
as arguments to theENTRYPOINT
or the only command to run in the absence of anENTRYPOINT
. The combined size ofCMD
andENTRYPOINT
must be less than 2048 bytes.container_type
- The type of container.entrypoint
- The container image run ENTRYPOINT as a list of strings. Accept theCMD
as extra arguments. The combined size ofCMD
andENTRYPOINT
must be less than 2048 bytes. More information on howCMD
andENTRYPOINT
interact are here.image
- The full path to the Oracle Container Repository (OCIR) registry, image, and tag in a canonical format.image_digest
- The digest of the container image. For example,sha256:881303a6b2738834d795a32b4a98eb0e5e3d1cad590a712d1e04f9b2fa90a030
image_signature_id
- OCID of the container image signature
step_infrastructure_configuration_details
- The infrastructure configuration details of a pipeline or a step.block_storage_size_in_gbs
- The size of the block storage volume to attach to the instance.shape_config_details
- Details for the pipeline step run shape configuration. Specify only when a flex shape is selected.memory_in_gbs
- A pipeline step run instance of type VM.Standard.E3.Flex allows memory to be specified. This specifies the size of the memory in GBs.ocpus
- A pipeline step run instance of type VM.Standard.E3.Flex allows the ocpu count to be specified.
shape_name
- The shape used to launch the instance for all step runs in the pipeline.subnet_id
- The subnet to create a secondary vnic in to attach to the instance running the pipeline step.
step_name
- The name of the step. It must be unique within the pipeline. This is used to create the pipeline DAG.step_type
- The type of step.
system_tags
- Usage of system tag keys. These predefined keys are scoped to namespaces. Example:{"orcl-cloud.free-tier-retained": "true"}
time_created
- The date and time the resource was created in the timestamp format defined by RFC3339. Example: 2020-08-06T21:10:29.41Ztime_updated
- The date and time the resource was updated in the timestamp format defined by RFC3339. Example: 2020-08-06T21:10:29.41Z
Timeouts
The timeouts
block allows you to specify timeouts for certain operations:
* create
- (Defaults to 20 minutes), when creating the Pipeline
* update
- (Defaults to 20 minutes), when updating the Pipeline
* delete
- (Defaults to 20 minutes), when destroying the Pipeline
Import
Pipelines can be imported using the id
, e.g.
$ terraform import oci_datascience_pipeline.test_pipeline "id"