Deployment using OCI Data Science Job
=======================================

ML Monitoring Application can be quickly set up using OCI Data Science Job and ML Monitoring Application managed container image.

- Configure required policy and dynamic rules
- Provision OCI Data Science Job
- Trigger Data science job run passing the required runtime configuration

Add Dynamic Groups Rules
--------------------------
User needs to create Dynamic Groups and add following rules for setting up OCI resources

.. code-block::

    ALL {resource.type='datasciencejobrun', resource.compartment.id = '<COMPARTMENT_ID>'}


Here COMPARTMENT_ID is the OCID of Compartment in which user need to setup OCI Data Science Job resource and push logs for ML Monitoring Application run .

.. seealso::
    `Dynamic Groups <https://docs.oracle.com/en-us/iaas/Content/Identity/getstarted/identity-domains.htm#identity_documentation>`_

Add Policy Rules
----------------------
Policy rules required to be added for authorizing and giving access permission to OCI resources within user’s compartment via dynamic-group . Here user is giving required permission using following policy to all the resources defined in the dynamic group created above.

.. code-block::

    Allow dynamic-group <ml-monitoring-application-dynamic-group> to manage log-groups in compartment <COMPARTMENT NAME>
    Allow dynamic-group <ml-monitoring-application-dynamic-group> to use log-content in compartment <COMPARTMENT NAME>
    Allow dynamic-group <ml-monitoring-application-dynamic-group> to manage objects in compartment <COMPARTMENT NAME>
    Allow dynamic-group <ml-monitoring-application-dynamic-group> to read buckets in compartment <COMPARTMENT NAME>


Configure OCI Data Science Job
----------------------
Steps to setup OCI Data Science Job :

- Create OCI Data Science Job using the `link <https://docs.oracle.com/iaas/data-science/using/jobs-create.htm#jobs_creating>`_.

    .. note::

        - Do not use BYOC while creating the job for ML Monitoring Application.
        - Do not use Fast Job. Only Custom Configuration is supported.
        - Do not use ARM shapes. Only Intel / AMD shapes are supported.


- Set the below environment variable while configuring the job:
    .. list-table::
       :widths: 25 15 25
       :header-rows: 1

       * - Key
         - Description
         - Value Example
       * - CONTAINER_CUSTOM_IMAGE
         - Managed Container moniker for ML Monitoring Application. `Available versions <../release_notes.html>`_
         - .. code-block:: bash

                  "dsmc://odsc-ml-monitoring-application:<version>"


Example

    .. image:: ../resources/job_created.png
       :width: 800
       :alt: Partition Data

    .. image:: ../resources/job_default_config.png
       :width: 800
       :alt: Partition Data


Run OCI Data Science Job
-------------------------------

After the OCI Data Science Job is created using ML Monitoring Application managed container, we can use OCI Data Science Job Run to repeatedly run the job.

.. seealso::
    `How to configure ML Monitoring Application config file <../config_setup.html>`_

Steps to configure OCI Data Science Job Run :

- Create OCI Data Science Job Run using the `link <https://docs.oracle.com/iaas/data-science/using/job-runs.htm>`_.
- Override the default job configuration that was defined when the job was created or add new runtime job configuration
- Refer the `input contract <../user_guide/input_configuration.html>`_ to add runtime configuration

Example

    .. image:: ../resources/job_run_created.png
       :width: 800
       :alt: Partition Data

    .. image:: ../resources/job_run_runtime_config.png
       :width: 800
       :alt: Partition Data


Other ways of creating Job and Job Runs
----------------------------------------
**CLI**

Use the `Data Science CLI <https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.40.2/oci_cli_docs/cmdref/data-science/job/create.html>`_ to create a job as in this example:

1. Create a job with:
    .. code-block:: text

	oci data-science job create \
	--display-name <job_name>\
	--compartment-id <compartment_ocid>\
	--project-id <project_ocid> \
	--configuration-details file://<jobs_configuration_json_file> \
	--infrastructure-configuration-details file://<jobs_infrastructure_configuration_json_file> \
	--log-configuration-details file://<optional_jobs_infrastructure_configuration_json_file>

2. Use this jobs configuration JSON file:
    .. code-block:: json

	{
	  "jobType": "DEFAULT",
	  "environmentVariables": {
		"CONTAINER_CUSTOM_IMAGE": "dsmc://odsc-ml-monitoring-application:<version>"
	  }
	}

3. Use this jobs infrastructure configuration JSON file:
    .. code-block:: json

	{
	  "jobInfrastructureType":"ME_STANDALONE",
	  "shapeName": "<shape_name>",
	  "blockStorageSizeInGBs": "block_storage_size_in_GBs"
	}

4. (Optional) Use this jobs logging configuration JSON file:
    .. code-block:: json

	{
	  "enableLogging": true,
	  "enableAutoLogCreation": true,
	  "logGroupId": "<log_group_ocid>"
	}

5. Create a job run with:
    .. code-block:: text

	oci data-science job-run create \
	--display-name <job_name>\
	--compartment-id <compartment_ocid>\
	--project-id <project_ocid> \
	--job-id <job_ocid> \
	--configuration-override-details file://<jobs_override_configuration_json_file>

6. Use this job run override configuration JSON file:
    .. code-block:: json

	{
	  "jobType": "DEFAULT",
	  "environmentVariables": {
		"CONFIG_LOCATION": "<HTTP_LOCATION_OF_OCI_STORAGE_APPLICATION_CONFIG_FILE_PATH>"
		"RUNTIME_PARAMETER": "{"ACTION_TYPE":"RUN_CONFIG_VALIDATION"}"
	  }
	}

**API**

The ADS SDK is also a publicly available Python library that you can install with this command:

    .. code-block:: text

	pip install oracle-ads


It provides the wrapper that makes the creation and running jobs from notebooks or on your client machine easy.

Use the `ADS SDK <https://accelerated-data-science.readthedocs.io/en/latest/user_guide/jobs/data_science_job.html>`_ to create job and run jobs

.. collapse:: job_and_job_run_example.py

	.. code-block:: code

		import ads
		import os
		from ads.jobs import Job, DataScienceJob, PythonRuntime

		# Setting OCI_ODSC_SERVICE_ENDPOINT
		os.environ['OCI_ODSC_SERVICE_ENDPOINT'] = 'https://datascience-int.us-ashburn-1.oci.oc-test.com'

		def create_job_and_job_run():
			rp_version = os.environ.get("OCI_RESOURCE_PRINCIPAL_VERSION", "UNDEFINED")

			# Setting authentication

			# Setting local authentication
			if not rp_version or rp_version == "UNDEFINED":
				ads.set_auth("security_token", oci_config_location="~/.oci/config", profile="DEFAULT")
			else:
				# Setting Resource Principal authentication
				ads.set_auth("resource_principal")

			job = (
				Job(name="Job_creation_using_Ads")
					.with_infrastructure(
					DataScienceJob()
						.with_compartment_id(COMPARTMENT_OCID)
						.with_project_id(PROJECT_OCID)
						.with_job_type('DEFAULT')
						.with_shape_name("SHAPE_NAME")
						.with_block_storage_size(BLOCK_STORAGE_SIZE)
						.with_log_group_id(LOG_GROUP_OCID)
						.with_job_infrastructure_type('ME_STANDALONE')
				)
					.with_runtime(
					PythonRuntime()
						.with_environment_variable(CONTAINER_CUSTOM_IMAGE="dsmc://odsc-ml-monitoring-application:<version>")
				)
			)

			# Create Job
			job.create()

			# Create Job Run
			job_run = job.run(
				name="ML_Monitoring_Validation_Run_ads_1",
				env_var={
					"CONFIG_LOCATION": "<HTTP_LOCATION_OF_OCI_STORAGE_APPLICATION_CONFIG_FILE_PATH>",
					"RUNTIME_PARAMETER": "{\"ACTION_TYPE\":\"RUN_CONFIG_VALIDATION\"}"
				}
			)


**Terraform**

Use the `Data Science Create Job Terraform <https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/datascience_job>`_ to create a job.

Use the `Data Science Create Job Run Terraform <https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/datascience_job_run>`_ to create a job run.