Oracle Cloud Infrastructure Documentation

The runtime.yaml File

In addition to score.py, the runtime.yaml file is required to be in a model artifact.

The purpose of runtime.yaml is:

To provide the necessary runtime conda environment reference for model deployment purposes. This is required to deploy a model using the model deployment feature of the Data Science service.

We've provided various model catalog examples including runtime.yaml files.

Following is a description of each field in runtime.yaml. As the model artifact structure changes, this file evolves and so does the version. These are the fields for the current MODEL_ARTIFACT_VERSION (3.0).

Field Description

MODEL_ARTIFACT_VERSION

The version of this artifact format. This version is 3.0. This is automatically extracted by ADS when the model is saved in a notebook session.

MODEL_DEPLOYMENT.INFERENCE_CONDA_ENV.INFERENCE_ENV_SLUG

The slug of the conda environment you want to use for deployment and scoring purposes. Usually, the inference environment is the same as the training environment thought that doesn't have to be the case. For example, you can train a model in one conda environment, serialize it as an ONNX model, and then use an ONNX conda environment to deploy it.

MODEL_DEPLOYMENT.INFERENCE_CONDA_ENV.INFERENCE_ENV_TYPE

The type of the conda environment you want to use for deployment and scoring purposes. Two possible values either data_science or published.

MODEL_DEPLOYMENT.INFERENCE_CONDA_ENV.INFERENCE_ENV_PATH

The path on Object Storage of the conda environment you want to use for deployment and scoring purposes. The path follows this syntax, oci://<bucket-name>@<namespace>/<file-path>.

MODEL_DEPLOYMENT.INFERENCE_CONDA_ENV.INFERENCE_PYTHON_VERSION

The Python version of the conda environment you want to use for model deployment. The supported Python versions are 3.7, 3.8, 3.9, 3.10, and 3.11.

runtime.yaml Schema

{
    "title": "Model Artifact schema",
    "type": "object",
    "properties": {
        "MODEL_ARTIFACT_VERSION": {"type":"string"},
        "MODEL_DEPLOYMENT": {
            "type": "object",
            "title": "Model Deployment",
            "properties": {
                "INFERENCE_CONDA_ENV": {
                    "type": "object",
                    "title": "Inference Conda Env",
                    "properties": {
                        "INFERENCE_ENV_SLUG": {"type":"string"},
                        "INFERENCE_ENV_TYPE": {"type":"string", "enum": ["published", "data_science"]},
                        "INFERENCE_ENV_PATH": {"type":"string"},
                        "INFERENCE_PYTHON_VERSION": {"type":"string"}
                    }
                }
            }
        }
    }
 
}

Example File for Model Deployment

This is a example runtime.yaml file for a Data Science model deployment resource:

MODEL_ARTIFACT_VERSION: '3.0'
MODEL_DEPLOYMENT:
  INFERENCE_CONDA_ENV:
    INFERENCE_ENV_SLUG: envslug
    INFERENCE_ENV_TYPE: published
    INFERENCE_ENV_PATH: oci://<bucket-name>@<namespace>/<prefix>/<env>.tar.gz
    INFERENCE_PYTHON_VERSION: '3.7'

Other Artifact Files

In addition to score.py and runtime.yaml, you can include any more files that are necessary to run your model in your artifact. These might include:

  • A serialized representation of your estimator object. For example, onnx, pkl, hdf5, or json.

  • A CSV file, a lookup data table, and so on.

  • Extra Python modules that are imported in score.py.
    Important

    Extra Python modules that are imported in score.py. Any code used for inference should be zipped at the same level as score.py or any level below that. If any required files are present at folder levels above the score.py file, they're ignored and could result in deployment failure.

If you're saving your model using the OCI Console, CLI, or SDKs, just zip the files along with the score.py and runtime.yaml files.

If you're using ADS to save your model, then copy all the files in the directory where the artifact is temporarily created before saving the model to the catalog. ADS compresses the entire artifact directory and sends it to the model catalog.

Note

The artifact now supports uncompressed models up to 6 GB when the model is saved using ADS, the CLI, or SDKs. The limit is 100 MB when using the OCI Console.