Logs

Job Logs

You can integrate jobs resources with the Logging service to record job run details in a custom log.

Both standard out (stdout) and standard error (stderr) outputs from your job artifact are captured and made available in the custom log. These outputs aren't available to the Data Science service. We highly recommended that you enable the logging integration for jobs, both for debugging any potential issues and for monitoring the progress of running your job artifacts.

Standard out messages are categorized under the "com.oraclecloud.datascience.jobrun.stdout" type, while standard error messages are under the "com.oraclecloud.datascience.jobrun.stderr" type. For both types, the log metadata includes the "source", which is the OCID of the job run that outputs the messages.

When logging integration is enabled in the job logging configurations, you can either:

• Provide your own log group and your own log to capture job run outputs. You can configure all the job runs to use the same log though we don't recommended it. Log messages are easier to read when each job run outputs to its own log.

  Or

• Provide your own log group, but enable the Data Science service to automatically create custom logs on your behalf at the start of each job run within your specified log group. If automatic log creation is enabled, there is no need for you to configure logs yourself before each job run. The log created is named in the jobrun-<partial-jobrun-ocid>-logs, but the name can be edited as needed. It doesn't impact the logging integration with the job run.

For either type of logging configuration, the job run resource principal must have permissions to write to your custom logs.

Model Deployment Logs

When you integrate the Logging for model deployments, access and predict log categories are created:

Access Logs

The access log category is a custom log that captures detailed information about requests sent to the model endpoint. This information includes the time the request was received, error codes, and so on. If you're experiencing problems with your model deployment, access logs are generally the first log you want to look at.

The metadata captured by model deployment access logs are:

logEmissionTime

The time, in UTC, when the log was emitted from the code.

message

The request path.

modelLatency

The time taken to process a request on the model server in milliseconds.

opcRequestId

The request Id. This value is the same requestId that's retrieved in the response.

status

The request response status code.

For example, an access log entry for model deployment could be:

"data": {
  "logEmissionTime": "2021-01-25T07:23:39.101Z",
  "message": "POST /predict 1.1",
  "modelLatency": 4.43,
  "opcRequestId": "0BC0860C17DC46D79A0A1A7B4F139829",
  "status": 200
}

Predict Logs

Predict logs originate from logging (stdout and stderr) calls made custom code execution in the model artifact Python files. Predict logs can emit useful information about the model and are entirely customizable. Configuring access and predict logs is part of the create and edit actions.

The metadata captured by model deployment predict logs are:

MD_OCID

The model deployment OCID value.

level

The logger level. Also referred to as the severity level of the log message.

logEmissionTime

Time, in UTC, when the log was emitted from the code.

message

A custom message emitted from the model artifact Python code.

name

The name of the logger used.

For example, a predict log entry for model deployment could be:

"data": {
      "MD_OCID": "ocid1.datasciencemodeldeployment.oc1.iad.amaaaaaav66vvniauqakarfnyvn6gd2qt4fjpv2ffdnrthqyhqpapevmmh6a",
      "level": "ERROR",
      "logEmissionTime": "2021-01-27T08:43:04.029Z",
      "message": "exception :: name 'function' is not defined",
      "name": "root"
    },

You can create two separate logs for predict and access or you can use the same one for both.