Bring Your Own Container
You can build and use a custom container (Bring Your Own Container or BYOC) for use when you create a job and job runs.
The maximum size of a container image that you can use with jobs is 25 GB. The size of the container image slows down the provisioning time for the job because of the container pull from the Container Registry. We recommend that you use the smallest container images possible.
Dockerfile
, and build an
image. You start with a Dockerfile
, that uses the Python slim
image. The Dockerfile
is designed so that you can make local and
remote builds. You use the local build when you test locally against the code.
During the local development, you don't need to build a new image for every code
change. Don't store files under
/home/datascience
directory. When the container is
provisioned and executed as a job run, they're overridden when in this
directory.Use the remote option to run the Dockerfile
when you think the code is complete, and you want to run it as a job as in this Dockerfile
example:
ARG type
FROM python:3.8-slim AS base
ENV DATASCIENCE_USER datascience
ENV DATASCIENCE_UID 1000
ENV HOME /home/$DATASCIENCE_USER
RUN python -m pip install \
parse \
oci
FROM base AS run-type-local
# nothing to see here
FROM base AS run-type-remote
COPY job_logs.py .
CMD ["python", "job_logs.py"]
FROM run-type-${type} AS final
Following is a sample job_logs.py
file:
import oci
import datetime
import os
import time
import sys
import uuid
from oci.loggingingestion import LoggingClient
from oci.loggingingestion.models import PutLogsDetails, LogEntryBatch, LogEntry
OCI_RESOURCE_PRINCIPAL_VERSION = "OCI_RESOURCE_PRINCIPAL_VERSION"
JOB_RUN_OCID_KEY = "JOB_RUN_OCID"
# switch
class Job:
def __init__(self):
rp_version = os.environ.get(OCI_RESOURCE_PRINCIPAL_VERSION, "UNDEFINED")
if not rp_version or rp_version == "UNDEFINED":
# RUN LOCAL TEST
self.signer = oci.config.from_file("~/.oci/config", "BIGDATA")
else:
# RUN AS JOB
self.signer = oci.auth.signers.get_resource_principals_signer()
job = Job()
print(
"Start logging for job run: {}".format(
os.environ.get(JOB_RUN_OCID_KEY, "LOCAL")
)
)
print("Current timestamp in UTC: {}".format(str(datetime.datetime.utcnow())))
print("Delay 5s")
time.sleep(5)
print("... another stdout")
print("Print all environment variables and values")
for item, value in os.environ.items():
print("{}: {}".format(item, value))
print("Docker Job Done.")
Before you can push and pull images to and from Oracle Cloud Infrastructure Registry (also known as Container Registry), you must have an Oracle Cloud Infrastructure authorization token. You only see the auth token string when you create it, so be sure to copy the auth token to a secure location immediately.