Install OCI Service Operator for Kubernetes for Service Mesh

The OCI Service Operator for Kubernetes makes it easy to create, manage, and connect to OCI resources from a Kubernetes environment. Kubernetes users can simply install OCI Service Operator for Kubernetes and perform actions on OCI resources using the Kubernetes API. The OCI Service Operator for Kubernetes removes the need to use the OCI CLI or other OCI developer tools to interact with a service API.

OCI Service Operator for Kubernetes is based on the Operator Framework, an open source toolkit used to manage Operators. It uses the controller-runtime library, which provides high-level APIs and abstractions to write operational logic and also provides tools for scaffolding and code generation for Operators.

For more information on the Operator SDK, see the Operator Framework site.

Install the Operator SDK and OLM

Install the Operator SDK on your local machine using the following steps.

  • Install the Operator SDK

    Go to the Operator SDK installation page and follow the installation instructions for your operating system.

  • Install the Operator Lifecycle Manager (OLM). The OLM helps users install, update, and manage the lifecycle of Kubernetes native applications (Operators) and their associated services running in clusters. Use the following command to install:
    operator-sdk olm install --version 0.20.0
    Note

    Local access to your Kubernetes cluster must be set up on your machine before you can perform this step.
  • Verify your installation of OLM by first checking for all the necessary Customer Resource Definitions (CRDs) in the cluster. Run the following command.
    operator-sdk olm status

    The command produces output similar to the following:

    INFO[0003] Fetching CRDs for version "v0.20.0"          
    INFO[0003] Fetching resources for resolved version "v0.20.0" 
    INFO[0021] Successfully got OLM status for version "v0.20.0" 
    
    NAME                                            NAMESPACE    KIND                        STATUS
    operatorgroups.operators.coreos.com                          CustomResourceDefinition    Installed
    operatorconditions.operators.coreos.com                      CustomResourceDefinition    Installed
    olmconfigs.operators.coreos.com                              CustomResourceDefinition    Installed
    installplans.operators.coreos.com                            CustomResourceDefinition    Installed
    clusterserviceversions.operators.coreos.com                  CustomResourceDefinition    Installed
    olm-operator-binding-olm                                     ClusterRoleBinding          Installed
    operatorhubio-catalog                           olm          CatalogSource               Installed
    olm-operators                                   olm          OperatorGroup               Installed
    aggregate-olm-view                                           ClusterRole                 Installed
    catalog-operator                                olm          Deployment                  Installed
    cluster                                                      OLMConfig                   Installed
    operators.operators.coreos.com                               CustomResourceDefinition    Installed
    olm-operator                                    olm          Deployment                  Installed
    subscriptions.operators.coreos.com                           CustomResourceDefinition    Installed
    aggregate-olm-edit                                           ClusterRole                 Installed
    olm                                                          Namespace                   Installed
    global-operators                                operators    OperatorGroup               Installed
    operators                                                    Namespace                   Installed
    packageserver                                   olm          ClusterServiceVersion       Installed
    olm-operator-serviceaccount                     olm          ServiceAccount              Installed
    catalogsources.operators.coreos.com                          CustomResourceDefinition    Installed
    system:controller:operator-lifecycle-manager                 ClusterRole                 Installed

Deploy OCI Service Operator for Kubernetes

The OCI Service Operator for Kubernetes OLM bundle contains all the required details like CRDs, RBACs, and ConfigMaps. Deployment installs the OCI Service Operator for Kubernetes in the Kubernetes cluster. First, create a namespace for your operator.

kubectl create ns oci-service-operator-system
Note

If deployed in your application namespace, the operator functions normally. This option is an alternative to creating an operator namespace.
Note

For disconnected realms, build your own OSOK Operator by following the steps from https://github.com/oracle/oci-service-operator/blob/main/BUILD_OSOK_OPERATOR.md. Replace the Bundle in the following command with the one that was built manually to install OSOK Operator.

Install the OCI Service Operator for Kubernetes Operator in the Kubernetes cluster in your namespace (oci-service-operator-system) using the following command.

operator-sdk run bundle iad.ocir.io/oracle/oci-service-operator-bundle:X.X.X -n oci-service-operator-system --timeout 5m
Note

Replace X.X.X with current version of the OCI Service Operator for Kubernetes. To get the current version, go to the GitHub release site at: https://github.com/oracle/oci-service-operator/releases
Note

Users must be logged into the Oracle Registry at iad.ocir.io in Docker to run the command. To ensure you are logged in, see Pulling Images Using the Docker CLI.

The command produces output similar to the following:

INFO[0036] Successfully created registry pod: iad-ocir-io-oracle-oci-service-operator-bundle-X-X-X 
INFO[0036] Created CatalogSource: oci-service-operator-catalog 
INFO[0037] OperatorGroup "operator-sdk-og" created      
INFO[0037] Created Subscription: oci-service-operator-vX-X-X-sub 
INFO[0040] Approved InstallPlan install-tzk5f for the Subscription: oci-service-operator-vX-X-X-sub 
INFO[0040] Waiting for ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" to reach 'Succeeded' phase 
INFO[0040]   Waiting for ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" to appear 
INFO[0048]   Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: Pending 
INFO[0049]   Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: InstallReady 
INFO[0053]   Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: Installing 
INFO[0066]   Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: Succeeded 
INFO[0067] OLM has successfully installed "oci-service-operator.vX.X.X"

To verify the installation, perform the following steps.

  1. Check for custom definitions with kubectl.
    $ kubectl get crds | grep "servicemesh.oci.oracle.com" | awk -F. '{print $1}'
    accesspolicies
    ingressgatewaydeployments
    ingressgatewayroutetables
    ingressgateways
    meshes
    virtualdeploymentbindings
    virtualdeployments
    virtualserviceroutetables
    virtualservices
  2. Verify the successful installation of webhooks corresponding to all the Service Mesh resources with kubectl.

    First, check validating webhooks.

    $ kubectl get validatingwebhookconfiguration | grep "servicemesh.oci.oracle.cloud.com" | awk -F. '{print $1}'
    ap-validator
    ig-validator
    igd-validator
    igrt-validator
    mesh-validator
    vd-validator
    vdb-validator
    vs-validator
    vsrt-validator

    Next, check mutating webhooks.

    $ kubectl get mutatingwebhookconfiguration | grep "servicemesh.oci.oracle.com" | awk -F. '{print $1}'
    proxy-injector

For more information on troubleshooting, see: Troubleshooting Service Mesh.

Install Metrics Server

If you are going to use Ingress Gateway, install metrics server. The Kubernetes Horizontal Pod Autoscalar uses the metrics server to scale the number of pods in the ingress gateway based on CPU usage. The metrics server is installed using the following command:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/high-availability.yaml

Bring your own Certificate for Service Operator (Optional)

As an alternative, bring your own certificate (BYOC) to secure communications. Kubernetes provides a certificates.k8s.io API, which lets you provision TLS certificates signed by a Certificate Authority (CA) that you control. The CA and certificates are used by your workloads to establish trust. To establish trust between OCI Service Operator for Kubernetes and Kubernetes, use certificates.k8s.io to create the CA and certificates. Patch the oci-service-operator-controller-manager-service-cert secret to use the new CA and certificates.

For more information, see: Manage TLS Certificates in a Cluster

To generate the CA and CertificateBundle use tools like cfssl(Cloudfare SSL) or OpenSSL. To use a user created CA, install the default configuration and patch the kubernetes.io/tls generated by OLM for OCI Service Operator for Kubernetes. The following steps demonstrate the configuration steps using CFSSL.

Note

In the following steps, replace {operators-namespace} with the namespace your operator runs in.
  1. Create a Certificate Signing Request(CSR) and private key with cfssl following steps:
    Note

    For more information on these steps, see: Create a certificate signing request.
    cat <<EOF | cfssl genkey - | cfssljson -bare server
    {
      "hosts": [
        "oci-service-operator-webhook-service.{operators-namespace}.svc.cluster.local",
        "oci-service-operator-controller-manager-service.{operators-namespace}.svc.cluster.local"
      ],
      "CN": "kube.operator.cert",
      "key": {
        "algo": "ecdsa",
        "size": 256
      }
    }
    EOF

    This command generates two files. The server.csr file contains the PEM encoded CSR. The server-key.pem file contains the PEM encoded key to the certificate that is yet to be created.

  2. Create CSR manifest and send it to Kubernetes following section.
    Note

    For more information on these steps, see: Create a CertificateSigningRequest object to send to the Kubernetes API.
    cat <<EOF | kubectl apply -f -
    apiVersion: certificates.k8s.io/v1
    kind: CertificateSigningRequest
    metadata:
      name: oci-service-operator-controller.{operators-namespace}
    spec:
      request: $(cat server.csr | base64 | tr -d '\n')
      signerName: oci.com
      usages:
      - digital signature
      - key encipherment
      - server auth
    EOF

    After running this command, the CSR created is in a pending state.

    kubectl describe csr {csr-name}

    Approve the CSR.

    kubectl certificate approve {csr-name}

    Validate the state after approving. Confirm that the value is: CONDITION-Approved.

  3. Sign the CSR.
    Note

    For more information on these steps, see: Sign the CertificateSigningRequest.
    1. Create a certificate authority key file (ca-key.pem) and certificate (ca.pem).
      cat <<EOF | cfssl gencert -initca - | cfssljson -bare ca
      {
        "CN": "My Example Signer",
        "key": {
          "algo": "rsa",
          "size": 2048
        }
      }
      EOF
    2. Issue a certificate by creating server signing request. Create the server-signing-config.json file.
      {
          "signing": {
              "default": {
                  "usages": [
                      "digital signature",
                      "key encipherment",
                      "server auth"
                  ],
                  "expiry": "876000h",
                  "ca_constraint": {
                      "is_ca": false
                  }
              }
          }
      }

      Use the server-signing-config.json signing configuration, the certificate authority key file, and certificate file to sign the certificate request:

      kubectl get csr oci-service-operator-controller.{operators-namespace} -o jsonpath='{.spec.request}' | \
        base64 --decode | \
        cfssl sign -ca ca.pem -ca-key ca-key.pem -config server-signing-config.json - | \
        cfssljson -bare ca-signed-server
    3. Upload signed certificate.
      kubectl get csr oci-service-operator-controller.{operators-namespace} -o json | \
        jq '.status.certificate = "'$(base64 ca-signed-server.pem | tr -d '\n')'"' | \
        kubectl replace --raw /apis/certificates.k8s.io/v1/certificatesigningrequests/oci-service-operator-controller.{operators-namespace}/status -f -

      After applying the preceding change, validate the CSR condition is "Approved, Issued."

    4. Download the certificate.
      kubectl get csr {csr-name} -o jsonpath='{.status.certificate}' \
          | base64 --decode > server.crt

      Running preceding command creates the server.crt file.

  4. Encode and patch server.crt from step (3)(d), the ca-key.pem file from step (3)(a), and the server-key.pem file from step (1).
    kubectl patch secret oci-service-operator-controller-manager-service-cert --namespace {operators-namespace} --type merge --patch }}{{'{"data":  "olmCAKey": "'$(base64 ca.pem | tr -d }}{{'\n')'", "tls.key": "'$(base64 server-key.pem | tr -d }}{{'\n')'", "tls.crt": "'$(base64 ca-signed-server.pem | tr -d }}{{'\n')'"}'
  5. Validate oci-service-operator-controller-manager-service-cert contains the latest values.
  6. With the preceding steps complete, the resources are created and working. However, if you get an error: x509: certificate signed by unknown authority, review the following steps to troubleshoot.
    1. Validate if the CSR has a condition of "Approved, Issued." Otherwise, start again from step (3)(c).
    2. Validate if base64 decoded olmCAKey matches the server.crt from step (3)(d). Otherwise, update the olmCAKey with the server.crt file.
    3. Validate if base64 decoded tls.key matches the server-key.pem file from step (1). Otherwise, update the tls.key with the server-key.pem file.
    4. Validate if base64 decoded tls.crt matches the ca-key.pem file from step (3)(a). Otherwise, update tls.crt with the ca-key.pem file.

Set up Resources Required for Service Mesh

Set up Steps

To start using Service Mesh, you have a few requirements to fulfill before you can create your own meshes.

If you are not an administrator, ask your administrator to give you access to resources of three services: Service Mesh, Vault, and Certificates. Your administrator can give you access to these services with Identity and Access Management (IAM) policies.

To prepare your tenancy to use Service Mesh, perform the following steps with your OCI account.

  1. Create a vault to store the encryption key for your certificate authority. See Managing Vaults for details.

    Add a Hardware Security Module (HSM) encryption key for your vault. See Managing Keys for details. For your certificate authority key, you can use a Rivest-Shamir-Adleman (RSA) 2,048-bit key or 4,096-bit key.

  2. Create a certificate authority. See Managing Certificate Authorities for details.
    Note

    The Maximum Validity Duration for Certificates (Days) setting on the certificate authority must be at least 45 days.
  3. Create a TLS certificate so clients can make TLS connections to your application. This certificate keeps HTTP requests to ingress gateways TLS encrypted. See Managing Certificates for details.

More Information

For more information on IAM policies related to Service Mesh, see:

Note

If you need to remove Service Mesh support from your application, see the Unmeshify Your Application (Optional) section in Service Mesh Overview Tutorial.

Next: Set up Policies required for Service Mesh