Install OCI Service Operator for Kubernetes for Service Mesh
The OCI Service Operator for Kubernetes makes it easy to create, manage, and connect to OCI resources from a Kubernetes environment. Kubernetes users can simply install OCI Service Operator for Kubernetes and perform actions on OCI resources using the Kubernetes API. The OCI Service Operator for Kubernetes removes the need to use the OCI CLI or other OCI developer tools to interact with a service API.
OCI Service Operator for Kubernetes is based on the Operator Framework, an open source toolkit used to manage Operators. It uses the controller-runtime library, which provides high-level APIs and abstractions to write operational logic and also provides tools for scaffolding and code generation for Operators.
For more information on the Operator SDK, see the Operator Framework site.
Install the Operator SDK and OLM
Install the Operator SDK on your local machine using the following steps.
- Install the Operator SDK
Go to the Operator SDK installation page and follow the installation instructions for your operating system.
- Install the Operator Lifecycle Manager (OLM). The OLM helps users install,
update, and manage the lifecycle of Kubernetes native applications (Operators)
and their associated services running in clusters. Use the following command to
install:
operator-sdk olm install --version 0.20.0
Note
Local access to your Kubernetes cluster must be set up on your machine before you can perform this step. - Verify your installation of OLM by first checking for all the necessary Customer
Resource Definitions (CRDs) in the cluster. Run the following command.
operator-sdk olm status
The command produces output similar to the following:
INFO[0003] Fetching CRDs for version "v0.20.0" INFO[0003] Fetching resources for resolved version "v0.20.0" INFO[0021] Successfully got OLM status for version "v0.20.0" NAME NAMESPACE KIND STATUS operatorgroups.operators.coreos.com CustomResourceDefinition Installed operatorconditions.operators.coreos.com CustomResourceDefinition Installed olmconfigs.operators.coreos.com CustomResourceDefinition Installed installplans.operators.coreos.com CustomResourceDefinition Installed clusterserviceversions.operators.coreos.com CustomResourceDefinition Installed olm-operator-binding-olm ClusterRoleBinding Installed operatorhubio-catalog olm CatalogSource Installed olm-operators olm OperatorGroup Installed aggregate-olm-view ClusterRole Installed catalog-operator olm Deployment Installed cluster OLMConfig Installed operators.operators.coreos.com CustomResourceDefinition Installed olm-operator olm Deployment Installed subscriptions.operators.coreos.com CustomResourceDefinition Installed aggregate-olm-edit ClusterRole Installed olm Namespace Installed global-operators operators OperatorGroup Installed operators Namespace Installed packageserver olm ClusterServiceVersion Installed olm-operator-serviceaccount olm ServiceAccount Installed catalogsources.operators.coreos.com CustomResourceDefinition Installed system:controller:operator-lifecycle-manager ClusterRole Installed
Deploy OCI Service Operator for Kubernetes
The OCI Service Operator for Kubernetes OLM bundle contains all the required details like CRDs, RBACs, and ConfigMaps. Deployment installs the OCI Service Operator for Kubernetes in the Kubernetes cluster. First, create a namespace for your operator.
kubectl create ns oci-service-operator-system
If deployed in your application namespace, the operator functions normally. This option is an alternative to creating an operator namespace.
For disconnected realms, build your own OSOK Operator by following the steps from https://github.com/oracle/oci-service-operator/blob/main/BUILD_OSOK_OPERATOR.md. Replace the Bundle in the following command with the one that was built manually to install OSOK Operator.
Install the OCI Service Operator for Kubernetes Operator in the Kubernetes cluster in
your namespace (oci-service-operator-system
) using the following
command.
operator-sdk run bundle iad.ocir.io/oracle/oci-service-operator-bundle:X.X.X -n oci-service-operator-system --timeout 5m
Replace X.X.X with current version of the OCI Service Operator for Kubernetes. To get the current version, go to the GitHub release site at: https://github.com/oracle/oci-service-operator/releases
Users must be logged into the Oracle Registry at
iad.ocir.io
in Docker to run the command. To ensure you are logged in, see Pulling Images Using the Docker CLI.The command produces output similar to the following:
INFO[0036] Successfully created registry pod: iad-ocir-io-oracle-oci-service-operator-bundle-X-X-X
INFO[0036] Created CatalogSource: oci-service-operator-catalog
INFO[0037] OperatorGroup "operator-sdk-og" created
INFO[0037] Created Subscription: oci-service-operator-vX-X-X-sub
INFO[0040] Approved InstallPlan install-tzk5f for the Subscription: oci-service-operator-vX-X-X-sub
INFO[0040] Waiting for ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" to reach 'Succeeded' phase
INFO[0040] Waiting for ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" to appear
INFO[0048] Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: Pending
INFO[0049] Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: InstallReady
INFO[0053] Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: Installing
INFO[0066] Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: Succeeded
INFO[0067] OLM has successfully installed "oci-service-operator.vX.X.X"
To verify the installation, perform the following steps.
- Check for custom definitions with
kubectl
.$ kubectl get crds | grep "servicemesh.oci.oracle.com" | awk -F. '{print $1}' accesspolicies ingressgatewaydeployments ingressgatewayroutetables ingressgateways meshes virtualdeploymentbindings virtualdeployments virtualserviceroutetables virtualservices
- Verify the successful installation of webhooks corresponding to all the Service
Mesh resources with
kubectl
.First, check validating webhooks.
$ kubectl get validatingwebhookconfiguration | grep "servicemesh.oci.oracle.cloud.com" | awk -F. '{print $1}' ap-validator ig-validator igd-validator igrt-validator mesh-validator vd-validator vdb-validator vs-validator vsrt-validator
Next, check mutating webhooks.
$ kubectl get mutatingwebhookconfiguration | grep "servicemesh.oci.oracle.com" | awk -F. '{print $1}' proxy-injector
For more information on troubleshooting, see: Troubleshooting Service Mesh.
Install Metrics Server
If you are going to use Ingress Gateway, install metrics server. The Kubernetes Horizontal Pod Autoscalar uses the metrics server to scale the number of pods in the ingress gateway based on CPU usage. The metrics server is installed using the following command:
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/high-availability.yaml
As an alternative, bring your own certificate (BYOC) to secure communications.
Kubernetes provides a certificates.k8s.io
API, which lets you
provision TLS certificates signed by a Certificate Authority (CA) that you control.
The CA and certificates are used by your workloads to establish trust. To establish
trust between OCI Service Operator for Kubernetes and Kubernetes, use
certificates.k8s.io
to create the CA and certificates. Patch
the oci-service-operator-controller-manager-service-cert
secret to
use the new CA and certificates.
For more information, see: Manage TLS Certificates in a Cluster
To generate the CA and CertificateBundle use tools like cfssl(Cloudfare SSL) or
OpenSSL. To use a user created CA, install the default configuration and patch the
kubernetes.io/tls
generated by OLM for OCI Service Operator for
Kubernetes. The following steps demonstrate the configuration steps using CFSSL.
In the following steps, replace
{operators-namespace}
with the
namespace your operator runs in.- Create a Certificate Signing Request(CSR) and private key with
cfssl
following steps:cat <<EOF | cfssl genkey - | cfssljson -bare server { "hosts": [ "oci-service-operator-webhook-service.{operators-namespace}.svc.cluster.local", "oci-service-operator-controller-manager-service.{operators-namespace}.svc.cluster.local" ], "CN": "kube.operator.cert", "key": { "algo": "ecdsa", "size": 256 } } EOF
This command generates two files. The
server.csr
file contains the PEM encoded CSR. Theserver-key.pem
file contains the PEM encoded key to the certificate that is yet to be created. - Create CSR manifest and send it to Kubernetes following section. Note
For more information on these steps, see: Create a CertificateSigningRequest object to send to the Kubernetes API.cat <<EOF | kubectl apply -f - apiVersion: certificates.k8s.io/v1 kind: CertificateSigningRequest metadata: name: oci-service-operator-controller.{operators-namespace} spec: request: $(cat server.csr | base64 | tr -d '\n') signerName: oci.com usages: - digital signature - key encipherment - server auth EOF
After running this command, the CSR created is in a pending state.
kubectl describe csr {csr-name}
Approve the CSR.
kubectl certificate approve {csr-name}
Validate the state after approving. Confirm that the value is:
CONDITION-Approved
. - Sign the CSR.
- Create a certificate authority key file (
ca-key.pem
) and certificate (ca.pem
).cat <<EOF | cfssl gencert -initca - | cfssljson -bare ca { "CN": "My Example Signer", "key": { "algo": "rsa", "size": 2048 } } EOF
- Issue a certificate by creating server signing request. Create the
server-signing-config.json
file.{ "signing": { "default": { "usages": [ "digital signature", "key encipherment", "server auth" ], "expiry": "876000h", "ca_constraint": { "is_ca": false } } } }
Use the
server-signing-config.json
signing configuration, the certificate authority key file, and certificate file to sign the certificate request:kubectl get csr oci-service-operator-controller.{operators-namespace} -o jsonpath='{.spec.request}' | \ base64 --decode | \ cfssl sign -ca ca.pem -ca-key ca-key.pem -config server-signing-config.json - | \ cfssljson -bare ca-signed-server
- Upload signed certificate.
kubectl get csr oci-service-operator-controller.{operators-namespace} -o json | \ jq '.status.certificate = "'$(base64 ca-signed-server.pem | tr -d '\n')'"' | \ kubectl replace --raw /apis/certificates.k8s.io/v1/certificatesigningrequests/oci-service-operator-controller.{operators-namespace}/status -f -
After applying the preceding change, validate the CSR condition is "Approved, Issued."
- Download the certificate.
kubectl get csr {csr-name} -o jsonpath='{.status.certificate}' \ | base64 --decode > server.crt
Running preceding command creates the
server.crt
file.
- Create a certificate authority key file (
- Encode and patch
server.crt
from step (3)(d), theca-key.pem
file from step (3)(a), and theserver-key.pem
file from step (1).kubectl patch secret oci-service-operator-controller-manager-service-cert --namespace {operators-namespace} --type merge --patch }}{{'{"data": "olmCAKey": "'$(base64 ca.pem | tr -d }}{{'\n')'", "tls.key": "'$(base64 server-key.pem | tr -d }}{{'\n')'", "tls.crt": "'$(base64 ca-signed-server.pem | tr -d }}{{'\n')'"}'
- Validate
oci-service-operator-controller-manager-service-cert
contains the latest values. - With the preceding steps complete, the resources are created and working.
However, if you get an error:
x509: certificate signed by unknown authority
, review the following steps to troubleshoot.- Validate if the CSR has a condition of "Approved, Issued." Otherwise, start again from step (3)(c).
- Validate if base64 decoded
olmCAKey
matches theserver.crt
from step (3)(d). Otherwise, update theolmCAKey
with theserver.crt
file. - Validate if base64 decoded
tls.key
matches theserver-key.pem
file from step (1). Otherwise, update thetls.key
with theserver-key.pem
file. - Validate if base64 decoded
tls.crt
matches theca-key.pem
file from step (3)(a). Otherwise, updatetls.crt
with theca-key.pem
file.
Set up Resources Required for Service Mesh
Set up Steps
To start using Service Mesh, you have a few requirements to fulfill before you can create your own meshes.
If you are not an administrator, ask your administrator to give you access to resources of three services: Service Mesh, Vault, and Certificates. Your administrator can give you access to these services with Identity and Access Management (IAM) policies.
To prepare your tenancy to use Service Mesh, perform the following steps with your OCI account.
- Create a vault to store the encryption key for your certificate authority. See
Managing Vaults for details.
Add a Hardware Security Module (HSM) encryption key for your vault. See Managing Keys for details. For your certificate authority key, you can use a Rivest-Shamir-Adleman (RSA) 2,048-bit key or 4,096-bit key.
- Create a certificate authority. See Managing Certificate Authorities for
details. Note
The Maximum Validity Duration for Certificates (Days) setting on the certificate authority must be at least 45 days. - Create a TLS certificate so clients can make TLS connections to your application. This certificate keeps HTTP requests to ingress gateways TLS encrypted. See Managing Certificates for details.
More Information
For more information on IAM policies related to Service Mesh, see:
If you need to remove Service Mesh support from your application, see the Unmeshify Your Application (Optional) section in Service Mesh Overview Tutorial.