Prerequisites for deploying the OCI Native Ingress Controller as a Cluster Add-on

Find out what you have to do before you can deploy the OCI native ingress controller as a cluster add-on to load balance and route incoming traffic to service pods running on worker nodes in a Kubernetes cluster.

Before deploying the OCI native ingress controller as a cluster add-on:

Setting Up an Instance Principal, User Principal, or Workload Identity Principal to Enable Access to OCI Services and Resources

To create a load balancer and route incoming traffic, the OCI native ingress controller, whether installed as a standalone program or as a cluster add-on, performs actions on other Oracle Cloud Infrastructure service resources (including the Load Balancer service and the Certificates service). To perform those actions on OCI service resources, the OCI native ingress controller pod uses the credentials of an authorized actor (or principal). You can currently set up the following types of principal to enable the OCI native ingress controller to perform actions on OCI service resources:

Note:

  • The use of instance principals to enable the OCI native ingress controller to access OCI services and resources is not supported in clusters with virtual nodes.
  • The use of workload identity principals to enable the OCI native ingress controller to access OCI services and resources is supported with enhanced clusters, but not with basic clusters.

Using instance principals to enable access to OCI services and resources

You can set up an instance principal to enable the OCI native ingress controller to perform actions on OCI service resources. Note that you can only use instance principals with managed nodes.

To set up an instance principal:

  1. Create a new dynamic group to contain the compute instances hosting the cluster's worker nodes:
    1. Open the navigation menu and click Identity & Security. Under Identity, click Domains. Under Identity domain, click Dynamic groups.
    2. Select the compartment to which the compute instances belong.

    3. Follow the instructions in To create a dynamic group, and give the dynamic group a name (for example, acme-oke-native-ingress-controller-dyn-grp).

    4. Enter a rule that includes the compute instances in the compartment, in the format:
      ALL {instance.compartment.id = '<compartment-ocid>'}

      where <compartment-ocid> is the OCID of the compartment in which cluster node pools are created.

      For example:

      ALL {instance.compartment.id = 'ocid1.compartment.oc1..aaaaaaaa23______smwa'}
    5. Click Create Dynamic Group.

Before you deploy the OCI native ingress controller, you will:

Using user principals to enable access to OCI services and resources

You can set up a user principal to enable the OCI native ingress controller to perform actions on OCI service resources.

To set up a user principal:

  1. If a suitable user does not exist already, create a user in IAM (see To create a user).
  2. If a suitable group does not exist already, create a group in IAM (see To create a group).
  3. Add the user to the group (see To add a user to a group).
  4. Get these items:

  5. Upload the public key from the key pair in the Console. See How to Upload the Public Key.
  6. Create a config file as a .yaml file containing credential information, in the following format:
    auth:
      region: <region-identifier>
      passphrase: <passphrase>
      user: <user-ocid>
      fingerprint: <fingerprint>
      tenancy: <tenancy-ocid>
    where:
    • region: <region-identifier> is the region where the cluster resides. For example, us-ashburn-1
    • passphrase: <passphrase> specifies the passphrase used for the key if it is encrypted.
    • user: <user-ocid> is the OCID of the user that the OCI native ingress controller is to use.
    • fingerprint: <fingerprint> is the fingerprint of the public key.
    • tenancy: <tenancy-ocid> is the OCID of the tenancy containing the cluster.

    For example:

    auth:
      region: us-ashburn-1
      passphrase: examplepass
      user: ocid1.user.oc1..aaaaaaaa_example
      fingerprint: 67:d9:74:4b:21:example
      tenancy: ocid1.tenancy.oc1..aaaaaaaa_example
  7. Create a Kubernetes secret resource in the cluster by entering:
    
    kubectl create secret generic <secret-name> \
    --from-file=config=<config-file>.yaml \
    --from-file=private-key=<private-key-file-path>.pem \
    --namespace <namespace>
    where:
    • <secret-name> specifies the name of the secret to create. For example, oci-config
    • --from-file=config=<config-file>.yaml specifies the name and path of the .yaml file containing the credential information that you created previously. For example, user-auth-config.yaml
    • --from-file=private-key=./oci/oci_api_key.pem specifies the name and path of the downloaded private key file. For example, ./oci/oci_api_key.pem
    • --namespace <namespace> specifies the namespace containing the OCI native ingress controller

    For example:

    
    kubectl create secret generic oci-config \
    --from-file=config=user-auth-config.yaml \
    --from-file=private-key=./oci/oci_api_key.pem \
    --namespace acme-namespace

Before you deploy the OCI native ingress controller you will:

Using workload identity principals to enable access to OCI services and resources

You can set up a workload identity principal to enable the OCI native ingress controller to perform actions on OCI service resources. Note that you can only use workload identity principals with enhanced clusters.

To set up a workload identity principal:

  1. Obtain the OCID of the cluster (for example, using the Cluster Details tab in the Console).

Before you deploy the OCI native ingress controller, you will:

Granting Permissions to the OCI Native Ingress Controller Cluster Add-on

The OCI native ingress controller requires permissions to access resources created by other Oracle Cloud Infrastructure services (such as the Load Balancer service and the Certificates service). The permissions you grant are the same, regardless of whether you install the OCI native ingress controller as a standalone program or as a cluster add-on. And the permissions are the same, regardless of whether you have set up an instance principal, a user principal, or a workload identity principal for the OCI native ingress controller. However, the way in which you grant those permissions does depend on the type of principal you have set up:

To set up permissions for the OCI native ingress controller, create a policy for the group (in the case of user principals), for the dynamic group (in the case of instance principals), or for the workload (in the case of workload identity principals), with policy statements to access OCI services and resources:

  1. Open the navigation menu and click Identity & Security. Under Identity, click Policies.
  2. Follow the instructions in To create a policy and give the policy a name (for example, acme-oke-native-ingress-controller-policy).
  3. If you are using instance principals or user principals, enter policy statements to allow access to the OCI services and resources used by the OCI native ingress controller, in the format:

    Allow <group|dynamic-group> <subject-name> to <verb> <resource> in <location>   
    

    where:

    • <group|dynamic-group> is either group (in the case of user principals) or dynamic-group (in the case of instance principals)
    • <subject-name> is either the name of the group (in the case of user principals) or the name of the dynamic group (in the case of instance principals). For example, acme-oke-nat-ing-cont-dyn-grp. Note that if a group or dynamic group is not in the default identity domain, prefix the group or dynamic group name with the identity domain name, in the format <group|dynamic-group> '<identity-domain-name>'/'<group-name|dynamic-group-name>'. You can also specify a group or dynamic group using its OCID, in the format group id <group-ocid> or dynamic-group id <dynamic-group-ocid>.
    • <verb> <resource> is one of the following (all of these are required, in separate policy statements):
      • manage load-balancers
      • use virtual-network-family
      • manage cabundles
      • manage cabundle-associations
      • manage leaf-certificates
      • read leaf-certificate-bundles
      • manage certificate-associations
      • read certificate-authorities
      • manage certificate-authority-associations
      • read certificate-authority-bundles
      • read public-ips
      • manage floating-ips
      • manage waf-family
      • read cluster-family
    • <location> is one of:
      • tenancy, if you want the OCI native ingress controller to have access to resources in the entire tenancy.
      • compartment <compartment-name> if you only want the OCI native ingress controller to have access to resources in the compartment with the name you specify as compartment <compartment-name>.

    For example:

    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage load-balancers in compartment acme-oke-cluster-compartment

    The syntax for the complete list of policy statements is as follows:

    Allow <group|dynamic-group> <subject-name> to manage load-balancers in <location>   
    Allow <group|dynamic-group> <subject-name> to use virtual-network-family in <location>
    Allow <group|dynamic-group> <subject-name> to manage cabundles in <location>
    Allow <group|dynamic-group> <subject-name> to manage cabundle-associations in <location>
    Allow <group|dynamic-group> <subject-name> to manage leaf-certificates in <location>
    Allow <group|dynamic-group> <subject-name> to read leaf-certificate-bundles in <location>
    Allow <group|dynamic-group> <subject-name> to manage certificate-associations in <location>
    Allow <group|dynamic-group> <subject-name> to read certificate-authorities in <location>
    Allow <group|dynamic-group> <subject-name> to manage certificate-authority-associations in <location>
    Allow <group|dynamic-group> <subject-name> to read certificate-authority-bundles in <location>
    Allow <group|dynamic-group> <subject-name> to read public-ips in <location>
    Allow <group|dynamic-group> <subject-name> to manage floating-ips in <location>
    Allow <group|dynamic-group> <subject-name> to manage waf-family in <location>
    Allow <group|dynamic-group> <subject-name> to read cluster-family in <location>

    Example of the complete list of policy statements:

    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage load-balancers in compartment acme-oke-cluster-compartment   
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to use virtual-network-family in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage cabundles in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage cabundle-associations in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage leaf-certificates in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to read leaf-certificate-bundles in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage certificate-associations in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to read certificate-authorities in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage certificate-authority-associations in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to read certificate-authority-bundles in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to read public-ips in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage floating-ips in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage waf-family in compartment acme-oke-cluster-compartment
    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to read cluster-family in compartment acme-oke-cluster-compartment
    
  4. If you are using workload identity principals, enter policy statements to allow access to the OCI services and resources used by the OCI native ingress controller, in the format:
    Allow any-user to <verb> <resource> in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}

    where:

    • <verb> <resource> is one of the following (all of these are required, in separate policy statements):
      • manage load-balancers
      • use virtual-network-family
      • manage cabundles
      • manage cabundle-associations
      • manage leaf-certificates
      • read leaf-certificate-bundles
      • manage certificate-associations
      • read certificate-authorities
      • manage certificate-authority-associations
      • read certificate-authority-bundles
      • read public-ips
      • manage floating-ips
      • manage waf-family
      • read cluster-family
    • <location> is one of:
      • tenancy, if you want the OCI native ingress controller to have access to resources in the entire tenancy.
      • compartment <compartment-name> if you only want the OCI native ingress controller to have access to resources in the compartment with the name you specify as compartment <compartment-name>.
    • request.principal.namespace = 'native-ingress-controller-system' is the name of the namespace created for the OCI native ingress controller during installation.
    • request.principal.service_account = 'oci-native-ingress-controller' is the name of the service account created for the OCI native ingress controller during installation.
    • <cluster-ocid> is the cluster's OCID that you obtained previously.

    For example:

    Allow any-user to manage load-balancers in tenancy where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = 'ocid1.cluster.oc1.iad.aaaaaaaa______ska'}

    The syntax for the complete list of policy statements is:

    Allow any-user to manage load-balancers in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to use virtual-network-family in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to manage cabundles in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to manage cabundle-associations in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to manage leaf-certificates in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to read leaf-certificate-bundles in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to manage certificate-associations in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to read certificate-authorities in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to manage certificate-authority-associations in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to read certificate-authority-bundles in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to read public-ips in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to manage floating-ips in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to manage waf-family in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
    Allow any-user to read cluster-family in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}

Installing cert-manager

The OCI native ingress controller, whether installed as a standalone program or as a cluster add-on, uses webhooks to inject pod readiness gates into pod specifications. To ensure security, the webhook server has to run in HTTPS mode, which requires a pair of certificates and keys. The OCI native ingress controller uses Certificate Manager (also known as cert-manager) to generate and manage the certificates and keys for the webhook server, so you have to install cert-manager on the cluster to use pod readiness gates.

Regardless of whether you install the OCI native ingress controller as a standalone program or as a cluster add-on, you can install and run cert-manager in two ways:

  • You can install and run cert-manager as an open-source product, by entering:

    kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml
  • You can install and run cert-manager as a cluster add-on. For more information about installing cert-manager as a cluster add-on, see Installing a Cluster Add-on.

For more information about cert-manager, see the cert-manager.io documentation.

Note that installation of the OCI native ingress controller as a cluster add-on fails if you haven't already installed cert-manager, either as an open source product or as a cluster add-on.