Troubleshooting Network Configuration Issues for Kubernetes Clusters Using Network Path Analysis Tests
Find out how to use network path analysis tests to help you resolve network connectivity issues with clusters you've created using Kubernetes Engine (OKE).
For a cluster you create with Kubernetes Engine to function correctly, the network resources specified for the cluster must be configured appropriately. When you create a cluster using the Quick Create workflow, Kubernetes Engine creates and configures new network resources for you, according to the guidelines described in Network Resource Configuration for Cluster Creation and Deployment. However, when you create a cluster using the Custom Create workflow, you specify existing network resources for the new cluster to use. These resources must be configured appropriately to enable network communication with the cluster, and to enable different cluster components to communicate with each other.
Configuring network resources for Kubernetes clusters can become complicated. Kubernetes Engine therefore provides a number of pre-defined network path analysis tests to troubleshoot network configuration issues. These path analysis tests examine virtual network topologies, walk through multiple route tables, and scrutinize security rules in network security groups (NSGs) and security lists. No actual traffic is sent, instead the configuration is examined and used to confirm reachability.
Using the path analysis tests, you can determine whether one OCI resource can reach another OCI resource (unidirectional connectivity from source to destination), and whether two OCI resources can reach each other (bidirectional connectivity). For example, you can use a path analysis test to determine whether a pod in a Kubernetes cluster that uses the VCN-Native Pod Networking CNI plugin can reach OCI services, and vice versa.
When using the Console, you can select the pre-defined path analysis tests from the Cluster details screen. For more information about the pre-defined path analysis tests that are available, see Pre-defined Path Analysis Tests.
Each pre-defined path analysis test requires a number of different test parameters. When you run a test, depending on the test you select, default values for some or all of the test parameters are derived from the properties of resources used by the cluster. In some cases, you can change the default values. You have to provide values for test parameters that do not have default values. Having run a pre-defined path analysis test, you can save the results as a JSON file, enabling you to compare test results over time. For more information about running pre-defined path analysis tests for a cluster, see Running Pre-defined Path Analysis Tests.
The path analysis tests are powered by Oracle Cloud Infrastructure Network Path Analyzer (NPA), which identifies virtual network configuration issues that impact connectivity. In addition to the pre-defined path analysis tests available from the Cluster details screen, you can also create your own custom path analysis tests using NPA. For more information about NPA, including permissions required to run NPA, and how to create and run path analysis tests with NPA, see Network Path Analyzer.
Required IAM Policies for Running Path Analysis Tests
To run path analysis tests to verify network paths for a Kubernetes cluster that you have created using Kubernetes Engine, you must belong to a group that has been granted permission to use Network Path Analyzer (NPA). Network Path Analyzer must also have been granted permission to access various network resources.
To find out the permissions to grant, and a recommended IAM policy, see Required Permissions in the Network Path Analyzer documentation.
Pre-defined Path Analysis Tests
You can use a number of different pre-defined path analysis tests to verify network paths for a Kubernetes cluster that you have created using Kubernetes Engine. In the Console, you can filter the list of path analysis tests by:
- the type of issue that you want to check for (such as DNS failures, Kubernetes control plane failures, nodes failing to register)
- paths that are essential, recommended, or optional for the correct functioning of Kubernetes Engine
The tests are grouped into the following broad test categories:
- Cluster API tests: Use to verify that one or more paths are available to allow the cluster's Kubernetes API endpoint to communicate bi-directionally with other cluster components and network locations.
- Node tests: Use to verify that one or more paths are available to allow worker nodes in the Kubernetes data plane to communicate bi-directionally with other cluster components and network locations.
- Pod tests: Use to verify that one or more paths are available to allow pods in the Kubernetes data plane to communicate bi-directionally with other cluster components and network locations.
- Load Balancer tests: Use to verify that one or more paths are available to allow an OCI load balancer or network load balancer to communicate bi-directionally with other cluster components and network locations.
Cluster components use different paths to communicate with other cluster components and network locations, depending on the cluster's network type (Flannel overlay or VCN-native) and the type of worker nodes the cluster contains (managed and/or virtual). In each test category, there are different path analysis tests to verify these different paths.
Note that the Console only shows you the path analysis tests that are applicable to the current cluster. For example, if the cluster's network type is Flannel overlay, you can only select path analysis tests that apply to the Flannel overlay network type.
Running Pre-defined Path Analysis Tests
You use the Console to run pre-defined path analysis tests to verify network paths for a Kubernetes cluster that you have created using Kubernetes Engine.