Dynamic Routing Gateways

Working with DRGs and DRG Attachments

When creating a DRG, you must specify the compartment where you want the DRG to reside. Placing the DRG in a compartment helps to limit access control. If you're not sure which compartment to use, put the DRG in the same compartment as a VCN you use regularly. For more information, see Access Control.

You might optionally assign a friendly name to the DRG. It doesn't have to be unique, and you can change it later. Oracle automatically assigns the DRG a unique identifier called an Oracle Cloud ID (OCID). For more information, see Resource Identifiers.

To use a DRG, it must be attached to other network resources. In the API, the process of attaching creates a DrgAttachment object with its own OCID. The DrgAttachment has a type field which denotes the type of object being attached to the DRG. The type field can be set to one of the following values:

• VCN
• VIRTUAL_CIRCUIT
• IPSEC_TUNNEL
• REMOTE_PEERING_CONNECTION
• LOOPBACK (See Loopback Attachments for more details.)

To attach a VCN to a DRG, use the CreateDrgAttachment operation or the Console to explicitly create the DRG attachment object. Attachments for virtual circuits, IPSec tunnels, and remote peering connections are created (and deleted) automatically for you when you create (or delete) the network object.

Working with DRG Route Tables and Route Distributions

A packet enters a DRG through an attachment and is routed using rules in the DRG route table assigned to that attachment. You can assign the same route table to many DRG attachments or create a dedicated route table for each attachment depending on the routing policies you want.

When you create a DRG, two default route tables are created for you: one for VCN attachments and one for all other attachments. When a route table is set as the default route table for an attachment type, the table is assigned to newly created attachments of that type unless an alternate table is explicitly specified. Route tables specified as the default for any type can't be deleted. Ensure that a route table isn't set as a default route table for an attachment type before trying to delete it.

A VCN attachment has two route tables: One DRG routing table for traffic entering the DRG, and one VCN routing table for traffic entering the VCN. The DRG route table exists in the DRG and is used to route packets entering the DRG through the attachment. The VCN route table is used to route packets entering the VCN through the attachment. If a VCN routing table isn't defined, a hidden implicit table always provides connectivity to all subnets in the VCN.

Dynamic Import Route Distributions

A distribution is a list of declarative statements that contain a match criteria (such as an OCID or an attachment type) and an action. You can use route distributions to specify how routes get imported from or exported to a DRG attachment. Two auto generated import route distributions are created for each DRG: one for VCN routes only and one for all routes. You can create other import route distributions.

DRG route tables contain both static and dynamic routes. Static routes are inserted into tables using the API, while dynamic routes are imported from attachments and inserted using an import route distribution. When a statement's criteria matches on an attachment, the routes associated with the network object being attached to the DRG are dynamically imported into the DRG route table assigned to the containing distribution. If the statement is removed from the distribution, the routes are withdrawn from the DRG route tables. Statements in a route distribution are evaluated in priority order: the lowest number has the highest priority. The order in which statements are evaluated doesn't affect the preference set for the routes they import.

When building route distribution statements in the Console, you can create a statement whose match type is "Match All". In the API, encode a "match all" statement by setting the match criteria to the empty list.

How do dynamic routes arrive at an attachment?

Routes to your on-premises networks are advertised from the CPE to IPSec tunnel and virtual circuit attachments using BGP. Routes are dynamically propagated from an RPC attachment on one DRG to an RPC attachment on another DRG through connected RPCs. Dynamic routes in a VCN include either all subnet CIDRs or all VCN CIDRs and all static route CIDRs configured on the VCN route table associated with the DRG attachment. The vcnRouteType property of the VCN attachment determines whether the subnet CIDRs or VCN CIDRs are propagated into the VCN attachment. The default behavior is to import the subnet CIDRs, but this behavior can be changed when creating or updating the VCN attachment. Refer to Route Aggregation for other details.

Dynamic Export Route Distributions

When an attachment is assigned to a DRG route table, the contents of that table can be dynamically exported to the attachment. If the default export route distribution is assigned to an attachment, the entire contents of the attachment's assigned DRG route table are dynamically exported to the attachment. To disable dynamic route exports to an attachment, use the API operation removeExportDrgRouteDistribution to set the attachment's exportDrgRouteDistributionId field to NULL. Dynamic route export to VCN attachments isn't supported.

One auto generated export route distribution is created for each DRG. You can create and attach other export route distributions using the CLI and API.

Route Propagation Restrictions

Routes imported from an IPSec tunnel or virtual circuit are never exported to other IPSec tunnel or virtual circuit attachments, regardless of how the export route distribution is configured. In a similar vein, packets which enter a DRG through an IPSec tunnel or virtual circuit attachment can never leave through an IPSec tunnel or virtual circuit attachment. Packets are dropped if routing is configured such that packets originating from IPSec tunnel or virtual circuit attachments are sent to IPSec tunnel or virtual circuit attachments.

ECMP

Equal-cost multi-path routing (ECMP) lets flow-based load balancing of network traffic over FastConnect virtual circuits or IPSec tunnels (but not a mix of circuit types) using BGP. ECMP allows active-active load balancing and failover of network traffic between a maximum of eight circuits.

Oracle uses the protocol, destination IP, source IP, destination port, and source port to distinguish flows for load balancing purposes using a consistent and deterministic algorithm. Therefore, several flows are necessary to use all available bandwidth.

ECMP is off by default and can be enabled on a per-route table basis. Oracle only considers routes with identical route preference as eligible for ECMP forwarding. See Route Conflicts for more.

Route Source

DRG routes originate as either static routes or as dynamic routes from VCN, IPSec tunnel, FastConnect virtual circuit, or RPC attachments. This origin defines their source, which is an immutable characteristic of the route. In the API, the source is referred to as the routeProvenance of a DrgRouteRule.

Routes are propagated between DRGs using RPC attachments.

Routes with a source of IPSEC_TUNNEL or VIRTUAL_CIRCUIT aren't exported to IPSec tunnel or virtual circuit attachments, regardless of the attachment's export distribution.

Routing a Subnet's Traffic to a DRG

The basic routing scenario sends traffic from a subnet in the VCN to the DRG. For example, if you're sending traffic from the subnet to an on-premises network, you set up a rule in the subnet's route table. The rule's destination CIDR is the CIDR for the on-premises network (or a subnet within), and the rule's target is the DRG. For more information, see VCN Route Tables.

Required IAM Policy

Peering VCNs using a DRG requires specific IAM permissions. See IAM Policies for Routing Between VCNs for details on the permissions needed.

DRG Versions

DRGs created before May 17, 2021 use the legacy software, and can be upgraded to the most recent version. DRGs created after that have the upgraded features by default.

The following summarizes the difference between an upgraded and legacy DRG:

A legacy DRG:

• Has no programmable route tables. It has a default routing behavior where all traffic is forwarded from on-premises to an associated VCN and from the VCN to on-premises.
• Can attach to a single VCN. The DRG can attach to another DRG for remote VCN peering using an RPC.
• Can attach FastConnect or Site-to-Site VPN, or both. You can only reach resources in the local region using these connections.
• Can support an RPC connection with a remote DRG-VCN pair in the same tenancy. See Remote VCN Peering using a Legacy DRG to learn about remote peering using a legacy DRG.

An upgraded DRG:

• Has two route tables by default, and more can be added later.
• Can have many VCNs attached to it within the same region. Local VCN to VCN traffic can pass through a mutually connected DRG instead of an LPG. See Local VCN Peering Through an Upgraded DRG for details.
• Can attach to on-premises using FastConnect orSite-to-Site VPN, or both. You can reach resources in both local and remote regions using these connections.
• Supports an RPC connection with a DRG/VCN pair in the same tenancy or another tenancy. See Remote VCN Peering through an Upgraded DRG to learn about remote peering using an upgraded DRG.

The rest of this article has recently been updated to reflect the capabilities of an upgraded DRG, as have the common networking scenarios. Remote VCN Peering using a Legacy DRG is the only routing or peering scenario specific to a legacy DRG.

Before you Upgrade a DRG

To take advantage of enhanced DRG features, upgrade the DRG. The DRG upgrade process is automated, but you must have the required permissions to start an upgrade. Upgrading a DRG is a one-way process with no option to roll back to a legacy DRG after the upgrade process has been started.

Expect each of the DRG's attachments to update in turn, forcing each upgrading attachment into a provisioning state. Virtual circuit and VPN tunnel attachments can no longer forward traffic when in a provisioning state. If you only have one virtual circuit, it could go down for about 20 minutes. Any existing BGP sessions for on-premises connections are also reset while the attachment and connections are also brought offline.

If you have redundant connections, expect traffic to failover to other circuits while one connection undergoes upgrade. Using a redundant configuration, the overall connection between on-premises and OCI is maintained during the DRG upgrade. For example, if a DRG has two FastConnect virtual circuit attachments to an on premises network with one configured as primary, while the primary virtual circuit is upgraded it drops connectivity but traffic can still pass over the secondary virtual circuit. After that first update has finished and traffic over the primary virtual circuit is restored, the process upgrades the secondary virtual circuit attachment and eventually both virtual circuits are back online. Expect the upgrade process to last up to 30 minutes per on-premises attachment. Likewise, if the upgrade process upgrades the secondary virtual circuit's attachment first, traffic stops on that circuit but can still pass over the primary virtual circuit until the secondary circuit connection is reestablished.

Remote peering connections and VCN attachments don't need to have a BGP reset the way virtual circuit or IPSec tunnel connections do, and the upgrade is faster. VCN connections to an LPG or other gateways are unaffected, as they're attachments on the VCN and not the DRG.

After the DRG upgrade process has completed, any Site-to-Site VPN IPSec tunnels are brought back online and all BGP sessions for FastConnect and Site-to-Site VPN are re-established. By default, the upgraded DRG has two auto generated DRG route tables and import route distributions enabled for the attachments. These resources are designed for backward compatibility with a legacy DRG and allow all previous communication to resume in the same manner as before the upgrade without further user intervention.

For step-by-step instructions on how to upgrade a DRG, see Upgrading a DRG.

Scenarios

We have provided some detailed networking scenarios to help you understand the role of a DRG in the Networking service and how the components work together in general.

Route Conflicts

If two routes with identical CIDRs are observed the same DRG route table, the DRG resolves the conflict using the following criteria:

1. Static routes always have higher preference than dynamic routes.

   Note
   
   You can't manually specify two static routes with the same CIDR in the same DRG route table, but it's possible for more than one route with the same CIDR to be imported dynamically.
2. Conflicts between dynamic routes are resolved by first analyzing the route's AS Path:
   • Routes with a route source of VCN or STATIC always have an empty AS Path.
   • Routes with a route source of IPSEC_TUNNEL or VIRTUAL_CIRCUIT have populated AS Path data (see Routing Details for Connections to Your On-premises Network for more details).
3. Otherwise, the attachment type that imported the route is evaluated according to the following priority based on the attachment type:
   1. VCN: the DRG makes an arbitrary but stable selection.
   2. VIRTUAL_CIRCUIT: If ECMP is disabled, the DRG makes an arbitrary but stable selection. If ECMP is enabled, all routes are added to the route table and the DRG makes routing choices using ECMP. The maximum supported ECMP width inside a DRG is 8.
   3. IPSEC_TUNNEL: If ECMP is disabled, the DRG makes an arbitrary but stable selection.If ECMP is enabled, all routes are added to the route table and the DRG makes routing choices using ECMP. The maximum supported ECMP width inside a DRG is 8.

   4. REMOTE_PEERING_CONNECTION: The DRG chooses the route with the lowest network distance.

      If two routes have identical network distances, the DRG selects the route with the highest priority route source (STATIC > VCN > VIRTUAL_CIRCUIT> IPSEC_TUNNEL > RPC).

      If two routes have the same route source, the DRG makes an arbitrary but stable selection.

4. If conflicting routes are imported from attachments of the same type, the conflict is resolved differently depending on the attachment type:
   1. VCN attachments: If identical CIDRs are imported from two VCN attachments, only one is selected using an arbitrary but stable decision procedure.
   2. VIRTUAL_CIRCUIT and IPSEC_TUNNEL attachments: If several routes with the same CIDR and different AS_PATH lengths are imported into a DRG route table, the route with the lowest AS_PATH length is selected. Otherwise, one route is chosen using an arbitrary but stable decision procedure.
   3. RPC attachments: If identical CIDRs are imported from two RPC attachments, one of them is chosen using an arbitrary stable decision procedure.

You can observe the results of conflict resolution by listing the contents of the route table. Deprecated routes are marked with the "conflict" status.

Route Aggregation

As described in RFC-1338 route aggregation lets you "advertise a single, aggregate, route which describes all of the destinations contained within it" instead of advertising a separate route for each client. The primary example of this in OCI Networking happens when you configure a DRG's VCN attachment to import VCN CIDRs instead of subnet CIDRs, thus simplifying what routes are advertised by BGP. Depending on the network architecture, you might also choose to:

• Use static routes instead of import distributions on the DRG to summarize even more narrowly (the scope for this is limited to that specific attachment)
• Aggregate across several attachments by introducing a second DRG and making the aggregated attachments only available through an RPC to another DRG

Other forms of route aggregation aren't available.

Limitations

Some functions might appear to be possible, but the following routing functions aren't allowed:

• Explicit creation or deletion of RPC, IPSec tunnel, or virtual circuit attachments
• Static routes in DRG route tables with next-hop of IPSec tunnel or virtual circuit attachments
• Use of non-default export route distributions
• Dynamic route export to VCN attachments
• Routes propagating through RPC through more than 4 DRGs