Setting Up Data Sources

You create a data asset from a wide range of supported data sources. To set up and manage your data assets, see:

Required IAM Policies for Data Integration Data Asset

Before creating an OCI Data Integration data asset, set the following policies to create, harvest Data Integration data asset and view the lineage for it.

As a prerequisite, create a dynamic group that includes the specific Data Catalog OCID as a resource in the group.

Any {resource.id = '<catalog_ocid>'}

Set the following policies to allow Data Catalog instance to read the Data Integration workspaces and harvest lineage metadata from them:

Allow dynamic-group <dynamic-group-name> to read dis-workspaces in compartment <DIS Workspace Compartment>

Allow dynamic-group <dynamic-group-name> to read dis-workspaces-lineage in compartment <DIS Workspace Compartment>

Note

If you are using the following Service Principal-based policies, you must modify them to the policies mentioned in the previous paragraph by February 28, 2024:

Allow dynamic-group <dynamic-group-name> to use dis-workspaces in compartment <DIS Workspace Compartment>

Allow service datacatalog to {DIS_WORKSPACE_LINEAGE_INSPECT, DIS_WORKSPACE_OBJECT_INSPECT} in compartment <DIS Workspace Compartment Name>

Viewing lineage requires CATALOG_DATA_ASSET_READ permission.

Allow <any-user> to read data-catalog-data-assets in compartment <compartment name> where target.catalog.id='<Data Catalog OCID>'

Required IAM Policies for Data Flow Data Asset

Before creating a data asset in the catalog for OCI Data Flow in a different tenancy, set the following policies.

In the Data Catalog tenancy, set:

admit any-user of tenancy <Data Flow Tenancy> to manage data-catalog-data-assets in tenancy where all {request.principal.type = 'dataflowrun'}

In the Data Flow tenancy, set:

endorse any-user to manage data-catalog-data-assets in tenancy <Data Catalog Tenancy> where all {request.principal.type = 'dataflowrun'}

Set the following policy when the Data Flow application and the Data Catalog are in the same tenancy:

allow any-user to manage data-catalog-data-assets in tenancy where all {request.principal.type = 'dataflowrun'}

Required IAM Policies for Metastore

Before you create Metastore data assets, you create policies to enable access to the required data objects.

As a prerequisite, create a dynamic group that includes the specific data catalog OCID as a resource in the group.

Any {resource.id = ‘<catalog_ocid>’ }

Allow Access to all the Metastores in a Tenancy

Create this policy to allow access to any metastore instance, in any compartment within the tenancy where the policy is created.

allow DYNAMIC-GROUP <group_name> to USE data-catalog-metastores in tenancy

Allow Access to all the Metastores in a Compartment

Create the following policy for the dynamic group to access all the metastores in a compartment:

allow DYNAMIC-GROUP <group_name> to USE data-catalog-metastores in compartment <compartment name>

Allow Access to a Specific Metastore Instance

Create the following policy for the dynamic group to access a specific metastore:

allow DYNAMIC-GROUP <group_name> to USE data-catalog-metastores in tenancy where target.metastore.id='<metastore ocid>'

Required IAM Policies for MySQL Data Asset

You can use the OCI MySQL Database service as a data source to create a MySQL data asset. In this section, learn about the policies that you need to for a group named Administrators, to work with the MySQL Database service.

Create this policy to allow members of the group Administrators to list and read the contents of all compartments in the tenancy:

Allow group Administrators to {COMPARTMENT_INSPECT} in tenancy

Create this policy to allow members of the Administrators group to read, attach, and detach subnets and read Virtual Cloud Networks (VCNs) in the tenancy:

Allow group Administrators to {VCN_READ, SUBNET_READ, SUBNET_ATTACH, SUBNET_DETACH} in tenancy

Note

You need access to these resource types to attach a DB System to a VCN.

Create this policy to allow members of the Administrators group access to all aspects of MySQL Database Service in the tenancy:

Allow group Administrators to manage mysql-family in tenancy

Create this policy to harvest data from MySQL database:

allow group <your-group-name> to use mysql-instances in tenancy

Required IAM Policies for Oracle Object Storage Data Asset

Before you create Oracle Object Storage data assets, you create policies to enable access to the required data objects. After creating these policies, when you harvest the Object Storage data asset, only those data entities that your data catalog instance has access to are listed. You can select the data objects you want to harvest from the displayed list.

At the least, you must have READ permission for all the individual resource types objectstorage-namespaces, buckets, and objects, or for the Object Storage aggregate resource type object-family. For step-by-step instructions, see tutorial Harvesting from Oracle Object Storage.

Create Resource Principal policies to allow Data Catalog access objects in Object Storage. As a prerequisite, create a dynamic group that includes the specific data catalog OCID as a resource in the group.

Example:

Any {resource.id = 'ocid.datacatalog.oc1..<unique_ID>'}

Required IAM Policies for Same Tenancy

Use the following policy if your Data Catalog instance and Object Storage are in the same tenancy:

Create this policy only for the root_compartment to allow access to any object, in any bucket, in any compartment within the tenancy. As the scope of this policy is the whole tenancy, a child compartment doesn't have access to the root or the parent compartments.
```
allow dynamic-group <dynamic-group-name> to read object-family in tenancy
```

To access specific buckets and compartments within the same tenancy, see Policy Examples.

Required IAM Policies for Different Tenancies

Create the following policies as a prerequisite if your Data Catalog instance and Object Storage are in different tenancies.

In catalog tenancy:

Define a tenancy tenancy-name1 to identify the Object Storage OCID, and then endorse the dynamic group dynamic-group-name1 to manage object-family in <tenancy-name1>.

Define tenancy <tenancy-name1> as <object-storage-tenancy-OCID>
Endorse dynamic-group <dynamic-group-name1> to manage object-family in tenancy <tenancy-name1>

In Object Storage tenancy:

Define a tenancy tenancy-name2 with catalog tenancy OCID. Define dynamic-group-name2 with the OCID of dynamic-group-name1 created in the catalog tenancy, and then admit the dynamic-group-name2 to manage object-family in the Object Storage tenancy.

Define tenancy <tenancy-name2> as <catalog-tenancy-OCID>
Define dynamic-group <dynamic-group-name2> as <dynamic-group-name1-OCID>
Admit dynamic-group <dynamic-group-name2> of tenancy <tenancy-name2> to manage object-family in tenancy

To access specific buckets and compartments for different tenancies, see Policy Examples.

Required IAM Policies for using OCI Vault

To use Oracle Cloud Infrastructure Vault, you create a dynamic group that includes the data catalog service as a resource in the group.

Any {resource.type='datacatalog' }

Create the following policy to allow the dynamic group to read the secret:

Allow dynamic-group <dynamic group name> to read secret-family in tenancy

To use an Oracle wallet with secrets in OCI Vault, you must:

Provide a wallet password when you download the wallet.
Remove the .p12 file from the downloaded wallet zip.
Use any base64 encoder to encode the modified wallet zip to base64.
Copy the base64-encoded data to a secret in a vault.
Create a secret for the database password.