Considerations When Using Export and Import
Ensure that you understand what is supported and any current limitations before using export and import operations in Data Integration.
During export, Data Integration clears sensitive information in data assets such as passwords. After the operation is complete, you must manually update the sensitive information fields of applicable data assets in the imported object.
Consider the following before exporting and importing objects in Data Integration.
- No objects are deleted in export operations.
-
You can export and import an entire workspace, a whole project, or different object types separately such as a folder, task, data flow, or pipeline.
-
Data Integration includes referenced objects in an export request. For example, when you export a data flow, by default Data Integration includes referenced objects such data assets. In an import request, you have the option to exclude existing data assets.
-
In an import request, Data Integration uses the identifier values of source and target objects to match objects, and performs the import using the conflict resolution option that you specify. The conflict resolution options are:
- Retain the conflicting objects
- Replace the conflicting target objects
- Duplicate the conflicting source objects
-
When importing projects, folders, tasks, data flows, pipelines, applications, and workspaces, you have the option to exclude importing data assets that already exist in the target. When you exclude existing data assets, Data Integration imports to the target only those source data assets that are not present in the target. Data assets that are already present in the target are not imported.
Before you initiate an import request, if you create data assets in the target by using the same data asset identifiers as used by the source data assets, Data Integration automatically wires the target data assets to relevant target objects when matching target and source data assets are found during the import operation.
-
When a project has many folders, we recommend that the first export and import operations be the following:
- Export the whole project (source)
- Import the whole project (target)
When a new folder is added in the original project (source), export the new folder and import that exported folder zip to the target.
-
When objects in a project's folder have references to objects in another folder, Data Integration exports and imports any dependent objects and their dependencies as well.
Suppose project
Project1
has three folders,Folder1
,Folder2
, andFolder3
, with the following dependencies:-
Objects in
Folder1
reference objects inFolder2
-
Objects in
Folder2
reference objects inFolder3
If you export
Folder1
, Data Integration exportsFolder1
,Folder2
, andFolder3
. FoldersFolder2
andFolder3
are included in the export operation because some or all objects inFolder2
andFolder3
are referenced by objects inFolder1
. Similarly, if you exportFolder2
, Data Integration exportsFolder2
andFolder3
; if you exportFolder3
, onlyFolder3
gets exported.In the scenario with folder interdependencies, the options are:
- Export the whole project first. As and when folders are created or updated, export and import folders to the target using the replace option. We recommend this option as the best practice.
- Export the folders in the following order:
Folder3
,Folder2
with the replace option,Folder1
with the replace option
-
Before exporting and importing, ensure that you have access to Object Storage and the relevant permissions and IAM policies, as described in Required Setup and Policies.