Troubleshooting Data Integration
Use troubleshooting information to identify and address common issues that can occur while working with Data Integration.
See also Known Issues for Data Integration.
Troubleshooting information:
Policy Messages
These are some error messages that you might encounter when policies are missing.
See also Policy Examples.
Possible messages:
Authorization failed or requested resource not found.
(404, NotAuthorizedOrNotFound, false) Authorization failed or requested resource not found (opc-request-id: <some-id>)
(404, NotAuthorizedOrNotFound, false) Authorization failed or requested resource not found.
Message location and resolution:
On the Data Integration Workspaces page or the Move Resource to a Different Compartment dialog:
The manage dis-workspaces permission is missing for the selected compartment. Add the following statement to the compartment policy.allow group <group-name> to manage dis-workspaces in compartment <compartment-name>
In the Move Resource to a Different Compartment dialog:
The service is missing the inspect compartment permission for the selected target compartment. Add the following statement to the service policy.allow service dataintegration to inspect compartments in compartment <target-compartment-name>
On a Object Storage Data Asset details page, when testing a connection:
The inspect compartments permission for the tenancy is missing. Add the following statement to the service policy.allow group <group-name> to inspect compartments in tenancy
On the Publish to Oracle Cloud Infrastructure Data Flow page or the Data Flow Service Publish History page for a task, when trying to view the Data Flow service application that was created after successfully publishing a task to the Data Flow service:
The read dataflow-application permission is missing for the selected compartment. Add the following minimal policy statement to the compartment policy.allow group <group-name> to read dataflow-application in compartment <compartment-name>
On the page of the Data Flow service application created for a task published from Data Integration, when trying to run the application:
The manage dataflow-run permission is missing for the selected compartment. Add the following minimal policy statement to the compartment policy.allow group <group-name> to manage dataflow-run in compartment <compartment-name>
-
On the Create Workspace page when enabling a private network, the following policies are needed for non-administrator users:
allow group <group-name> to use virtual-network-family in compartment <compartment-name>
allow group <group-name> to inspect instance-family in compartment <compartment-name>
Possible messages:
No items found.
No items found.
(404, BucketNotFound, false) Either the bucket named '<bucket-name>' does not exist in the namespace '<namespace>' or you are not authorized to access it (opc-request-id: <some-id>)
No items found.
DIS_DATA_XPLORER_0008 - Error in getting interactive job run for opcRequestID: <some-id>.
Caused by: java.lang.RuntimeException: java.io.IOException: Unable to determine if path is a directory.
Message location and resolution:
In Object Storage data asset details page, under Buckets:
Add the following statement to the compartment policy.allow group <group-name> to use buckets in compartment <compartment-name>
In the Object Storage Select Data Entity panel, after selecting a compartment and bucket:
Add the following statement to the compartment policy.allow group <group-name> to use objects in compartment <compartment-name>
In the designer Data tab for an operator node:
Add the following statements to the compartment policy.
allow any-user to manage objects in compartment <compartment-name> where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace-ocid>'}
allow any-user to use buckets in compartment <compartment-name> where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace-ocid>'}
Possible messages:
Error fetching data. No data available.
No data available
Message location and resolution:
On the Create Workspace page, in the Choose VCN and Choose Subnet fields:
The use virtual-network-family permission is missing for the selected compartment. Add the following statement to the compartment policy.
allow group <group-name> to use virtual-network-family in compartment <compartment-name>
In the Object Storage Select Data Entity panel, in the Bucket field:
Add the following statement to the compartment policy.
allow group <group-name> to use buckets in compartment <compartment-name>
Possible message:
0% completed with no steps mentioned
Message location and resolution:
On the Workspace Status dialog from the Data Integration Workspaces list:
The manage dis-work-requests permission is missing for the selected compartment. Add the following statement to the compartment policy.
allow group <group-name> to manage dis-work-requests in compartment <compartment-name>
Possible messages:
Cannot resolve schema reference for key <some-key>, when resolving bound data entity for operator <operator-name>.
Cannot find the schema when preparing data access operator <operator-name>.
Cannot resolve schema reference for key <some-key>, when resolving bound data entity for operator <operator-name>.
Cannot find the schema when preparing data access operator <operator-name>.
Execution of the task <task-type> <task-name> failed.
Message location and resolution:
In the designer Validation tab:
Add the following statement to the compartment policy.allow any-user to use buckets in compartment <compartment-name> where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace-ocid>'}
In the Log Message panel from the Task Runs page, after executing an integration task:
Add the following statement to the compartment policy.allow any-user to use buckets in compartment <compartment-name> where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace-ocid>'}
Possible messages:
There was an issue with the Data Entity for operator: <operator-name>. Either no data entity was selected or you no longer have access to the selected data entity, connection, schema, or data asset. Check the operator or data asset details.
There was an issue with the Data Entity for operator: <operator-name>. Either no data entity was selected or you no longer have access to the selected data entity, connection, schema, or data asset. Check the operator or data asset details.
Execution of the task <task-type> <task-name> failed.
Message location and resolution:
In the designer Validation panel:
Add the following statement to the compartment policy.
allow any-user to manage objects in compartment <compartment-name> where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace-ocid>'}
In the Log Message panel from the Task Runs page, after executing an integration task:
Add the following statement to the compartment policy.
allow any-user to manage objects in compartment <compartment-name> where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace-ocid>'}
Possible messages:
Task execution failed
Caused by: com.oracle.dicom.connectivity. framework.iface.exception.ConnectorException: (404,BucketNotFound, false) Either the bucket named '<bucket-name>' does not exist in the namespace '<namespace>' or you are not authorized to access it (opc-request-id: <some-id>)
Message location and resolution:
In the Task Runs page, when executing a task that has a target as an Autonomous Database:
Add the following statement to the compartment policy.
allow any-user {PAR_MANAGE} in compartment <compartment-name> where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace-ocid>'}
Possible messages:
DIS_SPL_0301 - Cannot publish to OCI Dataflow com.oracle.dicom.spark.execution.exceptions.SparkApplicationExecutionException:
DIS_SPL_0301 - Cannot publish to OCI Dataflow com.oracle.bmc.model.BmcException: (404, NotAuthorizedOrNotFound, false) Authorization failed or requested resource not found.
DIS_SPL_0301 - Cannot publish to OCI Dataflow com.oracle.dicom.spark.execution.exceptions.SparkApplicationExecutionException:
DIS_SPL_0301 - Cannot publish to OCI Dataflow com.oracle.bmc.model.BmcException: (404, NotAuthorizedOrNotFound, false) Unknown resource Unable to find bucket '<bucket-name>' in object storage namespace '<namespace>'. Please refer to the documentation and ensure that a bucket is available and configured. {ResultCode: UNABLE_TO_FIND_BUCKET, Parameters: [<bucket-name>,<namespace>]}
Message location and resolution:
On the Publish to Oracle Cloud Infrastructure Data Flow page or the Data Flow Service Publish History page for a task, after trying to publish a task to an application in the Data Flow service:
The manage dataflow-application permission is missing for the selected compartment. Add the following minimal scope statement to the compartment policy.
allow any-user to manage dataflow-application in compartment <compartment-name> where ALL {request.principal.type = 'disworkspace', request.principal.id = '<workspace_ocid>'}
On the Publish to Oracle Cloud Infrastructure Data Flow page or the Data Flow Service Publish History page for a task, after trying to publish a task to an application in the Data Flow service:
Create a bucket named '<bucket-name>' in the compartment where Data Integration has access in the tenancy (namespace).
Extraction from Oracle Fusion Applications
Troubleshoot common extraction problems when working with Oracle Fusion Applications data in Data Integration data assets, data flows, and tasks.
BICC Fusion Applications
When using the Data tab to view data from an Oracle Business Intelligence Cloud Connector (BICC) data asset, you might encounter reading or loading errors.
Inspect the Status JSON Files and Log Files
In the Object Storage bucket that you use as the external storage location for staging extracted data, you can inspect the status JSON files that are created.
You can also use the BICC Console to find diagnostic information to help with troubleshooting extraction failures.
- Log in to the BICC Console.
- From the Help menu, select Download Logs.
- Inspect the file
biee_extractor.log
in the log zip.
Value is Too Large for Column
Example message:
ORA-12899: Value too large for column
Possible cause: When extracting data for forward-engineering target loading in Data Integration, BICC might return inaccurate column length information for one or more view objects. When the data values for loading are longer than the maximum column length in the target, data loading fails.
Resolution: Create the target entity manually using larger column lengths.
BIP Fusion Applications
When using the Data tab to view data from an Oracle Business Intelligence Publisher (BIP) data asset, you might encounter reading or loading errors.
Preview of Data Fails to Load for a Data Entity
Example message:
DICOM_CONNECTIVITY_0144 - Exception in data-access manager while reading sample-dataset
Possible cause: In Data Integration, when viewing the details of a BIP Fusion Applications data asset, selecting Attributes returns the data entity attributes. Selecting Data, however, fails to load the report. The error message indicates that an error was encountered when retrieving sample data in the report.
Resolution: Test the data model in BIP. If the data model runs in BIP successfully, save the test dataset as sample data. Attach the sample data to the data model.
Task Runs
Troubleshoot common run problems when working with Autonomous Data Warehouse or Autonomous Transaction Processing data sources in Data Integration data flows and tasks.
For example, a task run might fail because of a loading issue.
Loading Fails
Example message:
ORA-20003: Reject limit reached, query table "ADMIN"."COPY$23_LOG"
Possible cause: Data errors such as differences in data length or type can cause staging and loading to fail.
Resolution: You can use the autonomous database log table that's generated during the run to discover the data error or errors. Click View Log to see the error message or messages. Query the log table (for example, "ADMIN"."COPY$23_LOG"
) to retrieve more information, including detailed logs for the load.
Creating of target data entity fails
Example message:
ORA-20000: KUP-04050: error while attempting to allocate 1073741823 bytes of memory
Possible cause: There's a limit of 4000 bytes for a VARCHAR2 column in Autonomous Data Warehouse. For example, if a VARCHAR2 source column has JSON data with a precision value of 1073741823, the task execution fails to create the ADW target data entity.
Resolution: Conversion between TEXT and CLOB is not supported in Data Integration. To work around both limitations, create a table in ADW with the CLOB data type. Then in the ADW target in the data flow, specify to use the existing data entity instead of creating a new data entity. Ensure that column mapping is appropriate.
Connections
Troubleshoot common connection problems when working with data sources in Data Integration data flows and tasks.
Before you begin troubleshooting, check the Policy Messages section for relevant error messages.
How to find the service console logs and opc-request-id
When an error occurs in the Console, and the error message does not show the opc-request-id, you can use the following steps to find the id.
- On the service console page in the browser, right-click and select Inspect. Alternatively, you can click the three dots icon and select More tools, then select Developer tools.
- On the toolbar of the Developer tools panel, click the Network tab.
- Select Filter Fetch XHR to only see the API calls made from the Console.
- Perform an activity on the service console page. Then on the Developer tools panel, click an item of interest in the Name column.
- Under Headers, you can see Request Headers and Response Headers. The opc-request-ids are listed in the headers.
- To see the service console logs, click the three dots icon on the Network tab and select Show console drawer. You can find the opc-request-id in the error block.
Database login fails
Example messages:
DICOM_CONNECTIVITY_0070 - OraError : ORA-28001: the password has expired
DICOM_CONNECTIVITY_0070 - OraError : ORA-01017: invalid username/password
Possible cause: The password has expired or an incorrect username and password pair was used.
Resolution: Verify that you have a valid username and password. Check that the connection details in the data asset has the valid username and password. If applicable, change the database user's password, then update the password in the data asset's connection details.
If the connection details are correct, retry connecting at a later time.
Test connection fails
Example message:
Timeout on polling the AgentManager Service for response of work request <request id>
Possible cause: Depending on the type of data asset (for example, Object Storage or autonomous database), possible causes include a missing networking configuration in a workspace that's enabled to use a private network, or you're trying to access a private IP resource from a workspace that's not enabled for a private network.
To check if a workspace is enabled for a private network, on the workspace home page, click Configurations. If enabled, Private network shows Enabled
, and the VCN and Subnet that are attached when the workspace was created also appear on the page. The subnet in use must be a regional subnet.
Resolution: In a workspace that's enabled for a private network, verify in the subnet's route table that a NAT gateway or a service gateway is present. For service gateway, check that the gateway is set to use the destination All <region_name> services in Oracle Services Network. Also check that the VCN has an egress rule for the appropriate port or ports.
If the network settings are correct, retry testing the connection at a later time.
For help to identify other options, see this Blog: Understanding VCN configuration for Data Integration
ADW test connection fails
Possible cause: A cause for an ADW test connection failure might be an issue with the host's fully qualified domain name (FQDN).
Resolution: Try again by using an IP address.
Error Code Reference
If you encounter an error or warning in Data Integration, refer to the error messages for help.
Error codes are organized by feature. Select a feature to view the list of codes:
Data Flow
Error Code | Message | Resolution |
---|---|---|
DIS_DATAFLOW_0001 | Unable to retrieve data flow with id {} in workspace {}. Possible cause: {}. | Review the possible cause. For example, if you see "Metadata object with key ' ' not found," the data flow either doesn't exist or might have been deleted. Use a valid data flow. |
DIS_DATAFLOW_0002 | Unable to save data flow, {}, in workspace {}. Possible cause: {}. | Review the possible cause. For example, if you see "Aggregator {} chosen for object {} does not exist," the project or folder chosen doesn't exist. Use a valid project or folder. |
DIS_DATAFLOW_0003 | Unable to update data flow with id {} in workspace {}. Possible cause: {}. | Review the possible cause. For example, if you see "Metadata object with key ' ' not found," the data flow either doesn't exist or might have been deleted. Use a valid data flow. |
DIS_DATAFLOW_0004 | Unable to delete data flow with id {} in workspace {}. Possible cause: {}. | Review the possible cause. For example, if you see "Metadata object with key ' ' not found," the data flow either doesn't exist or might have been deleted. Use a valid data flow. |
DIS_DATAFLOW_0005 | Unable to retrieve task with id {} in workspace {}. Possible cause: {}. | Review the possible cause. For example, if you see "Metadata object with key ' ' not found," the task either doesn't exist or might have been deleted. Use a valid task. |
DIS_DATAFLOW_0006 | Unable to save task, {}, in workspace {}. Possible cause: {}. | Review the possible cause. For example, if you see "Aggregator {} chosen for object {} does not exist," the project or folder chosen doesn't exist. Use a valid project or folder. |
DIS_DATAFLOW_0007 | Unable to update task with id {} in workspace {}. Possible cause: {}. | Review the possible cause. For example, if you see "Metadata object with key ' ' not found," the task either doesn't exist or might have been deleted. Use a valid task. |
DIS_DATAFLOW_0008 | Unable to delete task with id {} in workspace {}. Possible cause: {}. | Review the possible cause. For example, if you see "Metadata object with key ' ' not found," the task either doesn't exist or might have been deleted. Use a valid task. |
DIS_DATAFLOW_0009 | Unable to list data flows for aggregator {} in workspace {}. Possible cause: {}. | Review the possible cause. For example, if you see "Aggregator {} chosen for object {} does not exist," the project or folder chosen doesn't exist. Use a valid project or folder. |
DIS_DATAFLOW_0010 | Unable to list tasks for aggregator {}, in workspace {}. Possible cause: {}. | Review the possible cause. For example, if you see "Aggregator {} chosen for object {} does not exist," the project or folder chosen doesn't exist. Use a valid project or folder. |
DIS_DATAFLOW_0011 | Unable to complete task validation. Possible cause: {}. | Contact Oracle Support. |
DIS_DATAFLOW_0012 | Unable to complete task validation. Data flow does not exist for {}. Select a data flow and try again. | The data flow selected while creating a task doesn't exist. Edit the task and select another data flow. |
DIS_DATAFLOW_0013 | Unable to complete task validation. Validation for task type {} is not supported. | Validation is not supported for this task type. |
DIS_DATAFLOW_0014 | Unable to validate data flow. You must add and configure a data flow, and then try again. | The selected data flow doesn't exist. Select a valid data flow. |
DIS_DATAFLOW_0015 | Unable to validate data flow. Error occurred while processing the request. | Contact Oracle Support. |
DIS_DATAFLOW_0016 | Unable to validate data flow. Data flow with Id {} cannot be found. | The selected data flow doesn't exist. Select a valid data flow. |
DIS_DATAFLOW_0017 | Unable to complete data flow validation. Invalid Id {} provided. | Contact Oracle Support. |
Data Xplorer (Data tab)
Error Code | Message | Resolution |
---|---|---|
DIS_DATA_XPLORER_0001 | The following parameters cannot be null: {} {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0002 | {} is invalid. | Reopen the data flow and retry the same operation. If the issue persists, contact Oracle Support. |
DIS_DATA_XPLORER_0003 | Invalid Message Id returned from Data Gateway. Try again or Contact Oracle Support. | Retry the same operation. If the issue persists, contact Oracle Support. |
DIS_DATA_XPLORER_0004 | Data Gateway with workspaceID {} was not found. | Contact Oracle Support. |
DIS_DATA_XPLORER_0005 | Unable to connect to Data Gateway with workspaceID {}. | Reopen the data flow and retry the same operation. If the issue persists, contact Oracle Support. |
DIS_DATA_XPLORER_0006 | Error in getting interactive session for interactiveSessionId = {}, opcRequestID: {}, Caused by: {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0007 | opcRequestID: {}, caused by: {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0008 | Error in getting interactive job run for opcRequest ID: {}, caused by: {}. | Reopen the data flow and retry the same operation. If the issue persists, contact Oracle Support. |
DIS_DATA_XPLORER_0009 | Unable to decrypt autonomous database wallet file: {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0010 | Unable to decrypt connection password. | Contact Oracle Support. |
DIS_DATA_XPLORER_0011 | Error in deserializing the execution result response from Data Gateway for agentuuid {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0012 | Unable to create interactive session for agentId {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0013 | Deserialization error in Data Gateway response to get interactive session for agentID, {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0014 | Job execution failed for agentId {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0015 | Failed to generate code for node {}. DataFlow ID, {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0016 | The following parameters cannot but null, {} {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0017 | FlowNode {} does not exist in DataFlow {}. | Reopen the data flow and retry the same operation. If the issue persists, contact Oracle Support. |
DIS_DATA_XPLORER_0018 | Data Gateway error: {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0019 | Code Decoration Error for pipeLineName {}. | Contact Oracle Support. |
DIS_DATA_XPLORER_0020 | Call to Data Gateway failed for ID, {}. Try again. | Reopen the data flow and retry the same operation. If the issue persists, contact Oracle Support. |
DIS_DATA_XPLORER_0021 | Attempt to get interactive job run result failed. Invalid payload: {}. | - |
Data Execution
Code | Message | Resolution |
---|---|---|
DIS_EXEC_0001 | Unable to retrieve workflow with id, {}. | Contact Oracle Support. |
DIS_EXEC_0002 | Unable to update workflow with id, {}. | Contact Oracle Support. |
DIS_EXEC_0003 | Unable to delete workflow with id, {}. | Contact Oracle Support. |
DIS_EXEC_0004 | Unable to create workflow {}. | Contact Oracle Support. |
DIS_EXEC_0005 | Workflow object is not available in the repository for the given task id {}. | Contact Oracle Support. |
DIS_EXEC_0006 | Unable to retrieve task {} in Application with id, {}. | Check that the correct Application ID and task ID were used. |
DIS_EXEC_0007 | Unable to dereference task {}. | Contact Oracle Support. |
DIS_EXEC_0008 | Unable to list applications. | Contact Oracle Support. |
DIS_EXEC_0009 | Unable to retrieve summary for task {}. | Check that the correct task ID was provided. |
DIS_EXEC_0010 | An exception occurred while fetching ApplicationMetrics with multiple threads for this workspace. | Contact Oracle Support. |
DIS_EXEC_0011 | An exception occurred while updating the action run {}. Try again. | Contact Oracle Support. |
DIS_EXEC_0012 | Unable to terminate task run {}. Termination is not supported for any task run with status {}. | You can't terminate a Task Run that is in the TERMINATING or NOT_STARTED state. |
DIS_EXEC_0013 | TaskRun update failed. Another attempt to update will be made. | Contact Oracle Support. |
DIS_EXEC_0014 | Unable to update status of task run {} to {}. Another attempt to update will be made. | Contact Oracle Support. |
DIS_EXEC_0015 | There are no associated agents for instance {}. | Contact Oracle Support. |
DIS_EXEC_0016 | Unable to retrieve Task Run logs for task run {}. | Check that the correct task ID was used. |
DIS_EXEC_0017 | The operator is not a SparkExecutor operator. | Contact Oracle Support. |
DIS_EXEC_0018 | Unable to terminate task run {} with status {}. | Contact Oracle Support. |
DIS_EXEC_0019 | Workflow {} doesn't contain any nodes. At least one decomposable task operator is expected. | Contact Oracle Support. |
DIS_EXEC_0020 | Unable to find agent with id, {}. | Contact Oracle Support. |
DIS_EXEC_0021 | Unable to list agents. | Contact Oracle Support. |
DIS_EXEC_0022 | Unable to terminate task run {}. | Contact Oracle Support. |
DIS_EXEC_0023 | Unable to send message to agent {}. | Contact Oracle Support. |
DIS_EXEC_0024 | The executable workflow is null for task run with id, {}. | Contact Oracle Support. |
DIS_EXEC_0025 | Unable to generate decomposed workflow for task {}. | Contact Oracle Support. |
DIS_EXEC_0026 | Unable to create instance of class {}. | Contact Oracle Support. |
DIS_EXEC_0027 | The aggregator id for the task {} is null. | Contact Oracle Support. |
DIS_EXEC_0028 | Unable to perform Gzip compression. | Contact Oracle Support. |
DIS_EXEC_0029 | Unable to serialize the object {}. | Contact Oracle Support. |
DIS_EXEC_0030 | There was an issue with the config providers in SparkRuntimeDTO. | Contact Oracle Support. |
DIS_EXEC_0031 | Unable to create workflow run {}. | Contact Oracle Support. |
DIS_EXEC_0032 | Unable to create TaskRun object for task {}. | Contact Oracle Support. |
DIS_EXEC_0033 | Unable to serialize agent message payload for agentId {} and workspaceId {}. | Contact Oracle Support. |
DIS_EXEC_0034 | Unable to submit terminate request to agent for task run {}. | Contact Oracle Support. |
DIS_EXEC_0035 | Unable to create TaskRunLog Object for task run {}. | Contact Oracle Support. |
DIS_EXEC_0036 | Unable to create Workflow deletion validator for workspace {}. | Contact Oracle Support. |
DIS_EXEC_0037 | Unable to retrieve workflow deletion validator for workspace {}. | Contact Oracle Support. |
DIS_EXEC_0038 | Error while fetching Task {} published to application {} in workspace {}. | Contact Oracle Support. |
DIS_EXEC_0039 | Error retrieving pipeline objects during run of task {} published to application {} in workspace {}. | Contact Oracle Support. |
DIS_EXEC_0040 | Error retrieving connectivity parameters { Data Assets / Connections
/ Data Entities } for task with uuid, {}, published to application {},
in workspace {}. Validation Error: {} |
Contact Oracle Support. |
DIS_EXEC_0041 | Unable to post message to agent, retry attempt with different agent also exceeded. | Contact Oracle Support. |
DIS_EXEC_0042 | Unable to list Task Runs in Application. | Contact Oracle Support. |
DIS_EXEC_0043 | Unable to get Task Run with id, {}. | Check that the correct Task Run ID was used. |
DIS_EXEC_0044 | Unable to create InstanceMetric object for Task Run {}. | Check that the correct Task Run ID was used. |