If you're using only the recent logs for your search and analysis tasks in Oracle Logging Analytics, then enable archiving so that you can optimize the storage cost.

1. Open the navigation menu and click Observability & Management. Under Logging Analytics, click Administration. The Administration Overview page opens.

2. The administration resources are listed in the left hand navigation pane under Resources. Click Storage.

   The Storage page is displayed.

3. Click Enable Archiving. In the Enable Archiving dialog box, enter the count of the days after which the log data in the active storage must be archived in the field Active Storage Duration (Days), and click Enable.

   The count is calculated based on the timestamp of the logs. For example, if your logs have the timestamp November 4, 2020 23:43:12, and you've specified the Active Storage Duration as 30, then the logs will be typically moved to archive storage on December 3, 2020.

   Note
   
   

   It must be noted that even if you specify the Active Storage Duration of the logs to determine the logs that must be moved to Archive storage, the log index structure is based on the buckets that are used for storing the logs. In a typical scenario, an entire bucket is moved to the archive storage when all the logs in it are older than the specified criterion.

   For example, consider that the field Active Storage Duration is set to 30 days:

   • Bucket_1 has logs of age 40 - 80 days: The log data is eligible and is moved to archive storage.
   • Bucket_2 has logs of age 25 - 40 days: Although some of the log data is eligible for archiving, it is not archived until all the logs are suitable for the specified age.
   • Bucket_3 has logs of age 0 - 25 days: None of the logs are suitable for archiving. The entire bucket is archived when all the logs become eligible.

   In the above scenario, after Bucket_1 logs are archived, if more logs are collected which are older than 40 days, then they are typically appended to Bucket_2.

4. If you have enabled archiving already, and want to modify the archiving settings, then click Modify Archiving Settings. You can perform any of the following tasks:

   • You can change the value of the count of the days specified for archiving under Active Storage Duration (Days).
   • Click Disable Archiving to stop archiving.

   Click Save Changes.

If you want to use the logs that are archived for viewing and analysis, then you can recall the logs. The recalled data will count towards your active storage usage until you release it.

You can recall and release your selected set of logs multiple times. However, the recall feature is enabled only if you already have archived logs.

1. Open the navigation menu and click Observability & Management. Under Logging Analytics, click Administration. The Administration Overview page opens.

2. The administration resources are listed in the left hand navigation pane under Resources. Click Storage.

   The Storage page is displayed.

3. In the Storage page, on the left panel under Resources, click Archiving Recall Requests.

   The Archiving Recall Requests page displays the previously initiated recall requests.

4. Click Create Recall Request. The Create Recall Request dialog box opens.

5. Specify the Purpose of recall. This can help you to identify your recall request.

6. Optionally, if you have defined log set, then you can specify one or more Log Sets to filter the recalled data. To specify multiple log sets, use comma separation.

7. Select the time range of the logs that you want to recall, by specifying the User-defined start time and User-defined end time.

8. Click Estimate Recall Log Size. The Data set recommended for analysis section opens. The size of the logs that you've selected for recall is displayed adjacent to the heading Maximum recalled data size before filtering.

   Note that the start time and end time are extended to align with the log index structure based on buckets. So, when you view the list of active recalls or visit the activity tab, you may get the start and end time extended beyond your chosen time range.

   If your current recall time specifications overlap with another recall activity, then they can possibly get merged into a single recall activity and the resulting start and end time can get extended.

9. An alternative time range is recommended based on the availability of data. To select the time range you specified earlier instead of the recommended time range, enable the check box Do not use recommended data set for recall

10. Specify the Query to filter the data set. Exclude the time and log set from the query.

    Note that applying the filter does not impact the size of the data set estimated based on the time range.

11. Click Create Recall Request to proceed with the recall of the selected logs.

    The recall activity is listed in the Archiving Recall Requests page. The table specifies the status, time range, data size, and request date and time of recall activity, user who initiated the recall, and the purpose of recall. The individual recalls that have overlapping data are combined into a single collection. In such cases, the table displays the data size of the collection instead of the data size of the underlying recalls.

    Note
    
    

    If you keep the recommended and default data set for each recall, then the collection time range is the super set of the time ranges of the individual recalls. Otherwise, the collection time range may not be the super set of the individual recall time ranges.

    Watch the status of the recall activity. You can use the recalled logs for viewing and analysis after the recall activity is complete.

    If the data size icon for a collection is displayed in orange, then new additional log data is available for recall. Click the data icon and click Recall new data to initiate the recall of the new data. The Recall new data dialog box opens. The query to filter the data set and the time range for data recall are predefined. Specify the purpose of recall and click Create Recall Request.

12. After active use of the recalled logs, if you want to release them back to the archive pool, click the actions menu icon in the row corresponding to your recalled logs, and select Release.

    The recalled logs will then be released back into the archive pool. This will enable you to optimize your storage size and cost.

    Note
    
    

    When releasing the recalled logs using REST API, note the recall time range from console or CLI, and format the time as follows:

    • Recall start time: Round down (floor) the value. If the recall start time is From Mon, Mar 7, 2022, 05:45:33 UTC, then round down the time and specify it as from_time=2022-03-07T5:45:32.000Z.
    • Recall end time: Round up (ceil) the value. If the recall end time is To Wed, Mar 15, 2023, 17:26:53 UTC, then round up the time and specify it as to_time=2023-03-15T17:26:54.000Z.

Oracle Logging Analytics lets you purge log events that were loaded by agent or by an on-demand upload, to reduce the index size of the log data.

1. Open the navigation menu and click Observability & Management. Under Logging Analytics, click Administration. The Administration Overview page opens.

   The administration resources are listed in the left hand navigation pane under Resources. Click Storage.

   The Storage page is displayed.

2. In the Storage page, you can purge log data in one of the following methods:
   • Perform on-demand Purge:

     Click Purge Logs. The Purge Logs dialog box is displayed.

     • Select the compartment in which the logs must be purged.

     • Specify if the subcompartments also must be included in the Purge.

     • Select the date and time prior to which the log data that must be purged was collected.

       Purge action is performed on the log data from all the buckets in the selected compartment which were collected prior to the specified time period. For example, if you specify the date and time as November 2, 2020 12:00:00 and compartment Analyze, then the log data with the time stamp older than November 2, 2020 12:00:00 stored in the compartment Analyze is deleted.

     • In the Query field, enter the query to select a specific set of log data. For example, to select the logs from the entities of the type Linux Host, specify the query 'Entity Type'='Host (Linux)'.

       See Example Queries for Purging Log Data.

     • Click Estimate Reclaimed Storage to determine the size of the storage that can be reclaimed based on the selection you made in the previous fields.

     • Click Purge.

   • Create a purge policy to purge logs based on a query or age:

     Under Purge Policies, click Create. The Create Purge Policy dialog box opens.

     • Enter a name for the new purge policy.

     • Select the log group compartment to query for the logs. You can optionally specify if the subcompartments must also be queried for the specified logs.

     • Under Purge Logs Older than, select the time period from when the log data must be purged.

     • Under Schedule Interval, select the periodicity, and time of the purge action.

     • In the Query field, enter the query to select a specific set of log data. For example, to select the logs from the source Apache HTTP Server Access Logs, specify the query 'Log Source'='Apache HTTP Server Access Logs'.

       See Example Queries for Purging Log Data.

     • Click Estimate Reclaimed Storage to determine the size of the storage that would be reclaimed if the selections you made in the previous fields were to be applied now.

     • Optionally, click Show Advanced Options and add tags to your purge policy.

     • Click Create.

     The purge policy is created and will be run periodically as set in the previous steps.

     Note
     
     

     If a purge policy is paused due to lack of permissions, after modifying the policy statements as required, manually Resume the purge task.

   To delete a policy, click Actions icon next to the policy name, and click Delete.

   To view the purge activities performed, in the Storage page, under Resources, click Activity Report. The Activity Report page is displayed which summarizes all the storage activities. Use the Status and Time filters to view the preferred purge activities.

You can view the summary of your archive, recall, release, and purge activities to maintain close control of your storage use and also to track the status of your key logs that have been part of the activities.

1. Open the navigation menu and click Observability & Management. Under Logging Analytics, click Administration. The Administration Overview page opens.

2. The administration resources are listed in the left hand navigation pane under Resources. Click Storage.

   The Storage page is displayed.

3. In the left panel under Resources, click the Activity Report.

   The page displays the summary of the storage activities initiated such as purge policy, purge on demand, archiving, archiving recall request and recall release.

4. Use the Activity Type, Status, and Time filters on the left panel to narrow down your search for the storage activities.

5. Expand the storage activity row to view more details about it.