Enable Filters

Filter Options dialog box displays the list of fields used in the timestats chart. Enable the check box Show Search Filters. You can select one or more fields to display in the Filter panel. For example, consider the following search:

*
| eval 'Raw Size (bytes)' = unit('Raw Size', byte)
| link Time, 'Entity Type'
| timestats name = 'Entity Types'
            sum(Count) as 'Number of Logs',
            sum('Raw Size (bytes)') as 'Log Size'
        by 'Entity Type'

The Filter Options dialog box would now show the following fields generated by the timestats command and the fields used in the by clause: Entity Type, Number of Logs, Log Size.

In the fields panel, you can select one or more filters to view only those specific data points. You can reset the filter by toggling the Select All option.

After the filter is enabled for a specific field, to view the complete list of values for a string field, click the Settings icon next to the field name.

Customize Filters

Access the Customize Filters dialog box from the Settings menu next to the title of your time series chart.

In case of numeric fields, you can also customize the filter by clicking the Settings icon next to the numeric field name. The Customize Filter dialog box opens. The following Range Calculation Method options are available:

• Default: By default, the range is fixed to one-fifth of the maximum value and then rounding it to 2s, 3s, 4s, 5s, or 10s. For example, if the maximum value is 48, then the range would be 10. Then the buckets would be calculated as Below 0, Up to 9, 10 - 19, 20 - 29, 30 - 39, 40 - 49, Above 50.

• Logarithmic: To use logarithmic ranges such as 1 - 10, 10 - 100, and 1000 - 10000, change the range calculation method to logarithmic.

• Custom: Certain data sets need variable ranges. For example, consider the requirement to bucket CPU Average as 0 - 30% for under utilization, 30 - 80% as effective utilization, 80 - 95% as warning, and anything above 95% is extreme end. To configure variable ranges for this, you can provide comma-separated values 30,80,95. This creates the buckets 0-30, 30-80, 80-95, 95+.

You can optionally enable the check box Bucket Values <= 0 Separately to create a separate bucket for the field values less than or equal to zero.

Search in the Filter

Access the Customize Filters dialog box from the Settings menu next to the title of your time series chart.

For the string fields, you can key in the text for search. You can also provide regular expressions for searching. For example, searching for e$ will show all the field values that end with the letter e.

Check the Invert option to display only those values that don't match the criteria.

Understanding the Link timestats Syntax

In the detailed time series use case Time Series Analysis Using timestats Command, the following query is used:

'Log Source' = 'OCI VCN Flow Unified Schema Logs' 
| link span=1day Time, Action
| timestats name="Trend of Action" sum(Count) as Records by Action

Parts of the timestats command in the above example looks as below:

The following parameters can be used in the command syntax:

*
| eval 'Raw Size (bytes)' = unit('Raw Size', byte)
| link Time, 'Log Source'
| timestats avg('Raw Size (bytes)') as Size,
            sum(Count) as Records
         by 'Log Source'

Using the Fields in the Link timestats

There are two types of fields that can be used for a timestats function:

• Property field: Any field that is used in the link command, or created using stats, eventstats, eval, lookup, nlp, or delta commands after link.

• Log record field: A field that is defined in the log source, and exists in the log record. You can directly refer to such fields from a timestats function.

Only property fields are allowed in a by-clause. Large fields like Message, Original Content, Error Stack are not allowed in a function or in a by-clause.

Charting a subset of logs: Use the addfields command to generate charts for a specific subset of data. The fields used in addfields must be a property field. The following example illustrates the use of addfields:

*
| link Time, 'Log Source'
| addfields 
   [ * | where 'Log Source' in ('Linux Audit Logs', 
                                'Linux Secure Logs', 
                                'Linux Syslog Logs', 
                                'Linux Cron Logs') 
       | timestats name = 'Linux Logs' sum(Count) as Records by 'Log Source' 
   ],    [ * | where 'Log Source' not in ('Linux Audit Logs', 
                                          'Linux Secure Logs', 
                                          'Linux Syslog Logs', 
                                          'Linux Cron Logs') 
             | timestats name = 'Non Linux Logs' sum(Count) as Records by 'Log Source' 
   ]

Specifying the number of charts to be returned: Use the topcount or bottomcount parameters to specify the number of charts to be returned back to the UI.

• topcount: When grouping by fields, return N count of distinct groups with the largest aggregated values.
• bottomcount: When grouping by fields, return N count of distinct groups with the smallest aggregated values.

For example:

*
| link Time, 'Log Source'
| timestats topcount = 3 name = 'Top 3 Log Sources by Count'
            sum(Count) as 'Log Records'
        by 'Log Source'
| addfields
     [ * | where 'Log Source' in ('Linux Audit Logs', 'Linux Secure Logs', 'Linux Syslog Logs', 'Linux Cron Logs')
         | timestats bottomcount = 3 name = 'Bottom 3 Linux Logs'
                     sum(Count) as Records
                   by 'Log Source' ]

The following charts are generated as the result of running the above query:

Limits for Time Series

• Number of fields in the by clause: Only up to four fields are allowed in the by-clause.

• Allowed Fields: Refrain from using large fields in the function and in the by clause. Examples of large fields include the Message field, Supplemental Details, Error Stack and Original Log Content fields.

• Null Values: Timestats aligns the values in the link table using the values in the Start Time column. For example, a Duration field may have values only for 10:06 AM, 10:10 AM and 1:12pm. As part of the alignment, a timestats avg(Duration) command would average the 10:06 AM and 10:10 AM values into a single average for 10:00am. Since there are no values for 11am and 12pm, they are filled with zeros. The 1:12pm value is placed at the interval 1 PM.

  The actual alignment interval depends on the interval used for the query. An interval can be explicitly specified using the span parameter.

• Number of time series values: Only 100 series per timestats command is returned. Here are some examples that illustrate this limit:

  • The command timestats sum(Count) as Records by 'Log Source' returns only the top 100 Log Sources. The top 100 are identified by first sorting the values of each Log Source time series, and then sorting these time series. This means, if Records for a Log Source contains all zeros and a single large spike, then that is included in the returned results.
  • The command timestats sum(Count) as Records by 'Log Source', Label returns the top 100 unique combinations of the Log Source and Label.

  If partial results are returned, then an info icon next to the time series indicates the total number of series generated.

  The following are some of the options to handle large number of time series values:

  • Use the addfields command to filter the specific values to be used in timestats.
  • Use topcount or bottomcount to return the top 100 or bottom 100 charts.
  • Use the timecluster command to cluster the time series. This reduces the number of charts, since only representative samples are returned.

Configuration Options for the Time Series Charts

Similar to the histogram charts, the following options are available for you to explore using the charts. Click Chart Options icon :

• Hide / Show Charts: You can hide or show a chart.

• Selecting the Chart Type: You can select from different chart types.

• Adjusting the Chart Height and Width: You can increase the height of the chart using the Height option. Drag the Width control to increase or decrease the width of the charts. If there are more than one function, then reducing the width would cause more charts to be displayed in the same row. At full width, all the charts are displayed in individual rows. Reducing the width causes the charts to be arranged in multiple column order.

• Correlating across Charts: You can use the Chart Option Show Correlated Tooltips to show correlated tool tips across charts.

Additionally, you can also use the following options to configure your charts:

• Legend Control: You can turn on or turn off the legends using the Show Legend option in Chart Options.

• Chart Filters: You can use the filters to interactively analyze the time series. Enable the filters by clicking Chart Options, Filters, and Show Search Filters. The filter options show the list of fields used in the timestats chart. You can select one or more fields to be displayed in the Filter panel. After you select the Filter options, the fields generated by the timestats command as well as the fields used in the by clause are displayed above the chart.

  Select one or more filters to view only those specific data points. You can reset the filter selection by toggling the Select All option.

• Search and Customize Filters: Click Options icon next to a filter name to view the complete list of values for that filter. You can enter the text for search. You can also provide regular expressions for searching. For example, searching for e$ will show all the results that end in the letter e.

  Check the Invert option to display only those values that do not match the criteria.

• Select Colors: You can configure the color for each time series.

  Consider the following query that shows the trend of each Status in the OCI Audit Logs:

  Status != null and 'Log Source' = 'OCI Audit Logs'
  | link Time, Status
  | timestats sum(Count) as 'Number of Hits' by Status

  The chart displays the Status values and their default color. The values of the Status field are 200, 201, 404, 409, 204, and 400.

  
  
  
  
  

  Enable the filters using Chart Options to view the color option next to the chart.

  
  
  
  
  

  Click the Color link, select Status field, and select a palette that has as many colors as the values to configure.

  
  
  
  
  

  Update each color expression with the value or a regular expression you that want to match.

  
  
  
  
  

  You can repeat this process for each field in the Color drop down to configure the colors.

Understanding How Link timecluster Works

Clustering looks at each time series that has similar shape and values. Following are some of the characteristics used to cluster the time series:

• Peaks and Valleys in the time series: Time series with similar peaks and valleys tend to be clustered together.

• Values at each point: Even if two time series have similar spikes, they can still fall into different clusters, if the values are far apart.

Each cluster gets the following properties that are displayed in the console:

timecluster has similar syntax and options as the timestats command. For understanding the syntax and the configuration options for the time series charts, see Understanding the Link timestats Syntax and Configuration Options for the Time Series Charts.

For an example use case of plotting a time series using timecluster, see Time Series Clustering.