Time-series Anomaly Detection

Objective

This document provides instructions on how to enable anomaly detection using time series analysis on the metrics of your application. The F5® Distributed Cloud Time Series Anomaly (TSA) detection is supported for the Request Rate, Error Rate, Latency, and Throughput (RELT) metrics. The Distributed Cloud TSA is performed using advanced machine learning upon enabling through configuration. To know more about the TSA concepts, see Behavioural Firewall.

The TSA detection monitors and alerts about the following types of abnormal traffic patterns:

  • Unusually large spikes (DoS attack and genuinely high traffic)
  • Sudden drops — may indicate reachability issues
  • Seasonality patterns — these are periodic patterns
  • Missing periodic peaks — may indicate problems with client application
  • Unexpected peaks or drops

Note: The time series analyses include learning time of day usage patterns.

Using the instructions provided in this document, you can enable the TSA detection for metrics of your application and monitor the related detected anomalies in the Distributed Cloud service mesh.


Prerequisites

Note: If you do not have an account, see Create an Account.

  • One or more applications deployed on Distributed Cloud site and services configured.

Note: See App Management for more information. See Site Management for site creation instructions.


Configuration

The following image describes the configuration work-flow for enabling TSA detection for your application metrics:

seq tsa
Figure: Work-flow for Enabling TSA

Configuration Sequence

The following table presents the sequence of activities in enabling the TSA detection:

Activity Description
Create App Type Create app type and configure the TSA features.
Create App Settings Select metrics and components such as nodes, edges, or load balancers for TSA and associate them with the app type.
Monitor Anomalies and Alerts Monitor the service mesh or load balancer to check for anomalies detected and reported by TSA.

Create App Type

To enable anomaly detection for your application services, it is required to first enable TSA for those services using the app type object.

The app type object is created in the shared namespace. The load balancers of that app type in different namespaces need to be assigned with the label of the app type object.

Perform the following to create app type and enable generating the anomaly model.

Step 1: Navigate to the App Types page.
  • Select the Shared Configuration service.
  • Navigate to Security -> AI & ML -> App Types.

nav atype
Figure: Navigate to App Type Configuration

Step 2: Configure app type object settings.

Click Add app type and enter the configuration in the app type object creation form using the following guidelines:

app type cnf
Figure: App Type Feature Configuration

  • Enter a name for the app type. This is the value for the app type label to be assigned to the load balancers for which the TSA needs to be enabled.

  • Click Add item in the Features section and select a type for the AI/ML Feature Type from the drop-down list using the following guidelines:

    • Select API Discovery for enabling analysis on interactions between the services.
    • Select Timeseries Analysis for enabling analysis on RELT metrics.
    • Select Per API Request Analysis for enabling detection per API request.
    • Select User Behavior Analysis for enabling user behavior analysis.

Note: You can add all the features using the Add item option.

  • Optionally, select Enable learning from redirect traffic option for the Learn from Traffic with Redirect Response field in the Business Logic Markup Setting section.

  • Click Save and Exit to complete creating the app type object.


Assign App Type Label to Load Balancers

After creating the app type, it is required to assign the app type label to the load balancers for which you want to enable TSA detection.

Note: Enabling TSA detection for all load balancers in a namespace requires you to apply the app type label to all load balancers in that namespace.

Perform the following to assign the app type label to your load balancers.

Step 1: Navigate to load balancer management.
  • Select the Load Balancers service.
  • Select the desired namespace from the Namespace drop-down menu.
  • Navigate to Manage -> Load Balancers -> HTTP Load Balancers.
  • Click ...->Edit for the load balancer for which the app type label needs to be assigned.

lb edit
Figure: Navigate to load balancer Edit Configuration

Step 2: Assign the app type label.
  • Select ves.io/app_type for the Labels field and type.

at label
Figure: App Type Label Selection

  • Type the name of the app type object created in the previous step and click Assign Custom Value to add the app type label.

label value
Figure: App Type Label Addition

  • Click Save and Exit to apply the label to the load balancer.

Create App Settings

After creating an app type with the TSA feature enabled, it is required to associate it with the metrics and sources for which the anomaly detection is required. This is done by configuring the app settings object.

The metrics are RELT metrics and sources are of the following types:

  • Services
  • Service interactions
  • load balancers

Perform the following to create the app type object.

Step 1: Navigate to the App Settings page.
  • Select the Load Balancers service.
  • Select the desired namespace from the Namespace drop-down menu.
  • Navigate to Security -> AI & ML -> App Settings.
  • Click Add App setting for the load balancer for which the app type label needs to be assigned.

nav asetting
Figure: Navigate to load balancer Edit Configuration

Step 2: Enter configuration for the app settings object.
  • Enter a name for the app setting.
  • Go to Application Type Feature Configuration section and click Add item to configure an AppType.
  • In the AppType field, select the created app type object from the drop-down list.
  • Click Configure under the Timeseries Analysis Setting field. Click Add item in the Metric Selectors section of the time series analysis setting page.
  • Select an option for the Metrics Sources field from the list of options.
    • Select All Services for enabling metric analysis for all services.
    • Select All Service Interactions for enabling analysis for all service interactions between source and destination services.
    • Select All Virtual Hosts for enabling metric analysis for all virtual hosts.
  • Select an RELT metric for the Metrics field from the list of options.
  • Click Add Item to complete the Metric Selector.
  • Click Apply to finish adding Metric Selectors.

Note: You can add multiple metric selectors by using the Add Item button in the Metric Selectors list.

tsa metrics
Figure: Time Series Configuration for App Settings

  • Click Add Item to complete the AppType Setting.
Step 3: Complete app settings object creation.

Click Save and Exit to complete adding the Application Type Feature.

app settings final
Figure: App Settings Object Creation

Note: You can add multiple app settings using the Add item option.


Monitor Anomalies and Alerts

TSA detection happens based on your selection of sources in app settings and app type configuration. You can monitor the anomalies using the metrics or alerts or both. The TSA gets detected and displayed for service mesh or load balancer or both depending on your TSA configuration.

Step 1: Navigate to service mesh.
  • Select the Load Balancers service.
  • Select the desired namespace from the Namespace drop-down menu.
  • Navigate to Service Mesh -> Mesh.
  • Click ...->Edit for the load balancer for which the app type label needs to be assigned.
  • Click on your application tile from the displayed list to load its service mesh monitoring.

nav sm
Figure: Navigate to Service Mesh

Step 2: Load the service mesh metrics view.

Click Metrics tab to load the metrics view.

The metrics view presents trend information for your service metrics for a default or configured time period.

When the TSA is enabled for metrics, a shadow is shown over the metrics bars. This is called a Confidence interval. The confidence interval indicates that the metric value crossing this interval is treated as an anomaly. Such instances are marked in red color bars. Hover over or click on any bar to display the metric and confidence interval values.

sm metrics
Figure: TSA Enabled Service Mesh Metrics

Step 3: Load the service mesh alerts view.

The service mesh loads service graph by default. Click Alerts tab to load the alerts view.

Active alerts are displayed by default. Select the All Alerts option to display all alerts for default interval of an hour. You can also change time interval using the Last 1 hour drop down. The value Timeseries-Anomaly for the Group field indicates that the alert is an anomaly. The TSA alerts are generated for sustained anomalies.

Click > for any alert entry to load details in the JSON format.

sm alerts
Figure: Service Mesh TSA Alerts

Note: See TSA Alerts for information on time-series related alerts.

Step 4: Navigate to the load balancer monitoring.

In the Load Balancers service, select Virtual Hosts -> HTTP Load Balancers. Click on your load balancer from the displayed list to load its monitoring view. The load balancer dashboard is loaded by default.

Step 5: Load the load balancer metrics view.

The load balancer dashboard is loaded by default. Click Metrics tab to load the metrics view.

The metrics view presents trend for your load balancer metrics for a default or configured time period.

When the TSA is enabled for metrics, a shadow is shown over the metrics bars. This is called a Confidence interval. The confidence interval indicates that the metric value crossing this interval is treated as an anomaly. Such instances are marked in red color bars. Hover over or click on any bar to display the metric and confidence interval values.

vh metrics
Figure: TSA Enabled load balancer Metrics

Step 6: Load the load balancer alerts view.

Click Alerts tab to load the alerts view.

Active alerts are displayed by default. Select the All Alerts option to display all alerts for default interval of an hour. You can also change time interval using the Last 1 hour drop down. The value Timeseries-Anomaly for the Group field indicates that the alert is an anomaly. Click > for any alert entry to load details in the JSON format.

vh alerts
Figure: load balancer TSA Alerts

Note: See TSA Alerts for information on time-series related alerts.


Concepts


API References