Configure Data Intelligence

Distributed Cloud Data Intelligence processes log data collected by F5 Distributed Cloud Services and uses it to equip security and fraud prevention teams with comprehensive insights into user behavior, device identification, and network data. This enables the early detection of anomalies indicative of potential security threats and fraudulent activities.

Integrate Data Intelligence with your SIEM solution or Fraud Orchestration tools and use the log data processed by Data Intelligence along with additional context and signals from your other data sources to build better rules and models and to improve the quality of data and decisions.

Data Intelligence can send log information to the following data collection systems:

• AWS S3
• Splunk
• Datadog
• Azure Blob Storage
• GCP Buckets
• HTTPS

Configure Data Intelligence Data Delivery

Depending on the edition of Bot Defense that you have, there are two ways to configure Data Intelligence Data Delivery:

• If you have Bot Defense Standard Edition, follow the instructions in the sections below to enable and configure Data Intelligence Data Delivery. To get started, see the prerequisites.

• If you have Bot Defense Advanced Edition, you can contact your assigned Technical Account Manager for assistance enabling and configuring Data Intelligence Data Delivery. Your Technical Account Manager can perform most of the configuration process for you and guide you on how to get started with Data Intelligence Data Delivery.

  You can also optionally decide to configure Data Intelligence Data Delivery yourself. To get started yourself, see the prerequisites.

Prerequisites

• You must have a valid Distributed Cloud Console account. If you do not have a valid account, see Create a Distributed Cloud Console Account.
• You must have an Organization plan. To see plan information, from the Distributed Cloud Console Home page, click Billing.
• You must have an externally available data collection system.
• You must add the following IP addresses to the allow list on your firewall:
  • 35.247.100.206
  • 34.168.102.111
  • 34.168.48.1
  • 34.82.248.13
  • 35.230.48.185
  • 35.203.143.95

Enable Data Intelligence

To use Data Intelligence, you must first enable it in the Distributed Cloud Console.

1. On the Distributed Cloud Console Home page, click Billing.
2. Click Manage > Billing Plan and scroll to the Organization Plan.
3. Under the Organization Plan, click Data Intelligence. The Data Intelligence landing page appears.

   Note: If Data Intelligence is already enabled, the Data Receivers page appears, and you can begin using Data Intelligence.

4. From the Data Intelligence landing page, click Request Service.

Add a Data Receiver

Configure a data receiver to connect Distribute Cloud Services to your externally accessible data collection system.

1. Log on to Data Intelligence in the Distributed Cloud Console.
2. Click Manage > Data Delivery and then click Add Receiver.
3. Add a Name to identify the new data receiver and optionally add a Description.
4. From the Data Set drop-down menu, select the data set you want to send to your data collection system.
5. From the Receiver Configuration drop-down menu, select the data collection system to which you want to send data.
6. To finish configuring your data receiver, see one of the following sections:
   • Configure an AWS S3 Receiver
   • Configure a Splunk Receiver
   • Configure a Datadog Receiver
   • Configure an Azure Blob Storage Receiver
   • Configure a GCP Bucket Receiver
   • Configure an HTTPS Receiver

Important: When you configure your data receiver, for better security, F5 recommends that you select Blindfolded Secret instead of Clear Secret so that Data Intelligence does not have access to your password.

Configure an AWS S3 Receiver

1. Complete the steps in Add a Data Receiver.

2. In the S3 Bucket Name field, enter the exact name of the AWS S3 bucket where you want to send log data.

3. From the AWS Cloud Credentials drop-down list, select the cloud credentials you want Distributed Cloud Services to use to access the AWS S3 bucket

   If you need to add new credentials, from the AWS Cloud Credentials drop-down list, click Add Item. To add an AWS Assume Role credential, see Configure AWS Assume Role Authentication.

   Note: You must select credentials that are the AWS Programmatic Access Credentials type.

   In the AWS Region drop-down list, select the region where you configured your S3 storage bucket.

   

   Figure: Configure an AWS S3 Receiver

4. To save your AWS S3 receiver, click Save & Continue.

Configure AWS Assume Role Authentication

To help facilitate secure data delivery, you can add permissions to allow a specific IAM user in the F5 Distributed Cloud AWS account to assume a role in your AWS account. To configure AWS Assume Role Authentication, you must create a role and attach a trust policy in the IAM section of the AWS Console and then delegate it to the F5 Distributed Cloud AWS account.

1. Obtain the following service account and keys required for the Assume Role:

   • F5 Distributed Cloud AWS account number

   • Your F5 Distributed Cloud tenant ID

   Note: To request the F5 AWS account number, contact F5 Support.

2. Create AWS Assume Role with the custom trust policy.

   1. In the AWS Console, click IAM > Roles.

   2. Click Create Role and start creating the Assume Role.

   3. Select Custom trust policy and add the following JSON:

      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Principal": {
                      "AWS": "arn:aws:iam::<account-number>:user/svc-acct-f5xc-bot-and-risk-anagement"
                  },
                  "Action": "sts:AssumeRole",                  
              },
              {
                  "Effect": "Allow",
                  "Principal": {
                      "AWS": "arn:aws:iam::<account-number>:user/svc-acct-f5xc-bot-and-risk-anagement"
                  },
                  "Action": "sts:TagSession"
              }
          ] 
      }
      

      Copied!

      Note: Replace <account-number> with the F5 Distributed Cloud AWS account number.

   4. Create an inline role with the following permissions to push data to your AWS S3 bucket.

      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Sid": "Statement1",
                  "Effect": "Allow",
                  "Action": [
                      "s3:ListAllMyBuckets",
                      "s3:ListBucket",
                      "s3:GetObject",
                      "s3:PutObject",
                      "s3:PutObjectTagging",
                      "s3:TagResource"
                  ],
                  "Resource": [
                      "arn:aws:s3:::<<bucket name>>"          
        ]
              }
          ]
      }
      

      Copied!

   5. Finish creating the role in the AWS Console and record the ARN value (role_arn) that you created. You need the role_arn when you add the Assume Role credential in the Distributed Cloud Console.

3. Log in to the Distributed Cloud Console and add an Assume Role credential.

   1. When you add a AWS S3 data receiver, from the AWS Cloud Credentials drop-down menu, select Add Item.
   2. From the Select Cloud Credential Type drop-down menu, select AWS Assume Role.
   3. In the IAM Role ARN field, enter the same ARN value that you created in the AWS Console.
   4. In the Role Session Name field, enter a name for the role.
      

      Figure: Configure AWS Assume Role

   5. Click Add Cloud Credentials to save the credential.

Configure a Splunk Receiver

1. Complete the steps in Add a Data Receiver.

2. In the Data Delivery section, in the Splunk HEC Logs Endpoint field, enter the name of the Splunk HTTP Event Collector (HEC) that you want use to send log data to your Splunk deployment.

   

   Figure: Configure a Splunk Receiver

3. In the Splunk HEC token section, click Configure. Distributed Cloud Services uses an HEC token to authenticate with the HEC.

4. From the Secret Type drop-down list, select one of the following options:

   • Clear Secret: Enter the password you want to use. If you select Clear Secret, F5 strongly recommends that you use Base64 format.

   • Blindfolded Secret (recommended): From the Action drop-down list, select one of the following options:

     • Use Existing Blindfolded Secret: Enter the Blindfolded Secret.
     • Blindfold New Secret:
       1. From the Policy Type drop-down list, select Built-in.
       2. In the Secret to Blindfold field, enter your Splunk HEC token.
     

     Figure: Configure a Splunk HEC Token

5. Click Apply.

6. To save your Splunk receiver, click Save & Continue.

Configure a Datadog Receiver

1. Complete the steps in Add a Data Receiver.

2. In the Data Delivery section, from the Endpoint Selection drop-down list, select Datadog Site.

3. In the Datadog Site field, enter the URL for DataDog, datadoghq.com.

   

   Figure: Datadog Image

4. To configure the Datadog API key, click Configure.

5. From the Secret Type drop-down list, select one of the following options:

   • Clear Secret: Enter the password you want to use. If you select Clear Secret, F5 strongly recommends that you use Base64 format.
   • Blindfolded Secret (recommended): From the Action drop-down menu, select one of the following options:

     • Use Existing Blindfolded Secret: Enter the Blindfolded Secret.

     • Blindfold New Secret:

       1. From the Policy Type drop-down menu, select Built-in.
       2. In the Secret to Blindfold field, enter your Datadog API Key.
       

       Figure: Datadog Image

   Note: To obtain your Datadog API key, go to https://app.datadoghq.com/organization-settings/api-keys. For more information, see Datadog documentation: https://docs.datadoghq.com/account_management/api-app-keys/#add-an-api-key-or-client-token.

6. Click Apply.

7. To save your Datadog receiver, click Save and Exit.

Configure an Azure Blob Storage Receiver

Important: Before you begin, make sure you have access to your Azure Blob connection string. You need the connection string during the receiver configuration process. See your Azure documentation for information about how to obtain the Azure Blob connection string.

1. Complete the steps in Add a Data Receiver.

2. In the Azure Blob Connection String section, click Configure.

   

   Figure: Configure an Azure Blob Receiver

3. From the Secret Type drop-down menu, select one of the following options:

   • Clear Secret: Enter the password you want to use. If you select Clear Secret, F5 strongly recommends that you use Base64 format.
   • Blindfolded Secret (recommended): From the Action drop-down menu, select one of the following options:
     • Use Existing Blindfolded Secret: Enter the Blindfolded Secret.
     • Blindfold New Secret:
       1. From the Policy Type drop-down menu, select Built-in.
       2. In the Secret to Blindfold field, enter your Azure Blob connection string and then click Apply.

4. In the Container Name field, enter the name of the Azure container where the logs should be stored.

5. To save your Azure Blob storage receiver, click Save and Exit.

Configure a GCP Bucket Receiver

1. Complete the steps in Add a Data Receiver.

   

   Figure: Configure a GCP Bucket Receiver

2. In the Data Delivery section, in the GCP Bucket Name field, enter the name of the GCP bucket where you want to send log data.

3. From the GCP Cloud Credentials drop-down menu, select an existing set of credentials that Data Intelligence can use to access the GCP bucket.

   To add new cloud credential, select Add Item. For information, see GCP Credentials.

4. To save your GCP bucket receiver, click Save and Exit.

Configure an HTTPS Receiver

1. Complete the steps in Add a Data Receiver.
2. In the Data Delivery section, in the HTTP URI field, enter the URI for the HTTPS receiver. For example, enter, https://your_receiver_uri.com.
3. From the Authentication drop-down list, select how you want to authenticate to the HTTPS endpoint. Select one of the following options:
   • None: Select this if no authentication is required.
   • Basic Authentication:
     1. In the User Name field, enter the user name you use to connect to the HTTPS endpoint.
     2. In the Password section, click Configure.
     3. From the Secret Type menu, select one of the following options:
        • Clear Secret: Enter the password for the HTTPS endpoint. If you select Clear Secret, F5 strongly recommends that you use Base64 format.
        • Blindfolded Secret(recommended): From the Action drop-down menu, select one of the following options:
          • Use Existing Blindfolded Secret: Enter the Blindfolded Secret.
          • Blindfold New Secret:
            1. From the Policy Type drop-down menu, select Built-in.
            2. In the Secret to Blindfold field, enter the password for the HTTP endpoint.
   • Token Authentication:
     1. In the Token section, click Configure.
     2. From the Secret Type menu, select one of the following options:
        • Clear Secret: Enter the password for the HTTPS endpoint. If you select Clear Secret, F5 strongly recommends that you use Base64 format.
        • Blindfolded Secret(recommended): From the Action drop-down menu, select one of the following options:
          • Use Existing Blindfolded Secret: Enter the Blindfolded Secret.
          • Blindfold New Secret:
            1. From the Policy Type drop-down menu, select Built-in.
            2. In the Secret to Blindfold field, enter your authentication token.
4. Click Apply.
5. To save your HTTPS receiver, click Save and Exit.

Configure Advanced Options

Advanced settings include optional settings to enable compression and to configure batch options.

Enable Compression

Enabling compression reduces the size of the log data files delivered from Data Intelligence and can help reduce your data storage costs.

1. Click Manage > Data Delivery > Add Receiver.
2. In the Data Delivery section, enable the Show Advanced Fields toggle.
3. To enable compression, from the Compression Type drop-down menu, select gzip.
4. When finished, click Save and Exit.

Configure Batch Options

Batch options allow you to apply limits such as maximum number of messages, bytes or timeout. When you configure a limit, Data Intelligence sends log data to the receiver only when the system detects that a configured limit has been reached.

1. Click Manage > Data Delivery > Add Receiver.
2. In the Data Delivery section, enable the Show Advanced Fields toggle.
3. To configure the maximum age of a batch before it is sent to a data receiver, from the Batch Timeout Options drop-down menu, select Timeout Seconds and then enter the number of Timeout Seconds.
4. To configure the maximum number of log messages that can be added to a batch before it is sent to a data receiver, from the Batch Max Events drop-down menu, select Max Events and then enter a Max Events value between 32 and 100000.
5. To configure the maximum byte size of a batch before it is sent to a data receiver, from the Batch Bytes drop-down menu, select Max Bytes and then enter a value between 4096 and 104857600. Logs are sent after the batch is size is equal to or more than the specified byte size.
6. When finished, click Save and Exit.

Configure TLS

To encrypt the data sent from Data Intelligence to your Splunk, Datadog or HTTPS data receiver, you can enable TLS.

1. Click Manage > Data Delivery > Add Receiver.
2. In the Data Delivery section for your data receiver, enable Show Advanced Fields.
3. To enable TLS, from the TLS drop-down menu, select Use TLS.
4. The following options are enabled by default. You can disable them as necessary.
   • Perform Server Certificate Verification
   • Enable Server Hostname Verification
5. To upload a server CA certificate, select Server CA Certificates and then paste or upload the certificate (.pem, .cer, or .crt).
6. To optionally Enable mTLS, perform the following steps:
   1. From the mTLS drop-down menu, select Enable mTLS.
   2. Paste the contents of your Client Certificate (PEM or Base64).
   3. In the Client Private Key section, select one of the following options:
      • Clear Secret: Paste the Secret you want to use. If you select Clear Secret, F5 strongly recommends that you use Base64 format.
      • Blindfolded Secret (recommended): Select one of the following options:
        • Use Existing Blindfolded Secret: Enter the Blindfolded Secret.
        • Blindfold New Secret: From the Policy Type drop-down menu, select Built-in and then paste the Secret to Blindfold.
7. When finished, click Save and Exit.

Manage Data Receivers

Perform the following steps to manage, enable or disable, or delete existing data receivers.

Note: To add a new data receiver, see Add a Data Receiver.

1. From the Data Intelligence navigation menu, click Manage > Data Delivery.
2. From the list of configured data receivers, in the Actions column, click the Action menu (…) next to the data receiver you want to manage and then click one of the following options:
   • Manage: Update data receiver configuration settings. For information about specific settings, see the following sections:
     • Configure an AWS S3 Receiver
     • Configure a Splunk Receiver
     • Configure a Datadog Receiver
     • Configure an Azure Blob Storage Receiver
     • Configure a GCP Bucket Receiver
     • Configure an HTTPS Receiver
   • Disable: Temporarily disable the data receiver that connects the Distributed Cloud Service to your data collection system. This prevents the Distributed Cloud Service from sending data to your data collection system
   • Enable: Re-enable a data receiver that you previously disabled. This allows the Distributed Cloud Service to resume sending data to your data collection system
   • Delete: Permanently disable and remove a configured data receiver. This cannot be undone.

---

View the Data Dictionary

The Data Intelligence data dictionary provides information to help you understand the log data that Data Intelligence sends to your data collection system.

To view the information contained in the data sets in the data dictionary, from the Data Intelligence navigation menu, click Manage > Data Dictionary.

Click a data set to view the following details:

• Feature name
• Description
• Data Type

To subscribe to a data set, contact F5 Support.

---