This guide provides instructions on how to configure and use F5 Distributed Cloud Bot Defense (XC Bot Defense) on F5 Distributed Cloud Console (XC Console). For more information on Bot Defense, see About Bot Defense.
An Organization plan. If you don't have an Organization plan, upgrade your plan.
Enabling XC Bot Defense
Use one of these methods to enable XC Bot Defense:
Option 1 - WAAP add-on service
On the XC Console Home page, select Billing. The Billing Plan page appears.
Scroll to the right until you see the Organization plan.
Under the Organization plan, select the Bot Defense link. The XC Bot Defense service landing page appears.
Option 2 - Standalone service
If you have both an account and Organization plan, you can enable the XC Bot Defense add-on service by going to the XC Console Home page and selecting the Bot Defense card.
If XC Bot Defense isn't enabled, the Bot Defense landing page appears. Select Request Service to enable XC Bot Defense.
Otherwise, if XC Bot Defense is already enabled, the Monitor page appears.
Note: To manage protected applications, a ves-io-power-developer-role or greater is required in system and application namespaces. To access the Bot Defense Monitor page, a ves-io-monitor-role or greater is required.
Enabling mitigation (blocking mode) for XC Bot Defense
Typically, users configure XC Bot Defense in Flag mode to start, which logs (but does not block) any traffic. When ready, users can enable blocking mode (Block or Redirect) to start blocking malicious traffic.
Go to Home > Load Balancers > Manage Configuration > Edit Configuration > Bot Defense Config > Bot Defense Policy > Protected App Endpoints > App Endpoint Type > Edit Endpoint.
Under Select Bot Mitigation Action, change the Action for the protected endpoints to Block or Redirect.
If you selected Block, select a Status code and enter a message body. If you selected Redirect, enter a redirect URI.
Backend configuration
Common best practices
When configuring endpoints, use wildcards to terminate the match. For example: /login*
This prevents attackers from adding a slash to the end of the request (for example, /login/) when both variants reach the same application endpoint.
The mitigating response should try to mimic how the application responds when the request is invalid (for example, when invalid credentials are used).
WAAP or Distributed Mesh best practices
If your site is served from an apex domain, configure a redirect from the apex domain to the FQDN domain (for example, customer.com redirects to www.customer.com). This is required because your domain must point to a Distributed Cloud-provided CNAME. Pointing a CNAME from an apex domain is not recommended as that prevents you from having SOA and all other DNS records at apex.
Do not use IP-based session persistence to load-balance internally. F5 Distributed Cloud has a range of egress IPs and if you have IP-based load balancing at the origin, this may break your session persistence. Use cookie-persistence, if possible.
Lock down your infrastructure to Distributed Cloud egress IPs only. This prevents attackers from bypassing Distributed Cloud altogether and hitting your origin directly.
Validate your internal tools (such as logging tools or fraud tools) see the traffic accurately when going through XC Bot Defense.
If you have a CDN (such as CloudFront) implemented before F5 Distributed Cloud (Client > CDN > F5 Distributed Cloud > Origin), ensure:
The XC Bot Defense JavaScript file is excluded from the cached entities.
The CDN passes the origin User-Agent rather than injecting CDN User-Agent, such as "Cloud Front".
Web application best practices
Identify all URL paths to be protected by XC Bot Defense.
Use wildcards cautiously. Avoid enabling XC Bot Defense protection for more URLs than required by adding wildcards.
Mobile application best practices
Add XC Bot Defense headers only to those requests which are also configured for XC Bot Defense protection in WAAP or iApp. This helps avoid filling up bandwidth unnecessarily.
Before onboarding the mobile application, identify all URLs which need to be configured.
Ideally, F5 Mobile SDK is integrated into applications with forced-upgrade capability. This allows you to upgrade users to the protected application.
F5 recommends you implement the integration as described in the SDK documentation. If you plan to deviate from those recommendations, please explore the knowledge base for other integration considerations.
Initialize the F5 Mobile SDK as early as possible in the application lifecycle to ensure the SDK is initialized and ready to add headers before a protected request is made. If you use Push Notifications on Android, there are special integration considerations when push notifications are sent to many applications at once.
Include an application version marker in the User-Agent header of the request.
Example: User-Agent: sometext MyApp/3.3 sometext
This allows you to examine traffic filtered by a specific version of your application.
Execute parseResponseHeaders() for all responses returned to protected requests.
Do not send the same set of headers more than once since each set of headers contains a unique token.
The XC Bot Defense SDK is obfuscated. If you use code obfuscation, exclude it from being obfuscated again.
If the application is accessing endpoints by using WebView and those endpoints need XC Bot Defense protection, consider using the JavaScript solution (see Knowledge Base article on how to use JavaScript with the WebView).
In the test environment, consider keeping your endpoints in Block Mitigation mode. This helps you to discover SDK integration problems early during development.
Configuring XC Bot Defense to protect XC Mesh applications through an HTTP load balancer (WAAP add-on service)
In XC Console, go to the Load Balancers page.
Select the … in the Actions column next to your load balancer, and then select Manage Configuration.
Select Edit Configuration.
In the left navigation menu, select Security Configuration.
In the Bot Defense Config section, in the drop-down menu select Specify Bot Defense Configuration.
For load balancers with types “HTTPS with Custom Certificate” and “HTTPS with Automatic Certificate” set the Path normalize field to Enable path normalization. This is the default setting for new load balancer objects.
Be sure to clear the Disable under Service Policy.
Adding endpoints for XC Bot Defense protection
In the Bot Defense Config section, follow these steps:
Under Bot Defense Regional Endpoint, select the region where the endpoint resides.
(Optional) Select the Show Advanced Fields switch in the upper right corner. The Timeout field appears. In the Timeout field, enter the number of milliseconds Bot Defense should wait for the inference check to complete before timing out. The default value is 1000.
In the Bot Defense Policy section, select Configure.
In the Protected App Endpoints section, select Configure.
Select Add Item to add an application endpoint. You can create multiple endpoints.
Enter the following for each endpoint:
Name: The name of the message. Must follow DNS-1035 format.
Description: A human-readable description of the endpoint.
HTTP Methods: Which HTTP methods are monitored on this endpoint. Commonly used methods include POST/PUT/GET(XHR). GET requests are protected only if they are sent by XHTTPRequest from the page with XC Bot Defense JavaScript injected, not from direct navigation using the address bar or link. The ANY method should be used carefully and only when intended.
Endpoint Label: Endpoint labels help categorize and organize your endpoints. Endpoint labels also allow more granular attack intent identification and reporting when automation occurs. For example, to identify a gift card checkout flow on an endpoint, select Specify Endpoint label category > Shopping & Gift Cards > Purchase with Gift Card. F5 recommends that you configure endpoint labels immediately so that Bot Defense can begin collecting this information and reporting on endpoint details. For a full list of available endpoint labels, see Endpoint Labels.
Note: For web scraping labels, select Specify Endpoint label category > Search and select an available flow label.
Note: A subset of endpoint labels is included in BIG-IP 17.1. For information, see your BIG-IP product documentation.
Protocol: Which protocols are protected.
Domain Matcher: Since HTTP load balancers can serve multiple domains, you can specify domains here. Enter an exact value, a suffix value, or a regex value.
Path: Specify protected paths here. Enter a prefix, exact path, or regex value.
Bot Traffic Mitigation: Specify what action to take when a bot is detected.
Block: The endpoint returns a status code and message. You can select the code and edit the message here.
Redirect: The endpoint forwards the browser to a URI, specified here.
Flag: Select No headers to create a log record only, or select *Append Headers to create custom headers for Inference and Automation Type.
Mobile SDK for WAAP
There is a new Mobile SDK Settings section. This section is disabled by default. If a user adds a mobile endpoint in the Protected App Endpoints section, then they can enable the section and specify their specific header for mobile.
Users can select one of the following Traffic Channel options per endpoint:
Web Traffic
Mobile Traffic
Web and Mobile Traffic (needs to be enabled by the Show Advanced Fields filter at the top-right of the page)
Note: If Web and Mobile is selected, then the user needs to add specific mobile request identifier and select header. The Header is defined in the Mobile SDK Setting Advanced fields.
Mobile SDK settings
This is section of Bot Defense Policy lets you specify your mobile preferences. You must select Show Advanced Fields for the Mobile Traffic Identifier Header section to appear. Select Add Item.
You can specify the mobile header name and set their matcher preferences. Select from the following Match Options:
Match Values
Present
Not Present
If you want to set Transformers, select Show Advanced Features.
When done configuring, select Add item to save the details.
Injecting the XC Bot Defense JavaScript into your web pages
After you add the domains for which to apply XC Bot Defense protection, you must inject the XC Bot Defense JavaScript into the web pages.
To inject the XC Bot Defense JavaScript, follow these steps:
In the JavaScript Insertion section:
Under JavaScript Download Path, enter the path where the HTTP load balancer can find the JavaScript to serve to the client browser.
From the Web Client JavaScript Mode drop-down menu, determine how the protection JavaScript is loaded into the entry page. The JavaScript is loaded in two parts. The larger part can be loaded in two possible ways:
Asynchronously: The JavaScript runs as soon as it is loaded and does not block the page from loading.
Synchronously: The page does not continue loading until the JavaScript has been loaded and executed.
You can also choose to cache the larger part of the JavaScript so it does not have to be downloaded each time it is loaded into a new page.
Choose one of the following options:
Async JS with no Caching
Async JS with Caching
Sync JS with no Caching
Sync JS with Caching
Under JavaScript Insertion Settings, specify if the HTTP load balancer should insert JavaScript into all pages, or if some pages should be excluded.
If you select Insert JavaScript in All Pages:
Choose where you want to insert the JavaScript:
After <head> tag
After </title> tag
Before <script> tag
If you select Insert JavaScript in All Pages with the Exceptions:
Choose where you want to insert the JavaScript:
- After \<head> tag
- After \</title> tab
- Before \<script> tag
Select Add Item to add an excluded page. The JavaScript Insertion Exclusion Rule page appears.
In the JavaScript Insertion Exclusion Rule page, enter the following for each excluded page:
- **Name**: The name of the message. Must follow DNS-1035 format.
- **Description**: A human-readable description of the endpoint.
- **Domain Matcher**: Since HTTP load balancers can serve multiple domains, you can specify domains here. Enter an exact value, a suffix value, or a regex value.
- **Path**: Specify protected paths here. Enter a prefix, exact path, or regex value.
Select Add Item.
Note: The following options are visible only when the Show Advanced Fields switch is set to On.
If you select Custom JavaScript Insertion Rules:
Under JavaScript Insertions, select Configure. The JavaScript Insertions page appears.
Select Add Item and enter the following for each endpoint:
- **Name**: The name of the message. Must follow DNS-1035 format.
- **Description**: A human-readable description of the endpoint.
- **Domain Matcher**: Since HTTP load balancers can serve multiple domains, you can specify domains here. Enter an exact value, a suffix value, or a regex value.
- **Path**: Specify protected paths here. Enter a prefix, exact path, or regex value.
- **JavaScript location**:
- After \<head> tag
- After \</title> tab
- Before \<script> tag
When you are finished adding endpoints, select the Back button. The JavaScript Insertions screen appears.
Select the Back button.
Under Exclude Paths, select Add Item and enter the following for any paths you want to exclude from inserting JavaScript:
Name: The name of the message. Must follow DNS-1035 format.
Description: A human-readable description of the endpoint.
Domain Matcher: Since HTTP load balancers can serve multiple domains, you can specify domains here. Enter an exact value, a suffix value, or a regex value.
Path: Specify protected paths here. Enter a prefix, exact path, or regex value.
If you select Disable JavaScript Insertion, no further action is necessary. JavaScript is not inserted in any page.
Configuring XC Bot Defense to protect other applications through the Template Connector
Adding protected applications to XC Bot Defense
To add applications to XC Bot Defense:
On the XC Console Home page, select the Bot Defense card. The Applications page appears.
On the Applications page, if you haven't added any applications yet, select the Add Protected Application. Otherwise, select Add Application. The Protected Application page appears.
On the Protected Application page, enter the following:
Name: The application's name.
Labels: Select or add a label for this application.
Description: Describe the application.
Application Region: The region where the application's origin server resides.
Connector Type: The type of connector the application uses.
Users can add a new Connector Type called Custom when creating a new application. If added, the system displays Custom under the Connector Type on the Application Overview.
Users add metadata and region, and then select their connector type. A new Custom field appears. The user should select Custom if their specific platform is not listed. The Custom type provides the needed authentication parameters for use on other platforms.
If a Custom Connector Type is selected, the user has access to the following authentication parameters to use on the other platforms:
Copy App ID
Copy Tenant IF
Copy API Key
Copy Web API Hostname
Copy Mobile API Hostname
Copy Telemetry Header Predix
Note: Each Connector Type has different action, usually associated with the specific platform. Custom is intended to support many platforms and may have more fields than what is required to get started.
Select Save and Exit.
Managing protected applications
To manage a protected application:
On the XC Console Home page, select the Bot Defense card.
On the Applications page, in the Actions column, select the three dots next to your application. The Actions menu appears.
From the Actions menu, select one of the following:
Manage Configuration: Opens the Protected Application page. Select Edit Configuration to make changes.
Copy App ID: Copy the application's unique ID to your clipboard.
Copy Tenant ID: Copy the tenant's unique ID to your clipboard.
Copy API Key: Copy the application's API key to your clipboard.
Delete: Delete the application.
Download Template: Download an iApp template for use with BIG-IP.
The Protected Application page displays information about a protected application. You can choose to edit the application configuration, view additional information about the application, or view Bot Defense configuration information for your load balancers.
To edit a protected application, from the Protected Application page, select Edit Configuration.
To view Bot Defense configuration information for your load balancer, from the Protected Application page, click the name of your load balancer.
Client configuration
Adding XC Bot Defense SDK to your mobile applications
Native applications can be integrated with XC Bot Defense by following the two links below:
Use getRequestHeaders and parseResponseHeaders API, rather than the generateHeaders and analyzeResponse. This API is simpler and meets the needs of XC Bot Defense.
For body, you can pass in nil on iOS and null on Android.
For XC Bot Defense, getRequestHeaders always returns headers, regardless of which URL is passed. For this reason, the developer should only add headers to the requests which require protection.
Similarly, parseResponseHeaders should be called for every response to the protected requests.
Explore the knowledge articles for integrating XC Bot Defense with:
Flutter
React Native
Xamarin and Xamarin.Forms
Angular + Ionic
Verifying XC Bot Defense integration
Confirming JavaScript injection is working
To confirm XC Bot Defense is adding JavaScript to an entry page, inspect the page in your browser and look for XC Bot Defense-specific headers using the procedure below.
Obtain the following information from your load balancer configuration. You can find the entry page's host and path in your HTTP load balancer configuration by going to Home > Load Balancers > Manage Configuration > Edit Configuration > Bot Defense Config > Bot Defense Policy > Protected App Endpoints.
Entry page path (for example, www.yourwebsite.com/endpoint)
JavaScript download path (for example, /yourscript.js)
Go to the entry page path in your browser. For example: www.yourwebsite.com/endpoint
Right-click (Windows) or Control-click (iOS) near the outer edge of the page and select Inspect. Your browser’s inspector opens.
In the inspector, select the Network tab.
Refresh the browser. A list of requests appears in the inspector.
Look for three scripts in the list of requests named with your JavaScript download path, followed by ?matcher, ?cache, and ?async query parameters.
If you see three scripts with your JavaScript download path followed by ?matcher, ?cache, and ?async, XC Bot Defense is injecting JavaScript into the entry page.
Validating Mobile SDK integration
This section describes how to validate if F5 Mobile SDK is properly integrated. Complete these test cases:
Review the Integration documentation to make sure all integration steps have been followed.
SDK Initialization
Deploy the application onto a device for the first time (if the device already had the application, remove it before this test).
Start the application.
On the on the Bot Defense dashboard or the Bot Traffic Overview Dashboard, filter by your client IP. You should observe a GET request configuration fetch from your application.
Decorated Request
Make a request for every protected URL.
On the Bot Defense dashboard, filter by your client IP. You should see all protected requests marked as Human.
Mitigation
Temporarily alter the application code to not call getRequestHeaders.
Launch the application and make protected requests.
On the Bot DefesnseDefense Dashboard filter by your client IP. You should see all protected requests marked as an Automation. You should see a GET request configuration fetch occur right after each blocked mitigated request.
Update your code to call getRequestHeaders again.
Your integration is successful if you were able to complete all of these test-cases. Incorporate these Mobile SDK test-cases into your standard regression testing.
Before releasing a new application into the production environment, set mitigation action to Flag. Perform the false positive analysis before re-enabling mitigation.
False positive analysis
False positive analysis for web traffic
This section provides guidance for how to confirm XC Bot Defense is working as expected on your website after being enabled for real users. Specifically, you must determine if any legitimate traffic is marked as automation.
Although this documentation provides some guidance, this is a creative process. Explore your traffic reports and examine anything unexpected. Resolve all issues before you move to blocking mode.
Identify false positives
Review human and bot traffic on the Bot Defense dashboard or the Bot Traffic Overview Dashboard.
Is any traffic marked as non-human? If yes:
What is the automation type of the non-human traffic?
Does the traffic marked as malicious have a diurnal pattern which increases during the day and drops at night? This might indicate human traffic.
Look at the distribution of IPs and the countries they are from. Does this distribution look like it's coming in from your normal user base?
Look at the User Agent field. Are any suspicious user agents present? You can also identify wanted automation (like test tools or SEO bots) by using this technique.
False positive analysis for mobile traffic
Before you enable mitigation, be sure to confirm XC Bot Defense for Mobile is working as expected after it has been enabled for real users.
Specifically, in the Bot Defense dashboard, confirm the following:
Low or non-existent rate of requests with no XC Bot Defense headers and requests with headers are marked as Human.
Configuration fetch requests are made at an expected rate. Each configuration fetch corresponds to one user session.
Explore your traffic reports and examine anything unexpected. Resolve all issues before you turn on mitigation.
XC Bot Defense Dashboards & Reporting
F5 Distributed Cloud (XC) Bot Defense includes multiple ways to view detailed information about the traffic that Bot Defense analyzes. Automated Threat Briefings deliver insights to you automatically each month, while a variety of dashboards provide you with instant, detailed access to analysis of your data.
Automated Threat Briefings
Automated Threat Briefings summarize your monthly traffic. They are automatically generated by Bot Defense each month and sent by email to Bot Defense administrators.
Automated Threat Briefings include information about the percentage of your traffic that is automated, the numbers of good and bad bots detected, the number of human visitors to your protected applications and endpoints, and a breakdown of your overall web traffic that is analyzed by Bot Defense.
Figure: Automated Threat Briefings
To enable Automated Threat Briefings, in Bot Defense, from the navigation panel, click Manage > Applications and then click to enable Threat Briefing Email.
Figure: Automated Threat Briefings Toggle
Note: When you enable Automated Threat Briefings, Bot Defense sends the monthly email to all administrators.
Automated Threat Briefings are delivered from no-reply@botdefense.f5.com and have the title, “Your <Month> Bot Defense Summary.”
Load Balancer XC Bot Defense Dashboard
After you enable XC Bot Defense, the Load Balancer XC Bot Defense dashboard provides a snapshot of bot and human activity in your web traffic on your HTTP load balancers.
To view the Web App & API Protection Bot Defense dashboard:
From the Web App & API Protection navigation menu, click Overview > Dashboards > Security Dashboard.
In the Load Balancer widget, from the list of load balancers, click the name of the load balancer you want to view and then click Bot Defense.
Figure: Load Balancer Bot Defense Dashboard
The Load Balancer XC Bot Defense dashboard includes the following information:
Traffic Types: The total number of transactions in the selected time window and how many were malicious bots, good bots or humans. Hover over the donut graph for percentage information for each type of traffic.
Traffic Overview: The transactions per minute of human and bot traffic for the selected time period. Hover over the area chart or bar graph for the number of each type of event at a specific time.
Top Malicious Bots: The five bots that made the most malicious requests in the selected time period, including the source IP, ASN, user agent, country where the bot is based, and number of malicious requests in the selected time period. You can view each bot's Source IP, AS Organization, and User Agent.
Top Endpoints Attacked: The five endpoints being attacked most frequently by malicious bots, including the host name, endpoint path, and number of malicious requests in the selected time period.
Top Good Bots: The top good bots in your web traffic during the selected time period, including the name of the bot, type, and the number of individual events and percentage of events attributed to a specific bot.
Human Statistics: The devices, platforms and browsers used by your human visitors, and the geographic locations from where they are accessing.
You can specify the time period displayed in the dashboard and add the following additional filters to customize your view:
AS Organization
Automation Type
Country
Host
IP Address
Path
Traffic Channel
Traffic Type
User Agent
To view additional Bot Defense data for this specific load balancer, such as an overview of protected apps, human traffic by geographic region, and an overview of specific bad bot events impacting your applications, click View in Bot Defense to go to the Bot Defense Dashboard.
Figure: View in Bot Defense
Bot Defense Dashboard
The Bot Defense dashboard is a snapshot of all human and bot activity in the web traffic analyzed by Bot Defense for a specified time period.
To view the Bot Defense dashboard, in Bot Defense, from the navigation menu, click Overview > Monitor.
Figure: Bot Defense Dashboard
You can customize the time period and the region that is displayed by the dashboard. The Bot Defense dashboard includes the following information:
Traffic Overview: The total transactions of human and bot traffic for the selected time period. Hover over the donut chart to see percentage information for each even traffic type.
Traffic by Category: The traffic detected for each endpoint category for the selected time-period. Hover over the donut chart to see percentage information for each category.
Bad Bot Traffic Overview: An overview of bad bot traffic that was flagged and mitigated across your protected endpoints. Hover over the donut chart to see percentage information.
Protected Apps Overview: A breakdown of bad bot traffic by defended application. Hover over the donut chart to see percentage information.
Use the dashboard controls to view additional data.
Figure: Bot Defense Dashboard Tabs
Type
Traffic Visualized: Total number of events impacting the applications you protect with Bot Defense over the specified time range as an area chart, bar graph or line graph. Hover over the charts for information about a specific time.
Top Good Bots Found: Top good bots that are visiting your protected applications, including the name and type of bot and total events for each bot.
Bot API Latency Across Top Protected Apps: Information about traffic latency measured between the top protected application and origin server, including average time, application name, latency per application, number of transactions and trend information.
Endpoint Label Category
Traffic by Category: Shows the traffic detected per endpoint category for the time-period selected. Hover over the chart to see totals and percentages for a specific time.
Top Endpoint Labels: Shows the top 10 endpoint labels on which traffic was detected, and the percentage of traffic detected on the endpoint label in relation to all traffic detected on your system.
Top Attacked Endpoints: Shows the five endpoints that are being attacked most frequently by malicious bots. The table includes the host name, endpoint path, and number of malicious requests in the selected time period.
Bad Bot
Bad Bot Traffic Breakdown: A time series chart for mitigated and flagged bad bot events across all protected endpoints displayed as an area chart, bar chart or line graph.
Top Reason Codes: A list of the top reason codes indicating how an attack was executed.
Top Sources of Bad Bot Traffic: Bad bot traffic analysis by source type, including number of events, IP address and geolocation. Display by Source IP, ASN or User Agent.
App Protection
Top Applications: A list of the most requested protected applications.
Geolocation
Map Applications: A map that shows where your human traffic comes from. Hover over each country for additional information about your traffic.
When you filter the Bot Defense dashboard by a single load balancer, click View in Web App & API Protection to view the Load Balancer XC Bot Defense dashboard for that load balancer.
Figure: View in Web App & API Protection
Bad Bot Report
The Bad Bot Report provides information about malicious automation in your web traffic.
To view the Bad Bot Report, in Bot Defense, from the navigation menu, click Report > Bad Bot Report.
Figure: Bad Bot Report
You can customize the time period and the region that is displayed by the report. The Bad Bot Report includes the following information:
Bad Bot Traffic Overview: An overview of the flagged and mitigated traffic across your protected endpoints. Hover over the chart for percentage information.
Bad Bot Traffic Metrics: An overview of the different characteristics of the bad bot traffic that Bot Defense has detected across your protected applications and endpoints.
Bad Bot Events per Application: The number of bad bot events grouped by application. Hover over the chart for totals and percentages for each application.
Bad Bot Reason Codes: The top reasons why Bot Defense associated an event with a bad bot, including the number of events for each reason code and percentage, to help you understand how bad bots are attacking your endpoints.
Bad Bot Traffic Breakdown: An overview of bad bot events impacting your applications across the specified time range as an area chart, bar chart or line graph. Hover over the charts for totals and percentages for a specific time.
Bad Bot Events per Application: Shows bad bot events happening on each protected application as an area chart, bar chart or line graph. Hover over the charts for event totals and percentages on each application at a specific time.
Endpoints with Bad Bot Traffic: An overview of bad bots attacking your protected endpoints.
Bad Bot Events: Use grouping strategies to explore and investigate bad bot events. Display events by operating system (OS), browser, user agent (UA), autonomous system number (ASN) or IP address. Expand each row to reveal additional total and percentage information. Use Search to find specific events.
Protection Coverage Report
The protection coverage report provides information about your protected applications and endpoints and the traffic accessing them.
To view the protection coverage report, in Bot Defense, from the navigation menu, click Report > Protection Coverage Report.
Figure: Protection Coverage Report
You can customize the time period and the region that is displayed by the report. The Protection Coverage Report includes the following information:
Endpoint Summary: A summary of your protected applications and endpoints during your selected time period.
Protected Traffic Flow: A visual representation of all traffic visiting your protected endpoints.
All Protected Endpoints: Information about all of your endpoints that are protected by Bot Defense, including domain, associated application, number of events that took place on the endpoint, and percentage of your total traffic that passed through the endpoint.
Transactions Usage Report
The Transactions Usage Report allows you to see how many transactions Bot Defense has processed for the past year.
To view the Transaction Usage Report, in Bot Defense, from the navigation menu, click Report > Transaction Usage.
Figure: Transaction Usage Report
This report displays the following information by month or by quarter:
Traffic: All the traffic on the customer’s web applications that are monitored by Bot Defense.
Initial Config Request: Traffic that is generated by the Bot Defense JavaScript that is embedded on the end-user’s mobile app.
JavaScript Request: Traffic that is generated by the Bot Defense JavaScript that is embedded on the end-user’s web browser.
The report displays data for the past year, starting from a year previous to the current month until the previous month. For example, if the current month is December 2022, data is presented from December 2021-November 2022. Hover over the chart to see information for a given month or quarter.
The following summary data for the previous 24 months is displayed at the top of the report:
Highest: The highest number of transactions monitored in a given month in the past year, compared with the highest number of transactions monitored in a given month in the previous year.
Lowest: The lowest number of transactions monitored in a given month in the past year, compared with the lowest number of transactions monitored in a given month in the previous year.
Average: The monthly average number of transactions monitored in the past year, compared with the monthly average number of transactions monitored in the previous year.
You can display data as either an area chart or bar chart. Hover over the charts to see data from specific months.
Use the drop-down menu to switch between Monthly and Quarterly views.
Figure: Quartely or Monthly View Controls
Traffic Analyzer
The Traffic Analyzer report provides detailed insight into human, good bot, bad bot and other traffic on the HTTP load balancer. View traffic as a bar chart or an area chart. Hover over the charts to view traffic totals for specific times.
To view the Traffic Analyzer, in Bot Defense, from the navigation menu, click Report > Traffic Analyzer.
Figure: Traffic Analyzer
In addition to the chart showing transactions per minute for a specified time window, you can also view details about every HTTP request sent through XC Bot Defense. Each HTTP request includes the following information:
Time
Country
IP address
ASN
AS Organization
User Agent
Host
Path
Method
Traffic Channel:
Traffic Type
Automation Type
Header Reference
Rearrange and sort the columns by selecting them. Show or hide columns by selecting the gear icon to the upper right of the list.
Connectors
iApp Connector
Prequisites
Before you deploy or upgrade an iApp in the F5 BIG-IP system, you must meet the following requirements:
You have access to the F5 Distributed Cloud (XC) Console to download the latest iApp template.
You have BIG-IP versions 14.1 or later.
Note: The Distributed Cloud Bot Defense iApp template is required for BIG-IP versions 14 to 16. For BIG-IP version 17.0 or later, XC Bot Defense is integrated natively with BIG-IP and provides improved performance. For information, see https://techdocs.f5.com/en-us/bigip-17-1-0/big-ip-saas-bot-defense.html.
The virtual server that you want to protect with Bot Defense must have an HTTP profile and default pool attached to it.
To protect mobile endpoints, you must create your Mobile Base Configuration. See your mobile SDK documentation for information.
F5 BIG-IP Application Services Templates (FAST) are the next-generation successor to now deprecated iApp templates. F5 strongly recommends that you do not use both FAST and iApp templates together. They are not compatible with each other.
Download the iApp Connector Template v3.0.4 from the XC Console
Log on to the XC Console. From the Dashboard page, select Bot Defense.
Select Add Application. If you have not already added any protected applications, you are instructed to add one.
Add a Name and Description for the application.
Select the region: US, EU, Asia.
From the Connector Type drop-down list, select F5 BIG-IP iApp.
Select Save and Exit.
From the Action column in the list of applications, click the Action Menu … and then click Download template. Save the template in a location you can access later.
Next Steps
After you download iApp Connector Template v3.0.4, perform one of the following steps:
Log on to the BIG-IP where you want to add your application.
Select iApps > Templates > Import.
Select Choose File and navigate to the location where you saved the v3.0.4 template.
Select the template you want to import and click Open.
Then select Upload and OK to confirm.
Required Initial Configuration of a New Applications from iApp Templates 3.0.4
When you use the F5 XC Bot Defense Template v3.0.4 to create a new XC Bot Defense Standard Application instance, you must perform the following required initial configuration steps in BIG-IP before you configure any additional settings.
In BIG-IP, click iApps, Application Services > Applications and then click Create.
Enter a Name for the new application. The name cannot contain any spaces or special characters.
From the Template drop down, select f5.ibd.cs.
In the One-Time Install/Upgrade Setup section, make sure No, I will click the FINISHED button now is selected and then review the information in the Important section. Then scroll to the bottom of the page and click Finished.
Important: Do not make any other configuration changes before you click Finished. You can make only configuration changes after you click Finished.
Upgrade an application from iApp Template v3.0.3 to iApp Template v3.0.4
The following sections explain how to upgrade an existing XC Bot Defense Standard Application on a BIG-IP from Template v3.0.3 to v3.0.4.
Important: You can only upgrade to template version 3.0.4 from v3.0.3. To upgrade from a template version older than 3.0.3 and maintain your existing configuration settings, you must first upgrade your existing iApp template to v3.0.3.
Important: To upgrade a group of XC Bot Defense Standard Applications (more than one application on the same BIG-IP with the same F5 Security API hostname) contact F5 Customer Support for upgrade assistance.
Import the v3.0.4 Template for an Existing Application
Log on to BIG-IP where your application is protected and hosted.
Click iApps > Templates > Import.
Click Overwrite Existing Templates.
Click Choose File and navigate to the location where you saved the template you downloaded from the XC Console.
Select the template you want to import and click Open.
Click Upload and OK to confirm.
Upgrade the Template for an Existing Application
Click Application Services > Applications and then click the name of the application you want to upgrade.
Click the Reconfigure tab.
In the Template section, click Change.
From the Template drop down menu, select f5.ibd.cs.
In the One-Time Install/Upgrade Setup section, make sure No, I will click the FINISHED button now is selected and then review the information in the Important section. Then scroll to the bottom of the page and click Finished.
Important: Do not make any other configuration changes before you click Finished. You can only make configuration changes after you click Finished.
Next Steps
After you click Finished, you can safely make changes to the application configuration. See “Configure an application that uses iApp Connector Template v3.0.4.”
Configure an application that uses iApp Connector Template v3.0.4
The following instructions explain how to configure a new application that uses iApp Connector Template v3.0.4 or an existing application that has been upgraded from v3.0.3 to v3.0.4.
In the BIG-IP that hosts your applications, click iApps > Application Services > Applications.
From the list of applications, click the application you want to configure.
Select the Reconfigure tab.
The following sections explain how to configure each option in your application.
General Settings
Decide if you want to Activate Bypass Mode, which allows all HTTP traffic to bypass the Bot Defense processing engine. Only select Yes if you want to temporarily disable the Bot Defense service. For example, enable bypass mode if you suspect there is an issue with Bot Defense, such as too many false positive results.
Decide if you want to send all web form contents to the F5 XC Defense Engine API for analysis or if you want to send only telemetry data. You might decide to send only telemetry data (the default selection) if you must adhere to strict regulatory compliance rules, such as GDPR.
To see information about pattern and regular expression wildcard matching displayed in the user interface, select Yes from the drop-down menu. This information can be useful when you configure which client requests should be inspected by F5 XC Bot Defense.
Figure: iApp Connector General Configuration screen
Mobile SDK Options
If you have mobile endpoints, in the Do you want to enable F5 XC Bot Defense Mobile SDK support, select Yes (supports F5 XC Mobile SDK for mobile apps).
Decide how you want Bot Defense to identify requests from your Mobile SDK Clients and to help distinguish between mobile and web traffic through the same virtual server. Choose either headers or body keywords:
Enter the header Name and Value and click Add. For example, for the header name, you might enter sample-mobile-app, and for the value you might enter true. You can specify multiple headers.
Enter keywords that appear in request bodies. To enter multiple keywords, use the following format, “keyword1|keyword2|keyword3”.
In the Mobile SDK Reload Header Name field, enter the Reload Header Name from the XC Console. This value you enter is included in the response message when Bot Defense blocks a request. To find the Reload Header Name:
Log on to the XC Console. From the Dashboard page, click Mobile Base Configurations.
Copy the Reload Header Name for the appropriate mobile base configuration.
Enter the value in the Mobile SDK Reload Header Name field in the iApp template.
Figure: iApp Connector Mobile SDK Options screen
JavaScript Injection Configuration
To protect web-based endpoints, decide if you want your BIG-IP to automatically insert the Bot Defense protection JavaScript into your application pages or if you want to add the JavaScript manually.
If you want BIG-IP to insert the JavaScript, perform the following steps:
From the BIG-IP Handles F5 Client JavaScript Injections drop-down menu, select Yes.
Enter the path or URL where clients can access the F5 Client JavaScript to insert in your application. Enter a simple path that starts with / or a complete URL, such as https://example.com/customer1.js.
Select the location in the application pages where you want BIG-IP to insert the protection JavaScript. F5 recommends that you select After <head> so that the JavaScript is executed early. This helps to avoid conflicts with other scripts and allows time to for the protection JavaScript to be executed while the rest of page is rendered.
Note: If you select After </title>, be sure your application pages have the title tag.
To inject JavaScript into Telemetry data, in the Telemetry JavaScript Injection drop-down menu, select Enable. Otherwise, select Disable.
If you enabled Telemetry JavaScript Injection, decide if you want to inject the JavaScript in the
tag. If you select No, the JavaScript is inserted in the same location as the protection JavaScript.
Specify whether want to inject the protection JavaScript only in specific webpages:
To inject JavaScript in specific webpages, in the Do you want to choose which pages are injected with protection Javascript drop-down menu, select Yes. Then enter the path for the pages where you want to inject the JavaScript and click Add.
Note: Not all pages require JavaScript, but JavaScript must be inserted at all entry points, such as login forms.
To inject JavaScript in all pages, select No.
To exclude the protection JavaScript injection from specific webpages, in the Do you want to avoid injecting F5 Client Javascript into certain pages drop-down menu, select Yes. Then enter the path for the page you want to exclude and click Add.
To manually insert the JavaScript in your application, perform the following steps:
Note: Not all webpages require JavaScript, but JavaScript must be inserted at all entry points, such as login forms.
From the BIG-IP Handles F5 Client JavaScript Injections drop-down menu, select No.
Enter the path or URL where clients can access the F5 Client JavaScript to insert in your application. Enter a simple path that starts with / or a complete URL, such as https://example.com/customer1.js.
If you require Manual JavaScript Insertion, add the following tags to one of the recommended locations:
Specify pages and sources on the website that should be protected by Bot Defense.
The Mitigation Handler field is informational only. You do not need to make any changes to this field.
If you previously enabled F5 XC Bot Defense Mobile SDK support, select the type of client requests that should be inspected and then mitigated by F5 XC Bot Defense: Web, MSDK (Mobile Client), or Both (Web and Mobile).
Note: If you select Both, you must provide a header which distinguishes Mobile requests from Web. For example:
User-Agent: Android 10; MyApp/1.0
User-Agent: Android 9; MyApp/2.0
X-App-Version: 4.1
X-App-Version: 4.3
The specified header value can use a wildcard (*) to match a substring.
In the Host field, enter the host domain of the protected endpoint. For example, enter identity..com.
In the Path field, enter the path to the endpoint. The path must meet the following requirements:
Only uses lowercase letters.
Starts with ‘/’. For example, enter /login/.
Can use glob patterns: * ? [ ].
Closes any open bracket [ with ].
Brackets must include only valid URL characters.
Does not specify /* as a protected URL.
Uses / to protect the root.
For example:
Example 1: /application/version[0-9]/login
Example 2: /myapp/service/*
Example 3: /*/login.aspx
Example 4: /api/*/connection/*/*
Example 5: /api/connection/[a-zA-Z0-9_]*/set*
(Optional) In the Query field, if you want to protect specific sections of a page, enter the criteria for the section you want to protect.
For example, if the query string of a URI you want to protect is <yourdomain>/account.do/issue? account=moneytransfer, enter the parameter name and parameter value ‘account=moneytransfer’ to protect the “moneytransfer” function.
When you enter a Query parameter, the Bot Defense service looks at the Path and Query values. The Query field is regex matched.
Choose the HTTP Methods are monitored on this endpoint. You must set at least one of the options to Yes. HTTP requests are not routed if no methods are set to Yes.
Select Yes or No for each of the following methods:
ANY: Select Yes to protect the path with any type of method. Select No to limit protection to only certain methods.
GET: Select Yes to protect the path when it has a GET method.
POST: Select Yes to protect the path when it has a POST method.
PUT: Select Yes to protect the path when it has a PUT method.
Important: If you choose to protect all pages and endpoints (/*), F5 recommends that you select specific HTTP methods to monitor and that you do not select the ANY option. Protecting all HTTP methods for all pages can adversely impact performance.
In the Action drop-down menu, select the mitigation action you want to take for this endpoint if Bot Defense detects automation:
Continue (request continues to origin). For reporting purposes, you can optionally add a header to capture traffic that is going to the origin. You provide the header name in the Define Mitigation Actions section.
Important: When you initially deploy the iApp connector, F5 recommends that you set mitigation action to Continue for the first 24-48 hours and then monitor the system to confirm that it is functioning properly.
Redirect. The Bot is sent to a different page. You provide the URL where the Bot is redirected later in the Define Mitigation Actions section. The redirect must go to:
An absolute URL, for example, https://acme.com/redirect.html
A domain that you own.
Block. The Bot is presented with a message that it has been blocked. You provide the Response Code and Body of the response message later in the Define Mitigation Actions section.
Drop. This is not a valid selection. If you select Drop, Bot Defense defaults to Block.
When you select mitigation actions, you must be aware of the following important information:
Client Type
Valid Actions
Additional Information
MSDK (mobile)
Continue, Block
-- Redirect and Drop are visible but are not valid actions. If you select an invalid action, the system defaults to Block.
-- You must specify a header, otherwise the endpoint is treated as a web endpoint. For information, see “Define Mitigation Actions.”
Web Client
Continue, Block, Redirect
Drop is visible but is not a valid action. If you select Drop, Bot Defense defaults to Block.
Both (mobile and web)
If you select Both (web and mobile clients):
-- Drop is not a valid action. If you select Drop, Bot Defense defaults to Block.
-- You can select mitigation actions for each client type.
-- In the MSDK Req ID drop-down menu (see below), you must select either Hdrs or Body. Otherwise, Bot Defense treats the endpoint as a web endpoint.
-- You must specify a header, otherwise Bot Defense treats the endpoint as a web endpoint. For information, see Define Mitigation Actions.
In the MSDK Req ID drop-down menu, specify how you want Bot Defense to identity mobile endpoints.
If your endpoint type is MSDK (mobile), select Hdrs or Body. Otherwise, Bot Defense treats the endpoint as a web endpoint by default.
If your endpoint type is Both (web and mobile), select Hdrs or Body. Otherwise, Bot Defense treats the endpoints as a web endpoint by default.
If your endpoint type is Web, then you do not need to choose a setting. This field is ignored for web endpoints.
Specify whether you want to ignore letter case while matching request paths. If you select Yes, letter case is ignored when matching the path of a request URL. If you select No then matching is case sensitive.
After you finish adding endpoints, all protected endpoints are displayed in a table. You can modify or delete endpoints.
Web-Scraping Protection Endpoints
Configure web scraping protection for documents and pages on a web site that is accessible by GET requests without visiting the main page, for example, from a link on social media, from an email or from a saved bookmark. This direct access to resources prevents Bot Defense from inserting the protection JavaScript. When you configure an endpoint for web scraping, Bot Defense displays an interstitial page that is transparent to the user but that allows it to collect telemetry data about the requests.
The Mitigation Handler field is informational only. You do not need to make any changes to this field.
In the Host field, enter the host domain of the protected endpoint.
In the Path field, enter the path to the endpoint. The path must meet the following requirements:
Only uses lowercase letters.
Starts with ‘/’. For example, enter /login/.
Can use glob patterns: * ? [ ].
Closes any open bracket [ with ].
Brackets must include only valid URL characters.
Does not specify /* as a protected URL.
Uses / to protect the root.
For example:
Example 1: /application/version[0-9]/login
Example 2: /myapp/service/*
Example 3: /*/login.aspx
Example 4: /api/*/connection/*/*
Example 5: /api/connection/[a-zA-Z0-9_]*/set*
(Optional) In the Query field, if you want to protect specific sections of a page, enter the criteria for the section you want to protect.
For example, if the query string of a URI you want to protect is <yourdomain>/account.do/issue? account=moneytransfer, enter the parameter name and parameter value ‘account=moneytransfer’ to protect the “moneytransfer” function.
When you enter a Query parameter, the Bot Defense service looks at the Path and Query values. The Query field is regex matched.
In the Action drop-down menu, select the mitigation action you want to take for this endpoint if Bot Defense detects automation:
Continue (request continues to origin). For reporting purposes, you can optionally add a header to capture traffic that is going to the origin. You provide the header name in the Define Mitigation Actions section.
Important: When you initially deploy the iApp connector, F5 recommends that you set mitigation action to Continue for the first 24-48 hours and then monitor the system to confirm that it is functioning properly.
Redirect. The Bot is sent to a different page. You provide URL where the Bot is redirected later in the Define Mitigation Actions section. The redirect must go to:
An absolute URL, for example, “https://acme.com/redirect.html”.
A domain that you own.
Block. The Bot is presented with a message that it has been blocked. You provide the Response Code and Body of the response message later in the Define Mitigation Actions section.
Drop. This is not a valid selection. If you select Drop, Bot Defense defaults to Block.
Specify whether you want to ignore letter case while matching request paths. If you select Yes, letter case is ignored when matching the path of a request URL. If you select No then matching is case sensitive.
In the Specify redirect path field, enter the URL to which you want to redirect Bots when you select Redirect as a mitigation action.
In the Specify ‘Block’ response fields, enter a Response Code and then enter the body of the response message that you want to send to malicious traffic if you specify Block as a mitigation action. You must include the correct HTML tags in the body of the response.
When you select Continue as a mitigation response, specify whether you want to add a header to requests that provides F5 XC Bot Defense information. If you select Yes, add an F5 XC Bot Defense info header to requests sent to origin, then specify the header.
Allow-listed Entities
Optionally, create an Allow List that allows requests to reach the origin destination without being evaluated by Bot Defense. You are not charged for request that are not evaluated by the Bot Defense Engine. You can create a list by IP Address or by request headers.
To specify an IP Address Allowlist, in the IP field, enter the IP addresses you want to allow. Click Add after you enter each IP address.
Specify if you want to Use advanced header matching.
If you select Yes, Bot Defense uses regex to match the header name and value.
If you select No, then header names are matched exactly as you specify and header values are matches using simple * and ? wildcards.
Header name matching is always case insensitive.
To specify an HTTP Header Allowlist, enter a Name and Value for each header you want to allow. Click Add after you enter each name and value.
Figure: iApp Connector Allow List screen
Application Virtual Servers
In the Application Virtual Servers section, select the application virtual servers that you want to assign to your iApp. You iApp does not run if it is not assigned to at least one virtual server.
Note: You create virtual servers in BIG-IP. For information, see your BIG-IP documentation.
Know the following about selecting a virtual server:
The virtual server you select must have an HTTP profile attached to it.
The virtual server you select must have a default pool attached to it.
F5 recommends that each protected server have a Client SSL profile and SNAT profile, but these are not required.
Figure: iApp Connector Application Virtual Server screen
Note: If you have deployed an application on an iApp using a virtual server, you cannot deploy Bot Defense on an iApp using the same virtual server. You must choose a different virtual server.
F5 XC Defense Engine API Service Settings.
Define the API settings that allow you to route traffic to the Bot Defense engine and define pools.
The Application ID is automatically populated in the template.
Enter a Timeout value for API response. The default is 700 milliseconds. If the API response exceeds the configured value, then the request is allowed to continue without being checked for automation.
Enter the hostname of the F5 XC Web Defense Engine API service.
Enter the hostname of the F5 XC Mobile SDK Defense Engine API service.
Specify the Server SSL Profile that you want to use for outbound connections to F5 XD Defense Engine API servers.
To require that F5 XC Defense Engine API requests are routed through a proxy, select Yes, use an (explicit) HTTP proxy to connect to API servers, then do the following:
If the HTTP proxy requires basic authorization credentials, select Yes, send username and password in Proxy-Authentication header and then enter the Username and Password.
Select the HTTP proxy pool that you want to use. Perform one of the following tasks:
Select Create a new pool using a single proxy hostname and then enter the fully-qualified domain name (FQDN) and Port number of the new pool.
Select Create a new pool from one or more proxy IP addresses and then enter the proxy IP addresses and Port numbers you want to use. Click Add after each IP and Port number.
Select Choose an existing proxy pool and then select a proxy pool from the drop-down menu.
Figure: iApp connector F5 XC Defense Engine API Service Settings screen
F5 XC Defense Engine API Service(s) Health Monitor
If you do not require F5 XC Defense Engine API requests to go through a proxy, you can configure a health monitor to monitor the availability of your API service on both web and mobile clusters.
Health monitors display one of the following statuses for each cluster:
Available: The monitor is successfully monitoring the pool member.
Unknown: The monitor is unable to determine if the pool member is up or down.
Down: The monitor has determined that the pool member is currently down.
For detailed status information, see your BIG-IP Local Traffic Manager documentation.
To configure the health monitor, decide if you want to use the HTTP monitor or default monitor. The default monitor is not as sensitive as the HTTP monitor. The HTTP monitor is more sensitive but does not use TLS.
If you select Prefer HTTP monitor, review the following settings. You should not need to change the default settings.
Review which TCP port the HTTP monitor should query.
Review which HTTP path the HTTP monitor should query.
Review what HTTP status code means that the service is unavailable.
Figure: iApp connector health monitor screen
Advanced Features
Enter the IP address that the F5 XC Connector should use to contact internal virtual servers.
In the field, Do you want to update the HTTP X-Forwarded-For header, decide if you want BIG-IP to replace X-Forwarded-For IP address at the security perimeter.
Specify how you want to handle client requests with Expect: 100 header. Select Send prompt 100-Continue if you want BIG-IP to wait for larger requests to complete rather than time out.
Decide if you want to use an HSL log publisher or use the BIG-IP syslogd. If you choose to use the HSL log publisher, from the HSL log publisher drop-down menu, select the publisher you want to use.
Select a logging level to determine the verbosity of the logging data this is collected:
Error – Includes connection and network errors. Does not include transaction errors.
Warning – Includes information about malicious transactions. The Warning level also includes all Error-level log data.
Info – Includes information about both human and malicious transactions. The Info level also includes all Warning-level log data.
Debug – Full transaction logs that include all available log data, including all data from Error, Warning and Info levels.
Note: Under normal circumstances, you can set a lower logging level, such as Error or Warning. If you suspect that an incident is occurring, you can change logging levels to Info or Debug to help your investigation and defense, and then change it back later.
In the field, Which syslog ‘facility’ do you want to send log messages to, decide where you want the syslog file to be written.
In the field, Use what log-message format, enter the log format you want to use for syslog.
In the field, What request headers (if any) should request log messages include, enter a Header name. The header name can be exact or regex.
Figure: iApp connector advanced features screen
Save the application
When you finish making configuration changes, double-check your configuration settings and click Finished.
Delete an iApp Application
Deleting an iApp application is two-step process. First you must run a cleanup process. Then you can safely delete the iApp application. Note that deleting an iApp application removes all configuration information for the application.
In the BIG-IP that hosts your applications, click iApps > Application Services > Applications.
From the list of applications, click the application you want to delete.
Select the Reconfigure tab.
In the General section, select Yes to Clean Before Deletion. This removes or restores all the BIG-IP configuration components which were added/modified by the Bot Defense iApp template that are not automatically restored when you delete the iApp Application
Then scroll to the bottom of the page and click Finished.
When the cleanup process finishes, from the list of applications, click the application you want to delete.
Click Delete.
Enabling F5 Bot Defense on Cloudflare
To enable F5 Bot Defense on Cloudflare, you must complete the following high-level tasks:
Download a configuration file and template from the Distributed Cloud (XC) Console.
Create a Cloudflare Worker and deploy the connector in Cloudflare.
Prerequisites
You must have a functioning Cloudflare environment.
You must know how to configure and maintain your Cloudflare environment, including how to create and configure Cloudflare Workers. See your Cloudflare documentation for information.
To enable Bot Defense on Cloudflare, you must first configure the Cloudflare connector in the XC Console.
On the XC Console Dashboard, click Bot Defense.
Click Manage > Applications > Add Application. If no applications exist, you are prompted to add a protected application.
Add a Name and Description for the new application.
The name must be unique in the namespace and must follow DNS-1035 format.
Select an Application Region (US, EU, or Asia). This is the region where the origin server for the application is located.
From the Connector Type drop-down list, select Cloudflare and then select Configure.
Figure: Connector Type
Protected Endpoints
Configure the endpoints you want to protect with Bot Defense.
In the Protected Endpoints section, click Configure and then click Add Item.
Enter a Name and Description for the new endpoint.
Specify the Domain Matcher. The same Cloudflare website can be served from multiple domains. You can choose any domain or specify a specific host value. Enter an exact value, a suffix value, or a regex value.
Specify the Path to the endpoint. For example, enter /login. The path can include the wildcards, * and ?.
If you want Bot Defense to ignore capitalization when searching paths, select Case insensitive.
Figure: Case Sensitivity Setting
(Optional) In the Query field, if you want to protect specific sections of a page, enter the criteria for the section you want to protect.
For example, if the query string of a URI you want to protect is:
<yourdomain>/account.do/issue? account=moneytransfer, enter the parameter name and parameter value 'account=moneytransfer’ to protect the “moneytransfer” function.
When you enter criteria in the Query field, Bot Defense looks at the Query parameters in addition to the Path. The Query field is regex matched.
Choose the HTTP Methods that are monitored on this endpoint by Bot Defense:
GET (XHR/Fetch)
POST
PUT
You can select multiple methods.
Select the type of client that accesses this endpoint (Web Client, Mobile Client, or Web and Mobile Client).
Note: For endpoints that receive requests from both web and mobile clients, F5 recommends that you add a single endpoint and select Web and Mobile Client, rather than two separate endpoints for Web Client and Mobile Client.
Select the mitigation action for this endpoint when automation is detected. If you selected the Web and Mobile Client type, select a mitigation action for each type:
Continue. This allows traffic to continue to the origin. You can choose to add a header that enables you to monitor traffic in reporting and analytics dashboards. You can change the header name in the Header Name for Continue Mitigation Action field in the More Options section.
Redirect. This redirects automated traffic to a different page. You provide the appropriate Status Code and URI where you want to redirect the bot. Use this option to deceive bots into thinking their request successfully went through. You can also redirect to a page with more information for human users that may have been incorrectly redirected.
Block. This blocks automated traffic from proceeding to the origin. You can customize the Status Code, Content Type, and Body of the response message.
Note: Mobile clients only allow Continue and Block. If you select Web and Mobile Client, you can select mitigation actions for each client type.
Figure: Client Type
Note: If you select Web and Mobile Client, you must enter a Mobile Request Identifier Header when you enable Mobile SDK Settings. For instructions, see Mobile Settings.
When you finish configuring the endpoint, click Apply.
To add an additional endpoint, click Add Item and repeat steps 2-10.
When you finish adding endpoints, click Apply.
Web Client JavaScript Settings
After you add web-protected endpoints, you must configure how the XC Bot Defense JavaScript is injected into the web pages you want to protect.
Note: If you only protect mobile endpoints, then select Disable JavaScript Insertion and go to Mobile Settings.
From the Web Client JavaScript Insertion Settings drop-down list, select Specify JavaScript Insertion Rules and click Configure.
Add the Web Client JavaScript Path. The web client fetches the Bot Defense JavaScript from this path. This path cannot conflict with any other website or application paths.
From the JavaScript Location drop-down menu, choose the location where you want to insert the Bot Defense JavaScript in the application pages:
After <head> tag
After </title> tag
Before <script> tag
F5 recommends that you select After <head> so that the Bot Defense JavaScript is executed early. This allows time for the Bot Defense JavaScript to be fetched and executed while the rest of page is rendered.
Note: If you select After </title>, be sure your application pages have the <title> tag.
In the JavaScript Insertion Paths section, click Add Item.
Enter a Name, Description, Domain Matcher, and Path for each page in your application where you want to insert the Bot Defense JavaScript.
Note: Be sure to select paths to HTML pages that end users are likely to visit before they browse to any protected endpoint. In Bot Defense, these are called “entry pages.”
Note: F5 recommends that you use wildcards to select JavaScript insertion paths for HTML pages to which end users are likely to go, such as /index.htm and /login/*.
JavaScript insertion path examples:
/login/*
/catalog
Click Apply. The JavaScript Insertion Path is added to the table. Click Add Item to add additional JavaScript Insertion Paths.
(Optional) To exclude the Bot Defense JavaScript from specific web pages, in the JavaScript Exclude Paths section, click Add Item.
Enter a Name, Description, Domain Matcher, and Path to exclude.
Click Apply. This adds an item to the table. Click Add Item to add additional exclusion paths.
Note: F5 recommends that you carefully add JavaScript insertion paths to minimize false positives.
JavaScript exclusion path examples:
/login/images
/catalog/soldout/*
When you finish configuring JavaScript Insertion Rules, click Apply.
Manual JavaScript Insertion
Important: F5 recommends that you allow Bot Defense to automatically insert the Bot Defense JavaScript in your application pages. Only use manual insertion if it is required for Bot Defense to work in your IT environment.
To manually insert the Bot Defense JavaScript in your application pages:
In the Web Client JavaScript Settings section, from the Web Client JavaScript Insertions drop-down menu, select Disable JavaScript Insertion.
Manually add the JavaScript tags below to one of the following recommended locations:
Replace INJECTION PATH with the value you specified for Web Client JavaScript Path.
Figure: Manual Javascript insertion
Mobile Settings
If you added mobile endpoints to your configuration, select Enable Mobile SDK.
If you selected Web and Mobile Client as your client type during endpoint configuration, add a mobile header to distinguish the endpoints. This is not required if you selected only Web Client or Mobile Client.
Enable Show Advanced Fields.
In the Mobile Request Identifier Headers section, click Add Item.
Figure: Mobile Settings
Enter a Name and the corresponding Value. Specify if the value must be exact or if it can be regex.
Figure: Header Configuration
Click Apply.
More Options
Important: Enable Show Advanced Fields to see all options.
Trusted Client Rules (Allow List)
Trusted Client Rules add headers and IPv4 addresses to an Allow List. Incoming HTTP requests from the allowed IPs or requests that contain allowed header name/value pairs bypass Bot Defense engine for evaluation and proceed directly to the origin. There is no logging for pages on the Allow List.
In the Trusted Client Rules section, click Configure and then click Add Item.
Enter a Name and specify the Client Identifier. Choose either IP Prefix or HTTP Headers.
For IP Prefix, enter a string.
For HTTP Header, click Add Item, enter a Name and Value and then click Apply.
Note: You can add multiple headers to the table at the same time. You must add IP addresses individually.
Figure: Trusted Client Rules
Click Apply when you finish adding Trusted Client Rules and then click Apply again to configure More Options.
Set Logging Level
In the More Options section, click Show Advanced Fields.
Select one of the following logging levels to determine the verbosity of the logging data collected by the system:
Error – Includes connection and network errors. Does not include transaction errors.
Warning – Includes information about malicious transactions. The Warning level also includes all Error-level log information.
Informational – Includes information about both human and malicious transactions. The Informational level also includes all Warning-level log information.
Debug – Full transaction logs that include all available log data, including all data from Error, Warning and Informational levels.
Define Continue Mitigation Header
In the More Options section, click Show Advanced Fields.
The Header Name for Continue Mitigation Action field is the header that is added to a request when automation is detected and you have set the mitigation action to Continue. You made this selection when you configured your protected endpoints. For information, see Protected Endpoints.
Timeout and Body Sample Size Limit
In the More Options section, click Show Advanced Fields.
Specify the Timeout value. This value defines the maximum wait time for the Bot Defense engine to run the analysis and return the inference. If the timeout is exceeded, the request continues to the origin. By default, the field is set to 700 ms based on performance efficiency.
Download the Cloudflare Connector Configuration and Template Files
When you finish configuring the Cloudflare connector in the XC Console, you must download Worker and configuration files that you use to deploy the connector in Cloudflare.
In the Bot Defense dashboard, click Manage > Applications.
From the Action column in the list of applications, next to the application you just added, click the Action Menu … .
Figure: Action Menu
Click Download Config File and save the <app_name>.json file in a location you can access later.
Click Download Worker File and save the <app_name>.js file in a location you can access later.
Note:<app_name> is the name of the application you added to Bot Defense earlier. You can decide to save the .js and .json files with different names.
Configure the Connector in Cloudflare
Important: In Cloudflare, DNS must be in proxy mode.
Important: In Cloudflare, check your worker rate-limiting settings. Depending on your traffic volume, you might need to disable rate limiting. To disable rate limiting, contact Cloudflare support.
To deploy the Bot Defense Connector in Cloudflare, you must create a new Cloudflare Worker and then insert the connector code in the Worker.
In Cloudflare, create a new Worker to deploy the Bot Defense connector.
Navigate to the Worker you created.
Edit the Worker so you can replace the default Worker code with the code from the configuration files that you downloaded from the XC Console.
On the Worker edit page, clear the default Worker code and enter the entire contents of the <app_name>.js file you downloaded.
After you enter the code from the .js file, on the line const CONFIG = “__Config__”, replace “__Config__” (including the quotes) with the entire contents of the .json file that you downloaded from the XC Console.
Assign a route to the Worker you created for the Bot Defense service. Enter a wildcard or /* so the route matches all HTML pages and endpoints.
Save and deploy your changes in Cloudflare.
Confirm that Bot Defense is Functioning Properly
Perform the following steps to help ensure that Bot Defense is properly configured and evaluating traffic correctly.
F5 recommends that you enable stream logging in Cloudflare to confirm that requests are being evaluated by Bot Defense correctly.
On your website, inspect a protected endpoint to confirm that Bot Defense inserted the Bot Defense JavaScript.
Review the Bot Defense dashboard to see the types of traffic identified by Bot Defense
Run composer require f5/distributed_cloud_connector:VERSION (for example, f5/distributed_cloud_connector:1.0.2). The version of the extension can be found in the composer.json file.
Run php bin/magento set:up.
Run php bin/magento s:d:c.
Run php bin/magento setup:static-content:deploy. If Magento is in developer mode, add an -f argument. Developer mode is not officially supported in the extension.
Extension Configuration
To find the F5 Extension configurations, go to:
Admin Panel > Stores > Configuration > F5® DISTRIBUTED CLOUD SERVICES.
Configuration modules include:
Bot Defense
Account Protection
Authentication Intelligence
Bot Defense
The Bot Defense module contains these sections:
General
JS Insertion
Login Protection
Protected Endpoints
Web Scraping
Allow List
General
The General section contains options to enable and configure the connection of the Bot Defense module with the F5 Service.
The configuration options are:
Enable Service - enable/disable the module.
Mode - there are two options available:
Enterprise - the dedicated mode of usage where the “Application ID” and “Tenant ID” fields are not required.
Standard - the multi-tenant mode of operation.
The mode should be selected according to the F5 plan.
API Hostname - can be obtained from the F5 Distributed Cloud Console.
Backend API path - the path to the Bot Defense script on the website backend.
API Key - the key used for authentication of the API calls to the F5 system. The key can be obtained from the F5® Distributed Cloud console.
Telemetry Header Prefix - can be obtained from the F5 Distributed Cloud Console.
Application ID - available only when Standard mode is selected, and is available in the F5® Distributed Cloud console.
Tenant ID - available only when Standard mode is selected, and is available in the F5® Distributed Cloud console.
Timeout API response - the time (in milliseconds) that the module waits for a response from the F5 server. After that, the request is processed by the Adobe Commerce backend. The default is 700.
Source for Client IP address - select which source to use for getting the customer’s IP address:
Connecting IP - the IP address is gathered from the customer’s connection information.
X-forwarded-For - the IP address is gathered from the X-forwarded-For header.
Custom - define the custom header name to use for gathering the customer’s IP address.
Custom Ip address source - the custom header name for gathering the customer’s IP address. In the configured header, the first character of each word should be upper case.
Enable Debug - when enabled, the requests and responses to/from the F5 service are recorded into the log file on the website server.
Figure: General section of Adobe Commerce Cloud connector
JS Insertion
The JS Insertion section contains options to configure the JS tag insertion.
The configuration options are:
JS Path - either an absolute path, or a relative path to the JS:
Absolute example - https://<FQDN+JS_location>, such as https://www.foo.com/abc.js.
If using absolute tag insertion, verify the JS requests are routed to the F5 Bot Defense service.
Relative example - </JS_location>, such as /abc.js.
Insert JS option - specifies where in the page to add the JS tag:
Before <script>
After </title>
After <head>
JavaScript load mode - select the load mode of the JavaScript on the page:
Async with caching
Async with no caching
Sync with caching
Async with caching and defer XHR
Insert JS to specific web pages - displays the Specific pages list, a list of pages that should have the JS inserted.
If Yes, the Specific pages list displays:
Only the path to the page should be defined, such as /about-us.
Each page should be a new line.
Wildcards can be used for page path definitions, such as *, and so on.
To insert the JS tag on all pages put only an asterisk (*).
The JS tag is not inserted on any pages if this field is empty.
If No,the Specific page list field is hidden and the JS tag is displayed on all available pages that are not defined in the Excluded pages list field.
Exclude JS from specific web pages - displays the Excluded pages list, a list of pages that should be excluded from JS insertion.
If Yes, the Excluded pages list displays:
Only the path to the page should be defined, such as /about-us.
Each page should be a new line.
Wildcards can be used for page path definitions, such as *, and so on.
If No, the Excluded pages list field is hidden and the JS tag is inserted according to the Insert JS to specific web pages configuration.
Figure: JS Insertion section of Adobe Commerce Cloud connector
Login Protection
The Login Protection functionality configures Bot Defense for the Login action directly.
The configuration options are:
Protection type - defines the protection type, one of:
Disable - disables login protection.
Collect txn - customer verification is performed after sending the login request to the origin destination.
Ignore txn - customer verification is performed right after clicking the Login button on the storefront, but before the Login request is performed to the origin destination.
Login Url - defines the login action URL that should be protected by Bot Defense. By default it is set to /customer/account/loginpost*, which is used by the Adobe Commerce platform. It can be customized in cases when a non-default login functionality is being used on the website.
Mitigation - there are three options available:
Continue - the user is allowed to visit the destination page, but is flagged by the header defined in the Mitigation Continue header field.
Block - the user is blocked from reaching the destination page. They are routed to a blank page with the content defined in the Blocked Response Body field and the response code defined in the Blocked Response Code field.
Redirect - the user is redirected to the path defined in Mitigation Redirect path.
Figure: Login protection section of Adobe Commerce Cloud connector
Protected Endpoints
The Protected Endpoints functionality allows to define pages and sources on the website that should be protected by Bot Defense.
The configuration options are:
URL endpoint - the path to the page or action that should be protected. Wildcards can be used to define complex URLs.
Method - the type of the request being performed to the defined page or action. These types of request methods are supported:
GET XHR
POST
PUT
Mitigation - the mitigation action type. There are three options:
Continue - the user is allowed to visit the destination page, but is flagged by the header defined in the Flag Header name field.
Block - the user is blocked from reaching the destination page. They are routed to a blank page with the content defined in the Block Response Body field and the response code defined in the Block Response Code field.
Redirect - the user is redirected to the path defined in Redirect location.
Figure: Protected endpoints section of Adobe Commerce Cloud connector
Web Scraping
The Web Scraping functionality defines pages and sources on the website that should be protected by Bot Defense. This option allows you to protect documents and pages that are accessible only by GET requests.
The configuration options are:
URL endpoint - the path to the page or action that should be protected. Wildcards can be used to define complex URLs.
Mitigation - the mitigation action type. There are three options:
Continue - the user is allowed to visit the destination page, but is flagged by the header defined in the Flag Header name field.
Block - the user is blocked from reaching the destination page. They are routed to a blank page with the content defined in the Block response Body field and the response code defined in the Block response Code field.
Redirect - the user is redirected to the path defined in Redirect location.
Figure: Web scraping section of Adobe Commerce Cloud connector
Allow List
Allow List allows requests to get to the origin destination without performing verification by Bot Defense. There are two options that give this possibility, allow listing by IP Address or by request headers.
Allowed IP Addresses
Requests sent from the IP addresses defined in the Allowed IP addresses field are delivered to the endpoint without performing verification. Each IP address should be defined on a new line.
Allow Header
Requests that contain the [header]:[value] pair are delivered to the endpoint without performing verification. Each [header]:[value] pair should be defined on a new line. Wildcards are supported for both header name and header value.
Figure: Allow list section of Adobe Commerce Cloud connector
Account Protection
F5 Distributed Cloud Account Protection provides a converged solution for application security and fraud protection powered by a real-time, closed-loop engine and large-scale unified telemetry to reduce customer friction and stop fraud before it happens.
The Account Protection module contains these sections:
General
Protected Endpoints
Action
General
The General section contains options to enable and configure the connection of the Account Protection module with the F5 Service. The path to the module is:
API Hostname - the host name of the Account Protection service.
Script Tag - path to the F5 Account Protection script
JS Path - path to the JS file.
Customer ID - the customer identifier, can be obtained from the F5 Service.
Cookie name - the name of the cookie from the Account Protection service that should be decrypted and processed.
Decryption key - the key that decrypts the fr value in the cookie received from the Account Protection service.
Protected Endpoints
These options define the endpoints that should be protected by the Account Protection.
URL endpoint - the path to the page or action that should be protected. Wildcards can be used to define complex URLs.
Mitigation - the mitigation action type. There are three options:
Continue - the user is allowed to visit the destination page, but is flagged by the header defined in the Mitigation Continue header field.
Block - the user is blocked from reaching the destination page. They are routed to a blank page with the content defined in the Blocked response Body field and the response code defined in the Blocked response Code field.
Redirect - the user is redirected to the path defined in Mitigation Redirect path.
Method - the type of request being performed on the defined page. These types of request methods are supported:
GET
POST
PUT
Blocked Response Body - the message to display on the blank page when the Block mitigation action is performed.
Blocked Response Code - the status code to deliver when the Block mitigation action is performed.
Redirect location - the path the user is redirected to when the Redirect mitigation action is performed.
Figure: Protected endpoint options
Action
The Action section contains the mitigation actions settings, such as:
Missing Cookie - allows you to select the mitigation action for cases when the Account Protection cookie is missing. There are three options:
Continue
Block
Redirect
Invalid Cookie - allows you to select the mitigation action for cases when the Account Protection cookie is invalid. There are three options:
Continue
Block
Redirect
Authentication Intelligence
Authentication Intelligence extends the login session lifetime for non-malicious users, after they are verified by F5.
General
The General section contains options to enable and configure the connection of the Authentication Intelligence module with the F5 Service.
The configuration options are:
Enable Service - enables or disables the module.
API Hostname - the host name of the Authentication Intelligence service.
Script Tag - path to the F5 Authentication Intelligence script.
JS Path - path to the JS file.
Customer ID - the customer identifier, can be obtained from the F5 Service.
Cookie name - the name of the cookie from the Authentication Intelligence service that should be decrypted and processed.
Decryption key - the key that decrypts the c value in the cookie received from the Authentication Intelligence service.
Figure: Authentication Intelligence section of Adobe Commerce Cloud connector
Amazon CloudFront
Create a new Bot Defense application for Amazon CloudFront
On the XC Console Dashboard, click Bot Defense.
Click Manage > Applications > Add Application. If no applications exist, you are prompted to add a protected application.
Add a Name and Description for the Application.
Select an Application Region (US, EU, or Asia).
From the Connector Type drop-down list, select Amazon CloudFront and click Configure. Options appear to enter Amazon Reference Information.
Add AWS Reference Information
From the AWS Configuration drop-down list, select how you want to associate your protected application with your AWS distribution. Choose Select AWS CloudFront Distribution ID or Select AWS CloudFront Distribution by tag matching.
Click Configure and then depending on how you chose to associate your application with your distribution, enter a Distribution ID or a Distribution Tag.
Click Add Item to add another distirbution or click Apply.
Figure: Associate a protected application with your AWS distribution
Note: You can choose to manually associate your distributions after you download the integration package. For information, contact support@cloud.f5.com.
Add Protected Endpoints
In the Protected Endpoints section, click Configure and then click Add Item.
Enter a Name and Description for the new endpoint.
Specify the Domain Matcher. You can choose any domain or specify a specific host value.
Specify the Path to the endpoint. For example, enter /login.
Specify a Query parameter, if needed. If a Query value is defined, the Bot Defense service looks at the Path and Query values.
Choose the HTTP Methods that are monitored on this endpoint by Bot Defense. You can select multiple methods.
Enter Endpoint Labels to help categorize and organize your endpoints. Endpoint labels also allow more granular attack intent identification and reporting when automation occurs. For example, to identify a gift card checkout flow on an endpoint, select Specify endpoint label category > Shopping & Gift Cards > Purchase with Gift Card. F5 recommends that you configure endpoint labels immediately so that Bot Defense can begin collecting this information and reporting on endpoint details. For a full list of available endpoint labels, see Endpoint Labels.
Note: For web scraping labels, select Specify endpoint label category > Search and select an available flow label.
Note: A subset of endpoint labels is included in BIG-IP 17.1. For information, see your BIG-IP product documentation.
Select the type of client that accesses this endpoint (Web Client, Mobile Client, or Web and Mobile Client).
Select the mitigation action for this endpoint when automation is detected:
Continue (request continues to origin). For reporting purposes, you can optionally add a header to capture traffic that is going to the origin. Provide the header name in the Define Mitigation Header section.
Redirect. Provide the appropriate Status Code and URI.
Block. Enter the Status Code, Content Type, and Body of the response message.
Note: Mobile clients only allow Continue and Block. If you select Web and Mobile Client, you can select mitigation actions for each client type.
Note: If you select Web and Mobile Client, you must enter a Mobile Request Identifier Header when you enable Mobile SDK Settings. For instructions, see Define Mobile SDK Settings.
When you finish configuring the endpoint, click Apply.
Your protected endpoint is added to the table. To modify the endpoint, click Actions > Edit. To add additional endpoints, click the Add Item button.
When you finish adding endpoints, click Apply.
Define Web Client JavaScript Settings
Note: You only configure Web Client JavaScript Settings if you have web-protected endpoints. If you only protect mobile endpoints, then select Disable JavaScript Insertion and skip to Define Mobile SDK Settings.
Note: To manually insert the protection JavaScript in your application, enable Show Advanced Fields, select Insert JavaScript Manually, and enter a Web Client JavaScript Path. Then see Manual JS Insertion for information.
From the Web Client JavaScript Insertion Settings drop-down list, select Specify JavaScript Insertion Rules and click Configure.
Add the Web Client JavaScript Path. The web client fetches the F5 Client JavaScript from this path. The this path cannot conflict with any other website or application paths.
Figure: Web Client JavaScript Settings
From the Web Client JavaScript Mode drop-down menu, determine how the protection JavaScript is loaded into the entry page. The JavaScript is loaded in two parts. The larger part can be loaded in two possible ways:
Asynchronous: The JavaScript runs as soon as it is loaded and does not block the page from loading.
Synchronous: The page does not continue loading until the JavaScript has been loaded and executed.
You can also choose to cache the larger part of the JavaScript so it does not have to be downloaded each time it is loaded into a new page.
Choose one of the following options:
Async JS with no Caching
Async JS with Caching
Sync JS with no Caching
Sync JS with Caching
From the JavaScript Location drop-down list, choose the location where you want to insert the JavaScript in the code:
After <head> tag
After </title> tag
Before <script> tag
In the JavaScript Insertion Paths section, click Add Item.
Enter a Name, Description, Domain Matcher, and Path for each page in your application where you want to insert the protection JavaScript.
Note: Be sure to select paths to HTML pages that end users are likely to visit before they browse to any protected endpoint.
Note F5 recommends that you use wildcards to select JavaScript insertion paths for HTML pages to which end users are likely to go, such as /index.htm and /login/*.
Click Apply. The JavaScript Insertion Path is added to the table. Click Add Item to add additional JavaScript Insertion Paths.
To exclude the protection JavaScript from specific web pages, in the JavaScript Exclude Paths section, click Add Item.
Enter a Name, Description, Domain Matcher, and Path to exclude.
Click Apply. This adds an item to the table. Click Add Item to add additional exclusion paths.
Note F5 recommends that you carefully add JavaScript Insertions and that you do not add a long list of exclusions. A small cost is incurred for each inclusion request for AWS lambda to check for exclusions.
JavaScript insertion path examples:
/login/*
/catalog
JavaScript exclusion path examples:
/login/images
/catalog/soldout/*
When you finish configuring JavaScript Insertion Rules, click Apply.
Define Mobile SDK Settings
If you added mobile endpoints to your configuration, select Enable Mobile SDK.
If you selected Web and Mobile Client as your client type during endpoint configuration, add a mobile header to distinguish the endpoints. This is not required if you selected only Web Client or Mobile Client.
Enable Show Advanced Fields.
In the Mobile Request Identifier Headers section, Click Add Item.
Figure: Show Advanced Fields
Enter a Name and the corresponding Value. Specify if the value must be exact or if it can be regex.
Click Apply.
Configure More Options
Enable Show Advanced Fields to see all options.
Trusted Client Rules (Allow List)
Trusted Client Rules add headers and IP addresses to an Allow List. Pages that have a specific IP address or that contain specific headers are allowed to proceed to the origin and are not sent to the Bot Defense engine for evaluation. There is no logging for pages on the Allow List.
In the Trusted Client Rules section, click Configure and then click Add Item.
Enter a Name and specify the Client Identifier. Choose either IP Prefix or HTTP Headers.
For IP Prefix, enter a string.
For HTTP Header, click Add Item, enter a Name and Value and then click Apply.
Note: You can add multiple headers to the table at the same time. You must add IP addresses individually.
Click Apply when you finish adding Trusted Client Rules and then click Apply again to configure More Options.
Set Logging Level
Note: Enable Show Advanced Fields to see the following options.
Select a logging level to determine the verbosity of logs that are written to AWS storage:
Error – Includes connection and network errors. Does not include transaction errors.
Warning – Includes information about malicious transactions that show they were from bots. The Warning level also includes all Error-level log information.
Info – More verbose than Warning. The Info level also includes all Warning-level log information.
Debug – Full transaction logs that include all available log data.
Define Continue Mitigation Header
Note: Enable Show Advanced Fields to see the following option.
The Header Name for Continue Mitigation Action field is the header that is added to a request when automation is detected and you have set the mitigation action to Continue and Add Header for the endpoint. You made these selections when you configured your protected endpoints. For information, see Configure Protected Endpoints.
Timeout and Body Sample Size Limit
Note: Enable Show Advanced Fields to see the following options.
Timeout – This defines the maximum time to send the requests to the Bot Defense Engine for analysis. If the timeout is exceeded, the request continues to the origin (this is tracked in AWS CloudWatch). By default, the field is set to 700 ms based on performance efficiency.
Body Sample Size - This allows additional request body data (other than F5 telemetry) to be sent for analysis. By default, this is set to 0 MB. Max size limit is 1 MB.
Manual JS Insertion
If you require Manual JavaScript Insertion, add the following tags to one of the recommended locations:
Replace INJECTION PATH with the value you specified for Web Client JavaScript Path.
Figure: Manual Javascript insertion
Download Config File and AWS Installer Tool
In the Actions column of the Applications table, click the Action menu (…) for your application.
Click Download Config File.
Click Download AWS Installer.
Deploy the F5 Connector Using Amazon CloudFront
Log in to AWS Console home page.
Select AWS Region Northern Virginia (US-EAST-1).
Use the search to find Serverless Application Repository and click it.
Click Available Applications.
Click Public Applications.
Figure: Public Application Tab
Click the F5BotDefense tile to install the F5 XC Connector for Amazon CloudFront code module.
Important: Be sure to download the latest version of F5BotDefense in the Serverless Application Repository.
If there are too many tiles here, search for f5.
If the F5 connector tile does not appear, validate the AWS Account number you provided to F5.
Click Deploy to install the F5 Connector for CloudFront.
Deploying the F5 Connector creates a new Lambda Application in your AWS Account. AWS sets the name of the new Lambda Application to start with serverlessrepo-.
The deployment can take a few minutes to complete. It is complete when you see the F5BotDefense of type Lambda Function.
Click the name F5BotDefense to review contents of the installed Lambda Function.
Important: The available rate limit for Lambda functions is shared by all functions in your AWS account. If you reach the Lambda rate limit, an error displays in AWS. The rate limit for Bot Defense is 1000 per second. If you reach the Bot Defense rate limit, user traffic is allowed to continue to the origin with no mitigation from Bot Defense. F5 recommends that you increase the rate limit so that it accommodates all of the Lambda functions in your AWS account. For instructions, see your AWS documentation.
Configure the F5 Bot Defense Connector in AWS
F5 recommends that you use the F5 CLI tool to configure the F5 Bot Defense Connector in AWS. For best results, F5 recommends that you use the AWS CloudShell.
After you start AWS CloudShell, click Actions and Upload file.
Upload config.json and *f5tool, which you downloaded from the F5 XC Console.
When the upload is finished, run ls to confirm that the files were uploaded.
Run bash f5tool -–install config.json. Installation can take up to 5 minutes.
The installation tool saves the previous configuration of each CloudFront Distribution in a file. You can use the F5 tool to restore a saved Distribution config (thus removing F5 Bot Defense).
Note: Your F5 XC Bot Defense configuration is sensitive security information and is stored in AWS Secrets Manager. You should delete config.json after CLI installation.
AWS CloudWatch
AWS CloudWatch contains logs for Lambda function deployed by f5ConnectorCloudFront serverless application.
The Log group name starts with /aws/lambda/us-east-1.serverlessrepo-f5ConnectorCl-f5ConnectorCloudFront-.
The logs of lambda function can be found in the region closest to the location where the function executed.
For troubleshooting, look for error messages contained in the links under Log steams.
View Traffic
After your configuration is added, navigate to Monitor to view all traffic that the F5 XC Defense Engine has recorded for valid and invalid requests. This tool can help analyze thousands or millions of requests.
An asterisk * matches any sequence of zero or more characters.
A question-mark ? matches any single character.
For example, to find the word 'food' in the middle of a string, you must put asterisks before and after it. For example:
'*food*' matches both 'bird food' and 'food for birds'.
'food*' (single asterisk) only matches 'food for birds'.
'food' (no asterisks) does not match either one.
If a pattern expression does not match all of the characters in a target string (either exactly or by * or ?), the pattern reports no match.
Example query URL:
/catalog/socks?color=red®ion=emea&n=20
Example URL path match:
All pages that start with /catalog/ (note each / matters)
/catalog/*
Example URL query parameter matches:
Query-string must include region=emea
(^|[^w])region=emea([^w]|$)
Query-string must include region=emea OR region=latam
(^|[^w])region=(emea|latam)([^w]|$)
Query-string must include region=emea OR region=latam in
any mix of UPPER- and lower-case letters, like region=LatAm
(?i)(^|[^w])region=(emea|latam)([^w]|$)
Query-string must either be empty or include region=emea
(^$|(^|[^w])region=emea([^w]|$))
Incorrect Usage (Would match unwanted text)
Query-string must include region=emea
(DO NOT USE THIS--) region=emea (OOPS! MATCHES testregion=emea)
Match URL Query Parameters (and other regular-expression matches)
Basic regular-expression (regex) syntax:
A period . matches any single character.
A period followed by an asterisk .* matches any sequence of zero or more characters.
[abc] and [pqr] match respectively any character in or not-in the list.
[0-7] matches digits 0-7 inclusive.
By default, regex reports partial matches. For example, the regex 'food' matches both 'bird food' and 'food for birds'. To prevent a partial match, use a caret ^ at the left and/or dollar-sign $ at the right to force matches at the start or end of the target string. For example, ^food matches 'food for birds' and food$ matches 'bird food', while ^food$ only matches 'food' by itself with nothing preceding or following those four letters.
To match multiple distinct words separate them by pipe symbols | and surround the set with parentheses (). For example, enter ^(food|drink|toothp.*)$ to match any string which contains exactly 'food' or 'drink' or anything that starts with 'toothp', such as 'toothpick' or 'toothpaste'.
Regular expressions have many additional powerful features, such as a? to match zero-or-one letter 'a'.
This system supports most RE2 or TCL ARE regular expression syntax. For a full description of features, use a search engine to search for 'RE2 syntax'.
Javascript (ECMAScript) uses very similar syntax, but differs in a few ways. For comparison information, use a search engine to search for 'cmcdragonkai regular expression comparison'.
For endoint matches, use regex syntax common to RE2 and Javascript. The exception is (?i), which is specially handled for you.
When you look for several different words, for best performance, use a single regex like (red|blue|green) rather than searching for each word separately.
When you use regex, be aware of partial matching. For example, to match a URL path starting with /mobile you must enter ^/mobile, otherwise paths such as '/docs/mobile-info.htm' (/mobile in the middle) also match.
For regex endpoint matching:
To ignore letter-case, prepend (?i) to your expression. For example, enter '(?i)^food'.
Use only regex syntax common between RE2, TCL ARE, and Javascript. The exception is (?i), which Bot Defense handles specially.
Be aware of matching at word boundaries. The syntax [\w] works in all cases. For example, to find the word 'head' by itself, possibly surrounded by punctuation as with '<head>' but NOT embedded in a longer word like 'sleepyhead' or 'headlamp', enter (^|[\w])head([\w]|$). To match the word 'head' or the empty string, which is often useful with URL query strings, use (^$|(^|[\w])head([\w]|$)).
Endpoint Labels
Use endpoint labels to help categorize and organize your endpoints, and to allow more granular attack intent identification and reporting when automation occurs.
Category
Label
Description
Authentication
Login
When credentials are provided to gain access to a saved account.
Authentication
Login MFA
When multi-factor authentication is used to gain access to a saved account.
Authentication
Login for Channel Partner
Login to a saved account occurs through a partner via a different channel.
Authentication
Logout
When a session ends or saved account is exited.
Authentication
Token Refresh
Session remains valid; login is not requested again to access a saved account.
Account Management
Account Creation
When an account is created (for example, at a department store, a restaurant, or with a ride-share service).
Account Management
Password Reset
When a password is reset.
Profile Management
Profile Creation
When a new user profile has been created; some accounts allow for multiple profiles to be added (for example, health insurance with dependents).
Profile Management
Profile Update
When profile infromation (may contain personal or sensitive information, such as password, email, or payment info.) is updated.
Profile Management
Profile View
When profile information (may contain personal or sensitive information, such as password, email, payment info) is viewed.
Shopping & Gift Cards
Add to Cart
When an item has been added to the shopping cart.
Shopping & Gift Cards
Promo Code Validation
When a promotional code is validated.
Shopping & Gift Cards
Checkout
When payment is submited for items in the cart.
Shopping & Gift Cards
Payment / Billing
When payment/billing information is added.
Shopping & Gift Cards
Order Submit
When an order is submited to the vendor to be fulfilled.
Shopping & Gift Cards
Price Inquiry
When a price inquiry is submitted for a product, insurance, quote, and so on.
Shopping & Gift Cards
Purchase a Gift Card
When a gift card is purchased for a specified amount.
Shopping & Gift Cards
Update Quantity
When the cart quanity for an item has been updated.
Shopping & Gift Cards
Select Seat(s)
When a seat (for a flight, concert, movie hall, sporting event) has been selected.
Shopping & Gift Cards
Enter Drawing Submission
Used to submit entry for a drawing or lottery.
Shopping & Gift Cards
Gift Card Validation
Used when validating a gift card value.
Shopping & Gift Cards
Purchase with Gift Card
When a gift card number is entered as a payment option during checkout. This is also another way that can be used to validate the value on a gift card.
Financial Services
Apply for a Financial Service Account
When an application for a new credit card account, retirement account, bank account has been submitted.
Financial Services
Money Transfer
When a money transfer from one account to another has occurred.
Search
Flight Search
Used to search for open seats on a flight.
Search
Product Search
Used to search for a specific product.
Search
Room Search
Used to search for room availability.
Search
Reservation Search
Used to search to book a sports reservation, concert, and so on.
