Bot Defense Standard Dashboards and Reporting
F5 Distributed Cloud Bot Defense Standard includes multiple ways to view detailed information about the traffic that Bot Defense analyzes. Automated Threat Summaries deliver insights to you automatically each month, while a variety of dashboards provide you with instant, detailed access to analysis of your data.
Note: Bot Defense dashboard and report widgets display the message, "'Undefined' Data Discovered" when requests reach a protected endpoint that does not have an assigned endpoint label. Make sure you use the latest connector configuration and that you apply an endpoint label to all endpoints.
Note: Bot Defense dashboard and report widgets display the message, "Data Gap Discovered" when there is no data to display. Make sure Bot Defense is configured correctly and that you select a time period for the dashboard or report for which you have collected data.
Automated Threat Summaries
Automated Threat Summaries are emailed reports that summarize your monthly network traffic and Bot Defense activity. Automated Threat Summaries include information about the percentage of your traffic that is automated, the numbers of good and bad bots detected, the number of human visitors to your protected applications and endpoints, and a breakdown of your overall web traffic that is analyzed by Bot Defense.
Figure: Automated Threat Summaries
To enable Automated Threat Summaries:
- From the Distributed Cloud Console Home page, click Bot Defense.
- From the Bot Defense navigation menu, click Manage > Reports > Reports Management and then click Add Report.
- Enter a Name for the report and then, from the Report Type drop-down menu, select Bot Defense.
- Click Add Item and then choose the User Group to which you want to send the report. If you need to create a new user group to receive the report, see Users.
- Click Save and Exit.
Load Balancer Bot Defense Dashboard
After you enable Bot Defense, the Load Balancer Bot Defense dashboard provides a snapshot of bot and human activity in your web traffic on your HTTP load balancers.
To view the Web App & API Protection Bot Defense dashboard:
- From the Web App & API Protection navigation menu, click Overview > Dashboards > Security Dashboard.
- In the Load Balancer widget, from the list of load balancers, click the name of the load balancer you want to view and then click Bot Defense.
Figure: Load Balancer Bot Defense Dashboard
The Load Balancer Bot Defense dashboard includes the following information:
-
Traffic Types: The total number of transactions in the selected time window categorized by the following traffic types:
- Human: Traffic from human users and therefore not automated.
- Benign Bot: Bots that are harmless or non-malicious, such as SEO bots or test tools.
- Bad Bot: Bots that are malicious, often causing harm to legitimate users, businesses, or organizations.
- Telemetry Client: Traffic that comes from loading the Bot Defense JavaScript in application pages or from configuration requests (fetches) from mobile applications integrated with the F5 Mobile SDK.
- Challenged: Traffic from challenges served to clients on protected GET transactions to determine if the clients are human or automated.
Hover over the donut graph for percentage information for each type of traffic.
-
Traffic Overview: The transactions per minute of human and bot traffic for the selected time period. Hover over the area chart or bar graph for the number of each type of event at a specific time.
-
Top Bad Bots: The five bots that made the most malicious requests in the selected time period, including the source IP, ASN and number of individual events. You can view each bot's Source IP, AS Organization, and User Agent.
-
Top Benign Bots: The benign bots that appear most often in your web traffic during the selected time period, including the name of the bot, type, and the number of individual events and percentage of events attributed to a specific bot.
-
Top Visited Endpoints: The five endpoints being attacked most frequently by bad bots, including the host name, endpoint path, and number of malicious requests in the selected time period.
You can specify the time period displayed in the dashboard and add the following additional filters to customize your view:
- AS Organization
- Bot Reason
- Country
- Host
- IP Address
- Path
- Traffic Channel
- Traffic Type
- User Agent
To view additional Bot Defense data for this specific load balancer, such as an overview of protected apps, human traffic by geographic region, an overview of specific bad bot events impacting your applications, and information about the intent of an attack, click View in Bot Defense. Then, from the navigation menu, click Overview > Monitor to view the Bot Defense Dashboard.
Figure: View in Bot Defense
Bot Defense Dashboard
The Bot Defense dashboard is a snapshot of all human and bot activity in the web traffic analyzed by Bot Defense for a specified time period.
To view the Bot Defense dashboard, in Bot Defense, from the navigation menu, click Overview > Monitor.
Figure: Bot Defense Dashboard
By default, the dashboard displays data from the last six hours. You can customize the time period and the region that displays. You can also save and share filters that you apply to the dashboard.
The Bot Defense dashboard includes the following information:
-
Traffic By Type: The total number of transactions in the selected time window categorized by the following traffic types:
- Human: Traffic from human users and therefore not automated.
- Benign Bot: Bots that are harmless or non-malicious, such as SEO bots or test tools.
- Bad Bot: Bots that are malicious, often causing harm to legitimate users, businesses, or organizations.
- Telemetry Client: Traffic that comes from loading the Bot Defense JavaScript in application pages or from configuration requests (fetches) from mobile applications integrated with the F5 Mobile SDK.
- Challenged: Traffic from challenges served to clients on protected GET transactions to determine if the clients are human or automated.
Hover over the donut graph for percentage information for each type of traffic.
-
Traffic by Category: The traffic detected for each endpoint category for the selected time-period. Hover over the donut chart to see percentage information for each category.
-
Traffic by Action: Total number of automated traffic events that reach your protected endpoints and how many times each mitigation action is applied to these events. Hover over the donut chart to see percentage information.
-
Protected Apps: A breakdown of bad bot traffic by defended application. Hover over the donut chart to see percentage information.
Use the dashboard controls to view additional data.
Figure: Bot Defense Dashboard Tabs
Traffic Type
- Traffic Visualized: Total number of events impacting the applications you protect with Bot Defense over the specified time range as an area chart, bar graph or line graph. Hover over the charts for information about a specific time.
- Top Benign Bots Found: Top benign bots that are visiting your protected applications, including the name and type of bot and total events for each bot.
- Bot API Latency Across Top Protected Apps: Information about traffic latency measured between the top protected application and origin server, including average time, application name, latency per application, number of transactions and trend information.
Endpoint Category
- Traffic Category Visualized: Shows the traffic detected per endpoint category for the time-period selected. Hover over the chart to see totals and percentages for a specific time.
- Top Endpoint Labels: Shows the top 10 endpoint labels on which traffic was detected, and the percentage of traffic detected on the endpoint label in relation to all traffic detected on your system.
- Top Visited Endpoints: Shows the five endpoints that are being attacked most frequently by bad bots. The table includes the host name, endpoint path, and number of malicious requests in the selected time period.
Bad Bot
- Actions Breakdown: A time series chart for mitigated and flagged bad bot events across all protected endpoints displayed as an area chart, bar chart or line graph.
- Top Actions: The number of events to which each mitigation action was applied.
- Top Sources of Bad Bot Traffic: Bad bot traffic analysis by source type, including number of events, IP address and geolocation. Display by Source IP, ASN or User Agent.
- Threat Type: An overview of bad bot traffic by threat type over a selected time period. View as an area or line chart. Hover over the charts to see the number and percentage of each type of attack for a specific time.
- Top Threat Types: An overview of bad bot traffic by threat type listed in descending order.
- Top 5 Sources of Threat Types: A list of the top five sources of each bad bot attack type. Information includes source IP address, autonomous system number (ASN), geolocation, number of events from each source and the percentage of the total for that attack type attributed to the source.
Geolocation
- Traffic from Humans: A map that shows where your human traffic comes from. Hover over each country for additional information about your traffic.
When you filter the Bot Defense dashboard by a single load balancer, click View in Web App & API Protection to view the Load Balancer Bot Defense dashboard for that load balancer.
Figure: View in Web App & API Protection
Traffic Analyzer
The Traffic Analyzer report provides detailed insight into human, benign bot (good bot), bad bot and other traffic on the HTTP load balancer. View traffic as a bar chart or an area chart. Hover over the charts to view traffic totals for specific times.
To view the Traffic Analyzer, in Bot Defense, from the navigation menu, click Report > Traffic Analyzer.
Figure: Traffic Analyzer
In addition to the chart showing transactions per minute for a specified time window, you can also view details about every HTTP request sent through Bot Defense. By default, the report displays data from the last six hours. You can customize the time period and the region that displays. You can also save and share filters that you apply to the report.
Click on a date/time in the Time column to display the Transaction Detail panel. The panel contains the following information:
Name | Description |
---|---|
Timestamp and ID Fields | |
Time | The date and time of the request. |
User Name | The user name associated with the request. |
Client Token | Identifies a particular Bot Defense client session. For web clients, this value is shared by all protected requests made by the user from a webpage which executed the Bot Defense JavaScript. For mobile clients, this value is shared by all protected requests made by an app with an integrated F5 Distributed Cloud Mobile SDK within a four-hour window. |
IP Fields | |
IP Address | The IP address where the request originated. |
ASN | The autonomous system number of the IP address where the request originated. |
AS Organization | Name of the organization associated with the ASN. |
Country | The country of origin. |
Method and URL Fields | |
Method | The type of HTTP request. |
Host | The host where the request originated. |
Path | The path to the flow specified in the request. |
URL | The URL of the endpoint specified in the request. |
Referer | The web page where the request originated. |
Traffic Channel | The type of traffic (web or mobile). |
Attack and Inference Fields | |
Is Attack | If Bot Defense determined this to be an attack (True or False) |
Bot Reason | The reason that Bot Defense determined the request was an automated attack. |
Traffic Type | Whether the traffic was bot or human. |
Threat Type | Type of attack based on OWASP attack types, for example, scraping, carding, denial of inventory, credential stuffing and account creation. |
SDK Version | For mobile traffic, the version of the SDK used with the targeted application. |
Action Taken | The mitigation action taken by Bot Defense (Continue, Block, Redirect). |
Cookie Age | Time in seconds/ms indicating how long ago the cookie was set. |
Bot Cookie | Contains a unique id with expiration set by Bot Defense that enables F5 to connect multiple HTTP requests that share the same cookie value. Useful for detecting false positives and provides valuable data for threat intelligence. |
User Agent Field | |
User Agent | Information about the Client OS and the browser version. |
User Agent OS Family | The operating system on the user agent where the request originated. |
User Agent Family | The type of browser where the request originated. |
Fingerprints | |
Browser Fingerprint | A unique string based on characteristics of the browser that sent the request. |
User Fingerprint | Captures the hash value for each user's behavior, such as mouse movement, mouse click, and keyboard events. |
Header Fingerprint | Hash value, determined from various factors such as browser, plugins, headers and so on. |
DeviceID | A unique and persistent value which identifies a particular browser or a mobile device. |
Data Labels & Cluster | |
Flow | The endpoint label assigned to the flow. |
Agent | Indicates if the client is a legitimate user, illegitimate (failed to provide valid telemetry), on trusted allowlist (for example, your internal users), or on an untrusted allowlist (for example, third parties that you allow to access the application without valid telemetry). |
Application Name | Bot Defense Advanced only. |
Protected Application | The name you assigned to the load balancer, CDN or other application that you added to Bot Defense when you configured your protected endpoints. |
Headers and Response Fields | |
Request Headers | List of request headers. |
Response Headers | List of response headers. |
Response Code | HTTP response code for the transaction served from Bot Defense. |
Server Response Code | HTTP response code for transaction served from upstream. |
Transaction Result | Captures information based on success/fail criteria for protected endpoints. |
Latency Fields | |
Inference Response Latency | The time Bot Defense takes to process transactions and provide inference. |
Origin Latency | The time it takes the origin server to respond. |
Total Latency | Total of inference response latency and origin latency. |
Figure: Transaction Detail
Bad Bot Report
The Bad Bot Report provides information about malicious automation in your web traffic.
To view the Bad Bot Report, in Bot Defense, from the navigation menu, click Report > Bad Bot Report.
Figure: Bad Bot Report
By default, the report displays data from the last six hours. You can customize the time period and the region that displays. You can also save and share filters that you apply to the dashboard.
The Bad Bot Report includes the following information:
- Traffic Metrics: An overview of the different characteristics of the bad bot traffic that Bot Defense has detected across your protected applications and endpoints.
- Actions Taken: The number of events to which each mitigation action was applied, including percentage and trend information.
- Threat Type: An overview of bad bot threat traffic based on key OWASP attack types to help you see how attackers are targeting your endpoints. Also provides up/down trend information to show you the threat types that are peaking during the selected time range.
- Events per Application: The number of bad bot events grouped by application. Hover over the chart for totals and percentages for each application.
- Bad Bot Reasons: The top reasons why Bot Defense associated an event with a bad bot, including the number of events for each reason code and percentage, to help you understand how bad bots are attacking your endpoints.
- Bad Bot Traffic: An overview of bad bot traffic by threat type over a period of time.
- Bad Bot Events per Application: An overview of bad bot events that occurred on each protected application. View data as an area chart, bar chart or line graph. Hover over the charts for event totals and percentages on each application at a specific time.
- Bad Bot Traffic Breakdown: An overview of bad bot events impacting your applications across the specified time range. View data as an area chart, bar chart or line graph. Hover over the charts for totals and percentages for a specific time. You can also view bad bot events organized by operating system (OS), browser, user agent (UA), autonomous system number (ASN) or IP address. Explore and investigate detailed event data, including threat types and reason code information. Expand each row to reveal additional total and percentage information. Use Search to find specific events.
- Endpoints with Bad Bot Traffic: An overview of bad bots attacking your protected endpoints.
Protection Coverage Report
The Protection Coverage Report provides information about your protected applications and endpoints and the traffic accessing them.
To view the protection coverage report, in Bot Defense, from the navigation menu, click Report > Protection Coverage Report.
Figure: Protection Coverage Report
By default, the report displays data from the last six hours. You can customize the time period and the region that displays. You can also save and share filters that you apply to the dashboard.
The Protection Coverage Report includes the following information:
- Endpoint Summary: A summary of your protected applications and endpoints during your selected time period.
- Endpoints Category Breakdown: An overview of traffic passing through your secured endpoints. Use the drop-down menu to display data for specific flow categories and associated flow labels. Hover over the horizontal bars for information about traffic sources, action taken on the traffic, and the traffic channel (web, mobile and so on).
- Protected Traffic Flow: A visual representation of all traffic visiting your protected endpoints.
- All Protected Endpoints: Information about all of your endpoints that are protected by Bot Defense, including domain, associated application, number of events that took place on the endpoint, and percentage of your total traffic that passed through the endpoint.
Peer Comparison Dashboard
The Peer Comparison dashboard helps you understand how your bot protection compares to similar peer companies. Data includes comparisons of total traffic breakdowns, top benign bots, top reason codes (how Bot Defense determined the automation was malicious), and top threat types. For example, the Threat Types widget shows you your top bad bot traffic compared to an average of the top bad bot traffic at peer companies.
Use peer comparison data to help you identify areas where you might want to tune your configuration to help ensure the highest protection from current and future attacks.
To view peer comparison data, from the Bot Defense navigation panel, click Report > Peer Comparison.
Figure: Peer Comparison Dashboard
You can select the time period, sort the data in descending order based on either your threat information or your peer threat information, and view absolute values or view data as a percentage of traffic.
The Peer Comparison dashboard includes the following information:
- Total Traffic Breakdown: The total number of events impacting the applications you protect with Bot Defense compared to the average number of events impacting peer companies. Hover over the bar chart for exact numbers and percentages of each traffic type.
- Top Benign Bots: The top benign bots (good bots) in your web traffic during the selected time period compared to the average number of those benign bots at peer companies, including the name of each bot and the number of individual events attributed to a specific bot. Hover over the bar chart for exact numbers and percentages of each type of bot.
- Top Bad Bot Reasons: A list of the top bot reason codes indicating how an attack was executed compared to the average of those codes at peer companies. Hover over the bar chart for exact numbers and percentages of each code.
- Top Threat Types: An overview of the different types of bad bot traffic impacting your applications compared to the average number of those types of bad bots impacting peer companies. Hover over the bar chart for exact numbers and percentages of each threat type.
Note: Peer comparison information is not available for MSP and FedRAMP customers or for customers with private instances, and the Peer Comparison dashboard does not appear in the Bot Defense navigation menu.
Consumption Report
The Consumption Report lets you see how many transactions Bot Defense has processed for the past year.
To view the Consumption Report, in Bot Defense, from the navigation menu, click Report > Consumption Report.
Figure: Consumption Report
This report displays the following information by month or by quarter:
- Traffic: All the traffic on the customer’s web applications that are monitored by Bot Defense.
- Initial Config: Traffic that is generated by the Bot Defense JavaScript that is embedded on the end-user’s mobile app.
- JavaScript: Traffic that is generated by the Bot Defense JavaScript that is embedded on the end-user’s web browser.
The report displays data for the past year, starting from a year previous to the current month until the previous month. For example, if the current month is December 2022, data is presented from December 2021-November 2022. Hover over the chart to see information for a given month or quarter.
The following summary data for the previous 24 months is displayed at the top of the report:
- Highest: The highest number of transactions monitored in a given month in the past year, compared with the highest number of transactions monitored in a given month in the previous year.
- Lowest: The lowest number of transactions monitored in a given month in the past year, compared with the lowest number of transactions monitored in a given month in the previous year.
- Average: The monthly average number of transactions monitored in the past year, compared with the monthly average number of transactions monitored in the previous year.
You can display data as either an area chart or bar chart. Hover over the charts to see data from specific months.
Use the drop-down menu to switch between Monthly and Quarterly views.
Figure: Quarterly or Monthly View Controls
Save and Manage Dashboard Filters
The Bot Defense Dashboard (Monitor) and Traffic Analyzer report allow you to quickly and easily save and view past dashboard filter combinations. You can share your saved filters with other users and also view filters saved by other users.
Filters are specific to the report where they are created. For example, a filter created and saved for the Traffic Analyzer report is only available for that report.
Note: You cannot save filters in Bot Defense enabled in Web Application & API Protection.
Save a Dashboard Filter
-
Use the
Add Filter
option to filter the data displayed by your report. -
After you set a new filter that you want to save, click the
Save Filters
icon.Figure: Save filters icon
-
Enter a
Name
andDescription
for the filter and then clickSave and Exit
.Figure: Name and Description fields
The newly saved filter is available from the
Saved Filters
list to view and share with other administrators.Figure: Saved filters
View Saved Filters
To view a previously saved filter, click Saved Filters
and then select a filter from the list.
Figure: View saved filters
Note: Filters with custom date ranges older than 30 days expire and cannot be used.
You can also view filters saved by other users.
-
On the dashboard or report that you want to view, click
Saved Filters
. -
Click
View More Details
. -
Click
Others
and then click a filter from the list to view the filter.Figure: View other users' filters
Share a Saved Filter
- From the Bot Defense navigation panel, click
Manage > Saved Filters
. - From the
Saved Filters
list, click the name of the filter you want to share. - Click
Copy URL
and provide the URL another administrator to enable them to view your filter.