Configure Bot Defense Standard with F5 Distributed Cloud Web App and API Protection
This guide provides instructions on how to quickly configure a basic Bot Defense Standard test system using the Web App and API Protection service and how to protect a test application from automated traffic (bad bots).
Figure: Steps to Deploy a Bot Defense Test System
Note: To configure your production system with the Web App and API Protection service, see Configure Bot Defense on an HTTP Load Balancer for additional configuration information.
Note: If you use F5 BIG-IP or host your application on a content delivery network (CDN), see Configure Bot Defense (Connectors) in the Bot Defense Standard documentation for configuration instructions.
Step 1: Plan Your Bot Defense Deployment
Complete the following tasks to plan your Bot Defense deployment.
Meet the Prerequisites
To enable and configure Bot Defense, you must have the following accounts and permissions:
- A valid Distributed Cloud Console account. For information, see Create a Distributed Cloud Console Account.
- An Organization plan. To see plan information, from the Distributed Cloud Console
Home
page, clickBilling
. - The
f5xc-bot-defense-user-role
(orves-io-power-developer-role
or greater).
Choose an Application to Protect
Choose an application and endpoint that you want to protect with Bot Defense. For example, for your initial system testing, you can protect a login endpoint that users typically reach when they first visit your website.
You must have the following information:
- The application domain name.
- The login HTTP URL path.
- Which HTTP methods you want to protect. Bot defense can protect GET, POST, and PUT methods.
Next Steps
Enable the Bot Defense service in the Distributed Cloud Console and then choose the load balancer where you want to enable Bot Defense. For information, see Step 2: Configure Bot Defense.
Note: For information about how to create an HTTP load balancer, see Step 2 of the Web App Security & Performance Quick Start.
Step 2: Configure Bot Defense
To configure Bot Defense, complete the following tasks.
Enable the Bot Defense Service
To use Bot Defense, you must enable it in the Distributed Cloud Console.
-
On the Distributed Cloud Console
Home
page, clickBilling
. -
Click
Manage > Billing Plan
and scroll to theOrganization Plan
. -
Under the
Organization Plan
, clickBot Defense
. If Bot Defense is not already enabled, the Bot Defense landing page appears.Note: If Bot Defense is already enabled, the Bot Defense
Monitor
page appears. -
From the
Bot Defense
landing page, clickRequest Service
.
After you request Bot Defense, it can take up to 24 hours for F5 to enable the service.
Enable Bot Defense on an HTTP Load Balancer
- From the Distributed Cloud Console dashboard, click
Web App & API Protection
. - From the navigation menu, confirm that you are in the correct namespace.
- Click
Manage > Load Balancers > HTTP Load Balancers
. - From the
Actions
column, click the Actions (…) menu next to the load balancer where you want to configure Bot Defense and then clickManage Configuration
. - Click
Edit Configuration
and then clickBot Protection
. - In the
Bot Protection
section, from the Bot Defense drop-down menu, selectEnable Bot Defense Standard
.
Configure Protected App Endpoints
To protect web-based endpoints, such as a web page, in the Bot Protection
section, perform the following steps:
-
From the
Bot Defense Region
drop-down menu, select the region where the origin server for the application is located. -
In the
Bot Defense Policy
section, clickConfigure
. -
In the
Protected App Endpoints
section, clickConfigure
and then clickAdd item
. -
Enter the following information for each endpoint:
-
Name
: Enter the name of the endpoint. The name must follow DNS-1035 format. -
HTTP Methods
: Select the HTTP methods you want to protect on this endpoint. You can select multiple methods.ANY
: Includes all methods exceptGET(Document)
. To select all methods includingGET(Document)
, you must select each method from the list.GET(XHR/Fetch)
: Use when the protected application makes an XHTTPRequest or Fetch API call to get the content of the page. Note that GET requests are protected only if they are sent by XHTTPRequest from a page with Bot Defense JavaScript injected, not from direct navigation using the address bar or link.POST
PUT
GET(Document)
: Use to protect pages on a web site that can be accessible by GET requests without visiting the main page. When you configure an endpoint usingGET(Document)
, Bot Defense displays an interstitial page that is transparent to the user but that allows it to collect telemetry data about the requests. Note that you cannot useGET(Document)
with mobile endpoints.
-
Endpoint Label
: Select an endpoint label to allow more granular attack intent identification and reporting when Bot Defense detects automation.For a full list of available endpoint labels, see Endpoint Labels in the Bot Defense Standard documentation.
-
Domain Matcher
: Specify the domains you want to protect. Enter an exact value, a suffix value, or a regex value. -
Path
: Enter the path to the endpoint you want to protect. For example, enter /login. Enter a prefix, exact path, or regex value. -
Bot Traffic Mitigation
: Select Continue. This setting puts Bot Defense in monitoring mode and allows requests to continue to the origin. You can select Block or Redirect mitigation actions after you confirm that Bot Defense is correctly identifying automated traffic. -
Good Bot Detection Settings
: Specify whether the mitigation actions you selected above apply to both malicious automation and good bots. You can choose to flag good bots but allow them to continue to origin, or you can apply the mitigation actions to all automation. By default, mitigation actions are applied to all automation.
Important: Good bots, also called benign bots, are bots that perform specific, helpful tasks, such as search engine bots, social media crawlers and aggregator bots. Many of these bots can be useful to your business and therefore you might decide to allow them to continue to origin.
-
-
Click
Apply
. -
Click
Add Item
to add another endpoint or clickApply
when you finish adding endpoints.Figure: Configure Protected App Endpoints
Next Steps
Configure how Bot Defense injects JavaScript into your application pages.
Configure JavaScript Injection
To protect web-based endpoints, Bot Defense injects JavaScript tags. The Bot Defense JavaScript collects telemetry data about requests to your endpoint. You must configure how the Bot Defense JavaScript tags are injected into the pages that you want to protect.
In the JavaScript Insertion
section:
-
In the
JavaScript Download Path
field, enter a path to display to the browser and in the page source code in place of the actual download path for the Bot Defense JavaScript. This prevents malicious actors from determining what system you are using to protect your application.F5 recommends that you choose a URL or path that is similar to your existing JavaScript files, but that does not include “F5,” “Bot Defense” or other indications that it is used for security purposes. Enter a simple path that starts with
/
or a complete URL such ashttps://example.com/customer1.js
. The URL or path you choose cannot conflict with your existing JavaScript files. -
From the
JavaScript Insertion Settings
drop-down menu, selectInsert JavaScript in All Pages
. -
From the
JavaScript Location
drop-down list, select one of the following locations to insert the JavaScript in your application pages:After <head> tag
After </title> tag
Before <script> tag
F5 recommends that you select
After <head>
so that the Bot Defense JavaScript is executed early. This allows time for the Bot Defense JavaScript to be fetched and executed while the rest of page is rendered. If you selectAfter </title>
, be sure your application pages have the<title>
tag.Figure: Configure JavaScript Injection
-
Click
Apply
and then clickSave and Exit
to save your configuration.
Next Steps
When you finish configuring Bot Defense, test your deployment to make sure it is working as intended. See Step 3: Test Your Deployment.
Step 3: Test Your Deployment
Perform the following tests to help confirm that Bot Defense is identifying web traffic correctly.
Check JavaScript Injection
Open the application page where you injected the F5 Client JavaScript tags. Use the developer tools in your browser or view the page source to inspect the page and confirm that Bot Defense has inserted JavaScript tags with the following query string parameters:
?matcher
?cache
?async
For additional information about confirming proper JavaScript tag injection, see Test Your Bot Defense Configuration in the Bot Defense Standard documentation.
Figure: JavaScript Injection
Check for False Positives
Review the Bot Defense dashboard to see the types of traffic identified by Bot Defense and confirm that no legitimate traffic is being incorrectly identified as automation (false positives). From the Bot Defense Home
page, click Overview > Monitor
.
-
In the
Protected Apps
widget, check that your new application is listed and that the amount of traffic is appropriate. -
In the
Traffic by Type
widget, check that the level of human, benign bot, bad bot and other activity is appropriate.Figure: Bot Defense Dashboard
-
From the time-period drop-down list, select
Last 24 hours
. In theTraffic Visualized
section, check whether traffic marked as malicious increases during the day and decreases at night. If so, this might indicate human traffic.From the Bot Defense
Home
page, clickReport > Traffic Analyzer
. -
Look at the distribution of IP addresses and the countries of origin. Decide if this distribution looks like it comes from your normal user base.
-
Look at the
User Agent
column and decide if there are suspicious user agents present. You can also use this technique to identify wanted automation (benign bots), such as test tools or SEO bots. -
Click
Add Filter
. From the drop-down list, selectBot Reason
, selectIn
, selectToken Missing
and clickApply
. Review the traffic and determine if it looks legitimate.
For additional information about how to check for false positive results, see Analyze Bot Defense Results in the Bot Defense Standard documentation.
Next Steps
When you are ready to block or redirect automated traffic, see Step 4: Configure Mitigation Actions.
Step 4: Configure Mitigation Actions
When you are sure that Bot Defense is correctly identifying automated traffic, you can configure mitigation actions so that Bot Defense blocks or redirects automated traffic.
- In BIG-IP, click
iApps > Application Services > Applications
. - From the list of applications, click the application you want to configure.
- Select the
Reconfigure
tab. - In the section,
F5 XC Bot Defense Protected Endpoints Configuration
, from theAction
drop-down list, select one of the following options:Redirect
. The Bot is sent to a different page. You provide the URL where the Bot is redirected in theDefine Mitigation Actions
section.Block
. The Bot is presented with a message that it has been blocked. You provide theResponse Code
andBody
of the response message in theDefine Mitigation Actions
section.
- When you finish making configuration changes, double-check your configuration settings and click
Finished
.
Next Steps
Now that you have protected one endpoint, checked to make sure the system is functioning properly and configured a mitigation for the endpoint, try protecting additional endpoints.
For additional information about advanced configuration steps, see Configure Bot Defense on an HTTP Load Balancer in the Bot Defense Standard documentation.
On this page:
- Step 1: Plan Your Bot Defense Deployment
- Meet the Prerequisites
- Choose an Application to Protect
- Step 2: Configure Bot Defense
- Enable the Bot Defense Service
- Enable Bot Defense on an HTTP Load Balancer
- Configure Protected App Endpoints
- Configure JavaScript Injection
- Step 3: Test Your Deployment
- Check JavaScript Injection
- Check for False Positives
- Step 4: Configure Mitigation Actions