Perform a Site Analysis

As part of your Bot Defense planning process, F5 recommends that you perform a site analysis to identify all the endpoints you need to protect.

Step 1: Decide what flows you want to protect

First, identify the flows you want to protect. A flow is an area of functionality that you can protect with Bot Defense. For example:

Login flow
Forgot password flow
Check credit card flow
Sign up for an account flow

F5 recommends that you protect any flow that an attacker can exploit to cause harm to your business or your customers.

Step 2: Find the endpoints in each flow

After you identify the flows you want to protect, you must identify the endpoints in each flow that an attacker might try to exploit. Endpoints are URL paths that potentially expose your application to harm and that attackers are likely to try to exploit. For example:

For a login flow, to determine if a set of credentials is valid on for site, an attacker sends an HTTP request to the specific path that returns a result indicating whether the credentials are valid. The HTTP response usually includes a session cookie or authentication header.
For a money transfer flow, configure Bot Defense to protect the path that is used to perform that transaction.
For content scraping, configure Bot Defense to protect the path that returns the content that the scraper wants.

In some cases, there is more than one endpoint for a flow.

Wizards: Some flows involve a wizard where a user must complete several different forms in a defined sequence. For example, when opening a bank account or requesting an insurance quotation. For a wizard, you must make sure to protect the endpoint that prevents the attacker from causing harm. You need to identify this choke point and determine which endpoints are most critical.
Multiple pages with the same functionality: In other cases, there may be several pages in the application that perform the same function and use different endpoints. You might also have different websites with different domains that use different endpoints. You must identify and determine if you need to protect all of these endpoints.
Same data sent to multiple endpoints: In some cases, a site sends the same data for two different endpoints. In this case, contact your Technical Account Manager or F5 Support for assistance.

Record Information About Your Endpoints

When you discover an endpoint, record the following information to help you determine the steps necessary to protect the endpoint


HTTP Method	Allows you to define more specific request matchers. Requests to an endpoint are usually an HTTP POST request but can be a GET or PUT request.
XHR	Useful for determining mitigation actions. The most common mitigation action for a traditional form submit (whereby the browser refreshes the page) is a 302 with a location header redirecting to another page. For XHR requests, you might want to return a JSON body or other content that the application expects. If the JavaScript that triggered the XHR call does not receive the expected response, it can trigger an error that impacts the end user.
Path	The URL for the endpoint, including the domain. Also note if the website or mobile app uses the same URL path for more than one purpose. For example, the website might use the same path for logins, credit card purchases and booking a flight. In such cases, you must configure a matcher in a endpoint policy to determine which requests to protect by examining form body parameters.
CORS	If the request is sent using XHR and the domain of the endpoint does not match the domain of the entry point, then the request requires CORS. CORS enables the browser to send a special HTTP Options request to the server to ask whether it supports CORS and what custom HTTP headers it accepts with CORS. Make note of this as Bot Defense Advanced needs special configuration to handle CORS. (Note: CORS is needed unless the full domains match. That is, if the entry point domain is `products.example.com` and the endpoint is `purchase.example.com`, you must use CORS.)
Post Body Parameters	Note body parameters, especially the name of the username field.
Response	Determine how responses differ when a request succeeds compared to when it fails. This helps you define a mitigation action that mimics the normal behavior of the application.
Single-Page Application	A single-page application is a web site that is all one page and never refreshes. Page content updates dynamically by JavaScript and XHR calls. For single-page applications, you must allow for higher token reuse. Bot Defense normally limits token reuse to prevent replay attacks. However, you must raise the token reuse limit for an single-page application because the user stays on just that one page with just one token – due to the JavaScript loading and executing only once – and re-uses that token for many XHR requests.

Use a Web Browser to Identify Your Endpoints

Many web browsers provide tools for developers that allow you to view traffic in your browser. The following instructions are for Chrome version 126.0.6478.183, but other browsers offer similar tools.

Open the Google Chrome browser and click the More menu.

Figure: Google Chrome More menu
Click More tools > Developer tools and then click the Network tab.
Select Preserve log.

Figure: Developer Tools - Preserve log option on the Network tab
Open an HTML form page and complete the form. Make sure to use a memorable username to make the endpoint easier to find.
In the Chrome Developer tools, click Clear to remove all existing requests.

Figure: Developer Tools - Clear previous requests
In the Filter box, enter: method: POST.
In the browser window for your HTML form, submit the form.
On the Network tab, click Fetch/XHR, Doc and Other so that all three are selected. The endpoint is typically in one of these categories and is often one of the first POST requests after you submit the HTML form.

Figure: Developer Tools - POST method

You can confirm this is the endpoint because it has either query string parameters or body parameters that you typed into the HTML form. To see these parameters, click the path and scroll to the bottom of the window.
Make note of the Request URL. For example, www.example.com/bff/account/signin. This is the endpoint.
Repeat this process for each flow.

Step 3: Identify Entry Points

After you identify your endpoints, you must determine which is the entry point. The entry point is the URL of the page that a user or an attacker first encounters when they access your website or application. For example,

A login page where a username and password are required to access protected content.
A home page or landing page that users reach before visiting other pages on your site.

This is the page where you must load the Bot Defense JavaScript. Keep in mind that users reach your website in different ways, such as by following bookmarks or through Internet search results, so you may have a large number of entry points to protect.

Important: To avoid protecting more endpoints than necessary, use wildcards cautiously when entering endpoint URLs.

In most cases, the entry point is a page with an HTML form. To confirm this, look at the request to the endpoint and check for the value of the Referer header. This should be the entry point.

If the referer is not the value you expect, consider the following possible explanations:

Intermediate pages: It is possible for one page to make a request to another page, that returns a redirect. That redirect returns another redirect, and so on, before the request finally makes a post to the endpoint. To determine if there are intermediate pages and multiple redirects, use your browser tools. Start at the point of form submission and find the request to the identified endpoint. Check whether there are other requests in between. You can ignore XHR requests or requests for resources such as images, CSS or JavaScript.
XHR calls: Sometimes the page with the form makes a request to another page. That other page loads in the browser and makes an XHR call to the endpoint. In this case, the entry point is the last page that loads prior to the request to the endpoint. When there are multiple redirects, there is no specific entry point. In this case, you might need to use an interstitial page.
iFrame: When the HTML form is displayed in an iFrame, you can treat the page displayed in the iFrame as the entry point. To determine if a form is in an iFrame, use the developer tools in your browser to view and click on the form element. Check if the form element is nested in an iFrame element.
Dynamically Generated HTML: When JavaScript dynamically adds a block of HTML to a page that renders the HTML form, you can attempt to add the Bot Defense JavaScript tag to the block of HTML rather than adding it to all pages.

For assistance with this process, contact your F5 Operations team.

Next Steps