Alerts - PagerDuty

Published April 5, 2023 | Last modified November 25, 2024

Objective

This document provides instructions on how to configure alert notifications in F5® Distributed Cloud Services to route the alert messages to your PagerDuty service. For information on alerts, see Alerts.

Prerequisites

A valid Account is required.

Note: If you do not have an account, see Create an Account.

A PagerDuty account with administrative privileges to service configuration settings.

Configuration

Configuring to send alerts to the PagerDuty service requires you to first generate webhook for your PagerDuty service. After that, create alert receiver and policy in the F5® Distributed Cloud Console.

PagerDuty supports webhooks by providing service integration via API. This example shows instructions on generating the webhook configuration for a PagerDuty service.

Note: PagerDuty service integration generates an integration key which is used as webhook.

Step 1: Generate webhook configuration in PagerDuty.

Log into your enterprise PagerDuty portal, and follow the instructions in PagerDuty Webhooks page to generate the integration key for the service to which you want to post the alerts.

Step 2: Get the integration key from the webhook configuration in PagerDuty.

Navigate to your service configuration settings on PagerDuty, obtain integration key in Integrations section.

Figure: PagerDuty Service Configuration

The photo below is a sample service integration on PagerDuty.

Figure: PagerDuty Webhook Integration Key

Create Alert Receiver

Alerts can be created in different services and namespaces. Alert functionality is available in Multi-Cloud Network Connect, Multi-Cloud App Connect, Web App & API Protection, Distributed Apps, Audit Logs and Alerts ,and Shared Configuration.

This example shows alert notifications for PagerDuty setup in Shared Configuration.

Step 1: Create Alert Receiver in F5® Distributed Cloud Console.

Open F5® Distributed Cloud Console homepage, select Shared Configuration box.

Note: Homepage is role based, and your homepage may look different due to your role customization. Select All Services drop-down menu to discover all options. Customize Settings: Administration > Personal Management > My Account > Edit work domain & skills button > Advanced box > check Work Domain boxes > Save changes button.

Figure: Homepage

Note: Confirm Namespace feature is in correct namespace, drop-down selector located in upper-left corner. Not available in all services.

Select Manage in left column menu > select Alerts Management > Alert Receivers.

Note: If options are not showing available, select Show link in Advanced nav options visible in bottom left corner. If needed, select Hide to minimize options from Advanced nav options mode.

Select + Add Alert Receiver button.

Figure: Alert Receiver

Step 2: Set PagerDuty as receiver.

Enter Name, enter Labels and Description as needed.
Select PagerDuty in Receiver drop-down menu.

Figure: Alert Receiver Configuration

Step 3: Configure integration key for PagerDuty.

Select Configure link in Integration Key box.
Select Blindfold Secret bubble for Type in Secret info section.
Enter string you copied in text box obtained from applying Blindfold to webhook configuration.
Select Apply button.

Figure: Configure Secret

Note: See Blindfold your Application Secrets for information on how to encrypt your application secrets.

Step 4: Specify API for PagerDuty service, complete creating receiver.

Enter URL in PagerDuty URL box (to which you want to send the API request).
Select Save and Exit button to complete creating the alert receiver.

Create Alert Policy

Step 1: Create Alert Policy.

Select Manage in Shared Configuration service.
Select Alerts Management.
Select Alert Policies.
Select + Add Alert policy.

Figure: Create Alert Policy

Step 2: Add receiver to the policy.

Enter Name, enter Labels and Description as needed.

Note: Toggle Show Advanced Fields to show Configure Receiver Default Notification Parameters.

+ Select Item drop-down menu in Alert Receiver Configuration box.
Select receiver you created in Create Alert Receiver chapter.
Select + Add item button in Alert Receivers page.
Select your receiver if available in list.

Note: + Add item button to add multiple Alert Receivers.

Figure: Alert Policy Receiver Configuration

Step 3: Optionally, configure the notification intervals.

Enable Show Advanced Fields option in the Alert Receiver Configuration section.

Select Configure link in Receiver Default Notification Parameters section. This opens notification parameter configuration.
Enter configuration for the Notification Parameters.
- Toggle Show Advanced Fields in both sections to open configuration options.
- Enter a value in the [0-9][smhd] format in Notify Interval for a Alert. The [smhd] specifies seconds, minutes, hours, and days. An example value is 60m and default is 4h.
- Enter a value in the [0-9][smhd] format in Wait to Notify. The [smhd] specifies seconds, minutes, hours, and days. An example value is 2m and default is 30s.
- Enter a value in the [0-9][smhd] format in Notify Interval for a Group. The [smhd] specifies seconds, minutes, hours, and days. An example value is 5m and default is 1m.
Note: All fields must be in 2-digit form (60m, 60s) in Policy Rule Notification Parameters for intervals and group notifications for your policy rules to apply correctly and not error.
Select Apply to apply the notification parameters, return to the policy configuration.

Figure: Alert Policy Route Configuration

Step 4: Optionally, configure policy rules.

A minimum of one policy rule is populated by default with Any as match condition and Send as Action. You can select different match conditions, and actions from the drop-down list for the Select Alerts and Action fields respectively. You can use combination of more than one policy rule. Select Add item to add more policy rules.

Select Configure link in Policy Rules box
Select + Add Item button.
Select Alerts drop-down menu option, Any option is default.

Note: Matching Custom Criteria > Alertname Configure link > Select Matcher Type > select Apply button > select Apply button.

Select Action drop-down menu option, Send is default.
Select Apply button.
Order rules by dragging in Policy Rules page.
Select Apply button, order rules by dragging.

Note: You can also set the match conditions based on the severity, group name, and alert name.

Step 5: Complete creating policy.

Toggle Show Advanced Fields to show Configure Policy Rule Notification Parameters, select Apply button to add.
Select Save and Exit to complete creating the alert policy.

Create Active Alert Policies

Step 1: Start creating active alert policies.

In Multi-Cloud Network Connect > Select Manage.

Note: Active Alert Policies can be created and edited in Multi-Cloud Network Connect, Multi-Cloud App Connect, Web App & API Protection, and Distributed Apps services by selecting Manage > Alerts Management > + Select Active Alert Policies.

Select Alerts Management > select Active Alert Policies.
+ Select Active Alert Policies.

Figure: Active Alert Policy Configuration

Step 2: Add active alert policies to the policy.

Select item drop-down menu to select created alert policy needed.
+ Add Item button in Active Alert Policies pop-up window to add policy selection drop-down menus.

Figure: Active Alert Policy Configuration

Step 3: Complete creating the active alert policies.

Select Save and Exit button to complete creating the alert policy.

Verification and Example

You can verify the alert notifications by visiting your PagerDuty service page. The following image shows a sample of alerts sent to PagerDuty service:

Figure: Post of Alerts to PagerDuty Service

Concepts

Monitoring