Configure Streaming of Logs

Objective

This guide provides instructions on how to enable streaming of all system and application logs to an external log collection or monitoring system in the F5® Distributed Cloud Services. This is so that you can analyze events and measure application performance while establishing correlation with respect to other systems of your organization.

Note: Streaming of logs is supported only for the following types of logs:

  • Layer 7 request logs
  • Network logs if enabled through network policy

Using the instructions provided in this guide, you can configure log streaming through fleet and enable the streaming of logs for the sites that are part of the fleet.


Prerequisites

Note: If you do not have an account, see Create an Account.

  • One or more sites.

Note: See Site Management for information on site creation.

  • An external log collection or monitoring system reachable from the Volterra site.

Note: Ensure that the hostname is resolved and reachable from your F5 site.


Enable Streaming of Logs

The example shown in this guide sets up a syslog server in a docker container, and creates a log receiver object in the F5® Distributed Cloud Console for the syslog server. This example shown creates log receiver as part of the F5 fleet configuration. However, you can also create a log receiver object individually, and later can add it to a fleet.

Note: Streaming of logs is also supported for Elastic Search and Splunk.

Perform the following in the F5® Distributed Cloud Console:

Step 1: Start external log collection server.

This example starts a syslog server in docker container.

  • Start docker container for syslog and expose ports 514 and 601 for UDP and TCP connections respectively. F5® Distributed Cloud Console uses these ports by default for streaming logs when log streaming is enabled.
          
docker run -it --rm -p 514:514/udp -p 601:601 --name syslog-ng balabit/syslog-ng:latest

        

Note: You can configure a log collection system of your choice. However, ensure that the host is reachable from your site from which logs are intended to be streamed.

  • Optionally, log into the container start displaying tail logs.
          
docker exec -it syslog-ng tail -f /var/log/messages

        
  • Ensure that the syslog host is resolved from the site. This example adds an entry in the /etc/hosts file on your site with the hostname resolution.
          192.168.1.33 syslog.mydomain

        
Step 2: Start creating fleet.

Features can be viewed, and managed in multiple services.

This example shows Logs setup in Multi-Cloud Network Connect.

  • Open F5® Distributed Cloud Console > select Multi-Cloud Network Connect box.

Note: Homepage is role based, and your homepage may look different due to your role customization. Select All Services drop-down menu to discover all options. Customize Settings: Administration > Personal Management > My Account > Edit work domain & skills button > Advanced box > check Work Domain boxes > Save changes button.

NEW HOME PAGE C
Figure: Homepage

Note: Confirm Namespace feature is in correct namespace, drop-down selector located in upper-left corner. Not available in all services.

  • Select Managed in left-menu > select Site Management.

Note: If options are not showing available, select Show link in Advanced nav options visible in bottom left corner. If needed, select Hide to minimize options from Advanced nav options mode.

  • Select Fleets option in pop-up window.

  • Select + Add Fleet button.

STREAMING FLEETS 1 2
Figure: Fleets
  • Enter a name for your fleet and enter a label string for the Fleet Label Value field.

Note: The fleet label field is not available for editing an existing fleet.

  • Go to Advanced Configuration and enable the Show Advanced Fields option.

  • Select Enable Logs Streaming option for the Logs Streaming field.

  • Select on the Enable Logs Streaming drop-down menu.

  • Select Create new log receiver from the displayed options in the drop-down list. This starts a log receiver creation form.

Note: The drop-down list displays log receiver objects if those are already created. You can also select an existing log receiver. This example shows creating a new receiver.

STREAMING LOGS1 8
Figure: Enabling Log Streaming in Fleet

Note: For detailed instructions on fleet configuration, see Create Fleet guide.

Step 3: Create a log receiver and apply it to fleet.
  • Enter A Name for your log receiver.

  • Enter Fully Qualified Domain Name (FQDN) of your external log collection host in the Server name field.

STREAMING LOGSRECIEVER1 10
Figure: Log Receiver Creation

Note: The default port numbers for UDP and TCP are 541 and 601 respectively. System streams logs using these ports by default.

  • Select Continue button to create the log receiver, apply it to fleet. Return to fleet configuration.
Step 4: Complete fleet creation.
  • Toggle Show Advanced Fields in Enable Default Fleet Config Download section.

  • Check Enable Default Fleet Config Download box located at the bottom of the Fleet pop-up window.

  • Select Save & Exit button in the fleet configuration to apply the log receiver and create or update the fleet. The sites that are part of this fleet start streaming logs to the external syslog server specified in the log receiver configuration.

Note: Apply the fleet label to your sites from which you want to stream logs to the syslog server. For instructions on adding sites to fleet, see Create Fleet guide.

  • Verify that the logs are streamed by logging into the docker container as shown in Step 1.

Concepts