Continuous Delivery and Verification
Continuous Delivery
F5® Distributed Cloud App Stack integrates with your existing CI/CD pipelines to deploy applications across the fleet, ensuring minimal operational disruption. App Stack Virtual Kubernetes (vK8s) provides a single endpoint to the CI/CD tool, representing your fleet of sites. You can use standard deployment manifest files with annotations to select the sites on which your application needs to be deployed.

Figure: App Stack Continuous Delivery
The workflow to integrate vK8s with CI/CD pipelines:
- You create a vK8s object in a specific namespace. App Stack provides a kubeconfig file that represents the entire fleet of CE sites in that namespace.
- You then create segments of the fleet using virtual sites. For example, you segment the fleet into three virtual sites: dev, test, and production.
- Next, you link the kubeconfig file to the CI/CD tool. You might choose to have different CI/CD pipelines for dev, test, and production. You link the same kubeconfig file to each pipeline and leverages annotations in the deployment manifest file to specify the virtual sites (dev/test/production) on which to deploy applications. This scenario can be visualized in the diagram below.

Figure: CI/CD Using vK8s
App Stack provides complete flexibility for you to manage your applications and deployments. For example, you might choose to have three different namespaces for dev, test and prod, respectively. In this scenario, you will have a vK8s and Kubeconfig file per namespace representing the three specific segments of their fleet: dev, test, and production. In this scenario, you can link respective kubeconfig files with each of their specific pipelines for dev, test, and production. This can be visualized as shown in the diagram below.

Figure: CI/CD with vK8s and Fleet
Azure DevOps Integration
The next diagram shows an example of the App Stack kubeconfig file linked to the Azure DevOps CI/CD pipeline. The kubeconfig file was provided in the KubeConfig field as part of the kubectl apply job in the Azure DevOps CI/CD pipeline. Information about the Kubernetes cluster, such as cluster name, namespace, and the user is derived from the Kubeconfig file and shown in the Cluster Context field.

Figure: Azure CI/CD Example
Note: Ensure that you remove the certificate-authority-data entry from the Kubeconfig file.
Continuous Verification
vK8s is not only used for configuration, but it is also used to aggregate the health across distributed sites in a fleet. This is a powerful tool compared to other solutions where operators would have to write tooling to get status from each and every site one by one and aggregate the information. vK8s aggregates the health status across all sites, in a single pane of glass, reducing operational complexity for your operations team. Health status collected includes application deployment status, site health, and application health.
Continuous Verification for App Deployments and Site Health
The aggregated information on site health shows the distribution of CPU and memory load across the sites. This information is useful to IT or site administrators to identify which sites are being heavily utilized and in danger of running out of resources. Site administrators can set up alerts to be notified when the resource consumption exceeds thresholds.
You can use continuous verification to view the status of deployment, such as how many PODs were deployed, are in-progress, or have failed. Once deployed, you can also view the health status of the pod and whether it is running, waiting for an image, out of memory, and much more.

Figure: Continuous Verification of App and Health
Continuous Verification of App Health
The complexity of troubleshooting rises exponentially when debugging failures in a thousand instances of the application across a thousand sites. Traditionally, you would have to first determine which site and which application’s pod is failing. Next, you have to log onto the device, and hope that debugging was turned on and debug logs are available to determine the root cause. And you would have to compare the logs across healthy and unhealthy pods to determine what went wrong. This manual workflow does not scale beyond a few sites.
App Stack service offers the ability to continuously analyze your application logs to identify differences/anomalies between normal and misbehaving instances of the application. Once anomalies are spotted, App Stack offers the ability to alert on the anomalies so that you can get early notifications when the first deviation from normal occurs, reducing mean time to resolve issues. Continuous analysis of application logs is performed using supervised AI learning models that ingest logs from all the sites where the application is deployed.