Node Software Changelogs

Objective

This document provides the node software release versions and associated changelog.

Note: Any release version that is more than 6 months old is not supported. It is recommended to upgrade the software to the latest release.

Node Software Releases

crt-20240329-2727

Software Version: crt-20240329-2727

Release Notes

Description: Due to internal logging mechanism issue, traffic propagation is slow to reach CE Site.

Symptoms: Slowdown in configuration propagation during heavy load.

Conditions: System is trying to auto-mitigate L7 DDoS attack by blocking thousands of IP addresses.

Fix: Internal logging will be bypassed for relevant configuration object, thus unblocking configuration propagation.

---

crt-20240326-2726

Software Version: crt-20240326-2726

Release Notes

Node software is updated as part of March 26, 2024 SaaS Release

---

crt-20240318-2652

Software Version: crt-20240318-2652

Release Notes

Description : Managed k8s (PK8s) has an insecure private registry configuration. But those are updated only to docker configuration, causing nodes with cri-o unable to pull images from insecure private registry. And updating netapp version for adding missing permission on CRD.

Symptoms: TLS certificate error when pulling image from insecure private registry in PK8s.

Conditions: Site enabled with PK8s trying to pull image from insecure private registry.

Fix: Insecure registries from managed k8s configuration is updated to registry configuration in Site nodes.

---

crt-20240220-2651

Software Version: crt-20240220-2651

Release Notes

Description: When Site is created with specific Secure Mesh Site configuration, vhost0 cannot receive IP and lost connectivity.

Symptoms: During pod installation, Site lost connectivity after argo pod started.

Conditions: This issue is observed when Site is created with Secure Mesh Site configuration, and Secure Mesh Site does not have static IP for Ethernet interface of vhost0.

Fix: For this case, IP is chosen from existing Ethernet interface on that node.

---

crt-20240213-2648

Software Version: crt-20240213-2648

Release Notes

Description: Fixed an issue with local UI authentication.

Symptoms: Site local UI authentication does not work properly.

Conditions: Fixed an issue with local UI authentication.

Fix: The software version has fixed local UI authentication.

---

crt-20240123-2647

Software Version: crt-20240123-2647

Release Notes

Description: This fixes the tunnel flapping observed in CEs due to a warning being treated as error. Certificate validation fails in IKE (as OCSP/CRLs are not configured) and this was being treated as error (with the recent change in IKE) and that resulted in Vega flapping the tunnel.

Symptoms: IPSec tunnel is continuously flapping.

Conditions: NA

Fix: This hotfix suppresses the new errors reported from IKE to vega. This will prevent vega from flapping the tunnel.

---

crt-20240120-2644

Software Version: crt-20240120-2644

Release Notes

Description: ENVOY Pod crashed during JavaScript invocation from Layer 7 ACL.

Symptoms: Incoming request have NO User-Agent field and it leads to crash.

Conditions: Layer 7 ACL invokes Javascript challenge to safeguard from malicious activity. Javascript challenge relies on user-agent to handle it. Since user-agent is missing in the request, the system was unable to handle and leads to crash.

Fix: Fixed the Envoy Pod crash. After the fix, traffic disruption will not occur.

---

crt-20240117-2643

Software Version: crt-20240117-2643

Release Notes

Description: On the crt-20240117-2642 release, a virtual host route validation is added which may cause specific route to be incorrectly added.

Symptoms: Virtual host route configuration could not get deployed and load balancer may show 404 error.

Conditions: When virtual host configuration change or update happens.

Fix: This release reverted the route validation.

---

crt-20240117-2642

Software Version: crt-20240117-2642

Release Notes

Node software is updated as part of January 16, 2024 SaaS Release.

---

crt-20231223-2640

Software Version: crt-20231223-2640

Release Notes

Description: Pod is not evicted during node drain when software version upgrade is happening.

Symptoms: Pod is not evicted during node drain.

Conditions: During software upgrade and node drain option enabled.

Fix: Fixed node drain so that it is able to evict pod.

---

crt-20231223-2639

Software Version: crt-20231223-2639

Release Notes

Description: The following issues are fixed:

• Activate bond members when storage interfaces are configured with bonding
• When bond of network interface is configured, sometimes DNS configuration is not correctly applied

Symptoms: The following symptoms are observed:

• when bond is configured for storage interfaces, bond members are not activated until node is rebooted
• When bond of network interface is configured, when vhost0 interface is created, DNS configuration can become empty

Conditions: NA

Fix: VPM is updated to resolve the issues.

---

crt-20231223-2638

Software Version: crt-20231223-2638

Release Notes

Description: This version fixes the following issues:

• When multiple storage interfaces with static route are configured on RHEL9 Site, static route is not applied correctly
• When bond interface is used with RHEL9 Site, in some cases, the /etc/resolv.conf becomes empty
• When HTTP_PROXY is configured on the Site, software update to December 12, 2023 release version fails
• PVC creation does not work correctly on RHEL9 Site for HPE CSI

Symptoms: NA

Conditions: NA

Fix: VPM and CSI images are updated to resolve the issues.

---

crt-20231218-2637

Software Version: crt-20231218-2637

Release Notes

Description: Fixed Argo crash while fetching NAT IP.

Symptoms: Argo daemon used to crash while configuration for physical interface was getting updated, resulting in pod restart.

Conditions: Upon configuration change of a physical interface, Argo rebuilds the NAT IP list which would be used for flow setup. As part of change handling NAT IP update was not atomic, and in race case, forwarding threads could see an inconsistent NAT IP list which could be empty, causing segmentation fault.

Fix: NAT IP list update is made atomic by using a covering structure which holds both IP list and count of IP, making update consistent across forwarding threads.

---

crt-20231212-2636

Software Version: crt-20231212-2636

Release Notes

Description: Fixed docker service start issue after Site upgraded to the crt-20231212-2635 version.

Symptoms: When node is installed using 7.2009.27 ISO / OVA or 7.2009.45 OVA, and after it is upgraded to crt-20231212-2635, docker does not start.

Condition: Same as symptoms.

Fix: VPM is updated to resolve the issue.

---

crt-20231212-2635

Software Version: crt-20231212-2635

Release Notes

Node software is updated as part of December 12, 2023 SaaS Release.

---

crt-20231106-2588

Software Version: crt-20231106-2588

Release Notes

Delivered fix for following issue related to RHEL9 and CRI-O:

Symptoms: When static IP address is configured with OVF configuration, RHEL9 OVA does not correctly assign the static IP address. Conditions: This issue is seen when RHEL9 OVA is used with static IP configuration. Fix: Issue is fixed by updating VPM.

---

crt-20231106-2587

Software Version: crt-20231106-2587

Release Notes

Node software is updated as part of SaaS release performed on November 07, 2023.

---

crt-20231017-2542

Software Version: crt-20231017-2542

Release Notes

Updated attack signatures to mitigate Juniper Junos® OS and Confluence vulnerabilities CVE-2023-36845, CVE-2023-22515, and CVE-2023-22515.

---

crt-20231010-2541

Software Version: crt-20231010-2541

Release Notes

Delivered fix for an HTTP/2 vulnerability announced as CVE-2023-44487.

---

crt-20231003-2540

Software Version: crt-20231003-2540

Release Notes

Corrected issue caused by change of signature of logging of SSH login. The fix enables triggering of alerts with the new signature as well.

---

crt-20230912-2539

Software Version: crt-20230912-2539

Release Notes

Corrected issue caused by change to Geo IP data vendors. Use ISO country code GB instead of UK to fix previously configured service policies.

---

crt-20230910-2538

Software Version: crt-20230910-2538

Release Notes

Node software is updated as part of SaaS release performed on September 12, 2023.

---

crt-20230814-2477

Software Version: crt-20230814-2477

Release Notes

Fixed memory leak issue in the VER module when handling DNS requests in Site.

---

crt-20230811-2476

Software Version: crt-20230811-2476

Release Notes

The following issues are fixed:

• Vega CPU utilization was high due to redundant processing.
• Health check status was not propagated from Envoy to Vega correctly.

---

crt-20230807-2475

Software Version: crt-20230807-2475

Release Notes

Node software is updated as part of August 08 SaaS release.

---

crt-20230727-2449

Software Version: crt-20230727-2449

Release Notes

OpenVPN had an issue with resolving the proxy server IP from domain name due to missing libraries in the new container image. This update fixes the issue. This applies only to CE Site deployment.

---

crt-20230719-2448

Software Version: crt-20230719-2448

Release Notes

Fast ACL configuration in combination with specific load balancer configuration caused service crash. This crash is fixed now in this hotfix.

---

crt-20230709-2447

Software Version: crt-20230709-2447

Release Notes:

Node software is updated as part of the July 11, 2023 SaaS upgrade.

---

crt-20230628-2422

Software Version: crt-20230628-2422

Release Notes

ARP requests were erroneously sent as priority tagged packets, resulting in ARP resolution issue in some environments. This is resolved by ensuring that zero VLAN tagging is not performed.

---

crt-20230609-2421

Software Version: crt-20230609-2421

Release Notes

Change in configured route handling of load balancer led to issues when multiple routes pointing to weighted cluster are configured in the load balancer. This version resolved that issue.

---

crt-20230607-2420

Software Version: crt-20230607-2420

Release Notes

Node software is updated as part of the June 06 release upgrade.

---

crt-20230526-2380

Software Version: crt-20230526-2380

Release Notes

If an incoming HTTP request comes with x-request-id, same will be retained. Otherwise, the old behaviour of generating new uuid to use in x-request-id will be triggered.

---

crt-20230511-2379

Software Version: crt-20230511-2379

Release Notes

ALPN negotiation is disabled for HTTP Health check connections. HTTP health checks to origin server can fail if ALPN negotiated protocol and configured protocol in HealthCheck object are different.

---

crt-20230419-2365

Software Version: crt-20230419-2365

Release Notes

Flow update may get triggered in different threads and a race condition was resulting in crash. This is fixed with this release.

---

crt-20230418-2364

Software Version: crt-20230418-2364

Release Notes

The kubelet may fail to bring up a pod running with static manifest, which causes some expected K8s pod to not run in a desired state. This version fix that issue.

---

crt-20230417-2363

Software Version: crt-20230417-2363

Release Notes

A fast ACL configuration was resulting in a load balancer configuration to be not installed, impacting traffic. This is fixed with this version.

---

crt-20230410-2360

Software Version: crt-20230410-2360

Release Notes

Fixes an issue with creating a nodeport service. This does not impact any user traffic.

---

crt-20230327-2320

Software Version: crt-20230327-2320

Release Notes

Data path change to avoid drops due to "IP Fragment Too Small". Ensure that fragments are greater than 576 bytes.

---

crt-20230324-2319

Software Version: crt-20230324-2319

Release Notes

Checksum for fragmented UDP packets was incorrect, fixed with this version.

---

crt-20230323-2318

Software Version: crt-20230323-2318

Release Notes

When kubelet-proxy exited unexpectedly, kubelet are unable to bring kubelet-proxy up and causes worker nodes in a NotReady state. This is fixed.

---

crt-20230301-2265

Software Version: crt-20230301-2265

Release Notes

Fix for issue where cross-reference between WAF security events are created incorrectly.

---

crt-20230228-2264

Software Version: crt-20230228-2264

Release Notes

Fix for the following issues is delivered:

• Local kubeapi does not work after upgrading to the latest image.
• DDoS Auto Mitigation: Fast ACL action not working when the advertisement in load balancer is changed from Regional Edge to Customer Edge and vice versa.

---

crt-20230131-2238

Software Version: crt-20230131-2238

Release Notes

Delivered fix for an issue where a drop in access logs is observed.

---

crt-20230131-2237

Software Version: crt-20230131-2237

Release Notes

Software is updated to support new attack signatures that are added to F5 Distributed Cloud Services App Firewall.

---

crt-20230123-2236

Software Version: crt-20230123-2236

Release Notes

Delivered fixe for custom error issue and a proxy crash. Also increases ETCD keepalive timeout.

---

crt-20230119-2235

Software Version: crt-20230119-2235

Release Notes

Delivered fix for an issue where the request length was larger than 5k when analyzing the request ID header, causing coredump.

---

crt-20230113-2163

Software Version: crt-20230113-2163

Release Notes

Delivered fix for an issue where namespace-based labels are not getting resolved.

---

crt-20221207-2162

Software Version: crt-20221207-2162

Release Notes

Node software is updated as part of the SaaS upgrade.

---

crt-20221122-2120

Software Version: crt-20221122-2120

Release Notes

Fix for an issue where some virtual hosts are not generating any access logs is delivered.

---

crt-20221116-2119

Software Version: crt-20221116-2119

Release Notes

Fix for a K8s pod crash issue is delivered.

---

crt-20221109-2118

Software Version: crt-20221109-2118

Release Notes

Fix for the following issues are delivered:

• Requests to load balancer are failing with 5xx response.
• AWS TGW multi-node Site's Site Local Outside load balancer fails intermittently with upstream connect error.
• Pod in a specific node is moved to into a deadlock state.

---

crt-20221104-2117

Software Version: crt-20221104-2117

Release Notes

OpenVPN upgade is delivered.

---

crt-20221101-2116

Software Version: crt-20221101-2116

Release Notes

Site software is updated as part of regular SaaS upgrade.

---

crt-20221101-2115

Software Version: crt-20221101-2115

Release Notes

Delivered fix for an issue where requests from RE to vK8s endpoints are failing is delivered.

---

crt-20221006-2032

Software Version: crt-20221006-2032

Release Notes

Delivered fix for an issue where load balancer cannot set priority.

---

crt-20221006-2032

Software Version: crt-20221006-2032

Release Notes

Fix for an issue where load balancer does not follow the set priority for an origin pool member.

---

crt-20220829-1745

Software Version: crt-20220829-1745

Release Notes

Fix for a MULTUS issue with user defined Pod Security Policy (PSP) is delivered.

---

crt-20220829-1744

Software Version: crt-20220829-1744

Release Notes

Fix for an issue that causes post upgrade site failures is delivered.

---

crt-20220808-1693

Software Version: crt-20220808-1693

Release Notes

Fix for a bug involving pod-to-pod communication failure is delivered.

---

crt-20220803-1692

Software Version: crt-20220803-1692

Release Notes

Updated as part of enhancement to evaluate network policy label on NAT flow.

---

crt-20220803-1691

Software Version: crt-20220803-1691

Release Notes

Updated as part of SaaS update.

---

crt-20220704-1646

Software Version: crt-20220704-1646

Release Notes

• A software issue where in continuous addition of static routes is fixed.
• AWS Direct Connect enhancements to the Site software are delivered.

---

crt-20220704-1645

Software Version: crt-20220704-1645

Release Notes

Fix for an issue with cookie tampering detection is delivered.

---

crt-20220704-16445

Software Version: crt-20220704-16445

Release Notes

Fix for an issue with update logs is delivered.

---

crt-20220609-1615

Software Version: crt-20220609-1615

Release Notes

Fix for the issue where kubevirt pods are not reachable on a multi-node Site is delivered.

---

crt-20220607-1610

Software Version: crt-20220607-1610

Fix for an issue with certificate validation during HTTP to HTTPS redirect is delivered.

---

crt-20220510-1580

Software Version: crt-20220510-1580

Release Notes

Fixes for the following issues are delivered:

• Fleet deployment issue on multi-node Site
• App Stack Site deployment issue for GCP using Terraform

---

crt-20220510-1579

Software Version: crt-20220510-1579

Release Notes

Fixes for the following issues are delivered:

• Master nodes not reachable in multi-node Site
• IP Fabric connectivity issue between multi-node K8s Site to Regional Edge Site

---

crt-20220510-1578

Software Version: crt-20220510-1578

Release Notes

Fix for an issue deploying AWS TGW and Azure Sites is delivered.

---

crt-20220510-1577

Software Version: crt-20220510-1577

Release Notes

Software is updated to support the regular SaaS update.

---

crt-20220412-1548

Software Version: crt-20220412-1548

Release Notes

Software is updated to use user's password instead of default password in rewriting.

---

crt-20220412-1547

Software Version: crt-20220412-1547

Release Notes

Software is updated to address expiring client certificates.

---

crt-20220329-1460

Software Version: crt-20220329-1460

Release Notes

Fix for Site storage and node deletion issue is delivered.

---

crt-20220329-1459

Software Version: crt-20220329-1459

Release Notes

Update to mitigate CVE-2022-22965 (Spring4Shell) is added.

---

crt-20220329-1458

Software Version: crt-20220329-1458

Release Notes

A fix for container concurrency issue is delivered.

---

crt-20220329-1457

Software Version: crt-20220329-1457

Release Notes

A fix for container deadlock issue is delivered.

---

crt-20220217-1456

Software Version: crt-20220217-1456

Release Notes

A fix to ignore WAF detections in case of good traffic is delivered.

---

crt-20220217-1455

Software Version: crt-20220217-1455

Release Notes

A fix to redudelay in deployment time is delivered.

---

crt-20220217-1454

Software Version: crt-20220217-1454

Release Notes

A fix for an issue with CRL download is delivered.

---

crt-20220217-1453

Software Version: crt-20220217-1453

Release Notes

The site software is updated with security enhancements.

---

crt-20220217-1452

Software Version: crt-20220217-1452

Release Notes

Fix for a problem with route processing is delivered.

---

crt-20220217-1451

Software Version: crt-20220217-1451

Release Notes

Fix for route processing issue is delivered.

---

crt-20220217-1450

Software Version: crt-20220217-1450

Node software is updated as part of the SaaS upgrade. Click here for SaaS changelog.

---

crt-20220203-1433

Software Version: crt-20220203-1433

Release Notes

Node software is updated as part of the SaaS upgrade. Click here for SaaS changelog.

---

crt-20220120-1413

Software Version: crt-20220120-1413

Release Notes

Fix for healthcheck failures is delivered.

---

crt-20211215-1372

Software Version: crt-20211215-1372

Release Notes

Fix for a login issue with GCP Node is delivered.

---

crt-20211215-1371

Software Version: crt-20211215-1371

Release Notes

WAF violation exception enhancement for the node software is added.

---

crt-20211214-1367

Software Version: crt-20211214-1367

Release Notes

Fix for Healthcheck failures due to duplicated endpoints is delivered.

---

crt-20211213-1365

Software Version: crt-20211213-1365

Release Notes

Fix for WAF related container crash is delivered.

---

crt-20211213-1364

Software Version: crt-20211213-1364

Release Notes

Updated signatures and threat campaign for the Log4j Remote Code Execution (RCE) Common Vulnerabilities and Exposures (CVE).

---

crt-20210827-1282

Software Version: crt-20210827-1282

Release Notes

Fix for UDP proxy crash is delivered.

---

crt-20210811-1254

Software Version: crt-20210811-1254

Release Notes

Fix for the following issue is delivered:

Creating TCP load balancer with domain name is not possible for the Site on Kubernetes after the upgrade. As a result, servion external K8s is not created without SNI by the Site on Kubernetes.

---

crt-20210729-1221

Software Version: crt-20210729-1221

Release Notes

Fix to the following issue is delivered:

InterfaIP present in argo NAT pool even when strict mode is set. The fix checks strict mode and avoids adding interfaIP address.

---

crt-20210708-1192

Software Version: crt-20210708-1192

Release Notes

FRR is updated with fix for the following issue:

BGP peers are disconnected every 10 seconds.

---

crt-20210707-1191

Software Version: crt-20210707-1191

Release Notes

GRPC timeout is incremented to 1 second for internal modules.

---

crt-20210702-1190

Software Version: crt-20210702-1190

Release Notes

Delivered fix for the issue of gateway response with code 502. This fix reduces number of responses of type 502.

---

crt-20210625-1189

Software Version: crt-20210625-1189

Release Notes

Delivered fix for the following issue:

Node decomissioning does not delete master node.

---

crt-20210625-1188

Software Version: crt-20210625-1188

Release Notes

Node software is updated as part of the SaaS upgrade. Click here for SaaS changelog.

---

crt-20210518-1098

Software Version: crt-20210518-1098

Release Notes

Delivered fix for the following issue:

Master node is not deleted when the node is decomissioned.

---

crt-20210518-1097

Software Version: crt-20210518-1097

Release Notes

Delivered fix for the following issue:

After software upgrade, origin pools with servidiscovery objects stopped functioning.

---

crt-20210427-1052

Software Version: crt-20210427-1052

Release Notes

Delivered fix for the following issue:

Local site discovering the origin pool but does not work when trying to access.

---

crt-20210420-1051

Software Version: crt-20210420-1051

Release Notes

NetApp Trident and PSO stopped functioning in version crt-20210420-1049. This upgrade delivers the fix.

---

crt-20210420-1050

Software Version: crt-20210420-1050

Release Notes

Introduces allowing of persistent volume in admission by the platform manager.

---

crt-20210420-1049

Software Version: crt-20210420-1049

Release Notes

Node software is updated as part of the SaaS upgrade. Click here for SaaS changelog.

---

crt-20210413-1025

Software Version: crt-20210413-1025

Release Notes

A render problem with Netapp Trident app is fixed.

---

crt-20210413-1024

Software Version: crt-20210413-1024

Release Notes

Fixes for the following issues are delivered:

• TCP Loadbalancer is not functioning.
• Llink quality monitoring status is not correctly reporting.

---

crt-20210407-1023

Software Version: crt-20210407-1023

Release Notes

Fixes for the following issues are delivered:

• Trident backend job continues to stay in running mode.
• Pure Storage Servivolume expansion does not function. Fix is delivered by upgrading to PSO v6.0.5.

---

crt-20210407-1022

Software Version: crt-20210407-1022

Release Notes

Fixes for the following issues are delivered:

• Servimesh displays incorrectly in the UI.
• Session init packets originated by FRR are dropped.

---

crt-20210405-1020

Software Version: crt-20210405-1020

Release Notes

Servimesh graph shows node from another tenant. Fix for the same is delievered.

---

crt-20210403-1019

Software Version: crt-20210403-1019

Release Notes

Fast ACL performanenhancements are delivered.

---

crt-20210312-973

Software Version: crt-20210312-973

Release Notes

Port forwarding enabling for NetApp Trident CSI is added.

---

crt-20210312-972

Software Version: crt-20210312-972

Release Notes

Netapp trident version is upgraded to version 21.01.1

---

crt-20210312-971

Software Version: crt-20210312-971

Release Notes

Following issue fixes are delivered for managed K8s.

• kubectl drain does not function for managed K8s
• Cluster role, cluster role binding, pod security policy, and storage class creation is not allowed using managed K8s API.

---

crt-20210312-970

Software Version: crt-20210312-970

Release Notes

• Managed K8s resourlimit is enhanced to 8GB.
• AES encryption support is added for NetApp Trident.

---

crt-20210220-896

Software Version: crt-20210220-896

Release Notes

Fix for endpoint discovery failure for routes from REs is delivered. This is in case of HTTP proxy.

---

crt-20210220-895

Software Version: crt-20210220-895

Release Notes

Node software is updated as part of regular SaaS upgrade. Click here for more information.

---

crt-20210203-845

Software Version: crt-20210203-845

Release Notes

Endpoint reachability (via PSK tunnel) issue for AWS TGW is fixed.

---

crt-20210126-844

Software Version: crt-20210126-844

Release Notes

Header manipulation issue for request_headers_to_add value in case of HTTP load balancer is fixed.

---

crt-20210121-843

Software Version: crt-20210121-843

Release Notes

Node software is updated as part of regular SaaS upgrade. Click here for more information.

---

crt-20201204-725

Software Version: crt-20201204-725

Release Notes

Performanimprovements to F5® Distributed Cloud Mesh software are delivered.