Skip to main content
Version: 6.13.0

Release Notes

6.13.0 (2025-04-16)​

📦 SmoothGlue Features​

  • A new optional variable cloudwatch_log_group_retention_in_days has been added to the env.hcl files to configure the EKS cluster log group retention time. It can be configured as shown below.

    locals {
      cluster_inputs = {
        # Adjust as needed, default is 90 days. Valid value for X is one of:
        # [0 1 3 5 7 14 30 60 90 120 150 180 365 400 545 731 1096 1827 2192 2557 2922 3288 3653]
        cloudwatch_log_group_retention_in_days = X
      }
    }

  • IAM policies generated by the SmoothGlue IAC no longer apply tags to IAM policies when compatibility_mode is set to true. This change is to conform to limitations on high-side deployments

    • NOTE: On an existing cluster, you may need to delete the allow_kms and allow_cluster_autoscaler IAM policies to allow their recreation.

⏩ Upgraded Packages​

  • This release of SmoothGlue Enterprise v6.13.0 includes Big Bang Version 2.50.0. For more details on the features and updates included in Big Bang Version 2.50.0, please refer to the Big Bang release notes.
  • Upgrades the following Big Bang third-party apps:
    • Confluence LTS: 9.2.3
    • Jira LTS: 10.3.5
    • Nexus IQ Server: 1.189.0-01

🐞 Bug Fixes​

  • Fixed an issue that prevented user data scripts from running on AL2023 AMIs.
  • Fixed an issue that overwrote default values from SmoothGlue with customer overrides. This prevented SmoothGlue default values from being viewable at runtime. Overrides provided by customers are still overlayed on top of SmoothGlue default values, so this is a purely cosmetic change.

❗ Known Issues​

  • The following only applies to the initial deployment of the SmoothGlue IAC. No action is required for updates to already deployed clusters - Due to an upstream issue for the EKS module and when deploying a cluster using an AL2023 AMI, the System Integrator will need to manually generate and set a Zarf registry pull password. The following config can be added to the env.hcl file:

    locals {
      cluster_inputs = {
        zarf_registry_pull_password = "securepassword123"
      }
    }

  • When using a network load balancer (NLB) with the preserve_client_ip option enabled, the default routing rules for EKS nodes prevent nodes from accessing platform services hosted on the same node, which can cause failures when logging into Keycloak, particularly on clusters with fewer nodes.

    • More specifically, the default routing rules for nodes do not route traffic to the VPC router for traffic within the node’s local subnet, since these addresses should theoretically be reachable directly by the node. However, when using the preserve_client_ip option, the VPC router rewrites the source IP for traffic; when the node attempts to talk to the NLB, the traffic is rewritten so that it appears to come from the node itself, and the return traffic is not able to be routed correctly back to the NLB.
    • The following options are potential workarounds:
      • Disabling the preserve_client_ip option on the NLB will resolve the issue at the cost of losing source attribution for incoming traffic.
      • Removing the local subnet route on nodes will resolve the issue at the cost of increasing the amount and cost of traffic being routed through the VPC router.
      • Increasing the node count for the cluster will reduce the likelihood of the issue because it will become less likely for any given traffic to be routed back to the original node.

  • The Big Bang Istio Helm chart has a bug that prevents Istio Gateway deployments from properly being upgraded. During an upgrade, Istio Gateway deployments may get stuck as a result and will need manual intervention to complete the upgrade. To validate the issue in the cluster, check the health of the istiooperators.install.istio.io resource as follows:

    kubectl get istiooperators.install.istio.io -n istio-system

    If it is in Error status, delete all Istio Gateway deployments in the istio-system namespace to allow the Istio Operator to finish reconciling the upgrade and report a Healthy status. The deployments will be recreated automatically by the Istio Operator. For example:

    kubectl delete deployment.apps/admin-ingressgateway -n istio-system
    kubectl delete deployment.apps/passthrough-ingressgateway -n istio-system
    kubectl delete deployment.apps/public-ingressgateway -n istio-system

    Note: Deleting the deployments will entail some brief but non-zero downtime.

    https://repo1.dso.mil/big-bang/product/packages/istio-controlplane/-/issues/253 has been opened to track this issue, which was introduced in SmoothGlue 6.7 (Big Bang 2.44)

🌐 Compatibility​

  • The packages for this release were built using Zarf v0.46.0.
  • The packages were tested across the following Kubernetes distributions:
    • RKE2: v1.29.8+rke2r1
    • K3s: v1.32.3+k3s1
    • EKS: v1.30.9-eks-5d632ec

6.12.0 (2025-04-02)​

🚨 Upgrade Notices​

  • Kyverno
    • A new Kyverno Policy has been added which mutates pod specs to drop ALL capabilities in all containers if not already done. This policy works in tandem with the require-drop-all-capabilities policy to make it easier for SREs to securely deploy workloads to their clusters without having to explicitly modify the pod's containers' securityContexts to be compliant.
    • If Big Bang consumers are currently excluding certain workloads from the require-drop-all-capabilities policy due to incompatibilities with that policy, those exclusions should also be included for this new policy: add-default-capability-drop to avoid workload interruption.

⏩ Upgraded Packages​

  • This release of SmoothGlue Enterprise v6.12.0 includes Big Bang Version 2.49.0. For more details on the features and updates included in Big Bang Version 2.49.0, please refer to the Big Bang Release Notes.
  • console updated image to 54183
  • nexus-iq chart upgraded to 188

🌐 Compatibility​

  • The packages for this release were built using Zarf v0.46.0.
  • The packages were tested across the following Kubernetes distributions:
    • RKE2: v1.30.9-rke2r1
    • K3s: v1.30.9+k3s1
    • EKS: v1.30.8
  • The following AMI versions were used for testing:
    • RKE2 AMI: smoothglue-rke2-v1.30.9-rke2r1-rocky-8-base-v1.1.1-stig-2025-02-17T09-24-30Z
    • EKS AMI: smoothglue-eks-1.30.8-rocky-8-base-v1.1.1-stig-2025-02-10T09-21-19Z
    • Base AMI: base-Rocky-8-EC2-LVM-v1.1.1-stig-2025-02-10T0802

6.11.0 (2025-03-19)​

🚨 Upgrade Notices​

  • Flux has been upgraded to 2.5.1. Platform Operators should update their local Flux binary to a compatible version.

  • In-spite of the Istio HelmRelease being upgraded from 1.23.4 to 1.23.5, it is possible that the relevant deployments are still “stuck” at 1.23.4. If so, they will need to be manually upgraded. Check the health of the istiooperators.install.istio.io resource as follows:

    kubectl get istiooperators.install.istio.io -n istio-system

    If it is in Error status, delete the (relevant to your cluster) deployments in the istio-system namespace as follows to bring it back to Healthy status. The deployments will be recreated automatically by the Istio Operator.

    kubectl delete deployment.apps/admin-ingressgateway -n istio-system
    kubectl delete deployment.apps/passthrough-ingressgateway -n istio-system
    kubectl delete deployment.apps/public-ingressgateway -n istio-system
    kubectl delete deployment.apps/istiod -n istio-system

    Do a describe on the above deployments to verify that labels and images are now showing 1.23.5.Note that deleting the deployments will entail some brief but non-zero downtime. 

📦 SmoothGlue Features​

  • On RKE2-based clusters, the preserve_client_ips IaC variable is now set to false by default in order to allow pods to communicate internally using external DNS names. See the known issues section for more information.
    Note: This means that the client IP in any logs will appear to be the load balancer itself. To get the true client IP, you will need to setup monitoring on the NLB
  • Added documentation for manually rotating RDS/Aurora database passwords as a good security practice.

⏩ Upgraded Packages​

  • This release of SmoothGlue Enterprise v6.11.0 includes Big Bang Version 2.48.0. For more details on the features and updates included in Big Bang Version 2.48.0, please refer to the Big Bang Release Notes.
  • Update Gitlab to 17.9.2 (applied Critical Patch)
  • Update Jira to LTS 10.3.4 (addresses CVE-2024-38819)
  • Update JSM to 10.3.4 (addresses CVE-2024-38819)

🐞 Bug Fixes​

  • Fixed an issue with the SmoothGlue automated SSO feature for ArgoCD. SmoothGlue Admins should now be correctly given admin privileges in ArgoCD.

❗ Known Issues​

  • When using a network load balancer (NLB) with the preserve_client_ip option enabled, the default routing rules for EKS nodes prevent nodes from accessing platform services hosted on the same node, which can cause failures when logging into Keycloak, particularly on clusters with fewer nodes.
    • More specifically, the default routing rules for nodes do not route traffic to the VPC router for traffic within the node’s local subnet, since these addresses should theoretically be reachable directly by the node. However, when using the preserve_client_ip option, the VPC router rewrites the source IP for traffic; when the node attempts to talk to the NLB, the traffic is rewritten so that it appears to come from the node itself, and the return traffic is not able to be routed correctly back to the NLB.
    • The following options are potential workarounds:
      • Disabling the preserve_client_ip option on the NLB will resolve the issue at the cost of losing source attribution for incoming traffic.
      • Removing the local subnet route on nodes will resolve the issue at the cost of increasing the amount and cost of traffic being routed through the VPC router.
      • Increasing the node count for the cluster will reduce the likelihood of the issue because it will become less likely for any given traffic to be routed back to the original node.

🌐 Compatibility​

  • The packages for this release were built using Zarf v0.46.0.
  • The packages were tested across the following Kubernetes distributions:
    • RKE2: v1.30.9-rke2r1
    • K3s: v1.30.9+k3s1
    • EKS: v1.30.8
  • The following AMI versions were used for testing:
    • RKE2 AMI: smoothglue-rke2-v1.30.9-rke2r1-rocky-8-base-v1.1.1-stig-2025-02-17T09-24-30Z
    • EKS AMI: smoothglue-eks-1.30.8-rocky-8-base-v1.1.1-stig-2025-02-10T09-21-19Z
    • Base AMI: base-Rocky-8-EC2-LVM-v1.1.1-stig-2025-02-10T0802

6.10.0 (2025-03-04)​

SmoothGlue Features​

  • This release adds optional basic support for Amazon Linux 2023 (AL2023) AMIs in EKS cluster IaC. To use AL2023, add the following to the cluster_inputs section of your env.hcl file: 
    locals {
      cluster_inputs = {
        ami_id           = "ami-0123456789abcdef0"  # replace with actual AMI ID
        default_ami_type = "AL2023_x86_64_STANDARD"
      }
    }
  • This release adds optional support in IaC for provisioning GitLab's database using an RDS Multi-AZ cluster rather than a single database instance. As a single instance, RDS offers support for a single warm standby instance; however, provisioning the database using an RDS Multi-AZ cluster allows for a cluster of three instances. There is no automatic migration path from a single RDS instance to a Multi-AZ cluster, so we recommend enabling the Multi-AZ cluster during the initial cluster provision if possible. If migrating an existing cluster, you will need to perform a database import/export manually.
    • Note that there are instance class limitations when using a Multi-AZ RDS cluster; see the AWS documentation for more information.

⏩ Upgraded Packages​

  • This release of SmoothGlue Enterprise v6.10.0 includes Big Bang Version 2.47.0. For more details on the features and updates included in Big Bang Version 2.47.0, please refer to the Big Bang Release Notes.
  • Update Confluence to LTS 9.2.1 (Helm chart version 1.22.5-bb.0).
  • Update Jira to LTS 10.3.3 (Helm chart version 1.22.5-bb.1).

🪲 Bug Fixes​

  • Refactor IaC compatibility mode toggle to correctly disable NLB stickiness in ISO regions.
  • Exclude aws-ebs-csi-driver namespace from generate-networkpolicy-imds Kyverno ClusterPolicy so that RKE2 clusters can provision EBS PVCs correctly.
  • Exclude the following namespaces from the require-istio-on-namespaces Kyverno ClusterPolicy so that users may enforce the policy:
    • cluster-autoscaler
    • crossplane-system
    • kyverno
    • structsure-system
  • Extend HelmRelease install timeout for GitLab to 15 minutes.

❗️ Known Issues​

  • When using a network load balancer (NLB) with the preserve_client_ip option enabled, the default routing rules for EKS and RKE2 nodes prevent nodes from accessing platform services hosted on the same node, which can cause failures when logging into Keycloak, particularly on clusters with fewer nodes.
    • More specifically, the default routing rules for nodes do not route traffic to the VPC router for traffic within the node's local subnet, since these addresses should theoretically be reachable directly by the node. However, when using the preserve_client_ip option, the VPC router rewrites the source IP for traffic; when the node attempts to talk to the NLB, the traffic is rewritten so that it appears to come from the node itself, and the return traffic is not able to be routed correctly back to the NLB.
    • We are currently working on an Istio-level fix which should prevent VirtualService traffic within the cluster from ever leaving the cluster. Until that fix is available, the following options are potential workarounds:
      • Disabling the preserve_client_ip option on the NLB will resolve the issue at the cost of losing source attribution for incoming traffic.
      • Removing the local subnet route on nodes will resolve the issue at the cost of increasing the amount and cost of traffic being routed through the VPC router.
      • Increasing the node count for the cluster will reduce the likelihood of the issue because it will become less likely for any given traffic to be routed back to the original node.

🌐 Compatibility​

  • The packages for this release were built using Zarf v0.46.0.
  • The packages were tested across the following Kubernetes distributions:
    • RKE2: v1.30.9-rke2r1
    • K3s: v1.30.9+k3s1
    • EKS: v1.30.8
  • The following AMI versions were used for testing:
    • RKE2 AMI: smoothglue-rke2-v1.30.9-rke2r1-rocky-8-base-v1.1.1-stig-2025-02-17T09-24-30Z
    • EKS AMI: smoothglue-eks-1.30.8-rocky-8-base-v1.1.1-stig-2025-02-10T09-21-19Z
    • Base AMI: base-Rocky-8-EC2-LVM-v1.1.1-stig-2025-02-10T0802

🔗 Helpful Links​

6.9.0 (2025-02-19)​

🚨 Upgrade Notices​

  • During upgrade, you may get a SonarQube is under maintenance error message on the SonarQube UI.
    • To resolve this, once the HelmRelease upgrades, you will be prompted to visit your SonarQube instance at a <sonarqube_url>/setup URL.

📦 SmoothGlue Features​

  • Crossplane Upgraded the Crossplane and provider-kubernetes Crossplane components.
  • IaC: Added HA support for RDS Aurora modules:
    • Supported Applications:
      • Jira
      • Confluence
      • Mattermost
      • SonarQube
      • Nexus
      • Console
      • Keycloak
    • For any of the above modules, you can now add more than one RDS instance into a cluster. Additional instances will be Reader instances only. If the main Writer instance goes down, Aurora will automatically promote a Reader instance to Writer.
      • For each instance created, values such as the availability zone can be manually set; however, you do not have to specify AZ for each instance; Aurora will automatically place each instance in a different AZ.
      • All RDS Aurora storage is automatically replicated across multiple AZs regardless of DB instance count.
      • Examples

        • To create a writer instance and two reader instances for keycloak in your env.hcl:

          keycloak_inputs = {
            # Allows a specific number of database instances to be defined
            rds_instances = {
              primary     = {availibility_zone = us-east-1a}
              secondary   = {}
              replica1    = {}
              # ...
            } 
          }

        • Autoscaling of instances is also optionally available. Aurora autoscaling will NOT scale any instances explicitly defined in rds_instances; it will only add or remove reader instances up to the defined min and max limits. Autoscaling will use the target_metric scaling policy by default with a target CPU utilization of 70%. The following env.hcl provisions Keycloak RDS Aurora autoscaling with between 0 and 5 reader instances:

          keycloak_inputs = {
            rds_auto_scale = {
              enabled = true
              min = 0 # default is 0
              max = 5 # default is 5
            }
          }

⏩ Upgraded Packages​

  • This release of SmoothGlue Enterprise v6.9.0 includes Big Bang Version 2.46.0. For more details on the features and updates included in Big Bang Version 2.46.0, please refer to the Big Bang release notes.
  • Confluence: confluence-node:9.2.0 version: 1.22.3-bb.4
    • Removed duplicate jmx-initContainer
    • Updated cypress (source) 14.0.0 -> 14.0.1
  • Jira: jira-node-lts:10.3.2. version: 1.22.3-bb.0
    • Updated chart to 1.22.3
    • Updated cypress (source) 14.0.0 -> 14.0.1
  • Nexus IQ: Upgraded from 1.186.0-01 to 1.187.0-01
  • Crossplane Components:
    • crossplane - v1.16.0 to v1.19.0
    • provider-kubernetes - v0.12.1 to 0.16.2

❗ Known Issues​

  • If turning on new components, Zarf health checks are performed before unsuspending Big Bang. Manually resume the Big Bang HelmRelease, as required.
  • Big Bang 2.46.0 comes with a known issue relating to the gitlab-gitlab-exporter ServiceMonitor object. We are handling this issue as part of our upgrade process; no user action should be required. More information may be found here.

🌐 Compatibility​

  • The packages for this release were built using Zarf v0.46.0.
  • The packages were tested across the following Kubernetes distributions:
    • RKE2: v1.30.9+rke2r1
    • K3s: v1.31.5+k3s1
    • EKS: v1.30.8-eks-2d5f260
  • The following AMI versions were used for testing:
    • RKE2 AMI: smoothglue-rke2-v1.30.9-rke2r1-rocky-8-base-v1.1.1-stig-2025-02-17T09-24-30Z
    • EKS AMI: smoothglue-eks-1.30.6-rocky-8-base-v1.1.1-stig-2025-01-04T03-12-26Z
    • Base AMI: Rocky-8-EC2-LVM-8.10-20240528.0.x86_64

6.8.0 (2025-02-06)​

🚨 Upgrade Notices​

  • SmoothGlue packages are now built with Zarf v0.46.0, which is the minimum version supported. Please zarf init pre-existing clusters with the v0.46.0 init package before upgrading SmoothGlue.

  • The new Zarf version provides better package readiness checking. As a byproduct, the logic in the package has less control over when and what is evaluated. The default readiness timeout set by Zarf is too low for deploying a fresh cluster. It is recommended to add the following to the ZARF_CONFIG file:

    package:
      deploy:
        timeout: 30m0s

  • Due to the better readiness checks from Zarf, clusters that do not wish to use the automated SSO feature need to disable it from the config. Run clusters have it disabled by default, but for build clusters it is recommended to include the following to the ZARF_CONFIG file to opt out of the automated SSO feature:

    package:
      deploy:
        set:
          KEYCLOAK_CONFIG_ENABLED: false

  • This release will cause a node refresh to occur.

📦 SmoothGlue Features​

  • IaC Allow overriding EKS-calculated max-pods per node.

⏩ Upgraded Packages​

  • Upgraded Zarf to v0.46.0
  • Upgraded Confluence to confluence-node:9.2.0 version: 1.22.3-bb.2
    • Updated gluon from 0.5.12 to 0.5.14
    • Updated cypress dependencies 13.12.0 -> ^14.0.0
    • Updated registry1.dso.mil/ironbank/opensource/postgres/postgresql from 16.6 to 17.2
  • Upgraded Jira to jira-node-lts:10.3.2 version: 1.22.2-bb.4
    • Added gluon 0.5.12 -> 0.5.14
    • Updated cypress ^13.15.0 -> ^14.0.0
    • Updated registry1.dso.mil/ironbank/atlassian/jira-data-center/jira-node-lts 10.3.1 -> 10.3.2
  • This release of SmoothGlue Enterprise v6.8.x includes Big Bang Version 2.45.1. For more details on the features and updates included in Big Bang Version 2.45.1, please refer to the Big Bang Release Notes.
    • Promtail: Note: bumping promtail image/appVersion beyond the version used in upstream chart (v3.0.0 vs v3.3.2)
    • Mattermost upgrade from 10.4.1 to 10.4.2
    • GitLab upgrade from 17.6.2 to 17.8.1

🪲 Bug Fixes​

  • Standardize Terraform provider versions to resolve lookup inconsistencies.
  • Nexus can be enabled with nexus = true or nexusRepositoryManager = true, allowing for conditional enablement of Nexus Repository Manager.

🌐 Compatibility​

  • The packages for this release were built using Zarf v0.46.0.
  • The packages were tested across the following Kubernetes distributions:
    • RKE2: v1.30.8-rke2r1
    • K3s: v1.30.5+k3s1
    • EKS: v1.30.8-eks-2d5f260
  • The following AMI versions were used for testing:
    • RKE2 AMI: smoothglue-rke2-v1.30.8-rke2r1-rocky-8-base-v1.1.1-stig-2025-01-13T09-22-54Z
    • EKS AMI: smoothglue-eks-1.30.6-rocky-8-base-v1.1.1-stig-2025-01-04T03-12-26Z
    • Base AMI: Rocky-8-EC2-LVM-8.10-20240528.0.x86_64

🔗 Helpful Links​

6.7.0 (2025-01-22)​

📦 SmoothGlue Features

  • Kubernetes v1.30.x is officially supported and is the default version used to test SmoothGlue on EKS/RKE2. Additional testing is performed for Kubernetes v1.31.x using K3s.
  • IaC: allow autoscaling on a per-nodegroup basis with supporting documentation. Cluster autoscaler will be enabled by default on the main nodegroup. Additional nodegroups can be explicitly defined via tags.

⏩ Upgraded Packages​

  • This release of SmoothGlue Enterprise v6.7.0 includes Big Bang Version 2.44.0. For more details on the features and updates included in Big Bang Version 2.44.0, please refer to the Big Bang release notes.
  • console updated image to 39560
  • nexus-iq chart upgraded to 186
  • cluster-autoscaler upgrade to support Kubernetes v1.30.x

❗ Known Issues​

  • Kiali - ISSUE

    • On Kubernetes 1.29+, the Kiali Operator may fail with a 404 while running the kiali-deploy playbook if the cluster returns the flowcontrol.apiserver.k8s.io/v1beta2 API version (no longer served as of v1.29).

      In this case, removing the invalid API version should resolve the issue and allow the Kiali Operator to run successfully.

 $ kubectl delete apiservices.apiregistration.k8s.io v1beta2.flowcontrol.apiserver.k8s.io

🌐 Compatibility​

  • The packages for this release were built using Zarf v0.36.1.
  • The packages were tested across the following Kubernetes distributions:
    • RKE2: v1.30.8-rke2r1
    • K3s: v1.30.5+k3s1
    • EKS: v1.30.8-eks-2d5f260
  • The following AMI versions were used for testing:
    • RKE2 AMI: smoothglue-rke2-v1.30.8-rke2r1-rocky-8-base-v1.1.1-stig-2025-01-13T09-22-54Z
    • EKS AMI: smoothglue-eks-1.30.6-rocky-8-base-v1.1.1-stig-2025-01-04T03-12-26Z
    • Base AMI: Rocky-8-EC2-LVM-8.10-20240528.0.x86_64

6.6.0 (2025-01-07)​

🚨 Upgrade Notices​

  • :octagonal_sign: With a Major version update to Jira 10.3 you must also update the SSO addon, this is not provided for you if you are running Jira in a disconnected environment.

📦 SmoothGlue Features​

  • Adds Grafana Dashboard / Alerts for monitoring failed Keycloak login attempts by Username and IP
  • Jira has a major version update that changes how users SSO login, To force users to have to login again see this guide

⏩ Upgraded Packages​

  • This release of SmoothGlue Enterprise v6.6.0 includes Big Bang Version 2.43.0. For more details on the features and updates included in Big Bang Version 2.43.0, please refer to the Big Bang release notes.
    • Jira has received a major version to 10

🌐 Compatibility​

  • The packages for this release were built using Zarf v0.36.1.
  • The packages were tested across the following Kubernetes distributions:
    • RKE2: v1.29.8+rke2r1
    • K3s: v1.30.5+k3s1
    • EKS: v1.29.6
  • The following AMI versions were used for testing:
    • RKE2 AMI: smoothglue-rke2-v1.29.8-rke2r1-rocky-8-base-v1.1.1-stig-2024-09-23T08-14-20Z
    • EKS AMI: smoothglue-eks-1.29.6-rocky-8-base-v1.1.1-stig-2024-09-09T08-14-46Z
    • Base AMI: Rocky-8-EC2-LVM-8.10-20240528.0.x86_64