Variable Compositing

This document describes how configuration propogates through SmoothGlue. The general flow is as follows:

1. IaC/user-defined configuration is passed to Zarf.
2. Zarf passes configuration to Crossplane.
3. Crossplane manages SmoothGlue features, Big Bang configuration, and the Big Bang Flux HelmRelease.
4. Big Bang Flux HelmRelease manages and configures core, add-on, and third-party application Flux HelmReleases.
5. Application Flux HelmReleases deploys application resources into the cluster.

IaC

The SmoothGlue Terraform/Terragrunt IaC generates Big Bang values related to the managed infrastructure. Typical values may include:

• Storage configuration (EFS, RDS, or S3)
• EKS or RKE2 related configuration
• Networking configuration

By default, these Big Bang configuration files are created within the IaC's output folder (./outputs). Additionally, there are some statically defined values within the IaC's files folder (./files).

It is recommended to use a tool, such as yq, to dynamically merge the static/generated files with any user-provided configuration files:

SMOOTHGLUE_IAC_PATH=/path/to/iac/folder

# Compile BigBang secret files
yq ea '. as $i ireduce ({}; . * $i)' \
$(find ${SMOOTHGLUE_IAC_PATH}/outputs/ -name 'bigbang-secrets-*.yaml') \
user-bigbang-secrets.yaml > compiled-bigbang-secrets.yaml

# Compile BigBang value files
yq ea '. as $i ireduce ({}; . * $i)' \
${SMOOTHGLUE_IAC_PATH}/files/bigbang-values.yaml \
$(find ${SMOOTHGLUE_IAC_PATH}/outputs/ -name 'bigbang-values-*.yaml') \
user-bigbang-values.yaml > compiled-bigbang-values.yaml

Zarf

The Zarf SmoothGlue package manages the user-provided configuration. The package has logic to retrieve previously defined configuration from the Kuberenetes cluster. After initial install and configuration of the cluster, it is not required to pass the Big Bang configuration files to Zarf. However, it is recommended to use a Zarf config file to pass the configuration to correct drift when configuration has been modified directly in the cluster.

An example Zarf config file for configuring Big Bang would be as follows:

---
package:
  deploy:
    set:
      BIGBANG_SECRETS_FILE: compiled-bigbang-secrets.yaml
      BIGBANG_VALUES_FILE: compiled-bigbang-values.yaml

The SmoothGlue package will separate the configuration files into separate ConfigMaps and Secrets by application name within the structsure-system namespace. For example, if there is configuration for GitLab, it will be placed into the gitlab-overrides ConfigMap and the gitlab-overrides Secret within the cluster.

The ConfigMaps and Secrets in the structsure-system namespace can be modified in the cluster. They are dynamically retrieved by Zarf during the package deployment and can be a convenient method to reconfigure an application without having to deploy the SmoothGlue package again. However, Big Bang configuration passed by Zarf, if present, will be merged into the ConfigMaps and Secrets in the structsure-system namespace potentially overwriting configuration.

note

Since the IaC configuration is passed via Zarf, it will also end up in the <app>-overrides.

warning

Since SmoothGlue will retrieve configuration dynamically from the cluster, unsetting the configuration has some additional considerations detailed in Unsetting Custom Helm Values.

Crossplane

SmoothGlue has a set of Crossplane objects that is resposible for deploying, managing configuration, and correcting drift for applications deployed through Big Bang, as well as configuring SmoothGlue features, such as SSO.

Big Bang and any application deployed via Big Bang have their own corresponding Crossplane Claim. System integrators do not need to worry about these claims but should know they exist in the cluster. Each Big Bang application claim does the following:

1. Copies application override ConfigMaps/Secrets from the structsure-system namespace to the bigbang namespace.
2. Generates SmoothGlue default value ConfigMaps in the bigbang namespace for Big Bang applications.
3. Configures Big Bang to use the configuration sources, as shown above.

In addition, Big Bang's Claim also deploys Flux resources into the cluster to deploy the Big Bang helm chart.

If interested, Claims can be retrieved from the Kubernetes cluster with the following command: kubectl get claims -A.

Big Bang HelmRelease

The Big Bang HelmRelease resource is the root Flux resource which generates all the other HelmReleases in SmoothGlue (including external HelmReleases such as cluster-autoscaler). The Big Bang HelmRelease object merges all of the *-values and *-overrides ConfigMaps and Secrets in this order:

• ConfigMap bigbang-values
• Secret bigbang-values
• ConfigMap bigbang-overrides
• Secret bigbang-overrides
• ConfigMap global-sso-config
• Secret <application>-sso-config
• ConfigMap <application>-values
• Secret <application>-values
• ConfigMap <application>-overrides
• Secret <application>-overrides

To view the exact order of precedence, describe the Big Bang HelmRelease object and view the .spec.valuesFrom field. The ConfigMaps and Secrets listed there will be merged in order, with later entries overriding earlier entries. In general, though, the following rules should be applicable:

• Individual application Configmaps/Secrets will override the umbrella Big Bang values/overrides.
• For any given application, the *-overrides ConfigMap/Secret will override the *-values ConfigMap/Secret of the same name.
• Secrets will override the ConfigMap of the same name.

After all the values from various sources are merged together, they are then used to populate the individual application HelmReleases' values, with some logic from the Big Bang chart being layered on top, particularly to help integrate SSO into the various components. To view the Big Bang umbrella chart's logic for any given application, refer to the chart/templates/<application>/values.yaml file for that application. For example, Keycloak's values.yaml is located here.

To view the values for the Big Bang HelmRelease, refer to How to Inspect Helm Values and Manifests.

Application HelmRelease

The Big Bang HelmRelease resource generates child HelmReleases for each application it manages; for applications under the Big Bang umbrella, the HelmRelease will be created under the bigbang Namespace; for external applications, the HelmRelease will be created under a namespace specific to that application. Values for each application are populated from a Secret called bigbang-<application>-values, which is also created by the Big Bang HelmRelease. This Secret contains three fields in order of increasing precedence:

• common
  • This field is currently always empty.
• defaults
  • This field contains the default per-application logic in the Big Bang Umbrella Chart. Typically, this will set common values such as the domain, the SSO URL and CA certificate, and so on. Refer to the values-secret helper in the Big Bang Umbrella chart for how this is implemented.
• overlays
  • This field will contain all the merged explicitly-set values relevant to the application, as provided to the Big Bang HelmRelease. All values provided to the Big Bang HelmRelease under the .<application>.values key will be pass through to this field directly.

To view the values for each application's HelmRelease, refer to [How to Inspect Helm Values and Manifests].