Operator-less Istio Migration Guide
Background
In August 2024, the upstream Istio project announced the deprecation of the Istio Operator in Istio 1.24. In consideration of this, Big Bang 3.0 deprecates the Istio Operator, hence the major version bump.
Big Bang has released packages for migrating over to native Helm charts for Istio. This is included in the SmoothGlue 6.16.0 release, but the Istio migration is disabled by default. It is recommended that users enable and test out the migration on test/staging clusters (only) prior to the release of SmoothGlue 6.18.0.
Concerns
SmoothGlue handles all of the migration steps automatically for parts under SmoothGlue's control, such as Istio components. But there will be some downtime and manual steps (e.g., for custom applications using the built-in Gateways) as described below.
Downtime
There will be some down time, as the Istio Gateways will be recreated in a new namespace. During this time, applications that are utilizing the built-in Gateways will be inaccessible until after the migration is complete.
Custom Application Using Gateways
Custom applications deployed via Argo CD or other deployment mechanisms that use the built-in Gateways will need to update their VirtualService config to use the new Gateway namespace/name. It is recommended to add both the old and new Gateway namespace/name to existing application deployments before the migration to minimize downtime.
Pre-Migration
Applications that are deployed via Argo CD that are using the built-in public or passthrough Gateway should have their VirtualService updated to include the new Gateway namespace/name. For example:
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
name: my-app
namespace: my-app
spec:
gateways:
- istio-system/public
# add new public gateway
- istio-gateway/public-ingressgateway
# alternatively for the passthrough gateway
- istio-gateway/passthrough-ingressgateway
hosts:
- my-app.run.example.com
http:
- match:
- uri:
prefix: /
route:
- destination:
host: my-app.my-app.svc.cluster.local
port:
number: 80
Please back up the existing Istio ConfigMaps and Secrets from the structsure-system namespace before migrating in case you need to rollback:
kubectl get configmap -n structsure-system istio-controlplane-overrides -o yaml > istio-controlplane-cm.yaml
kubectl get secret -n structsure-system istio-controlplane-overrides -o yaml > istio-controlplane-secret.yaml
kubectl get configmap -n structsure-system istio-operator-overridess -o yaml > istio-operator-cm.yaml
kubectl get secret -n structsure-system istio-operator-overridess -o yaml > istio-operator-secret.yaml
Migration
Expected downtime for the migration should be around 15-30 minutes as SmoothGlue reconfigures Istio to use the new configuration. SmoothGlue handles all of the migration steps Big Bang documented here.
To migrate to the native Istio Helm charts, add the following to the zarf-config.yaml. This will be the default setting in SmoothGlue 6.18.0, but you can try out the migration on a test cluster for now.
package:
deploy:
set:
MIGRATE_ISTIO: "true"
Afterwards, proceed with the normal package deploy procedure to initiate the migration.
Example VirtualService output before migration, showing the old (existing) Gateways:
kubectl get vs -A
NAMESPACE NAME GATEWAYS HOSTS AGE
...
jira jira ["istio-system/public"] ["jira.eks-build-jgsw-17190-testupgrade.structsure.io"] 5h48m
keycloak keycloak ["istio-system/passthrough"] ["keycloak.eks-build-jgsw-17190-testupgrade.structsure.io"] 5h45m
...
Example VirtualService output after migration, showing the new (migrated) Gateways:
kubectl get vs -A
NAMESPACE NAME GATEWAYS HOSTS AGE
...
jira jira ["istio-gateway/public-ingressgateway"] ["jira.eks-build-jgsw-17190-testupgrade.structsure.io"] 13h
keycloak keycloak ["istio-gateway/passthrough-ingressgateway"] ["keycloak.eks-build-jgsw-17190-testupgrade.structsure.io"] 13h
...
See Upgrade SmoothGlue for more detailed steps on general upgrade procedures.
Post-Migration
After the migration, check all endpoint applications to ensure they are reachable. If some are inaccessible, ensure they have the updated Gateway listed in their VirtualService.
Additionally, Big Bang has a troubleshooting guide for the Istio migration.
Migration Rollback
If the migration needs a rollback, apply the backed-up Istio ConfigMaps and Secrets to the cluster:
kubectl apply -f istio-controlplane-cm.yaml
kubectl apply -f istio-controlplane-secret.yaml
kubectl apply -f istio-operator-cm.yaml
kubectl apply -f istio-operator-secret.yaml
In an effort to prevent data loss, SmoothGlue adds the helm.sh/resource-policy: keep annotation to the Istio Crossplane claims to reduce the likelihood of Zarf accidentally uninstalling Istio as part of a misconfiguration. These annotations will need to be manually removed in order to rollback to the Istio Operator packages:
kubectl patch istiocrds.app.structsure.io istio-crds-local -n structsure-system --type='json' -p='[{"op": "remove", "path": "/metadata/annotations/helm.sh~1resource-policy"}]'
kubectl patch istiod.app.structsure.io istiod-local -n structsure-system --type='json' -p='[{"op": "remove", "path": "/metadata/annotations/helm.sh~1resource-policy"}]'
kubectl patch istiogateway.app.structsure.io istio-gateway-local -n structsure-system --type='json' -p='[{"op": "remove", "path": "/metadata/annotations/helm.sh~1resource-policy"}]'
Remove MIGRATE_ISTIO: "true" from the zarf-config.yaml and deploy the package again to rollback to the Istio Operator configuration.