Troubleshoot the Pillars pipeline

How-ToPublicInterface: GitLab CI/CD

Use your application project's configuration, pipeline graph, job logs, and artifacts to diagnose a failure. You do not need access to the Pillars implementation repository. You need permission to view those records; changing CI/CD settings can require a maintainer.

Record the failing run

1. Record the project, commit, pipeline URL, pipeline source, and template ref from your project's include or the installation configuration supplied by your platform team.
2. Determine whether GitLab failed to create the pipeline, a job was omitted, a job failed, or a delivery output is missing. These are different failure points.
3. Open the affected job log and inspect its first substantive error. Retain the relevant report and image/package reference. Do not paste tokens or full environment dumps into a support request.
4. Use the matching section below. Make configuration changes on a branch and create a new pipeline to verify them.

Pipeline not created

Symptom	What to check	Action
not found or access denied for an include	Your application's include project, ref, file path, and the initiating user's permission to read it.	Ask the platform team to supply or repair the customer-accessible template. Adding a job token inside a job cannot fix pre-job include access.
No branch pipeline after creating a branch	The branch can be identical to the default branch, or a merge request already exists.	Check the merge-request pipeline and the workflow rules.
Invalid YAML or merged job definition	The pipeline editor's validation result for your project.	Correct the named field. Preserve required stages and remember that overridden arrays replace the included arrays.
A protected secret is missing only on a branch	Variable protection and environment scope.	Use the intended protected workflow or have the maintainer supply an appropriately scoped credential. Do not print the secret to check it.

Missing job dependency

GitLab can reject a pipeline when an enabled job requires another job that was disabled or excluded. These are the principal dependency relationships:

Dependent job	Required jobs or artifacts
sonarqube	dependency-check; framework-specific analysis/build inputs are optional when absent.
zap, cypress	container-image and crane.
npm-lint, ordinary npm-tests	npm-dependencies.
npm-tests-trigger-pipeline	npm-tests-generate-pipeline.
npm-tests-pipeline-artifacts	The trigger and the expected child npm-tests-recombine artifacts.
pip-tests	pip-dependencies.
poetry-tests	poetry-dependencies.
go-lint, go-tests, go-audit	go-dependencies.
cmake-lint, cmake-tests	cmake-dependencies.
dotnet-tests	dotnet-dependencies.
kustomize-kubeconform	kustomize-build.

Build and package jobs also consume earlier framework artifacts. A job using dependencies can be created but then lack required build output. If disabling an entire integration, prefer its framework switch. If disabling only one job, either retain the producer, disable its consumers, or deliberately replace their dependencies and supply the needed inputs. Verify the complete job graph after the change.

Expected job missing or skipped

1. Check the framework switch in project/group CI/CD variables and its detection files. A top-level YAML value cannot control an include. A pip package build needs more than requirements.txt, and a nested Go module needs GO_MODULE_DIR.
2. Check the specific disable switch in the framework or analysis reference.
3. For container-dependent jobs, check DOCKERFILE_PATH. For Helm, also check HELM_CHART_DIR/Chart.yaml and whether Helm support was included.
4. For publication, check whether the job is manual on tags, selected by release-pipeline variables, or selected only on the default branch. Do not assume every passing branch publishes every output.
5. Check whether a changed value exists only in a previous job's dotenv output. It cannot change job-selection rules already evaluated for the current pipeline.

Job image or registry access failed

A runner failing to pull its job image has not reached the job script. Ask the platform team to verify the configured image reference, runner registry authentication, registry availability, and architecture. Script-level buildah login is too late for this failure.

For image build, push, or scan failures, verify the registry hostname, image path, and permissions for that operation. A custom DOCKER_AUTH_CONFIG must contain all required registries. NeuVector additionally uses REGISTRY_USER and REGISTRY_PASSWORD or its NV_REGISTRY_* overrides. Check registry reachability from the component that actually pulls the image, which can be the runner or the scanner controller.

Framework build or test failed

Symptom	Corrective action
npm says the lockfile changed	Regenerate and commit the lockfile with the intended Node/npm toolchain, then rerun.
npm lint/test script is absent	Add the actual application script or disable the inapplicable job explicitly.
npm sharded reports cannot be collected	Keep ordinary tests enabled until the platform team validates template/API access, the generated child configuration, and the merge-command mismatch described in the framework reference.
Gradle task is missing	Supply the build, publish, check, and jacocoTestReport tasks used by the selected jobs or customize the affected job.
pip rejects versioning	Provide one literal [project].version in pyproject.toml; the helper does not support a purely dynamic version field.
Python coverage runner is missing or has no target	Install test dependencies; configure Poetry's coverage target, or use the pip job's documented unittest discovery layout.
Go builds the wrong module	Set GO_MODULE_DIR; provide the GoReleaser config in that module.
CMake chooses the wrong project	Set CMAKE_SOURCE_DIR explicitly and ensure required tools/dependencies exist in the job image.
.NET requires a publish project	Set DOTNET_PUBLISH_PROJECT to a .csproj when the build target is a solution.
.NET produces no .nupkg	Make the library packable, or set DOTNET_PACKAGE: "false" for a container-only application.
Build succeeds locally but staging publication fails	The build job may also publish. Verify the staging host, path, and credentials; disabling the later package job does not suppress the build's upload.

Scanner or application test failed

Determine whether the log reports findings, invalid configuration, authentication, connectivity, missing data, or a scanner execution error. Consult the scanner-specific thresholds; there is no single threshold shared by every tool.

• For SonarQube, check the root properties file, analysis token, server URL, and server quality-gate result. Preserve dependency-check when using the standard dependency graph.
• For NeuVector, check the API key, controller reachability, image credentials, and the report's count/score thresholds.
• For offline Grype or Trivy, verify that usable databases exist. Disabling updates does not create or refresh a missing database.
• For crane, inspect the Dockerfile's final image USER. The current check requires a numeric value; it does not itself reject UID zero.
• For Cypress or ZAP, verify that the built service starts, uses the expected HTTP port, and has its required application configuration. For Cypress, provide one config file and working tests. For a non-HTTP image, disable both web-test jobs.

Fix the reported cause, then run a new pipeline for changed code or configuration. For a transient external outage with unchanged inputs, retrying the job can be appropriate. A passing retry does not remove the need to retain and interpret the original evidence.

Evidence or release output missing

Symptom	What to verify
No SBOM files	The syft or Zarf build job actually ran, succeeded, and retained its artifacts. Look in the producing job before looking for a release archive.
No ZIP archive	The archive job was selected and received nonempty directories with the exact sbom or evidence basename.
Archives ran on a supposedly disabled release pipeline	The string RELEASE_PIPELINE: "false" is nonempty. Unset it when release-pipeline behavior is not intended.
attach-artifacts absent	A Git tag and exactly RELEASE_PIPELINE: "true" are required, and ATTACH_ARTIFACTS must not be "false".
Attachment cannot find files	Preserve default archive prefixes, complete the producers, and check retention. The attachment job uses default filenames and recognizes only the documented package directories.
No semantic-release job	The ref name must match SEMANTIC_RELEASE_REF_REGEX. The job is manual; an unmatched ref does not get it.
Release job runs but creates no release	Check releasable changes, the selected release configuration/preset, token permissions, and branch policy.
Zarf publication reports missing records or digest mismatch	Check the original build's records, their one-day retention, and the staging registry content. Run a new complete package pipeline when artifacts expired; do not bypass digest checks.

Manifest update failed

Inspect the target project's pipeline and commit history. Verify the project path, branch, token access, branch protection, file selection, and image name/tag. For Helm, ensure the yq expression matches the values structure. For Kustomize, remember that the current optional dependency names a different job from container-image; verify image coordinates instead of relying on that dependency to populate them.

If a downstream update receives empty values, check where they were set. The Helm and Kustomize triggers restrict top-level YAML variable inheritance and do not declare an image-build dotenv dependency. Set update values and verified image coordinates on the named trigger, following the delivery examples. Keep target project and branch values in project CI/CD variables for include selection.

A manifest update is a direct Git commit. To undo it, use your normal reviewed revert process in the target repository, then verify the deployment controller and application. Reverting the application's .gitlab-ci.yml does not undo a commit already pushed to another repository or revoke a token.

Verify the correction

Confirm that the new pipeline contains the intended jobs, that required jobs completed, and that skipped or allowed failures are understood. Inspect the expected reports, package or image reference, and any downstream target commit. Keep the pipeline URL with the verified result.

If platform support is needed, provide the non-secret configuration fragment, template ref, pipeline/job URL, first relevant error, and expected versus actual behavior. Your platform team can investigate the implementation; reading that repository is not a customer prerequisite.

Related information

• Use the Pillars pipeline
• Pipeline configuration
• Delivery and release configuration