Set up observability for a new application
What you'll learn
Ground Control (SmoothGlue's observability stack) is pre-wired for all platform components — but your application needs to expose metrics and structured logs for them to appear in Grafana. In this tutorial you'll wire a new application into the full observability stack: metrics scraping, log aggregation, and distributed tracing.
Before you begin
- A running SmoothGlue platform with Ground Control enabled (Run or Edge template)
- Console access with SRE or OrgAdmin role
- Your application deployed into a SmoothGlue project namespace
- Your app exposes a
/metricsendpoint in Prometheus format (or you'll use the default Alloy scrape profile)
How Ground Control ingests data
| Signal | How it reaches Ground Control | Storage |
|---|---|---|
| Metrics | Alloy scrapes /metrics on annotated pods | Prometheus (Thanos for long-term) |
| Logs | Alloy reads stdout/stderr from all pods automatically | Loki |
| Traces | App sends OTLP spans to Alloy sidecar | Tempo |
Logs are collected automatically — no configuration needed. Metrics and traces require annotation or code changes.
Step 1 — Enable metrics scraping
Add the following annotations to your application's Kubernetes Deployment or Pod spec. You can do this via the Console (Projects → my-app → Workloads → Edit Annotations) or in your Helm chart:
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "8080" # replace with your metrics port
prometheus.io/path: "/metrics" # default — change if different
Within 60 seconds, Alloy detects the annotation and begins scraping. Verify in the Console under Tools → Alloy → Targets — your pod should appear with status UP.
Step 2 — Verify logs are being collected
Logs from all pods in SmoothGlue namespaces are automatically collected by Alloy and sent to Loki. No configuration needed.
To verify: navigate to Ground Control → Logs in the Console (or open Grafana → Explore → Loki). Use the label selector:
{namespace="my-app"}
You should see your application's stdout output within 30 seconds of the pod starting.
If logs are empty: confirm the pod is running (smoothglue status or kubectl get pods -n my-app) and that your app writes to stdout (not to a log file inside the container).
Step 3 — Add distributed tracing (optional)
If your application supports OpenTelemetry (recommended), configure it to send traces to the Alloy OTLP endpoint:
Environment variable (set via Console → Projects → my-app → Environment):
OTEL_EXPORTER_OTLP_ENDPOINT=http://alloy.monitoring.svc.cluster.local:4317
OTEL_SERVICE_NAME=my-app
For applications without OTLP support, Alloy can inject a language-specific auto-instrumentation agent. Contact your Platform Engineer for the agent package.
Traces appear in Ground Control → Traces (Grafana → Explore → Tempo) within seconds of the first instrumented request.
Step 4 — Import a dashboard
Ground Control ships with pre-built dashboards for common frameworks (Node.js, Go, Python/Django, JVM). To import one:
- Navigate to Ground Control → Dashboards → Import
- Select a template from the SmoothGlue dashboard library or paste a Grafana dashboard JSON
- Set the namespace variable to
my-app
For a blank starting point, the SmoothGlue App Template dashboard includes panels for: request rate, error rate, latency (p50/p95/p99), pod CPU/memory, and log error count.
Step 5 — Set up an alert
Navigate to Ground Control → Alerts → New Rule:
- Name:
my-app high error rate - Query:
sum(rate(http_requests_total{namespace="my-app",status=~"5.."}[5m])) > 0.1 - Severity: Warning
- Notify: Mattermost channel
#my-app-oncall
Click Save. The alert is active immediately. Platform Custodian will also page you if it detects the pod has restarted more than 3 times in 10 minutes, regardless of this alert.
What you built
Full observability wiring for your application:
- Metrics scraped by Alloy and stored in Prometheus
- Logs collected automatically and queryable in Loki
- Distributed traces available in Tempo (if OTLP configured)
- Dashboard and alert rule active in Ground Control