Skip to content

Deployment Runbook: Staging

Deployed automatically by Jenkins on every release-tag build (v*.*.*) - this page documents what happens and how to reproduce/verify it manually; it is not a page this portal (or any unattended process) executes deploys from.

Preconditions

  • terraform/environments/platform/ and terraform/environments/staging/ already applied (see Terraform) - the shared K3s node and staging's own data VM must exist.
  • akshayabazaar-staging-secrets Secret already applied in the akshayabazaar-staging namespace.
  • Backend tests, frontend tests, and both dependency/image scans passed (Jenkins won't reach this stage otherwise - see CI/CD → Jenkins).

Backup/checkpoint

Automated nightly backup, when backup.enabled: true - see Operations → Backup. Before a specific risky release, a manual checkpoint is reasonable:

# Linux/Bash, from a host that can reach staging's data VM
mysqldump -h <staging-data-vm-private-ip> -u <DB_USER> -p ecommerce_db > staging-pre-release-$(date +%Y%m%d).sql

Build

Jenkins builds both images once per commit (see CI/CD → Jenkins) - never rebuilt for staging specifically.

Deploy

$env:KUBECONFIG = "path\to\kubeconfig-staging.yaml"
.\scripts\deploy-staging.ps1 -ImageTag <commit-sha>
Equivalent to what Jenkins runs (scripts/deploy-staging.sh on the Linux agent): helm upgrade --install akshayabazaar helm/akshayabazaar -f values-staging.yaml --set backend.image.tag=<tag> --set frontend.image.tag=<tag> --namespace akshayabazaar-staging --wait --atomic.

Database migration

Runs automatically as a Helm pre-upgrade hook, same as every environment - see Kubernetes → Helm → Migrations. Staging's migrations run against staging's own data VM only.

Verification

.\scripts\verify-staging.ps1
(defaults already point at real staging's namespace/release/host).

Expected result

Summary: 14 PASS, 0 WARNING, 0 FAIL - same target as local, against real infrastructure.

Smoke testing

Jenkins runs scripts/smoke-test.sh automatically immediately after deploy - a failure here stops the pipeline before production promotion is ever offered.

Monitoring

See Operations → Monitoring for what's wired up versus documented-but-off.

Rollback

.\scripts\rollback-staging.ps1
(dry run by default - prints helm history; add -Yes to actually roll back). Full detail: Operations → Rollback.

Common failure scenarios

Same NetworkPolicy/migration-hook classes of failure as local (this cluster runs the identical chart) - see Kubernetes → Troubleshooting.