Skip to content

Deployment Runbook: Staging (Legacy — Kubernetes/Helm)

SUPERSEDED - this is not the current staging path

This page describes the planned future Kubernetes/Helm/Terraform staging architecture, which remains on hold behind the domain-registration blocker documented below. It is kept for historical/future-architecture reference only - do not follow it to deploy or verify today's real staging environment.

VayzoMart's current, live staging environment runs on the shared-VPS Docker Compose architecture instead, at https://stg.vayzomart.com. For the canonical, actually-used runbook, see Deployment → Staging (VPS).

The two architectures are expected to coexist only until this Kubernetes path's own domain and infrastructure prerequisites are met, at which point a real migration decision (not a silent switch) will be made and this page updated accordingly.

Deployed automatically by Jenkins on every release-tag build (v*.*.*) - this page documents what happens and how to reproduce/verify it manually; it is not a page this portal (or any unattended process) executes deploys from.

⚠ Deployment is currently blocked - domain not yet registered

AkshayaBazaar's own public domain has not been registered yet. akshayathreadworld.in (the hostname this runbook, the Helm chart, and the deployment scripts previously referenced) belongs to a different, unrelated application and must never be used for AkshayaBazaar - not for DNS, not for CORS, not for TLS, not even as a placeholder.

Every domain-shaped value in this repo (helm/akshayabazaar/values-staging.yaml, Jenkinsfile, scripts/deploy-staging.ps1/.sh, scripts/verify-staging.ps1) now uses a clearly non-routable RFC 2606 "reserved for documentation" placeholder - staging.akshayabazaar.example.invalid - instead. This placeholder will never resolve, by design; it exists only so config files stay valid and self-explanatory, not as something to actually deploy against.

Do not deploy to real staging until: 1. A real domain is registered for AkshayaBazaar, and 2. terraform/environments/staging/terraform.tfvars (and platform's) is filled in with that real domain_name, and Terraform has actually provisioned the corresponding Cloudflare DNS record, and 3. helm/akshayabazaar/values-staging.yaml's ingress.host/backend.config.allowedOrigins/ backend.config.fileStorage.baseUrl/backend.config.r2.publicBaseUrl are updated (or overridden via --set) to that real hostname.

Until then, scripts/verify-staging.ps1 requires -IngressHost to be supplied explicitly (it no longer defaults to any real-looking hostname) or -SkipExternalHttp to run the in-cluster checks only - see that script's own header. scripts/deploy-staging.ps1/.sh will still deploy the placeholder hostname if run as-is; the resulting Ingress will simply never be reachable from the internet until the real domain is wired in, which is the intended, safe behavior in the meantime.

Preconditions

  • terraform/environments/platform/ and terraform/environments/staging/ already applied (see Terraform) - the shared K3s node and staging's own data VM must exist.
  • environments/staging/kubernetes/namespace.yaml and environments/staging/kubernetes/deployer-rbac.yaml applied by an operator after the cluster exists. Use the resulting namespace-scoped ServiceAccount token for the staging deploy kubeconfig; do not use K3s's raw cluster-admin kubeconfig for routine staging deploys. The namespace manifest is the canonical owner of akshayabazaar-staging; real staging deploy scripts require the namespace to already exist and do not pass Helm --create-namespace.
  • akshayabazaar-staging-secrets Secret already applied in the akshayabazaar-staging namespace.
  • Backend tests, frontend tests, and both dependency/image scans passed (Jenkins won't reach this stage otherwise - see CI/CD → Jenkins).

Scaling policy

Current HPA matrix:

Environment Backend Frontend
staging min 1 / max 2 min 1 / max 2
production min 2 / max 6 min 2 / max 4

Staging is intentionally conservative for cost and to avoid crowding production headroom on the shared K3s node. Production frontend and backend do not need identical limits: the backend is the CPU-bound API tier, while the frontend is lightweight nginx serving static assets.

Terraform validation

Safe, no-account checks before the Vayzo domain and account credentials exist:

cd terraform\environments\staging
terraform fmt -check -diff
terraform init -backend=false
terraform validate

Real planning, once manual account inputs are available:

cd terraform\environments\staging
terraform init
terraform plan -out=staging.tfplan

Do not run terraform apply during readiness review. A real plan requires:

  • Cloudflare account ID, zone ID, and scoped API token.
  • R2 remote-state access key/secret and the real R2 backend endpoint replacing ACCOUNT_ID.
  • Hetzner token and SSH/admin CIDR inputs.
  • terraform/environments/platform/ already applied so staging can read the shared K3s node and network outputs.
  • The real Vayzo/AkshayaBazaar domain, with staging using a configurable subdomain rather than a hardcoded hostname.

Backup/checkpoint

Automated nightly backup, when backup.enabled: true - see Operations → Backup. Before a specific risky release, a manual checkpoint is reasonable: 

# Linux/Bash, from a host that can reach staging's data VM
mysqldump -h <staging-data-vm-private-ip> -u <DB_USER> -p ecommerce_db > staging-pre-release-$(date +%Y%m%d).sql

Build

Jenkins builds both images once per commit (see CI/CD → Jenkins) - never rebuilt for staging specifically.

Deploy

$env:KUBECONFIG = "path\to\kubeconfig-staging.yaml"
.\scripts\deploy-staging.ps1 -ImageTag <commit-sha>
Both scripts/deploy-staging.ps1 and scripts/deploy-staging.sh (as well as Jenkins) enforce the canonical pre-deployment privacy gate (scripts/pre-deploy-gate.ps1 / scripts/pre-deploy-gate.sh) BEFORE executing helm upgrade --install. Direct, raw Helm commands against staging are strictly prohibited. The scripts set HELM_DRIVER=configmap so Helm release state lives in ConfigMaps; app, DB, R2, Razorpay, JWT, and backup Secrets remain pre-created outside Helm.

Database migration

Runs automatically as a Helm pre-upgrade hook, same as every environment - see Kubernetes → Helm → Migrations. Staging's migrations run against staging's own data VM only.

Verification

.\scripts\verify-staging.ps1 -IngressHost staging.<your-actual-registered-domain>
(namespace/release still default to real staging's; -IngressHost no longer defaults to any real-looking hostname - see "Deployment is currently blocked" above. Until a real domain exists, run .\scripts\verify-staging.ps1 -SkipExternalHttp instead, which verifies everything except the external HTTP check.)

Expected result

Summary: 14 PASS, 0 WARNING, 0 FAIL - same target as local, against real infrastructure.

Smoke testing

Jenkins runs scripts/smoke-test.sh automatically immediately after deploy - a failure here stops the pipeline before production promotion is ever offered.

Monitoring

See Operations → Monitoring for what's wired up versus documented-but-off.

Rollback

.\scripts\rollback-staging.ps1
(dry run by default - prints helm history; add -Yes to actually roll back).

Both scripts/rollback-staging.ps1 -Yes and Jenkins (ACTION=Rollback, ROLLBACK_ENV=staging) enforce the canonical pre-deployment privacy gate before executing helm rollback. Direct, raw helm rollback commands are strictly prohibited. Full detail: Operations → Rollback.

Common failure scenarios

Same NetworkPolicy/migration-hook classes of failure as local (this cluster runs the identical chart) - see Kubernetes → Troubleshooting.