Skip to content

End-to-End Deployment Runbook

This runbook ties together Terraform, Docker, Kubernetes, Helm, CI/CD, verification, rollback, and troubleshooting. Use the environment-specific pages for exact values; this page provides the execution order and safety gates.

AkshayaBazaar staging/production application deployment is currently blocked

This is a generic, environment-agnostic procedure. For the AkshayaBazaar application specifically, steps 3 onward (Terraform apply, container deploy, Kubernetes/Helm deploy) are not currently executable against real staging or production - AkshayaBazaar's own public domain has not been registered yet, so there is no real, reachable hostname to deploy ingress/CORS/TLS/media configuration against. See Deployment → Staging "Deployment is currently blocked" and Deployment → Production for the full explanation and what must happen first. This blocker does not apply to local development (step 4's Docker/local examples) or to publishing this documentation portal itself (step 12, Cloudflare Pages - a separately hosted, already-live Cloudflare Pages site, unrelated to the marketplace application's future domain).

1. Identify the change

Record:

Repository: <repository>
Commit/tag: <sha-or-tag>
Environment: <local|dev|staging|production>
Namespace: <namespace-or-N/A>
Change type: <application|infrastructure|configuration|documentation>
Rollback target: <previous-known-good-version>

Do not continue until the target environment is unambiguous.

2. Validate source and CI

For AkshayaBazaar, required CI is defined in .github/workflows/ci.yml:

backend: restore → build → tests
frontend: npm ci → lint → build
docs: install requirements → mkdocs build --strict

Production/staging deployment must use a reviewed commit that passed the required checks.

3. Infrastructure changes — Terraform first

Skip this section when the release contains no infrastructure change.

cd terraform/environments/<environment>
terraform init
terraform fmt -check -recursive
terraform validate
terraform plan -out=tfplan

Review the plan for:

  • unexpected destroys/replacements,
  • wrong environment/hostname,
  • firewall exposure,
  • DNS changes,
  • R2/resource deletion,
  • state/backend changes.

Only after review:

terraform apply tfplan

For production, follow the production approval process. Never reuse a staging plan file in production.

4. Container validation

Before Kubernetes deployment, verify the application can be built as containers.

Local/read-only validation examples:

docker compose config
docker compose build
docker compose ps

When images are published by CI, record immutable image tags/digests used by the deployment. Avoid relying on an ambiguous latest tag for rollback-critical releases.

5. Kubernetes pre-flight

kubectl config current-context
kubectl get ns
kubectl get pods -n <namespace>
kubectl get secret akshayabazaar-<env>-secrets -n <namespace>

Confirm the namespace matches the environment inventory.

For production, capture the current state before upgrade:

helm list -n <namespace>
helm history <release> -n <namespace>
kubectl get deploy,pod,svc,ingress -n <namespace>

6. Helm render and diff/review

Use the environment-specific values file.

helm lint helm/akshayabazaar
helm template <release> helm/akshayabazaar \
  --namespace <namespace> \
  -f helm/akshayabazaar/values-<environment>.yaml > rendered.yaml

Review images, ingress hosts, service ports, environment references, Secret names, replica counts, probes, resources, and namespace before upgrade.

Do not put secret values in Helm values committed to Git.

7. Deploy

Example pattern:

helm upgrade --install <release> helm/akshayabazaar \
  --namespace <namespace> \
  --create-namespace \
  -f helm/akshayabazaar/values-<environment>.yaml \
  --wait \
  --timeout 10m

Use the repository's environment-specific script/Jenkins job when one exists; do not replace automation with manual commands without a reason.

8. Immediate verification

kubectl rollout status deployment/<frontend-deployment> -n <namespace>
kubectl rollout status deployment/<backend-deployment> -n <namespace>
kubectl get pods -n <namespace>
kubectl get events -n <namespace> --sort-by=.lastTimestamp

Then perform application verification using Operations → Verification:

  • frontend responds,
  • API health responds,
  • authentication path works,
  • database/Redis connectivity is healthy,
  • critical marketplace smoke tests pass,
  • ingress/TLS/DNS point to the intended environment,
  • no unexpected error spike appears in logs.

9. Release acceptance

A release is complete only when:

CI                 PASS
Deployment         PASS
Pods/rollout       HEALTHY
Smoke tests        PASS
Monitoring/logs    STABLE
Version/commit     CONFIRMED

Record the deployed commit/image and verification result.

10. Rollback decision

Rollback immediately when there is a material regression such as:

  • failed startup/readiness,
  • authentication outage,
  • checkout/payment regression,
  • database migration incompatibility,
  • severe error-rate increase,
  • wrong environment configuration,
  • data-integrity risk.

Follow Operations → Rollback. Do not improvise database rollback: schema/data rollback needs the specific migration/backup plan for that release.

Typical Helm rollback:

helm history <release> -n <namespace>
helm rollback <release> <revision> -n <namespace> --wait

Verify again after rollback.

11. Troubleshooting order

When deployment fails, investigate from outside in:

DNS / Cloudflare
      ↓
Ingress / TLS
      ↓
Kubernetes Service
      ↓
Pods / readiness / logs
      ↓
Application configuration
      ↓
Database / Redis / external dependencies

Use read-only commands first. See Operations → Troubleshooting, Kubernetes Troubleshooting, and Docker Troubleshooting.

12. Documentation-only deployment

For MkDocs portal changes, do not run Terraform/Helm. The path is:

branch → PR → GitHub docs CI → merge master → Cloudflare Pages → portal verification

See Cloudflare Pages.