Zero-Downtime Deployment

1. Concept Overview & Systemic Problem

Primitive service updates via process restart (systemctl restart app or docker restart container) lead to technological downtime. For 5-30 seconds, while the new process initializes the runtime environment, connects to databases, and compiles JIT code:

• All current active HTTP requests are abruptly terminated mid-transfer.
• The load balancer or reverse proxy returns 502 Bad Gateway or 504 Gateway Timeout errors to users.
• User transactions (payments, state preservation, LLM generation) remain in a half-finished state.

Zero-Downtime Deployment eliminates downtime through orchestration of process lifecycle. The new version of the application is launched in parallel with the old one. Traffic is switched only after the new instance successfully responds to the Health Check probe, and the old instance correctly completes all ongoing operations in a Graceful Shutdown mode.

Phase 1: Active version v1.0
[Clients] ---> [Reverse Proxy / Nginx] ---> [Container v1.0 (Active)]

Phase 2: Launching v1.1 and Healthcheck
[Clients] ---> [Reverse Proxy / Nginx] ---> [Container v1.0 (Active)]
                                             [Container v1.1 (Starting... /healthz: 200 OK)]

Phase 3: Traffic switch and Graceful Shutdown v1.0
[Clients] ---> [Reverse Proxy / Nginx] ---> [Container v1.1 (Active)]
                                       \---> [Container v1.0 (Finishing active requests... SIGTERM)]

Phase 4: Completion (v1.0 turned off)
[Clients] ---> [Reverse Proxy / Nginx] ---> [Container v1.1 (Active)]

2. Architectural Taxonomy & Mental Model

Continuous deployment strategies:

1. Blue/Green Deployment:
   • Full duplication of the infrastructure layer.
   • Instant atomic traffic switching at the Nginx, Traefik, or DNS/ALB level.
   • Highest reliability, simplest rollback, but requires double the RAM and host resources.
2. Rolling Update:
   • Containers are updated sequentially one at a time or in groups.
   • Maintains constant cluster capacity (e.g., a minimum of 3 working replicas out of 4).
   • Resource-efficient, standard by default in Kubernetes and Docker Swarm.
3. Canary Release:
   • The new version receives a fixed micro-percentage of real traffic (e.g., 2-5%) or users from a specific internal group.
   • System metrics (Error Rate, Latency) are monitored. If no anomalies are detected, the traffic share is gradually increased to 100%.

3. Technical Pipeline & Internal Mechanics

Implementing Graceful Shutdown in Node.js / TypeScript

The process must correctly intercept operating system signals SIGTERM and SIGINT:

import express from "express";
import http from "http";

const app = express();
let isShuttingDown = false;

// Healthcheck endpoint for the orchestrator
app.get("/healthz", (req, res) => {
  if (isShuttingDown) {
    // Signal to the load balancer not to send new requests here
    return res.status(503).json({ status: "shutting_down" });
  }
  return res.status(200).json({ status: "healthy" });
});

const server = http.createServer(app);
server.listen(3000);

// Intercepting shutdown signal from Docker / systemd
process.on("SIGTERM", () => {
  console.log("SIGTERM received. Starting graceful shutdown...");
  isShuttingDown = true;

  // 1. Stop accepting new HTTP connections
  server.close(async () => {
    console.log("Closed all remaining HTTP connections.");

    try {
      // 2. Close database and Redis connection pools
      await dbPool.end();
      await redisClient.quit();
      console.log("Infrastructure connections closed. Exiting process.");
      process.exit(0);
    } catch (err) {
      console.error("Error during teardown:", err);
      process.exit(1);
    }
  });

  // 3. Fail-safe: force termination on socket hang
  setTimeout(() => {
    console.error("Forced termination: active connections timed out.");
    process.exit(1);
  }, 20000); // 20 seconds timeout
});

Docker Compose Healthcheck Configuration

To ensure traffic switching by the orchestrator, the health check parameter is configured:

services:
  api:
    image: my-company/api:v1.2.0
    restart: always
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/healthz"]
      interval: 5s
      timeout: 3s
      retries: 3
      start_period: 10s
    stop_grace_period: 30s # Time for Graceful Shutdown

4. Production Engineering Scenarios

01. Deployment via Coolify Without Traffic Interruption

The Coolify platform uses an embedded reverse proxy Traefik. When the Deploy button is pressed, it builds a new Docker image, starts a new container on a random port, probes its healthcheck endpoint, and only after confirming its operability, reconfigures Traefik routing on the fly, after which it sends a SIGTERM signal to the old replica.

02. Nginx Hot Configuration Reload (nginx -s reload)

When updating SSL certificates or adding new upstream configurations, the master Nginx process launches a new set of worker processes with the new configuration. Old workers stop accepting new connections, finish transferring active files to current clients, and only after that self-terminate without dropping a single packet.

03. Deploying Database Migrations Without Locking Tables in PostgreSQL

When adding indexes to large tables (millions of rows), the standard CREATE INDEX locks the table for writes (EXCLUSIVE LOCK), causing query queues to hang. Engineers use the CREATE INDEX CONCURRENTLY option, which builds the index in the background without blocking parallel read and write operations.

5. Pitfalls, Common Mistakes & Security

1. Lack of stop_grace_period in Docker: By default, Docker gives a container only 10 seconds to handle SIGTERM, after which it sends a fatal SIGKILL (immediate process termination). If a long request to an LLM or payment webhook takes 12 seconds, the operation will be cut off, resulting in data corruption. Increase the limit to 30-60 seconds.
2. Version Incompatibility Between Code and Database: Attempting to rename a table column (ALTER TABLE users RENAME COLUMN email TO contact_email) instantly breaks the old version of the code that continues to run during the parallel deploy. Always apply the Expand-and-Contract pattern in two separate releases.
3. False Positive Health Check Endpoints: If the /healthz endpoint checks the availability of all external third-party APIs (e.g., OpenAI or Twitter API) and they temporarily slow down, the orchestrator will consider its own service dead and enter an infinite restart loop (CrashLoopBackOff / Healthcheck Storm), destroying the operational production system. The liveness probe should check only the local process.