Building a Multithreaded Web Server in C++ with Docker

From a single-file POC to a production-grade, Dockerised service with Nginx

Vivek Bhadra  |  C++17 Docker Nginx POSIX Sockets

Introduction

Most tutorials on web servers show you a single-threaded accept loop and call it done. This post goes further. We will build a multithreaded HTTP server in C++ that handles concurrent connections using a thread pool, manages shared state safely with mutexes and condition variables, and packages the whole thing in Docker with Nginx as a reverse proxy sitting in front of it.

The project is called mt-webserver. It started as a teaching POC for a college internetworking course and evolved into something worth writing about properly.

What You Will Build

By the end of this post you will have a running C++ HTTP server that:

• Accepts concurrent connections using a thread pool of 4 worker threads
• Routes HTTP requests across 4 endpoints including a /slow endpoint to visually demonstrate concurrency
• Tracks live connections in a mutex-protected map, viewable at /connections
• Exposes a console thread for live management commands (list, stats, quit)
• Runs behind Nginx as a reverse proxy with rate limiting and security headers
• Ships as a two-container Docker Compose stack with health checks, resource limits and log rotation

Architecture Overview

Traffic flows from the browser through Nginx on port 80, which proxies internally to the C++ server on port 8080. Port 8080 is never exposed to the host; it lives only on the Docker bridge network.

  Browser / curl
        |
        v :80
  +-----------------------------+
  |  Nginx (reverse proxy)      |
  |  - Rate limiting 30 req/s   |
  |  - Security headers         |
  +-------------+---------------+
                | :8080 (internal only)
  +-------------v---------------+
  |  C++ Server                 |
  |  [Listener Thread]          |
  |       | push(fd)            |
  |  [SafeQueue]  <- condvar    |
  |       | pop(fd)             |
  |  [Worker Pool: 4 threads]   |
  |       |                     |
  |  [Connection Map] <- mutex  |
  |       ^                     |
  |  [Console Thread]           |
  +-----------------------------+

Docker network: webnet (bridge)
Port 8080 is internal only

Project Structure

The C++ code is split across six files following single-responsibility. One class per concern:

mt-webserver/
+-- src/
|   +-- main.cpp                   <- 4 lines: wire + start
|   +-- SafeQueue.hpp              <- header-only templated queue
|   +-- ConnectionManager.hpp/cpp  <- owns mutex + connection map
|   +-- RequestHandler.hpp/cpp     <- HTTP parse, routing, response
|   +-- ConsoleThread.hpp/cpp      <- stdin management commands
|   +-- WebServer.hpp/cpp          <- socket, thread pool, accept loop
+-- nginx/nginx.conf               <- conf.d drop-in
+-- Dockerfile                     <- multi-stage build
+-- docker-compose.yml             <- full stack definition
+-- Makefile

Class Design and Dependency Injection

The entire dependency graph is wired in main.cpp in four lines. Dependencies are passed as constructor references. No globals, no singletons, no shared_ptr overhead:

// main.cpp
ConnectionManager cm;
RequestHandler    rh(cm);
WebServer         ws(cm, rh);
ws.start();

WebServer owns the socket, thread pool, and SafeQueue. It constructs ConsoleThread internally since the console needs access to the queue and the running_ flag, both of which live inside WebServer.

The SafeQueue: Producer/Consumer Pattern

The SafeQueue<T> is the backbone of the concurrency model. The listener thread (producer) pushes accepted file descriptors onto it. Worker threads (consumers) block on pop() using a condition variable. They sleep when the queue is empty and wake up the instant work arrives:

bool pop(T &out, std::atomic<bool> &running)
{
    std::unique_lock<std::mutex> lk(mtx);
    cv.wait(lk, [&] { return !task_queue.empty() || !running; });
    if (task_queue.empty())
        return false;  // server shutting down
    out = std::move(task_queue.front());
    task_queue.pop();
    return true;
}

The condition variable serves two purposes: it wakes a worker when a new fd arrives, and it wakes all workers during shutdown so they can exit cleanly. This is why shutdown calls wake_all() before joining threads.

ConnectionManager: Owning Shared State

All shared mutable state lives exclusively in ConnectionManager. No other class holds a mutex directly. This is the single most important design decision for thread safety in the whole project.

void ConnectionManager::close_connection(int fd)
{
    shutdown(fd, SHUT_RDWR);   // drain both directions first
    ::close(fd);
    std::lock_guard<std::mutex> lk(conn_mutex);
    connections.erase(fd);
}

WebServer: The Accept Loop

The main thread becomes the listener thread once ws.start() is called. It uses select() with a 1-second timeout rather than blocking directly on accept(). This allows the loop to check the running_ flag periodically and shut down cleanly:

void WebServer::accept_loop()
{
    while (running_)
    {
        fd_set fds;
        FD_ZERO(&fds); FD_SET(server_fd_, &fds);
        struct timeval tv{ 1, 0 };
        if (select(server_fd_ + 1, &fds, nullptr, nullptr, &tv) <= 0)
            continue;  // timeout, check running_ and loop

        int client_fd = accept(server_fd_, ...);
        cm_.register_connection(client_fd, ip, port);
        queue_.push(client_fd);
    }
}

RequestHandler: HTTP Parsing and Routing

Reading the Full Request

A naive implementation calls recv() once and assumes it gets the entire request. This breaks when Nginx is in front since it sends headers across multiple TCP segments. The fix is to loop until the HTTP header terminator \r\n\r\n is found:

while (req.find("\r\n\r\n") == std::string::npos)
{
    ssize_t n = recv(fd, buf, sizeof(buf) - 1, 0);
    if (n <= 0) return "";
    req.append(buf, n);
    if (req.size() > 8192) return "";  // guard against oversized requests
}

Sending the Full Response

send() is not guaranteed to send all bytes in one call. We loop with MSG_NOSIGNAL to handle partial sends and to prevent a SIGPIPE crash if the client disconnects mid-send:

bool RequestHandler::send_all(int fd, const std::string &data)
{
    size_t sent = 0;
    while (sent < data.size())
    {
        ssize_t n = send(fd, data.c_str() + sent,
                         data.size() - sent, MSG_NOSIGNAL);
        if (n <= 0) return false;
        sent += n;
    }
    return true;
}

ConsoleThread: Live Management

The console thread runs on its own std::thread and reads from stdin. This lets you inspect and control the server while it handles live traffic:

make console     # docker attach mt-webserver
> list           # print all active connections
> stats          # queue depth + connection count
> quit           # graceful shutdown

When quit is entered, the console sets running_ = false and calls queue_.wake_all(). This unblocks all workers sitting on pop(), which then see the empty queue and running_ == false and exit. The main thread then join()s all of them for a clean shutdown.

Docker: Multi-Stage Build and Compose Stack

Multi-Stage Dockerfile

The Dockerfile uses two stages: a builder stage that compiles the binary with full build tools, and a minimal runtime stage that contains only the compiled binary. No compiler, no headers, no build tools in the final image:

# Stage 1: Builder
FROM ubuntu:22.04 AS builder
RUN apt-get install -y build-essential g++
COPY src/ .
RUN g++ -std=c++17 -O2 -pthread -Wall \
    main.cpp ConnectionManager.cpp RequestHandler.cpp \
    ConsoleThread.cpp WebServer.cpp -I . -o server

# Stage 2: Runtime (minimal)
FROM ubuntu:22.04 AS runtime
RUN groupadd -r appgroup && useradd -r -g appgroup appuser
COPY --from=builder /build/server .
USER appuser    # never run as root
EXPOSE 8080

Docker Compose

The Compose file defines two services on a shared bridge network. Port 8080 uses expose (internal only) rather than ports (public). Nginx is the only container with a public port, and it waits for the server health check to pass before starting:

server:
  expose: ["8080"]         # internal only
  stdin_open: true         # keeps console thread alive
  healthcheck:
    test: bash -c 'echo > /dev/tcp/localhost/8080'

nginx:
  ports: ["80:80"]         # only public port
  depends_on:
    server:
      condition: service_healthy
  volumes:
    - ./nginx:/etc/nginx/conf.d:ro

Getting Started

Prerequisites

• Docker and Docker Compose installed
• Port 80 free on your machine

Run It

git clone https://github.com/yourusername/mt-webserver
cd mt-webserver
cp .env.example .env
make up

Visit http://localhost. You should see the home page showing which worker thread handled your request.

Endpoints

Prove the Thread Pool Works

Open four browser tabs to http://localhost/slow simultaneously. All four should return in approximately 3 seconds, not 12. The thread pool runs them in parallel:

# Or from the terminal:
for i in {1..4}; do curl http://localhost/slow & done; wait

Key C++ Concepts Used

Industry Practices Applied

What’s Next

This is a POC, not a production HTTP server. Natural next steps if you want to take it further:

• CMake: replace the flat Makefile with a proper build system as the project grows
• TLS: terminate SSL at Nginx so the C++ server stays plain HTTP internally
• HTTP/1.1 keep-alive: currently every connection closes after one request; persistent connections would improve throughput significantly
• Dynamic thread pool sizing: adjust the pool at runtime based on queue depth
• Unit tests: ConnectionManager and RequestHandler are clean enough to test in isolation with Google Test

If you found this useful or spotted something worth improving, feel free to open an issue.