What a New Connection Costs: Pools and Keep-Alive

Opening a new connection costs round trips before any useful work happens, and a connection that is reused avoids paying them again. Keep-alive, which leaves an HTTP connection open for further requests, and connection pools, which keep a set of open connections ready for a program to borrow, both exist to turn that setup cost into a one-time payment.

Picture a web service that opens a new database connection for every request. It is fast in testing, on one machine, and slow under load in production, with the database spending more time accepting connections than running queries.

A new connection pays in three ways.

Handshakes. One round trip for TCP, then one more for TLS 1.3, or two for TLS 1.2, before the request leaves. A database adds its own login, often more round trips and a password check.

A cold congestion window. A new connection starts in slow start, so a large response takes several round trips to reach full speed, where a warm connection has already grown its window.

Work at both ends. Each connection costs memory and, for a database, often a process or thread on the server. Thousands of short connections can exhaust the server before the queries do.

Reuse avoids all three. With keep-alive, a browser or HTTP client sends the next request on the connection it already has. With a pool, a program opens a few connections at start-up, borrows one for each query and returns it after, so every query skips the handshakes and the login. A pool also caps how many connections the program makes, which protects the database.

Reuse has its own rules. A pooled connection that sat idle may have been closed by the server or a firewall in between, so pools check connections or retire old ones. And a pool sized larger than the database can serve only moves the queue.

So when a service is slow at establishing connections rather than at its work, reuse them.