Timeouts and Circuit Breakers for Remote Calls

Your endpoint is only as fast as the slowest service it calls, unless you set a limit.

When an endpoint calls another service over the network, the call's duration becomes part of the endpoint's own. Two tools put a limit on that: a timeout, which gives up on a single call after a set time, and a circuit breaker, which stops calling a service that keeps failing.

A ride-matching endpoint calls a pricing service owned by another team. The pricing service has nothing to do with your database, your code or your deployment, yet if it takes ten seconds to answer, your endpoint takes ten seconds too. By default the HTTP client waits as long as the other side takes, which during an outage can mean indefinitely.

Setting a timeout on the RestTemplate bean that makes the calls:

java

@Bean
RestTemplate restTemplate(RestTemplateBuilder builder) {
    return builder
        .setConnectTimeout(Duration.ofSeconds(2))
        .setReadTimeout(Duration.ofSeconds(3))
        .build();
}

The connect timeout limits how long to wait for a connection, and the read timeout how long to wait for an answer once connected. After that the call fails with an exception your code can handle, for instance by showing a standard fare, instead of hanging.

A circuit breaker, such as the one in the Resilience4j library, goes further: after enough recent failures it stops calling the service for a while and fails at once, instead of piling up slow, doomed requests. Neither tool fixes the pricing service; both stop its problem from becoming yours at the same scale.

So every call to a service you do not own needs a timeout at the very least, chosen from how long your own endpoint can afford to wait.

Spring Boot
Performance

← → move through sections and cards