GitHub Repository

​
Dependencies

  • java.net.http
  • software.sava.json.iterator

​
Configuration

Configuration parsers are provided for each component for the purpose of allowing users to configure the service.

​
Primitives

​
Time

Durations/windows/delays are ISO-8601 duration formatted PnDTnHnMn.nS. The PT prefix may be omitted.

​
Backoff

Backoff strategy in response to I/O errors.

​
Backoff Configuration

  • type: single, linear, fibonacci, exponential
  • initialRetryDelay
  • maxRetryDelay

​
Backoff Usage

var jsonConfig = "null";
var ji = JsonIterator.parse(jsonConfig);
var backoffConfig = BackoffConfig.parseConfig(ji);
var backoff = backoffConfig == null
    ? Backoff.fibonacci(1, 13)
    : backoffConfig.createBackoff();

for (long errorCount = 0; ; ) {
    try {
        // some I/O task.
        break;
    } catch (Exception e) {
        System.err.format("Failed to do something %d time(s)%n", ++errorCount);
        final long sleep = backoff.delay(errorCount, TimeUnit.MILLISECONDS);
        Thread.sleep(sleep);
    }
}

​
Request Capacity Monitor

Tracks request capacity available to a resource. Intended to be used with Calls to potentially wait until capacity is available before making a request to avoid rate-limiting issues. Capacity is reduced in a weighted fashion per call, and also potentially when response errors are observed.

​
Capacity Configuration

  • maxCapacity: Maximum requests that can be made within resetDuration
  • resetDuration: maxCapacity is added over the course of this duration.
  • minCapacityDuration: Maximum time before capacity should recover to zero, given that no additional failures happen.
  • maxGroupedErrorResponses:
  • maxGroupedErrorExpiration:
  • tooManyErrorsBackoffDuration:
  • serverErrorBackOffDuration: Reduce capacity if remove server errors are observed.
  • rateLimitedBackOffDuration: Reduce capacity if rate limit errors are observed.

​
Capacity Construction

var jsonConfig = "null";
var ji = JsonIterator.parse(jsonConfig);
var capacityConfig = CapacityConfig.parse(ji);
if (capacityConfig == null) {
    // 10 requests per second with a maximum complete backoff of 5 seconds.
    capacityConfig = CapacityConfig.createSimpleConfig(
        Duration.ofSeconds(5), // minCapacityDuration
        10, // maxCapacity
        Duration.ofSeconds(1) // resetDuration
    );
}

// Custom error tracker's may be implemented to handle responses other than HttpResponse<byte[]>.
ErrorTrackedCapacityMonitor<HttpResponse<byte[]>> capacityMonitor = capacityConfig
    .createHttpResponseMonitor("service_name");

​
Load Balancer

Somewhat opinionated generic load balancers intended to be used with Calls.

​
Error Count Backoff

Skips items proportional to how many errors it has. How many times an item is skipped is tracked so that it can move back up the queue to be re-tried.

​
Sorted

Sorts by error count and then sample call time. Must be driven with requests using nextNoSkip to sample request performance.

​
Single

Wrapper around a single item.

​
Load Balancer Configuration

  • defaultCapacity
  • defaultBackoff
  • endpoints: Array of remote resources.
    • url
    • capacity: Overrides defaultCapacity
    • backoff: Overrides defaultBackoff

​
Load Balancer Construction

var jsonConfig = "";
var ji = JsonIterator.parse(jsonConfig);
var loadBalancerConfig = LoadBalancerConfig.parse(ji);

var items = loadBalancerConfig.createItems((endpoint, capacityMonitor, backoff) -> {
    final var client = null; // Construct the client to your remote resource.
    return BalancedItem.createItem(client, capacityMonitor, backoff);
});
var loadBalancer = LoadBalancer.createSortedBalancer(items);

​
Remote Call

Facilitates retrying remote calls, using backoff strategies to delay between failures.

If the backoff strategy returns a negative number or the error count exceeds the max retries for the call context, the exception is re-thrown wrapped in a RuntimeException.

​
Composed

Simple retry logic, does not track any corresponding request capacity for the given resource.

​
Greedy

Records the consumption of request capacity, but does not wait until available.

​
Courteous

Waits until the resource has enough capacity, up until maxTryClaim attempts, after which it will either give up and return null or force the call.

​
Remote Call Usage

ErrorTrackedCapacityMonitor<HttpResponse<byte[]>> capacityMonitor;
var capacityState = capacityMonitor.capacityState();
var backoff = Backoff.exponential(1, 16);
var callContext = CallContext.createContext(
    2, // callWeight
    1  // minCapacity
);
var httpRequest = HttpRequest.newBuilder(URI.create("https://api.domain.me/endpoint")).GET().build();

try (var executor = Executors.newVirtualThreadPerTaskExecutor();
     var httpClient = HttpClient.newHttpClient()) {
  
    var callFuture = Call.createCourteousCall(
        () -> httpClient.sendAsync(httpRequest, HttpResponse.BodyHandlers.ofString()),
        capacityState,
        callContext,
        backoff,
        "retry log context"
    ).async(executor);
    
    var result = callFuture.join();
}

​
WebHook Client

Generic client to POST JSON messages.

​
WebHook Configuration

[
  {
    "endpoint": "https://hooks.slack.com/services/...",
    "provider": "SLACK",
    "capacity": {
      "maxCapacity": 2,
      "resetDuration": "1S",
      "minCapacityDuration": "8S"
    }
  }
]
[
  {
    "endpoint": "https://hooks.slack.com/services/...",
    "bodyFormat": "{\"text\":\"%s\"}",
    "capacity": {
      "maxCapacity": 2,
      "resetDuration": "1S",
      "minCapacityDuration": "8S"
    }
  }
]

​
WebHook Client Usage

var defaultRequestCapacity = CapacityConfig.createSimpleConfig(
    Duration.ofSeconds(13),
    2,
    Duration.ofSeconds(1)
);
var defaultBackoff = Backoff.fibonacci(1, 21);

var jsonConfig = "";
var ji = JsonIterator.parse(jsonConfig);

var webHookConfigList = WebHookConfig.parseConfigs(
    ji,
    null, // default body format
    defaultRequestCapacity,
    defaultBackoff
);

var httpClient = HttpClient.newHttpClient();
var webHookClients = webHookConfigList.stream()
    .map(webHookConfig -> webHookConfig.createCaller(httpClient))
    .toList();

var callContext = CallContext.createContext(
    1, // Call weight.
    0, // Minimum capacity needed.
    8, // Maximum times to try to claim capacity.
    true, // Eventually force the call.
    5, // Maximum retries.
    false // Measure call time.
);

var executorService = Executors.newVirtualThreadPerTaskExecutor();

var msg = "Something happened!";

for (var webHookClient : webHookClients) {
    var responseFuture = caller.createCourteousCall(
        webHookClient -> webHookClient.postMsg(msg),
        callContext,
        "webHookClient::postMsg"
    ).async(executorService);
}