Optimistic Concurrency in an HTTP API with ETags & Hypermedia

How do you implement optimistic concurrency in an HTTP API? There are a couple of different ways, regardless of what datastore you’re using in the backend. You can leverage the ETag header in the HTTP Response to return a “version” of the resource that was accessed. When a client then needs to perform some operation on the resource, they send an If-Match header apart of the request with the value being the result of ETag from the initial GET request. Another option is to leverage hypermedia by returning URIs for actions relevant to a resource that include the version apart of the URI. This enables concurrency to be completely transparent and does not require any knowledge from the client.

YouTube

Check out my YouTube channel where I post all kinds of content that accompanies my posts including this video showing everything that is in this post.

Optimistic Concurrency

In a concurrent environment like a web application or HTTP API, you have multiple concurrent requests that could be trying to make state changes to the same resource. The normal flow for optimistic concurrency is that clients will specify the latest version they are aware of when attempting to make a state change. As an example, two clients make a request for an HTTP request of GET /products/abc123

The HTTP API returns the data along with a “version” property. The version indicates the current version of that resource. Now when a client makes a subsequent call to perform any type of action that’s going to result in a state change, it also includes the version. As an example, the client performs an inventory adjustment by making an HTTP call to POST /products/abc123/quantity but it also includes the version it received from the prior GET request.

Now if the second client, which also did a GET request and also had version 15 of the resource, makes a similar HTTP call to do an inventory adjustment. It also includes the version it has, which is 15. However since the first client has already made a successful state change, the version is now 16.

This request by the second client will fail because we’ve implemented optimistic concurrency. There are various ways you can implement ways beyond just using a version number, such as a DateTime or Timestamp that represents the last change or most recent version. Using a relational database, this will be implemented by including the version in the WHERE clause and then getting back the value of affected rows. If no rows were affected then the version isn’t what is the current value.

ETags & If-Match

Another way of passing the version around from server to client is by leveraging the ETag and If-Match headers in the HTTP Response and Request.

A good example of this implementation is with Azure CosmosDB. When you request a document, it will return an _etag property but also include the ETag header in the response. This represents the version of the resource.

Here is what the response body looks like:

{ "id": "1f95478a1-4ce4-4cdb-9808-9bb24218dd6f", "customerId": "CodeOpinion", "status": 1, "_etag": "000000000-0000-d4f0-9fc1acb501d7" }

Here are the headers from the response.

Since we now have the ETag value, we can now use it when making a subsequent request to perform a state change. To do so, the request must pass the If-Match header with the ETag value.

The Cosmos SDK uses this exactly as illustrated within its API.

using Microsoft.Azure.Cosmos;

using var client = new CosmosClient("AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="); var container = client.GetContainer("demo", "orders"); var newOrder = new Order { Id = Guid.NewGuid().ToString(), CustomerId = "CodeOpinion" };

await container.CreateItemAsync(newOrder, new PartitionKey(newOrder.Id));

var readOrder = await container.ReadItemAsync<Order>(newOrder.Id, new PartitionKey(newOrder.Id)); readOrder.Resource.Status = OrderStatus.Processing;

// Works because ETag is correct await container.UpsertItemAsync(readOrder.Resource, new PartitionKey(readOrder.Resource.Id), new ItemRequestOptions { IfMatchEtag = readOrder.ETag, });

// Fails because ETag was changed when Upsert occured above. await container.UpsertItemAsync(readOrder.Resource, new PartitionKey(readOrder.Resource.Id), new ItemRequestOptions { IfMatchEtag = readOrder.ETag, });

Hypermedia

Another way to pass around the version is simply by using Hypermedia. Hypermedia is about providing the client with information about what other actions or resources are available based on the resources it’s accessing. This means when a client requests the product resource GET /products/abc123, the server will provide it the URI to where it can do an Inventory Adjustment. Since we’re providing the URI, we can include the current version in the URI.

using EventSourcing.Demo; using EventStore.ClientAPI.Exceptions; using Microsoft.AspNetCore.Mvc;

var builder = WebApplication.CreateBuilder(args); var app = builder.Build();

app.MapGet("/products/{sku}", async (HttpResponse response, string sku) => { using var stream = await WarehouseProductEventStoreStream.Factory(); var product = await stream.Get(sku);

return new { Sku = product.Aggregate.Sku, Quantity = product.Aggregate.GetQuantityOnHand(), Version = product.Version, Commands = new Command[] { new("InventoryAdjustment", $"/products/{sku}/{product.Version}/adjustment") } }; });

app.MapPost("/products/{sku}/{version}/adjustment", async (HttpResponse response, [FromRoute]string sku, [FromRoute]long version, [FromBody]InventoryAdjustment inventoryAdjustment) => { using var stream = await WarehouseProductEventStoreStream.Factory(); var product = await stream.Get(sku); product.Aggregate.AdjustInventory(inventoryAdjustment.Quantity, inventoryAdjustment.Reason); try { await stream.Save(product.Aggregate, version); } catch (WrongExpectedVersionException) { response.StatusCode = 412; } });

app.Run();

In the example above, I’m using EventStoreDB, which allows optimistic concurrency by passing the current version when appending an event to the event stream. If the version passed is not the current version of the stream, a WrongExpectedVersionException is thrown.

Here’s an example of what the response body looks like when calling GET /products/abc123

{ "sku": "abc123", "quantity": 40, "version": 3, "commands": [ { "action": "InventoryAdjustment", "uri": "/products/abc123/3/adjustment" } ] }

When we want to do an Inventory Adjustment, we aren’t constructing a URI, we simply use the response from the GET and find the URI in the commands array.