How to keep API idempotent while receiving multiple requests with the same ID at the same time?
From many articles and commercial APIs I've seen, most people make their APIs idempotent by asking the client to provide a requestId or idempotent key (e.g. https://www.masteringmodernpayments.com/blog/idempotent-stripe-requests ) and basically store a map of the request ↔ in the store. Therefore, if there is a request that is already present on this map, the application will simply return the saved response.
This is all well and good for me, but my problem is how do I handle the case where the second call arrives during the first call is still in progress?
So here are my questions
-
I think the ideal behavior would be a second call waiting for the first call to complete and the first call answer to return? Is this how people do it?
-
if so, how long will the second call wait for the first call to complete?
-
if the second call has a timeout limit and the first call hasn't finished yet, what should it tell the client? Should it just not return any responses, so the client will time out and try again?
source to share
For wunderlist, we use database constraints to make sure the query id (which is a column in each of our tables) is never used twice. Since our database technology (postgres) ensures that it would be impossible to insert two records that violate this constraint, we only need to properly respond to a potential insert error. Basically, we transfer this detail to our data warehouse.
I would recommend, whatever you do, try not to coordinate your work. If you try to find out if two things are happening at once, then there is a high probability that there will be errors. Instead, there may be a system that you already use that can make the guarantees you need.
Now, to specifically address your three questions:
- For us, since we are using database constraints, the database processes things in the queue and waits. This is why I personally prefer older SQL databases - not for SQL or relationships, but because they are really good at locking and queuing. We use SQL databases as dumb disconnected tables.
- This is highly dependent on your system. We are trying to set all our timeouts to about 1s on every system and subsystem. We would rather fail than queue. You can measure and then look at your 99th percentile for timings and just set that as your timeout if you don't know ahead of time.
- We will return the client's 504 http status (and the corresponding response body). The reason for having an idempotent key is that the client can retry the request, so we never worry about taking out the time and letting them do it. Again, we'd rather get things done quickly and quickly than let things stand in line. If things are queued, then even after fixing something, you need to wait a bit for things to get better.
source to share
It's a little tricky to know if the second call is the same client with the same request token or a different client.
Typically in the case of concurrent requests from different clients running on the same resource, you also want to implement a versioning strategy alongside the request token for idempotency.
A typical version strategy in a relational database might be a version column with a trigger that automatically increments the number every time a record is updated.
However, all clients must indicate their request token, as well as the version they are updating (for this, the IfMatch header is used, and the version number is used as the ETag value).
On the server side, when it's time to update the resource state, first check if the version number in the database matches the shipped version in ETag. If so, you record changes and version increments. Assuming the second request runs with the same version number as the first, it will fail with 412 (or 409 depending on how you interpret the HTTP specs) and the client should not try again.
If you really want to close the second request immediately when the first request is made, you are going down the pessimistic blocking route, which works well for REST APIs.
In the case where you are actually talking about the client retrying with the same request token because it received a temporary network error, this is almost the same case.
Both requests will be fired at the same time, the second request will be fired because the first request has not completed yet and has not yet written the request token to the database, but either way, the first successful completion is completed and the request token is written.
For another request, it will receive a version conflict (since the first query increased the version), at this point it should recheck the query token database table, find its own token there and assume that it was a parallel query that finished before and returned 200.
It looks like a lot, but if you want to cover all of the weird and beautiful failure modes when you are working with REST, idempotency and concurrency, this is the way to handle it.
source to share