Client-requested cut-off timeout

[ibnesayeed]

In applications where a quicker "good enough" response is more important than accurate and complete aggregation (which would take longer), the client might want to tell the aggregator a cut-off time after which any successful response (accumulated so far) is processed and terminated. The server will only wait longer than the cut-off time if no successful response is arrived from any upstream archive yet. In that case it would respond to the client as soon as the first successful response is arrived.

[ibnesayeed]

We need to investigate the means of how the client will convey the cut-off time. Possibly a request header would be utilized. We can try to find if there is one such header in practice that we can reuse or else we will have to introduce one.

[machawk1]

RFC7240 (Prefer header) has a defined preference for wait (Section 4.3) that might be useful for this.

[ibnesayeed]

As per the definition:

wait: Indicates an upper bound to the length of time the client expects it will take the server to process the request once it has been received.

It looks like utilizing wait preference of the prefer header could be a great choice.

Prefer: wait=10

[machawk1]

respond-async also seems like it would be useful for a client to specify not minding how long a response takes, so long as it's comprehensive of all known (to MemGator config) archives.

[ibnesayeed]

The respond-async preference is a cool idea, but not directly solving the problem described in this issue. One way to implement asynchronous response would be to return a 202 Accepted response quickly with a URL where the client can check back later if the response is ready. Services that utilize server-side caching can implement it easily. This could be useful in applications like CarbonDate, MementoDamage, and Warrick etc.