200 Response for "null"

[conraddit]

Hello Team,

Is it normal for "null" queries to be delivered as OK 200?

On our side, we are trying to cache Kupo requests, so that we can take load off of Kupo for repetitive queries. However, some requests return "null" as a 200 OK response, which results in "null" being stored in our cache.

Would it be possible for Kupo to return any other value, other than 200, for all "null" responses, so that we could cache only 200 responses?

Thank you!

[nielstron]

I assume you are talking about datums.

From my personal experience, an alternative is retruning 404 if a datum hash can not be resolved (I.e. on blockfrost). Personally I find this a much worse user experience than 200+null, because you can effectively not distinguish between putting a wrong URI, other mistakes, or a valid response where no resolution was found

BTW stupid suggestion: If you can decide to not cache non-200, can you also not cache null?

[conraddit]

That's correct. We're querying for a specific datum on Kupo, it's not yet on-chain, so it returns a 200 "null." Our caching system (HAproxy) detects that the answer was valid (200) and caches it. Next time we try that same datum, which might now be on-chain, our caching system doesn't know that it should request again, as it believes that "null" was a valid answer.

The solution, I believe, would be for Kupo to return any other API feedback, rather than 200 as it has been now.

[conraddit]

Also, for additional context, this is what HAProxy caches: https://www.haproxy.com/documentation/hapee/latest/configuration/config-sections/cache/, and the conditions within which it caches. Ideally, null responses would not return a 200 code, simply to avoid caching.
Does this make sense?

[nemo83]

Hello Team,

Is it normal for "null" queries to be delivered as OK 200?

On our side, we are trying to cache Kupo requests, so that we can take load off of Kupo for repetitive queries. However, some requests return "null" as a 200 OK response, which results in "null" being stored in our cache.

Would it be possible for Kupo to return any other value, other than 200, for all "null" responses, so that we could cache only 200 responses?

Thank you!

Hey Conrad, beside the 200+null issue, for all the other 200, is the content immutable right? So they should be able to be safely cached?

[conraddit]

Hey @nemo83, as far as I know, yes.

[KtorZ]

Content is actually NOT immutable; it's a blockchain. It's only eventually immutable. There's a section of the documentation which goes deeper into this explanation.

All-in-all, Kupo provides a ETag as part of every response (including and in particular HEAD requests). For a same URI, a same ETag means that responses are identical. Clients can use the If-None-Match request header to ask only responses that differ from a previous point. This is a relatively standard way of providing caching through HTTP and I would recommend to use it.

In the end, there's nothing wrong from an HTTP standpoint when fetching a a datum that doesn't exist. Hence why the answer is 200 despite the resource being null. Returning 404 in that case would make it impossible to distinguish from a genuinely wrong URI.

[nemo83]

Thanks both, it makes sense.

[koslambrou]

Just encountered this behaviour. However, the behaviour seems inconsistent. For some datum hash queries, it returns 404 Not Found, while in other cases it returns 200 Ok with null as the response body.

For example, in current preprod, I can run

http://localhost:3002/datums/e05f7af2ad81014f5049059163c0c5634b3b489f6ce629e134bb4fbe4e723fd5

which returns null as the response body with 200 Ok.

However, when altering the datum hash to something like e05f7af2ad81014f5049059163c0c5634b3b489f6ce629e134bb4fbe4e723fd6 (change last digit to 6), it returns 404 Not Found.

As a user, I would expect the behaviour to be consistent.

[KtorZ]

Are you sure that that last counter-example? The endpoint shall never return 404 actually; 404 is only returned for non-matching endpoints. Which might happen if you misspell something or perhaps if the datum doesn't properly parse (although the latter may be confusing).

e05f7af2ad81014f5049059163c0c5634b3b489f6ce629e134bb4fbe4e723fd6 does look like a valid datum, so this should parse just fine. Which is why I am surprised.

[koslambrou]

Mmm.. just checked again, and couldn't reproduce this. I think you are right, and I probably misspelled something. I can confirm that both valid datums return 200 OK