Experimental NextJS app-dir cacheTag to include ctx?

[dalechyn]

TLDR

ctx in queries is often being used as inputs when some are defined in the middlewares.

This is not taken in account with experimental app dir support, as the cacheTag only caches the inputs, but never the context.

Therefore, I am suffering a bug when a query that just uses ctx to retrieve user-specific data from ctx.session, is being cached with no inputs [], and I am able to see the cached data of another user.

Introduction

Experimental tRPC's NextJS AppRouter layout support involves caching of query inputs, and is implemented in the next way:

trpc/packages/next/src/app-dir/links/nextCache.ts

Lines 34 to 43 in b72c7aa

	const { path, input, type, context } = op;
	const cacheTag = generateCacheTag(path, input);
	// Let per-request revalidate override global revalidate
	const requestRevalidate =
	typeof context['revalidate'] === 'number' ||
	context['revalidate'] === false
	? context['revalidate']
	: undefined;
	const revalidate = requestRevalidate ?? opts.revalidate ?? false;

Problem

While it works for most of the cases, problems arise when your query logic depends on trpc's middlewares.

Here's an example implementation of a protected procedure taken from the tRPC's documentation:

import { initTRPC, TRPCError } from '@trpc/server';

export const t = initTRPC.context<Context>().create();

// you can reuse this for any procedure
export const protectedProcedure = t.procedure.use(async function isAuthed(
  opts,
) {
  const { ctx } = opts;
  // `ctx.user` is nullable
  if (!ctx.user) {
    //     ^?
    throw new TRPCError({ code: 'UNAUTHORIZED' });
  }

  return opts.next({
    ctx: {
      // ✅ user value is known to be non-null now
      user: ctx.user,
      // ^?
    },
  });
});

t.router({
  // this is accessible for everyone
  hello: t.procedure
    .input(z.string().nullish())
    .query((opts) => `hello ${opts.input ?? opts.ctx.user?.name ?? 'world'}`),
  admin: t.router({
    // this is accessible only to admins
    secret: protectedProcedure.query((opts) => {
      return {
        secret: 'sauce',
      };
    }),
  }),
});

As you can see, the ctx.user?.name in this example is implicitly becoming an input, because you use it in the query implementation, yet it's not being included in the cacheKey!

In this particular case, which is taken from the docs, the consequence is that the first user querying the hello query, would make it cache the response hello alice (i.e), then when the second user queries the same procedure (bob i.e.), he would see hello alice as it was cached with a cacheKey of [] as inputs to the functions simply don't exist.

Workaround

I was forced to always pass id of the user, and compare the input in the protectedProcedure's middleware with the value retrieved from the context, to enforce caching. But this is really awful as it forces me to always have the same parameter name in all the queries so that getRawInput doesn't perform a check against undefined.

Proposed Solution

Allow a user to explicitly state the cacheKey inputs.

State the cache key in the query

The revamped docs example could look like this:

import { initTRPC, TRPCError } from '@trpc/server';

export const t = initTRPC.context<Context>().create();

// you can reuse this for any procedure
export const protectedProcedure = t.procedure.use(async function isAuthed(
  opts,
) {
  const { ctx } = opts;
  // `ctx.user` is nullable
  if (!ctx.user) {
    //     ^?
    throw new TRPCError({ code: 'UNAUTHORIZED' });
  }

  return opts.next({
    ctx: {
      // ✅ user value is known to be non-null now
      user: ctx.user,
      // ^?
    },
  });
});

t.router({
  // this is accessible for everyone
  hello: t.procedure
    .input(z.string().nullish())
    .query((opts) => `hello ${opts.input ?? opts.ctx.user?.name ?? 'world'}`)
     // explicitly state data to use as `cacheKey`
    .cacheKey((opts) => [opts.input, opts.ctx.user?.name]),
  admin: t.router({
    // this is accessible only to admins
    secret: protectedProcedure.query((opts) => {
      return {
        secret: 'sauce',
      };
    }),
  }),
});

State the cache key in the middleware

The revamped docs example could look like this:

import { initTRPC, TRPCError } from '@trpc/server';

export const t = initTRPC.context<Context>().create();

// you can reuse this for any procedure
export const protectedProcedure = t.procedure.use(async function isAuthed(
  opts,
) {
  const { ctx } = opts;
  // `ctx.user` is nullable
  if (!ctx.user) {
    //     ^?
    throw new TRPCError({ code: 'UNAUTHORIZED' });
  }

  return opts.next({
    ctx: {
      // ✅ user value is known to be non-null now
      user: ctx.user,
      // ^?
    },
    // state which values from the context you want to cache
    cacheKey: (ctx) => ([ctx.user])
  });
});

t.router({
  // this is accessible for everyone
  hello: t.procedure
    .input(z.string().nullish())
    .query((opts) => `hello ${opts.input ?? opts.ctx.user?.name ?? 'world'}`)
  admin: t.router({
    // this is accessible only to admins
    secret: protectedProcedure.query((opts) => {
      return {
        secret: 'sauce',
      };
    }),
  }),
});

If you're read till the end, you're the hero, thank you.

[dalechyn]

Okay, here's another FAR simpler idea.

Let the user define the context values to cache inside of experimental_nextCacheLink.

Going to try this out on my specific setup, but similar has to be done in all the places where generateCacheTag is used.

type NextCacheLinkOptions<TRouter extends AnyRouter> = {
  router: TRouter;
  createContext: () => Promise<inferRouterContext<TRouter>>;
+  /** define values from the context that must be passed to `generateCacheTag` */
+ cacheContext?: (ctx: inferRouterContext<TRouter>) => any[];
  /** how many seconds the cache should hold before revalidating */
  revalidate?: number | false;
} & TransformerOptions<inferRootTypes<TRouter>>;

// ts-prune-ignore-next
export function experimental_nextCacheLink<TRouter extends AnyRouter>(
  opts: NextCacheLinkOptions<TRouter>,
): TRPCLink<TRouter> {
  const transformer = getTransformer(opts.transformer);
  return () =>
    ({ op }) =>
      observable((observer) => {
        const { path, input, type, context } = op;

-        const cacheTag = generateCacheTag(path, input);
-        // Let per-request revalidate override global revalidate
-        const requestRevalidate =
-          typeof context['revalidate'] === 'number' ||
-          context['revalidate'] === false
-            ? context['revalidate']
-            : undefined;
-        const revalidate = requestRevalidate ?? opts.revalidate ?? false;

        const promise = opts
          .createContext()
          .then(async (ctx) => {
+           const cacheTag = generateCacheTag(path, input, opts.cacheContext?.(ctx));
+           // Let per-request revalidate override global revalidate
+           const requestRevalidate =
+             typeof context['revalidate'] === 'number' ||
+             context['revalidate'] === false
+               ? context['revalidate']
+               : undefined;
+           const revalidate = requestRevalidate ?? opts.revalidate ?? false;
            const callProc = async (_cachebuster: string) => {
              //   // _cachebuster is not used by us but to make sure
              //   // that calls with different tags are properly separated
              //   // @link https://github.com/trpc/trpc/issues/4622
              const procedureResult = await callProcedure({
                procedures: opts.router._def.procedures,
                path,
                getRawInput: async () => input,
                ctx: ctx,
                type,
              });

And the generateCacheTag in shared.ts:

-export function generateCacheTag(procedurePath: string, input: any, context?: any) {
-  return input
-    ? `${procedurePath}?input=${JSON.stringify(input)}&context=${JSON.stringify(context)}`
-    : procedurePath;
-}
+export function generateCacheTag(procedurePath: string, input: any, context?: any) {
+  return `${procedurePath}?input=${JSON.stringify(input ?? {})}&context=${JSON.stringify(context ?? {})}`
+}

[dalechyn]

Figured out that it's working and changes the cache tag but it might be too long and have >256 characters.

I unfortunately cannot easily add a hashing function here, but I would consider seeing @noble/hashes sha256 hashing of the inputs, but it may look like this:

import { sha256 } from '@noble/hashes/sha256';
export function generateCacheTag(procedurePath: string, input: any, context?: any) {
  return `${procedurePath}?hash=${sha256(`input=${JSON.stringify(input ?? {})}&context=${JSON.stringify(context ?? {})}`})`
}

Such shouldn't affect the bundle size too much as the library is tree-shakeable.

[dalechyn]

Moved to #5455