initialize / main file for server side operations like db connections

[lior-amsalem]

Goals

1. currently if you wanna do a global db connection you must perform the call to the db connection from each reuqest which is very server-less first in mind
2. some devs add the code to config.next which is bad
3. some devs store the instance as singleton on the global obj of nodejs which is fine. but it's called again and again from each router.

Non-Goals

Background

it's annoying to call db connection every http request (unless your code is serverless which makes sense

Proposal

create a file to initialize or add a way to do so in the next.config or add an entity file called "server.ts" ? that perform server side actions only.

another option is to create a pre/post files for layout for instance for actions before rendering start

[icyJoseph]

There's this experimental feature: https://nextjs.org/docs/app/building-your-application/optimizing/instrumentation

I am not sure if it is meant to be used to start-up DB connections though.

[bbangert]

It can, but it works oddly. The instrumentation.ts file is marked as a client component by Next.js, which means even if you do the if statement toggle to check for the nodejs runtime, everything imported (even dynamically) cannot import server-only as a client component isn't allowed to do that, and the import chain is considered client.

The Next.js team really needs to address this, for a framework pushing server-side rendering and data access this hard, its puzzling to me why the server-side story is so poorly thought out.

[gregbrowndev]

it's annoying to call db connection every http request (unless your code is serverless which makes sense

Some serverless environments like Google Cloud Functions (v2) now allow handling multiple concurrent requests within the same function instance. See https://cloud.google.com/functions/docs/concepts/version-comparison#new-in-2nd-gen

This is a description of the problem I'm facing (making it tooggable to prevent spamming the thread)

I'm finding it extremely difficult to work with NextJS as a full-stack framework. What I'm doing might be a bit niche, but I want to embed a GraphQL server into NextJS (via an app route) and then configure the GraphQL client to perform a standard HTTP request when CSR and an in-memory request when SSR. Both Apollo Client and URQL support this using the SchemaLink and ExecutorExchange, respectively. All of this works great until you need to initialise anything on the server that is async.

It gets tricky to configure the GraphQL client for the server environment without completely breaking the SSR of the page or leaking server-side code into the browser bundle. Both of these occur when the server requires some async bootstrapping, e.g. creating a DB connection, and when dynamically importing your executable schema (to prevent the code being bundled into the browser) which itself returns a Promise. Another approach I tried was to wrap the client in a server-action, but again this can only return a Promise.

This forces creating the Graphql client to become async which means you have to use Suspense or conditional rendering deep within your component tree (i.e. in the root layout) while you wait for the promise to resolve. This basically means the SSR render always shows the loading state and you would then have to use streaming SSR on all of your pages to get any use at all out of SSR.

It seems excessively difficult to separate logic within a component that can only run on the server. Data fetching isn't the only use case where you want to have server-only code. In this case, the NextJS server has some dependencies that need to be initialised before the rendering can start (when server-side).

The two use cases that need to be addressed to make SSR feasible are:

• initialising dependencies within the server when all the server starts, e.g. DB connection, loading environment variables, loading the executable schema, etc. These should be shared by concurrent requests.
• initialising server dependencies that are used per request, e.g. a DB session, authentication, creating any other dependencies, without forcing you to use Suspense/conditionally rendering.

The former use case seems like it would be solved using the instrumentation hook previously mentioned, but it doesn't seem like the NextJS pages can import anything initialised in that file. I've seen this same problem when trying to share dependencies across Jest test suites because they run in isolation in the event loop.

I guess the former isn't easy to solve as it basically requires React to allow you to wait for the promise to resolve before rendering anything (like it would be on a traditional server). Of course, this increases latency, but that seems better in some cases than being forced to stream the content after the dependencies are available.

[bbangert]

They can use things initialized from there, using a singleton hack. It's not elegant, but it does work.... we have a singleton like this:
https://github.com/mozilla/fxa/blob/main/libs/payments/ui/src/lib/utils/singleton.ts

That we then use like this:
https://github.com/mozilla/fxa/blob/main/libs/payments/ui/src/lib/nestapp/app.ts#L39

The Nest dependency loading system is used to create all the services and the appropriate dependency graph for the database connections, etc.

Perhaps I should make a larger blog post about our setup at some point.... 🤔

[gregbrowndev]

Mind blown... I've just learned that React server components can be marked async, which tells React to wait before rendering any content: https://demystifying-rsc.vercel.app/server-components/async/.

This seems like exactly what I needed 😅

I will give this a try first because it should be pretty easy now to proceed, but I'm definitely interested in reading the blog post about how you did it!

[bbangert]

The instrumentation.ts has been fixed to allow loading server-only code. Since our code-base uses NestJS apps and we develop our services as NestJS Services, we have a loader that makes them available in our Next app that is triggered at startup. Once the 'app' object for all the desired services are loaded, they're available in the Next pages via a singleton.

It's a bit annoying to have to use 2 experimental features, but it does work. We have to flag all the server-only modules using the serverComponentsExternalPackages feature to avoid their inclusion in the client-side bundle.