Document and improve how custom fields are used for contacts

[0x80]

I just wasted quite a bit of time trying to use custom fields in my contact details. The error message said I was using invalid ids and then listed all my field names. But because in the UI you do not see anything related to ids, I was assuming that field name and id are the same things.

So now I learned that I need to use an id, for which I need to call an endpoint because it is nowhere else to be found.

This is such an odd API design choice that I think it really should be clearly documented.

I do not see why the API endpoint would be unable to internally look up the id based on the field name. It seems like a bad design choice to expose your internals like this and bother the API consumer with it. If I'm missing something fantastic you can do because of these low-level ids, please tell me. And otherwise, please consider simplifying your API. But I realize this is probably not the place for those kinds of feature requests.

To make things worse, I have a dev and prod Sendgrid account. I can look up the ids, but I have to do this at runtime, because if I deploy the same code to production, the ids will be different depending on what order the custom fields were defined in 🤦

[0x80]

So I managed to make it work. To illustrate what I had to do to get there, and to maybe help anyone arriving here with the same question, see the code below.

Pretty much all of it is written because I can't refer to fields simply by their name. Now for every change in user contact details, besides making a call to the Sendgrid API via the Firestore extension, the system now also makes an additional call to the field definitions API.

Because this background function is triggered on every user change, I was planning to memoize it as you see in the code, but it won't work like this because cloud functions are ephemeral and so the runtime context is likely gone by the next invocation.

Also, the code is now not typed anymore as it would be when using field names directly. My compiler will not warn me if I forget to include a field here or make a typo.

So all in all, I think this API design is pretty horrible and I hope you find a way to improve it.

But the fact that you have field ids, suggests that it is possible to create multiple fields with the same name and different types, so the source of the problem might be pretty fundamental. For the API I suspect it would be fine to use the first id match on the field name that occurs and leave it to the user to define their custom fields without naming overlap.

export async function writeUserToSendgridContacts(userId: string, user: User) {
  if (user.isDeleted) {
    await db.collection("sendgridContacts").doc(userId).delete();
    return;
  }

  const fieldIdByName = await memGetFieldIdByName();

  const data: SendgridContact = {
    email: user.email,
    first_name: user.firstName || undefined,
    last_name: user.lastName || undefined,

    custom_fields: {
      [fieldIdByName["invite_link"]]: user.inviteLink || undefined,
      [fieldIdByName["airline"]]: user.airlineId || undefined,
      [fieldIdByName["homebase"]]: user.homebaseAirportId || undefined,
      [fieldIdByName["is_registered"]]: user.isRegistered ? "true" : "false",
      [fieldIdByName["has_enabled_promotional"]]: user.settings
        ?.emailNotifications?.promotional
        ? "true"
        : "false",
    },
    modified_at: FieldValue.serverTimestamp() as FirestoreTimestamp,
  };

  await db
    .collection("sendgridContacts")
    .doc(userId)
    /**
     * This will preserve the meta field injected by the extension. Not sure if we need to.
     */
    .set(data, { merge: true });
}

type CustomFieldDefinition = {
  id: string;
  name: string;
};

type FieldDefinitionsResponseBody = {
  custom_fields: CustomFieldDefinition[];
};

const memGetFieldIdByName = pMemoize(async () => {
  // eslint-disable-next-line
  const client = require("@sendgrid/client");
  client.setApiKey(runtimeConfig.sendgrid.api_key);

  const [{ statusCode }, body] = await client.request({
    url: `/v3/marketing/field_definitions`,
    method: "GET",
  });

  if (statusCode !== 200) {
    throw new Error(
      `Call to Sendgrid field definitions failed with code ${statusCode}: ${response}`,
    );
  }

  const result = (body as FieldDefinitionsResponseBody).custom_fields.reduce<
    Record<string, string>
  >((acc, v) => set(acc, v.name, v.id), {});

  return result;
});