[RFC] URLSearchParams array support

[franky47]

Search params can appear more than once in the query string, like so: ?foo=bar&foo=baz

This reflects in URLSearchParams under a different API:

• getAll to obtain an array of values: ['bar', 'baz']
• get would only return the first entry (specification): 'bar'
• append does what it says, and adds a new entry when updating (eg: .append('foo', 'qux') will give ?foo=bar&foo=baz&foo=qux)
• set overrides any existing value (eg: .set('foo', 'qux') will give ?foo=qux)

In order to support this, some refactoring of the parsers, hook logic, and type definitions needs to be done, as it only ever worked with one-dimensional values.

Related discussions

• Array querystring output customization. #427

Originally posted by @ianldgs in #481 (comment)

If you have some feedback on a good API for this, I'd love to discuss more about it.

Perhaps a good API that woudn't require a lot of refactoring is:

type StringParser<T> = {
    parse: (value: string) => T | null;
    serialize?: (value: T) => string;
}
type NativeArrayParser<T> = {
    // inspired on `URLSearchParams.prototype.getAll`
    parseAll: (values: string[]) => T[] | null;
    serializeAll?: (values: T[]) => string[];
}

type Parser = StringParser<T> | NativeArrayParser<T>;

Then:

const parseNativeArray = (subParser: StringParser<T>) => {
  parseAll(values: string[]) => values.map(value => subParser.parse(value)),
  serializeAll(values: T[]) => values.map(value => subParser.serialize(value)),
}

useQueryState("id", parseNativeArray(parseAsString));

[franky47]

Coming back to this discussion: I think the behaviour needs to be strongly described as to whether the hook works with a single value or if it returns arrays.

Having an either-or case that handles everything, where the output type is T | T[] is the worst case scenario, as it makes it very difficult to consume values. In most cases, applications will have a strong opinion on whether a particular key is allowed to be repeated or not.

[tylersayshi]

Would it be possible to have an option on the parseArrayOf parser for checking for this multiple key scenario?

I'd expect it to serialize with the comma separated list still, but it would be nice to have an easy way to read in this multiple key scenario for backwards compatibility.

Or would middleware be the better place to own this sort of logic somehow?

[franky47]

The issue is not the parser itself, but the createParser which gives only a single value to the parsers:

nuqs/packages/nuqs/src/parsers.ts

Lines 104 to 111 in 4db0a10

	if (Array.isArray(value)) {
	// Follow the spec:
	// https://url.spec.whatwg.org/#dom-urlsearchparams-get
	if (value[0] === undefined) {
	return null
	}
	str = value[0]
	}

Middleware would help to convert ?foo=bar&foo=baz into ?foo=bar,baz, then any write would use the comma-separated convention: ?foo=new,state.

There are a couple of cases here:

1. Supporting ?foo=a&foo=b on read only (ie: convert to comma via middleware and use that for writes, for existing links)
2. Supporting ?foo=a&foo=b for both read & write (ie: using a server that expects this kind of formatting)

[jfsiii]

Just as another datapoint, this is a blocker to adoption in our (Remix) project. We use ?foo=a&foo=b and params.getAll('foo') in many places.

Part of the reason we use this pattern is because it's the default web/browser behavior for a form with input type="checkbox"es.

Clicking 3 boxes in a form like below and hitting submit will result in ?language=javascript&language=perl&language=php

Basic form with checkboxes

<form>
  <div>
    <label
      >JavaScript
      <input type="checkbox" name="language" value="javascript" checked
    /></label>
  </div>
  <div>
    <label>Perl <input type="checkbox" name="language" value="perl" /></label>
  </div>
  <div>
    <label>PHP <input type="checkbox" name="language" value="php" /></label>
  </div>
  <div>
    <label>Ruby <input type="checkbox" name="language" value="ruby" /></label>
  </div>
  <div>
    <input type="submit" name="submit" value="Submit" />
  </div>
</form>

I think it's important to support this long-standing web behavior

[tomasmenezes]

Also looking forward to this! It would definitely enhance out-of-the-box support for more readable complex queries such as: ?date=gte:10-02-2024&date=neq:15-02-2024 vs parsing everything as json, i.e., ?filters=[{date:{gte:10-02-2024}},{date:{neq:15-02-2024}}] or similar.

[fileformat]

I would definitely prefer option 2 (i.e. multiple query string params do not get converted to single param with commas).

I need to have the querystring set by nuqs match the output of a GET form submit.

[tylersayshi]

@franky47 if the desired behavior is split on this, maybe it could just be configurable via a prop in the NuqsAdapter?

[franky47]

Poll time, what would you think of an API based on parseAsArrayOf?

The idea would be to keep the same API we currently have for declaring array states, but add some indication to use the native key-duplication rather than a single key and separator for values:

// Option A (fewer changes, might be tricky to auto-complete right):
parseAsArrayOf(parseAsString, ';') // key=foo;bar
parseAsArrayOf(parseAsString, { nativeUrlArray: true }) // key=foo&key=bar

// Option B (more userland changes required, but more explicit):
parseAsArrayOf(parseAsString, { repeatKeys: false, separator: ';' }) // key=foo;bar
parseAsArrayOf(parseAsString, { repeatKeys: true }) // key=foo&key=bar

Feel free to suggest better names for nativeUrlArray and repeatKeys.

[SalahAdDin]

With debounce being mostly out of the door (see #900, feedback welcome), the next two big points on my internal backlog are this native array support and validation (see #446). Since validation might imply breaking (behaviour) changes, I'll focus on arrays first.

debounce out of the box? cool, we were doing it by ourselves.

[tomasmenezes]

Great to see some progress on this!

[stern-shawn]

Is it okay to ask what the status of this is? I've evangelized nuqs within our team, but the product we're about to launch just got a requirement to support some legacy query params which of course are in the native array format instead of comma separated 😓

Edit: found a temporary workaround by directly pulling from useSearchParams in a read-only fashion, but excited for when this is natively supported in nuqs!

[SalahAdDin]

Do we have any ETA?

[franky47]

I want to get v2.5 out first with all pending PRs (which include consequent refactors) before I can focus on this feature (which will likely be a large change).

PS:

Is it okay to ask what the status of this is?

This is the right way to ask for an update.

Do we have any ETA?

This is not.