Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON encoding #14021

Merged
merged 16 commits into from
Dec 10, 2024
Merged

Add JSON encoding #14021

merged 16 commits into from
Dec 10, 2024

Conversation

josevalim
Copy link
Member

@josevalim josevalim commented Nov 27, 2024

Note to code reviewers: no need to review the elixir_json.erl module, that was ported from Erlang for backwards compatibility reasons. It will be removed in future Elixir versions.

  • Add encoding
  • Add deriving
  • Add decoding
  • Figure out how to call encode_value

cc @michalmuskala

Comment on lines 111 to 112
assert ["{\"a\":", _, ",\"b\":", _, ",\"d\":", _, 125] =
json = JSON.encode_to_iodata(%WithOnly{a: :a, b: "b", c: make_ref(), d: [?d]})
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalmuskala you can see I chose to encode the longest possible part of the field as a binary, which includes the preceding field (either { or ,), the quotes, and the trailing :.

lib/elixir/lib/json.ex Outdated Show resolved Hide resolved
lib/elixir/lib/json.ex Show resolved Hide resolved
lib/elixir/lib/json.ex Outdated Show resolved Hide resolved
lib/elixir/lib/json.ex Outdated Show resolved Hide resolved
lib/elixir/lib/json.ex Outdated Show resolved Hide resolved
:elixir_json.decode(binary, acc, Map.new(decoders))
catch
:error, :unexpected_end ->
{:error, {:unexpected_end, byte_size(binary)}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered a JSON.Error or JSON.DecodeError exception with these, matching Jason.DecodeError? In Req we gracefully handle json errors like this:

iex> Req.get(plug: & &1 |> Plug.Conn.put_resp_content_type("application/json") |> Plug.Conn.send_resp(200, "{"))
{:error, %Jason.DecodeError{position: 1, token: nil, data: "{"}}

(which I'm still somewhat skeptical of cause it's not like users can do much with these errors anyway but this was a somewhat common request to have this graceful handling)

so it'd be nice to have an exception struct I could return instead. Not a big deal though, I can invent one for Req.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can raise a specific error on JSON.decode!, and I will make it so, but we don't use the {:error, Exception.t()} struct style anywhere in Elixir, so I am not sure we should do it here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, thanks!

lib/elixir/lib/json.ex Outdated Show resolved Hide resolved
lib/elixir/lib/json.ex Outdated Show resolved Hide resolved
lib/elixir/lib/json.ex Show resolved Hide resolved
lib/elixir/lib/json.ex Outdated Show resolved Hide resolved
lib/elixir/lib/json.ex Outdated Show resolved Hide resolved

@moduledoc since: "1.18.0"

@type decode_error ::
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call this decode_error_reason? We could also document the reasons here maybe.

lib/elixir/lib/json.ex Show resolved Hide resolved
"[123,\"string\",{\"key\":\"value\"}]"
"""
def encode!(term, encoder \\ &encode_value/2) do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spec?

josevalim and others added 2 commits December 9, 2024 19:06
Co-authored-by: Andrea Leopardi <[email protected]>
Co-authored-by: Adrian Salamon <[email protected]>
@doc """
This is the default function used to recursively encode each value.
"""
def encode_value(value, encoder) when is_atom(value) do
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is the same as the protocol, inlined for performance. But I am not a fan of its name. Suggestions? What about encode_callback? or default_encode? or default_encode_callback?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given it isn't technically an @callback, even if it is a generic term _callback might be confusing since its specific meaning in OTP.
default_encode captures the intent well I think.

@josevalim josevalim merged commit 793c114 into main Dec 10, 2024
21 of 24 checks passed
@josevalim josevalim deleted the jv-json branch December 10, 2024 11:14
@josevalim
Copy link
Member Author

💚 💙 💜 💛 ❤️

"[123,\"string\",{\"key\":\"value\"}]"
"""
@spec encode!(a, (a -> iodata())) :: binary() when a: var
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spec seems wrong, the callback takes two arguments, not one. Same on line 356

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

7 participants