Skip to content

Add JSON encoding - #14021

Merged
josevalim merged 16 commits into
mainfrom
jv-json
Dec 10, 2024
Merged

Add JSON encoding#14021
josevalim merged 16 commits into
mainfrom
jv-json

Conversation

@josevalim

@josevalim josevalim commented Nov 27, 2024

Copy link
Copy Markdown
Member

Note to code reviewers: no need to review the elixir_json.erl module, that was ported from Erlang for backwards compatibility reasons. It will be removed in future Elixir versions.

  • Add encoding
  • Add deriving
  • Add decoding
  • Figure out how to call encode_value

cc @michalmuskala

Comment thread lib/elixir/test/elixir/json_test.exs Outdated
Comment on lines +111 to +112
assert ["{\"a\":", _, ",\"b\":", _, ",\"d\":", _, 125] =
json = JSON.encode_to_iodata(%WithOnly{a: :a, b: "b", c: make_ref(), d: [?d]})

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalmuskala you can see I chose to encode the longest possible part of the field as a binary, which includes the preceding field (either { or ,), the quotes, and the trailing :.

Comment thread lib/elixir/src/elixir_json.erl
Comment thread lib/elixir/src/elixir_json.erl
Comment thread lib/elixir/src/elixir_json.hrl Outdated
Comment thread lib/elixir/src/elixir_json.erl Outdated
Comment thread lib/elixir/lib/json.ex Outdated
Comment thread lib/elixir/lib/json.ex
josevalim and others added 3 commits November 27, 2024 20:41
Co-authored-by: Eksperimental <eksperimental@autistici.org>
Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>
Comment thread lib/elixir/lib/json.ex Outdated
Comment thread lib/elixir/lib/json.ex Outdated
Comment thread lib/elixir/lib/json.ex Outdated
Comment thread lib/elixir/lib/json.ex
:elixir_json.decode(binary, acc, Map.new(decoders))
catch
:error, :unexpected_end ->
{:error, {:unexpected_end, byte_size(binary)}}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered a JSON.Error or JSON.DecodeError exception with these, matching Jason.DecodeError? In Req we gracefully handle json errors like this:

iex> Req.get(plug: & &1 |> Plug.Conn.put_resp_content_type("application/json") |> Plug.Conn.send_resp(200, "{"))
{:error, %Jason.DecodeError{position: 1, token: nil, data: "{"}}

(which I'm still somewhat skeptical of cause it's not like users can do much with these errors anyway but this was a somewhat common request to have this graceful handling)

so it'd be nice to have an exception struct I could return instead. Not a big deal though, I can invent one for Req.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can raise a specific error on JSON.decode!, and I will make it so, but we don't use the {:error, Exception.t()} struct style anywhere in Elixir, so I am not sure we should do it here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, thanks!

Comment thread lib/elixir/lib/json.ex Outdated
Comment thread lib/elixir/lib/json.ex Outdated
Comment thread lib/elixir/lib/json.ex Outdated
Comment thread lib/elixir/lib/json.ex
Comment thread lib/elixir/lib/json.ex Outdated
Comment thread lib/elixir/lib/json.ex Outdated
Comment thread lib/elixir/lib/json.ex Outdated

@moduledoc since: "1.18.0"

@type decode_error ::

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call this decode_error_reason? We could also document the reasons here maybe.

Comment thread lib/elixir/lib/json.ex
Comment thread lib/elixir/lib/json.ex Outdated
"[123,\"string\",{\"key\":\"value\"}]"

"""
def encode!(term, encoder \\ &encode_value/2) do

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spec?

josevalim and others added 2 commits December 9, 2024 19:06
Co-authored-by: Andrea Leopardi <an.leopardi@gmail.com>
Co-authored-by: Adrian Salamon <adr.salamon@gmail.com>
Comment thread lib/elixir/lib/json.ex Outdated
@doc """
This is the default function used to recursively encode each value.
"""
def encode_value(value, encoder) when is_atom(value) do

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is the same as the protocol, inlined for performance. But I am not a fan of its name. Suggestions? What about encode_callback? or default_encode? or default_encode_callback?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given it isn't technically an @callback, even if it is a generic term _callback might be confusing since its specific meaning in OTP.
default_encode captures the intent well I think.

@josevalim
josevalim merged commit 793c114 into main Dec 10, 2024
@josevalim
josevalim deleted the jv-json branch December 10, 2024 11:14
@josevalim

Copy link
Copy Markdown
Member Author

💚 💙 💜 💛 ❤️

Comment thread lib/elixir/lib/json.ex
"[123,\"string\",{\"key\":\"value\"}]"

"""
@spec encode!(a, (a -> iodata())) :: binary() when a: var

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spec seems wrong, the callback takes two arguments, not one. Same on line 356

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thank you!

@michalmuskala

Copy link
Copy Markdown
Member

Thank you for implementing this!
Once 1.18 is released to stable with the new JSON module, I plan to make a new release of Jason with some small fixes and then effectively deprecate it - it will still be supported for bug fixes, but I plan no active work on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

8 participants