Support Erlang/OTP 27 new features

[josevalim]

To be implemented when Erlang 27 is released:

• Ensure we can start observer without additional steps (List optional applications for wx apps erlang/otp#7441)
• Deprecate escaping of terminator in uppercase sigils as in EEP 66
• Support typespecs in docs from Add OTP 27.0-rc2 to CI #13437
• See if Task, DynamicSupervisor, and other behaviours need to include the label in its logger events
• Discussion about built-in JSON API and protocol

To be implemented when we require Erlang 27 exclusively:

• Add File.read/2 that wraps :file.read_file/2
• Remove backport of the jaro_distance implementation
• Remove backport of the elixir_json module
• Expose proc_lib:get_label
• Support multi-line prompt parsing: https://github.com/elixir-lang/elixir/compare/jv-get-until-parser-26 | stdlib: make the shell multiline prompt configurable erlang/otp#7601
• Consider using shell_docs_markdown to convert Markdown to HTML and use that for ANSI printing (stdlib: markdown conversor to erlang+html erlang/otp#8077)
• Remove usage of "application/erlang+html"

Please do not send PRs for those, as it will take a while until we can effectively accept them.

[lukaszsamson]

OTP 26.2 added this:

process_info/2 now supports lookup of values for
               specific keys in the process dictionary. For example,
               {{dictionary, Key}, Value} = process_info(Pid,
               {dictionary, Key}).

[josevalim]

In this case we straight delegate to process_info, so we don't need to do anything additionally.

[Eiji7]

Since logger is not mentioned here I would like to ask if such error:

Failure while translating Erlang's logger event

is known? Just want to give it a try and compile Elixir from main using asdf and Erlang version 27.0-rc1 and I got 11 errors with same logger problem.

1) test translates :proc_lib crashes with neighbour with name (Logger.TranslatorTest)
   test/logger/translator_test.exs:575
   Assertion with =~ failed
   code:  assert capture_log(:info, fn ->
           ref = Process.monitor(pid)
            send(pid, :message)
            send(pid, :go)

            receive do
              {:DOWN, ^ref, _, _, _} -> :ok
            end
          end) =~ ~r"Ancestors: \[#PID<\d+\.\d+\.\d+>\]
          Neighbours:
            Logger.TranslatorTest \(#PID<\d+\.\d+\.\d+>\)
              Initial Call: Logger.TranslatorTest.sleep/1
              Current Call: Logger.TranslatorTest.sleep/1
              Ancestors: \[#PID<\d+\.\d+\.\d+>, #PID<\d+\.\d+\.\d+>\]
          "
   left:  "\e[31m\n06:54:37.503 [error] Failure while translating Erlang's logger event\n** (MatchError) no match of right handside value: [pid: #PID<0.203.0>, registered_name: [], process_label: :undefined, error_info: {:error, %RuntimeError{message: \"oops\"}, [{Logger.TranslatorTest, :\"-test translates :proc_lib crashes with neighbour with name/1-fun-0-\", 1, [file: ~c\"test/logger/translator_test.exs\", line: 578, error_info: %{module: Exception}]}, {:proc_lib, :init_p_do_apply, 3, [file: ~c\"proc_lib.erl\", line: 323]}]}, ancestors: [#PID<0.202.0>], message_queue_len: 1, messages: [:message], links: [#PID<0.205.0>], dictionary: [], trap_exit: false, status: :running, heap_size: 610, stack_size: 29, reductions: 513]\n    (logger 1.17.0-dev) lib/logger/translator.ex:413: Logger.Translator.report_crash/4\n    (logger 1.17.0-dev) lib/logger/utils.ex:47: Logger.Utils.translate/5\n    (logger 1.17.0-dev) lib/logger/utils.ex:19: Logger.Utils.translator/2\n    (kernel 9.3) logger_backend.erl:102: :logger_backend.do_apply_filters/4\n    (kernel 9.3) logger_backend.erl:87: :logger_backend.apply_filters/4\n    (kernel 9.3) logger_backend.erl:33::logger_backend.log_allowed/3\n    (stdlib 6.0) proc_lib.erl:344: :proc_lib.exit_p/3\n\n\e[0m"
   right: ~r/Ancestors: \[#PID<\d+\.\d+\.\d+>\]\nNeighbours:\n    Logger.TranslatorTest \(#PID<\d+\.\d+\.\d+>\)\n        Initial Call: Logger.TranslatorTest.sleep\/1\n        Current Call: Logger.TranslatorTest.sleep\/1\n        Ancestors: \[#PID<\d+\.\d+\.\d+>, #PID<\d+\.\d+\.\d+>\]\n/
   stacktrace:
     test/logger/translator_test.exs:583: (test)

This happens on ==> logger (ex_unit) step with such info on the bottom:

Finished in 1.5 seconds (0.4s on load, 0.00s async, 1.0s sync)
3 doctests, 142 tests, 11 failures

Randomized with seed 488786
make: *** [Makefile:112: test_logger] Error 2
Build failed, cleaning...

I wonder if something was changed or it's just a bug on main branch? 🤔

[josevalim]

Erlang is passing more information to its logger, so it is something we will tackle once we start running our test suite against Erlang 27.

[nhpip]

Any plans to Elixirify the new Erlang json module?

json erlang

[josevalim]

We will most likely wrap it with a protocol, but that will be part of Erlang/OTP 28 as far as I know, so not a concern right now but rather roughly in a year from now.

[krns]

The target of the json module is OTP 27:

https://x.com/michalmuskala/status/1759666121320419743?s=46&t=FfKER0z4daYePSLFkNoU2A

[josevalim]

Ah, thank you. So I will add it to the list. We can probably backport the Erlang modules if we want to have it sooner, but we need to have discussions on the API. So maybe on Elixir v1.18.

[michalmuskala]

I'd be happy to discuss how to best integrate the new module into Elixir given the combination of Erlang stdlib, Elixir stdlib, and Jason. I was planning to release Jason 2.0 integrating with the new library, but if there are plans to integrate straight into Elixir, this is likely even better.

The current implementation in the PR contains some code patterns that benefit from some new optimisations in OTP 27, so there are some additional considerations around that as well, when back-porting to run on older Erlang releases.

[michallepicki]

I think this was released in OTP 26.2, it would be cool to expose getting a process dictionary value for specific key in the Process module (either another clause for Process.get or allow this in Process.info - currently a guard prevents it)

edit: I see this was discussed above but the problem with the guard was not brought up. It can be worked around by using a one element list though.

[josevalim]

@michallepicki please send a PR to relax the guard? We don't need to wait for Erlang/OTP 27 for that. :)

[lukaszsamson]

@josevalim We should also add File.read/2. OTP 27 added :file.read_file/2 with option :raw https://www.erlang.org/doc/apps/kernel/file.html#read_file/2