Change documentation to use ExDoc and EEP-59 style documentation #8026
Conversation
CT Test Results 38 files 983 suites 7h 23m 2s ⏱️ Results for commit 36dddd6. ♻️ This comment has been updated with latest results. To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass. See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally. Artifacts// Erlang/OTP Github Action Bot |
|
This is a most welcome change. Thanks, @garazdawi. I also love these stats. 👍 
|
|
Honestly it looks a little bit strange - the
|
|
Hello!
First of all, we are not integrating ExDoc into Erlang/OTP. We are using it as a tool, much like we use xsltproc+fop today, to generate html and epub documentation. The only thing a developer of Erlang/OTP1 needs to install is an escript archive that is downloadable from the ExDoc github page. So the analogy with rebar3 would be if we changed from using GNU Make to rebar3, rather than integrating the tools into the project.
There is one script that uses Elixir, and that is otp_check_html_links.exs. A script that checks that all links and anchors are correct for the generated HTML documentation. The main reason for writing it in Elixir was because floki is a neat library to work with HTML and I wanted to learn a bit of Elixir. If this becomes a problem, I'll write it in Erlang instead and use regexps to parse html.
This is only related to Erlang/OTP documentation. Third party projects are free to do as they please.
Not in any way that I can think of. Do you have any example that comes to mind?
Neither Tools that will be effected are things like ErlangLS and others that parse the AST of the code in Erlang/OTP. This because they now need to take into account for the EEP-59 style documentation attributes. In conclusion I would like to say that this is a step in trying to concentrate our relatively small communitys resources. Today we have (atleast) three documentation tools; erl_docgen, edoc and ExDoc. After this PR, we will only have 2 as erl_docgen will be removed. It is also our longterm vision (5-10+ years) that edoc will be removed, but we will see if that ever happens. Footnotes
|
|
Hi! @garazdawi many thanks for so extended answer!
Yes, let say we setup Elixir v1 which support Erlang v1, after some time, some issue was fixed in Elixir in v2(of course with support Erlang v2), and this changes are very important for some Elixir lib which was also updated till latest version, but locally we still have v1 - so, now if somebody will need make changes in Erlang/OTP documentation - he will need to upgrade Elixir version for make 3rd party libs works as well, because locally we still have old version of Erlang and Elixir - this is correct? |
Yes, I suppose that could happen. If that type of scenario happens a lot, we may have to start thinking about what to do about it. Until then I choose to be optimistic. |
Clear, many thanks again for all answers and for this big and amazing work! |
One potential alternative for these tools is to instead rely on EEP 48 and extract the documentation from the .beam directly.
Elixir and ExDoc supports the latest three Erlang/OTP versions (3 years), the same compatibility guarantees as Erlang, so that should give enough coverage. Plus I assume Erlang/OTP will always hardcode a particular ExDoc version, so you can always go back in time. Worst case scenario, we can build single-file executables with all of Erlang+Elixir+ExDoc, so you can always go back in time regardless of the currently installed Erlang. The only issue I can see happening is that a future Erlang version, say Erlang/OTP 28, removes something that Elixir was incorrectly relying on, breaking Elixir and consequently ExDoc. This means we need to fix Elixir quickly, but that's something we need to do sooner than later anyway. :) |
Yes, I also was think about this issue too. Many thanks for answer @josevalim it very useful! |
garazdawi
force-pushed
the
lukas/otp/convert-docs-to-markdown
branch
3 times, most recently
from
a29f34f to
bef1812
Compare
garazdawi
force-pushed
the
lukas/otp/convert-docs-to-markdown
branch
7 times, most recently
from
773aa31 to
5e12c58
Compare
The tools take application/html+erlang EEP-48 style documentation and convert it to EEP-59 style markdown. All erl_docgen types of documentation files are supported. This commit also adds some rudimentory support for generating HTML docs using ex_doc. The support is basically there to test that the XML to Markdown convertion works as intended and is not meant for production use.
We do this so that we also test that edoc still works as it should and that ExDoc can render edoc docs nicely.
Instead of using the preloaded beam files for the erts applications beam files, we build new beam files when we have a working compiler. The beam files in the erts application are only used for tools such as dialyzer and EEP-59 documentation.
We now use annotations instead.
garazdawi
force-pushed
the
lukas/otp/convert-docs-to-markdown
branch
from
5e12c58 to
36dddd6
Compare
|
@garazdawi I am looking for the "edoc → markdown+ExDoc" conversion tool. Is it available somewhere? |
|
@hcs42 It is part of edoc: https://www.erlang.org/doc/apps/edoc/edoc_doclet_markdown |
This PR changes the way that Erlang/OTP is documented from using erl_docgen to ExDoc.
This is a huge PR that uses a tool to automatically convert all our documentation
from XML to Markdown, trying to keep as close as possible to the original docs.
There are bound to be many small mistakes in the final docs, as the convertion
is not perfect, but it is a step into what we think is an easer to maintain
and evolve way of writing documentation.
The tool that was used to convert erl_docgen XML to Markdown can also be used
to convert edoc XHTML to Markdown. This tool will be made available in a later
PR for anyone who wants to migrate their edoc code bases to use Markdown and ExDoc
instead.