Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PDF builder: tectonic #9598

Closed
humitos opened this issue Sep 12, 2022 · 2 comments
Closed

PDF builder: tectonic #9598

humitos opened this issue Sep 12, 2022 · 2 comments
Labels
Needed: design decision A core team decision is required

Comments

@humitos
Copy link
Member

humitos commented Sep 12, 2022

This is a no-action issue for now. The idea is to start collecting ideas, user experiences and more from the community regarding the "The Tectonic Typesetting System", https://tectonic-typesetting.github.io/en-US/

I quickly tested by myself with https://github.com/readthedocs/sphinx-hoverxref and it worked out of the box without any configuration.

make latex 
tectonic -X compile sphinx-hoverxref.tex

The CLI interface was pretty simple and the output looks promising. Does it make sense to integrate it with Read the Docs somehow? Once that build.commands supports exposing other formats apart from HTML, it would be good to document and example about how to use it under https://docs.readthedocs.io/en/latest/build-customization.html
@humitos humitos added the Needed: design decision A core team decision is required label Sep 12, 2022
@humitos
Copy link
Member Author

humitos commented Nov 29, 2022

We are talking about building with Simple PDF at https://github.com/readthedocs/meta/issues/77#issuecomment-1301927685 and #9088 which is related to this issue as well.

humitos added a commit that referenced this issue Jan 11, 2023
@humitos
Build: standardize output directory for artifacts
db78cdf 
Always expose the final artifacts under `_readthedocs/<format>` directory, where
`<format>` can be one of the following:

- html
- htmlzip
- pdf
- epub
- json

This allows all users of the platform to have the same behavior. No matter if
they are using the default Sphinx/MkDocs builders or a custom one by using
`build.commands`.

Besides, this commit removes the "intermediary" directory used _before_
uploading the files to S3. This allows users to modify the output files directly
from `_readthedocs/<format>` using `build.jobs.post_build`, for example.

Some important notes:

- `BaseBuilder.type` is removed since it was not required anymore.
- There is no `old_artifact_path` anymore since there is no intermediary folder.
- The `sphinx-build` command is executed from path where the repository is
  cloned, instead from the the directory where the `conf.py` lives. This is to
  simplify the code and be able to use relative path for the build output
  directory (e.g. `sphinx-build -b html ... docs/ _readthedocs/html`).
- A new `BaseSphinx._post_build` method was added to cleanup the output
  directory (leave only one .epub, .html and .pdf file before moving forward).
- These changes should allow us to support _all formats_ for `build.commands`.
  We would need to check if these standard directories exist, set these formats
  in the version and upload their content.
- Upload step now validate there is one and only one PDF, ePUB and ZIP file when
  they are built. Currently, we only support one file per version. In the future
  we could remove this restriction. It fails silently for now, which is not
  great. We need to refactor this code a little to fail the build properly and
  communicate this to the user. I think it's _the correct way_ of doing it.
- This could also allow to support other ways to build PDF (SimplePDF or
 `tectonic`, see #9598) and
  also ZIP (e.g. zundler, see
  readthedocs/meta#77 (comment)).

Closes #9179
@humitos humitos mentioned this issue Jan 11, 2023
Merged
humitos added a commit that referenced this issue Jan 11, 2023
@humitos
Build: standardize output directory for artifacts
faec2a2 
Always expose the final artifacts under `_readthedocs/<format>` directory, where
`<format>` can be one of the following:

- html
- htmlzip
- pdf
- epub
- json

This allows all users of the platform to have the same behavior. No matter if
they are using the default Sphinx/MkDocs builders or a custom one by using
`build.commands`.

Besides, this commit removes the "intermediary" directory used _before_
uploading the files to S3. This allows users to modify the output files directly
from `_readthedocs/<format>` using `build.jobs.post_build`, for example.

- `BaseBuilder.type` is removed since it was not required anymore.
- There is no `old_artifact_path` anymore since there is no intermediary folder.
- The `sphinx-build` command is executed from path where the repository is
  cloned, instead from the the directory where the `conf.py` lives. This is to
  simplify the code and be able to use relative path for the build output
  directory (e.g. `sphinx-build -b html ... docs/ _readthedocs/html`).
- A new `BaseSphinx._post_build` method was added to cleanup the output
  directory (leave only one .epub, .html and .pdf file before moving forward).
- These changes should allow us to support _all formats_ for `build.commands`.
  We would need to check if these standard directories exist, set these formats
  in the version and upload their content.
- Upload step now validate there is one and only one PDF, ePUB and ZIP file when
  they are built. Currently, we only support one file per version. In the future
  we could remove this restriction. It fails silently for now, which is not
  great. We need to refactor this code a little to fail the build properly and
  communicate this to the user. I think it's _the correct way_ of doing it.
- This could also allow to support other ways to build PDF (SimplePDF or
 `tectonic`, see #9598) and
  also ZIP (e.g. zundler, see
  readthedocs/meta#77 (comment)).

Closes #9179
humitos added a commit that referenced this issue Jan 19, 2023
@humitos @ericholscher
Build: standardize output directory for artifacts (#9888)
239d890 
* Build: standardize output directory for artifacts

Always expose the final artifacts under `_readthedocs/<format>` directory, where
`<format>` can be one of the following:

- html
- htmlzip
- pdf
- epub
- json

This allows all users of the platform to have the same behavior. No matter if
they are using the default Sphinx/MkDocs builders or a custom one by using
`build.commands`.

Besides, this commit removes the "intermediary" directory used _before_
uploading the files to S3. This allows users to modify the output files directly
from `_readthedocs/<format>` using `build.jobs.post_build`, for example.

- `BaseBuilder.type` is removed since it was not required anymore.
- There is no `old_artifact_path` anymore since there is no intermediary folder.
- The `sphinx-build` command is executed from path where the repository is
  cloned, instead from the the directory where the `conf.py` lives. This is to
  simplify the code and be able to use relative path for the build output
  directory (e.g. `sphinx-build -b html ... docs/ _readthedocs/html`).
- A new `BaseSphinx._post_build` method was added to cleanup the output
  directory (leave only one .epub, .html and .pdf file before moving forward).
- These changes should allow us to support _all formats_ for `build.commands`.
  We would need to check if these standard directories exist, set these formats
  in the version and upload their content.
- Upload step now validate there is one and only one PDF, ePUB and ZIP file when
  they are built. Currently, we only support one file per version. In the future
  we could remove this restriction. It fails silently for now, which is not
  great. We need to refactor this code a little to fail the build properly and
  communicate this to the user. I think it's _the correct way_ of doing it.
- This could also allow to support other ways to build PDF (SimplePDF or
 `tectonic`, see #9598) and
  also ZIP (e.g. zundler, see
  readthedocs/meta#77 (comment)).

Closes #9179

* Build: decideif there is an output type based on the path existence

* Test: update build tests to match changes

* Test: check if the file exist before continue

* Build: simplify the code by running the commands inside the container

* Test: add checks for more commands

* Storage: use constants to make explicit artifact types

* PDF builder: raise an error if the PDF file is not found

* Build: use `relative_output_dir` and `absolute_output_dir`

* Builder: execute `sphinx-build` from the same directory as `conf.py`

I proposed to changed this CWD to be the one where the repository was cloned,
but that could lead to some unexpected behavior. So, it's better to be
conservative here and keep the CWD as it was.

There are some small downsides for this decision, but it's better than breaking
people's builds :)

* HTMLZip build: use `zip` inside the Docker container to build it

* Minor changes about docstring and final dot in a sentence :)

* Test: adapt them to match thew path and arguments changes

* pre-commit missing changes

* Sphinx builder: better default

* Update readthedocs/doc_builder/backends/sphinx.py

Co-authored-by: Eric Holscher <[email protected]>

* Update readthedocs/projects/tests/test_build_tasks.py

Co-authored-by: Eric Holscher <[email protected]>

* Minor changes to fix tests

* Docstring: explain why there is an exception handling at this place

Co-authored-by: Eric Holscher <[email protected]>
@humitos humitos mentioned this issue Jan 31, 2023
Closed
@humitos
Copy link
Member Author

humitos commented Jan 31, 2023 •
edited
Loading

Building with tectonic is now supported! 🥳

I created an example for this use case:

@humitos humitos closed this as completed Jan 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Needed: design decision A core team decision is required
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@humitos