[SPARK-30084][DOCS] Document how to trigger Jekyll build on Python AP…

…I doc changes ### What changes were proposed in this pull request? This PR adds a note to the docs README showing how to get Jekyll to automatically pick up changes to the Python API docs. ### Why are the changes needed? `jekyll serve --watch` doesn't watch for changes to the API docs. Without the technique documented in this note, or something equivalent, developers have to manually retrigger a Jekyll build any time they update the Python API docs. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? I tested this PR manually by making changes to Python docstrings and confirming that Jekyll automatically picks them up and serves them locally. Closes apache#26719 from nchammas/SPARK-30084-watch-api-docs. Authored-by: Nicholas Chammas <[email protected]> Signed-off-by: Sean Owen <[email protected]>
kailixu · Dec 4, 2019 · 29e09a8 · 29e09a8
1 parent 2ceed6f
commit 29e09a8
Show file tree

Hide file tree

Showing 2 changed files with 16 additions and 1 deletion.
diff --git a/.gitignore b/.gitignore
@@ -45,7 +45,7 @@ dev/create-release/*final
 dev/create-release/*txt
 dev/pr-deps/
 dist/
-docs/_site
+docs/_site/
 docs/api
 sql/docs
 sql/site
@@ -63,6 +63,7 @@ project/plugins/target/
 python/lib/pyspark.zip
 python/.eggs/
 python/deps
+python/docs/_site/
 python/test_coverage/coverage_data
 python/test_coverage/htmlcov
 python/pyspark/python

diff --git a/docs/README.md b/docs/README.md
@@ -102,3 +102,17 @@ using [MkDocs](https://www.mkdocs.org/).
 NOTE: To skip the step of building and copying over the Scala, Java, Python, R and SQL API docs, run `SKIP_API=1
 jekyll build`. In addition, `SKIP_SCALADOC=1`, `SKIP_PYTHONDOC=1`, `SKIP_RDOC=1` and `SKIP_SQLDOC=1` can be used
 to skip a single step of the corresponding language. `SKIP_SCALADOC` indicates skipping both the Scala and Java docs.
+
+### Automatically Rebuilding API Docs
+
+`jekyll serve --watch` will only watch what's in `docs/`, and it won't follow symlinks. That means it won't monitor your API docs under `python/docs` or elsewhere.
+
+To work around this limitation for Python, install [`entr`](http://eradman.com/entrproject/) and run the following in a separate shell:
+
+```sh
+cd "$SPARK_HOME/python/docs"
+find .. -type f -name '*.py' \
+| entr -s 'make html && cp -r _build/html/. ../../docs/api/python'
+```
+
+Whenever there is a change to your Python code, `entr` will automatically rebuild the Python API docs and copy them to `docs/`, thus triggering a Jekyll update.