Post exports hang or take a very long time when single org file gets large #551

kishvanchee · 2021-12-30T13:05:33Z

kishvanchee
Dec 30, 2021

Actual Behavior

The export is slow at times when I'm exporting the posts recently. This also takes up a lot of CPU (a few times my entire system hung up cos emacs took a lot of CPU). I have even had emacs hang after a point. Is this an expected behaviour? For context, I have around 100 or so org-subtrees (posts) and many of them are drafts (TODO / DRAFT). The no of lines of the org file comes close to 3k. I am using one post per org-subtree since that's the recommended method.

Is this a bug with ox-hugo / org-exporter? Anything I'm missing? Would splitting my posts into separate files be the solution here? If you have examples (sites) you know of who have large number of separate org files, that'd also be helpful.

I see this has been discussed here but no concrete answer on the scalability as such has been provided #428 Just linking it here for reference

Expected Behavior

I would expect a smooth export for every export I do.

How to Reproduce the Issue

Not exactly sure how to reproduce this, it's just a big org file. I've noticed this only recently. I didn't have this issue previously.

Ox-Hugo Debug Information

Debug Info

Debug information for `ox-hugo`

Emacs Version

GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, cairo version 1.17.4)
 of 2021-03-26

Org Version

Org mode version 9.6 (9.6-??-e0b1a05 @ /home/kishore/.emacs.d/.local/straight/build-27.2/org/)

Hugo Version

hugo v0.91.2+extended linux/amd64 BuildDate=unknown

Org `load-path` shadows

Warning: Possible mixed installation of Org

/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-C hides /usr/share/emacs/27.2/lisp/org/ob-C
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-forth hides /usr/share/emacs/27.2/lisp/org/ob-forth
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-core hides /usr/share/emacs/27.2/lisp/org/ob-core
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-dot hides /usr/share/emacs/27.2/lisp/org/ob-dot
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-fortran hides /usr/share/emacs/27.2/lisp/org/ob-fortran
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-octave hides /usr/share/emacs/27.2/lisp/org/ob-octave
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-lua hides /usr/share/emacs/27.2/lisp/org/ob-lua
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-groovy hides /usr/share/emacs/27.2/lisp/org/ob-groovy
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-colview hides /usr/share/emacs/27.2/lisp/org/org-colview
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-comint hides /usr/share/emacs/27.2/lisp/org/ob-comint
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-ocaml hides /usr/share/emacs/27.2/lisp/org/ob-ocaml
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-bbdb hides /usr/share/emacs/27.2/lisp/org/ol-bbdb
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-org hides /usr/share/emacs/27.2/lisp/org/ob-org
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-exp hides /usr/share/emacs/27.2/lisp/org/ob-exp
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-indent hides /usr/share/emacs/27.2/lisp/org/org-indent
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-duration hides /usr/share/emacs/27.2/lisp/org/org-duration
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org hides /usr/share/emacs/27.2/lisp/org/org
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-attach hides /usr/share/emacs/27.2/lisp/org/org-attach
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-emacs-lisp hides /usr/share/emacs/27.2/lisp/org/ob-emacs-lisp
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-icalendar hides /usr/share/emacs/27.2/lisp/org/ox-icalendar
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-screen hides /usr/share/emacs/27.2/lisp/org/ob-screen
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-compat hides /usr/share/emacs/27.2/lisp/org/org-compat
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-sql hides /usr/share/emacs/27.2/lisp/org/ob-sql
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-scheme hides /usr/share/emacs/27.2/lisp/org/ob-scheme
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-faces hides /usr/share/emacs/27.2/lisp/org/org-faces
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-ruby hides /usr/share/emacs/27.2/lisp/org/ob-ruby
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-table hides /usr/share/emacs/27.2/lisp/org/org-table
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-timer hides /usr/share/emacs/27.2/lisp/org/org-timer
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-list hides /usr/share/emacs/27.2/lisp/org/org-list
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob hides /usr/share/emacs/27.2/lisp/org/ob
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-md hides /usr/share/emacs/27.2/lisp/org/ox-md
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-src hides /usr/share/emacs/27.2/lisp/org/org-src
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-docview hides /usr/share/emacs/27.2/lisp/org/ol-docview
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-clock hides /usr/share/emacs/27.2/lisp/org/org-clock
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-org hides /usr/share/emacs/27.2/lisp/org/ox-org
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-beamer hides /usr/share/emacs/27.2/lisp/org/ox-beamer
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-html hides /usr/share/emacs/27.2/lisp/org/ox-html
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-ascii hides /usr/share/emacs/27.2/lisp/org/ox-ascii
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-latex hides /usr/share/emacs/27.2/lisp/org/ob-latex
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-sed hides /usr/share/emacs/27.2/lisp/org/ob-sed
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-loaddefs hides /usr/share/emacs/27.2/lisp/org/org-loaddefs
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-odt hides /usr/share/emacs/27.2/lisp/org/ox-odt
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-protocol hides /usr/share/emacs/27.2/lisp/org/org-protocol
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-crypt hides /usr/share/emacs/27.2/lisp/org/org-crypt
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-tangle hides /usr/share/emacs/27.2/lisp/org/ob-tangle
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-footnote hides /usr/share/emacs/27.2/lisp/org/org-footnote
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-rmail hides /usr/share/emacs/27.2/lisp/org/ol-rmail
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-feed hides /usr/share/emacs/27.2/lisp/org/org-feed
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-capture hides /usr/share/emacs/27.2/lisp/org/org-capture
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-macs hides /usr/share/emacs/27.2/lisp/org/org-macs
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-eval hides /usr/share/emacs/27.2/lisp/org/ob-eval
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-processing hides /usr/share/emacs/27.2/lisp/org/ob-processing
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol hides /usr/share/emacs/27.2/lisp/org/ol
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-texinfo hides /usr/share/emacs/27.2/lisp/org/ox-texinfo
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-clojure hides /usr/share/emacs/27.2/lisp/org/ob-clojure
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-entities hides /usr/share/emacs/27.2/lisp/org/org-entities
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-mhe hides /usr/share/emacs/27.2/lisp/org/ol-mhe
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-man hides /usr/share/emacs/27.2/lisp/org/ox-man
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-haskell hides /usr/share/emacs/27.2/lisp/org/ob-haskell
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-eww hides /usr/share/emacs/27.2/lisp/org/ol-eww
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-macro hides /usr/share/emacs/27.2/lisp/org/org-macro
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-datetree hides /usr/share/emacs/27.2/lisp/org/org-datetree
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-js hides /usr/share/emacs/27.2/lisp/org/ob-js
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-sqlite hides /usr/share/emacs/27.2/lisp/org/ob-sqlite
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-irc hides /usr/share/emacs/27.2/lisp/org/ol-irc
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-css hides /usr/share/emacs/27.2/lisp/org/ob-css
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-maxima hides /usr/share/emacs/27.2/lisp/org/ob-maxima
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-mobile hides /usr/share/emacs/27.2/lisp/org/org-mobile
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-mouse hides /usr/share/emacs/27.2/lisp/org/org-mouse
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-goto hides /usr/share/emacs/27.2/lisp/org/org-goto
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-python hides /usr/share/emacs/27.2/lisp/org/ob-python
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-calc hides /usr/share/emacs/27.2/lisp/org/ob-calc
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-agenda hides /usr/share/emacs/27.2/lisp/org/org-agenda
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-habit hides /usr/share/emacs/27.2/lisp/org/org-habit
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-sass hides /usr/share/emacs/27.2/lisp/org/ob-sass
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-lilypond hides /usr/share/emacs/27.2/lisp/org/ob-lilypond
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-lob hides /usr/share/emacs/27.2/lisp/org/ob-lob
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-w3m hides /usr/share/emacs/27.2/lisp/org/ol-w3m
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-eshell hides /usr/share/emacs/27.2/lisp/org/ol-eshell
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-eshell hides /usr/share/emacs/27.2/lisp/org/ob-eshell
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-plantuml hides /usr/share/emacs/27.2/lisp/org/ob-plantuml
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-R hides /usr/share/emacs/27.2/lisp/org/ob-R
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-lint hides /usr/share/emacs/27.2/lisp/org/org-lint
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-element hides /usr/share/emacs/27.2/lisp/org/org-element
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-shell hides /usr/share/emacs/27.2/lisp/org/ob-shell
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-lisp hides /usr/share/emacs/27.2/lisp/org/ob-lisp
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-bibtex hides /usr/share/emacs/27.2/lisp/org/ol-bibtex
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox hides /usr/share/emacs/27.2/lisp/org/ox
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-ditaa hides /usr/share/emacs/27.2/lisp/org/ob-ditaa
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-gnus hides /usr/share/emacs/27.2/lisp/org/ol-gnus
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-makefile hides /usr/share/emacs/27.2/lisp/org/ob-makefile
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-plot hides /usr/share/emacs/27.2/lisp/org/org-plot
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ol-info hides /usr/share/emacs/27.2/lisp/org/ol-info
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-perl hides /usr/share/emacs/27.2/lisp/org/ob-perl
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-pcomplete hides /usr/share/emacs/27.2/lisp/org/org-pcomplete
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-keys hides /usr/share/emacs/27.2/lisp/org/org-keys
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-archive hides /usr/share/emacs/27.2/lisp/org/org-archive
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-awk hides /usr/share/emacs/27.2/lisp/org/ob-awk
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-inlinetask hides /usr/share/emacs/27.2/lisp/org/org-inlinetask
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-publish hides /usr/share/emacs/27.2/lisp/org/ox-publish
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-java hides /usr/share/emacs/27.2/lisp/org/ob-java
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-id hides /usr/share/emacs/27.2/lisp/org/org-id
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-num hides /usr/share/emacs/27.2/lisp/org/org-num
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-ctags hides /usr/share/emacs/27.2/lisp/org/org-ctags
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-table hides /usr/share/emacs/27.2/lisp/org/ob-table
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-refile hides /usr/share/emacs/27.2/lisp/org/org-refile
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ox-latex hides /usr/share/emacs/27.2/lisp/org/ox-latex
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-matlab hides /usr/share/emacs/27.2/lisp/org/ob-matlab
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-attach-git hides /usr/share/emacs/27.2/lisp/org/org-attach-git
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-tempo hides /usr/share/emacs/27.2/lisp/org/org-tempo
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-ref hides /usr/share/emacs/27.2/lisp/org/ob-ref
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-install hides /usr/share/emacs/27.2/lisp/org/org-install
/home/kishore/.emacs.d/.local/straight/build-27.2/org/ob-gnuplot hides /usr/share/emacs/27.2/lisp/org/ob-gnuplot
/home/kishore/.emacs.d/.local/straight/build-27.2/org/org-version hides /usr/share/emacs/27.2/lisp/org/org-version

Study the output of M-x list-load-path-shadows.

`ox-hugo` defcustoms

org-hugo-section	“posts”
org-hugo-use-code-for-kbd	nil
org-hugo-preserve-filling	t
org-hugo-delete-trailing-ws	t
org-hugo-prefer-hyphen-in-tags	t
org-hugo-allow-spaces-in-tags	t
org-hugo-tag-processing-functions	(org-hugo–tag-processing-fn-replace-with-spaces-maybe org-hugo–tag-processing-fn-replace-with-hyphens-maybe)
org-hugo-auto-set-lastmod	nil
org-hugo-export-with-toc	nil
org-hugo-export-with-section-numbers	nil
org-hugo-front-matter-format	“toml”
org-hugo-default-static-subdirectory-for-externals	“ox-hugo”
org-hugo-external-file-extensions-allowed-for-copying	(“jpg” “jpeg” “tiff” “png” “svg” “gif” “mp4” “pdf” “odt” “doc” “ppt” “xls” “docx” “pptx” “xlsx”)
org-hugo-date-format	“%Y-%m-%dT%T%z”
org-hugo-paired-shortcodes	“”
org-hugo-langs-no-descr-in-code-fences	nil
org-hugo-suppress-lastmod-period	0.0
org-hugo-front-matter-format	“toml”

kaushalmodi · 2021-12-30T15:49:07Z

kaushalmodi
Dec 30, 2021
Maintainer

Does it hang when:

you export all the posts (C-c C-e H A)
you export the current subtree manually (C-c C-e H H)
you export the current subtree automatically on save (you have org-hugo-auto-export-mode enabled)

I see your Org version as 9.6. So that's the Org main branch version. Does this issue happen on the Org stable or bugfix branch?

Was this issue happening on emacs 26.x? Or did you start seeing it only on emacs 27.x (did you upgrade to emacs 27.x recently?)

I am just trying to understand what change caused this issue. If you think of the time when the export used to work fine, what were the emacs and Org versions and how much time back was that (so that we can estimate the version/commit of ox-hugo then). Was it the same OS or a different one?

Here's how you can get more debug info:

Set debug-on-quit to t. This will show the backtrace each time you hit C-g.
Try to reproduce the hang and while emacs is hung, hit C-g. Please share the backtrace you see next.

If you can share your Org file, I can attempt to reproduce the issue.

0 replies

kaushalmodi · 2021-12-30T15:55:21Z

kaushalmodi
Dec 30, 2021
Maintainer

Also, does emacs lock up permanently? (That should never happens and possibly points to some bug in emacs)

Or does it hang only for few seconds (15 seconds)? If you are exporting dozens of posts, it could take that long, but not if you exporting only 1 post (current subtree), regardless of the total size of the Org file.

0 replies

kishvanchee · 2021-12-30T16:20:10Z

kishvanchee
Dec 30, 2021
Author

I should have clarified. I always do C-c C-e H H to export only the subtree. I have exported all posts before but that was like around 10-15 or so posts which took less than a minute. I don't do that now, I just tried that out initially because I was curious.

I do not have org-hugo-auto-export-mode enabled. I wasn't aware of the feature. I don't think I'd be enabling it now either.

I see your Org version as 9.6. So that's the Org main branch version. Does this issue happen on the Org stable or bugfix branch?

I'm not sure about this. I would have to try it out and let you know. I currently use doom-emacs for my config so I'd need to configure this accordingly.

Was this issue happening on emacs 26.x? Or did you start seeing it only on emacs 27.x (did you upgrade to emacs 27.x recently?)

I don't think it happened in emacs 26.x. I updated to emacs 27.x as soon as it was available in the arch linux repositories. So my observation is restricted to 27.x. I've been using this version for close to a year now.

I am just trying to understand what change caused this issue. If you think of the time when the export used to work fine, what were the emacs and Org versions and how much time back was that (so that we can estimate the version/commit of ox-hugo then). Was it the same OS or a different one?

Looking through my git log for my blog, I think the last working version is dated Oct 25 2021. Let me link some issues I read up on when I upgraded my doom emacs and the configuration. You had also commented on one of them. OS config has not changed.

This doomemacs/doomemacs#5721 is the issue I came across when I hit the error org-element--cache-sync: Symbol’s function definition is void: backtrace-get-frames. I ended up doing unpin! org which solved that issue. Here's nobiot/org-transclusion#109 another related issue linked from there. At the same time I faced this issue doomemacs/doomemacs#5913 as well which was also solved by unpin! org.

Also, does emacs lock up permanently? (That should never happens and possibly points to some bug in emacs)

This happens when I try to save the file. Not when I export. Export happens successfully for a subtree but takes a very very long time (> 1-2 mins)

I passed pkill -USR2 emacs and it entered the debug trace. Here's the paste for that. This is for when emacs hangs on save.

Debugger entered--entering a function:
* #f(compiled-function () #<bytecode 0x155dc6088329>)()
  org-element--parse-to(4743)
  org-element--cache-verify-element((paragraph (:begin 4743 :end 4774 :contents-begin 4743 :contents-end 4774 :post-blank 0 :post-affiliated 4743 :mode nil :granularity element :cached t :parent (section (:begin 3984 :end 4774 :contents-begin 3984 :contents-end 4774 :robust-begin 3984 :robust-end 4772 :post-blank 0 :post-affiliated 3984 :mode section :granularity element :cached t :parent (headline (:raw-value "Course Review: UCB CS61A" :begin 3916 :end 4774 :pre-blank 0 :contents-begin 3984 :contents-end 4774 :robust-begin 4119 :robust-end 4772 :level 2 :priority nil :tags (... ...) :todo-keyword #("DRAFT" 0 5 ...) :todo-type todo :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 3916 :EXPORT_FILE_NAME "course-review-ucb-cs61a" :EXPORT_HUGO_CUSTOM_FRONT_MATTER ":series \"100 Days to Offload challenge\"" :title "Course Review: UCB CS61A" :mode nil :granularity element :org-element--cache-sync-key (35 . 3504) :cached t ...)) :org-element--cache-sync-key (35 . 3983))) :org-element--cache-sync-key (35 . 4743))))
  org-element-at-point(4774)
  org-element-cache-map(ignore :granularity headline+inlinetask)
  org-element-cache-map(#f(compiled-function (el) #<bytecode 0x155dbd1a1939>) :next-re "^\\*+ " :fail-re "^\\*+ " :narrow t)
  org-scan-tags(org-encrypt-entry (lambda (todo tags-list level) (progn (setq org-cached-props nil) (or (and (member "crypt" tags-list))))) nil)
  org-encrypt-entries()
  funcall(org-encrypt-entries)
  (condition-case e (funcall hook) ((debug error) (signal 'doom-hook-error (list hook e))))
  doom-run-hook(org-encrypt-entries)
  run-hook-wrapped(doom-run-hook org-encrypt-entries)
  (condition-case e (run-hook-wrapped hook #'doom-run-hook) ((debug doom-hook-error) (if debug-on-error nil (lwarn hook :error "Error running hook %S because: %s" (if (symbolp (car (cdr e))) (symbol-name (car (cdr e))) (car (cdr e))) (car (cdr (cdr e))))) (signal 'doom-hook-error (cons hook (cdr e)))))
  (let ((hook (car --dolist-tail--))) (condition-case e (run-hook-wrapped hook #'doom-run-hook) ((debug doom-hook-error) (if debug-on-error nil (lwarn hook :error "Error running hook %S because: %s" (if (symbolp (car ...)) (symbol-name (car ...)) (car (cdr e))) (car (cdr (cdr e))))) (signal 'doom-hook-error (cons hook (cdr e))))) (setq --dolist-tail-- (cdr --dolist-tail--)))
  (while --dolist-tail-- (let ((hook (car --dolist-tail--))) (condition-case e (run-hook-wrapped hook #'doom-run-hook) ((debug doom-hook-error) (if debug-on-error nil (lwarn hook :error "Error running hook %S because: %s" (if (symbolp ...) (symbol-name ...) (car ...)) (car (cdr ...)))) (signal 'doom-hook-error (cons hook (cdr e))))) (setq --dolist-tail-- (cdr --dolist-tail--))))
  (let ((--dolist-tail-- hooks)) (while --dolist-tail-- (let ((hook (car --dolist-tail--))) (condition-case e (run-hook-wrapped hook #'doom-run-hook) ((debug doom-hook-error) (if debug-on-error nil (lwarn hook :error "Error running hook %S because: %s" (if ... ... ...) (car ...))) (signal 'doom-hook-error (cons hook (cdr e))))) (setq --dolist-tail-- (cdr --dolist-tail--)))))
  doom-run-hooks(before-save-hook)
  apply(doom-run-hooks before-save-hook)
  run-hooks(before-save-hook)
  basic-save-buffer(t)
  save-buffer(1)
  funcall-interactively(save-buffer 1)
  call-interactively(save-buffer nil nil)
  command-execute(save-buffer)

I passed pkill -USR2 emacs and it entered the debug trace. Here's the paste for that. This is for when emacs hangs on export.

Debugger entered--Lisp error: (quit)
  format("%S" #s(avl-tree- :dummyroot [[[[[[... ... ... 0] [... ... ... 0] (headline ...) 0] [[... ... ... 0] [... ... ... 0] (headline ...) 0] (headline (:raw-value "Tools I use" :begin 12683 :end 13000 :pre-blank 0 :contents-begin 12703 :contents-end 13000 :robust-begin 12752 :robust-end 12998 :level 2 :priority nil :tags nil :todo-keyword ... :todo-type todo :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 12683 :EXPORT_FILE_NAME "tools-i-use" :title "Tools I use" :mode nil :granularity element :cached t :parent ... :org-element--cache-sync-key nil)) 0] [[[... ... ... 0] [... ... ... 0] (headline ...) 0] [[... ... ... 0] [... ... ... 1] (headline ...) -1] (headline (:raw-value "Learning The Art of Computer Programming" :begin 39962 :end 40122 :pre-blank 0 :contents-begin 40043 :contents-end 40122 :robust-begin nil :robust-end nil :level 2 :priority nil :tags ... :todo-keyword ... :todo-type todo :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 39962 :EXPORT_FILE_NAME "learning-the-art-of-computer-programming" :title "Learning The Art of Computer Programming" :mode nil :granularity element :cached t :parent ... :org-element--cache-sync-key nil)) 0] (headline (:raw-value "Comparing CSAPP and COA (Hennessey Patterson)" :begin 23190 :end 24083 :pre-blank 0 :contents-begin 23260 :contents-end 24083 :robust-begin 23342 :robust-end 24081 :level 2 :priority nil :tags (...) :todo-keyword #("TODO" 0 4 ...) :todo-type todo :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 23190 :EXPORT_FILE_NAME "comparing-csapp-and-coa--hennessey-patterson" :title "Comparing CSAPP and COA (Hennessey Patterson)" :mode nil :granularity element :cached t :parent (headline ...) :org-element--cache-sync-key nil)) 0] [[[[... ... ... 0] [... ... ... -1] (headline ...) 0] [[... ... ... 0] [... ... ... 0] (headline ...) 0] (headline (:raw-value "Resume in orgmode" :begin 70673 :end 76445 :pre-blank 0 :contents-begin 70743 :contents-end 76445 :robust-begin 70914 :robust-end 76443 :level 2 :priority nil :tags ... :todo-keyword ... :todo-type done :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 70673 :closed ... :EXPORT_FILE_NAME "resume-in-orgmode" :EXPORT_HUGO_CUSTOM_FRONT_MATTER ":series \"100 Days to Offload challenge\" :code true" :title "Resume in orgmode" :mode nil :granularity element :cached t ...)) -1] [[[... ... ... 0] [... ... ... 0] (headline ...) 0] [[... ... ... 0] [... ... ... 0] (headline ...) 1] (headline (:raw-value "Resume in yaml and jinja templates" :begin 98237 :end 103692 :pre-blank 0 :contents-begin 98322 :contents-end 103692 :robust-begin 98510 :robust-end 103690 :level 2 :priority nil :tags ... :todo-keyword ... :todo-type done :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 98237 :closed ... :EXPORT_FILE_NAME "resume-in-yaml-and-jinja-templates" :EXPORT_HUGO_CUSTOM_FRONT_MATTER ":series \"100 Days to Offload challenge\" :code true" :title "Resume in yaml and jinja templates" :mode nil :granularity element :cached t ...)) 0] (headline (:raw-value "Disc Math" :begin 80588 :end 80637 :pre-blank 0 :contents-begin 80602 :contents-end 80637 :robust-begin 80604 :robust-end 80635 :level 3 :priority nil :tags nil :todo-keyword nil :todo-type nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 80588 :title "Disc Math" :mode nil :granularity element :cached t :parent (headline ...) :org-element--cache-sync-key (3 . 80588))) -1] (headline (:raw-value "How to procrastinate" :begin 55270 :end 55358 :pre-blank 0 :contents-begin 55299 :contents-end 55358 :robust-begin nil :robust-end nil :level 2 :priority nil :tags nil :todo-keyword #("TODO" 0 4 (face ... wrap-prefix ... line-prefix ... ws-butler-chg chg fontified t)) :todo-type todo :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 55270 :EXPORT_FILE_NAME "how-to-procrastinate" :title "How to procrastinate" :mode nil :granularity element :cached t :parent (headline (:raw-value "Posts" :begin 4041 :end 112268 :pre-blank 0 :contents-begin 4049 :contents-end 112268 :robust-begin 4094 :robust-end 112266 :level 1 :priority nil :tags nil :todo-keyword nil :todo-type nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 4041 :EXPORT_HUGO_SECTION "post" :title ... :mode nil :granularity nil :parent ... :cached t :org-element--cache-sync-key nil)) :org-element--cache-sync-key (3 . 55270))) 0] nil nil 0] :cmpfun org-element--cache-compare))
  org-hugo--get-pre-processed-buffer()
  org-hugo--export-subtree-to-md(nil nil)
  org-hugo-export-wim-to-md(nil nil nil)
  (lambda (a _s v _b) (org-hugo-export-wim-to-md nil a v))(nil nil nil nil)
  org-export-dispatch(nil)
  funcall-interactively(org-export-dispatch nil)
  call-interactively(org-export-dispatch nil nil)
  command-execute(org-export-dispatch)

Please let me know if that was sufficient. If not I can execute your recommendation below.

Here's how you can get more debug info:
Set debug-on-quit to t. This will show the backtrace each time you hit C-g.
Try to reproduce the hang and while emacs is hung, hit C-g. Please share the backtrace you see next.

0 replies

kaushalmodi · 2021-12-30T16:36:38Z

kaushalmodi
Dec 30, 2021
Maintainer

This is for when emacs hangs on save.

This looks related to the doom config and the new Org cache feature in Org. This is unrelated to exports. Can you report this to the Org mailing list?

Here's the paste for that. This is for when emacs hangs on export.

org-hugo--get-pre-processed-buffer()

I am aware of that performance bottleneck in that function. That function is needed if you have Org internal links from one subtree to another. If you don't have links from one post subtree to another, I can come up with a variable that skips this pre-processing.

0 replies

kaushalmodi · 2021-12-30T17:23:55Z

kaushalmodi
Dec 30, 2021
Maintainer

If you have examples (sites) you know of who have large number of separate org files, that'd also be helpful.

You might find more than one example on how to do per-file exports on https://ox-hugo.scripter.co/doc/examples/ .

But if you are exporting only one subtree at a time, it should not take minutes to export! I would like to fix this in ox-hugo if possible or at least identify the root cause and have it fixed elsewhere (in Org or emacs). Your help in debugging this will be appreciated.

0 replies

kishvanchee · 2021-12-30T17:47:36Z

kishvanchee
Dec 30, 2021
Author

I am aware of that performance bottleneck in that function. That function is needed if you have Org internal links from one subtree to another. If you don't have links from one post subtree to another, I can come up with a variable that skips this pre-processing.

Unfortunately, I do have internal links from one subtree to another.

But if you are exporting only one subtree at a time, it should not take minutes to export! I would like to fix this in ox-hugo if possible or at least identify the root cause and have it fixed elsewhere (in Org or emacs). Your help in debugging this will be appreciated.

I did this subtree export once again and the CPU usage was so high my system hung. I had to restart now.

I have invited you on gitlab for my website https://gitlab.com/kishvanchee/personal-website/ to your username https://gitlab.com/kaushalmodi . Please feel free to clone and use it to debug ox-hugo.

Please let me know if you require anything from my end to help you debug this.

0 replies

kishvanchee · 2021-12-30T17:51:29Z

kishvanchee
Dec 30, 2021
Author

You might find more than one example on how to do per-file exports on https://ox-hugo.scripter.co/doc/examples/ .

Yes thank you I found it helpful. https://github.com/xianmin/xianmin.org/tree/master/orgpost is one that stands out that does both subtree and per-file exports (per year). If this doesn't get resolved in ox-hugo, most probably I'd have to adopt both approaches at the same time.

This looks related to the doom config and the new Org cache feature in Org. This is unrelated to exports. Can you report this to the Org mailing list?

I feel this is mostly related to only Org and not doom configs. I'll try to report this on the Org mailing list (I'm new to that 😅 ). Maybe I'll have to downgrade Org for now to 9.4 to see how it goes.

0 replies

kishvanchee · 2021-12-30T18:05:17Z

kishvanchee
Dec 30, 2021
Author

Since my system restarted, I'm not facing the delay/hang during the export of the subtree right now. Maybe I'll be able to give you a better traceback at a later point of time. Here's one traceback I could collect by pressing C-g on time.

Debugger entered--Lisp error: (quit)
  write-region(nil nil "/home/kishore/.cache/org-persist//27/a897ab-5f2b-4..." nil 0)
  org-persist-write((org-element--headline-cache org-element--cache) #<buffer content.org>)
  org-persist-write-all(#<buffer *Ox-hugo Pre-processed content.org *<2><2>>)
  org-persist-write-all-buffer()
  kill-buffer(#<buffer *Ox-hugo Pre-processed content.org *<2><2>>)
  #f(compiled-function () #<bytecode 0x1580fb51cc4d>)()
  org-babel-exp-process-buffer()
  #f(compiled-function (backend &optional subtreep visible-only body-only ext-plist) "Transcode current Org buffer into BACKEND code.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.\n\nIf narrowing is active in the current buffer, only transcode its\nnarrowed part.\n\nIf a region is active, transcode that region.\n\nWhen optional argument SUBTREEP is non-nil, transcode the\nsub-tree at point, extracting information from the headline\nproperties first.\n\nWhen optional argument VISIBLE-ONLY is non-nil, don't export\ncontents of hidden elements.\n\nWhen optional argument BODY-ONLY is non-nil, only return body\ncode, without surrounding template.\n\nOptional argument EXT-PLIST, when provided, is a property list\nwith external parameters overriding Org default settings, but\nstill inferior to file-local settings.\n\nReturn code as a string." #<bytecode 0x1580fc4f42d9>)(hugo :subtreep nil nil (:output-file "/home/kishore/kishore_repositories/personal-websit..."))
  apply(#f(compiled-function (backend &optional subtreep visible-only body-only ext-plist) "Transcode current Org buffer into BACKEND code.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.\n\nIf narrowing is active in the current buffer, only transcode its\nnarrowed part.\n\nIf a region is active, transcode that region.\n\nWhen optional argument SUBTREEP is non-nil, transcode the\nsub-tree at point, extracting information from the headline\nproperties first.\n\nWhen optional argument VISIBLE-ONLY is non-nil, don't export\ncontents of hidden elements.\n\nWhen optional argument BODY-ONLY is non-nil, only return body\ncode, without surrounding template.\n\nOptional argument EXT-PLIST, when provided, is a property list\nwith external parameters overriding Org default settings, but\nstill inferior to file-local settings.\n\nReturn code as a string." #<bytecode 0x1580fc4f42d9>) (hugo :subtreep nil nil (:output-file "/home/kishore/kishore_repositories/personal-websit...")))
  (let ((old-async-init-file org-export-async-init-file) (org-export-async-init-file (make-temp-file "doom-org-async-export"))) (let ((temp-file org-export-async-init-file) (temp-buffer (generate-new-buffer " *temp file*"))) (unwind-protect (prog1 (save-current-buffer (set-buffer temp-buffer) (prin1 (list (quote progn) (list ... ... ... ... ...) (cons ... ...)) (current-buffer))) (save-current-buffer (set-buffer temp-buffer) (write-region nil nil temp-file nil 0))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))) (apply fn args))
  +org--fix-async-export-a(#f(compiled-function (backend &optional subtreep visible-only body-only ext-plist) "Transcode current Org buffer into BACKEND code.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.\n\nIf narrowing is active in the current buffer, only transcode its\nnarrowed part.\n\nIf a region is active, transcode that region.\n\nWhen optional argument SUBTREEP is non-nil, transcode the\nsub-tree at point, extracting information from the headline\nproperties first.\n\nWhen optional argument VISIBLE-ONLY is non-nil, don't export\ncontents of hidden elements.\n\nWhen optional argument BODY-ONLY is non-nil, only return body\ncode, without surrounding template.\n\nOptional argument EXT-PLIST, when provided, is a property list\nwith external parameters overriding Org default settings, but\nstill inferior to file-local settings.\n\nReturn code as a string." #<bytecode 0x1580fc4f42d9>) hugo :subtreep nil nil (:output-file "/home/kishore/kishore_repositories/personal-websit..."))
  apply(+org--fix-async-export-a #f(compiled-function (backend &optional subtreep visible-only body-only ext-plist) "Transcode current Org buffer into BACKEND code.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.\n\nIf narrowing is active in the current buffer, only transcode its\nnarrowed part.\n\nIf a region is active, transcode that region.\n\nWhen optional argument SUBTREEP is non-nil, transcode the\nsub-tree at point, extracting information from the headline\nproperties first.\n\nWhen optional argument VISIBLE-ONLY is non-nil, don't export\ncontents of hidden elements.\n\nWhen optional argument BODY-ONLY is non-nil, only return body\ncode, without surrounding template.\n\nOptional argument EXT-PLIST, when provided, is a property list\nwith external parameters overriding Org default settings, but\nstill inferior to file-local settings.\n\nReturn code as a string." #<bytecode 0x1580fc4f42d9>) (hugo :subtreep nil nil (:output-file "/home/kishore/kishore_repositories/personal-websit...")))
  org-export-as(hugo :subtreep nil nil (:output-file "/home/kishore/kishore_repositories/personal-websit..."))
  #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>)(hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil)
  apply(#f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) (hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil))
  (let (before-save-hook after-save-hook) (apply fn args))
  +org--dont-trigger-save-hooks-a(#f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil)
  apply(+org--dont-trigger-save-hooks-a #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) (hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil))
  #f(advice-wrapper :around #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) +org--dont-trigger-save-hooks-a)(hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil)
  apply(#f(advice-wrapper :around #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) +org--dont-trigger-save-hooks-a) (hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil))
  (let ((old-async-init-file org-export-async-init-file) (org-export-async-init-file (make-temp-file "doom-org-async-export"))) (let ((temp-file org-export-async-init-file) (temp-buffer (generate-new-buffer " *temp file*"))) (unwind-protect (prog1 (save-current-buffer (set-buffer temp-buffer) (prin1 (list (quote progn) (list ... ... ... ... ...) (cons ... ...)) (current-buffer))) (save-current-buffer (set-buffer temp-buffer) (write-region nil nil temp-file nil 0))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))) (apply fn args))
  +org--fix-async-export-a(#f(advice-wrapper :around #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) +org--dont-trigger-save-hooks-a) hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil)
  apply(+org--fix-async-export-a #f(advice-wrapper :around #f(compiled-function (backend file &optional async subtreep visible-only body-only ext-plist post-process) "Call `org-export-as' with output to a specified file.\n\nBACKEND is either an export back-end, as returned by, e.g.,\n`org-export-create-backend', or a symbol referring to\na registered back-end.  FILE is the name of the output file, as\na string.\n\nA non-nil optional argument ASYNC means the process should happen\nasynchronously.  The resulting buffer will then be accessible\nthrough the `org-export-stack' interface.\n\nOptional arguments SUBTREEP, VISIBLE-ONLY, BODY-ONLY and\nEXT-PLIST are similar to those used in `org-export-as', which\nsee.\n\nOptional argument POST-PROCESS is called with FILE as its\nargument and happens asynchronously when ASYNC is non-nil.  It\nhas to return a file name, or nil.  Export back-ends can use this\nto send the output file through additional processing, e.g,\n\n  (defun org-latex-export-to-latex\n    (&optional async subtreep visible-only body-only ext-plist)\n    (interactive)\n    (let ((outfile (org-export-output-file-name \".tex\" subtreep)))\n      (org-export-to-file \\='latex outfile\n        async subtreep visible-only body-only ext-plist\n        #'org-latex-compile)))\n\nWhen expressed as an anonymous function, using `lambda',\nPOST-PROCESS needs to be quoted.\n\nThe function returns either a file name returned by POST-PROCESS,\nor FILE." #<bytecode 0x1580fc60cb09>) +org--dont-trigger-save-hooks-a) (hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil))
  org-export-to-file(hugo "/home/kishore/kishore_repositories/personal-websit..." nil :subtreep nil)
  org-hugo-export-to-md(nil :subtreep nil)
  org-hugo--export-subtree-to-md(nil nil)
  org-hugo-export-wim-to-md(nil nil nil)
  (lambda (a _s v _b) (org-hugo-export-wim-to-md nil a v))(nil nil nil nil)
  org-export-dispatch(nil)
  funcall-interactively(org-export-dispatch nil)
  call-interactively(org-export-dispatch nil nil)
  command-execute(org-export-dispatch)

I hope that is helpful, if not please let me know.

0 replies

kishvanchee · 2021-12-30T19:29:21Z

kishvanchee
Dec 30, 2021
Author

Temporarily I have downgraded Org to 9.4.6 from here - https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tag/?h=release_9.4.6. Hoping this would solve some of the issues 🤞

0 replies

reyman · 2021-12-30T21:49:30Z

reyman
Dec 30, 2021

You might find more than one example on how to do per-file exports on https://ox-hugo.scripter.co/doc/examples/ .

Yes thank you I found it helpful. https://github.com/xianmin/xianmin.org/tree/master/orgpost is one that stands out that does both subtree and per-file exports (per year). If this doesn't get resolved in ox-hugo, most probably I'd have to adopt both approaches at the same time.

This looks related to the doom config and the new Org cache feature in Org. This is unrelated to exports. Can you report this to the Org mailing list?

I feel this is mostly related to only Org and not doom configs. I'll try to report this on the Org mailing list (I'm new to that 😅 ). Maybe I'll have to downgrade Org for now to 9.4 to see how it goes.

I'm also present (and probably Nobiot) on emacs org mailling list. If you need some help to debug this problem with :w . Like you probably already see i also open an issue to doom, waiting for a reply : doomemacs/doomemacs#5924

0 replies

finex · 2022-01-03T13:08:52Z

finex
Jan 3, 2022

Hi, I'm experiencing the same issue with Doom Emacs (28-git from archlinux repo) and latest org 9.5.2. I'm using the auto export on save, but it does freeze even with the manual export. I've to press C-g, otherwise it stops endlessly.

If you can tell me how to downgrade org version at least we can understand if the problem is org 9.5.

Thank you.

0 replies

kishvanchee · 2022-01-03T13:27:00Z

kishvanchee
Jan 3, 2022
Author

@finex If you're using doom-emacs you can pin packages to a specific commit like this in packages.el and do doom sync -u. The following pins it to 9.4.6. You can pin it according to the commit from the releases tags here

(package! org :recipe
  (:host github
   :repo "emacs-straight/org-mode")
  :pin "652430128896e690dc6ef2a83891a1209094b3da")

0 replies

finex · 2022-01-03T15:47:14Z

finex
Jan 3, 2022

@kishvanchee Hi, I confirm that pinning the version to 9.4.6 resolve the problem. Do you think it's a problem on org-mode and it will be fixed in the future or should a fix be implemented on the ox-hugo project to maintain the compatibility with the latest org-mode version? Thank you very much!

0 replies

kaushalmodi · 2022-01-03T15:53:56Z

kaushalmodi
Jan 3, 2022
Maintainer

@finex Given that rolling back the version of Org mode fixes this problem, it's caused due to a change in Org mode. Now, the change could be good and could be pointing to an inefficient piece of code in ox-hugo.

But in order to fix it, we will need to share a reproducible example with Org mode list so that the folks there can take a look at this.

I have seen the same issue, but it's intermittent. So I am unable to come up with a recipe. If you have an Org file that you can share, that always shows this problem, I can share that with the list. Otherwise, I'll keep on looking for a way to reproduce this problem consistently.

0 replies

kishvanchee · 2022-01-03T15:58:07Z

kishvanchee
Jan 3, 2022
Author

@kaushalmodi I'm not sure if this would help, do you think an org file with say 200 entries help you debug? I'm thinking of a recipe like

* Posts
** Post 001
:PROPERTY:
...
:END:
Lorem ipsum ... <paragraphs>
** Post 002
...
** Post 200
...

If yes I can whip up a quick script to produce the file

0 replies

kaushalmodi · 2022-01-04T00:02:42Z

kaushalmodi
Jan 4, 2022
Maintainer

Today, I saw Emacs freezing due to the simple Org tagging command (C-c C-q). Tagging was snappy once again once I disabled caching on Org elements:

(setq org-element-use-cache nil)

@kishvanchee @finex If you have time, please update to the latest Org version from its main branch, disable cache and see if that slowness or hanging persists.

0 replies

finex · 2022-01-04T10:23:06Z

finex
Jan 4, 2022

@kaushalmodi Hi, I've updated to the latest Org version and disabled encryption like said on doomemacs/doomemacs#5924 (comment): (remove-hook 'before-save-hook 'org-encrypt-entries t) ... and it worked.

Now I will try to follow your suggestion. I will tell you the results later.

Thank you.

0 replies

finex · 2022-01-05T07:35:18Z

finex
Jan 5, 2022

Hi, I've just updated to the latest doom-emacs version + latest ox-hugo, removed both the temporary fixes (the cache one and the before-safe-hook one) and now both the org export and org tagging are working flawlessly.

I've tried to add the:

(setq org-element-use-cache nil)

and it still works fine.

The org version is still the same but the ox-hugo is the one from yesterday commits.

Let's see what it happens on the next days. If it is an intermittent problem or if it is really fixed with the latest version.

0 replies

finex · 2022-01-05T08:03:40Z

finex
Jan 5, 2022

Ok, it has freezed again after 30 minutes. Now I will add (setq org-element-use-cache nil). Let's see.

0 replies

finex · 2022-01-05T08:08:18Z

finex
Jan 5, 2022

Ok, using setq org-element-use-cache nil also fix the problem.

0 replies

rodelrod · 2022-01-29T14:57:58Z

rodelrod
Jan 29, 2022

I've been having this problem occasionally.

Just had it in a 26-line file. It was freezing saving but also just moving around the buffer. C-g wasn't working most of the time, had to pkill -USR2 emacs.

(setq org-element-use-cache nil) was the only way that allowed me to finally save the file.

0 replies

kaushalmodi · 2022-01-29T15:09:25Z

kaushalmodi
Jan 29, 2022
Maintainer

(setq org-element-use-cache nil) was the only way that allowed me to finally save the file.

@rodelrod Please report this on the Org mailing list so that it can be fixed in the devel or main branch of Org mode. The Org element caching is a bleeding edge feature under development.

0 replies

kaushalmodi · 2022-02-03T14:52:17Z

kaushalmodi
Feb 3, 2022
Maintainer

Hello all, I have converted this from an Issue to a Discussion as there's not much I can do from ox-hugo side to fix this. I encourage people facing this issue to create a reproducible example and email that to [email protected].

0 replies

kaushalmodi · 2022-02-25T14:56:33Z

kaushalmodi
Feb 25, 2022
Maintainer

@kishvanchee @finex Please see https://www.reddit.com/r/orgmode/comments/t15c3f/org_mode_profiling_meetup_on_sat_feb_26_9pm_sg/ -- there is going to be a live debug session related to performance when using Org built from the main branch (the one having the latest org-cache and org-persist updates).

0 replies

koustuvsinha · 2022-07-20T14:19:40Z

koustuvsinha
Jul 20, 2022

Hi @kaushalmodi, any updates on this? My org file is about 2000 lines long, and I faced the same issue of freezing while exporting and saving. Rolling back to Org 9.4.6 seems to work in my case (I already had (setq org-element-use-cache nil) which did not work). However, that way I miss using the recent features in Org :(

0 replies

kaushalmodi · 2022-07-20T14:25:22Z

kaushalmodi
Jul 20, 2022
Maintainer

@koustuvsinha Sorry, there are still no updates because no one has been able to create a consistent recipe to reproduce this problem. I use the latest Org mode from its main branch and set (setq org-element-use-cache nil) and it has been working alright for me. My site's Org file is at 11k lines right now and subtree exports finish in about 1 or 2 seconds.

0 replies

koustuvsinha · 2022-07-20T14:31:25Z

koustuvsinha
Jul 20, 2022

Thanks for the quick reply! Can you share the commit id you are using? I'll try to replicate it with the same version as you.

6 replies

koustuvsinha Jul 20, 2022

Thanks! I updated to 39005dc09 and it does seem the issue is not there (saving and exporting works). I'll observe if I hit the same issue again.

yantar92 Jul 22, 2022

For record. I try to prioritize all the performance-related issues on Org ML. They are usually fixed within days, given that enough information is provided after following the debugging suggestions. Reproducer is not always required - we have a specifically designed logger to debug errors in this area. Please report any performance issues to Org mailing list (see https://orgmode.org/manual/Feedback.html). Also, upgrading to the latest Org version often helps - we haven't had any reports in this area for weeks.

kamazeuci Oct 27, 2022

I have my org file that is giving trouble, who/where do I send it to?

kamazeuci Oct 27, 2022

Ok. I can send it to orgmode feedback

kamazeuci Oct 27, 2022

But the thing is that when running emacs -q , I don't have the problem... just with doom emacs...

kamazeuci · 2022-10-27T11:37:31Z

kamazeuci
Oct 27, 2022

I'm having the exact same issue.

Should I try to file an issue somewhere or this is already being worked on?
(I posted the problem on doom emacs forum but it triggered a spam filter, waiting for it to be reviewed)
https://discourse.doomemacs.org/t/org-encrypt-entries-before-save-hook-wont-let-me-save-my-org-file/3349/1

GNU Emacs 28.2.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.34, cairo version 1.17.6) of 2022-10-12
Org mode version 9.6 (9.6-??-86c4635
hugo v0.104.3+extended linux/amd64 BuildDate=unknown

3 replies

kaushalmodi Oct 27, 2022
Maintainer

Should I try to file an issue somewhere or this is already being worked on?

This isn't being worked on. Can you file a reproducible recipe for this issue to the Org mailing list?

See #551 (reply in thread)

kamazeuci Oct 27, 2022

thanks @kaushalmodi , I replied in thread.

What is the final proposed workaround in the meanwhile?

(remove-hook 'before-save-hook 'org-encrypt-entries t)
or
(setq org-element-use-cache nil)
?
This would just go anywhere on my ~/.doom.d/config.el file?

thanks

kamazeuci Oct 27, 2022

I tried
(remove-hook 'before-save-hook 'org-encrypt-entries t) on ~/.doom.d/config.el , then doom sync, and seems to have worked.

yantar92 · 2022-10-27T23:05:32Z

yantar92
Oct 27, 2022

kamazeuci ***@***.***> writes:

But the thing is that when running emacs -q , I don't have the problem... just with doom emacs...

This is an annoying issue. If you wish, we can try to debug via screen sharing.

…

-- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>

1 reply

kamazeuci Oct 30, 2022

thanks!
I really kinda worked around it, so it is not a problem anymore for me, but if by debugging it we help the org or ox-hugo project, I'd be happy to have an online session
thanks!

yantar92 · 2022-10-30T02:35:05Z

yantar92
Oct 30, 2022

kamazeuci ***@***.***> writes:

I really kinda worked around it, so it is not a problem anymore for me, but if by debugging it we help the org or ox-hugo project, I'd be happy to have an online session

Let me know when you are available and your timezone. You can contact me by email. See my github page.

…

-- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>

0 replies

Post exports hang or take a very long time when single org file gets large #551

Actual Behavior

Expected Behavior

How to Reproduce the Issue

Ox-Hugo Debug Information

Debug information for ox-hugo

Emacs Version

Org Version

Hugo Version

Org load-path shadows

ox-hugo defcustoms

Replies: 30 comments · 10 replies

kaushalmodi Dec 30, 2021 Maintainer

kaushalmodi Dec 30, 2021 Maintainer

kishvanchee Dec 30, 2021 Author

kaushalmodi Dec 30, 2021 Maintainer

kaushalmodi Dec 30, 2021 Maintainer

kishvanchee Dec 30, 2021 Author

kishvanchee Dec 30, 2021 Author

kishvanchee Dec 30, 2021 Author

kishvanchee Dec 30, 2021 Author

kishvanchee Jan 3, 2022 Author

kaushalmodi Jan 3, 2022 Maintainer

kishvanchee Jan 3, 2022 Author

kaushalmodi Jan 4, 2022 Maintainer

kaushalmodi Jan 29, 2022 Maintainer

kaushalmodi Feb 3, 2022 Maintainer

kaushalmodi Feb 25, 2022 Maintainer

kaushalmodi Jul 20, 2022 Maintainer

kaushalmodi Oct 27, 2022 Maintainer

Debug information for `ox-hugo`

Org `load-path` shadows

`ox-hugo` defcustoms

Replies: 30 comments 10 replies

kaushalmodi
Dec 30, 2021
Maintainer

kaushalmodi
Dec 30, 2021
Maintainer

kishvanchee
Dec 30, 2021
Author

kaushalmodi
Dec 30, 2021
Maintainer

kaushalmodi
Dec 30, 2021
Maintainer

kishvanchee
Dec 30, 2021
Author

kishvanchee
Dec 30, 2021
Author

kishvanchee
Dec 30, 2021
Author

kishvanchee
Dec 30, 2021
Author

kishvanchee
Jan 3, 2022
Author

kaushalmodi
Jan 3, 2022
Maintainer

kishvanchee
Jan 3, 2022
Author

kaushalmodi
Jan 4, 2022
Maintainer

kaushalmodi
Jan 29, 2022
Maintainer

kaushalmodi
Feb 3, 2022
Maintainer

kaushalmodi
Feb 25, 2022
Maintainer

kaushalmodi
Jul 20, 2022
Maintainer

kaushalmodi Oct 27, 2022
Maintainer