On the general state and direction of neorg and the ecosystem, especially GTD and Task Management

[langleyVa]

I currently have the short-term and personal need for a piece of software for writing, knowledge management, and task management a la GTD or Bullet Journaling. As an avid vim (and new neovim) user who always thought markdown was ugly, neorg looks wonderful to me. I love the format, I love the vision (especially the insistence on a rigid extensible specification that should help avoid the language splintering into dialects like markdown did), and the code looks nice and modular so that I am confident that I could whip out a few modules for my specific workflow needs.

On the other hand, I am confused and hesitant. Everything on the net about the GTD workflow in neorg refers to a module that was removed for the 1.0.0 language spec. Issue #695 is close to a year old now. In other repos, there's rudimentary code and extensive roadmaps for new parsers for the format, the norgoplis server, the norgberg database, and so on.

My question is, and I will just go ahead and ping @vhyrro and @danymat directly here: How stable and future-proof is the neorg code right now? I would be fine with adopting the format a little early, writing missing modules myself when I need them, encountering bugs, having to hustle a little to keep my modules up-to-date with neorg, and maybe even giving back to the project. But with the discussions about that whole server and database work and some other comments I saw e.g. on Reddit, I wonder if I would step on very shaky and moving ground if I enter the ecosystem right now.

The concrete external modules I would like to write would be 1. one that uses the journal and dirman modules to move/copy Todo items between an Inbox and the journals daily pages, or from one journal page to the next, like the bullet journal '>' marker does; and 2. a module that traverses the workspace via dirman and treesitter for other Todo items and generates a list of links at a certain spot (Like the old Table of Contents that could be generated under the "=TOC" marker). I'll try to find a syntax within the spec that allows for basic queries, "Put all open Todos here, put all open Todos scheduled for the next week here" etc. Both modules would be very narrowly tailored to my workflow at first, but might get more configurable over time, and some concepts or functions (or even just fixes for bugs, depreciation warnings or lacking documentation that I encounter) might be useful enough for everyone that I could offer them back in a PR.

I wonder e.g. if I write a module right now that uses treesitter queries, that whole module might need rewriting when tree-sitter-norg3 comes out. Every module I read, like dirman, I wonder if it even makes sense to rely on it now, because maybe that module is just there for the time being and will be thrown out completely once norgopolis is ready. I'm concerned that I might start off in the wrong direction and waste time and effort that could benefit us all more if I had just asked which direction you are going and tried to align and keep up.

If you say "Hey man, go for it, norgopolis will take months and when it's ready it will drop in with almost no friction!", I will. If you say "Okay, but lets talk first because things are indeed moving and we will tell you when and where so you don't end up under the wheels", let's talk. If you say "Sorry, but things are really moving behind the scenes. You'd waste your time right now and should check back in $TIME_FRAME when the big infrastructure changes are through", I'll have to clench my eyes and butt cheeks together and install logseq or vimwiki for the mean time.

Thanks for your time (reading this and writing neorg, it's awesome),
Thomas

[vhyrro]

Hey! Really good thing you're asking about this.

I think you're okay to purse your ideas, and here's why:

The functionalities of dirman and a few other modules are indeed being moved to norgopolis (aka the server backend), BUT that doesn't mean the core.dirman module will disappear - it just means that dirman will now shell out to norgopolis for its data, but its functions will remain the same. You can use the modules without much worry about them totally being changed.

If they ever are changed (not really likely), then the changes will definitely be put under a different version of Neorg, so as long as you're versioning everything in Neovim nothing should arbitrarily break. Sure, it may now take a while to rewrite some functionality to the latest version, but that's better than it totally failing haha.

The biggest breakage will be tree-sitter-norg3 for sure! If you write your queries and captures agnostically (aka (unordered_list1) @mylist) then it won't take much to just change the query later on to (unordered_list) @mylist.
Heck, after the breaking change feel free to just send me your queries and I can even rewrite them for you :)

Generally I would consider the Neorg code to be very solidly future proofed (that was my goal when writing the codebase). Things may break months/years down the road, but because everything is just a module the impact is nowhere near as bad as something like an entangled codebase where functionality is intertwined.

Issue #695 is close to a year old now. In other repos, there's rudimentary code and extensive roadmaps for new parsers for the format, the norgoplis server, the norgberg database, and so on

Yeah I feel you on this one. The GTD 1.0.0 issue is super old, but that's because there was soo much preliminary work to do to even consider a rewrite of GTD in the first place. New parsers, server backends, theoretical considerations, I even started contributing to the nvim-neorocks organization to try to get luarocks standardized in Neovim so installing Neorg's GTD is even feasible. It's been a long ride, but there's a lot of good news:

• All of the preliminary work is completely done. All the pieces will fall into place by the time GTD is implemented.
• I have been spending the past several days (and will continue) writing up a full GTD specification which describes GTD from head to toe (UIs, keybinds, features). I published a WIP document on the Neorg discord and we have some discussion going on there.
• After the GTD spec is done I and anyone else willing to help with the implementation will start working full force on implementing all of the described UIs, behaviours and databases. I expect this process to take about a month in of itself.

If you have any other questions feel free to ask!

[jgregoire]

I am very excited to switch my GTD system from Obsidian to Neorg. I have been lurking these repos for a while. 😄 I am willing to help with testing, proofreading, etc. if that is useful to you.

[vhyrro]

Super happy to hear! I'll let everyone on this thread know once we have a basic working version of gtd (if I don't forget of course!)

[langleyVa]

Vhyrro,

I love this answer. In a narrower it's just plain helpful, it contains all the information I was looking for. Hearing that GTD is well under way behind the scenes is also great. In a broader sense, I love the attitude that your answer displays: collaborative, balanced, really focused on the ecosystem-as-a-whole. I love that, it instills a lot of confidence. You're doing a great job.

I'll get myself sorted out: Line out the use-cases and features, have another look at the code and the specs, identify dependencies etc (anyone know a good notes app that I could use for that?) I'll keep the first steps small and the dependencies contained, doubly so for treesitter and the queries module (Thanks for the heads up!) I'll also keep half an eye on tree-sitter-norg3 so I see the waves coming before they hit. Thanks for the offer to help with the migration, I'll speak up when I need it. But, like, don't worry, it's okay if you focus on the bigger picture even if that means that I have to do a few updates every now and then.

When I find new dependencies and have new questions, I can either come back here, or (just an idea, either way is fine by me, you make the call) you could add a dev section here in the discussions to make those answers more searchable for other devs as well.

As to the upcoming GTD spec, you said you posted a WIP on discord. Are you referring to the VIP discord for your Patreon patrons? Is all GTD development going to happen there, or is there going to be a more public forum where the spec is published and development is organized? See also #1097 . Again, I think it would make sense if I align my stuff to the upcoming spec early. Maybe I can make my stuff play nice together with yours, or your features are even similar enough to mine that I toss mine and come over and help with yours instead.

[vhyrro]

Thanks so much for the kind words!

The discord is publicly available for anyone to join, no payment needed :p. Patreons and supporters simply gain access to some special channels where they can ask high priority questions. All GTD discussion is in the open on the #neorg-gtd channel.

The WIP GTD spec is also pushed to the dev notes, so check them out there! Here's a special version that I converted to pdf using the WIP pandoc parser: gtd-design-goals.pdf. I'll also post this PDF to the original GTD github thread so others can see.

If you have development questions along the way then you can always post them here or make separate github issues for each issue/oddity/inconsistency you may come across in the codebase, both options are fine :)

[simonvdm]

I'm a bit new to the forum etiquette here (haven't coded in decades) so forgive if it's the wrong place for this comment.

I just wanted to +1 the thanks and appreciation for the Neorg team.

Like many here I was very impressed when I first discovered the current implementation and vision for Neorg. I find the concealer to be an extremely elegant solution to the age-old plain text vs WYSIWYG conundrum. It convinced me to take the risk of moving my Life OS to Neorg even though the ecosystem is still very young.

The only thing missing for me is GTD.

My jaw dropped a bit watching the neorg-telescope-gtd and I was obviously disappointed that the code is not operational anymore.

Happy to know this is underway in a solid, future proof manner.

Unfortunately, I'm not able to contribute code or time so I guess I'll buy you guys a virtual beer ;-)

[max397574]

the gtd implementation is coming along really nicely
Within the next months (I hope 😉 ) we'll be able to get gtd things again.

I'll also update the gtd pickers (I originally created them) as soon as possible so they are operational again and perhaps more powerfull than ever before.

[flexagoon]

Hi, it's been 4 months since the last update on GTD I managed to find, are things still being developed?