-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
real time indexing #20
Comments
Might need to be careful about closing libmagic #124 (comment) |
Relevant: i've implemented 'almost realtime' indexing recently: promnesia/src/promnesia/dump.py Line 18 in c442081
E.g. you can have a separate config file only with your text notes (which should be indexed very fast). Then if you run
That means you can run it very often (e.g. every five minutes), or potentially combine with |
The last comment here needs to make it into main docs. Even better, if a new option is added like
But what about de-duplication? Are there any issues with updates? |
Yep, good idea to pass it in cmdline args! It was somewhat experimental at first, so I made it an env variable, but it seems to work pretty well (apart from one minor race condition I might need to fix first). Regarding deduplication -- not sure what do you mean? promnesia/src/promnesia/dump.py Lines 61 to 79 in e3b21cb
So it clears all the entries corresponding to the data source first and then inserts them. Hopefully shouldn't result in duplication! |
- CLI options described in the 2nd case explained in karlicoss#211, due to simplicity. - The precedence for deciding update/overwrite: env-var, --update/overwrite, --source exist? - Function defaults are false, as suggested in [karlicoss#20](karlicoss#20 (comment)). - Both index & demo updated. - Env-var now checks its value one of (update|overwrite). - All update/overwrite decision logic moved to __main_.
- CLI options described in the 2nd case explained in karlicoss#211, due to simplicity. - The precedence for deciding update/overwrite: env-var, --update/overwrite, --source exist? - Function defaults are false, as suggested in [karlicoss#20](karlicoss#20 (comment)). - Both index & demo updated. - Env-var now checks its value one of (update|overwrite). - All update/overwrite decision logic moved to __main_.
- CLI options described in the 2nd case explained in karlicoss#211, due to simplicity. - The precedence for deciding update/overwrite: env-var, --update/overwrite, --source exist? - Function defaults are false, as suggested in [karlicoss#20](karlicoss#20 (comment)). - Both index & demo updated. - Env-var now checks its value one of (update|overwrite). - All update/overwrite decision logic moved to __main_.
- CLI options described in the 2nd case explained in karlicoss#211, due to simplicity. - The precedence for deciding update/overwrite: env-var, --update/overwrite, --source exist? - Function defaults are false, as suggested in [karlicoss#20](karlicoss#20 (comment)). - Both index & demo updated. - Env-var now checks its value one of (update|overwrite). - All update/overwrite decision logic moved to __main_.
as suggested in karlicoss#20: - drop PROMNESIA_INDEX_POLICY env-var. - CLI options described in the 2nd case explained in karlicoss#211, due to simplicity. - Function defaults are false, as suggested in [karlicoss#20](karlicoss#20 (comment)). - Both index & demo updated. - Env-var now checks its value one of (update|overwrite). - All update/overwrite decision logic moved to __main_.
as suggested in karlicoss#20: - drop PROMNESIA_INDEX_POLICY env-var. - CLI options described in the 2nd case explained in karlicoss#211, due to simplicity. - Function defaults are false, as suggested in [karlicoss#20](karlicoss#20 (comment)). - Both index & demo updated. - Env-var now checks its value one of (update|overwrite). - All update/overwrite decision logic moved to __main_.
as suggested in karlicoss#20: - drop PROMNESIA_INDEX_POLICY env-var. - CLI options described in the 2nd case explained in karlicoss#211, due to simplicity. - Function defaults are false, as suggested in [karlicoss#20](karlicoss#20 (comment)). - Both index & demo updated. - Env-var now checks its value one of (update|overwrite). - All update/overwrite decision logic moved to __main_.
as suggested in karlicoss#20: - drop PROMNESIA_INDEX_POLICY env-var. - CLI options described in the 2nd case explained in karlicoss#211, due to simplicity. - Function defaults are false, as suggested in [karlicoss#20](karlicoss#20 (comment)). - Both index & demo updated. - Env-var now checks its value one of (update|overwrite). - All update/overwrite decision logic moved to __main_.
as suggested in #20: - drop PROMNESIA_INDEX_POLICY env-var. - CLI options described in the 2nd case explained in #211, due to simplicity. - Function defaults are false, as suggested in [#20](#20 (comment)). - Both index & demo updated. - Env-var now checks its value one of (update|overwrite). - All update/overwrite decision logic moved to __main_.
hmm seems that it was closed automatically by github -- we don't really have realtime indexing yet, so I'll reopen |
Perhaps for actual 'realtime' this would need proper HPI support. |
E.g. something inotify based. That would make the implementation quite a bit more complext that it's at the moment.
Also due to the nature of many exports (periodic), it won't be realtime unless the underlying exports are realtime.
Still it could at least detect source files changes, etc.
Also would work well in conjunction with Grasp.
The text was updated successfully, but these errors were encountered: