Sync only on ADD and maybe on MOVE

[alex6dj]

I opened a thread some time ago in the OMV Forum and didnt get any answer, so I will try here and copy the full question.

From OMV Forum:

Good mornig. After a year using SnapRaid on Windows Im doing the swith to OMV. Everything is set and working now, so no complaint here.

Im not using the internall diff tool, maybe becuase i missundertand the options. I dont want to autosync if something is updated or deleted, The internal tool say for the Update Threshold: Define threshold of updated files above which not to start the sync process. default = 0 to disable this check and always sync. So my understanding is that 1 will sync if one file was updated and zero will sync everytime, this is not what I expect. This is not the main question but an answer is welcome for the real behaviour.

Now the real questions.

Im using snapraid-aio-script.sh from https://github.com/auanasgheps/snapraid-aio-script for late night maintenance. Now I see several problems with this script I want to solve. Maybe the problem is a bad config, so Im here to check this, and maybe later if needed open an issue in the development branch.

• The Telegram notification I think only partially works. The only notification I get is SnapRAID Script Job started, no error, no warning, no script completed. Maybe this is really the only thing the Telegram bot notification support. I dont know, I was expection a little more info there.

• When the pool is in sync the scrub run fine, otherwise it dont run. Im ok with this but in my case I expect if the only thing I do was to add new files a pre sync should kick an later an scrub. Maybe the problem is the warning. Or maybe I should set ADD_DEL_THRESHOLD different fron zero.

238233 equal
      5 added
      0 removed
      0 updated
      0 moved
      0 copied
      0 restored
There are differences!

DIFF finished [dom 17 sep 2023 02:37:52 CDT]
**SUMMARY: Equal [238233] - Added [5] - Deleted [0] - Moved [0] - Copied [0] - Updated [0]**
There are no deleted files, that's fine.
**WARNING!** Updated files (0) reached/exceeded threshold (0).
For more information, please check the DIFF ouput saved in /root.
Forced sync is not enabled. **NOT** proceeding with SYNC job. [dom 17 sep 2023 02:37:52 CDT]
### SnapRAID SCRUB [dom 17 sep 2023 02:37:52 CDT]
Parity info is out of sync (deleted or changed files threshold has been breached).
Not running SCRUB job. [dom 17 sep 2023 02:37:52 CDT]

Here is my config file, at least the changed parts:

HEALTHCHECKS=1
HEALTHCHECKS_ID="*****************************"
HEALTHCHECKS_URL="https://hc-ping.com/"

TELEGRAM=1
TELEGRAM_TOKEN="**************************************"
TELEGRAM_CHAT_ID="**************"

SCRUB_PERCENT=3
SCRUB_AGE=0

[auanasgheps]

Hi there, I usually keep an eye on the OMV forum, but this is definitely the place to ask for help about this script.

I dont want to autosync if something is updated or deleted

With this script, you can choose what is the acceptable limit.

Keep to default this option: SYNC_WARN_THRESHOLD=-1

Change these to the maximum changed or deleted files you want to allow per sync. If one of these is exceeded, sync will not proceed.

DEL_THRESHOLD=500
UP_THRESHOLD=500

Regarding the Telegram notification: it sends a message when the process is started and when is completed. An upcoming version will display more stuff. Something is not working on your side regarding the final message. Do you see any error in the script logs towards the end?

When the pool is in sync the scrub run fine, otherwise it dont run. Im ok with this but in my case I expect if the only thing I do was to add new files a pre sync should kick an later an scrub. Maybe the problem is the warning. Or maybe I should set ADD_DEL_THRESHOLD different fron zero.

ADD_DEL_ TRESHOLD is an optional feature: if you don't want to use it, keep it to 0.
You have set a limit of 0 updated files. This does not make any sense, it means it will never be able to run a sync. You need to set an acceptable amount of updated/deleted files in order to run a sync!

[alex6dj]

Hi, first thanks for the answer and sorry for my bad english.

Just play a little with the thresholds, I was hopping a sync only run when an addition and maybe a move is detected. An addition is not a problem, a deletion either because Im using .recycle over SAMBA, but an updated (changed) file can be very hard or imposible to recover. If my conclusions are correct UP_THRESHOLD refers to addition and/or updates, and there is my problem. For now I will do manual syncs, this only happen every monday when I do some changes, all other days nothing change.

Im not using an email (yet) to receive notifications, but Telegram. The log dont show any exception or similar and the only line about Telegram is Telegram notification is enabled..

Curated Log:
SnapRAID-2023_10_20-0209.zip

The starting job notification is great, at last I know the job is running every night. But a little resume should be a good addition. I think It should have:

• Detected changes (count).
• If sync run without problems or not (without details, the log is for this)
• If status run found any error (again, no details needed)

It think this will work a lot better.

Any way, thanks for the hard work in this script. It is a must for everyone using Snapraid.

[auanasgheps]

Please, do not mis-usunderstand this feature.
SnapRAID is not intended to be a "Backup" solution, but more a "RAID" replacement.
This feature - sync under conditions (deleted, updated, etc) - is intended to be "don't sync if something very stupid has happened" such as a ransomware or human who deleted too many files.

It's meant to act just as safety net, not a backup solution. Don't assign SnapRAID a role that cannot fullfill.

If you want a history of your changes, you can keep files longer in your .recycle folder, use BTRFS snapshotting, Borg backup, Kopia Backup and/or many other!

Regarding the final notification that does not appear, I think you're facing #56, which is fixed in the next version. To workaround it, set a destination address, and within your email settings use a filter rule to delete the email or move it to a folder (I recommend the latter).

As of now, that's what you get

The new version will have some more notifications for info and in case of failures, such as #49 (comment) or #50 (comment)

Any way, thanks for the hard work in this script. It is a must for everyone using Snapraid.

Thank you, I always appreciate kind words like this :)

[alex6dj]

SnapRAID is not intended to be a "Backup" solution, but more a "RAID" replacement.
Yes I know the Snapraid limitations. Was running in windows for a year and save me several times from drive failures.

Well, I think this can be closed now. I will stick with manual sync and let the script run scrubs every night. I see the notifications changes, and like the results. I will test developer branch for fixes and whait a new release.

Thanks for all