Skip to content
This repository has been archived by the owner on Nov 28, 2020. It is now read-only.

SSH passphrase being asked multiple times #2

Closed
fernandodrf opened this issue Jan 27, 2019 · 6 comments
Closed

SSH passphrase being asked multiple times #2

fernandodrf opened this issue Jan 27, 2019 · 6 comments
Assignees
Labels
good first issue Good for newcomers

Comments

@fernandodrf
Copy link

Hi there, this is a nice script, thanks for sharing! I am using it to backup a self-hosted Nextcloud server. I keep getting asked for the passphrase for the ssh key that I use to connect to the remote server, even though I wrote it in the nc_borg.details file. Am I missing something?

Thanks a lot for the help!

@asifbacchus
Copy link
Owner

Hey, sorry about not getting back sooner. Are you connecting to an rsync.net server or some other provider? Also, maybe more importantly, have you checked the log file? There should a line in there right before the backups start that says "Borg SSH/REPO password... OK" assuming it's being read properly from the nc_borg.details file. Otherwise, is it showing an error? Finally, are you sure you are entering the password exactly on line 4? If you use a text editor, sometimes lines get wrapped and that causes errors, so just asking.

Let me know those answers and I'm happy to check into things further for you. Sorry you are having a problem getting things to work!

@asifbacchus asifbacchus self-assigned this Jan 28, 2019
@asifbacchus
Copy link
Owner

Just another thought: I actually never really tested the password using certain special characters like ( ) or any quotes like ` ' ". I don't recommend using these in passwords that will be scripted since they tend to confuse shells and ssh in general so I avoid them... but I never noted that in the readme or anything. Is it possible you are using characters like those in your password? Is the password still something you can change (i.e. new SSH keys). If you can change the key without having to lose data or something, then perhaps try a password that that is maybe longer but only alphanumeric or uses only 'safe' special characters like - _ ! % ^ * , .

Just throwing it out there... if this ends up being the problem, then I'll have to look into taking that into account in the script.

@fernandodrf
Copy link
Author

Hello again, I have the borg remote repository in an external server, the password is in the fourth line and I am actually not using any special characters for the SSH key/password. There are no signals of the password not being correctly read, I ran it a couple of times and I am getting an error because the version of borg in the remote server is too old, and some other directories not being found.

From what I understand, the SSH paraphrase and the borg password have to be the same, or is there any possibility to configure them different?

Thanks a lot!

Here is the log:


�[95m[2019-01-27 17:40:48]--- Start backup.sh execution ---�[0m
�[96m[2019-01-27 17:40:48]-- [INFO] Log file located at �[93m/var/log/borg.log�[96m --�[0m
�[96m[2019-01-27 17:40:48] -- [INFO] mySQL dump file will be stored at: �[93m/tmp/tmp.7WX9lGao2n/backup-20190127_174048.sql�[0m
�[96m[2019-01-27 17:40:48] -- [INFO] Web users will NOT be informed the server is down! --�[0m
Maintenance mode enabled
�[96m[2019-01-27 17:40:49] -- [INFO] NextCloud now in maintenance mode -- �[0m
�[39m[2019-01-27 17:40:49] Dumping NextCloud SQL database...�[0m
�[32m[2019-01-27 17:40:49] -- [SUCCESS] SQL dumped successfully --�[0m
�[39m[2019-01-27 17:40:49] Pre-backup tasks completed, calling borgbackup... �[0m
�[39m[2019-01-27 17:40:49] Verifying supplied borg configuration variables... �[0m
�[39m[2019-01-27 17:40:49] Borg base dir... OK�[0m
�[39m[2019-01-27 17:40:49] Borg SSH key... OK�[0m
�[39m[2019-01-27 17:40:49] Borg REPO name... OK�[0m
�[39m[2019-01-27 17:40:49] Borg SSH/REPO password... OK�[0m
�[39m[2019-01-27 17:40:49] Borg REMOTE path... OK�[0m
�[39m[2019-01-27 17:40:49] Processing referenced extra files list for borgbackup to include in backup�[0m
�[39m[2019-01-27 17:40:49] Found �[93m/root/NCscripts/xtraLocations.borg�[0m
�[39m[2019-01-27 17:40:49] Processed extra files list for inclusion in borgbackup�[0m
�[39m[2019-01-27 17:40:49] Found �[93m/root/NCscripts/excludeLocations.borg�[0m
�[39m[2019-01-27 17:40:49] Checking for tmp directory at �[93m/root/.borgbackup �[0m
�[39m[2019-01-27 17:40:49] tmp folder found within borg base directory �[0m
�[96m[2019-01-27 17:40:49] --[INFO] Executing borg with exclusions -- �[0m
Remote: Borg 1.0.12: exception in RPC call:
Remote: Traceback (most recent call last):
Remote:   File "/usr/lib/python3/dist-packages/borg/remote.py", line 142, in serve
Remote:     raise InvalidRPCMethod(method)
Remote: borg.remote.InvalidRPCMethod: RPC method get_free_nonce is not valid
Remote: Platform: Linux cluster1 2.6.32-042stab133.2 #1 SMP Mon Aug 27 21:07:08 MSK 2018 x86_64
Remote: Linux: Ubuntu 16.04 xenial
Remote: Borg: 1.0.12  Python: CPython 3.5.2
Remote: PID: 24776  CWD: /home/bunker
Remote: sys.argv: ['/usr/bin/borgbackup', 'serve', '--umask=077']
Remote: SSH_ORIGINAL_COMMAND: None
Remote: 
Please upgrade to borg version 1.1+ on the server for safer AES-CTR nonce handling.
/etc/nanorc: [Errno 2] No such file or directory: '/etc/nanorc'
/etc/msmtprc: [Errno 2] No such file or directory: '/etc/msmtprc'
/etc/msmtp_aliases: [Errno 2] No such file or directory: '/etc/msmtp_aliases'
/etc/apt/listchanges.conf: [Errno 2] No such file or directory: '/etc/apt/listchanges.conf'
/etc/logwatch: [Errno 2] No such file or directory: '/etc/logwatch'
/etc/fail2ban/fail2ban.local: [Errno 2] No such file or directory: '/etc/fail2ban/fail2ban.local'
/etc/fail2ban/jail.local: [Errno 2] No such file or directory: '/etc/fail2ban/jail.local'
/etc/php/7.0/cli/php.ini: [Errno 2] No such file or directory: '/etc/php/7.0/cli/php.ini'
/etc/php/7.0/fpm/php-fpm.conf: [Errno 2] No such file or directory: '/etc/php/7.0/fpm/php-fpm.conf'
/etc/php/7.0/fpm/php.ini: [Errno 2] No such file or directory: '/etc/php/7.0/fpm/php.ini'
/etc/php/7.0/fpm/pool.d/www.conf: [Errno 2] No such file or directory: '/etc/php/7.0/fpm/pool.d/www.conf'
------------------------------------------------------------------------------
Archive name: 2019-01-27_174049
Archive fingerprint: fbd17c98b6aa450868e742d60876f2756dad6d6b501b21946ab3ff90fa6084a3
Time (start): Sun, 2019-01-27 17:40:57
Time (end):   Sun, 2019-01-27 17:41:40
Duration: 42.90 seconds
Number of files: 2031
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:              779.50 MB            773.27 MB            773.26 MB
All archives:              779.50 MB            773.27 MB            773.26 MB

                       Unique chunks         Total chunks
Chunk index:                    2281                 2284
------------------------------------------------------------------------------
terminating with warning status, rc 1
�[96m[2019-01-27 17:41:41] --[INFO] Executing borg prune operation -- �[0m
Keeping archive: 2019-01-27_174049                    Sun, 2019-01-27 17:40:57 [fbd17c98b6aa450868e742d60876f2756dad6d6b501b21946ab3ff90fa6084a3]
terminating with success status, rc 0
�[32m[2019-01-27 17:42:58] -- [SUCCESS] Borg prune completed successfully --�[0m
�[39m[2019-01-27 17:42:58] Borgbackup completed... begin cleanup �[0m
�[1m�[39m[2019-01-27 17:42:58] ***Normal exit process***�[0m
�[39m[2019-01-27 17:42:58] Removed SQL temp directory�[0m
�[39m[2019-01-27 17:42:58] 503 error page never copied to webroot, nothing to cleanup
Maintenance mode disabled
�[96m[2019-01-27 17:42:58] -- [INFO] NextCloud now in regular operating mode --�[0m
�[1m�[32m[2019-01-27 17:42:58] -- [SUCCESS] All processes completed successfully --�[0m
�[1;93mbackup.sh generated the following warnings: �[0m
�[1;93m[2019-01-27 17:40:48] -- [WARNING] No webroot path was specified (-w parameter missing) (code: 5031) -- �[0m
�[1;93m[2019-01-27 17:41:41] -- [WARNING] Borg completed with warnings. Please check this script's logfile for details (code: 2200) -- �[0m
�[95m[2019-01-27 17:42:58] --- backup.sh completed ---�[0m
�[95m[2019-01-27 18:11:19]--- Start backup.sh execution ---�[0m
�[96m[2019-01-27 18:11:19]-- [INFO] Log file located at �[93m/var/log/borg.log�[96m --�[0m
�[96m[2019-01-27 18:11:19] -- [INFO] mySQL dump file will be stored at: �[93m/tmp/tmp.5WGKENf1Xt/backup-20190127_181119.sql�[0m
�[96m[2019-01-27 18:11:19] -- [INFO] Web users will NOT be informed the server is down! --�[0m
Maintenance mode enabled
�[96m[2019-01-27 18:11:20] -- [INFO] NextCloud now in maintenance mode -- �[0m
�[39m[2019-01-27 18:11:20] Dumping NextCloud SQL database...�[0m
�[32m[2019-01-27 18:11:20] -- [SUCCESS] SQL dumped successfully --�[0m
�[39m[2019-01-27 18:11:20] Pre-backup tasks completed, calling borgbackup... �[0m
�[39m[2019-01-27 18:11:20] Verifying supplied borg configuration variables... �[0m
�[39m[2019-01-27 18:11:20] Borg base dir... OK�[0m
�[39m[2019-01-27 18:11:20] Borg SSH key... OK�[0m
�[39m[2019-01-27 18:11:20] Borg REPO name... OK�[0m
�[39m[2019-01-27 18:11:20] Borg SSH/REPO password... OK�[0m
�[39m[2019-01-27 18:11:20] Borg REMOTE path... OK�[0m
�[39m[2019-01-27 18:11:20] Processing referenced extra files list for borgbackup to include in backup�[0m
�[39m[2019-01-27 18:11:20] Found �[93m/root/NCscripts/xtraLocations.borg�[0m
�[39m[2019-01-27 18:11:20] Processed extra files list for inclusion in borgbackup�[0m
�[39m[2019-01-27 18:11:20] Found �[93m/root/NCscripts/excludeLocations.borg�[0m
�[39m[2019-01-27 18:11:20] Checking for tmp directory at �[93m/root/.borgbackup �[0m
�[39m[2019-01-27 18:11:20] tmp folder found within borg base directory �[0m
�[96m[2019-01-27 18:11:20] --[INFO] Executing borg with exclusions -- �[0m
/etc/nanorc: [Errno 2] No such file or directory: '/etc/nanorc'
/etc/msmtprc: [Errno 2] No such file or directory: '/etc/msmtprc'
/etc/msmtp_aliases: [Errno 2] No such file or directory: '/etc/msmtp_aliases'
/etc/apt/listchanges.conf: [Errno 2] No such file or directory: '/etc/apt/listchanges.conf'
/etc/logwatch: [Errno 2] No such file or directory: '/etc/logwatch'
/etc/fail2ban/fail2ban.local: [Errno 2] No such file or directory: '/etc/fail2ban/fail2ban.local'
/etc/fail2ban/jail.local: [Errno 2] No such file or directory: '/etc/fail2ban/jail.local'
/etc/php/7.0/cli/php.ini: [Errno 2] No such file or directory: '/etc/php/7.0/cli/php.ini'
/etc/php/7.0/fpm/php-fpm.conf: [Errno 2] No such file or directory: '/etc/php/7.0/fpm/php-fpm.conf'
/etc/php/7.0/fpm/php.ini: [Errno 2] No such file or directory: '/etc/php/7.0/fpm/php.ini'
/etc/php/7.0/fpm/pool.d/www.conf: [Errno 2] No such file or directory: '/etc/php/7.0/fpm/pool.d/www.conf'
------------------------------------------------------------------------------
Archive name: 2019-01-27_181120
Archive fingerprint: dd73f0e8d945adb103c51bce5453d278b5274e9a18c06fb1cb79e238a2b394f5
Time (start): Sun, 2019-01-27 18:11:33
Time (end):   Sun, 2019-01-27 18:11:34
Duration: 1.29 seconds
Number of files: 2031
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:              779.50 MB            773.27 MB              1.32 MB
All archives:                1.56 GB              1.55 GB            774.58 MB

                       Unique chunks         Total chunks
Chunk index:                    2292                 4569
------------------------------------------------------------------------------
terminating with warning status, rc 1
�[96m[2019-01-27 18:11:35] --[INFO] Executing borg prune operation -- �[0m
Keeping archive: 2019-01-27_181120                    Sun, 2019-01-27 18:11:33 [dd73f0e8d945adb103c51bce5453d278b5274e9a18c06fb1cb79e238a2b394f5]
Pruning archive: 2019-01-27_174049                    Sun, 2019-01-27 17:40:57 [fbd17c98b6aa450868e742d60876f2756dad6d6b501b21946ab3ff90fa6084a3] (1/1)
terminating with success status, rc 0
�[32m[2019-01-27 18:11:39] -- [SUCCESS] Borg prune completed successfully --�[0m
�[39m[2019-01-27 18:11:39] Borgbackup completed... begin cleanup �[0m
�[1m�[39m[2019-01-27 18:11:39] ***Normal exit process***�[0m
�[39m[2019-01-27 18:11:39] Removed SQL temp directory�[0m
�[39m[2019-01-27 18:11:39] 503 error page never copied to webroot, nothing to cleanup
Maintenance mode disabled
�[96m[2019-01-27 18:11:40] -- [INFO] NextCloud now in regular operating mode --�[0m
�[1m�[32m[2019-01-27 18:11:40] -- [SUCCESS] All processes completed successfully --�[0m
�[1;93mbackup.sh generated the following warnings: �[0m
�[1;93m[2019-01-27 18:11:19] -- [WARNING] No webroot path was specified (-w parameter missing) (code: 5031) -- �[0m
�[1;93m[2019-01-27 18:11:35] -- [WARNING] Borg completed with warnings. Please check this script's logfile for details (code: 2200) -- �[0m
�[95m[2019-01-27 18:11:40] --- backup.sh completed ---�[0m

@asifbacchus
Copy link
Owner

Thanks for the log. Based on it, I see that the backups are completed successfully and the script is properly reading your REPO password. What's happening is my fault for my the wording of my script and documentation being misleading... I will fix that in a future release...

I'm 99% sure this is the situation: You are being prompted by SSH (NOT borg or my script) for your SSH key's password. This is outside the control of borg and, subsequently, my script. You would need to store your SSH password in something like a GNOME keyring or another option (check the SSH man pages) so that SSH can read it securely. Your SSH password and your borg repo password are two separate things, even if they share the same password.

There are two steps to connecting to your borg repo. The first is SSHing to the remote system. Then you supply a key and password to borg to decrypt and access your repo for updating/pruning/etc.

I recommend having an SSH key with NO password and then relying on your borg repo password and key for enhanced security. That way you're still connecting via an SSH key (securely) but the script will handle accessing your repo via a key and password with no prompts or required user interaction. Does that make sense?

If you want to continue using an SSH key with a password, you have to set that up via SSH itself (since borg just uses SSH as-is) and that is outside the scope of my script because the implementation varies so widely depending on your system setup. Simply put, unless you provide some kind of keyring system for SSH to use, it will sit and wait for you to enter a password for the key... unless there is no password :-)

If this isn't making sense, let me know and I'll try to explain better...

Regarding the other errors:
You can get rid of the errors regarding files not being found by editing the xtraLocations.borg file to remove the non-existent files. Also, you should probably update that file so that your PHP configuration files are included in the backup. The file has paths for PHP7.0 but you are probably (hopefully!) using PHP7.2 or PHP7.3. You'd only have to change the paths from .../7.0/... to .../7.2/... or .../7.3/....

Regarding the InvalidRPCMethod(method) error: You are correct, that is because you are using borg 1.1+ on your client and borg 1.0 on your server. You can ignore those errors or, better yet, update your server.

I hope this helps? By the way, thanks for the kind words about the script :-) I hope it helps you once you get this stuff figured out! Please let me know if you need any more help or if I totally lost you in the long reply.

@fernandodrf
Copy link
Author

Hi, thanks for the detailed explanation. I understand the difference between SSH passphrase and BORG password, but as you said, your readme is a bit unclear about it :P so I just assumed they had to be the same. But I managed to solve the error with your help and now I can set a cronjob.

Thanks!!

@asifbacchus
Copy link
Owner

Glad you got it all working. I'll fix the relevant sections of the in-script help and the readme soon to make this whole issue clearer and re-release -- thanks for pointing it out @fernandodrf :-)

@asifbacchus asifbacchus added the good first issue Good for newcomers label Jan 30, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants