Unable to update containers in Synology DSM running Container Manager

[firmlyundecided]

Running dockcheck on a Synology DS1522+ to evaluate docker containers running in container manager. Dockcheck works as intended, successfully identifying updates and sending email notifications using the DSM template. However, it is unable to execute updates for containers running in Container Manager.

When attempting to execute an update, I'm met with the following error:
Error response from daemon: Conflict. The container name "/container-name" is already in use by container "long-string". You have to remove (or rename) that container to be able to reuse that name.

This error persists regardless of if the update is triggered via -a or -y tag, or specific container selected via prompt. All Container Manager containers in question are built using docker compose files within in the Container Manager GUI.

Additionally, dockcheck identifies image updates that are not visible for updating in the Container Manager GUI. Unlikely this is due to image changes without new tags, as it's confirmed to affect images that have seen new releases tagged in github.

Referencing a comment made in an earlier issue, suggesting dockcheck will not work when using the Container Manager GUI.

On synology I use the GUI to run & update individual images which gives you a wizard to edit/store all the parameters for each container.
You can also create projects (docker compose behind the scenes) in the GUI.
You can also use docker compose entirely manually (no GUI).

From this reddit discussion with that same error (conflict between GUI and manual mode), it sounds like to use the command line (like dockcheck.sh) to manage updates he should probably exclusively go the docker compose manual (no GUI) route.

Note my other non-DSM docker systems use docker compose and dockcheck.sh normally.

Originally posted by @yoyoma2 in #161

[mag37]

Thank you for creating a new issue for further discussion.

Dockcheck works as intended, successfully identifying updates and sending email notifications using the DSM template. However, it is unable to execute updates for containers running in Container Manager.

I don't have any synology products myself so can't test - but I would assume the Container Manager is somewhat similar to Portainer in that it creates its own directory structure and handle things a bit different with mounts and compose.

You could check some of this by running the extras/errorCheck.sh script, it's lacking some but will show some paths.

Additionally, dockcheck identifies image updates that are not visible for updating in the Container Manager GUI.

That's probably because (as you mentioned) due to Dockcheck using registry hashes to determine updates while CM maybe uses tags/releases only. Dockcheck will see any change as an update, everything from a typo to a full rebuild.

As @yoyoma2 suggested - if you'd like to use Dockcheck for doing the updates, you should probably set things up using plain docker compose in the CLI.

I don't know if there's anything I can do or change to support the Synology setup better.

[yoyoma2]

Additionally, dockcheck identifies image updates that are not visible for updating in the Container Manager GUI.

On synology DSM I use dockcheck only for the notifications and manage the docker updates with the Container Manager GUI. I had noticed that the newly available images were not always seen by Container Manager so the following lines were added to the DSM notify template. Make sure your DSM notify template is up to date.

# This ensures DSM's container manager will also see the update
/var/packages/ContainerManager/target/tool/image_upgradable_checker

Since these lines were added, whenever I get a notification from dockcheck, I also see them from the CM GUI. You could create a scheduled task that runs that command as root on the schedule you prefer. I don't know how often DSM runs this check itself but it doesn't seem to be very frequent.

[mag37]

@firmlyundecided is there anything else I can do for you in this regard or have you resolved your issues?

[yoyoma2]

Perhaps dockcheck could detect DSM and refuse to update containers or at least warn to avoid using the DSM GUI.

$ cat /etc/VERSION | grep os_name
os_name="DSM"
$ 

I have zero user_managed_compose_project since I don't use projects (DSM's term for docker compose).

$ sudo /volume1/@appstore/ContainerManager/tool/synouserdata_docker | jq .project
{
  "total_compose_project": 0,
  "user_managed_compose_project": 0,
  "web_managed_compose_project": 0,
  "web_service_registered": 0
}
$

Perhaps the dockcheck update would work on containers run without a project like mine but I never dared to try.

[firmlyundecided]

I'm not sure what the best solution is. @yoyoma2's suggestion is interesting. perhaps it's even a call-out in the docs, to be a little more light-handed.

The actual ideal solution would be to figure out how to force Container Manager to play nice, but 1/ that's above my pay grade, and 2/ my experiences tell me that Container Manager and DSM simply isn't the nicest Docker environment to play in.

[mag37]

@yoyoma2

Perhaps dockcheck could detect DSM and refuse to update containers or at least warn to avoid using the DSM GUI.

$ cat /etc/VERSION | grep os_name
os_name="DSM"
$ 

While this is sensible - I'm a bit hesitant to start adding specific checks and warnings for different environments - I mean there's a bunch of different Hypervisors/VMs/LXCs/OSes/Environments that all can be modified by the user.
The user have to be responsible to know some of how their containers are managed.

@firmlyundecided

I'm not sure what the best solution is. @yoyoma2's suggestion is interesting. perhaps it's even a call-out in the docs, to be a little more light-handed.

See above - but yes, mentioning it in the docs sounds reasonable. A heads-up section.

The actual ideal solution would be to figure out how to force Container Manager to play nice, but 1/ that's above my pay grade, and 2/ my experiences tell me that Container Manager and DSM simply isn't the nicest Docker environment to play in.

As I don't have a DSM myself (someone gift me one please! 😁 ) I cant really dive in to that even if I'd have time. But yes I'm afraid it probably uses some custom layering / embedded subvolumes or similar.

[firmlyundecided]

Agreed re: documentation call-out, that users leveraging DSM's container manager may have issues automatically updating containers via the tool.

To help your documentation, here's the quick setup for automating Dockcheck via DSM's Task Scheduler:

1. Open Control Panel and navigate to Task Scheduler
2. Create a Scheduled Task > User-defined script
3. Task Name: Dockcheck
4. User: root
5. Schedule: User Preference
6. Task Settings:
7. ✔ Send run details by email (include preferred email)
8. User-defined script: export HOME=/root && cd /path/to/dockcheck && ./dockcheck.sh -n -i -I or other custom args
9. Click OK, accept warning message

Note: This setup will result in two emails sent, one by dockcheck (due to the -I flag) and one by DSM. The dockcheck email only includes the notification of available updates, while the DSM email shows the entire script as run. This is a user preference, and both are not necessary. Since Dockcheck cannot directly update containers in Container Manager, at least one email notification option should be enabled to use Dockcheck so you can be aware of manual updates available.

[yoyoma2]

A heads-up section.

I like that! Also nice documentation @firmlyundecided !

[mag37]

Thank you @firmlyundecided for the provided screenshots and writeup! I'll surely implement it somehow.
Considering how I should structure it, if I should add it into addons like addon/DSM-info/... so it'll be a proper readme in that directory.

I'll convert this issue to a discussion!

[mag37]

I've added some info in a new branch, please check it out and reply with any changes (or make PR to the branch)!
Thank you both for helping out.

next063 - extra plugins and tools

[mag37]

@yoyoma2 @firmlyundecided - bumping you if you'd like to have a look :) planning to merge to main in the coming days.

[firmlyundecided]

Updating my original Note with clearer caveats and changes below.

Using Dockcheck in DSM

Dockcheck cannot directly update containers managed in the Container Manager GUI, but it can still be used to notify you of containers with updates available. There are two ways to be notified, each with their own caveats:

1. Enabling email notifications within the Task Scheduler (step 6i below) will send an email that includes the entire script as run. This will not include the urls.list links to release notes, but it will show a full list of containers checked, up to date, and needing updates (following the args included in the scheduled task).
2. The DSM notification template will enable Dockcheck to directly send an email when using the -I flag. This is most useful when paired with an accurate urls.list file, and results in a neat succinct email notification of only containers to be updated.

This is a user preference, and both notifications are not necessary. However, regardless of the notification method, it is necessary to set up a scheduled task to run Dockcheck at a set interval (otherwise it will only run when manually triggered).

Automate Dockcheck with DSM Task Scheduler:

1. Open Control Panel and navigate to Task Scheduler
2. Create a Scheduled Task > User-defined script
3. Task Name: Dockcheck
4. User: root
5. Schedule: User Preference
6. Task Settings:
   1. ✔ Send run details by email (include preferred email) This is the optional step as described above)
   2. User-defined script: export HOME=/root && cd /path/to/dockcheck && ./dockcheck.sh -n -i -I or other custom args
7. Click OK, accept warning message

Set up the DSM Notification template

Copy the dockcheck/notify_templates/notify_DSM.sh to the same directory as where you keep dockcheck.sh.
Use as it (uses your default notification email setting) or edit and override manually.

[yoyoma2]

Great documentation! A typo?

Use as it (uses your default notification email setting) or edit and override manually.

Use as it is (uses your default notification email setting) or edit and override manually.

[firmlyundecided]

Good catch, correct.

[mag37]

Hey, I'll merge this to main now - but just to be sure this is correct.

1. Enabling email notifications within the Task Scheduler (step 6i below) will send an email that includes the entire script as run. This will not include the urls.list links to release notes, but it will show a full list of containers checked, up to date, and needing updates (following the args included in the scheduled task).

2. The DSM notification template will enable Dockcheck to directly send an email when using the -I flag. This is most useful when paired with an accurate urls.list file, and results in a neat succinct email notification of only containers to be updated.

1. Wont the urls.list be used if you use the -I (capital i) to also print the urls in the CLI-output?
2. I've changed the "when using the -I flag." to "when using the -i or -I flag." as it should work either way (just without the urls) - or is this wrong?

[firmlyundecided]

1. It doesn't print the URLs in that email, for whatever reason (just from my experience)
2. I defer to you and other contributors. Was written based on my understanding of the flags

[mag37]

Ah I might have misunderstood. (will change back to only lower i not either or)

But yes the -i flag will use the notify.sh template and send an email with that, if there's an urls.list available it will use it to enrich with urls.
The -I option paired with an available urls.list will print the URLs in the CLI output - though I just realized it doesn't when you've got it automated (with -n for example). I'll look into that!

[mag37]

Pushed this to main now! Thank you both.

I'll gladly receive any suggestions or improvements in the future, just hit me up with a PR, issue, discussion or message me directly (@mag37:matrix.org or @[email protected])