Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

28.0.3.2 → 29.0.0.19: issues about .htaccess and .well-known #2214

Closed
huangwb8 opened this issue May 2, 2024 · 16 comments
Closed

28.0.3.2 → 29.0.0.19: issues about .htaccess and .well-known #2214

huangwb8 opened this issue May 2, 2024 · 16 comments
Labels
needs info Additional info needed to triage question upstream

Comments

@huangwb8
Copy link

huangwb8 commented May 2, 2024

After update 28.0.3.2 to 29.0.0.19, there're 2 security & setup warnings appearing:

Your data directory and files are probably accessible from the internet. 
The .htaccess file is not working. 
It is strongly recommended that you configure your web server so that the data directory is no longer accessible, or move the data directory outside the web server document root.

and

Your web server is not properly set up to resolve `.well-known` URLs, 
failed on: `/.well-known/caldav` 
For more details see the [documentation ↗](https://docs.nextcloud.com/server/29/go.php?to=admin-setup-well-known-URL).

These warnings have never been shown in 28.0.3.2 or older version.

My nextcloud is built based on https://github.com/nextcloud/docker/tree/master/.examples/docker-compose/insecure/mariadb/fpm.

My update methods:

  • docker-compose down to set nextcloud down
  • remove old nextcloud image (including old nextcloud:fpm)
  • docker-compose up -d to update the nextcloud image

Here is the log:

Configuring Redis as session handler
Initializing nextcloud 29.0.0.19 ...
Upgrading nextcloud from 28.0.3.2 ...
=> Searching for scripts (*.sh) to run, located in the folder: /docker-entrypoint-hooks.d/pre-upgrade
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Setting log level to debug
Turned on maintenance mode
Updating database schema
Updated database
Disabled incompatible app: calendar
Disabled incompatible app: files_accesscontrol
Disabled incompatible app: tasks
Disabled incompatible app: theming_customcss
Updating <federation> ...
Updated <federation> to 1.19.0
Updating <lookup_server_connector> ...
Updated <lookup_server_connector> to 1.17.0
Updating <oauth2> ...
Updated <oauth2> to 1.17.0
Updating <password_policy> ...
Updated <password_policy> to 1.19.0
Updating <photos> ...
Updated <photos> to 2.5.0
Updating <files> ...
Updated <files> to 2.1.0
Updating <activity> ...
Updated <activity> to 2.21.1
Updating <circles> ...
Updated <circles> to 29.0.0-dev
Updating <cloud_federation_api> ...
Updated <cloud_federation_api> to 1.12.0
Updating <dav> ...
Fix broken values of calendar objects

 Starting ...

Clean up old calendar subscriptions from deleted users that were not cleaned-up

    0/0 [>---------------------------]   0% Starting ...

Updated <dav> to 1.30.1
Updating <files_external> ...
Updated <files_external> to 1.21.0
Updating <files_sharing> ...
Fix broken values of calendar objects


    0/0 [->--------------------------]   0% Starting ...
    0/0 [--->------------------------]   0% Starting ...
Clean up old calendar subscriptions from deleted users that were not cleaned-up


Updated <files_sharing> to 1.21.0
Updating <files_trashbin> ...
Updated <files_trashbin> to 1.19.0
Updating <files_versions> ...
Updated <files_versions> to 1.22.0
Updating <sharebymail> ...
Updated <sharebymail> to 1.19.0
Updating <workflowengine> ...
Fix broken values of calendar objects

    0/0 [----->----------------------]   0% Starting ...

Clean up old calendar subscriptions from deleted users that were not cleaned-up

    0/0 [------->--------------------]   0% Starting ...

Updated <workflowengine> to 2.11.0
Updating <admin_audit> ...
Updated <admin_audit> to 1.19.0
Updating <comments> ...
Updated <comments> to 1.19.0
Updating <firstrunwizard> ...
Updated <firstrunwizard> to 2.18.0
Updating <logreader> ...
Updated <logreader> to 2.14.0
Updating <nextcloud_announcements> ...
Updated <nextcloud_announcements> to 1.18.0
Updating <notifications> ...
Updated <notifications> to 2.17.0
Updating <systemtags> ...
Updated <systemtags> to 1.19.0
Updating <theming> ...
Fix broken values of calendar objects

    0/0 [--------->------------------]   0% Starting ...

Clean up old calendar subscriptions from deleted users that were not cleaned-up

    0/0 [----------->----------------]   0% Starting ...

Updated <theming> to 2.4.0
Updating <contactsinteraction> ...
Updated <contactsinteraction> to 1.10.0
Updating <dashboard> ...
Updated <dashboard> to 7.9.0
Updating <federatedfilesharing> ...
Updated <federatedfilesharing> to 1.19.0
Updating <files_pdfviewer> ...
Updated <files_pdfviewer> to 2.10.0
Updating <files_reminders> ...
Updated <files_reminders> to 1.2.0
Updating <privacy> ...
Updated <privacy> to 1.13.0
Updating <provisioning_api> ...
Updated <provisioning_api> to 1.19.0
Updating <recommendations> ...
Updated <recommendations> to 2.1.0
Updating <related_resources> ...
Updated <related_resources> to 1.4.0
Updating <serverinfo> ...
Updated <serverinfo> to 1.19.0
Updating <settings> ...
Updated <settings> to 1.12.0
Updating <support> ...
Fix broken values of calendar objects

    0/0 [------------->--------------]   0% Starting ...

Clean up old calendar subscriptions from deleted users that were not cleaned-up

    0/0 [-------------->-------------]   0% Starting ...

Updated <support> to 1.12.0
Updating <survey_client> ...
Fix broken values of calendar objects

    0/0 [---------------->-----------]   0% Starting ...

Clean up old calendar subscriptions from deleted users that were not cleaned-up

    0/0 [------------------>---------]   0% Starting ...

Updated <survey_client> to 1.17.0
Updating <text> ...
Fix broken values of calendar objects

    0/0 [-------------------->-------]   0% Starting ...

Clean up old calendar subscriptions from deleted users that were not cleaned-up

    0/0 [---------------------->-----]   0% Starting ...

Updated <text> to 3.10.0
Updating <twofactor_backupcodes> ...
Fix broken values of calendar objects

    0/0 [------------------------>---]   0% Starting ...

Clean up old calendar subscriptions from deleted users that were not cleaned-up

    0/0 [-------------------------->-]   0% Starting ...

Updated <twofactor_backupcodes> to 1.18.0
Updating <updatenotification> ...
Updated <updatenotification> to 1.19.1
Updating <user_status> ...
Updated <user_status> to 1.9.0
Updating <viewer> ...
Updated <viewer> to 2.3.0
Updating <weather_status> ...
Updated <weather_status> to 1.9.0
Update app contacts from App Store
Update app calendar from App Store
Update app documentserver_community from App Store
Fix broken values of calendar objects

    0/0 [>---------------------------]   0% Starting ...

Clean up old calendar subscriptions from deleted users that were not cleaned-up

    0/0 [->--------------------------]   0% Starting ...

Repair warning: Error while trying to rebuild fonts, if you had any custom fonts configured you'll need to run `occ documentserver:fonts --rebuild`
Update app files_accesscontrol from App Store
Update app onlyoffice from App Store
Update app theming_customcss from App Store
Starting code integrity check...
Finished code integrity check
Update successful
Turned off maintenance mode
Resetting log level
The following apps have been disabled:
=> Searching for scripts (*.sh) to run, located in the folder: /docker-entrypoint-hooks.d/post-upgrade
Initializing finished

Any suggestions?

@invalidplayername
Copy link

For the well known part, there are older issues where the issue has been solved by adding corresponding blocks in the nginx config. Although, the code hasn't made its way into the nextcloud repo yet:
nextcloud/server#38035 (comment)

@huangwb8
Copy link
Author

huangwb8 commented May 6, 2024

@invalidplayername

First, I used nginx as reverse proxy manager, where I had added :

location /.well-known/caldav {
    return 301 $scheme://$host/remote.php/dav;
}

location /.well-known/webfinger {
    return 301 $scheme://$host/index.php/.well-known/webfinger;
}

Second, my nginx configuration of nextcloud-fpm (raw in https://github.com/nextcloud/docker/tree/master/.examples/docker-compose/with-nginx-proxy/mariadb/fpm) had been added:

location ^~ /.well-known {
            location = /.well-known/carddav { return 301 /remote.php/dav/; }
            location = /.well-known/caldav  { return 301 /remote.php/dav/; }

            location = /.well-known/webfinger  { return 301 /index.php/.well-known/webfinger;}
            location = /.well-known/nodeinfo  { return 301 /index.php/.well-known/nodeinfo;}

            location /.well-known/acme-challenge    { try_files $uri $uri/ =404; }
            location /.well-known/pki-validation    { try_files $uri $uri/ =404; }

            # Let Nextcloud's API for `/.well-known` URIs handle all other
            # requests by passing them to the front-end controller.
            return 301 /index.php$request_uri;
        }

These settings work well in 28.0.3.2, where the .well-known error did not appear.

By the way, the .htaccess issue seemed to be a more serious problem. Any suggestions?

@joshtrichards
Copy link
Member

joshtrichards commented May 6, 2024

Does the DNS for "yournextcloud.com" resolve to the same IP address from within the Nextcloud app container as it does from the outside? The various setup checks have been migrating from being browser based to being server-side. They run through your trusted_domains and overwrite.cli.url and run from the container itself now.

Either way, these sound like either upstream (server) or config matters. I see nothing that suggests these are specific to the image.

@joshtrichards joshtrichards added needs info Additional info needed to triage upstream labels May 6, 2024
@huangwb8
Copy link
Author

huangwb8 commented May 7, 2024

@joshtrichards

The related contents in the config.php are as following:

'trusted_domains' =>
array (
0 => '192.168.1.100',
1 => 'yournextcloud.com',
),
'overwritehost' => 'yournextcloud.com:2443',
'overwriteprotocol' => 'https',
'overwrite.cli.url' => 'http://yournextcloud.com:2443',

These settings work well in the past versions of Nextcloud.

I checked the logs after I peformed a new security & setup warnings checking:

image-20240507115404641

According to the picture, the .well-known error seemed to be corelated with the availability of 80 port.

For some reasons, I cannot use 80 and 443 in my server, that's why I use the other port (2443) to replace 443. I don't known why the .well-known checking is involving URLs like http://yournextcloud.com:80, which I have never used.

Any suggestions?

@tinysun
Copy link

tinysun commented May 9, 2024

I have the same problem here, I deployed it directly on the physical machine.

@AGagarin
Copy link

AGagarin commented May 11, 2024

Same issue after installation on VPS with VM script. Your data directory and files are probably accessible from the internet. The .htaccess file is not working.

Has the identical setup on another VM with Nextcloud 28 no issues.

@joshtrichards
Copy link
Member

joshtrichards commented May 13, 2024

These settings work well in the past versions of Nextcloud

@huangwb8:

As I said: the checks used to all (mostly) run from your browser. They are now (mostly) migrated to a new API that runs them from the server/container. This isn't a Docker matter, but a Nextcloud Server matter. It's an incremental change in progress throughout the v28+ branches.

I do note your config looks odd:

 'overwritehost' => 'yournextcloud.com:2443',
 'overwriteprotocol' => 'https',
 'overwrite.cli.url' => 'http://yournextcloud.com:2443',

These contradict each other. One is using http on 2443 and the other https.

Since you're using the Docker image you may also want to confirm the output of occ config:list system (i.e. your real running config) truly reflects what you posted. For example, the overwrite* parameters specified via Compose would overwrite those since the reverse-proxy.config.php takes priority over the config.php.

I have the same problem here, I deployed it directly on the physical machine.

Same issue after installation on VPS with VM script.

@tinysun & @AGagarin: Then you're in the wrong place. This repository is solely for the Docker image.

As it stands this Issue appears to be for matters that are a mixture of (possibly) upstream bugs (Nextcloud Server not the Docker image) or, in most cases, config matters that are arising as new problems... due to changes with how the setup checks function as of Nextcloud Server v28/v29. I suggest following up at the Help Forum.

@witchent
Copy link

I use the nextcloud-fpm docker image with podman and nginx as proxy, and I get the same message about .htaccess which seems weird to me. Don't have the .well-known problem though, that still seems to work.

@hugalafutro
Copy link

config matters that are arising as new problems... due to changes with how the setup checks function as of Nextcloud Server v28/v29

I use different docker image and the issue is there as well from the day of update to NC29 so I'd say upstream, but more importantly;

is it then:
a) all good as it was before, just the config.php parser throwing an error in some configurations
b) my nextcloud data was exposed to internet for last ~3 years while nextcloud claimed all is well ?

Not trying to assign blame, just gauge the extent of the issue. I'm leaning towards a) as I didn't do any config changes for months (last change was to move the whole nc dir to new mount path of a new storage) and run my nc in cloud with fair amount of firewall probing going on and the server comes up A++ in the Qualys SSL test and all the firewall probers like nikto etc. I tried got blocked before they even started by crowdsec.

How could I try to access a file directly to see if it is indeed accessible without logging in to confirm whether this is indeed an issue and not just spurious warning?

@joshtrichards
Copy link
Member

joshtrichards commented May 14, 2024

@hugalafutro My guess it is a false positive since that check (the data directory security check) just got migrated to a new API, but it may not be a bug necessarily (though that's certainly possible). What it does it send an HTTP/HTTPS request to all of all your trusted_domains and your overwrite.cli.url with your datadirectory appended + .ocdata - eg.

https://domain1.com/var/nc_data/.ocdata
https://domain2.com/var/nc_data/.ocdata
https://ipaddress1/var/nc_data/oc.data

I say it could be a bug, but may not be one because it may be something weird like one of your configured trusted_domains actually resolves elsewhere.... and that "elsewhere" happens to return a 200 HTTP code to the above queries. That was what one person (below, in part) reported.

Related: nextcloud/server#45087

@joshtrichards
Copy link
Member

joshtrichards commented May 14, 2024

@huangwb8 Since your are getting both the data directory and the .well-known setup checks failing in your environment simultaneously, the above that I just posted may be relevant. Because both keep trying until they find a host that responds with an valid status code. Maybe audit your trusted_domains and overwrite.cli.url value fully:

  • Make sure there aren't any old no longer in-use hostnames/IP addresses.
  • Make sure also that they don't resolve to different things from the container's perspective (mismatched internal versus external DNS), etc.

@witchent
Copy link

witchent commented May 14, 2024

@huangwb8 Since your are getting both the data directory and the .well-known setup checks failing in your environment simultaneously, the above that I just posted may be relevant. Because both keep trying until they find a host that responds with an valid status code. Maybe audit your trusted_domains and overwrite.cli.url value fully:

* Make sure there aren't any old no longer in-use hostnames/IP addresses.
* Make sure also that they don't resolve to different things from the container's perspective (mismatched internal versus external DNS), etc.

Actually I just did that as well (deleted all trusted domains except for the one I am normally using) and it seems to have fixed the warning. Maybe I broke something (that list grew a bit over the years) but then I hopefully remember what I did. So thank you for giving me the push of doing this.

@hugalafutro
Copy link

Can confirm after removing all trusted_domains apart from the single one where nextcloud resides and restarting the container the warning is gone. Not sure why I had the local docker network and external IP there as well, but they're gone now along with the error.

In trusted_proxies I have the external ip of the machine (the reverse proxy runs on the same machine), 127.0.0.1 and ::1 - the latter 2 to make local push service working and none of these setting seem to trigger the warning as I didn't touch them.

In any case many thanks for pointing the right direction, although I believed there is no issue the existence of the warning kept nagging at the back of my head.

@whoamiTM
Copy link

whoamiTM commented May 16, 2024

Can confirm after removing all trusted_domains apart from the single domain that I have HA Proxy resolving to nextcloud, and restarting the jail, the warning is finely gone. I removed localhost and the jails local IP address.

@huangwb8
Copy link
Author

As you suggested, delete unavailable trusted domains also works for me by solving the .htaccess issue.

By the way, the .well-known still exists. In the 29.0.0.19 version, Calendar could not be installed in my nextcloud, which might be the chief culprit.

Thanks for all you guys! This issue can be just closed for a while.

@Banditen01
Copy link

Couldn't figure this out, then I found this and changed in previous working docker compose file. Maybe can help someone !!??

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs info Additional info needed to triage question upstream
Projects
None yet
Development

No branches or pull requests

10 participants