Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[bug] Regression - Conan 2.13-2.14 fail to extract tar files with read-only directories #17987

Closed
tttapa opened this issue Mar 19, 2025 · 9 comments · Fixed by #17998
Closed

[bug] Regression - Conan 2.13-2.14 fail to extract tar files with read-only directories #17987

tttapa opened this issue Mar 19, 2025 · 9 comments · Fixed by #17998
Assignees
@franramirez688
Labels
bug

Comments

@tttapa
Copy link

tttapa commented Mar 19, 2025 •
edited
Loading

Describe the bug

Conan 2.13 and 2.14 fail when extracting a tar file that contains read-only directories. Conan 2.12 and earlier handle this case without issues.

How to reproduce it

https://github.com/tttapa/conan-recipes/blob/45f1ab171a3450dce62bd470a9b04f949f9d2a3b/recipes/tttapa-toolchains/binary/conanfile.py#L115-L120

For example, downloading the file https://github.com/tttapa/toolchains/releases/download/1.0.2/x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz using the get() function fails with the following error on Linux:

tttapa-toolchains/1.0.1: Calling package()
tttapa-toolchains/1.0.1: Downloading 141.7MB x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz
tttapa-toolchains/1.0.1: Downloaded 67.7MB 47% x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz
tttapa-toolchains/1.0.1: Downloaded 136.8MB 96% x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz
tttapa-toolchains/1.0.1: Unzipping x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz to /home/pieter/.conan2/p/b/tttap5d52ca74e5127/p
ERROR: tttapa-toolchains/1.0.1: Error in package() method, line 115
        get(
        PermissionError: [Errno 13] Permission denied: '/home/pieter/.conan2/p/b/tttap5d52ca74e5127/p/x86_64-bionic-linux-gnu/build.log.bz2'
@franramirez688 franramirez688 self-assigned this Mar 19, 2025
@franramirez688
Copy link
Contributor

Hi @tttapa

Thanks a lot for reporting this. There's something weird because I got failures even using Conan 2.12 and 2.11:

Conan 2.12

conanfile.py (test/1.0): Unzipping x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz to /Users/conan/develop/conan/mmm
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 19%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 39%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 58%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 78%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 97%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 114%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 132%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 151%
ERROR: conanfile.py (zlib/1.2.1): Error in source() method, line 16
	get(self, "https://github.com/tttapa/toolchains/releases/download/1.0.2/x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz",
	PermissionError: [Errno 13] Permission denied: '/Users/conan/develop/conan/mmm/x86_64-bionic-linux-gnu/x86_64-bionic-linux-gnu/sysroot/usr/include/linux/netfilter_ipv6/ip6t_hl.h'

Conan 2.11

conanfile.py (test/1.0): Unzipping x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz to /Users/conan/develop/conan/mmm
ERROR: conanfile.py (zlib/1.2.1): Error in source() method, line 16
	get(self, "https://github.com/tttapa/toolchains/releases/download/1.0.2/x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz",
	PermissionError: [Errno 13] Permission denied: '/Users/conan/develop/conan/mmm/x86_64-bionic-linux-gnu/x86_64-bionic-linux-gnu/sysroot/usr/include/linux/netfilter_ipv6/ip6t_hl.h'

Could you provide the logs where it got a successful download? Ensure you use an empty folder where it's downloaded to verify that it does not happen in earlier versions.

tttapa added a commit to tttapa/conan-recipes that referenced this issue Mar 19, 2025
@tttapa
[CI] test for conan-io/conan#17987
f72e9a8
@tttapa
Copy link
Author

tttapa commented Mar 19, 2025

I've added a CI workflow that builds the package using different versions of Conan:

https://github.com/tttapa/conan-recipes/actions/runs/13945071556

It passes for 2.12 and 2.11, but fails for 2.13, 2.14 (with -vtrace for the full backtrace etc.).

@memsharded
Copy link
Member

I have been able to reproduce in Windows, Python3.13, latest develop2 source branch:

conanfile.py (mypkg/0.1): Calling source() in C:\Users\conanws\kk\kk
conanfile.py (mypkg/0.1): Downloading 142.8MB x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz
conanfile.py (mypkg/0.1): Downloaded 63.5MB 44% x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz
conanfile.py (mypkg/0.1): Downloaded 134.7MB 94% x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz
conanfile.py (mypkg/0.1): Unzipping x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz to .
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 117%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 241%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 369%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 494%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 618%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 743%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 869%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 993%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 1120%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 1245%
Uncompressing x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz: 1355%
ERROR: conanfile.py (mypkg/0.1): Error in source() method, line 12
        get(
        PermissionError: [Errno 13] Permission denied: '.\\x86_64-bionic-linux-gnu\\x86_64-bionic-linux-gnu\\sysroot\\usr\\include\\linux\\netfilter_ipv6\\ip'

Note also the wrong % reporting @perseoGI

But I have also tried with Conan 2.12, and it also fails in same way

@franramirez688
Copy link
Contributor

franramirez688 commented Mar 20, 2025 •
edited
Loading

Hi @tttapa

I just tried it in Ubuntu 24.04 (and Python 3.13), and I got the same result as the one mentioned above. It's failing for Conan 2.11, 2.12, 2.13, and 2.14:

2.11 and 2.12

conanfile.py (test/1.0): Unzipping x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz to /home/develop/conan/mmm
ERROR: conanfile.py (test/1.0): Error in source() method, line 16
	get(self, "https://github.com/tttapa/toolchains/releases/download/1.0.2/x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz",
	PermissionError: [Errno 13] Permission denied: '/home/develop/conan/mmm/x86_64-bionic-linux-gnu/x86_64-bionic-linux-gnu/sysroot/usr/include/linux/netfilter_ipv6/ip6t_hl.h'

Upper versions are indeed failing in a different way (always matching the bin folder):

2.13 and 2.14

conanfile.py (test/1.0): Unzipping x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz to /home/develop/conan/mmm
ERROR: conanfile.py (test/1.0): Error in source() method, line 16
	get(self, "https://github.com/tttapa/toolchains/releases/download/1.0.2/x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz",
	PermissionError: [Errno 13] Permission denied: '/home/develop/conan/mmm/x86_64-bionic-linux-gnu/bin'

Anyway, I'm debugging the code to try to understand what could be happening.

@tttapa
Copy link
Author

tttapa commented Mar 20, 2025 •
edited
Loading

Thanks a lot for looking into this issue!

I haven't personally encountered the ip6t_hl.h error you mentioned for 2.11 and 2.12, those versions work fine for me when running locally on Ubuntu 22.04 and Rocky Linux 9, and also using the Ubuntu 24.04 GitHub Actions image as shown in the link above.
Maybe it depends on the Python build as well? I'm building Python from source for my local tests, and using https://github.com/actions/setup-python for GitHub Actions.

Edit: That doesn't seem to be the issue either. I just tried Conan 2.12.2 with the system installation of Python on Ubuntu 22.04 (3.10.12), and it works fine. The same setup fails using Conan 2.13.0:

conan create recipes/tttapa-toolchains/binary --version=1.0.2 --build=missing --build-require -s compiler.version=14 -s compiler=gcc -vv
...
tttapa-toolchains/1.0.2: Downloaded /home/pieter/.conan2/p/b/tttap0bd740c6fb258/b/x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz from https://github.com/tttapa/toolchains/releases/download/1.0.2/x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz
tttapa-toolchains/1.0.2: Unzipping x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz to /home/pieter/.conan2/p/b/tttap0bd740c6fb258/p
ERROR: Traceback (most recent call last):
  File "/home/pieter/.local/lib/python3.10/site-packages/conan/internal/errors.py", line 35, in conanfile_exception_formatter
    yield
  File "/home/pieter/.local/lib/python3.10/site-packages/conan/internal/methods.py", line 64, in run_package_method
    conanfile.package()
  File "/home/pieter/.conan2/p/tttap54b1b56828eb6/e/conanfile.py", line 115, in package
    get(
  File "/home/pieter/.local/lib/python3.10/site-packages/conan/tools/files/files.py", line 131, in get
    unzip(conanfile, filename, destination=destination, keep_permissions=keep_permissions,
  File "/home/pieter/.local/lib/python3.10/site-packages/conan/tools/files/files.py", line 304, in unzip
    return untargz(filename, destination, pattern, strip_root, extract_filter)
  File "/home/pieter/.local/lib/python3.10/site-packages/conan/tools/files/files.py", line 393, in untargz
    tarredgzippedFile.extract(member, path=destination)
  File "/usr/lib/python3.10/tarfile.py", line 2319, in extract
    self._extract_one(tarinfo, path, set_attrs, numeric_owner)
  File "/usr/lib/python3.10/tarfile.py", line 2353, in _extract_one
    self._handle_fatal_error(e)
  File "/usr/lib/python3.10/tarfile.py", line 2349, in _extract_one
    self._extract_member(tarinfo, os.path.join(path, tarinfo.name),
  File "/usr/lib/python3.10/tarfile.py", line 2434, in _extract_member
    self.makedir(tarinfo, targetpath)
  File "/usr/lib/python3.10/tarfile.py", line 2467, in makedir
    os.mkdir(targetpath, 0o700)
PermissionError: [Errno 13] Permission denied: '/home/pieter/.conan2/p/b/tttap0bd740c6fb258/p/x86_64-bionic-linux-gnu/bin'

tttapa-toolchains/1.0.2: Error in package() method, line 115
	get(
	PermissionError: [Errno 13] Permission denied: '/home/pieter/.conan2/p/b/tttap0bd740c6fb258/p/x86_64-bionic-linux-gnu/bin'

@franramirez688
Copy link
Contributor

franramirez688 commented Mar 20, 2025 •
edited
Loading

Hi @tttapa

After several hours of looking into this, I can confirm that it's a regression bug so thanks a lot for reporting it.

I was quite confused when it failed for older versions but I realized that my file system is case-insensitive so that's why it was failing:

tar -tf x-tools-x86_64-bionic-linux-gnu-gcc14.tar.xz | grep -i "ip6t_hl.h"
x-tools/x86_64-bionic-linux-gnu/x86_64-bionic-linux-gnu/sysroot/usr/include/linux/netfilter_ipv6/ip6t_HL.h
x-tools/x86_64-bionic-linux-gnu/x86_64-bionic-linux-gnu/sysroot/usr/include/linux/netfilter_ipv6/ip6t_hl.h

Going back to the bug, it was introduced here: #17708 (specifically this line https://github.com/conan-io/conan/pull/17708/files#diff-9afbf22c7cefd3f7b642e9b1b988ba829d3c36e3092bfb6140d01ad0bba4c571R393).

tarfile.extractall() has a slightly different behavior compared with tarfile.extract() so that's the problem.

We'll try to propose a bugfix asap.

Thanks again for your feedback!

UPDATED: For the record, my host machine is macOS, and it's case-insensitive. However, as I said above, it was failing in my docker Ubuntu image, which is case-sensitive. Why? because I was building the test in a shared volume with macOS.

@franramirez688 franramirez688 mentioned this issue Mar 20, 2025
Merged
@franramirez688
Copy link
Contributor

@tttapa I'm proposing this #17998 to solve the issue (pending more testing).

If you want to give it a try using Conan from the source code, any feedback is more than welcome 😁

@tttapa
Copy link
Author

tttapa commented Mar 20, 2025

Thanks a lot, the frm/bugfix/untar_extraction branch fixes the problem for me.

@franramirez688
Copy link
Contributor

@tttapa this fix will be released in the next Conan 2.15.

Thanks again for reporting it and the useful feedback 👏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@franramirez688 franramirez688

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[regression] untargz fails extracting some folders franramirez688/conan
3 participants
@franramirez688 @memsharded @tttapa