[JK] Portable/Relocatable Python Linking #275
Conversation
jaswanthikolla
force-pushed
the
portable_python
branch
from
7e7b102
to
e36e941
Compare
|
|
||
| It "Relocatable Python" { | ||
| $semversion = [semver] $Version | ||
| $pyfilename = "python$($semversion.Major).$($semversion.Minor)" | ||
| $artifactPath = Join-Path "Python" $Version | Join-Path -ChildPath $Architecture | ||
|
|
||
| $relocatedPython = Join-Path $HOME "relocated_python" | ||
| $relocatedPythonTool = Join-Path -Path $relocatedPython -ChildPath $artifactPath | ||
| $relocatedFullPath = Join-Path $relocatedPythonTool "bin" | Join-Path -ChildPath $pyfilename | ||
|
|
||
| # copy the current build to relocated_python | ||
| $toolCacheArtifact = Join-Path $env:RUNNER_TOOL_CACHE $artifactPath | ||
| moveAssets -source $toolCacheArtifact -destination $relocatedPythonTool | ||
| try { | ||
| # Verify that relocated Python works | ||
| $relocatedFullPath | Should -Exist | ||
| "$relocatedFullPath --version" | Should -ReturnZeroExitCode | ||
| "sudo $relocatedFullPath --version" | Should -ReturnZeroExitCode | ||
| } | ||
| finally { | ||
| # Revert the changes for other tests | ||
| moveAssets -source $relocatedPythonTool -destination $toolCacheArtifact | ||
| } | ||
| } |
There was a problem hiding this comment.
I think this might fail on macOS.
@jaswanthikolla, can you run the full build in your fork to make sure this passes ?
It might save some time if it requires some modifications before maintainers validate the workflow / review the PR.
It might only fail when using a version not using the universal2 installer so worth checking a build of python < 3.11
There was a problem hiding this comment.
This is Ubuntu/Linux Specific code, So this test doesn't get executed for macOS. May be, I can simplify by removing powershell code and write bash instead.
There was a problem hiding this comment.
It does fail on macOS which is an UNIX (it's UNIX specific code, not Linux): https://github.com/mayeut/python-versions/actions/runs/11766830107/job/32774981352
|
Any update on this PR? Can it be reviewed and merged? |
|
Just adding a +1 here. This is a real issue for many users, @jaswanthikolla has put in the work to resolve it for many use cases, and it's been going stale for 5 months. |
|
Also echoing the need for this fix to be merged. |
What's the Issue?
Python binary is compiled with
rpaththat's/opt/hostedtoolcache/Python. Now, latest github runners defineRUNNER_TOOL_CACHE/AGENT_TOOLSDIRECTORYdifferently than/opt/hostedtoolcacheand that installs python at/home/runner/_work/_tool/Python/. So, with this, there are 2 issues.sudo python --versiondoesn't work ( See Error section) because most systems's doesn't allow passing LD_LIBRARY_PATH due to security issues.python --versiondoesn't work without setting environment variable LD_LIBRARY_PATHoutput of ldd :
Error:
Solution:
$ORIGIN is used to as reference to the binary path. So, we can use rpath that references relative path instead of absolute compile time path. With this relative path, Python binaries become relocatable.
Change:
--rpathchanged to-rpathas it's the standard documented one.How this is tested?
Why bother about this issue?
LD_LIBRARY_PATH can't be used with
sudo pythondue to security concerns. But, more details are here actions/setup-python#871